Microsoft 365 Community Microsoft Teams, 30 Mar 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Have fun with DALL-E generated art in your Microsoft Teams conversations

Description

In this 13-minute developer-focused demo, Lee Ford briefly reviews OpenAI - DALL-E, then proceeds to implement an AI scenario using a Teams Message Extension and an Azure Bot. Request an AI generated image based on a simple text description, receive back an image rendered in an Adaptive Card leveraging the Bot Framework to execute the transaction and accessing an Open AI API. A thorough educational code walk-through. Sample available.

This PnP Community demo is taken from the Microsoft 365 & Power Platform Development Community call recorded on March 30, 2023.

Demo Presenter
• Lee Ford (Symity) | @lee_ford

Community Call Conversation
• aka.ms/Mar30-Demo1

Supporting materials
• Sample - Teams DALL·E 2 Messaging Extension | aka.ms/msgext-dalle
• Article - How to use AI image prompts to generate art using DALL‑E | https://create.microsoft.com/learn/articles/how-to-image-prompts-dall-e-ai

Learn more
• Microsoft 365 Unified Sample gallery - https://aka.ms/community/samples
• Microsoft 365 Platform Community in YouTube - https://aka.ms/community/videos
• Microsoft 365 Platform Community - http://aka.ms/community/home

A

B

My name is Lee Ford and today it's a bit of a fun session. It's talking about having fun with AI in teams. So it's not a particularly serious conversation. It's more just kind of a bit of a fun to show. You know how easy it is to kind of implement AI into a teams app. So um we haven't got much time so I'm gonna get started so a quick bit about myself, I'm based in the UK.

B

um You can follow me on Twitter uh code. uh Put most of my samples on GitHub I've got a Blog which I'll blog very occasionally app, but I do do it. You know every other month or something like that. I mainly work in M365 workloads, hence being on this call.

B

um I do like to write code for a living and as a hobby. So hence why I'm talking today, I like to do a bit of exercise and a bit of DIY um uh just enough DIY to be dangerous uh anyway. So, okay, so what we're going to talk about is obviously we're going to talk about Dali um we're going to talk about what it is, uh what you can do with it and how you can use it um and then we're going to talk about how that then mapped into teams.

B

So how do I Implement into teams? And what can we do with it in teams? So um we haven't got a lot of time, so I'm going to uh kind of go through that pretty quickly, so um we're going to talk about open, Ai and we're not going to talk about.

B

um You know, chat, GPT and all that, but open AI um obviously offers different products. And obviously the main one that we're talking about here is is Dali. 2 allows you to generate an image from a description and we'll go over that in a bit more detail in a second, but um obviously we know the other ones like GPT and chat GPT and another one whisper. Those are different. Services down too, is just another one of those open AI services. So it's not its own company or anything like that.

B

It's just part of the open, AI Suite. um So if you've got access to the other, apis you'll have access to to Dali 2 as well. So um what is it? um It allows you to generate an image from a text input. So a dog eating an ice cream wearing a hat you can. You know you can type that type whatever you want and it will generate what it thinks you're asking for.

B

You can be a bit more descriptive, um so you can say it needs to look like an oil painting or it needs to look hand-drawn or whatever, and then it will kind of use the AI power to kind of try and get what you're asking for it's not always great. It doesn't always work exactly like AI. It doesn't always work exactly as you intend, but that's the the general kind of gist of it.

B

So how it's priced is depending on how high resolution the image needs to be depends how much the uh the image costs, um and, in addition to this, you can also request to have multiple attempts. So you could say a dog wearing a dog eating off from wearing a hats, and you could say: I want five different variations on that image and then you can choose which one you want to use, but ultimately, you've paid for all five to be generated. So you would pay in this case, let's say a higher resolution.

B

You pay two cents per image, even if you only really wanted one of the five to be used. So today we're just going to talk about a particular scenario, which is uh there's no easy way for me to find an obscure picture of a photo that I want to insert into a team's conversation. So you know maybe having a bit of a a laugh or a joke with with a colleague and you'll. Be I'll be good. If you, you know, there was a photo that existed of this particular thing.

B

Obviously it doesn't exist in the world, but I can use. uh You know the power of AI I can generate that that picture or that image and then insert it into a team's conversation. So um what we're going to do is we're going to use teams message, extension messaging extension.

B

It seems to be used interchangeably, but I'll call it message extension, um and the idea here is that you can access it by clicking the three dots on a message and then using it to generate an image, or um it's actually also at the bottom of the uh the message bar.

B

um You know with sort of compose message bar at the bottom of teams. You can also access it from there. So what a message extension is used for is it's interacting with existing or in this case, what we're going to do today is a new message. So the idea is that I want to insert something into the conversation um it can be used in a sort of a group conversation.

B

It could be used in a um personal conversation or it can be used um inside of a team, a conversation so um quickly go over what solution overview I have today. Obviously this doesn't have to be what you would use it's just what I've used for the the demo. So we effectively have someone using a teams, client they're, sending a request.

B

um I.E they're opening up the message, extension that then gets sent to bot services. So, even though we're not really using a bot in the truest sense, you know we're not having a conversation or anything like that. We are still using bot framework to uh to facilitate this things in the bot framework SDK from a from a code perspective, and the idea is that the team's client talks to the bot Services, which is hosted in Azure. That then gets relayed to wherever your code is running.

B

In this case, I've got it running in an Azure function and then I'm calling open, AI API to generate the image and then that effectively generates a link where the image is located. In this case, it's I think they use Azure, blob storage and then we then feed that back through to the team's client, with the image inside an Adaptive card, so we'll quickly go over. There actually have a lot of time. I'll quickly go over the uh it working we'll do a quick prayer to the demo guard to make sure it's.

B

It's all all good. But um yes, so I have done a few ones like I asked it to draw a dog saying: Choose, Love and we've got show of love, but you know it's it's near enough, um and we've also got some uh sort of personal conversations here. So just a bit a good friend Joanna. So these are previous ones I've tested earlier, so hopefully fingers crossed it will work now. So the idea is that we bring up the messaging extensions.

B

Obviously all these usual ones in there I'll create one called daddy two um and the idea was it'll, bring up a prompt and it'll say what what's the description of the image. So I'll just put a a cart and org slee on a couch. Okay, now I'm, not sure. What's gonna happen, hopefully it's safe for work, but this is AI. So who can who can say?

B

um Okay, uh I think that's a dog and a cat we'll go with it, I'm, not entirely sure. If it's half dog half cat but we'll we'll go with it, um the idea is I. Could edit and I can change the uh you know the description, but for brevity, let's just say: yeah, that's good and the idea is I can click Send and then that appears as a new message inside the conversation and obviously we include the description to say this is what I've asked for. This is what it's given me and that's about it.

B

From a demo perspective. I could write some more, um but ultimately it's yeah. It works that way, every time now, as far as uh what we need for it to work. uh As I mentioned, there is a a box service, Azure bot, whatever you want to call it and that sits in Azure and that's effectively the proxy between my code and the team's client.

B

So we have a given a very generic name, but you could call it whatever you wanted to in here. Obviously you can upload an icon and, from a config point of view, it's actually really really straightforward.

B

We actually just pointed at the um wherever the uh the the code is listening on a HTTP endpoint. So, in this case, I've got it tunneling back through to my local machine, where it's running in vs code But. Ultimately, this could be in Azure or whatever, um so that's it from a config except for actually just the channels. It's the only thing I've done in here is I've, enabled the team's channel to run out the the bot service to to receive requests from teams, and that's it for me, a kind of configuration point of view.

B

um As far as code goes, we have a couple of files really and that's about it as far as how it works, so we have a just going to zoom it out slightly just to make it over again. We have a sort of wrapper for the Azure function, so this is the the forward, slash messages, endpoint and effectively. What we're doing in here is that the main thing is we have a an activity Handler and that handles all the different types of activities that the message extension would send to on a HTTP request.

B

So I won't really worry about too much of how we set up the bot adapter But. Ultimately, the main part of the code is is this: is this bot file and in here we have a uh it's extending the team's activity Handler. So these are team, specific activities for what framework they're, not ones that you could use in um yeah web chat or anything like that and effectively. These methods are used for different purposes, so the first one is handle teens messaging extension fetch task.

B

So this is what gets run when you click on the message extension for the first time and it opens up- and if you remember it had a prompt to say please enter the you know what you want the AI to generate. In this case, I've got a just a helper function, that's um effectively creating an Adaptive card and I'll go through the adapter card in a second um and it's then creating um returning effectively the Adaptive card.

B

Alongside the message extension action response and obviously you can set the size and the height and what the title of the card is going to be and so on. And so that's what happens when you click the message extension um and then from Once. The message extension submit action is sent.

B

This is when I've completed so I want a dog and a cat lying on a couch when I click send that then triggers this activity, and in here we're effectively passing through the open, API key as a environment variable, and then we are effectively just creating using the open AI module where we're creating an image and all we're doing is we're passing through the image description, which is one of the fields in the in the Adaptive card um or saying this is the number of images that we want to be generated, so obviously we're only generating one.

B

We could generate five and have the user choose, but just to keep it simple, you're, just generating one and then again, just going back to the size of a major that we wanted to generate, and this again leads back to to the cost of you know. If you want a large image, it's going to cost more once we've got the response. We then effectively inject this back into the Adaptive card uh response and I'll go over the Adaptive cards in a second, but ultimately we're passing through the the URL and the description of the image.

B

That's what was asked for so you know I asked for an image to be whatever a type. This then sets that back into the Adaptive card response. So if I quickly look at the image response, we effectively have a really basic adaptive card. We've got the description as an accessible speech option, so it will describe it for uh um sort of accessibility reasons, but then also inside the uh the Adaptive card.

B

We have a text block with the description, and then we just have an image that links back to the image that the Open Eye API is generated of the image.

B

um So once we generate these out of the card, we then have the ability to create um an activity preview and the activity preview is when it says this is what I'm going to put in the chat and then you've got a send button or an edit button. If you click edit, it goes back to the previous step. If you click Send, then it inserts into the conversation. So that's what they call a bot method. Preview type of response handle editing of results.

B

That's when you click edit, so you've got actually that's not quite I wanted I click edit. That then sends it back to the original step. So again, we're then generating the prompt to say what would you like and then. Finally, the final method is when we've got the prompt to say this is what you know we're going to insert into the conversation. Are you okay? You click, send it then triggers this activity and like what this is effectively.

B

It's using the bot framework to basically say yep, okay, we're going to send the activity preview from the previous uh step and then effectively posts as a message into the conversation. So previously to this point, it hasn't nothing's been inserted into the conversation it's just been with inside the message extension sort of pop out this effectively then posts it into the channel or to the conversation, and we added on behalf of a bit of data, to show that it's not me personally, that's inserted a bit.

B

A bot has inserted it on my behalf and I think that is it from a code perspective. Really that's everything we have so I'll just quickly. Get back to the uh the slides and final thing to say is: it is already on uh PMP Dev samples. So if you do want to have a look try out for yourself, then please do and let me know how you get on and that's it for me.

A

Awesomely, very, very cool, stuff and I appreciate how you brought fun and Technology together and that's important for us to utilize uh as a community and in our work. All right. uh Questions put them in the chat for uh Lee and we'll have an opportunity to chat about those later how you can engage with them. Foreign.