Microsoft 365 Community Community Demos, 13 Jun 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: ChatGPT - but with YOUR data in Power Apps

Description

Get better factual information from CharGPT by injecting curated information from an internal datasource. In this 13-minute citizen developer-focused demo, Robin Rosengrün shows viewers how create that data source. Vecrtorize data by calling OpenAI Embeddings endpoint. See how to call ChatGPT (via a connector) and add context (your data) to the query. ChatGPT goes beyond standard search return with clear recommendations – it’s math! You will understand the math behind the scenes after seeing this demo.

This PnP Community demo is taken from the Microsoft Power Platform community call – May 2023, recorded on May 17, 2023.

Demo Presenter
• Robin Rosengrün (EnBW) | @power_r2

Community Call Conversation
• aka.ms/May17-Demo2

Supporting materials
• Connector - OpenAI (Independent Publisher) (Preview) | https://learn.microsoft.com/connectors/openaiip/
• Connector - Azure OpenAI Service | https://github.com/microsoft/PowerPlatformConnectors/tree/dev/custom-connectors/AzureOpenAIService
• Documentation – Embeddings | https://platform.openai.com/docs/guides/embeddings
• Tool - Long-term Memory for AI - The Pinecone vector database | https://www.pinecone.io/
• Linktree - Robin Rosengrün | https://linktr.ee/r2power

Stay connected
• Twitter https://twitter.com/microsoft365dev
• Call attention to your great work by using #PowerAppsCC and #PnPWeekly on Twitter.

Learn more
• Microsoft 365 Unified Sample gallery - https://aka.ms/community/samples
• Microsoft 365 Platform Community in YouTube - https://aka.ms/community/videos
• Microsoft 365 Platform Community - http://aka.ms/community/home
• Microsoft 365 Platform Community Blog - https://aka.ms/community/blog

A

B

Ty, because when you ask jgpt for facts, who knows how that Meme ends, you're gonna have a bad time. Well, this time we actually had a good time, because jgpt actually know the answer, but sometimes it's it's a big mix. You get mixed results, you get hallucination and for one for one thing, we can't get facts out of JT when we are using internal data that jgpd can know about, because it is not on Wikipedia or not on the web. So not in the chatgpt training data and a little bit about me.

B

My name is Robin I work at the German Energy company, um not a I.T guy I'm, a mechanical engineer and most important about that. I have no clue how all these AI stuff works, but what I do know it's that they have really easy apis that we can use and leverage.

B

So we don't know, we don't have to know how all of this works, but we can just connect to it and use it in our Power Platform projects, just like um the open, AI embeddings at point where you're gonna look at today, because we're gonna vectorize our data and yeah. Let's see how we can do that and how we can call that endpoint. But first of all, let's jump into a small demo. Video right here.

B

I made a movie database that consists of 7 000 movies and, as you can see, the the plots and genres and years of the movies and so on. And if we are now looking for a search term, then we will find movies that are similar to that search term. So um David I will need in a few minutes uh movie search term for you and we're gonna um search live for movies, and then it's a little bit like in the previous demo. Then we provide jet gbt with context of the movies.

B

So this could be any data, not um not data. That jgpt actually knows about like like movies, but this could be any any data, because in the prompt, we're gonna put the context in there and then let jet GPT make sense of that and tell it why we should watch those movies so um embeddings what are embeddings, what is vectorization and why do we need it? um Well, it's a cool concept where we get a text or just a word and we represent the meaning of the text of word with a vector.

B

So what can you think of that? How does that look like this is a simple Vector with two Dimensions. So um if we have an x value and a y value, we can show it in the plot. So for the word doc, we probably have this Vector for the word cat. We have this Vector, then we bring a third word into the mix. In this case it's Labrador.

B

It's probably here and now it's all just numbers, and then we can calculate the distance between those words and we see the distance between the word dog and Labrador is shorter than the distance between the word Labrador and Cat, which makes sense because it's a dog breed of course. In reality, it's not that easy, because we don't have two Dimensions open, AI use. It uses a model that has 1500 Dimensions, so a little more data, but the concept stays the same and the calculations stay the same and are easily doable for a computer.

B

So let's look exactly at that example. We had, but let's use real data so now I'm calling the open, AI API on this button and as we see we get a 1536 numbers back, which is the vector, the meaning of the word doc and we're doing the same for the word cat and then we're doing the vector, search and check the similarity for the word Labrador and as we can see, in fact, it has higher similarity to the word doc than it has to the word cat.

B

Let's see how the code looks like for that, because it's not that complicated at all make it a little bit bigger.

B

The actual API call really short and, as I said in the chat a little bit earlier, um this is not right now in the independent publisher connector, but will be in two or three weeks.

B

This is a custom connector I built, but luckily the documentation on open, AI page is a really good or you can just I already linked that in the chat and to the repo, where you will find all the connectors and there is the actual new version I put up there that will be deployed into a three weeks, so you can get started right now or wait two weeks and just use the independent publisher, connector and the platform, and all that the connector wants from us is the model which we want to use.

B

In this case, the embedding other model and the text we want to vectorize, which is the text box in here and on the other side, we're first doing the same thing get a vector, but we are not using this code, which would be the same as as the last time. But this time we can use the Azure open AI service.

B

So I saw a question in the chat where it just was like and we needed to be in this region, and if we have sensitive data, especially when we are from Europe, we have to be a gdpr compliant. So we don't want our information going to U.S service of open Ai and then, of course, we just deploy our own openai models in in Azure and use this and there's also a great connector you can use. This is not independent publisher, connector, but a custom connector that then your last give its build um the same.

B

That was speaking earlier and I've also added the embeddings endpoint there. So you can also download these from the from the repo and the whole calculation for the similarity. I will not go into detail right here, but it's not too hard just a few lines of code and we can check the similarity of those two terms and let's get back to the presentation before we get to the movie demo, because I want to show you what I did here.

B

um We won't look, don't look into that, but just looking at it schematically I downloaded the movie data of 7000 movies from IMDb. They have free data set. You can play around with really cool stuff. There was no summary or plot in there, so I. Let jet GPT write summaries for 7 000 movies and stored all the metadata up here into dataverse, and then I created embeddings for all of these 7000 movies. So yes, 7, 000 vectors with 1 500 numbers. Each and I did install them in dataverse, because dataverse can make sense of that.

B

They are specialized. Vector databases in this case. It's time cone which has a free tier, so get an account and start playing around with it, and this database doesn't only help you with storing them, but also retrieving them and calculating calculating the um the distances in the back end, and when we are retrieving the data, then again we were typing in a search term, like you saw in the demo earlier, create an embedding of that.

B

We get the 10 most similar movies and then we puzzled back the the data we got because here are only the IDS of the movie stored. Then we get our metadata from our database database back and then just like in a previous demo. um We give context to Jet GPT and let it make sense of that. So we we give the the plots and the summaries and the name and the years of the movie and then judge APK, should tell us why this is a great movie.

B

I have to watch when searching for the search term. Xyz, oh David. Yes, sir. What kind of movie do we want to watch? Well.

A

Let's see uh Guardians just came out and there's a wide spectrum. What do we do? What, if we do, a search on Marvel will that work.

B

Let's try them so um now we're getting our data from the database back any second Captain Marvel, of course, uh yeah really close, because it's in the title actually, but all these others don't have it in the title, and these are I, think all Marvel movies. So that seems to work, but still one sneaking in no all Marvel movies. I think is this different Avengers, don't know, um and now we're waiting for chat GPT.

B

This takes a little bit longer so about 20-25 seconds, and it will tell us why we should watch those movies when we are looking for a Marvel movie.

A

I wonder if jgbt can also add, like a post credit take like after we read it, it gives us like a sneak preview to what it see. What I did there.

B

Yeah um and yeah that that's given this a history um kept Marvel Fantastic movie uh and what it's about and I, also snuck in the URLs and let jgpt write an html text, so we can actually click on that. It's a little bit like the like the Bing, with the like the new Bing, with the numbers where we can click on the sources. So, let's really quickly look at the code on this button, also just few lines of code in here.

B

um First of all, again we get the embedding. Then we do. The pine cone query uh did a custom connector for that, but they also have good documentation. So it took 10 minutes to set everything up and pass the vector in here. We want the top 10 results back. We puzzle the data back together.

B

This is my dataverse table and then we let jet GPT write our review for the movies and here we're actually passing the data we got from the database back in here and yeah got got our jet gpts take for that movie and for the last slide, let's again jump back to the document to the presentation, because we want to get back to business so back from the from fun projects to probably something more business related. Let's think about support cases like regular customer support or I.T support.

B

We have a database full of successful cases where we actually help the customer and let's check GPT, write a summary of every of those cases with the problem and the solution create an embedding of that and store everything in a vector database. Of course we're using Azure open AI for that, because we want to have control over where the data goes to and on the output. Probably this time we have a power virtual agent, where the customer can type in a search term. Then there's a embedding created.

B

We get back similar cases from our Vector database puzzle the data back together, and then we provide the context to Jet GPT. We provide the question the customer asked and then we provide the solutions of the three most similar case we had in that department and then it can make sense of that and probably formulate a solution that will actually help the customer. So we put our data in jet GPT, so really amazing stuff. What comes all of these Ai and my take? Is we it's so easy to connect to those apis?

B

Let it be open, Azure, open air, IAI or other AIS that come out, and my suggestion is start. Some fun projects get your hands wet in that and yeah start building something with it. Everyone back to you, David Robin,.

A

Awesome awesome, awesome, stuff, I cannot uh I, cannot say how awesomeness has been uh by saying it more than three times right, but really cool stuff. I also appreciate how you showed it can be fun that helps people learn right, I think that's a secret sauce for everybody. Who's wanting to get in this technology find something that's maybe not business related. Just because we do have a couple minutes. I did want to ask you. Is that something that helped you learn to keep going with the movies you were passionate about?

A

It was fun, and then you kind of looked for the uh for the business side of it.

B

Now I I had the the business in mind, but um I couldn't I had no no database with business cases, so I looked for something that I was passionate about and I would have fun. With a little bit. Yeah made my own database absolutely.

A

Awesome that is great stuff and a great, uh a great tip for all of us to be able to use that, because sometimes it can be hard to say what am I going to use this for maybe not having the passion. In this case, uh you absolutely use that for something you're passionate about and it led to what is an amazing success. So thank you for that awesome, stuff, foreign.