Microsoft 365 Community Microsoft Graph, 26 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Building a custom data Loss Prevention (DLP) solution for emails with Microsoft Graph

Description

In this 16-minute developer focused demo, Rick Van Rousselt uses the new secure and customizable alternative to Microsoft Purview Insider Risk Management uses Microsoft Graph subscriptions for email (and other event) notifications and Azure.Messaging.EventHub as an incident reporting end-point. See how to create an EventHub and configure policies. Essentially 2 policies - Graph can only send to my EventHub and my custom code can only listen to my Event Hub! Solution uses an Azure Durable Function. This PnP Community demo is taken from the General Microsoft 365 Developer community call recorded on May 26, 2022.

Demo Presenter
• Rick Van Rousselt (Advantive) | @RickVanRousselt

Supporting materials
• Documentation – Get change notifications delivered in different ways | https://docs.microsoft.com/graph/change-notifications-delivery
• Documentation - Send events to and receive events from Azure Event Hubs - .NET (Azure.Messaging.EventHubs) | https://docs.microsoft.com/azure/event-hubs/event-hubs-dotnet-standard-getstarted-send
• Documentation - Azure Durable Functions documentation | https://docs.microsoft.com/azure/azure-functions/durable/

Learn more
• Microsoft 365 Unified Sample gallery - https://aka.ms/m365/samples
• Microsoft 365 Platform Community in YouTube - https://aka.ms/m365/videos
• Microsoft 365 Platform Community - http://aka.ms/m365/community

A

So data loss prevention solution with the microsoft golf. There is already, of course, a data loss prevention available in microsoft, 365 and part of it is insider risk management or it's pureview. These days, um insider is management. It's it's very good, but it's a very broad spectrum of alerts, so you not only email, checking and seeing who sends emails, for instance, externally, with attachments, but also sharepoint, and then teams, and then the us, even you can even see if somebody downloads something to a usb device stuff like that.

A

So it's very, very broad if, for instance, in our case, we needed to check okay, what if people are leaving and that's one of the main cases of insider this commandment, an employee leaves the company. What is he gonna take? I don't know the the the excel file with all the customers in it, uh along with him to his next job. Yeah. That's, maybe not a good idea. We don't want that, so you can create, what's called an hr connector.

A

So actually it's a csv file and you can upload that to powershell and then the system knows okay. This person has recently got this resignation or got fired or gave his designation himself, um so um we can set okay. This is the date he got this resignation and this is the date that he actually leaves the company and then he's monitored a little bit more, all very good, but yeah. As you all know, it's not customizable.

A

So um how to create your own email monitoring solution. Now, uh let's do demos, because I'm not a slight kind of person. So um let's go I'll. First show you the insider risk management. So if you want to do the uh hr connector as it's called, you can go to microsoft. Preview, say: data connectors. You create your own connector, which is creating an active directory, app registration, just client, id and secret. You don't need to give it any permissions.

A

Then you say this is a connector. You get a connected job id and you create your own csv file. I have it somewhere here, see just email address, resignation, date and last working day I took two people here. I upload that one with the script. So it's it's kind of also a little bit of custom development because yeah you see this script, okay, it's an example provided by microsoft, but it's one hell of a script. It's let me check 300. I don't need to zoom in, but it's 362 lines.

A

So it's not uh not perfect, and now everybody knows my client secret, but I'll change it afterwards and then in information risk management, insider risk management, you can say okay. This is the policy and I want to check. uh For instance, everybody who got uploaded microsoft recommends that the csv file- you upload, that then daily you schedule the script and daily things change. So every day you export that information.

A

It's all good but of course, security guys. We all know them. They need a lot more information. So how are we gonna? uh If somebody, for instance, uh sends an email outside of the company? How are we gonna check that? Because this was as you can see, let me see if I can create a post and show you the policy, for instance the. uh Let me check next next next next next, as you can see here, the thresholds, for instance uh at 10 to 20 events, it a day generates a low severity alert yeah.

A

We don't want an alert. No, we really want to see what people are doing so the classic way to do that is create a wrap hook or a subscription on somebody's inbox.

A

I think everybody most people this has been around for quite a while, so we create a subscription on somebody's inbox with the microsoft graph, meaning every time somebody sends an email. I think in this case it's in my inbox. So every time I get an email, but if you change these to send items also works. So every time I get an email uh this, uh the graph will notify my webhook. So an endpoint I put publicly somewhere with a there's something something has happened.

A

It will not say what has happened, but it will say something has happened and then I will have to uh get that information myself so check what what that? What is the event that occurred? So what was the email? Get the email and then do my rules. So we use an azure function for this one. As you can see here we have a notification function, um we get notified and of course that's the entire thing with the wrap hooks.

A

So the moment you create a subscription, the function has to be alive because it, the webhook, will check the subscription. Hey is it the li? Is the web hook alive. So is that azure function on live and then you get a token and you have to send that token back and then, once you ins once that's okay, the subscription will keep alive for a certain amount of time, depending on what server it we are really long. It's longer, I think exchange is only a week or something so every week you need to update that subscription.

A

So uh let's just uh create one I created one, so I will create a subscription and at the same time, of course, my function has to be online already. So I have my ng-rock running here somewhere. As you can see. I have my uh li it's this function.ng.io, so it's actually not simple to debug, build because you're creating a subscription. So where do I create my subscription? Let me delete all the subscriptions and then the moment I create my subscription. Of course my web book has to be alive, as you can see here.

A

If I go to all of this, there is a token in it and I have to send that token back. Otherwise the graph will say hey. This is not a valid endpoint, I'm not creating your subscription. So the moment I do that now I've returned that subscription is active and I have a valid subscription, meaning if I send an email to myself, I think it was on inbox. So let me just check so if I sent from an other email yeah, we don't need labels.

A

Of course, if I send an email to myself on my inbox, is it this one just test then we're good to go? Then we should get a notification from the microsoft graph. Hey there was an email coming in should happen every second. Now, if the email comes in should happen.

A

There we go takes a while for the email to come in now the email came in and now I should have a trigger if everything goes well.

A

Not yet no trigger okay, so, as you can see, this is the point I'm making it's hard to debug it's hard to create because you're working with ng rock and all all that stuff, you have to make sure that your function is already prepared before you can even start creating a subscription. So it's very hard to build as well. There is, of course, a new way to do this, that's with an event hub, so we can also say hey.

A

Instead of using my own um webhook or webhook yeah webhook that I created, we can also say: okay, microsoft graph go to the event hub, so I created an event hub and then the microsoft graph will say: okay, instead of going to the the endpoint of the webhook of the azure function, you created, let's go to the endpoint of an event.

A

It's also a little bit more secure and a little bit more stable because of course, if you let's say, for instance, turn off your magic function or you have to keep it alive because it goes to sleep. This event hub never goes to sleep. So how does it work? Actually, it's pretty cool because um you will first have to create a key vault and on that key vault, you will say: hey um the connection stream to my event hub. Well, that connection string, that's actually hidden. So this is the connection string to my event hub.

A

I put that in the key vault and I actually tell the microsoft graph, don't take my connection string to my event up, but that because that would not be secure. No hey, microsoft! Golf you get access to my e-vault as you can see, with access control, you should see here all assignments and there is a very specific one. uh Where is it?

A

There is a very specific one called graph.

A

I think it only has access to the secret, of course, called graph automation and that graph automation, app registration, that's already in your tenant, will have access to that connection, string and we'll get that connection thing for you first off and then we'll publish something to the event hub.

A

So if you can see that, if I go back to the event, sorry for this one, as you can see here with this event up, we have two policies: checked access policies, two policies, the graphs of the microsoft graph can only send to my event, love and my custom code can only listen on my event hub. So on an event up, you have centers and you have list next. So in this case it's very secure because my custom code can only listen. It cannot intervene with the event hub. It can send other stuff.

A

No only the graph can send stuff to that event hub. So, if we go back to our code, we have a doable function here. Also. We have a second problem that we also have what if so, as you saw, I created the subscription so where's, my this one, so I created my subscription.

A

This takes come.

B

A

For instance, let's say 10 seconds now: let's say let's stay two seconds that advance to create a subscription. What if we have like 50 000 users, we have to keep that subscription alive, it's meaning for exchange. Every week we have to do an update of the subscription because it's going to expire, so you give an expired date, but there's a max limit of course. So what if you have to do like 50 000 times two seconds yeah that takes a long time and you're running code again and again and again and again and again?

A

Well and of course, maybe we don't want to check all the users? Maybe we want to check, I don't know certain active directory groups, okay, you're building code again to check active directory groups uh who's a member of this group, the people you have to monitor and then maybe whatever rule you can find on people who to check you have to write that custom code yourself. Well, there's also a cool old trick to it and it's called transportation rules.

A

It's been around for millions of years and I created a simple transportation rule that says every time. Rick sends an email, so this is just me, but we can add multiple conditions, so there's more conditions of recipient if it has attachments. uh What are the message properties? Are they headaches?

A

This has been a lot around a long time ago, so it will cook you. If you go to an exchange expert, they will configure that perfectly for you. So every time rick sends an email, we'll send an incident report. It's called an incident report and you send it to a mailbox. This way, it's easy to configure.

A

If the sender is member of that group or if the sender is a shared mailbox whatever you can configure that in these conditions, this is the little bit of the ugly part, I'm still figuring that one out, but you get an incident report, meaning it looks like this. Let me open it up. It's just plain text, so rick sent an email, it was microsoft, build I sent it to this recipient and it was data platform.

A

So it generated an incident report because I created an email. I created a monitor, email inbox, meaning I just have to create a subscription from the microsoft graph on one on one inbox, because yeah, it's only one inbox that I have to monitor, making it easier. I just have to run that two-second subscription update at the end of the week. Every week now.

A

We can also create so the previous one was a function that got triggered by an http event. Now we don't do that anymore. We have an event hub tweaking so every time something is added to this event hub, it will run this event. Up is most it can take away more uh events than your own azure function or you have to scale it a lot or you're paying for a lot. So in this case this will the event.

A

Op will gather everything and then your code can easily simply do everything itself by on on a slow pace. It doesn't need to be super fast. So let me just start this. One up set a starter project.

A

Now, in this case, I needed to, of course, send an email. So oh there's already something in the event hub. I probably sent something out: let's go to the next one and let me just send an email just to be sure that it's the last one so I'll send a new email I'll, send it again to myself there we go test test. There we go.

A

Normally an incident report should come in, so it's a little bit slow, actually there's already. The incident report.

A

Now my event up, if my subscription is still alive, I hope so should be ticket.

A

A

Yeah see could be that my trigger is not working anymore.

A

There we go, he sent one more email, just to be sure you know what let's update the subscription, so I have my subscription. Let me quickly say this one, as you can see here, so this is the important part. Actually.

A

So if you, this is my normal notification url. So I have a function. A microsoft graph called this notification url in this case, no called this event hub and this event, as I already said, it's not, this is not the con. The connections thing to my event, no, this is to my azure keyboard. I put it a little bit bigger, so you can see it this to my azure key vault secrets and the one I called event top graph and, of course it my tenant id has to be in it.

A

So the moment I run this one, it will create a new subscription.

A

Oh, I of course have to put it through: go set a starter project and then we have a subscription. How am I doing on time? I have to stop talking within two seconds, so we have a subscription. Let me try the last one set a starter project good to go.

A

And, as you can see here, this event up, this is the magic.

A

So, as you can see here, let me go until there's the message, so this is actually not the email that got sent. There was a subscription I just created, but the subscription is already created. I didn't have to do anything I didn't have to return whatever code. So in this case the subscription got created. I have a subscription now I can send an email and it could it's good to go in the end, I'm not using a I'm, not running all my code at once.

A

No I'm starting a doable function so in this case a doable function, which is super cool. If you haven't used it use it, so it will get my data loss prevention entity. It will call whatever rule you want and it will report my email. So let me go back to the slides.

A

Let's do a quick recap, so we can use the microsoft graph subscription to get notifications about events in this case emails, but onedrive events, outlook events, security alerts, sharepoint themes, whatever you want, there's a lot of them.

A

We use event up to keys performance and stability and to make it a little bit easier for us developers. Also, the public url is not exposed because we don't say we, as you can see. Microsoft graph only knows our key vault and we can use transport rules to keep subscriptions to a minimum.

A

I've added the resources here you will get those later as well. Thanks.

B

Awesome very, very cool stuff. Rick really appreciate diving in deep. There really interesting to see and appreciate the extra effort.