.Net Foundation On .NET, 26 Mar 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cloud scale events with Azure Event Grid

Description

Azure Event Grid is a managed event routing service that can help distribute events across both Azure and non-Azure services in near-real time fashion.

In this episode Bahram Banisadr (@BAHRAwesoMe), a PM on the Event Grid team, comes on the show to talk about how to get started, discuss some of the common use cases, and show us some demos of the .NET SDK.
00:25 - What is event grid
01:57 - what are the supported scenarios outside of Azure ?
03:58 - What are the 3 most popular use cases of Event Grid?
05:24 - How does Event grid compare to other Cloud offerings
09:48 - How does EventGrid compare to Queues?
11:11 - Nuget Package
11:55 - Pub Sub Demo

Check out the following links to learn more:
Azure Event Grid homepage: https://azure.microsoft.com/en-us/services/event-grid?WT.mc_id=ondotnet-channel9-cephilli
Azure Event Grid docs: https://docs.microsoft.com/en-us/azure/event-grid/overview?WT.mc_id=ondotnet-channel9-cephilli
Event Grid NuGet package: https://www.nuget.org/packages/Microsoft.Azure.EventGrid?WT.mc_id=ondotnet-channel9-cephilli
Comparing messaging services in Azure: https://docs.microsoft.com/en-us/azure/event-grid/compare-messaging-services?WT.mc_id=ondotnet-channel9-cephilli
Create a Free Account (Azure): https://aka.ms/azft-net

A

Hey, welcome to another episode of on .net show.

B

Today, we've got bahram and he's going to be telling us about azure event. Grid., so quickly, tell us a little bit about yourself.

A

Yes., so I'm the pm for azure event, grid.

B

I've been with microsoft for about a year and a half now.

A

Okay and how's everything working out?, it's been pretty awesome,, so far. okay,, awesome. well,. You ended up on the show., so you know,. This is pretty good.. I made it to the promised land.

B

Yes, totally. so,, how about you tell us about event grid and what kind of cloud domain does that fit? Into?.

B

Yes. so, event grid is.

B

A messaging product and it's meant to be, basically,, a connective tissue for all of azure.

B

So,, what we're doing is building first class integrations with pretty much every azure service. and when there is a state change,.

A

Those services will emit an event,, and these can be pointed to pretty much any endpoint or built-in event handlers within azure.

A

okay., so pretty much all the azure teams. I.

B

Talked to told me that they're, the connective tissue of azure. so,, why are you really the connective tissue of azure?

B

yes. so,? The goal is here.

B

We're building a pub/sub system but within the ecosystem of azure.

B

We don't want customers to have to go build one.

B

Off connections or you know, even write a lot of code to get up and running.

B

so,. It's really a browse and select sort of operation.

B

You say,, I want to listen to my storage account anytime. A blob is created,, send an event to this event, to this azure function to handle it.

A

It's supposed to be very quick and easy to get set, up, subsecond and to end latency., right. so,. What I hear you saying: is, you know,.

A

If I'm a function or I have deployed a docker container or app service,.

A

It doesn't really matter., I can either participate in sending events or receiving them.

A

Exactly. yes. and, does this relate to just if my code is running in azure?.

B

Or are there some not in azure or scenarios like desktop that are relevant or even iot?.

B

Absolutely., so we do have first class integrations with azure, but we're also a general purpose: pub/sub system.

B

so,. You can omit events from your desktop application,.

B

You can receive them at pretty much any public http endpoint., so you are not restricted to.

A

Native azure event, types or azure as a system at all.

A

You can connect any services or anything that can run code, really. right.. So if I had the consoled app that.

A

Cared about blobs in a particular blob storage, account,.

B

Could I get events every time one of those was updated.

A

And then just write those to the console?

A

yes. absolutely. wow., you would., yes. and you're, saying.

B

Subsecond latency,, so pretty much, as those blobs.

B

Were being updated,, be up like almost instantaneous.

A

Yes., it enables real-time reactive programming, with your azure resources.

A

wow. now,, clearly with event grid,. I imagine people are using it within azure,.

A

With functions or whatever., have you seen people using it in.

B

This other way that we kind of talked about, either with desktop applications,, iot applications or even web applications that are running on-prem?, yes, totally.. We have a number of customers that.

B

Have various files that they're generating,.

B

They're dumping them into a store, and then they.

A

Want to go process them, on-prem, so,, they're, firing, events, picking them up and going and taking these files and processing them elsewhere.. I see. so,, clearly there's customers that are all on-prem and there's another set of customers that are all on the cloud., and so I guess, these scenario. We're talking about, is really interesting for these customers.

A

That kind of have the sliding window for how.

A

Much of their work is being done, on-prem and how much is being done in azure., yes., yes,, interesting.

B

So, back to event, grids.

B

Would you say maybe the three most popular ways that people are using event grid today that you see?

B

yes. so, the first one is certainly serverless.

B

So this is,, I have changes happening, the state changes happening to my various services in azure and I want to react to them in a way that requires writing code. But I don't want to host my own infrastructure..

B

I just spent up in azure function and point my event grid subscriptions to those functions or those logic. Apps.

B

And I can get up and running incredibly quickly..

B

The second big scenario we see is ops, automation.

B

Right. so, we've integrated with azure resource manager.

B

So that anytime, a control plan event happens within azure, azure resource manager emits events as well.

B

so,. We can basically subscribe to events on a resource group on azure subscription when a vm is created or sequel database is deleted, and we can point those events to any endpoint where we're running a script or a function, and do things like check for proper protocols,.

B

Tag the vm,, whatever needs to be done, there.

B

And then finally,. The third big scenario we see is general purpose. Pub/Sub.

B

so, we fully support--.

B

As we've talked about it, a little bit,. You can have an on-prem system that is using.

A

Event grid to publish its own events and consume them on-prem or those hybrid scenarios. We talked about as well.

A

right. so, if you're more familiar with one of the other clouds,.

B

What would be the analogous service that is in that cloud and does it have the same feature set as event grid?.

B

Yes. so,, if you're more familiar with google,.

B

The analogous service there would be google's pub/sub.

B

Google pub/sub is just a general purpose: pub/sub engine.

B

It doesn't have filtering on the wire like event grid, and it certainly doesn't have. Those first party integrations that event grid does.

B

If you're more familiar with aws,.

B

Their service over there would be sns, simple notification. Service.

B

They have recently introduced filtering on the wire.

B

And this is actually a pretty cool feature that both event grid and sns have is basically, you're, publishing events and you're, sending them to multiple different event, handlers that are subscribed to these events..

B

If a certain endpoint doesn't want to consume.

B

All the events before on the wire filtering,.

B

It would have to filter at its own end, point, you're, wasting cycles and compute.

A

Now, we filter on.

B

The wire, so you.

A

Only listen to specific event types, if you want.

A

great., and I presume that there might be a latency in there as well?.

B

Not, not really., it's in a milliseconds,, not something that customers will notice. okay.. It's more of the comp cost. yes. cpu costs. okay., yes.

B

well,. There would be latency if you were filtering on your endpoint and you had really high throughput,.

B

We scale out incredibly well horizontally, so we don't see latency issues.

B

sorry,, I mean on the other side. sorry, yes. there,. If you, in high throughput scenarios, you do start to see latency. yes. you get clogged up., yes., definitely., yes. and the other part of event grid is.

B

Those first class integrations that we have that doesn't exist in sns.

B

If you wanted to do something like that,, you would have to first use a lambda trigger and.

B

Then pipe it to an sns endpoint and then go from there.

A

Event grid instead,. We are trying to be kind of a uniform, ubiquitous eventing substrate., connective tissue. connective tissue. definitely.. Have I convinced you? yes,? I am definitely convinced.

A

so,. I want to go back to this vm scenario that you mentioned,.

B

Like I said,, I think the functions.

A

One is pretty straight forward., so I haven't done this particular scenario, so I'm going to kind of make something up, and you can tell me if this is true.. I kind of imagine that there's these applications that.

A

People have built that do need to create vms who haven't found the one true way yet.

A

With event grid.- and they get to this point- where they have to create a vm,, they actually have to have another vm running that actually kind of manages this..

A

It creates the vm through arm.

A

And then kind of like connects that vm and has a script that runs,.

A

That makes sure that vm is created. Correctly.

A

It feels very procedural.

A

And very kind of like,.

A

Both has to create the vm and then configure it and potentially has to have this other vm running all the time.

A

With the thing you mentioned,, it seems like it's much more reactive that.

A

Maybe there's vms created of one type that may be.

A

Configured one way and then vms.

B

Of another type that need to be created in another way., and then you just have these listeners that do the right thing in the right situations.. Is it kind of like that?, yes,, pretty much.

B

even before event grid,, we did have azure automation in azure.

B

So you could run your scripts there, but you still have the problem of your need to know that a vm has been created..

B

You have to actually send something to that script and trigger it..

B

So with event grid, now,, you don't have to have prior knowledge,.

B

You don't have to make sure you're hitting the right script.. It will just listen to all control plane. Events coming.

B

In to whatever resource group or azure subscription, you pointed at. and anytime, a vm is created,. You don't really have to do. Anything, it'll, just.

A

Trigger that automation, script, it'll run the script and tag the vm,.

A

Send the message to a slack channel,, whatever you really need., yes. teams.

A

teams.. Can you tell me,.

B

Some of this feels a little similar to queues, which is obviously thiis existing feature in azure..

B

Can you compare and contrast? totally, yes. at a high-level,? The big difference is that a queue is push-pull, semantics and event. Grid is push-push semantics..

B

The other big difference is that queues,.

B

We put them out there as a general tool, and you write your own code to push things to it,. You write your own code to pull from it,.

B

While you can't do that with event grid., you can get up and running and use our first class integrations without needing to do that.

B

in fact,. One thing we're actually working on is.

B

A queue integration for event grid, so you could push your native events to a queue.

B

so,. If you have a scenario where you're not up.

A

All the time to receive these events,, you could wake up,. Look at your queue, say, oh, event grid has pushed some events to my queue,.

B

I'll pull them now and consume them.. It's a little bit like when you land in.

B

The plane and turn your phone on and then all of a sudden, like all these text, messages, so yes.

A

Totally, yes., it also works for buffering scenarios. If you can only handle certain throughputs for spiky workloads and anything else.

A

makes sense. yes. so,, I'm guessing that. There's a visual studio story for this., yes, absolutely. and maybe even for.net. so,. How about we take a look?.

B

I think you've got a sample here. yes., just before we dig deep into it,.

B

Can you show me? Is there a nuget package that people can look for?, so our nuget package is available on nuget.org.

A

We have both a run-time and a management nuget packages.

A

Right., so this one is called microsoft.azure. eventgrid. and,. Can you click that "dependencies" now so we can see? okay,. So it looks like it's supported to all the way back to .netframework, 4.5.2 and there's also a.netstandard library.

A

So, that will work on.

B

Also.Netframework but.netcore and xamarin.

B

So,, that seems to be covering a lot of bases.

B

Nice. okay.: let's look at visual studio.

B

Yes. so,, what we have here is a pretty simple sample,.

B

It's just using our published sdk., it's connecting to an event grid topic and a topic is.

B

Just your entry point to event grid if you are publishing your own custom events to event, grid.

B

And we're just publishing a very simple new employee event.

B

That I have here created.

B

we can go over to the azure portal and.

B

Create event subscriptions on this topic: we're publishing to, point them at a function.. I can tell you how quickly this happens in real time.

B

yes,, please do. and we can actually go consume. Those events.

B

So,, this is the function that we're interested.

B

In sending our events to and handling them, from.

B

So we just need to endpoint url for this, I'll copy. That- and I have in this tab- are event grid topics..

B

So this is just a dns endpoint that we have the keys to,.

B

To send to our events., I create a new event. Subscription., I can say it's "netevents".

B

I give it the endpoint that I want to aim it at and I hit "create".

B

and as soon as that is done, deploying,.

B

We will see a new event,.

B

A validation event of pop up here.

B

So,, the validation event is our kind of security measure.

B

What the validation event does is it's-- so event, grid, in a way,.

A

Because we have a massive fan out.

B

Capabilities and very high throughput, is kind of the world's largest [inaudible].

B

So,, the first thing that our code is going to do here is.

B

Actually handle validation event that we send every new endpoint that we register.

B

So, we're just going to look for that validation event and when we receive,.

A

It we're just going to echo back the validation, code.

B

And that just means that I own this endpoint.

A

Is it a kind of 2fa for event, grid?.

A

B

Okay. definitely. so., it seems like we have to use 2fa for basically everything these days.

B

yes. so, our script.

B

Here is set up to actually handle a number of different event: types., I'm just using a-- if check.

B

To see if we're having a validation, event, a blob created event or this new employee, event.

B

So is just a really simple function.

B

That could handle a bunch of different event types being thrown at it.

B

so,. If I open my log and then I go back to my visual studio, projects,.

B

I can just hit this endpoint with an event that.

A

We're spinning out and we'll see it.

B

Appear pretty much instantaneously on the other end.

A

Okay,, so this is the side that's sending. The event. is that correct?

A

yes. we're sending the event from visual studio..

A

A

You might have the console locked by that block?.

A

Yes, I've just started it down.

B

Sometimes, when you select-- yes, there, we go. that's totally what happened.

B

so,. We should see an event.

B

We have our own subscription here.

B

Everything is configured correctly?.

B

Yes, just checking out if everything's configured correctly..

B

These are the events that I had published. Earlier.

A

Interestingly enough,, there is some reason we're not getting an event through.

B

Hopefully we're connected to the internet, here.

A

B

Just create a second event: subscription.

B

right. so,, we're demonstrating the diagnostic kind of patterns that you would use for. This., yes. so,. We have our endpoint here.

B

Let me just make sure,.

B

Yes, I'll just overwrite.

B

That old endpoint to make sure that we have the right one.

B

save the changes., it has succeeded. So we.

B

Should be getting a validation event.

B

Right now and we're not. just interesting.

A

Yes,, let's run the client side code.

B

Yes., let's see if I got it, running.

B

Okay, there we go. so,, we just got that event. Through.

B

So maybe, you just somehow,: you ran the client side wrong, the first time?

B

yes. either I had run the client side wrong. The first time or I just didn't properly- connect it.

B

So, we should now see,.

B

I think I might just not copy the full url or something.

B

If we click on "metrics" here,, we would see the way we diagnose this is,.

B

This has a little bit of latency,.

A

We're working on improving that, but soon,.

B

If we wait a minute or two,, we would see our successful and failed delivery. Count.

B

It will show up on this graph., but all we did was we matched.

B

Our event type as our new employee event and we wrote to the line,.

B

To the console,, their name,.

A

Who they're reporting to and their email here.

A

obviously,? We could do much more interesting actions on receiving an event.

A

totally,, but this kind of demonstrates the fundamental., the fundamentals, yes. yes. awesome. right. so,. This.

A

Is obviously live in azure.

A

people can go and use this today.

A

yes. so,. They obviously.

B

Would download the nuget package, but from an azure standpoint,, do they create a new event grid subscription??

B

Is that kind of what happens? yes. so, there's a couple of ways to use it.

B

So if you want to send your own custom, events,.

B

First, you would create a new topic, and this pretty much just gives.

B

You your entry point into a event: grid., you just give it a name,, a research group.

B

You want it to live in and what location you want it to be. In. and now,. You can send all of your events from different sources or a whole bunch from a single application to this endpoint.

B

and then,. Once you have that topic,.

B

You can go create event subscriptions on it.

B

The other option is in,.

B

If you go to a storage account, for example, you'll, actually.

B

See event grid as an option in your "resource" menu here.

B

So that would be that blob scenario. You talked about earlier., yes. exactly., and here you just create new events. Subscriptions., you again point them to whatever endpoint you want and you click "create".

A

And now, anytime, someone creates new blobs or deletes them in this storage account,.

A

You get an event fired to this endpoint. Every single time., awesome. yes.

B

Okay,, I think we've learned.

A

On quite a bit about event, grid today..

A

I certainly think I've picked up enough knowledge.

A

That I could use that amazingly enough.

B

Any closing thoughts or resources that people can look at that you'd like to share? yes. so, azure.com/eventgrid takes.

B

You to our product page, where you can view our documentation,, learn more about it, and we're very excited for where we're going in the next year or so..

B

We're working quite closely with a number of people's favorite product teams. In.

A

Azure to have more and more integrations.

A

So,, it's only going to become a more powerful tool as we go.

A

right. awesome. okay,, thanks for coming by bahram and telling us about event grid..

A

It has been another episode of the on.net show. thanks for watching.