AsyncAPI AsyncAPI Talks, 10 Jun 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Live Coding: Let's Talk AsyncAPI and Code Generation

Description

Video markers below! 👇

Join Solace Dev Advocates Tamimi and Marc and Fran Méndez, Founder of @AsyncAPISpec, for an interactive live coding session. We’ll check out the design-to-code experience using Solace PubSub+ Event Portal and the AsyncAPI Code Generator to build event-driven microservices using Spring Cloud Stream.

00:00 Countdown
2:15 - Introduce Fran Mendez & his cat
3:35 - Introduce Marc
5:00 - Q/A
7:15 - Journey to finding AsyncAPI
10:55 - How is Solace supporting AsyncAPI
13:40 - How are organizations and devs using AsyncApi
19:45 - Talk about the AsyncAPI Generator
27:30 - Marc gives an overview of application
31:25 - Design architecture using Solace Event Portal
52:00 - Generate Spring Cloud Stream application from AsyncAPI spec
1:17:34 - Explain loosely coupled microservice
1:21:15 - Start NodeJS microservice
1:25:55 - Fran explains AsyncAPI components
1:36:20 - Using validators in AsyncAPI
1:46:00 - How to stay up to date
1:49:00 - How to contribute

If you have any questions, join the Community! https://solace.community/ and follow @SolaceDevs on Twitter

Find the code at: https://github.com/mrc0113/live-coding-asyncapi-codegen
Sign up for your free cloud account at: bit.ly/try-solace-free

A

Well, hello, everyone and good morning, good afternoon and good evening to anyone who's watching us live right now on YouTube and on twitch I'm, so excited to be your host. For today my name is Timmy me and I am a developer advocate at solace today. This is gonna, be our first event. In our first episode in a series of virtual events, we're hosting at solace.

A

We have two types of virtual events: we have a live coding, events and workshop events, and today's live coding event is going to we're going to be talking about async, API and learn how we can design and generate code to build an event-driven application. I'm so excited to have this event today and we're gonna have everyone to be interacting with us online. Just a few housekeeping things do to say: we have a pool we would like you to fill in.

A

You can see the QR code and the the link down on your screen so feel free. Please pick up your phone scan this QR code and fill this bowl. I assure you'll take less than 12 seconds, even less than 10 seconds to go through the questions on the on the poll. The reason why I wanted to do this is kind of to see where we can find you a better and kind of like tell you about our upcoming events. So without further ado, I would like to introduce our amazing guests. For today.

A

First, we have the founder of async API, our friend Mendez who's gonna, be joining us to today in the in the Caudill. So I would like to introduce friend to the call with us.

B

Thanks for inviting, as I was saying that you'll see, you will see me here today, doing some nasty coding things and with the soleus, folds and and probably with my cats, because you probably might going through this.

C

B

Now trying to take attention so I'm.

A

So excited to active session, where you know, google chat with you and your cat, you and your cat on a couple of things. um Next, I want to introduce my tour manager, developer relations team, so Mike is in here and Mike. You also have.

C

Around me as well sweetie.

A

C

To kick off this first episode with yourself and with Fran joining us, obviously from async API and for anybody doesn't know. I'm a developer advocate at souls with jhemini based out of Orlando Florida.

A

Fantastic thanks thanks for joining before we get starting with the you know, the ice cricket questions up a couple. Questions I'd like to remind the audience. Whoever is watching us right now on YouTube and Twitter. You can ask any questions, so your audience questions are accepted in chat, so feel free to put some comments in your chat, whether you want any questions, emojis or anything.

A

You want to tell your to tell me or mark or friend, just type it into the chat and we'll see it so before we get started, I wanted to ask a quick question to Fran and mark what do you guys do outside work outside async, API and outside so let's go friend I would like to know what do you do outside async API before we delve into the nothing goodies are facing API.

B

So far don't do much. We can not go out. We can already go out. Just kiddin I usually like to just play volleyball, mostly indoor volleyball, but now it's gonna be soon the time for beach volleyball as well so ice.

A

B

My other passion and I'm terrible as well involved volleyball.

B

Afterwards, right, yeah.

C

B

The part that I.

A

Like them, I go for the VI yeah.

C

So for me, it kind of kind of the similar thing right, like I'm, a sports person I like to watch sports and I like to try to play sports. It used to be good at basketball, but you know it's been a while and I like going around to it once we open back up I like going around some of the colleges and traveling stuff. Traveling is a fun hobby for me, so kind of a shutdown by the pandemic right now, but hopefully that will go back to normal soon. Oh man.

A

Okay, well thanks for telling me you guys do outside what your day-to-day work is.

B

Sorry to interrupt, but you have to tell us yeah.

B

A

On my free time, when I'm not doing like work, apart from just like hanging out with friends, I I picked up DJing recently, so I've been like a bunch of DJing so because I'm not going out I've been DJing at home, practicing a bunch. So this is something that I'm like really interested in I, really like music. So yeah. That's that's what I do outside solace so.

C

Over the air, horn and clapping came from exactly.

A

I, don't worry, I just put like air horns next.

B

Event will have to me on the.

A

Fire and mixing beats.

A

um Well, did you to me me, that's a good segue for a next question. Oh friend, um miss question is for you I'd, like to uh you, could give me like an overview on your journey to finding a sink API, and why did you find a sink API and what's the gap that you were trying to fill with um finding a sink API and making it like a big issue.

B

That's a good question: I actually I, never thought that I was going to found I, think API or anything to me. I was just working on a company, it was called each. It was an API change law company and, as it was an API chase, look working with open API, mainly right, so we were used to open API for everything and and we had a distributed architecture with RabbitMQ in the middle, so yeah. So we have an event-driven architecture.

B

You know API, as a change law is actually a series of events right, so it made sense. So what happened is we didn't have such tools that we have progressed api's for even driven ate the ice right or even micro-services? In this case, and therefore what I did was: let's try to adapt open API for our use case when I say get, it means subscribed when I say post is a public.

A

B

Did the job less for for documentation, but not for cogeneration, and it was like a big hack on top of open API right right.

A

And this what's the timeline: are we talking about it? Three.

B

Years ago yeah it was 2016 4 years ago. Okay,.

A

B

And then I asked the open API people like you, have plans to include other protocols and methods and stuff, and they say like no way man, it's too much it's already too much as it is and handles them. So yeah I decided to just hack myself, open API and remove what was needed and add the needed staff and created it's in KPI, but I. Don't I, never thought it was going to be used by anyone else. Other than me like, but my default always was with with a thing.

B

Keep bearing with any project, and you can see my my github profile is I was always doing everything open-source by default. Okay, unless there's a reason to keep it private yeah, we gonna reason something. So it was also open source and I was speaking about it on own events, because I thought it was. It was a cool idea that somebody had to explore. I, never thought it was going to be me how.

C

Much how much work it really is. I.

B

Wasn't really really like this is too much. Nothing I could do this in my spare time. Famous last words.

B

C

B

Up the reason of my spare time until it was that's.

A

A nice thing about open source, you know sometimes, like you just put things open-source and then you start getting support from the community so, like you found it a sink API in 2016 and fast-forward.

A

Now you have a bunch of different organizations, and people are supporting a sink API and contributing to the open source repositories on github, and you guys have like a very cool website right now, and this is, this is kind of e than the nice thing about this I know like I, wanted to kind of see when, when did soulless kind of like get into this timeline, so mark to you like how is full of supporting the async API initiative, an organization and like how did it like through this timeline, starting, they said 2016, where, when it SOLAS get involved in this and how it still is supporting basic API.

C

Yeah yeah for sure so, I don't know the exact times. I have to go, go, look that up, but I kind of had a similar background myself a little bit to fran and that you know I worked for large organizations in like the aviation space and the weather space in the healthcare space. And you know all of these large organizations were global and we needed that.

C

You know if we had event-driven systems because we needed to keep up with the real-time data and make real-time decisions, and we came to the kind of the same conclusion at those organizations in that, like the tooling, wasn't there that we needed, and so in some organization we tried to adapt service registries that were available to kind of like store your topics and other organizations. We built custom systems to kind of handle what async api does now, and so you know that kind of drove me myself.

C

Actually I was a customer of Solace and a user of other messaging systems before I worked for solace, so I came up and work started, working in solace, I think it's been about a year and a half ago now, when I heard about these all said that portal and the initiative that they were working on with ace Nvidia and so so since then, I think for my solace perspective at least since I joined I know you know, we've been contributing the documentation, the helping you know Fran and the scheme figure out, maybe where to take the spec in certain cases, giving our opinion.

C

Of course, you know one of the cool things for us is code, generators that we'll talk about more in a little bit. So we one of the members of our team, Michael Davis was the lead developer on the spring cloud, dream code, generator template and then, of course, we're doing other things like sharing via conference talks, this virtual developer workshop and we've also done some hands-on workshops with a Cindy I. uh You know with some of our customers and then of.

A

C

Think Fran, let me know if I'm wrong, I believe solace. You know, as one of the platinum sponsors is still the largest. We have the largest amount of funding towards the initiative.

B

Through open collected, yes, I'm.

C

B

C

We're also funding.

C

Funding and helping you know: Fran work his magic on his end and push forward nice.

A

Yeah, no that's great and I see. People like are commenting on now and twitch Aaron is like he found a syncope about three to four years ago and surprised. Nobody else was talking about it and he's happy that it's going thanks, Aaron for the comments and.

C

A

Impotent on this speaking of that Fran, how our organizations and developers using async API like yes facing API now's, an open standards and open source- and you know it's defining the direction of where asynchronous development is come, how our organizations the app like any examples on how organizations or developers even are using async API. No no days yeah.

B

We find like a very broad set of use cases if you want, mmm for instance, you have very you- have a lot of use cases more than I thought there will be about using you know using WebSockets right, API, it's public WebSockets api's like it's the case for slack, for instance, and an HTTP to server sent events, and when I curative is in kpi. In my mind, it was all going to be around a Q P and Q T Kafka at the time was loading but I never thought about this.

B

Other protocols right and patterns right.

A

B

I see I, see unexpectedly, I see a lot of people using the request response, not inquest response, but more like the client-server type of architecture with messaging like in the case of WebSockets or server, sent events right. Where you don't have topics, you have a single connection and everything happens there and and then you also have these other kind of companies and organizations. You can hear my answer, so you you have all the other kind of organizations that use it for something completely different.

B

That's on the on the technical aspect, let's say on the business aspect or use cases more or not. Super related to the technology will be some people. Do it to coordinate it or use it to coordinate the teams. You know develop teams of develop of the characters and agree on documentation on the message format and on the fields that you can find their respect their you know as a developer portal. If you want right, yeah.

A

B

That's to me, that's the main, the main use case aside from providing a public, a public website where people can consume your documentation right in the case of Internet of Things, for instance. This is very common yeah.

A

You bring in master, you mentioned it very good, like you know the documentation where teams could use it to like document their asynchronous application, so when I first got introduced to async API. So what are the things that you would expect? The first thing that came up to my mind is swagger, an open API. You know like how swagger is used. An open API is used to define your rest api's async api. Is there used to define and document your asynchronous API?

A

You know so the way that I first thought kind of when I, when I was introduced to a security a couple months ago, so not so long. The the way that I was able to understand and wrap my head around is like all right, there's a rest way of doing things using you know open, API and swagger, and you could document it. You could generate code like REST API code from your swagger aspects and the same idea.

A

There is a sink API that you can like document and kind of have like a set standards on your event-driven application, who not.

B

Only yeah that's.

A

B

And- and you mentioned won't say, is that I think that the the the easiest way to start using async API in through documentation, safe right, it's it's very safe. It doesn't affect your production systems and it it's useful. It's already useful for your teams right, but we're seeing more and more demand for people wanting to use async API for management capabilities right. So you know like, like you, have rested guys, gateways yeah powered by open, API or swagger descriptions, yeah.

A

B

Thing right so gateways that sit in the middle between the consumers or the producers right.

C

B

The broker and and basically apply policies to the messages.

A

In the make that documents, how you want the whole like the flow of the messages and how the messages rate to each other and what are the different kind of schemas, are beyond dummy messages: yeah, okay, yeah! No, that's great! You had your first async API conference a couple of weeks ago, yeah.

B

Regulation, that's thanks. That was crazy.

A

About like how people were very happy about that conference and I was so impressed how it was all done within what three weeks unless to.

B

Two weeks of health, yes weeks.

A

And a half, then three weeks you have to be precise, it's not at.

B

The we must say something we have nothing else to do. We were in a confinement like we're in a lockdown, so we were actually bored at home. It's like I, don't want to think more on the on the lockdown. So let's do something to forget about that.

A

B

That was a little bit of excuse, yeah and also at the time we thought that this is probably gonna end up soon, it's gonna be finished soon, the lockdown. When all this they make and all this stuff, we should do it quick right. We do it in two or three months from now. Maybe nobody is gonna tenth right anyway,.

A

Organized so well.

C

Done on that, you know it was crazy. Cuz I, remember like it was on a like a Tuesday sick meeting. We're like hey is interested and then by Thursday like the website, was up well.

B

I mean you know.

A

You did mention in your talk Fran during a sink API conference that async API you're working on a bunch of different tooling there's a bunch of different tools with a sink API, one of which is the generator and I know. My kind of like mentioned touched a little bit on on the generator. Can you talk about tooling and in specific about the the generator as a tool to be used with with ASIC API cuz? This is kind of what what I'd like has to kind of like focus on today's session.

A

So tell us more about your like the tools and.

B

A

With a syncope I, please sure.

B

So I think our our way of thinking about async API. In the beginning, it was more about the spec. Only the spec and the suspect was the primary thing and everything was running spec, but more and more we're seeing that as it's normal. Nobody cares about the spec right. It's saying: yeah people want to use tools.

B

It's like just give me give me something that tangible right, something that I can that I can use and yeah it's powered by the spec and all the stuff, but I don't want to learn the spec right all right, I'm, not in that precisely I might have to learn the spec, because I want to use that tool right. It's not it's, not the elbows, it's not the other way around right. So so we started thinking more on the use cases that people and and taught them to different people.

B

What were they use cases they were expecting from its in KPI and and also I, didn't define the the easiest part that parts that we can start working on and again the documentation generation cogeneration seem to seem to to ask like the the easiest want to start with that it provides a lot of value, and so we started there and that's generate there.

B

Actually, so generator is it's a tool, it's so it's, so it can be so generic tool if you think about it, that it works for its Inc API, but we with varied a little tweak. You can go for any other specification, so what it does actually is. It takes a sinc API as the input and the template of files. You know a set of files as that's what we call a template and it generates a result or a set of files right, and so, with this system in place, you can generate ducts and code.

B

Of course, you can generate whatever you want and, as I was saying that we were even joking, the other day, Lucas and I that, with very little tweak there, we can even make it work with a pen. Api ramble, JSON schema protobuf, whatever we want.

A

B

We can, we can add more rendering engines as well like we're now exploring on making it work with react and yeah.

A

Yeah I put that up. That's it's.

B

Like it's super easy to black new stuff in there yeah it, we probably that will not go in this direction, because then it will become something like very generic. Yes,.

A

B

Gonna be difficult to explain, but but yeah it's like this is more or less the main tool that we have and and bila or behind this there's the parser right. The parser is the one taking care of reading the file. They sing, keep a file and convert them into a model right that you consume yeah I.

C

Think right now all right so that, basically, when you use the generator you choose which template you want based on what you want to generate and so I think right now: there's generator templates for nodejs, yeah, Python, Java, spring cloud stream and then from a Doc's perspective, I think it's HTML or markdown right.

B

Yep so there's another Java Java, sprint Java.

C

Springs well, okay, yeah! So today, when we jump into the live coding session, we're gonna be looking at the Java spring cloud stream, template and then I believe the node.js template, or is it the node.js WebSocket template or a night? If no cheese.

B

Template is a generic one. Okay, we're gonna be using both you're gonna, be using that you have sprinkle.

C

B

B

A

Is great thanks for, like touching on the the code generator and the kind of need, a polyglot support of multiple different programming languages that it supports before we jump into the live coding, because I'm, pretty sure I'm excited to get to the live coding section of here? I just want to remind anyone who just joined us right now on YouTube and twitch to fill in the pool. So you will see we have the QR code at the top over there.

A

Next to friends, fitties goofy, yeah, there you go friend thanks for putting to this will be. Do.

C

A regular Vanna, White yeah.

A

Or you could just go to this link, you know five seconds. It's just two questions that you know like. Where did you find? Where do you get about this and what's what's role so filling this this pole and we could go ahead and jump into the the core of what we're going to do today with live coding before we do that friend, a quick question for you: did you ever go to New York City before no.

B

A

You did, he did well. I was a conic thing that you've seen in, like you know when you do when you think of New York City. What what is things that, like pop in your mind, you.

B

Don't want to know.

B

Now I can tell you a brief, like.

B

Good about the CP that happened to me first time, I've been in New, York City. It was like, like 12 or 13 years ago, and it was to participate in a break dance contest.

B

Yes, I was the professional break dancer like yeah like around 15 years ago where and we were also competing in Los, Angeles, so yeah so.

C

Now we know what can happen if we're gonna take a break in a little bit and then friends, gonna breakdance I.

B

Mean is gonna, be a.

B

That's what happens when you so much.

C

A

To know one more thing about you that you were like deci New York, one other things that I think of as well when I think of New York is like they have a lot of taxis over there, and you know when I'm gonna watch like movies and shows about New York everybody's standing on the street and hailing a taxi, a yellow taxi, yellow pad. So I will let mark introduce our application about India in the mean time.

A

So there is the QR code, you could check it out and you can join our solace community as well in the solace care community, we'll stay up to date with what we're up to so you can see down low. We have the solace community website and you can also use your phone to scan the new QR code up there too. Yes, thanks fans to to.

C

A

It's always like this is what we have and without further ado Mike. Can you please introduce us what we're going to be doing today? What is the idea that you have in mind that we're going to be working on and, in the meantime, Mike yep go ahead and talk about the application audience? If you have any questions, comments, suggestions or anything feel free to put them like ask ask you, questions I will be monitoring and we could either and highlight you a question on the live stream. Your if.

B

A

Lucky one through the chosen lucky one.

C

We had some more than one question yeah, so if you want to share my screen so Mimi sure, and so essentially all the all the code route, this livestream we're gonna go ahead after and throw in this github repo. So a lot of coding, async API code gen, but what we're gonna do today, since you mentioned New York, City taxis, essentially we're gonna. We're gonna use some data here that the New York City, Taxi and Limousine Commission kind of made public.

C

So they they had an open data release of over a billion taxi ride, records and essentially Google, took those and said, hey. You know in the release of data they basically released where pickups happen and where drop-offs happened right and since we all know, Google knows how to do maps, they decided to map the routes of each of those rides and it made that data available. So what we're doing is we're taking that data and and publishing it into Sol's, pub/sub, plus and so essentially, taxis are sending in taxi updates.

C

That include information like the right idea. You know where they're at with latitude and longitude the meter increments. A number of passengers etc, and so that that information is being published on to a topic, hierarchy, installous, and so it's an asynchronous manner. The taxes are publishing in they don't know. If anybody is consuming this events or if 100 you know, applications are consuming those events, so in a asynchronous manner, Fran and I are going to build out two applications that use.

A

Those events markup it. What can you do me? um Maybe.

C

I think I've already all right here you got better yeah.

A

That's about it, yeah.

C

So what we're gonna do is we're gonna receive those events, and so the initial events are being published on a topic hierarchy that uses variables here that allow us to do filtering on the exact events that you want.

C

um So we're gonna, create a ride, drop-off processor, using spring cloud stream and so I'm gonna consume on this teach view topic and so teach he was basically filtering on just ride status of drop-off and then I'm gonna use a greater than sign, which is a wild card, and, and so that will get me, it doesn't matter how many passengers, what the right ideas or what the longitude and latitude are of, where the taxi is I'm going to get all of those events.

C

And so what this application is going to do is basically window those events and calculate some averages. So it's going to publish events back in to our event, broker that include, like the average meter reading and the average passenger count over a given window. mmm So for this, I'll probably use a window of like 20 seconds or so just to keep things moving along and then Fran is going to use.

C

The node Jas generator template to generate an application that listens over and QT I believe and it's gonna listen for these drop off average events right.

A

C

So I guess without further ado, we'll jump into the design here. So we have this. You know this high level architecture, diagram kind of like a business level of what you want to do right now so now, of course, we need to kind of design that, and so we're going to use this all as event portal for that so I'm logged into a saltless cloud here and so for those of you that are not familiar with Sol's cloud I'll quickly show like you can launch messaging services in here.

C

You can build your your event, mesh kind of around around the globe to decide. You know where your, where your events can go and where consumers can you're connecting to the US or to a service in Europe or in or in Asia right to consume the events. But what we're gonna use today is the designer mm-hmm and so we're looking at New York City taxi company, and so, if I go into New York City taxi company right now,.

A

Cut you off a bit, so we have a question coming in so with this architecture that you just showed us earlier. What I? Let you filter based on passenger counts, require this off the right status so based on that make a hierarchy that you showed us earlier. Yes,.

C

For sure, so so, essentially in mqtt or in any of the solo supported api as you these these slashes here, the back slashes represent different levels, and so you can wild card at an individual level or use like the greater land to get everything. And so so. For me, we're gonna use the greater than because I just want to drop off. But if I want to pass just passenger count, I could do V, 1 asterisk and then passenger count of like 2 and then slash and whatever else. I want right.

A

C

And I also want to say one thing here, as well as from a I do say filtering now from a filtering perspective, the way the sauce broker works. Is it actually filters at the broker, so it doesn't just send all those events over the network to you and then your application is filtering and throwing it on the floor. It is actually on the broker side.

C

It knows what your application wants and it will only send the events to you over the network that you have selected, but you subscribe to essentially them so I think that should answer that question. But definitely if you have fall, questions through go ahead and throw those in the chat right, yeah the problem, and so, if I go back to the event portal here, we basically just had taxis sending that taxi status update event into the portal right now. So what we want to do is we want to add our ride.

C

Drop-Off processor, so I'm going to go ahead and add that first one. So if I create an application here and it'll say ride, processor.

C

I'll start with version zero one and we'll say this is an experience, remap or a micro service.

A

If you're creating this, this application, that I will consume the the stream on the topic that you showed us in the architecture diagram abroad, yeah.

C

Exactly right, yeah so from a design perspective right, because you know a lot of times, your architects will be the ones putting in the designs. You know you want to have. You know a name of an application, an initial version. Add your description in this case.

C

I'll go ahead and add myself as the owner here and I can tag it, and so the the tags in the portal you can tag for whatever you want in our kids, I like to use sc/st just at Stanford spring cloud stream, it's a little bit shorter for your France. We don't say spring cloud stream over and over again and so that you can filter online.

B

C

Yeah exactly and then so you know in here, we want to consume those taxi events.

C

So if I come in here, I can I can basically search my catalog of events and I've cactus starts update, so I'll subscribe to, though, and I want to publish you know, according to my my Architecture from before, I want to publish kind of this event, so I'll call that event, maybe like a ride average update, so I need to create that because this event doesn't exist yet so, if I say, add event and we'll say, ride average update, and so we do have a shared checkbox here, and this will allow you to share it across application domains which is kind of a concept in the event portal as a way of like separating your organization.

C

So it could be different projects, different verticals, your business or different software development teams, and for now I don't want anybody else to mess with it since I'm, just developing it so I'm. Just gonna leave it internally to me to to modify into my New York City taxi company application domain, and for this I'm just gonna say the others event contains the average cost and fast computers.

A

You're, just kind of like designing your architecture, still not coding, you're, still not kind of like doing any cogeneration, you're just designing the overall architecture and how you want your different micro services and different applications to interact with each other. Basically, exactly.

C

And so I'm defining the event I'm, defining which topic it will be available on. This allows you to kind of have that that decoupling, that is kind of like sold to you when you go when you want to go a vent driven, you know one of the benefits is being decoupled, so you know my publishers, don't have to care where my subscribers are or what they're subscribing to, and vice versa.

C

My subscribers don't care who's publishing what so, in this case, I'm saying, as you know, this events can be published on this topic and from a publisher perspective. I'll do that and then, as a subscriber, you know they can look at the topic space available and decide which events they want to subscribe to them. And then you know, do the filtering, like we talked about before, just to pick these specific ones that they're interested in or not interested in um so I think what I said is with these stats draw-off average.

C

So I look at my picture. It's going to make sure of magic here, so we said taxi New, York, City, v1, drop-off.

C

Sorry, if you won stats drop off average, so yeah anyone stature up on average, perfect and then I need to add my payload schema, and so you know as part of my event of course, and include the payload and that's where the actual data is stored. So we'll call this ride. Average payload and it's gonna be in JSON, and so just to make this easy, it's easier to create kind of a sample message.

C

So I said alright, what's gonna be in this event, we want to know the timestamp, the average of meter we're gonna make but zoom in a bit unless ah yeah me me just likes to make fun of me, though I suck at zooming in.

A

It's also better I. Did it all.

C

Right, hello, I'm, Stan.

A

C

Reading average passenger count window duration seconds and the window ride counts. This would be kind of the number of drop-offs during my window and so I'm just using JSON schema net I'm gonna click Submit, and this creates a giraffe schema for me. So just to make sure everything looks good here, I can see. Let's see my timestamp is a string average meter reading is a number window. Duration is an integer which makes sense. I'm gonna do full seconds for my duration and ride. Count again is an integer.

C

We're not gonna have partial rides right, so the all this looks good, so I'm go ahead and copy this schema and we'll throw that in over here and I'm. Just gonna say this is a subscription payload, so I'll have to make up something there right. So if I save that it created my payload and automatically attached it as the payload schema to my event, right.

A

C

Again, I'll go ahead and throw myself as an owner here and I'll, go ahead and say about have any tags off the top of my mind, all right. So now I've created my new event.

C

It's tagged as new and I say this is the event that my application is going to publish, and so when I click Save now I can see my new application, the ROI drop off processor is created and the ride average update events that we just created is also on the graph and so now just to wrap up our architecture in the design portal. We need to add the ride. Both consumer and so I'm gonna go ahead and add that right.

A

So literally, what you're doing is you're just kind of like still again kind of defining your into end architecture. How you want is that what are the different microservices that are in your architecture and how they interact with each other like do they subscribe or produce to subscribe or produce any events on particular topics and your you're just defining every topic and what are the the payload schema for for the topic? So it's all like right now, we're in the architecture, design, phase, yeah.

C

Yeah exactly so, you know I'm, adding my applications in and I'm choosing which events they subscribe or publish to okay.

C

So the reason for this one is because ahead of high level architectures now I need to define it further in one place that we can actually develop that right, but also because it's going to allow me to make that development process quicker, which I'll show in a second via the use of a sink API of course, and it also allows me to really visualize my architecture because um the so let me go ahead and grab this tax and event so I want to the ride average updates when I was subscribed to here, and so, if I save that we add this new event or the new application that subscribes that event, but because these applications are decoupled, it's hard to sometimes understand.

C

What's going on in an event-driven architecture, you know, historically, you have events flowing over your broker, whether it be solace or IBM, MQ or Kafka, or a rabbit, and you kind of go to the middle, our team and say alright who's sending what or what am I receiving or you have to go, dig up like old documentation. But in this case, because this event Perl tool is easily adaptable on. You can have your architecture in here and you can collaborate whether you're, an architect or a developer, with each other to really understand.

C

What's going on so, let's say as a developer, you know if I need to develop an app now. I can of course, come in here and say all right where I drop a processor I need to develop this on. So I can come in here after my architect to find it. I can see the event I subscribe to. I can see the event that I am publishing and I can dig into those events, and you know see the schema information and the topics that that it uses.

C

So, of course, I can go develop this by hand manually, but I don't want to do that right as a developer. You know I'd like to do things the most efficient way, and so what I'm actually gonna do is there's an async API option here if I right click and I'm going to download the document as an Amal, and so we just quickly created our architecture in the event portal and I can export an async API specification file, and so what this specification file gives me is the schemas that define my payloads.

C

So I've got the right average payload on that we created and I have the taxi status, update, payload, which already existed, because the taxis were already sending updates into our broker.

C

So I have those two payloads that if and those payloads are attached or messages, so we have to up the rat ride, average event or message: ride average update event or message which references our payload for the ride, our payload and the same for the taxi status update, and so we have information here that we can that basically defines all our messages.

C

So we've got those two messages and then we have the channel information here, like it's taken from the portal and dropped in face and KPI document that defines you know where we can receive or or send those events to, and they of course, reference. The messages themselves anymore.

A

More information from the portal so like right after you designed your architecture. All we had to do is you have like a different application architecture. You right-click and you download the async API spec. That has the definition, the async FBI definitions in in there exactly.

C

So, like I was able to use the event portal tool in a more like human, readable, human access accessible. It just won't use your friendly way to design my architecture and now, of course, machines. Don't have those same. You know they don't care what it looks like. They need a machine readable way of basically processing your architecture of your application and that's what AC API provides here. It's we can export this in a way. That's not super nice for a human to read.

C

We may be as technical humans, but a non-technical person will look at this. Like you know what the hell is that, but no we export this ack pyar document and now.

B

Using something better about this in kPa, no, no just in front.

C

Of me, yeah, yeah, yeah, mole and JSON is not it's not human readable right likes regular sentences, a non-technical person would read, but this is a machine, readable specification rate, and so it's cooling for broader basic API I can now use this and myself as a human kind of like Fran was actually saying earlier. You know the specification, isn't our favorite part? Look we want to use the tooling. We want actually do things with it. Not this like fill out a specification and then end our day and have a drink.

C

Yeah I can read all the specifications yeah like.

B

People like people do with kubernetes nowadays.

C

A

B

Backbone of everything, but right.

A

You bring up a good point friend like it's like you know, the specification could be is used as a back one of everything, but it's not be in a process. It's it's part of the journey. Not the destination. Yeah write this on a spare or something a.

C

New shirt in the in a syncope, our swag store it's one step along the way. Now it's a big step, it's stuff, that a lot of people in organizations have had a challenges with along the way and clearly including myself at my past roles. So we have description and other information. You know about your application here so now, as a developer, I want to.

C

Actually you know develop this, and we said you know architects said, and we talk to our development team, and we said: oh, we want to do this in using spring cloud stream, and so, if you're not familiar with spring cloud stream, essentially, if you're familiar with a spring framework, it's a you know a Java framework that allows you to do a whole bunch of stuff I mean you practically do anything now using the spring framework. What spring cloud stream is built.

C

You know, on top of two pieces of that three much so spring boot and and spring integration are ways that you develop highly. You know. Micro services using spring brute and spring integration, allows you to kind of implement the different integration patterns that have kind of become best practices in the enterprise over the years, and so spring cloud trim is based on those two frameworks and that allows you to create. You know highly scalable event-driven microservices, so it's kind of Springs recommended way to create these event-driven services and one of the things that it does.

C

Is it abstracts the developer from having to learn the messaging API, it's kind of, as we talked about before, there's been a lot of tooling and a lot of frameworks around to create you know restful api and synchronous API, so you know rest and soap and other things before that, but not as much for event-driven it. So you know with the combination of async a pion spring cloud stirring. That's really makes this process easy for a developer.

C

That has not done a lot of design driven things in the past right, so I want to show that and so I'm gonna use the music into.

A

The generator I just want to remind everyone who just joined us right now that your questions are, if you have any questions, feel free to ask them on the chat, whether you're on YouTube or on Twitch, and we also have a poll so for those who just joined us right now, we have a poll so feel free to like fill in fit on the poll.

A

You just have like two questions to ask you, so the QR code is at the top, get your phone scanning or go to the thanks friend, oh, go to the to the link below just three questions will take you less than five seconds. This is for us to kind of like no we're we're way to target our next events, so go ahead. Mike continue after you've kind of like designed your architecture and got this back. You.

C

Know you know: I got LASIK API document right that defines my application, and so now you know I want to generate my spring cloud stream, app and so I come over to the async API github organization and if I go into the Java spring cloud stream, template there's all sorts of documentation here that you know defines like basically how you can use this template to do the code generation, and so it allows you to define to see how methods will be created in your app.

C

You know how they'll be named and, of course, how to use this temple it's kind of the important part. So first we start with installing the AC API generator, which I already have installed so I'm not going to reinstall that, and then you run the generator again essentially against the spring cloud stream using the spring colostrum template and recent document. So this is the most basic way if you didn't want to specify any settings.

C

But of course you know, we want to specify some configuration options to get the most value kind of out of the generation.

C

So when you scroll down there's a whole bunch of configuration options that you can go, take a look at here that allows you to have add the actuator capabilities to your spring projects, define your artifact type and artifact ID artifact ideas generates a maven project, so artifact, ID and group ID are things that you can have generated for you, instead of how to fill them out afterwards, and so I'm gonna go ahead and use this template. um Just let me go ahead and jump over to the COI.

C

Now, I'm, actually before I, do that in the CLI. Do you want to touch on two of these configuration options that I want to use, and so one of these, let's see yes functioning.

C

So this one here, the operation, access, CS function name, and so what this allows you to do is essentially specify on your operation, so an operation being a publish or subscribe.

C

What you want, the name of the function to be that processes, events that are like from that operation in your application, and so, if you use this on it on a publish operation and a subscribe operation, the generator will actually tie those together and say: alright, I want to subscribe to one event feed and for each of those events that comes in or for some windowed group of those events that comes in I want to create this other event, and so it creates kind of a stream processor for you.

C

So in our case, because we had our our architecture here, of we're receiving taxi status updates into our I'd drop off processor and as we receive those we want to publish ride average updates.

C

We are going to do that. And so let me go ahead and draw over to our HTTP on file, so just to show you how that would work. I do an ex SES function, name and we'll call this maybe are the name of our method, we'll call process drop off bride apertures that sounds good and so I want to use that same function, name both on the public and the subscribe operation and then also on the on the subscribe side. I'm gonna go ahead and you can set up a destination.

C

So if you want to explicitly name your queue, maybe I'll call this like a taxi update queue. Let me go ahead and save that file, and so now I.

A

Am accessing it for the spring spring cloud right Cillian again, these are all the requirements that I needed from the spring cloud perspective.

C

They're requirements to kind of tune, the app, so you don't have to specify I'll show you in a second after I, generate the app. What that did. Let's make it a little bit more clear, select me, so you guys don't have to watch me type out all the parameters. I want I'm gonna copy it in here and then I will explain it. um We zoom in a little bit.

C

Alright, so essentially the AG command. So after you installed the AC API generator, the HG command is the command to run the generator and then I'll go through here and I want to output to the ride, drop-off processor directory and missing its face here. I want to use the solace binder so essentially spring cloud stream as a framework allows you to use different messaging systems under the covers and there's the concept of a binder that basically handles the messaging API is for you, so it abstracts you've developers from having to learn those api's.

C

So this generator template supports the use of of cough, go rabbit or solace. So in this case, I'm going to use solace, of course, there's two kind of paradigms to spring cloud stream, so kind of the regular imperative kind of programming, style or reactive. On my figure, what the heck try reactive as part of this I, don't have tons of experience using it, but maybe it'll make it a little bit more fun if I mess up a few times we'll see and then I want to include spring actuator.

C

That's allowed for to monitoring endpoints I've got my maven artifact ID and group ID, and then I've got my look like a group IDs. That's five here twice: I'm gonna. Take that out my java package that I want the generated objects to go into and then my connection information here so I've got my host, my username, my password and my message: VPN. So that's to connect to that Salt's burger itself and then I'm, pointing to the a to keep you gotta file. That is down a little bit the ride.

C

Drop-Off cost about you know file and then the template that I want to use so Java spring cloud stream. So let's see so if I throw that in wow. That was quick. So it generated my new files in the ROI drop off processor directory and so I can see that here and and so what this actually generated generated was a maven project. So for Java developers you know a lot of you, probably use maven and so I go over to my IDI IDE, my ete, my IDE and import.

C

These projects, I, can choose existing maven projects and, if I browse here and go to my git repo for the live coding session, where I drop off processor, it generated a pom for me. So I'm gonna go ahead and click finish on that and I basically have my job app and so just to show real quick I'll start with the palm.

C

So so in the palm here, because I specified so all.

B

This has the bond mark yeah mark. Can you please throw me into kids? Hey, we can, let's see it them. ah Yeah.

C

For sure, thanks is that butter, yeah.

A

That's weather all.

C

Right perfect: this was the outline here as well a little bit more screen space. Alright, so we got a group ID or artifact ID. That came from the specification, the name of our application itself and then the the generator you know knows how to include which dependencies you need. So it includes spring cloud and in this case we use using the solace finder, so it included. You know that's to order, so basically all the stuff that you got to go figure out for yourself, because the generator does it for you. It also generated.

C

Remember we had our two excuse. Me are two different payloads, so it generated Java pojos for lows. So essentially everything that was in our schema became.

C

You know, instance variables in our Java POJO and it created JSON, mappings cuz. The incoming data is in JSON and getters and setters to get and set those. So we can use those while we code without having to you know manually, create these classes and then application Java. Here you can give it a specific name via Kramer. If you want I, just left it as the default application.

C

Java is where you'll actually build your app. So earlier, when I mentioned, when I added the function name, the xsds functioning. This is actually the name of the method that was generated right here and so by specifying it on the publish line and on subscribe side it added. You know the taxi status, update payload as the incoming event to this method and on the ride average payload blocks of each of these.

C

As as the parameters to this function now- and this is a job- util function, function, object which is a functional interface, so it expects you know one thing in one thing out and kind of basically follows a functional interface method or a paradigm in Java. If I did not specify those things in the async, API suspect it would have created a consumer for a consumer method or being to handle the taxi status, update, palin and it would have created a separate method which was a supplier of the ride average payload.

C

So because I wanted to handle them kind of in order or under one method. That's why I included that and then before I jump into actually doing the coding piece, so you can see basically it set up full skeleton for me right and it says, add my business logic here, because obviously we can't generate your business logic. You have to add light yourself, Thanks.

A

Massena nai to generate your business logic.

B

For the next version, yeah.

C

Version 2, yeah, yeah, yeah.

B

Okay, so it's gonna, be it's gonna, be ready soon,.

C

So so, when you look at this might say what the heck there's no there's no messaging anything in here and that's kind of where spring cloud string comes in. You know it abstracts that away from you. So this is just a regular spring beam, but if you go over to the application dog am will file.

C

This is where the spraying configuration takes place, and so the generator actually generated the skeleton and in the configuration behind the scenes for you, so it wires up that function that that it created and it wires up the the channels here that you are publishing or subscribing to to receive or send events, and so the input side is listening on my cue that I named in the ammo file and the output side is outputting to the topic that I defined, and so because we are using the solace binder.

C

Our best practices are published at topics and consume from queues, which is you know why the binder, which is handling the messaging API part for you. It does that, and so install is on a queue you can basically specify subscriptions.

C

So if we remember back on our on our diagram on Skoda this diagram here we have incoming events on topic one and they have this hierarchy, so I actually want to listen on just drop off greater than so in my file here, I'm just gonna go ahead and and change the topic that I'm subscribing to you to use that wildcard and then I only get those events when I start up on so I'll go ahead and save that and so I don't have to do anything else from a configuration perspective to actually connect to the solace broker and start receiving events once I have my business logic created, so the generator don't left for me all right so now for an application perspective, you know what do we actually want to do here now?

C

We want to like write our code right, so so what we want to do, probably if you want to break with a stream of taxi updates on into Windows and then probably aggregate the events in those windows to basically collect that information together for all the events that happen in that 20 second interval, and then once we have that information for that 20 second window will calculate the window. Averages in the ride. Average payload objects. So we can you know stream modes out, so I'm going to go ahead and to do this.

A

This is where an architecture diagram, this stream of events, that's coming from taxi updates. They have like a payload and what your Spring cloud micro service would do is kind of like just consume. This JSON object like the streams and then over over a window of 20 seconds, so it'll just consume it over to a window like a window regardless how long it is, and then it will calculate the stats right. Yeah.

C

Yeah exactly so, all the events are flowing in continuously. This broad drop of processor is subscribing to only drop-off events, because you know it doesn't make sense to calculate like the average cost per ride. You know, maybe, throughout the the I, only care about the cost at the end of the ride, right like when I pick up initially I, don't really care about that cost or like the hundred or you know, fifty or a thousand updates along the way right throughout the ride.

C

I just want the end when I drop the passenger off what was the total cost, and so those are the events I'm interested in, and so this application is listening for just those events and it's gonna receive all the drop-offs in a window and it's going to process the averages and republish those out on the drop off average topic.

C

So if I go back to my IDE, the first thing we want to do is break it up into Windows and so I guess. Let me touch on one thing: real, quick. If you're not familiar with project reactor, essentially when I chose reactive, the right reactive parameter on the generator it use flux here so so flux essentially in project reactor, is a stream of 0 to n elements. So because we have an infinite number or we're gonna keep getting updates from taxis forever.

C

In this case, mi we're using a flux, and so we want to process that stream of events and we're gonna. Have you know continual windows going out so we're using a flux there as well? The other kind of common publisher in project reactor is mono. So if you know you're gonna have a stream of 0 or a stream of one event, then you would use mana, but we're gonna break the stream up. So what I'm going to do is all the returned flux we get rid of the return null and I want to window that.

C

So, if I, actually, let me just to make sure we see things as we go, I'm going to add a log and bang window on that.

C

So what am I doing wrong here ah box, it's a lambda, lambda function and so I want to do a window, and so in my window, I said I wanted to do 20 seconds, so I'll do I'll, second 20, and so what this does is it's breaking my flux up into Windows of 20 seconds and then on those I'm gonna call it flat map function to say now for each of those windows.

C

I want to execute only calculate the average I'll just make a separate method to calculate the average doesn't exist yet so I'm gonna go ahead and put that down and so for each window. I know I'm gonna have one average, which is why I'm going to turn on mono from this so I'd say calculate average I'm going to take in a my flux of my taxi face so grab that here we're calling the flux as well.

A

This is all Java magic right, yeah.

C

Java magic, the cool way, is just with project reactor. It allows you to do some of us, windowing and stuff, so so kind of by default in in a lot of event, brokers when you process advanced and indistinct manner. So, like one event, so it's fine when they're separate to most brokers and so project reactor allows me to do these things like windowing and processing those events in a window which is which is pretty cool and I wanted now I wanted to get more experience with it.

C

So I was using that as part of this, so so now that I broke it into a window and I basically created this function and down here. I want to aggravate the events that are in these windows to collect the information. So in order to collect the information I'm just going to create a accumulated class to keep that information, and so in order for us to calculate the averages, we need the ride count.

C

So we can basically total up the cost of all of our rides in a window and divide by the ride, count and same from the passenger count. So I'll do a let's see a double total meter, so total meter, and then we need our total passengers and you're never going to have partial passengers so I'll make that an integer.

C

All right and so I, don't think, read all these getters and setters I'm going to use project long block, which is kind of come something cool that's from spring, so it allowed me to well. Actually I need to add it, so this is actually recognized as a spring project, so I can add edit starters here and add in Lombok. So this is pretty. This is pretty cool if you have a IDE that you're using a spring tool to be then, and so now.

C

What this allows me to do is so essentially now in this you know I'm using a Lombok here to generate the debtor's setters and constructor. So I don't have to do that. Myself keeps my code short right. So is.

A

This kind of like a package that you kind of include with your application, yeah.

C

And automatically gets added by the compiler, so just to show you so essentially you know usually in Java. Historically, you know, if you had your your class members here, you know in my in my accumulator class I had to I would have to create the getters, the setters, the to string, the constructor and all that, but using project Lombok and see if it. Why is it not loading it? For me, okay, I think I have to actually have a clock that compiles here.

C

What do I have wrong.

C

What a a nice accumulator bike and using Lombok.

C

Mm-Hmm missing a semicolon.

C

If anybody sees what I'm missing, please feel free to jump in ah here they come pits missing, yeah poor in there and.

A

C

Maybe let's see where does this close? That close is there no.

C

So, let's calculate average what.

B

Does it say when you hover over a flat map because it looks like the earth stops there? Oh.

C

Here's my here's my mistake: there we go, I have extra paren in there all right. So now, if I import- that's great, so you can essentially see you know by default when I import these. So this is Lombok data and then I, say I want on all arcs construct. I want to create this object. You know specifying all the parameters. It actually creates. All of these getters and setters for me and I, unfortunately can't zoom in here, but I guess you can trust me, there's a getter and setter for each of these three.

C

It also creates equals objects and hashcode and two strings and stuff. So I don't have to do that myself, which is nice to keep it short. So now that I have a something to aggregate these into I actually want to return the flux, and so I want to reduce those things using my accumulator. So I want a default accumulator and I'm going to start with the zero of everything right so I have a ride count of zero before I get any events.

C

A meter count zero passenger count of zero and then the next parameter here as part of my reduce function, is to actually populate or decide what to do for each on the event that comes in in my window. So the a will be accumulator and I guess the do be like my next. A I won't use.

C

Do these tax in the update, but for each taxi, updated I, essentially, you know want to create a new accumulator object and I want Kate, hey you got it ride count so my accumulator before and I'm, adding one more ride to it for each event, ride that we get and I'm gonna get the total meter and I want to add the of the meter from the new events that tool that meter reading and.

C

Then the last thing is the taupe Asterix I want to get the get the total passengers, what I've accumulated so far and add the number of passengers in my latest taxi update on so passenger account. We add that format down a little bit and.

A

To go for this is you will have this in your payload once you publish the message to do your topics? Basically, yes,.

C

So my goal here is that I am accumulating the data, so I can actually calculate the averages so now that I've accumulated the data, I want to actually map it, and let me just format this little bit here. So it's a bit nicer.

C

No one's great yeah there we go so so I, basically it calling a reduce. So what the reduce is doing. It's taking this flux, which remember of flux, is zero too many events so I'm getting the taxi update events I'm accumulating those into a new into an accumulator, so I'm returning an accumulator object that we created here. So at this point, I have one object. That knows the total amount of rides in my window. I know the total meter amount and I know the total number of passengers.

C

So now I just want to map that accumulator. So call it simulator key here and I need to create my new ride. Average payload, and so it's part of that we don't want it to be. I.

C

Just basically need to calculate my averages now, and so let me when I create the new payload and I want to say: can you later see, I want to get the total meters? That is the first thing and then it can either go get ride count. So that's the first thing and then.

C

The second thing is the title of window, so in this case we're doing 20 seconds as we did that above and then again we want to do the accumulator at total passengers, so will it create the average for the passengers so total passengers and again the average will be the total passengers divided by the number of rides.

C

Ride counts and then, lastly, we just need to add a date to this, and so for the date we'll go ahead and just use the current date and so to do that. And if you think the top here yeah.

A

C

Format, academic.

A

Before rank, I've never used one book before, and it's actually pretty cool. How like it, does all your getters and setters for you and it just facilitates the the process. The overall process makes it more efficient, yeah.

C

A

C

Nice, because, if you, if you update things, it's all in one spot, so so I'm going to point now, I'm mapping, my accumulator and let's see, what's it complaining about the kind of because I'm doing a I need to catch my queue later to a double so I get a double returned.

C

And n, divided by an ant I believe reached him now. What is it.

C

Or it's been a double integer double integer string. Oh.

C

um It's an argument. So, let's see here, if I look at my ride, average payload the constructor actually wants an average, a meter reading, which we have window duration, which we have average passenger count which we have on missing the window. Rod cap, so the fourth parameter should be a window ride cap, so I just want to get the numerator dot, um get right count there, and now, if I can format that now we're happy all right. So we have now, if I put the comment in the right spot, so.

C

Let's look at what we did real quick. We broke the stream of taxis into Windows and then for each windows we called calculate average and then, in the calculate average we basically aggregated the events in those windows into an accumulator, and so we collected our you know total number of rides in the window, the total meter and the total passengers. Then we use that information from our accumulator objects. You create the ride, average payload and so that's what's being returned.

C

You know out of the calculate average and so for each window, we're returning a mono or one average payload and that's collected up and returned as a flux. So, of course your business logic might be different. You might also just include the imperative or might decide to use the imperative Java style if you're more familiar with that, like I am but I decided to embarrass myself in front of everybody by learning something new and so now I've added my business logic.

C

I can go ahead and just run this as a spring boon app, and so theoretically, this should just start up and connect and run, and so we can see because I added the dot log options for every taxi status, update payload that comes in essentially internally. You know, flux is getting called with a a on X. So for each event we get a non next, so we can see those words coming and then after 20 seconds we should see the window and there we go.

C

So we received events for 20 seconds and we see our motto being created here, where it created the rod, average payload and it you know we calculated our average meter reading was 1414 73 over that time period for the window, duration of 20 seconds. We got a ride, count of 6 drop-offs. During that time and a timestamp you know tagged for our events and so just to make sure that I'm getting those events.

C

Let me go check it out in the Tri meet ABS, to show that and I'm using a instance in Azure and so in. Try me I can actually subscribe to those events, and so, if I connect and my topic so.

A

Now your app is running your micro services running mark right and your.

C

A

And it should be publishing to the topics that you defined in your micro service. Now, what are you doing? What's the try me half that you're you're, showing us yeah.

C

Exactly so so, essentially you're on the app in in sauce cloud allows you to connect and publish or consume events are flowing across a given broker. So in this case I'm now publishing my events on the stats draw both average topic. So just to just to prove to myself. Like my I know, my apps running and I'm, seeing logs but I, want to make sure that it's actually running before I hand off to phrase nobody's.

A

C

These events, yeah I, don't know you know. I'm a developer advocate I code quite a bit, but um you know it doesn't mean I, don't test my code. I've got a production, maybe.

A

C

Might go to production. You know you never know all right. So I want to listen on this topic, so taxi New, York, City view on stats drop off average right, so tax, New, York, City, d1, stats drop off average, and somebody scribe to that.

C

I should get something like every 20 seconds here and there we go so I can see that I'm getting an event, and so at this point you can see that how basically how easy it was for me, go from design to using the AC cameo dinner to create the app skeleton and then I had to add my business logic, which for me was not as easy, because I had to use how to learn how to use projects reactor a little bit, but once I got that going, I, basically just hit run and my microscope is running receiving the events from the taxis and publishing the average events.

C

So we've now, you know created our first micro service using spring cloud stream and they said yeah generator. But now you know if I go look at our architecture here in the event portal. You know. That's that's the first piece, so we have the taxis sending the events in. We have the ride: drop-off processor, creating the rad ride average updates, but now we want the ride, drop-off, consumer created, and so in the description here we said we knew that Fran is going to develop this and you know Fran's, not a Java guy.

C

He likes to use node and other applications. So we said this is a node.js application and you know that's fine, because the smallest broker allows you to use different different languages and different protocols and EPA right I'm using spring cloud stream direction. I can show you really quick and then I'll hand off to Fran, but we support different, a Piazza and protocols. While you wait, that's meaning zoom:1.

C

A

C

Our spring cloud stream binder here actually uses our our java api under the covers. So it's a solace sdk publishing into solace, but just because I'm using some special format and I'm using a Java API I'm allowed I can communicate with these other applications. So, for example, like the taxis are probably would probably be using C or Python, maybe to publish events over MQTT just as examples now, Fran is actually going to use JavaScript or nodejs to receive events over in QT and the broker itself is doing the protocol translation.

C

So he does not to care his application doesn't have to care, but I'm using spring he's just gonna use mqtt and connected to the broker. So again the applications are decoupled. He doesn't care think about the publisher other than that. The events exist on the topic that is in the design. I think.

A

About this, like it's a pure micro-services design, where, as application developers yourself and friend, you're completely decoupled from the taxis portions, we don't even know how and we don't need to know how the taxis send their data and what programming language use or what protocol we use as long as they use a programming language on protocol and they send it to the broker these different micro services even like, if we're in a real in a real world where Fran and Mark could work on the micro service in parallel, where Mark and because Fran already high already knows, for example, application developer or developing the micro service already know the schema and everything is defined from the technical requirements.

A

So what are the requirements for your application? Your architecture, it's already there. It's already defined in a spec and you can see like the the other micro service, is not dependent on all on how Mike's micro service is being built. As long as Mike's micro service follows the the spec and follows the the schema and that's the nice thing about it. We're like Fran now I would like to kind of like see like your your input and there's like what do you think of the whole process and passing on to you yeah.

C

B

Was the whole idea about about async API right so defining the interfaces right? We don't care about the implementation, so that's why we can communicate it. You have a service with a loggia service or with whatever other service, and we care about the interface and the interface is composed by primarily first or let's say. First is we need to understand where the broker is?

B

So that's where what we call servers on the on the rise in PDF specification right then, what's the protocol, of course, what's the protocol that this ever is speaking or the protocols right can be many like like so as that's, for instance, I supposedly used for many protocols right yeah.

C

Sorry I went I, went ahead and push the stuff into the github repo okay, so you can pull down if you need to use on the live stream branch also.

B

C

B

I already did it so perfect.

C

A

Wait how do mine you could check out the github repo over here we're putting everything live you can follow along, but then you know feel free to Jenna to contribute and let us know what you think so far so friend now I'm gonna pass it to you. So let me know if you want to show your skin or not. Why.

B

Is it good news yeah for.

A

B

Can start crawl.

A

Down to the architecture, diagram, okay, show what we did now. Yes, this is it friend could take that piece now on the bottom right, if you zoom in perhaps maybe.

B

It's um let me try dang.

C

So yep and then friend you want to go ahead and share your screen and then yes.

B

I'm on it, which window it is okay,.

C

Yeah, so so what brands gonna do now is is basically those events that I subscribe to in the try me Tom, the ones that are being put out on stats drop off average he's going to consume those in a in a nodejs application right.

A

Question to mark and friend came from twitch from Aaron does.

C

A

Api have a preference or capability specifically for hierarchical topics or doesn't matter flat. Topics are okay, too lot.

B

Topics are okay to, of course, because they have to because not every protocol supports hierarchical topics right so yeah that they, it was masked like, for instance, you have WebSockets, which is a very common case. You don't have that. You only have the event name, which is map to the topic, and it's just a plain string. It doesn't have any meaning and no characters have any special meaning yeah.

C

No I would say yeah so easy. Give you guys definitely supports either way from a.

B

C

Perspective, like are, we have taught the best practices that are out there so definitely like using the different levels is, is great best practice for solace and it's good for allowing. But you want the the hierarchy and I saw this best practices perspective to define what the actual events are so kind of defines your adventurine, so your subscribers can choose which bits and pieces of the event stream that they want absolutely.

B

I mean my my personal recommendation is always if you have them use them, because they can save you a lot of overhead, let's say yeah, especially especially in performance. Right like we were saying that the taxi. The taxi topic was a great example that we had the reduce that the the window or the number of passengers, the number of passengers, that no.

C

Pass here yes,.

B

You don't want to you, don't want to probably you're not interested in in right that have, or than three passengers, for instance. Thank.

C

B

Route, otherwise, you will have to filter out in your application and then.

C

B

Be receiving a lot of messages that you're not interested in actually yeah.

C

Exactly and I was like, and so I subscribed so like the drop off events, when I was doing that, if I did not do that, like I, think the this event feed has like 2,000 once a second. Almost so you know in a 20-second window, I got six drop-offs yeah. If I didn't filter on just the drop-offs I would have got what 40,000 events know how to throw away exactly so.

B

C

Network overhead was saved there all right. It's.

B

Important yeah, of course, it's important right.

A

B

Yeah sure, okay, perfect so yeah we have here the the I think EPA file that mark just exported and, as you can see, this is yeah. This is a. Let me just come here. This is the file that it is supporting there with the right averages, update message and it has defined a payload here and we're gonna be getting the messages here on the drop of average topics right. So let me just go through the generation process, so I'm gonna use the async API generator and I'm gonna specify that I want to generate this specification.

B

Keep a file and I'm gonna use the official notice templates in KPI. So that's how you'll do it and this this template I'm sharing, sorry I'm sharing just a code editor window, otherwise I'll go for so in the commentation, but this template requires you to have to specify which server you want to generate. So let me try to find it here. You see that there is a servers section on yeah.

C

B

Those and keep a document you can have multiple here right, so one way to optimize your code is to tell the the code generator, which several do you want right, because now this one is soulless, but imagine that you have three here and this one is not called salt, but it's controlled by staging and and development right.

B

You don't want to get all of them right. You would just probably want to generate one of them, so so yeah, that's that's the reason you have to specify l7, so I'm gonna pass a turn to the the code generator and it's every parent and I'm gonna. Specifically, this is the soulless one.

B

Let me also put the the result of this generation of this generated code in a folder called code. Maybe just just inventing its name right now and it's like whatever I just don't want to generate the code in the in the exact same way and in this India's directory right. So so we don't mess up. Basically, I guess, let me run it.

C

So for my note from for this generator that you're using what are the other protocol options besides I'm.

B

There's no match there's much sheer mass of them. Maybe let me try to find. Oh sorry.

C

I can look it up. It was true.

B

Yeah, it's because I'm always I'm always sharing the the code window. Let me just I.

C

Stumped I stumped the founder here I'll lock it up there, quick, okay,.

B

C

Amqp and keeps it's me WebSockets a cough that looks like.

B

Oh yeah yeah regarding protocol yeah. Oh sorry, I thought you were meaning about all the params. You can pass I.

B

Don't think there are many, but.

C

From a source perspective, mqt is definitely the way to go so yeah.

B

So so summer isiliye, you can see that we got generated code here on the code folder, so I'm gonna get there city code and for those who are familiar with no Jesus, come on right, npm install I'm gonna install all the dependencies and yeah it shouldn't take too long cool. So one of the things that I'm gonna make it fail just for you to see what's happening so I'm gonna put something: I'm gonna run npm start, and this is gonna fail. Okay, where you get the problem here, stop it.

B

It says connection refused by the username of or password. Of course we have the solar server. The only information we have about the stolen server is this dead. We don't have a username or password right, so we have to go to the config file here and specify it right. We have to specify the username and the password. Let me just give it a happy. I'm here somewhere, yeah.

C

I think it would actually work if you're running a local solace, but obviously from a cloud service perspective. We make sure you have to authenticate to use the cloud service right.

A

There is a parameter if you scroll down as well the configuration parameter under the the broker. So you see here in the comment: oh yeah mul there's the broker section and then, when you you go down, there's the configuration parameter as well that you could that you need to add to this.

B

So yeah this case also I had already the username in the past: okay, okay, yeah, the cool guys from SOLAS gave me beforehand so so yeah we have the username and the password I'm gonna write it again. Just recap again: let's see so cool we get success here. It says it's subscribed to. It's subscribed to the topic. It's ready and it's listed the mqtt adapter, because it's an MPPT server like because we're saying it here and then it's a PPA file, it's connected! So it's listening and we got a message right so that.

C

B

Pretty easy right that we got, we got here, timestamp the average. You know all this. The information that mark is pushing from the from the server he just created and yeah. You can see we're receiving messages every once in a while. It's every 20 seconds, yeah yeah. We got another one so perfect, so yeah you saw how easy it was to actually generate the code.

B

Just a quick walkthrough of what's happening here, we're using a library called Hermes, and this is for those familiar with nogf. This is like suppressed platform messaging, so yeah and it's just using middle words like likes press. So it's it's saying the application to use this middle word, which this meter were. This middleware in turn, is actually a route. So it's actually yeah!

B

You see it's subscribing it's using this this when it says use it's like it's subscribing today's topic and then you get a message here right and you can do whatever you want here on the handlers it's another handler is created and this will be the equivalent where Mark was really right, all the business logic and as you can see, you get implement your business logic here.

B

Something that we will have sorted out soon will generate the business project.

C

This is pretty nice. I actually had not seen this before correctly.

A

From run so similar to like, when you develop.

C

A

Nodejs application, if you're developing a node.js application- and you want a handle, REST API calls. So let's say you Bill your you're developing your like JavaScript application, whether it's front-end or back-end, and you want to handle requests from your rest, api, so use Express for you and in from an asynchronous perspective, you can use your you were using kermis to handle any messages or any events that are coming from any messages are coming from the topics that we are kind of, like listening to or subscribing to yeah so yeah to compare like I.

A

You try to kind of compare.

B

A

B

Actually, a little bit a little bit more because it's not chairs to handle the incoming messages, but it also it's capable of handling they out outgoing messages. So whenever you try to send a message to a topic, you can apply certain middlewares to manipulate the message before it's actually sent, and one of the things that you can do is to validate that.

B

The message is valid: it's well-formed compared to to the async ap the finish, and so so you make sure that you don't publish any any message that it's not looking correctly or well formatted right, right and thing on the incoming messages. You can validate them as they arrived to your application and make sure that it has the correct format right here.

C

Does that where does that validation happen? Can we know that? Is there anything.

B

That that's something that we want to include next yeah.

C

B

Have some sort of.

B

Type that we can use if you want right now and with this with this example, and if you want, we can use it. If you have time, I think.

C

B

Can make it fail if we want and.

A

Right now we have like the schemas being defined the schemas being defined in the async, API, spec and friend. I, don't know: do you have access, so you have access to the github repo just to show the overall infrastructure Kotecha.

A

What we'd like we literally did right now to micro-services one in sprint cloud one in nodejs and ojs just subscribes to the topic and puts it out. So what we did, because there are some stuff in the async api, expect that we haven't used yet so such as the the schema. So, for example, like we showed right short earlier that you know you could define the schema. The payload schema with the JSON schema like what what you have so I'm, actually kind of intrigued and curious to see how, from nodejs perspective.

A

How can we make sure that mark is sending the right schema on the.

C

Top, don't trust me.

A

B

A good point and I want to just to remark when thing is we're really using this information about the schema? If you look at the screen right now, I'm sharing, you have all this documentation right about average method. Reading equal default value is zero and at the written second default value is zero right and.

A

This information is.

B

Taken so all these documentation on the carry out is generated from the schema right, SJS Doc's talk.

A

About driven development, where you have your your scheme is already there, so you could literally even like have tests in advance. Unlike you could you could you know your micro service to even confirm other micro service to see if, like the the keynote is coming right because yeah, for example, let's say in a hypothetical situation mark um put the like a parameter, for example like a time stamp as an integer?

A

This would fail because, like the we put in the schema that it sits there, so we could like go ahead and see what we could do with this. Just a quick kind of update to anyone.

C

Who's watching.

A

Us live right now, your questions are accepted. If you have any any questions, if the overall architecture wasn't clear, let us let us know, because what we did right now is to recap: we went from designing an architectural in the event portal and this all is clouded event portal, defining all the schemas downloading, the async API spec, installing a or developing a spring micro-service spring quad micro service and a node.js micro service and our harmoniously asynchronously working together. Right now and let's try to challenge friend a little.

C

B

Let me just I'm, not sure I'm I'm, not sure, I'm sharing the right window. Okay, so it's like okay, correct so yeah, so I'm here you'd see me moving stuff around right. Oh you're,.

C

Using a future version, essentially what you're gonna do now? What you're gonna use a future version like a new feature.

B

Now, no it's actually it's actually a package that will lead one of the our members like creative and it's an odious package that allows you to take, take async API from input as an input and then whatever Jason value role, Jason value and compare if they're valid right.

B

That's that's worth this and it's kind of maps to async API right, so not gonna, extend so much and how it works. Really, because this could be really boring.

C

Relative, like our basics, out of perimeter and say, hey validate and then the generator will plug it in for me. So.

B

Basically look so if you, if you look here on the on the left side, you we have a middleware, its folder and this these are the ones that are logging the messages as they arrived right, and one of this is validate right. So here I'm using the is library of wali that it's called a syncope evaluator right and what it's doing is.

B

As we get a message, it's gonna use the library to validate it and if it fails, it's gonna continue the pipeline right, because this is a this- is like a middleware pipeline, but with an error. So it's gonna notify that there never happened.

B

Okay, so you have the error of lager, which is the one that we have here in the left side will take care of login it properly right or doing whatever you need to do with it, and if everything goes well, it just silently continued the pipeline and that's it, and it will at some point, arrive to your business logic. Okay, okay, so.

C

B

A pretty simple concept: okay- and we important here the middle word just to make sure and we're using it here. So one of the things that I've done beforehand is say, validate this message. So this message right average averages update, is it's taken from here from the fundation, keep a file. It's this one right, so I added this name here, so it knows which one is it and let me just save it and cross my fingers.

A

My thing is for you to.

C

If it says my starting correct, I'm gonna deny it.

B

Yeah, it even failed so.

A

B

Of the things one of the cool experiments of the week will do here. Right now is we can actually do the opposite. You can change your code if you want and make it fail, go.

A

Ahead, all right, let's, let's.

C

Change, I, don't think Java will. Let me like unless I change, the objects themselves right. Okay,.

B

Let me do something else.

C

If you change something in the stack all right, yeah.

B

I'm gonna change this back. Exactly it's gonna be easier, so yeah listen.

C

This is cool, though, because, like historically like Alana, again kind of back to the know, if driven applications are decoupled and when you have problems in these systems in operations a lot of times, everybody just points the finger at the middleware team and the middle or teams like I, don't know. There's events.

A

C

I see stats going up, yeah.

A

B

Allow you to actually.

C

Like validate like and figure out, you.

B

Know the yeah if.

C

What's flowing across the pipes actually matches the design yeah we'll help you troubleshoot that much quicker yeah.

B

Exactly so here you see that we have this average meter reading and it says it's the type number we're gonna change it to a string and the for the default value. It's gonna be that's.

A

B

Zero, so that's it I think that will be enough. Sorry I actually have to do it here on this on this file, the other one is the original one. Where.

A

B

I generated the comment from the the curved front, so.

A

B

Me just make sure that everything is here and that I think it is yes here is so it's the fold zero getting nice texture. Okay, so let me start it.

A

So now technically I don't we'll send it will say.

C

First, as a string right, yeah.

B

It's probably doing that casting or something like this automatic typecasting. You know no GS beautiful things.

A

Happen we make it so pager to blue.

C

B

Yeah, it's true. Let me if.

C

It says that my ride count of three is somehow a valid boolean, that I think the validator.

B

Has a problem exactly if it casted the boolean true right well for a number it's possible as well. You.

C

Know I guess when.

B

I do an object. You're.

C

The timestamp change the timestamp, the timestamp, won't be a 1 or 0 and to.

A

Give a shout out to community members: this validator was created by Lalit, so you can see, see, put a handle down there. uh He contributed to the validator on a sink API. This is a pure example of how open source is built by the community and anyone could go ahead and develop so believe. If you're watching shout out to you thanks for developing this son, thanks for contributing to open source.

B

I can't get it to something really different, which is an object right. I said.

A

B

Subject: yeah actually type is a number so yeah, you can see it block here anyway like where is it is letter reading? What is it one yeah, so this this one is it's number as its OB right, but on the icing keep a file, we're saying that it's would be an object, that's right because we're saying that it's not respecting an object right like you can see here. So that's why it's failing right!

B

That's why it's correctly fraying failing so, yes and I, think that will be like all from my side, in terms of so in life code, I'm! Happy to take any questions. Of course, I found great.

A

I mean it is kind of like and to application, to show how we could you know, work in different micro services together design, architects, even validate things, so we we covered like a good ground over here and we're in time. It's it's good.

A

We have a good amount of time, which is which is great, so I want to given a quick reminder to anyone who's watching right now, all right to join our solace community there's like updates there to stay up-to-date with our future events, will be posting events, so you can see near friends head there's a QR code where you could.

A

Scan it right now over here, which is great so thanks for that no I just wanted to kind of you know real.

C

Quick to me or not as well, we are, we are the sauce community, we'll have our event scheduled pretty soon, and so we do plan on doing weekly events, so I think most the time they are going to be on. You know, on Wednesdays, so by by joining the community also be notified of future events and be able to go in there and see the scheduled stuff as well. Yes, arts me be good. All.

A

Right, we've got a question from twitch that just game, so he this person uses node, no dev one two three, but also plays with spring Java for backend. What other language is gonna use with async API.

B

So so far, I think it's it's an odious Java with two different frameworks. It's Python right. It's the other, template and I.

B

Remember a community, a community members saying that they had a C++ code generator but I, don't I'm, not sure they open source it well.

A

In any case, this.

B

Is this is something we're working on we're continuously working on actually we're actively creating new code generators ourselves and so us as well we're in there right and actually there's a Python and the Java spring South Stream code generators were created by the, but.

A

Continuing as well so, this is actually like. Yeah go.

B

And and and would like to remark here that most of the times, the only thing you need to do if you want to have a code generator with your favorite language, you only have to put a simple project in a repo and and tell us to review it. Even if we don't know so much about the language we can help, you convert it into a template right, so it doesn't need to be perfect.

B

It's just a starting point, because once we have a starting point, then we'll people will be more keen to contribute and improve it. But so so that's that's that's instance. How I did it with with an OG as a code generator right, but it was like that maybe I created a proof of concept and then people start it started complaining about it, but.

A

B

How it works, it's like I knew that it had failed, it had bags and it had really bad things and decisions, but the moment you have something working and or you feel that you could have something working. You only have just to fix this. Then people are more keen to go and fix this only, but to tell people to start a new project from scratch and I contributed and all that it's usually it takes more suite I'm. Damn right. Thanks.

C

No time one, two, three Thanks I, just kind of on that note like Fran. If, if people want to contribute to a sink API like how should they do so, yeah.

B

So one of the things that we always say if you want to really contribute join, our slack I was like workspace. I think you have the URLs somewhere yeah there.

C

B

If not, you can always go to the website and you'll find it on the top menu but yeah the I. Think the first step will be this. If you're not a fan of slack, we can understand it. It's also. It's also fine to just open an issue on the on the generator on the generator repo, so that so that we can help you and we can ask answer your questions or we can.

B

We can actually collaborate because, but well to be honest, slack works better right. It's like more real-time chat in Seoul. It's not like github issues, you might. It might take one day for us to reply or right. It's not like it's not that life right, yeah I will.

C

Say, like I, saw I'm part of this and keep your slack workspace and really like. It is a very yeah.

C

Like it has been a very welcoming community, like people are always introducing themselves and it seems to be you know growing with with the initiative and definitely I. You know I feel like sometimes I'm asking questions like late in my day and since I'm in Orlando. You know, I, don't expect answers till the next day, but Fran and Lucas and everybody else they're like I, don't think they sleep I, think they. You know, answer slack messages like all night.

B

Developing the next generation generator with where it generates the business logic and one of the critic. One of the core features is, is a machine learning and you know and processing. So it actually replies to you. It's about replying.

A

That's that's fantastic! Well, I'm, really happy that we were able to get this working from design and run. You said you, you have something else to do mentioned. Yeah.

B

Just one thing is: we are we're slowly rolling out a new tool, which is the async API have and I think you saw you should subscribe there. You should sign up here.

C

B

It's it's a it's. A cool I think it's gonna, be a really cool application and tool right: it's open source and and yeah. So let's also collaborate there. It's a it's like an IV! 'if you, if you want or assess where you can store your files right and have projects and share with your team and yeah, so you can share the async API files with your team and you.

C

Know operate on them right, I'm excited to see to see hub right is it.

B

C

It's gonna be available to everybody, I.

B

Hope soon I mean it's gonna be soon, but I hope when I say soon is within the next month within the next next month. Time frame. So it's let's say during the next 30 days. Let's say I'd like to again say not computer, so.

A

B

The so Christmas yet more rest.

C

B

So the reason is not yet opens it's not yet available for everyone and it's not ready it's as stupid as we don't have a website. Yet we don't have a website. We have the application, we don't have landing page and and also because we want to open search it with some decent documentation right, so people can start building on top of it, because I love to explain things about this and KP I have, is it's an it? It's a noun, a pluggable or it's made of a pluggable architecture. If you want or if.

A

B

Like Google architecture, so you it's easy to create your own plugins for half the do whatever you need to do all have right so and so yeah as one of the things that we have to explain really. Well, it's what's the what's that what are the plugins API, how they work, how to develop plugins for the UI plugins for the server-side plugins for the database, how to store custom day down the database and.

C

We wonder why you've no time to do anything else, all right, and then we were wondering why you didn't do anything else outside of basic.

B

And I and I even I, didn't even mentioned half of the things that I'm doing. Sir.

B

Right now so like in the next days, I move into my hometown, which is 1,000 kilometers from here and we're in the middle of a pandemic and and an emergency status State on in Spain, so yeah, it's fun good.

C

Luck, hopefully, you know, congratulations, hopefully, for moving back to your hometown, yeah.

B

I know Emily so.

C

What we're playing tools I just want to plug one more thing: I guess is I, don't know if I mentioned it earlier, but the the console that I was showing an install is cloud you can sign up for free and you can launch like a free developer messaging service and you can use the event portal. There's like a free trial period. I think at 60 days use that to generate your ASIC API spec and you know check it out, just like we did just like we did on on this session. So yeah.

A

I think this kind of like mine demise, is everything so friend again. Thank you very much. Thank you.

B

Invited me and thanks.

B

Three-Body is actually another there, but thanks for all the work, yeah.

A

Thanks Joe, yes,.

C

Thank you. Okay. Thank.

A

You forever this and again for anyone who is didn't get the chance to fill in the the pool. Please go ahead. This is your last chance to fill the the pool, but, apart from that, all I have to do is just once one thing.

C

A

C

He's been like, like all the meetings we've had like last two weeks, he keeps having outro music I'm like how can you use it for the meetings.

B

Interesting concept: I'm gonna start doing this.

A

It's a good idea.

C

A

Just like have your music.

A

C