AsyncAPI AsyncAPI Conf 2022, 5 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: AsyncAPI Conf - Day 3 - 5th Nov 2022

Description

https://conference.asyncapi.com/

Contents:

00:00:00 Waiting for starting
00:04:13 Opening
00:05:45 Let Me In: Open the Doors of your Organization for Student Contributors - Karuna Tata, AsyncAPI Initiative
00:42:25 Automating Governance for Asynchronous APIs - Nauman Ali, Stoplight
01:21:57 Catalog your Events with AsyncAPI - David Boyne, AWS
02:02:06 Contract Driven Development in AsyncApi, Tools and Procedures - José Enrique García Maciñeiras & Pablo José López Rivadulla, Sngular
02:50:21 AsyncAPI Recipes for EDA Gourmet - Laurent Broudoux, Microcks.io
03:38:20 The Future of Messaging? "Store-Carry-Forward" (SCF) Protocols, Delay Tolerant Networks, and AsyncApi - Ismael Velasco, Adora Foundation & Green Software Foundation
04:31:48 Quality of Service in Asynchronous Services: An AsyncAPI Extension with Service Level Agreement Support - Marc Oriol Hilari, Universitat Oberta de Catalunya (UOC)
04:58:38 Break
05:32:06 AsyncAPI and the World Wide Flow - James Urquhart, VMware
06:11:48 Generating and Using AsyncAPI Specs from Real-World API Data - Natalie Edson & Rob Galanakis, Lithic Technology
06:40:10 Closing

A

A

A

Hello, oh great.

A

um I'm, so sorry about these, um so I think I always have trouble with restream um yeah, so I'm glad that got resolved as fast as possible.

A

um So uh it's been a long ride uh over the last couple of days and finally, today is the last day of the asynk conference, 2022 and yeah, so I'm so excited um over the last couple of days. We've had. You know amazing folks come on come on stage and we've had so many interesting presentations from various people from the community and also from various people that are actually using the um some of the tools in the async API ecosystem and yeah.

A

So since today is the last day uh before I start I would love to give out a shout out to all the people that made this conference possible, which includes amazing folks from the from the community. um It's better shout out to and Lucas so we have Ben.

A

We have have karuna, we have I, don't know, I can't keep going on, because the list is so long um shout out to Alejandra and the banners of folks that made this conference so so interesting and also I've, given a special shout out to our sponsors, uh which includes um Postman, which is the sapi Platinum sponsor for conference 2022 and also ugly, which is a gold sponsor for this conference and yeah. So why not? Let's get right into it.

A

So the first person um for this presentation um for these for the first person presenting today is our own very star, which is karuna Tata, um so I'm bringing her upstage um karuna. Are you ready?

A

Okay, so coming up stage now is karuna, so hello, karuna.

B

C

How you doing I'm doing good and really really excited for today.

A

Yeah same here uh you know I'm, so I'm so I I'm, so fortunate to be the one hosting you because I know um you've done so many amazing stuff and with the community and yeah so I'm so excited for you to get started. um Do you have anything to share before we get right into it? Would you like to say anything.

C

um Yeah do enjoy the last year of conference and we might have some surprise for you, uh which we did yesterday, but today might be different. So keep.

D

A

D

A

Yeah absolutely she said we might so don't don't guess your bars raised okay, so.

D

A

Might we might not so who knows so yeah, so one of Let's get right into it, um karuna. So oh boy, Karim.

E

Hey everyone, it's so good to see you at a sling API conference. I am karuna and my topic is: let me in open the doors of your organization, fostering contributors.

E

Let me just quickly introduce myself: hey I'm, karuna and I am a coach at Major, League hacking and a technical writer at async API as a part of G sword. uh This order is short for Google season of dogs and I have been contributing to a single API. Apart from this, or two and I have been part of multiple student communities and open source organization.

E

Also I have done lots of events in the past and I have been like streaming on YouTube doing like uh lots of podcasts and Twitter spaces and fun fact I like drawing anime characters.

E

Now uh before we get into the real part of this session. Let me ask you one question: what is open source now? I think a lot of you might know what is open source, but still, let me explain you can see this mean right here, saying open those programmers and they are just lifting up the society now. What does that mean uh I'll see because oh we, we might be using a lot of Open Source tools, you might you might even like not know about it. Let's say there is Firefox.

E

There's VLC editor uh video editor and there are a lot of tools right and all these are developed by open source contributors and open source maintainers.

E

Now, if you want to become this open source programmer who's lifting up the society, what I will recommend is contribute to open source? Now you will say: buy, should I contribute to it. Just because, uh like I do not want to be a hero, I'm, fine, being a normal person, then you might want to reconsider that because it helps you in a lot of ways.

E

Let's see you, you are using an open source tool and it keeps on crashing, and you might be wondering why and you can do a great job by reporting bugs on GitHub issues. Let's say: VLC VLC has its own repo on GitHub.

E

You can just head on if it crashes sometimes or you see some issues happening on VLC, you can just go on to the repo erase an issue and though that issue will be fixed in no time, because a lot of contributors and maintainers see those issues, keep track of it and those issues won't be like that. Like closed Source, what happens in closed sources?

E

If you wanna report a bug saying that hey my Google Chrome is on crashing, but you can see the code of Google Chrome and you just have to send the report to them saying it keeps on crashing now. You don't know how much time it will take for them to ship the next patch, because only few people will be working in that group, so in GitHub in open source. You don't have to do that.

E

You can just go on to the repository and yeah that like say, there's a bug and then making meaningful contribution contributions to open source software. This leads to like elevate your skills and you can make it better like let's say if I know, Python and I wanna contribute to python related software or tools. I I might know, not know some Concepts and then I go on Discover. Some Concepts in the repository and I will ask hey. How can I do this they uh to the maintainer and the maintenance?

E

My Supply here you can check out these these resources you have and then I will learn the concept and then I will make my contribution. Go live right. This can be a great opportunity to put your skills and see that you can grow a lot more in open source, and also this puts your work in public like open source is viewed by everyone right. So whenever you are contributing to an open source, you can see that these are my issues. These are the pull requests.

E

These are the files which I changed, or this is the code. You can also show it to your um job. You can also show it on your job resume or you can show it to your HRS or any in any technical interviews. You can just say: hey I have done this before, so this helps you a lot and then getting started in open source is a great way.

E

So one thing I'll tell you: if you have never never ever contributed to open source, you can just do fixed fixing typos like lot of Open Source contributors who are maintainers now or GitHub stash. Now start it somewhere right. Even I started somewhere. So you, you might be thinking they are already great. They know so many things. It's not like that. Everyone gets started at some point, so fixing typos in documentation or read me sorry code. If you see it, you can just like ship up a request and that's a great way to get started.

E

You will know how it works. Github works. You will like collaborate with others. Saying okay, I made this you'll get to know like how to comment what language you should use, what are the do's and do Nots, so you can just go ahead and do that now. I'll be talking about some problems which I see where students face while contributing to open source projects.

E

Let's say one student like done everything like or they did fixing typos or finding good first issues and doing it, but they still find it hard like not finding right projects to contribute to. So these are some questions which I get on daily basis, or let's say this is just repetitive questions which I get every time from all of those students.

E

So tell me: how did you get into open source or community uh beginner friendly projects are not really friendly and then what language did you learn? So how did you get into open sources just by getting into it? uh I mean once you get started in it start having conversations with them in Discord or in slack or whatever platform, the organization uses, let's say, I'm contributing to a Sync API and in the in the start.

E

I might not know what conversation to start so I might pick an issue which is labeled as good first issue and then I will contribute, make a PR to it and they might say hey. These are the changes and the conversation goes on and what you can do is you can just hop on to the slack or Discord and start a conversation. Saying: hey I fixed this PR.

E

But uh do we have any more issues or do we have any anything else where I can contribute to if you are into design or if you are into code, they have everything for that. So that's not a problem and then beginner friendly projects are not really friendly. Yes, I have seen those projects and uh I mean.

E

If you look uh from the point of view of a beginner where you don't know code or you don't know how to like talk to maintenance or comment out or something, then you might see this big depository with lot of files, folders and all look all of the code everywhere, and you might find it really difficult to do that.

E

But let me tell you one thing: even if you see a really big repository with lot of files, folders and stuff, you can just go on to uh picking up like uh good first issues and see that there might be few comments already left by the maintainer saying that this is a good first issue, and this can be solved in this way. Even if you do not get how to solve it, you can just comment there that I am stuck here.

E

Will you please tell me how to solve this part right and what language did I learn? Does not? It does not matter really, because each and every organization has its own language on Tech stack.

E

So um I will say that when you are picking a project or if you are just really interested in doing that project, then you might have to learn the language or you might have to know the concepts which you already like know in other language and then convert it to let's say I, know, Python and I want to convert these Concepts, which I learned in Python to JavaScript, because I am contributing to a project with JavaScript.

E

Right now, so these are some changes you might have to do it, because if you want to contribute it, you have to uh make sure that you know everything regarding to that project and solution for this, as I mentioned good first issue and had to professed labor. So if you don't know, October 1st is a month long uh open source program where anyone can come and contribute to open source, and they have projects listed out there on their website.

E

And then you have to just make uh three pull requests and you'll get swag swag As in t-shirts like this or anything regarding to that. So if you do a like a contribution to an open source organization with hacktoberfest label, then you get swag and that's a great motivation for you to get started and here's a comment of async API from someone from async API saying uh that they don't mind having a contributor doing that issue or giving that full request.

E

So this is also a great way to make your organization infusion like every time a maintainer comes and says: no. This issue is only done by the core people, or this issue is just meant for these people and you are not welcome to do it. That makes your organization not very inclusive and closed.

E

So it just looks like a very exclusive organization and, as you can see in the comments, so they don't mind having a contributor do the work for them if if they want to- and this makes it really uh amazing to look at and then we have few more resources listed out here, which is upload for grads.net.

E

So this uh this repository is meant for new contributors only and you can see all the issues or repositories where they have uh things for new contributors to get started in open source and then first timers only is another resource. Just like up fog wraps, you might want to check it out and do hack to buffets, because it's still going on and as I speak, October 1st is like half completed some. Some people might have even gotten meal saying that hey your triple request, Summers.

E

So if you are still not doing hacktoberfest go join it right now, here's another one. So all your open source contribution is type of pull request, but this helps a lot because some typos do not look good in uh any code or in documentation. So it's still a valid um open source contribution.

E

Another problem which I see here is organizations not being inclusive, just as I talked in my previous slide. So you can see here are few organizations which I find really really inclusive. One is Eddie Hub community. This is really inclusive and like they also promote speaking inclusive language in their own Discord server. They had a bot actually where, um if you write some non-inclusive language, the bot Flags you out and says hey. This is not inclusive, and once you edit your message, the bot message goes away.

E

It's I really like that bot and uh it makes people like get used to it once, let's say, I make mistakes twice, saying some non-inclusive language and about life now, two times three times and like fourth time I'll get used to it like slowly saying oh yeah, I, learn, inclusive language and I will start using it everywhere, regardless, if the bot flag me or not, I'll still use it everywhere.

E

Also, here are a few another comments saying that some contributor contribute like come and say: can I work on this issue and they say no need to ask for more information. You can just go ahead. This is like really great way to make contributors welcome in in your issues or like that, and then I I had a contribution.

E

uh I was making a contribution to async API, which is like the issue was outside of the sort of program and I wanted to contribute to it, because I found it interesting and then they were getting started there.

E

Making the ambassador program page so uh I wanted to make that page correctly and I did what I like understood from uh from previous PSC page, so I just copied, pasted it and then change changed few context or text, and then people started giving on reviews one after the other saying that hey this could be like that and other maintainers hop on and say that hey, if you don't know how to do it, uh would you like to like us to show to you like?

E

Would you like to hop on a call and show show it to you? They help us a lot saying this like makes a contributor feel welcome and at the same time it does not make the contributor feel like they haven't. uh They don't have the knowledge or they can't contribute to it.

E

So, and this is a great way to say that a maintainer of an organization saying that uh uh we will help you, even if you don't know we have people who can help you here as a mentors or here at like people who can guide you into this stuff. That's a great way to make make your organization inclusive and another problem I see, is losing motivation or Direction. So uh where does this come from losing motivation or Direction so and we'll? We know.

E

We all know that uh open source is voluntary right, so people just come and do it there, no one waiting paid for it or no one's really taking it very, very seriously like it's not their full-time job. uh It should like it. It should be great if open source was full-time job, but it's not like that people have their bills to pay and for most of the people I know it's not a full-time, so they just do it for Community or they just do it for their projects which are running on right.

E

So, as for students, we have lot of things to like focus on. We have admix, we have uh open source work and we might have some other coursework or we have to like get 10 or 10 on in every subject and then graduate or look for jobs. So there are a lot of pleasure on students, so in this uh open source might just go in a side and sit there, and we might not even think about it anymore. So what makes uh like what makes us lose?

E

This direction is all of this pressures, and what can we, as an organization, do to help these students make a proper contribution and then also cheer them up as a motivation? That's the solution. Again few means saying that we might buy a three thousand dollar laptop, but five thousand dollar open source called donation is like ah no way I'm, not doing that, and open source developers do not get paid.

E

They might be contributing to a lot of lot of stuff and uh making your tools better and better, but they do not get paid, and this uh this way like really losing motivation to do that after some time, because we have jobs which pay us, then why would you do open source right now for students, I'll, say swags are some kind of motivation. Let's say they get stickers or let's say they get um t-shirts after some point of time.

E

Those things are a little meaningless and then we come to a point where we have to look for a job or pay our bills or if you, if some student, wants to do a course, they might want to pay for it, but they not do not have the right resources for uh things to pay for it.

E

So uh GitHub sponsors is such program where you can sponsor a student if they are doing a really really great job in open source, and this will help them in a lot of ways right and also there are some programs like gsod, where your organizations like async API, hire technical writers and then Google pays them at the end of the time period so that they can just come and say that we contributed here here and then Google pays them right.

E

This is a great way and another thing is: we have Google sees a Google summer of code which is similar to gsort, but it's not about documentation it's about code, so you write a proposal to uh the open source organization and to the project saying that hey I can help with this. uh This is the feature I propose and uh if you get selected, you can start with your code contributions, and you can learn a lot of skills. You can make a lot of connections networks and then, in the end you get paid.

E

So this also helps you in the job perspective. Plus you can pay your bills, so this is a great way and there are a lot of lot of other programs where you get paid. So if you might want to explore it, you should because and as I told that GitHub sponsors is another great way where, if you think someone's doing good, you can just go on and sponsor them.

E

This little donation will help them a lot in growing right and if you are wondering after hearing all these problems and solutions say how can I bring the change to my open source organization like how can I be inclusive? uh It's not really hard to do it, because, once you start adapting to the ideas or ideas, then you, your organization, will grow in no time and you'll get a lot more contributors than you uh do today. So first point add a code of conduct.

E

Now code of conduct clarifies organization's mission and vision, and it also tells you that this organization is a safe place for everyone, and it shows that the standard standards of professionalism, also it articulates the values of your organization, and so it it tells all the contributors that this is the desired behavior and if you go out of it, you might not get to contribute to open source.

E

So uh code of adding a code of conduct is a great way to tell people about your open source organization and also uh to make contributors aware that if someone goes out of their way and like violates code of conduct, we will take action against them, so that and eradicating problematic language from open source code and documentation. uh There's an article by Red Hat, where they say that the problematic language is becoming a non-inclusive for everyone and they should remove it and they are taking action on it for like in code and also in documentation.

E

So this this makes uh our organization more inclusive.

E

Next is ask don't tell uh this is for contributors and also maintenance. Let's say uh you should not tell people something to do. You should just ask them a request them politely saying that hey I made this pull request. If you are like uh this, if this is a valid contribution, yeah, could you please merge it? uh The this is not the right way of asking saying that uh Hey merge my pull request, hey merge, Maple request or they can just Spam.

E

Your comment section by saying: hey, merge, hey merge and that's not a great way. So you don't you don't tell anyone to do you just ask them so that you will get reviews on it or you'll get feedback on it saying next time you can do better or you get next time. You can use like contribute on this issue and you get feedback like that. So and another thing is being specific now this helps in a lot of ways, because one thing I see is: if you are not specific people might get a confused.

E

Let's say: I have a problem regarding something ABC and then uh I might just text the maintainer or just hop onto your Discord server and say: um hey I made this PR. Could you help me out? This is a really vague way to put your problem out there, so you should be specific in defining what your problem is. So when you hop on to Discord saying that hey this is my PR link, the pr or say the pr number, and then you come to the point where I am stuck at this point.

E

I do not understand this or you couldn't be very specific. This saves the lot of time of you and the time of the maintainer like or any Mentor who guides you right. So if you are specific, you get the solution to the proper thing where you are stuck at. If you are not specific, then you might get a vague answer just like that, or they might not even uh make an effort to solve your problem.

E

Then next is explain yourself. If you are making a pull request, then you should know that you, you can paste before and after screenshots. If it's a code or documentation contribution or if it's a UI, ux design contribution, then you can explain what you have done properly in a great way, so that this helps in merging PRS very quickly, and this makes uh the work of the maintainer easier right and offer Solutions.

E

If, uh if you see an issue which is like stuck from a very long time- and you know how to solve it or for them saying that hey I worked and on the same problem last time or I, did a PR similar to this and linked the pr or uh right, like I, have written a Blog on this problem.

E

I think I can help you and then say you can do XYZ steps and they might give you a chance to like contribute to uh your their organization, and it might end up being a great contribution to the organization right and then have diversity.

E

uh I, like this point of in async API, where like when I was interviewing for my gsort and I, saw the document of the like in the GitHub report. They wrote. We do not want to be a barrier for any of the technical writers, and everyone is welcome to interview. They got 200 applications, but still uh they like did not leave the applications in between saying, oh, they they won't make it. They won't make it and they did not do that. They took all the interviews of 200 people and it's very diverse Community.

E

Let's say like uh I'm I'm, coming from some different background and some other the shortcomings contributors are coming from a different background. So it's just not like everyone's like uh great at technical writing or they have a PhD degree or something we have. We came from very different backgrounds and when they saw it so this uh this diversity uh makes Community very intensive and also I would like to mention that async API did not let other contributors who like interviewed for this go away. They said they have another program, called mentorship uh mentorship session.

E

So once the g-sword interviews were over and Technical writers were selected, they launched another program for mentorship and saying that the other people who did not get into uh get into this this word they can uh check out these maintenance here mentorship uh program which we launched, and they can still contribute to it right. This is a great way to keep your contributors in your organization and then setting up a Wiki or repo where people can contribute their ideas or of suggestions.

E

uh Nowadays, people don't use, make keys much I think, but there is a feature called discussions uh by GitHub. So if your organization have a discussions, tab, please make sure you use it properly, because uh many many contributors can come and leave their ideas or suggestions. Let's say you start a discussion saying Us in kpi conference now you you will see like many of the contributors coming commented and shared their ideas and saying that this time works. This does not, or they just gave to make this event a success.

E

Okay, ideas to make this event a success, so that is also a great way to make your contributors keep getting involved in your organization and then thank you for listening to my talk and ending with a small name fixing a breaking bugs versus fixing typos in documentation. You get started with fixing typos in documentation and there's no shame in it like you can just go ahead and do that one. Now you can just find one typo in a Sync, API repo already and then fix it.

E

So you can get started in open source right now and right here right. So thank you. So much and bye.

A

Hey karuna you there.

A

um So what an amazing presentation from karuna um so I can see. We have a couple of questions for karuna and yeah, so I'm checking. If Corona is there um are you? Are you ready to come off stage?

A

A

Yeah, what an amazing presentation karuna, um you know so enlightened that even me, I get to learn a lot of amazing stuff. For my talk, a couple of questions um for you from the audience, um so the first question is from Liberty Aziz. He said a common misconception about contributing to open source is that you need to start contributing code. In fact, it is often that the other parts of a project that are most neglected and overlooked.

A

What advice? Would you give that type of contributors.

C

So my advice is open. Source does not limit just to code. uh You can go beyond code. As you can see, we are.

E

Doing documentation at async.

C

Api I have seen people who contribute to design too. We have barbano uh who, who you might have seen in uh slack, doing design.

E

C

E

uh People who do Community Management, even though being a project manager at open source, is also a contribution, and uh other types of contribution include open source.

C

Advocacy so if you are going to going out to give a talks like this, uh that is also a contribution, because you are educating people about open source. So.

E

This is another contribution, I see. So these are my advices. If you are new to open source, do not back away just saying it's only for code, uh you can contribute in many many ways, even if it's not good.

A

Awesome, what a very very interesting um answer and Corona and the other question you've got from this same person- is what are some things you should look out for before contributing to an open source project.

C

So I got to learn this the hard way but uh I I.

E

Joined an organization which helped me get uh get uh familiar.

C

With all of these, so I'm gonna share the advice which they gave so whenever you see to contribute to an open source project. uh First thing you might see is the idea if it fits fit into your text tag or you, you might want to learn the tech stack, but you like that idea and you want to.

E

Contribute so first things you will see is go to their project report and see if the project is.

C

Still active because if sometimes the project is not active and you have made a pull request, all your efforts will be wasted, so I'll say just.

E

Go to the issues, tab or pull, request, Tab and see if the product is still active and.

C

Again, uh just as I said, uh make your organization.

E

C

uh Whenever you want to contribute to a project, see that their pull requests are the closed, build requests are.

E

Closed with a reason, uh let's say some pull requests got just closed and you might say all of the work which, uh which is good and some maintainer just closed it without uh explaining it. So that.

C

Will discourage you a lot and you might feel bad? Oh, my efforts are all in vain.

E

Why did I contribute to this project? So that is another um factor which you should consider before contributing.

C

E

C

Things are dumb if.

E

You are finding good mentors uh good maintainers contributors. You can just go on and talk with them before contributing, so that.

C

You will get the context behind uh the project and do some homework before contributing, because.

E

um Other maintainers and uh all the people who are in the organization will appreciate that that.

C

You did some homework before.

E

uh Contributing.

C

To their open source project and then you can go ahead and ask your questions.

E

So that is my advice.

A

Oh awesome, awesome karuna. That was a very, very interesting one. uh Thanks for your time, we are so glad to you know having you in today's conference uh and your presentation was awesome. Absolutely awesome. uh We are looking forward to seeing you doing more amazing stuff with the community and also looking forward to seeing you present uh at the second player Conference next year. Thanks for having you uh so one more thing before I go, I would love to give a remark.

A

A closer remark to karuna's talk, uh so open source isn't an exclusive club right. It is made by people like job it is made by people. Just like you. Open source is just Advanced term for treating the worst problem as fixable, so feel free to contribute to open source private contribute to listen kpi. We are always and currently accepting contributors. Thank you see you guys in five minutes back on the short break, thanks karuna see you later.

A

A

A

Because the next presenter should be presenting now and yeah, so so sorry we are going at a very, very short break, so we are back again and yeah. The next speaker in line with me, is now man. So now man is a very, very interesting fellow and I think he is also some of the people pushing um you know it's in the piano forward, with some amazing stuff they've been doing with the community and outside the community right, so I'm gonna bring Naman up so I'm checking if Naman is up.

A

A

Hey Naman hi how.

F

A

How you doing I'm good? How are you uh I'm doing I'm doing fine? You know I'm, so excited. You know to be able to see you today and I'm, pretty sure you have something interesting for all the people watching and every community members today.

A

G

Thank you so much for having me I'm going to be talking about API governance. Today, I know it's a it's a topic. That's discussed a lot. I want to get into the Practical details of how you can potentially achieve some of some bits of it.

A

Of course, you know I'm looking forward to learning more awesome, perfect, okay, going backstage now.

G

Hi everyone um I'm the one um so I'm a product manager at stoplight and I'm, primarily looking at the Epi governance piece, um EPA governance piece within stop light here, um I know: API governance is a term, that's thrown around a lot and it has a lot of different pieces to it um from how your workflow Works, within your API design, how reviews work to actually managing and deploying apis and eventually deprecating.

D

Them so there's a lot of different.

G

Pieces to it, what I really want to get to today is actually automating automating, some of those some of those pieces and primarily getting into API Styles or style guides, and how you can potentially use them as an effective tool to to make API governance better in your organization.

G

um So before I before I get started with um before I get started with um how to get into actually creating these style guides and using them. I just want to set a bit of context into why we're talking about what we're talking about and I took a quote from the state of API report that smartware does and standardizing, and what it says is. Standardization is the number one challenge that teams building apis are looking to overcome this year.

G

Now, when we talk about standardization, it's primarily especially when you go on scale, let's say: You're, Building, hundreds or thousands of apis. How do you make sure that people across different geographical regions um and new developers or designers coming in make sure follow similar guidelines and your apis as an organization? Look.

H

G

So that the developer experience is called for whoever's, consuming it internally or externally, and API governance like I talked about, has a lot of different pieces to it. But one General goal that um we're looking to achieve with all the processes that we build around API design and development is consistency. You really want to be consistent in the apis that you're building out um and while, while there are multiple ways to go about governance, some centralized teams or pushing it more down to the teams that are building apis.

G

Good API governance, in my perspective, is more about making the right thing to do, which might be following some different some principles or some specific guidelines or a workflow easy to do and how.

I

G

Can make it easy as I think that's the theme of my uh what I'm going to be talking about is automation if you're automating the different piece, um the different pieces of your workflow you're, really making it easier for people who are whose main goal is to build apis, make sure that they're doing it right and from what we've seen from customers within the community. One effective tool to achieve that is setting up any API style guide. Be it if you are a small company, just starting out start with something small and Define.

G

What good looks like in your organization, and especially if you are at scale you'd, want a style guide out there. Much like you might have a style guide for how you um how you develop, apps or other things, just to make sure that um everybody has a good sense of what best practices and um what good looks like in your organization.

G

There are a couple of different governance strategies. What we have seen over the years for large companies like you'd, see Google PayPal. Putting out these. These API guides um is a large PDF that might be a large PDF. That might be that might contain all those different um all those different best practices.

G

The problem with that is that one there's it reduces the time to Market or reduces the time it takes for you to put apis out because there's a central team that has to look at all API designs and make sure that, um if that style, guide is being followed and feedback is always late in the process plus another piece being people who need to follow these style guides. People who are building out apis need to read large in PDF documents, just to make sure what the style guide looks.

H

Like and now you cannot expect.

G

Everybody to be doing that in system or some general, so the other piece which is um would just come and gain popularity of late is automating some of these um some of these guidelines. Now you build out these API standards and we'll talk about more about what these contain and how um how you can potentially make.

D

I

G

I

G

General idea being, if you can automate some of these rules, your reviews go faster, because feedback is much faster, maybe within the design environment and within your review process.

G

There's visibility across the board as to what those rules are when those appear and it scales well, especially in larger organizations, so the general idea being automate what you can and it's going to bring you a lot of a lot of advantages over time, especially as you scale your program um just reiterating the point and how important automation is to a um API governance and we've seen that across different people that we talk to all right. So I'm I talked about this a bit but just to put into perspective.

G

This is this: is Google style guide out there in terms of API design and what it is is a large website with different topics and how to go about versioning? What naming conventions to follow? What errors? um How do? How does Google do errors within the organization? This is great because at least there's an idea out there on what code looks like, but then again for somebody coming in New Or buildings. These things out.

G

They have to go and find the right topic that they need an answer for and really really make sure that they read through wire before actually implementing it. Cumbersome takes time now what you can get to is the automated kind of style guides, which is on the left. What you see is a set of rules.

G

um These are built using spectral which supports both open API and using API and other Json things too, or.

H

G

Out there like open, API CLI from um where you can, where you can codify some of these rules um and then run them in your CI, run them in your design, environment and get errors um as perfect. So if you automate and automate the guidelines itself, it really um like we talked about really makes your life easier. So you could map what Google has as a PDF into some of these codified rules and run them in your make it part of your process.

G

Now now, we've I've talked about why you should be creating an automated style guide but big question that um that is certainly hard to answer in terms of like the complete. um What should an API style guide include what everything that is include, but I'll, try and get it give a sense of some of those things that you can include in an API style guide to make sure that you're achieving the right things and, in my perspective, I break it down into two two different buckets.

G

There are two different goals with these guidelines, especially with um um with us as an industry, the API industry or people building apis, adopting open, API and async apis formats to write these design. These apis one goal is to really churn out good API specification documents, be it open, API racing kpi to make sure that the tooling that you're using you might be using a code generator or you might be using some other tooling, that all documentation generated to power, your workflow, so the quality of the API spec matters a lot and there's.

G

This is one bucket of um one bucket of rules that you can create an example being some async API best practices. First of all, you should have validation for your async API, so you're at least following the right syntax. When you're writing these, but then you can go beyond that and say all right. I'm I want only this version within my organization.

G

That might be the latest async API version, um using a particular protocol, adding valid examples, making sure the operation ID is unique and adding information for like who the contact or is for this API and the license information itself. Just you can go through the async API document itself and say: okay, these fields are really important for tooling uh for what we do within the organization with these specs, so you create routes for it.

G

um The async API team has done a really good job of actually creating the async API rules that they've been contributing to within Spectra. So you can go check it out, which has some of these rules already pre-built, so you can start with them and keep adding on to those to make sure that the async apis that you're building are really good quality.

G

um Another bucket another bucket within this, um this importance of building could API facing kpi documents is maybe you're using a Gateway right. So if you're using.

J

G

Gateway you there would be particular um particularly nuances of that, like the Amazon API Gateway, not supporting one off within the Json schema.

H

Or some X extensions.

G

That you might have for your.

H

Code generation.

G

Setup or your gateway setup again, you can codify these to make sure that everybody who's building out these apis are adding these Fields within within the document, which really helps make sure that the tool and downstream works well. um A big goal of building these documents is documentation itself so um make sure to add some documentation. Rules like you should always have examples on payloads ADD descriptions. What what the length of the description should be.

G

Maybe add a grammar, a spelling check just to make sure that the descriptions are not making typo you're, not making typos within the descriptions itself, and if there are any other. If the tech writing team has any other requirements in terms of what good documentation looks like within your organization, you can add those into the rules itself and make sure that the documentation that you're producing is a fire quality.

G

um This is one bucket um and I think it's really important to build out these rules to make sure that your documents are of high quality and the tooling works well, but then I think where it really gets interesting with these style. Guys and automating is some building out. Some Epi design guidelines and.

F

I would look at it as.

G

The first level as maturity level, one where you're building out rules to uh make better apis, open apis and API documents. But then going ahead and saying this.

D

G

What good API design looks like with an organization and one bucket of it might be API URLs. Now every company wants to do it or does tend to do it differently. There's no one right way of doing what casing you should use or what your url structure should look like but decide on whatever.

F

G

Want to follow if you want to follow some particular company like say you want to do really like how stripe does it to take those conventions and build those rules out like what what casing should my parameters be? What should I completely avoid within my URLs and um what the structure of my URL looks like now?

G

If you have a good sense on of rules on how your URLs are anybody, who's using your apis would find it far more intuitive as to what the next and what the next call might look like the next, the next piece, which is always a tricky topic with an organization's versioning and while again there are multiple ways of doing how doing versioning within organizations it's good to at least decide whatever works for well, for you and codifier so say.

G

If you want to go for path level, versioning or you want to do versioning within headers, whatever it.

F

G

Make sure that everybody's doing it at the same way again the goal being consistency itself?

G

um This is a big one in a bucket that contains a lot. Thus I have all this I won't go through all of it, but the general idea being make security, a big part of your style guide and what I've, really? uh What I've really found helpful, is just to make sure that you're, following wasps stops top 10 the rules there. You can create those rules and then go from there and say this is the security scheme we use within the organization HTTP basic? We stay away from it.

G

um We always add limits to our limits to our schemas in terms of a string. What's the limit of that um if there are minimum maximum properties and objects just to make sure that some of these uh basic security concerns don't fall through the holes, it's it's they're, um they're easy, but they're still easy to miss too.

G

um If you have some great limiting conventions, you should add that too, and this this list can go on and on depending on what what the security posture of your organization looks like um error handling is another another piece that you can add to your add to your API style guide.

G

Make sure you save in your style guide that you shouldn't be added and adding um adding errors into successful responses, but also saying that this is what your error object looks like, and what the popular responses error response is that you want to have um always within your within your API all right. So these are um just to just to summarize data, but um there are two different buckets that you can build.

G

Style guides are off, go to go to API documents and then go to API design uh codify both of them and put them into one style guide that your organization as a general encourages people across the board to cross the board to use and implement, but.

H

G

Piece is building these out and it's it's certainly a big task to take up, but, let's say you've gotten the point where your style guide is ready. How do you roll it out? The first idea would be to socialize it within your organization using some sort of documentation. Considering this is enough. This is uh this.

G

Is a machine readable spec that you're building a style guide and you can probably generate documentation for it and some down put it on um put it somewhere, where everybody can go, find it and look at what um how how style guys look within your location. This is just one more one to get the idea of the style guide out there and to have conversations with people who are going to be actually using it um as to if this makes sense or not, especially in their use cases.

G

Now, what the next step would be to bring it where your designers or developers are the idea being, instead of it being a document like we saw with Google style guide, bring it to your design environment, so, let's say you're using vs code or some other tool to actually build these async API or open API files out.

G

um You can bring those errors into the design environment itself. So, as people are writing these specs, they can get get these errors and warnings within their environment. What this does is it gives real-time feedback. Much like you'd have es link when you're writing code. It makes it easy for.

H

G

To follow the style guide itself, like we talked about governance, is making about making things easier in this really. This really puts it right where nobody needs to go and look at something to actually follow these rules, and you can divide them into some being errors, some being warnings just in case of what do you think about this Rule and the severity of it and people can choose to ignore it.

G

This is more encouraging people to use it rather than enforce I get, but then um the next bit that you can do is make it part of your review process. So let's say you do your reviews on GitHub you're, following that workflow, where somebody changes and on chasing it changes. The spec makes a pull request. You can run these um this style guide within the pull request itself. You get errors right there and then have a conversation around.

G

Why is this not fixed, or why is this error here and if we need to ignore it then have a conversation around it, but the general idea of anything that passes those checks and those conversations can be skipped as part of the review process and you can focus more on um rather more important things and that's. Why do you still need to review API designs? Yes, but the review can now focus on business capabilities use cases.

G

So what are you actually building rather than how you're building it or what the casing looks like or you're missing out in an error or not using the right um right convention when it comes to.

F

G

It comes to API design, so more about more about. Does this API make sense? Does this solve the use case, rather than focusing on um what the conventions might be, so your reviews are more useful, they're, faster and um I. Think.

K

In building API.

G

Is much better and faster too, with this now as a style guide, is a really effective tool, I think one that has good tooling around there to actually start doing this, but.

H

L

G

Else can you do, and these could be separate conversations in terms of governance, but having an improved diff system so that people can look at what has changed when as part of the review process in a better way and some sort of breaking change detection. Just so that you don't you're not putting outbreaking changes without knowing about them, so make this part of the review process as to a better deaf, hopefully not Jason and some sort of breaking transaction. And there are open source tools out there that do that and.

M

Then going from.

F

G

um There's a you, can build out a concept of a reusable API components: Library which might have models, parameters, headers error responses that everybody should use just to just so that people can get started with it quickly and keep reusing stuff rather than building it out, so that there are five different types of errors that your organizations API is through um and templates being another I think next step. Once you have a style guide, a good sense and what good looks like you have reusable components.

G

You can put out templates and say async API documents and open API documents, saying if.

L

You're building this.

G

Kind of an API use this template get started with it just to make the process even faster than what it would be without these thank.

F

G

This is this is what I want to talk about. Hopefully it was useful if you all have any questions. Let me know.

A

um Thanks Naman, okay, um so we've we had an amazing. uh What's going on.

A

um So we've had you just you just like enlightened me, so much I learned so much about uh you know how to automate API governance yeah on in your talk and yes, um I think we have I think a couple of folks are actually like. Inspired by your talk and in church. You have two questions right. So the first question is from um wukash and it says a big company is still doing manual, governance approach.

G

You know um I'm, hoping more and more are moving towards the automation bed, but yes, um I, think there's. um There has been more of um more of a push towards automating um some of these processes, the the concept of Legacy apis, the overhead of building some of these style, guys and automating them, and just generally I think um more knowledge around what you can do with these tools out there um is helping, but we can. We, as a community can certainly do a lot more.

D

M

I, certainly in.

G

The past year to have seen a lot more big companies put out their own style guides that are automated and are really putting.

F

It out there I think um one thing that I'd stress on.

G

If people people who have built out style guys, we should put these style guides out there for public, just so that other people can get ideas from and we, as in an industry, can grow on. What good looks like when it comes to API design.

A

Okay, awesome awesome, I love that no one I love that um so the question question is actually from a folk called illegal Aziz. um So it said, apart from automation, what are some drivers? We should be focusing on to quickly mature our governance approach.

G

A

So and like I said I.

G

Think there's a lot of different pieces to governance, putting out a style guide and some of those pieces of automation that I talked about help, but one one big piece: I: think that really really does help um organizations as visibility so make sure um if you're using the spec itself put out, create a catalog of all your apis. So you know what apis exist, but.

L

When you know what exists.

G

You can actually find problems and make find solutions to how you can potentially improve your.

M

Maturity until.

G

General I think um sometimes assuming um that every across the board there's knowledge around these tools like open, apis and kpi. What tools out there that assumption can be wrong at times and just general sharing knowledge across this um across what is coming up and what these tools can do across your organization really helps.

A

Interested interesting um thanks for clearing that up now Norman, and so, if you have any question for Naman, um you can easily reach out to Norman and I'm pretty sure if you want to learn more about API, Governors and I bet. um Norman is your go-to guy and yeah. So um would you like to can folks reach out to you um on Twitter or on slack what was a slack Channel or at the AC API platform.

G

Oh yes, um I'm on the async API slag, so you can always reach out there and on Twitter and Linkedin too so anyway, I'm happy to answer any questions. We'll have discussions around API governance or design in general.

A

Awesome so I I bet. You would eventually see me in your inbox also, because I would love to learn more about API governors, perfect I'm, looking forward to that all right! um Well, we are so glad to having you you know present today.

A

um Now man and also we are looking forward to seeing you doing more stuff with the community and helping position kpi forward and also with the all the cool stuff you are doing uh so we are so excited to have you, and we are also looking forward to know to having you next year at the Singapore conference 2023.

A

A

All right. Thank you um see you next time. No man.

A

Yeah so an amazing presentation from Norman, and so you can reach out to Norman on slack on Twitter. If you have any question about API governance or you would love to learn more about um the presentation you just gave.

A

um So that's everything on normal's presentation and now we'll be going on a short break and we'll be seeing you guys at 120, see you guys later bye.

A

A

A

A

F

F

F

F

F

F

F

F

F

F

F

F

A

A

A

A

A

A

A

A

So I hope you guys have enjoyed your break and yeah. So today um now we have the next presenter, which is our own, very David. And yes, so, let's just get right into it. uh David are you ready to come online foreign.

N

A

David, how you doing yeah great, how are you yeah, I'm doing fine? um You know I'm, so I'm so glad to be the one hosting you man all.

N

Right, it's good to meet her. That is awesome. Yeah.

A

Awesome so so glad to have you back today.

N

Yeah, no, it's great yeah I'm. Looking forward to uh doing the talk and yeah see what people think for sure.

I

A

I

A

Yeah, so for most folks, who don't know David so David I would say: David is a rock star at sa kpi, so he has done some amazing projects and that has helped push the attack, API initiative forward and yeah. So that's why I'm so excited to have David today.

A

um So um before we get started, do you have anything to share.

N

uh Not really I just hope you all kind of uh enjoy the talk and thanks thanks so much for having me here as a conference. uh Any questions you have on kind of on the on the channel as we go um I'm happy to answer any questions afterwards or anything so yeah I hope you like it.

A

Awesome awesome man yeah, so let's get right into it.

I

Hello: everyone it's in kpi conference, uh my name is Dave Bowen and thank you for having me today and kind of watching this uh for the next half an hour. What I'm going to be talking about is cataloging your events with async kpi, what do I mean by cataloging and and documentation, and that's exactly what we're going to be diving in to today. So here's a quick agenda for you. What we'll be looking at we'll be diving into some initial problems.

I

Actually, that you'll see what I see in the community kind of talk about, and problems I've had in the past. With that initial setup of event, driven architectures and the problems you start to see and emerge as time goes on. So before we dive in I'm going to introduce you to myself, my name is Dave Boyne. You can find me on Twitter at boyne123 I'm, a senior developer Advocate at AWS focused on serverless, Technologies and also event driven architectures I'm, a father of two.

I

um So you might hear them scream in this video how they are in the house and um so listen out for that uh ex-guitar player and previously before AWS I was working in open source full time actually part of the async kpi team for a while and before that, working in startups and some large organizations mainly focused around tooling developer experiences and all sorts of things. I also enjoy kind of writing, open source software and tools for developers and focusing on event driven architectures.

I

So, let's dive into this talk and talk about the problems of event, driven architectures beyond that initial setup and things that you might see- and maybe some of these things might resonate with you unless, as you look around in the community at the moment, you can see. Actually the adoption of Avenger of an architecture is growing right. More and more people are using it more and more solutions are being developed with a kind of this event, architecture in mind and when you start diving into the reasons.

I

Why is because I believe technology is just making it more accessible right? um The adventure of an architect has actually been around for decades, so it's about the technology and how that's helping us do that, and if we just have a look at some AWS Services as an example. Here you know we have things like Amazon eventbridge, which is a serverless event, bus which allows us to send and receive events and map them up to other serverless kind of Technologies there.

I

We have kind of Step functions that are out to the orchestrate some workflow based on events and even things like I'm, the right, um which allows to kind of do some compute, based on based on events funny enough. um So you know the the event. Architectures themselves are actually becoming more and more accessible, and then, when we start looking at the benefits of event architectures, you start seeing these kind of words come up. You know we.

I

We aim to have Loosely coupled systems, especially in this kind of microservices architecture, especially because we have different teams doing different stuff event. Architectures allow us to kind of produce these events and consume them in different teams and process them in different ways right and having this Loosely. Coupled systems that allows us to have this extensibility, which allows us to have consumers, come and go as time goes on, so as requirements change in our organization, we can change who's, listening or add more listeners and do more and more things and event.

I

Architectures just really really help us from just from my own experience. You know it just help us get there and then, when you start getting into that kind of flow, you start seeing this agility arise, which is you know we need to do X when y happens. Well, if we are, we are already already publishing. Why? Then we can just do the X anyway, because we because there's no work that needs to be done there and, as you add events, and you start designing your architecture in this particular way.

I

You start to see this kind of agility that emerges now. What tends to happen is once you go beyond that initial setup of your event. Architectures, you start to see some weird kind of problems emerge right and normally it's not around the technology, but more around kind of behaviors and and people in general. Really. To be honest, um you start to kind of question.

I

You know the producer and consumer responsibilities as an example, so who is who owns the contract here right, or maybe it's validation, do you validate at the producer level or the consumer level or both, and then in the middle?

I

Here we have coupling services with events, so even the event design itself can actually couple you to your producers without you, even knowing lack of governance and standards as time goes on, should you be thinking about your event, design and the impacts that actually has on your organization even to the point of the the event naming itself the unclear event in 10? What is the name of our events? Are they past tense? Are they things that have happened? Is it very? Is it clear what we're looking at exposed domain domain information?

I

Are you exposing too much um integration or internal information within a particular kind of bounded context or a particular service? Should you be exposing? What kind of domain information should you Expose and where's the line there? Finally, things like you know focusing on implementation details rather than the behavior of your system. Looking at the events and things like event storming, there can help now, when you start diving into event architectures and you start reading books or white papers or whatever it might be.

I

You start seeing the um I started to see this term arise, which is producers should not know about consumers and, although that's kind of true technically you know, we want to create this decoupled architectures. What I believe happens you know, is we take this too literally, and what happens over time? Is you start to ask you yourself who is producing what, especially if we have hundreds and hundreds of events and we've scaled this out in our organization? You want to know some basic stuff who is producing what event?

I

What version of the event are they producing? Who can I contact? uh Who owns this event? What is you know what information and context around this event? Why was this event created? It could have been created two years ago as an example. So, as you start to look and come into these issues, you start to look and wonder: is there any kind of event driven documentation that can help us solve these initial issues and these problems that we have now?

I

Luckily for us, you know this is why we're all here at the conference is that and we have the async API specification, which is great. It allows us to kind of document. The producers and consumers are using various different kind of protocols and document our channels and be able to have a place where we can actually say. Okay, here is my service, and this is what you can do to it as a producer or a consumer. This is the kind of stuff that it's work. This kind of stuff is doing now.

I

The best thing about this kind of specifications and standards in general is. We can actually create Tools around the specification to help us get to that next level of documentation on our next level of like CLI tools or whatever it might be, and that's what I'm going to talk about next is introducing event catalog and what is event catalog now to explain this.

I

What I want to do is I actually want to just focus your attention on these three different areas that I've been kind of exploring for the past few years, and these kind of the things that kind of get makes get me excited personally, which is event, architectures, um developer tools and developer experience.

I

Now, over the past few years, I've been diving into event, architectures reading books, experimenting playing and developing tools and all sorts of stuff and I come across this book, in particular, recommended to me actually and by James urquhart, which is called flow. Architectures now James looks into kind of the future of event streaming and event-driven Integrations within our organizations, and, if you think about this for a minute, you think about data and cross-boundary data within your organization. We are always integrating with third-party providers.

I

You know, especially it kind of startups we're looking at different SAS providers and things we can integrate with there or data lakes and all sorts of things so to how we do that today are we mainly focus on like web hook, Integrations or whatever it might be, um but but there is becoming a more of a need to have this data kind of real time and as James kind of explores in this book, which really resonates with me is like the future of these streaming platforms and the future of events and data sharing.

I

And how do you get access? Well, obviously, if I can just subscribe to a stream of data rather than setting up all this stuff all the time this integration and testing and the things we have to do, but anyway, a part in particular in this book that really stood out to me was this worldly map and, as you can see here, I've highlighted on the left.

I

Is this discovery phase discovery of producer and consumer and the need for that and where that is along this whole life cycle, and because it's on the left, it's kind of indicating that this discovery phase still kind of needs more work, and it's not really ready for us to kind of move along in that sense. So it kind of just got me really thinking about.

I

How can we can we look at tools or ways in which we can, with event-driven architectures in mind, allow us to discover events and document events and producers and consumers easier, and that's the key really here is easier and having that developer experience in mind now, as I was continuing to dive deep into a vent driven, architectures and standards and various things like that.

I

I come across Fran Fran Mendes on um on Twitter right and he was like so I was building a tool and he was like Hey would be cool if you could integrate a base in kpi I was like what's asyn kpi and I've come across. You know the async API specification, which is great. It literally solves a lot of the problems and things that I've been thinking about and things that I kind of want to want to use and want to explore more.

I

You know the ability to document the producers and consumers the ability to go somewhere and see exactly what this application is doing from an event driven point of view and as I mentioned the great thing about the specification, it allows us to write tools and kind of see what else is possible. So I started to look at some tools and build some open source documentation tools that kind of really resonated with the community and people online.

I

That allows you to take like schemas or schema Registries, for example, with eventbridge and generate some documentation for you, some static content where you can go and host these things and kind of read and kind of document. Your schemas to the next level and I also integrate this with async API, which takes you far and and transforms it into some kind of 101 documentation and static website as an experiment and as I started.

I

Exploring this more and more people kind of found, it useful, which I found very interesting, which kind of mapped to that the whole Discovery phase of event driven architectures and that need to have something there to help these organizations scale their adventure of an architectures in general really and as I focus on on these tools and focus on kind of Open. Source Tools in general I spend a lot of time diving into what is a successful, open source tool.

I

Look like, and it's kind of a lot of emphasis goes around documentation of the tool itself, but also the developer. Experience that you have and when I start thinking about how people make websites and that experience with markdown files and also MDX files which allow us to have components within markdown files to allow us to custom render things on the screen. I started thinking about.

I

Can we use markdown and mark down our MDX files to actually generate a vent driven documentation and also include the async kpi specification in that to help this whole Discovery and documentation phase that I've been thinking about so I started.

I

Thinking about how can I kind of line the three up, which is event driven architectures line, that we've developed tools and standard standards and specifications like async API and also create, try and create this developer experience where people already know how to kind of use the tool, because they might be familiar with markdown, for example, I really wanted to try and give something back to the community which allows uh organizations and people to really document these producers and consumers with visualizations and components and all sorts of things, and that's when I decided to kind of build and experiment with this all called event catalog now.

I

What is a van catalog is an open source project that helps you document your events, services and now domains. It can actually take your async API file or files that you have and actually generate into a static website right, and you can also the static website will also host mermaid, diagrams and schemas and all sorts of things and 3D node graphs and all and the and integrates with async API as a plugin.

I

So here's an example page of an event that I look uh on event, catalog and what it might look like, and you can see everything here you can customize, so your producers and consumers on the top right here is the event called item removed from car.

I

As an example is a version 1.5.5 event and it producers is the shopping API and we have a consumer which is the customer portal, and this is all documented using markdown and also you can also all document this using async API files and what the plugin will do is it will take that and generate this for you, you can download schemas, and you can also see the schemas or see custom ndx components which is like mermaid, diagrams 18, kpi file, renderers and all sorts of things there.

I

So here is the async API plug-in. This is how it works. So when you initiate an event catalog, you install the plugin and you pull it the path to your async kpi file and then, when you run generate what it will do is it will pass the file and generate your events, your producers and your consumers. So what I'm going to do now is actually take you through a demo.

I

What we will do is create a new catalog We'll, add an async API file, install the plugin and we'll generate our catalog from our ancient kpi file and then look into what else we can kind of do there. So let's have a look now and and see that in action. So what we're going to do now, we're in the terminal is actually create our new event catalog, and we do that by running MPX at event. Catalog create event catalog at latest or without the latest command.

I

If you want and then we're going to call it my catalog async API, and what I do there is when I press return that will kind of set up the initial foundations of our event catalog. What I'll do for this video is kind of fast forward it uh does it take a minute or two to do that and then I'll show you how we, what it looks like initially and how we can use an async API file with the plugin that we use cool so now what we're going to do once?

I

That's done is we're going to go into our 18, kpi event, catalog and then we're going to run npm run Dev to just start it up and see what it initially looks like out of the box. So, let's over head over to the browser great so we're now in the browser- and you can see we're on Port 3000 and our application is now running and out of the box, we get some standard stuff, we get events, Services domains and visualizer. If we click on them, we can see.

I

If we go into the events, we can start to see all the types of events that we kind of get out of the box. We can use the filter here to start looking for events. If we want to and also filter by events by different services, we can also dive into Services themselves and be like okay.

I

um What is this service doing? So if we click on the basket cervix, for example, we can see the basket service itself is actually publishing these three different events, as we come down here, we're using MDX components to visualize the basket service and the corresponding events that it's raising and also we can provide some metadata inside of this page, which is like the repository link who owns the event, and maybe we can view it in the visualizer.

I

If we head over to viewing the visualizer, this will allow the visualize of our whole event catalog, and we can start going down here and exploring our events and our domains. For example, the basket service publishes this file publishes this event, which is subscribed by the data Lake. We can just start using markdown files to generate visualizations and documentations into event catalog, which is an awesome way to kind of just render some static content around your event driven applications.

I

So, let's take a look at the code and just see exactly what it is that we get out of the box before we Implement async, API cool, so now we're over in the code and what happens outside the box. What you get is you get domains, folder events folder and a Services folder, and all this inside of those markdown files. So, as you can see, we've got our events and we have an order complete event. We have a markdown file here and this markdown file.

I

We have some metadata and everything is documented online and documentation, but you can say: okay, the name is order, complete. We have a version, we can put a summary there. Our producers are, these are consumers is this service and we have also have some owners and then we can use some custom MDX components that the community has built on top of event, catalog to be able to kind of render different things inside of our event, catalog itself.

I

Now what I'm going to do is we're going to actually delete these and I'm going to install the async API plugin and we're going to use an async kpi file to generate this document, this catalog ourselves so step one. What we'll do here very quickly is just delete, domains, delete our events and delete our services and then what I'm going to do is head back over to the terminal and we're going to install the async kpi plugin.

I

Okay, so now we're here. What I'm going to do is just install the plugin for async kpi, and we do that by doing npm install our own catalog plugin generator async API. Let's go back to the code and set up the async kpi plugin and the way we do that is, we go to see event catalog config file, and now we have to actually install and use the generator and the way we do that is. We include the generators.

I

um They generate uh part of a file which is this so generators and we point to the async API generator and we need to pass it the path to our specification. So, let's put the let's put the AC kpi file directly into the root of the project here.

I

So we're going to do now is create a new async API file.

I

Inside the root of our project and I'm, going to copy an example, async API file into here, so this is using the street lights, Kafka API that you can find on the async API project, an examples repo we're going to use this to generate our event catalog. So what I'm going to do now is go back to the terminal and now what we're going to run is npm run generate.

I

What this will do is it'll, take the async kpi file and parse all the events and the services and make and turn them into event, catalog information. So let's now run the catalog and see what it looks like in the browser.

I

So now, let's head over to the events folder- and we can see here that we have dim light, light, measured and turn off light, which is all version one which is our async API version. If we click on dim light, we get shown the event itself, a commander, particular street light to dim the lights, and if we scroll down, we can see the schema, and we can see this nice diagram here. We can also use event catalog to actually download the schemas themselves.

I

So if we click on that, it'll allow us to download it locally to our machines. We can also see the producers here, so we see the street lights Kafka apis the producer of this particular event. So if we go inside there and now we're in the services folder, we can now see the street lights Kafka API, which is all documented for us again. Everything you see here is from the async API file.

I

We can see that the AC, the API itself consumes this light measured event, but also kind of produces this dim light and turn off event. We can navigate to the events and the producers uh um over here in the messages we can also use the visualizer to kind of scroll through the um all the events and kind of visualize our whole landscape.

I

Using this all events and services feature up here, we can drag our nodes and kind of fish start visualizing and seeing what is going on using this particular tool and if we're interested in a particular service, we can come down.

D

I

And click on that and see that itself. We can also start using the 3D node graph as a way to kind of visualize events going to and from this particular service. Now this is great, but what happens if we make changes to our async API file over time? What happens to event catalog? Well, let's make some changes to event catalog first and then we'll make some changes to async API and we'll see how the two work hand in hand.

I

Let's go back over to the code, and what we'll do now is we'll make a change to dim light markdown file. So maybe we want to add some more context here. This is where event catalog can kind of really help. You document your event, driven architectures alongside async API, for example, because this is markdown we could add any information we want here. We can. We could just have a Hello World from my dim light if you are going to use this, make sure you follow the instructions found in the project readme, whatever.

H

I

It might be, we can save that and when we go back to the UI we'll see that rendered. So, let's have a quick look now, as we navigate to dim light. We can see here here is our changes there. So let's make some changes to our async API file and see what happens so, let's head over back to the code in our async API file, let's update the version to version 1.1.0 and let's say we want to change dim light to have some different kind of summary um command of a particular street light.

I

Let's just say: let's have a look at the dim light payload and let's change some stuff here. Let's say the maximum is no longer zero to 100, but for whatever reason, as a zero to a thousand, we say the changes here, we've updated the version and what we're going to do now is run generate again to generate our new events and what event catalog will do with the configuration that we set. It will actually understand that the versions have changed and it will version the events for us.

I

Let's go ahead and run this command and then I'll go back to the browser to show you exactly what's happened, so what I've done here? I've run the generate command again, which I didn't show you, because it's just a console, um but what's happened here, is actually got. We've gone back to our event and it's actually upped it to version one one: zero. Now the readme file hasn't changed.

I

It still stores that information everything below this line is stored, that we regenerate this top bit using the metadata of your event and also what you can do here is. You can also go back and see previous versions of your async kpi file. So an example here would be dim light version 1.00 if I click on that, we can actually see the old event the old information, and we actually get a warning that we're currently looking at an old version here and the great thing about this.

I

We can actually diff the changes and view the changes using event catalog. So you can see here. Schema version is updated from one to one, one: zero latest and even better. What we can do is we can either change log to this version itself. So let's just go to the code quickly to add a change log, so example of adding a change log. What we do is we go to dim light. We see this new version folder and we can see one zero zero and if we come in here and we add a change log.

I

And we can add any information we want here again, it's just markdown files, so we can say why this particular change was made.

I

We save that we go back to the UI and, as you can see, the change log is now rendered, and this is another way to give developers and give people in your organization kind of just information about why your events are changing using your async API file, so another a great addition there to just help. You discover and understand what is going on. So the final thing I want to show you is an addition. Recent addition to the event catalog, which is using async API without the plugin.

I

So what you might do there is, you might Define a service and actually use the async API MDX file to generate the async pi documentation. Let me just show you a quick example of how you might do that.

I

So if we go back to the code and what we're going to do is create a new service, let's create a new service called my service, and what we're going to do is we're going to create a new index file I'm, going to copy what we had here and put that there we're going to change the name to my service and, let's just change the summary in a minute just to be my demo service, okay, and what we're going to do here is we're actually going to not render a node graph, but we're actually going to render async async API component.

I

Now, what I'm going to do is put the async copy, this async API file and put it inside this service. Now what this component will do is we'll render out the ace and kpi file that's directly against the service. So if you have multiple services and multiple async kpi files, you can start using event catalog to render these things side by side. So, if I save that I'll show you exactly what I mean when we go back to the browser.

I

So now, if we click on services at the top, we will see my service, the one we just created and if we click on this, he will show us that our new service, my our demo service, what we just defined and what it would do, is render this async API file for us um using that the async API tools from the community. So you can come down here. You can see. Okay, here are the servers um I hear the different types of operations, look at the parameters and everything else like that.

I

You can combine this with event, catalog components if you want to render more documentation or render graphs or visualizers or metadata or owners or Services whatever you want to do, you can use event catalog with or without without that plugin to kind of render these different services.

I

So that's a quick demo of how you can use async API files and event catalog to generate static content that you can host anywhere for your development teams. You can use event catalog to document events, services and domains, use your async kpi files to help generate that. So hopefully you find it useful and if you want to get started head over to eventcatalog.dev and what we're going to do now is just quickly summarize what we spoke about in this talk and extra resources to help you get started.

I

So, as I said earlier, the adoption event adventure of an architecture has grown, we can sit in the community. We can see that with the technology and what it's allowing us to do with kind of less really within our organizations. Again, you can kind of see this definitely in the AWS community, in the serverless community using event architectures to kind of boost their organizations. Here when we look at event architectures in general, you kind of see these kind of themes that emerge that it's allowing us to do to have.

I

Loosely coupled systems extend our systems with this whole producer, consumer relationship and I can and that itself provides agility to our organizations. But the problem is, as I said earlier, that when you go beyond that initial setup of event, driven architecture's problems really start to emerge in more. You know, often than not it's not really technical problems, although there are some, don't get me wrong, it's more kind of behavioral problems or design problems or architectural kind of problems, um and don't worry you know, the great thing is async.

I

Api is actually here to help there's a great Community behind it and great kind of passion and drive um that that that comes from this project, so I really recommend kind of scan.

I

This QR code, if you haven't already I, mean I, guess you're the async API conference, if you're watching this have a really dive in and check it out and check out the tools that the community have have built there, which is awesome if you're interested in documenting or a documentation tool for event, driven architectures I'd love to get your feedback on event catalog, if you're using async, API an event catalog or if you've got any more information or anything like that, just check out eventcatalog.dev and have a play with it check out the static content.

I

That's already hosted on the website as a demo. Hopefully it can help you in your organizations um if you want, if you want to kind of, join the communities, come and say hello or got any feedback or ideas just get involved on the left. Here we have async API slack channels so again scan that code there and sign up and then on the right. We have event catalog Discord channel, so you've got any ideas for plugins or any ideas or features or problems.

I

We have a small community there kind of really helping each other, which is awesome to see uh and kind of just exploring event catalog and just seeing how it can actually help your organizations kind of cross, this boundary of this Eda kind of documentation and Discovery kind of problems that we have and thank you so much for listening. If you kind of made it this far and what haven't got to the end, my name is Dave Boyd and again you can find me on Twitter, boyne123 and I just want to.

I

Thank you all for listening to this talk, and also thank you all kind of for inviting me to the async API um conference to to give this a short talk to you. Also I hope you enjoy rest of the sessions and thank you very much for listening.

A

Hi Dave, um are you, are you ready to come up line.

N

A

I just learned so much um in your presentation, and it was an amazing talk. um You know I'm so excited about what uh what event catalog can do and also the future of event catalog and how it is also happening. You know it's in kpi gets more recognition in the event driven um ecosystem right. So do you have anything to share.

I

um Not really no event, catalog everything's open source, uh obviously like the async kpi spec um I, feel like with all this stuff, we're kind of just riding out we're just at the beginning right so I.

N

Think there's loads of stuff to explore loads of things to do um and if you're interested in joining you know either.

I

Of these communities, or both these communities or helping out you know, just feel free, feel free to reach out uh if you've got any ideas or problems or if you're kind of interested in any kind of event-driven solutions, yeah just come, come join us and figure out how we can make this world better right.

N

I

Just come together, it's brilliant, um so yeah feel free to reach out to me whenever you want.

A

Great I'm David, um as you can see, um folks like David and the rest of the community, are actually the people shaping the future of event-driven apis and yeah. So I can't wait to see much more things get done by you, David and the whole communities behind even catalog, and also the whole comment is behind the Sync API.

A

um Thanks for oncoming um David, um we're looking forward to also seeing you um in the next conference, 2023 I'm, so excited yeah.

I

Thank you very much yeah thanks for having me it's.

N

Been great yep all.

A

N

A

Just so, um the next speaker will be coming after a quick short break um and the next speaker I will be talking about contract, driven development in async, API tools and procedures. um So, looking forward to that presentation, um so I'm seeing you guys at 13 20, oh sorry, I think 14 o'clock. Okay, so seeing you guys later.

A

A

A

A

A

M

M

M

M

M

B

B

And welcome back um hope, you guys are having a great day and our next speaker um is Jose who will be giving us the next session called Contact driven development in async API um Jose. Are you ready to come on stage?

B

You can just wave your hand, okay, cool. Let me bring you up.

J

Hello, how about you.

B

How are you doing.

J

ah So far, so good and yourself.

B

I'm perfect, thank you. um Would you like to say anything before we play a session? Yes,.

J

I'm very grateful to be here with you talking about contact our development and it's absolutely uh honor just to share this knowledge with you guys. Hopefully, people like it.

B

I'm off I'm grateful that you're here today and you're joining us today and I'm sure the audience will learn so much from you today.

N

B

Yes, so without any further Ado, let's tune in.

J

Hello world, thank you for hosting us here in this amazing conference. uh We're gonna talk about Contra, Revenue, Sync, API tools and procedures, something that we use in our day today in order to create our amazing software.

J

But first of all let me introduce myself who I am my name is I am principal civil engineer here at singular, coordinated by singular for the moment, with more than 16 years, in this mess fighting in different places and in different trenches, open source initiative, the OSS also the special service office here in singular I, am the father of kaloging, the multi-pa generator and the multi-pa contact converter.

J

These three projects are one of the main one that we deliver to the community, so we can share with with you our amazing coding, Styles and our good to do the things. Okay, I also are the Master's late for my two cats.

J

Just another way to do the same another tool to do the same thing. You are doing every day, another way of thinking another path to try. Maybe another thing to discover or not. Maybe so. First of all, let me tell you a story when you was a jungle kit, it was a fat Heidi, John fella and, as usual, the people started to call me names like what called gorilla monkey chin, blah blah blah blah. So you have two choices regarding to this point: fighting or embrace them I.

J

The first thing I do was fighting them, so it was amazing, fights, kicking and kicking, but at the end, icon, I embrace the gorilla thing. So let me introduce one of my friends once upon a time there was a mono Carlito, Mona Lisa was in church of the of his back, so he feed it back. He defended back from others, he communicated with all other packs and he was doing all monkey mono stuff.

J

The time passed, Lister was a successful. Mono hispak grew more and more with most Maize, more monkeys teams. He was a really nice and big pack other packs. Looking as a successful mono, so they wanted to make business. We think she was busy and he was happy the business expanded. Everyone want to work with monolito requesting food requested protection requesting all monkeys, different Apes, stuff and different Apes start contacting to him to work together. So he faced The Province. There are different names for Manana. There are different names for three.

J

There are different names for whatever thing the Apes do. Every day, other types of food were requested, not only bananas, just only nuts or different fruits and more stuff. We were requested.

J

So what he made Lisa has a problem, but Lisa was ismar gorilla, so he started thinking and thinking and thinking, and he got a big idea. He will find a way to talk to each other, so he can communicate with different Apes with different monkeys.

J

He has two to them as a system in order to communicate the data or the message between all of them. So I think that that was easy to read easy to understand. All the Apes mono sketched what he called the Apes petition interface or the APA, defend it as an instructor way to request bananas and other stuff from me on my pack, Mona Lisa was happy.

J

Every ape in the younger was starting to use his APA in order to request information and send information in Apes from other jungles started to communicate with them, so he expanded his business to other jungles. He sent his small Apes there, so he can start working in a in a huge corporation like really happy and really rich. For now,.

J

Monolito age- and he neither more has to help me, so he started to to delegate his all his knowledge and expertise in different small Apes, like a micro Apes that started working in different angles.

J

What can go wrong? We have the APA here, so he can, inter exchange information, exchange data, May request. It happened that other species appear on the stage and he wanted to talk with him and cooperate and increase the business and and share information between each other. So he needed to improve the APA. He need to come to the transactions to avoid the APA to be tweaked and broken, so the information to keep stable between all the transactions.

J

So monocreative way to control transactions or contracts Define it as framework to guarantee decorated of the message is changed through the APA monolith. Thunder colleagues were happy really really happy, because the funny way that keep the APA stable, that the information we shared between all the different apps and different groups and different packs doesn't matter which one the language they will understand each other okay. Let's gonna talk about contract testing, what's meaning contrasting. What is contract testing?

J

There are different definitions based on where you made this testing and you can make the testing for the consumer part they can make the testing for the producer part to a different approach. I know that development tried to join both approach in one. So taking the common point that is, you have two actors: consumer and producer is based on his key message. You need to Define, what's it they distributed or the data that you are sharing between the different actors?

J

There are some kind of field buildings, so you have defined your field with some limitation with some constraints, let's say a part of the types stream integer flow decimal. You can specify some radius, you can specify some limits W bit or on Lower limit, and the main important thing is. This is an agreement of mini mode. So what you are guarantee is that you at least will send the specific data Define the data that is in in the contract. You can send more of that.

J

There is no problem, but you cannot send less than what is specific in the country.

J

Excuse me right now: there are two big pledges in this war, but with the pack broker and spring Cloud contract. That is an open source solution from from Spring, but is is a pay solution, so you pay for it to have your broker and you lose the system. Sprinkled contracts is Super Source, so you need to install it and do it by yourself. This one is my favorite one, so we are gonna talk about it.

J

So, as I said before and content testing warranty is the current of the measures, meaning that I, um maybe a bit hard on it, that you guarantee that the message is sent and receive it, as has been defined in the API.

J

So you will receive the fields with this type and we'll you will add some kind of validation. For that, let's say that you are sending an extreme, you will not send a number, so the test that are generated are verifying that it is going to happen. It allow us tests both actors, so you will have validation for producer and test, for the producer part generate the stat for the testing and also validate the consumer part, so it tests almost every part of this.

J

In of this Exchange, the one strong point of the contract, private development is that they give you independent between the teams to develop. So once you have defined your API, you generate the the contract. This contracts are shared between all the teams, so producer teams, provider teams will have more requests that we requested data and it will validate the response. In this case, do you will receive a specific event and we we and we, we validate the event that is produced by the provider regarding the consumer? It will receive uh event reviews for the work.

J

So what does that mean that my consumer will be listening to a queue or a channel and will receive a a date that that model is is defined in this context, so it can test his Sound business logic.

J

So that's the main point of of and the strong point of the content. One once is defined it and share between the things they can start working independent without no problem, meaning that when they reach production, maybe it's not 100 guarantee, but the list says 1995 percent. All the communication will work with well.

J

Contracts are integrated in the build part of the project, so in theory, because you know that something can happen always all the code will be validated in the in the build stage, so nothing will will reach to production if all the things are done properly. Okay.

J

So what is a contract? Well, we have here uh like a sample about the contract. What how a contract is. As you can see, there are two different sections: the input section and the output setting. The input is made for the provider to receive the message or the consumer that is receiving dimensions and the output is based for is made for the producer, so the producer will send the message to that channel. Do you find it in the center part and the body will have those fields?

J

You can see here that there is some data. This data will be used for provide so to create the steps for the consumer, so the consumer will receive a message with name Adama, the account number 0002 and the update 2022 10 21 at 4, half past four am to the morning what happened with the input path. The input part will mean that the pro the provider will receive a message in this Channel and this measure will have that body an amount of two to point 513c blah blah blah. The last update is character.

J

I, don't care about that, because there is no no bonded about that. The name will be Adama. Id. Second number: all this data is created or defined based on what is specify in in the contract. So, as you can see, uh this kind is made in yaml language, so it's kinda readable. We can also Define it in Ruby that is more or less as well as readable or in Java. Okay, this is maybe not a readable language, but it's not so complicated right.

J

About how we can trigger those tests, there are different. There are three different ways, so I can trigger a test run into a contract testing environment, so I can have a triggers function. I will see later the samples in in the real code. Okay, so I can define a phantom that will produce and output meshes. So the the contact framework will receive this message and we validate it.

J

We can produce a method based on an input, so the system will receive an input Define it in the in the half part of this sample. The business logic will do all the magic and we produce an output method that will that should match the one that is specified in the output message or I can receive an information without no output. Let's see that we are creating a survey. That is a sync, so we only receive message.

J

So we only have the input power, the meaning that this message will be um distributed by the system and I I. We should provide the function in order to make the validation okay.

J

So, in a sprinkler of contract, there is no attachment to a specific product, meaning that they can keep the contacts whatever you want. If we're talking about the path system, you need to have the contact in the path broker, so you pay for it, but in the spring Cloud you have it in your rigid. You can have it in the file system. You can have it in your local folder or whatever. So there is nothing that you need to pay for you. You will provide it.

J

Okay, but I mean it's free, okay, happy news and also it's uh it's a polyglot support different language, so I can have it in the back end working in Java and they can have a team in for the front end working in in node.js, for whatever thing I recently have so the test will be defined and created for both for both people. You can have another team working in Python that will we also receive the ground testing, so it's open to whatever language right, but okay. This Comfort is about about a Sync, API and contract.

J

How these two content, how these two walls can match? Well, I think API is as an expedition interface. We Define how the communication will take between different apps contracts will provide the necessary tools to ensure the communication is the effectivity so how it can be done? How I should I can use the power of those tools of those specifications for guaranteed and messes are provided and are working on. This system is perfect.

J

Well, let's go into this another another thing: they are prefers, paneling, meaning that that you should Define first DPA, then we should create the content, and then we start working on building our Solutions. That can be a bit uh controversial right because we will have a working API defined it by the by the ape petition interface.

J

We will have a framework for testing the service, but we should keep updated different set of files once that we Define the APA and the ones for the final contracts and remember I, said you define the contract in general in Ruby or in Java files, but the Sync API is made in Json or in Channel. So is different. It's more work for the for the teams in order to to handle this.

J

So is there any solutions? It's complicated. We face that having the single Pi files in one repository and then the contact another repository and we were, we were updating the Sync API and someone forgot that they the contracts, so the tests are failing. The products start not be deploying so usual like a really headache.

J

So we found a solution. We create the multi-appy tools, fantastic.

J

So what we have here, we create a a spring Cloud stream, multiplayer plugin, maybe an engrader that generates code based on a Sync API. So we can generate our own code, Java code for the moment, producer and consumer, and we create a spring cloud content, multiple converter, so we can convert apis based on open APA and a sync APA into real contracts.

J

So at least we will have a set of contracts based on those apis from scratch, without nothing to do just only configure the system for that, so the ACA multiview converter can work without the multi-apa generator. So, let's focus in this component. We passed this thing API file. We create a Content based on the defendant operations. We create a stat based on the sample, provides from the asyncabe file or in case you don't have defined any examples, because you are in the earlier stage of the definition we will Generate random data. Matching this definition.

J

Ideally, you should put examples on your files in order for the comment. Okay, remember that, but you know sometimes it doesn't happen that you don't have time or there is not enough definition, but you still your teeth. Need your teams need to be working quite soon because their lines is almost there, so you define the same API and keep working. So in order to avoid you waiting for we generate some random database on okay. Do you want to see how it's working? That's gonna, go.

J

Let's gonna do a demo and we'll see later how it works.

J

Okay, then, let's gonna do a small demo to see how this our tools is working, so we're gonna show you how the apps can create their own business and conquer the world.

J

So now I am moving to the presentation thing here. We are so, and you can see here, we have a single Pi version, 230 graph awareness of info about the banana petition service. Okay, this is our API, and here we have a couple of channels, the account channel, so we will have our app account that will send through different system.

J

So we have our petition. So here we will receive our petitions. Okay, and here we will receive the new accounts, so this will be an app account, so this channel will gonna pourish all the information about the new about the account that have been created and have been updated. We received the petition for bananas in this in this channel, so we are subscribed here and our account system or user system or whatever system that Apes have in place.

J

They will send the new account uh using the the same AP Jammer here so check it that we are referring a new file that is here in this place. So let's gonna see what is inside so here we have an app account event. This is an object with those properties, normal async API file. Here we have a pattern that we use later on, but you can see that here we are defining something interesting. Okay in this one, we have the banana petition.

J

So here we have another object, it's a normal object, so we have an example: okay, yes, it will do it for you. So have an example. Every time we Define them floats for for the amount we're defining the the petition tied at this car cost credit debit. We can use also paper for the Apes. They are very modern in the lifetimes. Okay, um nothing else here. So yes, my palm my palm file, we have here um two things.

J

First, at all, is our Maven plugin for Generate the code. So this is the group ID artifact. This is the standard configuration and here we can specify the spec files.

J

Sorry, so here we are. We are specifying the file for our Sync API. We are choosing the IDS of the operation that we we will consume and we Define the idea of the operation that will create an.

F

J

Bridge, meaning that we, though we will not follow what is specified in the given API by default. We will say, if you have a publish here, we will get a producer. If you have a subscriber here to, we will create a consumer, but we allow in case you are using this API in the other side of the of the, in the other part of the communication, to specify to produce in the Subscribe in the Subscribe operation. So with only one file, we can cover both sides of the of the Saga or different sagas.

J

We configure here the Epic package and the model package, so all the classes will be built in these in this path. The model will be here in this path and the same happened with the stream Bridge. But what about the contract?

J

The contest here? We are using spring Cloud contracts, so our converter is set as a dependency, so automatically we will press. We will get the file Define it here in this folder, okay and we will create all the the tests necessary for that. The package that will be used for for creating the testing classes is specifying here. Okay, one thing that we need to to create is this content very fair base, so this class will be used for create and for running.

J

The the test in the producer part for the consumer is is another word, is different class I. Don't have it right now, but for you can see how it works. We can see a producer and we can see the validation for the for the the consumer part uh one thing interesting and we will create a.

J

Let me put it in that way, so we can see each other okay, so we have here the operation, ID store, petition, new account. You will create this function with this name based on your calcium, plus the validation suffix. So here you can put the code view for validate the message that you are receiving in the input same happened here.

J

Okay- and this is a comment for later- something that I I will need to show you so here uh when the test will be created. It will call this store petition once the data once and stop is injected into the system for this specific operation ID.

J

In this case we are testing the producer, so we are producing a message with this specific uh class, this specific entity, so here my service will convert it to the to the message that we will create with the with the employee and will validate that this Fields match the the schematic that has been defined.

J

So, let's hope how it works. We're just gonna. Do the standard, Living, Clean style, close fingers to see something is, is broken or not, because this is a live demo. Each.

J

So things are running, we have some warnings here, because by default the spinning cloud is also reading a yaml files. So, but we got here some Sun warning that the file will not be able to be processed because not is not a standard screen cloud and contract file. We are working on it. We still have plenty of things to do, but what we want to show you is these two things, so our generator has created the emitter and The Listener for the estimate producer.

J

We have this producer here and he is waiting for offering this demo this method. Sorry, so you can have this 7 and will produce into the account that has been defined here in the channel. As you can see, excuse me here, so we we already have a code, so can be used to review messages and inject it into the channel.

J

Excuse me regarding the subscriber.

J

We have a subscriber class that configure all the wins for the subscribers okay, and we offer some interface that you need to to implement this interface. What they have is the method Define it here for the starter petition and for restore the new account: okay, let's wanna close those ones, and so you can see based on the APA that we have definite here.

J

We have the store petition from tune to start with the celebration ID. We will receive a banana petition event that is defined here in this in this class. That is here, okay, and we will call the the object that you have implemented following this interface. Okay, this interface- we can see here all we have in this method, so here implementing this interface, you will introduce your logic business into the system, so you will receive an event that has been created as well from with our plugin okay. So this is Json.

J

We have here the end type with the values we have all the methods, so you can have the the Builder. The Builder is also implemented here. So all the code you need to to work with this um with this API is already implemented and offer it for us to do okay, but what about the contracts? Because something I was telling you that is Magic. That is fantastic, that is wonderful.

J

Contracts are generated here in this piece of code, so first of all that we got is the contract that we created in a general form. So this input and output message is a spring Cloud Contra Journal file con contract, sorry for the contact of the Contracting by default. If we don't have a radius or we don't have something to to follow, we added a standard stream validation for the string that we are adding. If there is a number like here, we added their their rejects to validate that.

J

This is a is a proper number, so here we will not receive a word or a letter or or something we just will receive uh a number if we are trying to push something different, uh it will fail since here that here we have our Regis for testing the Ico uh date time, but how we can ensure that this is working well here we have the contrast class. This is extending the verify that we showed you before.

J

Let me close here and there, so this account emitter is triggering and uh method with this payload. Okay, so here we have. The, when part is when triggered this function here, the very five of the nurses is receiving in this destination. That is also defined in the. Let me show you here we have defined the channel, so we are linking this class to the same channel that has been defined here in the in the async API.

J

We are converted from this account emitter journal, and here when we are, what we're doing is getting the Json from this specific response. Okay, once we have it, um what will happen? Is these measures will pass against the region that we have defined it in the account emitter?

J

So here let me put it one side by side here we have so this ready that we have defined here will appear in the Matched part of the string, so every field that has been created and push it in the in this schema will be validated same happened with the numbers same happening with the string and also the same happening with the time, the time that we have here in a SEO format right. So this is regarding the the producer part that we have triggered here and call it this test.

J

What happened with these two, as we told you before, when you create a consumer, that's only a consumer without output method, meaning a sync method.

J

So we will create a message and we will put this message into the system, so we create the implementations based on the payload with the headers, and so we inject this message into the same channel that has been defined here. Okay,.

J

Following the content is that has been stipulated in this file here that we can open right here because it's besides beside us, so why I am doing that, because these measures, this contact function, what it makes is validate that these input messages match this uh specification that we have here. We have any match in this case. We don't have any much. So it's okay. We are doing what happened here once the message has been sent into the channel. We will call this this pattern here.

J

The store, petition, validation, the stop Edition validation is one that I implemented here. So using the verify from mojito I will catch up the the parameter that call it from this method. We get the value and we make the expectation. So you are sure you are asking about about these two lines. Well, let's gonna do a break here and go to the code in the code.

J

We have our listener here in this case, is the banana listener, so my bandana listener is this Cloud that is implemented the stock Edition that we've we saw earlier that will receive the will receive the the banana petition from from the channel, because this is a beam that you now what is.

F

J

Okay, so here we have the petition process service that will handle all the petitions. So once this method is called, we will save the petition. We will increment the counter and we saw logins on because we will implemented your specific business logic right.

J

The thing is that, because we have defined that as mob being, this is a mock, so we are capturing the parameter that we are saving, so here we have a mock. We are calling the sales method, but here this object that we are creating here is captured by this uh argument cutter here, so once we get it, we can validate and assert all the values and blah blah blah. So it happened with a new account validation.

J

That is the next method that we have here. Okay, so again we create the contract, the contract also validate that the measure that we created is is, according to the definition here is sending into the channel. We receive it in our code because we have here a mock. We capture the the input we get, the value of frontal and the validate that is not lower. Okay. This is not a super.

J

Complex validation, but this is only for the demo if we want some complex validation, we just need to open and commenting this code and do some import thing here and if, let's see is this one? Is this one here as well or not I? Think I make a mistake here. Yes, because this is the event one.

J

So now it should be something like.

J

A be something light: okay, Finish will be okay, point.

J

Petition type, so now we are a square okay. So what happen? If I run this this here, because we can run it the straight away from here, so we already have the code and inside the unit code the IntelliJ started, doing his magic, compiling and so blah blah blah. So we start working and.

J

If we have a plan, we can start him doing the the noises. Okay, all tests are fake, our passing, but this one is failing. Y is failing because because because the validation is failing, because we are expecting to find a 10.45 as fluid, but we are getting these kind of things. Oh, my goodness why this can happen? Well, you can start discussing later, but this is like a problem with Java regarding um how it represents the the flows, and so this is a problem that you need to solve.

J

I mean doing the proper comparison and here, but let's gonna do from something more easy. That's gonna change here for the 10, because we want here at 10 and in the banana petition. We can send here attend, so he I make another install.

J

I will get a flooding colors.

J

Compiling compiling doing the test- and this is fairly old because they test those- you know you can see that we are really really testing what we are expecting and what we are we are receiving now he's complaining, because oh the dates is something that is always old.

J

Okay, what is failing here, the dates so we have here and I played the null, and here we have a moving day and a pleasure date. So let's gonna go to my validation, thinky.

J

That is here. So what we can see if I am receiving uh banana petition entity that has this kind of validation field that is, but here I don't expect to receive any data. In that case, I am receiving data. So this is this should be my minimal data to validate, because it's the the thing I am receiving, but here I don't consider anything any data.

J

I can add it here. Some some functions to validate the data in case this is random data or I can come here to the banana petition. Another is an example for the data and blah blah. There are different solution, but my point here is I am receiving two fields that is mandatory in in the in the contract moment, data and apply data, and here they are not expecting so. The tests are broken.

J

Let's say that someone is modifying the the contact with some real data and adding more field that that they need as a producer or in this case, as a consumer I should receive, but I don't take. Taking care of. My test will grow, so I will note it as something is wrong. I need to implement something like that: okay, guys, let's gonna, move to a new thing. I want to show you, let's say here. Another test I have defined here: I have a account number.

J

The account number now has a pattern that need to be fulfilled. If you can see here, the account number that I am sending in the meter that someone has came to the APA and Define a format. Okay. So now everything I generate will match this format here, meaning that my contract here will move from this radius expression, okay to the new one that is in the file.

J

Okay. So now I should I should see how these things break when I executed so I validate the test.

J

And my account emitter is failing because my my expecting author is DDD and match and it doesn't match, but the pattern and one or three Oaths and some numbers. So let's gonna replace the feel that I am using salt, zero, zero, zero, zero, two three: that should make the the thing I think so, let's gonna try.

J

uh I said cute in the test.

J

Is still complaining.

J

They still come. Okay, yes, because I am I, am a big dump myself. The two is not in the radius expression. So, yes, we are sending morally the same, but we are still not matching the same expression. So here that you can see there is not only numbers it's numbered, but with a specific format. Okay, that's gonna. Try again.

J

To see how the thing is working, I cannot um tweak system.

J

Okay, now he's complaining because I expecting one two: three: okay, okay, that's my problem! The problem is the piece of meat between the table between the chair and the computer, so this person say number between three and nine okay and a two number between three and nine and A2.

J

Let's gonna try again, hopefully I will be kindly smart. Maybe a smart ape.

J

Flooding colors now we are generating a contract and an object that is matching all the specifications that we have here. So, as you can see, just in real time, I can verify that the information that they are sending is the one that I am receiving and you can be able to get your teams talking together.

J

The front end the back end or the other microservices and Define the the API first, and then we can generate the contract, so you can start working really really fast to working to waiting, because one thing doesn't have the producer other data they have the consumer, so I think we can ensure that that the speed of the of the development will be something something improve it and really rate.

J

So this is for us. This is the the demo that we have. So if you have any any questions, uh now is the time to to do it. So thank you very much for hosting us, and here I will be waiting for your questions. It's a pleasure to show you our tools. Thank you very much.

B

All right, thank you so much Jose for this wonderful talk, I'm sure a lot of people have learned from you.

M

B

Would you like to say something to wrap up your call you're a talk. Sorry well,.

J

um Sorry we have those two tools in our GitHub, so it's open source everybody can use it. Everybody can criticize and also collaborate with them just because we were working with different apis. At the same time, we decided to make one tools that handle all of them. That was.

H

J

Main reason that we create this multi API tool, so you say absolutely interested under development. There are still Mass things to do.

B

Okay, great, um would you mind sharing the link um on slack where everyone can get um the tools and the resources? No.

J

Problem I will share it right now.

B

Okay, good um for everyone and you can join us um under the conference. uh 2022 um Channel user will share the resources and the links over there and if you have any questions, you can also send them through slack you'll, be happy to answer all your questions. Thank you so much Jose for coming through today and sharing your talk as well.

J

Give me the opportunity to talk to you.

B

Thank you very much. I hope you enjoy your day or morning.

J

Afternoon here afternoon,.

B

Okay, great, thank you. Thank you all right, so we're now moving on to the next talk, um Laurent I hope, you're ready. You can wear your hand if you are okay, perfect. uh Let me bring you on stage.

O

Yes, I can see me.

B

Okay, great I can see you as well. How are you doing.

B

O

Are you doing yeah, very fine and a good afternoon or good morning, everyone it's afternoon here.

B

I'm also in the afternoon, but some day in the morning, difference in time zones.

O

B

All right, so Laurent will be presenting a session about a Sync API recipes for Edie, Gomez, I. Hope I got that right, that's perfect! All right! Would you like to say something before we start your session.

O

Yeah, just uh maybe some words, but I, give also an introduction at the at the beginning of the recording.

O

So uh he has the the goal of this talk is to give you um uh trying to give you a wide overview of the different use cases where you can use Eda and Sync API, because from my experience in the field, I I tend to talk to people that are very uh uh that that were very focused on specific use case and most of the time, we're forgetting about any other use cases where, where we can apply and use a Sync API. So this is a yes.

O

This is a wrap-up of different uh different scale of of use cases.

B

Thank you so much Lauren and I'm sure the audience is ready to learn from you and I'm also ready to learn. And if you have any questions for learning during the talk you can send them through on slack or you can send them on on YouTube you'll be happy to reply them yeah sure that in mind, we're going to move backstage and we'll hear from Lauren in a second.

O

Welcome to this session called async APA recipes for idea Gourmet. My goal today is to give you a panorama of many different use. Cases where event driven can bring benefits in your complex Enterprise information system.

O

In this session, I'll first give you my opinion on what Eda stands for and what could be an adoption map using a Sync API specification we'll again go through different recipes, but I like the features of a Sync API that can be used in different use cases and also during the talk I'll try to illustrate those features in a pragmatic way using a tool called micros.

O

So, first a little bit more about me. My name is Lauren and I'm working at Google on daytime, where I support fintech startups scale-ups in their in making their application design. Cloud native I'm also involved in open source and spare time being a member of the async API TSC and the leader of microsmore heard that in a few minutes before joining editors or Cloud providers, I was working as an Enterprise architect in financial services. So I think I have a very pragmatic background.

O

Very pragmatic experience on many topics related to distributed computing, Enterprise, Integration and API programs, and, as mentioned my talk last year, I like gardening as a hobby but I also like food as most French people do and this topic of of recipes, so maybe you're you're angry, but before dropping to the recipes, just go back to the fundamentals for a few minutes.

O

What's Eda and it's pretty simple, it's Heaven driven for the win and even driven it's a parodying man, bring new Concepts, bring common Concepts that are all about decoupling, decoupling in time, decoupling, the technology, but also decoupling, the the space, the location and it brings a new Concepts like publisher sender, of producers like uh subscriber receivers of consumers of events and, of course, the main one, the the stimulus or the the event that are triggering messages.

O

But, throughout my experience this last year, I also come to to a somewhat provocative question. When discussing with customer or colleagues. What does the a of Eda stands for? Of course everybody knows it's for architecture, but uh what does architecture mean to you? Is it technical architecture? Is it related to the the logic of interaction?

O

Is it the architecture of your information system or the the organic design of your application and wouldn't the amine application? Sometimes if you focus on interacting with the user, so it leads to the question: what's an application if you're looking for a user perspective or lifecycle perspective, but in short, those questions make me think that there are different levels where Eda can make sense.

O

Yet Eda, Is, Not, A, New, Concept I, remember first meeting it nearly 20 years ago, when we were talking about SOA service oriented architecture, but at that time Eda was quite different and was mostly seen as a as a carrier pigeon.

O

It was not that considered reliable, remember it was just the beginning of free class system and the era of two-phase commits and at the end of the day, people tend to prefer pure web services or batch exchanges that were way easier to to operate at that time. So Eda was just on the test and on a few use case, mostly one usage, exchanging flows with Partners or Legacy stem, and mostly in financial services, but and also even there is very similar, all pigeons or or EDS Solutions.

O

There were very different because of the lack of standards for describing their usage, but luckily, as you may know, with this conference and as you may now all be aware of a Sync API saved the day, really it saved the year, because this spec has this fantastic power of allowing us to preciously describe every situation.

O

What we used to have in traditional Eda are the information system level represented here by this API. That is triggering an event that can be consumed by an email service on a CRM service.

O

It also allows us to manage what we see new industry, iot Edge Computing use cases at the application Level having a distributed applications with a central herb and distributed devices. For example. It allows you to address what we have in modern application: interaction like streaming applications having an API that stream event to a browser or a mobile phone, and yes, it also helps specify complex or long-running workflows, like loan application like mod cache application and so on. So you see that it can apply to to a wall range of use cases at very different levels.

O

L all these examples and others makes me think about some kind of adoption map for Eda, a map that would Encompass the application, Level or tactical use case, as well as the architectural level or strategic adoption drivers, and it could go from local consideration that may be driven by user interaction or deployment, because, after all, as soon as you are doing, serverless stuff, you're doing events, even if you don't really see them, if you don't really specify them, and this can lead you to more uh Regional ones, trying to optimize resource consumptions or access to Technologies and then to more uh govern one where you can find the the most large-scale application, modernization, initiatives or techniques like transformation of monoliths to microservices, for example, and finally, to Global ones.

O

When you really see the the heavens as the building blocks for supporting your I.T transformation or enabling more agility into the into the business, huge category comes with its own victories of badges. If you manage it as a progression, but don't get me wrong, I I, don't necessarily see this as an adoption path. It's a map and, in my experience, I've seen people that were really well addressing the path on the right, but totally neglecting the one on the left.

O

So my opinion is that there are benefits to addressing all of these and, of course, with great Powers come great responsibility or challenges, and it's not that different with Eda, and we see many two difficulties on the field.

O

One comes from the the multiplicity of protocols and formats that are supporting Eda I will not say we're still in the carrier, pigeon era, but we now address a wide variety of use cases and we know that one size doesn't fit all so we end up with a lot to to handle, and you have once crammed from the the consequent complexity of fine, very often, environments for developing testing, viewing, QA and so on, and because we have decoupled things, we now have a ton of run times to to manage.

O

So the latest element I want to introduce before talking about recipes is this microx project I'm working on this is of greatest importance as I truly think that even the greatest spec needs, pragmatic, tooling, to unleashes Value to a reactor on the life cycle and in a few words. Microx is a tool that help to help simulation and testing of all kinds of API, including the the even driven ones, for the support of async API.

O

Of course, how we are doing that, and we are doing that by leveraging the power of contracts an example contracts, because we are using a Sync API and it provides all the syntactic rules in it and examples, because uh examples provide illustrative real world scenario for free and they can be embedded into the spec as well most of the time in development we we neglect or or in design phases. We. We neglect the importance of examples.

O

We let a tools generate examples for us uh and we have basic examples having a random values, having uh always the same values, but we do think that it's a best practice to pay great attention to two examples, because when they are carefully chosen, they can represent real life and business expectations in a very uh clear way.

O

So in microx we're um we reuse these examples to First, provide you with simulations or mocks and we are able to turn them into actionable assets so that we reuse them to realize conformance testing or contract testings.

O

So that's a few words on my Crocs and you'll see it in actions in a few in a few in a few minutes. So now that we have our three elements: Eda async, API and microx, we can deep dive into recipes and I specifically choose six recipes.

O

That I think are the the most common ones and but every architect every every practitioner should be aware of foreign as I try to make them a bit generic to to Encompass the most situation. Possible. You'd probably need to put some context in here, but I hope they will really represent actionable patterns that you may uh apply later on.

O

So my first recipe is the the examples you you always see when dealing with a Sync API, it's dedicated to user interaction and how Eda and async API can help enhance the the user or the the customer experience, or at least the experience of the developer. That is developing enhanced user or customer experience. So this is a mini receipt to to set up the structure. That would be the same for each one. So we'll talk about the.

O

What what are the Eda patterns we use and the motivations we'll go through the Sync API ingredients on the right, the ingredients that can be used to implement them? And finally, we'll have a word on how microbes can help in those situations.

O

So in this recipe we are looking you're looking to make a better user experience, pushing events in real time to the user through full browser or a mobile device, and this one involves the definition of a subscribe operation using async apis async API, subscribe operation means that the users of the API will need to subscribe a channel to get the events to get the notifications to get the messages.

O

But you can also specify user LED interactions using Eda and async API. Imagine you want to implement a chat room or something like that, and this time you have to Define publish operation on the channel, so that users can send messages and because a Sync API this notion of protocol bindings you'll be able to precisely describe the one you choose.

O

It can be a simple HTTP for server side events for notifications or it could be AWS a simple notification system for for mobile or even web sockets in the case of a chat, so a Sync API is very flexible for that. It also allows you to handle different content types for your messages, but in that case, you'll probably end up with Json encoded messages.

O

How can microx help in that? Well, let's focus on a simple point for a moment. Microx will be very helpful, allowing you to to speed up the work of the fronted team, while the backend team is developing the the notification system, for example, so we are going to to see how microbes can bootstrap your async API demo time and hope the Democrat will be with me today.

O

So let's switch to to the micro. So my this is the the UI, the dashboard of microx presenting you, the different apis which are in your system and micros, is very uh contract. Trust driven. We promote contract first approach, but we recently developed this new feature to allow you to having a rapid feedback loop on a new API. This is called direct API direct API allows you to choose an API type here. We are going to choose 11 driven API and we are going to publish a notification API.

O

This will be her first release 0.1 and we are going to handle notification.

O

Simply and with microx, as it is example driven, you can just specify one example that is the the reference payload but will be used to produce simulation and to produce your async API. I can use specific notations in micros. Here, for example, I want to generate a universal, unique identifier for my uh for each event, for each message- and here I want some Randomness, uh generating a first name to greet the the newcomer in the chat.

O

So I'm gonna hit the next button and if everything is okay, yes, I got this new notification API that just right here, so micros has bootstrapped in a Sync API specification. For me. Okay, this is your Sync API file and you can have access to the documentation.

O

It has generated a subscribe operation with different bindings and you can see that by default we have a websocket binding just right here, so copying the the websocket endpoint right here and just testing it. I have developed a basic chat.

O

uh Front end and I can connect to my micro service and hopefully start to receive messages.

O

Not yet, let's refresh things.

O

And hopefully it doesn't work? Oh yes, it works, so you can see with my box. I can really uh really quickly bootstrap my Eda project really bootstrap my async API project, whether for web sockets or other bindings. Here, for example, the Kafka bindings I will talk about in a few minutes, so this I have uh some endpoints that are generated for me, so that ux team can start working on and I have also a basic contract that has been generated for me and I can start to to enrich it to to complete my use case.

O

So, let's get back to the to the presentation and move on to the second recipe, how hidea and a Sync API can help on resource usage optimization.

O

This is very hot topic today, as most traditional companies are moving to the cloud and they are trying hard to to lower the cost due to the current context- and this is where Eda can be used at an application Level as a new as new programming patterns emerge. So we are talking a lot these days about reactive programming, that is using techniques like non-blocking IO when talking to databases and dependencies, but reactive also introduces a lot of asynchronous structures to avoid having threads waiting for processors in the in the chain of invocation.

O

So in this type of design, each processor you can see in the chain will benefit from clearly defining its interface through an async API publish operation. Of course, you will have to prefer to proceed in reverse order. Last processor publish its interface and so on reverting the the chain, and in the case you were implementing the request response paradigmund this way this is something we we used to called stage event driven architecture when using queuing technology in the past.

O

A Sync API also allows you to clearly Define a correlate correlation identifier, but may be used to to match the outgoing response from a subscribe operation. This time for this type of situation, you may want to to choose some kind of traditional broker like rabbit and Q or imqp, as those patterns may require durability, but not the long-term Persistence of messages so also on the side of message: content type, A, Sync API as also first class support, forever messages that are binary encoded.

O

So this is a very interesting for performance here, because messages are internal to an application and not necessarily read by humans. So you can have a bet on average for performance.

O

Regarding microx usage, in that case, you can easily use it for testing different processors in in isolation to to each other. Therefore, it becomes a straightforward to define a set of example, message corresponding to expected input and output and using microx testing features. You can really spot buggy processors in your chain. So let's see that in action with uh with our use cases, so um as a developer of a processor, I can easily get incoming messages without having to wait for the previous processor to be to be ready.

O

So, uh if I go back to my notification, API I've used previously I can, for example, see that there is also a Kafka binding that has been provided and I can easily start to browse the Kafka topic that was created and managed by micro. So as an example here, I can just here using the the kcat um here is the KitKat tool to see that I start receiving messages and I get start developing my processor so very, very easy.

O

Now, um let's imagine I am I have developed. My processor I have developed my processor and it is. uh It should respect this user signed up, API, okay and I can test this processor very easily. So, let's imagine I start this processor.

O

It is a producer and it produces messages and on my topic on a local topic, called user signed UPS, you can see here an outgoing messages and using micros I can easily uh test it out. So, for example, I can launch a new test specifying a Kafka, um a cafe Point here, the endpoint that is used by my application.

O

I can say that I want to check this operation and I'm using in a Sync API schema Runner, and what microx is going to to do is to connect to one or different brokers in your uh in your ecosystem in your environments, and it will start to collect the different messages that are sent on this broker and for each and every message it will receive it will check If.

O

The message is confirmed to the schema, and here you can see that my processor is correct, because the messages that he is sending to the to the broker are conformant to the schema. So it's very easy to spot the the buggy processor to to check here that everything that your produce are confirmed to your API and interfaces you you define so really easy to to test in isolation.

O

So now, let's go to the to the first recipe that is about how hidden a Sync API can help Legacy and Technology integration.

O

We all have Legacy Parts in our information system and even if a Sync API is quite new, I saw very interesting usages of of it in Legacy context.

O

Technology integration is quite related to this as it's basically the same concern of integration and managing unreliable Network. That pushed us to be to be asynchronous. So the first use case is about Legacy integration.

O

You remember it was all the mq series, Webster and Q Brokers that allow access to your Mainframe. This is what I'm talking about and I see a lot of people, retro engineering, those chains to take advantage of formal description and tooling acceleration that may that we, they may have with a Sync API eterogeneous.

O

Unreliable and long distance networks are also now a very common case in Industries, smart cities or hedge use cases, so this can really benefit from a Sync API being there for describing the the common part of the interaction, the the command that are that you that are sent by the help to the different device.

O

So this can be done through subscribe operations and it can be used to to describe the the data Gathering part through publish operation for for describing the the data that should be sent by the different devices, for example, and here again, multi-protocol support in a Sync.

O

Api is really big thing, as we are able to support things such as IBM, Q, amqp or mqtt protocol bindings on the message, content type side of things that XML is still a big thing regarding those Legacy systems and a Sync API can handle a XML message as well and of course, Json rises in many hatch cases where the interoperability between different devices or different device. Vendors is the key.

O

One thing that I really wanted to highlight today is the usage I, see of micro simulation capabilities on the field. uh Having issues with the cost and availability of Mainframe partitions for testing having concern about the availability of remote devices, all this can be easily simulated using a Sync API example supports in microx.

O

We see many people be building a complete partner, sandboxes using simple but significant mocks with micros, for example, in the the banking World. A lot of Banky are providing open banking sandboxes to allow their partners and their users to to play with their apis and instead of connecting those sandboxes to existing services on Mainframe. Some of them are have replaced the these expensive sandbox environments by just mocks that are served by Michael, so really really a cost-casting uh costing solution.

O

uh Okay, just switch to demo but go to the recipe. Number number four um recipe. Number four is how Eda and a Sync API can help modernizing a monolith in two different micro services, hence managing the the complexity that arises.

O

Let's say you have the need to externalize some modules of your application for a better scale or for an easier way to apply changes or refactoring it and extracting microservices using Eda can be very, very handy.

O

One nice thing about event driven is that it allows you to to decouple the place where events are consumed from the place where they are produced. So if you have introduced events between services and modules, you can transparently extract the modules and still have them consume events, and it's then very easy to retire existing modules in a in a very Progressive Way as a side effects. Please note that events may also be useful in microservices work to to break the dependencies to improve the uptime and the resilience of an application.

O

The drawback of this being that you, you may change the way you handle transactions switching to more Loosely, coupled methods like, for example, The, Saga orchestrations.

O

This may be judges of a kill, but it's really help anticipating the the scale up of your application. The scale up of your of your traffic as you refactor into micro services, so really nice thing of uh a Sync API, is that it will allow you to to to precisely describe your orchestration and your composition uh process. You, you may Implement using Saga.

O

Embedded into your Sync, API spec may be totally appropriated for that and uh you can see utilize you can use bindings like rabbit and Q or or not, if you're, very Cloud native you still, you probably don't need long Persistence of messages. In that case too, on the micro side, we can specify that we tend to see more and more the need to use local development laptop situation that.

O

A developer that can't or doesn't want to be connected to True complex systems, for example a case of Mobility or offline situation, and for that we we have also developed a Docker extension for that. So uh just to to give you a brief overview, you can install micros in a centralized way so that the same instance can be used by different developers, or you can easily use the docker desktop tool and browse the different extensions in the marketplace and install the microxim tension.

O

That way, you have a ready to launch micros and it's just as easy as eating the the launch button, and just after that, you get your your locally running instance ready to be filled with any mock. You need for your local development experience so very handy to have this new way of deploying and using micros recipe. Number five is all about pursuing this modularization, but scaling it to the organizational level by adopting some kind of domain or product driven orientation.

O

The usage of Eda and events tend to be generalized as the de facto pattern for East-West Communications once you cross the boundary of domain. Okay, I'm really talking about cross-domained boundaries, and this usually goes hand to end with the adoption of choreography as the way to to assemble multi-domains processes in the organization. And here again there is a lot of interesting things in the pattern called Saria choreography.

O

So I really really encourage you to to have a look at this also to to notice a great approach that we have foreseen organization is the definition of all the the capabilities of of a domain through manage API. Could it be for a classical rest, API for the command or graphql API for the queries or a Sync API for all the things relating to access of unit or data in unit chunks or in a streaming mode?

O

Most of the time we see that the data related capabilities are provided by Common infrastructure, common Services, common team of attenders ingestion, cleaning, anonymization and so on, and I really think it's also a great fit for a Sync API. A search service provider can easily describe publish operations contracts that can be simply reused by all the domains that want to consume these centralized services.

O

On the protocol side of things, the async API can support Kafka bindings, as we already see. It fits particularly well with the injection and streaming use cases, but uh please note that for this use case, the deployment of a schema registry with either a Json or Avro looks like like a mandatory step, as schemas must be available and governed across the organization.

O

We specifically added the schema registry support for Kafka in the latest async API release, that is the 2.5 release of the spec. So you should really have a have a look at it. How can my Crocs can help you with that? It can help a lot because of its multi-protocol support. We mentioned Kafka in that case, but where socket is also of interest, if you need to to expose some data to to the outside of the organization, because please don't provide external access to your Kafka broker.

O

Use web sockets instead, for example, and as micro supports, both really really easy. It really is the pain testing, the different Exposition, but microcan also help with the schema registry integration, as it is able to to connect to the central registry for automation, of the the publication of the schema updates, and it can also connect to distributed registry to to test that the records read from different Brokers can be correctly deserialized using a corresponding schema.

O

uh Let's see that in action uh here so, for example, I got a schema registry that is connected to my current microx instance, and this registry is empty at the moment and on the micro side, I have preloaded nothing API that is handling overall messages.

O

Okay, so you can see there is a in a Sync API spec and there is a network schema that is attached to this. Spec here is the schema, so uh microcs can publish mocks message for now they are disabled and you will see there that as soon as I enable the publication of mock message, it will also handle the synchronization of the schema registry with the corresponding schema. So, let's, let's check with the property Edition I can now enable the mocking just save here.

O

So it should start mocking sending new messages and as soon as it has started, mocking if everything- okay, yes, it has a synced registry with new schema that is attached to the Sync API specification. So it's very handy to ensure the propagation of your schema on the registry and as microx can be also connected to your git repository. As soon as you have updated, your your async API file into your source of Truth git Repository.

O

You can be sure that it will be propagated to your schema registry as well, and here we are finally on recipe number six and has a cherry on the cake. I wanted to finish with this one talking about how PDA can be used in large scale to provide some help in a move to Cloud transformation, because in move to Cloud there's, usually this problem of um data gravity.

O

You want to start with the easy part moving one application to the cloud. The data is still on premises used and produced by so many hover apps. So, as said before, one nice thing about events is that they are easy to replicate and you can decoupage where the data is produced from where it is consumed.

O

We we, yes, we already saw that, and in that case, let's imagine, I have my on-premises and my Cloud environment, and the first thing that we have to do is to to set up a replication of events to the cloud or to the clouds of my choice for that I probably select a globally available work broker, like the one from Soleil from Google pepsilver from AWS or even Kafka, with Milo ring, for example, if you're managing different regions and the nice thing is that we support all of them in a Sync API.

O

So once the replication is done, you can start deploying new consumer C on your cloud provider. This is transparent, totally transparent.

O

Then you can move adapted consumer B on the cloud before retiring it from the on-premise and when, finally, you are ready or you want to move the service that is on the top, you just have to to move it and to revert the replication okay. Now the master is the cloud environment and events are replicated to the on-premise environments.

O

Finally, you can retire your on-premise services and maybe the consumer, a that uh that sticks on the premise or but will stick some premise forever, continue to receive environments in a very transferred way, as with the previous recipe have a role in Json with schema registry are perfect candidates for that, but not that it may also be of interest to consider something like Cloud events that can ease the the registration or the the interior.

O

Interoperability of your home events with the one of your Cloud platforms you chose, and luckily async API is also compatible with Cloud events as I describe it extensively in a previous talk last year. So you may may have a quick search on YouTube search for my name and the cloud events or a Sync API, and you will probably find this this detailed video showing how they can be combined to yes to to provide great power and great interoperability.

O

Finally, where to use the microx in this picture and I would say everywhere, of course, because it also supports the different bindings and it's for cloud-based broker. So it could be of great help for anticipating and preparing migrations, as well as ensuring there is no regression once you, you've started moving components or revert replications, for example.

O

Basically, you can see micros everywhere, so uh uh time is flying and the time to to Rapid things so uh I say it's a journey and if I succeed in my demonstrations, uh You Now understand that Eda plus a Sync API plus microx, can be a very helpful combo in a lot of situations.

O

From purely tactical to more strategic approaches are said in the introduction. I chose today to have a very Progressive approach, but um this is not necessarily a path. This was easier to have this Progressive approach to to introduce concept one by one, but you can really start from the right and then also consider recipes on the left starts having a global approach and then a local optimization approach for going the other way.

O

It's really up to you to let's say to be your home menu, depending on your on your scope of action, depending on the maturity of your organization and and a lot of event, Criterion that that matters for you just finer words of microbes, we open, we welcome everyone to come to share ideas and to contribute uh the project is purely community and usual driven. So this last month we implemented a lot of stuff regarding uh users requests regarding the supports of new broker the supports of new approaches of test time testing approaches.

O

So it's really up to you to to come and join the the community, try it and give us feedback- and this is the final words many thanks for for attending this session. I hope it has been worth your time and I hope to to see uh many many many questions in the chat, bye.

B

And thank you so much Lorien. This was very informative and I learned a lot about async, API, edas and macrocks and I'm sure the community and everybody else learns a great deal from you and so to wrap up your session.

B

um What's key takeaways, would you like to mention or to give to the community and to the users.

O

Yes, so um just to to repeat uh some piece of my conclusions: yes, I wanted to show here that there is a lot of different use cases where Eda and async API can shine.

O

uh I I realized that, during the preparation of the talk that there was in fact, a lot of other recipes and I have to to select and just to pick six of them and now I've got a lot of material for a blog post series and so on, but yes, I, basically I wanted to yes to show that it can go from very uh scoped, very narrow concerns regarding user interaction to very broad, very wide strategies around microservices migration, Cloud migration and so on. So yes, I hope you you've enjoyed this.

O

These different recipes and I think there is a lot of material for some others.

B

That's true, and if you want um to speak to Lorraine, if you have any questions, please join using API, snack, I'm sure the link is shared on the YouTube platform Twitter and the socials, and you can also DM um I'm sure. If everyone wants to have any questions for you, you can DM. Lawrence is available on slack as well, and you can also check him out on hashtag conference2022 under async API. Thank you so much Lauren for your time today and I hope to we hope to see you next year as well, with more recipes.

O

Hey I hope, so thanks a lot for the opportunity.

B

Thank you very much and with that in mind, we're going to take a two minute, breather and Usher in our new, our next guest Ishmael, who is going to give us the next talk. Please stay tuned.

B

B

B

B

On our next talk, you'll be giving us um a talk called the future of messaging um Ishmael. If you're ready to come on stage, could you give me a wave okay, perfect?

B

How are you doing.

K

Hello I'm very well. Thank you really good to be here. Thank.

B

You so much for making the time to be here as well um I'm. Looking forward to your talk, would you like to give us a summary of what you'll be talking about before we start.

K

Certainly I'll be looking at um something called delay, tolerant or disruption, tolerant networks which is being used to build um the interplanetary internet, but also it's being used more locally, so to speak in our planet, to reach those very hard to reach and cut-off communities which don't have easy access to the internet, for example, or in the middle of the ocean.

K

Ships Etc wherever the connections are very poor and it uses a an approach called store, carry forward, which is a growing Paradigm in networking and I will be exploring whether and how far async API could articulate this new approach and bring to that new community so that the two communities have delay to learn networks and the async API Community can begin to talk to one another.

B

That's perfect, I'm, really looking forward to your talk today and um Ishmael will be presenting us live so if you're ready, the flow is yes.

K

Thank you, I'll just share my screen.

K

Can you see it okay, just checking on this platform.

K

Yeah I think so: okay, well really fantastic to be here. I love, async, API I came across it a few years ago and uh became a fan of all its possibilities and the amount of progress since then, when I sort of just came across it and in Postman Galaxy and today, with the full Linux foundation and a huge community of volunteers, it's it's been a very inspiring thing to to watch and it's one of the most efficient, effective and friendly, welcoming open source communities. I've had the privilege of being involved in.

K

So it's a great thing to be here with you all and to be learning for the incredible talks and uh to be sharing some thoughts about myself. I am the director of the Adora foundation and I'm leaders, researcher right now at the green software Foundation working on the state of green software report and software engineer by background also social entrepreneur and very focused on sustainability, and particularly the um the role of tech and digital Emissions on the environment and what we can do about it. And that's what led me to really begin to explore these.

K

The late, tolerant networking as I will explore or explain in a minute, so um delay disruption, tolerant Networks. When do you use them? You use them when messaging involves very high, latency, think Earth to Moon latency or whether it's very low data rate or a lot of this connection. Think the middle of the ocean or limited connection range I once lived in a very remote Maasai Village, and they had um phones, mobile phones, maybe two in the village. They had one cassette tape.

K

I spoke about this in a in an earlier API I, think Asic API conference, but they had very low connectivity. They weren't within range of uh Wi-Fi and similar, and so um this is another use case and then what is called time shifting in carbon aware Computing. So the electricity today is a mixed thing. It's mixed between dirty fossil fuel, coal, burning plants and green, renewable, wind and solar and hydropowered plants, and the problem with nature is that you have to be a partner you're, not really an owner or a controller of nature.

K

So you can't really decide when the wind will blow or the sun will shine and therefore, during any given day, the electricity will have moments where it's mostly powered by Renewables or only powered by Renewables and moments where it's mostly are only powered by Dirty energy time. Shifting involves making sure that you're Computing only happens at the times when the electricity is cleanest and therefore this notion of delay or disruption.

K

Tolerance can be helpful if you're, a gadget or device is, um can process the information or receive the messaging at clean times and doesn't have to be on at all times. This will be another great value, so the late disruption, tolerant networking, is an approach to rooting messages in an asynchronous way, hence async API and in an event driven way, but in a way that does not require a direct, constant link or um or range of connections. So it uses something called a store, carry forward, relay routing architecture and I'll explore a bit more.

K

What this means in detail, but it's uh the idea that you don't just send the signal. You don't just send this the signals store it and then convey it to your subscribers, but you can convey it to subscribers, which then carry it on to another area and hit subscribers in that remote area that are connecting not to your original Source or your broker, but to that mobile device, for example.

K

So it is used for The Internet of Things. It is used for vehicle to vehicle communication for carbon, aware Computing, as I mentioned, and areas where there is very low infrastructure in hard to reach locations like outer space, mid-ocean and remote regions, or in sort of easy to reach locations, but uh in circumstances that impose interruptions, latency or delays.

K

So this is a paradigm that I think we are pretty uh familiar with in even driven architectures. The message goes to a streaming service and the streaming services publish it to all subscribers within range at the time. So if that message has been streamed at 11, anyone who is connected at 11AM will receive that message, because it's just forwarding it on the device number two that looks in at half 11 will not have met received that original message.

K

So it's like uh jumping in halfway through uh terrestrial TV program- you, um you are just hitting it at the point: you're hitting it, so the previous messages are gone.

K

Another Paradigm that is very familiar is the store forward Paradigm. So a queue is a good example. So you, the message, goes into a queue and then all of the subscribers will receive that message as soon as they connect to that queue. So it could be at 11 am and then someone else comes at 11 30 a.m and they get their SNS notification or whatever it is that you are relying. So here your broker is not just forwarding, but storing. So the architecture is a store forward.

K

Paradigm device number three is Out: Of Reach. It is never going to connect to that queue, and so it will never receive the message, no matter at what time it connects, because it will never be in range of that broker and therefore the store forward mechanism will never reach it, and that will be the case whether you are in the International Space Station or in um uh very remote Village in Bangladesh or in the middle of the ocean. In your Cruiser, so um store carry forward will send a message to your queue or your stream.

K

It doesn't matter, I think it should say Q, because I've got a device there at 11 30. So it's the queue and it will send it to all of the subscribers that uh that come on. So this device number one gets it at 11 device number two gets at 11 30. and there is some moving object, which is device which is a device.

K

It might not be an end device, it might not even be a kind of um a destination device, but it will store the message and carry it and when it's carrying it it will now be in range of device three and transmit that message so that device 3 never comes in touch with your broker is near in range of the message, but it still receives it via that moving object that you that stored it carried it and forwarded, and this is a powerful Paradigm in so many ways.

K

So um the internet, for example, requires all of the protocols to align in order to stream one message from one from your server to the end device, for example, and that can create an element of latency with a store carry forward.

K

You might just store the first layer until that second layer aligns with the next layer and not necessarily with the first layer, and that will mean that you can actually send a lot more messages in between in an asynchronous way and that's why it is of such interest for interplanetary communication, which is where the Paradigm started. So it began in NASA. It began in space research it extended with tarpa into defense and therefore into terrestrial applications, and then it went beyond defense and space into all kinds of uses where these uh paradigms are useful.

K

So there are a number of relay strategies um that do not necessarily reflect this. This kind of streaming API from Twitter to to the browser. So, instead of just that direct communication, your intermediary objects that moving object in between it could be an opportunistic relationship. So um if the browser is entirely out of uh reach of Wi-Fi, for example, it may be that you are relying on uh that. Moving object is a random mobile phone and you don't know which mobile phone will be in range when so.

K

This is an opportunistic relay you're, basically looking out for any device that is willing to carry your message and instead of you'll, send it potentially to lots of devices all of the devices there so that they in turn opportunistically relay it to n devices that happen to be along their route, or it could be a Cooperative relay strategy. In this case, you have very specific um devices that are cooperating so you're, not waiting on a random mobile phone.

K

You know that there is going to be a train passing at a at the same time every day in the same place, and that that train will be passing through a neighborhood that has no connection to the internet normally, but will be in range of that train for just enough time. For that train to receive that Twitter message, that's probably more important things you can do uh with it, uh but that's just one example. So here you have a Cooperative relay strategy where those machines are consciously cooperating with you to send it.

K

And finally, you could have a competitive strategy where um your there are perhaps some time pressures or um perhaps priority of messaging Etc. So you might have five potential machines that are collaborators that you know are going to go there and you send it to all five and the one that gets there first, transmits it. It's also not all or nothing. For example, it could be that um part of the message is streamed by that first strain and the last part of the message is streamed by a sort of drone that is passing Etc.

K

So there are many architectures that you can create, depending on your use case, and people have created sort of a mix of all of this, where you use opportunistic mobile phones, but you also have a relay boxes in between, so that the mobile phone transmits it opportunistically to a relay box that collaboratively cooperatively, transmits it to another relay box where you potentially have a central point.

K

That has a that treats different relay boxes in a competitive way, and this is, for example, a use case in disaster Communications, where you might have sort of Emergency Services with their own backup, Communications infrastructure in the middle of nowhere, and they are integrating multiple streams of information and then communicating a an integrated narrative back to headquarters. So there's a lot of possibilities around these, so um I I mentioned here a few examples already now. This is important. For example, at the moment, most of our uh Internet of Things is very energy and Communications done.

K

It is basically on all the time, even if it only needs to be on for a particular moment of time, and that means that it is generating digital emissions. It is uh burning through battery when it might not need to so being able to identify which connected devices could be delayed. Tolerant is hugely important.

K

Likewise, connected devices are very often mobile and therefore being able to identify which connected devices could act as carriers for signals and therefore bring in our messages Beyond those people who get to connect in range of whatever infrastructure we are using to communicate again becomes quite instrumental so um designing connected devices that can both be delayed, tolerant and also act as carrying relays both for messaging that is directly relevant to them.

K

So it might be that you have um a watch that requires to receive messaging for all of the apps that are running on its on its Smartwatch, but it could also be that you are an altruistic that I forgot that Paradigm that's another Paradigm is altruistic uh the lateral Network, so your phone, for example, your your watch, for example, might agnostically, say I am happy to take messages in general and there are there's thought around security that has been going on and in fact it's been sort of built into a lot of the networks from the start.

K

um It might be that it's a certain sub section of signals that you're open to, but essentially you are saying, I'm open to signals that don't benefit my user that benefit third parties, and my user is happy to use his watch for that.

K

um So that's another way in which uh connected devices Internet of Things up would be using this paradigm vehicular networks vehicle to vehicle communication. This is something that is growing in significance as um iot as electric vehicles as smart cities, uh smart networks, all of this ecosystem is accelerating and does seem to be the face of things to come. We are not yet at a point of having reached critical mass, so a lot of this vehicle to vehicle networking is still at the proof of concept stage.

K

But in whatever scenario we are looking at then delay, tolerant, networked communication and routing has a potentially very big role to play. So the idea would be that a car, for example, that is coming in the opposite direction from you- will have information about traffic conditions ahead of you and so that car could communicate with your car to transmit that information, which, in turn is aggregating the input from that car in the opposite direction.

K

With all the other inputs, it has access to and transmitting them to other cars, which means that the level of detail and richness that you can get in terms of Road information is incomparable in relation to what we have today. So this would make self-driving cars, for example, in conceivably, more safe and accurate and powerful, and there are, of course, uh non-vehicular advantages to being able to create such Dynamic mesh networks between cars.

K

Carbon aware Computing, I have uh already sort of mentioned it, so um this could apply to your connected devices to the internet of things or it could apply to your software. So um if a particular process or more specifically, in our case, a particular subscription to a message- is not time sensitive, then you could describe your apis in such a way that that transmission only happens at times when the electricity grids, both in the emitting the publishing side and the consumer subscriber side are at their greenest.

K

So if you could create a set of apis that communicate with each other, not just in terms of their task, but also in terms of, for example, their level of battery, but also in terms of their their emissions and their the grid and the the local electricity conditions. This could be a game changer to the um gigantic contribution that our sector is making to uh climate change and where most other sectors are gradually diminishing their contribution to climate change at the moment, not fast enough to avoid really painful scenarios but decreasing our industry.

K

The digital uh technological industry is still growing, so the data that we're using is growing the um electricity we're consuming is growing.

K

We are becoming more efficient, so we are consuming less energy per process as time goes by, but our growth is so extensive that our net emissions keep on growing, even if we become more efficient so being able to be carbon aware in our devices and describe that in some way and ensure that our communication imposes delays as opposed to tolerate delays, welcomes delays, Embraces delays, but does so in a way that is coordinated and that allows us to essentially run on green energy, which will give us more and more availability to more and more, we green, the um our our energy consumption globally, then that's fantastic and finally, I have mentioned examples of low infrastructure.

K

This is a case for digital inclusion at the moment, in spite of this tremendous Mark that we're making on the world positively and negatively, fifty percent of the planet is not connected. This is huge and given what happens on the internet, what happens in Digital Life that digital exclusion is a financial exclusion, is a knowledge. Exclusion is a gigantic Injustice and at the moment, our approach to reaching those hard-to-reach areas to connecting those disconnected areas has tended to be.

K

We just have to grow more, which, of course, is you know going to create even more problems for those areas, as they are the main sufferers of the impacts of climate change, so delay.

K

Tolerant networks are a paradigm that allows us to envisage ways of creating connectivity in areas of the world that are currently excluded from the digital conversation, without necessarily having to think about investing in major infrastructure projects in every single area and major servers, and all of these things so being able to create more distributed processes is that next civilizational step as it were, that I described in my last async API, took this sense that we are moving toward a much more nuanced reactive, distributed event-driven society and not just technology.

K

So what are the cases for async API in delay and disruption? Tolerant networking, as far as I have been able to tell I, haven't, come across any use of facing API in the delay, tolerant, networking, community and- um and this is a pity on both sides- I think that async API has a huge role to play in the Paradigm that I've just described.

K

Async API, of course, is there to describe event-driven Communications And Delay, tolerant networks are quintessentially event-driven routing protocols. Moreover, async API is protocol agnostic. This is absolutely critical, because at the moment we are at risk of a proliferation of protocols and there will be a proliferation of protocols. Delay, tolerant networks in uh the internet of things will use different protocols for different types of Internet of Things Different protocols for vehicular um communication, different protocols for interplanetary communication.

K

So this is, on the one hand, natural, and it makes sense you might not want to use the same protocol for your smart watch as you are using to communicate with Mars.

K

But on the other hand, it also creates a huge amount of complexity and constraints, because one of the key Promises of the store carry forward Paradigm is interoperability. Is this idea that my watch or my mobile phone or my car could help deliver Emergency Services information in a massive flood or heat wave or earthquake um to emergency services and to sort of up the to the central coordination side of the government and if all the protocols are not able to talk to one another?

K

Our ability to do this is hamstrom so being able to have an a language to describe our apis. That is, protocol agnostic that allows my car and my watch to talk to one another and to the emergency services and to the International Space Station as need arises, is is really powerful and really significant and being able, particularly when we're talking about, for example, opportunistic or altruistic uh routing strategies being able to say I am open to these kinds of signals.

K

I welcome these kinds of apis and I am transmitting these kinds of apis and these kinds of messages um to these kinds of service. This is really important and at the moment, as far as I'm aware and I'm, certainly not an expert in this field, but as far as I've been able to tell there isn't anything like it. There is no way there. There is thought going on into creating common standards, and there is uh work going on into thinking about this, inter interoperability, but not in a protocol agnostic way.

K

So async API offers to the delay, tolerant, Network community, the ability to talk to one another, not just now, not just for users, they all ready foresee, but for the future for all the issues, all the uses and all the applications that are emergent that will become possible and that no one can predict, because that is the nature of an event-driven society that we are using and collaborating and harnessing information from one another for uses that we never thought of before, and that the aggregation of previously on related information makes possible Innovations and applications that people would not have been able to even conceive or imagine before that data or those information streams could be aggregated.

K

So async API provides a language to be able to aggregate these kinds of communications and informations and innovate and expand, and um because async API has been working on this fantastic tooling, some of which we've had demoed and highlighted in this fantastic conference and will continue to have.

K

um Therefore, the the late tolerant, Network Community can also greatly benefit from advances in this tooling to simplify and uh shorten the amount of work involved in the processes. Most of the delay tolerant, Network Community, as far as I've been able to tell s to be much more Hardware focused and software focused and infrastructure focused and software focused, and that means that if they had software focused tooling that could accelerate and improve the quality and the interoperability of the hardware they are building.

K

This is surely a massive win and, in addition to the interoperability aspect, that I've mentioned I think another thing that async API can offer. The uh delay tolerant. Network community is the possibility of design first approaches and a specification first approaches.

K

If people already know what the entire data structure and what the entire communication protocol and what the requirements of communication interaction will be for a particular technology, then an entire ecosystem can already start building, knowing that that will be in place and purely on a combination of the async API, spec and its tooling, and that will accelerate Innovation and again in ways that are potentially unpredictable.

K

So if I can read the async API spec of the next Samsung smartwatch I might already know that I would be able to use it for uh emergency services in uh in conditions of disasters, for example, and I might be able to reach out to the Samsung team and say hey. This is what we're thinking we are sort of the United States government we'd like to be able to do something like this. This is our async API spec.

K

Would you like to talk so the possibilities that they sign first makes possible not just in terms of speeding, speed to Market and uh sort of defect, reduction and all of the values that we already know about also has a potential in terms of the Innovation and collaboration process that this very Innovative and networked ecosystem is having and will have so. These brings questions with it to us.

K

Can async API as it stands describe effectively the store, carry forward routing strategies? Can the async API tooling be used for delay? Tolerant Networks is the specific the existing spec fit for purpose, or could it benefit from extending I will not fully answer these questions today, I? If anything, I think that the main contribution I would like this talk to make in terms of the audience here today, which is async API practitioners and experts, is for you to take these questions forward.

K

I will obviously give it a stab, but really I think this is something that can be can be explored a lot more, and this is something where I think if the async API initiative, where to create, for example, some good thought pieces, some good um materials that the delay tolerant, Network Community, could get hold of ideally collaboratively, ideally with someone from that Community from um whether it be sort of there are some massive networks now outside of just NASA and DARPA Etc.

K

There are uh equivalents of the async API as it were initiative, a non-profit foundations that are um having this conversation, so I think if there could be the kind of conversation and documentation that would put this on the radar of this sort of very important part of our emerging technology landscape.

K

This would be huge, so I hope that this awakens the Curiosity of some of the people here today to explore this further than I will now looking at this from a sort of a very introductory perspective and I won't dwell any in detail, you can see I'm using a the 2.0 here from from the slide, and we are ahead now so uh I think the the presentations that have been given today and that uh and that will continue to be given, will sort of Flesh this out a lot more, but at the high level, I think this provides, in my view, what you need essentially to describe the core of a delay, tolerant network of a store, carry forward protocol, because you are saying this is what this application is.

K

This is a a store, carry food and it is being served from here. This may need perhaps a bit of refinement, because, obviously, if um foreign portion of it that vehicle that has taken the message with it, uh my understanding is that here it would become the server essentially because it no longer has any connection to the original server and it is emitting the message that it is carrying for its publisher and subscribers. So how would you describe the servers is something I would love your input on?

K

Is the server purely the uh originator? Do we have our sort of you know, production, staging and along the same lines, would the carrier sort of inject into the server description, its own self I'm, not sure? But so, if that's where, where that would go right and then, of course you have uh your channels, and uh instead of that again is fairly straightforward in principle, because um essentially, every carrier of messages is both a subscriber and a publisher, and essentially a broker really and uh and components.

K

I think become particularly valuable in this Paradigm, insofar as they might be reusable cross-device cross application. There might be components that can be um reused by a whole range of completely independent uh connected devices, for example, which are intending to talk to one another as part of their protocol. So it might be that the components part of the specification gains importance within specific domains in the uh delay, tolerant, networking world.

K

So um you know, if you are using delay, tolerant networks for communication with uh boys at Sea, then it might be that even if the manufacturers of each of these devices are completely independent, they might use certain common components for their async API descriptions to maintain um interoperability and and facility, and this um design first approaches.

K

Now the channels again I think my intuition. My instinct is that uh they are um good enough as they stand, because you are able to say uh I subscribe to these operations um as a carrier. For example, you know I'm, focusing on carriers, because I think the other components in my diagram are already being used by async, API and- and we know they work, but it's this uh temporary mobile carriers that are not just publisher and subscribers, but also brokers.

K

But my understanding is that really a broker is still essentially a publisher and a subscriber, so um they would presumably have to specify uh what this what they subscribe to in order to be able to carry that original message from the broker and what they publish and in and what are the sort of?

K

What is the data formats that they publish, and this could potentially be really quite complex again, an area where components come into their own, because if you are acting as a carrier potentially for things that you are not going to consume, you don't really care about the data format yourself, you're, just passing it on. For anybody who happens to use that data format. So again, this is where components could become really valuable because you might say, uh I am a publisher of this component.

K

I am a publisher of this component I'm, a publisher of this component and each of those components might have completely different data structures and payloads and headers Etc in accordance to the multiple protocols that that carrier may or may not interact with and um so I. Yes, I think I've already mentioned here, and here you have.

K

When we look at the components you have the schemas the messages, the security schemes and the bindings I think uh my understanding of bindings is that bindings are the um the protocol, specific requirements, the requirements that you wouldn't say in a generic case in kpi that are specific to one particular system that we'll be using uh your protocol. So I think this is probably one of the most crucial elements were. Thought needs to be given, so these are.

K

Perhaps this is the place where you specify, for example, whether you are carbon aware whether you are going to be operating. You know publishing only when your uh electricity grid is clean or you know sort of accepted subscriptions only when you hear that the electricity grid is agreeing up from your uh subscriber, for example, um I I'm, not sure, but I do think that this is the area.

K

The bindings area is the bit that makes async API so usable already for the late, tolerant networks, because anything that is not covered in the async API specification can be added to the bindings and to the extensions um I think this is very important for those who might be listening to this presentation and are not aware of extensions.

K

The fact that you can take the async API spec and extend it for your purposes, and my understanding is that in the community the async API Community, you might often begin with extensions and if some of those extensions proved to have really General value, and not just for the people who extended it, they might sort of become part of the next Universal lacing API spec, and this is a real opportunity again for people in the very emerging, very new, very fast, changing delay, tolerant, Network space, because they, if there's something that is missing or does not quite fit in the current ACI async API spec, and that is not handled in The Binding section.

K

Then you can always extend it, and it might be that when one of these um store carry forward, implementers extends it in some way. You have you know: 3 000, delay, tolerant, network products, saying me too please and that becomes sort of either stays as an extension becomes mainstreamed into the async, API spec, so I I, think sort of my. My summary of this would be to say that um foreign.

K

For proof of concept, I think we have enough now to use async API to describe delay, tolerant networks and store carry forward protocols. um Do we have enough to describe all store, carry forward protocols, I'm, not sure? Do we have enough to be the best use case for all delay, tolerant, Networks, I'm, not sure this is an area where I really hope.

K

This starts a conversation, and um more generally is in this example, for example, um without thinking just of the store carry forward protocol when we're thinking about the carbon aware Computing, which is growing and growing and growing, and will continue to grow.

K

What would be the best place to do this now? Would it be in the uh in the overall present in in our sort of uh description in our extensions in our bindings, or is this something that should become part of the async API specs, some form of signaling, this uh carbon aware sensitivity to different devices? That's my last question. So thank you very much for having me and I look forward to whatever conversation ensues.

B

Thank you so much Ishmael. um This was an informative talk, especially when it comes to async API and the role that it has to play in store, carry forward and delayed Terraria networks.

B

um We have a question um from James, um it reads: wouldn't store and Kerry wouldn't store, carry forward with something communicated by the events metadata and is not part of the API, except as a parameter.

K

Maybe I'm entirely open to that possibility, but the reason I would the doubtful is because I think you would want to be able to describe that intermediate layer. You'd have to be able to describe how to communicate with that moving object, so you are not just receiving that agnostically as it were. It might not be. It might be that in some cases, yes, it's literally storing the same message for the same end user and it just happens to be carrying it. Then.

K

Yes, it might just be in the event metadata, but if that moving object is also acting as a broker and it is using multiple protocols, then it might be that you actually need to describe it in a richer way.

B

Okay, thank you so much and I hope um James. um Your question has been answered um before we close. We would like to say something before we wrap up.

K

um No just uh please reach out I'm part of the slack group on on async API. Please tag me because I don't always follow all the threads. um Also it'd be good to just. Have those discussions I'd be happy to join a community group to have that conversation further. uh But if you think this is invaluable, then you know ignore me: take it Forward yourself uh and get talking to the delay, tolerant, Network community.

K

Wouldn't it be great yes,.

B

True, and if you want have any questions, if you have any thoughts uh any contributions, please reach out, join the async API community and join us on slack Ishmael. If you have any questions for Ishmael as well, he'll be happy to answer them on slack as well. Thank you so much Ishmael for coming today and we hope to see you next year on the next conference. Fantastic.

K

Thank you for having me thank.

B

You, and with on that note, um we'll be heading on a six or five minute break while we prepare for the next um presenter. Please stay tuned.

B

B

B

P

P

P

Hi everyone, my name is Catalunya and I'm going to present this talk, titled quality of service in it.

L

L

A

A

A

The next presenter, which goes by the name Mark and today mark, would be working on through and deep in, details about quality of service in a symptom of services and secretary extension with service level agreement support, so um I'll be bringing up Mark of stage now uh Mark. If you're ready, you can have a thumbs up.

A

uh Hey Mark, you ready okay, great, bring your upstairs now.

A

L

How you doing hi fine? How are you.

A

I'm doing fine I'm super excited to have you here today.

L

Excited to be here.

A

Yeah I can see your presentation, looks promising and I can't wait to learn. You know a lot about what it is. You have to teach us today and I'm pretty sure. The whole Community also is super excited to listen to your presentation. Before we get started. Do you have anything to share with the community and or with the audience.

L

Yes, that I'm super excited to be here and to be part of the presenting here in the in the Asic API conference and I hope you, you enjoy my presentation and don't be shy to ask questions.

A

Definitely I I know the question to the eventually coming so I hope, you're ready for the questions perfect.

A

All right so without further Ado um Mark, so we're going backstage now and we'll bring a presentation up.

P

Service level agreement support in the context of and driven architectures. We might have several services, in this case, a synchronous Services where the service clients can subscribe to several topics or events, and these events are received as soon as they have been generated. However, these systems might encounter problems of quality of service. For instance, we might have latency problems and the events are received much later than it should be expected. Also, we could have availability problems, so the events are not received at all.

P

These quality of service problems might impact not only the final end user of the service client for a particular service, but might also impact the whole architect with the whole Software System, where we have dependencies between different asynchronous services.

P

Therefore, we need to tackle this problem and we need to guarantee the quality of service for asynchronous services, and here I underline here asynchronous, because in the field of service oriented systems and in particular those web services that are synchronous, there is a lot of support for quality of service in terms of service level, agreements, monitoring and so on and so forth.

P

But that is not the same degree of support when we talk about asynchronous services to tackle this problem, we provide here- or we proposed an async API extension with service level agreement support in order to address it. We first need to do two steps. The first one is: we need to identify what is the quality that is relevant for a synchronous Services? What are the characteristics, attributes and metrics that constitute the quality of service for this type of services, and when we got that?

P

The second question is how we can Define conditions over over them in order to ensure that the whole systems works as expected.

P

Moreover, what we want is to provide these features, a seamlessly integrated with the async API specification, so let's go with the first step, identify the quality, so what is quality in the field of the asynchronous services? What are these characteristics that are relevant here now in order to identify the quality? First, we need to understand how quality of service can be structured.

P

So here what we have is that the quality of service is a structure in a hierarchical way in different categories, so the quality of service we may say that is a totally totality of the quality characteristics of a service and is quite the characteristics, for instance, could be a performance, are groups of quality attributes so, for instance, for performance we would have things like time, Behavior or resource utilization. The quality attributes are features of the service that affects the quality and they can be measured in terms of quality metrics.

P

So, for instance, if the quality attribute is, for example, this time Behavior, a quite imagery could be, for example, the latency of the web service. So quite a metrics are the quantitative or qualitative measurements for a quality attribute.

P

Following this hierarchical structure, there have been several proposals, and these proposals are basically quality models. That instrument and characterizes the different quality characteristics attributed metrics. The standard for the software are systems of product quality is the Ico 250010. So here we have. It is a standard that the software product quality Can it have. It has several characteristics such as functional suitability, performance efficiency, compatibility and so on, and for each of these characteristics there are several quality attributes.

P

The problem with this model, if we want to apply it in the domain of asynchronous services, is twofold: on the one hand, it's for software systems, so it's not specific for synchronous services, so it might be. There might be several attributes that might not be here or the other way around. There might be here several attributes or characteristics that are relevant for a software system, but not for synchronous services and, moreover, this quality model doesn't go deep down to the level of the quality metrics.

P

So we have quality characteristics and attributes, but it doesn't go down to the level of metrics to address this issue and as part of our previous work, we conducted a systematic mapping study where we revised the scientifically territory of quality models that are specific for services and what we did is we analyzed these quality models, 47 of them and we integrated them and mapped them into these Ico Stander. So basically, what we did is for each of the quality models that we identified in the literature we integrated, all of them and uh basically a set.

P

We map them to this to this standard. Here it is an example of the mapping that we conducted so here we have the list of an excerpt of the quality models that that we studied and here for each quality model. We identified two. We still really support different characteristics and attributes of the of the Ico standard, and for each of these attributes we identify different metrics.

P

So, for instance, for the attribute of functional correctness, there might be metrics for correctness, Precision transaction integrity and so on and so forth, and this was done in the field of quality models for services, but not specific for asynchronous services. So if we draw a Venn diagram- and yes, this is actually a Venn diagram. What we have see, what we can see is that we have the quality model for software. This is the standard Ico 250010.

P

This is the result of our systematic mapping study where we have a quality model that is specific for services, and what we want to achieve is a quality model for asynchronous services.

P

So what we have done in this work, we have basically have a look at the Quality model for services that are Services of any kind, both synchronous and asynchronous and identify those characteristics, attributes and metrics that are relevant for the asynchronous services hearing is an excerpt of the different attributes, metrics that we have identified that are relevant in the domain of asynchronous services, so, for instance, for time Behavior.

P

Obviously there is no such thing as response time, because there is no response to provide in this kind of even driven architecture, but there are several metrics in the time behavior that are relevant, like the latency the round trip time, if there is efficas in the messages, the Jitter, the throughput and so on, and here we have a list of different metrics for the different quantity attributes and the quality characteristics.

P

Now that we have the list of these Matrix attributes and characteristics that are relevant for the um asynchronous Services, we need to define the conditions over them and to do so, we basically propose to use the same principles that are being used for synchronous services, so in synchronous Services. There is this service level agreements where the service level agreement.

P

It is basically a contract between the service provider and a service consumer for the provisioning of a web service, and that specifies what are the conditions and in also the quality of service, that the service provider promises to their clients.

P

So, in the there are several specifications, several languages to write service level agreements, but the de facto standard in terms of uh synchronous, Services, is the Ws agreement and our goal is to use this same. Double use agreement in the field of asynchronous services, adapting the particularities that the event-driven architecture poses, so that ws agreement is basically a document that has this structure, so it has a name. It has a context. The context provides, for instance, who is the service provider?

P

What are the services being provided, and it might also talk about, for instance, the duration of this W's agreement, contract and so on, and then we have the terms. The service terms provides the definitions of what are the particular service in the in the field of uh synchronous services. This service terms will be the services, the methods of those Services, the endpoints and so on, and the guarantee terms is where we specify the conditions in the field of asynchronous services.

P

What we have is basically the services that provide events into particular topics and channels, so the the concepts that we are familiar with in the async API specification are the ones that relate to the service terms and the guarantee terms are the ones that, where we put the conditions that we must guarantee now the guarantee terms, it has several um several things to to be included in that. So the first one is the services scope. So when you put the condition, this condition applies to a particular scope in the field of uh synchronous Services.

P

The scope can be, for instance, that this condition must apply to these particular methods in the field of asynchronous services. The services scope might, for instance, apply to particular topics to particular channels, or you might also say, state that the service scope is the whole asynchronous. What the holy synchronous Services is providing to the whole set of set of Channel set of topics.

P

The second aspect is the qualifying condition, so this is some sort of a precondition that must be uh true in order to enforce the service level objective, so it's some sort of a precondition. It's stating that my service level objective, when I'm going to put my condition, should be true. If the query thing is, the qualifying condition is also true.

P

So, for instance, we might have specified service level objective that should only be applied during, for instance, weekdays specific hours or even we might also include here some conditions for some particular metrics, so, for instance, if we want to enforce that, the latency should be below a certain number. uh If, let's say there are less than a certain amount of people subscribed to that topic.

P

So this, if could go to this qualifying condition in the service level objective, is where we put the condition that we want to assess, and finally, we might have optionally uh what is called the business value list, and here we basically can put uh rewards and penalties if the commander conditions with when this service level objective are not are not fulfilled.

P

Now the question is: how can we integrate this WS agreement in async API? That is the basics, the the idea of of our proposal. Now the thing is in the field of how W's agreement has been defined. It is as a standalone document. So that means that, for instance, if you have a swap web service which is specified in whistle or if you have a restful service specified in open API, the Ws agreement is uh used as a standalone document.

P

In our case, we believe that it was much more convenient to integrate WS agreement inside the async API specification, so instead of defining a standalone document for asynchronous services, what we are doing here is integrating these WS agreement within the async API specification. Now, how can we do that? The thing is as part of our previous work: here you have the QR code. If you're interested in in the paper, we defined a nursing API so, but we basically extracted the async API meta model.

P

So here we have the uml model uh of the async API specification, with all the different elements of the async API notation, and from that previous work, what we have done is we have extended this meta model with the concepts of the Ws agreement.

P

So here we conclude, as you can see the SLA with the guarantee terms, the scope, the scope that uh pinpoints to channels the guarantee terms might be qualifying conditions or service level objectives and they are basically pulling Expressions over quality of service metrics and this quality of charismatics are the ones that we have seen uh are the ones that are or Maps or to speak to the elements that we have seen in the in the quality model.

P

Here we have an example of an async API, with this extension that conforms to the meta model that we have shown previously. So, for instance, here what we do is in the components part of the async API specification. We add as a component, the Qs Matrix. So basically, here we are defining what are the metrics that we are interested to put conditions over in this async API specification and, for instance, in this course metric? We can talk, for instance, for the average latency. We can put a description and then the metric type is latency.

P

These metric types is the list of the elements of the quality model for a synchronous Services. We can include what is the unit that we want to put the conditions or measure that would be, for instance, in millisecond, and also one relevant thing here is that we can group this matrix by event. That's something that doesn't happen for synchronous services, and the thing is that in asynchronous Services there is one event generated by the service provider and this event is received by multiple service clients.

P

So when we talk about latency, you might have several latencies one for each service client receiving the message or you can compute the average latency grouping all those messages by events. So if there are 10 subscribe clients to a particular topic for a particular event, you do the average of all these of the latencies of all these of all these clients, and these defines basically the different metrics that that we are interested in another important aspect that is uh different with respect with the Ws agreement. Since here we are integrating WS agreement with async API.

P

There is no need here to read repeat again: what are the channels? What are the servers and everything that are specified in the async API, because it's already here in the document, so here in the term, in terms of the terms the service terms that we need to Define, we only need to define the quality of service metrics, and then we go to the service level agreements to the guarantee terms.

P

Now, in the guarantee terms, we can Define, for instance, here the scope where our service level objective applied to, and here we can Define our service level objective that we are referring here. The average latency that we have defined here previously should be below, for instance, 300, and the 300 is, as you can see, in milliseconds.

P

So this extension, uh we have developed uh two support for that and we basically had a tool that we started to develop uh years ago, which is the async API toolkit. Now the async API toolkit is an eclipse based tool and it provides an editor with content SDS. It also checks the grammar and everything it has code generation capabilities, so we have basically a rest only in this world. What we have done is we have extended this async API toolkit with the new metal model that includes the Ws agreement uh integrated with async API.

P

So here you can easily write and specify your async API documents with all these functionalities that are provided through these instruments of eclipse. If you are interested in the tool, you can have a look here at the QR code and in terms of the code generation capabilities. One of the things that we are working right now is on the generation of monitors, so in the current version, what we have is that the code generation capability of the async API toolkit is capable of generating boilerplate code for mqtt.

P

So basically from the async API specification, this async API toolkit is capable of generating uh both the server side code and the client-side code, very simple, very boilerplate code that can be extended by the uh obviously by the developers of the of the service, and what we are doing is we are extending this code generation capabilities to also generate the monitoring instruments, in the sense that when you put the conditions that you want to be enforced in your asynchronous service, we are generating this also monitoring instruments that are instrumenting this boilerplate code that we are generating in order to monitor the different metrics that are here index, async API specification.

P

So this part is still a work in progress, but the part about the um the uh specification of async API with service level agreement support. It is already available.

P

So as a conclusion we have presented here, what is the quality model for a synchronous Services, where we basically identify those metric attributes and characteristics that are relevant for this type of services? We have presented an async API extension with service level agreement, support based on the Ws agreement standard, and we have also briefly describe the tool support that we have the async API toolkit that provides these two support to Define this async API documents with this service level agreement support.

P

So that's all for that. Thank you very much and now it's time for questions.

H

I

H

You can hear me well um welcome, after the break I'm here to replace Ace because he run out of Internet uh with him, we'll send him a few buckets with internet later um so yeah. Let me let in Mark Mark you ready to join the live stream.

H

Okay, perfect! So.

K

L

Sorry I cannot hear you well now, I, don't know if it is me, but I could hear you well before I joined now. I cannot tell you how.

H

About now, can you hear me well now.

L

Better, yes, okay, perfect, so.

I

Yeah, um basically, thanks.

H

A lot for uh for taking time to present and um the most important taking time to show up at this late hour on Saturday, that's um I'm super grateful.

K

H

I bet other members of the community appreciate it and.

L

H

There are some I think there are some questions for.

M

H

um I think there's a question from Ravi. So can we apply us SRE methods to async API.

L

Okay, for us re methods, I guess it means the site, reliability, engineering method,.

P

And yes, obviously, I mean, of course you can apply other methods to async API and actually our contribution and our work could contribute in this field. So, for instance, the fact that you can Define uh specifically what are the conditions that must be met in order to guarantee that the quality of service is adequate for your asynchronous services and then, for instance, these things that we are working on that is automatically generating monitors from that specification, so for sure it can be applied and our contributions aim actually at supporting or providing some support to this.

P

Today's practices.

H

Okay and and I see that.

F

Ravi also mentioned.

H

Open Telemetry: do you have something to add in case of open telemetry.

L

uh I haven't checked too much about open Telemetry, but uh yes, basically I.

P

Guess it could be a use, I mean in the sense that uh the AC.

L

Kpi specification.

P

L

uh Towards with respect to the technology, and in our extension, when we Define.

P

These service level agreements for async API, it is also technologically agnostic, so the independent from the technology, the component that we are generating that is based on our previous tool, to generate these monitors, and let's say they are in-house monitors that we built here in the universities.

L

But, of course, I guess that we could, for instance,.

P

Generate the components or the elements required in order to configure the open, Telemetry or any kind of monitoring systems that.

H

Are out there? Okay, perfect um Ravi, I hope that answers your your question. I, don't see any other questions um so Mark. Do you want to add something at the end like a wrap up or, like summary, something super important people should remember from your talk.

L

So if you are interested, you have in the presentation the QR codes of the different elements, our papers, as well as our tools, and it's just glad to be here and be able to share with all of you, our our extension, our proposal to stand the sink API and more than welcome. If you have feedback or comment or if you want to pin me in the slack Channel I am more than available to.

P

To answer or get in touch with you.

H

Awesome thank you and, from my side, huge thanks that you're building also Tools around.

I

H

Because, like yeah spec itself without loss, is nothing really so yeah thanks a lot thanks.

L

H

So I'm um yeah see.

L

F

H

Folks, um so now it's time for a for a break. Let me like so I'm I'm, really not in sync with schedule. So let me have a look it's 555 in Poland, so the next talk is hmm oh in no. No, actually, we have break in five minutes, so we ah we're gonna have a longer break. Oh we're gonna have a super long break awesome uh because I just got fried um from my wife, which is amazing so like we, so we have.

H

The next topic will be uh delivered like the next presentation will be delivered by James and we have like 34 minutes break. So um the talk is at 17, 30 UTC, uh so James uh from VMware will talk about async, API and the world wide. Not web flow. So yeah see you around in a half an hour.

H

H

H

H

H

M

M

M

M

M

M

M

M

M

M

F

F

F

F

F

F

M

M

Thank you, foreign.

M

M

M

M

M

M

M

M

F

F

F

F

F

F

F

F

F

F

F

F

B

B

B

B

B

B

B

B

B

B

B

B

B

B

B

B

And we are back hello, everyone welcome back and I hope you guys are still having a good day and you're enjoying the talks that we are having so on our next. Our next speaker is James. You will be giving us a talk about async API and the world's wide flow um James is with me backstage James. Can you wave your hands so I can bring you on stage perfect.

Q

B

Hello James: how are you doing I'm.

Q

B

Good thanks, thank you so much for coming today.

Q

Yeah happy to be here and and excited to talk uh to everybody uh about. uh Basically, a topic that came off of uh the the book. I wrote for a rally called flow architecture is the future of streaming and event. Driven integration.

Q

um Give you guys all an idea of sort of this concept that I think they think API can play a huge role in of uh the worldwide flow, um and that which will be explained in some depth in the course of the presentation. So I won't give too much away, but uh yeah excited to share it here and excited to be a part of this event.

B

And thank you so much James and I hope. The audience is excited to learn more from you um without further Ado. um We'll get um right straight to the talk. Thank.

F

Q

Hi I'm James erkert, uh welcome to my talk today about AC kpi and the worldwide flow, and uh so we hope everyone is doing well wherever you are in the world and whatever time it is really. The topic of this conversation is the idea of driving business, speeding up the flow of business through event, driven integration and basically integrating the communication of changes of things of activity and changes of state between organizations and, ultimately, as organizations connect to each other kind of creating a giant uh set of those connections which I call the worldwide flow.

Q

um I'll start by defining the term flow a little bit because it's a heavily overloaded term, and we all know that you know you can find that word uh used in Tech in a million different places.

Q

In this case, I want to be very specific, with the definition I'm talking about flow in terms of an event driven integration across organization boundaries, meaning either between uh and from one business division to another business division within a company, but even more excitedly To Me is from a company to another company or a company to a government institution or any of those sort of cross, uh different organizational entity boundaries that might happen and then doing this through standard interfaces and protocols which we'll talk a lot more about here in a bit but but using standards so that everybody it's as simple as possible to make these Integrations happen.

Q

Now the term WWF has a little bit of a history, at least in the United States right, there's a big battle between um the World Wrestling Federation or the wrestling entertainment organization and the World Wildlife Fund, which was this wonderful, peaceful protector of of animals across the planet.

Q

um So, unfortunately the word, the letters WWF have a little bit of an overloaded connotation, but in reality what I'm talking about is is a concept that I go through in great depth.

Q

In my book flow architectures, the future of streaming and event-driven integration um and really is encapsulated by this sort of tagline right as HTTP linked the world's information and created the world's web flow standards, will link the world's activity and create the worldwide flow and I really think it has the chance of having the same or bigger impact that HTTP had on the way that we link what's happening in the world, who we are the ways that we interact with each other in the world, um not to oversell it.

Q

But but really, if you lower the cost of this integration, greatly you're going to create this explosion of new Integrations that happen and who knows where that's going to take us.

Q

um And another way to picture this is really just in terms of Eda, so we, you know, here's friend presented this earlier in the conference as sort of a very simplistic picture of an Eda and I. Want you to kind of take a look at this, and you know think about this: okay, instead of being within an application system in a very simple set of software services.

Q

What if it's companies so imagine that the European weather center for weather um predicts sets of predictions to DHL, a transportation company and uh and that transportation company in turn, send signals to suppliers to manufacturers to shippers to say Hey. You know these are one trucks are going to be available. These are the capacity elements are going to be available. This is what weather effects will have in terms of travel times and then also at the same time, can send signals to the consumers about hey.

Q

Your package may be delayed a little bit because of this weather event or um or hey we had. You know we had a really easy travel conditions, so we um predict that this will be at your front door. You know in this 10 minute period window period.

Q

um You know that there's a lot of other activities like this, that you could imagine uh things in finance where people are exchanging information about transactions that have occurred uh or movements in stock price and other things like that.

Q

um Healthcare there's the obvious condition: sort of tracking patient conditions and and being able to set off, alarms or react quickly or automate corrections to things when, when Health deteriorates but I think just as interesting are the signals are going to be sent between Healthcare organizations about available beds about supplies needed about.

Q

uh uh You know the the the current state of uh and changing state of their um their emergency room capacity and their ability to handle new ambulance loads right. So most of the time that may be very mundane, there's not much to do it's. It is what it is. But if there's a crisis, then that data may be very, very good at signaling that this a crisis underway and, at the same time also enabling or these organizations to coordinate with each other better and faster.

Q

um I started this exploration by sort of asking the question: okay, if you're going to do flow, if you're going to do event driven integration across organization boundaries, what are the pieces and parts that you need and really a large chunk of what needs to be? There is the stuff that's sort of more to the right right. The the sources and sinks um you know, six database storage is is readily available from any cloud provider in several forms as a managed service.

Q

It's just a utility pretty much at this point, even the ability to process um uh data and or or the cues that we use to sort of buffer event data and uh and broker event data. Those things are sort of. Very you know, open source projects. Apache alone is, you know, probably got 15 or 20 projects that kind of fit into the Q and processor space in some way.

Q

um There's a there's a lot of really really powerful things out there that kind of rarely available, so the pieces are set up on the board for uh flow to to be out there and be successful. What's missing, though, is the stuff that's to the left? What's missing is the stuff, that's related to the interface standard interfaces and protocols that are needed to make this integration as simple as possible, and so this is what I explored in the book as sort of what would it take?

Q

What are the elements that have to be there and what?

Q

What is the landscape today and how is how will that potentially encourage um the movement of of these things and so a little get a little bit more kind of definition, of what I mean by interface and protocols, so protocols we're all very kind of aware of sort of the wire level protocols, TCP and then, on top of that things like mqtt and uh and amcq, and you know, and all these different other things that are really critical, that we use all the time um but uh but packaging the event data itself is extremely important.

Q

On top of it. The thing that's carried by those protocols on the wire is very important, so metadata is just setting the context of the event, not only where did the event come from. Is it connected to any other events, but also things like um you know what? What does it take to be able to process event what you know? What's the format of the payload?

Q

What's the uh what's the size of the payload and uh What uh encryption is being used right, there's all kinds of things that could be represented in metadata and then the payload itself, which is you know whatever the data, is that we're communicating this the state change of an individual value, uh the new state of a complex set of data structure, whatever it may be, they're communicating and then, on the interface side, there's really two interfaces that I was able to kind of narrow things down to so one is discovery which seems like it's very kind of extra at first, but it turns out to be really really important in order to understand not only that there exists a stream that has a certain type of information that you want to consume, but also what does it take to be able to to consume that stream?

Q

You know how do you filter for information or how much volume can you expect to receive what the compute capacity would you calculate off of that?

Q

Excuse me um and then there's the subscription API, which is sort of what everybody you know thinks about right away. The subscription interface right so um which is really important to allow a publisher to say, hey, I'm, making this data available um and allows the consumer to say: hey I want to connect to that topic.

Q

Maybe, with this filter and I, want to be able to consume that data and and by the way, to authenticate that they are who they are and that they're allowed access and so on, and some examples of that then and sort of really narrowing down on kind of how close we're getting here. So we've got standard wire protocols all over the place. Mqtt mqp I talked about web socks and sockets is really big um and increasingly I see that out there, especially for point-to-point connections when you're not going through a broker.

Q

um There's also um the metadata and I really think and I'll make the argument through the rest of this talk that cloud events really has the inside track here, um it's just by far um not only the best expression of uh the metadata required that um I think by far, but also the most used and utilized by vendors and Cloud providers and so on. So it's really got kind of a very it's beginning to be adopted very quickly. Then I think um for the payload.

Q

My belief is, is that ultimately, the standards in payload are going to not be sort of General standards, but are going to be specific to Industry. So you know it may be. You know, embedding fix financial transaction information into a payload. It may be, uh you know Health Care related. It could be um custom protocols that get built by say a small group of vendors that are working together and a special way to accomplish something, and it may be options that we can't even imagine today right.

Q

There may be government defined uh payload standards for certain types of transaction or certain types of information exchange. We don't know, but but the payload has to be flexible enough for all these different kinds of things to be represented as they need to be represented, so they can be consumed and then I the interface side, I really think async kpi, all the evidence I have points they Sync API having the inside track um and and I say that and I'll explain more in detail.

Q

Why, in a second, but really the kind of the big things here is instant kpi kind of combined with um a number of things is uh with something that allows you to get to the async API documents to discover they Sync API documents.

Q

um It's really like a self-contained set of information. That's consistently communicated right, so you can get a description of the API fairly easily. Now, how do you find those async API files? Well, that might be somebody sets up a registry service or or something like that I think personally, the most likely thing is that people will kind of make them available on what just developer website somewhere, right and uh and people will Google for them. Google will be the the primary search mechanism for the discovery.

Q

Async API will be the representation of the API that is either machine or human consumable out of the result of that. So again, speculative but I think there's a solid evidence that, uh at the very least, async API docs have a very likely chance of being a part of that and then subscription async API again I see that way. More than anything else, that's out there. There is other options simpler, more specifically defined for Pub sub, for instance, so the pub sub API from Salesforce is the example I give here.

Q

There's some you know the eventbridge API, there's the Google Pub sub API, there's other things Kafka API, but that could potentially kind of win out um for the subscription API, but I really think it needs something like async API, it's more General, more um and I'll. You know: descript adaptable, extensible I'll talk about this stuff, a little bit more here in a second foreign. Now one of the things you can do with these worldly maps that I've been showing you here is.

Q

You can begin to think about hey if everything's sort of being custom built or it's product based or whatever. Now, what happens if, as the energy begins them to mature, that technology, to the point where those things become basic Commodities? And so it's especially interesting that all that stuff that I had circled on the right before is all needs to evolve. This is the opportunity.

Q

This is why there's a door open here for a standard to appear for the interfaces um so async API plays, plays the role of interface really well, it's a way of describing topics and connections. It's it's exactly what you need right, here's some information about it. Here's the you know the servers you can go connect to here. The channels that are available on those servers here are the components that are described.

Q

The elements that described as part of these events- it's it's solid, I mean this is the 2.0 spec I know, there's some changes coming with the 3.0 spec, um but the basic pattern here um and what we're learning as we Implement these patterns uh leads to a really good way of sort of a basic description of what the API is for um for the interface to a flow service of some sort.

Q

I also like the fact that so far, it's it's fairly consistent right. We've got some major releases happening on a fairly good clip now, hopefully with 3.0, you know third time's a charm. We begin to slow down uh kind of the major changes to the way things work, but um it is something that has a solid Community around.

Q

It has vendors that depend on it that consistency is almost going to become guaranteed by um by just the the um the the momentum that's created by its usage and the fact that that making breaking changes will be harder and harder as we go forward.

Q

um It's also very adaptable, which is incredibly important, because who knows what event driven integration is going to look like 20 years from now. um It needs to be able to be extendable. It needs to be able to.

Q

um There needs to be a process by which the standards can change if they need to change um and, and that will help it live longer and and be important and and survive as a standard for a long time. It's also Discovery friendly, which I talked a lot about in a little bit earlier, but that's just a super really cool thing to add into it.

Q

It's also- and this is maybe the most important reason why I sent kpis inside track- is it's getting adopted. It has a community, that's consuming it. It has the usage of these different vendors, these different uh uh implementers of a Sync API that are happening and as that grows as the number of organizations that are using async API grow and as they begin to offer those uh connections to other organizations and use it as a flow standard um that that's how something becomes a standard in the modern area.

Q

You don't just declare it as a standard. You have to go, get people to adopt it and validate it. Running code wins right, um and so you can say some similar things about Cloud events as well by the way, and it's important to kind of note them right. So, first of all, it's a way of describing perfectly you know near perfectly well the metadata of an event. It's it's really well thought out.

Q

There's been a huge amount of time put into it, a huge amount of kind of experimenting with different ways of doing things, and ultimately it boils down to a very few um required elements. These three elements on the left are the only required elements in this list and then a whole bunch of stuff that can be added or extended uh in the definition of the of the metadata. For that uh that object, which all this makes it consistent.

Q

There's a community and a standard document and a way of updating the standard that allows the community to kind of move along together as a group and it's extensible and adaptable to what's happening. But the big thing with Cloud events is really nicely. It also adapts to different usages of events, so you may have some systems that have a kafka-like story where they need a connection of events that are related to each other, and that dependency is understood so that you can track through that.

Q

As you step through the to the topics um and and sort of you know rewind and play through it again, but others may just be sending a state change.

Q

That's captured by a digital twin model that just essentially forgets when capture that say change records it and says: okay, next next event, next event, next event, and that connection between events is much less important to the way that processing happens um and it works really well with Lambda events, and it works really well I mean you know, there's this huge list of stuff of types of processing.

Q

um In fact, in the book, I have a really good diagram. That kind of explains what are the different types of processors you use for different ways: you want to consume events and and five events adapts to all of it, and that title is wrong. Let's just say: glottopense.

Q

Also, by the way why Cloud events it's being adopted right, it's it's their implementation out there. This list is just a very small partial piece of what they have on their website, indicating people who've actually built Integrations that use cloud events as the metadata packaging for the events that they either transmit or consume.

Q

um There's great list of vendors that are involved in the on the working group that is maintaining the standard, there's a great list of vendors who have come around and integrated some form of uh of adapter or or core functionality of their system.

Q

That is cloud event aware and Cloud event friendly, and you have things like k-native that make building platform elements that can consume those events, much much easier just to eliminate it's a huge amount of the toil required to be able to capture and interpret those the metadata in order to present the payloads correctly foreign.

Q

Look at what happens when you bring those two together, and this is what kind of excites me again. We want to go back to this idea that we want to make it super easy for everybody to connect into this web of streams that's available out there, and so um so, bringing Cloud events and uh async API together kind of gives.

Q

You I think that that proper collection of the right extensibility, the right uh separation of concerns to be able to enable um uh to be able to enable flow to happen, and there are a few areas where there's still some. You know some overlap, uh certainly um uh protocol bindings like MP3 and amqp protocol bindings, both both sets of Standards generate their bindings.

Q

um There has to be some reconciliation around that, but for the most part, you know really well set up to happen, and you can hear our discussion. This is uh paren um from the async apsi Clemens vasters, representing from Microsoft, uh representing the cloud events group and then myself, uh sort of representing the flow architecture, author and kind of talking about these.

Q

um These ways that these things can work together to create that flow future. What's the opportunity, then, here and I think this is the most exciting part of this whole thing is: there's a ton of opportunities, so, first of all, async API becoming part of cncf. What a what a just a wonderful wonderful collection of Technologies to bring together um and I think it really simplifies the ability for these working groups to get together and have these conversations about.

Q

How can we facilitate a solution to these forms of integration as they as they become more aware, more uh obvious, um and so you know and I think you know.

Q

I've talked to the guys at Nats uh that work on that um a few times about this they're, very, very open, as as the flow standard appears, they want to make sure that that supports it very well um and I know that the you know the API Community could um could really use not only what async API is doing today for all forms of event exchanges, but as we get into um to doing these flow exchanges, I think it's gonna be incredibly important to get uh to to get the API Community what they want in a way that they can consume it easily.

Q

Today so again, really well set up um some usability things that are missing right. We don't have that way of just sort of saying: hey here's a topic I want to connect to it right, there's no libraries um that really bring async, API and Cloud events together, yet into a just super, easily consumable piece.

Q

We do you know some code, Generation stuff, that's great and I and I think that that work is really important, but I think there's no sort of like hey, that's just built into a programming language or it's just built into uh an operating system or or a framework or whatever.

Q

um So we'll see, I think some of that stuff started to happen. As you know, as sort of the flywheel of demand and and need sort of starts to spin out um security, authentication I think is still not um doesn't have quite the spotlight.

Q

It needs I think it's probably the most important problem that really needs to be solved well and I know there's stuff about that in async, API I think you know just real Clarity in terms of especially if there's other standards that are being consumed and sort of highlighting that secured the secure way of connecting in a standard way is really important to building trust in a commercial setting and in a government setting and then adoption we just need to you know we just need to make it as easy as possible for people to adopt flow, and initially that's going to be a little bit rough because we got we don't know what all the pieces are.

Q

Yet we haven't got it all working together, but I think in five years. The goal would be really good to say: look we want to make it super simple.

Q

If somebody, you know wants to write an application that they want to connect to a flow compatible um stream, that it's super simple to, do it and here's all the code ready to go to do it um or here's the library that does it for you, whatever that may be so adoptability onboarding is really important and then some future opportunity here that I think is super super exciting right, so I, you know maybe longer term than some of the others here.

Q

Monetization is one that I look at and, like you know, it's not just how can I charge to let somebody connect to my stream, but are there ways to charge for people to share that data with others or um or to charge people to take? uh You know uh to license a piece of of the data, that's in there for use and other purposes entirely um or to pass it on to you know whatever maybe process it in certain ways. um There's some exciting possibilities. There I think observability um is a really hard one.

Q

Not only how can I see my you know my the things I control, but how can I see where my data is Flowing outside of my control, and uh you know lots of questions about you know what's privacy, what isn't? um What's you know? What's what's trade secret?

Q

What isn't that have to be answered there, but ultimately I think there's a great opportunity here for um for being able to visualize how your activity is triggering activity elsewhere or or what you're waiting for whether you're getting accurate information from a source and uh and you're you're, getting all the information you should be from a source.

Q

uh Where did all this data come from right and then that brings me to Providence and I really think the next billion dollar company that comes out of Eda is going to be somebody who really kind of solves core provenance not only who's the origin and can I validate the origin of the data, but is the data as I receive it still as it was intended to be sent? Has it been altered in some way?

Q

Has it been added to in some way that's unexpected um and also as things this data moves through kind of multiple? You know it gets maybe re-posted in a different topic for some by a different party, whatever it may be, that's kind of the core origin of of each of the the data elements um is in some way validatable as as having come from the source as a part of that event context, so um Providence I mean everybody.

Q

I talk to in the space is that's one of the biggest problems is how do I solve the problem of making sure my data provenance is, is understood and that I am not working off of some mocked data or or you know, or malicious data of some sort, and with that I think uh I'll wrap up and just say. Thank you very much again. Opportunity is huge here. uh Async IPI very much has an inside track. It's a really well designed uh standard for the purpose.

Q

I hope the 3.0 stuff takes it even further I'm going to be very excited to kind of watch. What happens with that and then? um But ultimately, you know where we want to get is again that ability, if we can make it super cheap for people to consume events from other organizations, there's a whole bunch of things that I think you know I certainly couldn't imagine that can come out of that some bad, maybe but mostly good, and mostly really fundamentally changing the speed and which business happens on the internet. So thank you very much.

A

um Yeah so James, are you ready to come up.

Q

Hi, how are you.

A

Yeah I'm doing great um what an amazing presentation James um so I really enjoyed your session. To be honest, it was it was amazing, I learned so much about how you know the flow whatever what it's all about and yeah I think you actually answered a question on on slack right, but I would love to like make you answer the question also on this live stream. So I'm gonna really question today.

D

A

The audience now yeah, so um one of you asked a question saying: are you considering the instant API to replace EDI, slash as2.

Q

Yeah so I think there's a number of existing data Exchange pieces that are largely used to push State change data across the wire and EDI is probably the biggest one that everybody knows, but there are standards in the insurance industry and the healthcare industry.

Q

I put some of them in the in that slide, where I talked about fix, for instance, in the financial industry um and uh and absolutely um uh my argument would be if it is way easier, cheaper and reliable to connect to a stream of that real-time data by the standard that everybody knows as well and uses well known to be secure, etc, etc, um that it opens up ecosystem and it reduces you know, risk of obsolescence and a number of other things by moving to the stand that standard, as opposed to sticking with something that's very custom for a given industry.

Q

Now it will take a long time when something really works fairly well for an industry like fix, it may take a really long time, but if the the ecosystem of stuff that could consume your event, explodes especially for things like routing security, uh monetization, all that kind of stuff, that's not specific to that industry, um then um then I believe that there will be an impetus to make that move.

Q

So you can take advantage of that ecosystemic capability around the standards um as move forward, and it could be you know, I I, use the weather stream as an example.

Q

There's a lot of data that and you think, it's very specific to a certain kind of class of problems, and it could and turn out to be really useful in a place that nobody ever guessed before, and so, if it's some special cut, you know specific to an industry standard that that data is available for and everybody's got to go, find it and figure out how to program.

Q

To that, then, um then, that's really expensive, whereas if somebody can just say, hey I'm, going to point to that stream and start receiving stuff and see what I can do with that data. um The experimentation allows for a lot of innovation to happen much faster.

A

Interesting interesting, James, I, love that and and to add right. So you actually uh I think your your session with um Fran on the taking out loud episode was one of my favorites right. So you were in the session with Clements and and I see you guys talk so much in depth about ocean kpi and Cloud events and all sort of stuff and I really love this session. So uh shout out to that um Stream So.

A

If you haven't watched um the the live stream uh about taking out loud, where Fran hosted James and Clement so I'm gonna share that in the link after the short break right thanks for joining us, James I'm. Looking forward to you, know having you in the next conference.

Q

Hey I really appreciate it. Thank you and and love the show lots of great presentations thanks.

A

All right have fun, James see you. Thank you.

A

A

A

A

A

D

D

D

D

D

D

D

A

A

A

Yeah welcome back so thanks for hanging in there folks. um So finally, uh we are at the last. We are currently at the last session of the AC kpli conference, 2022 and yeah so excited it's been a long ride and yeah I'm so happy to be presenting the last um presenters for today and which there will also be the last presenter for the executed by a conference 2022 and with me. Yeah I have amazing folks, um Natalie and Rob.

A

um So are you guys ready to come off stage? Can I get a thumbs up?

A

Oh awesome, so bringing you guys up stage.

A

Hi guys hello, hi Natalie, how you doing.

E

I'm doing good I'm doing good I I'm wondering if things are pretty slow, oh looks like Rob might be having some audio issues, yeah I'm checking in which we'll be back, I'm, sure, okay, yeah, definitely so I think Rob is back yeah. Now my mic works.

E

Awesome, hey Rob, how you doing I'm great? How are you yeah, I'm doing great too I'm so excited to have you guys here with me today and very very, very, very interesting. You know you guys are the last presenter for the async UPI conference 2022 and you know I can't wait to actually get right into it. So how you guys feeling now we're great great yeah.

E

It's been a awesome conference. Awesome recording everything was uh relaxing, so great I'll be looking forward to doing way way more better next year. uh I know with amazing folks, like you guys, will definitely be more better next day.

E

So before before, I play your life before I play your video, your presentation, would you like to share anything, uh not really I, think we we walk through it and set up the context, uh and we don't want to give away the the surprise. I will mention that, there's like a fully functioning demo, you can use to you know short of it as you generate uh full async API specifications from data instead of uh ahead of time. It comes at the end uh and there are Links at the end.

E

You can use to try it out yourself and there's a hosted Service, uh just like an endpoint. We have where you can be like. Does this? How well does this really work and you throw your own data into it um in their their Links at the end, so we encourage folks to try it out.

E

Oh awesome, I'm. Definitely gonna trade, our house, man for real I'm, doing it um so for folks, I'm, actually watching this live stream from everywhere around the world, so you should feel free to do whatever it is. Rob and Natalie has for us after this session. uh Okay playing the video now.

E

My name is Rob gallonakis. My name is Natalie Edson we're the founder and Senior software engineer at webhook TP. Today we have something fun to present. At least we hope it's fun, definitely a bit different than many of the other talks, uh we're going to build a complete and comprehensive async API specification without writing a single line of it ourselves.

E

So, let's set the stage. There are two things you should know: first about our company. It's just the two of us right now, uh we'll talk about our product web hook to be in a minute, but we also run a software consultancy, lithic technology, we're juggling clients, customers and support, and life there's always quite a lot going on.

E

The second thing that you should know is that we're pretty lazy and lazy, of course, meaning that we want to spend time doing things that we enjoy doing and then avoid spending time doing things that we don't enjoy doing I'm a published poet you can see here in these pictures uh me reading in a poetry. Reading uh Rob is the primary parent to his two children. You can see Rob here being very silly.

E

Thankfully, we also enjoy programming, but when we do program, we like to really let our creativity flow. We're not specification people on top of that I'll come out and say: I'm, not a huge fan of code generators, I'm, also skeptical of any technical documentation. Artifacts I, run across inversion control and my eyes run right over this sort of specification language that reads like someone's lecturing a misbehaving child.

E

There's no question, though, that specifications are absolutely vital to the world of programming and I'm very glad they're people that enjoy building them just because I do not is not a criticism of specifications as a Pursuit, but it does make us very strange speakers at a conference designed around a specification like how did we get involved with async API in the first place, to answer that we need to tell you a little bit about webhoe DB, our software product I'll keep this short since this is not a sales pitch webport DB is a self-hosted or fully managed data Reservoir that instantly synchronizes, schematizes and normalizes data from third and first party apis and can be queried using SQL or put another way.

E

We take data from apis jam it into a postgres database, so it's normalized and structured, and then you query that database the biggest challenge of an integration product like this is the sheer variety of programmer expression that exists every time. We start a new integration. We have to be prepared for anything, no documentation, examples or staging environments are ever complete and many apis lack them entirely.

E

This is a particular problem for webhook DB, because we also support what we call the API 99, which is the long tail of companies that offer apis as an additional thing, rather than their main product. Integrating your stripes and your shopify's isn't so difficult, but most apis aren't up to those standards.

E

This, for example, is part of an actual payload from an API. We support there's a lot going on here and it's trickier than it seems at first look. You can't really tell with a start date an end date referred to in terms of AM and PM. You can only know for sure by seeing more data, so every time we have to integrate an API we'd have to carefully go over documentation and examples.

E

Implement, based on that hope, everything's correct and tell me if you've heard this one before continually find out all the ways the production API differs from what's documented, but we're lazy and a small company and I don't like fixing production issues at my kids soccer practice and we like building silly but sometimes useful stuff. So we stepped back and said: is there a better way? Yes, there is well.

E

Maybe there is what we realized is that, instead of integrating an API and then testing it live, we could capture a bunch of data from production and use that to drive the integration from there. We realized it's actually quite difficult to understand an API by looking at 1000 different, individual API requests. We needed a way to classify them.

E

This is what led us to async API. Instead of using a specification to drive and control the development of a service, we could use it to classify and describe a service that already exists. That is, we could feed individual requests into a program that generated an async API spec. Even before we started writing real integration and then we decided we could use the specification to create fixtured events and publish them against webhooktb as a way to regression test. Every single one of our Integrations.

E

It was a real aha moment that, instead of using the spec to circumscribe development, we could use it to explore unfamiliar systems and navigate how we integrate with them. This could bridge the gap between the highly structured procedural world of specifications and the free or flowing more artisanal style of programming that we like to do so. We set out to build a tool that could accomplish this.

E

For us to do this, we depend on Cutting, Edge techniques across the disciplines of machine learning, genetic programming, multivariate statistics and formal proof, modeling just kidding it's a lot of. If statements, the tool we built is called mocks, populi or moxpop. For short, when we think about authoring specifications, there's normally a central group that makes decisions, and then it is up to other devs to ensure that their services conform flips, that on its head, the implementation is what drives the specification moxpop generates, specs for how your service integrates with an external service.

E

This perspective has been important for us, while working on webhookdb, because we're working primarily with third-party apis that don't have formal specs and cannot be modified.

E

Here's how all this works. Conceptually events are published by external services on the Left To Your Service in the center, when you use Mock's populi to build a stuck for your service. These events end up describing subscription messages, as events from the external service are handled by your service, you sample and record them at full fidelity those recorded events get fed into moxpop, which generates async API specifications and then incrementally improves them using newly recorded events.

E

Finally, you can use those specs to generate fixtures that you can publish back to your service as part of integration tests or to generate data for unit or fast testing.

E

That was a lot, so, let's break it down into four steps. First step is recording events. Then we use the recorded events to generate Json schema for message payloads. Then we use the recorded events to generate the rest of the async API spec and finally, we turn generated specifications into fixtured events and publish them to our service.

E

This first step is the most straightforward capture, full Fidelity events, in whatever format they're being published, for example, if these are HTTP events, you'll need to capture the method, URL, headers and body, even if you think you only care about the body if you've captured enough information so that you can replay the event you did it right also you'll probably want to sample the events you capture, there's no reason: you'll need to run moxpop for each event, step two generate Json schema for message payloads.

E

This is where moxpop enters the picture, and things start to get interesting in this step, we're not actually focused on async API, we're just focused on Json schema. We take a Json object and generate the Json schema that describes it now without using box top here's. The Json schema that we would expect on the left. Here you can see. We have two different integers from our experiences developers. We know that these integers probably mean very different things, but in the schema that gets generated, they're both described in the same way.

E

So the the schema is very accurate, but it's not very precise to generate schemas that we can use to generate fixtures. We need to be both precise and accurate. These targets from left to right, Show, Low, precision and low accuracy like a Json schema with just a type, which is what we just saw.

E

It doesn't provide much detail about anything that we have high precision and low accuracy like if we generate constraints that are wrong low Precision, but high accuracy is like if we use the format property, knowing a string is in a date, format is useful, but it doesn't tell us anything about the actual date being represented, and finally, we have high precision and high accuracy whatever. That means we need to achieve High, precision and high accuracy. That's what moxtop schema generation is all about.

E

This is where we start to get creative. We can use async, API extension properties to provide a new layer of precision to the data we've seen using moxpop. This is the Json schema we'd end up with for the previous example, you can see here that we have an extremely high level of precision, but probably too high, we've made a lot of assumptions. These assumptions are likely to drive down accuracy, we assume X will always be a 32-bit integer and that Z will be an integer and a timestamp. We can't know these things for sure, though.

E

It's very likely we're wrong about one or all of the assumptions we've made. So in order to improve accuracy, we can generate schemas for many payloads and merge them together. This resulting schema has a decreased Precision but increased accuracy. For example, what we thought was an integer. It may actually be a floating Point number or it could be a string in some cases. We keep fitting the schema to the data we see the fitting process is commutative, so we can keep doing this incrementally.

E

As we record new events, the resulting schema will always be accurate for all events ever processed.

E

However, there are a few cases where we.

E

Can random string of ASCII letters and numbers? We can't tell what it is right away. It might be an identifier or maybe a free-form text. Entry in these cases, maxpop will store and accumulate the values we observe in each payload. As we see on the left here.

E

Eventually, after we've sampled enough data, we can make a decision about the schema. For example, if every value we've seen has the same length and they'll seem like identifiers, we can assume it's an ID field.

E

We do the same thing for enumes at some point. We've seen enough samples and seen few enough distinct, email-like values that we can assume the field as an enum. These enum values get put into the enum Json scheme of property.

E

Recording information in the X scene. Values property is tricky because there may be sensitive values. We don't want to include sometimes access tokens or IDs come through in an event. Whenever we get a string value that looks like gibberish, we generate a new string value of the same size and use that so that we never write out IDs tokens or anything else, sensitive.

E

One last thing: Max talk does is record copies of payloads into the examples field. This allows you to see actual event payloads in the spec. We try to be clever about when we collect them like when a property format, changes being selective about examples, Bridges the gap between reviewing the API and the abstract. Only through the specification and reviewing it only through Live Events you get the best of both worlds. The API is condensed into a spec and every example shows you some unexpected change vector so putting it all together.

E

Here are some of the formats and properties moxpop handles. You can see full examples in the GitHub repo. The version here uses a super set of async API and Json schema formats, plus some formats describing data we've seen in the wild like numerical strings and integer timestamps. These additional formats, coupled with the constraints and the scene values, give us an accurate and precise description of the data we see from an API.

E

So the previous step was all about generating Json schema payloads we're going to take the same idea and apply it to the rest of the async API specification.

E

We can actually generate most of the spec from the full Fidelity events. We record. We can't handle sections like info that require out-of-band information, but we can do quite a lot.

E

Oh wouldn't you hear that the examples in this section are HTTP specific, because webhubdb is focused on web hooks. Most of our usage is HTTP based. All of these examples should work for other protocols, but we wanted to warn you as we'll be using HTTP specific terms ahead.

E

We can take each component of the HTTP event and figure out what parts of the spec the component applies to path, method and body map directly to channels Channel, bindings and message payload, but request. Headers are more interesting. The HTTP request, header is mapped to multiple parts of the spec we categorize headers into special protocol and application. Headers special headers are those with a special purpose, for example. Host and accept can be used in the server object described in the server URL and protocol versions, the content type headers.

E

That's the message, content type and we can figure out the message correlation ID by guessing which header, if any, looks like it's the correlation ID for protocol headers, we keep a list based on HTTP standards. Any protocol headers are set on the message HTTP bindings headers, as in this example, we include the actual header value in the Json schema, so the spec is easier to understand. Protocol headers, as you may guess, can't really be fixtured since the values meaningful for the protocol.

E

In these cases we end up ignoring them or using the X last value verbatim. Finally, anything we don't identify as a protocol or special header. We treat as an application header for application headers. We use the standard moxpop deriving and fitting, so they get as much Fidelity as the message payloads.

E

When you put this all together, we can take some recorded events and generate a totally usable and valid async API specification.

E

You can see the full example in the GitHub repo, which is linked to at the end of The Talk, all right we're. Finally, at the reason we built moxpopular in the first place, which is to use the generated async API specs to fixture and publish a huge number of events against the service, there are two types of fixturing going on and here's the first one mozz pop can create fixtured objects for the Json schema we saw generated previously.

E

It uses all the extension properties to make sure the fixtured value looks pretty close to real, there's, obviously a lot of fake data here, but from a technical standpoint, this looks legit and probably exercises the same code and database paths that real data would the detail in the fixtures comes mostly from the number of custom formats which we saw earlier, but also the extension property Fields acts as constraints that narrow down the fixture values.

E

For example, if you have a created field, that's an integer timestamp, all the fixtured values would be between the minimum and maximum. That's been seen in any recorded event. You won't get created values from 20 years ago or 20 years in the future.

E

Finally, here's what happens when you run a regression test against moxpop Vox and your services async API spec. It publishes the configured number of events against a server. We use this fixed string to both load test and integration test. Our back end.

E

What you see here is the request and response dump of a fixtured HTTP request sent to the server and the actual server response. Again, you can see we're getting pretty nice fixture and data and the server accepted it happily.

E

So what have we learned? Building Max populi, three big lessons first, is that async API truly can describe any kind of API. This was our first time using it and we were super impressed. I've done pretty extensive work with the open, API and async API has some subtle and significant improvements. I think all of them are great. I know I'm preaching to the choir, but I did want to mention it and affirm the amazing work. So many folks here have done as Outsiders to the world of specifications.

E

uh This work all gets a huge thumbs up from us. Our second takeaway is probably what you started thinking a couple minutes into this presentation. Building mock spot was a lot more work than not building it. While it's been useful, I wouldn't say it's been essential for us at webhoe TV at least not yet.

E

We do expect this to change as it's developed further and we get to lean on it for more and more new Integrations, uh we may have been able to solve some of our problems with other tools or with a little manual labor. However- and this is a big, however, the third takeaway is whether or not it was worth it. uh Building moxpot was a whole lot of fun. Software development at scale is frustrating and stressful I.

E

Don't have to tell all of you uh and two of the best ways we have found to make it more fun are to build better, automated testing and tooling, and to work on tightly scoped projects. That'll allow us to be creative within that scope. In the case of box pop, uh when your project is a tightly scoped testing tool, that's the best of all worlds. Voxpop helped fill out integration testing around webrtv and it was a ton of fun to work on.

E

This is largely because async API is so well done from the spec to the tooling to the documentation and the community. I really can't say enough, good things you're watching us today you can go check out. Mux, popular and GitHub go project you can use as a CLI or a library, but even better, there's a link in the readme that allows you to try this whole thing out yourself using curl.

E

Really you should go grab some events from your own Services run them through the end point and see what sort of async API spec you end up with. We hope it's as useful to your teams as it has been to ours.

E

You can also reach Rob and me via email. If you want to follow up we're happy to answer any questions or just talk shop.

E

Finally, if you're dealing with any third-party API Integrations, you should check out webhosetv and if you're ever in Portland Oregon, please let us know we'd love to buy you a coffee and tea or just chat thanks again to async API econ. If this has been great.

E

Hey folks, um what an amazing presentation from um Natalie and Rob so I'm bringing you folks back up. um I have Freddy.

E

Go it's coming up.

E

Hey guys come back, hey no I can see, I can say, um Rob um has the better version of himself. Oh where'd. She go.

E

Is it gonna be representing you Rob? She don't have to answer some questions. Yeah yeah, she's all brought up to speed.

E

um So you guys just gave an amazing presentation and I absolutely love it and yeah, but I have a question right. So you said you have you know some experience with open apis but of course I I I think API is far more better. If you can say it, I can say it right. So second player is from a better in my opinion.

E

But um so are you saying even if I want to talk to men, if I want to, you know generate um for restful apis as I'm still going to get um my results in the sick, API specification right, yeah I, uh so as part of building so I mentioned before the uh presentation we have a just like a hosted service. That's basically just runs the CLI for whatever your your request.

E

Input is um I in order to build the Swagger for that I went into a or open API 3.0 for that I went into a project I built uh a couple years ago and I was like: oh man, I know, I can't use open. It's so there's so much complexity to it compared to uh async API, which I've done a lot less work in, but just felt so much easier to use that if I were to go back, we build a lot of apis.

E

We don't do a ton of like Enterprise type of event-driven architectures, um but if I were to just build a spec for an API I was building, I would I would do async API and then just rely on people understanding. You know it's a request response I mean you can document the request response and everything, but it handles um it, handles everything and handles it. Super well.

E

Interested interesting I love that you know because then, technically you know I've been in this um kind of conversations where I keep telling folks. You know you can use async API to also with restful apis right, even though its main idea is just to for event driven apis right, but I can say it does everything right. So why can't we just use it instead of you know, which is an open API.

E

You know just have everything in just one single place but yeah as I say folks telling me you know just to use open API for restful purpose and Justice event, API with async, API and I'm like why not you could just use both with SDK API So. Based on that. Would you suggest folks, just focusing on the zip API for both grsful purpose and eventually purpose yeah I would just stick with async kpi for uh documenting restful.

E

Apis I, don't see a compelling there's nothing that I see in open API that I am unable to represent effectively with async.

E

Oh great, so uh so I think I, don't think we have okay I think. Can we don't pay to see if we have a question for you on on the channels? um Let's see.

E

E

So I'm so excited to you know to be to be the person to host you guys, and you guys gave an amazing presentation and I can say um you guys you guys are very, very awesome and yeah, so I I will be I'm. Working like I'll, be testing out this amazing tools, you guys are built- and you know, maybe create some content around it to also share with the community about an amazing stuff that is actually going on with people that are helping push in the async apis initiative forward with such tools and yeah I'm.

E

Looking forward to doing that, and also collaborating with you guys, I know, I can't wait. I can't wait to do that. All right, excellent, uh so um I think that's yeah, of course. Definitely um so I think that's everything. So do you guys have anything to share before you guys say goodbye just thanks for the opportunity.

E

um Yeah, this uh I think Mark's popular is fun, so I hope you guys do too same. Of course you know looking forward to the future of um your Web book, um DB and I'm pretty sure the future is exciting and yeah. So we as a community also are here for you guys. You know um so looking forward to seeing you guys next year at the secret conference, I'm pretty sure by next day you guys would have some amazing features so like demo for us. So all right, bye, guys bye-bye.

E

So we just had a wonderful session um from Rob and Natalie and yes, that would be the end of the Asian Cafe conference. 2022, that's the last presentation, but Herman. um It's been a long run.

E

You know, since the last and two days, we've had amazing presenters and we've learned so much personally, I've learned so much and I've had so many ideas for some stuff for me to work on after this conference from presenters and I can see people doing numbers and stuff building amazing tools with eating API, spec and also so much community support from this kpi community and yeah. So before we call it a day, um I'll be bringing um an amazing person. I know you folks have seen a face so much um on stage.

E

um So truly you hope.

E

Remember: hey Ace, hi Julie, so um so I'm, so I'm, so I'm so glad you know, you know um for your contribution and it did amazing for this conference. I can I'm giving you that I'm, giving you a five star and I'm so sad to bring up um not being able to bring up Ben because Ben has to go. It would have made sense if the three of us were here um so for folks watching watching this live stream um totally did an amazing job being I.

E

Think she's following I wouldn't say one of the best I would say the best host so far for the conference 2022. She did an amazing job shout out to you Julie at off. So.

E

Would you would you I would love you to give a closer remark for the sinking back conference? 2022 hey! This has been fun: I love the community, I love the speakers. Everything was so valuable and everybody has been giving us have been dropping nuggets after nuggets. After nuggets and I'm sure everybody who has been watching since day, one will benefit from each and every speaker that is spoken on this platform.

E

So, yes, shout out to the organizers to the community, the people off stage shout out to them for being behind the scenes and making sure that everything rolls for us on stage as well. So big apps and kudos to everybody who made everything possible and Flow.

E

So thanks Julie, um so I would I would also. Can you hear me yes, I can hear you hey that okay awesome, so, um as truly has given us a closing remark for the instant API conference, 2022 and I think days have been a long and fun ride so much knowledge shared so many stuff. um So Tulia has given a long shout out to the amazing folks amazes that made this conference possible and I will also be adding an additional shout out to the amazing folks behind instant kpi.

E

um So many you know, I, don't even know what to call them. I think they are, they are they are superheroes and they deserve a cape right. So I'm bringing you guys a cape, you guys did awesome and yeah so foreign, but we don't have a choice. Yes all good things of the county and, unfortunately yeah it definitely comes my name all right guys, um looking forward to seeing you guys next year and I'm so open up this in Japan next 2023 conference would be hybrid to be in person and also online and yeah.

E

So if it is the best and then I get to have that beer with Fran, um so looking forward to seeing you guys next year, bye guys. Thank you so much peace.

E