AsyncAPI Community, 21 Jan 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: AsyncAPI SIG meeting 15 (Jan 21, 2020)

Description

This is the recording for the AsyncAPI Special Interest Group (SIG) meeting #15.

Attendees:
- Fran Méndez
- Jonas Lagoni
- Michael Davis
- Robert Diebels

Moderation:
- Fran Méndez

Agenda:
- Channel deprecation
- Generator templates maturity and ownership
- Q&A

Chat dump:
18:11:43 From Robert Diebels : Was lagoni.
18:21:36 From Robert Diebels : Are there any implementations of this at a protocol level?
18:22:04 From Robert Diebels : Other than HTTP.
18:26:06 From Fran Méndez : I don’t know any other protocol implementing this
18:30:21 From Robert Diebels : Shouldn't a template support everything that's in the version of the spec it's templating for? I don't think a valid spec should lead to generated code that's incomplete.
18:32:10 From Fran Méndez : https://app.quicktype.io/
18:34:26 From Robert Diebels : They have an NPM package afaics.
18:38:14 From Fran Méndez : https://github.com/quicktype/quicktype/issues/1293
18:42:54 From Robert Diebels : What about creating plugins?

A

So, let's start by looking at the agenda so.

A

Someone today proposes I, don't remember, to be honest, the channel duplication, property, I say it was you yes, yes,.

B

Based on the usual race and slack.

A

Yeah, yes, nyan nyan praise to you sue.

A

A

My so my current thoughts about this is.

A

Channels can be defined across many files, not just one so, for instance, channel might be defined, one is in KP a file for the producer and another is in keep a file for the consumers, and you know all the stuff. What happens if you deprecated it in one side, but not in another? You know what I mean.

A

But that problem applies to everything. To be honest, like what happens, if you define some messaging one side and another message in another side, so it's not really a problem, but maybe that the suggestion here will be that you define your channels in separate file and just reference them right. So you always have just a single division.

A

So sorry go no.

B

It's just one problem with that is that, even if you define it in a separate file, you will still have the issue that you can have the exact same channel.

B

Okay kind of help can put like you get a problem with the producer and the consumer. Who is what and yeah.

A

So that makes me think again about the structure of the spec itself, like, maybe because, when I started, this I copied so much open API. Maybe it made sense, but now the more I work with this is the more I realized that it can create a lot of problems because of the decentralized nature of the messaging messaging. Api is right, as opposed to REST API, so HTTP API, so so yeah, oh I, think we can probably or we we should probably think about it for future versions feature major versions, that's something that we can produce.

A

B

A

Something like this, because that will require a rewrite of the spec and start almost start from scratch, with with how it is structured right anyway, regarding regarding the deprecation property for channels, I, don't see any inconvenience aside from the one that I mentioned, but that again is an inconvenience that it's not just with the deprecation it's with everything inside the channel definition, so it will make things worse for heck. This is just a problem that already exists. It doesn't create a new problem, so yeah I, don't know other than that.

A

When you duplicate the channel, you don't you usually don't deprecated it and that's it, and you usually offer your teams more information right like, for instance, you can use this channel until this date or you can or you will be able to connect, but nobody will be producing messages in this channel until certain date or whatever right so instead of true or false, maybe it will contain more information or it so allow you to put more information aside from for false, like the date and if I'm not mistaken, there was a there's.

A

A similar debate on json schema for the deprecated work for the deprecated keyword and there's also something that attic fielder proposed I. Think it's a sunset header for HTTP.

A

We can probably learn from this about the potential needs that we will. We will have in case. We want to duplicate something right, so sunsets.

A

Yeah, it's a it's a date, so the sunset header. Let me just share my screen I- think it's gonna be easier.

A

So this is the sunset header for HTTP, and this is an example here, so you don't say you don't say that something is deprecated.

A

Instead, you you say when you're the sunset something right like, for instance, the date in this case, the date and I. Don't I, haven't read this thoroughly. I mean not thoroughly. I haven't read this yet, but I think this is just a date, so maybe, instead of having deprecated, we can have something like sensit hmm sounds like or deprecated, but it contains a date instead of true or false right. What do you think.

C

Sounds reasonable yeah.

B

Yeah I agree it's more than just a true or false, and I kind of like that sort. Mice might solve other problems as well, also with the regards to the deprecation flag or the sunset flag for the channels. What if we? What if we add the channel trade.

A

Yeah, that's nothing yeah, no I thought about that. I think it was too late in the process and I didn't add it, but but yeah that makes sense to have like the channels right now.

A

Yeah in case you want to have this like I, don't know you can you want to apply a trade for four channels right with the deprecation notice? That's another thing, the date and the notice you might want to leave a message there right, alright, something to say to the teams instead of instead of this channel, you subscribe to this other channel. Something like this.

A

This could be a message like kind of a human-readable message like description or it can actually be, is both a message and also another property indicating what's the new channel, if at all like where it's like, consume this information after this is deprecated right after this is sunset actually so like. What's the other channel name that this will be subscribing or the rice will be Pendleton yeah.

A

So, as you can see, it's not as simple as it seems when someone comes and say hey, why don't we add deprecated right.

A

Yeah, let's just add it and for some reason you had true or false, and then you want to add, there's a the message, something else and suddenly you have a breaking change, because the true or false can not allow anything else, so yeah. So that's why I'm taking these consciously like? Let's, let's because just because.

A

This came, this can be a problem in the future right if we don't design it correctly so yeah anyway, um so I think I'm, gonna I, think I'm gonna create the issue, so we don't forget about it and in the meantime, there was.

A

So you had some concerns units also explain why I do this, so you don't have to get.

B

So it's regarding, but simply it's like I'm still working on the Nets client, both for typescript and for dotnet and I, was wondering when that gets done and gets merged into the generator I guess we consider it kinda an official template, since it is directly in included in the generator yeah, because so it's more like when, when other people come and use this template, and what can they expect quality wise from these templates?

B

Can the issue are the issues being fixed when I create one who are the responsible person for it as soon as it's handed up office and official template.

B

A

Can reply this, like instance in, in my opinion, if you move the template to the generator repo as a let's call it official or whatever template, then it stays in KPI team.

A

The responsible to maintain it right and the way to open issues is.

A

Well, it's actually the way to communicate issues is actually by opening these using github right. So that's a that's. What I will think I think? That's that's how how people might what people might expect right? The baton will be the responsible to to handle this and quality wise in the beginning. I, don't think people the beginning, I, don't think we will be able to provide a huge quality right, but as long as.

A

The templates received more issues and we fix them and all the stuff quality wise. It will become better right over time. So.

B

So yeah that was also a concern like you. Have all these kind of scenarios, with Jason schemes like all off one off and off and everything and see do they need to be included in the template for it to be accepted in the first place, for example,.

A

Are you considering donating something I'm.

B

Considering adding all of the support for all of them, that's more! No.

A

I, don't think honestly at this point in time, I don't think we should be so picky about it like the easiest. It is for people to use something, even if it's not perfect the better and then, if you have something or anyone has some template and it doesn't support all of and one-off and all the stuff. That's fine actually I think the only one supporting this is the HTML template and the markdown template, but the code generators I, don't think they are supporting a lot of templates.

C

I'm working on I'm, not supporting those yeah.

A

C

I could put in rules, you know in the java, the java pojos that I'm generating to sort of say well, yeah I mean this is a required field, but I think that's more trouble than it's worth right now. You know that kind of level of validation is, you know it's a very fine meant we could put in later, but I think just to get things working nicely for people and to have something that people can actually use.

C

It's I think there's little value to be gained from from that kind of you know very detailed validation, so I've chosen not to not to bother with those things. Yet you know myself yeah.

A

Yeah I think it's easier to stuff. Like this, you started with the most common cases you avoid going so deep into the one up a lot and all the possibilities into decision schema also decent schema also allows, if then else and all the stuff, and that that's that's another thing.

A

My recommendation is and start with something it might not be good for everyone, but it might be good for muscled people. So that's all right.

B

One one concern I have is that then it's kind of it's gonna, be the templates dictating how you should write your async API file and different simply or your async API files will not be.

B

You won't be able to go from, for example, if we have a calf and official Kafka client template and then go over to using nets just by one command, basically, and that's kind of like and I guess, an issue from the.

B

From the users kind of perspective, you.

A

Wind it if you mean that, for instance, one template supports, for instance, Olaf and the other doesn't.

B

Yeah, for example, and if they already have I, don't know 20 clients they're, not gonna, rewrite everything in there, because our template doesn't support it like this. So it's more like in that way. Maybe.

C

The solution of that is to use the bindings.

C

You know on each channel where you can have protocol specific extra attributes, so you're saying well, if I'm using Kafka, then I can add, like the consumer group attribute that doesn't make sense with an QT or something like that so yeah that may be, if maybe that's the way to get around it, I mean I'm grappling with the same thing, but unfortunately guys I have to go to another meeting, so I have to break off now, but it's unfortunate because this is something that I'm having to think a lot about to just this very issue, so I hope.

C

Maybe I'll watch recording later to see. If there's any more insights about this, but anyway see I have a great day.

A

So um I think it's what Michael said is correct, but there's still a problem and the problem is it's not with specific protocol with protocol specific information, it's more like you said, for instance, Jesus keema stuff that we don't completely support on every level. Yeah.

B

I expect them yeah.

A

You're right, I mean I, think I will not say that it will dictate how you write this in KPI. That's probably a lot, but it can happen for sure. Like you, you know that the templates don't support all of, for instance, and you try to avoid using Allah during second.

A

I'm trying to think about or I'm trying to guess in which situations you can decide to both to avoid using a lot right and how you will prevent it as an example but yeah I.

B

Mean there are workarounds, but if you already have us Inc API file, a company won't spend the time to convert. That document.

A

Yeah and what Robert is saying that so the new template support everything. That's in the version of the spec, it's templating for I, don't think about specs will lead to generated code. That's incomplete! That's right! The problem with this is again when that's the goal we the end goal, is to reach that point, that level of activity, but to get there. It requires a huge work correct in each template.

A

On top of that right, I think once we have, for instance, I think that the the case of HTML and markdown, this is sort of solved with some minor issues, but it is easily solvable, but in the case of code generators, the thing here is that we will have to figure out how to map jason schema, two types to a class definition to an object definition or whatever you want to call it right.

A

There are libraries for that. Specifically there is this one, quick type, I, don't know if you, if you've heard of this but I'm gonna link it here, quick type.

A

So I'm gonna link the playground. They have so useful to understand what they do and.

A

The thing is that they think they don't pound this their library, it's open-source, but it's not I, think it's not maintained as a separate library in the core of their app of their app code.

A

So I don't know it's. We can probably have a chat with them and and and see if we can help extracting this functionality to a separate library. So we don't have to recreate everything again, sort.

B

Of showed this before I had indeed with this yeah.

A

You haven't seen this: no I haven't yeah yeah, it was for me. It was like yes, why I haven't seen this before yeah. This is this is amazing and yeah and ends a lot of work, as you can see so yeah. Why not taking advantage that it's open source, yeah and probably extracting this to a separate library with their permission or probably them doing it or I, don't know I, don't know, maybe they are not in in the mood to do it whatever.

A

If, if that's the case, we will have to create a because it's open source will have to create this separate library and we will have to maintain get them ourselves. I.

B

Mean I can easily write if I just copy pests of the scheme. In here the JSON scheme, I could I mean all I need is like at an old template and then I can copy/paste it exactly.

A

So that's that's the cool thing like if you have it as a library you can yeah your template, can leverage this library and and say hey just create a class for these schema right and it will create a java class or it will create a go class or go. Oh, they.

B

Already have a.

A

B

A

They have, they have lots of stuff, so but I'm not sure they are in the mood to maintain them separately. I have seen some issues in the in the yeah. They have an NPM package, but what I'm saying the issues is that they don't maintain, so they didn't fall semantic versioning and they break the the API all the time. Is they just doing this for their for their app and then I? Don't think it's core.

B

A

Business to maintain this library, it's a separate thing, so I don't know: maybe they change it. They totally changed the perspective with this regard, and maybe we can just offer our help like hey. We can just help. You strap this, so we didn't have to do it right and we can maintain it. We can come maintain this right. I, don't know, sounds like this, and and also there's the the swagger products so for for open API. So so he also has a code generator for almost any language, any primer language.

A

So they really do that, but as far as I know, I think it's Java, it's not know Jesus so yeah the code is there. We will probably have to just translate this just translate this code no dear, so we can eat it on the generator, which is not bad, but it's far from ideal right, so yeah, but I'm yeah.

B

So this QuickTime is actually. This is what we need yeah. This is exactly what we do. Every language yeah.

A

That's exactly what we need and we can maintain this and we are in the mood to to cooperate. That's fine! If not, we can probably just fork it and maintain our own copy. Yeah.

B

A

Like this I don't know, I will prefer not to hope not to fork it yeah. So we can all benefit right from this, not just ourselves or or Ben, but all together, but yeah I can understand that maybe it's not their business, so yeah. Let me try to find this I.

A

Remember seeing an issue.

A

Yeah, it's let me check. Let me share this with you, so there's this issue here, someone asking for this and they're not exactly asking for data they're asking for documentation for the quicktype core library.

A

But the thing is it.

A

Or it seems that for a common dry print here.

A

Actually, it's been follow up in another issue from sounds of faith. We might have to do it ourselves and then yeah and then the guy says again I'm like yeah. My problem with the with using all those undocumented internal library, AP, is, is that they might break without any notice, which is true. I will be concerned as well like to rely on the library that might break every every month or in a few weeks, so so yeah anyway, the opportunities there for sure yeah. So just let's just explore this, and maybe in yeah.

B

I mean from now is you can just just use the size play around with it find that template that kinda a template of what you want and then use that yeah.

A

That's one, and another thing that might be related is the fact that, for instance, so the other day I got mentioned that some people might just want to generate this one: a template to generate Java projects, so they don't gone well, they're old. They just want this.

A

The class, the class right so I suggested creating another template and we could be just add just the Java file use and that's it then an alarm triggered in my head like hey, then you will have to maintain this template class you'll have to maintain all the Java templates and they also they also behave the same right when generating the Java classes. So I wonder if we can probably create something like a dependency system between templates. Maybe it's a lot right.

A

Maybe it's over engineering this bit like having a Java template, depend on the padule template yeah template. So we don't duplicate things. I.

B

Mean if it do it like that, instead of having something like quick type in the back end like it's the same, whatever is the same.

B

It's the same result either way yeah the one thing we just have: control over yeah.

A

Exactly so, in any case, if we use quick type, something similar, then it's perfect, you don't need to worry about it. Just update all the templates to the latest QuickTime version or you make sure that all of them are in the same version and and that's it that's fine, but yeah, and that that also raises the issue of template specific dependencies right.

A

So, for instance, in order for my template to perform this task, which is so complex, I would like to npm install this library right, which is not in the it's not in the in the package, Jason initially right on the generator presentation, but I need just for this template this specific template- and this is already happening by the way, with a java with a java template that Michael is working on. So he will need to do some stuff that will be used useful just for him, and that makes sense. I mean this way.

A

You don't have to code everything yourself right. You can use other practicals, but then we we entered that Landwehr.

A

Again, templates can have dependencies specific dependencies, but I think these then like Roberts saying what about creating plugins, then this opens also the door to defining a plug-in system, apply an API and we have we sort of have some kind of plug-in system, which is a the hooks for the templates kind of.

A

So probably we can, we could use the hooks to npm install some specific dependency like it will install the libraries before before anything is generated.

A

But then I was thinking then there's a problem. Yeah I was sort of too many things my head so then there's another problem. The problem is that some people use the templates even ourselves, with a playground not to generate coal in the file system, but to retrieve the result as a string, and we don't want to install anything on the system right.

A

We don't want to install additional packages. We don't want to run additional commands in system because we might not have permissions to do to do these things so, for instance, if you're running the generator in a CI system, you might not have permission to run impedance so there or things like this right so or imagine, for instance, in the playground you want to generate.

A

You want to generate the specific you want to generate code from from specific from your racing, keep a file using the playground. You don't want to run the command there right on the server like running, npm, install' and waiting for install- or in this case maybe it's it's. A different approach can be done in a different anyway.

B

A

Thing the thing here is that I'm jumping from too many things to other things. So the thing here to summarize is it: there are needs for template dependencies.

A

There are needs for the types generations or classes generation and.

A

And there might be even and something that I find myself repeating all the time in different templates is that some of the some of the filters or helpers depends on what you call it I think called filters. Now. Filter are the same basically the same therapy in each template, so some of this functionality might be worth struct into generator itself. So you don't have to repeat this whole time and you maintain it on a single place. If you need something totally different than you create your own filter, that's still.

B

A

But but I think what I'm seeing here is that, in terms of repeating code, we can still do better right, like we've got a lot of care, which is fine, I mean we still find our way and and the generators not even in version one, and for that reason, because something is gonna break something right like we're.

A

Gonna have to break backward compatibility so for sure, and that's I'm sure about it and and maybe even templates will become packages on its own, so you can reference them with a URL or with an NPM name or that Eaton NPM install template. Something like this so yeah. So the to summarize I will say that is. Is this three things like? We need to avoid repeating some curves in the filters generate code, generate types of classes and and also.

A

Make sure that ten plates have there are dependencies right, so I think it's. This is food for thought, so maybe we can keep chatting on slack, maybe and instead of moving everything. I mean I'm, just saying, because here we are just the three of us.

A

It will be a little bit also.

B

It's not something that we have to address right at this moment exactly as soon as we get more templates instability in a rhythm we.

A

B

Of begin to see a pattern so.

A

What I'm gonna do now is I'm gonna, create some action items or food for that right. So we think about it and we can discuss in in the channel with more people involved and also what day today is twenty-first. Okay, so I cannot say anything yet. But someone is it's very amazing KPI full time in February, first I'm, just respecting his privacy.

B

A

So yeah he will be in charge of these things as well, so like he will be participating on this so yeah. So maybe this is something that we can keep discussing. What do you think yeah.

B

Definitely I still think it's something that we don't need or have to address at the moment, yeah exactly.

A

Not at the moment, but I think it's worth addressing, it soon definitely yeah before it. Yet the problem gets bigger, yeah and it's gonna. It's gonna be a bigger pain to migrate to the new system of templates or new way of doing things, yeah cool. So anything else you want to erase today.

B

I, don't think I have anything else.

A

Let's wait for rubber.

A

He might be receiving the package now I'm.

B

A

So yeah well in the meantime, and good luck with this exo-m's. Why not to think so much about async api or anything related until you finish now, that's also why.

B

I, haven't done pretty much anything since the tambour yeah.

A

Now, that's that's fine, just focus on that and getting this done and.

A

Cool so I'm gonna stop recording. In the meantime, we can then chat.