Add a meeting Rate this page

A

Hey.

A

Tommy.

B

Hey, oh, hey, remy how's. It going pretty good.

B

We technically, I think, only have an hour because I'm pretty sure that the workflow group uses the exact same zoom call at that uh at 1.

C

Pm eastern, so I have a hot stop at 10 pst anyway, so yeah.

B

Well, I gotta eat lunch, so yeah, it's all important stuff.

B

So, let's see who's gonna join.

D

Did you have a chance to see the video.

B

No, unfortunately, I did not I apologize. I just saw it this morning and I haven't had a chance to actually run it. So how long is the video, by the way three minutes.

C

Well, I can, I can do it live yeah.

D

Maybe.

B

Yeah.

C

That's that's like people join first then we'll.

B

Do that, just out of curiosity, did you um did you implement a ping service like I described here.

D

uh Yeah, it's just. I need to check the the ping is not exactly the same. I have a pink service and a cat service which I can call the way I want. So I call it garfield and then basically, every 15 seconds, the cats send an event of like an action like you're sleeping sitting, yarning or whatever, and I also did the gateway so, but I I can, while we wait, I can show you what I did and I can restart if someone joins yeah. I don't mind.

B

Racism yeah go, go ahead and go ahead and start we can always redo it again and it's only three minutes. Let me stop sharing stop shooter there we go.

C

What okay.

D

Okay, there's like those uh max with like a broken keyboard. It's super annoying.

D

Okay, you see my screen, so basically I have three services, the gateway, garfield and the pinging service, so they all are on. You see the url, so I just connect them on several parts.

D

In the discovery you basically see the slash service, it's just a display of services, so garfield does like as soon as this and all those type events.

D

Correct me whenever I say wrong thing about, like the words, I think I need to repeat the spec to get every words perfect, then the pink service is basically with this source and it's sending this event.

E

So.

D

From this ui I can basically subscribe when I subscribe here, so the ping is every five seconds, so you can see that I start receiving the the pink events cool.

D

I can also subscribe to barefield and in that case, uh we'll start receiving so thing is every five seconds while garfield is every 15, so in uh the next five seconds we should see a garfield here, excellent, so that's nice. But for me it's just like one part.

D

uh Now I have a gateway uh which is intermediary right now. He has nothing like in the discovery. You can see it's empty, but I want the gateway to start proxy and uh like being the intermediary of garfield, so I'm gonna ask it to connect to garfield. So now you can see that in the discovery I have this.

D

So if I subscribe to the gateway so now like the little, the green thing is just to show that I was subscribing to this. So I should see the events arriving here within the 15 seconds. So you see here one event, of course, if, as a consumer I subscribe to birth, then in the next 15 seconds we will see a double event, because I will receive it from the gateway and also from the garfield service. So here that's why we see a double one.

B

So, just just be clear: you said you subscribed to the gateway. What you really mean is you're, subscribing to the gateways version of the garfield service right.

D

Yeah like yeah, so basically it's like I'm calling this. um So I did some simplification because it's still quite a bit of work to implement all that and do the ui uh but, um like I didn't, send the subscription config so like, and I raised some questions by doing like by doing that, even on the sdks side. uh The way we emit the the event. So I did publish everything on github and my goal is also like to open the discussion to understand so like in that gateway.

D

I can also connect to ping and then suddenly, so you see the gateway who will expose basically those two sources and send events from both and like it allows me. My scenario is basically garfield or ping will be github and, like other of our provider, my gateway will be like an xo gateway, and so my employee will go to the gateway like my teams and they don't care about how the gateway connects to garfield.

B

Just like cassidy, when you subscribe to the gateway's version of garfield, is the url pointing to garfield or is it from the client's perspective? Is the url pointing to the gateway, or is it pointing to the real garfield to the gateway directly.

D

Because, like I want to be able to have network isolation between so and like this gateway should like, there is way more feature that I should add. I was just staying at like a simple first level, but so when you connect it automatically subscribe to the service for now without filter. So it's already start to proxy, even if no one has subscribed to the service yet so like. I don't do cleaver matching to understand.

D

If I need to subscribe in advance or not but like, and you see as soon as I disconnect it will remove from the discovery service and then now I disconnect the two of them. So the gateway is not doing anything anymore and then I can still see my two services.

B

So, in your case, the the connect disconnect is making it. So the gateway is just aware of the service.

D

Yeah.

B

Exactly available, okay,.

D

So this part is basically an api uh that is not in the spec. It's like a proxies. What I call that is the api ic for the cloud events gateway just to say: okay, like go and discover this service for me and just start proxying it and same thing for everyone. So whenever you click connect, that's where normally, if I had a schema on a subscriber subscription configuration, I will pop like a form that for you to feel what you have to feel and to subscribe. But um yeah.

D

I just want, like that's a sim simplification of the old case, but it was to have something concrete to work on.

B

This is.

D

Cool I.

B

Like it a lot so so let me answer some questions as you're coding. This up, um since your ui thingy here can subscribe to all three things: um did you have to put in any specialized code depending on what you subscribe to or is the subscribe, 100 generic and you only used metadata from the discovery endpoint.

B

So it's.

D

uh um Only use for now I just put the url, and I consider that the subscription, because this subscription api, like there is a few things that are weird in my opinion, is we put for now. The api of the subscription is about posting on a slash subscription endpoints, but yet on every service we put the url of the subscription url. So in fact it's the same for every service when you're in the discovery panel.

D

So that's like small stuff, um but it's I was like. Okay, can you? Can you repeat that more time you lost me yeah? Sorry like I can probably show a bit of the code before it might uh explain what I do so. The ui basically talks to this service, where it's like just a demo, to get like all the events, because, as inside the ui, I cannot expose the port to get the evidence.

D

So this is a small service for the ui to be able to subscribe and subscribe, and and do that and for the rest when you subscribe inside. So this button that is here.

F

So.

D

Let's click subscribe. I will basically end up here sending a subscribe with the url I want to subscribe and it will go to this. Do a post on that subscription url subscription with like the thing to be able to retrieve all the events and then it will keep it inside the service.

D

So I store it in like the events buffer and the ui is basically for now just pulling because, like I didn't, have the time to do a full web socket, but it's pulling from this standpoint to get the latest events every time.

D

So that's, but that's using the endpoint like subscription here you can see. So that's the spec definition of the subscription.

B

But okay! Well, I want to make sure understand, though, because every every service in the discovery endpoint should have a unique url that you're sending the subscribe to. Yes,.

D

The url is coming from the body. So basically, if you look here.

D

In a network, I think it would be great so when I unsubscribe when I subscribe, I give him the url of the service.

D

Here so I say: okay, I want to subscribe to this service url, so we will go to that. Do slash discover like on that one. I do agree, it's a shortcut because it should do the slash, discover it's like service to get the subscription url, but basically as if you look every time, the subscription url for me is the url subscriptions.

D

Okay,.

B

Okay, so you cheat because you know you're talking to yourself.

D

Yeah on that one yeah like if I was not, then I would do the like, in fact I would use here. I because I have also that data, so I'm sorry to it's when I click.

D

So if I click here where I'm sending is in fact uh you can see the response I have the full. uh I think preview is better. You have the full stuff with the subscription url. So I could. I could use that. So it's just I do agree. I cheated a bit because uh I finished yesterday at 10 am so.

B

No because because my question was because the reason I asked that question was because I wanted to make sure that sure your implementation may choose to cheat, because you know you're talking to yourself and whatever. But I wanted to make sure that there wasn't something missing in the specification so that you could have done it through discovery as opposed to shortcut and sheets.

B

But like.

D

My shortcut is really uh a quick like I could almost fix it in a few sec, because uh right now, I'm taking the url of the service that I display here. Why? Basically, the only thing I have to do is to take the url that is shown in this pattern. So it's really quick, but what I was saying is more um like I think when I I'm gonna subscribe to this one.

D

If I sub connect those two so now I have like the gateway with like the two uh two type, and here you will see that the subscription url will end up being uh now. I did an overwrite because it should be that's where. Basically, I should override the subscription url to say.

F

That.

D

Now it's a gateway, it's not the original service right, that's basically what I do because my shortcut is enabling that by just uh because when I click on that it's gonna be called the right fueler.

D

So that's just I need to override this part, but it was not really in part of the spec, because it's more part of the service that I call a gateway which is basically not this one, it's in the discovery, but what I really noticed also is the current sdk doesn't match, doesn't fit well with description and discovery, because the way you send the events once you emit the event you need to already know. What is your current subscription to send to the current subscriptions and the way the sdk was designed?

D

I don't think it is a great match for that, so I need to talk with the sdk people if we think that this case is the right case, if I understood the thing correctly um and what I wanted to show, you is.

B

So I'll make sure you understand that um because scott's on the call, when you say the sdk, do you mean specifically the javascript or typescript sdk? Or do you mean all sdks probably have the same problem.

D

I cannot talk for the other sdk. What happened is um I had to do when I aim it. So I, like the dummy service, are here. So basically I have my ping service, it's just creating. So that's just register the way I was showing you. It allows you to create easy discovery, service.

F

But.

D

Then I just create this new cloud event, but I had to create the cloud event emitter because in fact, the way the sdk was I was not able to plug to the emission of the events. So in fact, then this thing will call and that's why I created. I created the emitter where basically I say: okay emit me this cloud event and then it will aim it as an event.

D

So then, my subscription service can subscribe to this uh type of events to redistribute it to all its subscription, because if you don't do that, then you only consider it's a static emission. You only name it to one endpoint to yeah.

D

I hope I'm clear, maybe I'm not, but.

B

Scott does that make sense to you and do you think the golang sdk would have a similar issue.

G

uh Golang doesn't have that issue. Okay,.

D

Because, basically, whenever, like once you, when you receive a like, that's that the thing I had to call inside the subscription service is you have to go through all your subscriptions if they match the filtering to push the event to them, and so you end up doing several requests when you emit, depending on how many subscriptions you have in your service.

H

Sorry, maybe I missed what the what the problem is so.

D

Did you see the demo by the way.

G

I I came in about 10 minutes late uh forgot about this meeting, because dougie doug did not put it on the calendar. All right.

B

So it's in the note, though, so I did. I did.

C

At least do something just not enough, so basically I did.

D

The ui: what was the issue with the sdk? The way it emits the the the events were not pluggable. It seems to me so you cannot implement the subscription api nicely because you have no way to to connect to it to redistribute to all your subscriptions.

D

So this.

G

One is expression I see.

D

So.

G

In in go in the go sdk, you can set up this client and stuff, but you can change what the target is. If that's the question.

D

Yeah but the subscription api creates multiple targets because you have multiple subscriptions.

I

Sure.

G

Yeah, that's fine.

I

Yeah.

D

It was just like, um so I need to find the because I don't think the example are good enough.

D

Yeah, I can come back to that. I suppose, if I try to go too much in the deep there and I will lose everyone, but uh if you haven't seen scott, I can we show quickly this.

D

I did see this yeah okay, um so the nice thing is like this: uh ui could be plugged almost everywhere. So I just need to add like something to add new services, and then you can define your own service and you can play around to see the events and the discovery.

G

The one thing in the in the subscription spec I didn't like and I didn't implement, was I I think in the spec it says you have to use a query, parameter to get the subscription id.

B

So yeah change that my pr changes that last night. I did not like that at all.

G

Yeah it's it's! I didn't implement that because I was like this is garbage, I'm not doing it.

D

Yeah for me, I implemented to be honest. This way uh I propose also like a refactor just to say it should be this way with, like you create a new subscription, you get the list of subscription, you get one, so I didn't implement this way because, uh like it was not making sense to to use the old endpoint because, like you are, you are the one specifying the id or stuff like that, where I don't think it should be. So I.

I

This is basically what I implemented.

D

Too okay, so maybe we should merge that.

G

Because there's a symmetry with the discovery.

D

Api too, like the other thing, was in the open api in the schema. There is also something about proxy, where basically, it should not be like it's not me as a client who should define the proxy the server will use, so I don't think it should be inside.

D

Like if you look in the http settings here, there was like proxy credentials and proxy url, where I should not be able to define those.

D

So.

C

Yeah, I can stop sharing sorry, I don't want to eject the whole meeting but um yeah. So, okay, let me do this, since we only have 40.

B

Minutes left before get kicked out. You want to see my quick demo, oh yeah sure go for it.

B

Okay,.

G

Cool.

G

Okay, so I showed you the the ring buffer last time, but we'll set that up again.

G

And if you don't remember, it's so service one down streams to service two so basically like this is the resulting catalog of this service and it's pulling from this middle service. And then the middle service is going to pull from the rightmost service. So uh note, c, service outdated, is here outdated and before before we had epoch uh that outdated service would would propagate the ring as a little bubble of uh incorrect data.

G

But now we have epoch and we can see that the outdated service that the middle server hosts out of its normal catalog has been replaced by the actual authority of that service. So cool. Now it's steady neat.

G

um So that's just kind of proving that the epoch thing.

B

uh Works so when you say the I want to make sure on the same page, let me say: epoch thing works. Is it because you assume that if you change any metadata about the service, you now own the service.

G

Well, my assumption is that you don't change the metadata or the epoch.

J

If you're aggregating it oh or the epoch, okay, yep, okay, cool, makes sense.

G

Oh okay, so uh and then I was like, would it wouldn't it be cool if this discovery service could actually, you know, host itself?

G

And so, if you notice it has a z, but also this discovery service from netcloud minute cloud. Meta is just the stupid name that I invented for this thing, so I can curl the roll closed.

G

On port 80 and then look at services and pipe this through the gq, and we see that we can look at this.

G

So this this is a particular service and it is the subscription endpoint for this discovery service directly. So we can leverage that uh over here, I'm gonna, I'm gonna run a little bit.

B

Okay, go back for a cycle, let's see the output there, so it's interesting that so that queried the discovery. Endpoint.

B

Entry, but your your list of events is interesting. um Is that the is that the list of events that were generated or is that the list of events that you can get.

G

These are the lists of events that I can get I can subscribe to these. I invented these right. Okay,.

B

I'm sorry I just yeah that that that's correct it's just the way. I the way the text was written there it it it. I thought, was the list of events as they were being generated. So there was a live stream. Okay, never mind. I was the description. Threw me got it nevermind. I was just revisiting it.

G

Oh yeah, it's probably red herring with this, the stream. um So I have this little uh little client program. That's gonna go and it's gonna register on that particular host to this sockeye service, which is running here and I'm going to do no filtering yet and so we'll we'll see that happen.

G

uh No okay, so the only 8080 is running and then I'm going to run the clients make sure that sockeye is visible, and so I get that service subscribe. It has no content type, no data, it's just uh I subscribed, and now I can do the the same demo over here where middle service propagates the c and d services, and they show up that's neat and then I can run the final service in the ring, and I should see changes every year on sockeye.

G

Once it propagates, it probably gets every 10 seconds for this particular demo.

G

And so sockeye collates on subject and the subject happens to be the service that it's talking about, because this particular event stream is about service entries in the discovery catalog, and so you can see when c was added and it was the outdated description and now it's it's been updated and it's. The description is the correct and we also got uh we added d and a and b all right. So that's basically my demo.

G

I can oh, oh right. Oh sorry, okay, wait! One one! One change! uh So we'll shut this down. There's no storage! On my this. This thing uh we will add the filtering, so I only want to see updated events too much and then we'll register here.

G

And then we'll go and run turn these, so we should get no events until I start up this service and stuff starts getting propagated. So we'll just wait. A second for.

G

That.

G

Cool, so we only got one event because the filtering works and we get that one updated event from the internal discovery. So why I'm showing you this is. We can get both push and pull for catalog changes based on this, because you can pull the the end point or you could register you could subscribe to it and get changes which is kind of cool.

B

Very cool, excellent! Okay, so let me.

B

Okay, so let's go back to here then so you, both you guys, showed really cool demos, but I'm trying to figure out is what pieces of those demos do.

B

We need to actually write down in this document for other people to implement, or what do we need to do in order to get people to start sort of playing in part of this game right so, for example, if if I I figure at a minimum, people need to expose or register or let us know what their discovery endpoint is, which is why I was assuming everybody needs to add their own section down here, but do we need to formalize what people do with those endpoints in terms of what queries we expect them to do, or should we leave a little more free form?

B

So, for example, you guys both did something completely different with your discovery, endpoints right um scott, you got the circular thing you do subscribe to it remy you did a gateway thing going up to it, and maybe maybe we should leave a completely free form, so people can do completely different things and try to push the system.

B

But what do you guys think is the bare minimum someone who wants to play in this game should be forced to implement? Is it just a discovery endpoint? Is it discovery, endpoint discovery and subscription.

D

Yeah discovery and subscription because I think without it it's it's harm to them or.

B

Something that is nice, so when you okay, so obviously they need a discovery, endpoint list of services in there, but you then said subscription. You mean subscription for each of those services right. Well, I think the discovery endpoint.

G

Maybe one of the qualifications is that the discovery endpoint hosts services that you can actually subscribe to. Yes,.

D

It's a subscription url, but then that means you need to have a subscription api. The only issue there is like, but it seems that we are on the same page and with scott is uh we need to upgrade the subscription api because the current version is honestly not good to implement so.

B

Huge plus one yep, so I'm sorry I was taking notes, say that again.

D

Like we need to validate the new change in the subscription api because we don't want people to uh implement the current version of the subscription api, like oh.

B

You mean the query parameter thing.

D

Right yeah, like yeah, it was like the whole api was not correct, like it was not resource based for the end points and uh yeah. So.

K

Honestly dude, I I ignored.

B

Everybody came to the same conclusion: that's great yeah, so so hold on a minute. Let's.

D

Just at least, uh let's try to enjoy the same way all together by following.

L

A new.

K

One okay, so this just a quick feedback. I mean I uh because even I started working on my demo, I was not able to finish it on time. uh I kind of don't agree that having discovery and points should be a minimum criteria, because I stopped I was assuming the fact that there's already a discovery endpoint which is enlisting the subscription url. So I started working on the second half so does it really make sense to have that as a mandate.

B

So when you say second half elaborate, what you mean by the second half.

K

uh The implementation of subscription manager and, basically assuming that that url is already available in the discovery endpoint, so you still need to connect the event producer with the consumer, and that is the logic of subscription managers. I wanted to implement just the subscription api and basically have a messaging back and connect to it.

B

So if you wanted to do just the subscription api- which I think I agree with you- I think that's valid, then you would be responsible for figuring out how to register yourself with some discovery. Endpoint correct.

K

Yeah I mean I'm still, assuming that there is a discovery service behind it. It says I'm not implementing it as a part of the demo, but.

D

The thing for me is like because the next mine is still under locals, but I saw that scotch is using uh like already a public endpoint, so we could all make them publish and then to subscribe. We will need to have the discovery that works so depending for me. If you cut it or not, it's fine, it's just it needs to be there. At the end point.

F

For us to be able to explain it.

D

And one thing that, while I was implementing the discovery service is in fact relying on the subscription service, because the subscription service is the one defining the subscription url and the subscription config.

D

And that's where I started to have some interdependency between all the spec in my implementation and I think it's normal. But I was not sure it was part of the intent originally.

K

Yeah, I think it does make sense so because you really need to have the url that needs to be published as a part of your discovery uh response right. So it does make sense to have the subscription manager or the implementation of the subscription api. First before we start having a response from the for the discovery, endpoint.

B

So I'm sorry nisha, I'm I'm not following you. If you don't, if you only have the subscription manager api, you have to have them accessible, someplace or you have to have it. How are people.

K

Going to find you, if you don't have discovery.

B

Endpoint.

K

No, no I'm not this, I'm not saying that we should not have it. I'm saying it should be as part two. So first you need to have the url to which you can subscribe to and then basically, then you would create a metadata right. So implementation comes before metadata or the metadata comes before implementation. It's just.

G

Okay, positioning it. Okay, um I don't. I was able to implement the discovery and the subscription apis with no dependencies on either.

D

But how do you feel the.

K

Subscription you are earning subscription consuming, I mean that's probably because you already had the insight about what url you want to subscribe to. Right I mean, but in a case where we don't have the information that what you are to subscribe to. In that case, the subscription av implementation should go first right.

B

Well, I think it seems to me this is just a choice and how you just implement it, but it seems to me if you want to participate in the demo.

B

If you, if you're only going to do the subscription api, then you need to work with somebody to get your service registered with their discovery endpoint or if you want both, you have to implement.

D

Both.

B

Yeah.

D

Correct.

B

Right so the.

D

Order.

B

Doesn't really really matter, I mean just your.

K

Choices: yeah yeah makes sense. Okay,.

B

Okay, so if that's the minimum, people need to do just to participate, do we need to talk at all about uh what do we do to actually test this out to make sure, for example, people can talk to the discovery service to do a subscription, or should we just say no, everybody sort of write their own client tester thingy, and we want variations there to push the boundaries.

B

What I.

D

Was thinking is, maybe I can also put the ui uh like, even as a separate project, because the ui is not specific to typescript. So if we want to iterate on that to add more stuff like I liked the more detailed version of scott he's just doing the ui, it takes a few hours, at least so, maybe if we just have a common, uh because it's agnostic to the technology, so the ui should be almost a separate stuff that we can all rely on. You know.

B

Yeah, if you want to make your ui available for other people to to use, I think with the only thing. That's missing is the ability for someone to then register their service with you through through the.

D

Year.

G

Yeah.

B

Yeah.

G

But uh yeah I can say.

D

It's just a.

G

Quad events viewer- I we use it for k native, mostly.

D

ah Okay, that's right, like I, I didn't know the project yeah well,.

H

Nothing to do.

D

With these it's it just uh presents cloud events in the.

H

Ui.

D

Because what I like with the one I did is like you really see the discovery like you have the discovery plus the subscription.

D

uh So I wanted to display that at first I wanted to have a small maps of the service, but it was getting too late to do the graph part.

B

So, scott, the ability to subscribe to the discovery, endpoint itself to get changes. um Do you want to make your uh the metadata about the discovery service itself available? So people can basically copy that and steal it if they want to spit out events for the discovery endpoint? That way, we have some consistency.

G

But yeah it's it's all in I'll paste. The repo.

A

Okay, cool. Thank you.

B

Okay,.

C

Oh, thank you.

B

Hold.

C

On.

B

Oh wait. That's something really! That's your sucker thing! There we go. Thank you.

C

Okay, so let's do this scott.

G

I need to work on getting some like the the fake other supporting services up and running. I do have a version of this that's running in the kubernetes cluster that you can subscribe to updates for it, but you don't get updates. If you don't have a ring right.

D

If, uh like on my side, I was proxying directly uh without cache, so I didn't really had an issue with the epoch because in fact, whenever you query, it requires all the service which wouldn't be sustainable in a full production environment. So I need to to implement more, like the caching and stuff like that.

D

Okay,.

K

um Just a quick question to scott uh scott, where, where are the events finally resting? Is it so? Is there a messaging system behind the implementation of this poc.

G

No, no, no there, it's all! It's all just um fire and forget.

K

Okay, this is where I am. I wanted to reduce.

G

The number of dependencies on this, my implementation, the config, does ship it out to a k native service. My thinking is that I'm going to set the um the scale to min 1 and then you know um maybe make it live for like 15 minutes or something, and then when no one's touching the demo, it goes away and resets.

B

Okay, um what else do we need to talk about, then? Here I mean I assume the biggest hurdle for at least well, actually not a hurdle with skanoremi. It seems like you guys need to make yourself available on the internet, so people can talk to it. Yeah.

C

I'm not but.

B

um So if we wanted to register some so scott, let's say I wanted to take your services and register them into my discovery. Endpoint. Is that something that is possible, and we should do that way? I can add more stuff and test out to make sure your subscription manager can can be hooked up to my discovery, endpoint that I can then subscribe to without having to go through your discovery, endpoint yeah, I might have to add, like.

G

Some sort of helper page to help you register your your service into my discovery service. If you want to be aggregated into it, and that might be interesting.

D

On my side, like with the ui, I've shown you if I make it available online, and I just add you the plus, then you can do plus and then you create you, put the url of your service and then automatically will be available on the ui, and you can also click on the gateway and ask it to connect and do right. So it's really just adding a button on my side, and I can put it if I put it online, then we can all play around with it. It's just.

D

uh There is no authentication and we will all share the same ui because under the hood, it's the same service. Unless I manage session on it.

G

Some sort of api.

D

That your ui could call yeah, that's already what I did no right but like on my ui. Basically, what I will say is: if I send let me if your service stays at like scott.com, then I will put scott.com into the ui. It will ask my api to do a discovery.com services from that you will get your all your services and when you click subscribe, you will use the subscription url to subscribe to it and then it will display automatically.

G

Yeah but you're not going to get my update events because you're you're being the aggregator for me and I what I want to show is: you can register random registry services.

D

But I will get the, why do you say I won't get uh because for now it's just a proxy. So every time I ask to the gateway, we'll ask you and you will just proxy the call.

G

Right, but uh so I need these downstream services or upstream services to be able to register into the my discovery endpoint, because it itself is an aggregate.

D

Well, like uh I can put the two other services like the ping service, and uh so we can all uh display our ping services and uh like for me it was a cat but uh yeah. I can continue with this one, and the thing is those services like a small discovery and an endpoint. They already have everything for me. They are ready to go those ones they manage subscriptions and uh and discovery.

D

So you can connect to them, so you could, on your side, add them to your discovery on your service. Once mine are online, you can probably try to reach to them and add them to yours.

G

Oh yeah, I see yes, I I need to implement that piece because it's it's dynamic, but only through movement.

D

Because and then what I was thinking is just to put the ui to be accessible for everyone. So this way you don't have to rebuild it. You can just use it because the ui is just uh if I reach, basically it proxy the discovery proxy also, so it allows you to in one ui, uh it's kind of a client for the discovery and and the subscription.

B

But remy you, um you just do a simple get to the discovery endpoint to grab list of services and to aggregate right.

D

Yeah when I aggregate it's, uh if I so the aggregation, so what I call the gateway we'll do the get on all the different services that he knows to get all the the source and displayed. It will call the slash service on each proxy and then aggregate them into one view.

B

Right and you, you periodically, do a reget to get the updated list right.

D

For now what I do like, I didn't cache it and it's basically, if the client, so the web browser is asking to discover, then it will redo the it will launch 10 requests in the background and answer with one okay. So I don't. There is no cache, so there is no interest of uh it's always up to date, because I'm always asking to the final service, which is discovery, is okay: okay, cool.

B

Okay, all right, um I think that's probably enough for today I think, once we get our various discovery endpoints up there, people can actually start hitting them and playing with them. um I'm sure we'll need to get back together and complain about each other's implementation choices.

D

For me, it's really more the subscription api, where I was uh my my two biggest uh I would say issue is the subscription api. We need to validate the new version that we probably both implement alike, but to make sure that we have exactly the same now, and the second is the sdk and the javascript sdk, because I think I should push back some of the modifications I did inside the sdk, but I would like to have more architectural discussion with the sdk people on that, because I don't want to change code without them.

D

Agreeing with the fears of things.

D

Okay,.

K

Let's do.

J

This question.

K

Sorry, do we have the receiver implementation in the sdk already for discovery and subscription api.

D

Oh, I had to build it yeah thing and I that's why I'd like to push it back. I had to appear on the sdk, the javascript sdk for the discovery service, but it's yeah and in fact, while doing the full implementation, like with the dependency with subscription api, I think there's something to discuss there.

K

Yeah I mean because I was wondering that once we are done with the pc, that's where we start creating tasks in the sdk, where we create the receiver implementations for all the relevant sdks right.

B

Wait wait, can you can I'm sorry you guys? Aren't you lost me when you say receiver, what do you mean by receiver.

K

So I mean basically, you would have something by the end of the project. We would have something like, let's say, a subscription api listener receiver or subscription api receiver. So you don't have to implement your own uh uh on receiver for the subscription api. It should have been offered by the sdk out of the box. Okay,.

B

Wait wait. I apologize I'm being really dense but when, when you say a receiver for the subscription api, do you mean an implementation of the subscription manager? Or do you mean a sync to get events.

K

uh

B

No basically.

K

So it's it's a registration endpoint where you simply start a server and then the user of the sdk registers their functions like what, if you receive a get request or if you receive uh get because it's a subscription or a create, subscription, request and stuff like that right and in that way. So basically we wanted to abstract some of these uh http protocol specific uh boilerplate into the sdk. That's what I'm, assuming that we would head on down the line. Okay,.

B

So you're talking about a subscription manager, sdk kind of a thing yeah.

K

Yeah, okay, so like something what we have a cloud event receiver as of today, we would have something similar for the subscription api, as well as for the discovery endpoint. So in that way, that we try to isolate as much boilerplate as possible into the sdk itself.

B

Understood.

K

Okay,.

D

Got it cool yeah? Another thing I did implement was just like a simple security check on my side like when I subscribed. I uh already had like a header with like a secret when I received the event I verified that that secret is, is the one I sent before like when I subscribe because yeah. I know that we try to keep the security aside the spec, but uh maybe it's because I work in security field.

B

But I think, but I think that'd be an interesting use case for that stuff, that we talked a little bit about on the phone call last thursday right, which is in the subscription api spec. It talks about certain headers that you can ask to be sent or echoed back to you. Yeah.

C

That's exactly what I use.

I

Yeah so I used to be a good test. Yeah.

B

Okay, um okay, is there anything else? You guys think we need to talk about um because it sounds like people just need to go out and finish up. Their implementations add their metadata for like a better phrase to this webpage or to this google doc, and then I think this week's call is a discovery call after the usual one, so we can continue the discussions there.

B

um Obviously, if we have any issues we want to bring up, we can use the slack channel yeah anything else. You guys want to talk about right now,.

D

No, I think it's good. If someone can review the api I have on the subscription api, it would be great.

K

Okay, very.

F

Nice.

K

You could drop the qr in the channel, that's the cloud channel that we can have a look.

G

How's your implementation going doug.

B

I have um I have a lot of the same thing you guys do. Obviously um I have a little test. Client allows me to do subscribes and stuff. I need the biggest thing for me. I think, is just to make it available on the internet, which I have not done yet um and.

L

I don't have filtering.

B

I need energy to add filtering support, but most of the other stuff, I think, is there. I just it's just not in a dimmable demo, mobile form.

K

Oh, I think I didn't realize you have to make it available over the internet.

B

Well, something has to be available right for us to be able to talk to you one way or another.

G

What was the change around uh the filters array and then the inner filter.

B

Oh okay, yeah here, let me see what you guys think, because I.

D

Yeah, that's why I didn't rush the filtering part because I was like, I think it's moving.

B

Yeah, so here.

G

So because I wonder.

J

If I.

G

Just ignored what's what's in the spec, but I don't think I did.

B

Okay, so to me, I think the discovery endpoint filters array should look like this, where it's an array of filters where each filter expression, you can specify a dialect and then the type and then the property you want to assert or compare and the value you want to compare.

G

This this is much much harder to implement in a language like go.

B

Is it because these properties are dynamic yeah?

B

How else could we do it to make it easier and go the way it was the way it was um hold on? Let's go back to the way it was so the way it was was this.

G

I see a difference. I think this example was wrong. It's supposed to be filter at the top, just with a no s right and then inside. That is an object that says dialect and then filters.

B

Yeah but the filters was still dynamic based upon the dialect.

G

Yeah, that's that's! Okay. I can understand how to marshal an object internally into it. If I can marshal the whole thing at once and based on violence.

B

Oh, are you saying, okay, let me back up you're. I think suggesting that I made life harder by moving dialogue inside of it yeah.

D

Because then, the class is not.

B

But see the problem that I had with it is that what that means is I can't do multiple. I can't do multiple dialects in one subscription. Yeah don't do that? Well, why? Why do you want them, because.

M

I think.

B

Because because if as a subscriber, I support multiple dialects, why should I not allow somebody to use them all? At the same time,.

D

Yeah, I do agree with you dude, I'm sorry for good, but.

B

I'm.

D

Sorry.

B

For go, that's the entire reason I moved it in there. Scott was because it made no sense to me to say if I support five of them, but you can only use one at a time that seems silly.

D

The only thing is just uh if we do the json schema. Basically, a filter is defined by a dialect and all the property depends on the dialect correct like because, if I create my own dialect, like let's say french like maybe it's not gonna be type property and value, but something completely different. Correct.

B

That was my assumption as well. Yes, yeah.

D

Okay, yeah so yeah I mean I'm aligned with you dude yeah. Also last week there was someone saying- and I I find it interesting- that it's really hard to fill down and to limit on the number of filters.

D

While filtering can be compute intensive and it was easier for him to put a limit on the number of subscription, and so he wanted to have only one filter capabilities or like to find a way to to avoid someone sending you like 100 filters inside one subscription.

G

Does it say if the filters are on and on it says an and.

B

I specifically says, and.

D

Yeah, if it's an end, uh that's kind of uh yeah.

B

I I moved the text around, but this is basically the text right in here.

D

Yeah, so I mean, if you put 100 conditions, it means at the end. You are not selecting anything well.

B

So I think it makes sense for maybe the the discovery endpoint metadata to say. If there's a limit to the number of filters, you can specify per subscription right that way. So that way a discovery, endpoint can say, hey this. This is this. Subscription manager only supports one filter per subscription yeah and leave it to the implementation yeah. I think you're right, it's better.

G

Does it say anything about the the filters, yet it doesn't look like it. What do you mean about the filters like what what dialects do the do which filter dialects? Does it support and things like that.

B

Well, according to what clemens wrote it says, you must support the basic one right, but how do you know if you can use other ones? So, there's? Okay, that's actually something else.

B

Here uh wait: where was it I added some wait did doing a different pr. I thought I added something, so someone could discover the list of dialects hold on well, maybe I put in the same pr oh yeah here I create. I create a subscription dialects, entry or field in the discovery thing.

G

I would just call it dialects.

B

uh We could I don't care about the name that much I was just trying to be consistent, but yeah. We call it dialects.

D

Actually that one is like we should have an object that is called subscription with url dialect and config idling. I would say, because the more we add it start becoming ugly yeah. We can definitely talk.

B

About.

A

That yeah.

G

I don't know big as well.

B

Setting in scott.

G

Yes, subscription url subscription config sufficient protocols, which is also a subscription.

D

But it didn't get grouped yeah exactly yeah, that's something yeah you're right. I forgot about that. But yeah the protocol is basically the subscription service we can say which protocol is.

B

Managed yeah.

D

We can definitely.

B

Switch all that up cleaning up is always good. Okay,.

D

um The spec version, also there is one thing, is: if we have 0.3 and 1.0. What does that mean? It's? When you're subscribed, you should ask which format you want. Yes, part of the subscription yeah. I think too yeah.

D

I think.

C

Yeah, I mean actually, we have a which I remember hold on.

D

So I have a hard stop in three minutes, but I think it's really super helpful for me at least to participate in those goals. Thank you.

B

Oh and I wish he had a sample subscribe message someplace, I don't think he does.

D

In the subscription, no there was a but like this one is completely it's not yeah.

B

Yeah, maybe I put it in the bottom.

B

Yeah, so here's what a subscribe looks like, I think.

D

Oh yeah, it looks something like that.

B

Yeah but notice there's no way that maybe I messed up but there's no um spec version in there.

D

No, you.

B

Know with.

D

It and we need to put it.

G

I might be okay, because the if the sender sends it should be doing the the options, negotiation and it shouldn't be in the subscription.

G

What do you mean by that.

G

uh When this, when the subscription service makes the initial post it should be asking, you know what respect version it wants, and so, if it's it tries to send version one and that doesn't work. It might fall back to something that is supported.

D

Yeah, that's right! For me, it was. It would be nice to be in the subscription. So this way I don't have to fall back, because if I generate one million even and I fall back every time on one million events, if I'm not smart, so it's.

G

What happens when you want to upgrade to the next version, but all your subscriptions are locked into the old one.

D

You put a subscription, you just update the subscription to say now. I want the v1 and on your endpoint. The sync is basically managing to the two version. Now.

D

Or you re-subscribe, depending on your case yeah, I think we're getting kicked out. Yeah.

B

Okay: okay, we'll talk, we'll talk! You guys uh I'll talk to you guys again on thursday or through the slack channel.

D

Okay, bye.

B

Everybody.

L

Hello, hello,.

N

Hi, this is for the service serverless worker.

L

Yes, exactly uh is it nicola coutu.

N

Sorry, ed: stop google, uh google home! Yes, my name is I'm mostly just interested to attend out of curiosity, so I'm going to be very.

L

Okay, great welcome to the call, and do you want to appear on the attendees list.

F

With.

L

uh Yeah, do you want.

F

To be associated.

L

With company it's free too so.

N

Okay, I'm doing this on a personal basis, unpaid time.

O

Yeah, I think, there's a quality.

M

In a lot.

O

Of countries, so I don't know how many people are actually going to join today, like I know in brazil, there's holidays, so maybe ricardo won't be here.

N

How many people usually attend the meeting.

O

um We have a pretty small community, we have about between five and maybe 10 people, typically in the call, but we just changed the weekly calls as well. So, okay.

N

I'm from montreal canada, oh very nice,.

L

Oops was a mute and we have falco hi fargo, hey guys.

L

So yeah this is the first time that we switched to our weekly calls, and two weeks ago we had an additional one. Last week was the regular bi-weekly and the cncf calendar was updated. I also sent out a note in our slack.

F

Channel.

L

And I sent an updated invite to the serverless working group email list- let's see if people already have it on their calendar, as we discussed last week.

L

But we also have a few low hanging fruits. I think for the.

L

Prs.

O

Yeah just bingo, so we have.

L

Sorry so we have david hi, david wow, lots of new names. It's great.

L

David, can you hear me.

L

Oh malik, hello malik, this is malik. Yes, I can hear you hi. Do you want to be associated with a company.

M

Yes, I from american express actually, yes sure.

C

Thank.

L

You.

L

I'm trying again david david: can you hear.

L

Me.

L

Hey and we have jorgen hi.

A

Hey guys.

L

Okay, then I think we can start, I'm not sure if jorgen do you know if k wanted to join today.

F

I will ping him and see.

L

Okay, but I think we can start so the whole call. Let's start with the oh, we have one more wow, hello uh karina. I think you were first uh no she's not connected yet maybe now karina. Can you hear me.

B

Hello, yes, I can hear you hi.

L

Hi is this your first time on serverless workflows.

B

Yes, correct uh tahumia invited me to the meetings, so thank you for having me.

L

Okay, and is it correct that you are with red hat.

E

Yes, correct.

L

Okay, thank you.

L

And we have funky e again hi welcome back hi, no.

L

All right trying david one more time david. Can you hear me.

L

Now? Okay, let's start so roll call we're through community questions. Are there any questions to the community to the.

L

Group, no okay. Now, let's get to our first topic, uh so we've been discussing a release plan for quite some time and we wanted to make at least one bump in the release, because a lot has happened and we haven't put out a new release and we wanted to get one before kubecon.

L

Now um we already wanted to freeze the current version and work on bug fixes last week, but um I think we we'll have to get to it now, because there are only two more weeks left to um kubecon and we initially we wanted to do 1.0, but a lot of issues have come up and there is a little bit of work going on.

L

That was also pushing out critical fixes that we wanted to do for the retries and the error handling.

L

So the idea was maybe to do a version that is not yet a 1.0, but something below the next logical one would be 0.2. But I also feel that too much has happened. We could go higher. I think. Last week we had a suggestion to use 0.9. I am really not.

L

We really just want to signal that we have a new version, uh so I'm opening the mic for suggestions.

P

So is the question just uh which version which number.

L

Yeah yeah, because we had a discussion. I don't want to put any.

Q

So something between 0.1 and less than 1.0 yeah. Okay, I I mean 0.2 sounds fine to me the signal new version, um but I also get the uh the part about showing that there's been a lot of improvement or a lot of new. A lot of change. um 0.5 might be a candidate.

L

That's funny. I just mentioned that same number to hear me on the chat on slack and tier me also made a suggestion that maybe we could do a release candidate personally, I'm in favor of really doing an actual release and then working towards 1.0 after kubecon.

Q

Same for me as well like an actual number more than a release: candidate, okay,.

L

So tear me up: um would you be okay with that point five? Does it sound good yeah, yeah.

O

If everybody's okay with that definitely and I can then start creating the branch and and and uh once we go through the prs, we can the ones mentioned here- we can just do a freeze until this work is done and then move on um also in the road map as well.

L

Perfect: um okay last chance for objections.

L

Okay, then point five: it is, um and also we've been postponing a demo on how to use open api function. Definitions for two weeks now and today would be a good time to have it. Tell me how long should we schedule for that is that 15 minutes so.

O

If I can have maybe about 20 last 20 minutes I'll be happy with that. Okay, great okay with you guys um for today's talk.

L

Okay, I have to stop sharing right. We've been there last week.

O

Oh, do you want me to like do it now or you want to go through the piece yeah.

L

Yeah yeah.

O

Oh.

L

Okay: okay, okay, let me just see.

O

If you're ready.

L

Or.

O

If.

L

If you need more time, then let me.

O

No, no, no, no! No! No! Okay, I'll share my screen and let me just open up some slides that I have um one. Second, I think I have it going on all right. It doesn't matter so I I share my screen share.

O

Alright, you guys see my screen somehow, yes, okay, great okay! um I I really would like for this to be some sort of interactive type of talk. I don't want to be a monologue and if I talk too much, which I sometimes tend to do, uh please stop me and tell me that, like that I talked too much but anyways. So uh what I wanted to show recently, basically for people that are new here, uh serverless workflow uh defines and I'll show this language. I mean the slide here.

O

What we show when we define serverless workflow, what it is. We really define a declarative and a domain-specific language, uh workflow language declarative is it's not represented in code or, if else, statements or low-level code blocks, but it is expressed in json-yama format and domain-specific language, as in our target domain, is workflows, orchestration of event-driven distributed services and the reason why I say that is when we see a look at workflows and what they do and using orchestrating uh things using workflows.

O

We really have to understand our domain, which is a lot of times governed by our architecture and things like that and also see how this language really fits uh to translate or to express uh the control flow logic and the whole workflow language in that particular domain. So recently, what we made a change is basically uh introduce that we're really uh specification in standards based for events.

O

uh We define them using the cloud event specifications and we also made a recent change for function, definitions or the definition that tell the runtime implementations how to execute or invoke distributed services during workflow execution, and we move that to offload. It are any sort of possibility to find some custom parameters, use some language that is not really appropriate to our domain or really be carry on some custom definitions.

O

The longer long run will just mean maintenance to the open api specification, which is 100 times better suited for defining invocation of restful services that we ever could do in our language uh on our own. So I wanted to show a little demo in this demo.

O

We will code a process together and I think I'll process the workflow together, and I thought this would uh start some discussion, material and also kind of let you guys see some things about the workflow other than just looking at uh the specification and reading the documents and staring at the examples.

O

So is this something I just wanna before it continues this something okay to do right now, or you guys find this boring and you don't want to look at it at all. uh Thank you speak now. I mean.

N

I I'm interested.

O

All right so just to show you guys uh this uh thing is okay, so what I'm going to describe to you guys is a use case all right, so the use case that we're looking at is in this particular demo: patient onboarding, for example, in a hospital system, so you're dealing with registering a new patient to assist and assigning doctor to a patient based on their conditions and also scheduling uh appointment for that particular patient with a doctor that was assigned to it.

O

It's a very small uk use case, of course, in real world scenarios you would have a lot more rules and and and the business logic would be more complex, but for this 20 minutes. This is good enough. uh Since we said that a service worker is really a domain specific language, we have to look at now. What is the architecture of our applications and see how workflows can fit into actually orchestrate to solve the business problem on patient or onboarding?

O

So in this slide, you will see that in the application that I will show you uh that will start soon is that we have three services running they can be completely distributed, but the way I have it, I have a lot of issues with internet, so I'm writing them locally right now, so it doesn't really matter. uh We have three patient services and we have to orchestrate them in order to uh solve our business problem on the bottom.

O

You will see that in a lot of cases we have different triggers or applications or services which need to invoke uh this orchestration process. They can be as we'll see also in the demo web uis, or they can be some particular devices uh they might be used in a hospital other web uis, where you enter a new patient information to trigger onboarding.

O

uh The way we're going to do it in this demo is we're going to through different uis or different devices, uh push cloud events to a message broker in this demo. I use mqtt uh mosquito, but you can use kafka whatever the heck. You want to really do now. Our workflow. The way we define it is can be either triggered and we will see as a service.

O

So it has a restful end point, but we really want to be defining our workflow language that we're going to listen to this new patient cloud event, which is going to trigger our workflow execution and then orchestrate the services that we have available in our system. So that's kind of like what it is and um I'm just gonna go. I don't know from yet I'm gonna go ahead and just start my application.

O

It doesn't have any workflows defined yet oh well um hold on. I forgot to do something real, quick.

I

One sec.

O

While this is running, I have this on a branch, uh we're gonna look at uh vs code now in vs code I have a simple maven project and this is currently what we're going to do is run it on uh quarkus.

O

You can also run it on spring boot or micro naught or whatever it's a java based runtime, currently that I've been working on to to add the the the implementation or the runtime implementation for our serverless workflow language, and one important thing when you start working in vs code is, I would urge you to go to the extensions and type in serverless workflow, and in this case you will see the first one is our extension.

O

This is the extension that we provide and it's in our github repository, and it gives you things like code hints code snippets, it lets you generate an svg diagram for both json or yaml uh formats, and things like that. So this is kind of a must to have, especially if you're, looking at uh coding in vs code. um So let me see if this is completed, real, quick yep. So now I can probably start my app sorry about that. I wasn't prepared really anyways. This should just take a sec.

O

All right so thank you. We started it all right. So what the first thing I want to show you is the swagger ui of our application and just like we showed on this particular slide. We have the three services we have them here.

O

The end point of our patient services, slash patients, the endpoint of our doctor services, doctors and the appointment service is under slash appointment. That's the end points of this particular services. I don't have any workflows, I don't have any orchestration. So that's what we're going to do together this new patient event service. It's just a little service that from web ui you can hit it, give it new patient information, it's going to create a cloud event and push it onto uh and mqtt so anyway.

O

So let's go back to our um jesus again, all right vs code- and this is the maven project, because this is a java runtime. If, if we ever end up having runtimes in different languages, this could be a different project structure but under source main resources.

O

uh We're going to create this svg real, quick, we're going to create a new workflow and let's call it on boarding uh workflow and we're going to use json now and let's go ahead and start now, when you start using uh this in vs code and get our extension, you'll see, you're gonna get code completions right and it's going to show us first, the top level uh parameters that we can use for our workflow. So let's give our workflow unique id.

O

Let's call it onboarding, we can give it a name, let's say onboarding workflow, and at this point we can start doing our control flow logic. So, let's take a look at that. The way we define control for logic and serverless vertical language is basically a number of states. Each state does particular control flow logic, assignment or piece that it's responsible for and then transitions between those states. So let's go ahead and define our states for our orchestration.

O

uh Let's give a state a name, so we have let's say, uh let's onboard state and each state has a type. Now you will see that currently, in our specification, we have nine different types and each one again irresponsible, reduce some piece of control for logic that that it's assigned to do since in our uh onboarding business requirement, we have to wait on an event or new patient event, we're going to use an event state.

O

So once you define a type, the extension is going to give you the parameters that are associated with that particular type, so you're not going to get parameters for different types of states. So in our case uh we are looking for a parameter called on events. This parameter really means that, okay, we define the events that we want to uh wait for and then the actions they're going to be executed once we receive this particular event.

O

So in this case we have uh an event traffic, let's call it new patient event all right and then what we only have one. Now you hear you can define multiple you can define, joins and, and things like that, whether multiple events can arrive together. If you need more than one, if you just want to act on one, uh the actions are associated with singular event, and this is what the case is. Then we define actions.

O

The actions in our case are really invocations of our distributed service or the three services that we have available in our demo. So for that, let's define our function reference and we're going to give it a reference name. So this is the first, so the first uh service that we want to invoke. We can give it a a name- let's say, add new patient to system, and if you see so far, this is all logical names.

O

These are domain specific description of our services that make sense to your domain uh at your company or your projects. Are you using? There is nothing special here so now we want to. I have two more and since for onboarding, I have to first add a patient and assign a doctor and then um schedule the next appointment.

O

We need to execute these functions in sequentially, but you can also in this action, say that you want to execute actions in parallel as well, so the next one we want to execute is, let's say, assign doctor to patient all right. So we got that and the third one we say we say schedule.

O

Patient appointment- and this is really it with one single state where we're able to describe all of the control for logic, we're waiting for the particular new patient event. If we want to execute the three or invoke the three services sequentially, the only other thing we have defined in our language is: we have to tell the workflow which state is the starting state in which state is the ending state as far as ending workflow execution. So we just since in this case we only have one state um oops uh we want to.

O

There are different ways of starting workflows. There's a scheduled way and jurgy knows a lot about that. He will help us with that, but in this case, since we're waiting, I went, we just have a default start and on the bottom, for example, we can define that this state also ends uh workflow execution and again, let me show you this too.

O

If you're interested, we can create an event when workflow execution ends, we can terminate all the threads like if you have some parallel state running branches, still in the background or different threads, everything will be killed or for us once we perform the three actions or invocation of our services. We just want to end the workflow execution and see with like 30 lines of yaml code. We've defined this or I mean json code and yaml will be less. The only other thing now is that we do have to find the control flow logic.

O

We have to give our runtimes a little bit more information about the event new patient event and about the services that we want to invoke. So for that what we have is for events. We have an events array, and this contains our event definition now right now we're defining him in line, but the language itself defines also that this can be a string type where you can offload. You can define your events and and function definitions and separate files and reference them here, so they can be reused between multiple workflows right.

O

So in this case, what we're going to do we're going to give our one event a name now this has to match uh the declarative or the domain specific name here, and at this point we use uh cloud events, context attributes. So we have a type attribute that has to match the type of the cloud events format of the event that comes in so let's say we call it new patient events, for example type, and we also want to have a source parameter.

O

This is the primary defines who produces this particular cloud event and one cool thing that you can do, especially when you're using message, brokers and stuff like that, and we have in our requirements uh uh we have described.

O

We want to be able to receive events from multiple sources, so, instead of defining this event five times each one for each different device that we have one thing that we can actually do is say new patient, which is our topic uh for our message: broker that will receive events, and we can just put a plus sign here, and this basically means that we want all the events on all their message broker topics that let's say we have the example of new patient, slash web new patient patient, slash device x, device y.

O

All of them is actually going to trigger our uh instance of our workflow execution. So that's really it we for our single event. This last thing we have to do is define our functions or the functions are tell the runtime a little bit more information on how to exit actually execute the three services that we want to invoke. So let's go ahead and define our first function.

O

You guys stop me with any questions. If you guys have any questions, don't you know don't I'll go off otherwise forever. So we have a first function definition and we talked about functions. Being a, we use the open ia api definition, so we have an operation the parameter here now. If I go back to my web page, you will see that things like quarkus and micronaut and swagger, and things like that, allow us to. We don't have to uh hard code, the open api definition.

O

Sorry, I have to move, um but it will be generated for us. So in a lot of these new types of architectures and and things like that, you don't have to deal really with the overhead of uh generating this open api stuff yourself. So what I have here is, I also have the same thing in our um project, but in json format, so I call it services.json.

O

So what we want to do is an operation parameter. Give a uri in this case it's a relative path to our open api definition. So it's uh api, services.json and then what we want to do is give it a unique identifier of the operation of this service that we want to invoke. So if we look at our open api definition, we see under slash patients, which is the first service.

O

We have a post operation here that we want to invoke and its operation id is called, add patience, so we're just going to where's my workflow all right here. We just want to give it that, and that's really enough information for the runtime to know exactly what needs to be done in order to execute this particular function. So the only other thing I'm gonna do is create two more one for this one and for this one it's assigned doctor to patients. So it's our doctor service. We look at the post method assigned doctor.

O

That's that and our third one uh is our scheduled patient appointment service, which has to deal with.

O

Where is our appointments? The post method is schedule, appointments all right, and this is it. This is our workflow definition. There is nothing more to it now. One cool thing with a serverless workflow vs code plugin, which you can do is we can see if we can actually visualize this particular workflow and right now what we have done is we have generated an svg diagram from our json definition. The same thing can be generated from yaml as well, and this looks very simple: it just uses a state uml diagram.

O

We can see, we have a single state and the border color you're, seeing the legend uh it matches the type and we see that our event state waits for a single event called new patient event, all right. So one thing I have to do now: real, quick, I'm sorry, since we added the workflow, I have to just restart uh my app in order for it to get picked up, and this really only takes a couple seconds. Any questions so far.

L

Yeah.

E

One thing I noticed.

L

Sorry, no, please go ahead.

E

Yes, I have a question um you apparently associating functions to uh to the different steps. I understand that now. Let's say that I'm I'm trying to improve my my workflow and I'm as a result, I'm developing a new function.

E

How would I uh and you know, obviously the function is already deployed and possibly running, and but I would like to test my functions so what? What would be the approach to do that? Do I need to yeah.

O

Well, there could be, in my opinion, multiple approaches uh you could have. Workflows are developed just like services, and I was going to show them as well. Workflows also have a version parameter, which I don't have here. You can version your processes test, different ones and the cool thing about workflows or right now is they're not separate as as your code. So, just like you would test out your different version of a microservice that you deploy in go or python or whatever the workflows are really the same.

O

The world of this monolith, type of workflow deployments where you click a button, and I don't know what happens- is over and I was going to show you guys actually that right now I redeployed the application, and if we look at the swagger ui right now, you will see we also have an onboarding service. And if you look at our workflow definition, this is the unique id of our workflow.

O

So with this modern type of workflow orchestration, is you really write your workflows within domain specific language rather than code, but you deploy them just like any microservice that anybody on your team would write and deploy so your workload itself, even though they define some sort of a part of a solution to a business problem, or uh it really then can be deployed as services and be used by other uh parts of your teams, then, let's say have onboarding is just a part of their business requirements.

O

Just like another service that you have deployed in your system, and I think that is really kind of like a very important feature right now, workflows that there is really no separation between what we call a micro service, written, let's say in code and deployed on a container or a cloud platform, or you know workflow at the end. You really don't know right.

E

Okay, let me a little bit more specific understand what you're saying and that makes sense. For example, there is a concept of a b testing when you deploy some new uh version of the new containers right. So the way this is done, I mean, after you need to play with the routing of the queries, uh possibly here writing of events. Again, I don't know exactly how that will be done.

E

Essentially, I would want to do some a b testing and I would say only my requests are my events, so I don't know how to do that either. Maybe I have to create a different ui. I don't know only my request should go to this particular workflow and the people that are currently using my infrastructure.

E

They still, they still should be using uh the workflow that that is in place now right, but only for the new workflow that I'm trying to develop myself because I'm the developers uh I want the uh the routing to be slightly different. Is this something that you can address at your level? It can be addressed, possibly at you know at other levels, but definitely.

O

One way that we're addressing this in this particular java based runtime, is via versions. So, if you, if I added a version here, oops sorry, I'm typing standing up right now: um conversion oops and you can write- let's say 1.0 this will be deployed under the endpoint, would result in one one underscore o slash on boarding, so you can have multiple versions of your workflow running it.

O

Also you can switch out for the language definition, the function and event definitions, so you can keep your control flow logic the same, but here instead of having events, one thing you can do is say events and you can actually say source main resources, abc.json or yaml. So you can even switch those out. One can be the events for your deployment system production.

O

What some can be also for your development or testing uh environments, and, of course, we are looking for contributions in this area as well, uh because you seem to be an expert, I think it would be really nice if you can look at what we have and see what kind of improvements we can make on the language level to make it easier for users to do exactly what you're describing it does that make sense yeah but anyways um any other questions.

M

Quick, this is molly here, so you said uh what is the request and responses for this this election, he understood like implicitly not. He was specific, explicitly.

O

I didn't understand.

M

The input input the input from event a to event b to event c.

O

Yeah yeah, that's a good question, and- and this is one cool thing about uh serverless workflow- is that you don't really have to. If you see in my function, definitions and also my event definition here, I don't really have any parameters defined right and this is because by default each state holds its data. So what happens? Actually here is when we receive the new patient event.

O

The payload of this new patient event is going to become the state data now by default, through the specification.

O

If you execute functions, you pass the entire state data and this is what our workflow is actually doing. However, when you have a function ref, you can define parameters explicitly so here, if your function- let's say you know from your open api definition takes in two or three or five parameters. You can. Let's say we. Could you also define it like this right and then you can give an expression which is a json path? Expression.

O

uh The query is your state data or in this case the payload of the new patient event, which became the state data and, I can just say, for example, dollar sign dash patient, so you can use expressions to find define parameters as well. So what happens after each action execution?

O

The results of this execution of a service will be then merged with the state data and then the next function uh execution is going to either receive the entire state data as as as payload or it will. You can again define your parameters yourself, but because the for this demo, each service that I've defined can just take the entire state data as it comes from from the new patient event, I don't have to define anything so yeah yeah. Does that help.

M

Oh yeah, that's good! Thank you.

O

That's a good question, so just just to end this demo, I'm sorry, I'm probably taking way too long. uh Now, let's actually see if we can run this demo uh here I have a very simple ui. It just has a form. It's where you enter in. Let's say somebody working at a hospital in your patient. We can say his name is, let's say michael, and this guy unfortunately has breathing problems. Now, when I click on here, we're going to hit create a cloud event.

O

The cloud event is going to be set on our new patient, slash from web topic mqtt, and it's going to trigger workflow instance, and let's see if this actually works whoops. Why did I get two? I don't know right now, but it doesn't matter so we'll see we on board and michael. He has breeding problems. I think I clicked it twice sorry, and so this is our first execution of our first service or patient service. We our workload, the store, the patient or system this guy, this, oh, the appointment service, didn't really work.

O

Just I don't know what's going on right now, but anyways and we didn't really get a call from the third service. But I don't know this worked last night and I'm getting some.

A

uh I think.

O

Also, your.

A

Workflow definition, the function reference should be, there's a table here there in the function. Reference see the first one is assigned doctor to patient, but there's.

A

A is that correct, which one yeah the line line 43 a signed doctor, but uh the operation is a patient. Is that right.

O

Oh no, this just matches this guy right here uh they seem to be out of order yeah. It doesn't me doesn't matter, but this actually worked. I can just check it. The assign this is bad, now schedule appointment, let me just see, add patient or is it add, patience.

A

Should the add a new patient to system the operation is.

F

Yeah the functions and the operations are mixed up. I think which line uh that should be a sign doctor and line 48. Oh.

O

Sorry, that's why our third one wasn't called yes, thank you guys so much I apologize, so this is correct now yeah! Thank you guys, sorry, so, let's just go ahead and restart this guy and actually run the demo correctly. Thank you guys.

M

Are restoring these data anywhere in the database.

O

uh Right now, no, but you can for the simple demo and just have uh in memory. Okay in memory, I have an application scope class that stores all this stuff in there. So yeah just a demo. So let's go ahead and do this again.

O

So, let's michael and he is breathing problems woohoo there we go so we successfully executed our three services in sequential order, as per our workflow control flow logic, and the only last thing I want to show is the other devices. So here I have a mosquito I'm going to publish directly to mosquito topic new patients from client, and we have a patient of name, new patient from client to make sure it comes from here and here's the condition of irregular heartbeat.

O

So when I send this directly, this will be converted to a cloud event and should trigger also our workflow instance. So, let's see, if you actually did it, if I refresh this page and it seem it didn't, oh, I have something wrong: didn't even send it all right. Let's try here with the yeah.

F

There.

O

You go so now. If we send it all right here we go so we triggered our uh new workflow instance. We had this event directly that was pushed into the mqtt topic, and here is our new patient. He was assigned to some general physician in the scheduling service scheduled him for the next appointment.

L

So yeah.

O

One question.

L

About the um so, you copied this swagger ui into your local pass and you're referring to that file locally, but typically the open api spec would be hosted right, yeah yeah. So when you have subsystems uh exposing api functions, how? How typically is it to have um this rebecca ui definition with with a hosted service? You mention what do you? What are you using to generate that information internally.

O

um Okay, sure, so what I showed you guys earlier, um we have uh the new kind of like things like quarkus and spring boot, and things like that once you deploy your services, a lot of these things generate the swagger definition or the open api definition for you. So not only can you generate you, the open api or the swagger ui, so you can visually see. What's going on, you can also hit an end point called open api and you see it just downloaded a file for me, and here is it is so.

O

This is the same thing that I have locally in my project, uh but this is a yamo, so in my workflow I could have.

O

Referenced.

O

Instead of saying api services.json, I could have actually had a full path to my open api endpoint. You see so at that point the runtime implementation would get the ammo open api definition, parse it and know how to execute or invoke this particular uh and particular operation on the service.

O

Does that help.

L

Yeah, I think that was also um the the reason why we wanted to uh also have this demo right. You should motivate um that the use of open api is not that difficult.

L

um I don't know, do you think we should reference or endorse any of the libraries tools, or is this a swagger io library that you're using.

O

Yeah open api has a bunch of different tools for many different languages, uh since I have this java project, I'm just using the built-in stuff that quarkus has for me, I'm not adding anything special, so I'm just using the quarkus uh open api um extensions and basically everything built in spring boot really has the same stuff and I'm sure, if you're, not even in the java space but you're, using like a node.js or or or go or whatever open api has a lot of stuff.

O

For that, too, um I'm not familiar with the tools and everything on every language, but um I've kind of like been around like java for too long to to to do a demo in something else. At this point.

L

Okay, cool thanks so much for the demo. um Are there any other questions.

M

Yeah, that was a cool demo. Yes, sir,.

E

Yeah, I just have one question which maybe is beyond what you presented but, for example, for monitoring the uh the workflow, the workflow engine. Right, for example, I want to know where my events are in which they are, and I've possibly have some matrix about. You know how much time it takes to go from state one to step two to state three.

E

Do you have tools that you recommend to do that.

O

No, but one thing that the serverless workflow language defines is a set of extensions.

O

If you go to our github repository and go to the specification project, you will see a directory called extensions, and the idea is: is that the way we want to define those is uh as extensions to the to the to the language itself, and currently you will find an extension called kpi key performance indicators uh where you can enhance your workflow uh information with information about expected versus actual uh runtime results, such as cost and performance and times, and things like that.

O

We are also working on uh tracing extension. Currently, uh so you know we're very dependent on the community, you know so any help on any of that uh would be huge. You know from you guys if you want to get involved, just ping us on on on the slack channel, and I would love to work with anybody.

E

When you mean tracing you mean tracing, uh you know like uh jager type of tracing or are you talking.

K

About.

E

Okay, um okay, because here we are talking about events right monitoring events, I mean that's what that that was what my question was about. Tracing is one problem in itself and uh some people just solved it, but maybe you know there's still a lot of work to be done. I agree with that here. I'm talking about. I want to have visualization of my events and see you know what's happening in my workflow, specifically on the workflow right.

O

Yeah, but I mean I'm sure that if we define an extension, that is, uh that is very kind of like defines a general structure of of defining like what you want we're looking for and what your expected values are. uh I assume that you can use it with with the with the services that you have available, let's say on kubernetes or your or your cloud platforms that actually uh allow you to to to obtain this information during runtime and and we're basically trying to work with them as well.

O

We can't define a service on our own. We just deal with a workflow language, but the set of extensions that we write should be able to be used in multiple container or or or environments that you're running this in.

L

Thanks again to me, I'm sorry in the interest of time I'd like to move on. If there are any more questions, please just contact, hear me on the slack channel or open an issue if it's for the spec and so on. Okay, um let me share my screen and get to the pr. So we have uh two very easy pr's. I believe um one is fixing an image where the output of the let me show it.

L

Sorry, I think the output of a workflow example was listing a single layer, a single um value in a single map instead of an array of maps, because I think the input was three. uh Is it yeah?

L

I think this is better okay, so input for this was an array of maps and the output then said it's just a single map, but it should have been an array with just one map inside yeah, okay, and I think to me- you fixed it and, and the pr that is listed here right, state data filter examples.

L

Any objections to merging it.

L

So approved and the next one is an ambiguous, json pass example, and let me check so this one was uh doing an evaluation to, I think, derive the is adult property and then do a transition based on that.

L

And the let me show the fixed, I think if I should change this, it's easier.

L

um

L

Okay, yeah.

L

So, instead of this, it is changing to the condition that is correctly uh selecting elements and then, when that good condition is not null, so anything would render the as a result of this query would render the condition true transition is fired as better than this one and yeah okay.

L

I hope everybody had enough time to review it. uh It was on for a while any objections to merging this.

L

Also approved the third one is a little bit yeah, so I would have had a question to lucas who is not on the call today. um This is about uh the conditions at the end of a state, uh the transitions and the conditions for it. Oh, which one was it. Do you mean? uh Was it the switch state or was it conditions.

O

Yeah.

O

Or event based conditions, so the the issue that he was fixed here was that we didn't have in our language and ability to define uh what to do or which transition or which condition to take uh on a switch or, if you want to think a gateway.

O

When there's two conditions that evaluate to true two or more so what we added is two ways number one is the uh order based decision. So if you just have a number of conditions in order, uh the the priority would be the aligned with the ordering of the way you define the conditions, and we also added the ability here via priority, parameter to declaratively uh define a priority specific.

O

This same thing like what bpmn is doing really, um but to define a declarative order on each condition in order to to to uh define what to do in cases when there is multiple conditions that are.

L

So my argument last week was that we use order in uh many points in specification. For example, the actions list is an ordered list and this would not only apply to the switched state, I believe, but also than to transitions, and uh I I can't really find the use case, because, if we use priority, numbers we'd also have to clarify um what happens if I have the same priority assigned to two transitions and they both fire.

L

um This again would resolve to order. So I'm not sure why we would want priority to begin with, but I'll ask that um to luca and.

O

Yeah, let's.

L

Play offline.

O

And move it to next week, if you want.

L

Okay, then maybe we can cover one or two issues, um especially those two. They are both all the 125 and 135, because I think they're both going in the same direction. So the first one is from tme to update the error. Handling section, I think error handling uh includes also uh what happens if call failed and whether this has to be retried like a function.

L

Call I meant uh like an action, and um there was also a suggestion to move this. The retry definition, which is currently described in as part of the event state to a general section on re-twice, and I think both are targeting more or less the same. That is tidying up. The error, handling and introducing a new um section void. Is that correct, you're going um to me.

O

Yeah and it's my fault- I've been on this for too long and I'll try to get it this week done before we cut the release, but yeah jurgen has added a bunch of cool information and and input to this and and the retry section right now, it's it's kind of like exactly the same as what, for example, aws provides. There is no more or less uh that we do at this point, but with jurgen's input it has to change.

O

We have to redefine like how we actually handle errors and retries and especially associated with different timeouts in our language. I think, currently, it's not very well. It's ambiguous. It's some cases, so it's not a small task. So if anybody else wants to help with that, I'm all yours.

L

uh So.

O

Tell.

L

Me do you think we should do this before branching out or cherry pick from the dev branch, then for the version 0.5, because I think this is something we would want in version 0.5 right.

O

I don't know, I think I think just um just.

L

Talking about these.

O

Yeah, I think the pr will be so big that it might take weeks for us to make a decision on it, but I think 0.5 could be going out with what we have currently, and this will be a change for maybe the next either one o or whatever we decide. The next release would be just my opinion, but again whatever we just because.

L

Another thing: are they retries, so maximum attempts interval attributes and the exponential back off, uh which I believe are very good suggestions, and we just need to work on pr's for that. But it would be good to branch out. um Do the reordering first and then add it um to a changed version of the spec description.

F

It definitely seems like the we don't want to be having people modifying retry stuff, while the document's being reorganized yeah, let's either fix the the second two on retries or or do the updates to the organization of the document, but one needs to happen before the other and the the that second one, the proposal to remove the retry definition.

F

um I just think it's confusing where it's at so I don't know if that should be a required require something required to fix before we cut a release, um whereas I I feel like the retries for max attempts and exponential back off. Those are something that would be nice to be in. If, if we were picking stuff to go into the into the next cut or not.

L

But we also added quickly these um the uh the jitter to uh intervals right. So I think introducing those max attempts and interval attributes would also be an easy pr right.

L

I just don't want people to start work and then uh get conflicting changes that have to be sorted out in lengthy sessions, so I just want to make sure if we maybe want to reorganize to me it sounded as if the the retry definition in the event state really just has to be moved to a workflow error handling section so to have it as a general section and then later we can work on other error handling.

L

Apart from the retry stuff.

L

But um let me see if I let's take this offline um and we definitely do the 0.5 branch and freeze and because this also it's just reorganization in my opinion, so it's really sort of a um bug fixing for understanding, so people can readability right and then the retries maxitam's interlight will be the exponential bank of would-be additional features.

L

uh So then do this: let's do this shortly after.

F

That makes sense: okay,.

O

Manuel I just before we end I just wanted to since karina is still here. I just wanted to kind of introduce you guys. Karina is from red hat and, and one thing that we really need is like a lot of help on the community side of things for a project and she's. Like a pro on that, so I would like to just say.

L

Yeah come on you.

O

Know you are, and just a little you know if we have ideas and I'm sure karina will have how to improve uh just the overall community section uh and how we we interact with with the community try to get more exposure. She can help us there a lot so just letting you guys know.

B

Yeah exactly thank you for the intro and got me in guys and thanks for the demo. Today too, it was great.

L

That's awesome. Welcome aboard it's great to have you, okay, and um I think, for the remaining uniqueness constraint is also something that we have targeted with the correlation token. But uh you made some good points here and the action and events we have. We had an outcome that uh we would want two separate prs out of this right and then the research to add support to jsonpatch schema is something that ricardo who is not with today is um working on. So this is definitely ongoing, and this completes my list sorry to rush through these.

L

uh Let's definitely follow up on this and we're out of time anyways um any other business. Anything pressing.

L

No okay and then let me do a final roll call for the late commerce uh emmanuel. um Do you want to be associated with any company.

E

I am actually surprisingly working at american express- and I see malik malik- is here- I'm at american express yes.

L

Okay, great welcome and david. um Can you hear me now.

J

I can.

L

Okay, great great, I I tried at the beginning of the call you're with adobe right, you're, fine.

R

Yeah I just switched to my phone, so I was having some audio problems at the beginning.

L

Okay, perfect now just making sure you're here all right and subban hey hey! Do you want to be associated with any company.

J

Yeah um chegg c-h-e-g-g.

P

Like that.

M

Yeah.

L

Okay and thank you, everybody and hear you next week.

L

I think we also have a zuban here right, yeah yeah, oh.

O

Sorry.

L

I I missed you buddy.

O

Have I missed anybody.

L

No, I think, that's complete. Yes,.

E

Can you share the link to this document so in the chat channel or something like that.

L

It's um yeah, it's also in the community calendar of the cncf and in the meeting I invite I shared today. Okay,.

F

And.

L

On slack, but here there you go. Oh tell me you bit, oh no. I sent it to you gosh here you go yeah. Thank you. Link.

E

It's in a meeting and right otherwise, okay, I have it as well, but yeah. Thank you following.

L

Cool then thanks: okay,.

N

Thank you. Everyone bye, bye, have a great.

H

Day.

H

You.
youtube image
From YouTube: CNCF Serverless Working Group 2020-11-02

Description

CNCF Serverless Working Group 2020-11-02