AsyncAPI AsyncAPI Conference 2021, 17 Nov 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Building and managing an extensive API for Robotics and IoT - W.Vanheste & D.Hubrecht, Zora Robotics

Description

AsyncAPI Conference 2021 - Day 2
17th November 2021

This session will describe how we evolved into using AsyncAPI in a Kotlin code-driven approach, how we manage the scale of our API, how we use it in our documentation portal and no-code visual programming environment in ZBOS Control.
Zora Robotics specializes in robotics software. With the universal robot operating system, ZBOS, Zora Robotics is the worldwide leader in consumer targeted robot software. Our extensive API is based on MQTT.

A

Hey there, uh let me start with saying what is zorobots: zorobots is a robotics company, it's short for zora robotics, it was founded in 2011 and the focus is on making robotics accessible to all.

A

It started really simple: with a tv remote controlling the now robot, the one you see on this slide and it evolved into a crazy operating system called zvos. Now what is said boss, that boss is the zerabot's operating system, we started working on it back in 2017 and uh well. It was based on on all the prior work, of course, but it was a completely new system meant to be universal cross-platform and well you can.

A

You can compare it to like windows for robotics, basically, um so there's an android java virtual machine and a web version via kotlin multi-platform.

A

This is, uh these are most of our robots that we support. uh You have cruiser here, pepper james, the small one is robin then now I actually saw the first slide. Billy billy, which is an interactive flower pot. uh Aimbots, helps with the pandemic, and sawyer is a big, robotic arm.

A

We also support virtually any android device and we have a version running on the web. With virtual robots, you see um the virtual zora there in a virtual home we'll have a control application which works on your phone or on the web. It allows you to control your robots, the virtual robot or an actual robot.

A

This is the website if you want to check it out and everything runs on a single api. That's what we're here for today. So our api is based on mqtt, there's a broker on the robot and the broker on the cloud.

A

It's rather huge, considering it's 600 channels across 26 categories, and I hear a lot of people in the essec api development are using it for benchmarking things, because it's so huge. So that's quite funny. You can see our documentation on the link below.

B

So uh essentially, um we have two brokers: running uh mqtt brokers, one on the robots which you can see on the left here and the one in the in the clouds and these two brokers, uh sync each other- and we mainly made one in the cloud. So we could remotely access the robots to our control app um since we access and control the robots through an mqtt api. That also means that third parties can integrate to it.

B

We, you can either run an app on the robot itself and connect to mqtt or you could even have an external app running somewhere, for example, on an iot device, something maybe that monitors the air quality and then reports to the robot or stuff like that.

B

um These are some examples um or api of our topics. uh For example, um what we do a lot is we have a published topic to request something uh in this case. We do battery cuts because we want to get the current battery state and then below it. You see the response.

B

We always work with passing a key in the payload and then the response contains that key. So you know that the response is for you, since it's a broadcast system, you have to do something like that and the payload is mostly are jsons.

B

For example, in this case you get your battery level and percentage. You know if it's charging or not- and you know if it's docked or not, um because we don't want to pull to uh for uh the battery states. You can also listen to the battery event topic, which does the same as a response but publishes its once in a while every time it changes.

B

um So that's just one example: we have a lot of topics. You can control movements, you can play audio or a set of volume levels. You can open a camera stream. Take a picture turn on the lights, change, the core of the lights. If the robot support it, I can play media play audio or show videos on him or images on the screen. If robot has a screen, can let robot say something to text to speech which we support for multiple languages?

B

You can listen to sensors, some robots have uh like bumpers or buttons, uh which can listen to uh and also motion, detection and phase recognition to the builds and cameras uh or listen to voice commands. If a person says something to robots, you can and then do something.

B

um So we also support automations. We have to our control. We have a composer uh system where you can essentially play place, blocks, uh think uh scratch or note red but specifically made for zed balls, um and you also have a scheduler, so you can start automations on specific uh time points.

B

uh We also integrate uh domotics through home assistance, uh so home assistant publishes data to uh our robots and you can uh act on it. um We also support uh third party apps. They can integrate into uh said bowls uh by, for example, adding themselves as an app. um They can register send cells as an app. So the user can see the app visible on the when he fetches all the apps and control.

B

You can add settings which the users can then change. Custom translations, um you can add monitoring if you have a monitoring device, for example co2 meter or something like that,.

B

uh Then, um as we have an extensive api, we surely had to document it in the early days we just updated a page on confluence where we had a list of all topics and a description of them. What payload had to be sent in it? Of course, it was hard to maintain it had to be updated. Every time you changed something in codes. It was always out of sync and and well it contradicted uh the actual uh code.

B

um Also, it wasn't uh easy to uh for third parties to read out the api, um so we started using a sync api. um So uh at first we just made an uh manually. We made a yaml file uh where we put in all the topics and payloads and stuff. It was quite easy to maintain, but all developers had to learn the sync api syntax because they had to update the documentation every time you change something and the code, and it happened that something was forgotten and it was still out of sync.

B

So what we essentially did was uh we made multiple yaml files files per category, and then we used um the yaml ink tool to uh sync those files or merge those files into one bitcoin which we then published.

B

um So we wanted to make that progress better. We wanted to.

B

Automatically generate the documentation, so we went one step further and we let the async api file generate from code, so it was at that point. It was easy to maintain because we didn't have to manually update it. uh The code and documentation was always in sync, and we uh because we generated the code, uh the file from code. We didn't need to make any uh code changes.

B

um We didn't need to change our data classes or something like that um um downside as we need a lot of annotations in our code because we had to say: okay. This topic has this description, so we had to annotate that and we had to add some custom codes uh to make this all work um and it doesn't always form nice as docs. Sometimes you can make a more proper dock when you manually make it, but we found it wasn't that big of an issue.

B

So uh why did we decide to essentially uh work on a code driven uh way? So in a perfect world we could have just made an sync api and then let the or kotlin codes generate from that. For example, we could have defined a battery event and a sync api, and we could then export it to a kotlin class and it would look something like this. For example, we would have our battery event with a level uh if it's charging or not, if it's stopped or not.

B

Of course, that's not how the real world world works, I'm afraid, because yeah our data classes, aren't that easy. We had, for example, to add some annotations. We had to add today at serializable annotation for our json parser. We had to add the js gs export annotation, because we wanted to make the data class available from javascript for our content, multi-platform uh integration, um some other problems.

B

Again we had some annotations, like the serial name, um json parser, which says scotland, not platform, uh doesn't automatically uh convert uh camel case to snake case, so we had to add serial names. uh Sometimes we had um our functions in the doctor class itself, for example this one, if it if the request is valid or not,.

B

Or, for example, these ones where we made some helper functions, if you get a jar and it's no new and then instead return zero, so we could work around it, but that would mean we had to do a lot of code changes some classes, we, I don't think we would be able to work rounds. For example, we have an enum here um which we give a parameter to.

B

We have a custom serializer, which is also added to an annotation and because some bug and kotlin we also had to add a companion object to make that serializer work. So this would be hard to generate from a sync api spec file.

B

um So that's why we chose to generate the sync.

A

B

File from the code, so um how do we do it?

B

We essentially have a kotlin mqtt library, um where all our mpt topics and other classes are defined and we made a generator a sync api, generator, app or library. If you will that's integrates that and then generates a sync api json file from it. um For that we made use of uh the jsync api um library link, you can find there.

B

How does that work? Well, essentially, you can define in the sync api and encode instead how you would normally do in a yamal or json file, so, um for example, below in the codes, we would create um an example npt api class and then we would serialize it to json and then written to a file. um So how does a an mpt api or an sync api class? Look like so here on the right you can see.

B

um We have a class that extends the sa async api class, which comes from the library and and there we have to overwrite some of some implement some functions.

B

um For example, the get info function um requires you to return an input object which you can find on the left side of the screen, which you just fill in with some meta data. You normally find in a sync api file, for example the title: the description of the api uh contact information, the version stuff like that, um the same is essentially done for other things, like servers or all available tags.

B

However, it's the components, parts that we wanted to sorry, the channels parts that we wanted to automate.

B

A

B

uh For uh we made separate classes where we grouped some of our uh mqtt topics, um for example, we have here the battery channels class uh and there we return um I'll, go to the codes uh next few slides, but we generate some publish and subscribes uh topics and we pulse our mqtt topics which we already had in the in the library.

B

um So we create a publish here for battery cats, subscribe for better response and subscribe for battery events.

B

um So what is that constant? For example, we have the events, so the battery event I mentioned on previous slides is here: it's just a static string. The the static string is used uh and multiple over projects where we pull in that scotland library and we then consume those constants to do our subscribes and publishes.

B

But we had to add extra information, of course, to make um to make sure we could uh generate a sync api docs from it. So we opted to add annotations to each uh constant.

B

um So, for example, we would have the at the topic summary uh annotation, which would then add the the summary uh and the I think api docs the description and then the data where you would pulse a class uh and um uh the class as uh the payload. He would send. uh Typically for that topic.

B

So in the in the class itself, we also added some um some, um for example, if you wanted to add a description to a property, uh you could add by using a property description, or we also made some small helper annotations, for example, the property schema, which you could then add. Okay, this is percentage, the level of a battery is percentage and it would then automatically generate like the minimum s0. The maximum is 100. Add a description to it. Stuff like that, so you didn't have to add it.

B

Every time you had a percentage like property and a class.

B

um So how do we process those annotations, um as uh we use reflection for that, um so we uh in kotlin you can get uh the properties of a class um or of fields, and then you get uh you can get annotation from it and can and then get the data in that annotation.

B

These are three examples, the summary description and the data class.

B

um Now these methods that I just mentioned, you can then call um so uh we will uh by using the jsync api library, we would build an operation and then adds the summary description, a message to it and we just get the information from the from the annotation from it um the message so the payloads required some extra codes because that's more complex.

B

And it's in simplified version looks something like this, so you would build a schema and in that schema you wanted to set the properties.

B

The properties are then retrieved from the primary constructor of a kotlin class, which is essentially the constructor.

B

The only constructor data class normally has in kotlin. um Then we would uh loop overall uh parameters in it uh in the constructor. We will check what type it is. uh We will check if it has uh annotations and and get those and fill the information in as well.

B

uh If we see that um the the type of that property is not a string or an integer or something like that, we would but it's an object. We would then recurse and get properties from that one. So we would have a nice three of uh in the schema um of course, um more had to be done, for example, the serial names we used for our serialization.

B

We have to use that one- those ones instead of the uh the camel cased names we used in the code itself. uh We also had some non-constructor fields and some places we had to add. We had to add some special handling for arrays and enums and objects. We also had to prevent infinite nesting.

B

um Sometimes a property in the class has the same clause as a clause itself, so it would just keep looping and have an infinite nested codes. So, of course, that's we had to break up and there are some other cases as well.

B

The only thing we still added manually its examples, we weren't able to add those through annotations, but we just added them and after we generated the sync api schema, we just added an example for, for example, battery events, then level and and all the properties filled in um so it would show up nicely as an example in the sync api documentation.

B

um The outputs from that library has done one big json file, essentially containing all topics, but we also opted to create a json file for each category separately. The reason we did that is because we want it in our web portal in the documentation portal. We wanted to split up everything, hence into separate categories and then load one of those json files, depending on the selected category, um showing one big uh page: wasn't: user friendly and not performant uh either. So we opted to do that.

B

um The big json file uh mainly exists for third parties who just want to integrate everything while they can use that file. um So it's also integrated into our ci cd. um Every master builds. We do. We run the the code to generate the documentation and it's then published to our public kit repo. So that means it's available immediately.

B

Every time we do a release and it's also tagged with the version that we released at that point. It's also integrated into our documentation portal and will give more information about that.

A

Thanks davey, so what we see here on this page is our documentation portal.

A

On the left hand, side, you see all the categories in our api on the right hand, side you see all the topics you can subscribe or publish on and, of course, the details in the center.

A

You can find this site on sorabot.e everything is built with antora fontura is open, source documentation, tool um and all documentation in it is written in ascii doc. Oskidok is like markdown or other markup languages, except that it's quite advanced to make structured documents. So it's perfect for manuals and technical writing.

A

What what it does basically is aggregate documentation from multiple sources as our documentation. Our asic api api is just published to a git repo as well.

A

We use the tools created by david jenks to convert it to ascii doc and then integrate it into our doc portal. It's really nice because it also generates the navigation and all the cross links between different topics. So it's a it's a really nice tool.

A

We also consume our own asic api in the said: boss, control application. So there's um an mqtt block like a building block in our um programming environment, where you can publish and subscribe to topics you can um connect to multiple mqtt brokers, if you like.

A

um So this is how it looks basically.

A

All the topics that are in our api can be searched easily. So, as you can see in this list, if you click the information icon, you get some more information about this topic. All that all those things are are rendered from the async api spec.

A

And then, if you select to use one of these things, it ends up in the block and you can change the payload if you like. In this case, you are checking the battery status of a robot.

A

As for future plans, uh we have we've been working a lot on a virtual environment and virtual robots the past year, and I would say that 95 of what we've been doing is not even live yet. So 2022 looks really nice when it comes to releases.

A

We want to further expand on our composer application, because there's a lot of opportunities in education, especially because everybody knows robots are expensive and if you can have a virtual robot that does all the same.

A

It's perfect for stem classes and stuff like that, so our focus will continue on virtual robotics as well as it's just all the same codes as for the real robots and part of that is building a community of people where they can share their own compositions as we call them, so they can share their own programs. They make with our software, improve upon it and another step there is to allow third-party integrators to easily interact with our system, make their own composer blocks, which are like the fundamental building blocks of our system.

A

So these are the things that in 2022 we are. We are really looking forward to. um If you um are interested in more of that, I set up a mailing list, you can, you can subscribe to it like we can give you a free trial to our virtual robot platform. The moment we launch it. So please do don't worry. We won't spam, you with anything else. This is basically I made this list just for this presentation.

A

If you like what you heard and you want to participate in our projects, please feel free to reach out on jobset storables dot. Be. Thank you very much for your attention. If there's any more questions me and davey will be there just after this thanks.