OpenJS Foundation OpenJS World 2021 - Automation, 2 Jun 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Internet of Things (IoT) with Node: Both Practical and Fun! - Jesse Gorzinski, IBM & Michael Dawson

Description

As we know, IoT devices are all around us. Don’t get left behind as companies around the world create competitive advantage by gathering valuable data through IoT connected devices! This module will show you what software you need and how to start building the programs that will allow you to gather and process data.We will show you some simple devices as well as sample programs that can be used to integrate those devices with business systems.

A

So welcome to openjs world and today jesse- and I are going to be talking to you about internet of things with node.js, both practical and fun. Before we get started a little bit about ourselves, I'm michael dawson, the node.js lead for ibm and red hat. What that means is I get to be an active community member, I'm a collaborator member of the technical steering committee and generally active across the project.

A

It also gets to me that mean it means that I get to work with great teams across red hat and ibm, for example, jesse in terms of our support for node.js on different platforms as well. As you know, all sorts of things in terms of making node.js first-class citizens in our offerings.

B

And thank you michael I'm jesse. I'm uh work with ibm as michael mentioned, I'm the business architect of open source on a platform known as ibm- I, uh which you may have not heard about before, but it's definitely a a platform with a strong im like strong footprint and it powered banks, insurance companies, all kinds of fortune 100 and fortune 500 companies as well. uh But in my role here at ibm, I get to set the strategy around open source technologies. For that ibm. I platform around things like open source languages and iot.

A

So, just before we get started a little bit of an idea of what we're going to go through, so we're going to start with a brief introduction to iot and mqtt. So what are they? Why are they important, we'll look at using mqtt with node.js, because they're a great fit we'll take a look at some real physical devices, we'll talk about the anatomy of a simple mqtt light and temperature sensor sensor that might fit one of your business use cases.

A

We'll then show you mqtt and devices in action uh in a demo. Part we'll, then touch on leveraging mqtt with the cloud and we'll finally, we'll finish up with a few thoughts on reactive applications and kafka.

A

So to start with what is the internet of things? Generally, it's what's used to refer to a network of physically connected devices. These devices provide data. Often you know a constant stream of things like temperature pressure and so forth. They can also often be controlled, though you can send a message back to the device and you could turn a light on or off uh open a door, those kinds of things, and if you want a more comprehensive description, you can go and check out the wikipedia listing for the internet of things.

A

Now, one of the most commonly used protocols with iot devices is mqtt. It's uh mq. It stands for mq telemetry transport was actually invited and invented by uh somebody at ibm quite a while ago, and it's really popular because it's really lightweight you can do a publish and subscribe.

A

um So it's it's a message based where you, you publish the topics and you can subscribe, subscribe to talk topics, but it's very small. It's got a small footprint, it requires low bandwidth, so you can get it into small devices and it's fairly easy to use and that's you know kind of why it's so popular again. You know. If you want to go back to a reference, you can look at mqtt.org in terms of the the formal definition there. The great thing is it's asynchronous, which is a really good fit for node.js.

A

As we know, so, let's dive into some of the specific terminology. So the first thing you need to know is the client and the broker. So clients are basically you know, devices applications running on san ibmi or some other platform who connects to a topic either to publish or subscribe messages.

A

There's some great clients out there so faux and mqtt js. So file is one which provides clients across a number of different platforms and languages. Mqtt js is a npm module, I'm written by one of the the members of the node ecosystem, and you know it's a great choice when we're using them. The broker is the piece that actually provides the the message. Cues and the persistence and sort of the back and often mqtt is provided as a protocol, along with a bunch of other uh protocols.

A

So later on, jesse's going to show us uh amq and amq supports a number of different protocols. Mqtt being one of them mosquito is a you know, a javascript, pure javascript broker that you can also use for experiment, experimentation as well. So that's the client in the broker topics are basically, you know an id that we subscribe to when we want to listen for messages or what we publish on when we want to send messages.

A

Messages themselves are free form, so they can basically be any any text. So it's up to you to figure out what that formatting is going to look like and then qos stands for. Quality of service and mqtt provides three different qualities of service, 0, 1 and 2, which we'll talk about in the subsequent slides.

A

So let's look at topics. Topics are uh you know one or more levels, separated by a topic center separator and in in this example? You know we have factory is one level floor is another level and machine is another level, and I can you know, ask when I subscribe. I can I can subscribe to a very specific level.

A

You know so I can subscribe to factory floor machine one in which case I'll just get uh messages for that one topic or I can subscribe using wildcards, which I show a little bit below, so that I can get like all messages for topics that are underneath factory floor. One, for example, there are a few restrictions.

A

um They must be at least one character, each each level and they are case sensitive. So it's good to be aware of wild cards, which are very useful when you're subscribing on topics are the plus which matches a single level. um So in in this case you know, factory plus slash plus slash machine. One will match factory floor, 1, machine 1, but not you know something where there's two intermediate levels.

A

Like the second example, the hash sign number sign matches multiple levels, but it's only allowed at the end, so it basically says I want to get messages for all topics below this route that I'm specifying and those end up being quite quite useful.

A

At very least, if you're trying to figure out what topics are available and messages are, are coming into you on quality of service. uh There's three levels: they're very simple: you know zero is at most once one is at least once and two is exactly once and if you think about simple devices at least often you know at most once is, is all you need? If I'm sending a you know a stream of temperature readings, it really doesn't matter.

A

If I me, if I miss, you know a message, because the next one will come along and you know I'll still have the data, but depending on your application, you can use um some of the other ones, which is like at least once you're guaranteed to have it delivered, although it may be repeated in that case, and then two is like its own you're only ever going to get it.

A

Of course, as you pick the higher levels there's going to be more overhead, and so you know, unless you really need to you know, zero is probably the way to go. The other thing is is that the qs is determined by the receiver. So even if you actually ask for a higher quality of service when you're sending you may not get that on. On the other end so now over to jesse to sort of give us a little bit of. Why are people interested in iot in our context,.

B

Yeah thanks michael and you know, obviously it's easy to see that iot continues to play a growing part in everybody's day-to-day lives right, I'm guessing each person attending today uses iot in some shape or fashion and probably with multiple devices right. We have smart watches, smart appliances, smart tvs, smart cars.

B

All of these things we call smart, but we have iot devices uh more and more prevalently available in our life, but from a business perspective, if you are a developer or a manager or an architect and you're trying to see you know, am I in an industry that cares about iot. The short answer is most likely. Yes, because iot technology now is really transcending across many different industries right, so we have ibm.

B

I clients, for instance, in the trucking industry, who are managing their fleet to make sure that things like the temperature of their meat when it's in shipment is good to keep the product safe and so on. You know we have clients in the medical industry and they're investing in smart, smart devices that are easier to maintain, have lower rates of failure, and things like that as well. So iot is uh very prevalent in many many industries these days.

B

A

Now, just oh yeah, there you go. We lost your video there for a second, so you're back.

B

Sorry, I'm back all right, fantastic, um so.

A

We do have in there.

B

Lost me again, all right all.

A

Right now now we see again all.

B

Right, fantastic, um so a simple publisher example that we have here is uh written with that mqtt js project. That michael mentioned, that is relatively easy to use relatively flexible and it's kind of the tool of choice for us because of those reasons right, um oh lost. My.

A

Video again we're getting a little bit of technical glitches, a.

B

Little bit of technical glitches, uh the iot technology works much better than.

A

B

A

B

Webcam technology, but um top to bottom. It's a pretty simple example in the sense that we just require some pretty common modules that you see with fs and path we require in this mqtt js module- and you know most of the code is actually setting up the ssl bit because we chose in our demos that we set up to encrypt the communications with tls, and so that requires us to set up the options with uh the client certificate and certificate authority, trust stores and things like that.

B

And so really you can see we're just setting that up in this mqtt options object, and then we call the mqtt.js library to connect to the broker of choice. In this particular case, it's an ibm. I system we use for demo and then once we're subs connected.

B

We can subscribe to a topic, and in this case we just spit that output out to the console just for quick demo, debug purposes.

A

Right and the previous one, the previous page, just showed you know actually publishing on the same topic.

B

Yep absolutely and the code is pretty much the same. It's just the only thing. That's different is: what are you going to do once you get connected, you register a callback for a message received versus um calling the readily made apis for doing that. Simple publish to that mqtt broker.

A

That's right: we just didn't want people, you know anybody being able to publish to our broker, so we added some security.

A

So now we can move on and actually look at some some devices, the the you know the the the iot devices you can use range from all sorts of different things and sizes. If you look at the the lower right, that's an esp-01, that's a very small, but it has built-in wireless and you know enough memory to be able to run a small program, and you know the one beside. It is a slightly larger one that already has like a usb port.

A

It's uh it's one that I commonly use because you can just plug it in and now you've got something. That's got some pins that you can use to turn things on and off take readings and then all the way up to something like on the top left, where we have a raspberry pi where you can run.

A

You know a real operating system and you know all sorts of things in in between I kind of like the the the socket one on the on the right, because that's one where actually, you can load in your own javascript code, connect to an it iot network and turn the light on and off and stuff like that, in addition to some of the ones that we've built here.

A

uh This is another one. I've worked on more recently. It shows that uh the wemos mini d1, where it's it's a a weight sensor, and so basically the the sensor that that reads, the the weight um you know connects up to that little device which can then connect and send over mqtt the uh the weight of whatever you put on top of it.

A

You might also ask well what about some existing devices so, instead of trying to to build my own, you know build my own there's a bunch of devices, but not all of them use mqtt. So quite common pattern is to use a mqtt bridge where you have that bridge, converting from whatever protocol the devices are using into the mqtt protocol.

A

In my example, there's a lot of devices out there that you can get that run on 433, megahertz and so like we have a door sensor, a motion sensor, actually a fire alarm that will go off and again send messages on 433.

A

The most right one is a is a temperature sensor and I have a bridge written in um in in node.js. Actually that can take the messages that come in through the fourth 433 megahertz, and you know we can convert them to uh to mqtt and then use those messages, and actually this one isn't written in node, but this one's uh you know built on it on esp8266. That can can do that conversion.

A

So now, let's look at what it would take to say, build your own device just so you can see that it's really not that complicated. What's going on in the devices, if you want to build something, that's purpose fit. um You know, in industrial cases, you'll more likely want to buy a commercial device for support and so forth. But you know if you want to start experimenting and sort of proof of concept what you might be able to do in your business. This is a good way to go starting get started.

A

So, let's imagine that, for example, your business grows plants, maybe things like temperature humidity and light intensity throughout the day might be interesting, and you know in this case this is something that I've set up. I'm growing a few plants in my basement.

A

You know mint and basil, some nice spices stuff, like that, and I ended up you know through through that device which you can see there. It's built on again one of those esps and it collects slight temperature and humidity and, like you know, through the mqtt network, I can fairly easily get that data out and then you know, I put together a grafana dashboard that consumes that data and shows me.

A

You know the trends of the humidity temperature and light um you could set up alerts to say: okay, wait, a sec, the lights went off when they're not supposed to, or if things are getting too hot, too cold and so forth.

A

This is the version of devices that we're actually going to show you in in the demo um a slightly earlier version of those I'm just trying to raise it up, so you can see it um and it builds in the dsp has a light which blinks every time it's sending the message and it collects light and temperature information.

A

This is the circuit diagram of what it looks like.

A

So we have that wemos, which has a nice usb port you just plug in it's got a light sensor, which is that gl device, it's basically just a resistor that changes resistance, based on the amount of light that there is it's connected to the analog pin on the the wemos through a you know, a resistor divider, so that you get a different value depending on how much light is being presented and then, on the right hand, side is a ds 18v20, which is a integrated temperature sensor and basically through sort of a one wire protocol.

A

You can talk to it, ask for the temperature and it gives that back to you and then you know for uh as shown when I'm showing the device there's also an led that we can turn on turn off and it's set up to turn on and off whenever it's sending you some data looking at. What's actually in the device itself, uh we saw uh topics. So when I introduced the concepts, uh topics were one of the the important topics and you can see there the late topic, the temp topic and the led topics.

A

So those are the topics that the device will publish on when there's a light. Reading a temperature reading or or you know, basically, you want to tell it to turn the light on or if the light changes on the right-hand side. We see some of the code and again just like the case that that jesse showed us earlier in terms of the node client. A lot of this is just boilerplate so that we can get our connection to the network and our certificates.

A

Again over here is still more sort of boilerplate. The nice thing that you'll notice here is like I didn't have to write the code to to query that that temperature sensor there's already a dallas temperature sensor, um sort of library that's out there, and so we can leverage those in in a lot of the things that we write.

A

These tend to be. um You know they tend to work very much like you would expect an asynchronous system and if you're familiar with node, um you know through callbacks. So in this case you know, part of the code is a callback that says when I get a message posted to a particular topic.

A

um This is the code that I want to run and in this case it says, like you know, when I get a message, I'm going to tell I'm going to check whether that's a message telling me to turn the light on or turn the right off and associated. You know. It then calls the function that will actually write the pin to make that change.

A

Now we start to get into the the code itself and you can see that I'm creating the pub sub client and I'm passing here that callback, which we saw on the previous page to say, okay, this is the callback when you get a message that I want you. I want you to run along with the connection info. That says, you know: what's my mqtt port and server string, the code itself here now is mostly around getting the certificates all set up and passing in a unique id.

A

The one of the key things is: every client really should have its own id. Otherwise I found that some of the brokers get confused. So in this case we use the the mac address as a unique id, because that's going to be unique for every day, every every device- and you can see here that uh when we create our client, so we you know, create a client we've created a client and within this loop we basically at the the first we check to say: are we connected to the the wi-fi?

A

If so, let's start you know if, if not, let's actually say, let's connect and here's where we're passing in that unique id and once we're connected, we subscribe to the topic in this case the led topic that we can send then inside that loop. The other thing we do is every so often so we've got our transmit interval seconds and delay. We want to take the temperature sensor, the temperature and light readings and publish them to a particular topic. So we saw those topics uh earlier: the temp topic and the light topic.

A

So this code, basically, you know, interacts with the library that I mentioned converts it to a string, because we said that the messages were just plain formatted text and then publishes to the temp topic and the light topic and then toggles the lead so that you get some feedback that is actually sending the data out.

A

Now I showed you c plus plus, if you don't like c plus, and you want to do it in javascript. I won't go through the same description, but you can do that through esprino and you can write in javascript. The code looks very similar. You have a similar loop and in fact I did that to push the same code into a demo type that device like this, and if you want to go check it out, you can go check out my github repo to take a little more look and more detailed.

A

Look through the code there. So now we're at demo time I'll hand it back over to jesse.

A

So jesse you still there.

B

Sorry about that, yes, I am here- um we are going to walk you through a really quick demo and then we'll show you the code behind it. uh What we're actually going to do is we're going to show uh consuming data from these light and temperature sensors that michael just talked about, and then we're going to actually store that information in a database to be used by other database applications so set up to get this.

B

To this point, I have active mq running on my system and happens to be an ibm I system, but that's not uh necessarily relevant. It can be anywhere because these are all very portable and we wrote a little bit of node code that I'll show you in a little what bit node.js code. Does it make sure that the database exists and you can see in this case?

B

It already has the schema and everything because we've run this before and now, because those devices are sleeping and sending data as temperature changes or light changes or every 10 seconds. You're going to see this light and temperature sensor data coming in on a periodic basis right, and so, when that light and temperature sensor comes in, we see the console output here and that information is also available in the database. And so I'm going to pop up an sql tool here.

B

Where I'm doing a simple select from this table that I created this iot records table for temperature and center data, and you can see there's a couple tabs that popped up down in the bottom, and you can actually see uh the historical time behind the the various sensor values for temperature and for light as well.

A

B

If you're interested that's.

A

That's the temperature in my uh my office and I can actually put my hand over the uh the sensors here. If we want to see the light, go down a fair amount, um but maybe we'll come back and take a look at that afterwards. If you want to go ahead and show the code yep.

B

If we have, if we have time to do that, we'll circle back and do that in a little bit, but the code behind it is going to be very simple and very familiar to the code. We showed you earlier in the presentation today um we're going to set up the connection to the broker in exactly the same way. We showed you the example uh client certificate set up and the ssl setup and michael. I need to stop sharing.

A

B

Know I did stop sharing, we need you. We need the slides back up right.

A

There we go sorry.

B

The only thing that's different is um you know when we're consuming data. We showed you this one before we're consuming the data, we stick it in console.log. You see it in the terminal, but we're going to take that a step further like I just showed.

B

So, if you go on to the next slide, we actually wrote some code to store it to the database and I'm not going to walk you through all the details here, we're using an ibm, I database connector creating a table if it doesn't exist and just doing some basic database setup there, and if we go to the next slide.

B

All we did is we modified our callback for when we receive a message to do a simple insert into statement with sql and, as you can see, it's just a few lines of code to get that level of integration, and so now any database application can interact with this iot data.

A

Yeah- and that's really, I think one of the reasons why it's so popular is that really very small amount of code, just a few steps, and now you've got data flowing from the devices all the way into your database, where you can, you know, take some more sophisticated action and stuff like that.

B

A

So you know, we've shown you actually connecting the devices directly to um the to a system which is running a broker. If you don't want to set up that that yourself um or you want to make it something, that's like can go across. uh You know it's not necessarily stuck within your organization. One good thing to do is leverage leverage, the cloud, so you could sort of do um what's shown in this first picture, which is to move some of the pieces from your.

A

You know internal system out into containers running in the cloud, and that's actually what I did for some of the applications I've been playing around with is. I ran an mqtt broker in the cloud and then a node application, and in this in this case it's an example where you know either through through a ui on a computer or on my phone. I can it can talk to that controller. That controller will convert.

A

You know whatever buttons, you press to say, turn on lights, turn off lights into an mqtt message, publish it to that broker, that's in the cloud and then device that device which is a plug.

A

You know plugged into my house, you know and is, is connected to a wi-fi that can get out to the outside world connects to the broker and you you know now, I'm controlling everything without having to be running a broker on my own premises, but even better uh there's a number of iot platforms that you can get from from organizations and they will provide you all those pieces so that, basically, you can just say, here's the devices.

A

I want to connect up to my my um my iot system, and you know both from the devices and from the clients. The thing I did want to mention here is that there's you know the one thing that's a little bit different.

A

Is they often have a few more constraints over things like the topic, so, for example, in this particular part of the code, I show that um you know you need to change the light topics, so they're structured in a way, for example. This is for the watson iot cloud in a way that that fits uh their their environment, because it's a multi-tenant environment. So, in this case you know, I need to say iot2 event.

A

Lite and they even sometimes add some extra features in this case- it's formatting, so I can tell it what format the message is coming in, and so you know I had to change the format of my my topics and I had to change uh the format of my messages the nice thing about. That, though, is that, then they let you very easily start to chart graph and work with that data, because you know having it being structured as json, for example, they can extract it and you can put it on charts and so forth.

A

So if you're you're getting into this, that's another good way to sort of have a quick bootstrap um to getting charts, graphs and data and all that kind of stuff.

B

All right and I'll quickly just mention a couple other tools that are available for doing iot programming. One of my favorites is node red, which actually allows a low code graphical type of designer that allows you to just define your application logic as a flow of data through all these various nodes, and I worked with a client once who actually set up a very large manufacturing integration with plc devices all linked up through node-red, and it is.

B

uh It works like a charm and it's very simple- to use very easy to prototype and deploy to production. So that's something definitely worth looking into um and another trend that we're seeing is this whole notion of reactive systems? And so, if you go out- and you read the reactive manifesto it comes in uh you know, various key attributes that are are defined as a reactive system right.

B

It needs to be responsive, elastic, resilient and message driven right and so we're seeing an uptick in interest in messaging technologies and kafka is quickly becoming the mo this. The standard mode of operation, the messaging component of choice, um and so as you're defined, defining your plan for iot. You need to also maybe take this into consideration as well, because your iot data might need to very, very well integrate with some kind of messaging system.

B

There's a product out there called apache camel, which is really great, as I say, for tying anything to anything, including tying iot devices and technologies to messaging systems as well so I'll hand it over to michael to wrap things up.

A

Yeah, I you know just to add on the reactive systems. I think you know the structure uh that we've seen in like the mqtt uh side of things. You know jesse and I you know joked that we we only use these devices every so often, but they keep working. I unplug them. I plug them back in, and so the reactive systems are sort of looking to leverage.

A

Some of that same same approach where the pieces are detached and decoupled um and sort of the message broker is, is the piece that lets you glue it all together. But it's it's a very resilient.

A

um You know approach and way that you can scale and and yeah we see that um kafka is, is quickly becoming the the choice there. So, when you're thinking about your your iot systems today, it would be good to think about like how are you going to bridge that data into maybe a bigger?

A

uh You know kafka backbone, that's serving your enterprise applications as well. I think that's all the time that we have today.

A

We we hope you you enjoyed this sort of world whirlwind introduction to what iot is um some example of what some of the you know, the the what it looks like in terms of of you know devices that you could build and experiment with how you use uh mqtt with node.js and how it's such a great fit, because you know uh you know, iot is all around being asynchronous and asynchronous events and node fits really nicely with that, um and then you know.

A

We hope that you see that, like it's really, you know, there's it's not that hard to get started to connect those devices in get the data into your database as just jesse showed, and then even you know, leverage the uh you know larger cloud-based, iot systems or you know, plugging into your more uh enterprise-oriented applications that might be using uh reactive approaches in kafka. um So thanks for watching- and we hope you enjoy the rest of the conference.