Cloud Native Computing Foundation KCD Indonesia 2021, 18 Dec 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Breakout Session: Understanding Node.JS Performance using OpenTelemetry - Doni Rubiagatra

Description

How can you tell your Node.JS cloud app is slow? broken? or want to improve your code quality? Observability is the answer!, especially using OpenTelemetry. OpenTelemetry lets us generate, collect, and export telemetry data (metrics, logs, and traces) for analysis in order to understand your software's performance and behavior.

A

B

So, okay, first of all, I want to thank you to everyone that joined my talk. Welcome so I've agreed to deliver this talk in english, so be with me with that. So I also want to say it's been a long time since my last conference back in 2019 before panamic and okay, let's, let's get started about this talk.

B

A

Once again, hello, my name.

B

Is and today we're going, we are going to talk about understanding, node.js performance using open telemetry. This talk is not about how we set up our infrastructure for disability tracing, but mainly understanding how important of survivability is. I think this is a good start. If you want to like start microservices uh before you doing that, you must know how your application or service behave and yeah it's it's it's. I think it's safe, but to understand open, telemetry, okay, so today agenda.

B

So I need some introduction, because my stuff and my company is relatively new. It's only three years old, it's actually this year's uh it becomes three years and we need some branding what what about our company? uh Okay! The second part we are going to explore the definition of observability and a little bit the story of open telemetry and why we need open elementary to understanding our surfaces and the third one. We are going to define what kind of data that we need of certifiability, and are you aware about this?

B

Standardization of data and the third part of this talk. We're gonna talk about.

A

B

What's next and also the q a so who is so I'm working for zero one group back january and 2020, I helped building a remote engineering team, and this year we doubled the size of the engineers, and we are proud of that.

B

To put this simple 01 group is, we are helping you to build a great product, so, instead of only software that we created, we built an ecosystem to help your product to success.

B

We have like consultancy, uh technical, due diligence. uh Technical officer is under zero technology for digital marketing content creator. Breast strategy is under zero one digital and for me, I'm part of zero one technology, so we mainly build some products and research data analytics research, mathematical, modeling, that enterprise need and right now, I'm as a lead product engineer and for the next part for 01 technology. I think we are focusing on building products from node.js ecosystem.

B

uh Previously we use clojure and google script in our workflow. We love functional packet. We love functional programming actually, and anyone in here already try functional programming on the back and the front end. Please, please say you love it too, and but it's hard to ask to scale the team when we use closure and quantuscript, so we migrated to node.js ecosystem and we also proud supporting opengs foundation by contributing on numerous projects such as node.js, not add-on, api festival or unity.

B

All the opengs foundation projects that we proudly support, and I also individual supporter of node.js foundation, javascript.

B

Okay, let's get start to the talk, the first one, so what is observability?

B

Do you put it simply? Observability is help. You see what looks weird so imagine uh like on every maybe on every 8 a.m. Your service, doing something weird, throwing some error. What do you need to understand? What make it happen?

B

So absolutely answer that so in terms of uh in terms of definition, actually absorbability is term of mathematical control theory. So, once again, we as a software engineer, borrow a lot a lot of term from another um another side of jobs, so observable is measured. How well internal state of the system can be inferred from knowledge of its external output. So it's quote from charity majors, a cto of honeycomb io, so honeycomb io is also provided service.

B

That can observe your services and do you uh and anyone here already try honeycomb yeah, it's good to try so for the next. So I I'm still don't understand what what is observability. So, if you can.

A

B

uh For example banana, so you can tell where banana is unwrapped, not ready for consumption or the banana is ready for the consumption. So, as you can see on the left, side of the scale is looks green, it's unripe and towards to the right it will become yellower or- and there is a black spot in india. So you can understand how the state of banana only from its output, that is observability.

B

So as simple as that- and I I put some not random tip actually but yeah, the david also the everyone uh definition of observability, and he is baron. Schwartz uh creator of book mysql high performance tuning monitoring tells you whether the system works observability lets you ask why it's not working, so it's kind of different between monitoring and observability, so monitoring is not the same as observability, so why we need this because you know our system maybe become more complex and I believe at zero. One group.

B

uh If, if you watch previous uh talk about the cloud native yeah, we use that approach that we we create the product for the clients that hope the product is success not being complicated.

B

So uh is if, if the product success, let's start to get some more complex for scalability and somewhere else, and when we talk about system complexity, uh maybe somewhere exponential more complex, you add more engineers, you add more technology, it's really great for the products, but I think it's really hard on humans to understand it.

B

The exponential growth of products is really great for users. There are so many options, but it's hard to choose one. So you can imagine right now. The cncf landscape is a lot. There is a lot of things going on uh for maybe for storage option. There is many a possibility for like a pub solution.

B

There is a nuts and yeah. There is there's a lot of product. Is it's great for us as a user and once again, why it's so important, because uh it gives more visibility, alerting a better workflow, less time in meetings, and you can say that engineers more confident to deploy or ship our new features, because it's accelerated difficulty.

B

We will know when a bad thing happen. So observability is important because it gives you greater control over growing your complex system.

B

So the question is: why open telemetry, the short answer is we need to standardize the data?

B

I don't know back in, like 2010 2011, everyone instrumented by themselves, they're, not such thing as standardization, about how the data that we need to observe, and I will I will tell some a little bit story about open, telepathy.

B

So actually, opera telemetry is a merger from open tracing and open senses. So I was back in 2000 existed. I I forgot what exact time uh open dressing and open sensors was were, but if I remember correctly 2019 they must become open telemetry, because open tracing and open sensors have complementary features, so they are not contradictory and and in in on uh cncf uh foundation, it become uh open, telemetry.

B

And open telemetry answer this question: how will we generate the data.

B

Yeah, that's that's the simple, uh simple, uh open, telemetry answer and after that, so what's exactly the opportunity. So it's a set of api and sdk tooling integration, the design for the creation and management of telemetric data such as stresses metrics and logs. So to put on perspective, it will instrument your data, standardize. It generate the data, collect all the data that you need and export to your exporter.

B

Zipking. You can name it uh it's it's a open, fender or agnostic vendor, and why open telemetry is good choice for you. It's provided a lot of sdk.

B

Not every sdk is metal enough, but it's fender acne sticks, so you can configure with any back end. You want put on your put the edge on your code or you can put it on your vm virtual machine um yeah. It's full super quality of popular open source projects, including g, and this is the only example of not gs.

B

Yeah, because in zero one we also not, but you you really to try another language, sdk and.

A

B

What's uh it's good or not?

B

Okay, the next part is to define to define node.js, apps and service data. So what kind of data that we need to become observable?

B

Imagine if you visit japan and there there are a lot of vending machine. We we we will give give some example in what's the behavior of vending machine what what data that we can uh explore or gather from uh finder measuring system, so the data is called melt. So what is smell so melt is stand for matrix, even locks and traces.

B

To put it simply, methods are numeric measurements, so it's simple as like a cpu use memory or just some kind of aggregation of the data one minute time or events per minute.

B

For the example is the table on the pillow of the slide. So you have the timestamp. You have the aggregation, what kind of data that they produce and yeah? Maybe some mathematics- I don't know, but that's kind of data that represents metrics.

B

Second, one is events so conceptually and even can be defined as the degree action happening at the moment in time. So when we start our vending machine, maybe there is some something even is going on, uh for example, at 3 p.m.

B

A bag of barbecue chips was purchased for one dollar, so the event it will produce the timestamp what kind of event what kind of item and what the value. So uh it attach the metadata that you need uh what's going on with what with the event and third one is locks similar to events.

B

Log database discrete is always produced in our applications. It's not aggregated it's more granular than event and actually one event can produce multiple logs. So let's look through once again the example of the event.

B

Okay, one barbecue chips first purchase for one dollar. What so? What happened? um Okay, you should use your price uh user press the button b and then press the button forward and it will prompt and we will select sdb, ruby chips and user prompt user to pay one dollar. So you can.

A

Imagine that yeah.

B

This, I think it is more more close to our as a software engineer. We we rely on the log. What's uh what's going uh to happen in our application, then uh this is like just a like a tiny part of what kind of data that we need to observe and the last one is traces, so traces are stitched together from special different color span.

B

So I think it's really uh to put it simple, it's kind of a several chain of special events and the special channel of event and gator, and we call it span so.

B

If defending machine using microservices, the the traces uh as as you can see on the slide, the first span is is is on the vending machine how it process, after that, after defining mission, uh you you, you put some uh payment. What what? What chips that you choose? uh It will call the backhand and the data.

B

That, after doing some logic stuff, it will call the credit card company like uh yeah the surface, the third party service, and it will give back some data and it will issuing uh the remove from the bank.

B

So these four kind of data that we need to pay attention, how we can understand our application or our services and okay for the last part, let's get start for the demo so yeah this works.

B

I I pray for before before before I'm going to talk, I pray to the god to to make it works about the demo and you can you can you can check it on my github repo, but it's only interesting part, and I want to give some note on the on the on the javascript, uh no jesus ecosystem.

B

I actually tried to use the news sdk uh to give some example how we use uh on javascript, but I reaches version one if I'm not wrong, and there are a lot of uh breaking changes uh when when I use it so so, when I use the new sdk, it's not what what it's not what I intend to do to the demo. So I I wrote back to the like the last.

A

B

That I already tried it. It works so yeah, for example, on that that the the node the node module is become sdk, trace, node uh yeah, you can, you can search it right now and actually not. I think it's not really yet for not not no gsr system, because I I I must like yesterday.

A

B

Cannot reproduce what I need for the demo? Okay? uh Let's look our demo just wait a minute! uh Okay, so this is in my terminal.

B

This is my: um let's see what happen in this repository, so we have simple server on this. I give some like express server web server. uh There is a endpoint of slash endpoint and dead endpoint and listening on 8080 port. That's simple right and stressing the gs in here. Let's see that open telemetry provided as javascript sdk, so you can initiate the new threshold provider in here and register the provider, and you can for the first example. We just give some output on the uh under console. Okay, let's try it so, for example,.

B

So, for example,.

A

B

Dev actually yarn app to run our program. It's listening to localhost 8080. Let's do some curve.

B

8080, so on the right side of the terminal, you just simply get some get requests from uh to the surface and on the left side you can see there is a data that produces by open diameter. The timestamp duration attribute is standardized on uh open, telemetry yeah. As simple as that, you can attach upper telemetry on your surfaces basis on your programming language, and let's give some example, if you, if we using zipkin, so I already prefer provided docker compose in here.

B

uh So, let's check yeah it's running on the background. This is the kin. uh Let's try your app zip kit, so it's also also run the express surface and run in logan source 80d. But if you can see the translucent skin, it has a zipline exporter to it. So we can add you jagger promotions, all this all the together for the information in here. So this is example. If we using a zip code and.

A

B

On the terminal, we do some curve.

B

Actually it's just nothing on the console. Let's try to use a second file host 9411 and we run the query. Yeah it works. It will put the trace on the sipkin part. Okay. This is simple uh how we can add uh the trace on on our application and one more one. Last time we gonna give some example of micro services.

B

So in here I provided dust board, gs and movie gs, so movie gsas, just just return the movies or what the list of movies and the dashboard yeah. It's called the movie services.

B

So, okay, let's, let's start to explore, so uh we uh we run app, sorry just store at first and then the another surface run uh movie your movies. Okay, there is two services uh run right now and then let's go http localhost, so we're gonna, try, dashboard, uh 8080, sorry d: where is the support? Okay, 3000, okay, 2001?

B

And that's what okay it's written, because it's it's just on the console! Look what the data it give and, let's see through the zip kit, let's find traces run the query there is a dashboard service. So, as you can see, uh if, if you start to uh doing some microservice, I think open telemetry is a really good uh choice to understand. What's the behavior of your uh surface and it's really easy enough to to getting started.

B

Okay, I think that's it for the demo. Maybe I'll present my slide once again. So what's next, you can define the method, so actually the demo is only for tracing.

B

You can define the metrics and then export it to promoters and try using jager, not only zip can explore provided sdk on respective language and maybe for the people on the infrastructure. Maybe you will find some way to make it scale when we use open telemetry in our surfaces, and this is my resources when I created this talk, all the blogs, all the documentation. I put it in here if you're curious. What what I needed to to build this talk, and thanks maybe I'll start to try to answer the q, a okay.

A

Great great, uh thank you adoni for the for the presentation and, yes, we can start the completion from uh to creation. uh First from uh masjid, kunsaro.

A

He asked about uh if I have an n number of gs services service or possibly there are other language as well, and can you share? How can I use open telemetry to help me trace the call across different services, for example,.

B

A

Sensor, visible service, a surface b and two surfacing.

B

Okay for yeah formless, giri uh yeah, actually I'm not try it before, but technically it can be done. So if my surface, a I'm using node.js and on surface b, I'm using, uh I don't know golang, you can use the same api same sdk on the respective language, but you need to give some the surface.

B

The first one is the name of the surface and.

B

Which which exporter that we need to we need to address. I'm not I'm not uh try it before, but I think it's possible I'll. Try it later today, yeah.

A

A

Good okay, uh another question uh from: uh if I'm rosie, uh hey masoni can open telemetry detect both problems. Oh.

B

That's interesting, uh actually, opponent also provided so you can put open doorbell on the browser, but once again, I'm not really exploring that. Maybe you can like check out what the headers or from the request, maybe maybe you can.

A

B

Can like doing some logic stuff on your code to to know that this is the bot? Not not not people request, not human request, the resources yeah. Maybe you can try it uh to put some logic on your code.

A

Okay, yeah: it's about the behavior that uh called the ap.

B

Yeah yeah yeah. We.

A

B

What kind of bot that request our resources right, yeah you can you can you can differentiate what human and what uh both doing? Okay.

A

Yes, yes, yeah and.

B

That's all about data.

A

So yeah yeah, okay and let's go from here one uh discourage why you choose not yes to observe.

B

Okay, the first one because our company used node.js, that's the simple. uh This is the simple answer for the simple question, but if you want to ask why not gs, why not yeah like like like the factor is using go on on the back? I think I think, because our our company is more focusing on products instead of technical excellence, we need to build mvp as fast as we can, as as as robust as we can doing some tests in the same language with the back end on the front end.

B

So we want to limit overhead to learn new language, uh but we we never left the best practices like tdd uh cicd automatic deployments. We believe that whatever.

A

B

Technology, if you do a clean code, it will be fine.

A

Okay uh last last question: uh it's from ksp algorithm can the can opener limit the monitoring db and if they could queue process.

B

So I just read the combination, like all the the the provided sdk, but not try it before I I just I just opened the git github there. There is a open telemetry for database yeah, but I never try it and you can put opportunity on the virtual machine. So it's instead on the instead on the code level level, you can put it on a virtual machine, so.

A

B

Can you can like monitor some processes from machines, okay, yeah? So I think the summary of this talk is to let everyone know that you need to understand your service using data. I think that's that's my my summary of my.

A

Okay, okay, pretty good! Thank you cool, uh so I think it's uh enough time for us and yes uh thank you. Thank you for uh the presentation about the open tournament on the gs uh uh and uh have a nice day and enjoy.

B

A

Thank you. Thank you. Everyone. Thank you.

A