Kong API Gateway 101, 7 Oct 2021

Previous Meeting

⏯

youtube image

►

From YouTube: The Life of an API Gateway Request (Part 1)

Description

In this two-part series, Kong Engineer Enrique García Cota abstracts how an API gateway processes requests into four different layers. This first video will go into infrastructure and nodes. Try Kong Gateway: https://bit.ly/3rYb9SV

▬▬▬▬▬▬ TIMECODES ▬▬▬▬▬▬

0:00 Intro
0:49 Infrastructure Layer
3:05 Node Layer
13:11 End of Part 1

▬▬▬▬▬▬ COMMUNITY ▬▬▬▬▬▬

⭐ https://github.com/Kong/kong
🐦 https://twitter.com/thekonginc
🌎 https://konghq.com/community/
🍻 https://www.meetup.com/pro/kong
❓ ️https://discuss.konghq.com/
💯 https://konghq.com/kong-champions/

#API #APIgateway #KongGateway

A

It's difficult to talk about the lifetime of a request without taking into account very.

A

Different abstraction levels on the on the lifetime.

A

So that's exactly how I ended up structuring the talk.

A

There is going to be four different layers where we talk about how requests go through.

A

The system. by the way,, I'm enrique,, I'm on the open source side of things on the core.

A

Team, and I develop on the open source gateway.

A

So this talk is going to be mostly about that,, but I might insert some parts about the.

A

Enterprise side of things as well.

A

So going into the infrastructure,, this is going to be a very high level, view.

A

You are most certainly familiar with this.

A

This is the usual setup when you have a client server architecture in the world wide web.

A

Or in the case, using kong's parlance,, a consumer and an api that gets consumed.

A

This is literally a worldwide level structure.

A

Requests and responses routinely go over the atlantic ocean or other land masses. Every.

A

Day., so what happens when you put kong in the middle is that kong acts as a reverse.

A

Proxy., it does nothing by default,, which is good.

A

It's more or less transparent.

A

This schema is used a lot to explain how things work,, but it's far from complete.

A

On the side of the consumer on the left,, what you really end up having,, it's lots of.

A

Consumers., that is why we need a consumer entity in kong.

A

And on the right,, you have the same thing., you don't have one single api., you have many., and that is why we need a service entity on kong to differentiate between.

A

Those., even in the middle,, that is not as simple on a real production system.

A

It is not unusual to have multiple common nodes working together in a cluster.

A

Before them,, there is a load balancer, which even sometimes it's also a kong node.,.

A

So obviously we could make a whole talk about how to organize this. On kubernetes with.

A

Docker images or other things related with infrastructure.

A

But that is not what this talk is about., so I'm going to restrict it to a single kong node.

A

And that is the next step on the presentation of the next chapter.

A

So now we are not only planet wide level, we are on a country level.

A

A node is more or less a country.

A

It says germany here it can be another country, or a state on the united states as well.

A

This country, in particular, is very concerned about efficiency, doing things as fast as.

A

Possible with the least resources possible., so the way things got started was by looking at what was available,, which was the.

A

Operating system. so in a modern operating system,, the largest unit of.

A

Execution is called a process.

A

They are expensive to make.. They are expensive to switch over to when you have several courses or you can do.

A

Multitasking and they cannot share memory with each other.

A

They can, however, have child processes.

A

There is this special instruction in the os called fork that makes a copy of the process.

A

And also makes a copy of the memory of the process.

A

So this is a way of sharing memory,, but it's a one way: way.

A

You cannot share memory back upwards.

A

And it is still somewhat expensive,. Even switching between siblings is.

A

Expensive. the next thing that oppressive systems gives you our threads.

A

Threads are less expensive.

A

they're cheaper to create and to switch over to,, but they're still not free..

A

One process can have many of them.: they can share memory,, which is an advantage that we will talk about a bit later.

A

But they can get blocked.

A

And when a thread gets blocked, performance is lost.

A

The thread blocked is not a huge loss,, but when all threads in a process get blocked,.

A

You can have potentially a big performance loss because the process gets switched.

A

And, as I said, before, that's more or less expensive.

A

So is there a way to avoid both of them that that was the initial question that was asked?.

A

And it turns out there there is.

A

We'll see about how to do that.

A

In kong, there is a an initial or master process that is in the in the.

A

Node country, the master process is, is a city,, a capital city like paris.

A

And it does a lot of very critical things like loading, the initial configuration and.

A

Setting the initial memory layout that then will get copied.

A

These include the lua virtual machine and other things.

A

But its most important job is to create and manage, manage in this case mostly means.

A

Resurrecting if one of them dies, workers.

A

So what are the workers?.

A

Workers are child processes from the master process.

A

This means that they get a copy of the memory that was initialized by the master process,.

A

But it's a copy., it's not.

A

It's not shared, or at least not shared upwards.,.

A

They have these very special set up in which they are single threaded, and their main.

A

Task is to actually do the work to to transmit and work with the.

A

Requests., the interesting thing about them is that they are built with what's called.

A

Non-Blocking io,, which basically means that they never get blocked.

A

So that's how we avoid the problem. I was telling about you about some slides ago.

A

Now we're going to do another deep dive.

A

we are going to do to go inside one of those threads.

A

If the we were on a city level before now, we are on a district or even office.

A

Building level., so one of these worker threads initially is empty and that empty space is.

A

Are connections. they are available connections.

A

and what happens after this worker is created,. Is that a request arrives to kong.

A

And eventually it arrives to one of those available connections.

A

So you get an unblocked or new connection that needs to be treated.

A

Since we are a thread,, we can start processing it.

A

And that's why we changed the color..

A

But while we are processing new requests, come and get basically put into a queue.

A

Once we get blocked,, we blocked on the connection,, but the thread itself can.

A

Continue processing other available connections, for example,. This one.

A

And this process continues again and again.

A

Until we get to a place like this, where we have several available unblocked connections.

A

That are waiting for the thread to do the work on them.

A

We'll talk about what this work means a bit later.

A

And we also have some blocked connections.

A

Maybe I should talk a little bit more about what blocking means on the operating system.

A

Blocking means things that take time.

A

It can be reading from the hard drive of file, or it can be send in a request through the.

A

Network and waiting for the response.

A

Those things on the traditional operating system tend to block threads,, but here they.

A

Block the connections., but the thread can continue. Processing.

A

Now, there is one more wrinkle into this set up,, which is that some of those blocked.

A

Connections are called keep alive.

A

This is related with tls or secure connections.

A

When one of these connections is established,, there is what's called an ssl handshake.

A

And that process is not super expensive,, but it has some cost.

A

So, in order to save processing speed,, we sacrificed a little bit of memory by keeping.

A

Those connections open., so if the same customer with the same ssl properties arrives,, we can start.

A

Serving him immediately without having to redo that ssl handshake again.

A

So that's why some of them are blocked,, even if they are not actively being worked. On.

A

And there is yet another wrinkle on this set up,, which is timers.

A

Every worker has a list of timers, which are pieces of code that need to be executed on.

A

His own schedule., so sometime in the future,, some of them are recurrent,, so they happen again and again,.

A

Like every one second or every 10, seconds.

A

This piece of code needs to be executed.

A

And that execution,, it's also done by that single thread. We have been talking about.

A

So this is how it looks when the thread is executing one of those timers.

A

Notice that when a timer is being executed,, no connection is being executed.

A

So this means that if you create too many timers,, you might have a problem in performance.

A

This actually has happened in the past.

A

One pattern that we have learned very painfully to avoid is creating one timer parent.

A

Connection., that's what used to happen in our health checks if my memory serves, and that.

A

Has been changed, so that doesn't happen anymore., but what ended up happening is we ended up creating so many timers that the performance.

A

Got degraded because the connections were not being served appropriately or fast, enough.

A

Ok. and that does it for that level, we are going to talk about time now and then we'll.

A

Keep digging downwards in the abstraction layer.