Kong API Gateway 101, 21 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: The Life of an API Gateway Request (Part 2)

Description

In this two-part series, Kong Engineer, Enrique García Cota abstracts how an API gateway processes requests into four different layers.

In part 1 of this series (https://youtu.be/gnX_uBh0KOc), we started a journey from the planet-spanning infrastructure layer to what happens inside a single Kong worker, similar to an office building in complexity.

In this second part, we’ll dive a bit deeper—we'll see who the occupants of that office building are and the kind of life they live.

Try Kong Gateway: https://bit.ly/3rYb9SV

▬▬▬▬▬▬ TIMECODES ▬▬▬▬▬▬

0:00 Intro
0:49 Phases
3:05 Plugins
13:11 End of Part 2

▬▬▬▬▬▬ COMMUNITY ▬▬▬▬▬▬

⭐ https://github.com/Kong/kong
🐦 https://twitter.com/thekonginc
🌎 https://konghq.com/community/
🍻 https://www.meetup.com/pro/kong
❓ ️https://discuss.konghq.com/
💯 https://konghq.com/kong-champions/

#API #APIgateway #KongGateway

A

We are going to talk about time, now.

A

And then we'll keep digging downwards in the abstraction layer.

A

So phases are periods of time in which something interesting happens and.

A

What you can and cannot do changes depending on which phase you are in.

A

So the init phase is the first one that happens right when the master process is created.

A

There is nothing else,, no workers, yet.

A

and the master process needs to do it's stuff,, and then this kong related.

A

Stuff, like you, can see the whole list, here.

A

I've already mentioned quickly before that one of those things it does is creating the.

A

New lua virtual machine., it's worth also noticing that this particular phase is not available for plugins.

A

I will be marking the phases that are available for plugins and those that aren't usually.

A

At the last line of these bullet points., I'm not going to go over every single bullet point,, but I think this slides can serve as.

A

As a good reference., I will talk about the highlighted ones: only.

A

I mentioned before that the virtual machine is created by the master process and then gets.

A

A

This setup is one of the most confusing, or was one of the most confusing, things for me,.

A

To understand when I was trying to develop my first plugins.

A

Until I understood this,, it was very difficult for me. To, in particular, understand how.

A

Global variables worked on a plugin.

A

Sometimes they seemed to be working just fine and suddenly they stopped working and.

A

I didn't understand why.

A

ok, next phase after init is the init_worker phase.

A

This is executed once per worker.

A

A reminder that workers get a copy of the master virtual machine.

A

This means that any global variable that was initialized on the interface they get a copy.

A

Of., but if they try to make modifications,, those modifications are local to the worker.

A

And new connections don't necessarily reach the same worker every time they reach a node.

A

So it's possible that those modifications are not there when the next connection arrives.

A

Somewhat difficult to explain,, I can explain with another diagram at the end.

A

Notice in blue at the bottom that the init_worker phase is available on plugins.

A

Now,, when a request arrives to the kong node,, it goes down to a connection and it starts.

A

Getting treated.

A

now,, we are not on the office level, anymore.

A

We are now on the person level.

A

Each connection can be thought of as a person and the phases on which we treat a request,.

A

Which are several, can be thought of more or less as the hours of a day of a person.

A

So., when the request arrives to kong,, there are several pre-proxy phases,, which are.

A

Similar more or less to things that people do in the morning before work, like having.

A

Coffee, getting a shower and so on.

A

First, one is the certificate phase.

A

I have already mentioned at the on the previous abstraction level. The keep alive.

A

Connections., this is the phase that it's working with them.

A

When there is a tls connection,, there is this tls handshake, and then we get this.

A

Special phased executed,, but it does not get executed on every single connection,. Only.

A

When the tls handshake is done,, and since we have the keep alive,, it doesn't happen. Every.

A

Single time. as a result, one special thing that this phase does that no other phase does.

A

Is that the context,, which is a variable that leaves for the whole duration of the of the.

A

Request, gets cleared after this phase gets done.

A

This means that if you create a plugin that sets something on the context on the.

A

Certificate phase: it is not going to survive until the next phase.

A

So you have to take that into account..

A

Probably don't touch the context on the certificate phase.

A

The next phase is the rewrite phase..

A

This one is kind of special.

A

on a human,. It would be similar to waking up and not knowing who you are yet.

A

And I say this, because on this phase, the router has not been executed.

A

That means that you still don't know which service or route got matched by the request.

A

That just arrived., so there is very little things you can do.

A

I tend to think about this phase as an early exit phase.

A

For example,, if you wanted to,, if you are sure that your kong node will only answer to.

A

Get requests, you could implement a very simple plugin that, on the rewrite phase rejects.

A

Any non-get request.

A

this would be a bit more efficient than doing it later, because the router won't execute,.

A

So it's going to be faster,, but that's more or less it.

A

I have not been.. I have not seen this particular phase being used by any of our open source. Plugins.

A

It's true that I didn't look very, very deeply.

A

Probably one of them uses it,, but most of them don't.

A

Access, on the other, hand, is the most usual one.

A

This is the one that I think is the coffee or the breakfast, one that almost no one.

A

Skips., it happens after the router has been executed,, so you know which service or route.

A

Is being matched.

A

It also has an access phase on plugins,, so this is probably the most popular phase to.

A

Implement on a plugin. and we'll talk about plugins a bit later on the next chapter.

A

You can see that there are lots of things that happen on the access phase. Besides those.

A

Two that I mentioned.

A

Preread is the same or the equivalent of access,, but for tcp or udp connections,. The.

A

Router is also executed and you have a preread phase on plugins for those plugins. That.

A

Support tcp and udp.

A

Ok, so now we are ready to proxy or we are going to proxy send,, and that means sending.

A

The request to the upstream api in this case.

A

That is kind of similar to commuting to work and then doing work outside your home more or.

A

Less., the content phase is implemented by a module built in c,.

A

And it's kind of hard coded,, it's kind of not available to lua,. So as a result,, it is not.

A

Available to plugins., in parallel with this phase,, there is a pseudo phase of sorts called balancer.

A

This balancer phase happens several or can happen several times per connection.

A

Basically,, what's happening inside this phase is that we try to connect to one upstream,.

A

And if it doesn't work,, we try another one and if doesn't work with another one, until we.

A

Run out of upstreams or or one of them sends a valid response.

A

So that is why we call it pseudo-phase,, since it happens inside the content phase,, we.

A

Cannot, for now make it available for plugins.

A

And when a failure happens, a series of not very important things, happen.

A

You have this as a reference. If you need it.

A

Now, how do we send a response??

A

The response is very interesting because it's different from how we treat the request.

A

As you can see,, both lines are highlighted, here, and that's because we treat the response.

A

As a chocolate bar that we don't eat in one go.,.

A

We split it into small pieces and we process it piece by piece.

A

So the first piece is the one containing the headers.

A

This means that as soon as the last bite from the headers,, the response headers arrive to.

A

Kong,, it will start processing this phase or this phase starts.

A

And at the end of this phase, kong will send the headers back to the consumer, so it can.

A

Have the headers faster., the consumer, doesn't have to wait for the whole request to go through kong.

A

It has a header_filter phase of a level on plugins.

A

And now we get to the body_filter phase and the body is even crazier, because you don't.

A

Have you don't get a single phase for processing the response? Body, you get multiple.

A

So you get the first chocolate block.

A

And then, since potentially responses can be huge.

A

I'm talking about imagine video responses or even images, or sometimes even javascript,.

A

Zip files., so one thing you don't want to do is having to allocate memory or disk space for the.

A

Whole thing, if you can avoid it, so you treat it chunk by chunk.

A

As a result of this treatment,, you only have to allocate the memory or space for each.

A

Chunk individually.

A

That is why you do it,, but it is kind of uncomfortable.

A

If you want to make a transformation on adjacent response,, this way of treating the body.

A

Is kind of not very user friendly.

A

So recently, because of this, and because of needs on the non-lua plugins,, which are a.

A

Recent addition, we have now another sudo phase called response.

A

Response, basically replaces header-filter and body-filter.

A

It does not split the chocolate bar into small pieces.

A

It eats the whole chocolate bar in one go headers and body filter at the same time.

A

This means that it has the inconvenience that I mentioned before.

A

You have to have space to hold the whole thing. Up.

A

And if you are serving videos,, this can be huge.

A

It has an extra thing to take into account,, which is that when any of your plugins has a.

A

Response phase that activates buffered responses, globally.

A

This means that all your connections from all your kong.

A

Workers in the node will wait until the chocolate bar is complete before sending.

A

Them back to the customer., so don't use this willy nilly, be very aware of the constraints.

A

Log phase is interesting because it happens after the the response has been sent back to.

A

The consumer., so it's free time in the sense that any time you spend here, it's not going.

A

To affect the user experience of that particular connection.

A

However,, keep in mind that any time you spend here is time that you don't spend serving.

A

Other connections or on other timers,, so it still affects the global efficiency of the.

A

System. and one particular wrinkle, the log phase has is that cosockets,, which are what.

A

Used to communicate with the outside world of kong cannot be used on this phase,, so you.

A

Have to get around that limitation by using timers,, but I mentioned before that timers.

A

One time per request is an untidy pattern,, so things get tricky that way.

A

The log phase would be similar to go into bed at night dreaming, so you don't do any more.

A

Work. last sudo phase in this case is the handle_error phase.

A

Anytime, something unexpected happens like an unmatched route or the upstream, not.

A

Responding. basically, this gets run and then the header filter, body, filter, and log.

A

Phases happen,, but with a default,, not user, configurable response.

A

It's not available for plugins, partially, because a plugin can throw an error. So this is.

A

On top of that., so if you wanted to see this on the schema altogether,, it looks like this.

A

I decided not to do this presentation on top of this, because it's kind of difficult to.

A

Talk about a schema like this,, but feel free to use this as a reference.

A

Now, last thing was plugins, and I wanted to mention that plugins,, which are pieces of.

A

Code inside kong that change stuff are similar to hands.

A

If connections are people, plugins, are there ways of changing things?.

A

So the first thing plugins can do is talk with an external system.

A

All our logging, plugins and the tracing ones like zipkin and prometheus are basically.

A

Doing this on the log phase.

A

So,, so the user isn't impacted.

A

The next thing plugins can do is modify the request when they are talking to the external.

A

Api., we don't do this often..

A

These are the two plugins I found.

A

Not surprisingly, the access phase is what gets used. Here.

A

The other thing we can do is modify the other arrow that exits kong.

A

Again, very few examples of this as a pure mechanic response transformers.

A

And cors are the two plugins I found that are purely doing this.

A

It's surprising that cors in particular,, that's setting of headers on the access phase.

A

That's something that is a bit surprising. Sometimes you can set the headers of the.

A

Response before you have even sent the request to the upstream server.

A

. and one special case of sending a special response is actually the most useful or most.

A

Used structure of plugins,, which is shortcutting.

A

All of the authorization plugins reject requests. If the.

A

Credentials are incorrect., but others, like bot, detection or ip restriction, also help alleviating the traffic on.

A

The upstream and keeping our customers happier without traffic that they don't want.

A

Again,, the most popular phase in this case is the access phase.

A

So in conclusion, kong is full of very tiny people who do most of their work, either on.

A

Breakfast or dreaming at night, and they like to eat their chocolate in small chunks.