solo.io Hoot Livestream, 18 Jan 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Episode 19: eBPF: A Top-Down View

Description

Join Lawrence Gadban, Field Engineer at Solo.io, on our next Live Hoot Episode on January 18th at 10AM Pacific/1PM Eastern.

In this Hoot we will explore extended Berkeley Packet Filter (eBPF), the Linux technology that is quickly gaining popularity throughout the industry. We will first discuss at a high-level what eBPF is and how it works, then with that foundation we will take a hands-on look at building and running eBPF programs.

To learn more about us, please visit https://www.solo.io

We are hiring! https://www.solo.io/company/careers/

Questions? https://slack.solo.io

Code Samples: https://github.com/solo-io/hoot

Suggest a topic to cover here: https://github.com/solo-io/hoot/issues/new?title=episode+suggestion:

A

A

Let's see here should be live for any of any of you all out. There apologies for the technical difficulties, um but hopefully you're seeing a screen and you're. Seeing me my name is lawrence. If you've seen me on camera, um I think we should be good to get started again. Apologies for the late start. I promise we'll make it up with the with the content that we'll be talking through, um but okay, so welcome to our hoot.

A

uh My name is lawrence, like I said from solo, I'm a field engineer here at solo and today we're going to be talking about ebpf, a top-down view, um and so the goal for today is to kind of get a high-level overview of what ebpf is as well as kind of some a hands-on approach to to actually building and um running some ebv ebpf programs.

A

So let me go ahead and get this slideshow going should be able to see the slides, okay, so ebpf a top-down view. So, like I mentioned we're going to kind of get into an overview kind of the you know the the fundamentals of what ebpf ebpf is, um then the top down view portion will actually dig into um you know. What's what really is happening and why why this technology is so powerful and then we'll see a hands-on demo so to get started? Ebpf ebpf?

A

What is it? It stands for extended berkeley, paclet packet filter, which is it actually is kind of a evolution of uh t of the original bpf which stands for berkeley packet filter.

A

The reason it's called that is, um you know, as the name sounds it's for it was originally for filtering packets, and so, if you think of a tcp dump, tcp dump- or you know several other of the tool of tooling in the you know, linux, world or unix world for filtering packets- that that is how that be bbpf is how you do that, and so with tcp.

A

Don't, for example, you know if you've ever used that in order to um to you know capture, network uh flows and so on, what you're really doing is you're you're telling tcp dump to to? Basically say I want to capture packets, that are, you know, tcp or udp and they're from this source address to this destination address and so on, and the way that that worked is by using the berkeley, packet, filter, subsystem and so ebp ebpf is an extension of that.

A

Where now, rather than only working for you know, packet filtering it, it essentially is a way to to to handle any type of custom logic that you want to essentially embed into the linux kernel without having to change that, and so here on this second bullet point, uh this is really more. I actually saw um some of the maintainers for edpf have talked about. You know the the technology has kind of outgrown the acronym. It's a you know it's a technology, not an acronym and really what it is.

A

Is it's it's a technology that enables you to embed custom logic and attach it to various points in the kernel and elsewhere. Even, for example, one of the things that that you can do with with ebpf is you can actually attach to user space applications, um but I would say the majority of the you know the most common use case, at least for now, is for attaching to various places in the kernel.

A

It's also event based, so I call out, you know, think think javascript. So, basically, as an event happens in some type of kernel functionality, you can then do something in your evpf code and one of the other really important things about this extended. You know, ebpf, that's really important. Is that the it?

A

The the programs that you write are are verified to be safe, which means that now you can, um you have a guarantee, uh or at least you you have a close to guarantee that you know the the code that you're writing the programs that you're running won't crash the kernel they're guaranteed to return. You can only access, specific sections of memory and so on. So it gives you a lot of confidence to run.

A

You know current you're, essentially writing kernel code, but it gives you some some safety that you're not going to essentially crash your kernel, and this is one of the really important benefits of evpf compared to things like kernel modules, where you know, if you, if you load a kernel, module and it's faulty, you could crash the kernel, in other words, bring down the entire system.

A

So there's a lot of value here that ebpf has provided- and you know recently it's become a very popular technology for for valid reason, because it essentially gives you um kind of endless flexibility of being able to modify and run custom logic in the kernel without having to actually make changes in the kernel um and we'll get into. You know some of the more specific uh things that you can do with it here in this in the rest of this talk.

A

So um what is an ebpf program so ebpf, like I said you know, the technology has kind of outgrown the acronym there's a lot of components to it, but at the most core level, what you're doing is when you write an edpf program, is you're writing some code that will again run in the kernel, and so you know I call out this quote here. This is from um kind of the unofficial, bpf documentation.

A

There's a link here, we'll you know I can send this out, but basically an ebpa program is simply a sequence of 64-bit instructions, and so when you, ultimately you know, ultimately what you're doing is you're writing some code and we'll get into what that code looks like, but ultimately it gets compiled.

A

Or you know, what have you into a set of 64-bit instructions and ebpf runs as a vm in the kernel which, with its own instruction set, you can also think of it as bytecode, and so you have essentially what looks like assembly right that are specific to evpf, and when you, you basically will write code in you know higher lingual level, language, compile it to this assembly and then that assembly will contains the the you know that that's the actual instructions for what happens for your program and in most cases on a lot of the you know most common architectures.

A

You actually can that that byte code will actually be jet compiled to essentially run natively. So if you're writing some ebpf code and you compile it to your assembly, then on you know, common architectures that will get jit compiled into native. uh You know native basically native code native machine code, you're, essentially running as pretty much as fast as you can go.

A

So now you have this custom logic that somehow is running as fast as it possibly can on your system and that's really important for the types of things that you want to do with evpf. You know whether that's tracing or whether that's network policies and so on, you're getting the you know the flexibility of being able to load this in um kind of outside of outside of like making changes to the kernel, but it also will run at a you know, native speed um here on the bottom. This is uh this really.

A

This example is a little um it's kind of in the weeds, but the main point here is that, given some ebpf program that's been compiled into assembly, we can actually look at the assembly, and this is what you know. This looks like, so you can see that there's registers, um there's function, calls and so on, and so it's not too different from you know any other assembly that you may be familiar with.

A

But again this is specific for the ebpf vm running inside of the kernel so kind of taking all that- and you know what thinking about. What's like a standard, workflow you'll write some ebpf program code and, and here we're calling out in c what you'll see as we go through this. Is you typically kind of the most common way of writing ebpf code today?

A

Is you write in a c like language, and I call it c like it's referenced to see like because the syntax is you know c, as you may know, but you don't really use things like libsy, like you, don't have the standard library and so on. You can only use specific ebpf, helper functions and and built-ins and things like that so you're not using you're, not writing like system c, but you're writing uh using the c language. You're. Writing your your ebpf program, then you'll compile it uh probably using clang. I actually don't know.

A

If there's I don't know if there's another compiler that works today for compiling to ebpf assembly, but you know you, you write your code in c you use clang to to compile it to ebpf assembly or bytecode and that's stored as an elf file. We kind of you know briefly looked at that in the last slide. That was the end result was that that ebpf code or program was compiled into an l file that contained your evpf uh assembly.

A

Then you'll. Take that elf file, and you know the various uh programs that are in that file. Will you load them into the kernel and you also create the necessary maps, we'll go into more detail of what the? What maps are? What ebpf maps are here in a second and this? This process is typically done for you by a component that can be called.

A

That's typically called a loader which makes sense because you're loading programs into the kernel, but you can also do it through system calls and we'll take a look at some of those here in a second then, once the code is loaded into the kernel, the programs are loaded into the kernel, then you actually attach them to whatever hook point you you are you you need to so we'll again we'll get into more details. But you know this is basically loading. The the program into the kernel is is not enough right.

A

You have to load it into the kernel and then you actually have to attach that program to do something, and you do that by attaching it to the hook points and that you know, as we mentioned, think event based right. So the hook points are essentially more or less saying. Okay, based on this hook point I'm going to receive an event for say when I receive a packet right and then you can do whatever you want with that data and then that's kind of the last step right.

A

You read and analyze that data, so those apps that we talked about come into play here. But ultimately you know you you want to do your program will do something right and more than likely it's going to be operating in some sort of data. So then you need to read and analyze that data and that's kind of the high level workflow of an ebpf program.

A

So getting a little bit more into the details of loading and attaching so like I mentioned the loader usually takes care of loading, your program into the kernel, but under the covers. What that's really doing is it's using this bpf and you're you, you make this syscall with your program and you get back a file descriptor uh kind of similar to you know pretty much every like either. You know the common saying is everything in linux is a file um or more accurately, a file descriptor. What happens?

A

Is you load your program into the kernel with that syscall and you get back a file descriptor that references, the program that you just loaded and so that's what you'll use to kind of do all of the rest of the plumbing, for uh you know whatever it is that you need to do with that program.

A

So the hook points that we talked about depend on the type of program that you're writing. So some of the exam. This is a you know very small list of examples, but things like trace points, k, probes, socket filters, xdp, which extend, which stands for express data path there are. There are lots of types of ppf programs that you can write and based on that type.

A

That is how you can, then that determines where you can attach them to and we'll in this in through the rest of this example, we'll be using k-probes a lot so I'll go into more details there, and but it's important to call out that, based on this type of program that you're writing and you know what, where you're, attaching it to you, you actually will use a different different tools and different system calls to be able to attach to the attach your programs to those hook.

A

Points- and so I mentioned k-probes- is what we'll be using k probes I'll come back to this I to call out what a k-probe is if you're not familiar. This is straight from the the kernel documentation k-probes, enable you to dynamically break into any kernel routine and collect debugging and performance information non-disruptively.

A

What what's important about that is that basically any kernel function, you can use a k probe to basically get a copy of all of the memory or data being accessed by that kernel function and that's really powerful, because you know pretty much anything that that anything. Fundamentally that happens in your system. You can theoretically get access to it with a k probe and so and we'll go more into detail of like what k what they look like.

A

But you just know that you know k-pro gives you access to the um basically the access to the data being passed into any kernel function, and so, when you, in order to attach that for k-pro, specifically one of the ways you can do, that is by using this perf event, open, syscall and then followed by io, ctl syscall, and what you're really doing there is since k-probes. You know pre-date ebpf ebpf and they are part of like the perf subsystem and so really what you need to do.

A

Is you need to tell that system that hey? I want to attach a k probe to this function, more or less right, and so with ebpf. What you're doing is you're attaching um first you're you're you're, creating you're telling the kernel that you want a k-probe on a given function, and then you are using ioctl to attach your ebpf program to that k-probe.

A

So we talked about evpf maps. um This is kind of your. These kind of are your fundamental.

A

um This is how you really get data to and from your bbef programs- and this is important because for you know, especially since we're talking about um things like uh observability and tracing and so on, you need to be able to access that data to make sense of it right, and so the bbpf maps are a way to do that. um I call it here. You know their data structures used to store gather data gathered from the program share with other programs et cetera.

A

um There are lots of types of maps, and these, you know, represent different types of data structures.

A

um You can also get into things where, like you, can have maps that are created on certain cpu, cpus and so on, um and they can be accessed by the bbpf program, as well as the user space program, and that's important because you know, if you want to do something with that data, you need a way to access it from user space, and these maps are the way to do that, similar to the other components that we've been looking at.

A

It's also created with that bbpf syscall, so in other words, you can create it from user space right, so you can make a syscall to create it again. This is also something that's typically done for you through your loader and we'll get into more of that later.

A

So, to kind of give you a more visual representation of what is actually happening when you build in the ebpf program, um this is kind of the I'm not sure if that yeah, so so.

A

This is a very small snippet of what a program would look like right and we'll get more into the code uh later on, but just know that you know you have some, you you're writing some ebpf code and then you compile it using clang and you're, calling out that the target's bbpf uh and the output file is in this case this tcp tcpconnect.o file and then, when we run that command.

A

If we look at what that file actually is you'll see that it's an elf file- and I just you know, make sure elf is uh the and actually let me make sure that you can see that last yeah.

A

Okay, so elf is the um standard kind of executable format on linux and so you're getting what looks like a standard, executable format file, even though it's really just a way of representing some compiled ebpf code right, and so, as you can see this is um we have an elf file that contains the the code that we just compiled and then what you want to do is if you, if we, if we kind of look in that file, what we can see is here we go, I think, yeah, okay, so it's kind of lagging a little bit.

A

I apologize, um but if we, if we actually look inside that file, you'll see that- and this is a standard, elf format, um you basically have sections that represent different pieces of of data more or less, and in this in this file we have all of the you know various things like a light, the license uh we talked about maps and then in our example. In the previous example, we have various k, probes and uh k, red probe, which is essentially so the k-probe is when you enter your kernel function.

A

The k-rep probe is right, as you return from that function, and so you can see that we have the k-probe and the k-rep probe for tcpv for connect, which is what we wrote our probes for, and so, when we compiled that program, we got an l file that has um assembly more or less for each one of these programs and then what we'll do is. We will take that and load that program into the kernel and that's how we get our ebpf program and, more specifically, the the actual um type of program into the kernel.

A

So, for example, if we, if we focus in on this tcp v4 connect k probe right, what we're really going to do is, as we parse through this file or more accurately, you know, as the loader parses through this file, it will look for sections that have, for example, k-probe tcp v4 connect. It will know that. Okay, I need to attach this as a k probe and the content of this k probe or the the actual code that represents this k.

A

Probe is in this section in the elf file, so what our loader will do and again you could do this by hand, but there's no real reason why you would want to do that unless I guess you were trying to test something out, but what you do. Is you take that that that that probe and that section and that you know byte, code or assembly that represents that program, you'll make a call to the bp bbpf, syscall and you'll tell it.

A

I want to do a program load, and then you pass the program details, and so in this case, what the program details look like is it's a bbpf, prog prog type k pro so you're saying that I know that this is a k probe because of the section name, for example, and here is the bytecode, which is in the elf file at this section.

A

So what you do is you need to load that into the kernel and that's what the syscall accomplishes.

A

The last step is that your program is essentially validated by what's called the verifier and the verifier is the component that you know make sure that your your all the things that we talked about at the beginning that make up the safety of the program.

A

So things like you know that it returns, and you know it's not an endless loop.

A

uh It only accesses memory in the correct space that it's supposed to, and things like that and the verifier is what actually performs that verification and if it so, if your program doesn't pass the verifier, this call will fail and the uh program is not loaded into the kernel and that's how you that's, how you get that safety of um unless you get that safety of not you know of making sure that your program will not blow up the kernel or do something nefarious. Something like that.

A

It's worth calling out that the verifier can be.

A

You know a bit finicky. So, even if you write, if you're writing code, that you think is accurate, uh the verifier may reject it, and that just might be, because you are not initializing um memory in a way that it expects or you're not you're, not you know doing something with pointers in a completely correct way or you're inferring.

A

So the verifier is probably something that will give you trouble, um but that's where it helps to kind of dig into what what commonly is done. The verifier will tell you exactly where it it's rejecting your code, but it does this in a pretty low level way.

A

Essentially, it tells you know what in the assembly, which, for example, which register is not passing the verification, and then you can take that and kind of work backwards and figure out where you need to make your code more explicit or where you need to change something, but once that is done now you get back that file descriptor, that that more or less points to your loaded, edpf program in the kernel, so to take a quick break from kind of the you know, the actual specifics of getting the the programs into the curl and so on to actually write this stuff.

A

It's important to kind of know how you can do that, um and so one of the things that uh so these are. These are kind of some of the options that you have, the the top one is like one of the it's probably not a realistic option, but you, since the ebpf program, is ultimately just represented as as assembly or byte code, you could, in theory, write that by hand.

A

There are actually some really interesting use cases uh that that do exactly this, but it's more! It's not something that you probably don't want to do that unless you have a really good reason to or if you're just interested in you know, learning the assembly um there's actually a really cool, I probably should add a link to it. There's a there's, a user space implementation of the evpf uh vm.

A

So if you wanted to you, could you know, write some assembly by hand and run it through the the user space vm to make sure that it it would work. um But again, that's not really something that uh it's not really for realistic, uh or at least the standard use case. um The next one is bcc.

A

This is kind of one of the original uh frameworks. If you want to call it that for writing. Bbpf programs, um it's the I have the you know description straight from the the read me here, uh but basically what you do with bcc is it's very tightly coupled with the system that so you write some code, that's in the bcc style and what you do. Is you more or less write it? You ultimately get it says here you can do python or lua I'll use the python example.

A

You basically have a python script that contains bbpf code as uh c that is treated more or less as a string in python and then, when you actually run your python script, bcc will take that that string, that has your code, do the compilation with um it basically has clang and llvm as part of it, and it will do the compilation on the fly when you run it, and so it actually I mean it's a pretty. You know solid tool, but there are downsides to it because of exactly what I just said.

A

You have to compile it on the fly and you ship around playing an llvm as part of the binary more or less, and so it works, but it also has some down down. uh Basically, some cons as far as like portability and so on.

A

um Another important thing is that, when you're writing code for bcc, you typically are writing your your bb bbpf code to use the kernel structures directly from the linux headers, more or less, and so what that means is that you're writing your code for a specific version of the kernel and you are depending on the headers to be present. So that's kind of two downsides right there, because the headers are not present on that's more of like typically that's more of a dev environment. Type thing running that in production is a your production.

A

Servers typically will not have the kernel headers present and then, on top of that, that the kernel, the the actual kernel structs change, pretty much every version, and so, if every kernel version- and so if you're writing it for one version, what the way that the fields are structured or the way that they're ordered and so on may not be the same on other versions of the kernel, and so there's some downsides there as far as portability goes and that's actually what the next one lid libby lib bpf is trying to solve, which uh more or less is the kind of production style, more modern way of writing bbpf programs, and so this is kind of an example here, where bbpf program code is normally written in the c language, with some court with some code organization conventions added to let libby pf make sense of bpf code structure and probably hand everything into the kernel.

A

This is taken from uh lib, bpf, bootstrap kind of which is a getting started guide more or less for lib bppf, and this. uh This blog here that I link out, is a really great uh resource for everything. Pretty much live bbpf. This is, uh I believe, is I don't know how to say it. I don't want to butcher it, but andre nekrico.

A

He is one of the core maintainers, I think, is also kind of like the tech lead of the libby pf structure, and so what this really does is this? Will this attempts to solve the portability challenge with two components, called btf and ko re or compile once run everywhere?

A

uh Btf stands for bpf type format, and this is this: is you know we don't have time to get into the details here, but really what it's intending to do is allow you to to compile your bppf code into a binary that you can then ship around and with with the btf bbpf type format and ko re.

A

That means that, as long as the kernel supports bpf type format, you can run your code on any version of the kernel that supports that that type format and it allows you to do things like you- know, you're, not depending directly on the kernel structures, this library, more or less knows how to handle different versions of the of the kernel headers or of the structs, and will automatically kind of relocate based on which version you're running on so kind of again to you know: condense.

A

It bpf is the kind of future forward um per more production style way of writing.

A

uh Bbpf programs, um then just quickly also call up their high level tools like bbpf trace, which is an interesting tool that kind of essentially is trying to give you a higher level higher level language you can think of something like awk is very similar that allows you to write bbpf code without getting into the very you know the weeds of writing like c and talking to kernel structures, so that's another option as far as like kind of higher level um tooling.

A

So that's we'll move on from this there's another tool that I'm going to touch on at the end um that that you know we as solo uh are, are we wrote that will help the goal is to make it even easier for getting started with with bbpf and we'll touch on that here in a second um quickly to get into the the top down view?

A

This is kind of like so we've kind of talked about what the what the technology is, and you know what it does, but how it actually works is something that I think is a little confusing and when you see what it actually, what we're actually doing, it makes a lot of sense. In my opinion, it helps kind of make everything click.

A

So if we want to do something like trace a network connection um in this case we're using curl and we're just making a curl to 1.1.1.1- and you know that that's just a server that lives somewhere over the internet. So if we want to actually trace this, you know what we could do is we could have um some kind, some type of instrumentation in curl, uh that that will keep track of you, know, connections and so on to a given server or a given destination.

A

um But let's see how so what we'll do is we'll kind of track through go down into the linux stack to understand, what's actually happening and how we can use ebpf to help with that so kind of the at the top level.

A

You know, ultimately, under the you know, when you make this curl, can this curl request to connect to 1.1.1.1 curl itself has to actually establish that connection and the way that it does it is with um in the curl there's a lib for connect.c, and what that really ends up doing is calling the syscall connect, which uh more or less is going to just take an open, socket and then give it an address that you want to connect to, and so from kind of the user space and kind of from you know the the application level.

A

That's it it says. Okay, I need to make a syscall. I need to call connect. I want to establish connection to this server, but then what happens is it starts going into the linux stack, the the network stack, and so the actual syscall makes a request here to do on the socket. I want to call this connect function more or less for this socket and what the in this case, you know we're talking. uh 1.1.1.1, that's a tcp connection with curl, and we want to do this.

A

This ipv4 right, so there's a specific function in the in the kernel for making a tcp connection over ipv over ipv4. So this is not, you know directly on the path but down in the weeds. There is a struct that basically points when you make a connect call. What that really does is it calls this function, tcp underscore v4 underscore connect, and so, when your socket connect option is being called what actually happened?

A

What actually is happening is it's calling this tcp v4 connect function, and this is what actually performs the you know, the real connection. So when you make this connect call, it goes to the syscall. This is call under the cover is going to call this tcp v4 connect and that's kind of the lowest level. That is what is actually making the connection.

A

So if we want to trace you know, network connections and if we want to not have to instrument the application itself, what we can do is write a bp bbpf program that will use this basically we'll we're going to. In our example, we'll use a k probe to take the you know basically get an event more or less when this function is called and then do something with it.

A

So, in our case, we just want to log it, or you know we just want to see that there is a connection being established and so going back to our earlier example.

A

If we have the k-probes and a k-rep probe on this specific kernel function, when we you know, we attach that to our on our system to that, to that to the right function. Calls when I, as a user, make a curl 1.1.1.1 without any changes at the application level, the we will automatically get an event and then, whatever we're doing with the event down here, that will automatically be uh called more or less, and so what we can do is you know we'll go look at an example. We can just.

A

We can, let's log it right, so we'll be able to know, for example, which uh which pid is making a request to what server, and so the kind of the power comes from the fact that we're be we're able to do this transparently to the application.

A

We can, the the attachment and the loading and all that is happening outside of the the application process and since, as we touched on it, basically gets compiled down to native code or native machine code, we don't have or there's very little performance impact, um and so it's a really powerful thing. And then this is just you know one very kind of not contrived example, but it's a very simple example and it, but it hopefully illustrates kind of some of the power that ebpf provides.

A

So uh we're kind of running a little bit late, I'm going to keep going, but what we'll do is at solo, we we released an open source, a project called bumblebee which is essentially a a way to get started with edpf that hopefully makes the developer experience the getting start, experience extremely simple and so we'll jump into a quick demo. With this, I encourage you to go check out the website, the git repo, the github repo, if you're interested.

A

So here's our github repo go check it out. The goal really again is to be very straightforward and it also provides some pretty cool things that we'll we'll see, um but really you have to you. Should you don't have to do much?

A

Basically, all the user space stuff that I was talking about, it's kind of abstracted away for you. You can also get metrics automatically prometheus style metrics, but uh let's get into a demo, make some of the concrete.

A

So I have. uh Let me make this a little bit bigger. I have the uh command line tool b, downloaded um already. You can get it just by running our install script. You can also do um yeah here's the excel script. You could also uh you know, use go if you want to, but essentially I have that downloaded here on my system, and so if I were to do b you so I have it here and then what we'll do is I want to write some code to do exactly what we just talked about.

A

I want to see network connections, and so I have a single file here. This probe.c. If we look at it, um it's very it we'll go into some of the of the details here, it's very much adapted from our tutorial.

A

So if you are at the repo, if you're interested it's too big in our docs folder, there's links on the readme, but we have a getting started guide. That kind of talks through pretty much everything that I'm gonna show so go check. It out, go, try it out, but to get into the code. What we want to do is we want to log the pid and the ip address of every connect, call every tcp connect, and so to do that.

A

This file is, let's see, 47 lines of code and what we will do is we're going to create a map.

A

Remember the map is how you pass data to and from uh you know, either the kernel or wherever you're at and so from the kernel to user space, or vice versa. If you wanted, in our case we're going to make a map that will just simply print out whatever whatever data we get from our event, and so in our case again, we want to. We want to log the ip address and the pid for the destination, that of a of a connect call, and so we're going to create this map.

A

That's called events, and it is a map of type ring buffer and the value is this event struct, which is defined up here, and this event struct contains two pieces of data, an ip address which represents the destination address and the pid, which is the process id of the of the process.

A

Making this connection right, and so our map, which is called events, uh is going to receive events that contain the destination address and the pid of those um you know of the connect calls and then uh just also call out the section.

A

uh If you remember, when we were looking at the elf uh file for the bppf code, um the this dot maps represents that it's a ebpf map that we want to load into the kernel and we'll be using this to pass data to and from this dot print is a bumblebee kind of specific convention, and this will tell our loader to automatically print out any event in this map, and this will make more sense when we actually run it. But this is basically a convention that says you know anything.

A

That's in this map I want you to in this example, print it out to the screen automatically. So we're not writing any code to like you know, display it or anything that's handled for us um and then the the actual probe itself is pretty straightforward.

A

um Since we're connect we're adding a tcp or k probe to the tcp v4 connect. These following arguments are more or less the arguments that the actual function call the kernel function gets.

A

So, in this case, it's a socket and a socket address, and that is you know what is actually the kernel is actually using to make to establish the connection, so we have access that in our kpro, and so what we'll do here is we'll initiate or initialize the event that's going to contain the data the the destination address and the pid, and then we're also initializing some memory to hold the destination address.

A

uh We do a bp bbpf, core co-re read, which is a lib epf kind of specific helper.

A

This is again handling the the portability aspect, where this allows us to write code that will run on multiple kernel versions, more or less, but really what we're doing is we're getting data from the arguments that were passed in to that that tell us what the destination address is.

A

So ultimately, this deader variable after this call will have the correct value on whatever kernel version you're on for the destination address ip address, then we need to reserve some space in that ring buffer and then we simply need to write the data so to our event, we're writing the destination address and we're uh writing the pid. The pid we're getting from um this is another helper function. Again I mentioned earlier. You can't use kind of standard. um You know since you're running this is running the kernel.

A

You don't have access to like the lib c right, so most of the kind of data that you need to get you get through helper functions. This is an example of one of those helper functions which will give you the pid. Essentially the name is a little different because when you're in the kernel space, the pid is not kind of the pid that you would expect from the user space. But essentially this function gives us what we want, which is the the user style pid, and so we we have the destination address.

A

We have the pid. In our event, let's go ahead and submit that to the ring buffer. So I know that was a little quick again. The tutorial that I mentioned on on the github kind of goes into the weeds and will show you um kind of we'll walk you through in more specifics. What is actually happening so then, what we want to do. We have this code again, that's the only that was all the all the code we need to write. um We need to actually build it.

A

So what we're going to do is we're going to build using uh the b command the bcli and what we can do is we can do a b build and we want to build the probe.c file and then what we can do is we one of the other kind of cool features of bumblebee? Is that so in the in the slides we were looking at. You know the fi, the the actual output from a standard ebpf build process.

A

Is that elf file, but what we did is we actually made it to where we have a an oci image spec that allows you to kind of package up the the bbpf program in an oci image that you can then ship around or store using any of the kind of standard.

A

You know cloud native style, oci, image, uh workflows, and so what we'll do is we'll build this program that we just wrote we're going to build that into an oci image and we'll call that um my underscore probe and since an image will tag it we'll tag it with v1, so b build the file that we want to build and then the oci image that we're going to package it as, and so we compiled it and as you can see it says you know, we wrote it to this probe.o and you can actually see that that file does exist right.

A

So what we actually could even do is do this file on it and you'll see that it is an elf that contains ebpf, and then it's also saved to our oci image, my underscore probe v1, so that that that is an image right that could be pushed to a red container registry that can be pushed to. You know whatever you wherever you want it.

A

It's also still stored locally on my system, so we can do is we can run it now with b run and then the name of the image, so my probe v1 right and so the runner. This is the runner. This is the actual.

A

You know tool that we built that again, since we didn't in with bumblebee, we didn't write any user space code, but it does know that there is a map called events that we want to print events for right, and we also know the format of that based off of the code based off of how we define that struct.

A

So if we open up another pane- and we make some network requests, for example to that that 1.1.1.1- that we were looking at right as we make that request- and actually here you go- I got another one. Just you know from standard system. um You know whatever is happening on the system.

A

There was a request from pid 459 to this 108 address and then my curl call down here was uh you know this pid, which, which was the curl command, that I just run ran, made a request to the destination address of 1.1.1 and again you know this is formatted as an ip address. This kind of happened all dynamically. We didn't have to write any code to do that. So if we send some more traffic right, we can send more 1.1.

A

We can go to somewhere else like google.com.

A

Right, and so this is one of the ips for google.com, I'm sure we can run this a few times, we'll get different ones yeah, so 138 101, but all of this is happening without having to write any code to handle kind of the the reading of the map. But then it also is uh we're also, basically we're getting. You know, network tracing network observability without having anything in curl. To do so, and so there's a lot of power there um again bumblebee is intending to make it simple to get started.

A

What I wrote again is based off of the getting started guide. You should be able to get started as well, um but this is just kind of like a you know, the very very uh intro level to kind of the the power that edpf has, and um you know some of the the tooling that is being built around it to make it easy to get started. And then we haven't even touched on things that you know you can do things like I mentioned in the talk in the slides there's.

A

You know different places to attach programs to that. Allow you to do things like network policy or, if you want to um you, know, handle a packet at the earliest possible point. If it's from you know, say a bad actor, you can drop it immediately before it even goes through the stack.

A

So there are a lot there's a lot of options, there's a lot of power, but with that and we kind of ran over because we got started late, so I'm going to stop there. I hope it was illuminating or interesting.

A

If there are any questions, please let us know but yeah, that's all I have thanks for watching and let's see, doesn't look like there's any questions.

A

Nope, okay, all right! Well! Thank you! Everybody see you next time.

A