Cloud Native Computing Foundation KubeCon + CloudNativeCon + Open Source Summit China 2021, 18 Dec 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: BPF: Introduction, Programming Tips and Tricks - Wenbo Zhang, PingCAP

Description

Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

柏克莱封包过滤器：简介、编程技巧和诀窍 | BPF: Introduction, Programming Tips and Tricks - Wenbo Zhang, PingCAP

柏克莱封包过滤器 (BPF) 已成为 Linux 内核中最重要的子系统之一，其被广泛应用于追踪、联网和安全。其被用于安全有效地扩展内核的功能，而无需更改内核源码或加载内核模块。然而，柏克莱封包过滤器编程远比听起来要费劲的多，这可能会让很多人感到沮丧。在本演讲中，Wenbo 将与您分享 BPF 的前世、今生和未来，以及使用 libbpf 库开发 BPF 工具的一些技巧和诀窍。

Berkeley Packet Filter (BPF) has become one of the most important subsystems in the Linux kernel, widely used in tracing, networking, and security. It is used to safely and efficiently extend the capabilities of the kernel without requiring to change kernel source code or load kernel modules. However, BPF programming is far more strenuous than it may sound, which can be discouraging for many. In this talk, Wenbo will share with you the past, present and future of BPF, and some tips and tricks to developing BPF tools using the libbpf library.

A

Hello, everyone thank you all for joining me today, I'm very honored to share how to develop ppl tools with live bpf and bpm, siri and kubecoil. First of all, let me introduce my little update.

A

My name is mumbo john. I come from pcat the company behind thai db and an actual contributor to live vpn tools. I was fortunate to get a lot of help from andrew nikurico, who is sir bpf and bpf's cre project leader. Then I was contributing to live vpf jews.

A

I learned a lot from him and to community and I enjoyed communicating with them. Now, let's get down to the topic, I will share my experience about writing. Bpf applications with live ppf.

A

I hope this topic is helpful to those who are interested in live ppf and it could inspire them to further develop and improve ppf applications with live ppf.

A

So, let's first see what's ppf cy first before talking about this concept, let's think about what ppf portability is. We know that the development which kernel is very varied, although backwards compatibility can be guaranteed as a system core layer changes within the kernel. Some system don't guarantee this compatibility.

A

When we use bpf for system tracing, we often need to get function, parameters read fields in the struct and so on. Obviously, ppf programs need to fix this compatibility problem.

A

Therefore, we need to carry uh clarify the probability of bpf bpf productivity is ability to write the bpf program that will successfully compare past kernel variation and will work correctly across different conversions. We thought the need to recompile it for each particular kernel. The solution to bpf probability is ppf cre.

A

Some people may have some doubts on the necessary uh necessary necessary day of bpmcy when they use pcc. It seems like we don't encounter any capability issues, so why do we need this new thing? The answer is yes, we needed.

A

The reason is that all those are emergencies of bcc is a major improvement in the bpf development experience. It has some not for shortcomings.

A

It used to seal on front end to modify your writing ppf programs when the program occurs when the problem occurs, it's difficult to find the problem and figure out the solution.

A

You have to remember naming conversations and automatically generated transport structures, because the like pcc library contains a huge llvm or cylon library. When you usage, you might encounter some issues.

A

When the two starts, it takes mining, cpu and memory resources to compile the bpf program. If it runs on a server light, lags system resources it might trigger a problem.

A

Bcc depends on kernel header packages, which you have to install on each target host.

A

If you need an exported context in the kernel, you have to minority copy and paste tab definition into the ppf code, because ppf programs are compiled during rentham.

A

Many simple company compilation arrows can only be detected at the runtime. This affects your development experience.

A

By contrast stage, bpm's ui has this advantage. When you implement bpm cre, you can directly use the live, bpf library provided by kernel developer developers to develop ppf programs.

A

Their development method is the same as writing ordinary c user mode programs. One combination generates a small library, a small binary file, lab ppf acts like a bpf program. Loader and relocates lose and checks. Ppf programs, ppf data firm developers only need to focus on the bpm for programs, correct correctness and performance.

A

This approach minimize overhead and removes huge developments, which makes the overall development process smoother. This is why we need bpm, cye and y is the future of ppf.

A

Bpm coie only consists of four parts: ppf tab, information, compiler, bpf, loader kernel.

A

I will work you through one by one bpf tab information which allows to capture roofer pieces of information about kernel and ppf to program caps and code enables all the other parts of bpf coie puzzle what's exciting is that in the latest, clone version, 4.11 kernel has also implemented.

A

Support for model ptf compiler uh is long provides means for ppf program c code to express their intent and record relocation. Information, bpf loader ties btfs from kernel and bpf program together to adjust compiled bpf code to specify kernel on target hosts kernel while staying complete, uh completely bpmcrie.

A

Agnostic provides advanced, vpn features to enable some of the more advanced single celebrities.

A

That's the hollywood with live ppf, as mentioned before the ppf application is divided. uh It's divided into two parts. The ppf program that needs to be loaded into the kernel and the control part on the left. Part of the finger is very simple. Bpf creasing program is triggered when the right system call is initiate.

A

It gets the person's id that triggers the right request and prints it out. The path with the right is less than 30 percent. Let's talk more about it.

A

Bpf application typically goes through the following fast open that ppf object file is passed, ppf maps, bpf programs and global variables are discovered, but not yet graded.

A

After the bpf app is opened, it's possible to make any attentional adjustments citing ppf programming tabs, if necessary.

A

Please setting initial values, bookable values and so on before all the intent before all the entities are created and loaded, loot pad load fast, ppf maps are created, barriers, relocations are resolved, ppf program problems are loaded into the kernel and the very first at this point also has a bpf application uh by decant and existed in curl, where no ppf program is yet executed after the load fast, it's possible to set up initial bpf map state without reaching with the bpf program called execution.

A

Attachment fast, this is the fact at which bpf programs get a attended to various bpf hope points such as stress points, key probes, cigarette hooks network package, proceding pipeline and so on.

A

This is the face at which bpf starts, performing useful work and read, or update ppf maps and global variables, kill down fast bpf programs and are detached and unloaded from the curl ppf maps are destroyed and all the sources used by the ppf app are freed generated. Bpf security has.

A

Corresponding functions to trigger it fast.

A

Name, open crease and opens bpf application, name, load, installation loads and verifies vpf application. Parts named attach attaches all out to attack attachable bpf programs.

A

It's optional. You can have more control by using level bp as apis directly named, destroy detaches all ppi for programs and freeze up or use the resources.

A

We replace the name with minimal, which looks like the picture on the right in attention to these typical faces. There are few motives about suffices to share with you.

A

The first is combine open and load faces. We mentioned earlier that some setup work can be done after the open test, but if there are no such requirements, we can compare the load fast and the open fast gather.

A

The example in the filter comes from the live bpf project, which is used to measure the course of autonoma.

A

Selective attach by default bpf skeleton will automatically attach all bpf programs, but sometimes we don't want to do this. We'd rather selectively attach certain ppf programs based on com command line parameters, so we can attach manually, as shown in the finger.

A

This is a tool from the live ppf tools, project to detect the delay of the block layer, selective load earlier. We might change that the program can be attached mainly, but this had two obvious job comings.

A

One is that we load an unnecessary ppf program. In addition, we can't use standard passes well and the code looks unclear, so the community has a new api to control writer to load the ppf program automatically.

A

This is a tool from the live ppf project for starting the delay distribution of the of some visors in sd4.

A

If these two runs are sitting working with kernel, I suppose model btf, then we choose to load the fm3 bpf program. Otherwise we choose to loads. The keyboard programs.

A

Custom load and attach security is suitable for almost all scenarios, but there is a special case. First events in this case, instead of using links from a struct name bpf, you need to define an struck array. Bpf links struck a picture of link links.

A

The reason is that perf event needs to be opened, celebrate uh separatedly on each cpu after this open and attach ppf event by yourself.

A

Finally, during the teardown fast remember to destroy each link in the links and then destroy links.

A

Multiple hundreds for the same event, starting in version 0.2 level dpf, supports, multiply entry, point, bpf programs between the same executable and the linkable format sessions.

A

Therefore, you can attach multiply ppf4 programs to the same event such as twist points or keyboards. This was boring about erf session name clash for details, see at bpf for support for bpf to bpf. First now you can actually define multiply headers for an event action on the right before that you need to define different event. Styles for the same event,.

A

Reading kernel structures fields, you may be aware that at the present we have only seen level bpf api related and didn't feel that it has anything to do with the probability of bpf.

A

Then we will now introduce some programming content related to bpfc.

A

Let's first look at how to achieve bpf probability in reading structure members.

A

The first is pccoe bcc will.

A

Can we replay rewrite task point to pid into accord to bpf appropriate, which is great, though sometimes might not work, depending on the compatibility of an expression used with like btf, because it doesn't have vcc called relative magic at its disposal?

A

There are a few ways you can achieve the same result. If you are using recently. I did bpf programming television, ppf programs.

A

Then you have a smartness of bpf, very fair on the other side, which now understands and checks bpf caps negatively and allows you to do. Follow, pointers and read kernel memory directory and simply avoiding ppf will appropriate course. So you don't need compiler, rewriting magic to get some nice and familiar snack list.

A

You can also use slab, ppf and ppf program characteristic way: pairing this functionality with bbmcre to support portal such as relocatable field race. You have to include this code into built-in prisoner exercise, intense comparability.

A

You can also use ppf programming, calculating and ppf's urban. That's it. It will work as you expect that will be portable between different current versions.

A

But given the bleeding age decency of bpf3 program uh type chasing, you might not have the luxury of using it yet so you have to use level of ppf property accessibility inside.

A

Non-Coie lab behalf way now with seriale and wpf.

A

There are two ways to do this: one directly replacing ppf book rate and bpf core rate bpf core corridor is the simple macro which passes both arguments directly to bpf appropriate, but it also makes a c long record build or offset relocation for third argument by parsing it through built the embracer exercise index.

A

So the luckiest example is actually just this under the hood.

A

You might you may have a question if a field has been removed from the structure. Is there a way to deal with this situation? The answer is yes, you can use. Ppf call field exists macro to do this.

A

Here is the example.

A

From kernel uh file point 50 was the member was struck, has changed so we can use this to existed to find if it existed.

A

Another scenario that causes ppf capability issues is kernel. Api changes for this case bpfcre provides two compare.

A

Camper mercury solutions like ppf, clearwise, extern, kick kick config values and the struct flavors let the bpf provides externs as a simple idea. Epfo program can define an external white variable without name such as conversion, uh linux, conversion to extract a running conversion, all a name that matches one level quick fix. I kis case such as config hc, the value of hers that kernel was built with and if the ppf will do its magic to set everything up in such a way that your ppf program can use such external variables as any other global variable.

A

This varieties will have correct values. Writing to a true kernel. Your ppf program is executed in additional.

A

Ppf welfare will track those variables as known customs and will be able to use them for advanced control. Flow analysis, and that's called an emergency here is an example for dealing with transparent interface compatibility.

A

After introducing how to deal with ppf compatibility issues, let's take a look at how to pass control information to bpfo programs.

A

Pcc's approach is achieved through string replacement, because the ppf program is the screen for vcc and can be modified at about as well.

A

If a pcc is not reused, the traditional achievement is to write configuration information into the my this mystery is not efficient. Although the information writing the mic map is static, the bpf welfare can't reorganize this situation, so some optimization can't be made the solution to such a demand.

A

A monumentally uh competitive use case is, though, using read-only global date. It is site once by the control application before the bpf program is loaded into a kernel from the bpf program side. This looks like a normal global variable excise.

A

There won't be any ppf map. Lookup overhead global variables are implemented as directly memory. Excise control application site view site intestinal initial configuration by use before ppr4 program is loaded, so by the time ppf verifier will get to. With addition of program.

A

Configuration variables will be well known and read-only. This will allow bpf fairfield to track them as known, consists and uses advanced control flow analysis. So here is an example.

A

We can naturally have some future conditions to the ppf program, such as the pid that we want to filter.

A

Next, let's talk about its storage related contrast, beginning in the curl, need six bpm hash marks, perform memory, play uh election fight board and introduce the bpf and no free alert flag.

A

The motivation for doing so is to avoid vps with a corrupt kick group with bpf deadlocks.

A

The community had tried other solutions, but in the end, pre-looking, as my elements was the same placed solution and didn't affect the user space with full behavior when fulfilled a collection, is turned memory. Expensive defines a meet the bpf, no prelog flight to keep old behavior for details, see bps map playground.

A

When the map size is not large such as my entries is 256, this flag is not necessary. Bpff no break look is slower.

A

One advantage overlapping bpf tools is that it's portable, so the maximum space required for the wife may be different for different machines. In this case, you can define survive results specified in the size and resize age before load. For example, in name ppfdlc defines my ads as this. After the open test code, ppf map, resize.

A

Not only can you use global variables to construct bpfl program logic, you can use the instead of maps to make your pro programs simple and more efficient global variables can be analyzed.

A

You just need to set global variables to be a fixed size, or at least with bonded maximum size. If you don't mind wasting some memory, for example, because the number of soft lq tabs is fixed, you can defend global errors to seal coins and stir gram in software coding, vpf dc.

A

Then you can traverse the ram directly in user space.

A

If you want to know more details, you can read the following articles, but first two articles were written by bpf martiner, which is very valuable and kills a lot of knowledge related to principle.

A

The last one was written by me mainly related to actual compat. If you want to try it out, you can start with following two projects.

A

Otherwise pincap is a software where servers provider commented to delivering one stop experience to create deep business solutions.

A

Thaidb is an open source, distributed new circle database for elastic skill and real-time analysis.

A

That's all thank you.