Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2022, 11 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Keynote: What We Learned Dissecting the World’s Most Popular Containers - Ayse Kaya

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from April 17-21, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Keynote: What We Learned Dissecting the World’s Most Popular Containers - Ayse Kaya, Head of Strategic Insights + Analytics, Slim.AI

Speakers: Ayse Kaya
Data scientist and container enthusiast Ayse Kaya and her team at Slim.AI analyzed more than 100 of the world’s most popular public container images using open source tools to better understand what developers encounter when running containers in Kubernetes. What they found was a vast, varied, and complex world that gives developers massive opportunities to scale, but also presents risks to both security and productivity. This talk shares the data, visualizations, and insights they generated from their research. Kaya shows the current paradox in software supply chain practices (i.e. taking advantage of abstraction vs. knowing what’s in the software you ship), and that even small, special purpose containers could have thousands of packages, libraries, and licenses, not to mention critical vulnerabilities. Finally, she’ll highlight the current trade-offs teams make between “developer experience” and “production readiness”, and open a discussion about how we can improve as an industry.

A

Have you ever felt infinitely small in front of something vast and complicated like the midnight sky ever since I was a little girl? I've been fascinated by complexity, especially how complexity emerges from simple, humble beginnings, from Elemental particles to galaxies at MIT, focusing on emerging Technologies I was fascinated activated by research like steppenwolfrom and John kanime's Game of Life tying the computer systems fit biological world with life itself.

A

After all, what could be more complex than biological systems right, except maybe kubernetes if kubernetes is the organism that containers are the cells, and maybe that takes the analogy a little too far, but we know that containers are being used all the time everywhere in nearly every software development project across all verticals see I'm, not a devops engineer, I'm, a data scientist working for a startup called slim.ai at slim. We are obsessed with container optimization and security, and this can hundreds of thousands of containers regularly.

A

This year alone we have scanned more than 900 000 containers and, as a data scientist working with this data, I would like to provide you a window into the world of containers, as developers experience. It often for the first time.

A

I would like us to expand our senses about this world. I would like us to look at some perspective. Shifting data about this world take, for example, the 60 percent. That's the percentage of top publicly available containers that have more vulnerabilities in them today than they did a year ago.

A

60 percent- and this is after a year of intense focus on social Air, Supply Chain Security in the aftermath of multiple security incidents.

A

The rest of the containers haven't improved that much and we'll be talking about that in a moment. But before, let's couple this inside with this other statistic here, this one is from a global randomized study that we designed asking questions to developers and devops engineers.

A

Seventy percent said their customers are asking demanding their containers to have zero vulnerabilities.

A

On top of that, 88 of the developers said it is getting more challenging to remove the vulnerabilities from their containers the number one contributing factor being the complexity, the numerous components with dependencies in them. So, as you can see, there's a story that's unfolding before our eyes. Containers are becoming more vulnerable.

A

Customers are demanding Perfection more than ever, and developers are feeling the pressure, and although developers Are In the Heat of the battle, there is an undeniable Rising tide when it comes to developer interest in kubernetes and containers and container adoption across companies, large and small. In fact, developers are beginning to look at a technology like Dockers, as native to their workflows, like a tool like git or an IDE, to stack overflows, 2022 developer survey.

A

Docker is the number one technology developers love and want to learn, and you can see this in the usage stats as well. The old time pools on Docker Hub has tripled in just a year, but while there are lots of tutorials out there to learn about shipping and and building containers, there is precious little understanding around. What's in them, like cells, containers are being seen as these Atomic units a little building blocks of living organisms where requests go in data comes out.

A

It's just that simple, but then something like log foreshall happens and before we know it, 2022 becomes the year of software supply chain security and we are all more invested in understanding the not only the principal components of our software systems, but also their second order effects their dependency trees.

A

So how did this renewed sense of awareness change the container landscape? My research team at slim.ai have been deconstructing analyzing, observing containers in an effort to understand what makes these containers developer friendly and production ready.

A

Last year around this time we put a magnifying lens onto the top publicly available containers on Docker Hub and published a report with analytical insights.

A

We had some shocking findings in that report and I'm not going to list them all, but I'm going to tell you this. As container enthusiasts, we were expecting to see some outliers certain containers, certain container categories, having a large number of vulnerabilities packages, libraries licenses special permissions. What have you, but even the averages were surprisingly High.

A

And one thing to note about these containers is that, yes, there are 10 million images on Docker Hub with more than 318 billion pools, but this top 165 containers account for more than 30 percent of all use. So it's a very special plaster.

A

So today we published a sequel to dodge report this time, focusing on the Delta between last year and this year. So let's go ahead and look into the data and some of our learnings findings.

A

The first finding is that we have more high and critical vulnerabilities than ever across all categories now, I already mentioned that 60 of the containers have more vulnerabilities today than they did a year ago. And yes, we did remediate certain vulnerability, certain incidents, but we are detecting new incidents four times faster than our remediation rate and the new ones. The new incidents that we detect are mostly high and critical vulnerabilities, but the high severity vulnerabilities have increased by 50 this year.

A

So today the average containers has almost 300 vulnerabilities, thirty percent of which is high or critical up from 20 percent last year, and we thought 20 was too high.

A

Let's move on to the next finding component complexity has also increased significantly. The average container today has 400 packages and, if you think about it, this number is supposed to be the tip of the iceberg.

A

Every package might have thousands, sometimes hundreds of thousands of dependencies, as shown by multiple studies, but it turns out that even the tip of the iceberg is an iceberg and it is not just the packages the licenses have increased by three and a half X. The containers have more layers, they're larger. The scanning of these containers takes longer times, and even the metadata they're s-bombs have increased by 40 percent.

A

Now don't get me wrong these tools? These packages are necessary for experimentation. It helps developers to debug and test and build and have fun, but they also represent a tight surface and implied in need to have an automated process to remove this so that that attacks your first surface never makes it to production.

A

Data is data, but what's the impact on our people, how does this work complex and more vulnerable landscape impact? Our devops engineers and Developers I mentioned that 88 percent said they find it more challenging to remove the vulnerabilities.

A

Only one in four Developers said they fully understand how containers living and hardening works and, interestingly, that seems to be a mismatch. A disconnect between Executives and the front line. Engineers 49 of the executives said they are slimming and containing they are slimming and hardening containers in their companies. But the people who do the actual work are reporting significantly lower numbers.

A

Governments and companies may be demanding a world with zero vulnerabilities, but that land feels Out Of Reach, given our tools and techniques. The sobering bottom line is that we are no more secure today than we were back in 2021.

A

The Silver Lining, however, is that we have woken up and we're more aware of these challenges than ever before us at slim. We believe you should ship, you should know what's inside your container, and you should ship only what's needed to production you're seeking to solve this challenge for all of the world's containers through Automation and intelligent, optimization and look I. Do not think complexity is the enemy here. Ignorances there are times why we need to take a step back and enjoy the poetic beauty of the complex systems around us.

A

There are other times where we need to step in fearlessly and deconstruct and redesign.

A

I know that we have the resolve the decision, intelligence, the build power to solve these issues, and most of the decision makers are in front of me in this room.

A

My friends say: I'm an incurable optimist, but that's not why I think the future is bright. I know there are relentless brilliant competent teams around the world who are losing their sleep thinking about these problems, so many teams that I couldn't even fit the names that we leveraged for this project alone. In my thank you card here and as humans When We Gather, like we did at this fabulous location when we put our hearts and Minds together, we can make even ridiculously tough beats.

A

Look easy and that's why I know when I'm on this stage next time, I will be bringing better news. Thank you.