Red Hat OpenShift Security | OpenShift, 22 Feb 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: London OpenShift Commons Gathering 2019 Lightning Talk Detecting a Runtime Breach with Sysdig Secure

Description

London OpenShift Commons Gathering 2019 Lightning Talk Sysdig
Using Sysdig Secure to Detect a Runtime Breach

A

Yeah good afternoon, everyone, my name's Tom Llewellyn I'm, a sales engineer at cystic, um but prior to that I spent about ten years on an off at Red Hat, and my final role at Red Hat before I joined Cystic, was as an open shift architect, and one of the main reasons why I chose to join Cystic was some of the issues that I was seeing as I was trying to resolve problems for customers of open shift, as they move their workloads into production and start getting more and more complex operations.

A

On the platform that the the sort of complexity that that introduces when I saw what cystic does and how it works. There was a sort of penny drop moment of, of course, yeah. This is how you should do it, and so what cystic is doing is effectively for these container platforms, he's actually building this intelligence platform based off an open-source tool set that you see there so things like Prometheus, cystic, inspect, Falco the policy enforcement engine and call for scanning images.

A

We pull all that together and then build this intelligence platform that actually helps you to make sense of the dynamic environment and the complex environment of a micro services architecture, so that you can then actually properly a observe, what's happening and be secure. What's going on inside and being be assured and gain confidence in what's happening on the platform, so the way that works in terms of the commercial product.

A

We have this this platform that presents this both of our monitoring and a security perspective, but what I thought I would do today and there just a five-minute talk, would be sort of concentrate in on one aspect of one of the things that we can do so: the Falco security engine. So this is now a cloud native computing, so CN CF incubator project, so we've actually donated that to them.

A

It's now becoming the de facto way of implementing run time, security on containers and just work through an example of how we use that in the product to enforce runtime security.

A

So, just to give you an overview of how cystic works as a platform essentially you're on open shift, we would deploy as a daemon set and that daemon sets deploys a privileged container to each in the cluster and then that demon that that container will start instrumenting the colonel. So we are pulling an event stream from the colonel on the host, which means that we see every system call that happens on that host and from that we can then build a picture of what is happening.

A

So when we give you a metric of you know the network latency of a connection, that's because we've actually measured where we've seen the read and the the corresponding write and we've measured the time between them, so we're actually generating real time information and from a security perspective. That means that also nothing can happen inside a container without us, seeing it, because everything that you do in the container is also making a system call to that. One curdle, because I've had containers work.

A

So that gives us this rich source of information and also the ability to spot anomalies in behavior and enforce that behavior. As well so we can actually, you know, do two things we can alert you when suspected breaches happen, but we can also stop the container or pause it. So you know group, so we gives you this some enforcement of that ability as well.

A

So, switching to a quick view of how that looks in practice, so on the right, you'll see the actual yeah Mel syntax for a Falco rule, how that appears in the GUI that we give you on the web. So these are just two views of the same thing effectively, so this particular what example is somebody has managed to start an interactive terminal, shell inside a container that might be in your dev environment perfectly acceptable, but in a production environment it's viewed as a suspicious activity.

A

This could be, for example, somebody's managed to actually breach your code, and you know buffer overflow or something dropped to a shell prompt, and so you know we have. This is a real-world example of the sort of attacks that can happen, and when this happens, we can actually then do two things. We can notify you that it's happens through whatever channels you define.

A

We can also stop or pause the container, but we can also, which is a unique feature to Cystic as well activate to capture, and what this is is essentially a time-based complete dump of all system activity on the host, where this container is running for the time that we define- and you can see here that in this example, we can also start from before the event triggered as well, because we're maintaining a ring buffer of that of that system information.

A

So we can get the events leading up to the trigger and then for a time period afterwards. So you can see what happened and then that capture file can then be uploaded to the backend or downloaded to wherever you want to, and you can run another tool called cystic inspect, which allows you to then analyze that information. So this is the Cystic inspecting for interface. You can run it in on the web.

A

You can run it on your laptop and you can see that this then gives you quite a bit of information about it's, the full dump of everything that happened on that system in that 30 seconds. So what you can then do with this is you can start clicking on the individual boxes and, as you do so, it populates the timeline at the bottom, and you can then narrow down into looking at specific portions of that time frame.

A

So, as you drag on to that time, timeline at the bottom, you can actually drill in and look at specific bits of. What's happened in that in that capture, and so for this one here we know that it's a terminal, shell in a container, so drilling name to see when the alert happened and then looking at their dwell. There was a bunch of commands that ran after that a load of files got deleted, so we can then start to analyze. Well, what was going on?

A

What was that person doing that attack that happened and if I clicked onto the executed, commands tab?

A

What it will do is drop me into this view, which actually didn't just gives me a list of the commands that have been run by that shell script, and so you can see exactly what's happened there and that this entire attack took place in less than a second and what they've done is they've dropped to a shell prompt, downloaded some content that was base64 encoded deleted a directory in the container, replaced it with their own version and then dropped the connection so had you not had something like this in enforcing that attack could have happened, you'd be none.

A

The wiser is still there running in your environment. So in terms of the richness that this gives you and the ability to detect anomalous behavior and enforce it in in production, there's nothing else out there that that works in this way. So, if you'd like to know more, come and see us on Table seven and in the back hall or you can reach me, my contact details are at the beginning of the slide, be happy to talk to you more about it. Thank you very much.