Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2021, 29 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: A Safer Curl | Bash for the Cloud - Carolyn Van Slyck, Microsoft

Description

Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

A Safer Curl | Bash for the Cloud - Carolyn Van Slyck, Microsoft

Most of us have used curl to download a script and run it immediately. Using curl | bash provides instant gratification. We can quickly get up and running with an application without requiring a steep learning curve or a strong attention span. Unfortunately, the common advice is that this is not safe! But what if it was?

Let's walk through how we can work with people's natural tendencies, keep the one-liner and make it more secure. We will use Porter and Notary to transform an example cloud-native application deployment from a dicey bash script, executed with bash and hope, into a safer one-liner installation that was designed to be used in production.

You will learn:
Why curling a script to bash is insecure, and why bundles mitigate those risks.
How to reuse existing tools and scripts in a bundle, without starting over from scratch.
What a safer one-line user experience could look like.

A

Hi, I'm carolyn van slick, a principal software engineer at microsoft. Today. I want to think through how we can have our cake and eat it too, namely I want to install software quickly and securely.

A

Now I have to admit that I'm kind of a sucker for running an installation script by using curl pipe bash- I mean I know I shouldn't, but it's just so easy and I'm so lazy.

A

Now this laziness extends beyond just installing devtools on my laptop. A lot of cloud native software could benefit from having a more straightforward installation process. Common wisdom says that curling a script piping, its bash, isn't safe. So what can we do to make this work? Instead?

A

What if we could make installers that are as easy to use as curl pipe bash, but we're a whole lot safer.

A

First, let's talk about what curl pipe bash really is. So this is what it looks like: that's when you copy and paste a command. That looks like the one right here now. We all use scripts like this because they install quickly and it helps you a software that you aren't familiar with curl downloads, the script and then the script contents are piped to bash and they're immediately executed.

A

Personally, I really like installation scripts. I can copy paste my way to victory with just a single command. They require little to no knowledge of the installation, logic or tooling and yeah. I really do believe that the project maintainers can automate the installation way better than me on the first try.

A

So, let's review, why curl pipe bash is a security risk? What if someone on the internet offered to help install an application that you really wanted and they would do it for you? All you need is to hand over your laptop for five seconds and they'll type super fast and boom it's installed.

A

Now I don't know about you, but I don't hand over my laptop to my husband, and I trust him mostly when we think about handing over your laptop to a stranger, it's easy to see why it's risky. They would have access to everything on there, documents, pictures environment variables with security tokens and passwords you're logged into a bunch of sites.

A

Maybe even your work vpn, it's clear that they could do anything with your laptop that you could do and that doesn't change when you switch from a physical laptop to a quick copy paste of an installation script when you use magic installation scripts, you are giving the script full access to your laptop and anything your laptop has access to oops.

A

So, okay, what? If you read the script first and then run it? You open the script in your browser, you look it over and go yup.

A

This looks like how I expected, and then you run the script.

A

So here's a fun fact: a web server can detect that you are piping, that web request to bash a malicious server can detect that you are piping to bash and then decide to give you a different file, not that file that you just read in your browser, but whatever it wants and then you're right back to where you started letting randos run amok on your computer.

A

Instead of using that handy one liner, here's a safer way to run an installation script first download the script, then, if they provided a checksum of the script, um match against it and make sure you've downloaded the right file, now read the script figure out if it is legit. If it's what you want to be running and then you know run it.

A

My frustration with this is that the copy paste experience from that one liner is completely lost. No one really wants to read bash and, depending on the complexity of the script, even after reading it, you may not completely be sure what it's going to do.

A

Most importantly, when you do finally run the script, it's executing as you and again it can still do anything that you can do all right. So we aren't supposed to curl pipe bash. But what, if I really want to anyways, have you ever wanted to try out some software and the installation?

A

Instructions are like 15 steps where you have to install three clis copy, your credentials into more environment variables, you're gonna, forget about them and leave them around forever, and you know it requires a database that you don't feel like figuring out how to set up and really now that you think about it. Maybe you'll just try out this some other time, because this is just one more yak to shave and you have work to do you know who could really benefit from a one-liner cloud-native deployments, often they're complex with dependencies dedicated, clis magic configuration credentials.

A

Most of us don't want to read a series of docs and install extra tools just to try something out. We really don't want to figure out how to script, coordinating all these tools, for example, creating a database and then passing the connection string as a helm. Chart value just a little example, but I don't want to do it.

A

Lots of potential users stop there and decide. This is more than they're willing to invest, especially when they're just figuring out is this software. They want to use and they're installing a couple and trying them out to see how they work.

A

So I know that I just said we shouldn't curl pipe bash, but if we could just provide our users a great experience to quickly set up and try out our software, that would be great right.

A

So, let's flip the reflexive. You can't do that. That's not safe to how can we make it safe? What would we need to do to make that happen? We need to isolate the installer only giving access to what it needs instead of being able to read. God knows what that is hiding on my laptop. We want something that does the boring stuff like checking signatures or digests for us.

A

We also want reliable insight into what is actually in the installer so that they don't have to read the source. There are a couple of tools out there that can meet most of our requirements here now. One of them is the cncf sandbox project. Porter.

A

Let's see how close we can get to making a safer installation experience for our users. Porter is a command line tool that makes cloud installers. It packages your application, which could be a website, a microservice, an iot, app whatever, with all the necessary tools, configuration and deployment logic necessary to properly install your application.

A

The bundle is versioned it's distributed by oci registries, and it can be installed with a single command. Now it doesn't replace any of your existing tools and porter works with any tooling or stack. You can use vms uh serverless iot bare metal. Whatever you can script, it all goes in the bundle.

A

Porter provides packaging distribution, a layer of security around what you already use today. Let's say that I have a nifty app that I'd like to share with the world tabby cats. Tracker tabby cats is a browser extension that shows you a different cute cat, like you see here on each new tab that you open, tabby cat's tracker is an app that lets. You save your favorites and share your top tabby cats with your friends.

A

Tabby tracker runs on kubernetes and it's installed with a helm chart and because I was resume padding a bit, it uses the cloud database to save your tabi's too. So it's got everything in it. Now, let's use porter to install tabby tracker safely, no curl pipe bash for us for unfamiliar bundles. You can use porter, explain to learn how to install it.

A

Bundles have meta data that describes what the bundle is its version, the credentials it needs to install and any parameters that you can use to. Customize. The installation explain, gives you enough information to know what you need to pass to the porter install command or later to the upgrade command.

A

Now, looking over my bundle, I see that I need an azure service principle. I can pick which region to use, though it'll default to my favorite central us.

A

I can also see that the bundle uses terraform and helm and based on the outputs it's going to create a kubernetes cluster and a database. All right that makes me feel a bit more comfortable running this bundle. Since I can see exactly what credentials and tools, the bundle will use.

A

Next, I'm going to tell porter where it can look up any credentials that the bundle needs, in this case an azure service principle. Instead of storing them in environment variables. On my computer, I'm going to put them into a secret store.

A

Here I have azure key vault set up with secrets for the service principle, I'm using azure, but I could have used any secret store like hashicorp. Vault porter has something called credential sets, which is a mapping between the credentials used by the bundle and where porter can look up those credentials when the bundle is run. Porter looks up the credentials just in time and injects them into the bundle.

A

This is great because it keeps sensitive values off my dev and ci server.

A

Now I'm going to run porter credentials generate to make a credential set and pass it to my bundle.

A

Porter is going to walk through each credential and prompt for where that value is stored. Now mine are all in a secret store, so I can just enter in the key name for each value, and this tells porter where to get my credentials when it needs it later now. I realize this is an extra step installing something that you wouldn't have to do with curl pipe bash, but remember that one of our goals was isolating the installation script from our local computer.

A

We don't want the installer to be able to snoop around and find juicy environment variables with tokens and credentials in them when porter runs a bundle. The bundle executes inside a docker container any credentials that the bundle needs must be explicitly passed in a credential set. Lets me store a hint for porter on how to find my credentials.

A

Maybe I have one for my azure credentials, another for aws or github.

A

I can define this once and then, when I run a bundle that needs access, I can give the bundle that credential set without having to specify each time how to read them out of a secret store.

A

Now that all this security stuff is out of the way, here's the one liner to replace our curl pipe bash can run porter install, give it my credentials and the bundle to install that's it.

A

So, even though the bundle is using helm and terraform, I don't have to have those installed on my laptop, because the bundle remember a docker container has the right versions of the cli installed. Alongside with my terraform files and helm, charts everything my app needs to install is inside that bundle. It doesn't matter what I'm installing it's the same, porter install command.

A

Now that my app is installed, let's check it out the porter list command. Lets me see everything that I've installed with porter, and I can see more details about that installation with porter show.

A

So I can see when it was installed if it was successful and if I had specified any custom parameters when I installed it, they would show up here too.

A

What I'm looking for next is the url of tabby cats tracker that I installed I'm going to look for the outputs generated by the bundle with porter with installation, outputs list.

A

So the bundle saved all sorts of useful things and the connection string for the database, cubeconfig, terraform state and here's. What I was looking for, the endpoint for my app.

A

Now you're gonna see that I'm kind of an inspire aspiring web designer, but hopefully the cute tabby cats make up for my css skills seriously, even if you aren't into porter. At this point, I hope that I've tricked you into installing the tabby cats browser plug-in at the very least.

A

Now, let's go look at the cluster that it made for me, I'm going to save the cube config and then I'm going to use it with cubectl and see what was deployed.

A

Here I want to call out just two things.

A

First, the image used by my deployment uses a digest, not a tag, that's something that porter can help you manage so that you aren't copying digest around. You know, because it's not quite as easy to work with as tags and I'll show you in a minute. Just how porter does that. The other thing that you can see is that porter injected the connection string for the database that the installation script created now, let's go see the source for the bundle and how it was made.

A

A bundle is defined by porter.yaml file. This manifest contains basic information about the bundle and also has all the data that you saw from porter explained here. You can see that I've declared that we need an azure service principle and also a region parameter.

A

Keep going, let's look around a bit. It also declares what tools are inside. The bundle by declaring mix-ins mix-ins are building blocks for bundles and they help you use existing tools inside of a bundle and have it work. The way you'd expect mix-ins install the tools inside the bundle's docker image and map between the yaml that you see here and the final commands executed inside the bundle when it's run mix-ins can make it easier to to write the bundle.

A

For example, since we are using helm inside of the install action, it will automatically use the helm, install command mixons help. You extract data from a step and pass it into the next step of your installation.

A

Here, terraform created a cluster and a database, and I'm grabbing those out of that step and making them available to the helm chart which I'm running right next here.

A

Normally, this type of stuff is all done with a bit of glue work inside a bash, and you could still do that if you wanted to the mixins, just make it easier to focus on the deployment logic.

A

So pretty much everything you see here helps you create a bundle with just one file and the yam will kind of help to piece everything together. You don't have to do this like. Quite honestly, you could keep the bash script. You wrote today put in the bundle and go, but for a lot of people. This extra metadata and structured view of passing data between steps can be really helpful.

A

So did we hit the mark if I'm looking for a safer way to quickly install rando software from the internet? Does porter meet our requirements?

A

Now we wanted to isolate the installer, automatically check signatures and digest, and we wanted insight into what the installer was actually going to do. Bundles really just borrow security features from existing tools. We can check off isolated installer, they run in a container and they only have access to what we explicitly provide like cloud credentials.

A

Bundles are distributed over oci. You know docker registries and uh this existing infrastructure that we already know how to run. We can scan it for vulnerabilities.

A

We can check digest and porter can help do this for you, so we porter we're also working on signing bundles with notary, so that porter can check the bundle, signature and verify that it was created by a trusted publisher, so that checks off our first two security concerns.

A

So for the rest, with porter explain we can see what's in the bundle and how to run it, which is way better in my opinion than reading bash great, so it's safer than curl pipe bash. But is this the experience what we were originally aiming for?

A

Full disclosure? It isn't always a one-liner. The first time you run a new bundle. You're going to run porter, explain to see what credentials it needs and then create a credential set with the credits it needs. If you don't have one set up already, but for any bundle that you've worked with before it's just that one command porter install, I mean I'm calling it close enough.

A

It's always the same command. Just porter install regardless of the text stack, tucked away inside the bundle, so I think we're doing pretty good here and finally, what we really wanted. All along was a magic installer, someone with no experience with the app or any of its deployment tools.

A

They can run, install and get a working installation of software. In very little time. I mean kind of magic. So there you go it's a safer curl pipe bash for cloud native deployments mission accomplished.

A

I said that this is a safer curl pipe bash originally, but there was a there's a little qualifier on the end of that which is, I said, for the cloud: porter isn't really intended to install just say a command line tool on your local dev machine.

A

What bundles can help with and make safer and easier would be things like quick starts for your application, you're trying to help someone get your software set up for the first time, so they can try it out or customer installations, you're, not a sas, you're shipping, something that the customer needs to install.

A

So, in addition to writing an installation guide, you can give them the installer that helps them get up, get set up really the first time, but it also tracks upgrades over time so that you can manage additional operational concerns from day to day, uh like a bundle could help you do troubleshooting for the status, see what's up or down or do a backup even your own internal applications, just like an external customer would kind of appreciate a little help. Setting up and running your software. You know the same applies to the people on your team.

A

The people managing your software in production aren't always the developers who wrote it being able to upgrade to a new patch quickly, flip a parameter and like crank the log level to 11 with just a single command, I think, is kind of useful teams can also share an installation of porter so that when say someone stalls the app in a new environment on monday, another team member can see what was done and pick up where they left off without any explicit handoff.

A

You can also use bundles for a side project.

A

Now I don't know about you, but when I work on a project in a weekend, you know I'll figure out how to set up all the infrastructure and like configure the secrets for my app and then I get distracted like life happens, so I tear it all down because I don't want to be paying for it and then, when I come back to it a few weeks later, I don't know I've forgotten how to install the darn thing.

A

To be honest- and this is where bundles are kind of cool, I can capture all the logic for deploying this app that I was playing around with and when I come back to it later, I can just run the bundle and I didn't lose any progress trying to re-figure out what I did a couple weeks ago, and I mean this is why I like bundles.

A

I can ship not just a binary or a docker image and if I'm really like going above and beyond giving someone an installation guide, that's actually useful instead, not instead, but in addition to I can just ship an entire installer. It has all the tools it needs. It knows how to do a proper installation, not just what was easiest to document and come on. I mean you know, there's a gap between the two.

A

What's in the docks and what's a real production installation, you could put that real production insulation in the bundle, even if it's awkward and hard to describe here, are some resources for getting started. Making your own installers, the quick start walks you through how to use porter to install bundles or you know, if you like videos, our learning page has demos and talks that cover porter just in general and when you're ready to make your first bundle run.

A

Porter create boom you're up and running then head over to our authoring, bundles page for more details on what you can do inside of a bundle. Now, if you have any questions, our community page has links to slack our forums, the mailing list. You can ask questions and connect with other users.

A

Thank you for going on a bit of a user experience journey with me asking ourselves an important question that I think is relevant to any software. How can we make the tasks that people are going to do regardless?

A

No matter what you say, how do we make that more secure, rather than telling someone nope that's a bad thing, you're a bad person for wanting to do it. Let's aim to understand why they want to do this task in the first place, what were they really trying to accomplish, and maybe there's an alternate user experience that we could provide and make it safe thanks? Everyone.