Continuous Delivery Foundation GSoC, 5 Aug 2021

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: GSoC Phase 1 Try Spinnaker io

Description

For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/

A

Hello, my name is daniel and today I'll be presenting phase one of my project for google summer of code 2021, my project name is try.spinnaker.io explore spinnaker in a sandbox environment.

A

I like to start off my presentation, giving a little um primer on what spinnaker is so spinnaker describes itself on its website as an open source, multi-cloud continuous delivery platform that helps you release software changes with high velocity and confidence.

A

As you can tell, it is quite a mouthful and I like to break down these buzzwords one by one. A simplified, high level. Explanation of what spinnaker is is a tool that allows you to deploy applications in a very fast and safe way.

A

Spinnaker supports deployments on all major cloud providers such as aws, azure, google cloud provider and oracle spinnaker's biggest selling point is its continuous delivery features. It supports advanced deployment strategies such as red black rollouts, which deploy a new version of your application with the existing version, and it destroys the old version once the new version is ready to go.

A

It also supports automated canary analysis, which rules out a change to an application to a small subset of users, and then metrics are collected to see if everything is running properly. You can also define your own deployment process to your heart's content.

A

Spinnaker also supports rollbacks, which allows you to revert to an older version of an application if the new version has gone catastrophically wrong.

A

There is also a manual judgment feature which makes all updates require human approval, and you can restrict updates to a certain time period. There are tons of other features that are available in spinnaker and I recommend you to read them.

A

If you're interested on our website, spinnaker was originally developed by netflix to serve as their own private deployment platform, but it was released to the public in 2015 and since then many other companies such as google and airbnb, have also adopted it as their own primary deployment platform and in 2019 spinnaker was donated to the cd foundation.

A

The motivation for this project actually comes from my personal experience.

A

I clearly remember the first time when I tried to install spinnaker and it was extremely difficult to say the least, and I spent countless hours searching through random, get up issues and looking through stack overflow and digging up random messages from slack just to get the main ui of spinnaker to appear on my computer, and I think, probably one of the biggest reasons why it's so difficult is because there's so many dependencies required to actually get spinnaker running. So you need a external storage provider like an s3 bucket.

A

You need to have a kubernetes cluster that has at least 16 gigs of ram and four cores. You also need to set up cloud providers that you want to deploy to, and you do a lot of networking to expose the ui, the api and whatever services you're. Providing.

A

If you compare this to like a project like jenkins, all you need to do to run jenkins on your computer is to have java installed and double click. The jar file having a sandbox environment where users can go in and deploy some pipelines and test out the spinnaker ui, is something that I really wish I had when I first heard about this project.

A

I think other open source projects are the importance of lowering the barrier to entry to having some kind of hands-on experience with their own project, which is why we see services like the go playground and play with docker being available to the public.

A

Regarding the infrastructure of our project, I decided to go with a multi-tenant solution on an aws eks cluster. This means that all the users will be sharing a single spinnaker instance on the cloud.

A

All the infrastructure is codified, using terraform and as simple as running one command to get tridot spinnaker.io running on aws spinnaker and its associated configurations are installed using armory's, open source spinnaker operator.

A

Here are the list of features I have implemented so far, and I'd like to give you a quick demo on what they are when you first enter our site, you'll be prompted to log in with your github username and.

A

Password and after you log in um here's, the spinnaker ui so right now we support um deployment to a kubernetes cluster using manifests. So here is an allowed app and we can deploy this.

A

And if we go in here, we can see what exactly this pipeline is doing. So we don't really have any special configurations, but we have one important stage, which is the deploy manifest stage, and here it deploys nginx to our cluster and it also creates a load balancer. So we can access nginx.

A

And this is finished deploy here, so we can actually access this by going into the clusters and going to deployments- and here, if you click this little button right here, this one is for the load balancer and you can just go to the link and our nginx deployment should show up.

A

Yeah, the redirect, I guess, isn't working well right now, but here we go and another feature we have is is um our private uh ecr registry, which stores all of our docker images and, as you can see, this is not the uh the default docker hub that many people are used to this.

A

One is um hosted by us and the reason why we do this is so that we don't hit any um rate limit issues and so that users have to deploy um images that we have already hand selected for our application, so users can't deploy their own malicious, bitcoin miners, or anything like that, and to show that um all other public images are blocked.

A

We can try to deploy this unknown app, and here this tries to deploy just the regular nginx from docker hub, which is public, and if we try to deploy this, we should get an error.

A

So we can see the execution details, and here it says that our um our security enforcement has denied the request, because this um image is not part of our image policies, which is exactly what we want, and we also have an auto resource cleanup pipeline here and what this does is every 20 minutes.

A

It deletes all the deployments, all the services, all the storage, related things and secrets that have been deployed by spinnaker, which means it has been deployed by the users on our application.

A

So if we run so, this runs every 20 minutes by itself, but we can just run it manually.

A

Ourselves and yep this ran successfully, so since everything is destroyed, if we go back to our deployment here, yep our load balancer should not be here anymore.

A

I think, if you just give it a refresh, you'll delete the cache, so the load balancer and the nginx container we just deployed are now deleted from our cluster and so that yeah that helps clean up unused resources.

A

So here are the upcoming features for phase two, I plan to add authc rules to spinnaker, so all the public users have to deploy to a specific namespace that we've defined in our kubernetes cluster and admin users can create separate applications in a different name space that basically can destroy everything, as we saw earlier with the the cleanup pipeline.

A

I also plan to create some test applications with default pipelines that users can use and add more images to the private repository that we're allowing users to access, and I also plan to create a car which injects these default pipelines into spinnaker, and I also plan so that users can also create their own pipelines in spinnaker.

A

But this sidecar would also auto delete them after a certain period of time, similar to how I auto deleted it, the the resources that users have created, I also plan to install falco onto our kubernetes cluster, which is a security tool that logs any specif suspicious activities, and I also plan to collect some metrics.

A

Thank you for coming to my presentation and if you have any questions, concerns or any suggestions for me. Please feel free to email me or reach out to me on the cdf slack channel. Thank you.