Cloud Native Computing Foundation Kubernetes Community Days Bangalore 2022, 11 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Leveraging Kubernetes For Remote Development Environments

Description

Kubernetes has become the go-to container orchestration system when it comes to deploying microservices-based applications to production. Due to this, being able to replicate production like Kubernetes-based environments during development is becoming tremendously valuable. These development environments are particularly useful because: - They ensure that while developing, developers see the application exactly as it would look in production - Using Kubernetes features like Volume Snapshots in these environments allows developers to replicate huge production databases during development in a matter of minutes - They free up a lot of resources on machines by providing a way to avoid running resource-intensive tools This talk will cover what Kubernetes-based development environments are, why they are useful, and how you can set them up easily using open source tools.
This talk will be beneficial for the ecosystem for the same reasons the adoption of dev environments has been growing - it significantly increases developer productivity! Ever since switching to development environments, developers have reported: - Being able to completely deploy huge applications in minutes by leveraging cloud resources - A super quick onboarding experience for new and existing developers alike - Extremely fast data replication thanks to Kubernetes. A production database of 100 GBs can be replicated in dev environments in a matter of minutes - giving developers access to real data to work with during development After the talk, the viewers would not only understand how Kubernetes is shaping the future of modern-day development but also learn how they can get started using these development environments!

A

Hi everyone and welcome to kubernetes community days bengaluru. We hope you have been having an amazing conference so far. I am ash and I work as a developer experience engineer at octato, I'm joined today by my good friend daniel, who is a scale engineer at honeybook in this talk, we're going to introduce you to the world of remote dev environments and highlight the crucial role kubernetes plays in them before we start the actual talk.

A

I want to give you a brief overview of what we'll be covering so that you have an idea of what to expect we'll start by going over the what of remote dev environments, so, basically what they are and why the need for them even arose in the first place.

A

We'll also explain what role kubernetes plays in them and how it makes these environments possible.

A

Then, once we know the what we'll jump right into action and see a demo on how you can get started using them with the help of existing open source tools, we'll also cover a bit of how they work behind the scenes, so that you have a better understanding of the whole process.

A

The demo we have for you should ensure that you have no trouble replicating the entire thing and trying them out on your own after the talk then. Finally, once we have seen all of this in action, we'll briefly go over the, why and cover why using them could be a game changer if you're looking to develop cloud native applications with that out of the way, let's get started so a bit of a history lesson here, but a development environment refers to the whole systems, tools and processes that you use when writing code for your application.

A

Now, traditionally, this dev environment would be local. That is, everything would run on your personal machine. So, let's say you're a backend engineer working on some web application and you need to make changes to the back end, but you also want to know how they would change the front end.

A

What you would need to do is you would need to bring up the back-end and the front-end microservices locally on your machine and then code out and see how your changes look.

A

Now this wasn't a problem for the applications. We were developing five to ten years ago. It was the norm at that time, but ever since kubernetes came into picture, it enabled us to ship more and more complex applications easily.

A

As a result, we started seeing a change in the way we architect our applications from huge monoliths. We started coding applications which consisted of large number of microservices, and the interactions between these microservices started becoming more and more complex. Not only that, but bringing these microservices up locally was no longer an easy task.

A

Not only did they take a lot of resources from the local machine, but a lot of configuration needed to be done each time you wanted to develop so put simply the bare bones: local dev environment we had been using initially just wasn't able to cope up with this new world of containers and kubernetes.

B

Yeah, so all these things that are just talked about in this complexity that we now see with our development environments and our production environments arose to all sorts of problems right with ever-changing ecosystem of tools and technologies, around system architecture and design. The inner dev loop with that loop of developing building, running testing and repeating is no longer really self-contained.

B

It's gradually becoming harder and harder to even simply validate that everything still works after enforcing long pauses to the dev cycle, waiting on ci pipelines, build pipelines, deploy processes right, so numerous tools rose with the idea of bridging this gap, among them tokyo, docker, compose local, kubernetes and more, but as systems progress and the complexity rises. The toll on machines trying to do it all was just too great, leading to very expensive dev machines that still really don't measure up at times forcing developers to opt out instead of in to using services.

B

We've all heard this. Oh, this service isn't required to run that service I'll. Just leave it off for now making the gap between my local development and the production environment that I'm going to actually go on this code in uh that much farther away- and you know it works on my machine. The dreaded it works in my machine uh is a sentence. We've all heard right. Docker came by for this exact cause out of many, but the build and deployment of containers isn't enough really anymore. The ecosystem is so much more complex than that.

B

We have api gateways, web application, firewalls identity providers, uh content distribution networks, object, storages and many more uh cloud infrastructure and uh services that we're using in our day-to-day production environments today, even when perfectly replicating the container ecosystem to the dot and working actually very hard to do so, production always has its differences and bridging that gap locally is harder than ever right and another thing that arises here as a problem is limited in slow inner dev loops, a key part of developing is focus right. Developer productivity is affected immensely by constant, constant issues.

B

More often than not waiting for the build test deploy flow can throw that dev in a context switch. That is going to affect his productivity and in this day and age, this is just too classy to ignore.

B

So we did try and come up with solutions right, but I call the solutions. We came up with more of half measures than actual solutions only because they don't actually bring back the convenience we had when developing simple applications on local machines, like we used to in simpler times and soon you'll see what I mean. We came up with solutions like relying on ci builds and staging clusters, and while these are really good solutions, we'll help ensure that we don't ship broken code to production, and we really need to do them.

B

They have their own set of problems when you try to use them, while still sticking with your local dev environments. The problems they have is that they are slow and not well connected to the development process right. You always need to wait up for our the staging environment to be up and running with the ci to start running the build process to end and the test to actually run in a sense they appear kind of after you're done coding. Your changes, you'll, make your changes, commit and push them and then wait for that.

B

Build and wait for that pipeline to to finish, and only then see if their work is expected or not. They're useful, no doubt and they solve a different kind of problem, but they don't feel the gap caused in the development phase by modern deployment tools and practices.

A

To add to what daniel said, the solution we need is for the code writing phase and not after it. So this is where remote dev environments come into the picture. The word remote should give you some idea that we would not be depending on our local machines resources anymore.

A

Instead, we'll be leveraging cloud resources, but don't let this word remote fool you into thinking that all we are suggesting is ssh into a remote machine and then using its resources.

A

Daniel already covered the problem with that approach, the modern dev environments we refer to in our talk, utilize, kubernetes clusters.

A

Instead, the way they work is by first bringing up any services you require on the cluster and then allowing you to code your changes and see these changes reflected, live in the deployed version, we'll see this in action and get into the technical details in the next slide, but I think you can now start to see how this solves the problems we were complaining about earlier, since we are using remote machines to power the deployment of the required microservices.

A

The whole thing is just extremely fast leveraging. A kubernetes cluster during development means what we see in the dev phase mirrors exactly what we'll get into in production.

A

Immediate feedback is made possible by the fact that, as soon as you make a code change that change gets transferred to the container running in the kubernetes cluster, and you see the effect of that change in the deployed version. All of this will make much more sense once you see it in action, but uh to come to the problems we were trying to solve.

A

You see how this has two big advantages: over using local dev environments, so just relying on ci and staging clusters, it's better than local dev environment, because your changes, you see them exactly how they would look once you ship your code to production, so no more hidden, surprises and gotcha is waiting for you once you deploy, and this method is definitely more effective than relying on ci bills and staging clusters, because it first is very fast and, secondly, it happens in the actual development phase when you're writing code and not a step after it, so no need to build.

A

Commit push or deploy, it really is the best of both worlds.

A

So before we see this all in action in a demo, let me flesh out some details about how these dev environments work and then explain a bit more about the container replacing step we talked about earlier. The idea in in of itself is pretty simple. The way our dev environments would work is that first, we'll deploy the microservices. We need to develop onto a kubernetes cluster, then in the deployment running in the cluster, we'll replace the container by a different container that contains all the dev tools we might require, like npm python, go compilers, etc.

A

This container also inherits the same secrets, config maps or any other configuration value. The original kubernetes deployment might have.

A

So what this means is that it is ready already set to work with other surrounding microservices and requires no configuration on the developers part. This is also what makes sure that the remote dev environment, exactly mirrors what you will get after shipping to production.

A

With the replace container, we set up a bi-directional file synchronization service, to keep your changes up to date between the local file system and this dev container. This means, whatever code you'll write on your machine, will get synced with the code running in the con in the container part of the kubernetes cluster.

A

So you'll see the change of your code as soon as you hit save on a production like environment. How cool is that? No okay? I know you're wait uh eagerly waiting for the demo time. So let me just go over the tools we'll be using to handle the complicated bits of replacing the container and setting up the file synchronization service, we'll use an open source tool called octet or cli for the kubernetes cluster, we'll use to deploy our micro services we'll go with google kubernetes engine.

A

But the good thing is that octatocli works with any kubernetes cluster, so you can go ahead with whatever is available to you. So let's proceed to the actual demo. Now.

B

um So this is the demo and what we're gonna show here is actually what we talked about up till now, how we can use our own kubernetes infrastructure right, a cluster that I already have using an existing deployment that I already have- and let's say you could trust me that maybe I have that introduction and uh I'm going to show you how I'm going to connect to those uh deployments that I have change code in my own ide and see that changes change live right.

B

So that's the idea of what I'm going to show you we're going to do this using, um like our said, octet cli, uh which is an open source tool to do these things, and also the gk for the kubernetes cluster. So just to show you around, um I have this cluster, I'm connected to it. This guy um I'll show you. um I have a helm chart already deployed. Okay.

A

B

A

I think we can only see the slide right now. Could you show your entire screen yeah? It's up.

B

Sorry um so I have uh a combination already deployed. I have a hum chart already deployed as well. um Within this helm. Shot has right now, two deployments um as you can see. I actually gonna have three here, but you can uh uh disregard this one, so we have an api and a front end. So two micro services, two services and deployments running right now, as you can see these, these are the code parts right here. This is the helm chart just so you can have a look at it.

B

uh The values, as you can see, I have one api one front end and a mongodb running and the app itself just to show show it is a simple movies, app right, so this movie's app, I can reload and I can show you these in the movies. This is like what I can continue to watch all sorts of buttons. Nothing, nothing really works, but it does look really good. um So let's show you how I'm going to change some stuff, um let's start with the api.

B

So if you're looking at the api, I have a simple node.js server running just a simple like api movies, api watching and gets all sorts of data. So let's show you how this works I'll be moving here and what I'm going to do is going to run a potato up.

B

What this does is the really cool thing I've talked about. This actually replaces the deployment that we have right now. The api service deployment with a new deployment called api octato, the what it does is. It has a pod that has the same environment, of course, and everything else is the the the other deployment before, but I connect to it via ssh with remote file synchronizations.

B

uh As you can see, the files are synchronized. Everything is running, really cool and now I'm in this remote plot. So let's do yarn start.

B

Running on start started, uh nodemon running our node right, so let's just uh see that it still works. Everything works as we expect, but now we can see our console logs running right. These console logs they get watching, get api watching. So you can see that we're in the container. Let's show you some changes so just as an idea, let's add a wrap. Okay, so app.get slash api fun.

B

And let's do something cool here.

B

Isn't this fun awesome so now I'm doing something really fun. I change. I change my code and, let's see it happening, so if I go here and I access the api, slash fun route that I just did, isn't this fun so just to prove to you that I didn't do this before and I'm not 90 right now. Let's do this, isn't this really fun?

B

I know surprising refreshing right now and it is really fun right. um So what we did we received right now? We saw that we connected to a remote pod. We changed code locally in my id and without extra anything really we saw those changes being replicated live to our uh kubernetes cluster and just to show you another tiny bit, let's go ahead and do a ui change right, so I'll go to the front end sort find it is another micro service. I have it right here. It's a simple react app.

B

You can see all sorts of words right, so I already have that running and already connected to it just to expend some time. So, let's do some stuff. You see these movies on my list. Let's go here and see movies in my list. I I'm telling you it's the same guys: okay, let's go ahead and change it, so my list is really cool as well awesome as you can see, the changes went up and webpack changed and compiled the resources already and going back here.

B

It's already changed just uh just to show you again a small really thing. I can show you how we use webpack to actually change this live.

B

Okay, as you can see, my list became my list again and finishing off and all this uh let's say I finish developing the only thing I really need to do is exit the remote container right and run potato down potato down.

B

This will deactivate my development environment and bring back the deployment I had before. So if I go back right now, I should give it a second.

B

You can see that my old app is up and running the one I had in my container before um awesome so yeah. So what did you see in this demo? We saw that we had a kubernetes cluster. uh We had a deployment running with my own helm, charts that we already wrote before in my own application and utilize those hum charts in that component cluster to develop these applications live from my local ids, so.

A

Do so, let's get back to the slides stress out some facts? All the changes you saw were made on the local files, but those changes got transferred to the kubernetes cluster and then they affected the real deployed version of the application.

A

So switching to the next slides we just like to wrap up by telling you what advantages you have with this method.

A

So at this point we think, like you can start to appreciate.

A

So uh I think at this point you can start to appreciate how much better this way of developing modern cloud native applications is compared to any of the existing approaches. So right from the very start of the dev process, it just makes everything super convenient when you want to develop. It is extremely easy to bring up all the micro services you require during development using just one command.

A

This frees up developer time spent you in configuring all these micro services and gets you directly to the development phase of writing code in a matter of minutes and if you're already using docker, compose files or helm, charts or any other kubernetes related manifest in production, you can simply use all of those to power, your remote dev environments, so that means no additional configuration required.

A

Then, when you are in the writing code phase remote dev environments give you an environment which exactly mirrors production while showing your code changes, reflect immediately.

B

And all of this together really lets you do a better dev experience right, you're going to develop faster, you're, going to have much less friction uh in the development workflow and you're gonna. Have your inner development loop be as tight as as tight as it could be, and all this really really is part of uh uh I, I guess uh a sentence.

B

I'd like to leave you all with developing for the cloud in the cloud creates a mindset, shift that betters how you code- and I truly believe that if you write code, utilizing cloud resources initially you're going to utilize those resources better right, you're going to do applications that are cloud native in a better way.

B

um So thank you very much for attending this. This presentation- um I was daniel, and this is ours. If.

A

You have any questions, please feel free to reach out to us after the session or later on twitter. Thank you so much.