Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2019 (San Diego), 22 Nov 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cloud Native Architecture: Monoliths or Microservices? - Goutham Veeramachaneni & Edward Welch

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Cloud Native Architecture: Monoliths or Microservices? - Goutham Veeramachaneni & Edward Welch, Grafana Labs

Microservices are the rage right now and for very good reasons. But microservices are not without drawbacks, requiring a complicated configuration and deployment, increasing the barrier to entry for both developers and users alike. This poor user experience can slow the rate of adoption for a project and hinder developers. There is a solution to this problem seeing a lot of success. A single binary app which can act as a monolith but can also be scaled as microservices. Thanos being a great example, where the kickstart is super simple yet can be scaled out as required. The Loki project was patterned after a similar model and we’ve since re-architected Cortex as well. In the talk we will explore how an application can be architected to be both a monolith and microservices, improving both adoption and ease of use while still allowing to scale as a cloud native microservices application.

https://sched.co/UaX6

A

A

So today, I'm going to talk about we're going to talk about micro-services and monoliths and we might actually be advocating for monoliths at cube con, but we'll see so it's not working.

B

We fix this a minute ago.

A

So yeah I'm gone I'm. A Prometheus maintainer I then started working on cortex, which is another CN CF project, which is a horizontally scalable Prometheus, which is extremely micro-services based and then I also maintain Loki and when I have free time which I don't given I maintain three open-source projects. I try to bike just working on.

B

My name is Ed I also work at Gravano labs with Gotham I'm. The tech lead on the Loki project, I try to contribute a little bit to the cortex project and I used to build Java monoliths. So we're gonna talk to you about. Basically, this order, monoliths micro services and mano micro lists, which we just trademarked a couple minutes ago. So I'm gonna give you a little bit of history about monoliths and I'm gonna.

B

Tell you why they're the greatest thing ever and I'm gonna do that through these story about a pet walking service, so my friend and I had decided to build a pet walking service. My friend is not Gotham. In this case he comes into play later. We decided to build this pet walking service and, like anything, we build a website. We build a back-end, we put it all in one repo. It was real basic, real, simple.

B

We needed a proof of concept and it wasn't a whole lot to it right so easy to develop two of us working on it. You know we had the journey on to our monolith like the way most applications would start. Nothing really terribly complicated. You know one team, basically working one person. Sorry, let me read my notes: real quick, easy to develop a single application, so we deploy our application as simple as well right. We put it on one server. We have a simple, my sequel database behind it.

B

We pointed ping da manha, we pointed Nagios at it. Monitoring was easy. Deployment was easy. Updates were easy because it would really care about a lot of other stuff. At the time right, we didn't really care about high availability. We needed a proof of concept we needed to make our dog walking service famous, but then things got a little more complicated, so we found success, walking dogs, but then we found success, walking cats and we found success charming snakes. We were walking Lamas, we had tons of customers all over the world. Our business was growing.

B

We were backed up against um my sequel database and you know we're we're doing all right, like the monolith was still really nice right like we. We had a code base. We could develop from one big repo, but you know we're starting to see people that are having you know: conflicts about working on the same files working in the same space. We didn't really build our app with you know snakes and cats in mind, so it got a little bit trickier. The deployment at scale got a lot harder.

B

We started looking at ansible to do scale deployments, but we had trouble keeping our ansible playbooks running. People didn't keep them updated and people didn't run them to completion. Every time I ran in my instable playbook I got a different error. I had to troubleshoot it. You know we're not really sure like how do we move from here right, like database was getting a little bit trickier to work with.

B

We had to use sticky sessions on our load balancers in order to do some kind of parallel scaling, because our app wasn't really built to be stateless.

B

Additionally, when I give this introduction to this talk to my manager at Cortana labs, Tom, milky he's like how can you give this talk without talking about Conway's law? Like that's great, it's great, Tom, I, don't exactly know what the Conway's law is so went to Google and I. Look I'm like yeah he's right. You know we have this problem too. You know our organization is developing into different business units. We have you know sales. We have marketing, we have training. You know people have to learn how to charm snakes.

B

Everybody wants something from our software, you know and it's getting harder and harder to sort of the lines keep getting blurred. The application keeps you think bigger and we're really really struggling right. So you know I open up the internet and I look at it and it says I should move to microservices. So we went out on the internet. We posted a job offering and we got this guy here.

A

So I was still in school and I wrote a blog post that showed micro services for create, and they gave me VP of engineering, so I dropped out of school and I was like I'm gonna micro-services your world man, and so, if you look at it, micro-services actually started taking off only in about 2015, it was still. It was just starting to take off and I started, implementing micro services everywhere and put like. Why was I doing micro services? Well, simple answer scale yeah, it's just.

A

We raced a bunch of hard bank money and we scale insanely fast. We added millions of users and it's just this one monolith. It's not able to keep up with the scale of that. We are adding we're adding new cities new pets. We are doing to take snake-charming. All of that. But beyond all the Swedish mediator is a bunch of money, so we can actually have more instances. We can have bigger instances. We can also still scale because, of course we are vc-backed, but it's not just that. It's just not scale in terms of throughput.

A

It's also scale in terms of organization. It's it's like, as ed was mentioning there's now a sales team there's now a marketing team, there's a training team building team, there's a team for everything and they were all working on the same code base and the code base was becoming bigger and bigger. They were like five different apps for different things. It was all running off a single codebase. We were trying to release every week, so we cut a release on Monday and by Wednesday the snake charmers.

A

Could snakes are many more so we had to roll it back. It is just getting super untenable. Everyone was trying to get their feature sent. Nobody was trying to test their features and everyone was blaming everyone else that didn't work out. So I was like I'm gonna split this organization into micro services, one for each team or a set of micro. Sorry, services for each team, so building at two microservice snake-charming gets microservice.

A

Dog-Walking gets a big micro service because they're the most popular everyone gets a new micro service. I'm, like you got a micro service, you got a micro service and in the end we turn we ended up. Actually doing pretty well, we hide smart people and everyone was taking care of their own micro services. It was kind of sort of working and then it actually got out of hand. We had a small micro service that was doing addition right, yeah.

A

If you look at the real world. Actually it's kind of sort of true. If you look at mon-sol, that is their micro services path. So, apparently, if you use, you actually are going to hit 1500 different micro services, which is insane and awesome, but I was like. Oh, this is too much I'm going to go back to school finish my degree this is this is not for me and I actually went back to school. Graduated and I joined the fauna, labs.

A

Sorry about that yeah and I joined to form labs and when I joined I was going to work on hosted Prometheus, which is a CN CF project called cortex. It is a pretty simple application, as you can see it just has seven or eight different micro services. That's like I snake-charmer for I can do this. I can work on cortex and I was like. Let me see how how to contribute to it. How to build it run, make I cannot make run, make tests.

A

I can do that and then it said, run mini cube and then run all of these different services and stateful sets all these different things in mini cube. I'm like wait. What I'm in college with a four year old, laptop and I need to run mini cube, wait, I, need to run kubernetes on it and then put seven or eight different applications on it. Just so that I can run a simple query called up what is even happening in this world.

A

It's just super confusing, but anyways as I said, I was pretty smart, I actually ended up, building and working on it and month or two later I went on call that was fun. It's like Oh buddy, so slow, I'm, like okay, okay, I had no clue what to look at. There was like 8 or 10 different things. I need to look at what locks do I look at. Do I, look at all of them. Do I, look at some of them. Why are we debug logging at debug level? What is this new distributed?

A

Tracing thing if it's just getting untenable and I actually wanted to go back to monolith where everything is on a single single process. I can easily debug everything. I know what's going on, and all the logs are super clear and in one place and that's when I read this amazing blog post by segment. That's a really good engineering team- and this was right about the time.

A

I was getting super frustrated, they're, like from hundred problem to children, to one superstar and I was like yes, this makes so much sense like wow, and then I noticed something else. The rise of the mana microts or Micro monoliths. So I think these are they. These applications are the future and they're amazing. So what are they? I did not do that.

A

Anyways, so a mana micro lid is a single binary monolith composed of micro services. What do I mean so? Basically, let me give you an example of a very popular CN CF project called Thanos, which also does hosted like scalable Prometheus, but with different trade-offs. The real thing that I love about Thanos is how simple it is to use. So essentially you have your Prometheus's in different clusters. You want long-term storage. You want to aggregate data between your different Prometheus's. You can use Thanos, you can also use cortex.

A

So basically, you first start running a side car with your prometheus and then the side car by itself does nothing. So you need to add a courier service to it. So this courier service talks to these side cars talks to Prometheus and make sure that it aggregates data from different Prometheus servers. It D duplicates data if there's a chair pair Prometheus pairs, so you are so the first step is ok. I want global aggregation between my three different Prometheus's in 3d and clusters. You add the query layer and then you can decide hey.

A

This is not enough for me. I want long-term storage, you add the object, storage layer which uploads these blocks into the object, storage, and once you are happy with that, you add the compactor layer which actually lets you compact these blocks and object storage into bigger blocks.

A

The thing I really really like about this is it gives you incremental value, and you can start with something super simple that you understand and then you can add one more component that you understand and then one more component and you can come up with a much complex architecture and when something breaks you have confidence that you can fix it, because you actually grew from a small app to something more complex. You gain confidence there and then you went to something more complex, which is amazing, and we realize that this is the user experience.

A

This is the developer experience we wanted. Thanos is a single binary just to go, build Thanos, dot, slash Thanos, and you have Thanos working on your local system and we attempted to do that with Loki. It was super successful, like Loki has been adopted, mainly because it's super simple to use. We don't need you to run twenty different micro services, just one binary and we went back and we are connected cortex to be simple to use.

A

So if you go and look at the cortex readme right now, it looks like this go: build cortex, dot, slash, cortex, that's as simple as it is, and most of my time I'm not running on people. It is anymore and just like doing most of my stuff locally and then push into dev in a different, much bigger kubernetes cluster and testing it at scale there. The yeah, the development experience completely changed and I have a lot more confidence in cortex. Now you will see why but cortex Thanos Loki at really good examples.

A

Similar example is Jaeger. You can just start with the Jaeger. All in one binary run all of you ger as a single monolith, and the moment you realize, okay, Jaeger, 1 and 1 is not scaling. Let me scale the ingest path. You can run a bunch of queries alongside you're all in one and then a bunch of collectors, and then you can also add a bunch of query, query or Jager pods. Slowly, you can grow the complexity of your application. Similarly, I for the elastic is the same.

A

You can independently scale the query, path and ingest path, so these are really good applications that have seen incredible adoption because they are mono micro, LEDs, so Eddie's gonna now talk about what it takes to build a one of my product, so.

B

I fast forward now, I've, given up on dog walking, is a career and I, and the picker fauna, labs and I start working on the loci project and right out of the gate. This is kind of new to me, so this is a single binary application that we have the ability to scale horizontally and, as this project is growing, this is seeing that you know a huge amount of success because it has a really low barrier to entry, but it has a lot of other benefits too, who are able to deploy or sorry to run.

B

You know the application locally we can I can run it from my editor with a debugger. You know it's like all the things that I loved about my monolith, but I can still, you know, have the ability to scale horizontally. So I'd looked at the codebase and I've kind of tried to summarize like how did we get here right like what makes low key? You know, whatever the kind of core components to building an application that has this functionality and the probably first important part is a modular codebase kind of a you know.

B

It's kind of an old word in software right, modular code has been around for a long long time. Books have been written in the 60s and 70s, and you know: there's nothing really new or fancy about it, and you know we're all supposed to do it and it's kind of hard right like like building a application.

B

This modular is not necessarily hard, but knowing where to draw the lines of modularity is, and so you know, one of the things that I think that makes sense in the world of horizontal computing is to build modules around how your application scales so low key and cortex our database applications effectively. So you know we heavily modularize the read and write paths so that they're isolated and that can scale horizontally separately. If you, you know, do a lot of heavy calculation processing, you know scheduling like if you have workloads in kubernetes.

B

Those are generally good module boundaries to define.

B

Another sort of really valuable aspect of the modularity is how you do config management. So all this kind of ties together with a single binary concept, but you have a composable config object. That's made up of all of the config objects of each module, so for cortex you know it has a querier and gesture stores, all these different layers. They keep their configuration within their modules, but we pull all that up into a composed config object, and we do that because that's what allows you to do this sauce, slash, cortex, run with a single file.

B

The single config file contains two config for all of the different modules, and similarly this is the single binary aspect of this, which is kind of a really important part for that ease of use. The ease of entry experience. This is what it lets me run this on my laptop with dot. Slash cortex well lets me run my debugger again I'm, not you know installing docker I'm not running from a docker container or time to attach to a remote process, I'm not running an app server and then trying to attach to that I.

B

Just have a binary and a config file and then, when I get to the point, where I'm actually ready to scale my application out, we manage that through config options, so there's different arguments that get passed into the same binary and effectively internally. We just load the modules from the code that we care about in case we target it to be an adjuster or a distributor.

B

These are terms related to kind of cortex, but it's about reading and writing effectively, and we can add these components as we need to and the modular code base if you notice is still using the same in this case, docker image, because we just wrapped that binary with one image and the same config file, because we're loading the same config into everything and they just read the sections that they care about a big part of what enables this, at least in this case, and there's a few ways to do this.

B

But G RPC is a fantastic way to sort of enable this modularity to go horizontal. So what a couple things that make G RPC really really good for this is the the interface that it defines. It was a big fan of soap back in the day and it laughed off the stage for it, but I love, wisdoms. You know one of the things I loved.

B

It was the wisdom right like we moved to rest and the rest had a waddle, and it was just never the same and GRP see kind of reintroduces that that feature that I really liked, which is this contract. You know, and it's an interface and inside of just a low key, for example, because I'm more familiar with that, we, when we load these modules in the single binary, we're still using G RPC, we just effectively leasing golang. We register the handlers.

B

Essentially, the paths for the handlers are separately, even though they're loaded in a server and local host, and so when that process runs as a single binary. It effectively makes G RPC calls to itself on localhost on a local, socket I'll point out that this is not obviously the most effective way to scale an app right like you're. You know serializing and deserializing to the same process. You have, you know, duplicate memory in places, but you know in the bigger picture.

B

We don't really care that much right like the idea here is ease of use right, like I, want to be able to run it. I want people to be able to run it easily. You wanna be able to debug it when I want to scale it because I'm, you know running out of limits. That's when I start going horizontal.

B

Another aspect we call is kind of airplane mode, so you know in terms of good, like application design, it's really really a good idea to abstract all of your. What I call third-party dependencies through your own interfaces. In this case, like you know, dynamodb or s3, or anything that you have that you're, you know we were writing to or you require, and the airplane boat concept is that you should be able to run your app without internet access or with really really poor internet access.

B

If you have a dependency on an api or a third party, abstract it through an interface and then make a simple implementation of it, something that writes to the file system or something that produces data that you can configure it's worth the time, because this really speeds up your ability to sort of develop this app and in the case of both cortex and loci, they have a you know: a file system, storage, for you know both the objects in the database store and like it's not super performant.

B

You know it's got its limits of scalability, but it kind of doesn't matter because it just lets. You run the thing you know like I, don't need anything I, don't need to connect anything any just a penny counts, I just run the thing you know this is the the monolith world that I really missed right, like the ability to just do something without having to have ten different micro services and run them in containers and getting them all set up.

B

Another thing that I would say is a little. Our opinionated side is the mono repo low-key cortex. They knows these are mono. Repos, you know the the client, the server all of the pieces, everything we you know plug-ins third-party stuff. We just keep stuffing in the same repo. It's just it's been making our life a lot easier, there's kind of a few reasons, I think that that helps one in the modularity sense.

B

In my experiences, when I took a Java app and I decompose it into a bunch of micro services years ago, I put them all in their own repo, they all had their own. You know maven coordinates and files, and you know what I tended to find is like over time as an application evolved and that interface change, and it got to the point where I'm like okay. This should really be two separate modules and then really feel like setting up a new project. That was a huge pain in my butt.

B

You know I had to make a new source control project I had to configure a new maven file, it's a lot of copy and paste and like I, just didn't, do it right, so keeping that code in the same codebase and limiting the sort of you know, effort required to refactor your code base is a big advantage. It's a big advantage for sort of consistency and tagging put a different way. It's hard to see the advantages of breaking these things up into separate repos.

B

So we get back to the cortex story here and essentially, cortex went from this micro services application and, following on the success of projects like Loki and Yeager and Thanos, we've re architected it basically to operate in this micro services or sorry, this single binary but scalable approach, and it ends up working where you can run the thing you connect to your database or your file system. In this case, then you know when you want to run more of them you, you can actually scale the single binary horizontally.

B

There are some caveats in the cortex and Loki case. For that matter, too, we're scaling up and down, and some of the communication is still work-in-progress, though it is possible, it's better if you point it at some kind of a cloud: storage or local storage like cassandra, dinamo or s3, your BigTable or all of those things, and that's part of that abstracting, your third, your storage interfaces or any of your external vendors through your own interfaces. To make sure you can swap these things out.

B

So and then you get to that point again where it's like: okay, now I'm, you know, my monolith works great and till it doesn't right and that's where you start breaking things up, but to Gotham's point like I can run the thing and be confident in it, and you know I can point people at it. I can point new developers at it.

B

They can use the thing and then, when I get to where I'm deploying it you know at the big data scale that I want my company to be at I, can add these targets and configure the services and scale them horizontally.

B

Do you push yeah nice and then we can add other services like the front end and then, ultimately, we get back to this architecture that we have here, which is what we really want. Micro services in many ways really are the way we want to be in you know a kubernetes environment and a cloud native environment.

B

It does work really really well. It solved a lot of the problems that our pet walking service had. It just also created a lot of problems that were already solved right, like you know, the development experience the onboarding experience and the you know, proof-of-concept type experience so in in the end at least we're proposing here with you know some of these kind of ideas of these projects. You can effectively have your micro, lift micro, mono, mono micro, lift we're working on that stuff, but you can have your application be sort of boast of these things.

B

You know you're, making some trade-offs and like how perform it is in one direction or another, but it gives you a lot more. Flexibility.

A

Yep, to summarize being like having a mono repo is super nice. It helps you actually build this single binary that you can do dot, slash, cortex, dot, slash Prometheus and it all works, having modular called G RPC. All of this is create, but always remember like when you're onboarding new developers make it easy for them to run this run stuff. If you have, if someone's coming onto your team- and you tell them hey, you need to run just 17 different services in kubernetes in your own namespace to do any kind of development.

A

You need to build an image, push an image. You do all of this they're going to quit and don't make them do that. Make make the development process super easy and make the adoption process incremental. Like let people incremental e, add complexity, ask the need- and if you do, that, things will change.

A

If you have an open source project, make sure it's easy to use, and it's incremental when you're scaling it out and once we actually move cortex from micro services to a mono micro, let world the number of blog posts, the adoption, the users everything like just like shot up, and it actually super it matters that you make the development experience and the adoption experience super seamless.

A

A

Okay, there's a bunch of questions.

C

Yeah, if you can head to the mic, that'd be great Suzanne. Two quick questions like how big are the teams here, because part of why one might do separate, repos and do this? Is you just don't have disciplined engineers that are capable of working the way you're talking about, but you still need to make progress? Could you talk to those things so.

B

The in terms of like a mono repo I mean if your code is, is modular enough, that your teams are disciplined like it shouldn't in theory matter what repo you're in right, like like, if you're committing in two different files, like you're, all working out of the same file, I, would say. Maybe you've got to look at how your code is architected, so you can break it apart into files where people aren't colliding generally, the source control doesn't really care.

B

How you know big, it is if you have enough ability to keep people out of the same place. At the same time, yeah.

C

Well, I think the question is more so these teams that are working on these mono micro lists how bigger these teams.

A

These teams are actually super small, at least the open source projects that we are working on. Yeah well,.

B

I guess the question is: what would it matter.

C

Well, it matters to communication and discipline and how they understand what they're doing with a small team. It's quite easy to understand what you guys are talking about. Do I'm, not criticizing I'm, just saying there's this thing of artificial forced boundaries to help people be more disciplined where they don't cheat across modular boundaries. That's what I was wondering about.

A

D

So I I'm not a hundred percent clear on what you're advocating for, but what it sounds like is you're saying: build your code in a modular way, almost as if it was a microservices architecture, but in one place and then because it's so modularly build. You can easily break out a micro service. If you have to down the road that what you're saying yes, okay and.

A

Make sure that it's easy to run it locally as a single process single binary, so that people who are new to the codebase? They can quickly run it instead of like and they can debug it by hooking up a debug that locally, instead of having to who can hook up the debugger up to a Cuban, Indies cluster. Okay,.

D

So it's a time when you do break out a micro service. Would you put that in a different repo, nope.

A

I would actually saw, as ed was saying, it would be like dot. Slash, cortex target query your target in gesture. It would be set in the same same binary, but with a different flag or a different command. It could be cortex in gesture or cortex career or cortex all in one. So the new micro service will still be in the same repo.

D

All right, thank you.

E

So you've taken a micro service architecture and turned it into a single binary. One of the benefits of micro services is how easy they are to scale I, wasn't quite clear on what you're doing to make that single monolithic, binary that you've now deployed to kubernetes or wherever scalable, in the same way that micro services are so.

A

Like we are not going to scale a single binary, so essentially, if you can.

B

A

If you look at this, you can just start with running all the components in just the wind one layer and the movement you suddenly notice. Oh queries are a bottleneck now and I can't scale this out without affecting other things. Let me just break the queries into a separate microservice just by adding a new command dot. Slash cortex target query here, and then this is a new micro service that just as querying and then Oh queries are not fast enough.

A

I need to add some caching paralyzing splitting munging, you can add a front-end which is another micro service. So you can incrementally add this complexity and you can incrementally add micro services su scale. So.

B

The I mean the way that actually works is the just the argument on the command line. So when that initiated that module, the argument on the command line determines this actually like how many gr BC endpoints, you register what code is initialized, etc. So there is some kind of glue work upfront that determines how many of those modules are actually active in that binary in order to determine what kind of micro service it is. Okay, thank you.

F

Conceptually speaking, this is like super interesting you're, drawing this wonderful third camp, that we can all pitch fork against, and but you seem to be advocating for like some sort of boundary line between like the monolith and and microservice purism ideals. But what would like? What would your guiding principle be of like how to draw the lines between where you need a totally separate service or separate repo I'm sure there are separate debates? What what guiding principles would you just come up with, based on your experience of okay?

F

That's probably time to start a new command to run completely separate, so.

B

I mean we're bounded by an application, with kind of a purpose right like if you're, architecting and I think something more like like you. How would you run your business right, I like back to our pet walking business right, like you know, I, don't know that you need to be I, don't know that the limitation needs to be the source control like you could put all of your code in one repository companies. Do this. The advantage of that I think is. It does make it sort of simpler to kind of move around.

B

What makes would you run an entire company with one binary I? Don't know that I mean I. Guess you could like I, don't have a great answer for I mean this is why the modularity problem is hard right like like again I would just kind of look back to say like what is the scalable unit of this and like in back to Conway.

B

You know like what are my business units centered around and you know what are my team is focused around I mean I, don't think, there's a limit to how big you could make this I. Don't know that there's an advantage to making it infinitely big unless you regularly have people moving all over the company and all over your teams and all over your codebase kind.

F

Of the same problem of like what is a pod and like how big do you make it, and what do you put in it and all that stuff, but the modular set yeah.

B

I mean, generally speaking, the smaller your module is the better right, the smaller the interface, the better that you know the smaller you can make any compose. This is the micro-service idea right, like the smaller you can make that the sort of in theory, the easier it is to do, rollouts the less likely you're gonna make changes easier just to test.

B

So you know, thank you, yep no time for two more questions. Hi.

G

There a quick question about moving from a monolith to microservice structure. How did you I guess maintain progress over you know, weeks and years of you know, keeping your monolith running, keeping it maintained, putting out new features while simultaneously, you know migrating it into a micro service structure. How did you guess keep crop progress going instead of you know stopping and never getting back on the project.

B

We cheated by having a small project that.

G

There any advice that kind of like really helped you guys on track. I.

B

Mean I guess like, depending on how your your codebase is built. If you can continue like, if you can pull out functionality, start it as a new service, start slowly migrating people to that service and sort of continue. That trend, like sort of define where you want your modules, to be like what your business units are start pulling function now, you're gonna basically be replicating it at some point right until you can in write.

G

B

Your app into I'm, unfortunately I, don't know yeah I've imagined, there's a talk out there on how to decompose your York monolith, but.

G

Okay, so no so no magic pill. Okay, thank you.

H

B

Can't quite hear you I'm afraid sorry.

H

Like it's possible to integrate radius.

B

Integrate them into it, I mean I, would consider those like the external dependencies and I would abstract those through an interface or more like in a Postgres, and it's like it's trickier. If you're using you know lots of like like cortex and loci abstract through reads and writes simpler interfaces, I mean it's not that simple right, but if you can build your application around the the units of like what's your storing and how so that you can implement any storage mechanism behind them.

B

That certainly makes that easier to hide those out like that being said like, if you were app like in the old monolith days and the original story like if you're only a dependency is running, something like Postgres, you can run that locally pretty easily. So that's not a huge barrier to entry and I.

A

Would like to add that we use DynamoDB and memcache D or Redis in cortex, but the single binary, no dependency version has in mem caching and bold DB as an alternative. So you could implement an interface into SQLite yeah and then have everything done locally without having to run this for screws, and this is only for development or for quick testing to check if your code works and all of that, so you can. You can always abstract that away with the node event. No dependencies version.

B

Thanks just give the speakers a round of applause and please fill out your breakout session.