GitLab Incubation Engineering, 15 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Incubation Engineering Jamstack -- Update June 15th 2022

Description

TL;DR: https://gitlab.com/gitlab-org/incubation-engineering/jamstack/meta/-/issues/46

A

A

Hello, this is the jamstack intubation engineering update, I'm janis. Thank you for tuning in today. Let's talk about a sidetrack I have been on recently and I've even made a little slideshow for this.

A

I was recently asked a question that in 2022, I'd expect to have a simple answer turns out. It doesn't not really someone asked me what's the most straightforward way to add comments to an otherwise static site, maybe discuss, but what, if it? If I don't want a proprietary service or if it isn't about comments, there should be a way to create an extremely simple api in minutes. Don't you think.

A

Let's take this question apart for a second all I need to do. This is an api and some data storage.

A

For the api part, there are a million beautiful and fast solutions out there that are adequately sized to the task, but what about the data? Maybe s3- maybe azura, but isn't there a nice little sql way to do this? That isn't proprietary.

A

If I want to implement sql way to do this, I'm usually standing before a dilemma. Let's say I want to make it simple. That means I'll deploy one database on one server somewhere in the world, depending on where my clients are from we're introducing a huge latency with this approach. That's shame. If we have an otherwise static, sitemaps may be distributed, distributed via cdn, and then we have to wait multiple seconds for the api data to propagate.

A

I mean more complex sites solve this with database replication.

A

Now we have super low, latency database connections everywhere, but we end up having to manage the db replication, keeping those instances in sync, even if we use a sas platform to do this. It seems like a huge overhead for say a few hundred comments on a static marketing site.

A

I'm obviously modeling this because there's an elegant solution on the horizon, it's the edge again cloudflare is working on d1. That's an sqlite database running on the edge network.

A

This is solving the exact problem from above it's a distributed database. So it's super performant. It's also very lightweight because it's sqlite, so it's finally, a database optimized for small but distributed data sets.

A

I've talked with the folks from cloudflare yesterday and I expect to get better access to this in a few weeks, and then I can tell you more about how this all works, but with this database architecture possibility in mind, let's get back to the original question that was posed to me.

A

What's the most straightforward way to add comments on an otherwise static site, let's focus on the straightforward for now, because creating an api on cloudflare workers even with d1, is fun, but not always straightforward.

A

And this is where my next idea comes in. Creating an api is something that in 2022 should be so straightforward that you shouldn't have to think twice about it. What I'm proposing now should make that happen. I've been exploring how to build an api generator, let's open a figma model at this point and look at an early draft of what I'm imagining let's say somewhere in the gitlab ui. We have a page like this here I can create a new model. This corresponds to a database table, but the user doesn't have to create that table.

A

Once I've created the model, I can add fields aka columns.

A

All this will not only create a database structure but a graphql api as the interface for it not only the query, but also the four crud methods, as mutations and queries create the model read or query the model update the model delete the model here in the ui. We could maintain various permissions for each model within the api who can execute each of the mutations.

A

Only me gitlab, admins or maybe even include a short javascript function for authorization.

A

Now this is how I imagine the architecture behind it once the user creates that model in the gitlab ui, a job will create the migrations for the db.

A

At the same time, we will generate javascript app from those model definitions and deploy that to a cloudflare worker at app interacts with d1 for data storage and has a graphql api as a front-end. This is mostly invisible to the user.

A

A self-hosted client could connect their own cloud flag credentials to get the same functionality.

A

The idea is that this generated worker is also source, controlled, mit licensed and accessible to the user. If at some point the application outgrows the capabilities of the generator, they should be able to continue to maintain their application by hand. So they're not locked into this service.

A

This is an early idea and I'm extremely open to feedback at this stage, please feel free to reach out to me with ideas or issues currently trying to extract what is the mvc in this idea. So if you have a use case to dog food, this, please let me know, I'm hoping that experimenting with this idea will also spawn ideas around how to implement serverless functions for pages and other approaches on how to connect with cloudflare.

A

Now we're really short on time right now, for this week's update, so next time I'll show you the poc app I've been working on so far until then. I hope you enjoyed this update and see you next week. Bye.