.Net Foundation On .NET, 12 Feb 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lesser known features of the Cosmos DB SDK for .NET

Description

In this episode, Jeremy chats with software engineer Matías Quaranta about some of the lesser known features of the Cosmos DB SDK for .NET. Matias discusses some useful patterns for managing the lifetime of the client, implementing custom serializers and some other interesting features.

[01:09] - Using the fluent pattern with the Cosmos client
[05:28] - Custom serialization
[11:08] - Working with Dependency injection and Stream

GitHub repo with the sample code
https://github.com/ealsur/ondotnet-cosmosdb

Performance tips for Azure Cosmos DB and .NET
https://docs.microsoft.com/en-us/azure/cosmos-db/performance-tips?WT.mc_id=ondotnet-c9-cxa

Work with NoSQL data in Azure Cosmos DB (MS Learn)
https://docs.microsoft.com/en-us/learn/paths/work-with-nosql-data-in-azure-cosmos-db?WT.mc_id=ondotnet-c9-cxa

Cosmos DB .NET Standard SDK release notes
https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-sdk-dotnet-standard?WT.mc_id=ondotnet-c9-cxa

A

We are talking fluent api's, custom, serialization and streaming API s and cosmos TV. These are underdog features of the cosmos DB net, SDK that you won't want to miss check it out.

A

The cosmos DB dotnet SDK has been out for several months now, but there are some underdog features that are perhaps very valuable, but not as well known. We're kicking off a miniseries today on on.net, where we cover those underdog features and expose them to you, I'm here with Matthias from the cosmos TV team. Thank you thanks.

B

For the invitation, oh.

A

Absolutely so tell us a little bit about what you do on the team. Well,.

B

I'm an I'm, an engineer in the SDK team. So the idea of this talk- and this episode is to kind of shed some light on on api's and features that we have in the SDK that are not really well known and are amazing. In some particular scenarios sounds.

A

Great, let's go ahead and jump in I, don't think it needs any introduction.

B

Awesome so the first one is the Builder pattern or a little fluent pattern. Basically, the we have me come to the code directly, so we have in the Kosmos client.

B

You know the the most common way is just new in app because most client, but this other builder way is creating an instance of the cosmos builder, and then we can start using the extensions that, with extensions to add features into the client, for example, the first one is I'm defining, which is the name of the application that I'm going to use the client on, and this is used across all the requests that the application does. The good thing about the application name. Is it changes or actually adds the information into the user agent?

B

So if we are then using a thermometer or other login solutions, and we want to track which are the requests that are coming from our different applications where we are using the SDK, we can start to filter by the name appearing in the user agent when we define the application name. Other things I can do is I can change the connection mode between gave way and the reg mode I rather say that the cosmos to be a b3 SDK compared with the b2st, can use a stirring mode by default.

B

So if I want to change to Gateway I need to explicitly tell the client to do so. I can oh and.

A

Then the the gateway mode is the the more direct mode. So it's not layered on top of HTTP, no.

B

The contrary, so gay.

A

B

Gateway mo is the HTTP base connection mode and the direct mode is a TCP one.

A

So by default we're doing a direct TCP connection.

B

A

We can figure Gateway we're going over HTTP exactly.

B

Okay, then we have the application region. The idea of the application region is to define where the application is deployed to so, from the ack point of view we can select, which are the accounts endpoints, which are closest to the region you are connecting to. Then we can change if we want the consistency level and we can customize things like the the retrying behavior based on throttle, like I mentioned before this exact same thing. That I did here with the Builder pattern. I can do it with the most know.

B

The normal or, more known way of you know creating the Kosmos client and then creating an instance of Cosmos claim options and then setting on the properties.

A

Sure so it's basically a question of which convention that you like to work with, except.

B

A

Do the same thing with both I personally love fluent API design, I think it's very clean and readable so having an option to be able to do it that way. In fact, if that wasn't there, I might actually be expansion.

B

Itself, so it's nice to have it, and there is another thing we do is we are when we are working working because of the way we normally created at arrays and inside that the database will create.

B

Containers idea is that you can create a container with again, you know doing new container, the phone in the partition, key and so on and so forth, but we also have support for this fluent API, where we are basically saying, for example, which is the default time to live or if I want to find the index in poli-sci I want to define, which is the indexing mode or I want to define the exclude paths or included path if I'm customizing, which fields are index L and which are aren't and I, can even define, which is the unique key that I want to be used with the container.

B

So again, this is a a more friendlier way to define the container and change its configurations, but I, and we can do the same thing with just a normal ap. Is that do a new container and pass all the properties? Okay,.

A

But at the end of the day, what you get back is either Kosmos client or Carlos.

B

Container it's the same result, but we basically take the approach from the developers point of view that it's more akin to what we like. Okay, make sense awesome. So that's the first one, the second one and I'm, showing each cap at the moment, because everything that I'm going to show you all the code, it's on github at the moment, so you can clone it and use it. The second one is custom serialization in the beach.you SDK in Causton CBE.

B

We basically took into hurt listening to our customers and identifying, which were the pain points that they you know most commonly went through, and one of them was sterilization. The B 3 SDK, as the v2 SDK uses JSON donut as the default serializing mechanism, the currency relation engine, but we provide two options and, with this I'm gonna jump again to the code and.

B

Go back so the first one is: we can define options that will customize how the sterilizer works. So we have a Sandow net and we will use it to serialize an item into the database, but that serialize is really says realizers or really we'll have some particular behaviour that I can define again using this builder pattern that we've used in the previous example. We can do the same thing without the Builder, but it since we are talking about the others and Florence initially will show that the builder approach so.

A

I can do some things like by default if values know instead of.

B

A

Property and showing it as as now it's just simply not stored in.

B

The database and we I can define the naming policy if I want to use camel case for a.

A

Capital, first or not exactly.

B

So yeah, in order to test that I have this shader model. Here, it's just a basic class with no decoration, the curators whatsoever, and what it's going to do is it's going to just get the container and create the item based on the C sharp model and the other I'm going to just I'm gonna run both at the end. So just first explain both options, and then we can just run the code and see how it behaves.

B

Okay, the other option is basically, you define your own custom sterilizer, we expose on an abstract base class called custom serializer, and the only thing you need to do is just override the front string and the to a string. The twist three, maybe three is used when you are saving an item into the database and the from a stream is used when the database is responding with the stream of information, and you want to you know, convert that stream into a class.

B

Okay, in this case, in this example, I'm using system test system detects that JSON and its own serializer and I have basically another model, and this one is using, for example, decorators that are only available in the system: dot, X, dot, Jason. So.

A

Am I understanding correctly, the the stream that ultimately gets passed? Cosmos should be JSON exactly. However, I am able to manipulate that so I get a class and I can control if I wanted to, instead of an array to make that just to set a unique properties. Yeah.

B

Whatever I have I think like exactly and you can either use the built in JSON donates utilizer with some custom options or basically take the the abstract class implemented yourself and when you are new in the cosmos, client define your own custom, serializer. Okay,.

A

B

It's again the same code, we are just creating an item and it will use two separate serializers. So let me just basically do the dotnet run and this should create and it's what this runs. Let's go to the usual portal, which obviously expire decision.

A

Refresh that session we are doing live in the portal, cosmos, DB interactions, and it's up- and you know the trick right of moving your mouse clockwise to speed.

B

A

B

A

B

Try are, you sure, looks like a charm okay. So now, I have my two items. If I go back to the console, I can see that it created two items, one with the custom sterilizer and one with the sterilization of serialize eruptions. So if I go to the first one, which is using the custom sterilizer I, can you I can see that the ID untitled, let's zoom this in I, can see that the ID untitled or not? You.

A

B

Okay, awesome: okay: we can see that the ID untitled are using the shades on properties that I was defining in this system takes JSON decorators inside my model and I.

A

B

To the other one, we will see that it is using I go again to the Chaisson donut I was using upper case, and it's now using lower case I need it didn't serialize the property that we have, that wasn't being assigned. So it was basically following the ignore known values and camelcase property naming policy rather.

A

B

It's two ways to deal with custom serialization, so.

A

B

You like chase Internet, you can customize it. If you don't like she's on it, you can build well.

A

This would be great, too if we have a legacy set of classes that were bringing the Kosmos TBN.

B

You're already.

A

Using a serializer format excited.

B

A

To plug that right in and use that yips seems to make a lot of sense yep. It.

B

Does and finally, the last one that I have is basically hundred streams, so there are two major pain points that we've seen you know with users across time. One is handling the Kosmos client instance and the other is working with middleware applications and when I mention Miller world applications is those. There are maybe we're API, where I have a web client or an application. That is talking to my boy API and it's my way, a peekaboo API who talks with Kosmos to be so.

A

It's like a proxy ammos accessing.

B

It through exactly and in previous SDK versions. The only way to do this conversation is, you had to have your model class in your mirror world and you had to use that model to basically serialize the information pass it to consume, to be get the response, serialize it and pass it back. So.

A

We're serializing twice once to get it to the cosmos, DB format once to return it exhale.

B

The other side of a and you would end up creating you know, code in the middle, where which would be probably seizure classes that were mapping payloads coming from the client, but you weren't really using it. It was just in order to do the serialization, so the OB 3 SDK exposes some streaming API, so this is trim ApS will let you basically send streams directly to the cosmos, DB database and receive streams from the Congo to be database and pass it back and forth without really doing any kind of sterilization.

B

So it takes away all the performance that was being taken up when you were serializing and deserializing without really needing to so. Basically, what I'm going to show you is I'm gonna, press f5. This is a normal and simple Web, API asp net Corvo API application, where I have two actions. The read item and I have the save item. Now.

B

If, if we go up here to the controller definition, you will see that I'm getting this cosmos client in. So the idea of this cost client in is that I'm, using the di or dependency injection in asp net core, to define a single tone. Instance of the Kosmos client, one of the most common pitfalls that I've seen users fall into. Is that when you were near creatine, Kosmos related applications, they might end up mooing or pitting multiple instances of the same client connect into the same account.

B

So what we want to do what's a performance optimization is just create one client and reuse it across the lifetime of your application, which.

A

Is very straightforward in MVC applications or web api because, yes,.

B

A

A registration exactly.

B

So I create the instance of the client and I. Do services dot, add singleton. Now the only when I do this approach, the only thing is I'm getting in the controller that costs claim instance. Now there are some users that would like to maybe their account have multiple containers right.

A

B

They would like to have some way to you know predefined, which are the containers they want to work with, because, if I, if I get the client I need to do a client get container and basically get the in in terms of the container I need to know, which is the name of the container that I want to use and we've seen people that they just want to have that those container distances.

B

You know preparation beforehand, based on some logical configuration, so another approach that I've seen is you can create a proxy class, and this proxy can do all the internal logic that you need to. You know pretty pretty, please select or pre, create the container instances and then again use the single pot pattern to pass that proxy around. So you can either pass the Kosmos client or create a proxy instance and habit. Sorry, this is the proxy and have it been injected into the controller's.

B

The main takeaway is make sure to use one client instance across the lifetime. Okay, so the idea is. This is exposing a read item and this is exposing a save item and if you, if you take a look here, we'll see that we are calling create item stream and we are not passing any tea or any type and I'm just passing the body of the request that is coming from the client directly to the cause on TV SDK. Okay,.

A

Skipping the entire serialization.

B

Exactly and if I this is postman, so what I'm gonna do is I'm gonna. This is a simple JSON payload, nothing, nothing fancy, so I click send and this guy is hitting the back end. I will just move forward and I will see that the response contains basically information about the cosmos, TV response from the server. If I click play and I move forward, I can get the headers well. Actually, yes, I can get the response down here. Sorry and I can just get all the cosmos II be related.

B

Headers like, for example, the request charts to see how many are used. That operation took on my cosmos. We account, but the important thing is I'm not doing any kind of serialization on the server, because I really don't care I'm, just servicing a request from coming from the client serving the same in the stream, getting the response, passing it back, I'm not making.

A

It just pure JSON that you sent, and so this will work directly with any type of front-end client exactly.

B

And the same thing can be done with read operations. There is a read item stream acing, which takes the ID and the partition key and I get just a response. For example, if I this is passing an ID that doesn't exist, I'm gonna click send, and this guy here will basically forward and I will see that there is a response of 404 not found because that document doesn't exist. But if I use the ID that I just saved then and I click play again.

B

I will get this JSON document that was stored in the previous step. Nice.

A

B

The important thing is again: this middleman is not applying any kind of realization. It's just taken the stream from the server passing it down, and then it's the client-side enos in this case postman, which basically renders that information or consumes that information. So.

A

It should make it incredibly easy for me to stand up a set of web api's interface with my database. That's just basically a proxy to pass through because I don't want the full client connecting directly from my front end. Yeah.

B

A

End can securely access the web api that can pass through the cosmos DB exactly, but we're not adding those extra serialization and deserialization steps. That's.

B

The point of this this particular topic, most of the people, will basically do something like go into the read item and doing read, item async and then pass a setter type. Some model here right, but the thing is, you need to create that model into your boy API, and if the client for some reason changes the model, then you need to remember to update the model here and you have let me mention before the caveat sauce. You know lower case camel case and all that you.

A

Lose the flexibility of having a non structured or semi structured database.

B

Excited to say exactly: okay, those were kind of like the big tips that I wanted to show on this first episode, awesome.

A

So those have been some underdog features of the cosmos DB SDK. We are going to give you the link to the github repo that has these links. We also have a few more features that are coming up in the next episode, so stay tuned for that, and thank you for joining us.