►
From YouTube: Streaming and Batching with the Cosmos DB SDK for .NET
Description
In this episode, MatÃas Quaranta returns to talk to us about some more features of the Cosmos DB SDK for .NET. This time, his covers some interesting features like the estimator API, the change feed processor, running bulk operations and also transactions.
[01:00] - Using the change feed processor
[05:00] - Scaling with multiple processors
[06:43] - using the Estimator API
[09:08] - Implementing transactional batch
[15:33] - Support bulk operations
GitHub repo with the sample code
https://github.com/ealsur/ondotnet-cosmosdb
Performance tips for Azure Cosmos DB and .NET
https://docs.microsoft.com/en-us/azure/cosmos-db/performance-tips?WT.mc_id=ondotnet-c9-cxa
Work with NoSQL data in Azure Cosmos DB (MS Learn)
https://docs.microsoft.com/en-us/learn/paths/work-with-nosql-data-in-azure-cosmos-db?WT.mc_id=ondotnet-c9-cxa
Cosmos DB .NET Standard SDK release notes
https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-sdk-dotnet-standard?WT.mc_id=ondotnet-c9-cxa
A
A
A
B
B
Basically,
the
change
feed
just
to
give
a
small
conceptual
description
is
kind
of
like
a
time
limit
where
we
have
all
the
items
that
have
been
created
inside
the
container
across
time
and
they're
update
operations.
So
we
can
do
with
change.
Feed
is
basically
create
applications
that
react
upon
changes
that
are
happening
inside
a
particular
container
and
for
the
B
3
SDK.
What
we
have
is
something
called
the
change
it
processor,
which
is
part
of
the
B
3
SDK,
and
you
might,
when
I
describe
what
it
does.
A
A
B
A
B
The
idea
is
that
I
will
create
an
application.
This
application
will
again
create.
We
have
a
Cosmos
client
instance
and
I
will
be
fine
and
I
will
actually
define
two
containers,
one
which
will
have
the
items
that
I
will
be
basically
writing
to
and
another
called
the
least
container.
This
container
is
being
used
to
a
store.
The
state
and
the
state
is
basically
kind
of
like
a
pointer
in
time
were
I
were,
which
is
the
current
point
in
time
where
everything
they
changed
again.
B
A
A
B
The
thing
is,
the
code
is
rather
simple:
we
get
the
container,
we
want
to
listen
for
changes
own.
We
call
gate
change
the
processor
builder
again,
a
winter
pattern
with
the
fine
which
is
the
the
type
of
or
actually
the
C,
sharp
type
that
we
want
to
map
our
events
to,
and
we
define
an
a
delegate
or
or
a
function.
This
function
will
basically
receive
the
list
of
changes
that
are
happening
at
a
particular
given
time.
Okay,.
A
B
If
I
want
to
listen
for
changes,
I
get
the
reference
of
the
container
called
the
changing
processor
and
then
I
call
a
start
async,
so
that
what
I
will
do
is
I
will
come
here
to
one
of
my
screens
and
I
will
do.
This
is
basically
pointing
to
the
source
code
of
this
project
that
it's
also
available
in
github
and
will
start
a
processor.
The
processor
will
start
to
listen
for
changes
in
the
particular
container
and
the
other
thing
is
I'm.
Gonna
show
you
this
one
is
the
one.
B
A
B
A
B
This
is
important
because
if
those
instances
grow
or
shrink
because
I
need
you
know
this,
these
changes
are
basically
being
received
in
the
container
at
a
given
rate.
Maybe
a
hundred
hundred
updates
per
second
or
a
thousand
or
a
million
if
I
have
only
one
instant
processing.
Those
changes.
Cpu
is
a
resource
that
can
be.
You
know,
constrained
I
mean
it
could
be.
You
can
have
a
so
much
CPU
on
a
single
instance.
B
A
B
Let's
write
a
thousand
documents
and
the
idea
is
that
the
processors
that
are
on
the
top
half
of
my
screen
will
start
to
receive
those
changes.
So
there
are
some
that
goes
to
one
of
them
instances
and
the
other
goes
to
the
sick
one,
because
there
is
a
load
balancing
happening
inside
the
application
and
the
final
interesting
thing
is
called
the
estimator,
so
it
wouldn't
be
great.
B
If
I
have
all
this
code
deployed
across
many
instances
and
if
I
couldn't
know,
you
know
how,
if
I'm
lagging
behind,
what's
the
rate
at
which
these
processors
are
processing
changes,
so
there
is
one
more
thing
I
can
I
can
show.
You
is
total,
run
estimator
there
or
not
esteem,
a
tour.
So
the
estimator-
and
let
me
quickly
show
you
the
code,
it's
very
similar
to
the
processor,
but
it's
down
here
the
cemeteries.
B
Instead,
instead
it
changed
his
estimator
builder
and
the
estimation
the
delegate
it's
going
to
receive
a
long,
which
will
be
the
amount
of
changes
that
my
current
deployment
of
change.
The
processor
instances
is
lagging
behind
of
the
actual
point
in
the
changes.
That
means
that,
let's
say
my
processors
are,
you
know,
running
processing
the
hundred
changes
per
second,
but
if
the
rate
at
which
at
which
my
documents
are
being
ingested
in
the
DB,
it's
a
thousand
changes
per
second,
then
I
will
I
can
use
estimator
to
detect
that
lag
and
maybe
provision
more
instances.
This.
B
B
A
B
B
Store
procedures
exactly
and
again,
if
you
need
to
burst
in
your
code,
if
there
are
things
that
need
to
change,
you
need
to
remember:
hey
I,
need
to
burn
my
store
procedure
or
remember
to
update
the
logic
in
the
store
procedure
when
I
deploy
me
an
expression
of
the
application
right.
So
the
idea
of
the
transactional
watch
is
to
actually
do
that
within
the
scope
of
your
application.
So,
let's
jump
on
the
code.
What
I
have
here
is
I
have
a
container,
let's
just
press
f5,
to
make
it.
B
You
know
more
easy
and
obviously
make
room
for
the
code.
So
basically,
this
code
will
create
a
transactional
watch
and
we
all
usually
start
with
a
container.
We
have
a
container
and
I
want
to
create
a
transaction
and
watch
for
operations
within
that
container
and
I.
What
I
do
is
I
define,
which
are
the
operations
that
I
want
to
do
inside
that
batch
in
this
case,
I
create
a
show.
I
have
one
show
for
the
on
the
net
and
I
create
three
episodes:
episode,
1
2,
1,
3
4
that
causes
TV
SDK
features
so.
A
B
Yep,
let
me
it's
easier,
I
think
it's.
If
I
do
it
from
the
console,
so
that's
2
today,
but
see
the
batch
and
then
let
me
run
and
like
we
can
use
a
code
so
outside
oops,
so
we'll
press-
and
this
will
show
you
basically
goes
down
to
here-
I
get
the
response
from
the
batch
and
I
can
extract
from
within
the
batch,
because
the
batch
will
say:
ok,
everything
was
ok
everything
committed,
but
then
I
want
to
get
the
individual
operations
out
of
the
batch
to
get
their
individual
results.
B
A
B
I
can
extract
those
those
results
and
basically,
what
I'm
doing
is,
in
this
case
I'm,
showing
the
current
last
update
of
the
show
and
the
next
operation.
What
I
will
do
is
I'm
gonna
update
the
show
and
I'm
going
to
create
another
another
episode.
The
problem
here
with
this
batch
is
that
this
create
item
will
have
the
same
ID
as
the
was
one
of
the
ones
that
I
previously
created.
Ok,.
A
B
The
expectation
of
this
batch
is
that
it
will
fail,
so
it
will
neither
create
this
and
it
will
neither
update
the
show
with
an
updated
last
update
date
with
ok,
so
it
will
basically
extract
the
result.
I
can
show
that
the
result
fail
and
the
replace
operation,
the
one
that
was
updating
the
display
show
for
the
show
it
has
an
a'
status
code
of
fail
dependency.
That
means
that
that
particular
operation
went
through,
but
the
subsequent,
the
one
that
came
before
after
with
a
result
in
e
conflict.
A
B
B
Exactly
and
finally,
what
I'm
gonna
do
is
this
is
a
basically
an
example
of
mixing
and
doing
different
operations.
Now
I
have
3
replace
operations
that
are
updating,
episode,
1
and
2,
and
updating
the
show,
because
we
are
releasing
it
now,
but
the
episode
1,
maybe
the
air
that
it
is
in
seven
days
and
episode.
Two
is
in
14
dates,
but
I
also
delete
in
episode
3,
because
I
decided
that
I
will
only
do
for.
B
The
basically
the
operation
went
through
episode,
3
was
deleted
and
the
story
after
I'm
committing
the
batch
I
can
see
that
the
store
date
is
the
one
that
I
defined
on
the
watch.
So
the
idea
of
patch
is
you
have
this
idea
of
you
know
having
all
these
operations
be
one
or
one
or
nothing?
You
know
all
or
nothing,
and
you
can
programmatically
describe
that
operation.
You
can
put
that
in
your
code.
You
can
version
it
and
you
don't
have
to
rely
on.
You
know
putting
your
code
everywhere,
just
to
write.
A
B
B
B
B
B
Finally,
the
last
but
not
least
important
one,
it's
called
bug,
so,
let's
zoom
in
a
bit
just
to
make
it
more
clear.
So
the
idea
of
bulk
is
and
I
like
to
have
the
code
here.
So
the
day
of
work
is
there
are
scenarios
where
you
want
to
just
you
know:
dump
it'd
be
a
amount
of
data
into
the
database.
You
need
to
do
maybe
a
nicely
processing
of
data.
A
B
You
know
reach
in
that
point
where
you
are
getting.
You
know
all
the
all
the
throughput
out.
It's
sometimes
hard
you
need
to
maybe
rely
on
well
individual
CK.
We
have
the
bulk
executor
library,
which
is
an
external
library,
that
you
need
to
alter
your
code
and
more
dependencies,
and
it
has
some
restrictions
in
the
amount
of
operations
you
can
send
at
each
time,
but
miss
basically
Balki's
is
define
us
operations
that
need
to
just
push
push.
A
B
B
The
idea
is
that,
let's
quickly
zoom
to
the
cold
and
I'm
gonna,
basically,
this
is
this
is
the
same
code
for
I'm
gonna
run
two
different
windows:
one
is
going
to
save
300
K
items
into
my
container
with
bulk,
and
this
I'm
gonna
run
the
same
code
without
bug
and
the
you
can
see
the
basically.
The
difference
here
in
the
code
is
basically
toggling
that
bulk,
enable
feature
on
and
off,
depending
on
a
command
line
attribute
the
I'm
gonna
send,
but.
A
A
A
B
B
A
B
I'm,
basically
creating
all
these
operations
in
parallel
and
then
at
the
end,
avoiding
them
all
in
a
normal
scenario,
when
you
are
using
this
DK,
this
will
basically
trigger
one
service
call
per
operation.
So,
if
I'm
creating
300k
tasks,
each
one
to
say
one
document,
we
do
300k
requests
to
the
backend.
Okay.
A
B
And
if
you
think
about
it
in
terms
of
latency,
those
are
300k
requests.
So
what
the
bulk
feature
is
doing
is
just
detecting
that
we
are.
Basically,
we
have
a
lot
of
operations
concurrently
trying
to
go
through
the
database.
It
will
create
these
batches
in
memory
and
group.
These
operations
by
partition
affinity,
I
kind
of
like
to
think
about
this,
but
it's
kind
of
like
a
train
station,
so
we
will
pull
in
trains.
B
A
B
These
operations
concurrent,
so
we
create
the
strains
we
send
the
trains
and
the
thing
is,
each
train
represents
only
one
request
to
the
database,
so
we
are
reducing
drastically
the
amount
of
frequency
we
are
sending
to
the
other
is
we
are
sending
all
these
operations
and
then
we
are
resolving
and
unwrapping
the
response
and
basically
wiring
up
to
the
original
tasks,
and
this
is
important.
So
in
a
normal
operation
in
normal
C
sharp
operation,
you
do
the
create
item.
You
get
a
task.
B
That
task
represents
a
promise
of
a
service
request,
going
back,
you
know
going
to
a
service
and
back
with
bulb,
we
basically
kind
of
like
light
to
you.
We
give
you
a
task,
but
a
task
doesn't
map
to
a
service
request.
That
task
is
the
promise
of
us
and
putting
your
operation
in
the
Train,
sending
that
train
along
that
train.
Coming
back,
we
are
wrapping
the
the
result
of
the
train
and
worrying
the
result
to
the
task
we
gave
you
originally.
B
And
let's
go
back
and
we
can
see
the
difference
so
the
the
one
with
back
234
seconds
and
the
one
we
know
about
same
code,
two
minutes
and
15
seconds
and
again
this
is
my
small
notebook.
This
code
is
running
locally
in
my
machine
connecting
to
an
account
that
it's
somewhere
in
with
us.
So
obviously
there
is
latency
going
on
and
everything
sure,
but
we
can
see
the
real
difference
between
you
know
sending
all
these
300
items
through
the
wire
with
bulk
and
without
bulk.
This.
B
B
Give
us
your
feedback,
absolutely
awesome.
That's
that's!
Basically,
all
that
I
have
I
really
hope
that
you
can
try
these
features.
We
really
need
the
user
feedback,
because
we
we
we
take
to
heart
what
our
users
are
actually
doing
and
we
listen
to
your
feedback.
So
the
idea
is
that
we
need
you
to
test
it.
We
need
to
find
which
are
the
best
scenarios,
maybe
some
one
things
about
the
different.
It's
an
area
that
we
really
in
think
about,
and
that
makes
you
know.
Let
us
polish
this
feature
even
more
sure.
A
So
this
is
version
3
p3
is
of
the
SDK,
so
go
out,
grab
version
3
of
the
cosmos
DB
net
SDK,
as
Matthias
mentioned.
We
want
your
feedback,
we
want
you
to
use
these
in
your
applications.
Share
your
candid
feedback.
This
team
is
passionate
about
updating
these
SDKs
to
meet
your
needs.
We
just
reviewed
a
ton
of
underdog
features
that
provide
tremendous
value
and
just
have
them
been
getting
the
press
they
deserve.
So
hopefully
the
word
is
out
there
share
this
with
your
friends.