►
From YouTube: Scalability Team Demo - 2021-09-30
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
B
A
Yeah
there's
this
yeah,
it's
it's
it's
something
that
has
been
on
my
mind
for
a
while,
and
I
had
a
bit
of
downtime
and
I
thought
just
going
to
see
if
I
can
get
this
working
and
it
looks
like
it
works.
A
So
this
started
when
a
while
ago,
matt
and
igor
did
an
investigation
into
cpu
spikes
on
one
gitly
server,
where
they
noticed
that
sometimes
it's
at
100
cpu,
and
that
a
lot
of
it
is
because
of
system
calls
in
italy
and
in
git,
because
it's
all
about
the
final
tags
rpc
in
that
case,
so
there's
a
repo
with
a
lot
of
tags
and
something
is
fetching
those
tags
in
a
loop,
and
it
turns
out
that
gitly
implements
this
type
of
rpc,
where
it
has
to
yield
many
objects
in
a
very
inefficient
way.
A
A
Is
there
a
way
to
do
this
better
and
actually,
one
or
two
months
ago?
Somebody
also
asked
me
if
somebody
from
the
kitly
team
asked
me
if
I
had
any
ideas
of
how
to
make
final
attacks
more
efficient,
because
it
was
something
that
kept
coming
up.
So
I've
been
walking
around
with
this
idea
and
now
I
finally
thought,
let's
put
something
together,
and
it
turns
out
that
we
can
do
better.
A
So
I
made
a
benchmark
where
I
took
the
gitlab
or
kit
lab
like
a
local
clone
in
gdk
and
in
a
rails
runner,
I
fetch.
I.
I
built
an
array
with
all
the
tags,
which
is
what
the
client
side
of
final
tax
looks
like
and
then
my
benchmark
was
to
do
that
100
times
in
a
row
and
see
how
fast
it
goes
and
with
the
status
quo
that
was
taking
59
seconds
for
100
calls
where
we
built
an
array
of
old
attacks
in
github
or
kit
lab.
A
A
Well,
I
can
actually
just
show
you
that
what
the
benchmark
is
so
the
benchmark
finds
project
23.
It
takes
the
role
repository,
so
there's
no
caching,
and
then
it
says
repository.tags
100
times
so
that
that
makes
yeah.
A
So
it's
kind
of
I
think
it's
a
reasonably
end-to-end
benchmark,
but
it
doesn't
include
the
overheads
of
if
you
would
make
api
calls.
You
also
have
json
serialization
on
top
of
that.
A
So
that's
nice
because
that
means
it
goes
it's.
So
this
is
single
threaded
in
a
loop
and
it
means
that
goes
six
times
faster
and
I
also
took
flame
graphs.
So
after
I
did
that
so
the
benchmark
is
just
100
calls,
and
then
I
turned
on
another
script
where
I
do
the
same
thing,
but
with
an
infinite
loop.
So
then
you
enter
a
steady
state
of
single
threaded
traffic
and
then
I
run
a
30
second
profile
to
see
what
the
server
is
doing
and
in
those
profiles
you
see.
A
So
this
is
one
from
the
status
quo.
I
think
you,
you
see
yeah,
it's
probably
interesting
to
look
at
the
sales
of
that
for
a
moment,
so
in
the
status
quo,
ruby's
actually
not
doing
a
lot
of
work,
but
italy
is
doing
a
lot
of
work
gets
doing.
A
lot
of
work
and
prefect
is
barely
visible,
but
the
thing
I
was
focusing
on
was
the
kit
and
the
gitly
part,
because
that
runs
on
the
gt3
server
and
that
is
not
horizontally.
A
Scalable
prefect
is,
and
it
prefect
is
also
not
the
biggest
part
of
the
pi
here.
So
in
my
table,
I
sum
the
number
of
samples
for
these
two
things
and
there.
So
the
status
quo
was
in
30
seconds,
4
500
samples,
either
gate
or
italy
was
doing
stuff
on
the
cpu,
and
when
I
put
this
improved
version
or
this
alternative
implementation
in
steady
state-
and
I
take
a
30
second
sample,
then
git
and
gitly
are
together.
A
3
300
samples
on
the
cpu,
but
if
you
take
into
consideration
that
it's
doing
more
calls,
that
means
it
is
10
samples
per
coal
instead
of
89
samples
per
coal,
so
that
means
9
less
cpu
workloads.
I
think
that
means
90
9
times
less
cpu
workloads
so
from
89
to
10
on
the
kitly
server.
A
C
A
B
B
A
C
A
Well,
the
funny
thing
is
that
in
the
status
quo,
you
can
actually
see
that
prefect
is
doing
very
little,
and
that
is,
I
think,
because
the
batching
is
working
well.
Prefect
has
to
copy
very
few
very
few
grpc
messages,
and
if
I
look
at
the
improved
version
and
its
profile,
then
prefix
has
more
work
to
do
and
gitly
has
more
work
to
do.
A
B
Doing
more
work
yeah,
it's
yeah
working
working
efficiently,
not
what
is.
A
The
thing
not
hard
yeah
and
it
could
even
be
made
more
efficient
because
well,
you
can't
see
anything
here,
but
I
know
that
gate
is
using
a
four
kilobyte
buffer.
On
its
end
and
in
other
scenarios
I
tried
to
git.
Currently
doesn't
let
you
make
that
buffer
breaker?
But
if
you
do
you
also
reduce
this
call,
so
I
think
we
can
probably
squeeze.
A
About
how
it's
done,
what
what
the
magic
is
or
well
what's
different,
but
you
have
go
ahead
well,.
C
My
my
my
the
sort
of
the
trend
that
I
see
here
is
that
we
are
taking
all
the
grpcs
and
the
most
the
ones
that
are
the
most
heavy
and
the
most
expensive
from
a
grpc
point
of
view,
perhaps,
and
turning
them
into
binary
streams
of
of
a
sort,
and
obviously
we
can
kind
of
do
this
one
by
one
by
one.
Is
there
like
an
alternative?
C
That
is
something
like
grpc
but
is
like
a
lot
more
efficient
where
we
can
get
the
benefits
of
of
because
obviously,
grpc
has
huge
benefits
in
terms
of
like
you
know,
understanding
the
the
structure
of
the
data
coming
to
you
and
repeatability
you,
you
know
what
I'm
talking
about,
but
obviously
it's
got
a
massive
downside
as
well,
but
it's
kind
of
like
we
could
say
we
can
turn
everything
into
assembly
and
it'll
be
much
faster
downside
to
that
as
well.
A
A
Byte
streams
yeah
in
that
case
we're
taking
rpcs
that
already
are
byte
streams,
they're
just
byte
streams
encapsulated
in
grpc
messages
and
if
you're
transferring.
C
A
So
that
is
there
we're
saying
we're
going
from
a
byte
stream
encapsulated
in
grpc
to
a
byte
stream
in
yamax,
and
that's.
We
expect
that
to
be
much
better
by
the
time.
We're
done
in
this
thing
we're
using
still
using
a
byte
stream
encapsulated
in
grpc,
so
we
could
cut
the
improvement
from
the
other
thing.
A
On
top
of
this,
if
we
really
wanted
to
except
yamax,
is
a
go
library
and
there's
only
a
go
implementation,
so
we
can't
really
use
the
stuff
we're
building
for
the
git
fetch
efficiency
project
for
this,
unless
we
reimplement
yamax,
which
was
yeah.
But
of
course,
if
we
had
something
that
was
not
grpc
and
that
was
primarily
oriented
towards
byte
streams,
then
both
of
these
things
would
work
well,
but
just
by
putting
a
byte
stream
on
encapsulated
in
grpc
messages.
C
A
There's
a
git
command
called
cad
file
and
that's
as
a
batch
mode,
where
you
give
it
a
bunch
of
object.
Descript
object,
names
on
standard
input,
one
per
line,
and
then
it
dumps
them
out
on
standard
outputs,
and
that
is
the
format.
So
that
has
a
header
line
that
you
can
parse.
So
you
just
read
until
the
new
line
and
you
break
it
on
the
spaces,
and
then
you
know
yeah
next
and
then
that
headline.
A
B
A
A
So
the
format
of
the
stream
is
the
more
or
less
the
output
format
of
git
cut
file
batch
and
and
that
format
is
easy
to
parse.
It
is
one
header
line
like
I
said
one
header
line
followed
by
a
new
line,
so
you
can
read
that
with
buffered
io
and
then
you
have
a
length
prefix.
So
you
know
that
the
next
x
bytes
are
the
the
raw
binary
representation
of
the
git
objects.
A
A
Basically,
a
text
format
that
you
can
parse
with
awk,
it's
a
it's
a
very
friendly
to
parse
it
it's
not
a
difficult
format
to
parse,
but
that
is
actually
one
of
the
disadvantages
that
I
list
listed
here,
which
is
that
for
this
to
work,
the
client
side
needs
to
have
this
parser
for
the
get
object
formats
and
currently
gitly
has
a
parser
for
that
which
we
had
to
build
there,
and
I
hacked
one
together
that
works
well
enough,
but
I
didn't
write
a
test.
Suite
so
I'm
sure
it
does
bugs.
A
Yeah-
and
I
I
think
that-
and
this
is
a
bit
of
a
crazy
idea-
because
we
made
this
design
choice
early
on
that
we
parse
on
the
gitly
server
and
we
return
structured
objects
and
we
use
protobuf
for
the
structure
because
well
we
have
protobuf,
it's
grpc,
that's
what
that's
what
you
do
in
grpc,
but
all
that
parsing
happens
on
a
non-scalable
resource
and
it
can
just
happen
just
as
well
on
the
clients
and
the
other
problem
is
that
because
of
the
parsing
parsing
is
just
an
intensive,
cpu
intensive
job
and
what's
happening
now.
A
A
C
C
Design
choice
like
it's
it
you
want
to
encapsulate
your
your
you
know,
because
that
means
you
can
replace
the
back
end
of
git
with
you
know,
other
inputs.
A
Now
but
the
question
of
why
are
destroyer
commit
structured?
I
think
it's
just
a
natural
design.
I
think
it's
the
most
obvious
design
like
if
you,
if.
A
An
api,
a
json
api
for
getting
commits
out
of
gitlab.
Then
you
don't
get
one
big
blob
in
the
native
kit
object
format.
You
get
a
structured,
json
object
that
is
easy
to
get
fields
out
of
it's
just
easier
to
consume.
For
the
a
structured.
D
So
it's
a
case
that
it
was
a
reasonable
design
choice
at
the
beginning.
It's
just
that.
It's
gotten
to
the
point
that
we
can't
that
from
a
scaling
perspective,
it's
just
not
the
best
choice,
anymore,.
A
Yeah
it's
that
with
what
I
know
now
I
would
say
it
is
not
the
most
efficient
choice
and
it's
yeah.
It
forces
you
down
a
path
it
and
andrew
asked
about
batching.
So
we
we
create.
These
go
objects
one
for
commit,
and
then
we
batch
them
before
we
send
them
out.
But
that
means
that
we
need
to
allocate
memory
for
each
of
these
objects.
Well,
in
order
to
parse
these
objects,
we
need
to
keep
allocating
new
buffers
and
we
can't
reuse
them
until
the
whole
batch
goes
out.
A
So
with
a
batching
interface,
it's
harder
to
reuse
memory
and
you
need
to
worry
about
where
the
boundary
is
and
if
you're
just
copying
bytes,
then
you
don't
care
where
the
binary
is.
You
just
say
a
bite.
The
by
the
boundary
can
be
at
any
byte.
You
send
out
the
message
you
go
to
the
next
one,
so
it
adds
all
this
complication
to
to
italy
and
with
where
we
are
now
it
added
up
to
make
things
slow.
C
C
You
would
have
a
dom
and
you
would
build
up
your
your
objects
and
you
generate
your
stubs
and
everything
like
that
and
exactly
it
was
basically
just
grpc
before
grpc
right
and
then
you'd
populate
it
and
then
at
some
point
the
service
would
be
scaling
up
and,
and
you
would
have
a
big
problem
with
the
performance
of
the
server
and
then
at
that
point.
C
The
nice
thing
that
you
had
in
the
rpc
times
was
that
you
could
replace
it
so
that
you
weren't
doing
that
part
of
creating
an
object,
an
array
of
objects
and
then
populating
it
and
then
passing
it
to
the
parser.
You
would
just
switch
over
to
a
jack's
emitter
right
where
you
would
just
you
know,
basically,
centered.
B
C
And
it
would
stream
those
down
and
then
you'd
get
like
a
100x
speed
up,
which
is
exactly
what
you're
doing
here.
But
the
nice
thing
is
that
the
cert,
this
the
client
side,
would
kind
of
be
the
same,
because
the
client
didn't
have
to
you
could
kind
of
independently
change
those
two
sides,
and
then
the
client
could
actually
switch
over
to
a
jack's
parser
as
well.
And
then
you
get
the
speed
up
on
both
sides.
A
It's
not
exactly
the
same
problem
because
with
the
stream
parser,
the
problem
is
that
you
don't
want
to
build
this
huge
intermediate
state
when
you
can
do
the
job
in
the
stream
yeah.
C
B
A
Go
much
faster,
but
the
difference
here
is
just
not
parsing
at
all.
It's
as,
if
imagine
you
had
a
log
for
json
lines
and
you
wanted
to
send
that
somewhere
else
and
you
parse
each
json
line
of
object
in
your
programming
language
and
you
dump
it
back
to
json
before
you
write
it
into
a
socket.
It's
faster.
If
you
never
parse
it
in
the
first
place,.
C
Yeah
I
mean
I
still
think
that
there
is
a
benefit
overall
to
to
having
an
abstraction.
You
know
like
say
you
wanted
to
change.
Git
form.
I
don't
know
you
know
it's
it.
It's
a
sound,
it's
a
sound
abstraction
to
to
to
have,
and
if
you
look
at
every
other
grpc
service,
you
know
we're
not
just
sending
back
raw
files.
It's
a
set.
C
We
we
use
that
same
way,
because
it's
a
sound
way
of
building
things
that
allows
you
to
to
to
change
your
server
without
always
impacting
your
client
and
and
that's
a
a
good
thing
like
architecturally,
but
I
think
in
this
case
you
know
the
benefit.
Is
there
an
advantage
in
the
fact
we
get
value
from
kind
of
dropping
that
abstraction,
but
I
don't
think
it's
like
automatic
that
you
should
drop
it,
but
in
this
case
no.
A
No,
like
I
said
it
is
the
it
is
a
very
it
is
the
natural
design
choice
to
do
what
we
did
but
yeah
for
the
sake
of
efficiency.
It's
it's
just
not
always
ideal.
C
B
It's
this
thing
that
would
like,
what's
what's
causing
it
to
be
so
much
slower?
Is
it
the
the
the
deserializing
or
like
the
parsing
on
the
italy
side
or
the
putting.
A
A
I,
the
unbuffered
io,
is
the
biggest
is
the
biggest
problem.
So
that
means
that
you
have.
You
have
one
git
process
that
is
creating
a
list
of
all
the
tags
and
what
gitly
does
now
is
it
reads
one
item
of
the
list,
so
that's
a
system
call
and
then
it
writes
it
into
another
process,
and
then
that
returns
the
tag
and
then
it
reads
directly
reads
it
back.
So
that's
two
more
system
calls
and
then
it
parses
that
and
it
puts
it
in
a
grpc
message
and
throws
it
on
the
batch.
A
And
the
difference
is
that
now
we
have
the
git
process
that
generates
the
list.
It
talks
directly
to
the
via
pipe
into
the
git
process
that
fetches
the
objects.
So
it
writes
still
one
line
at
a
time,
but
they
come
out
as
four
kilobyte
chunks.
So
it
only
does
the
right
every
four
kilobytes
and
then
the
process
that
generates
the
objects
also.
A
C
C
Right
and
people
use
find
all
tags,
because
we
also
have
problems
with
find
all
tags
in
redis
and
the
cache
that
we
have
and
the
amount
of
data
that
we
transfer
in
and
out
of
that
cache
and
what
it
is
is
is
somebody
needs
to
display
a
page
and
it's
got
a
list
of
tags
on
the
page
and
they
do
find
all
tags,
and
you
know
in
their
mind,
they're
thinking
that
there's
going
to
be
10
tags
on
the
repository,
but
there's
30
000
tags
and
it's
putting
pressure
on
redis.
It's
putting
pressure
on.
C
Obviously
on
italy,
it's
expensive
for
the
application
and
use
lots
of
memory
and
most
of
those
cases
like
a
user.
Can't
you
know
if
it's
a
obviously
there
are
merge,
requests
and
sidekick
jobs
and
stuff
that
might
need
all.
Thirty
thousand,
but
most
of
the
time
a
user
doesn't
need
to
see.
Thirty
thousand,
you
know
so
it's
almost
like
they,
they
shouldn't
be
yeah.
They
can't
see
it.
C
There
shouldn't
be
a
way
to
call
that
endpoint
from
the
web,
because
it
means
that
something's
gone
horribly
wrong
in
the
architecture
or
the
design
of
a
of
a
controller,
but
it
is
used
all
over
the
show
and
it
causes
us
a
lot.
B
I
think
I
think
that
the
the
reason
that
it's
used
like
this
is
because
there's
no
nobody
has
built
something
like
a
paginator
for
tax,
yet
yeah.
A
B
C
And
it's
yeah,
it's
redis
as
well.
You
know,
there's
so
many
things
that
you
know,
because
we
have
that
z-set
implementation,
that
checks,
existence
of
tags
and
everything
there's
so
many
things
that
are
really
expensive,
because
people
assume
that
there's
like
10
tags
on
a
repository
and
it's
only
going
to
get
worse
as
time
goes
on
right
people
never
delete
tags.
A
Yeah,
this
has
been
broken
forever,
so
yeah
that
it's
it
can
and
should
also
be
improved
on
the
on
the
client
side.
B
The
thing
that
I
was
reviewing
on
the
client
side
was
a
high
deadline,
exceeded
errors
for
final
tax,
basically.
A
Yeah
well
thanks
anyway,
for
for
listening
to
this,
I'm
not
really
sure
what
to
do
with
it,
because
we
have
lots
of
other
projects
going
on,
and
it's
also
quite
the
radical
departure
of
how
the
rest
of
kittley
works.
So
I'm
not
sure
if
it
would
be
easy
to
sell
this
to
the
kittley
team
or
where
this
even
fits
into
our
other
priorities.
But
I
thought
it
was.
I
wanted
to
see
if
it
worked,
and
so
I
can
happily
say
that,
yes,
it
works.
D
It
might
be
that
this
is
one
of
the
things
that
we
we
keep
in
the
back
of
our
mind,
for
when
we
start
to
have
problems
in
this
area.
We
know
that
this
is
something
that
we
can
do
to
alleviate
it,
because
I
think,
with
all
the
other
things
going
on,
I
I
don't.
I
agree
with
you.
I'm
not
quite
sure
how
we
would
get
this
some
priorities
to
do
unless
it
was
very
like
a
very,
very
clear.
We
have
to
do
this
or
that's
going
to
explode.
A
Well,
one
practical
thing
I
can
say
is
that
there
matt
created
an
infradev
issue
about
these
final
tag
problems.
So
we
could
take
this
as
a
possible
solution
to
solve
to
resolve
that
infra
death
issue,
but
that
doesn't
tell
us
who's
going
to
do
it
and
when.
B
B
C
Branches
and
find
all
tags,
is
it
branches
or
refs
whatever,
but.
C
B
C
Whatever
that's
become
since
I
last
looked
at
it
a
long
time
ago,
but
basically
like
when
you
do
the
find
all
tags
from
the
caller's
point
of
view,
the
the
abstraction
remains
the
same.
It
basically
gives
you
an.
B
A
No,
no,
so
you
don't
it's,
I
it's
it's
almost
all
hidden
and
gets
declined
right
now
and
not
in
a
completely
natural
way.
But
that's
where
the
that's
where
the
response
stream
comes
in
and
yeah
and
an
array
of
git
tag,
gitlab
git
tag,
objects
comes
out.
A
A
Yeah,
I
and
I
I
don't
know
how
people
how
a
wider
audience
would
feel
about
this
concept
of
not
emitting
structured
data
or
emitting
data
in
a
custom
yeah
as
a
byte
stream
like
this,
but
anyway
I
I
just
wanted
to
like.
I
said
I
wanted
to
share
it,
and
I
think
we
have
enough
other
things
going
on
before
this
before
we'd
have
time
for
this.
D
Now,
thanks
for
taking
us
through
that,
it's
it's,
I
was
quite
surprised
by
the
difference
in
the
results,
so
it
was
good
to
see.
A
One
of
the
weird
things
by
the
way
was
that
I
just
sorry
just
one
more
thing,
but
I
started
building
this
and
at
first
it
was
twice
as
fast
and
I
thought
wow.
This
is
amazing,
and
then
I
started
fiddling
with
the
ruby
codes
and
then
it
went.
I
got
it
three
times
faster
after
that,
and
then
I
ended
up
at
six
times.
C
C
A
Yeah,
it
might
make
sense
to
move
it
over
to
get
that
board
gitlab,
because
that's
what
the
the
infradev
issue
lives.