►
From YouTube: GMT 2017-10-18 Performance WG Meeting
Description
Agenda and Notes:
https://docs.google.com/a/mesosphere.io/document/d/12hWGuzbqyNWc2l1ysbPcXwc0pzHEy4bodagrlNGCuQU/edit?usp=drive_web
B
C
A
Dimitri
was
working
on
those.
It
looks
like
he's
not
in
the
call,
so
I'll
just
describe
what
what's
being
done.
So
the
first
thing
was
that
we
added
support
for
arenas
and
we
started
making
use
of
those
in
the
install
handlers
in
the
process
where
we're
taking
constant
messages,
so
that
just
provides
a
bit
of
an
optimization
there.
A
A
A
A
E
C
E
I
basically
put
together
some
benchmark
for
the
agent
rich
broom
attrition
this
week
and
the
results
and
hasn't
been
conclusive,
but
I
think
it's
it's
a
good
start.
We
can
I
have
WEP
which
I
can
I
can
share
shortly
after
and
I
guess.
I
can
guess,
gather
some
feedback
here
by
just
talking
about
it
briefly.
So
the
benchmark.
E
Is
basically
having
a
real
master
with
a
bunch
of
fake
agents
which,
because
even
for
a
full-fledged
mock
slave
class,
it
would
involve
a
lot
of
states
recovering
and
I'm
complexity
there.
So,
basically,
what
I've
done
is
just
to
create
a
a
prototype
process
which
is
called
a
test
slave,
which
does
only
two
things.
One,
it's
consent
read
register
message
to
the
master
and
it
it's
installing
a
instant
is
installing
a
handler
for
slavery,
registered
message.
E
E
But
at
least
this
captures
the
path
from
that
point
on
to
the
next,
a
couple
of
asynchronous
processing
steps
where
it
consults
the
authorizer
and
registry,
for
example,
and
then
does
handle
processing
all
of
the
states
sent
over
by
rivers.
Your
message
reverses
your
slave
message,
so
it
captures
all
of
that
so
right
now
the
knobs
are
the
number
of
agents
to
to
register
with
a
master
and
then
just
try
to
simulate
what
the
real
slaves
done.
E
There's
the
the
real
agent
captures
a
number
of
frame
work
number
of
a
number
of
frameworks
on
this
agent,
and
there
are
tasks
running
on
these
framework
running
from
these
frameworks
and
also
they
are
completed
frameworks
and
they
are
tasks.
History,
including
completed
tasks,
so
I'm
per
hosts
I'm,
giving
five
frameworks
and
for
each
of
these
framework,
I
give
five
tasks
and
also
I,
give
50
completed
frameworks
and
I
give
five
tasks
per
these
completely
frameworks.
So
just
a
few
simple
knobs
to
simulate
the
behavior
and
so
far
the
result
hasn't
been
all
that
you.
F
E
Just
so,
first
you
have
a
master
and
then
I
create
you
know
thousands
or
tens
of
thousands
of
agents.
These
were
like
fake
agents,
which
are
basically
just
actors
I
to
simulate
the
slave
object
and
by
simulation
I
mean
it
will,
when
I
create
it,
it
will
send
a
really
register
slave
message
to
the
master,
all
of
them
without
the
real
back
off
and
all
that
without
retries.
F
E
A
A
E
E
F
E
It's
going
to
mark
a
future
as
being
completed
so
I
basically
have
four
two
thousand
agents:
I
have
two
thousand
futures
I'm
doing
a
weight
on
all
2,000
of
them
and
I
stopped
watch
once
that,
all
of
them
all
of
the
futures
are
completed.
So
that's
the
measurement
and
it
took
a
minute
to
to
read
register
to
process
fully
process.
All
of
these
and
and
I
I
cannot
see
any
difference
between.
E
I
haven't
which
I
totally
do,
but
so
with
flame
graphs,
I
think
a
fly
and
I
I
told
Ben
that
our
environment
has
some
problems
which
basically
I
Jenna
I,
try
to
generate
ride
to
generate
flame
graphs
from
our
production
assistants.
But
the
the
graph
came
out
with
the
whole
bunch
of
unknown
symbols,
and
we
know
what
it.
What
is
the
problem,
but
then
right
now
we
cannot
fix
it,
but
still.
E
Graff
seems
to
me
can
only
tell
you
this
much
because
previously
with
even
with
that
graph
with
a
bunch
of
whole
bunch
of
unknown
symbols,
we
could
see
protobuf
processing
being
a
very
wide
bar
and
so
I
guess.
It
is
a
known
fact
that
these
copying
or
constructing
is,
is
taking
a
lot
of
taking
a
lot
of
cycles.
I
guess.
E
F
A
good
question:
this
is
the
first
time
I've
attended.
One
of
these,
so
I'm
probably
missing
a
bunch
of
context.
But
what
is
what
is
the
context
for
this
study?
Was
there
some
behavior,
you
observed
in
you're
like--you're,
so
you're
registering
2,000
or
20,000
agents?
Were
you
seeing
slow
agents,
registration
and
that's
what
motivated
the
trying
to
figure
out
what's
slow?
Is
that
what's
going
on
right.
E
So
we
have
already
done
a
bunch
of
optimization
which
improved
the
failover
time
drastically,
but
previously.
So
what
motivated
this
in
front
of
their
beginning
was
when
the
master
fills
over
the
agent
would
all
start
reconnecting
registering,
and
but
it
would
take
sometimes
a
very
long
time
for
them
to
be
fully
to
be
fully
registered.
I,
see
basic
things.
E
Right
so
in
in
this
benchmark,
and
so
that's
why
I'm
open
for
feedback,
but
in
this
in
this
version
it
just
each
agent
would
just
send
one
rate
registered
slave
message
and
wait
for
the
response,
which
is
the
slavery
registered
message.
It
wouldn't
do
back
off,
it
doesn't
do
rate
rise
and
one
because
I
found
it
simpler
to
create
a
basically
new,
fake
and
slave
object
and
which
has
the
logic
to
do
all
of
these.
E
G
C
G
G
G
F
F
A
That
what
happened
in
that
patch
was
instead
of
using
arenas.
We
construct
a
a
message
on
the
stack
and
we
move
into
the
vector
for
repeated
fields,
but
I
think
what
will?
What
will
do?
There
is
just
have
move
support
for
moving
the
whole
message
into
the
handler.
Accomplish
the
same
thing
so
we'll
have
that
at
some
point,
but
we
don't
have
it
yet.
G
E
A
G
A
G
Maybe
the
better
option
would
be
just
push
all
the
messages
to
master
queue
and
then
I'm
just
run
it
along
and
push
the
message
queue
and
then
what
strung
the
master,
like
the
only
one
thread
would
be
running
like
you
construct
the
messages
then
push
them
like
message
event.
You
push
that
to
inject
it
to
master
queue
and
then
like
when
everything
is
constructed
you
just
let
it
run.
E
B
What
I
did
I
created
two
benchmarks
that
their
purpose
was
to
compare
the
performance
of
the
old
the
process
message-passing
based
scheduler
driver
and
the
new
v1
scheduler
HTTP
api
for
the
implementation
of
the
those
benchmarks?
I
used
the
old
HTTP,
the
old
lik
process
based
driver
that
all
framework
Harry
are
using
right
now
and
then
you
HTTP,
based
the
one
scheduler
driver.
That's
called
Isis
so
because
those
those
benchmark
benchmarking
tools
are
using
the
process
and
parts
of
for
the
mazes,
the
performance
of
those
benchmarking
tools,
effects
and
benchmarks
as
well.
B
So
yeah
I
used
the
latest
version
available,
which
is
at
this
time
1.4
and
I
compared
the
performance
in
two
scenarios.
First
scenario
is
when
the
master
is
generating
a
lot
of
events
for
this
benchmark.
I
send
one
huge.
I
reconcile
cold.
That
contains
a
lot
of
non
existing
tasks
and
for
all
of
those
tasks,
the
master
will
respond
with
update
and
update
event
and
I
measure.
B
The
time
that
best
from
sending
the
first
from
sending
the
coal
and
until
the
last
updated
event
that
scalar
scheduler
receives
the
other
scenario,
is
ingesting
a
lot
of
scheduler
calls
by
the
master
for
this
test.
I
sent
many
revive
calls.
I
chose
revive
because
their
processing
is
relatively
cheap,
so
it
won't
affect
that
much
and
after
I
sent
all
the
revived
calls
that
had
I'll
send
one
reconcile
call
with
one
on
existing
tasks.
B
Sending
of
many
events
by
the
master,
the
performance
is
roughly
the
same,
so
reconciling
100,000
of
tasks
takes
relatively
six
six
point
four
seconds
if
we
are
using
old
or
scheduler
driver,
and
it
takes
around
six
seconds
if
we
use
the
news
the
new
HTTP
scheduler
driver.
So,
as
you
can
see,
the
performance
is
somehow
even
better
just.
B
A
B
Yeah,
okay,
okay
yeah
one
interesting
thing
that
I
found
when
doing
this
benchmark
is
that,
right
now
the
HTTP
scheduler
driver
calls
the
received
call
back
in
an
a
synchronous
manner
and
interesting
thing
that
if
we
switch
to
the
synchronous
calling
of
that
callback,
we
gain
roughly
forty
percent
before
improvement.
As
you
can
see
in
this
benchmark,
the
time
that
took
to
process
the
same
amount
of
reconcile
reconcile
calls
is
three.
B
So
in
this
scenario,
I
think
the
the
new
scheduler
drivers
is
good,
so
move
to
the
second
benchmark.
That
measures
the
performance
of
call
ingestion-
and
here
the
results
are
not
that
good,
because
when
we
are
using
the
new
HTTP
scheduler
scheduler
driver,
the
performance
of
it
is
nearly
three
times
worse
than
if
we
use
the
old
leap
process
based
driver.
B
B
B
A
B
A
A
A
B
B
H
Well,
I
guess,
while
I'm
here,
just
I
would
like
to
figure
out
what
Demetri
wants
to
do
with
the
defer
dispatch
stuff.
We're
essentially
like
the
context
I
have
is
just
that
we
want
to
move
the
arguments
through
the
front
dispatch,
they're
scheduled
work
or
investigation
going
on
anything
else.
I.
Just
briefly
looked
at
the
code.
H
You
have
that
will
be
useful.
Okay,
all
right,
I'll
reach
out
to
you
and
slack
I,
guess,
Demetri
I,
guess
essentially
essentially
what
we
want,
though
I
think.
First,
we
need
to
build
a
one-shot
bind
utility
right,
as
you
mentioned,
like
stood
behind,
doesn't
work
for
us
because
it
assumes
that
it
can
be
called
multiple
times.
So
we
can
kind
of
build
the
one
start
buying
the
utility
in
isolation
somewhere
in
stout,
and
then
we
can
use
that
in
the
process.
H
H
Yes,
so
what
I
want
to
do
there
first
is
the
CI
is
pretty
unstable
at
this
point
and
I
I
have
people
who
don't
want
me
to
update
upgrade
to
students
or
teen
just
yet,
because
we
want
to
stabilize
the
CI
first
I'm,
not
sure
how
long
that's
gonna
take,
but
the
other,
but
but
the
other
thing
is
that
we
actually
need
to
upgrade
this
update
the
CI
to
begin
with,
to
have
support
for
the
new
compilers,
because
otherwise
it's
just
gonna
break
everything.
So
so
the
CI.
H
G
H
H
I
was
mainly
just
thinking
about
just
looking
at
how
things
like
where,
where
and
how
it's
called
like,
for
example,
for
example,
we
presumably
you
know,
take
the
result
of
defer
and
pass
it
to
then
alright.
So
we
should
look
at
the
Ben
code
to
make
sure
that
yeah
we
actually
do
what
we
call
it
once
and
that
kind
of
stuff
I,
almost
wonder.
D
D
Yeah,
it
is
always
on
us,
but
I'm
just
saying
that's,
basically
that
could
I
think
that
would
be
a
sane
way
to
think
about.
It
is
if
someone
creates
a
called
one
sort
of
deferred
once
function.
We
will
move
your
arguments
into
it
now.
I
want
to
be
clear.
We
we
can't
move
the
future
arguments,
and
so
we
can't
your
value
that
one
we
still
have
to
pass
and
as
an
value
traveler,
because
there
could
be
multiple
people
that's
hold
onto
future.
C
D
D
We
end
up
holding
on
to
the
the
stood
function
inside
of
future
and
we
ultimately
execute
like
it
gets
it
gets.
It
gets
implicitly
converted
basically
to
a
state
function
in
them:
okay,
okay,
so
this
would
be
like.
If
somebody
gives
us
a
you
know,
call
them
all
once
or
whatever
it
is
yeah
I
didn't
under
the
covers.
When
that
gets
executed,
we
could
move
in.
We
could
move
through
I
guess
any
parameters
that
pass
to
it:
yeah,
yeah
yeah,
but.
D
D
Would
go
if
we
moved
it.
That
means
you
couldn't
call
the
second
one.
So
I
think
there
needs
to
be
something
completely
different.
If
we
want
to
have
the
concept
of
I
have
a
basically
a
future
that
nobody
else
has
ie
I
have
like
the
the
r-value
of
this
future.
Nobody
else
has
it
and
there's
no
other
copies
of
it.
Call
it
like
an
owned
future
if
you
lost
your
you'd,
be
future
and
therefore,
when
I
pass,
my.
C
D
H
I
get
what
you're
saying
I
think
yeah
I
mean
the
the
part
that
the
problem
essentially
is
just
that
futures
are
inherently
shared,
and
so
we
we
have
the
same
problem
as
like
stood
bind
where
we
have
to
be
able
to
call
it
multiple
times,
essentially
calling
dot
gun
multiple
times
on.
It
is
yeah
you
have
to.
We
can't
move
it
into
the
bend
because
you
have
to
be
able
to
come
well.
D
H
E
Typical
handling
message,
handling
methods
were
like
doing
partial
processing
and
then
at
the
end
of
it,
basically
more
often
than
not
at
the
end
of
it,
because
we
need
to
invoke
another
asynchronous
component,
would
jump
out
of
the
current
actor
and
go
to
another
and
then
later
come
back.
So
does
that
mean
that
then,
for
most
of
for
a
lot
of
these
arguments,
we'll
just
do
just
and
remove
on
on
all
of
them
if
they're
not
shared
they're,
just
they're
just
parameters
within
the
scope
of
the
method,
we're.