►
From YouTube: Kubernetes for Mac: How to Consume Shiny AWS Mac Shapes for iOS Builds - Madhuri Yechuri & Zach Gray
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Kubernetes for Mac: How to Consume Shiny AWS Mac Shapes for iOS Builds - Madhuri Yechuri, Elotl & Zach Gray, Flare.build
iOS builds have traditionally run on manually managed Mac servers sitting in brick and mortar data centers. Availability of Mac compute shapes on AWS presents a unique opportunity to move these builds into the cloud along with the ease of managing them via Kubernetes, thereby simplifying Operations. This talk describes Flare.build’s journey of evaluating manually-managed vs Kubernetes-managed Mac compute shapes on AWS, lessons learnt, and suggested best practices.
A
You
know
working
at
the
companies
that
you
see
up
here
and
so
a
little
bit
about
myself
and
flair
to
kick
things
off
so
yeah
a
little
bit
about
flare.
So
you
have
to
forgive
me
for
looking
up
at
the
screen.
There's
no
mirroring
here
so
yeah
a
little
bit
about
flair,
we're
part
of
bazel's
experts
program,
which
is
a
community
program
organized
by
google
to
sort
of
connect.
A
Another
thing
that
we
do
is
we
actually
offer
some
value-added
services
in
sort
of
a
sas
or
infrastructure
as
a
service
model
to
folks
using
bazel
to
build
and
test
applications
at
scale,
so
we'll
dive
in
a
little
bit
into
bazel.
I
won't
go
too
deep.
I
don't
want
to
bore
you
too
much,
but
suffice
it
to
say
bazel
is
a
is
a
great
build
tool
that
you
know
connects
to.
A
A
We
also
have
to
deal
with
sort
of
unifying
lots
of
different
compute
types,
so
you
know,
for
example,
mac
os
when
we're
doing
ios
builds
so
there's
a
lot
of
underlying
complexity
here
to
the
product
that
we're
building
and
so
we're,
of
course,
thrilled
to
have
partnered
up
with
a
loadle
to
help
us
with
some
of
the
underlying
infrastructure
stuff.
So
we
can
really
focus
on
building
out
our
product.
B
I'm
madri,
I'm
founder
of
ilotal
makers
of
nodeless
kubernetes,
the
vision
for
nodeless
communities
is
we
have
transitioned
from
treating
applications
as
pets
to
treating
applications
as
cattle.
The
vision
for
nodeless
is
to
do
the
same
for
compute,
so
not
to
have
pre-provisioned,
always
on
compute,
that's
hand
curated,
but
compute
that
comes
up
and
disappears
according
to
application
life
cycle,
so
it
falls
in
it
works
in
two
levels.
B
The
first
level
is
at
a
single
cluster,
where
you
just
have
a
kubernetes
control
plane
that
is
deployed
and
no
compute
that
is
pre-provisioned
at
all
the
right
size.
Compute
for
the
part
comes
up
when
the
port
starts.
So
if
an
ios
build
is
scheduled,
a
mac
one
metal
instance
comes
up,
and
if
you
want
an
arm,
compute
shape
for
your
app
and
arm.
B
Compute
comes
up
and
it
could
be
in
the
shape
of
an
on-demand
spot
or
fargate
launch
type
on
aws
and
similarly
for
other
cloud
providers
as
well,
and
it
also
works
in
commoditizing
workload
clusters
in
a
federation
of
workload
clusters.
So
you
can
have
policy
driven
application,
scheduling
across
multiple
workload
clusters,
so
you're
not
treating
individual
workload
clusters
as
pets.
So,
having
said
that
I'll
hand
it
over
back
to
zach
to
go
more
about
the
problem
statement.
A
Great
thanks
so
yeah,
a
quick
tl
dr
on
bazel
and
distributed
builds.
So,
like
I
said
you
know,
bazel
is
google's
open
source
build
system.
Some
of
you
may
be
aware
of
it.
I
assume
it
was
used
briefly.
I
think
in
the
kubernetes
project,
although
I
think
they
hit
the
eject
button
on
that
at
some
point
recently.
A
So
this
build
system
is
universal,
extensible
and
fast.
If
the
parameters
are
right
and
that's
where
flare
comes
in,
something
that
is
a
little
lesser
known
about
bazel
is
it
has
great
ios
support
and
broad
adoption
through
the
mobile
native
foundation
and
there's
a
lot
of
great
work
in
the
community
to
sort
of
make
bazel.
A
So,
like
I
said,
this
requires
a
server
implementing
a
few
of
the
remote
api
protocols
and
that's
what
we
offer-
and
one
really
important
anecdote
here-
is
that
these
macro
actions
really
need
to
be
running
on
apple
hardware.
So
it
might
be
tempting
to
to
try
to
run
these
actions
on
some
linux
hosts
and
there's
some
some
emulation
and
things
that
folks
are
looking
into,
but
really
long
story
short.
A
So
you
know
we,
we
launched
this
company.
You
know
just
a
few
years
ago
so
right
around
the
time
that
these
new
aws
mac,
one
metal
instances
were
coming
out.
So,
of
course,
you
know
we're
super
excited
to
see
this.
You
know,
especially
when
we
look
at
folks
building
ios
apps
at
scale.
This
means,
potentially
you
know,
no
more
big
messy
closets
jammed
full
of
mac
minis.
A
Of
course,
aws
is
well
known
for
auto
scaling.
We
have
that
in
scare
quotes
here
because
that's
up
for
debate,
but
in
theory
this
is.
This
is
a
feature.
That's
that's
provided
and
then,
of
course,
you
know
standardizing
the
devsecops
on
aws
is
awesome.
Rather
than
dealing
with,
of
course,
any
other
infrastructure
providers
or
bringing
your
own
hardware.
A
So
yeah
so,
like
I
said,
these
instances
are
great.
We're
super
excited
to
have
launched
our
company
requiring
mac
compute
right
when
these
were
available,
but
there's
unfortunately,
still
a
few
shortcomings
that
we
had
to
sort
of
overcome,
especially
in
the
earlier
days
here.
So
you
know,
I
think
the
first
is
the
pricing
model
right
now,
even
with
committed
use
discounts.
A
The
pricing,
as
can
you
can
see
clearly
here,
is
a
bit
extravagant.
You
know
when
you're,
when
you're
dealing
with
you
know
a
large
number
of
nodes,
especially
so
this
is,
of
course,
one
of
the
biggest
sticking
points
that
a
lot
of
people
have
when
it
comes
to
adopting
these
mac
minis
they're,
expensive.
A
Another
issue
is
that
there
is
actually
some
hard
limitations
on
auto
scaling,
so
one
of
these
is,
of
course,
the
24
hour
minimum
allocation.
So
if
you
kind
of
squint
at
that
really
what
this
is
saying.
Well,
that's
not
really
auto
scaling
at
all
right,
so
you
know
when
we
turn
on
a
machine,
we're
stuck
billing
in
24
hour
increments.
So
we
don't
want
to
turn
on
10
machines,
just
to
turn
nine
of
them
back
off
and
still
be
billed
for
the
24
hour
period.
A
So
as
a
result,
what
we
see
is
these
instances
are
often
underutilized
after
we
scale
them
up.
You
might
hang
them
around
for
another
23
hours
after
turning
them
on
one
of
the
biggest
issues.
Right
now
that
we
see
using
sort
of
out
of
the
box
tooling,
is
just
that
the
existing,
auto
scalers
are
really
they're
meant
for
linux
and
they're,
just
not
a
good
fit
for
the
shape
of
the
mac.
Compute.
A
A
Of
course
the
jobs
are
coming
in
we're
spiking
up
close
to
90,
100
cpu,
but
unfortunately
there's
just
these
big
areas
of
the
graph
that
are
totally
underutilized,
which
is,
of
course,
one
of
the
big
challenges
that
we
would
like
to
address,
because
we
don't
want
to
just
you
know,
be
burning
cpu
cycles,
kind
of
like
they
mentioned
in
the
keynote
so
yeah.
So
with
these
mac
and
one
metal
instances,
of
course,
management
became
a
whole
lot
easier,
but
it's
still
not
easy.
A
A
Also
again,
configuring
the
auto
scaling,
for
example,
for
ci
and
remote,
build
workers
that
we're
working
with
is
still
fairly
complex.
Even
with
you
know,
aws
primitives
that
we
have
in
terraform,
it
still
can
be
a
bit
of
a
challenge.
Obviously,
we've
got
this
big
terraform
template.
We
need
to
get
all
that
stuff.
Just
right
set
up
all
the
auto
scaling
groups,
all
that
still
not
super
straightforward.
A
So
to
sort
of
recap
on
some
of
the
problems
that
we've
seen
at
our
company
while
adopting
you
know
the
mac,
compute
is
really
one
configuration
management
while
it's
easier,
it's
still
a
challenge.
Two
auto
scaled
instances
are
underutilized
and
they're,
also
expensive,
and
then
three,
you
know
what
about
bazel,
so
some
solutions
that
we've
come
across
here
working
this
space
would
be
so
for
as
far
as
the
configuration
and
management.
A
Well,
one
of
the
one
of
the
hacks
that
we
have
here
is
actually
utilizing
the
the
same
workers
both
as
the
ci
runners,
so
the
ci
agents
that
might
be
hooked
up
to
a
ci
solution
as
well
as
executing
the
bazel
remote
actions,
and
so
that
way
the
under
utilized
cpus
can
now
be.
You
know,
joined
in
and
work
as
part
of
the
remote
build
farm
when
they're
not
actively
running
ci
jobs
and
then
yeah.
What
about
bazel?
A
Well
now
that
we've
got
this
sort
of
unified
compute,
we
don't
have
to
have
a
separate
pool
of
of
workers
running
over
here
for
our
bazel
agents,
then
another
one
for
ci
agents.
It
all
just
runs
on
the
same
infrastructure,
all
managed
under
kubernetes
yeah,
so
yeah
so
kind
of
a
quick,
a
quick
look
at
how
you
know
the
solution
sort
of
evolved
for
us.
So
of
course,
if
you're,
if
you're
working
on
doing
ios
builds,
you
should
probably
have
a
ci
system.
A
B
Thanks
zach,
so
when
zach
and
others
wanted
to
consume
mac
one
metals
on
aws,
we
wanted
to
do
an
objective
evaluation
of
hey.
Do
we
really
need
kubernetes
here?
We
are
all
here
for
kubecon,
so
we
are
all
we
all
love
kubernetes,
but
for
an
end
user
we
wanted
to
do
a
thorough
evaluation,
as
is
this
really
required?
Is
communities
really
required
for
managing
these
mac
one
metal
instances
or
is
manual
management
good
enough,
so
we
did
benchmarking
along
four
dimensions.
B
The
first
one
is
graceful
termination
of
both
the
build
agent,
as
well
as
the
compute
node.
The
second
one
is
build
agent
configuration
and
ongoing
operations,
and
the
third
one
is
auto
scaling
of
both
the
build
agent
as
well
as
the
underlying
compute,
and
the
fourth
axis
is:
how
do
you
reclaim
bad
nodes
which
happen
to
be
a
more
frequent
scenario
on
for
mac
when
metal
instances
has
compared
to
arm
nodes
or
any
other
gpu
devices
or
any
other
kinds
of
compute?
So
let's
go
into
the
details
of
each
of
these.
B
Let's
start
with
graceful
termination,
so
for
build
agent.
If
you
want
to
gracefully
terminate
it
if
you're,
if
you're
managing
your
map
on
metal
nodes
manually,
you
would
have
to
configure
ec2
life
cycle
hooks
and
you
have
to
maintain
these
life
cycle
hooks
and
you
have
to
constantly
monitor
and
edit
these
life
cycle
hooks,
which
is
huge
amount
of
overhead
for
operation
teams,
whereas
with
kubernetes.
B
We
all
know
that
if
you
are
running
a
build
agent
as
a
kubernetes
pod,
even
if
you
get
a
termination
notification,
the
workload
that's
running
inside
the
pod
will
will
complete
before
the
part
is
gracefully
terminated.
So
you
get
that
out
of
the
box
with
in
a
kubernetes
managed
system
and
let's
move
on
to
graceful
termination
of
the
nodes
themselves.
Again,
if
you
are
managing
your
mach
1
metal,
computes
manually,
you
would
have
to
create,
configure,
manage
and
update
your
ec2
lifecycle
hooks
and
it's
tightly
coupled
to
your
build
agent.
B
Whereas
with
kubernetes
you
you're
able
to
cordon
off
the
node,
and
you
can
drain
the
node
and
you
can
you
can
gracefully
terminate
the
node.
So
you
get
a
lot
of
these
graceful
termination
requirements
for
both
the
workload
build
agent,
as
well
as
the
underlying
compute
node
out
of
the
box.
If
you
use
a
kubernetes
based
system
next,
let's
talk
about
build
agent
config,
so
build
agents
would
need
certain
kind
of
config
information
for
where
your
log
files
are
located
and
stuff
like
that.
B
So
if
you're
man,
if
you're
manually,
managing
your
build
agent
on
a
mac
one
metal,
you
would
have
to
bake
this
config
information
into
the
ami
at
creation
time
and
during
ongoing
operations.
If
you
have
to
update
the
config,
you
will
have
to
provision
a
brand
new
node
and
update
the
config
on
that
node
and
and
create
a
new
ami
out
of
it.
B
And
then
you
have
to
churn
all
of
your
existing
nodes
with
the
new
ami,
which
is
huge
amount
of
overhead
for
operations
team
with
respect
to,
if
you're
managing
it
using
kubernetes,
you
are
automatically
able
to
do
it
using
kubernetes.
So
you
get
much
more
ease
of
operations
for
the
config
management,
because
you
can
use
config
maps
and
secrets
to
pass
in
the
config
information.
So
you
simply
have
to
update
your
config
map
and
secret
and
roll
out
a
new
deployment
for
your
build
agent.
B
B
The
third
dimension
is
auto
scaling,
so
we
wanted
to
also
evaluate
how
auto
scaling
of
the
build
agent
pods
can
can
work
in
a
in
a
manually,
managed
macro
metal
scenario
and
how
auto
scaling
of
the
underlying
compute
would
work
in
a
manually
managed
versus
kubernetes
managed
scenario.
Let's
look
at
auto
scaling
of
the
build
agent
first.
B
Groups
and
you'll
have
to
expose
the
metrics
from
the
build
agents
through
cloud
watch
and
you'll
have
to
configure
your
xposed
metrics
through
cloud
was
through
a
lambda
function
or
something
and
plug
it
into
your
auto
scaling
group.
So
it's
it's
quite
a
bit
of
operational
overhead
involved
in
in
setting
up
all
of
these
plumbing
networking
in
place,
whereas
with
kubernetes
you
can
have,
and
you
can
have
an
hpa
that
is
configured
to
scale.
B
B
Again
it's
tightly
coupled
in
the
manual
scenario
to
the
build
agent,
so
you'll
have
one-to-one
correspondence
between
build
agent
and
macro
and
metal
if
you're
manually,
managing
them,
whereas
with
kubernetes
and
nodeless
kubernetes
in
particular,
the
auto
scaling
comes
out
of
box
because,
based
on
pending
jobs
in
your
kubernetes
cluster,
needless
kubernetes
cluster
will
spin
up
new
nodes
and
scale
down
new
nodes,
and
it
is
cost
aware-
and
it's
aware
of
the
24
hour
billing
cycle
of
the
underlying
compute.
So
you
you're
able
to
get
auto
scaling
out
of
the
box
as
well.
B
B
We
know
we
noticed
up
to
10
percent
of
their
north
spun
up,
weren't,
really
functional,
and
if
you,
if
you
were
using
manually,
if
you
were
managing
your
mach
1
metal
instances
manually,
you
would
someone
had
have,
would
have
to
notice
that
hey
my
bill,
provided
my
my
bill
failed
and
have
to
triage
the
failed,
build
and
then
figure
out
that
oh,
the
node
is
not
really.
It
was
never
really
able
to
come
up
on
this
on
this
failed
node,
whereas
with
kubernetes.
B
B
So,
having
evaluated
manually
managed
kubernetes
manually
managed
versus
kubernetes,
we
decided
to
build
the
following
stack
for
managing
mac1
metal
instances,
four
build
workloads
orchestrated
by
a
ci
orchestrator
using
the
kubernetes-based
framework.
So
at
the
top
we
have
bill
scalar,
which
is
an
hpa
so
bill
scalar.
B
What
it
does
is
it
looks
at
pending
bills
in
your
build
queue,
and
it
also
looks
at
what
percentage
of
your
build
agents
are
busy
and
looking
at
the
metrics
collected
from
the
build
agents,
it
will
scale
up
number
of
build
agent
parts
up
and
down
based
on
pending
jobs
in
the
build
queue.
So
if
there
are,
if
there
is
an
unexpected
spike
or
if
there's
a
spike
in
the
number
of
build
jobs
that
have
been
provisioned,
it
will
increase
the
number
of
replicas
for
the
build
agent
parts
up
and
it'll
scale
down.
B
The
build
agent
part
count
down.
If,
if
not,
that
many
build
agents
are
busy
and
going
down
one
level,
the
next
level
is
no
less
kubernetes.
No
less
kubernetes
takes
care
of
auto
scaling
the
underlying
compute
based
on
pending
pending
parts
in
the
control
plane.
So
if
there
is
an
there
is
a
spike
in
the
pending
parts,
based
on
what
hpa
has
advised
kubernetes
control
plane.
B
No
less
kubernetes
will
auto
scale
the
mach
1
metal
nodes
up
if
there
is
an
increase
in
the
number
of
pending,
build
jobs
and
the
third
level
is
kubernetes
control,
plane
itself,
which
gives
us
a
lot
of
the
graceful
termination
and
bad
node
management
and
monitoring,
and
all
of
those
nice
kubernetes
goodness
out
of
the
box
and
at
the
fourth
level,
is
the
mach
1
metal,
node
itself,
so
mach
1
metal
node.
When
it's
running
in
the
nodeless
kubernetes
mode,
it
is
going
to
run
a
cubelet
stack.
B
Now,
let's
look
at
each
of
these
four
layers
in
action.
So,
let's
start
at
the
hpa,
build
scalar
build
scalar
is
an
open
source
project
from
ilotal.
The
links
are
provided
at
the
end
of
this
and
in
the
final
slide.
So
here
you
see
that
the
running
job
count
is
listed
in
red.
So
you
see
that
the
running
job
count
and
the
total
agent
count
are
tracking,
pretty
synchronously,
which
means
that
bill
scalar
is,
is
spinning
up
new
agents
creating
new
agent
agent
parts
based
on
pending
jobs
in
the
build
queue.
B
B
The
second
layer
is
nodeless
kubernetes,
where
we
are
going
to
look
at
so
it
provisions
just
in
time
compute
when
a
part
starts
up
and
it
terminates
the
compute
when
the
part
terminates
and
the
stack
is
comprised
of
of
the
compute
order,
scalar,
that
auto
scales,
mach
1
metal,
compute
and
the
on
the
node.
You
have
cubelet,
plus
the
cri.
That
is
max
specific,
and
this
is
of
course,
there's
a
free
tier
where
you
can
provision
and
manage
up
to
two
mac,
one
metal
instances.
The
entire
kubernetes
stack
for
free.
B
So
now,
let's
look
at
nodeless
kubernetes,
hey
luna,
in
action.
So
what
we
have
here
is
a
screenshot
of
a
production,
no
less
kubernetes
and
managed
macro
and
metal
environment,
and
you
see
that
the
the
cost,
the
price
of
macrometals
is
trending
up
and
down,
based
on
time
of
day,
day
of
week
and
and
day
of
year
as
well.
B
So
let's
try
to
super,
impose
what
we
saw
with
the
hpa,
with
the
pending,
build
jobs
and
pending
pending
build
parts
versus
the
cost,
and
you
see
that
when
there
are
more
bills
you're
paying
more
and
when
there
are
less
bills,
you're
paying
less
so
the
the
lulls
in
the
bills
right
here
correspond
to
the
low
amount
of
cost.
That
is
being
that
is
being
spent
on
your
mark
on
metal.
B
So
the
nodeless
kubernetes
component
is
smart
enough
to
understand
that
your
mac,
one
metals,
are
charged
at
a
24
hour
cadence,
so
it
it
does
predict
that
okay,
I've
finished
my
build.
I
just
provisioned
my
my
mark
on
metal,
but
a
lot
of
pending
jobs.
So
I'm
going
to
keep
this
mac
one
metal
instance
on
and
to
schedule
future
builds.
So
it
doesn't
churn
the
mac
on
metals,
as
often
as
it
would
churn
on
an
arm
based
vm
or
an
intel-based
vm.
B
So
now
look:
let's
look
at
the
entire
thing
in
action,
so
we're
going
to
start
with
looking
at
build
scalar
and
nodeless
less
kubernetes,
no
less
kubernetes
in
action.
First
and
then
zach
is
going
to
talk
about
flare
components
and
the
build
a
flare
build
agent
and
how
you're
optimally
utilizing
the
compute
that
you
provisioned
for
the
mac.
B
Thanks
zack,
so
in
the
top
window
we
are
going
to
execute
the
operations
in
this
in
this
architecture.
The
bottom
left
window
is
where
we
are
going
to
watch
the
nodes
in
the
system,
the
bottom
right
window.
We
are
going
to
watch
the
parts
in
the
system,
so
we
currently
have
the
we
currently
have
one
mach
1
metal,
that
is
running
in
our
environment
and
we
have
the
mac
auto
scaler,
which
is
nodeless
kubernetes
component.
That's
the
hpa
that
is
scheduled,
that's
configured
in
the
environment.
B
So
if
we
look
at
the
hpa,
the
hpa
say
has
a
target
of
90
and
the
min
number
of
parts
is
one
max
number
of
parts
is
three
and
we
want
to
have.
The
current
replicas
is
one.
What
is
what
that
means
is
that
we
want
the
hpa
to
scale
from
one
pod
to
three
parts.
Currently,
it
is
at
one
spot,
one
part,
and
we
want
we
want
it
to
perform
a
scale-up
operation.
B
If
the
mean
utilization
of
the
build
agents
is
at
90
percent.
So
if
the
mean
utilization
of
the
build
agents
across
two
build
pods
or
three
billboard
parts,
if
it
if
it
hits
90,
that's
when
we
want
to
perform
the
scale-up
operation,
so
let's
go
ahead
and
schedule
a
build.
If
we
kick
off
a
build,
what
we?
B
What
we
would
want
to
see
is
that
the
hpa
should
realize
that
okay,
a
new
build,
came
into
the
build
queue
and
we
want
to
scale
the
number
of
of
the
of
the
build
parts
from
one
to
two,
because
the
average
utilization
of
the
current
build
agent
should
hit
should
go
above
90..
So
we
should
let's
look
at
the
new
pod
being
scheduled,
and
if
you
look
at
the
bottom
right
window,
the
pod
is
in
pending
state
because
it
doesn't
have
a
machine
metal
compute
to
run
on.
B
So
here
is
when
nodeless
kubernetes
kicks
in,
and
it
sees
that.
Oh,
there
is
a
pending
pod
in
the
kubernetes
environment
and
there
are
no
compute
nodes
available,
so
it
will
go
and
schedule
a
create,
a
brand
new
mac
one
metal
instance
and
that
microphone
metal
instance.
You
see
in
the
bottom
left
window,
it's
ready
and
that's
where
the
the
part
is
scheduled.
B
A
Cool,
so
so
here's
a
quick
look
at,
of
course,
some
of
the
the
metadata
coming
out
of
bazel,
and
so
here
we're
taking
a
look
at,
of
course,
at
the
ci
output
here
and
so
I'll.
Just
explain
a
little
bit
more
about
what's
going
on
under
the
hood.
So
what
we
see
actually
happened
here
was
the
the
build
ran
in
the
ci
environment,
but
it
actually,
you
can
see
there
if
you
look
closely.
A
11
actions
were
executed
remotely,
so
we
had
a
lot
of
cache
hits
because
this
was
a
pre-built
example
here,
but
we
can
see
that
we
actually
had
a
lot
of
well.
11
actions
were
executed
remotely,
and
so
this
is
a
build
of
bazel
itself,
the
open
source
project
against
one
of
our
environments,
and
so
we
went
ahead
and
clicked
a
link
there
to
link
off
into
some
of
our
our
ui.
A
That's
actually
gonna
allow
us
to
see
a
little
bit
more
deeper
into
what's
going
on
behind
the
scenes
and
that
bazel
build.
So
we
can
see
a
bunch
of
metadata
about
the
build
there.
The
build
logs
some
some
information
about
the
cache
hits
and
all
that
stuff,
so
this
is
powered
by
an
api
built
into
bazel
to
expose
that
data
to
us
here.
So
that
was
an
example
of
like
a
success
case.
This
is
an
example
of
when
there's
some
failures
in
bazel.
A
So
again
in
the
ci
system,
we've
hooked
into
some
some
life
cycle,
events
in
bazel
we've
extracted
and
actually
parsed.
Some
of
the
errors
that
were
dumped
out
during
this
build
bubbled
those
up
right
in
front
of
the
user.
So
in
this
case
this
failed
because
we
tried
to
execute
a
mac
os
build
of
bazel
without
any
mac
os
workers
available
at
all.
So,
of
course,
the
build,
failed
and
so
again
clicking
that
link.
Here
we
go
back
in.
A
We
see
the
you
know
the
build
details
here
in
our
platform,
and
we
actually
see
that
we
extracted
this
error,
so
we'll
we'll
take
a
dive
in
there
to
kind
of
look
at
some
of
the
the
information
there
that
we
are
gathering
about
errors.
So
when
errors
occur
in
the
system,
we're
actually
like,
I
said
we're
extracting
those
with
a
proprietary
algorithm
and
we,
while
the
stack
trace,
doesn't
look
great.
This
is
the
stack
trace
that
was
dumped
out
of
bazel
and
so
we're
actually
kind
of
cataloging
this
error,
and
so
we
can
see.
A
You
know
the
last
time
that
this
this
error
was
encountered
in
the
system,
so
it's
sort
of
like
you
could
think
of
it
like
sentry,
but
for
build
time
errors
rather
than
runtime
so
yeah.
So
that's
pretty
much.
You
know
a
quick
snapshot
of
what
the
ui
is
able
to
do
and
some
of
the
metadata
coming
out
of
bazel
so
I'll
hand
it
back
over
to
madri.
I
guess
to
talk
a
little
bit
more
about
what's
going
on
here.
B
B
B
So
we
are
working
on
supporting
m1
instance
types
that
have
been
coming
to
market,
so
that
is
a
work
in
progress.
Currently,
we
do
want
to
add
that
support
pretty
quickly
in
the
environment,
so
that
should
be
coming
out
pretty
soon
as
well.
So
the
whole
idea
of
nodeless
kubernetes
is:
is
you
as
the
end
user
of
communities
platform
shouldn't
worry
about?
What
are
the
new
newer,
better
kinds
of
compute
shapes
coming
into
market?
A
Sure
yeah
so
on,
of
course,
on
the
future
road
map.
You
know,
we've
got
some
items
we
want
to
get
to
at
some
point.
So
of
course
one
of
those
would
be
some
basal
specific
optimizations
for
our
use
case.
So
one
of
the
things
that
we
want
to
do
is
maybe
find
a
way
to
intelligently
share
some
of
the
the
bazel
agent
outputs
directly
with
the
ci
job
and
sort
of
short-circuit.
Some
of
the
round-trip
network
calls
that
would
make
maybe
a
potential
impact
there.
A
Of
course,
the
other
big
item
that
we're
really
excited
about
is
that
a
little
open
sourced
the
build
scare
scalar
framework
itself.
So
that's
that's,
of
course
super
awesome.
So,
of
course,
there's
a
link
here,
we'll
share
the
slides,
definitely
check
it
out,
and
then
you
know
some
other
stuff.
Of
course
you
know
the
upcoming
m1
support.
We're
super
excited
about
that
from
a
bazel
perspective.
Of
course,
these
m1
chips
are
really
really
great
for
for
basal
builds.
Things
are
quite
a
lot
faster.
A
B
That's
about
it
so
bill
scalar
it.
It
is
designed
in
such
a
way
that
you
can.
It
can
pull
in
external
metrics
from
any
ci
orchestrator.
So
it
currently,
it
supports,
build
kite
circle.
Ci
flair
build,
but
it
can
be
extended
to
any
ci
any
ci
metrics
provider.
So
it's
it's
basically
getting
in
the
metrics
from
the
ci
agent
and
and
converting
it
into
metrics
that
can
be
in
injected
by
the
hpa
cool.
A
B
Yep
yeah,
that's
about
it,
so
there's
a
free
tier
for
mac,
one
metal,
nodeless
kubernetes,
like
I
mentioned
earlier,
you
can
run
it
for
up
to
two
nodes,
so
bill
scalar
is
open
source,
which
is
the
hpa
that
is
open
sourced
and
the
free
tier
for
the
mac.
One
metal
compute
is
open
source,
so
the
the
only
thing
that's
not
a
free
tier
is
is
free,
obviously,
and
the
only
thing
that's
not
not
open
sourced
yet
is
the
mac
auto
scaler
component,
and
that
is
tbd.
A
A
D
B
C
Yeah
and
the
follow-up
question
is
then,
in
my
use
case,
I've
already
built
my
osix
software,
but
I
just
want
to
test
the
configuration,
and
so
my
use
case
is
like
to
install
and
reinstall
and
reinstall
the
same
software.
Does
this
system
allow
that
to
keep
reinstalling
the
software
that
I
have
on
the
mac
hardware.
B
B
So
you
see
the
build
agent
config,
where
you
want
to
be
able
to
to
inject
different,
build
agent
config
in
data
into
your
current
port.
That
would
become
much
much
more
easier
by
using
this
system
than
doing
it
manually
because
with
kubernetes
you're
going
to
be
passing
in
all
the
config
data
as
config
maps
or
secrets
stuff,
like
that,
so
you'll
be
able
to
inject
varying
configurations
dynamically
at
runtime.
So.
B
Yeah,
that's
a
really
good
question,
so
the
question
is:
what
cri
are
we
using?
So
this
is
a
cri
that
we
built
at
elotal
for
mac,
one
metal.
So
it's
a
cri
that
basically
takes
in
the
sierra,
is
basically
image
service
and
container
service
implementations
right,
so
that
talked
to
the
cubelet
agent.
So
the
cri
that
we
built
is
a
custom
cri
for
mac
for
mac,
one
metal,
that's
part
of
the
the
closed
source,
one
open
source
strategy,
tbd.
B
A
Yeah
one
one
additional
comment
I
would
have
on
that
is
so
from
a
bazel
perspective.
Of
course
it's
important
for
us
to
allow
folks
that
have
big
existing
fleets
of
macs
to
bring
that
hardware
to
our
solution.
So
that's
definitely
some
functionality,
that's
sort
of
core
to
our
offering
and
that's.
I
think
one
of
the
original
reasons
we
reached
out
to
a
loadable
is
because
we
were
looking
for
a
way
to
have
sort
of
hybrid
cloud
setup
support
out
of
the
box.
B
Yeah
yeah
yeah
yeah,
so
the
nice
thing
about
kubernetes
in
general
and
no
less
in
particular,
is
that
since
we
can
define
hey,
I
want
xcode
was
version
abc
and
these
other
build
packages
or
any
other
dependency
packages.
There
are
pre-built
amis
already
available.
So
if
you,
if
you
mention
the
the
dependencies
that
you
that
are
needed
for
your
pod
for
running
the
workload
in
the
pod
manifest,
the
right
ami
is
picked
at
runtime.
Based
on
that
information.
B
Yeah,
so
that's
a
good
question.
So,
if
you're,
if
you're,
if
the
ami
is
being
used
for
flare,
build
plus
build
other
kind
of
build
agents,
then
it's
managed
by
flare,
flare
and
us.
If
you're
using
it
for
non-flare,
build
related
use
cases,
then
it
would
be
managed
by
us.
So
it's
basically
an
ami,
that's
already
in
our
fleet
and
we
white
list,
your
your
aws
account
to
consume
it.
Okay,.