►
From YouTube: Microservices Patterns with NGINX Proxy in an Istio Services Mesh [I] - A.J. Hunyady, NGINX Inc
Description
Microservices Patterns with NGINX Proxy in an Istio Services Mesh [I] - A.J. Hunyady, NGINX Inc
Building a cloud native application is only half the battle; running it reliably is the other half.
NGINX, the leading provider of ingress controller functionality in Kubernetes environments, has partnered with Istio to enhance Sidecar proxy capabilities in the Istio' Services Mesh architecture.
A service mesh is highly dependent on the strength of the proxy, and NGINX is the most powerful service proxy in the market. It offers a small footprint high performance engine with advance load balancing algorithms, caching, SSL termination, API gateway, extensibility through broad range of third-party modules, sciptability with Lau and nginScript and various security features with granular access control.
Microservices also require a Web Server to be deployed side-by-side with the service proxy. While optional, deploying NGINX as Web Server technology provides additional benefits in performance, manageability, security and the overall monitoring of the Application.
NGINX is already used by more than half of the top 100,000 websites and this talk will describe how NGINX in Istio environments is a natural extension of this technology.
Our demo will show a sample application running in a Kubernetes/Istio/NGINX environment and we will answer questions from the audience.
About A.J. Hunyady
A.J. Is a technology enthusiast and a Silicon Valley veteran. He founded Zokets where he developed software for managing containerized services in highly dynamic environments. A.J. is now at NGINX, where he leads innovations in new product development.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
A
So
I'll
try
to
cover
as
much
as
and
pretty
much
I
try
to
speed
the
conversation
into
to
have
15
minutes
with
the
marketing
material
and
someone
provided
a
bit
of
a
background
in
terms
of
who
the
company
is
and
what
are
some
of
the
reference
architectures
we
built
for
micro
services,
infrastructures
and
then
the
demo,
so
how
many
of
you
are
nginx
users
shorthand
all
right?
Well,
I,
guess
I
hope
not
to
bore
everyone
here
and
may-maybe
interesting.
So
first
slide
it's
all
about
what
nginx
the
company
is
for.
A
Those
of
you
who
don't
know
nginx
has
been
around
for
quite
a
while.
It
started
in
before
2004
when
our
founder
and
CTO
Igor
C
serve.
Try
to
solve
the
sea
tank,
a
problem
which
was
the
ability
to
improve
the
number
of
concurrent
sessions
for
a
web
server.
At
that
point,
web
servers
could
do
maybe
in
100
or
web
server
connections,
concurrent
connections
with
nginx.
He
was
able
to
bring
that
number
to
tens
of
thousands,
so
he
opened
the
first
version
of
that
in
the
java
source
code
in
2004
fast
forward
about
seven
years.
A
Clearly
it
was
very
well
received
by
the
community
and
he
couldn't
no
longer
manage
the
ecosystem
on
his
own.
So
him
and
a
couple
of
his
friends
they
got
together.
They
created
a
company.
The
company
was
called
Angeles
Inc.
Now,
if
you
see
what
two
years
later
after
they've
played
to
the
company,
we
build
a
commercial
product
which
is
called
ngx
Plus,
which
helps
us
keep
the
episodes
going.
It's
worth
mentioning
that
around
80%
of
our
functionality
that
we're
building
even
today
it
goes
into
open
source
first.
A
Hence
this
is
it's
great
to
be
part
of
this
conference.
Whatever
I'm
going
to
be
presented
today
is
gonna,
be
open
source.
If
you
fast
forward
today,
nginx
is
a
globally
distributed
company.
We
are
backed
by
first-class
VCS,
NEA
being
one
of
them.
We
have
offices
all
around
the
globe
our
headquarters
in
San
Francisco.
Our
offices
are
in
the
UK
in
Ireland,
Moscow
and
Singapore.
A
A
A
We
anticipate
that
around
2018
we
will
be
over
50%
of
top
1
million
of
websites,
so
you
may
be
asking
what
helped
us
gain
such
tremendous
momentum
in
the
marketplace,
because
it's
one
thing
to
have
a
a
small
footprint
infrastructure,
but
some
trend
has
to
be
shifted
in
the
market,
so
we
believe
that
the
trend
was
really
the
adoption
of
micro
services.
So
if
you
took
a
look
what's
happening
in
the
market,
nginx
is
very
small
footprint.
It's
fast.
A
If
you
see
what's
happened
in
the
marketplace
first,
it
was
the
packaging
portability
and
the
ability
to
share
containers
across
ecosystems.
Darker
has
done
a
tremendous
job
covering
that
as
the
second
part
of
that
of
its
it's
really
dark
additional
orchestration
aspect.
What
do
you
have
to
date?
The
containers
you
distribute
them
across
the
ecosystem
and
then
you
need
to
stitch
them
together
into
applications.
Micro
services
based
applications.
I
would
argue
that
kubernetes
has
done
an
outstanding
job
in
building
a
solid
orchestration.
A
Can
that
can
spray
containers
across
the
ecosystem,
but
then
those
containers,
the
orchestration
alone,
as
you
guys
know,
because
you're
part
of
this
conference
is
simply
not
sufficient.
You
have
to
build
a
tremendous
amount
of
custom
tooling
in
order
for
you
to
stitch
the
applications
together.
In
order
for
you
to
run
a
CIC
D
pipeline
right
right,
run
multiple
versions,
side
by
side,
in
order
for
you
to
monitor
the
AYGO
system
and
last
but
not
least
in
order
for
you
to
secure
them
and
all
that
has
to
happen
in
dynamic
context.
A
So
this
is
where
the
services
match
comes
in.
It
enables
you
to
stitch
those
services
together.
It
enables
you
to
provide
a
high
level
of
security,
enables
you
to
monitor
them
effectively,
and
it
enables
you
to
be
able
to
break
down
multiple
applications
running
side
by
side
through
this
ecosystem.
So
nginx
has
been
offering
various
micro
services
architectures
in
order
for
vendors
to
move
from
a
simple
deployment,
maybe
a
legacy
deployment
where
they
have
just
one
egress
load,
balancer
to
more
sophisticated
deployments
where
they
actually
care
about
the
east-west
traffic.
A
So
let
me
take
you
to
the
progression.
So
initially,
we've
deployed
the
proxy
model.
Maybe
I
should
ask
before
I
go
forward,
how
many
people
are
actually
familiar
with
some
of
the
reference
architectures
that
nginx
has
been
pitching
for
the
last
several
years.
Brigid,
it's
quite
a
bit
of
shorthand,
so
proxy
mother.
It's
probably
the
simplest
of
the
three
of
the
four
that
are
here.
A
It
enables
you
to
deploy
an
engine,
as
instance
at
the
egress,
and
in
this
case
you
probably
only
deal
with
traffic
that
comes
into
your
network
and
gousa
goes
out
your
network.
You
allow
the
services
to
communicate
with
each
other,
so
you
don't
really
get
in
the
way
in
the
ingress.
Proxies
is
more
like
if
you
are,
if
you
have
an
agile
old
man,
sir,
as
an
f5
and
you're,
not
happy
with
the
way
that
that's
working
out
for
you,
because
it's
very
hard
to
make
changes
to
it.
A
Then
we
see
customers
are
actually
taking
nginx
like
sitting
in
front
of
it
or
front
of
an
F
a
load
balancer
and
enables
you
to
I
should
stitch
out
workloads
based
on
application,
work,
application
payload.
The
second
is
the
router
mash.
If
the
router
mash
is
where
you're
taking
a
monolithic
application,
you're
trying
to
you're,
beginning
to
peel
it
off
into
micro
services.
In
that
case,
you
end
up
in
a
situation
where
you
have
different
environments.
A
Working
together,
maybe
half
your
network
is
working
in
with
half
of
your
applications
are
actually
VM
based
and
the
other
half
you're
moving
into
micro
service.
Then
in
that
case
you
will
have
an
ingress
load
balancer
that
handles
the
traffic,
and
then
you
spread
the
traffic
inside
the
micro
service
for
for
management.
A
For
instance,
if
you
have
a
kubernetes
environment
running
side-by-side
with
a
VM
environment,
you
can
split
a
traffic
and
English
side
and
stitch
together,
an
English
controller
in
the
kubernetes
environment
to
do
the
dispatching
of
your
applications
in
the
in
the
us
controller
of
kubernetes.
Now
the
third
iteration
into
this
it
becomes
when
you
are
really
interested
not
only
in
the
north-south
traffic,
on
the
English
side
and
simple
Cabrini's
deployments,
but
you're
interested
in
managing
the
traffic
in
between
services
themselves.
A
In
this
situation,
we
are
recommending
the
fabric
model,
where
you're
deploying
an
engine
existence
on
every
one
of
your
micro
services
and
these
nginx
instances
communicate
with
each
other
and
also
the
service
discovery
protocol.
So
it
enables
to
stitch
together
services
very
much
like
Easter
is
doing,
but
without
a
control
plane.
Now.
The
fourth
iteration
of
this
is
where
Easter
comes
in
ER,
it's
the
same
fabric
model
with
the
ability
to
stitch
together
services,
but
also
with
a
control
plane
and
the
conversation.
A
The
whole
presentation
today,
it's
going
to
be
at
our
services
match
going
forward
so
in
the
services
match.
If
it
is
for
you
to
take
a
look
at
what
happens
under
the
hood,
you
have
a
control
plane,
as
was
described
in
various
other
sessions
today,
in
the
control
plane,
you
are,
you
have
three
components
in
the
East,
your
control
plane
itself.
A
You
have
pilot,
you
have
mixer
and
you
have
auth,
so
it
really
enables
you
to
provision
the
services
match
data
plane
and
enables
you
to
monitor
it
and
names
it
to
secure
it,
so
that,
since
that's
pretty
well
described,
not
going
to
spend
most
of
the
time
looking
at
the
services
nation
that
they
are
playing
on
the
data
play
side,
you
actually
do
have
to
implementation
choices
when
it
comes
to
the
services
match.
Data
playing
one
is
service.
A
Product
service
itself,
plus
the
proxy
in
a
container
where
you
deploy,
and
maybe
a
para,
such
as
unit
which
will
announce
back
in
back
in
September
unit,
actually
enables
you
to
run
workloads
within
the
same
container
that
the
services
match
is
happening
across.
So
in
essence,
you're
deploying
just
one
element
in
your
network.
You
don't
have
to
deploy
a
sidecar
and
the
second
implementation
pattern
which
are
going
to
be
describing
here.
It's
when
you
run
the
service
and
a
proxy
side-by-side
in
a
sidecar
like
environment.
So
what
have
we
done
at
nginx?
So
we
are.
A
We
had
various
that,
as
we
said,
we
have
four
patterns
that
we've
we've
recommended
our
customers
and
the
last
one
is
the
one
that
actually
comes
with
a
control
plane
that
you
can
manage
yourself.
Instead
of
building
a
lot
of
custom
toolings,
you
have
now
the
option
to
deploy
sto
and
deploy
nginx
and
as
the
services
proxy.
So
in
the
demo
that
I'm
going
to
show
is
we're
going
to
be
using
nginx
a
sidecar
proxy
instead
of
the
standard
proxy
that
comes
with
the
Eastern
environment,
and
some
people
actually
asked
us.
A
There
are
mainly
in
the
non
nginx
users.
Why
are
we
using
nginx
to
the
psycho
proxy?
Well,
there
are
several
reason
why
number
one
is
is
battle
tested
it's
been
around
for
13
years.
He
has
a
lot
of
use
cases
that
we've
accomplished.
It
has
powerful
configuration
directives
over
650
of
them.
The
current
easy
to
implementation
exposes
some
of
the
patterns,
but
there
is
lot
more
to
be
done
and
we're
going
to
be
showing
you
how
the
configurations
are
stitching,
stitched
together
with
no
services
match.
We
have
also
highly
programmable
interface.
A
We
are
able
to
easily
add
looool
and
you're
able
to
add
ng
script
in
order
for
you
to
program
the
proxy
itself
and
least
but
not
last
I
mean
last
but
not
least,
is
the
strong
community
backing
with
many
third-party
modules
and
in
the
demo
I'm
going
to
show
you
one
of
the
integrations
that
we've
done
with
Zipkin
from
our
partners
at
life
that
it
does
it
the
architecture?
What
does
the
implementation
look
like?
This
probably
looks
pretty
familiar
to
you.
It's
really
the
easier
control
plane
running
with
an
engine
next
cycle
proxy.
A
So,
instead
of
actually
you
deploying
envoy
you'll
be
deploying
nginx.
In
this
case,
we
want
to
make
sure
that
what
we
do
is
taste
compatible
with
with
East
your
adapters,
so
it's
pretty
much
transparent,
except
for
the
Installer
there's
gonna
bring
in
a
sidecar
for
you.
The
whole
thing
should
look
pretty
much
the
same
way
from
the
users
perspective.
We
want
to
be
as
transparent
as
possible,
so
the
cycle
injections
just
happen,
just
as
you
would
throw
this
your
deployment
and
then
we
want.
A
We
have
support
for
rules
for
various
policies,
MPLS,
monitoring
and
tracing
in
terms
of
the
architecture,
as
we
mentioned
that
we
we
have
it's
a
transparent
proxy,
the
proxy
implementation.
So
the
way
this
is
being
implemented,
we
have
an
agent
that
is
deployed
together
in
our
site,
with,
with
the
nginx
container
in
a
sidecar,
the
agent
doesn't
translation
from
Earth
and
from
pilot
and
generates
those
engine,
s
configuration
files
and
you
can
deploy
nginx
as
an
HTTP
proxy
as
well
as
transparent
TCP.
A
A
A
So
another
thing
that
I
believe
that's
very
important
regards
to
this.
This
is
an
open
source
project
and
you
can
visit
it
today
at
github,
slash
engine
mash.
That
I
mean
this
is
the
codename
the
project
name,
for
it
has
various
components.
Engine
mesh
is
actually
the
core
component
as
an
agent
that
runs,
so
you
can
make
modifications
to
it.
If
you
like
to
augment
it
with
additional
directives
right
now,
it
does
a
bare-bones
installation
enables
you
to
run
and
voice,
as
if
you
would,
with
wood.
A
Wood
enables
you
to
run
run
an
eStore
as
if
you
would
with
them
boy
it's
better
quality
is
comparative.
Is
it's
compatible
with
sto
0.2
to
12
and
we
encourage
you
to
participate,
we're
looking
for
for
contributors,
we're
looking
to
address
some
of
the
use
cases
you
may
have
that
you'd
like
to
see
nginx
running
with
so
now.
It's
time
for
a
demo.
A
A
A
So
we're
gonna
install
engine
mesh
initializer
in
order
for
us
to
swap
the
default
proxy
we're
going
to
deploy
SEO
as
a
demo
app,
which
is,
if
you've,
seen
the
easier
presentation.
It's
really
the
same
dem
up
we're
trying
to
make
sure
that
we're
compatible.
So
you're
gonna
see
how
everything
what
how
things
are
happening
under
the
hood
or
running
the
same
application
will
examine
the
nginx
sidecar
without
traffic
all
traffic
to
our
one.
A
We
only
want
to
run
use
keys
now
to
save
time
and
we'll
also
demonstrate
monitoring
with
Ravana
and
prometheus
as
well
as
Zipkin.
So
there
are
a
few
things
that
I've
done
before
I
got
started,
is
I've
actually
downloaded
each
do
0
I
mean
SEO
dice
0-2
that
dot-to-dot
12
an
engine
mesh
from
our
from
our
ng
mesh,
posit
or
e
and
I'll
say
install
kubernetes.
So,
let's
see
you
shouldn't
be
anything
here,
but
I
want
to
make
sure
that
command
comes
up.
Okay,
there
are
no
resources
found.
A
A
So
this
is
an
installation,
france,
installation
of
Sto,
except
for
installer,
which
I'm
going
to
do
separately.
You
have
East
your
certificate,
related
stuff,
egress,
ingress,
mixer
pilot,
and
you
also
have
parameters
and
refiner
for
monitoring
as
well
as
Zipkin,
which
currently
runs
independently.
It
has
not
yet
been
integrated
into
mixer,
so
we're
going
to
run
as
such.
So
let's
go
ahead
and
and
follow
the
first
step
and
install
ng
mesh
installer.
A
A
So
what
it
does
is
actually
tells
sto
that
issue
start
engine
X,
which
is
engine
mesh
for
us.
It
should
pick
it
up
from
dark
repository
and
install
that
as
a
sidecar
beside
proxy.
Clearly,
you
have
two
images
you
have
the
in
it
and
you
have
the
debug
for
this
demo
we're
going
to
be
using
the
debug
okay.
So
that's
the
only
change
I
she
had
to
make
before
installing
the
book
in
for
app
pretty
much.
Everything
else
should
be
transparent.
It
should
be
compatible
with
what
is
taught
us
today.
A
A
Oops
its
product
page,
so
this
is
the
book
Evo
app
and
if
I
go
ahead
and
refresh
I
should
see
different
type
of
reviews,
so
this
is
version
1
version
2
and
version
3.
So
this
allows
you
to
have
a
little
bit
more
contest
in
terms
of
what's
happening
what's
happening
here.
You
have
a
you,
have
various
pods
and
then
you
have
one
details:
pod,
one
product,
page,
pod,
our
ratings
and
three
versions
of
reviews.
So
let's
take
a
look
at
the
product
page.
A
So
if
you
take
a
look
at
what's
happening
in
that
container,
you
have
two
containers.
You
have
the
product
page
where
the
business
logic
is
really
for.
This
is
the
service
logic
for
the
product
page
itself,
and
then
you
have
these
two
approxi.
This
is
the
ng
mesh
box
official
currently
running.
So
let's
go
ahead
and
navigate
into
the
into
the
container.
A
A
The
configuration
of
her
has
several
things
in
it:
one
is
loadable
module
for
the
mixer
configuration.
This
is
the
Ross
model
module
which
I
described,
and
then
you
have
open
tracing
for
Zipkin.
That
has
been
done
by
the
elapsed
time
guys.
So
if
you
are
looking
into
that
for
those
of
you
that
are
familiar
with
the
HTTP
building
block,
you
have
all
been
tracing
on
and
a
bunch
of
open
tracing
configurations,
and
then
you
also
see
mixer
mixer
server
configuration
all
these
changes
have
been
pushed
down
to
us
by
the
pilot.
A
A
Okay,
so
by
convention
there
are
two
type
of
various
files
out
here,
and
some
of
the
files
are
actually
for
internal
consumption
by
Sto.
But
the
files
that
are
interesting
for
us
is
is
the
files
that
are
managing
the
book
info
app,
which
actually
help
us
to
run
important
90-80.
So
by
convention,
everything
that
has
to
do
with
English
sports
is
going
to
have
an
IP
address
the
IP
address
of
the
pod
and
port
980.
A
So
if
you
look
at
that,
you
look
at
HTTP,
10,
dot,
36.2,
dot,
31,
and
you
see
that
it
has
an
upstream
which
is
which
actually
serves
the
information
from
the
sidecar
proxy
interlocal
hall
support
90
80-
and
this
is
it's
listening
currently
on
a
port
that
is
provisioned
by
sto
in
in
the
IP
tables,
to
actually
to
push
information
to
the
local
host
at
twenty
thousand
and
seven.
And
then
it
has
the
mixer
configurations
that
have
also
been
pushed
by
pilot
and
we've
actually
configure
them
automatically.
Ok
and
then
the
other
one
part.
A
Ups
and
the
upstream,
you
see
that
you
have
3d
views,
so
this
is
the
reason
why
one
actually,
when
I,
have
down
refuel
reload
on
the
reviews
page,
you
see
that
you're
seeing
three
views
pages
do
not
want
those
are
the
directives
in
nginx
that
are
controlling
that.
So,
if
I
do
this
again
see
there
are
multiple
reviews
version
one
two
and
three.
So
let
me
go
ahead
now
and
change
the
reviews
to
push
everything
into
one
into
one
single
review
of
version
one.
So
if
I
I
can
do
that
by
running
easier
control,
still.
A
Here
we
go
now
what
this
has
done.
If
you
think
a
look,
what
happened
nginx?
If
you
do
load
this
file,
you
see
that
the
out
and
the
upstream
has
been
modified
now,
and
it
only
passes
one
IP
address
and
port
number.
So
if
we
go
back
to
the
application
itself
now
you
can
see
that
I
can
do
multiple,
pepper,
refreshes
and
only
version
one
is
being
shown
now.
If
you
go
ahead
and
change
this
again
and
remove
the
route
rules,
oops.
A
And
I
reload
the
file
again.
So
what
I'm
trying
to
display
here
is
that
what
I'm
trying
to
show
here
is
that
each
tio
passes
the
information
as
it
would
through
the
normal
channel
and
engines
picks
them
up
and
it
does
it.
It
creates
its
configuration
files
real
transparently.
So
you
don't
have
to
make
any
other
modifications
norther
for
you
to
swap
out
the
default
proxy
with
with
the
nginx
proxy.
Now,
as
you
saw
previously
in
the
demo,
I
have
actually
provisioned
graph
Anna
and
provision
Zipkin,
so
I'd
like
to
show
how
they
operate.
A
So
since
it's
been
running
in
the
background,
I
would
expect
to
see
already
some
graphs
just
be
deployed
and
in
the
sto
configuration
file,
so
I
pasted
your
dashboard
yeah
it
is
you
see,
graphs
showing
up.
So
if
I
refresh
the
page
a
few
times
just
to
show,
this
is
real
I,
don't
just
make
them
up.
You
should
see
a
spike
in
the
graphs
here
we
go
see
the
grass
coming
in
okay,
an
extra-credit,
let's
see
what,
as
it
can
also
works.
A
Don't
think
I
open
up
or
think
you
can't
control
zip,
you
find
traces
here
they
are
now
you
can
trace
the
traffic.
You
trace
the
packets
that
are
moving
all
the
way
in
from
the
English
side,
all
the
way
in
through
the
system,
so
you
can
find
out
what
I
have
any
delays
or
anything
else.
So
that's
pretty
much
it
from
the
demo
side.
A
A
But
right
now
the
go
so
to
restate
the
question:
what
is
the?
What
are
the
things
that
the
NGO
mesh
does?
The
default
proxy
does
not
provide.
The
goal
for
us
was
to
keep
everything
for
backward
compatible,
meaning
that
I
want
to.
We
want
to
make
sure
that
whatever
we
do
in
nginx
doesn't
break
anything
that
is
provides
for
us,
so
we
try
to
stay
in
par.
So
at
this
point
it
doesn't
do
things
differently,
because
there
was
never
a
goal
moving
forward.
Now
we
want
to
start
exposing
nginx
directives
such
as
single
sign,
ons.
A
Caching,
we
have
several
dozen
third-party
libraries
and
you've
seen
one
that
was
with
Zipkin.
We
won't
expose
a
lot
of
others,
but
we
need
to
do
that
in
a
way.
That's
compatible
resume,
so
we
need
to
provide
it.
We
need
to
start
mapping
it
at
the
top
level
and
then
building
it
down
to
a
system,
and
that
requires
some
code
changes.
So
if
you're
interested
in
some
of
that,
then
you
have
you'd
like
to
accomplish
use
case
with
what
nginx
were
more
than
happy
to
to
take
a
feedback
and
build
out.
A
At
this
stage,
we
do
reload
config
updates,
but
we
are
very
tactical,
meaning
that
if
all
we're
doing
is
actually
change
it
upstream,
we
actually
can
do
it
through
the
API,
but
we're
keep
in
mind
and
checks
the
way
it
operates.
It
reloads
the
configuration
it
has
a
soft
reload
and
therefore
the
workers
will
continue
processing
traffic
and
we
can
decide
how
quickly
we
should
age
and
how
aggressively
we
want
to
age.
Those
workers.
A
A
A
A
A
So
passenger
application,
if
you
integrate
a
passage
or
application,
it
should
be
passage.
Application
should
be
branch
transparent
because,
as
you
seen
when
we
are,
if
you
have
a
kubernetes
file,
a
yeah
Mel
file
that
you're
trying
to
push
just
like
booking
for
would
be
you
push
it
in
and
nginx
will
digest.
It
will
create
a
configuration
files
for
you,
so
you
can
run
it
transparently.
In
a
sense,
it's
gonna
stitch
up
the
security
channels
as
well
for
you.
A
External
okay,
oh
I,
got
it
so
part
of
the
part.
You've
mentioned
that
part
of
the
the
deployment
I
have
that
I
look
to
the
service
to
pick
out
the
IP
address.
Yes,
that
is
something
that
we
are
running
this
all
in
Google
Cloud.
So
it's
provided
by
it's
everything
is
done.
Live
I
mean
it's
an
environment
that
we
set
up.
It's
visible.
You
probably
can
right
there,
pure
self.
So
it's
it's
provided
by
Google.
A
A
How
difficult
was
it
right
time
makes
a
mixer,
glider,
rust
I
would
say
it's
taken
us
a
few
months,
so
it
wasn't.
It's
not
a
simple
exercise:
you're
free
to
take
a
look.
It's
it's
an
open
source.
It's
an
engine
mesh!
You
can
open
it
up
and
see.
How
here
is
the
architect
you
can
have
and
outside
the
conversation
with
him,
he
he's
the
one
and
and
a
small
team
with
Michael
and
a
few
other
people
have.
Are
the
ones
writing
itself?