►
From YouTube: ROS 2 Hardware Acceleration Working Group - meeting #8
Description
Eighth meeting the ROS 2 Hardware Acceleration Working Group.
We discuss the group's progress during the last month showing how updates of REP-2008 to reflect cloud extensions, discuss ROS 2 perception Nodes performance using AMD KV260 and NVIDIA Jetson Nano and discuss how benchmarking results hint that FPGAs obtain 500x performance than GPUs.
The session also includes a guest talk provided by Jeffrey Ichnowski from the RISE lab and AUTOLAB at the University of California at Berkeley, which together with his team led “FogROS 2: An Adaptive and Extensible Platform for Cloud and Fog Robotics Using ROS 2”. This work led to ROS VSLAM (ORB-SLAM2) latency reductions of 50%, grasp planning optimizations going from 14s to 1.2s, and motion planning speedups of 28x.
For more including source code, check https://github.com/ros-acceleration.
00:00 Welcome
00:30 Intros
03:07 Ultra96-v2 port status report
06:20 ROSCon 2022 Hardware Acceleration Workshop
08:05 Robotic Processing Unit update
09:45 ROS 2 perception Nodes performance benchmarking
16:45 REP-2008 cloud extensions
21:55 Jeffrey Ichnowski, FogROS 2: An Adaptive and Extensible Platform for Cloud and Fog Robotics Using ROS 2
52:13 Q&A
For a discussion about this meeting, refer to https://discourse.ros.org/t/hardware-acceleration-wg-meeting-8/25147.
A
Welcome
everyone
to
the
eighth
hardware,
acceleration
working
group
today
with
asphalt.
We
have
jeff
and
eric
from
berkeley
they'll
be
telling
us
about
fantastic
work.
They've
been
leading
on
essentially
cloud
robotics
and
we'll
listen
to
that
in
the
second
half
an
hour
of
the
meeting
as
usual,
let's
kick
off
the
meeting
by
letting
people
that's
new
to
the
group,
making
some
self
interaction.
So
folks,
the
stage
is
yours,
maybe
jeff
and
eric.
Would
you
like
to
to
get
started
with
intros.
A
D
E
F
G
Hi
I'm
delay
so
I'm
currently
working
as
an
embedded
systems
engineer
interested
in
robotics,
I'm
also
doing
a
master's
by
research
in
the
area
of
machine
learning,
so
I'm
quite
interested
in
implementing
stuff
on
fpga.
Well,
I
saw
like
a
publication
on
this
robotics
solutions
you
guys
working
on
and
that
got
me
interested
in
joining
this
group.
A
A
A
What's
gonna
drive
the
reference
hardware
platform
so
far,
most
of
the
contributions
have
gone
towards
these
amd
kriya
kv-260,
which
made
it
the
reference
hardware
platform,
but
we
are
constantly
evaluating
and
considering
other
boards
so
that
they
get
added
also
as
co-official
boards.
For
that
we
have
a
number
of
guidelines
which
typically
get
summarized
in
this
ongoing
standardization
effort
for
hardware
acceleration
architecture
and
conventions.
I'm
gonna
paste
this
in
the
chat
as
well.
A
For
those
of
you
know
everyone's
welcome
to
review
this
document
and
add
his
or
her
view
to
it.
In
a
nutshell,
it
is
an
open
document
and
follows
the
common
guidelines
of
the
ross
enhancement
proposal.
That's
what
rep
stands
for
going
back
to
the
reference
hardware
boards,
one
of
such
board
that
has
been
going
on
for
quite
a
while
is
the
ultra
96
v2
pretty
early
within
the
working
group.
There
was.
A
There
was
a
number
of
folks
actually
that
have
shown
interest
and
I've
been
supporting
them
on
the
side
to
make
sure
that
this
gets
enabled
pedro
martos
has
been
very
active
in
leading
us
as
a
community
to
get
support
for
the
av
net
ultra
96v2
and
it's
still
an
ongoing
work.
As
of
last
message,
I
think
pedro
managed
to
get
some
decent
progress
and
build
some
of
the
first
examples.
A
Regarding
the
hardware
acceleration
workshop
for
the
upcoming
roscom
2022,
that's
going
to
happen
in
tokyo
in
a
few
months.
Essentially
that
is
tracked
in
here.
So
last
time
I
informed
about
the
intention
to
present
a
workshop.
Unfortunately,
after
re-evaluating
it
due
to
a
pure
lack
of
resources
on
my
side,
I'm
gonna
have
to
step
down
on
the
organization
of
the
workshop
for
the
on
the
hardware
acceleration
workshop
for
roscon.
A
Prospect,
I
think,
what's
left
is
essentially
to
come
up
with
a
description
and
to
invite
a
number
of
guest
speakers
more
than
happy
to
facilitate
that
list
and
even
to
help
engage
them.
The
only
thing
is
that
I'll
be
over
overloaded,
very
overloaded
with
work,
and,
frankly
speaking,
I
don't
see
myself
finding
the
bandwidth
to
do
the
proper
organization
that
formal
workshop
requires
and
use
fyi
from
past
experiences.
Workshops
require
a
significant
amount
of
work.
A
You
can
expect
from
30
to
50
participants
on
each
one
of
the
workshops,
and
this
happened
the
first
day,
the
first
day
of
the
conference.
It's
a
three-day
conference,
so
typically
the
first
day
is
where
workshops
happen.
I
definitely
encourage
everyone
if
attending
to
to
show
up
in
any
of
the
workshops
in
one
because
they
happen
simultaneously.
A
At
least
it
really
saddens
me
that
we
will
not
be
organizing
a
hardware
acceleration
workshop,
or
at
least
that
I
will
not
be
behind
it,
but
I'm
more
than
happy
to
support
if
anyone
has
interest
to
drive
it.
Just
let
me
know,
and
I'll
be
more
than
happy
to
hand
over
this
to
whoever
wants
to
have
or
wants
to
do
that.
A
Okay.
So,
regarding
the
robotic
processing
unit,
which
is
a
project
that
we
announced
in
one
of
the
past
meetings,
there's
been
pretty
interesting
feedback
provided
by
some
of
you
and
some
others
who
are
not
participating
in
the
working
group
meetings.
Special
thanks
to
theo
guglielmo
joe
and
a
few
others
in
the
working
group
there's
been
your
input
has
been
received
and
I'm
slowly
integrating
it
for
anyone
that
deserves
or
sorry
desires,
to
provide
additional
feedback.
A
A
A
To
most
of
you
might
seem
obvious,
but
to
many
out
there
in
the
robotics
communities
is
still
something
to
be
discovered.
The
fact
that
essentially
hardware
acceleration
is
critical
for
edge
perception
and
most
of
the
fancy
robotic
behaviors
we
tend
to
see
out
there
actually
do
include
robotic
perception.
This
is
one
of
the
many
and
some
of
you
may
know
this
one.
This
is
atlas.
A
Atlas
uses
exhaustively
hardware,
acceleration
to
ensure
that
that
it
can
perform
these
movements.
This
dexterity
on
the
go
and
at
the
edge
of
course
it
offloads
certain
computations,
but
still
it
is
those
capabilities
that
hardware
acceleration
capability
at
the
edge.
What
really
empowers
many
of
the
fantastic
movements
we
we
actually
see.
So
with
that
motivation
in
mind,
one
of
the
open
questions
that
I've
been
receiving
so
far
has
been
well
so,
which
is
the
right
accelerator.
A
Should
I
be
using
gpus
or
fpgas
and
previous
reviews
we
performed
within
the
working
group,
analyzed
this
from
a
roster
computational
graph
perspective
and
so
to
build.
On
top
of
that,
this
new
study
that
I
disclosed
a
while
ago.
Sorry
not
this
link,
but
actually
this
one
essentially
builds
on
top
of
those
graph,
computational
graph
inspections
and
looks
into
particular
roster
notes.
A
In
particular,
it
tries
to
answer
that
question
and
looking
specifically
at
fpgas
and
gpus
in
particular
for
the
fpgas.
We
select
the
amd
kriya
kv-260
reference
platform
for
gpus
we
use
or
select
for
benchmarking
purposes
the
nvidia
gets
on
nano.
The
rationale
behind
this
is
because
those
are
the
ones
I
have
easily
available
and
also
because,
frankly,
speaking
from
a
cpu
centric
perspective,
the
roster
performance
is
somewhat
comparable
in
those
two
development
platforms.
A
The
amd's
one
features
I
believe,
and
a
quad-core
a53,
whereas
the
nvidia
one
an
a57
and
quad-core
a57.
So
so,
when
testing
things
roast
too
wise
and
again
from
a
cpu
perspective,
the
performance
is
similar,
so
that
allows
us
to
get
also
a
decent
comparison
of
of
the
actual
accelerators
themselves,
also
to
be
specific,
specifically
fair.
A
In
this
case,
the
benchmarking
approach
that
has
been
used
is
the
exact
same
one
that
we've
been
using
so
far
in
any
of
the
past
few
meetings-
and
you
can
see
described
in
here
in
the
wrap
2008
that
I
pasted
before
in
the
chat.
So
you
can
see
here
the
proposal,
the
rationale,
the
reasoning
behind
it,
and
how
to
perform
this
systematic
assessments
and
benchmarks
for
performance
benchmarks
by
following
this
methodology.
So
by
following
this
and
more
specifically,
to
be
more
precise
and
isolate
things
to
the
acceleration
kernels.
A
What
we
are
doing
in
here
is
actually
focusing
and
discarding
part
of
the
compute
time
and
I'll
touch
on
that
in
just
a
bit
from
an
implementation
perspective,
some
of
or
actually
many
of
the
of
these
nodes
are
actually
available,
the
acceleration
examples
or
similar
packages
linked
in
the
working
group
organization,
but
in
a
nutshell,
what
they
do
is
they
labor?
They
leverage,
sorry
amd's
by
this
vision,
library
and
nvidia's
vision,
programming
interface,
to
put
together
a
number
of
perception
notes
that
are
compared
one-to-one.
A
We
are
essentially
capturing
and
discarding
both
the
roster
message,
passing
infrastructure
and
the
host
device
interactions
on
data
transfer
overheads.
This
happens
both
between
the
cpus
in
the
amd
solution
and
between
the
ones
in
the
nvidia
solution
as
well.
It's
just
whether
they
interact
with
the
fpga
or
the
gpu
now
going
into
the
results.
A
Then
we
have
more
complex
algorithms,
such
as
the
histogram
of
oriented
gradients,
wherein
the
difference
is
significant,
very
significant
and
others
were
in.
Essentially,
we
also
get
interesting
speed
ups
for
us
to
consider
when
looking
beyond
pure
runtime
and
diving
deep
into
actually
the
power
consumed.
A
The
observation
overall
and
on
average
is
that
in
general,
the
fpga
counterparts
are
more
power
efficient
or
five
times
more
power
efficient.
This
is
a
rough
average
there's
there's
there
are
some
nodes
where,
in
the
actual
performance
per
watt,
difference
was
significantly
higher
than
five
times
and
there
were
others
where
it
wasn't
that
high
beyond
these
results
shown
and
as
we
have
repeatedly
been
mentioning,
at
least
this
is
my
viewpoint.
A
A
A
So
in
a
nutshell,
the
extensions
have
added
a
fourth
tyler,
as
you
can
see,
and
some
of
you
may
remember,
the
the
overall
architecture
proposed
was
based
on
three
pilots:
extensions
to
the
roster
pl
system,
extensions
to
reverse
two
meta,
build
tools
to
ensure
that
everything
can
be
orchestrated
directly
by
following
the
roster
development
flow
extensions
or
better
than
extensions.
The
addition
of
a
new
firmware
layer
which
allowed
to
switch
from
accelerators
by
simply
just
enabling
one
flag
while
keeping
sorry
the
same
development
flow.
A
And
finally,
with
the
latest
extension
or
addition,
we
have
now
fourth
pilar
the
cloud
pilar,
which,
once
again
dives
into
ensuring
that
cloud
capabilities,
specifically
hardware
acceleration
in
the
cloud
can
be
pushed
in
the
same
way.
We
build
accelerators
for
edge
devices
technically
from
a
technical
perspective.
A
The
only
thing
that
changes
is
the
fact
that,
instead
of
targeting
an
embedded
edge
target,
we
need
to
pass
a
flag
that
targets
a
particular
instance
in
the
cloud,
so
it's
instance
specific
and
also
cloud
specific
in
a
way.
But
regardless
the
complexity
is
abstracted
away
from
the
roster
developer,
and
the
only
thing
you
need
to
do
is
pass
the
right
flag
for
the
right
instance
automatically
all
of
the
build
system
will
get
configured
for
you.
A
It
will
generate
the
corresponding
artifacts
automatically,
so
many
people
jumping
in
sorry,
guys
and
and
yeah
once
those
artifacts
are
generated
for
the
cloud
you
can
directly
just
push
them
into
your
corresponding
cloud
instances,
and
that
again
is
something
that
jeff
and
eric
will
tell
you
more
about
again
just
to
highlight
it.
Some
of
the
notes
I
took
in
here
this
effort
of
the
cloud
extensions
to
the
hardware.
Acceleration
architecture
aims
to
bridge
the
gap
and
simplify
the
use
of
hardware
acceleration.
A
Also
in
the
cloud
we
believe
there's
lots
of
potential
in
here
and
and
certainly
as
ross
2
computational
graphs
get
spread
across
essentially
edge,
workstation
and
cloud.
This
is
going
to
be
a
big
need
again.
The
way
we
are
pursuing
this
or
approaching
this
is
in
a
completely
vendor
agnostic
and
hardware
agnostic
manner.
So,
regardless
of
whether
you
are
working
with
gpus
or
fpgas,
you'll
be
able
to
do
the
same
in
the
cloud
provider.
Of
course,
your
cloud
service
provider
offers
those
instances,
but
that
is
your
choice.
A
And
last
but
not
least,
this
effort
also
seeks
to
achieve
a
unified
way
to
leverage
hardware
acceleration,
so
that
people
choose
needs
to
learn
things
once.
This
is
one
of
the
biggest
entry
barriers
that
I've
been
encountering
and
that
people
has
been
telling
me
that
they've
been
encountering-
and
this
is
also
based
by
the
way
on
the
survey
we
conducted
earlier
this
year-
an
fyi
that
the
data
is
being
processed.
As
of
now,
and
once
it
gets
finalized,
it
will
be
edited
by
a
design
team
and
then
it
will
be
published.
A
I
think
that
it
shouldn't
take
too
long
for
that
to
to
be
finalized,
but
yeah.
This
is,
in
a
nutshell,
the
exciting
pieces
that
I
brought
for
you
all
today
very
excited,
frankly
speaking
about
the
cloud
extensions,
and
hopefully
the
talk
today
from
jeff
and
eric
will
further
enlighten
this
topic
and
bring
your
interest
as
well.
A
None
all
right
so
then
it
is
a
huge
pleasure
of
mine
to
introduce
and
hand
it
to
jeff
and
to
eric
who
are
essentially
working.
As
I
said
in
berkeley
connected
to
the
auto
lab
and
together,
they
will
tell
us
about
focus
2,
an
adaptive
and
extensible
platform
for
cloud
and
fog
robotics
using
ross2,
which
I'm
very
optimistic
about,
is
hopefully
going
to
get
accepted
within.
I
was
the
classic.
B
Great
thanks
for
the
intro
and
thanks
for
having
us
here.
This
is
great,
so
I
do
want
to
preface
this
talk
by
saying
you
guys
are
getting
the
first
view.
First,
access
to
the
rough
cut
presentation
of
this
work,
we're
definitely
open
to
feedback,
criticisms,
ideas
and
so
forth.
I
would
invite
you
to
interrupt
me
at
any
time
during
the
talk.
If,
if.
E
B
B
All
right
awesome,
so
yeah,
so
I
want
to
tell
you
about
cloud
robotics
and
this
project
we're
calling
fogros
ii
and
I
had
a
little
trouble
trying
to
gauge
the
you
know
the
right
level
of
content
for
this
audience.
I
do
understand
you
guys
know
ross
pretty
well,
but
I
wasn't
sure
about
the
cloud
side.
So
I'm
gonna
start
off
with
you
know.
What
is
the
cloud?
Probably
a
pretty
basic
question?
B
Well,
the
cloud
you
know,
maybe,
as
simply
as
I
can
put,
it
is
on-demand
pay-per-use
access
to
network
computing
resources
and
the
software
and
databases
that
run
on
them.
So
one
of
the
main
selling
points
for
it
is
really
that
cloud
computing
freeze,
user
from
having
to
manage
and
purchase
hardware
and
from
running
the
software.
B
But
from
what
I
hear
from
people
who
are
experts,
the
fpgas
that
they
provide
are
big
now.
Another
part
that's
really
kind
of
exciting
about
the
cloud
is
that
you
get
essentially
unbounded
access.
To
these
I
mean
really.
Your
bounds
are
going
to
be
on
your
budget,
but
if
you
need
100
fpgas,
you
can
get
100
fpgas
and
then
use
them
for
an
hour
and
then
stop
using
them.
B
B
But
if
you
think
about
the
possible
number
of
cloud
configurations,
it's
actually
kind
of
daunting
and
you
multiply
that
by
the
number
of
robots
and
tasks
that
they
can
perform,
that
just
becomes
even
bigger
numbers.
So
I
think
one
of
the
ideas
that
I
was
thinking
through
is
like.
How
can
we
get
an
idea
of
what
cloud
robotics
mean
by
just
going
through
sort
of
a
and
what
the
future
is
by
just
going
through
a
brief
and
borrowed
history
of
robotics
in
the
cloud?
So
I
found
this
great
visual
on
the
internet.
B
B
B
So
that's
a
really
interesting
way
of
thinking
about
the
cloud,
but
the
cloud
is
essentially
a
utility
of
computing
and
the
history
is
somewhat
mundane
for
a
little
while,
yet
arpanet
network
operating
systems
internet
are
forming
over
the
next
couple
of
decades.
The
main
history
really
starts
in
2000,
with
the
launch
of
aws
and
from
then
you
can
start
to
see
this
explosion
of
growth.
It's
as
its
utility
and
popularity
has
grown.
B
So
I'm
sure
you
have
other
key
takeaways
feel
free
to
interrupt
with
them.
I
think
it
would
make
for
a
great
discussion
too,
but
at
this
point
I
want
to
compare
robots
and
the
cloud
and
see
how
they
might
complement
each
other
and
why
I'm
excited
about
this
and
why
I'm
actually
excited
about
hearing
like
the
hardware
acceleration
cited
in
perspective
of
this.
So
just
looking
at
how
we
compare
these
or
contrast
these
the
robot
has
a
huge
upfront
cost.
B
It's
usually
you
know,
let's
say
a
hundred
thousand
dollars,
it's
all
capital
expenditure,
you
just
pay
for
it
up
front,
and
then
you
got
a
robot
that
maybe
will
last
you
know
10
plus
years.
On
the
other
hand,
the
cloud
gives
you
on-demand
pricing,
so
you
get
the
computing
as
you
need
it,
and
what
computing
you
need
and
it
seems
to
be
trending
along
with
moore's
law.
So
as
you
wait,
you
know
a
few
years
you
get
access
to
more
and
more
computing
capabilities,
and
you
know
previous
sort
of
high-end
computing
becomes.
Cheaper.
B
Robots
are
often
designed
up
front
to
consider
power,
weight
and
mobility
requirements
and
that
often
limits
their
computing
capability,
whereas
the
limits
on
the
cloud
is
really
the
cost
limit.
So
you
have
a
lot
of
computing
capabilities,
but
it's
just
how
much
you're
willing
to
spend
on
it
on
the
robot
side-
and
I
think
this
is
where
you
know
some
of
the.
What
victor
was
mentioning
earlier-
that
hard
real-time
safety
critical
aspect
is
really
important
to
address
on
the
robot.
B
Just
the
latency
to
the
cloud
is
probably
an
instrumental
barrier
at
this
point
in
time,
maybe
not
in
the
future,
but
if
you've
got
something,
that's
absolutely
absolutely
safety
critical
you've
got
to
run
on
the
robot,
whereas
if
you
want
to
get
more
out
of
the
robot,
you
want
to
get
the
robot
to
do
more
things
per
unit
of
wall
clock
time.
You
can
look
to
the
cloud
for
more
performance
and
parallelism.
B
You
can
look
at
to
bring
new
capabilities
that
maybe
aren't
safety
critical
to
the
robot
and
maybe
finally,
in
that
last
row,
the
the
fun
idea
I
was
thinking
about
is
that
algorithms
and
robots
are
outpacing.
The
capabilities
of
robots
you've
got
a
10
year
old
robot.
You
cannot
run
a
modern
neural
network
based
program
on
it
efficiently
and
meanwhile
the
cloud
is
the
available
computing
is
always
outpacing,
algorithms,
like
even
the
largest
language
models,
which
is
something
like
half
a
billion
weights,
and
it
is
still
under
the
capabilities
of
what
the
cloud
can
provide.
B
So
now
that
I've
talked
about
cloud
and
robots,
what
is
cloud
robotics
then
kind
of
revisiting
that?
Well,
the
definition
is
open,
but
here's
a
few
ideas
so
with
big
data
that
we
can
get
in
the
cloud,
we
can
start
to
think
about
doing
collective
learning
with
on-demand
computing.
We
can
start
to
take
on
more
tasks
faster,
better
more
efficiently.
We
can
start
to
think
about
human
robot
interaction
using
these
large
language
models.
You
know,
take
your
amazon
alexa
as
an
example.
B
Sorry,
if
I
triggered
it
with
a
keyword-
and
you
know,
fleet
management,
you
can
have
network
management
of
large
distributed
fleets
of
robots
and
at
present
we're
sort
of
increasingly
dependent
on
the
cloud
and
remote
presence
like
like
right
now
on
the
zoom
call
or
I'm
sorry.
Google
me
call-
and
this
has
really
been
leading-
this
has
been
something
that's
also
led
to
increased
adoption
of
the
cloud
you
know.
B
It's
also
created
huge
demand
for
more
computing
networks,
data
things
that
are
increasingly
only
available
through
the
cloud
and
we've
also
seen
an
explosion
of
internet
of
things,
things
with
low
computing
capabilities
that
often
need
the
cloud
to
gain
any
capabilities,
and
you
could
put
you
know,
low-end
robots
in
there
as
well,
and
so
one
of
the
main
concerns
that
we're
often
thinking
about
is
the
cloud
has
network
latency.
So
how
do
you
use
it
on
a
robot?
B
But
one
of
the
things
that's
kind
of
exciting
is
that
new
technologies
in
5g
and
up
like
5g
and
the
upcoming
6g.
This
concern
sort
of
diminishes
and
enables
new
approaches
that
we
may
not
have
considered
in
the
past
so,
for
example,
consider
self-driving
cars
by
most
definitions.
These
are
safety,
critical
robots,
a
timing.
Failure
would
lead
to
harm
or
death,
but
when
self-driving
cars
run
into
problems
that
their
ai
cannot
handle,
remote
operators
can
take
them
over
and
drive
them
remotely
and
safely.
B
So
the
future
of
cloud
robotics
is
looking
bright
and
I
would
argue
necessary
to
get
to
a
future
where
robots
are
increasingly
helpful
in
everyday
life.
We
can
start
thinking
about
the
nature
of
robots
in
the
cloud
and
how
we
can
get
the
the
best
performance
benefit
out
of
the
cloud.
Can
we
start
looking
at
the
edge
or
fog
layers
to
get
more
capabilities
at
reduced
latency,
and
can
we
take,
for
example,
like
the
aging
population
as
the
population
ages,
and
we
have
fewer
and
fewer
active
people
relative
to
retired
adults?
B
We
need
to
start
looking
for
new
ways
to
to
meet
labor
demands,
or
maybe
we
could
just
start
thinking
about
things
like.
Wouldn't
it
be
great
if
you
left
the
house
in
the
morning
and
it
was
a
mess
and
a
robot
took
care
of
everything
for
you,
while
you're
gone
and
cleaned
up
everything
for
you-
and
these
are
things
that
we're
looking
at
in
the
cloud
to
enable
and
we're
starting
to
see
these
capabilities.
We
have
compute
where
the
compute
demand
is
really
high.
B
We
can
start
leveraging
the
cloud,
so
I'm
gonna
skip
through
the
details
of
this,
but
this
is
some
of
the
research
that
we
did
in
the
auto
lab
at
berkeley,
where
we
looked
at
how
we
could
distribute
the
processing
of
that
decluttering
task
that
I
showed
you
where
we
could
do
a
lot
of
learning
using
the
computing
power
of
the
cloud.
We
could
use
edge
networks
to
make
adaptation
and
inferencing
much
faster
in
lower
latency,
and
this
is
a
part
that
I'm
really
going
to
skip
and
I'm
going
to
get
to
another
example.
B
That's
actually
kind
of
a
running
example
that
we
use
in
the
the
fog
ross
part
which
I'm
building
up
to
but
say
you
have
a
robot
that
wants
to
get
around
a
room.
Well,
one
of
the
really
effective
ways
to
do
this
is
called
sampling
based
motion
planning
and
you
can
see
from
this
animation
I'll,
just
restart
it
that
it
works
by
just
randomly
sampling
configurations
in
space
and
connecting
them
into
a
tree
and
sort
of
improving,
as
you
see
it
more
and
there's
a
really
effective
way
of
speeding.
B
This
up
is
just
parallelize
the
process
sample
more
samples
concurrently
and
build
the
tree
faster
and
I'm
going
to
skip
this
part,
but
basically
I'm
going
to
say
that
we've
been
able
to
overcome
some
scalability
limits
in
some
private
research
and
are
able
to
do
processing
like
this.
So
you
have
a
one
core
machine.
You
might
get
a
motion
plan
that
looks
like
this.
You
put
four
cores
on
it.
You
get
a
plan
that
looks
like
this.
B
You
put
32
cores
on
it
and
you
get
what
looks
to
be
a
near
optimal
plan
and
the
same
amount
of
wall
clock
computing
just
by
putting
more
computing
more
parallelism
on
it.
We've
even
extended
this
to
very
high
dimensional
problem
plan
planning,
and
you
put
it
on
the
cloud.
So
here's
the
the
key
fun
part
of
that
project.
B
So
let's
say
we
go
back
to
that
decluttering
problem
and,
let's
say
a
robot's
like
decluttering,
your
your
lab
space,
your
office
space,
your
school,
wherever
it
is,
and
it's
got
to
navigate
between
rooms.
Well,
navigation
is
a
pretty
easy
problem
for
a
robot
to
solve
on
board
most
of
the
time.
But
when
you
get
to
a
manipulation
problem,
where
it's
got
to
move
that
multi,
you
know
multi-degree
of
freedom
arm,
it's
becomes
a
higher
difficulty
problem.
The
the
difficulty
of
motion
planning
grows
exponentially
with
the
degrees
of
freedom.
B
So
you
start
to
get
these
high
cpu
time
problems
and
maybe
looking
at
another
way
if
you've
got
a
robot
that
has
this
sort
of
cpu
usage
over
time.
It
might
solve
a
simple
problem
and
then
not
be
using
computing.
While
it's
moving,
you
know,
switches
to
a
complex
problem
and
then
again
the
robot's
moving
and
not
computing
anything
and
you
might
get
to
a
very,
very
complicated
process
problem
where
you've
got
to.
B
B
Now
it
has
a
bunch
of
limits
I'm
going
to
skip
through
all
of
them,
except
for
one
of
the
key
details
where
it's
got
very
limited
processing
capability.
It's
only
got
two
hardware
threads
there's.
As
far
as
I
know,
there's
no
gpu
or
fpga
offering
yet,
but
I
suspect
there
will
be,
and
one
of
the
things
that
we
looked
at
is
well.
Could
you
well
if
one
multi-core
lambda,
you
know
it's
probably
too
slow.
It's
probably
going
to
be
about
the
same
computing
as
you
have
on
the
robot.
B
Just
imagine
the
one
lambda
being
what
the
robot
can
compute
by
itself
and
in
the
same
amount
of
wall
clock
time.
We
just
get
a
vast
amount
of
improvement
if
we
use
100
lambdas
and
this
trend
increases
shows
across
the
board
with
all
these
different
problems,
but
putting
it
back
onto
that
motivating
example
of
the
robot
decluttering
at
desk.
If
you
have
the
same
amount
of
wall
clock
computing
time,
you
do
one
lambda
in
parallel
or
just
the
robot
itself,
10
lambdas
in
parallel
or
100
lambdas.
B
In
parallel
using
this
process
that
I
sort
of
skipped
through
quickly.
You
can
see
that
the
hundred
landos
gives
us
a
faster
solution
to
our
target
grasp
point
for
that
robot
than
we
got
with
the
10
lambdas,
and
in
that
same
amount
of
wall
clock
time,
the
robot
could
not
compute
a
single
solution,
so
we
can
go
from
not
computing
a
solution
to
computing,
a
very
good
solution
by
just
adding
literally
a
few
pennies
to
the
computation
using
the
cloud
using
these
methods
and
that
sort
of
exciting
point
goes
back
to.
B
I
think
that
point
where
you
have
this
hundred
thousand
dollar
robot
and
you
want
it
to
say
declutter
the
hall,
but
it
can't
do
it
all
in
one
night,
so
you
need
to
get
two
robots
using.
You
know
onboard
computing,
but
if
you
can
speed
up
efficiency
enough,
you
can
say
well
using
the
cloud
spending
an
extra
buck
50
a
night.
We
can
get
the
robot
to
declutter
the
whole
place
at
once
in
one
go
so
that
sort
of
led
to
this
thought
process
well
using
cloud
robot
cloud
for
robots
isn't
easy.
B
B
Robot
parts
in
the
cloud
and
ross
in
particularly
ross2
became
a
really
good
place
to
start
with
that.
Maybe
the
best
way
to
do
this
or
talk
about
this
is
just
go
through
an
example.
So
here
we
have
this
robot
application.
It's
got
four
nodes
in
it.
It's
got
a
sensor,
node
a
control.
Node,
it's
got
a
grass
planer
node
that
really
benefits
from
the
gpu.
B
It's
using
a
neural
network
to
look
at
an
image
to
compute
where
it
should
grasp
an
object,
stably,
and
it's
got
a
motion
planar
node,
which
is
like
one
of
those
ones
that
I
showed
you
before,
which
really
benefits
from
a
lot
of
cpus.
But
the
robot
itself
only
has
two
cores
and
no
dq.
So
what
do
we
do?
Well
with
fog
ross?
B
We
say:
let's
look
at
this
launch
script
and
let's
just
change
the
launch
script,
we're
going
to
say
we
want
one
of
these
nodes
to
run
on
one
of
the
cloud
instances
we're
going
to
say
at
the
top.
Here
we
want
to
use
the
us1
region,
that's
the
one
closest
to
berkeley.
We
want
to
use
this
particular.
You
know
magic
string
of
g4dn,
which
translates
into
a
machine
with
the
gpu
and
I'll
get
to
that
in
a
second,
and
then
we
just
say
on
the
the
bottom
part.
B
We
want
that
robot
node,
that
grass
planner
nodes
to
run
on
that
machine
that
we
just
specified
and
after
we
do
that
and
we
launch
it
bob
ross
takes
care
of
the
rest.
It
moves
those
nodes
into
the
cloud.
It
sets
up
all
the
machinery
that
you
need
in
the
cloud
that
sets
up
secure
communication
between
the
the
robot
node
on
the
robot
and
the
robot
nodes
on
the
cloud
and
essentially
makes
it
all
happen
and
from
the
coding
perspective,
there's
not
been
a
line
of
chain
code
change.
B
B
B
Then
two
it
will
deploy
all
the
ros
dependencies
and
the
application
workspace
dependencies
into
that
virtual
machine.
Then
three
it
will
set
up
the
virtual
private
networking,
the
vpn
between
the
robot
and
the
cloud
and
then
number
four.
It
will
copy
over
the
workspace,
in
particular
the
nodes
that
you
want
to
run
on
the
cloud.
B
Then
five
it
will
set
up
the
dds
networking
between
the
robot
in
the
cloud
essentially
telling
the
robot
that
it
needs
to
communicate
over
the
vpn
and
the
cloud
that
it
needs
to
communicate
over
the
vpn.
Then
six
it
will
set
up
any
dockers
that
are
needed.
This
is
a
part
that
I'm
kind
of
skipping
over,
but
we
also
support
dockers
and
then
seven
and
eight
it
will
launch
the
nodes
in
cloud
and
robot
and
so
that
that's
what
fog
ross
does
for
you
and
again
now
you
could
potentially
do
this
all
yourself.
B
That's
how
we
prototyped
it
but
fog
ross.
It
was
really
just
those
changes
of
lines
of
code.
In
I'm
sorry,
just
the
changes
in
the
configuration
script
and
it
launches
in
the
cloud.
So
we
we
ran
it
on
a
few
examples.
We
took
a
slam,
simultaneous
localization
and
mapping
process
and
ran
it
on
a
32
core
cq
in
the
cloud,
and
we
got
half
the
latency.
B
So
now,
I'm
going
to
go
through
a
walkthrough
of
how
you
might
go
about
using
fog,
ross
and
again
feel
free
to
jump
in
at
any
time,
but
the
first
part
of
the
process
would
be
to
say
figure
out
what
instant
stuff
you
want
to
use.
So
I'm
just
going
to
the
amazon
website
to
see
what
instance
types
they
have
available.
B
B
So
I'd
actually
encourage
you
at
this
point,
open
up
another
browser
tab
and
go
to
cloudping.info
and
hit
the
http
ping
button
and
you'll
see
the
round
trip
latency
to
all
the
different
data
service
outlets,
different
data
centers
that
amazon
provides.
So
this
one.
I
ran
just
last
night,
I'm
on
the
east
coast
right
now,
and
you
can
see
that
I'm
getting
41
milliseconds
to
us
at
least
one
58
to
usc
2,
so
I
can
kind
of
choose
between
those
two.
B
When
I
was
in
berkeley
just
a
couple
weeks
ago,
I
was
getting
15
milliseconds
to
us
west
one.
So
once
you've
figured
out
that
instance
type
you
then
go
to
select.
Well,
we
want
the
fpga
in
us.
You
know
whatever
region
you
are
or
eu.
Whatever
region
you
are
or
so
forth,
and
you
might
select
of
the
options
available,
the
ones
that
best
suits
your
need
or
the
lowest
cost
that
will
satisfy
what
you're
trying
to
accomplish
so
here
we
might,
for
example,
choose
this
fpga
based
instance,
with
a
3.3
per
hour.
B
B
Now
we
get
to
the
actual
pocker
example,
so
the
talker
example
is
pulled
straight
out
of
the
tutorials.
We
have
a
slight
modification
here.
I
wish
my
mouse
worked,
but
I
would
be
pointing
out
that
we're
getting
the
host
name
and
ip
address
of
both
the
talker
and
the
listener.
The
talker
is
upper
right
and
you
can
see
what
it's
doing
is
on
that
timer
callback,
which
it's
called
every.
B
Second,
it's
going
to
be
sending
a
message
that
says
hello
world
from
host
name
host
ip
and
then
the
counter
and
on
the
subscriber
on
the
lower
left.
You
can
see
that
it's
going
to
say
I
am
and
then
print
out
its
host
name,
and
I
p
and
say
I'm
going
to
say
I
heard
the
message
that
it
heard
so
now
we're
just
going
to
walk
through
this
screencast
and
hopefully
it
works
of
walking
through
the
demo
and
so
first
thing
we
do
in
this
demo
as
unfortunately,
I
dropped
it.
B
A
little
early
is
we're
just
going
to
look
at
that
launch
script.
That's
part
of
that's
included
as
part
of
the
fabros
demos
and
in
particular
we
just
want
to
look
at
you
know.
How
does
this
launch
group
look?
There's
nothing
magical
here.
This
is
just
straight
out
of
the
examples.
Again
you
can
see
it
has
a
launch
description.
It
creates
a
talker
and
listener
nodes
and
so
forth
and
just
adds
them
and
runs
them.
So
now
we're
going
to
build,
as
you
have
to
do,
ignore
all
these
errors.
B
This
is
the
early
cut
of
the
you
know
a
couple
weeks
ago
where
this
video
was
taken
from.
We
don't
see
those
anymore
in
the
latest
release
and
then
we're
going
to
launch
it
and
right
now
we're
not
using
any
of
the
fog
rest
capabilities,
we're
just
launching
the
talker
example
that
showed
you.
You
can
see
that
it's
going
to
say
hello
from
127.0.1.1
to
well.
It
went
a
little
quick,
but
it
was
from
localhost
to
localhost.
B
The
robot
was
communicating
with
itself
so
now
we're
going
to
go
in
and
modify
that
launch
process.
So
we're
going
to
say
we're
going
to
use
fog
ross,
launch
description,
we're
going
to
need
a
machine
type,
so
in
this
case
we're
going
to
use
an
aws
instance
and
we're
going
to
say
we
want
region
in
this
case
we're
going
to
say
us
west
one.
So
that
was
the
region
that
I
just
pointed
to
before
now
instance:
type
we're
going
to
select
something
just
a
basic
instance
called
t2
medium.
B
So
again,
these
are
the
only
changes,
we're
making
we're,
not
changing
any
code,
we're
just
changing
the
launch
and
of
course,
we've
got
to
add
our
import.
So
we
do
an
import
from
frog
ross.
We
import
aws
in
cloud
node
and
then
we
save,
and
then
we
run
it
so
well,
then
we
build
and
I'm
going
to
skip
the
build
part
and
just
see
if
I
can
get
to
the
running,
and
now
I'm
not
going
to
try
to
hide
any
details
from
you.
B
B
It's
you
know
going
through
all
the
different
processes.
You
need
to
get
a
cloud
instance
essentially
ready
to
go
and
gonna
finally
build
the
workspace
that
it
moved
to
the
cloud.
Oh
there's
a
docker
image.
It
set
up
the
vpn.
We
skipped
over
that
and
now
it
started
the
launch
process,
and
so
now
I'm
going
to
go
to
the
real
time.
You
know
speed,
video
and
you
can
see
that
you've
got
the
talker
and
listener
going
from
different
ips
to
talk
to
each
other
and
I've
not
changed
again
any
lines
of
code.
B
The
robot
one
of
the
nodes
is
on
the
cloud
and
the
other
node
is
on
the
robot
and
they're
talking
to
each
other.
Just
fine.
So
now
I'm
pressing
ctrl
c
and
doing
a
command
line
interface
to
see
all
the
different
instances
that
I
have
running.
I
have
just
that
one
that
I
showed
and
now
I'm
going
to
terminate
the
instance,
so
I
don't
get
charged
for
it
anymore
and
that's
it
for
the
demo.
B
So
next
steps
are,
we
are
working
actively
and
diligently
and
very
hard
to
get
it
into
ros.
To
humble
we
welcome
contributors.
We
welcome
early
access
testers,
you
know
anyone
who's
willing,
not
to
say
hey.
This
doesn't
work,
it's
horrible,
anyone
who's
willing
to
say
hey,
it
doesn't
work,
here's
a
here's,
something
that
doesn't
work.
Please
fix
it
or
here's
an
idea
for
effects.
We
also
have
a
iros
22
workshop
on
cloud
robotics
that
we're
going
to
be
going
to,
and
hopefully
to
roscon
as
well
and
then
further
down
the
line.
B
B
Some
of
our
own
planning
is
more
command
line,
interface,
tooling,
monitoring,
maybe
looking
at
better
performance
and
instance
for
use
or
increasing
the
capabilities
to
make
use
of
the
cloud
such
as
using
multi-agent
or
that
serverless
computing
paradigm
that
I
went
through
and
with
that
I'm
going
to
try
to
wrap
it
up
with
some
time
to
maybe
talk
a
little
bit
and
put
up
a
url
and
feel
free
to
reach
out
to
us
at
the
email
addresses
listed
below
and
say.
Thank
you.
A
Fantastic,
thank
you
so
much
jeff.
That
was,
that
was
a
fantastic
presentation,
all
right
folks.
So
I
hope
you
are
as
shocked
as
I
am.
I
must
admit
that
I
knew
a
few
of
these
things,
but
still
getting
getting
jeff
on
stage
is
certainly
very
impressive.
So
it's
time
for
the
q,
a
questions,
comments,
pointers.
I
B
Is
a
really
really
good
question?
I
don't
know
if
I
can
answer
it,
though,
because
I
think
it's
pretty
application
specific.
There
are
definitely
break-even
points
right.
You
could
imagine
that
everything
can
run
on
the
hardware
on
the
robot
and
you
can
spend
a
little
extra
time
and
not
use
the
cloud
at
all.
That's
definitely
a
possibility.
B
The
other
extreme
is
that
you,
you
know,
can
use
one
robot
instead
of
15,
because
the
cloud
speeds
it
up
so
much
so
I
don't
have
a
really
good
answer
for
that.
I'm
sort
of
leaving
it
open
to
like
the
application
implementers,
but
that's
one
of
the
things
that
we're
thinking
about
as
far
as
like
the
tooling
of
making
it
sort
of
easy
to
sort
of
to
make
that
computation
figure
out
for
your
application.
What
is
the
the
break-even
point.
I
And
the
other
question
I
have
is
basically
on
the
communication.
You
you
touched
upon
that
and
then
you
you
were
saying
5g
6g,
I
mean.
Did
you
analyze
the
environment
in
which
this
communication
could
be
a
little
bit
difficult?
I'm
not
saying
I
mean
a
house
environment,
but
a
little
bit
more
harsh
environment,
like
you
know,
places
in
which
5g
doesn't
reach
properly.
I
mean,
do
you
expect
a
sort
of
you
know
downgrade
of
performance?
How
how
you
articulate
this
functionality.
B
B
The
other
interesting
point
that
I
I
think
is
a
really
good
one
to
start
thinking
about
as
well
and
victor
mentioned
this
earlier
is
the
the
power
requirements
of
it
like
there
is
a
power
requirement
to
using
the
network
and
does
that
power
requirement
overcome
the
the
benefit
that
you
get
from
using
the
cloud.
So
I
think
I'm
not
trying
to.
Basically
what
I'm
trying
to
say
is
I'm
not
trying
to
sell
this
as
like
an
end-all
solution
for
everything.
H
B
Well,
so
one
of
those
one
of
those
demos
that
I
realized
I
went
through
very
fast
was
using
a
gpu
that
was
the
the
middle
one.
The
grass
cleaner
and
I
went
from
doing
inferencing
on
the
robot
cpu
to
using
inferencing
on
the
gpu
in
the
cloud,
and
we
could
do
that
because
we're
using
like
pi
torch
under
the
hood,
so
pi
torch.
You
know
we'll
use
whatever
hardware
within
limits
and
it
went.
You
know
12x
improvement,
including
which
we
were
excited
about
the
round
trip
latency.
H
H
A
E
B
So
harry,
thank
you.
That
is
that's
a
great
question.
It
is
on
our
roadmap.
We
have
a
plan
for
how
to
do
that,
but
we're
not
doing
it
yet
and
we
definitely
have
use
cases.
It's
it's
one
of
those
things
where
it's
like
it's
on
our
future
list.
It's
just
where
to
prioritize
it,
and
so
you
know.
Obviously,
if
we
get
a
lot
of
people
saying
hey,
we
want
to
do
it.
You
know
to
nearby
computers
that
would
definitely
influence
our
our
thought
process
on
prioritizing.
A
Yeah,
I
agree
with
that
and
what
I
would
point
out
harry
would
be
that
I
mean
both
jeff
eric
and
I
know
the
architecture
of
ross
pretty
well.
We've
been
working
on
that
for
the
past
few
months
and
there's
no
technical,
I
would
say
hurdle
that
stops
you
from
jumping
in
and
contributing
an
extension
that
actually
does
exactly
that
from
from
a
technical
perspective,
you'll
just
need
to
identify
where
exactly
we're
engaging
with
the
cloud
and
point
towards
a
local
server.
A
So
so
the
actual
amount
of
code
contributions,
I
would
say,
is
pretty
reasonable.
So
if
you're
excited
about
that,
why
don't
you
open
up
a
ticket,
an
issue
and
maybe
start
asking
for
pointers,
and
maybe
that
is
something
that
you
can
get
out
of
the
way
from
the
berkeley's
folks
team
contributions,
as
jeff
said,
I
think,
are
super
welcome,
so
yeah.
I
would
encourage
you
to
do
it.
A
More
questions
or
comments,
maybe
I
can
make
a
quick
one
and
we're
out
of
time,
which
shows
the
level
of
excitement
jeff,
that
your
talk
raised.
One
of
the
things
that
I
would
point
out
for
those
of
you
more
experienced
with
ross
is
that
I
personally
believe
that
this
work
opens
up
the
door
for
a
new,
actually
communication
paradigm
for
those
of
you
following
or
that
have
been
following
robs2
interactions
for
the
last
few
years.
A
You
would
probably
recognize
that
there's
typically
two
three
ways
that
people
often
consider
the
first
one
is
intra
process.
Communication
wherein
nodes
are
written
within
the
same
soc
are
and
within
the
same
process.
Then
there
is
inter
process
communications
and
we've
seen
the
community
a
bunch
of
contributions
to
make
that
more
efficient
with
icrx
and
other
projects
related
to
that,
and
that
is
in
a
nutshell,
essentially
nodes
within
the
same
soc,
but
in
different
processes.
A
Then
we
have
the
intra
network
interactions
when
you
have
various
socs
connected
with
a
local
network
typically-
and
I
personally
think
this
intra
network
next
boundary
or
cloud-
let's
just
call
it
cloud-
is
what
this
work
really
really
empowers.
I
think
jeff
was
quite
on
spot
by
showing
first
early
during
his
presentation
that
the
latencies
actually
are
not
that
much
and
especially,
if
you
compare
to
some
of
the
alternatives
we
have
right
now
in
inverse
2
depending
on
the
computations
and
the
computational
graph.
A
That
is
the
next
boundary
we're
going
to
be
working
out.
I'm
happy
to
report
on
that
that
I'm
hoping
to
find
some
time
to
contribute
a
few
additions
in
a
nutshell.
There's
a
working
prototype
that
beyond
essentially
leveraging
a
cpu
in
the
cloud
also
leverages
cloud
instances
that
do
use
hardware
acceleration,
and
this
flow
that
I
described
when
you
build
the
accelerators
directly.
A
B
Cool
well
thanks,
so
much
for
having
me,
I
realize
we're
way
over
time
now.
I
would
like
to
say
feel
free
to
reach
out
to
eric
or
me.
We
I
put
my
email
address
in
the
meeting
notes,
I'm
not
sure
that's
a
good
idea
or
not
for
like
public
posting,
but
definitely
happy
to
answer
your
questions
and
the
the
followers
link
should
be
somewhere
as
well.
A
Thank
you
very
much
once
again,
jeff
and
eric
thanks,
everyone
and
we'll
see
each
other
next
month,
just
as
an
fyi,
a
humble
is
releasing
in
a
few
weeks.
So
those
of
you
tune
to
the
community
start
testing
things
out,
start
reporting
about
issues
and
and
yeah
stay
tuned
for
for
more
good
stuff.
Coming
up,
have
a
good
one
guys
take
care.
Thank.