►
From YouTube: ROS 2 Hardware Acceleration Working Group - meeting #2
Description
Second meeting of the ROS 2 Hardware Acceleration Working Group (HAWG) where we review the open architecture that Xilinx contributed to the ROS community. We also look at additional architectural possibilities through Xilinx's Kria Robotics Stack (KRS), touch on a few acceleration use cases and cover the recipe for adding new boards to the community. We wrap up enjoying a guest talk from Christian Lienen and Marco Platzner, from Paderborn University, who shared with us their research on reconfigurable computing and parallel architectures.
For more including source code, check https://github.com/ros-acceleration.
0:00 Welcome and agenda
02:57 Recap of ROS 2 HAWG progress - resources
05:03 Recap of ROS 2 HAWG progress - architecture
06:06 Kria Robotics Stack (KRS)
10:22 Adding new boards to the HAWG
14:30 ROS 2 hardware acceleration use case, a simple publisher acceleration
21:00 ReconROS, flexible hardware acceleration for ROS 2 applications
46:50 Q&A
For a discussion about this meeting, refer to https://discourse.ros.org/t/hardware-acceleration-wg-meeting-2/21789.
A
A
Then
we'll
have
a
pause
and
we
will
it's
not
going
to
be
a
part.
We'll
jump
actually
right
away
into
a
30-minute
speech
from
two
guests:
four
from
father-born
university
who
are
researching
on
reconfigurable
computing
and
viral
architectures.
We'll
have
a
pleasure
to
have
christine
linen
and
marco
platzner.
A
A
Kindly
ask
you
to
please
keep
all
of
the
questions
noted
down
and
we'll
try
to
address
them
all
at
the
end
of
the
q,
a
with
the
q,
a
both
for
the
initial
section,
as
well
as
the
ricon
ross
section
and
talk
we'll
have
from
from
our
guests
I'll
stay
at
longer
as
needed.
If
people
there's
people
has
still
question
so
so
no
worries
we'll
try
to
to
tackle,
at
least
all
that
I
can
alright.
So
let's
kick
it
off
officially,
I'm
gonna
start
sharing
my
screen
here.
A
So
recap
of
the
progress
so
far,
there's
been
quite
a
few
contributions
and
interaction.
Following
some
of
the
requests
heard
in
the
community,
we
created
a
matrix
chat
for
interactive
discussion.
Still,
the
challenge
is
pretty
much
getting
started,
but
if
you
do
need
you,
you
do
feel
you
don't
need
this
sort
of
like
capability
to
have
direct
interaction.
We
welcome
you
there
we'll
be
happy
to
to
receive
your
input.
A
There's
also
been
essentially
a
backlog,
backlog
management
created
board,
and
let
me
bring
that
up
for
you
and
in
this
board.
By
the
way,
the
slides
also
will
be
available,
probably
I'll
I'll
push
them
at
least
my
own
and
then
I'll
ask
our
guest
also
to
provide
there
so
that
we
can
have
them
all
here's
the
backlog.
A
As
of
now,
you
can
see
there's
some
tasks
that
have
been
fulfilled,
while
some
others
are
still
a
work
on
progress
contributed
both
by
by
some
of
the
ones
driving
the
community,
as
well
as
by
some
newcomers
such
as
pimartos,
who
later
on,
will
touch
into
his
endeavor
hardware.
Acceleration
working
group
minutes
are
also
kind
of
like
kept
over
there
and,
as
always,
the
tags
and
everything
you
need
to
know
to
follow
up
our
work.
The
objective
remains
the
same
and
will
not
change.
A
This
group
is
to
drive
the
creation,
maintenance
and
testing
of
acceleration
kernels
on
top
of
open
standards,
c,
plus,
plus
and
opencl,
mostly
four
optimized,
roster
and
gazebo
interactions
over
different
compute
substrates,
including
fpgas,
but
also
gpus
and
others.
It's
important
to
mention
that,
though
we
are
emphasizing
the
use
of
open
standards,
we
of
course
welcome
and
want
to
interface
with
technologies
such
as
hls
cuda
and
others
to
make
sure
that
we
essentially
can
get
up
to
speed
with
industry
standards
out
there.
Still.
A
Typically,
we
just
need
to
do
the
usual
with
we
are
used
to.
We
need
to
build
the
workspace.
We
don't
need
to
source
the
corresponding
resulting
workspace
as
an
overlay,
as
we
can
see
in
the
second
line,
and
then
we
can
use
flags
to
select
to
which
specific
hardware
target
we
want
to
build
this.
This
accelerators
and-
and
this
is
all
together
possible
thanks
to
this
architecture
proposed
which
separates
specifically
the
targeted
technologies,
with
the
overall
rush
to
specific
things.
A
With
the
corresponding
abstraction
layers
that
allow
essentially
to
keep
things
as
separated
as
possible
so
that
they
can
remain
scalable
now,
we
at
xilinx
are
looking
at
this
with
lots
of
enthusiasm,
and
we
are
also
looking
beyond.
We
believe
that
the
hardware
acceleration
capabilities
can
be
leveraged,
of
course,
on
top
of
ros
2,
but
also
below
roster
at
the
under
layers.
That's
why
we
are
putting
together
blocks
and
boxes
that
aim
to
add
value
across
the
stack
across
the
robotic
stack,
all
the
way
down
to
silicon.
A
To
give
you
a
particular
example
of
those
things,
we're
cooking
and
we're,
putting
together
we're
already
thinking
and
working
actively
towards
giving
you
the
opportunity,
the
tools,
the
capabilities
to
deliver
a
more
deterministic
real
time
more
real
time,
if
any
of
the
real
time
is
real
time,
it's
not
more
real
time,
but
essentially
real
time
lost
too
by
analyzing
dissecting,
the
complete
robotics
stack
and
going
from
the
application
layer.
All
the
way
down
to
the
data
league
layer,
considering
the
sources
of
indeterminacy
and
stylings
has
answers
for
all
of
this.
A
We
believe
many
others
also
have
answers,
but
we
are
trying
to
leverage
all
these
answers
in
a
roster-centric
manner,
so
that
you
can
use
call
con
and
email
tools.
You
can
extend
them
and
you
can
still
leverage
all
these
capabilities
capabilities
such
as,
for
example,
the
last
level
cash
contention
preventions.
We
do
have
as
part
of
our
solutions
and
many
many
many
more
again,
everything
in
a
roster-centric
manner,
and
this
is
pretty
much
what
comes
down
the
road
and
what
we
are
working
on.
A
To
give
you
a
particular
example
of
what
is
possible
today,
you
can
today
actually
build
mixed
critical
systems
with
the
result
and
the
work
already
available
in
the
hardware
exploration
working
group
repositories.
You
can
with
a
single
comment,
as
you
can
see,
in
the
slide,
change
from
a
vanilla
default
root
file
system
based
environment,
to
a
fully
preemptable
preempt
rt
based
root
file
system,
which
gives
you
essentially
the
capabilities
to
run
certain
type
of
systems
and
robotic
systems.
A
Now,
if
you
need
to
explore
things
further,
if
you
need
to
dissect
things
more,
then
you
can
do
it.
You
can
do
it
with
what
we
are
building
together
and
particularly,
we
are
facilitating
the
use
of
the
send
hypervisor,
also
in
a
roster-centric
manner.
You
see
in
the
example
how
you
can
very
easily
create,
essentially
three
virtual
machines
you
can
assign
to
each
one
of
these
virtual
machines,
the
root
file
system
that
you
want
or
the
ones
we
propose
as
default
ones.
A
This
is
totally
up
to
you
on
your
use
case
on
your
robot,
and
you
can
do
much
much
much
more
you
can
do.
You
can
do
much
much
more
around
this,
essentially
with
what
comes
down
the
road,
real,
quick.
This
vision,
we're
trying
to
push
together
forward
and
to
the
community
internally
at
silence.
We
are
calling
it
as
the
korea
robotics,
because
we
are
focusing
very
much
on
what
we
know
best.
A
We
know
our
technology
and
we
know
that
the
korea
portfolio
is
setting
a
significant
value
to
the
overall
rush
to
ecosystem
with
its
adaptive
computing
capabilities,
so
with
it
we're
trying
to
bring
roster
down
to
the
silicon
level
and
we
are
making
it
extremely
easy
to
use.
So
we
see
that
essentially
installing
krs
today,
with
its
alpha
release
and
all
of
the
capabilities
I
was
just
describing
before,
is
as
easy
as
creating
a
roster
workspace.
You
just
need
to
fetch
the
corresponding
packages,
build
a
workspace
and
start
using
it.
A
That's
pretty
much
it
as
complicated
as
it
gets
to
already
start,
creating
your
own
specialized
computer
architectures
and
now,
if
you
wish
to
learn
more
about
krs
and
overall
about
what
we
are
doing
in
the
hardware
exploration
working
group,
there's
a
couple
of
resources.
We
have
been
creating
to
make
you
more
aware
and
closer
to
our
view
and
our
strategy
as
well
to
ensure
that
you
all
can
leverage
this.
There
are
these
two
two
white
papers.
A
This
was
a
point
that
was
raised
quite
nicely
in
the
last
meeting
and
also
in
the
forums
as
well
as
in
the
github
organization.
So
we
took
it
very
seriously.
We
went
ahead.
We
grabbed
all
of
our
thoughts
and
dumped
them
into
a
series
of
conventions,
an
architectural
proposal
which
is
right
now
available
in
this
full
request.
A
If
I
can
get
my
here
this
okay,
so
it's
available
in
here
and
I'm
gonna
bring
it
to
your
attention
and
so
right
in
this
pull
request,
you
can
check
out
directly
the
aspect
and
the
architecture
proposal.
You
can
review
it.
You
can
annotate
it.
You
can
make
your
comments.
You
can,
if
you
have
interest
of
using
this,
also
add
your
view.
Maybe
we
missed
something
et
cetera,
et
cetera.
If
you
maybe
have
some
pointers
about
the
architectural
pilots,
we
should
be
considering.
A
This
is
the
right
place
to
do
it
and
we
are
hoping
to
get
the
community
input
in
here.
We
use
this,
and
also
we
are
using
the
governance
policies.
We
believe
makes
sense
for
now
for
the
hardware
acceleration
working
group
to
create
this
recipe.
This
recipe,
essentially,
is
composed
of
four
very
simple
steps.
A
We
also
suggest
this
is
probably
not
mandatory,
but
if
we
think
it's
a
good
practice
that
you
take
number
step
number
three,
which
essentially
entails
going
ahead
and
assessing
the
capabilities
within
the
conventions
we
proposed,
we
wanted
to
make
it
very
simple
for
consumers
or
roster
packages
to
give
them
a
hint
of
what
capabilities
each
hardware
platform
provides,
and
so
for
that
there's.
This
simple
template
which
you
can
check
again
in
the
rep
2008
pull
request,
and
this
this
is
just
the
current
status.
A
This
is
not
something
we
just
came
out
of
the
blue,
but
actually
something
that's
being
used
already
by
by
someone.
That's
porting
things
to
the
ultra
96
version
2,
so
pimartos
is
actually
already
taking
the
endeavor
for
this
board
and
he
actually
came
up
with
a
pretty
similar
list
in
south
and
we
just
fine-tuned
it
a
bit
to
make
sure
that
it
aligns
with
the
with
the
hardware
acceleration
working
group
architecture.
A
A
A
So
here
is
here's
leo,
who
is
a
roser,
a
roboticist
and
leo?
Has
this
simple
publisher
who
is
trying
to
essentially
publish
stuff
at
10?
Hertz
he's
got,
however,
one
function
he's
calling
the
vector
add
function
where
he's
adding
two
sources
of
input
from
the
computational
graph
in
one
and
into
two
vectors,
and
unfortunately,
the
operation
within
this
function
is
somewhat
expensive.
Computationally.
In
this
case,
we
are
artificially
making
it
similarly
computationally
expensive,
but
there's
many
many
of
these
examples
within
the
ross2
packages,
so
in
this
case
leo
has
identified.
A
What
is
the
issue
in
his
note
and
he
wants
to
optimize
it.
He
wants
to
make
sure
he
meets
his
goal.
His
target,
this
10
hertz
of
publication
of
publishing.
Sorry.
So
traditionally,
if
leo,
were
to
use
either
silences
technologies
or
any
anyone
else
else.
To
be
honest,
he
would
need
to
do
much
of
the
heavy
lifting
himself.
He
would
need
to
essentially
ensure
that
ross
chew
is
shipped
in
the
right
format,
with
the
right
version
in
the
corresponding
board.
A
He
can
select
the
target
because,
again
in
his
work
space
in
his
roster
workspace,
he
could
have
multiple
targets.
You
could
have
the
kv-260,
the
ccu102,
the
ultra
96
vitu,
and
he
could
try
amongst
them
all
so.
The
first
thing
he
selects,
which
one
he
wants
to
target.
He
then
can
easily
build
a
raw
sd
card
images
again
very,
very
easily,
no
crazy
downloads
or
complex
processes.
No,
he
builds
it
customized
for
his
particular
use
case.
He
can
pick
and
configure
the
kernel.
A
The
linux
networking
stack
everything
again
looping
back
to
what
I
was
pointing
out
before
about
determinism
and
ensuring
we
can
leverage
a
more
deterministic
real
rush
to
set
up
overall
now,
once
he's
done
that
he
can
build
his
workspace
as
he
will
do
in
his
workstation,
he
can
source
the
workspace.
He
can
get
it
as
an
overlay
effectively
speaking
and
then
he
can
just
by
adding
the
flag,
get
everything
right
away
and
ready
to
be
shipped
into
the
board,
and
he
can
do
this
manually.
A
He
can
use
net
booting,
there's
lots
of
capabilities
that
can
be
used
in
here,
but
in
a
nutshell,
after
doing
all
of
this,
which
again
is
very
simplified,
with
the
work
we've
been
doing
with
hardware
acceleration
working
group
compared
to
the
previous
approaches
that
we
still
see
with
many
other
technologies,
he
still
observes
that
the
publication
rate
is
below
his
target.
He
was
aiming
for
10
hertz,
but
he
gets
two
hertz
and
here
is
where
hardware
acceleration
really
comes
to
play.
A
Of
course,
we
are
analyzing
a
very
very
simple
example,
but
in
most
cases
some
level
of
tracing
and
benchmarking
is
required
to
ensure
he
selects
the
right
things
to
accelerate,
and
this
is
something
by
the
way,
we're
working
very
actively
on
so
so
expect.
Good
news
on
this
front
from
our
side
from
the
hardware
acceleration
working
group
now
going
back
to
leo's
example
leo
decides
to
create
an
acceleration
kernel
and
he
uses
hls
for
this,
and
in
this
case
he
just
adds
these
three
lines
which
allow
him
to
optimize.
A
The
data
flow
of
this
particular
kernel.
There's
no
real
exploitation
at
this
level
of
parallelism
within
the
code.
This
just
optimizes
the
data
flow
and
just
by
doing
so
and
again
using
the
same
command
so
see
the
the
flow
of
development
is
actually
very
rash.
Geocentric,
it's
very
simple:
to
follow
up
and
to
optimize
your
robotic
systems,
that's
what
we
wanted
to
do.
We
were
obsessed
with
ensuring
that
we
do
provide
the
best
experience
out
there
for
hardware
acceleration
and
with
this
tiny
change
he
gets
to
six
hertz.
A
This
may
sound
familiar
if
you
attended
the
past
meeting,
because
this
is
what
we
showed
directly
with
source
code,
but
as
promised
he
can
do
better.
He
can
do
better.
He
can
not
only
optimize
the
data
flow
but
also
exploit
code
level
parallelism
and,
as
you
can
see,
the
fixes
or
the
changes
are
tiny,
but
by
doing
so
he
already
can
reach
his
desired
target.
A
We
are
not
just
providing
these
capabilities
for
you
to
accelerate
things,
we're
also
leveraging
silence's
expertise
in
hardware
to
leverage
hypervisors
to
create
mixed
critical
systems
to
leverage
time-sensitive
walls
to
core
layers
such
as
rcl
cpp,
with
a
deterministic
executor.
We
are
also
working
on
providing
a
more
deterministic
dds
solution
and
even
a
data
linked
layer
that
is
time
sensitive
through
leveraging.
Time-Sensitive
networking,
so
end-to-end
rush
to
real-time
is
what
we
are
aiming
for
here,
also
using
and
leveraging
adaptive
computing
and
hardware
acceleration.
A
Now,
with
that
intro
sorry
to
speak
so
much,
I
think
it's
time
we
give
the
room
to
our
guests.
So
so
yeah
we'll
push
the
questions
to
the
end.
As
pointed
out,
and-
and
hopefully
everyone
will
enjoy
very
much-
I'm
sure
you
will
the
session
now
on
on
rick
and
ross.
So,
marco
christian,
would
you
like
to
take
a
stage.
C
Okay,
I
think
you
can
see
that
now
we
can
see
it.
Thank
you.
Okay,
excellent
yeah,
thanks
for
for
thanks
victor
for
having
us
today
and
allowing
us
to
present
our
work
here.
So
we
are
a
kind
of
an
academic
research
group
that
in
parabona
university,
which
is
in
germany
in
case
you
don't
know,
and
we
have
recently
started
working
on
something
that
we
called
recon
ros.
C
So
we
do
have
a
background
in
reconfigurable
computing
and
personally,
I'm
dealing
with
fpgas
more
than
25
years
now
and
recon
ross
is
kind
of
a
combination
of
reckoners
and
ross.
So
and
accordingly,
we
will
have
two
parts
in
our
presentation.
First,
I
will
give
a
very
short
intro
into
reconos
that
stands
for
reconfigurable
hardware,
operating
system
or
reconfigurable
computer
operating
system,
and
then
christian
will
dig
over
and
explain
how
he
built
recon
ross
by
combining
reckoners
with
ross.
C
C
So,
in
software
terms,
this
is
effectively
a
remote
procedure,
call
right,
and
that
has
a
number
of
issues.
First
of
all,
the
hardware
accelerators
are
rather
passive,
but
just
sit
there
and
wait
to
be
to
be
called
for
execution.
They
cannot
initiate
things
on
their
own
and
second,
the
interface
between
software
and
hardware
is
is
always
to
do
a
fresh
and
kind
tedious,
because
the
software
has
to
be
changed.
We
have
to
insert
the
commands
that
deal
with
the
with
the
reconfigurable
part
on
on
that
aspect.
C
C
So
these
threads
either
hardware
or
software-
can
now
communicate
and
synchronize
with
the
other
threads
and
don't
even
have
to
know
where
the
counterparts
where
they
are
mapped
to
whether
the
software
and
hardware.
So,
for
example,
if
what
we
did
use
posix
threading
library.
As
that,
then
we
have
a
kind
of
a
standardized
interface
between
the
software
and
hardware,
and
so
the
term
hardware
thread
might
be
a
bit
misleading.
C
Because
in
the
software
world,
the
threat
is
typically
a
small
execution
unit
with
a
sequential
thread
of
control
and,
of
course,
in
our
hardware
accelerator,
we
do
not
want
to
have
a
sequential
control.
We
want
to
have
massive
parallel
implementation
and
in
order
to
explain
how
we
establish
that,
I'm
showing
a
bit
more
elaborate
slide
an
example
architecture
how
it
looked
like.
C
So
on
the
left
top
side,
we
see
the
processing
system
with
the
cpus
they're
running
linux,
for
example,
and
on
top
of
that,
our
reconnaissance
layer
and
software
threads
and
on
the
right
hand,
side.
We
see
the
programmable
logic
with
two
so-called
reconfigurable
slots,
which
are
the
areas
on
the
fpga
we
use
to
map
the
hardware
accelerators
into,
and
now
we
have
the
hardest
reds.
So
two
two
in
this
case
and
these
hardware
threads
accelerator
circuits.
C
They
communicate
on
one
side
to
the
operating
system
by
the
so-called
osif
operating
system
interface
and
on
the
other
side,
to
the
memory
by
the
memory
interface
and
now.
The
way
this
works
is
that
the
hardware
thread
implementation
itself
is
hopefully
highly
parallel.
That's
why
we
achieve
kind
of
faster
runtimes,
but
what
is
sequential
is
the
communication
with
the
operating
system
and
as
in
a
software's
right
right.
C
There
is
one
operating
system
called
after
each
other,
and
it's
really
sequential
and
this
behavior,
so
the
sequence
of
operating
system
calls
is
captured
in
what
we
call
it.
Os
fsm
operating
system,
finite
state
machine,
and
that
is
the
same
kind
of
structure
we
have
in
a
software
thread,
and
the
second
important
thing
is
on
the
software
side.
We
have
so-called
delegate
threads
and
there
is
one
delegate
thread
for
each
harbor
threat
and
the
delegates
are
very
lightweight
software
threats.
C
What
they
do
is
they
call
the
operating
systems
call
services
in
behalf
of
the
hardware
threats,
so
we
do
have
a
communication
between
the
osfsm
and
their
corresponding
delegates
threat
that
then
actually
executes
the
operating
system.
Call
that
has
pros
and
cons,
maybe
a
a
small
con-
is
that
this
takes
some
time
right.
We
have
to
raise
an
interrupt
and
to
go
through
the
interrupt
system
on
the
cpu
through
the
linux
kernel
until
we
arrive
at
the
delegates
right.
C
So
since
this
takes
some
time,
this
is
not
maybe
intended
for
hardware
threads
that
do
extremely
frequent
os
calls
right,
but
that's
anyway,
typically
not
the
case
for
hardware
accelerated
functions.
The
on
the
pro
side
is
flexibility,
so
we
can
make
basically
any
operating
system
call
available
for
hardware
stretch
if
that
makes
sense
right.
So
it's
very
simple
to
add.
C
One
thing
I'd
also
like
to
mention
is
preemption,
so
on
software,
of
course,
we
have
full
preemption
by
operating
systems
in
hardware.
Full
preemption
is
in
principle
possible,
but
extremely
tedious
to
implement
and
expensive
in
terms
of
memory
and
time.
So
what
we
follow
here
is
a
cooperative
multitasking
approach
in
the
sense
that
the
hardware
threats
they
indicate
when
they
want
to
or
can
or
want
to
yield
control
to
something
else
right,
and
then
the
software
part
can
react
on
that
and
a
bit
more
advanced
by
fpgas
that
can
use
partial
reconfiguration.
C
We
can
use
that
feature
to
load
or
remove
the
hardware
threads
at
runtime,
so
this
reckoners
project
has
been
running
for,
as
I
said,
15
years
or
so
we
have
in
the
course
of
that
being
able
to
do
a
lot
of
different
versions.
They
all
share
the
same
idea
of
that
rekonos
is
the
multi-threaded
model
in
software
and
hardware,
so
we
are
academic
group
and
obviously
we
have
limited
resources
for
maintaining
an
open
source
repository
and
providing
all
the
examples.
C
So
that
has
also
been
side
projects
by
phd
students,
but
as
you
can
see
that
we
have
covered
several
generations
of
operating
systems
of
of
spgas
with
processor
cores,
sometimes
even
with
soft
processor
course,
and
the
current
version
we
have
is
for
the
sync
and
for
the
creation
of
the
hardware
threads
we
can
use
either
vhdl
varilog
or
hlf.
So
we
do
have
basically
libraries
for
doing
this.
State
machines
operating
systems
change
machines
for
both
of
them,
a
recognized
version
for
the
ultra
scale,
is
underway
for
the
64-bit
architecture.
C
So
over
the
time
we
have
done
a
lot
of
student
projects
and
also
other
cooperative
projects,
some
even
with
industry.
Using
rekonos
and
the
experience
from
these
years
is
that
recognized
was
extremely
useful
in
especially
especially
three
settings.
The
first
is
that
has
also
been
kind
of
mentioned
already
by
victories
that
this
supports
a
kind
of
a
step-by-step
application
process.
C
We
can
just
change
configuration
files
and
then
rebuild
the
system
and
that's
very
easy
to
create
all
these
different
versions,
because
the
other
threads
that
the
hardware
sets
communicate
with
they
don't
have
to
be
changed
right
and
also
the
operating
system
doesn't
have
to
be
changed.
It's
the
only
communication
by
the
posix
library
calls
and
the
last
one,
which
is
why
advanced,
is
that
using
by
using
partial
reconfiguration,
we
can
also
do
runtime
migration
of
of
threads
between
software
and
hardware
or
hardware
software.
C
So,
as
shown
in
this
example,
for
example,
we
started
with
a
system
that
had
one
hardware
thread
and
then
maybe
because
the
workload
changed
and
the
performance
was
not
sufficient.
We
decided
to
run
a
second
thread
in
hardware
and
later
on,
maybe
again
the
workload
changed.
We
decided
to
go
back
to
all
in
software,
and
so
this
kind
of
freedom
that
we
have
for
migration
allows
us
to
change
the
partitioning
between
hardware
and
software
stretch
and
also
change
the
number
of
threads.
B
B
I
want
to
first
talk
about
the
different
options
for
enabling
yeah
for
enabling
hardware
accelerations
in
ros
nodes.
So
when
we
have
a
closer
look
into
the
recent
literature,
we
mostly
see
the
first
two
options
from
this
figure.
So
a
and
b
in
this
case
yeah
the
rust
2
node,
is
more
or
less
executed
in
software.
And
if,
if
it's,
if
we,
if
it's
neat,
for
example,
hardware
acceleration
for
offloading
computational
intensive
things,
it
runs
a
fpc
call
to
this
hardware.
B
But
what
we
did
or
what
our
intention
or
idea
was
in
recon
ross,
was
to
initiate
the
ros
behavior
of
the
node
or
to
run
the
ros
2
node
fully
into
hardware
to
so.
The
communication
is
in
this
case,
initiated
by
the
ros2
node
and
not
by
the
rus2
node
in
hardware
and
not
by
the
software
implementation.
B
B
For
example,
we
can
directly
connect
peripheral
cores
to
our
ros2
nodes,
so
we
can
read
sensor
data
within
the
fpga
and
we
don't
have
to
copy
it
to
software
first
or
into
the
software
domain
first
for
sending
it
to
other
nodes,
for
example,
and
for
example,
we
could
also
use
fifos
or
other
communication
primitives
for
communicating
between
ros2
nodes
and
hardware,
and
we
don't
have
to
go
the
way,
go
the
way
through
the
software
and
now
the
question
is
how
we
did
that.
Let's
see
we
that
we
will
see
in
the
next
slide
yeah.
B
I
think
you
can
remember
the
slide
from
marco
a
few
slides
ago
where
the
ros
archit
reckon
ross
architecture
was
shown
and
what
we
did
to
to
build
record
noise
was
basically
adding
two
main
things.
The
first
thing
is,
our
reckon
was
stack,
which
adds
ros2
related
objects
to
recognize,
so
this
objects
are,
for
example,
rs2
nodes
that
are
publisher
subscriber.
B
There
are
services
and
actions
both
as
service
as
server
and
clients,
and
the
stack
also
includes
things
like
initializations
of
these
objects
or
the
memory
management.
When
you
think
on
the
scl
library,
for
example,
you
have
to
to
declare
allocators
and
so
on
and
then
so
on,
and
that
is
basically
done
by
our
reckoners
stack
and
for
access
for
accessing
this
stack.
B
B
And
additionally,
we
have
software
sources
which
are
written
in
c
and
c
plus
plus,
and
we
have
message
definition,
files
that
are
the
same
message:
definition
files,
which
you
also
use
for
us
2,
for
example,
where
you
can
specify
messages
and
hardware
sources
as
well,
which
can
be
yeah
c
c,
plus
plus
hls
sources,
and
also
very
lock
and
vhdl
descriptions
for
very
optimized
designs
and
for
the
cross
compilation.
We
use
chemo
emulated,
docker
container.
B
B
This
tool
for
brings
the
advantage
that
we
can
use
the
message
definition
or
the
message
message:
definitions
for
the
hls
tool
flow.
So
we
have,
for
example,
if
you
want
to
access
an
image
message
where
some
data
is
lying
around.
We
can
use
this
message.
Definition,
header
files
to
access,
for
example
the
payload
or
the
image
data
part
of
the
message,
without
calculating
all
the
offsets
by
ourselves
and
the
output
products
of
our
tool
flow
then,
is
then
at
first
an
arm
binary
to
run
it
on
the
on
the
processing
system.
B
Yeah
now
what
are
the
benefits
for
mapping
ross
notes
to
hardware?
So
in
our
opinion,
we
have
at
first
a
raw
speed
up
which
depends
more
or
less
on
the
node
function
and
the
effort
we
spend
for
the
optimization
so
optimizing
things
in
hls,
it's
not
very
trivial.
So
there
you
have
consider
the
whole
architecture
for
getting
the
best
performance
and
on
the
other
side,
we
have
also
to
consider
the
overall
speed
up.
B
B
B
In
the
next
experiments,
we
programmed
some
yeah
smaller
application
examples.
The
first
one
was
inverse
kinematics
and
the
sorting
algorithm
and
the
sobel
filter,
which
and
endless
classifier,
which
classifies
handwritten
images,
and
we
implemented
this
application
without
yeah
hardware,
optimization
so
without
any
hls,
pragmas
and
so
on
and
this
evaluation.
All
those
experiments
showed
that
there
is
in
most
cases
the
speed
up,
for
example,
for
the
inverse
kinematics.
We
have
a
raw
speed
up
of
6.3.
B
Demonstrators
which
are
able
to
belongs
a
ball
on
its
plate,
and
in
this
scenario,
we
used
three
different
fpga
boards,
where
we
distributed
threats
randomly
more
or
less
between
the
hardware
and
software
domain
and
on
the
different
boards,
and
this
example
should
show
the
ability
of
reckoners
to
handle
also
larger
products
pro
eject
and
also
there,
where
some
measure
measurement
where
we
could
or
where
we
could
show
that
that
yeah,
the
deterministic
behavior
of
the
system
is
better
for
using
hardware
threads
and
software.
Only
implementations.
B
To
sum
up,
recognize
enables
ros2
application
developers
to
use
flexible
hardware
acceleration
with
fpgas,
because
we
provide
a
consistent
programming
model
across
a
hardware
software
boundary.
So
in
the
best
case,
you
can
just
copy
your
your
code
from
your
from
the
hls
domain
to
the
c
domain
and
back.
B
A
Thank
you
very
much
christine
and
marco.
That
was
that
was
fantastic
and
very,
very
exciting.
The
you're
pushing
for
quite
a
few
years
already,
but
definitely
recon
brush,
is
I
think,
of
interest
to
everyone
in
this
community
and
and
we
definitely
look
forward
to
cooperate
closer
with
you
and
try
to
bring
some
of
these
capabilities
closer
into
the
hardware
exploration
working
group
and
into
the
mainline
work
we're
putting
together.
So
so,
thank
you
again
for
for
sharing
your
thoughts.
A
D
B
D
B
A
It
was,
and
I
I
got
that
the
first
time
I
read
your
paper
and-
and
it
was
interesting
because
that's
the
first
thing
I
tried,
while
we
were
architecting
our
initial
steps
in
the
hardware
acceleration
working
group
overall
architecture,
but
I
did
see
that
not
just
you
yourself,
but
also
other
groups,
are
using
docker
containers
actively
for
cross-compilation.
It
seems
to
be
a
a
common
practice
overall
and
and
I'm
personally,
typically
very
in
favor
of
dockerizing
as
much
as
possible.
A
But
the
reality
is
that
my
experience,
my
experience
and
by
the
way,
there's
also
an
ongoing
project
in
the
roster
community.
I
think
pushed
by
aws
if
I'm
not
wrong,
which
did
actually
across
compilation
two
of
us
two
based
on
docker
containers,
so
there's
definitely
ground
for
for
going
in
that
direction.
But
when
I
tested
this
experimentally,
what
I
obtained
was
that
since
you're
building
artifacts
in
emulation
the
overall
and
that's
exactly
what
I
did
as
well,
I
think
also
what
what
is
proposed
in
the
community.
A
Then
the
times
that
I
was
experiencing
were
exploding
like
an
order
of
magnitude,
above
or
more
due
to
the
fact
that
you
were
essentially
using
emulation
for
this
process
when
compared
to
just
doing
cross
compilation
natively
in
your
workstation,
so
so
I'm
just
comparing
cross
compilation,
natively
versus
doing
it
on
on
the
emulation.
So
what
are
your
thoughts
so
in
there?
Maybe
my
experiments
need
to
be
repeated.
I'm
definitely
open
to
your
thoughts
about
that.
B
A
Yeah,
I
fully
agree.
I
think
that
there's
a
trade-off
and
I
think
both
can
co-exist.
Indeed,
the
experience
that
I
mostly
observed
was
that
in
iterative
development
processes,
where
you
essentially
are
just
touching
hls
and
iterating,
the
overhead
that
you
do
experience
by
going
the
docker
path
and
emulation
path
was
something
that
was
impacting
very
much
my
development.
But
I
can
also
get
the
fact
that,
when
you're
facing
a
very
complex
structure
of
packages,
the
cross-compilation
can
also
be
somehow
kind
of
like
reaching
middle
ground
there.
A
E
Double
muted,
sorry,
I
was
just
making
an
observation
that
use
of
docker
is
orthogonal
from
cross
compilation.
You
can
use
docker
and
cross,
compile
or
not
cross
compile.
You
don't
need
to
commingle
the
two
we
in
nvidia
use
docker
for
our
work
and
we
use
cross
compilation
and
don't
use
cross
compilation
depending
on
the
circumstance.
So
I
don't
think
there's
any
any
correlation
there
between
cross
compilation
and
docker,
but
there
are
increase
in
in
sort
of
resource
requirements
to
run
docker.
If
you
want
to
docker
as
you
work.
A
Yeah
you're
totally
right
yeah.
I
think
the
the
edge
we
were
tackling
in
here
is
not
just
docker,
it's
having
docker,
so
an
emulation,
so
sorry
simulation
and
then
within
the
simulation
and
emulation
and
then
natively
compile
things
within
the
chemo
emulation.
So
that's
actually
the
comparison
doing
things
based
on
compiling
things,
natively
in
an
emulation
versus
doing
the
cross
compilation,
so
so
you're
right
about
the
fact
that
docker
is
orthogonal
to
things,
but
in
this
case,
docker
is
used
as
the
tool
to
facilitate
building
the
artifacts
natively
through
emulation.
F
A
All
right,
so,
if
not,
it's
9
p.m
in
spain
guys.
So
I'm
gonna
wish
everyone
a
good
morning,
good
day,
good
night
to
those
of
you
in
cet
time.
Thank
you
so
much
for
joining
today,
we'll
be
having
probably
another
meeting
soon,
thanks
very
very
much
to
marco
and
christian
for
the
great
presentation.
I
personally
enjoyed
it
very
very
much.