►
From YouTube: ROS 2 Hardware Acceleration Working Group - meeting #6
Description
Sixth meeting the ROS 2 Hardware Acceleration Working Group where we discuss a methodology for ROS 2 Hardware Acceleration that has been tested in various commercial solutions for hardware acceleration (including Xilinx’s and Nvidia’s). The session will also includes a guest talk provided by Christian Lienen who tell us about a hardware-accelerated ROS 2 executor and his novel approach that introduces an FPGA-accelerated event-based ROS 2 executor which schedules callbacks of ROS 2 nodes and utilizes a reconfigurable slot model and partial runtime reconfiguration to load hardware-based callbacks on demand.
For more including source code, check https://github.com/ros-acceleration.
0:00 Welcome and agenda
00:50 acceleration_examples package
03:50 Methodology for ROS 2 Hardware Acceleration
27:25 Event-Driven Programming of FPGA-accelerated ROS 2 Applications
54:53 Q&A
For a discussion about this meeting, refer to https://discourse.ros.org/t/hardware-acceleration-wg-meeting-6/24207.
A
This
is
a
post
recording
of
the
sixth
meeting
of
the
roster
hardware,
acceleration
working
group.
Unfortunately,
the
first
half
an
hour
of
the
meeting
wasn't
recorded.
So
here
is
a
summary
or
a
walkthrough
on
the
beats
we
discussed
for
those
first
30
minutes
the
meeting
started
as
usual
with
the
introduction
of
new
folks
to
the
working
group.
We
had
about
30
participants
as
it's
been
the
norm
in
the
last
few
meetings,
and
I
encourage
you
to
check
the
list
of
attendees
that
we
recorded.
A
The
first
topic
we
discussed
in
the
working
group
was
the
progress
that
had
happened
over
the
last
30
days
approximately
last
month,
and
particularly,
we
started
discussing
acceleration
examples.
Acceleration
examples
is
a
repository
we've
been
maintaining,
which
collects
and
puts
together
rush
to
acceleration
examples.
Hardware,
acceleration
examples
demonstrating
the
use
of,
hopefully
various
hardware,
acceleration
technologies
down
the
road
for
now
mainly
focused
on
the
official,
officially
supported
hardware
reference
boards
within
the
working
group,
but
that
is
expected
to
grow
over
time.
A
The
examples
that
we
have
put
together
over
this
past
year,
working
within
this
group
have
been
summarized
here
for
quick
access.
We
have
examples
demonstrating
how
we
can
do
single
rows
to
node
acceleration
and
also
complete
rush
to
computational
graph
acceleration
with
a
first
example
that
we'll
discuss
just
in
a
bit
again.
A
The
examples
are
described
in
here
and
go
from
very,
very
simple
publishers
to
more
complex
constructs
involving
offloading
acceleration
in
the
programmable
logic,
using
fpgas,
or
even
fully
exploiting
the
capabilities
of
hls
to
accelerate
c
plus
plus
code
and
offload
it
to
to
the
fpga
gaining
a
decent
advantage
time
wise
and
determinism
wise.
We
also
have
examples
that
demonstrate
advanced
capabilities
that
you
can
leverage
if
you
use
fpga,
such
as
creating
custom
stream
interfaces
that
share
information
between
acceleration
kernels
directly
through
dedicated
channels
through
streams,
the
stream
interfaces
using
xc4
stream
interfaces.
A
A
This
has
been
done
by
following
a
case
study
so
by
by
illustrating
it
directly,
and
the
intent
of
this,
of
course,
is
to
receive
community
input
feedback
so
that
we
can
then
leverage
this
into
the
ongoing
standardization
effort
for
hardware
acceleration.
That's
happening
in
the
rep
2008
pull
request,
that's
hoping
to
add
value
to
the
community
and
essentially
put
together
a
document
that
summarizes
the
architecture
and
conventions
one
can
follow
and
and
is
recommended
to
follow
to
leverage
hardware
acceleration
in
robotics.
A
Hopefully,
this
will
be
adopted
by
more
and
more
manufacturers
and
silicon
vendors,
and
that
is
why
we
are
pushing
hard
for
this.
So
back
to
the
methodology,
which
is
the
main
topic
that
we
wanted
to
discuss
today,
the
methodology
sorry
essentially
starts
by
describing
the
case
study.
The
methodology
itself
is
composed
by
four
big
steps,
and
some
of
them
have
sub
steps,
and
the
case
study
we
will
be
touching
on
is
based
on
the
roster
perception
stack
start
by
taking
a
look
at
the
actual
methodology
proposed.
A
As
you
can
see,
it
has
four
steps,
third
of
which
has
two
sub
steps.
The
first
one
depicted
with
the
a
letter
it
says
kind
of
like
start
tracing
your
computational
graph,
and
some
of
you
might
be
wondering.
Why
do
we
start
by
tracing?
What
is
the
difference
between
tracing
and
benchmarking?
This
is
confusing
right.
A
Well,
let
me
clarify
benchmarking.
Is
the
act
of
running
a
computer
program
to
assess
its
relative
performance,
whereas
tracing
is
that
is
a
technique
specifically
used
to
understand
what
goes
on
on
a
running
system
in
the
context
of
hardware
acceleration
for
robotics,
it's
fundamental
to
be
able
to
assess
both
tracing
helps
you
determine
which
pieces
of
ross
you
know,
are
consuming
more
compute
cycles
or
generating
in
determinism
and
are
there
by
good
candidates
for
hardware
acceleration.
A
Similarly,
and
also
benchmarking,
helps
comparing
acceleration
kernels
across
acceleration
technology
solutions
such
as,
for
example,
the
kriya
kv-260
versus
the
jetson
nano
or
similar.
So
it
does
help
you
compare
acceleration
kernels
across
hardware,
but
also
across
kernel
implementations
again
in
the
iterative
process
that
hardware
acceleration
is
so
again
going
back.
A
The
first
step
is
to
just
instrument
your
computational
graph,
introduce
the
corresponding
pros
and
trace
it,
trace
it
to
help
determine
which
pieces
of
a
particular
node
and
computation
on
graph
are
consuming
more
cycles
and
are
thereby
good
candidates
to
start
considering,
hardware
acceleration,
and
that
is
the
first
step.
The
second
one
is,
of
course,
generating
a
cpu
baseline,
and
we
do
this
via
benchmarking,
by
using
the
prior
probes
that
we
introduce
with
with
tracing.
We
can
do
this
all
together,
pretty
pretty
simple,
and
this
will
be
available
well.
A
Once
we
have
obtained
the
cpu
baseline.
The
next
step
is
then
to
start
implementing
hardware
acceleration,
and
we
do
that
by
essentially
choose
two
sub
steps.
The
first
one
is
accelerate
individual
notes,
generating
the
corresponding
acceleration
kernel
for
each
one
of
them.
Then,
once
we
have
generated
those
acceleration
kernels,
we
can
pass
into
the
next
sub
step,
which
is
acceleration
the
computational
graph,
accelerating
the
computational
graph,
and
we
do
that
by
optimizing.
A
The
data
flow
from
node
to
node
through
various
techniques
and
we'll
touch
into
a
couple
that
we
explored
down
in
in
the
case,
study
that
I'll
describe
just
in
a
bit.
The
last
step
which
actually
drives
us
into
kind
of
like
a
loop
until
we
converge
into
appropriate
results,
is
of
course
tracing
the
hardware.
Acceleration
results
against
typically
the
cpu
computational
baseline
we
generated
in
the
second
step.
A
So
this
is
pretty
much
the
methodology.
Let's
now
look
a
bit
into
the
case
study,
so
the
case
study
we'll
be
exploring
and
discussing
as
part
of
this
proposed
methodology
for
ros
2
hyper
acceleration.
Is
this
a
specific
one?
It
is
essentially
a
super
simple,
two
node
computational
graph,
which
uses
two
of
the
packages
that
are
contained
within
the
image
pipeline
meta
package.
That's
part
of
the
roster
perception
stack.
You
can
see
that
the
images
are
generated
using
kosivo
and
pushed
into
the
computational
graph.
A
The
first
node
that
received
the
images
is
the
rectification
node,
which
essentially
corrects
the
distortion
that
comes
from
the
camera,
pushes
that
data
into
an
image
wrecked
topic,
which
is
then
subscribed
by
a
resize
node
which
again
might
be
in
the
cpu
or
in
the
fpga,
as
it's
depicted
in
this
figure,
and
that
node,
of
course,
also
performs
the
corresponding
operation
in
the
image.
In
this
case,
the
resizing,
enlarging
the
image
by
twice
its
size
and
finally
pushing
the
data
into
a
resize
topic.
A
Also
the
corresponding
roster
notes
in
the
perception
stack-
and
you
can
see
this
illustrated
in
here
in
this
pull
request,
which
was
merged
and
which
include
not
only
lttng
probes
and
a
baseline
for
introducing
additional
trace
points,
but
also
the
trace
points
that
correspond
with
with
those
two
nodes.
Then
we
are
interested
in
analyzing.
A
I
guess
we
have
the
the
overhead
introduced
by
the
to
message:
passing
infrastructure
across
layers
in
the
user,
land
level
in
the
rcl
cpp
level
and
below
as
well.
So
with
that
kind
of
like
layout.
For
the
first
step,
we
jump
into
the
second
step
of
our
methodology,
which
is
generating
a
cpu
baseline,
and
we
do
that
by
using
the
same
trace
points.
A
We
use
the
same
trace
points
and,
of
course,
we
script
things
out
so
that
we
can
stack
the
timings
together,
and
this
is
how
it
looks
as
we
can
see
it
takes
about
on
average.
By
the
way,
it's
important
for
me
to
highlight
that
here
I'm
plotting
the
average
results
for
each
one
of
these
trace
points
obtained
for
this
pipeline.
This
this
computational
graph
running
for
about
60
seconds
on
each
one
of
the
implementations
that
we
will
be
considering.
A
So
we
are
plotting
averages
and
we
can
see
how
the
average
the
total
average
takes
about
a
bit
more
than
90
milliseconds.
Here
we
are
depicting
the
first
step.
We
think
first
step
within
the
third
phase
of
the
methodology,
which
is
the
hardware
acceleration,
so
we're
going
a
bit
faster,
plotting
wise.
A
That
corresponds
with
this
first
bit:
accelerate
raw
two
nodes
and
components,
and
that
was
done
again
by
picking
up
the
two
nodes
that
are
involved
in
the
computational
graph,
the
rectify
and
the
resize,
and
proposing
providing
accelerated
versions
of
each
one
of
them
by
leveraging
the
programmable
logic.
The
fpga
and
you
can
see
here
how
we
proposed
an
implementation
for
the
rectify
node,
which
is
slightly
different
than
the
default
one
that
you
can
see
today
in
your
default
image
pipeline
package,
we're
using
opencl
to
exchange
information
between
the
host,
the
cpu
and
the
fpga
device.
A
This
is
done
again
for
both
the
rectify
and
the
resize
kernel,
and
the
results
that
we
obtained
again
average
results
for
the
graph
running
for
about
60
seconds,
at
a
clock
used
at
a
clock
of
250
megahertz
in
the
acceleration.
Kernels
is
about
this
big,
so
we
can
see
how
there's
a
slight
improvement.
A
It's
not
huge,
but
there's
a
slight
improvement
and
essentially
the
the
reason
why
this
is
not
mind-blowing
is
because
obviously,
what
we
are
doing
in
here
is
just
pure
off-loading
of
the
resize
and
rectify
operations
to
the
fpga,
and
so,
if
we
want
to
further
leverage
parallelism
or
exploit
the
capabilities
of
of
hardware
acceleration,
we
need
to
look
into
more
than
just
pure
offloading,
regardless.
Pure
loading
will
give
us
some
interesting
advantages,
especially
additional
reliability
on
the
determinism
of
the
execution
of
those
operations,
resides
and
rectifies.
It
is
executing
in
hardware.
A
Let
me
bring
that
up
again
into
the
third
step
of
our
sorry,
the
second
sub
step
of
our
third
phase
in
the
methodology.
So
again,
we've
now
just
accelerated
the
the
notes
and
and
to
be
precise,
that
that
gives
us
a.
I
think
I
have
a
table
somewhere
here
yeah
here
it
is
so
just
doing
that
gives
us
mean
essentially
a
total
improvement
of
about
6.22
percent,
which
is
interesting.
But
again
it's
not
mind-blowing.
A
If
we
consider
the
actual
effort
that
it
takes
to
bring
things
to
hardware
acceleration.
So
again
we
can
do
more
and
that
more
comes
by
accelerating
the
flow
starting.
The
flow
of
the
information
that
goes
from
node
to
node
and
visually.
That
means
that,
instead
of
going
back
and
forth
from
the
cpu
to
the
fpga
after
executing
the
operation,
meaning
that
the
information
flow
comes
from
the
cpu,
it
then
gets
transferred
into
the
fpga.
A
The
rectify
operation
gets
computed
in
the
fpga,
then
the
result
gets
back
into
the
cpu
so
that
it
can
then
be
passed
to
the
roster
message,
passing
infrastructure
and
throughout
the
corresponding
topic
it
reaches.
The
next
note
there's
lots
of
waste
of
compute
cycles
in
doing
so,
especially
because
both
nodes
are
running
acceleration,
kernels
and
thereby
they
could
leverage
some
additional
approaches:
additional
techniques
that
provide
more
determinism
and
avoid,
maybe
the
message
passing
infrastructure
or
ross
2,
which
certainly
is
a
limiting
factor
for
many
use
cases
and
again
that
can
be
visually
inspected.
A
So
again,
with
that
in
mind,
with
that
philosophy
in
mind,
we
we
dive
down
into
hardware,
accelerating
the
computational
graph.
We
now
have
some
initial
measurements
of
what
accelerating
the
nodes
gives
us,
but
we
want
to
get
better
and
faster
interactions
in
a
computational
graph.
So
we
look
at
accelerating
the
computational
graph,
and
particularly
the
results
obtained
are
interesting.
We
can
see
how
let
me
bring
these
two
figures
bigger
a
bit
in
here.
A
So
this
is
one,
and
this
is
the
other
one.
So
we
can
see
we're
gonna
explore
two
approaches.
They
indeed
provide
an
interesting
upgrade
when
compared
to
just
pure
offloading
to
the
fpga,
but
each
one
has
its
ups
and
downs.
So
we'll
we'll
just
speak
a
bit
about
both
of
them.
The
first
one,
the
first
one
we
evaluated
is
the
one.
We
call
the
integrated
approach
for
accelerating
the
computational
graph
and
this
integrated
approach
in
a
nutshell.
A
What
it
does
is
it
smashes
both
kernels
together,
so
both
kernels
and
both
nodes,
resize
and
rectify
into
a
new
node.
So
precisely,
what's
needed
in
here
is
to
re-architect
the
computational
graph,
and
we
end
up
instead
with
two
nodes
and
a
topic
passing
information
from
node
to
node,
a
single
node
that
contains
all
of
the
logic
of
rectify
and
resize.
A
A
So
we
dig
a
bit
deeper
and
what
we
did
is
we
explored
an
alternative.
We
explored
an
alternative
that
consists
on
instead
of
smashing
both
nodes
together,
we
keep
them
as
they
are
in
the
in
the
purely
fpga
of
loaded
case,
and
what
we
do
is
we
design
and
leverage
xc4
stream
interfaces
to
create
what
we
call
an
intra
fpga
reverse
two
communication
queue
which
connects
accelerators
in
the
first
node
to
the
ones
in
the
second
node.
A
A
The
only
complexity
in
this
case
relies
on,
of
course,
leveraging
these
xe4
stream
interfaces
to
create
this
intra
fpga
was
two
communication
cues,
but
fortunately
that's
something
we're
looking
at
and
we
are
providing
reference
implementations
for
so
all
together.
Here
is
the
results
of
the
case
study
we've
been
exploring.
We
can
see
how,
indeed
the
one
that
optimizes
the
most
is
the
integrated
approach.
A
That
comes
with
the
downsides
of
having
to
re-architect
the
the
computational
graph,
and
we
saw
how
using
essentially
a
creative
approach
using
this
intra
intra
fpga
node
queues.
We
are
able
to
get
similar
results
and-
and
it
kind
of
like,
brings
the
best
of
both
worlds.
So
pretty
interesting
results.
This
was
done
again
systematically
by
using
the
trace
points
that
we
introduced.
A
So
this
is
what
we
discussed
in
the
first
piece
of
the
meeting.
There's
already
community
members
here
asking
and
sharing
their
thoughts.
A
pretty
interesting
discussion
so
definitely
encouraging
you
to
join
and
share
your
thoughts
or
contribute.
Of
course
that
was
the
second
topic
we
discussed
and
also
I
highlighted
the
fact
that
other
hardware,
acceleration
solutions
are
on
the
process
or
in
the
process
of
being
evaluated
and
added
to
these.
A
A
A
There
seemed
to
be,
though,
some
interest
around
the
ultra
96
p2,
so
I
pointed
people
and
folks
again
to
this
ticket,
which
is
the
one
where
I'm
supporting
with
a
few
extra
cycles,
the
port
of
the
hardware,
acceleration
architecture.
We
are
maintaining
to
a
new
board
to
the
ultra
96
p2
board,
which
is
which
is,
I
believe,
manufactured
by
av
net,
so
so
yeah.
A
I
I
called
out
for
contributions
and,
of
course,
also
for
people
to
jump
in
and
share
interest,
because
if
there's
enough
interest,
we
could
reach
out
to
avnet
and
try
to
essentially
get
them
to
commit
some
engineering
resources
about
it.
The
last
bit
shared
in
this
first
half
of
the
meeting
was
concerning
the
updated
architecture.
So
we've
added
more
bits
to
the
architecture
for
clarity
and
clarification.
You
can
check
it
out,
as
always
in
the
community
repo
of
the
group
and
and
yeah.
A
C
Okay,
yeah
hello
and
welcome
again
to
my
talk
about
our
most
recent
work.
The
raccoon
ross
executor
yeah.
As
victor
said,
my
name
is
christian
lien,
I'm
phd
candidate
at
pederborn
university
in
germany
and
thank
you
for
the
opportunity
to
speak
here
in
front
of
the
audience
all
right
so,
as
I
think
all
of
us
know,
there's
a
high
demand
for
computational
power
in
robotics
applications,
especially
we
have
energy
sensitive
applications.
C
So,
if
you
think
about
very
large
applications,
our
reconfigure
every
fabric
could
be
too
small
or
it
could.
Oh
sorry
could
not
be
able
to
fit
the
whole
application,
and
because
of
that,
we
need
some
kind
of
time.
Sharing
for
our
resources,
so
we
have
to
re
use
it
during
runtime
for
different
applications,
and
our
luck
is
that
modern
platform
fpgas
are
able
to
provide
a
dynamic,
partial
reconfiguration.
C
So
we
can
just
reconfigure
parts
of
the
reconfigure
every
fabric
during
runtime
and
in
the
following.
We
want
to
leverage
that,
but
one
question
is
now
how
to
use
hardware
accelerations
hardware
acceleration
in
robotics
in
combination
with
partial
reconfiguration,
so
especially
when,
and
how
does
this
reconfiguration?
C
C
So,
as
maybe
some
of
us
know,
reckoners
was
a
project
which
was
proposed
by
our
research
group.
I
think
10
years
ago,
or
something
and
the
main
contribution
of
recon
rose
is
the
introduction
of
so-called
heart
rate
threats.
So,
hardware
threats
are
basically
areas
in
the
reconfigurable
fabric.
They
are
predefined
during
design
time,
and
the
special
property
of
this
hardware
thread
is
the
fact
that
they
are
combined
with
the
delegate
software
thread.
C
C
So,
basically,
the
hardware
thread
can
access
the
virtual
memory
due
to
this
mammoth
interface.
In
the
same
way,
like
any
other
software
thread
and
two
years
ago,
we
extended
this
work
to
recon
ross,
which
basically
adds
yeah
ros2
functionality
to
recognize
and
using
recon
ros.
The
user
is
able
to
use
raw's
communication
paradigms
like
publish
subscribe,
communication
or
services
and
actions
in
this
hardware
threads,
and
the
large
benefit
of
this
approach
is
that
we
have
now
a
consistent
programming
model
for
hardware
and
software
nodes.
C
C
We
both
yeah
support,
vhdl
and
high
level
synthesis,
so
the
programming
model
is
only
available
for
for
high
level
synthesis
because
we
have
no
vhdl
in
the
software
domain,
but
using
hls.
We
are
able
to
yeah
to
have
a
consistent
programming
model,
sorry,
this
other
than
that
we
have
the
whole
set
of
messages
available.
So
we
can
make
use
of
standard
ros
messages
like
in
any
other
software
application.
So
we
can
use,
for
example,
image
data
types
and
so
on,
but
we
can
also
yeah
define
the
user-defined
message.
C
Types
yeah,
similar
to
the
standard,
ros
flow
tool
flow
using
callcon,
and
the
conclusion
is
that
we
now
can
implement
whole
ros
notes
using
hardware
threads
in
the
yeah
in
the
hardware
domain
without
yeah.
Some
remote
procedure
calls
paradigms
which
are
very
common
in
this
field,
and
now
we
have
this
open
question
how
to
leverage
partial
reconfiguration
and
that
also
the
questions
which
parts
of
the
systems
we
have
to
exchange
and,
for
example,
when
and
how
to
start
the
reconfiguration
process.
C
So
we
need
a
kind
of
scheduling
here,
and
our
basic
concept
in
this
work
is
that
we
introduce
hardware
callbacks.
So,
as
many
of
us
know,
the
event
driven
programming
is
the
defective
standard
in
the
ros
world.
So
the
standard
way
of
programming
ros
application
is
to
use
a
ros
executor
to
define
callbacks
and
nodes
and
then
attend
these
callbacks
to
our
executor,
and
the
executor
looks
for
for
new
events
and
then
schedules
our
callback,
and
that
is
basically
the
same
thing
that
we
want
to
use
here.
C
C
The
termination
is
then
signaled
using
the
osf
interface,
and
then
our
executor
knows
when
to
start
and
the
hardware
architecture
which
we
propose
for
this
approach
is
shown
in
this
figure
red
on
the
right
side.
In
general,
our
platform
fpga
provides
m
cpu
cores,
as
we
know,
for
example,
for
the
sync
7
000.
There
are
two
cpu
cores
which
we
can
use
for
software
corvex
and
on
the
other
side,
we
have
this
red
reconfigurable
slots
which
are
going
to
yeah
to
run
our
callbacks
later.
C
C
This
is
also
possible
in
this
in
the
slot,
and
the
second
element
of
the
of
the
hardware
architecture
is
a
combination
of
the
icap
interface,
which
is
the
inter
internal
configuration
access
port,
which
yeah,
which
allows
for
the
configuration
of
yeah
of
the
programmable
fabric
from
the
programmable
fabric
and
for
getting
the
data.
So
the
configuration
data
for
this
slots
into
this
a
cap.
C
C
As
of
that,
as
I
said,
the
event
driven
programming
mythology
is
very
common
in
russ2
applications,
for
especially
if
we
use
the
rcl
cpp
or
the
python
library,
and
these
callbacks
are
then
triggered
by
the
standard
executor
for
timer
subscribers
service
servers
or
service
clients
and
yeah.
The
the
standard
scheduling
algorithm
is
shown
in
this
figure
and
it's
very
yeah
special
for
ros
applications
that
it
follows
this
schedule.
C
C
C
C
C
Finished
before
new
non-timer
callbacks
are
collected
so,
except
for
timer
callbacks,
which
are
always
executed,
and
this
standard
rs2
executor
is
available
in
a
single
threaded
and
multi-stranded
version
in
the
multi-threaded
version
it
has
basically
more
worker
threats
and
the
central
instance
collects
the
callbacks
and
distributed
to
the
to
the
worker
callbacks
and
for
our
recon
ros
executor.
We
basically
try
to
use
the
same
scheme
of
of
scheduling.
C
We
have
here
also
the
privileged
timer,
which
is
checked
in
each
iteration,
and
after
that
we
go
through
a
list
of
callbacks
and
yeah
search
for
a
ready
callback
and
then
execute
it.
Another
special
thing
here
is
compared
to
the
standard
executors
that
we
also
handle
with
bit
streams
so
due
to
the
yeah
fvga
property,
we
cannot
just
run
the
callback.
We
also
have
to
check
first,
if
this
callback
is
already
written
in
the
in
the
reconfigurable
logic,
and
if
not,
we
have
to
start
a
a
bit
right
bit
stream
first.
C
C
C
We
have
basically
one
central
recon
ros
configuration
file
which
is
shown
on
the
left
side,
and
I
printed
the
exemplary
callback
node,
which
is
here
the
filter,
node
or
the
filter
callback
code,
and
for
get
our
system
running
now.
We
have
to
specify
the
configuration
file
where
we
yeah
special,
especially
fi,
one
reconfigurable
slot,
which
comprise
some
slices,
some
dsp
units
and
some
block
memory
and
other
than
that
we
have
some
primitives
for
the
yeah
for
our
ross
node.
C
We
have,
for
example,
some
image,
yeah
image
data
types,
and
we
have
as
well
as
subscriber
and
a
publisher
for
each
node.
So
from
line
6
to
10,
we
have
the
filter
node
and
from
12
to
16.
We
have
the
inverse
kinematics
node,
which
are
targets
for
our
application,
and
our
examplery
callback
code
is
now
very
simple.
C
We
have,
or
the
first
instructions
instruction
in
our
record
is
basically
getting
the
the
ready-to-use
image
data
and
calculates
the
position
of
the
payload
of
this
image
data.
Getting
that
payload
data
into
our
block
ram
of
the
fpga
then
on
in
line
12,
calculates
the
solvable
filter
operation,
which
writes
back
the
data
to
the
ram,
and
then
we
can
write
the
data
into
the
main
memory
and
initiate
the
publish
operation
to
provide
the
viewer
node.
C
Our
data
and
the
last
last
step
of
our
callback
is
a
thread
exit
command
where
we
signal,
through
the
osf
interface,
to
the
delegate
thread
and
to
the
executor
that
we
are
done
and
that
the
executor
can
schedule
the
next
callback
and
that's
basically
it
so.
We
have
yeah
a
callback
which
is
quite
similar
to
to
a
standard
callback
where
we
have
the
position
of
the
image
data
as
on
some
parameter
of
some
callback
and
yeah,
and
that
should
make
it
very
simple
for
the
evaluation
purposes
of
our
work.
C
C
And
for
our
measurements,
we
first
yeah
measured
the
the
raw
speed
ups,
so
we
just
yeah
more
or
less
measured
the
blocking
time
of
the
of
the
executor
and
look
how
long
the
execution
takes
for
each
callback
and
we
see
that
we
have
for
each
application
a
more
or
less
large
speed
up
so
yeah
example.
Hash
calculation,
for
example,
brings
only
a
speed
up
from
1.2
compared
to
the
software
implementation,
but
on
the
other
side
we
have
a
number
sorting
reference
application
where
we
gain
the
speed
up
from
up
to
yeah
48.2
times.
C
But
there's
also
a
disadvantage
or
yeah
a
thing
that
we
have
to
consider
during
design
time,
because
we
have
the
over
overhead
of
the
reconfiguration.
So
if
the
callback
is
not
already
written
into
our
reconfiguration
slot,
the
executor
has
eventually
to
do
a
reconfiguration
and
we
also
measured
the
reconfiguration
time
for
for
the
our
slots.
So
we
see
that
our
slots
have
different
sizes
from
2.8
megabytes
to
5.2
megabytes,
and
we
see
that
the
reconfiguration
time
more
or
less
grows
up
linearly,
but
we
also
yeah.
C
We.
We
also
see
that
the
reconfiguration
time
is
yeah
much
higher
than
expected.
We
only
gained
a
160
megabytes
instead
of
400
megabytes,
and
we
still
think
that
this
is
a
the
reason
of
that
is
yeah,
a
very
bad
written
linux
driver
which
yeah
generates
some
overheads
because
of
the
yeah
mem
copy
from
from
the
user
land
to
the
kernel
land
and
there's
still
room
for
improvement.
C
C
Yeah
that
was
basically
my
talk.
The
summary
is
that
the
recognized
executor
provides
a
solution
for
hardware
accelerator
scheduling,
so
we
basically
solve
the
scheduling
problem
by
just
extend
the
recon.
The
standard
recon
standard
was
to
executor
by
by
reconfigurable
hardware
properties
and
therefore
we
bring
the
event
driven
programming
paradigm
for
hardware.
C
So
in
our
reference
application
there
are
no
migrations
between
hardware
and
software,
so,
for
example,
if
we
have
both
available
hardware
and
software,
we
could
also
change
yeah
the
execution
domain
during
runtime,
but
we
didn't
do
that
until
now,
and
one
yeah
future
work
could
also
be
the
integration
into
the
standard,
ros
executor
so
in
the
package
for
that,
but
we
didn't
do
that
until
now,
and
then
thank
you
for
attention.
If
you
have
any
feedback
or
comments,
just
ask
me
now
or
write
me
an
email
and
I'm
happy
to
hear
from
you.
B
C
B
C
Yeah,
so
I
think
there
is
so
the
natural
bottleneck,
I
think,
is
the
icap
interface,
so
it
limits
the
overall
reconfiguration
time
for
the
bandwidth
for
the
reconfiguration
time
to
400
mega
bits
per
second,
and
we
have
just
one
of
that,
so
that
is
basic
yeah
overall,
that's
a
bottleneck
of
the
overall
thing.
So
if
we
fully
leverage
this
icap
interface
yeah,
we
have
no
chance
to
get
it
faster.
C
C
A
I
was
gonna,
I
was
gonna.
Add
that
so
the
model
that
has
been
presented
sounds
at
least
to
me
and
out
of
intuition
something
very
interesting
for
things
like
ross
actions
or
maybe
even
services,
actions
and
services
are
are
typically
expected
to
run
for
a
bit
longer
you,
you
essentially
send
an
action
request.
A
Something
happens
in
the
robot,
such
as
you
know,
a
series
of
of
movements
and
manipulation,
calculations
that
essentially
bring
your
end
effector
into
a
required
position,
so
that's
taking
significantly
higher
times
than
the
ones
presented
today
and
I
think
for
those
scenarios
using
reconfiguration
in
an
isolated
or
series
of
isolated
slots.
I
think
there's
value
lots
of
value
in
that
yeah
to
be
honest,
but
but
but
there's
indeed,
there's
indeed
merit
in
in
the
question
that
you're
posing
because,
obviously
not
for
every
single
interaction.
B
Yeah,
so
the
visual
channel
processing
seems
to
be
an
example
that
you
may
not
want
to
lead
with,
because
that
got
me
right
over
the
edge
yeah
now
makes
sense,
makes
sense
excellent
yeah
by
the
way.
I
really
love
it,
because
that
you
know
standardization
of
the
programming
model
fantastic.
So
I
was
just
trying
to
see
how
we
would
be
able
to
use
this
so
well
done.
Christian
thanks.
A
Yeah
class,
one
on
that
christian,
your
work
is
always
super
exciting.
Maybe
a
quick
question
from
my
side
is:
could
you
describe
briefly
so
that
there's
ground
for
more
questions
and
we
already
passed
time,
but
could
you
describe
briefly?
How
did
you
benchmark
things
and
how
do
you
get
out
these
numbers?
A
D
Yes,
I
saw
that
even
for
this,
for
the
all
hardware
executor
for
this
sort
of
sorting
application,
there
was
some
outliers,
which
was
it's
like
a
really
low
bar
and-
and
I
wondered
what
is
the
content
or
have
an
idea?
What
makes
for
that
like
jitter
in
the
processing
time?
That
definitely
is
not
the
fpga
execution,
because
that
should
be
a
mistake.
C
So
I
don't
know
if
there,
for
example,
this
is
a
thought
application
which
transfers
large
amounts
of
data
between
the
two
boards
or
the
two
compute
platforms.
We
might
have
some
congest,
the
congressions
there
and
the
interface
and
the
roth
layers
and
so
on,
and
therefore
we
have
this
outliers
there
in
the
jitter.
D
Okay,
thanks
yeah,
very
cool.
I
also
thought
that
when
you
have
this
ros
computational
graph,
I
guess
you
could
also
anticipate
when
you
would
receive
certain
messages
like
if
you're
getting
frames
at
at
20
frames
per
second,
you
could
program.
You
use
the
I
cap
to
program
the
slot
before
you
would
receive
the
frame
unless
you
didn't
have
to.
E
C
In
this
approach,
some
kind
of
direct
connect
is
not
that
easy,
because
we
have
changing
notes
or
changing
callbacks
in
the
slots
and
therefore
it's
a
bit
harder
to
generate
a
bypass
from
one
slot
to
another.
So
I
already
thought
about
that,
but
I
didn't
got
a
or
I
didn't
get
a
yeah,
a
good
solution
for
that.
A
So
so
maybe
I
can,
I
can
add
a
few
bits
in
here
robert
and
thank
you
for
the
questions.
So
so
we
need
to
differentiate
between
exactly
the
types
of
interactions.
At
the
ros
level,
there
is
essentially
four
types
of
interactions,
but
but
roughly
going
from
small
to
big
in
terms
of
networking
interactions,
the
first
one
is
is,
what's
typically
called
intra
process
interactions.
A
Okay,
these
are.
These
are
two
nodes
both
living
in
in
the
same
process,
and
in
this
case
there
was
middleware
framework
and
there
was
client
library
actually
not
even
the
ros
middle
framework.
The
rcl,
the
rose
client
library
does
that's
pointer
passing
so
there's
zero
copy
strategy
implemented
already.
So
in
that
case,
there's
not
much.
We
can
do
and
and
the
latencies
shouldn't
be,
that
impactful
for
the
intro
process
scenario
for
the
inter
process.
These
are
these
are
running
in
two
different
processes.
In
this
case
it
depends
on
the
actual
implementation.
A
Typically,
a
roster
communication
message.
Passing
infrastructure
was
relying
on
the
dds
implementation.
So
so
again,
if
we're
looking
at
the
the
different
processes
situation,
then
ross2
is
relying
on
the
dds
implementation
and
in
that
particular
scenario,
there's
huge
huge
margin
for
improvements,
because
you
don't
really
need
to
use
the
networking
stack
for
that.
A
You
can
use
various
other
strategies,
such
as,
for
example,
relying
on
the
fpga
or
the
gpu
to
to
pass
this
particular
information
by
using
shared
memory
or
related,
and
this
is
something
that
some
companies
are
implementing
out
there.
There
is
the
icrx
project,
which
is
supported
in
the
community
as
well,
and
it
does
it
does
that
for
inter
process
communication,
so
so
over
there
by
default.
We
are
suffering
from
that
and
they
are
smarting
for
optimizations.
A
Now
beyond,
inter
process,
there
is
intra
network
and
internet
work
interactions
and
in
that
case,
there's
not
really
lots
of
things
we
can
do
because
dds
implementations
are
already
optimized
for
those
types
of
things
and
of
course
we
could
come
up
with
a
new
communication
middleware,
but
that
defeats
the
purpose
I
think,
of
of
building
on
top
of
dds
due
to
the
interoperability
aspect.
So
so
I
think
there
is.
E
A
A
That
is
correct.
That
is
correct.
Absolutely
so,
hey
folks,
thank
you
once
again
for
joining.
It
was
an
exciting
call.
I
will
follow
my
colleagues
recommendation
and
do
myself
again
alone,
another
one
and
record
myself
so
that
it
can
be
completed
this
time
and
yeah
we'll
keep
its
rating
through
this
course
to
the
official
channels
through
the
ros
acceleration
github
organization.