►
From YouTube: ROS 2 Hardware Acceleration Working Group - meeting #5
Description
Fifth meeting of the ROS 2 Hardware Acceleration Working Group (HAWG) where we focus on reviewing progress the group made during 2021, including the presentation of new objectives for 2022. We also touch on recent updates over the last few weeks and then enjoy a guest talk from professor Emad Samuel Malki Ebeid who told us about his research using ROS 2 in applications leveraging Drones for Infrastructure Inspections and Interactions and how hardware acceleration is helping them in their endeavours.
Due to technical difficulties, the first part of the meeting was NOT recorded but all the information is available in the minutes (https://docs.google.com/document/d/185Cy1xjpAOgJygEOnlf5OCgOQTywmF0qgSpS3GiW16Q).
A
A
A
So
he
kind
of
wearing
a
faraday
cage
around
his
body
and
then
he's
using
his
hands
and
eyes
to
look
for
defects
and
his
daily
job
is
to
go
back
and
forth
to
look
for
these
defects
and
yeah,
and
that
also
comes
to
other
countries
like
the
previous
one
was
in
the
united
states.
This
one
is
in
asia
and
it's
like
kind
of
a
common
job
yeah
these
workers.
They
do
it.
A
So
the
first
thought
is
like
okay,
why?
We
should
why
we
cannot
just
use
thrones
and
instead
of
sending
humans
online
and
then
make
them
do
a
tedious
work.
And
then
that
is
the
first
project
idea.
I
came
is
to
mainly
build
drones
and
let
this
drones
fly
around
the
power
lines,
but
as
we
all
know,
that
drones
can
fly
for
a
short
ter
for
a
short
time
like
20
minutes,
and
then
our
part
is
to
use
a
magnetic
field
to
recharge
the
drones.
A
So
if
you
can
build
a
drone
that
can
land
on
the
power
line,
harvest
the
energy
and
then
they
can
fly
again,
so
we
can
do
a
job.
And
here
we
came
up
with
from
hoover
ims
in
germany
to
build
a
harvesting
mechanism
that
could
be
attached
to
a
drone
and
also
still
to
be
a
lightweight.
So
if
they
can
recharge
the
battery
to
make
it
smart.
A
We
connected
this
cloud
services
in
this
way
that
drone
knows
where
to
go
and
when
to
go
and
when
it
collects
data
it
sends
back
to
the
cloud
and
in
order
to
speed
up
this
process.
So
we
also
run
ai
algorithm
onboard
so
to
detect
this
defects,
and
we
fly
these
drones
in
ns1,
so
we
can
yeah.
We
can
cover
all
of
the
four
angles
and
the
power
lines,
and
then
we
let
these
drones
communicate.
A
So
we
use
a
lower
one
like
a
low
power,
long
range
wires
area
network
to
send
the
coordination
data
between
the
drones.
So
you
can
fly
along
with
the
lines,
so
we
can
combine
in
this
project
the
swarm
system,
energy,
harvesting
and
ai
in
the
project,
and
that
was
funded
nationally
in
denmark
and
they
gave
us
a
budget
of
around
3
million
euro
back
in
the
days.
A
If
you
are
interested
in
this
work,
you
can
of
course
read
it.
We
put
it
all
all
of
our
data
open
source,
so
you
can
check
this.
Article
was
published
in
ikra
is
a
well-known
conference
in
robotics
after
we
made
the
drone
quite
advanced
yeah.
So
in
this
case
we
we
built
a
custom
drone,
so
the
full
drone
that
you
see
is
just
this
is
built
from
scratch.
A
Here
we
added
on
the
top,
the
energy
harvester
and
also
we
used
an
nvidia
card,
because
now
we
need
to
run
the
a
iron
board
and
we
see
the
raspberry
pi
cannot
do
much,
rather
than
for
do
we're
running
a
very
lightweight
control,
algorithm
to
track
a
line
and,
of
course,
plus
we
use
the
rows
and
methods
to
communicate
with
the
flight
controller
yeah.
So
in
this
case,
we
we
moved
to
nvidia.
A
A
A
I
just
stay
here
for
a
second,
so
this
unit
is
weights
around
one
kilogram
and
it
can
charge
the
drone
battery
which
in
our
case
is
10
amp
hour
within
three
hours.
Two
hours
and
40
minutes
so
expect
here,
800
amp
to
pass
through
the
line
that
represent
the
primary
current
and
then
the
drone
is
harvesting
energy
and
then
the
drone
flies
15
20
minutes
afterwards.
A
So
we
still
have
some
optimization
to
be
done,
so
we
could
save
in
the
nvidia
card
a
little
and
make
the
drone
lighter
and
try
to
incorporate
more
activities
together
to
make
it
better
and
that's
what
we
actually
aim
for
for
the
next
or
the
current
work
regard
beside
this
work.
We
also
test
again
is
a
electromagnetic
interference,
so
we
built
a
small
drone
covered
by
faraday
cage
and
then
we
touch
it
here,
150
kilovolt
lifeline,
and
then
we
thought.
Okay,
all
are
good,
so
we
can
do
it.
A
We
tried
with
the
400
kilovolt
that
is
the
highest
in
in
our
region
here
in
in
finn,
but
yeah
when
we
were
doing
this,
the
drone
just
crashed.
That
was
too
much
electric
field
passing
through
the
drone.
So
we
understood
what
went
wrong
and
then
we
protected
better,
and
then
we
can
see
that
ron
was
working
under
the
arcs,
the
electric
arcs
and
the
touch
of
the
line
without
issues.
A
A
A
So
if
I
could
see
a
futuristic
drone
that
we
can
charge
and
then
go
to
the
bridge,
make
inspection
and
we
come
back.
So
that
is
a
scenario
here
in
this
project
called
drones
for
safety,
and
that
is
a
new
project
that
started
in
june
2020
and
we
aim
to
do
the
same
to
respect
the
british
and
apply
the
same
for
railways
so
that
ron
can
inspect
the
railways
similar
as
it
does
with
power
lines.
A
A
One
of
the
advantage
that
an
fpga
or
or
a
hardware
accelerator
give
is
mainly
the
reconfigurability
part,
so
I
could
imagine
that
drone
could
be
running
one
task,
every
task
at
a
single
time,
so
inspection
maybe
later
to
do
control
to
drag
a
line,
maybe
later
to
send
data
to
the
cloud
and
so
on.
So
we
have
several
activities
that
they
can
run
one
after
one.
If
we
do
it
in
smart
and
hardware,
we
could
benefit
a
lot
from
the
reconfigurability
aspect
and
the
hardware
accelerators,
while
also
doing
it
in
real
time.
A
So
this
is
our
current
setup
is
to
have
a
classic
drone
approach,
low
level
flight
controllers
that
keep
the
drone
on
air
communicate
with
the
ground
station,
the
classic
approach
we
that
we
in
the
past
we
used
to
do
is
to
use
a
raspberry,
buy
or
an
nvidia
card
to
to
interact
with
sensors
and
then
communicate
with
other
nodes
and
our
current
activities
to
replace
these
two
is
to
use
we
working
mainly
with
alter
96
and
hopefully
in
the
future.
We
will
also
work
with
the
kve
260.
A
And
yeah,
so
if
you
can
run
the
ros
2,
we
accelerate
some
nodes
and
we'll
communicate
and
accelerate
some
activities
as
ros
nodes
and
the
hardware
that
could
be
making
accelerated
sensor,
fusion
data
or
running
a
dbu
and
then
looking
for
defects
and
work
around
that.
So
we
could
have
a
high
performance
drone
system.
At
the
end
we
took
the
first
step
last
year,
so
I
had
a
master's.
A
A
So
he
checked
the
functionality
was
working
and
then
he
imported
that
to
the
altera,
96
v2
and
then
use
the
ros.
So
it's
px4
and
make
the
same
thing.
The
target
is
the
boxy
now
the
unit
here
so
he's
looking
for
a
boat
here,
a
picture
of
a
boat
and
then,
when
it
details
the
picture
it
lands.
So
I
could
see
the
performance
of
the
drone
is
quite
nice.
It's
very
stable
and
it
detects
the
land.
A
A
A
A
Having
this
ubuntu,
we
can
run
ros2
foxy
on
top.
That
is
also
needed
for
the
px4,
and
then
we
use
rtbs
to
acce
to
communicate
between
the
flight
controller
and
the
cvu
in
between
we
use
axibus
to
change
data
between
sensors
sensor,
fusion,
algorithms
here
and
also
an
equalized
testing
with
fin
a
neural
network
to
to
detect
objects
and
then
based
on
the
results.
This
data
is
going
up
to
the
software
taking
decision
and
then
we
steers
the
drone
right
left
based
on
what
we
want
to
do.
A
You
can
check
out
this.
You
can
check
out
this
activities
and
and
our
github
group,
the
travel
isdu
group
you
can,
you
can
see
how
far
we
went
so
for
now
we
have
a
working
prototype,
we
can
send
commands
back
and
forth.
We
can
control
a
drone.
We
can
also
build
custom
ips
and
also
to
change
data
through
xbus
and
bramps
to
between
the
bs
and
pl
sites.
A
So
we
tried
with
millimeter
wave
high
resolution
time
of
flight
cameras,
three
cameras,
eight
segment
lighter
and
the
classic
cameras,
and
we
make
analysis
to
see
how
they
react
against
the
power
line,
and
we
could
see
here
a
stereo
camera.
We
have
a
lot
of
noise,
eight-segment
light.
That
was
quite
okay,
but
we
also
were
impressed
about
the
millimeter
wave
that
gives
us
really
accurate
results.
A
A
A
B
All
right,
fantastic,
thank
you
so
much
madsen!
Well,
that
was
that
was
fantastic,
really
really
exciting
work.
You
guys
are
doing
and
looking
forward
to
see
those
drones,
perching
the
power
lines
recharging
themselves
and
ensuring
that
they
are
in
good
health,
good
good.
So
let's
open
it
up
for
questions.
I
guess.
C
From
a
software
libraries
utilized
in
the
different
projects
there
can
I
find
that
information
in
the
papers
I'm
just
trying
to
identify
where
to
go?
Look
just
so.
I
can
identify
and
perhaps
prioritize
in
terms
of
what
to
what
to
optimize
for
similar
projects,
because
I
really
connect
with
what
it
is
being
used
for
right.
A
B
B
I
mean
sorry,
that's
kind
of
like
the
the
quickest
way.
It
comes
to
mind
to
me,
at
least,
if
you
want
to
introspect
the
the
computational
graph
to
identify
the
resources,
as
as
it
was
being
questioned,
because
that's
that's
the
easiest
way
right
to
get
it
up
to
speed,
to
get
it
in
gazebo
and
then
to
to
actually
instrument
things.
Yeah.
A
A
B
I
mean
for
what
concerns
and
I'm
sure,
like
others,
might
be
interested
as
well,
but
for
what
concerns
my
interest,
I
definitely
would
be
up
for
spending
a
few
hours
guys
and
try
to
reproduce
your
use
case
in
simulation
in
gazebo
and
from
there,
of
course,
try
to
help
with
with
changes
in
the
actual
architecture
and
proposing
changes
here
and
there.
But
I
think
that's
kind
of
like
the
baseline
for
for
trying
to
set
some
like
common
baseline,
and
we
haven't
yet
reached
that
point
in
the
working
group.
B
D
A
No
not
yet
so
for
for
the
moment
to
follow
the
line.
We
use
eight
a
pair
of
eight
segment,
solid
state
sliders,
so
this
eight
segments
tells
you
in
which
segment
as
as
the
line
is,
and
then
we
have
a
pair.
So
we
have,
we
can
draw
a
line
and
we
use
this
few
bits
coming
from
the
two
lighters
yeah,
with
a
classic
bib
controller
running
in
a
cpu,
so
that
was
okay
to
track
a
power
line
yeah.
A
So
our
next
is
to
do
the
acceleration
using
millimeter
wave
and
also
cameras,
and
that
will
become
heavy
for
a
classic
cpu
approach
like
a
raspberry
buy,
and
our
target
is
to
do
that
using
hls
and
in
the
hardware
side.
So
we
could
have
a
very
low
latency
in
the
fused
data,
and
thus
we
can
speed
up
the
flying
speed
of
the
drone.
So
for
now
we
fly
like
a
0.5
meter
per
second,
it's
really
really
slow.
That's
because
also
yeah.
A
A
A
So
it's
it's
moving
through
the
hardware
and
then
doing
the
acceleration
the
hardware
side.
So
this
way
we
can
save
some
time
instead
of
going
through
the
cpu
using
the
software
libraries
to
get
the
data
and
then
put
it
to
the
memory
and
pass
it
to
the
hardware.
So
we
do
that
as
a
hardware
to
millimeter,
with
data
processing.
E
A
Yeah,
so
in
this
paper
that
I
showed
the
survey
it
was,
we
had
the
different
tests.
Well,
usually
it's
a
power
line
really
gives
us
really
good
result
with
this
millimeter
wave.
But
if
you
put
the
millimeter
or
even
a
room,
you'll
not
really
get
any
good
data,
that's
because
of
the
bike,
ground,
reflections
and
yeah,
and
it
needs
some
more
poster
processing
in
order
to
get
a
good
data
out
of
it
what
we
tested
this
was
merely
during
winter
time,
so
it
was
a
kind
of
cloudy
snowy
day.
A
It's
not
really
visible.
It's
not
a
really
a
clear
sky,
the
millimeter
wave
we
are
around.
I
think
20
meter
far
and
then
up
to
one
meter,
far
from
the
power
lines,
and
we
could
detect
the
lines
yeah
so
based
on
the
data.
The
texas
instrument
is
showing
that
it
can
go
through
the
clouds
and
can
do
good
work
when
it
comes
towards
different
climate
conditions.
B
So
there's
here
another
question
from
joe
in
the
chat
he
asks.
Gazebo
simulation
is
all
digital.
Are
there
plans
to
build
a
computer
in
the
loop
simulation
to
benchmark
node
execution
on
device
and
by
computer
in
the
loop
simulation?
I
presume
you
mean
hitle
hardware
in
the
loop
joe
that
that's
a
common
practice
with
uivs.
F
Yeah,
so
I
I
guess,
there's
there's
the
computer
in
the
loop
bit,
which
I
was
talking
about,
that
would
have
sensor
information,
basically
dummied
up
for
it.
With,
with
like
a
data
creation,
node
say
for
imus
or
cameras.
My
question
was
more
about.
Let's
say
we
mock
up
interfaces
to
a
simulation
engine
like
gazebo
for
those
signals
then
plug
basically
use
the
ultra
96
in
this
case,
as
a
a
device
for
actually
just
running
all
of
our
processing
nodes
and
then
benchmarking.
F
A
Yes,
I
mean
we
follow
this
approach
for
well
with
ross
and
raspberry,
buy
in
general
that
what
we
see
in
simulation,
what
we
get
in
the
raspberry
pi,
it's
not
a
hardware
in
the
loop
simulation.
That's
because
it's
two
separate
things
why
we
don't
do
that?
That's
because
of
the
drone
itself.
The
drone
we've
seen
the
simulation
flying
nicely.
It's
not
the
same
drone
that
we
have
it
built
flying
outside
different
ways,
different
orientations
offsets
here
and
there
can
not
give
us
the
same
performance.
We
test
the
control.
Algorithms
are
working,
fine.
A
We
don't
really
have.
We
don't
have
really
plans
to
do
hardware
in
the
loop
simulation.
I
think
it
will
be
really
great
if
we
can
put
the
altera
96
as
an
accelerator
platform
with
with
the
communication
with
the
gazebo
through
through
uart
or
rtbs,
and
then
we
see
yeah.
The
drone
is
flying
in
gazebo
and
the
data
is
accelerated
and
ultra.
G
F
Yeah,
I
guess
I'm
thinking
about-
maybe
the
inverse
of
that.
I
want
a
simulation
engine
to
create
synthetic
sensor,
data
for
all
of
the
onboard
sensors
and
then
I
would
communicate
that
information
to
my
my
processing
device,
my
song
and
then
just
be.
Ideally,
the
the
competing
platform
wouldn't
know
that
it
was
being
driven
by
simulation
data
like
in
the
idealized
world.
It
would,
it
would
run
as
close
to
the
actual
in-flight
setup
as
possible.
G
Can
you
share
a
link
for
that?
Please,
benjamin
yeah
tell
you
what
I
might
shoot
you
I'm
not
I'm
not!
I'm
sort
of
indisposed
of
being
able
to
send
stuff
easily
is
my
email
addre.
Let
me
just
shoot
you.
My
email
address
right
here.
If
you
want
to
shoot
me
an
email,
I
can
send
you
stuff
on
that
yeah.
That
sounds
great.
G
B
B
So
so
definitely
interesting
that
that
path
and
also
happy
to
see
there
is
there's
excitement
in
here,
so
we
reached
the
top
of
the
hour.
Thank
you,
everyone
for
meeting
once
more
emma
samuel.
Thank
you
for
sharing
your
use
case
very,
very
exciting.
Thank
you
to
you
and
to
your
colleagues
for
this
fantastic
work.