►
From YouTube: Integration of Edge and Cloud IoT Platforms - Drasko Draskovic & Janko Isidorovic, Mainflux
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Integration of Edge and Cloud IoT Platforms - Drasko Draskovic & Janko Isidorovic, Mainflux
As network processing functions move towards the edge, it becomes important that IoT middleware enables connectivity, device management and data filtering in the form of IoT platform running most often on the IoT gateway HW. However, these gateways must be controlled and managed from a cloud application, and they themselves become "things" in the larger IoT-schema. Moreover, cloud IoT platform should also aggregate and store data from thes gateways and devices connected to them, but also the data of other devices - sensors and actuators that directly connect to cloud, without a need for an intermediary gateway. Using an example of LF EdgeX Foundry project for IoT gateways and Mainflux project as a cloud IoT platform for gateway management, we will present a typical architecture of an end-to-end open-source source solution for IoT edge-fog-cloud continuum.
https://sched.co/Nrul
A
And
the
edge
are
starting
to
be
integrated
and
edge
today,
edge
computing
being
especially
a
hot
topic,
and
so
this
call
this.
This
talk
is
titled
the
integration
of
edge
and
cloud
IOT
platform.
So
we
want
to
examine
what's
happening
in
the
cloud,
but
what
is
also
happening
in
the
edge
and
how
we
can
potentially
unify
this
edge
for
cloud
continuum
when
we
are
building
IOT
solutions,
my
name
is
Josh
Kokoschka,
which
I
come
with
a
lot
of
background
in
building
oil
tea
system
and
especially
a
long
time.
A
Involvement
in
open
source,
community
and
I
started
this
project
about
five
years
ago.
The
project
I
will
mention
today
and
together
with
a
group
of
enthusiasts
and
and
and
open
source
experts
built
a
platform
that
today
is
in
in
wide
use
all
over
the
world,
and
today
with
me,
is
young
kyunki
sidorovich.
A
He
is
a
co-founder
of
maine
fox
company,
a
company
that
was
built
on
the
top
of
this
project
and
also
very
important
for
this
talk.
His
background
is,
is
in
in
edge
computing
and
he's
been
a
chairman
of
one
big
linux
foundation's
project
called
Ajax
foundry
he's
been
sitting
in
technical
steering
committee
and
he's
been
sharing
application
work
group,
so
a
few
words
about
main
flux
project.
A
It's
an
open
source
project
that
is
Apache
2
license
completely
patent
free,
open
source,
IOT
platform,
and
it
has
been
growing
steadily
and
with
over
30
con
tributaries,
contributing
to
it.
Often
it's
very
live
project.
It's
very
active
and
some
choices
were
made
when
we
started
building
this
healthy
cloud.
It's
written
in
go
programming.
Language
which
gives
us
certain
features
that
turn
out
to
be
important.
A
On
the
top
of
this
open-source
project,
a
company
was
built
at
startup
that
offers
professional
services
simply
because
when
you
build
an
open-source
project,
you're
looking
for
a
business
model
and
then
there
is
a
there
are
a
lot
of
companies
that
are
addressing
you.
They
start
using
the
software,
and
so
basically
in
order
to
have
a
continuous
support
to
the
open
source
project.
A
company
called
main
flux.
Labs
today
exists
and
building
its
building
solution
on
the
top
of
the
Memphis.
A
But
it's
also
maintaining,
together
with
with
the
community
the
the
core
of
the
project
and
now,
with
this
introduction
in
place.
I
would
like
first
to
address
the
cloud
portion
of
the
IOT
middleware
and
the
way
that
we
are
building
in
general,
the
IOT
solutions
and
then
I
will
leave
the
floor
to
the
anko
so
that
he
drills
down
into
edge
part
of
what
has
been
done
with
that
jax
foundry
project
and
then
we'll
talk
again
about
the
integrations
of
those
edge
solutions
and
the
cloud
solutions.
A
So
this
has
a
nice
consequence
that
actually,
the
micro
services
themselves
are
very
small
and
very
tight.
They
they
can
be
a
few
megabytes,
maybe
in
size.
So
it's
good
for
the
cloud
because
you
easily
distribute
those
micro
services,
not
necessarily
only
with
Dockers
but
go,
builds
statically
compiled
binary
that
you
just
take
it
put
it
on
some
other
server
run
it
and-
and
it
runs
right-
the
everything
is
compiled
into
the
binary-
and
this
is
a
nice
consequence
also,
yes,
that
you
can
actually
deploy
the
same
code
on,
for
example,
IOT
gateway.
A
You
don't
have
to
change
any
single
line
and
have
the
same
code
base
that
runs
on
the
in
the
cloud
and
on
the
gateway,
and
this
is
like
the
ultimate
destination
where
we
are
going
at
least
with
this
project,
is
building
a
unified
iot
platform
that
can
spread
all
around
all
around
this
continuum.
From
the
edge
from
the
modest
gateway
that
can
have
maybe
a
few
few
mats
of
ram,
so
we
easily
run
the
whole
solution
on
the
raspberry,
pi
type
of
computer
and
also
the
same
solution.
A
We
scale
it
out
to
hundreds
of
machines
in
the
cloud
geographically
distributed,
but
to
go
back
to
the
to
the
architectural
blocks.
Is
that
usually
there?
Some
clients
which
are
actually
the
things
and
those
things
are
are
connecting
through
some
security
through
some
proxy?
There
are
different
protocols
to
the
back
end
of
the
system,
and
then
usually
you
have
some
device
management
right
and
then
the
there
there
is
a
plethora
of
what
we
call
protocol
adapters.
A
The
point
is
that
the
things
what
we
call
them,
the
things
the
clients
part
of
the
IOT-
are
different
devices
right
and
those
different
devices
speak
different
protocols
and
those
protocols
in
IOT
are
mostly
unusual
for
a
typical
cloud
solution.
So
you
you
here,
for
example,
MQTT
protocol
or
constraint,
applications
protocol
and
those
protocols
are
a
bit
specific.
Some
are
pure
tcp,
like
MQTT
coop
on
the
other
scan
is
UDP,
so
it
demands
a
DTLS
as
a
encryption
mechanism
and
so
on.
A
There
is
there,
of
course,
clients
on
the
on
the
input?
Sorry
I'll
just
go
back
its
clients
on
the
input,
but
there
there
is
a
part
which
can
be
a
proxy
security
authorization,
user
management,
device
management
and
so
on.
One
big
part
is
of
course
messaging
part
is
this
is
where
you
accept
all
those
connections
from
different
devices
and
then
sending
those
messages
internal
in
the
system.
A
It
then
sends
a
message
to
one
of
the
protocols:
they
say
it's
an
MQTT
device.
It's
going
to
the
nuts,
which
is
extremely
performant
message,
broker
it's
a
cloud
native
message
broker
and
then
it
sends
it
to
different
applications,
different
consumers
or
write
it
in
a
database
and
so
on.
And
these
all
these
here
are
the
micro
services
that
we
implemented.
So
they
are
there
they're
in
the
in
the
matrix
core.
A
A
The
engineers
who
are
used
to
this
kind
of
platforms
would
understand
it
easily
and
and
operated
on
on
on
any
computer,
whether
it's
a
very
complex
cluster
in
the
cloud
or
it's
just
a
simple
IOT
gateway
right
so
going
into
the
deeper
into
details,
as
I
mentioned,
what
what
are
the
the
users?
Let's?
Let's
take
a
look
at
this
at
least
part
here,
so
these
this
might
these
micro
services,
which
is
mostly
user
and
device
management.
A
But
things
are
more
interesting
like
they
are
actual
devices
and
then
those
devices
must
be
as
well
represented
somehow
into
database
so
that
you
can
as
some
entities,
so
you
can
manage
them,
give
them
specific
attributes
and
so
on,
and
one
interesting
thing
with
the
main
flux
IOT
platform
with
this
cloud
is
that
for
messaging
in
order
to
simplify
it
and
to
make
it
very
abstract
and
practically
abstract
away
all
those
protocols.
We
we
it's
not
invented
but
introduced
a
notion
of
channels,
so
channel
is
basically
very
similar
to
a
topic
on
a
broker.
A
Let's
say
a
topic
on
an
MQTT
broker,
but
it's
but
it's
parallelized
for
any
of
the
incoming
protocols
so
that
you
can
actually
use
this
channel
and
make
two
different
devices
communicate
between
them.
They
post
on
the
same
channel
through
completely
different
protocols
so
that
you
can
subscribe
on
on
TTT,
for
example,
protocol
and
then
send
some
messages
via
HTTP
and
you
will
get
those
messages
on
the
on
the
device
and
basically,
with
this,
we
completely
abstract
away
the
protocols
and
simplified
the
creation
of
the
applications
and
and
the
firmware
on
the
devices.
A.
A
Very
interesting
part
is
definitely
security
and
IOT
platforms
are
coming
with
different
implementation
of
the
security
primitives.
One
of
the
typical
that
we've
so
so
so
far
in
the
open
source,
software
in
office
or
society
platforms
were
certificate.
Authentication
authorization
via
some
tokens,
some
credential-
that
better
thing
have
and
very
nice
example
of
very
high
security,
is
something
that
typically
Amazon
AWS
IOT
had
and
they
what
they
did
is
they
introduced
a
mutual
TLS
for
authentication
and
authorization
of
the
devices.
A
So
we
started
you
know
going
through
the
different
IP
platforms
that
are
out
there
today
to
find
a
similar
implementation
and
when
we
could
not
find
it
we
implemented,
we
implemented
it
in
main
flux.
So
I
think
that
it's.
The
main
facts
today
is
pretty
unique
in
in
that
sense
that
it
implements
a
mutual
TLS
for
the
device
connections
and
basically,
what
it
means
is
that
you
you
create
a
certificate
for
the
device,
is
a
client-side
certificate
and
then
authenticate
the
device,
while
this
certificate,
so
that
every
device
practically
has
a
different
certificate.
A
So,
in
order
to
do
this
four
different
protocols,
we
have
to
introduce
some
tricks
or
rather
specifications.
Where
should
we
keep
some
information
about
the
device
itself
in
which
field
of
the
certificate?
We
should
put
this
one-
and
this
is
all
well
documented-
on
the
in
on
the
project
documentation,
but
it's
nevertheless
it's
it's.
It's
a
building
block
that
we
are
very
proud
of
today,
because
it
introduced
really
an
industrial
level
of
security
into
into
this
open
source
platform.
A
But
as
ice
mentioned,
that
the
platform
itself
like
an
building,
an
IOT
middleware,
it's
not
only
about
messaging
it's.
There
is
a
lot
of
thing
to
do
about
device
handling,
but
also
those
difficult
infrastructure
Plumbing's
like
and
when
I
talk
about
this
and
I
mean
mostly
about
DevOps
right.
How
do
you
deploy
platform?
How
do
you
maintain
it?
What
do
you
log?
How
do
you
log
and
so
on?
A
So
all
the
stuff
that
are
that
you
need
in
order
to
have
a
reliable
e
running
cloud
platform
today,
right
and
so
main
flux
itself
implements
by
design
through
the
domain
driven
design
I
mentioned
and
through
those
features
that
go
and
go
kit
offer
us
structured
logging.
It
provides
all
the
metrics
and
we
are
capable
to
today
to
push
those
metrics
into
Prometheus,
for
example,
or
or
panda
logs
through
a
fluently
into
the
elastic
search
and
then
visualize
them
with
cabana,
but
one
other
very
interesting
thing.
A
A
so
called
a
so-called
sourcing
is
basically
that
whenever
some
event
happens,
device
connects
or
disconnects
or
sends
a
message
or
something
like
this.
You
would
like
to
inform
the
other
micro-services
that
this
event
happened
and
we
use
for
this
new
red
extremes.
So
red
is
five-0
recently
introduced
so-called
red
streams,
which
is
a
log
based
broker
or
storage
that
that
you
can
use
for
this
kind
of
integrations,
very
useful
and
and
and
already
present
in
this
in
this
implementation.
A
A
B
B
It's
open
source
it
wants
to
be,
it
is
vendor
neutral
and
it
architecture
is
quite
similar
to
the
architecture
of
the
of
the
main
folks.
So
it
really
fits
well
with
what
we
do
on
the
on
the
cloud
side
on
the
server
side,
with
what
we
do
with
the
hedrick's
boundary
and
the
edge
side
edge
expounder
ii
is
a
set
of
api's
loosely
coupled
microservices
and
educate
andre
has
implemented
those
api's
using
the
go
language,
so
go
programming
language.
B
So
if
it's
the
same
all
programming
languages
we
use
on
the
main
park
side,
you
can
also
deploy
expander
using
docker
containers.
So
the
same
way
as
you
can
deploy
main
flux,
it's
hardware
and
operating
system
agnostic.
You
can
run
Ajax
on
arm
or
Intel
on
Windows
Linux,
Mac,
even
a
real-time
operating
systems.
B
If
you
need
and
as
I
said,
it's
Linux
Foundation
project,
it's
a
party
to
license
the
goal
of
the
project
is
to
enable
companies
to
quickly
create
add
solutions
and
to
integrate
a
different
type
of
sensors
to
be
Vanderloo,
vendor-neutral
and
and
to
help
reduce
time
to
market
with
using
ajax
foundry.
So
in
essence,
when
you
take
a
look
when
we
talk
about
the
architecture
of
the
edges,
foundry,
it's
a
set
of
10
plus
micro
services.
A
B
Have
devices
that
are
connecting
to
the
system
using
many
one
of
the
many
protocols
and
we
have
rest
OPC,
UA,
Modbus,
a
back
net,
ZigBee,
Bluetooth,
low-energy
and
PTT,
etc,
and
then
those
device
services
push
the
data
to
core
services
and
then
core
services
push
the
data
to
distribution,
services
and
distribution
services
then
can
push
the
data
out
of
the
edge
to
next
year
in
your
IT
platform
or
to
cloud
you
have
management
and
you
have
security
also
in
the
system.
So
how
would
that
work?
B
If
you
have
a
thermometer
that
recall
temperature
and
it
would
send
data
using
what
is
this
back
net
to
the
back
net
device
service,
then
the
vice
service
would
do
its
part
and
push
the
data
to
the
core
data.
Core
data
would
push
the
data
to
distribution
micro
service,
and
then
you
can
connect
this
distribution
micro
service
to
many
different
clouds
like
as
or
AWS,
Google
etc,
but
and
Lane
Fox.
Of
course.
B
However,
distribution
can
also
push
data
back
to
local
analytics
and
then
this
local
analytics
can
do
some
processing
on
the
edge.
So
in
case
you
lose
Internet
connectivity.
Your
edge
would
still
be
able
to
do
some
computing
and
do
some
decisions
independent
of
the
internet
connection
and
if
required,
then
the
local
and
engines
could
issue
a
command
to
a
command
micro
service,
which
would
then
push
that
command
to
an
PDT
adapter
and
maybe
like
stop
machine
because
it's
overheating.
B
A
Things
Hank
offer
for
this
presentation
of
the
Ajax
foundry,
and
so,
let's
now
take
a
look
at
the
possible
integration
of
one
system
that
has
main
flux
in
the
cloud.
It
has
main
flux
on
the
edge
as
well
and
and
it
has
expounded
awning
on
another
edge
computer
IOT
gateway.
So
we
connected.
So
what?
What
are
the
scenarios
that
can
actually
have
happened?
When
we
talk
about
the
device
connectivity
from
just
from
the
connectivity
perspective,
we
can
connect
devices
directly
to
the
cloud
right.
A
If
we
have
some
a
little
bit
more
powerful
sensor,
let's
say
Wi-Fi
connectivity
on
it.
It
can
go
directly
through
your
home
router
or
through
some
industrial
router
and
connect
directly
to
the
application
in
the
cloud
or
you
can
connect.
Let's
say
something
that
produce
much
more
data,
a
connected
vehicle
or
or
some
other
machine.
But
what
happens
when
you
have
a
devices
that
maybe
don't
don't
have
the
IP
connectivity,
a
Bluetooth
devices,
ZigBee
devices
and
so
on?
A
You
would
like
to
to
connect
them
first
to
the
local
gateway
and
then
connect
them
to
the
cloud.
There
are
many
reasons
why
you
need
those
edge
computers.
You
need
them
for
local
connectivity.
You
need
them
to
save
the
bandwidth,
because
maybe
you
don't
want
to
send
everything
on
the
cloud
you
want
to
do
some
processing
here
right
to
save
the
save
the
bandwidth
of.
What's
between
the
edge
and
the
cloud,
you
want
to
have
a
low
latencies
between
some
mission-critical
machines
and
decisions
that
are
made
on
the
edge
processing.
A
So
what
will
main
flux
as
a
initially
cloud
platform?
But
now
look
more
in
this
unified
IOT
platform
context
provide
you
on
the
edge
well,
it
will
provide
you,
those
all
those
characteristics
that
I
mentioned
before
when
I
explained
what
main
flux
has.
So
it
has
five
hundred
nine
certificates,
mutual
TLS.
A
If
you,
if
you,
if
you
put
this
system
on
your
edge
gateway,
then
you
can
connect
a
huge
number
of
various
different
protocols,
devices
that
are
talking
different
protocols
to
your
edge
gateway
and
then
send
data
to
a
main
flux
in
the
cloud
through
these
distribution
services
as
Yonko
mentioned.
But
when
you
do
this
kind
of
integrations
again,
you
will
need
some
kind
of
IOT
agent,
a
small
micro
service
that
is
running
on
your
gateway,
to
enable
this
kind
of
communication
on
the
control
plane,
not
only
on
the
data
plane.
A
Okay,
you
can
use
this
micro
service
to
pump
data
out
to
the
cloud,
but
how
will
you
control
your
gateways?
How
will
you
manage
thousands
of
Gators
that
you
deployed?
Well,
you
use
this
for
this
in
order
to.
In
order
for
this
to
happen,
you
have
to
have
some
kind
of
firmware
that
is
only
used
for
a
control
plane
between
the
cloud
and
your
gateway
and
in
a
jets.
A
So
what's
missing
today,
but
we
is
coming
very
soon
for
the
ajax
is
a
UI
so
that
you
can
actually
have
a
nice
user
interface
of
the
all
the
all
the
systems
that
are
running
on
your
gateway
and
typically,
all
those
micro
services
in
Ajax,
as
in
main
fox
running
dockers.
So
you
can
actually
deploy
them
with
with
either
compose
or
maybe
kubernetes
some
kubernetes
variant
and
basically,
with
this
release
integration
explication
side.
I
would
like
to
finish
this
talk,
because
I
would
like
to
leave
a
few
minutes
for
eventual
questions
if
you
have
one.
C
A
And
main
flux
itself
provides,
as
I
said,
the
facilities
for
integration
to
true
event
sourcing.
It
is
very
easy
to
use
so
whatever
happens
in
the
system
can
be
connected
and
and
sent
as
an
event
to
other
parts
of
the
legacy
legacy
systems
and
then
one
other
point
of
integration.
If
you
needed
would
be
on
the
on
the
very
edge
or
on
the
Gateway
themselves,.
C
A
Channel
is
an
entity
that
you
provision
in
the
in
the
system
that
you
configure
in
the
system
and
it's
up
to
you
as
a
creator
of
the
system
to
provision
as
many
channels
as
you
like.
So
typically,
you
can
give
maybe
your
device
to
channels
or
or
10
or
100,
it's
it's,
because
it's
an
analogous
to
MQTT
topic.
You
can
imagine
that
one
device
can
be
connected
to
many
any
amputee
in
subscribe
or
publish
too
many
amputee
3
topics
and
analogous
to
this,
your
device
can
subscribe
or
publish
to
many
main
facts
channels.
A
That's
the
idea,
and
very
often
for
the
Gateway
would
have,
for
example,
two
channels,
one
that
you
will
use
for
the
data
data
plane
for
the
communique
for
communicating
data,
maybe
sending
them
to
the
cloud
and
the
other
channel
can
be
used,
for
example,
for
the
control
plane,
where
you
would
set
some
control
messages
over
this
channel
to
the
Gateway
so
that
it
can
stop
start
local
services.
Give
you
back
logs
in
the
cloud
and
so
on.
Just.
B
To
other
thing,
you
can
also
have
a
single
channel
where
multiple
thermometers
send
the
temperature,
and
then
you
have
one
channel
where
multiple
devices
send
the
pressure,
and
you
will
main
Fox
has
the
notion
of
publisher
of
a
device
that
sent
the
data.
So
you
can
have
a
single
channel
for
the
temperature,
the
other
channel
for
pressure
third
channel
for
voltage,
etc.
So
it's
up
to
you
to
configure
it
the
best
way
you
need
it.
It's
really
flexible
and
you
can
have
multiple
channels
per
device
single
channel
device.
A
Yes,
so
I
think
I
put
it
on
on
one
of
the
sides,
but
so
far
we
saw
one
big
deployment
with
one
of
the
biggest
retailers
in
USA,
which
is
deploying
main
flux
practically
in
every
store.
So
we
are
talking
about
two
thousand
stores
and
then
connecting
those
main
fluxes
in
and
managing
them
with
another
main
flux
instance
in
the
geographically
distributed
cloud.
So
this
is
the
scenario
of
unified
IOT
platform
that
I
spoke
about,
which
which
we
want
to
go,
go
yeah.
So
this
is
one
example.
A
The
other
example
is
one
big
company
in
the
domain
of
oil
and
gas.
That
is
you
that
is
using
main
flux
in
managing
a
lot
of
gateways
as
well.
The
gateways
are
not
a
main
first
gate
race
yet,
but
they
have
their
proprietary
software,
but
they
have
some
pieces
of
the
software
that
is
necessary
to
communicate
with
main
flux
on
the
control
plane.
A
The
HD
IOT
agent
that
I
mentioned
so
this
is
this
is
another,
that's
a
use
case
or
business
case,
and
then
there
we
saw
there
with
so
many
flux
applied
in
smart
buildings,
smart
offices,
as
well
so
for
to
control
a
lot
of
sensors.
That
would
be
connected
either
directly
through
to
mailbox
in
the
cloud
or
to
some
kind
of
gateway
and
didn't
miss
something
we're.
D
B
One
more
thing:
with
the
main
flux
edge
D:
you
don't
need
to
have
VPN
to
your
gateway
anymore,
because
this
edge
D
enables
bi-directional
secure
communication.
So
you
don't
have
to
manage
virtual
networks
anymore.
You
just
put
the
demon
on
your
gateway
and
you
take
it
in
the
field.
Connect
it
and
it's
gonna
be
fine
and
secure,
and
we
have
just
a
single
port
that
is
using
used
for
all
the
communication
so
attack.
Food
footprint
from
the
outside
is
just
as
small
as
possible.