►
Description
Presented by Xi Yang and the SENSE team
May 6, 2020
The Software-defined network for End-to-end Networked Science at Exascale (SENSE) is a model-based orchestration system which operates between the SDN layer controlling the individual networks/end-sites, and science workflow agents/middleware. The SENSE system includes Network Resource Manager and End-Site Resource Manager components which enable advanced features in the areas of multi-resource integration, real time responsiveness, and workflow middleware interactions.
The demonstration will show the status of ongoing work to integrate SENSE services with domain science workflows, such as those envisioned for DOE Superfacility operations. A common vision for these integrations is the provisioning of SENSE Layer 2 and Layer 3 services based on knowledge of current and planned data transfers. SENSE allows workflow middleware to redirect traffic at granularities ranging from a single flow, specific end-system, or an entire end-site onto the desired SENSE provisioned services. The SENSE Layer 2 services provide deterministic end-to-end resource guarantees, including the network and Data Transfer Node (DTN) elements. The SENSE Layer 3 service provides the mechanisms for directing desired traffic onto specific Layer 3 VPN (L3VPN) for policy and/or quality of service reasons.
A
Okay,
we'll
now
have
a
short
presentation,
a
demo
by
she,
dr
sheehan,
is
a
network
engineer
at
usn.
Until
last
year
he
was
a
senior
research
scientist
and
director
of
rnd
for
max
foreign
atlantic
crossroads
at
university
of
maryland.
Sense
is
a
collaboration
project
among
usnet,
caltech
formula,
argon,
burnout,
company
and
university
of
maryland,
and
the
project
is
led
by
inder,
the
director
viasnet
and
the
rest
of
it.
She's
gonna
give
a
show,
presentation:
okay,
she
it's
all
yours.
A
B
B
So
since
is
a
project
we
started
about
five
years
ago,
and
this
is
a
collaboration
between
many
institutes
and
so
when
we
started
on
the
next
project,
I
think
the
the
whole
idea
is
really
to
and
look
at
the
how
and
science
workflows
and
we
are
use
the
the
network
basically
and
many
workflows.
They
really
have
a
and
distributed
facilities,
devices
and
people
right
so
that
they
have
to
deal
with
the
network
very
frequently
and
so
the
by
looking
at
the
network.
B
Nanoscale
landscape
is
really
really
complex
right.
So
even
like
for
the
for
the
dual
environment,
we
have
a
more
normalized
network.
We
still
look
at
like
wider
error
networks
and
regional
networks
and
campuses,
and
also
your
and
now
your
your
your
local
hosts
in
the
systems,
your
instruments
and
clusters.
So
it's
it's
very
complex.
It's
typically
a
multi-domain
and
much
resource
and
distributed
infrastructure
all
right.
B
And
so
this
takes
a
lot
of
a
lot
of
operation
and
we
do
not
want
your
workflow,
your
science
workflows,
really
to
to
worry
about
this
next
big
activation,
and
so
that's
why
we
have
a
sense
to
and
to
come
into,
filler
filler
the
gap
right
so
since
really
want
to
operate
between
your
workflow
and
and
the
sdn
layer,
and
so
since
as
a
architecture
design.
So
it
really
has
two
and
layers
and
so
and
the
then
there
above
is
called
sense
orchestrator.
B
So
here
we
are
looking
at
the
service
oriented
architecture
with
a
consumer
layer
above
and
with
a
provided
layer
and
at
the
bottom
right.
So
it's
really
like
like
a
market,
and
you
are
trying
to
exchange
your
services
and
what's
unique
for
census,
we
are
using
a
model
driven
approach,
so
every
domain,
every
service
provider
is
going
to
describe
the
resources
as
a
information
model.
B
We
are
actually
using
a
semantic
web-based
model
to
make
transport
and
standardized,
so
this
model
is
exchange
across
the
system
and
the
orchestrator
is
going
to
aggregate
the
model,
and
so
we
have
like
a
so-called
end-to-end
infrastructure
model
and
because
we're
using
semantic
models.
It's
very
information
rich.
B
So
you
can
do
a
lot
of
things
from
formulas,
information
that
you
do
and
the
authority
discovery
and
patch
funding
your
all
kinds
of
like
orchestration
reasoning
right
so
that
it
can
become
very
weary
and
sophisticated,
and
so
at
the
oxidation
there
actually
and
since
really
doing
a
lot
of
negotiation.
This
is
a
typical
negotiation
and
a
transaction
system,
and
so
from
here
we
can
actually
give
a
user
a
good
interface
both
from
the
web
ui
and
also
from
the
from
a
rest
api.
B
B
We
need
to
actually
help
the
providers
to
to
describe
their
and
their
capacity,
their
capabilities
and
what
the
availability
of
the
resources
really
in
you
know
in
real
time,
so
yeah,
here's
and
just
repeat
what
I've
had
it
is,
so
we
have
all
kinds
of
resource
managers.
At
this
point
we
have
network
resource
managers
and
we
have
and
two
flavors
one
for
our
cars,
one
for
nsi
and
developed
by
yes
net.
B
B
Firstly,
we
want
integers
integrate
sense
into
in
the
application
workflow
agent,
so
we
have
a
register
api
and
for
the
northbound,
so
you
can
actually
use
and
add
to
your
application
codes
or
scripts
or
web
ui
and
and
and
secondly-
and
we
can
integrate
sense
into
other
middleware
tools
and,
for
example,
now
graph,
ftp,
globals
and
right
now.
We
also
have
integration
with
big
data
express,
which
is
another
data
transfer
tool
developed
by
by
ferminar
and
besides
transfer.
B
In
this
case,
it
could
typically
be
someone
using
touch
over
the
the
it
for
your
for
your
for
your
science
domain
right.
It
could
be
a
postdoc
or
could
be
summer
and
a
network
admin.
So
we
help
them
to
to
to
work
on
the
workflow,
scripting
and
and
compose
their
intents
are
suitable
for
their
workflows.
B
B
So
this
is
a
connection
dedicated
to
you
and
we
provide
like
guaranteed
binaries.
So
typically,
it's
gonna
have
no
latency
and
no
jitters.
So
it's
it's
very
good
for
for
notch
transfer
and
so
in
the
in
today's
demo.
I'm
going
to
actually
start
with
my
with
my
web
browser
and
and
and
after
this
demo,
I'm
also
going
to
present
a
layer
2
amino
3
service
from
our
service.
B
I'm
going
to
use
the
rest
api,
I'm
going
to
use
command
line,
but
by
doing
both
I
can
just
show
you
all
the
all
the
possible
combinations
over
you
provide
different
different
services
or
different
apis
or
interface.
So
this
is
the
sense,
orchestrator
and
ui.
So
what
you're
seeing
in
this
screen
is
no
visualization.
B
B
So
the
service
catalog
and
there
are
different
category
services
we
can
do
so
for
since
we're
more
focused
on
the
dynamic
network
and
collection
so
go
in
there.
We
can
say
there
are
three
headsets
and
connection
services
are
supporting
right
now.
Now
a
multi
pass
point
to
point
and
we
line
that's
what
we're
going
to
do
today.
B
We
can
also
do
multi-point
winner
bridge,
that's
basically
multiple
terminals
and
put
them
under
on
the
same
user
to
design
a
broadcast
domain,
and
we
can
also
do
a
monopoly
near
three
vpn
and
so
for
the
furnace
and
important
connection.
So,
first
on
add
one
connection
here:
all
right
so
and
it's
gonna
have
two
terminals
terminal
one.
I'm
gonna
pick
from
my
my
directory.
The
contact
put
here
then
from
here.
B
We
also
get
a
merlin
here,
so
I
use
we
don't
take
any
on
both
sides,
which
means
a
system
is
going
to
pick
us
for
me.
So
from
whatever
is
available,
so
you
can
complete
randomly
and
for
this
nail
tool,
I'm
also
going
to
assign
the
ip
addresses
on
both
sides.
So
once
it's
created,
we
can
actually
start
to
pass
traffic
without
manual
configuration
and
let's
do
a.
B
B
We
can
also
do
scheduling,
but
you
know,
but
in
this
case
I'm
just
going
to
make
it
happen
immediately
and
we
are
suggesting
app
range
for
assignment
on
both
sides
and
if
you
don't
do
the,
then
it's
going
to
be
ascend
automatically
out
of
a
dynamic
pool
and
which
we're
defining
our
our
information
model
right
here.
I'm
just
marking
homework
control.
So
I'm
going
to
put
this
manually
used
and
based
on
my
memory.
B
B
Then
we're
going
through
two
and
two
phase
and
commit
and
process
to
to
propagate
the
service
to
owner
and
resource
managers
to
disrupt
distributed
transaction.
Then
it's
going
to
commit
in
parallel
to
make
things
happen,
and
that
means
that
resources
will
get
allocated,
and
so
during
this
process
we're
going
to,
we
can
actually
say
and
all
the
information
is
in
in
the-
and
you
know
you
know,
knocking
and
table
right.
B
B
We
can
verify
those
information
has
been
applied
in
a
system
and
it
has
been
pulled
up
from
the
model
channel
and
so
now
what
we
can
do
is
we
can
so
visualization.
We
can
actually
say-
and
let's
go
here
so
in
a
big
visualization
window,
we
can
just
say
how
individual
resources
have
been
used
for
this
service
right.
So
here
we
talk
about
we're
going
to
go
through
maryland.
B
And
we
use
a
yes
net
and
scenic
and
we
go
to
a
car
tag.
So
what
I
can
do
here
is
you
can
use
your
your
visualization
and
tool
to
look
into
which
resources
have
been
involved
in
this
service.
So
I'm
not
sure
how
this
looks
in
a
room
video,
but
here
I'm
my
screen.
I
have
a
bunch
of
resources
have
been
highlighted,
highlighted
by
the
by
yellow
color.
B
You
can
actually
look
at
the
the
resource
per
service,
but
you
can
actually
look
here.
You
can
actually
see
only
the
resources
allocated
for
this
net
service
and
it's
another
very
useful
tool
is
called
manifest
and
this
actually
tool
and
the
if
you
are
using
the
application,
client
right
through
api
you
can
actually
and
to
query
and
to
get
manifest
and
because
we
just
said,
for
example,
we
tell
the
system
where
we
are
asking
for
any
winner
on
both
ends.
B
B
B
Yeah
about
17
milliseconds
of
dna
yeah,
so
so
that's
the
natural
service
here
and
when
you
are
done
with
the
service,
you
can
practically
just
cancel,
that's
going
to
determine
your
service
and
put
the
resource
back,
but
when
the
source
is
terminated,
you
can
always
bring
them
back.
If
it's
for
deleting
this,
you
can
always
re-install
your
service,
so
that's
the
layer
tool,
so
we
go
back
to
our
slides.
B
However,
I
have
one
screenshot
here,
which
is,
which
is
the
the
experiment
are
on
my
I
just
have
to
learn
nothing
light
actually
give
a
presentation
to
and
to
know,
cms
and
team
this
morning.
I
think
summer
folks
are
also
here
today
around
me
right
now
yeah,
so
I'm
gonna
skip
the
actual
performance
test
because
of
time,
but
I
even
look
at
this
next
screenshot.
B
You
can
see
why
we
are
using
science
near
to
service
in
this
case,
and
so
he
was
using
exactly
the
same
tool
and
two
size.
Umd
and
a
car
takes
foreign
experiment.
So
when
you
are
using
the
public
interface
and
have
a
traffic
traffic
send
over
the
public
internet,
so
this
is
a
the
panel.
Is
the
throughput
you're
getting
so
understood
shoulder
you
can?
Yes,
you
start
a
service
before
and
after
and
the
shoulder
you
can
say,
this
is
your
performance
of
our
public
internet.
B
And
then
you
can
use
sensories
to
guarantee
your
traffic
really
get
us
through
with
not
much
benefits
and
yeah.
In
some
other
demos,
people
say
you
can
you
can
actually
show
if
you're
doing
the
transfer?
At
the
same
time,
once
you
start
a
sense
service,
you
you
get
a
constant
and
finally
for
sense,
then
your
best
effort
service
is
going
to
get
squeezed.
B
So
this
is
really
very
important
because
we
are
not
only
guaranteeing
binaries
from
the
white
arrow
network.
Like
yes,
now
a
lot
a
lot
of
times,
actually
you're
going
to
be
competing
with
others
and
sometimes
intentionally,
sometimes
not
intentionally,
but
you're
going
to
be
competing
with
others
even
from
your
in
the
host.
You
really
want
to
have
some
guarantee
here,
especially
for
some
mission,
critical
services,
all
right.
B
B
Yet
so
we
simply
just
I
recognize
that
you
know
we
are
going
to
have
some
pre-config,
also
vpn
segment
already
in
the
snap.
So
what
I'm
going
to
do
is
we're
going
to
have
a
sense
to
set
up
and
now
four
near
two
tunnels.
Those
that's
four
separate
point
point:
there
are
there
two
services
just
get
a
tunnel
from
each
other
side
into
the
and
the
earth
elsewhere,
vpn
and
segment
in
my
es
net.
So
if
you
look
at
my
catalog,
you
can
say
we
already
have
four.
B
There
are
two
services
be
dynamically
set
up
by
sense
and
so
from
here
we're
going
to
create
a
new
vpn
by
dynamically
and
config
all
the
other
segment
like
on
every
side.
You
can
cut,
you
can
configure
the
three-way,
pin
routes
and
and
make
sure
you
can
rotate
across
all
of
them
and
so
for
for
since
actually
the
the
main,
the
main
work
here,
we're
going
to
base
our
our
model
and
information,
we're
going
to
say.
B
B
So
we
can
do
it
and
exact
same
thing
from
the
web
gui.
So
this
is
how
typically
like
the
and
the
application
is
going
to.
If
you
are
using
your
program
code
or
using
your
script
and
first
one.
Let
me
just
show
you
how
the
nearest
event
intent
will
look
like
it's
very
simple
right,
so
we
you
have
four
in
the
sides
you
find
here,
those
those
are
like
the
identifiers
for
your
for
your
dt
house
and
for
each
one.
We
also
with
the
identifier
of
ipv4
prefix
you
can.
B
B
So
let's
do
the
intent
for
neutral
api
across
all
four
sites
and
the
first
step
is
called
create.
Whatever
what
really
happened
here
is
and
the
service
is
going
to
be
zoomed
into
and
since
northbound
there
this
whole
intent
is
going
to
be
interpreted
and
computed
through
the
oxidation
process.
It
can
take
some
other
computation.
So
what
you
say
here
is
actually
the
model
generated
by
minus
computation
right.
So
you
start
from
very
simple
description
of
intent.
B
B
B
So,
since
has
a
transaction
business
system,
it
uses
the
two-phase
commit
so
the
first
phase.
I
will
try
and
transaction
we're
going
to
propagate
and
this
model
to
cross
the
owner
resource
managers
in
a
distributed
transaction
and
handling
right.
So
that
way
we
can
guarantee
and
if,
if
we
we,
we
say
yes,
everybody
say
yes,
then
then,
and
we
have
we
have
a
agreement
and
if
any
of
those
rms
and
does
not
agree,
then
we
can
roll
back.
So
this
is
pretty
much
a
soft
reservation.
B
So
after
this
we're
going
to
do
and
the
hard
reservation,
which
is
basically
a
commit
stage
so
next
stage
where
the
the
allocation
happens,
so
this
normally
is
going
to
be
a
a
longer
process,
because
this
this
is
the
phase
the
other
result.
Managers
will
actually
allocate
the
resources,
and
so
we
and
through
api.
B
It
takes
much
longer
time
so
you're
not
in
in
my
case,
you
may
not
want
to
read
but
wanting
to
just
file
and
then
leave
and
come
back
to
check
check
status.
So
we
also
give
you
that
not
async
kind
of
operation,
but
here
we're
just
using
synchronous
and
to
wait
for
and
finish
and
once
once
it's
finished.
We
know
the
service
is
ready
for
for
for
for
the
user
to
come
in
here.
B
So
now
we
can
see
the
services
committed,
but
we're
still
waiting
here,
because
we
don't
only
wait
for
service
being
committed
to
api,
but
also
make
sure
all
the
models
we
hear
back
from
the
from
rms
are
actually
actually
what
we
are
expecting.
So
we
are
not
right
now,
where
you
know
actually
verification
stage.
That's
wait
for
all
the
model
comes
back,
so
we
can
make
sure
the
service
actually
gets
activated.
B
So
until
we
typically
take
take
like
one
minute
or
two,
and
so-
and
let's
just
wait
for
a
little
bit
here
and
and
go
back
to
the
visualization,
it's
the
same
process.
You
can
actually
see
all
the
information
from
the
from
the
service
has
been
showing
up
here
so
that
this
military
can
start
seeing
in
this
case,
because
we're
not
actually
receiving
back
the
model
we
are
not
showing
enough
highlight
here.
Let's
go
back
here
again,
just
wait
for
it
yeah.
So
now
service
is
ready.
B
B
You
can
say
here
three
for
this
specific
dt,
node
and
three
routes
have
have
been
and
config
each
responsible
for
taking
the
traffic
towards
one
of
the
other
three
in
the
sets,
and
we
also
have
a
local-
and
this
is
this
manually
configured
previous
name.
We
have
mo
have
a
local
name
space
and
used
to
just
pretend
we
have
a.
We
have
a
routing
segment
here,
and
so
we
can
do
a
pin
test
from
our
form
nest.
Routing
from
this
name,
space
towards
all
other
routing
areas.
B
And
to
to
cern,
as
you
can
see,
for
people
who
are
not
familiar
with
the
air
three
service,
I
mean
it's
given
a
lot
of
flexibility
for
for,
for
ap
ranges
right
so
for
for
layers.
We,
you
have
to
put
every
aperture
in
the
same
broadcast
domain,
but
here
you
can
have
a
separate
address
and
prefixes.
That
makes
the
addressing
much
easier.
B
B
C
B
Right
so
and
yeah,
so
I
sort
of
put
a
little
bit
highlight
here
but,
and
since
is,
is
a
middleweight
is
a
mediation
layer,
so
basically
for
people
who
are
doing
the
service,
which
requires
to
cross
a
network
layer
right
and
typically
like
a
data
transfer
and
data
movement
and
the
data
caching,
and
so
they
can
use
sends
for
and
to
to
get
a
high
quality
and
network
services.
Dynamically.
B
Like
one
example
here,
I
just
mentioned
about
the
big
data
express,
which
is
a
data
transfer
tool
from
a
formula,
so
they
integrate
the
sense,
an
api
client
so
when
they
negotiate
data
transfer,
so
they're
going
to
and
they're
going
to
allocate
their
local
storage
and
transfer
resources
on
the
on
a
dt
nodes.
At
the
same
time,
it's
gonna
negotiate
the
white
area
and
connection
from
a
sense,
and
then
I
just
talked
about
the
pharmacist.
You
can
get
a
qrs
guaranteed
and
dedicated
mail
to
service.
B
So
so
that's
a
service
account
your
leverage
for
for
for
for
notch
transfers,
and
so
and
then
there
are
other
and
potential
use.
Cases
like
you
can
integrate
sensing
to,
and
direct
data
caching
system,
and-
and
also
I
just
heard
this
morning,
like
people
from
cms
and
they're
planning
to
also
use
sends
for
to
integrate
into
our
ftls
and
and
lucia
and
systems.
They
can
use
to
use
that
for
as
their
and
their
business
of
network
and
data
transfer
and
yes,
but
beyond
beyond.
B
Just
networking
and
and
as
we
described
since,
actually
is
a
is
a
semantic
model,
basic
system
and
you
can
actually
integrate
into
a
system
modern
network
elements
you
can
integrate
into
the
system
like
clouds
which
will
already
have
an
hp
cluster
storage
and
anything
you
want
to
automate,
and
since
it's
really
automated
our
automators,
so
basically
just
put
those
things
together
and
try
to
reason
about
how
they
should
be
connected
and
what
kind
of
services
they
can
support.
What
kind
of
capabilities
they
have.
C
C
A
B
Yes,
so
I
mean,
since
has
its
own
web
gui
you
can
actually,
you
can
actually
use
a
sense.
We
have
guitar
observation,
and
one
important
point
here
is
that
since
it's
not
actually
a
single
oscillator,
so
since
our
picture
is
and
when
we
have
our
own
twitter,
maybe
the
the
background
here
is
actually
misneeding,
but
actually
since
user
many-to-many
and
architecture,
which
means
you
can
you
have
many
operators,
I
work
with
many
resource
managers,
so
everybody
can
be
a
service
consumer.
B
Everybody
can
be
a
service
provider,
so
you
can
actually
deploy
your
own
sensor
orchestrator
and
work
with
any
resource
managers.
Where
you
have.
You
have
a
trusted
relationship
with
so
then
we
do
have
a
very
good
web
web
gui
and
you
can
use,
but
on
another
time,
if
you
want
to
develop
your
own,
I
mean
we
have
our
open
api.
So
everything
we
do
here,
we
we
use
our
own
api.
B
You
can
also
use
api
to
develop
your
own
web
gui,
but
from
this
one
we
already
have
we
solve
a
lot
of
features,
authentication
and
alteration.
You
can
use
a
cla
gun
to
get
in
you.
Can
you
can
do
raw
management
management,
and
this
is
the
role-based
access
control
system?
You
can
do
a
lot
of
different
things
for
different.
B
Yeah,
if
I
understand
question
you're
asking
for
and
by
any
researchers
you
mean
you
mean
scientists,
and
yes,
I
think,
as
I
just
said,
and
this
capabilities
eventually
can
be
used
by
the
scientists.
But
I
we're
not
expecting
a
scientist
is
going
to
use
this
directly
and
think
we
hope
they
can
work
with
with
enemy
switzer
to
to
use
this,
and
it's
not
necessarily
a
satellite
administrator,
because
for
sunday
administrators
we
can
work
with
them
to
to
make
a
initial
deployment
right.
B
You
need
that
you
have
a
software
on
your
on
your
system
and
to
make
it
work.
Those
agents
should
be
deployed
and
be
be
monitored
for
them
to
be
working
normally,
but
at
the
same
time,
actually,
I
think
it
takes
some
minimal
hand
holding
for
for
integrated
sense
into
your
workflow
into
your
existing
tools.
B
For
some
users,
you
may
want
to
come
to
a
sense,
auction
web
ui
to
do
things
directly,
which
is
also
possible.
We
will
give
you
your
and
we
can
help
you
compose
those
intents.
You
can
use
them
again
and
again
without
doing
any
modification,
and
those
are
are
certainly
possible,
but
otherwise,
once
we
have
a
sense
and
already
I
don't,
we
hope
you
can
just
use
your
existing
workflow
and
without
without
actually
packing
and
to
your
salary
illustrator.
Unless
you
have
because
troubles.
A
A
B
And
okay
yeah
thanks
alex
and
also
the
question
about
extension
to
cloud
providers
actually
and
as
a
sense
project
we
do
not
have.
We
did
not
have
the
the
responsibility
for
cloud,
but
as
a
software,
actually
we
do
support
aws,
google
and
azure,
and
from
microsoft
we
our
primary
support
is
for
aws.
We
support
direct
connect
services.
We
also
support
someone
and
unlimited
services.
B
You
can
do
vpc2
and
s3
from
from
the
amazon
cloud
using
your
own
account
and
our
support
for
google
and
microsoft
is
is
more
limited
than
aws,
but
we
do
have
some
early
work
there
as
well
and
so
yeah.
So
if
anybody
is
interested
in
that
area,
I
think
certainly
a
future
extension
where
we
will
include
your
requests.
B
All
right
right
so
and
hps
centers,
we
are
right
now.
I
don't
think
we
have
since
deployed
to
any
production
and
service.
Yet
we
have
we
had
access
since
resource
managers
as
some
of
those
sign
up
boxes
like
we
had
one
at
at
the
dtn
server
in
inverse,
but
right
now
we're
not
actually
doing
a
production
deployment
to
hvc
centers.
B
Yet,
but
our
hope
is,
we
can
start
some
some
parallel
deployment,
which
means
that
we
can
actually
put
one
experimental
box
next
to
it
is
a
production
details,
so
we
can
have
both
them
to
mount
the
same
storage
and
if
some
user
won't
have
want
to
take
this
as
a
as
a
risk
production.
We
can
use
this
and
to
try
some
some
different
network
paths
and
yeah.
We
are
right
now
we
actually
not.