►
From YouTube: 2020-02-28 - The Superfacility Project Team - The Superfacility project: 2019 year in review
Description
NERSC Data Seminars: https://github.com/NERSC/data-seminars
Title: The Superfacility project: 2019 year in review
Abstract: The Superfacility Initiative was a key component of the CS Area strategic report, and described the research and engineering required to connect experimental, networking and HPC facilities to accelerate scientific discovery. The Superfacility project was created in early 2019 in response to this Initiative. The project tracks, coordinates and communicates the work being performed across the CS Area to address the needs described in the Strategic Plan. This includes close partnership with several science teams whose needs are driving our work. In this talk, we will introduce the Superfacility concept and project structure, and the project leads will present their highlight achievements in 2019, and plans for 2020.
A
A
A
All
right
so
I
want
to
just
start
off
by
introducing
this
idea
super
facilities,
a
word
that
has
been
in
circulation
at
Berkeley
Lab
for
a
quite
a
number
of
years.
Now
this
is
a
concept
that
Cathy
Garrick
was
a
big
proponent
of
there's
a
lot
of
different
there's,
a
few
different
ways
in
which
people
use
the
term
super
facility.
So
I
want
to
go
through
a
little
bit.
A
What
we
mean
when
we
talk
about
super
facility
and
how
this
is
fed
into
the
Cs
area,
strategic
planning
around
super
facility
and
how
that
develops
into
the
super
facility
project
itself
and
then
Cory's
going
to
talk
a
little
bit
more
about
what
the
project
it's
doing.
Technically
so
and
it's
kind
of
most
basic
definition.
The
super
facility
is
a
set
of
connected
facility.
A
The
set
of
connected,
perhaps
do
a
user
facilities,
experimental
and
observational
facilities,
so
this
would
mean
an
experimental
facility,
perhaps
connected
to
es
Net
connected
to
an
HPC
facility
like
nurse,
but
the
super
facility
concept
is
really
wider
than
that.
It's
not
just
simply
sticking
these
pieces
of
equipment
together.
A
So
that's
both
within
the
networking
and
in
computing
sites,
and
then
there
is
a
component
of
computing
at
the
edge,
so
developing
the
hardware
and
the
capabilities
to
do
appropriate
computing
at
the
at
the
experimental
sites
itself.
So
this
new
facility
project
was
really
developed
in
response
to
this
initiative
to
try
to
think
about
how
are
we
going
to
implement
this
vision?
How
are
you
actually
going
to
make
this
work?
So
the
aim
of
the
super
facility
project
is
to
coordinate
the
work,
that's
being
done
within
the
CS
area.
A
To
answer
this
vision
and
to
develop
these
kinds
of
capabilities
and
a
key
part
of
this
is
that
we're
engaging
very
closely
with
seven
key
science
applications
that
are
driving
our
requirements
and
are
helping
us
to
understand
what
it
is.
They
really
need
from
from
a
super
facility
type
functionality
and
that
we
working
very
closely
with
them
to
make
sure
that
we're
able
to
develop
tools
that
they
can
actually
use
and
that
they
can
use
easily.
So
our
goal,
our
level
one
goal
is
by
the
end
of
the
project,
so
we're
envisioning.
A
This
is
a
three
year
project.
By
the
end
of
2021,
three
or
more
of
our
seven
science
engagements
will
be
able
to
demonstrate
automated
pipelines
that
can
take
data
from
a
remote
facility,
analyze
it
at
large
scale
on
HPC
resources
without
human
internet
intervention,
and
that's
a
really
key
point.
I'm.
A
lot
of
these
experimental
facilities
can
use
nurse
at
the
moment
from
use
es
net
now,
but
it
takes
a
lot
of
people
in
the
loop
to
get
things
running
and
it's
not
easy.
A
It's
not
seamless,
and
so
a
big
part
of
our
aim
here
is
to
make
this
easy
and
routine,
not
just
for
the
engagements
that
we're
all
ready
for
the
wider
community
of
deary
user
facilities.
So
this
capabilities
here
that
are
needed,
including
real-time
computing,
which
supports
dynamic
and
high-performance
networking,
both
in
the
wide
area
network
in
the
local
network
and
data
management
and
movement
tools,
an
API
driven
automation
and
authentication
using
federated
identity.
So
these
are
all
areas
that
you're
going
to
hear
about
later.
B
So
you
know,
we've
all
worked
with
technology
for
for
a
long
time,
some
of
us-
and
so
you
know,
there's
sort
of
this
classic
problem
when
you're
trying
to
do
any
sort
of
innovation,
if
you're,
interacting
with
a
user
or
someone
who
needs
something
and
you're
trying
to
find
out
what
they
need,
but
they're
also
trying
to
find
out
what
the
possibilities
even
are,
and
sometimes
there's
this
exchange
and
a
gap
that
you
can't
you
can't
get
past.
What
do
you
need?
B
But
while
what
I
wanted
to
talk
about
first
was
the
areas
of
technical
work
that
that
we
actually
arrived
at
through
our
initial
engagements
with
them.
Our
initial
conversations
with
them
to
say
like.
Where?
Are
you
heading?
What
are
the
kinds
of
work
you
do?
Where
are
some
of
the
challenges
so
that
we
could
distill
that
into
a
set
of
new
capabilities
that
we
could
build
as
generalized
sort
of
building
blocks,
new
capabilities
for
the
center
that
all
of
these
projects
could
take
advantage
of?
And
these
are
the
areas
of
technical
work
and
new
capabilities?
B
There's
they're
sort
of
grouped
in
two
and
two
by
similar
themes
there,
but
the
the
first
here
is
the
idea
of
being
able
to
schedule
real-time
computing
far
in
the
future,
not
a
job
submission,
but
fire
fire
in
the
future.
When
you
know
that
a
beam
time
is
scheduled
or
the
run
of
a
detector
is
scheduled,
so
that
that
is
something
that
we
can
accommodate
on
a
big
compute
system.
The
second
line
here,
software-defined
networks,
sense
project,
our
relation
to
that
and
more
forward-looking
self
managed
systems
are
all.
C
B
B
Within
the
center
and
in
and
out
of
the
center,
and
if
it
were
only
easier
to
visualize
what
my
data
usage
is
to
be
able
to
get
a
handle
on
how
to
manage
that,
and
if
I
had
better
tools
for
producing
data
in
the
first
place
and
a
lot
of
the
most
tedious
work
would
be
simplified.
That's
the
third
category.
B
The
fourth
is
spin,
is
a
new
service.
That's
operated
in
in
my
group,
that's
containers
as
a
service
platform,
and
this
is
meant
to
respond
to
a
need
to
science
projects
that
need
something
beyond
just
traditional
HPC,
a
persistent
service
like
a
database
or
a
workflow
engine
or
or
a
service
that
works
in
in
partnership
with
a
real-time
detector,
or
something
like
that.
B
So
we
won't
really
want
to
make
automation
kind
of
built
in
from
the
ground
up
in
these
initiatives,
and
so
that's
why
that's
why
this
is
described
as
everything
you
see
here,
automate
it
all.
So,
if
you
can
imagine
interacting
anything,
you
do
with
no
risk
doing
that
via
API,
that's
sort
of
the
long-term
vision
and
while
we're
at
it,
blending
in
use
of
modern
authentication
techniques
for
for
leveraging
federated
authentication
to
log
on
with
your
home
institution
and
use
that
identity
with
with
nurse.
B
So
this
is
the
broad
set
of
capabilities
and
we're
gonna
do
lightning
talks
that
go
into
each
each
one
of
these,
but
you
can
see
the
relationship
that
we
built
between
the
the
requirements
gathering
we
did
initially
and
translated
those
into
this
initial
set
of
taking
technical
capabilities.
If
we
deliver,
we
feel
I
can
lift
all
these
projects
up.
It's
an
ongoing
process,
though,
so
as
we
deliver
and
build
new
functionality
or
even
come
up
with
our
plans.
C
B
How
to
do
it?
We've
tried
to
have
conversations
back
to
the
science
projects
and
say
this
is
sort
of
what
we're
planning
will
this
work
free?
You
plan
to
use
what
we're
planning
to
deliver
and
if
the
answer
is
yes,
then
we
know
we're
on
the
right
track.
But
if
the
answer
is
no,
then
we
know
we
have
to
come
back
and
think
of
new
ideas
or
adjust
what
our
plans
are.
B
So
it's
this
process,
like
I,
said
at
the
beginning
of
trying
to
find
that
intersection
point
between
unmet
needs
and
things
that
could
exist
if
we
put
effort
toward
them
to
reduce
this
demonstrated
outcome.
So
so
here's
where
we
go
into
the
lightening
talk
section
and
let's
see
who
is
first,
it's
going
to
be.
You
talk
about
streaming,
it's
going
to
be
streaming.
People
in
here.
A
Standing
in
for
Roland
Thomas
he's
not
here
today,
so
one
of
the
things
that
we
want
to
emphasize
in
this
this
work.
The
computing
element
is
important
here
and
so
part
of
the
that
we're
tracking
within
the
city
project
is
work,
that's
being
done
within
the
nice,
app
programs
and
nice
app
for
anyone
in
the
room.
Who's
not
familiar
with
it.
A
It's
a
nurse
program
and
for
actually
don't
remember
what
the
acronym
is
right
now,
but
it's
application,
readiness
and
for
for
some
particular
teams
that
we're
working
with,
and
we
have
several
teams
and
that
a
part
of
the
super
facility
project
that
are
part
of
the
nice
app
program,
both
a
nice
app
for
data
program
and
also
the
nice
acta
simulations
program.
So
what
it's?
A
part
of
a
big
part
of
the
work
that
we've
done
this
year
in
the
nice
app
program
has
been
recruitment.
A
That's
part
of
the
nurse
9
project
requirements
and
it's
also
an
important
base
level
of
seeing
how
the
applications
perform
on
the
Edison
hardware
and
then
working
on
developing
the
capabilities
to
run
on
Perlmutter
when
that
arrives.
There's
also
been
significant
participation
in
the
nice
app
hackathons
by
super
facility,
science,
engagement
teams
and
so,
for
example,
the
summer
pi
team
from
ALS
participated
was
able
to
get
a
30x
speed
up
on
their
GPU
and
reconstruction,
which
is
fantastic
results.
A
We've
also
had
XFL
participation,
desi
and
LZ
participation
hackathons
last
year,
and
also
in
planning
to
come.
So
this
is
kind
of
the
elements
of
this
super
facility
project.
That's
really
helping
getting
the
code
ready
to
run
on
pel
Mattox.
That's
a
big
focus
for
us
for
the
future.
Alright,
so
the
next
slide
is
Jupiter,
Astraeus,
I
think
you're
all
gonna
take
this
one
yeah.
D
So
part
of
the
making
this
super
facility
idea
useful,
is
writing
people
with
easy
interfaces
to
be
able
to
access
their
data
to
be
able
to
run
their
analyses
and
do
it
in
a
visual
way
from
wherever
they're
sitting
and
in
talking
to
a
bunch
of
users.
I
think
we
found
out
that
Jupiter
actually
fits
a
lot
of
these
requirements
and
so
we're
engaging
with
a
couple
of
projects
to
drive
some
of
these
requirements
to
figure
out
what.
D
How
is
it
that
we
can
make
better
and
notebook
of
environments
useful
for
these
kinds
of
distributed
super
facility
projects?
So
the
first
group
that
we
engaged
with
were
the
folks
from
enzym,
and
they
already
had
some
notebooks
that
they
were
running
kind
of
on
a
single
node
basis
to
look
at
their
data
and
but
but
they
really
would
like
to
or
wanted
to
be
able
to
do
this
on
a
much
larger
data
set
running
at
nurse.
D
D
It's
we
get
was
a
significant
speed
up,
but
it
also
had
the
effect
of
being
able
to
capture
a
pattern
that
we
can
start
to
use
over
and
over
again
with
other
projects,
which
is
here's
a
notebook
based
workflow.
How
do
I
take
certain
steps
in
the
notebook
and
then
push
them
out
to
a
bunch
of
HPC
nodes
on
the
back
end,
to
be
able
to
get
scalable
workflows,
one
of
the
other.
D
Some
of
the
other
tools
that
were
useful,
I
think
that
came
up
were
this
idea
of
having
parametrize
notebooks,
where
you
can
run
a
notebook
and
essentially
do
a
perimeter
sweep
that
we
generate
notebooks
across
a
bunch
of
different
examples
and
then
the
idea
of
a
curated
notebook
environment.
So
if
you
think
of
a
project
as
taking
data
at
the
source,
they
want
to
be
able
to
take
a
canned
analysis
and
be
able
to
spin
that
up
at
nurse
and
run
that
and
a
notebook
service
is
a
good
way
of
doing
that.
D
D
So
seven
do-e
labs
lots
of
different
facilities,
all
of
the
major
kind
of
parties
that
we're
talking
to
and
various
others
in
terms
of
how
they're
using
notebooks
to
facilitate
these
kinds
of
workflows,
lots
of
good
discussion
that
came
out
of
it
lots
of
sort
of
potential
future
collaborations.
So,
overall
we've
had
you
know:
we've
had
a
lot
of
nice
little
stories
and
successes
and
we're
continuing
to
sort
of
build
out
the
suite
of
tools
and
capabilities
that
that
Jupiter
is
doing,
and
also
besides
all
this
stuff.
D
We've
also
been
you
know,
pushing
for
various
modes
of
deployment
of
Jupiter
ignore
itself,
so
that
includes
things
like
access
to
different
classes
of
nodes.
You
know
being
able
to
run
notebooks
on
your
GPUs
or
on
your
back
end
nodes
or
in
a
quarry,
login,
node
or
and
spin,
so
that
works
also
been
going
on
all
right,
I
think
that's
mostly
that
covers
Jupiter
at
least
the
highlights.
There's
lots
of
little
details
which
we
can
talk
about
later.
If
you
have
questions
hey.
C
Slurm
simulator
was
to
study
the
basic
operational
bottleneck
of
supporting
real-time
workloads.
So
if
we
allocate
a
portion
of
quarry
to
support
real-time
needs,
then
it
affects
the
scheduling
for
all
the
other
jobs
who
do
not
have
real-time
needs
and
as
it
is
people
wait
for
weeks
to
get
their
job
to
run
on
quarry.
So
the
purpose
of
slurm
simulator
was
to
see
if
you
could
somehow
simulate
quarry
and
see
how
the
waiting
times
would
be
impacted.
C
If
you
were
to
allocate
a
portion
of
the
machine
to
real-time
workloads
and
we
did
analyze
the
last
last
year
in
the
summer,
I
spent
some
time
studying
some
simulators
and
we
analyzed
two
of
them
and
some
is
a
really
monolithic
piece
of
code
and
you
have
to
run
the
slam
controller,
simulate
various
state
of
jobs
and
provide
a
dummy,
configuration
file
and
then
I
sort
of
cycles
through
those
jobs
and
tells
you
how
invent
it's,
gonna
schedule
them
and
it
takes
a
bit
of
time.
So
just
to
simulate,
let's
say
a
hundred
cores.
C
C
E
C
E
You
cause
a
time
where
there
may
be
no
usage
at
all
and
you
could
productively
use
that
time.
So
what
we
looked
at
is
a
number
of
options
for
how
we
could
get
the
best
performance
that
occurring
without
impacting
traditional
workloads,
whilst
not
delaying
real
time
and
experimental
stuff.
So
we
looked
through
a
series
of
things
and.
C
E
Aims
the
idea
that
the
best
way
to
do
this
would
have
some
form
of
a
reservation
where
a
experimental
job
can
request
that,
but
you
allow
other
workload
into
it
in
the
interim
and
then
evicted
when
the
real
workload
comes
along.
So
we
talked
to
scaredy
about
that
and
I
did.
He
did
a
lot
of
that
negotiation
and
contract
stuff
and
we
basically
asked
them
for
this
functionality
and
that
was
to
live
in
insulin,
2020.
So
I've
done
testing
already
with
the
master
branch
about
a
month
and.
E
Just
to
validate
that
what
they
were
merging
would
work-
and
it
does
I
mean
it
was.
It
was
a
very
basic
testing
because
it
was
running
all
dirty
and
not
in
the
standard
install,
but
it
was.
It
was
great
I
could
credit
reservation
as
myself
and
submit
some
jobs
as
Doug,
and
they
were
running
in
that
reservation
until
I
came
along
around
something
and
it
was
Phillies,
and
the
idea
is
that
the
group
who
needs
a
reservation
for
the
experimental
work
plate
says
I
can
tolerate
the
latency
from
my
job
sites
in
them.
E
As
long
as
the
length
of
time
of
the
job
is
less
than
the
latency,
if
this
can
tolerate
it
will
run
in
there
and
will
run
until
the
real
workload
comes
along
get
so
that's
it.
So
the
other
thing
I'll
mention
is
also
in
2002.
It's
nothing
we
funded,
but
scaling
we've
been
working
on.
This
is
there's
a
REST
API
coming
there,
which
is
great.
E
A
All
right
next
up,
it's
me
again,
so
one
of
the
key
areas
in
super
visibility
project
is
workflow
resilience,
and
this
has
a
number
of
different
components,
and
what
we've
been
focusing
on
this
year
is
helping
our
experimental
engagements
to
develop
the
appropriate
level
of
resiliency
in
their
workflows
and
us
who's
going
to
be
unavailable,
sometimes
working
really
hard
internally
to
minimize
the
impact
of
down
times
a
software
update
of
power,
outages
on
our
users
and
there's
a
lot
of
work.
That's
going
on,
especially
this
year
focused
working
facilities
around
that.
A
But
one
of
the
things
we
need
to
be
able
to
do
is
help
the
experimental
teams.
We
work
with
to
understand
what
what
could
what
would
we'll
be
up
and
down
at
nurse
and
what
they
should
do
in
order
to
to
mitigate
the
down
times
how
they
can
have
best
practices
in
workflow
resiliency.
So
we
developed
a
set
of
advice
for
this
published
a
webpage
around
it
and
there's
a
number
of
tools.
For
example,
Kelly
Rowland
develops
a
tool
to
containerize
a
condor
environment
to
allow
to
be
transferred
elsewhere,
which
is
a
neat
little
tool.
A
It's
called
CTD.
It's
on
our
nurse
github,
if
anyone's
interested
in
that
we've
also
submitted
an
a
LCC
request
and
for
time
across
the
the
oscar
computing
facilities
to
trial
cross
oscar
facility
workflows,
just
something
that
Katie
and
tipis
was
leading
so
we'd
like
to
be
able
to
demonstrate
workflows
running
on
multiple
sites
this
year.
A
H
Yeah,
so
the
API
design
aims
to
have
scientific
facilities
be
able
to
submit
workflows
and
charts
as
if
they
had,
you
know
done
in
the
past
via
SSH,
and
so
the
API
design,
part
of
it
came
from
talking
with
the
experimental
groups
and
part
of
it
came
from
looking
at
what
else
is
out
there.
So
there
are
other
projects
similar
to
super
facility
that
already
have
an
API,
and
so
we
so
what
we
have
today
is
we
have
demo
side,
that's
the
URL.
H
You
can
go
there
and
browse
the
documentation
and
try
some
of
the
functionality
its
swagger
based.
So
if
you're
familiar
with
that,
it's
it's
a
way
to
describe
an
API
and
serve
self
document,
the
code.
It
runs
on
spin,
and
our
hope
is
that
the
next
version
of
slurm,
as
chris
mentioned,
has
a
REST
API,
so
that
currently
we
submit
jobs
and
talk
to
North
Korea
as
a
sage,
but
we're
hoping
that
in
the
next
version,
will
be
able
to
do
it
through
the
REST
API.
H
H
Api
endpoints
authentication
will
give
you
a
currently
it
takes
a
username
password,
but
mark
we'll
talk
about
how
that
will
involve
in
the
future.
It
will
give
you
back
a
job
which
is
an
encrypted
piece
of
text,
accounting
interfaces
with
iris,
so
you
can
get
your
allocation
data
etc.
Health
will
show
you
the
kind
of
a
breakdown
of
system,
components
at
Northcote
and
with
their
current
status
is
and
then
jobs.
Let
you
submit
jobs.
Transfer
data
and
reservations
is
more
for
an
experimental
facility
creating
a
future
reservation
and.
H
I
I
Federated
identity
and
federated
authentication
is
a
very
fairly
ambiguous
term
and
so
defining
exactly
what
it
meant
and
what
was
important
to
the
scientific
projects
and
the
technologies
that
we
would
use
it
at
nurse
was
really
the
first
part
of
this,
and
so
we
decided
on
an
architecture
that
uses
sam'l,
which
is
kind
of
the
federated
protocol.
That's
been
in
use
and
in
academia
and
research
and
education
for
decades
now
and
we're
all
fairly
familiar
with
it's.
I
So
then,
we've
started
in
on
the
implementation,
and
the
first
step
really
is
to
have
Federation
where
Identity
and
Access
Management
components-
and
this
was
part
of
the
the
nem
replacement
project,
and
so
the
actual
banking
aspect
of
this
is
further
ahead
than
the
Identity
and
Access
Management
components.
We
had
some
NRI
for
an
open
source
project
in
order
to
make
it
more
appropriate
for
nursing,
and
so
that's
still
in
progress,
and
so
we're
phasing
in
these
kind
of
core
I
am
components
over
time
and
integrating
those
with
iris.
I
The
open
ID
connect
identity
providers
of
grand
new
technology
for
us
who,
as
I
mentioned,
only
been
sam'l
in
the
past,
and
so
we
now
have
what
I
term
a
pilot
quality,
open,
ID
connect,
identity
provider
alongside
our
sam'l
identity
provider
and
then
we're
also
working
on
an
identity
provider
proxy,
and
this
basically
is
kind
of
an
intermediate
point.
I'll
show
a
diagram
of
it
in
just
a
second,
but
it
allows
the
applications
to
connect
to
this
proxy.
The
proxy
will
then
direct
users
and
allow
them
to
like
their
home
identity
and
then
they'll
authenticate.
I
But
when
it
comes
back,
it
combines
it
aggregates
attributes
that
are
obtained
from
the
home
identity,
but
also
puts
in
nurse
specific
attributes
that
are
necessary
for
these
workflows
and
that
will
include
things
like
your
nurse
ident,
your
nurse
user
ID,
what
projects
you're
part
of
and
other
authenticate
or
authorization
decisions.
And
then
a
big
part
of
this
is
really
just
policy.
I
There's
also
which
authentication
sources
we're
willing
to
accept,
and
so
that's
kind
of
an
ongoing
discussion
right
now
and
I
think
we're
getting
closer
and
closer
to
having
a
proposal
and
then
there's
also
kind
of
I.
Don't
know
if
it's
a
Holy
Grail,
but
it's
it's
something
we're
looking
forward
to.
How
would
we
streamline
account
provisioning
for
people
that
are
users
of
experimental
observational
facilities
and
that's
well
we're
just
starting
those
discussions
here,
they're
also
a
topic
of
discussion
with
rich
Carlson's,
D
CDE
working
group.
I
So
this
is
more
or
less
how
it
works
for
federated
login
for
sam'l
or
to
get
a
Open
ID
connect
off
to
token,
so
the
user
basically
interacts
with
a
tool
like
iris
and
is
redirected
to
the
proxy,
and
then
the
proxy
allows
a
user
to
go
out
and
select
any
home
identity.
In
this
case,
this
is
the
uncommon
Federation
which
is
the
u.s.
research
and
education
Federation
and
that
we
believe
just
from
a
quick
survey
of
our
users
that
covers
about
75%
of
our
users
between
the
uncommon,
Federation
and
do-e,
one
idea.
I
Adp
that
could
serve
for
for
the
authentication,
then,
when
the
authentication
comes
back
to
the
proxy,
the
command
to
grouper
components
or
the
internal
components
that
provide
these
nursed.
Specific
authorization
account
info
and
returns
that
to
the
to
the
actual
application,
and
it's
an
opportunity
also
to
request
a
multi-factor
step.
If
all
my
Dendi
provider
didn't
include
that.
B
Okay,
so
spin,
like
I,
mentioned
earlier
in
the
overviews
of
platform
where
users
can
come
and
run
services
that
they
design
that
are
built
on
docker
containers,
and
so
we
actually
launched
the
pilot
has
been
in
the
mid
2018
and
so
over
2019.
Our
real
activity
was
just
ramping
up
the
service,
so
we
conducted
a
lot
of
workshops
through
the
year
built
the
user
base
up
to
about
a
hundred
and
thirty-five
users,
and
and
as
of
the
end
of
the
year,
there's
probably
hundreds
ervices
running.
B
It's
been
an
a
variety
of
stages
from
you
know:
prototyping
to
production
services.
So
within
the
super
sylheti
project,
specifically,
we've
had
well
we've.
We've
shown
some
interest
here
and
engage
with
some
of
the
some
of
those
projects
to
help
them
get
their
services
running
in
spann
and
I've,
just
kind
of
highlighted
here
which
of
the
projects
that
we've
that
were
partnering
with,
and
the
super
ability
project
and
sort
of
what
they're
doing
in
spin.
So
we're
happy
to
see
that
that
uptake.
B
F
So
I
want
to
talk
about
self
managed
systems,
and
really
this
is
figuring
out
which
knobs
are
on
the
system
that
impact
our
applications,
performance
and
figuring
out
how
to
tune
those
automatically
and
so
for
discovering
what
the
knobs
are
in
the
systems.
We
rely
on
domain
experts,
so
these
are
people
who
understand
the
networks
or
power
or
file
system
or
scheduling,
and
then
there's
that
second
component
of
understanding
how
it
impacts
performance.
F
F
You
don't
really
want
to
have
that,
so
the
question
we
asked
is:
can
we
predict,
given
the
counters
on
the
network
in
the
current
network
state
what
the
impact
is
on
application
performance,
and
so
we
went
through
a
prototype
of
that
and
I
guess
paper,
but
there's
many
other
ongoing
works.
There's
people
looking
at
power
and
other
things
really.
This
is
my
chance
to
advertise
the
working
group.
If
you're
interested
in
this
sort
of
discussion,
we
meet
monthly
and
there's
a
link
at
the
bottom
there
that
in
Google
Groups.
C
G
Technology
areas
to
make
network
as
a
scalable
resource,
so
one
of
the
projects
we've
been
helping
with
this
NLCS,
we
work
with
the
es
net
to
help
LCLs
to
establish
dynamic
network
pads
to
mask
so
LCLs
when
they
want
to
start
running
an
experiment.
We
can
establish
a
path
onto
our
backup
link,
yes
net
and
we
can
chat
it
down.
This
is
all
transparent
to
us
and
doesn't
involve
manual
intervention.
Another
project
we
on
board
has
Enslin.
We
actually
rolled
out
our.
C
G
Center
to
the
molecule
of
boundary
and
if
nected
them
at
400
gig
the
specifics
of
the
project
I'm
going
to
talk
about
in
a
future
slide
and
another
thing
which
we
work
with
CSC,
especially
under
the
Doug
and
James,
was
to
create
a
plugin
in
Salam
for
bandwidth,
intensive
compute
experiments
so
that
they
can
schedule
compute
nodes,
reserve,
compute
nodes
in
a
bandwidth
balanced
manner.
Basically,
what
the
plug-in
does
is.
It
looks
at
all
the
bridge
nodes,
the
health
of
the
bridge
nodes
and
maps
compute
in
an
optimal
bandwidth,
optimal
manner
across
the
bridge
nodes.
G
So
this
would
help
with
real-time
computing
experiments,
which
needs
some
bandwidth
balanced
operation.
So
these
are
a
few
of
the
areas
which
Sdn
is
looking
at
and
if
there
is
a
need
for
something
internal
in
terms
of
a
special
tunneled
path
or
something
like
that
in
internal
to
our
network,
please
let
us
know.
A
Alright,
so
next
up
incense,
so
Alex
and
shin
couldn't
be
with
us
today,
but
they
sent
me
some
notes
to
read
out
so
I
can
explain
a
little
bit
about
what
the
sensor
project
has
been
doing.
So
sense
stands
for
software-defined
networking
for
end-to-end
network
science
at
the
exascale,
so
they've
been
working
very
closely
with
LCLs
it's
what
ashley
was
mentioning
just
earlier.
A
They
want
to
build
an
intent,
driven,
multi
domain
orchestration
framework
to
allow
distributed
science
workflows
to
provision
and
manage
the
full
end-to-end
network
services
so,
and
that
includes
on-site
and
slight
edge
data
caching.
So
a
lot
of
the
work
this
year
was
around
working
with
LCLs
to
demonstrate
these
capabilities
for
an
XML
CP
workflow,
and
they
also
had
a
demonstration
at
SD
19
which
used
dynamic,
I'm
reading
this,
because
this
is
not
my
area
of
expertise,
dynamic
attachments
of
n
site
resources,
2
layer,
3,
virtual
private
network
connections
advertised
by
es
net.
A
A
A
sense
of
components
was
driving
the
connections
there,
so
plans
in
2020
involve
working
closely
with
the
nurse
group
and
the
SDN
group
in
the
the
last
markets,
though
called
last
mile
problems
and
making
sure
that
everything
connects
up
all
the
way
from
the
edge
of
nurse
all
the
way
into
the
compute
node,
so
that
the
path
from
the
DTN
into
a
compute
node
is
as
performance
in
perhaps
as
controllable
as
the
path
from
the
experiments
to
nurse.
This
is
one
of
the
things
that
they
weren't.
A
Looking
at
another
thing
that
actually
has
a
nice
cross-pollination
with
other
work.
That's
going
on
in
the
super
facility
project
is
they're
also
looking
into
federated
ID,
so
integrating
prototype
elements
of
federated
ID
with
in
sense
to
allow
this
kind
of
capability
for
authentication
in
the
network
as
well
as
in
the
computing.
J
The
globe
was
sharing,
endpoint
was
deployed
and
the
ALS
team
used
this
for
together
four-day
run
where
they
were
sharing
seven
terabytes
of
data
and
they
sort
of
automatically
set
up
sharing
endpoints
to
share
with
users
from
UCLA
and
Boulder,
using
this
framework
that
Bjorn
Enders
and
his
students-
and
this
is
done
with
the
ALS
scientists
with
minimal
people
in
the
loop.
They
just
use
the
interface
to
create
this
and
they
were
able
to
share
this
data
out.
J
We've
also
deployed
command
line
data
transfer
tools
that
use
Globus
on
the
back
end,
so
these
allow
for
large-scale
data
transfers
to
be
integrated
more
easily
with
workflow
tools.
If
you
have
a
bunch
of
data,
you
need
to
move
into
our
out
of
the
center
or
between
the
layers
of
the
file
system,
and
you
want
to
do
it
programmatically.
You
can
invoke
this
command-line
tool
instead
of
going
to
the
Globus,
UI
and
dragging
and
dropping,
and
these
are
going
to
be
the
building
blocks
for
the
interface
into
the
super
facility.
C
J
J
Members
in
the
storage
group
I
see
Greg
here
to
deploy
the
prototype,
GPFS
HP,
SS
integration,
and
what
this
does
is
it
for
the
user
teams,
a
file
system,
a
very
file
system
like
interface
for
HP
SS,
you
put
the
files
on
GPFS
they're
automatically
moved
into
HP
SS
in
a
way
that
is
sensible
for
tape,
storage
and
so
we're
looking
at.
This
is
one
of
the
ways
to
take
a
lot
of
the
heavy
lifting
of
putting
a
copy
of
your
data
into
the
archive
and
doing
it
properly.
K
Dashboard,
which
is
something
that
we
put
up
together
for
nerd
users
as
something
completely
outside
of
the
super
facility
project
initially,
because
it
was
just
something
people
wanted
to
be
able
to
do.
It
gives
people
the
ability
to
kind
of
manage
their
data
use
using
a
graphical
web-based
interface,
with
just
a
little
more
simple
to
use,
then
some
of
the
other
tools
we
may
have,
and
especially
more
performant,
compared
to
like
going
in
and.
K
Side
is
an
example
of
our
PI
interface,
so
this
is
a
way
that
pis
will
be
able
to
go
in
and
browse
through
the
files
that
they
have
in
a
specific
directory
like
a
project
directory
level,
and
then
they
can
drill
down
through
there
up
and
down
to
like
things
operations
on
the
things
they
selected.
So
it's
a
pretty
simple
interface
at
this
point
and
we're
hoping
to
soon
be
expanding
that
into
things
weren't
really
to
actual
moving
at
the
data
leveraging.
K
The
stuff
you
saw
on
the
previous
slide
and
this
example
on
the
right
is
just
another
piece
of
the
data
dashboard
that
will
also
probably
find
its
way
into
the
Sutro
facility
API
interface,
and
this
is
a
way
for
anyone,
not
necessarily
just
API,
to
go
in
and
be
able
to
see
what
files
you
have
and
visualize
them.
And
you
see
the
structure,
the
the
directory
structure
has
and
then
there's
a
place.
K
L
I'll
get
some
in
advance
the
slide.
Oh
there
yeah
sorry
hdf5
is
a
popular
I/o
middleware
package
that
many
applications,
both
at
nurse,
can
broadly
within
the
dealy
and
internationally,
of
course,
used
and
as
part
of
the
super
facility
focused
efforts.
We're
trying
to
do
two
different
sides
of
the
equation
here:
the
quick
same
coin:
almost
we're
developing
a
virtual
object
layer
connector
for
hdf5
that
will
open
up
xtc2
files
to
be
read
within
and
analyzed
within
the
hdf5
ecosystem.
L
L
In
a
you
know,
this
is
the
core
of
the
super
facility
effort
right.
How
can
we
stream
our
data
from
a
remote,
since
you
know
experimental
site
into
a
supercomputer,
and
so
as
part
of
hdf5
that
we've
implemented
another
connector?
Allow
you
to
plug
in
that
again,
transparently
to
the
application,
will
just
take
hdf5
files
they're
being
written
locally
at
SLAC
and
stream
them
over
the
network
and
write
them
also
at
nurse.
So
this
is
a
mirror.
A
M
Sure,
thanks
Debbie,
so
Debbie
asked
me
to
add
a
couple
of
slides
on
some
of
the
work
we've
been
doing
in
collaboration
with
nurse
and
I.
Just
have
two
slides
on
this
I'm
gonna
be
really
quick,
because
I
do
realize,
there's
more
slides
and
less
time
at
this
point,
so
we've
been
working
with
LCLs
workflows,
trying
to
analyze
what
they
have
running
on
nurse,
but
also
trying
to
get
some
of
workflows
up
and
running.
M
The
focus
in
this
particular
project
is
looking
at
sort
of
data
patterns
and
policies,
so
we're
using
darshan
to
analyze
the
I/o
patterns
and
identify
the
different
patterns
of
computational
and
storage
use
and
trying
to
understand
what
these
workflows
would
need
on
earth
systems
going
forward.
We've
been
also
looking
at
the
batch
job
log
qs
q,
information
to
analyze,
sort
of
what
their
workflows
are.
M
M
Here
we,
what
we're
doing
in
the
with
this
is
really
using
darshan
logs,
where
available,
which
is
only
a
small
percentage
to
reconstruct
workflows.
So
we
can
actually
start
viewing
the
world
more
as
workflows
rather
than
jobs,
which
will
allow
us
to
do
some
optimizations
when
it
comes
to
IO
Network
things
like
that,
and
so
we've
been
doing
some
analysis.
There.
M
We've
also
started
thinking
about
implicit
workflows
where
we
look
at
the
job
logs
with
time,
windows
and
sort
of,
say,
hey
what
did
a
user
run
in
a
certain
period
of
time
and
how
did
their
workflows
perform
in
that
so
I
presented
most
of
these
at
the
December
meeting.
We
have
some
more
analysis
since
then,
but
this
allows
us
to
look
at
these
logs.
Understand
sort
of
failure.
N
N
N
A
N
A
That's
a
great
point,
so
the
timeline,
the
timeline
of
the
project
is
I
was
always
designed
to
be
a
three
year
project
so
by
the
end
of
the
three
years.
So
by
27
two
years
from
now,
we
intend
to
have
everything
all
the
capabilities
that
we've
been
developing
well
out
of
demo
mode
and
into
production.
But
a
lot
of
those
capabilities
are
going
to
be
in
production.
Well
before
that,
so
we
have.
We
were
a
real
project.
We
have
milestones.
A
We
have
timelines,
we
have
Gantt
charts,
so
we
have
a
time
frame
for
which
various
different
tools
are
going
to
be
ready
and
production
ready.
A
lot
of
the
capabilities
that
we're
thinking
that
we've
been
talking
about
now
and
we're
hoping
to
have
in
production
in
the
next
six
months
and
so
I
don't
have
the
exact
dates
for
every
tool
that
we're
talking
about
two
hands.
I
A
So
the
question
point
from
Casey
is
explaining
what
we
expect
the
application
teams
to
do
you
compared
to
what
we
expect.
What
nurse
is
going
to
provide
so
desk
is
providing
the
capabilities,
the
tools,
the
infrastructure
of
the
the
nurse
centric
tools.
So
a
good
example
of
that
is
is
the
data
movement
tools,
so
developing
tools
to
automate
data
movement
between
the
nurse
file
systems,
but
how
an
application
or
an
it
science
team
can
use
that
tool.
It's
that's
up
to
them.
N
A
Right,
weird
effect,
with
we're
aiming
to
develop
tools
that
are
going
to
be
sustainable.
These
aren't
sort
of
one-off
demonstrations
that
we
won't
be
maintaining
that
we
won't
be
continuing
on
so
yeah.
The
choice
is
always,
of
course,
up
to
the
science
team:
it's
whether
they
the
tools
and
meet
the
needs
that
they
have,
but
we're
aiming
for
this
to
be
sustainable
tools.
An
ask
is
going
to
be
supporting.
A
All
right,
well,
we've
managed
to
leave
absolutely
no
time
at
all
for
the
science
highlights.
So
what
I
will
say
is
that
we
have
our
first
annual
meeting
and
so
in
a
sense
you
can
think
of
all
the
work
that
you've
heard
about
today
is
kind
of
being
a
trailer
for
our
first
annual
meeting
day
is
gonna,
be
on
March
31st,
we're
hosting
the
first
super
facility
projects
annual
meeting,
and
there
will
be
talks
from
a
whole
range
of
people
from
our
science
partners.
A
We
have
a
panel
discussion
of
facility
directors,
so
talking
about
the
the
importance
of
HPC
and
networking
and
experimental
science
and
we'll
have
a
lots
of
demos
and
interactive
sessions.
So
there's
particular
tools
that
you've
heard
about
today
that
you're
interested
in
seeing
how
this
worked.
You
can
come
along
and
play
around
with
what
that
looks
like
and
have
demonstrations
from
the
nurse
staff,
and
also
this
is
will
be
containing
inspectors
from
outside
of
do-e.
A
So
we
have
people
coming
from
outside
of
the
DA
complex
to
talk
a
little
bit
about
how
they
see
the
connection
between
HPC
and
experimental
science.
So
everyone
who
worked
is
welcome
the
website,
which
is
a
little
perhaps
hard
for
you
to
pass
here,
but
the
website
is,
is
up
and
registration
is
open.
So
please
everyone
come
along
if
you're
interested
right,
thanks
to
all
our
speakers
for
doing
such
a
fantastic
job.