►
Description
Date: 3/1/2019
Presenter: Ray Idaszak
Institution: RENCI
South Big Data Hub
B
And
that
is
a
testament
to
the
you
know,
the
leadership
that
has
put
that
forward
that
it's
having
such
good
adoption
in
the
hydrologic
science
community
and
as
value
and
starting
to
have
adoption
in
other
domains.
So
this
is
what
I'll
be
talking
about
and
what
we're
going
through
and
kind
of
the
current
status
and
some
of
the
challenges
that
we
face
in
what
are
some
of
the
differences
that
we've
had
to
put
into
the
architecture
between
hybrid
share
and
this
with
this
new
version
that
we're
calling
common
share.
B
Okay,
so
for
hydrogen
share,
when
when
David
harborton
was
doing
the
introductions,
he
introduced
himself
as
the
he
is.
The
overall
I'm,
the
Hydra
sure
project
he's
on
the
line
here
with
us,
so
it's
structured
as
an
NSF
collaborative
board,
so
I'm,
the
I'm
one
of
two
collaborative
P
is
for
the
current
Hydra
share
award,
show
and
weighing
it
and
to
say
is
the
other
collaborative
di
and
you
know
getting
on
April,
6th
and
2018
and
I.
This
is
an
actual
link,
but
there's
a
YouTube
video
I
wanted.
A
B
So
that
was
the
some
audio,
but
that's
a
link
to
to
the
YouTube
video
of
Dave's
presentation,
and
so
so
Hydra
share
is
cyber
infrastructure
that
you
know
the
software
to
service
data
service
model
to
service
and
I'm
gonna
go
through
some
more
this
in
a
positions,
its
datasets
and
models,
as
social
objects
have
been,
can
be
comment
on
ok.
So
this
is
the
usage.
B
As
of
the
middle
of
last
year,
where
we
had
twenty
six
hundred
users,
you
can
kind
of
see
the
breakdown
of
the
resources
and
the
type
of
audience
that
we
have
and
they,
as
of
today,
I
understand
that
it's
over
three
thousand
users.
So
it
has
three
thousand
registered
users.
So
it's
in
it's
in
production,
and
it's
hosted
here
at
Red
Sea,
which
is
an
Institute
at
the
University
of
North,
Carolina,
Chapel,
Hill,
and
so
guys,
ma'am.
B
You
know
again
it's
a
baseline,
getting
these
slides
from
from
what
David
presented
and
often
presents
as
the
overall
pie.
But
hydrologic
is
a
team
sport,
so
you
can
kind
of
think
of
a
lot
of
domains,
including
biomedical
science
and
informatics
that
require
collaboration.
So
you
know,
hybrid
share
address
is
a
grand
challenge,
better
hydrologic
forecasting,
and
you
can
map
that
to
where
genomics
there
there
are
similar
Grand
Challenges.
Okay,
so
can
we
apply
this?
You
know
this.
This
value
is
the
value
provided
in
Hydra
sure
to
other
domains.
B
So
when
we
look
at
how
people
share
the
internet
now
YouTube
Facebook
Instagram
Dropbox
so
forth,
but
is
there
such
a
thing
for
hydrologic
data?
So
that's.
You
know.
That's,
of
course,
how
we
position.
Hydra
share
data
and
models
used
by
hydrated,
hydrolyzed
neurologists
are
diverse.
These
are
examples
of
some
specific
types
of
data
that
are
supported
by
Hydra
share
and
we
also
support
in
Hydra
share
generic.
A
B
B
So
it's
a
platform
for
sharing
hydrologic
resources
in
collaboration
again,
some
facets
that
would
be
nicely
applicable
to
other
domains.
People
are
very
familiar
with
Dropbox
kind
of
capability,
so
you
know
we
want
to
make
these
kinds
of
interfaces
as
intuitive
and
understandable
for
the
end-user
to
to
use
and
I'm
gonna
watch
my
time
here.
B
So
here
is
this
a
story
here
for
how
Hydra
sure
is
used
occult
by
Steven
Holl
story,
so
he
collects
estate
in
the
field.
You
can
kind
of
think
of
a
hydrologist
is
doing
bad.
You
know
again
similar.
Can
a
researcher
collect
data
off
of
a
sequencer,
for
example,
I'm,
just
kind
of
drawing
parallels
between
Hydra
share
and
other
communities
posit
the
data
in
the
Hydra
sure
repository
verify
the
data
and
metadata
correct,
but
keep
the
data
private.
It's
not
public.
B
Yet
so
the
hydrogen
air
supports
different
access
controls
for
private
discoverable
in
public
discoverable,
meaning
that
the
metadata
only
is
discoverable,
but
the
data
is
not
downloaded,
then
submit
the
paper
for
publication
at
that
point,
the
paper
that
the
data
in
Hydra
sure
can
receive
a
DUI
and
then
publish
when
the
paper
itself
receives
the
eyes
like
I
just
mentioned.
So
can
the
data
in
the
hydrogen
infrastructure
receive
a
DUI?
So
here
are
some
things
that
Hydra
sure
can
do
and
I'm
going
to
ask
you
to
sort
of.
B
You
know
take
a
mental
note
of
this,
because
what
we
did
in
the
subsequent
work
is
to
sort
of
try
to
look
at
this
as
a
set.
So
you
don't
have
these
sharing
data
models,
access
controls
strikes
us
visualize
use.
Web
services,
API
published
with
the
DOI,
discover
these
web
apps.
This
notion
of
an
app
that
can
act
on
the
resource
within
within
the
Hydra
sure.
B
Environment
and
then
so,
the
Hydra
share
summary
of
what
it
can
do.
Is
it's
a
web-based
environment
as
it
supports
the
multiple
hydrologic
data
types
using
standards,
flexible
discovery,
model
sharing,
meaning
hydrologic
models,
and
these
models
can
be
executable
or
they
can
be
discoverable
and
as
well
as
discoverable,
facilitate
ease
of
access
to
use
of
high-performance
computing,
social
meaning
collaboration
and
links
to
other
data
models.
Enable
more
rapid
advances
through
collaborative
data
sharing
Hydra
share
is
sustained
by
the
consortium
of
universities
for
Ben's
hydrologic
science
incorporated,
which
is
over
a
hundred
member
University
Consortium.
B
B
That
starts
to
get
into
how
we
started
to
look
at
generalizing,
Hydra
sure
to
comment
share
and
it
and
what
we're
looking
here
is
it
kind
of
take
the
capabilities
that
have
been
so
well
received
used
by
the
hydrologic
science
community
and
see
if
those
can
be
generalized
and
abstracted
so
that
they're
not
hydrology
specific
but
can
be
applicable
to
other
domains
like
biomedical
science?
And
you
know,
and
the
mapping
obviously
has
turned
out
quite
well
for
us-
we've
been
doing
this
for
you
know
over
a
year
with,
with
with
promising
results.
B
So
it's
you
know.
Our
current
environment
has
early
users
early.
You
know
friendly
users.
We
taste
that
we're
learning
from
we
don't
consider
it
a
production
environment
but
but
we're
learning
as
we
go
and
the
hydrosphere
community.
You
know
the
Hydra
shirt
code
base
is
not
insignificant,
it's
over
a
half
million
lines
of
code
and
so
we're
able
and
and
it's
and
you
know-
and
it's
not
funded
code.
B
It's
very
nice
that
it's
berkeley,
bsd
three
claws
slice
open
source
license,
so
that
allows
us
to
pick
it
up
with
continuing
attribution
to
the
Hydra
sharing
quasi
team.
So
here's
a
slide
from
crease
castillo
who's
on
the
line
as
well.
That's
our
key
a
provided,
the
architecture
for
the
common
sure
platform,
and
here
we
start
to
see
some
of
the
differences
that
we
had
to
address
for
the
biomedical
science
and
informatics
community.
B
There
was
additional
requirements
for
intermediate
identifier.
Some
of
the
search
capabilities
had
to
be
augmented
with
ontology
based
search,
which
we've
done.
We
definitely
definitely
had
to
a
very
significantly
address
sensitive
data.
So
obviously,
if
we're
going
to
put
pH
higher
or
sensitive
health
data
into
the
system
that
has
to
be
treated,
it's
not
necessary.
It
doesn't
necessarily
have
a
parallel
in
the
hydrologic
science
community.
B
So
here's
another
slide
that
has
the
architecture
that
Clarice
did
for
the
platform
in
more
detail.
I'm
not
I'm,
going
to
leave
this
as
an
exercise
to
the
user,
but
it
shows
that
there
there's
quite
a
bit
of
capabilities,
and
you
know
an
additional
needs
with
respect
to
things
like
federated
authentication,
because
if
we
attend
to
cater
one
system,
that
is
the
common
share
would
be
great
if
we
could
also
have
that
as
an
authentication
to
other
systems
that
may
be
hosting
sensitive
data
without
having
to
have
everything
some
of
the
words
there.
B
Some
of
the
requirements
there's
a
lot
of
requirements
for
auditing
and
keeping
logs,
because
those
are
requirements
that
have
to
do
with
governmental
requirements
like
FISMA
when
an
institution
is
hosting
government
data
and
so
forth
and
okay.
So
here's
the
the
kind
of
the
high-level
overview
of
what
we've
done
is
we've
taken
hydrology
and
we've
started
with
common
share,
but
we're
also
saying
that
the
common
share
with
the
biomedical
sciences
that
shouldn't
be
limited
to
that
and
whether
you
know
common
share
being
for
a
NH
funded
data.
Commons
has
its
name
derived
in
that.
B
But
it
could
just
as
easily
be
something
else.
You
know
share
that,
then
it
has
to
do
with
the
get
it
in
a
different
community.
It's
really
important
to
stress
that
when
you're
dealing
with
at
any
given
time,
you
know
8
to
10
organizations
distributed
around
the
United
States,
actively
contributing
code
with
production
releases.
Everyone.
You
know,
spanning
between
every
three
to
five
three
to
six
weeks,
that
software
engineering,
proper
software
engineering
is
really
important.
B
Industry
derive,
but
not
necessarily
industry
exact,
because
it
does
have
to
adapt
academia
with
code
reviews
in
absolute
versioning.
You
know
so
on
and
so
forth.
Continuous
integration,
continuous
deployment
that
becomes
really
important
and
also
a
necessary
precursor
to
doing
something
like
taking
a
codebase
and
deploying
it
for
a
different
community.
B
So
we're
right
we're
right
at
15,
minutes
and
I
know
this
goes
till
4:00.
So
here's
some
some
links
to
learn
more,
which
have
put
in
the
talks
and
and
quite
a
few
people
spanning
both
the
Hydra
share
and
comment
share,
works
efforts
that
contribute
to
this.
So
I'll
stop
there
and
ask
if
you
know
I
can
take
a
little
bit
more
time
to
do
just
some
quick
demonstration
of
the
capabilities
or
I
can
stop
now
and
answer
questions,
because
I'm
ready,
15
minutes.
A
Well,
why
don't
we
just
pause,
real,
quick
and
see
if
there
are
questions,
and
so
thanks
for
that
presentation
there
and
I
was
just
going
to
comment,
haven't
seen
it
in
the
chat
you
feel
like
just
typing
in
the
chat
window.
Rather
than
speaking,
we
can
take
questions
that
way
and
pause
for
a
second,
and
if
not,
we
can
dive
into
a
demo.
B
Some
stuff
in
action:
okay,
so
here
I've
created
some
files.
I've
got
some
just
artificial
public
data's
in
the
artificial
private
data.
So
here's
you
know
first
is
hard
to
hide
your
share
and
again
there's
a
link
in
the
slides
that
will
allow
you
to
see
if
they
have
carpet
in
stock
on
Hydra
proper
that
was
presented
on
April
6th
of
2018.
B
So
I
refer
you
to
that,
but
you'll
notice
that
you
know
the
discover.
The
collaborate
up
here
on
the
top
of
Hydra
share
you
know
starts
is
effectively
the
you
know
very
similar
tabs
here,
but
you
know
coupling
new
ones,
like
concept
search
the
ability
to
searchable
text
cover
here.
One
thing
I
wanted
to
mention
that
we
had
to
do
different,
so
here
we
have
type.
This
is
again
one
of
the
things
that
we
do
differently
is
that
we
have
an
authentication
scheme
that
uses
your
post
institutions
so
I've
been
here.
B
Native
credentials,
so
in
our
case,
they're
called
onions
and
it
probably
goes
over
there
and
pumps
for
that
case.
So
I'm
already
authenticated
here
in
comment,
share
I'll,
open
up
a
new
window,
and
when
we
say
you
know,
we
want
this
interface
to
be
as
easy
to
use
or
familiar
with
something
like
Dropbox.
The
drag-and-drop
part
is
similar,
but
we
don't.
We
do
some
things
different,
that
we've
directly
from
this.
So
we'll
put
like
my
public
hurricane.
B
A
B
And
at
this
point
it
says:
congratulations,
you
can
make
it
public
and
sweep
over
here
now.
It
allows
us
to
do
that.
We
make
the
user
accept
an
agreement
and
there
it
can
be
now
it's
public,
which
means
that
it's
also
discoverable
and
I
think
I've
been
Harvey
before
bed
and
the
thing
is
I
have
two
resources
there.
One
is
that
I
used
as
a
test
earlier
and
then
the
one
I
just
created
just
just
now,
I.
B
Click
this
we
have
the
ability
to
a
couple
options
here
is
we
can
have
different
users?
So
if
I
wanted
to
make
our
here,
somebody
who
can
view
the
data
or
edit
on
it,
we
can
do
that.
That's
I'll
Drive
from
from
Hydra
sure,
and
then
we
can
do
to
see
gene
so
like
we
can
add
whole
groups
of
people
that
can
also
had
add.
You
know
at
the
whole
group
can
yes
associated
with
this
level
of
access
control.
B
It's
you
eyes
and
minutes,
because
there
is
much
more
of
a
need
for
persistent,
published
intermediate
data
which
the
arcs
and
the
minutes
are
good
for
so
we've
we've
done
that
so
I'll
show
that
real,
quick,
another
Amen
to
accept
and
I'll
do
publish
with
a
minute
and
you'll
see
this
thing
down
here.
This
the
grid
to
32
by
quit
is
going
to
change
to
an
arc
here
and
if
we
click
that,
although
it's
one
of
those
timing,
things
I
just
needed
to
give
it
a
little
bit
more
time
to
get
through
the
system.
A
A
A
B
So
with
hydrogen,
so
two
comments
there
with
hybrid
share.
Everything
is
always
associated
with
the
32
by
quid
from
the
onset,
and
it
remains
mutable
with
that
32
by
quid
to
be
to
be
able
to
change
it
until
it's
published
with
a
dua
and
that
at
that
point
it
becomes
immutable.
Okay,
that's
in
hydra,
share
okay
and
comments.
Here
we
have
the
addition
of
when
I
published
here
so
like
I
do
the
I'll
do
the
difference,
because
I
have
two
different
ones.
This
one
has
not
yet
been
minted
a
minute.
B
It's
still
got
the
32
by
quid
and
notice
that
I
can
you
know
very
readily
edit
it
you
know
change.
You
know,
add
more
your
david,
whereas
with
the
version
that
has
the
minute
which
we
can
tell
by
here
that
is
published,
it's
going
to
start
blacking,
feasibilities,
there's
no
pencil
icon
I
can
make
a
copy
and
the
new
version
of
it,
but
I
can
no
longer
edit.
It
becomes
because
it's
mu
with
that
with
that
minute
now
so.
B
Okay,
so
alright,
so
let's
see
how
we
doing
yeah
continue
to
ask
questions.
Cuz
I'll,
just
do
one
more
part
of
the
demo,
which
is
the
sensitive
data
which
is
very
different
from
Haider
share
and
show
you
some
of
the
capabilities
that
were
recently
implemented
so
to
create
new
and
I'll
put
here.
Artificial
sense.
B
B
Okay,
so
here
it
says
the
research
can
be
made
public
right,
so
at
this
point
I
can
I
could
make
the
data
public
just
like
that.
Unfortunately,
it'll
block
me
seeing
like,
can
I
really
make
this
sensitive
data
public
turns
out
I
can't
I'm
not
going
to
agree
to
that,
but
note
that
here
I
can
still
download
it
right.
I
can
download
it
here
and
it'll.
You
know
download
to
my
desktop
and
for
sensitive
data.
We
needed
to
preclude
that,
so
it
has
been
implemented.
Recently.
B
Copy
or
create
new
versions
is
blocked,
so
those
are
the
kinds
of
things
that
are
very
different
than
then
then
hydrosphere,
and
we
can
also
require
things
like
a
data
use
agreement,
there's
that
in
in
Hydra
tree
there's
a
very
similar
capability
to
do
that.
One
of
the
things
that
we
did
differently
also
is
we
overloaded
or
added
additional
capability
to
the
where
we
have
the
groups
where
we
can
add
whole
groups
with
two
access
controls,
but
a
lot
of
the
way
we
position.
B
The
groups
in
common
share
are
as
as
whitelist
like,
who
is
authorized
to
have
access
to
a
particular
data
set,
and
so
we
can,
you
know
in
groups
can
also
be
private,
and
so
we
Hydra
share
has
no
human-in-the-loop
to
request
access.
You
know
to
create
like
how
you
can
just
sort
of
click
through
and
in
your
account
is
created
with
common
tier.
B
Even
if
some
were
on
some
paperwork
or
some
policy
in
another
organization,
they
already
have
that
access,
but
this
gives
us
the
provenance
train
for
the
audit
trail
that
keeps
track
of
these
things
that
we
can.
You
know,
always
show
that
the
user
didn't
create.
You
know
that
user
agreed
to
those
to
that
data
use
agreement,
in
that
we
have
a
system
that
required
that
and
then
they're
given
access
to
this
data
and
then
some
other
things.
Let's
see
where
we
have
four
minutes,
there's
some
other
things
that
we
really
concentrate
under.
B
There
are
things
that
are
in
Hydra,
sure
they're,
like
works-in-progress
of
being
able
to
run
Jupiter,
notebooks
and
things
on
HBC
resources.
We
have
a
little
bit
more
flexibility
in
common
sure
because
it's
not
a
production
system,
so
we
can
try
a
lot
of
things
so
there's
the
ability
to
take
the
work
of
scientists
which
clarice
on
the
phone
crease,
because
the
is
the
rent
CPI
up
and
we've
integrated
side
s
here
within
common
term
called
pivot,
so
that
it
allows
us
to
to
run
those
things
cloud
agnostic
lis.
B
B
About
how
much
work
that
took
to
get
that
Bart,
we
also
have
the
ability
to,
like
instance,
a
an
individual
ec2
instance
per
user
in
Amazon,
which
we
can
use
to
host
a
variety
of
software
that
is
useful
to
that
community.
So
for
them,
for
example
like
viewing
DICOM
images
or
di't
that
come
image
format
which,
in
the
case
of
the
users
of
this
system,
represents
COPD
gene
patient
data,
which
is
basically
has
to
do
with
image
imaging
the
lungs
all
right.
So
we're
two
minutes
to
go.
A
That's
good
I
think
we
have
a
couple
little
things
just
to
announce
the
end
here,
but
I
want
to
thank
you
for
a
good
presentation
and
just
make
sure
there
aren't
any
other
questions
or
other
things
before
we
move
on
I
a
quick
question.
First
of
all,
thanks
very
much
that
was
a
great
demo
and
again
sort
of
I
think
gives
us
the
opportunity
to
see
what's
out
there.
The
question
I
have
is
Hydra
share
as
far
as
sort
of
its
its
overall
functionality.
B
It
is
great
for
the
sustainability
model,
so
you
know
that
a
lot
of
what
is
being
presented
here
is
that
hydro
chair
is
so
well
thought
out
that
it
just
you
know
very
naturally
those
capabilities
lend
themselves
and
are
really
well
positioned
to
see
other
domains
and
aren't
necessarily
hydrology
specific.
So.