►
From YouTube: 2021-10-19 - DeRose - Scaling out HPC with On-Premise Performance in the Oracle Cloud Infrastructure
Description
NERSC Data Seminars Series: https://github.com/NERSC/data-seminars
Speaker:
Luiz DeRose, Ph.D., Oracle
Title:
Scaling Out HPC with On-Premise Performance in the Oracle Cloud Infrastructure
Abstract:
The continuous increase in complexity and scale of high-end systems, together with the evolving diversity of processor options, are forcing computational scientists to face system characteristics that can significantly impact the performance and scalability of applications. HPC users need a system infrastructure that can adapt to their workload needs, rather than having to constantly redesign their applications to adapt to new systems. In this talk, I will discuss the current trends in computer architecture and the implications in the development of HPC applications and programming and middleware environments. I will present the Oracle Cloud Infrastructure (OCI), which provides availability, resiliency, and performance at scale, so HPC users can easily choose the best option for their workloads, and will discuss hybrid on-prem/cloud options, which facilitate workload migration from on-premise to the cloud. I will finish the presentation with a discussion of some of the challenges and open research problems that still need to be addressed in this area.
Bio:
Dr. Luiz DeRose is a Director of Cloud Engineering for HPC at Oracle. Before joining Oracle, he was a Sr. Science Manager at AWS, and a Senior Principal Engineer and the Programming Environments Director at Cray. Dr. DeRose has a Ph.D. in Computer Science from the University of Illinois at Urbana-Champaign. He has more than 25 years of high-performance computing experience and a deep knowledge of programming and middleware environments for HPC. Dr. DeRose has eight patents and has published more than 50 peer-review articles in scientific journals, conferences, and book chapters, primarily on the topics of compilers and tools for high performance computing.
Host of Seminar:
Jonathan Skone, Ph.D.
Data & Analytics Services Group
National Energy Research Scientific Computing Center (NERSC)
Lawrence Berkeley National Laboratory
A
So
that's.
He
brings
a
particularly
interesting
perspective
to
the
cloud
space
having
this
hpc
experience
and
so
today
he's
going
to
provide
an
overview
of
the
oracle
cloud
infrastructure
and
discuss
the
design
features
for
hpc
that
are
incorporated
into
oracle's
cloud
infrastructure
that
make
the
workloads
particularly
attractive
for
the
hpc
user
base.
With
that
I
hand
over
the
virtual
floor
to
you,
louise.
A
A
B
I
was
talking
to
jonathan
just
before
I've
been
involved
with
nursed
for
many
many
years.
In
fact,
even
before
gray,
so
nurse
users
have
used
the
stuff
that
by
being
involved
since
seaborg,
because
I
was
at
ibm
for
a
while
and
and
we
had
in
fact,
a
tool
that
I
developed
of
the
hpm
toolkit
that
was
available
on
that
platform.
B
So
most
of
you
probably
don't
even
use
that
platform,
but
I've
been
involved
in
that
and
then
yes,
16
years
at
cray,
have
been
involved
with
with
many
of
the
the
nurse
systems
right.
So
yes,
today,
the
plan
is
to
talk
about
kind
of
a
risk
retrospective
on
what's
happening
on
what
in
the
industry
and
and
also
talk
about,
partly
why
I
moved
to
the
cloud
and
why
I
believe
the
clouds
is
is
really
the
the
future
of
hpc
cloud.
You
have
a
huge
component
of
that
on
that
right.
A
B
Please
I
I
don't
know:
what's
the
process
on
the
the
seminar
series
normally
on
presentation,
I
I
don't
mind
if
people
interrupt
me
with
questions
or
stop
me
with
questions
on
virto,
I
don't
know
how
easy
that
this
is
but
feel
free
to
if
there
is
a
way,
feel
free
to
interrupt
me
for
questions
at
any
time.
Otherwise
we
will
have
some
time
for
questions
so
about
in
in
around
2010.
B
Horowitz
and
team
came
up
with
these
data,
which
is
quite
interesting
right.
So
basically
it
showed
that
in
around
2005,
when
the
the
multi-core
era
started,
so
this
kind
of
a
inflection
point
on
a
lot
of
these
curves
right,
the
curves
for
transistors,
the
curves
for
the
because
for
number
of
cores
the
power
and
what
we
see
is
basically,
as
I
said
in
2005,
the
the
power
consumption
on
processors
got
to
a
point
that
people
could
not
increase
the
frequency
anymore.
B
B
But
what
we
saw
is
that
the
the
the
power
curve
got
to
a
point
that
we
could
not
continue
raising
the
the
frequency
and
that's
when
multi-core
started
right,
so
basically
2005
at
the
time
at
craig.
We're
dealing
with
amd's-
and
I
I
believe
was
barcelona-
was
the
first
force
of
the
multi-cores
and,
as
you
see
the
trend,
there
is
that
the
number
of
cars
started
to
increase
and
every
other
curve,
with
the
exception
of
transistors
kind
of
flattened
out
right.
B
The
single
thread
performance
frequency
power
more
recently
in
2017
a
sorry.
So
these
trends,
as
I
said,
but
basically
there
were
like
moore's
law
talking
about
the
increase.
There
was
also
dennard's
law
that
says
that
the
performance
per
watch
will
double
every
two
years,
but
again
due
to
the
physics
that
couldn't
happen
and
the.
B
B
B
So
if
I
was
developing
a
programming
environment-
and
let's
say
I
was
developing
compilers
pretty
straightforward
and
basically
you
got
the
current
architecture
and
you
optimize
for
that
architecture
and
come
up
the
next
one.
X86
was
dominating
dominating
at
the
time.
So
you
didn't
have
a
lot
of
stuff
to
worry
about
other
than
getting
the
best
performance
out
of
that
particular
architecture.
B
B
Because
we
talk
about
arm
but
armies
design,
the
the
architecture,
but
then
the
the
providers
build
on
top
of
that.
So,
even
if
you
are
developing
forearm,
you
have
different
ways
of
of
optimizing
depending
on
if
the
fujitsu
it's
an
ampere,
it's
a
marvel
and
so
forth
right.
So
so
this
this
huge
diversity,
it's
quite
interesting
and-
and
I
spend
quite
a
bit
of
time,
also
in
academia
right.
B
B
If
you
are
developing
system
software,
it's
really
really
hard
right,
because
now
you,
instead
of
doing
just
for
one
architecture,
you
have
to
think
about
this.
This
diversity,
if
you
are,
in
fact
I
I
put
myself
on
if
I
put
the
hat
of,
if
I'm
buying
a
system,
I
can
see
that's
also
very
hard
right,
because
now
you
need
to
map
the
the
workloads
for
for
your
center
to
the
best
architectures
that
that
are
available
and
I'll
talk.
B
B
B
I
believe
one
of
the
first
meetings
that
I
had
about
that
was
08
or
09
already,
and
the
the
main
concern
was:
oh,
we
needed
gpus,
but
it
has
to
be
usable
right,
and
so,
with
the
the
I
mean,
the
first
problem
is
again
the
the
the
diversity
right.
So
a
cpu
core
is
is
kind
of
heavy
weight,
while
the
the
gpu
core
is
lightweight.
B
B
Creation
or
or
appearance
of
different
programming
models,
like
one
of
the
things
that
nvidia
did
quite
well,
was
cuda,
but
cuda
was
high
low
level.
So
then
there
were
new
programming
models
that
were
developed,
one
of
them
being
high
level
ways
of
programming
the
gpus,
the
problem:
that's
it
took
too
much
too
long
to
be
adopted,
so
we
are
not
having
two
of
them
right.
So
there
was
like
open
acc.
B
We
were
able
to
kind
of
put
all
of
this
together,
try
to
make
this
happen
on
openmp,
but
took
too
long,
so
open
acc
showed
up
and
later
on,
openmp
target
directives.
So
just
these
different
sets
of
programming
models
created
yet
another
level
of
complexity
right
and
then
adding
to
that
they.
You
still
want
to
do
hybrid
programming
with
cpus
and
gpus,
and-
and
there
were
situations
where,
even
at
clay
we
had
libraries
that
would
explore.
B
B
Many
many
codes-
and
even
I
mean
if
you
write,
if
you
look
at
the
right
sides,
the
one
of
the
top
codes,
the
the
community
earth
system
model-
is
already
a
couple
model
right,
so
even
on
that
code
is
really
three
different
things
right.
There
is
atmospheric.
There
is
land
there
is
ocean,
so
you
see
that
the
diversity
of
codes
makes
it
very
hard
to
decide
on
which,
which
processor
to
to
to
have
on
your
main
system
right.
A
B
Doing
even
your
mpi
right
so
or
how
do
you
do
mpi,
plus
a
threaded
model
like
openmp,
and
how
do
you
take
advantage
of
of
vectorization
or
accelerators
right
gpus?
So
all
of
these
just
added
are
different
layers
of
this
whole
to
the
complexity
right.
The
other
thing
that
happened
in
the
industry
and
at
the
nurse
with
quarry,
you
had
knl
that,
for
example,
have
two
levels
of.
B
Right,
so
how
do
you
optimize
for
that?
How
do
you
take
advantage
of
deciding
what
to
put
on
hbm
or
what
to
put
on
on
regular
memory,
so
a
lot
of
the
users
of
of
know
and
that
not
using
the
hbm
as
a
memory,
but
just
as
a
yep
another
layer
of
cache?
But
if
you
want
to
use
as
a
memory,
then
again
more
complexity
right.
B
So,
as
I
said,
the
the
getting
a
system
or
for
a
diverse
world
load
is
really
really
hard
and-
and
those
are
the
three
layers
that
I
can
think
on
or
where,
where
it's
it's,
let's
call
three
pillars
right.
So
so
the
the
main
one
of
the
complexity
is
capacity
planning,
so
one
can,
when
one
is
going
to
to
decide
on
a
system.
B
If
you
go
to
the
maximum
capacity,
then
what
you
see
is
probably,
at
least
in
the
beginning,
a
lot
of
under-ritualized,
which
basically
you
lose
on
return
of
investment,
because
you're
paying
quite
a
bit
to
to
get
that.
So
it's
interesting
that
if
you
hear
people
saying
oh,
my
my
system
is
close
to
100
utilization.
B
B
I
I
was
at
ibm
at
the
time
and
used
to
interact
quite
a
bit
of
the
folks
at
ncsa,
and
one
of
them
once
told
me
that
I
was
wasting
my
time
doing
performance
tools,
development,
because
people
would
invest
much
much
much
more
in
trying
to
figure
out
how
to
cheat
the
queue
and
get
the
job
to
run
twice
as
to
trying
to
optimize
the
code,
because
you
would
gain
much
more
by
being
able
to
do
that
right.
Of
course,
he
was
joking,
but
it's
it's
something
that
I
always
remember
like
yeah.
B
If,
if
you
are
waiting
on
the
kill,
it's
just
research
or
work,
that's
not
being
done
right.
So
the
the
other
big
point
is,
is
the
cost,
and
one
thing
that
I
will
talk
next
is
in
fact
more
on.
The
cost
is
what
we
call
the
cost
of
lost
performance
right
so
and
what
what
happened
is
normally
on-premise,
what
the
systems
are
three
to
five
years
and
the
technology
changes
very
very
rapidly
right.
B
So-
and
I
have
I
slide-
that
I
will
talk
more
about
that
and
then
the
other
pillar
is
the
the
it's
hard
to
optimize
operations
right.
So
you
need
to
basically
take
care
of
of
things
like
security,
which
always
a
pain
and
you
you
need
to
take
care
of
of
the
optimization
or
the
the
how
to
get
the
best
return
of
investment,
which
is
again
it's
their
conflict
interest
there,
because
the
best
return
of
investment
may
not
get
the
best
productivity
and
so
forth.
B
So
if,
if
we
do
a
similar
analogy
on
the
right
side,
you
see
the
number
of
production
jobs
per
year
on
the
oracle
clouds.
And
if
I
go
back
to
looking
into
2016-
and
you
can
say:
okay
16,
we
had
broadwell
17,
pascal
and
then
sky
lake
in
18,
with
volta
19.
We
had
naples
20,
we
had
rome
and
umpir
in
vision,
pier
and
and
nice
lake.
B
B
So
that's
that's
really
something
to
to
pay
attention
so
that
comes
into
okay.
Based
on
all
of
these
hpc
in
the
clouds
seems
to
be
a
very
appealing
idea,
right
and-
and
the
view
is
basically,
the
users
really
need
an
infrastructure
that
can
adapt
to
their
workloads,
as
opposed
to
the
users
of
developers,
have
to
adapt
to
each
new
architecture
every
time
that
the
new
architecture
shows
up.
B
The
clouds
is
based
a
lot
or
has
have
has
been
based.
A
lot
on
virtualization
and
virtualization
brings
things
like
jitter
and
a
lot
of
overheads,
which
is
not
necessarily
an
issue
for
for
commercial
computing,
but
for
scientific
computing
is,
is,
is
a
big
problem
right.
The
other
thing
is
specialized.
Networks
were
not
available,
so
all
of
these
were
kind
of
impactful
to
try
to
run
hpc
applications
on
on
the
cloud
more
recently.
B
This
is
changing
and
I
will
talk
why,
but
things
that
we
we
see
is
cae
and
bio.
Science
have
have
been
the
early
adopters,
but
the
the
cloud
market
is
growing
quite
a
bit
right
so
and
hyperion
research,
which
is
one
of
the
top
areas
or
companies
that
provide
informational
kind
of
future.
Looking
their
prediction
is
that
the
hpc
markets
in
2024,
so
three
years,
will
be
around
46
billion
dollars.
B
So
that's
quite
a
bit.
Another
researching
report
is
for
intersex
360
and
that's
a
little
older,
but
basically
saying
that
the
cloud
is
the
fastest
growing
segment
of
the
hpc
markets.
Right
and,
and
then
hyperion
was
saying.
That
was
on
the
the
publication
at
isc
this
year
that
the
cloud
revenue
is
projected
to
reach
close
to
nine
billion
dollars
in
2024,
and
that
reflects
a
compound
annual
growth
rates
of
17.6.
B
B
But
then,
whether
geoscience
academia
are
showing
in
fact
the
highest
growth
over
days
up
to
2024
right
so
overall,
if
I
add
academia
and
government
labs
and
weather
and
defense,
so
the
public
sector
is
supposed
to
grow
aggregates,
like
20
year-over-year
right.
So
that's
that's
quite
a
bit
of
a
growth
and
quite
interesting
growth
right.
So.
B
And
then
why
hpc
is
on
the
cloud
is
unappealing
is
better
than
on
premise
right,
so
one
I
I
mean
I
thought
the
cost
of
lost
performance
right.
So
one
of
the
the
great
things
about
the
clouds
is
that
you
have
the
latest
generations
all
the
time
right.
So
basically,
with
that
one
can
eliminate
the
cost
of
lost
performance,
you
don't
need
to
do
refresh
cycles.
B
B
B
You
get
out
of
the
the
data
center
business
right,
so
you
don't
need
to
to
worry
about
electric
bills.
You
don't
need
to
worry
about.
I
I
mean
throughout
my
career,
ibm
and
then
cray
how
there
were
many
times
that
a
site
had
to
wait
for,
because
the
data
center
wasn't
ready
because
the
building
wasn't
ready
and
and
the.
In
fact,
this
is
interesting
because
the
the
site,
with
more
experience
when
they
are
planning
an
upgrade
or
a
new
system,
sometimes
the
first
thing
that
they
would
walk
on
was
the
building
right.
B
So
so
I
remember
one
of
the
sites
that
that
I
worked
with
that
basically
got
the
system
and
the
system
basically
well.
They
had
like
14
cabinets
and
they
had
to
only
put
three
operating
because
they
didn't
have
room
for
the
other
11.
They
had
to
wait
for
a
whole
year
which,
like,
if
you
go
back,
remember
the
cost
of
last
performance.
A
A
B
Clouds
and
basically
a
little
bit
of
oracle
backlog
right.
So
it's
interesting
that
a
lot
of
people
think
about
oracle
as
a
database
company,
but
oracle
has
been
involved
with
hpc
for
quite
some
time
right,
so
article
bought
back
a
while
ago
long
time
ago
bought
some
in
2010
and
with
that
also
bought
kind
of
steak
on
on
melanox.
B
On
the
other
hand,
as
I
mentioned-
not
I
didn't
mention-
I
was
talking
to
jonathan
earlier-
so
one
of
the
things
that
oracle
did
was
to
bring
a
lot
of
experience
to
the
team
right
people
with
experience
on
the
cloud
with
experience
on
hpc,
and
the
idea
was
to
look
into
how
do
we,
what
is
working?
What's
not
working
and
how
we
can
make
things
better.
B
So
so
the
article
clouds
we
do
have
our
focus
is
really
on
bare
metal.
We
do
have
vms
because
we
also
provide
support
other
communities,
but
for
hpc
the
focus
is
really
on
getting
bare
metal,
which
provides
a
much
better
price
performance
and
provides
in
fact
similar.
I
will
have
a
slide
that
I
will
talk
about
that,
but
provide
similar
performance
to
on-prem
environments
or
even
better,
depending
on
how
you're,
comparing
oracle
is
really
started
as
a
database
company.
B
A
B
B
You
want
the
low
latency
and
the
the
high
bandwidth,
and
you
want
these
predictability
right.
So
we
have
cpu
servers
and
gpu
servers
and
the
excel
data.
That's
basically
oracle
prime
product
all
connected
to
the
to
the
network,
and
we
use
these.
What's
called
the
rocky
v2.
So
basically
is
rdma
over
converted
internet
as
our
network
base.
B
B
B
B
B
What
what
they
do
now
is
plan
for
the
average
or
even
lower
utilization,
and
then
you
do
cloud
bursts.
So
you
are
connected
to
the
cloud
and
when
you
need
more,
so
you
just
burst
into
the
cloud
so
at
oracle.
We
have
this
thing
that
you
call
the
off-scaling
that
allows
you
basically
to
go
from
on-prem
directly
to
the
to
the
cloud
whenever
you
need
and
that's
one
way
that
you
can
address
the
the
weighting
right
that
you
would
have
otherwise
on
the
old
prem.