►
From YouTube: Programming platforms: Kokkos, Raja and OpenACC
Description
Rahul Gayatri (LBNL), Sunita Chandrasekaran (University of Delaware) and David Alexander Beckingsale (LLNL) present a panel discussion on Programming platforms: Kokkos, Raja and OpenACC. Recorded live via Zoom at GPUs for Science 2020. https://www.nersc.gov/users/training/gpus-for-science/gpus-for-science-2020/ Panel Chair: Dossay Oryspayev
A
A
We'll
have
two
parts
main
two
parts
first
part
will
be
presentations.
Many
short
presentations
of
about
five
to
seven
minutes
by
each
of
our
panelists.
Then
we'll
have
about
slightly
more
than
30
minutes
for
discussion
and
q
and
a
session
and
then
all
of
the
audience.
I
encourage
to
actively
ask
your
questions
and
participate
in
the
discussions.
A
You
can
do
this
either
using
the
q
and
a
box
or
you
can
raise
your
hand,
and
I
will
unmute
your
mic
so
that
you
can
directly
ask
question
to
one
of
our
panelists
of
or
all
of
them.
Okay,
that
being
said,
I
would
like
to
first
introduce
rahul
kumar
gayatri.
He
is
a
application
performance
specialist
in
the
apg
group
at
nurse
where
he
works
mainly
on
helping
application
teams
optimize
their
codes
for
the
next
generation
architectures.
A
A
B
Earlier
requested,
okay,
so
today
I'll
be
talking
about
openmp
and
openmp4
gpus
in
the
context
of
this
workshop
right
and
the
coco's
programming
model.
So
let
me
start
by
brief
introduction
about
what
I
wanted
to
talk
about.
Openmp,
I'm,
assuming
that
most
of
us
are
pretty
familiar
with
openmp
as
a
framework
and
in.
If
you
want
to
know
more
about
openmp4
gpus.
I
would
encourage
you
to
check
out
chris
daley's
talk
from
yesterday's
performance
and
put
productivity
and
portability
a
panel.
B
I
know
the
title
says
performance
portability,
and
I
don't
want
to
get
into
that
discussion
because,
as
we
saw
in
yesterday's
panel,
it
has
like
multiple
viewpoints
and
there
is
no
single
accepted
definition
of
what
performance
portability
is
of
yet,
but
at
least
you
can
be
sure
that
the
there
will
be
some
sort
of
portability
when
you
use
these
openmp
for
offload
directives
across
these
multiple
architectures,
how
performant
they'll
be
and
how
close
to
the
peak
performance
of
the
architecture
they
will
reach.
That's
yet
to
be
seen.
B
They
have
open
forums
where
you
can
submit
these
bugs
and
most
of
the
times.
It
is
generally
something
wrong
that
the
programmer
is
doing
rather
than
the
compiler.
But
if
there
is
a
case
when
there
is
a
genuine
compiler
bug,
they
are
very
diligent
in
in
fixing
that
and
releasing
it
with
the
next
compiler
release.
B
So,
in
that
sense,
it's
a
very
active
community
in
terms
of
users
and
in
terms
of
compiler
developers,
some
of
the
advantages
and
disadvantages
of
using
openmp
offload
directors
is
that
first
thing
is,
as
we
all
know,
is
open.
Av
is
relatively
easy
to
use.
It's
not
too
invasive
into
your
code,
like
you
can
just
annotate
blocks
of
code,
saying
this
has
to
this
loop.
B
For
you
in
that
sense,
cocos
is
a
much
more
advanced
framework
where
it
allows
you
to
to
have
final
grain
control
over
the
code,
as
well
as
doing
a
lot
of
these
performance
tricks
in
in
in
the
back
end
by
itself.
So
for
those
of
you
who
do
not
know
what
cocos
is,
it
is
a
c
plus
plus
based
programming
model
for
writing
portable
app
performance
for
double
applications.
B
It
allows
you
to
expose
abstract
hierarchies
of
parallelism
in
your
code
and
then
it
is
the
job
of
the
framework
to
map
these
hierarchies
onto
whatever
target
architecture.
You
are
running
and
it
it
provides
a
portability,
some
sort
of
portability
across
all
major
hpc
platforms
and
the
way
it
does.
This
is
by
supporting
different
backends.
So
the
backends
that
are
already
available
are
the
serial
backend,
which
is
basically
sequential
code.
The
p
thread
back
end
the
openmp
3.0
back-end
cuda
back-end
coda
uem
back-end.
B
The
fact
that
there
is
already
a
open,
mp3
and
p
p
thread
back-end
implies
that
the
cocos
code
would
already
run
on
almost
most
of
the
available
cpus
for
hpc
that
are
there
and
the
cuda
and
cuda
uvm
back
ends
implies
that
you
can.
You
can
run
the
cocos
code
on
all
the
nvidia
gpus
that
are
available
right
and
then,
apart
from
these
backends,
there's
an
active
development
of
openmp
target
backend.
B
B
How
many
one
minute?
Oh
I'm,
okay,
okay,
and
then
it
lets
you
do
this
1d
or
multi-dimensional.
You
know
loop
computations,
where
it
lets
you
tile
the
loops
and
it
divides
the
available
parallelism
into
three
different
hierarchies
which
in
cuda
you
can
simulate
you
can
analogous,
which
is
analogous
to
like
teams
that
unlock
this
to
thread,
blocks
threads
to
thread
x,
dot
y
and
vector
to
thread
x,
dot
x
and
allows
you
to
do
atomic
operations.
B
This
is
a
simple
example
of
of
where
of
how
a
simple,
c,
plus
plus
code
can
be
written
in
cocos.
On
the
left
side,
you
can
see
the
c
plus
plus
code,
where
we
have
an
array,
integer
array
of
ten
elements
and
it's
being
updated
inside
of
for
loop
and
on
the
cocoa
side.
You
can
actually
choose
your
execution
space.
You
know
the
coco's
default
execution.
B
A
Thank
you
rahul,
and
our
next
talk
will
be
given
by
sunita
chandrasekharan.
She
is
an
assistant
professor
with
the
department
of
computer
and
information
sciences
at
the
university
of
delaware.
She
received
her
phd
in
2012
on
tools
and
algorithms
for
high-level
algorithm
mapping
to
fpgas
from
the
school
of
computer
science
and
engineering
ntu
singapore
rich
her
research
spans
high
performance
computing,
parallel
programming,
benchmarking
and
data
science.
Applications
of
interest
include
scientific
domains
such
as
plasma
physics,
biophysics,
solar
physics
and
bioinformatics
sunita.
A
C
Okay,
thank
you,
everybody
for
joining,
and
you
know
being
able
to
do
this
online,
so
yeah.
So
following
up
with
rahul's
previous,
you
know
talk
on
the
openmp
and
cocos.
This
is
about
open
acc,
where
the
idea
is
again.
It's
a
directive
based
programming
model,
one
of
the
two
directive-based
drawing
model,
the
other
one
being
openmp
and
just
skimming
through
some
of
the
ongoing
things
that
we
have
been
up
to
with
open
acc.
C
C
There
is
definitely
many
more
things
to
do
with
respect
to
supporting
other
features
in
these
languages,
so
this
is
pretty
much
a
start.
Then
we
also
improved
multi-device
support
through
direct
memory,
copies
and
synchronization
as
another
added
feature.
Prior
to
this,
there
was,
if
in
case,
if
you
wanted
to
copy
data
from
one
gpu
to
another,
we
had
to
copy
to
cpu
and
then
to
gpu.
So
by
enabling
this
improved
multi-device
support,
we
won't
need
to
synchronize
back
and
forth
with
the
cpu.
We
won't
need
to
block
the
cpu
twice
in
case.
C
I
also
wanted
to
step
through
some
of
the
activities
with
respect
to
using
openacc
on
scientific
applications.
Some
of
these
data
are
not
2020
are
not
super
current
but,
for
example,
the
18
of
insight
applications
at
summit
are
from
data
is,
from
november
2019
the
top
five
hpc
intersect.
360
research
is
a
couple
of
years
old
number,
but
gaussian
vasp
ansys
fluent
are
some
of
those
hpc
top
five
top
three
applications
platform
supported.
C
That
is
a
list
there,
but
I
would
also
you
know,
plus
one
to
rahul's
previous
comment
where
you
know
we
could
get
into
the
performance
portability
aspect
and
talk
about
it
for
several
hours
and
we
will
never
come
to
a
consensus,
but
so
that
those
are
a
list
of
targets
that
is
that
currently
open
acc
supports
and
open
acc
applications.
As
you
can
see,
the
trend
has
grown
from
you
know
about
30
applications.
C
C
We
are
also
running
an
openacc
slack
channel,
which
has
grown
over
the
past
couple
of
years,
quite
a
bit,
especially
those
participating
in
gpu
hackathons.
We
invite
them
to
the
slack
channel
where
they
have
basically
used
as
a
stack
overflow
if
you
like,
and
we
have
been
debating
between
you,
know,
keeping
slack
and
moving
to
stack
overflow,
but
bottom
line
is
trying
to
answer
questions
as
of
when
the
users
have
them
and
an
easy
mode
of
communication
to
you
know,
get
them
up
to
speed.
C
Ever
since
this
was
released
and
made
you
know
available,
you
can
see
the
total
number
of
downloads
have
steadily
increased
and
I
myself
use
them
as
part
of
my
teaching.
You
know
parallel
computing
or
computer
architecture
courses,
just
like
you
would
use
say.
Gcc
openmp,
for
example,
for
you
know,
parallel
computing
classes.
C
This
is
an
ongoing
effort,
a
bit
more
on
the
gpu
hackathons.
There
are
a
couple
of
varieties
that
any
of
you
could
participate,
boot
camps
and
hackathons.
One
is
a
couple
of
days
event,
the
other
one
is
a
five-day
event
and-
and
you
basically
bring
your
code,
you
bring
your
team,
there
are
mentors
assigned
to
your
team,
it's
two
mentors
per
team
and
just
sit
down,
and
you
know
and
hack
code
and
get
it
to
working
on
the
systems
which
the
particular
hackathon
host
supports
or
whatever.
C
So
this
has,
you
know,
been
very
successful
and
you
can
see
the
plots
as
to
how
the
number
of
participants
number
of
courts
have
been
steadily
increasing.
I
myself
did
one
of
them
in
ud,
back
in
2016,
had
six
teams
and
definitely
the
codes
took
off,
and
it
was
a
nice
way
of
exposing
domain
scientists
to
the
different
ways
you
can
program
either.
You
know
accelerate
and
parallelize
their
code
on
large-scale
systems
and
thanks
to
julia,
we
got
this.
C
I
got
the
slide
from
her
just
literally
just
yesterday,
where
we
have
the
bunch
of
hackathons
coming
up.
So
take
a
look
at
the
gpu
hackathons.org
and
you
would
find
you
know
a
range
of
applications
and
it's
not
just
open
acc.
It's
cuda
open,
acc,
openmp
cocos.
There
are
applications,
there
are
even
sometimes
python.
C
So
if
you're
interested
you
have
a
code
you're
looking
for
help
and
mentorship
to
move
your
code
to
large
scale
systems,
you
have
more
than
one
place
to
go
to
to
participate,
and
several
of
these
hackathons
are
going
on
virtual
right
now,
and
I
hear
it
is
pretty
successful.
So
do
make
use
of
you
know:
gpu
porting
and
the
last
one
I
wanted
to
draw
your
attention
to
is
an
open,
acc
teaching
kit
which
people
like
me.
C
You
know
who
teach
and
I'm
sure
there
are
several
of
you
on
the
call
who
probably
educate
and
teach
you
know
next
generation
workforce,
there's
a
bunch
of
teaching
materials
that
I
work
with
nvidia
to
put
together
and
there
is
darkerization
as
well
available.
Google
slides
available
and
some
codes
for
lab
exercises,
and
these
are
some
of
the
modules
there's
room
for
improvement.
But
there's
something
for
you
to.
You
know
start
off
and
get
your
hands
dirty.
A
Thank
you
sunita,
and
thank
you
for
staying
within
the
time.
Okay,
our
next
programming
model
talk
will
be
given
by
david
alexander
beckinsale.
He
is
a
computer
scientist
in
the
center
for
applied
scientific
computing
at
lawrence,
livermore
national
laboratory.
His
work
focuses
on
programming
abstractions
and
he
is
a
project
lead
for
empire
and
chai,
and
the
core
raja
team
member
david
received
his
phd
computer
science
from
the
university
of
warwick
uk
in
june
2015.
A
A
D
D
So
in
terms
of
back
ends,
we've
made
a
real
good
progress,
pretty
much
supporting
all
the
current
platforms
as
well
as
being
well
underway
to
supporting
the
machines.
You'll
be
expected
to
run
on
that
are
coming
up
soon,
so
we
have
regular
sequential
loops
some
simile
stuff
openmp
support
both
for
the
cpu
and
for
target
offload
with
openmp
4.5.
D
So
the
initial
goal
was
to
get
these
big
old
codes
running
efficiently
on
sierra
and
at
the
time
this
porting
effort
was
started.
You
know
a
complete
cuda
rewrite
was
never
going
to
fly
because
it's
not
portable
and
when
you
have
a
million
lines
of
code
you
just
can't
afford
to
maintain
multiple
versions
and
when
the,
when
these
efforts
were
started,
openmp
offload
wasn't
really
viable,
and
I
think
this
was
mentioned
before,
but
the
you're
really
heavily
dependent
on
compiler
support.
D
If
you
want
to
use
something
like
openmp,
and
so
that's
you
know,
wasn't
a
route
that
our
application
customers
really
wanted
to
go
down
and
then,
if
you
look
out
kind
of
into
the
future
platforms
that
are
coming
down,
the
pipeline
are
based
on
gpus
from
different
vendors,
so
really
having
some
abstraction
that
insulates
you
from
this
is
changing.
Technology
is
critical.
D
D
You
turn
that
into
a
lambda
expression,
and
these
the
numbers
here
in
the
bottom
right
are
just
comparing
the
the
speed
ups
of
this
this
loop.
You
can
write
it
obviously
like
natively
in
sequential
openmp
or
cuda,
or
use
the
raja
version,
where
the
only
thing
that
changes
is
the
template
parameter,
and
you
can
see
there
then
you're
taking
the
time
of
the
native
implementation
over
the
time
of
the
roger
we're
pretty
close
to
to
parity
across
those
back
ends.
D
In
the
same
way
you
have
the
kernel
function,
that's
templated
on
an
execution
policy,
but
instead
of
a
single
iteration
space
and
a
single
lambda
function,
you
can
actually
pass
in
an
arbitrary
number
of
iteration
spaces
and
an
arbitrary
number
of
lambda
functions,
and
then
the
execution
policy
describes
and
how
those
are
iterated
over
and
the
kind
of
the
order
that
you
want
things
done.
So
this.
E
D
You
know
in
the
previous
code
examples
they
just
determine
where
that
loop
is
going
to
run
and
the
data
accessibility
is
the
responsibility
of
the
programmer
and
again
this
goes
back
to
kind
of
the
initial
work
working
with
the
the
code
teams
at
livermore,
where
they
already
had
code
to
manage
their
data
in
the
way
they
wanted
to
right
and
we
weren't
going
to
come
in
and
tell
them
that
they
have
to
to
move
everything
to
some
special.
You
know
raja
managed
data
type.
D
So
the
the
two
projects
that
that
I
work
on
that
are
in
this
space
is
chai,
which
provides
an
array-like
object
that
coordinates
with
raja,
so
that
the
data
moves
back
and
forth
to
the
cpu
and
gpu
implicitly,
depending
on
where
your
raja
loop
is
going
to
run,
and
then
we
also
have
the
umpire
project,
which
is
a
portable
api
for
accessing
different
types
of
memory.
So
you
know
umpire
is
to
kind
of
memory
resources.
D
So
really
what
raja
has
given
us
in
this?
You
know
having
a
an
abstraction
on
top
of
these
various
other
programming
model
technologies
is
to
be
able
to
write
code.
That's
high
performance
on
sierra,
but
that
we
know
is
going
to
be
portable
to
future
platforms
right,
we're
insulating
all
the
the
application
developers,
the
computational
physicists
from
all
this
kind
of
underlying
churn
on
on
programming
models
right.
You
know
where
you've
got
cuda
and
hip
and
sickle.
D
That's
really
not
something
that
you
want.
The
application
developers
to
have
to
deal
with
right
and
one
of
the
things
that
we've
really
focused
on
is
making
it
easy
to
do
the
most
common
use
cases.
So
incrementally
adopting
this,
you
know
one
loop
at
a
time
as
you
gradually
port.
Your
application
is
easy
and
was
a
critical
feature
in
moving
applications
right
onto
sierra,
but
through
some
of
our
apis.
D
So
I
know
historically,
people
have
viewed
the
raja
project
as
as
fairly
livermore
focus,
but
we
really
are
making
an
effort
to
collaborate
and
continue
to
to
develop
in
the
open
and
to
onboard
more
external
users.
So
you
know
I
just
closed
with
saying
that
we
we
really
welcome
users
and
collaborations,
so
we
hope
to
hear
from
from
some
of
you
thanks
very
much.
A
Thank
you,
david
and
I'd
like
to
thank
all
of
our
panelists
on
behalf
of
everyone
for
presenting
this
very
nice
overviews
of
the
different
programming
models
and
joining
being
able
to
join
us
during
this
uneasy
times,
and
with
that
I'd
like
to
start
taking
questions
from
the
attendees
or
any
other
questions
feel
free.
I
mean
the
panelists.
You
can
also
ask,
please
feel
free
to
discuss
some
topics
with
other
panelists
as
well.
A
A
Do
we
have
any
questions?
Okay,
so
looks
like
we
have
a
bit
of
the
first
question:
okay,
so
the
first
question
is
coming
from
one
of
our
organizers
mwazawan.
He
is
asking
a
question
in
this
soup
of
languages,
where
each
claims
of
ease
of
use
and
portability,
if
someone
who
is
new
to
gpu
porting,
might
get
overwhelmed.
A
C
Sunita
here
chiming
in
plus
one
to
what
david
just
said
coming
from
a
standpoint,
you
know
teaching
literally
teaching
students
in
class
about
gpu
reporting.
I
think
my
another
question
would
the
person
who
is
who
is
posing
this
question
is:
what
is
the
person's
background
right?
Are
we
talking
about
somebody
who
is
a
beginner
to
gp
reporting
or
somebody
who's
in
an
advanced
stage
or
an
intermediate
stage?
That
will
also
depend
to
where
you
want
to
start
for
the
to
the
students.
C
I
would.
I
would
rather
throw
a
directive
than
cuda,
for
example,
and
I
can
see
them
starting
to
use
gpus
at
the
end
of
three
hour,
three
three
seven
three
months
of
a
semester,
and
they
did
they
did
projects.
We
did
a
very
basic
open,
mpr
flooring
and
they
did
projects
on
open,
acc
gpu
and
they
were
happy
to
have
used
gpus.
So
I
think
it
would
also
depend
on
you
know
what
kind
of
background
the
person
is
coming
from
in
order
to
use
a
particular
framework
to
begin
gpu
protein.
B
A
B
Hi
this
is
rahul.
I
I
agree
with
what
sunita
and
david
said.
That
first
thing
is:
what
is
the
base
language
that
your
code
is
in,
and
second
thing
is:
how
much
time
do
you
have
to
spend
on
that
like
cocos
and
raja?
Both
are
a
bit
more
intensive
in
the
sense
that
you
will
have
to
spend
a
slightly
longer
time
to
just
use.
B
Just
get
your
code
ready
to
start
using
these
frameworks,
then
something
like
open,
ec
or
openmp,
where
they're
a
bit
more
easier,
initially,
but
then
again,
cocos
and
raja
support,
more
back-ends
than
openvcc
or
opening
people.
So
so
that's
that's
the
choice
like
how
much
time
do
you
have
to
work
on
this.
A
A
D
D
B
Okay,
apart
from
the
obvious
yeah
fact
that
openmp
was
like
until
a
couple
of
years,
back
openmp
was
more
sort
of
concentrated
in
in
on
the
cpu
side,
sorry
and
open
ecc
more
on
the
gpu
side,
but
now,
with
the
with
the
rapid
development
of
these
target
directives
by
different
compiler
vendors
for
different
hardware,
I
would
say
like
at
least
for
nvidia
gpus.
Both
open
mp
and
open
acc
kind
of
are
pretty
much
similar,
but
open
ecc
has
like
especially
the
pgi
implementation.
B
Open
ec
has
a
has
been
around
for
a
longer
time,
so
you
in
some
cases
you
might
find
that
it
is
more
optimized
in
its
implementation
compared
to
openmp,
because
mp
is
still
a
bit
new
with
respect
to
that.
But
openmp
is
also,
but
from
my
experience
both
of
them
can
be
used
to
achieve
the
same
sort
of
performance.
C
Thank
you
rahul.
I
just
I.
I
thought
I
should
let
you
say
first,
because
obviously
openmp
has
been
existing
for
a
longer
much
longer
time
than
sorry.
Openmp
has
been
existing
for
a
much
longer
time
than
opengcc
right.
So
obviously
open
mp
has
been
you
know,
prevalent
for
cpu
for
many
years,
so
there
are
concepts
like
tasks
in
openmp
where
you
could
probably
do
the
same
with
openacc,
but
it's
a
little
bit
more
convoluted.
C
So
if
you
were
to
have
a
you
know
application
you
want
to
break
down
with
respect
to
tasks,
I
would
use
openmp
with
respect
to
gpu
programming.
Openmp
offloaded
offloading
compilers
are
evolving
as
we
speak.
We
know
they
are
a
priority
for
many
different
reasons
and
there
are
codes
beginning
to
exist.
When
I
say
codes
I
mean
more
than
benchmarks,
you
know
I
mean
real
quotes
in
comparison
to
open
acc,
which
has
been
existing
since
2011
2012
onwards.
C
Implementations
began
to
exist
so
predominantly
targeting
gpus,
and
so
the
adaptation
and
usability
of
open,
acc
features
and
implementations
for
gpus
are
are
more.
You
know
readily
available
for
large
cores
for
production
codes
as
well.
Openmp
is
playing
catch
up
and
pretty
sure
they'll
get
there.
But
if
you
want
to
you
know,
use
move
your
100,
I
mean
tens
and
thousands
of
lines
of
code
to
gpu
open
acc
has
been
there
done
that.
So
that's
my
two
cents.
A
D
I
think
some
of
the
the
main
differences
are
the
first
kind
of
philosophical
writing,
like
I,
I
kind
of
was
getting
at
it
in
my
slides,
where
we
tried
to
make
it
the
simple
thing:
simple
right,
easy
to
to
put
raja
just
on
one
loop,
and
you
don't
have
to
change
any
of
your
data
structures
or
your
memory
management
code
right.
We
don't.
We
don't
want
to
take
control
of
that,
and
I
think
the
other
place
where
there
are
some
differences
are
just
some
of
the
kind
of
specific
features.
D
So
I
think
the
kind
of
the
stuff
we
have
in
the
kernel
api
in
terms
of
what
you
can
do
with
nested
loop
patterns
and
how
you
can
map
them
specifically
to
various
parts
of
the
underlying
programming
model.
Back-End,
are
kind
of
distinct
to
raja
and
potentially,
following
on
from
that,
the
way
that
we
express
execution
policies-
it's
not
in
terms
of
like
this
is
a
parallel
loop.
D
A
Okay,
can
we
okay
so
because
of
the
time,
let's
continue
with
the
next
question,
which
is
along
this
similar
lines?
Jack
is
asking
a
question
on
similar
to
questions
that
came
up
yesterday.
Do
david
and
rahul
see
a
path
towards
getting
the
c
plus
plus
standard
to
adopt
some
of
the
ideas
in
caucus
and
raja.
D
Yeah-
and
I
would
just
add
to
that
real
quick-
we
have
an
ongoing
collaboration
between
the
rogers
and
cocos
teams
to
try
and
come
up
with
some
of
these
features.
So
the
thing
we're
working
on
right
now
is
portable
atomics.
That
would
work
across.
You
know
all
these
different
hardware
back
ends,
but
have
the
the
kind
of
semantics
in
the
api
of
what
will
be
in
the
c
plus
standard.
A
B
E
E
B
D
Yeah,
so
we
have
some
stuff
in
development
right
now.
Actually,
where
and
it's
it's
taken
us
a
while
to
to
figure
out
the
api
to
this,
but
basically
yeah
you'll
have
a
kind
of
some
portable
object.
That
represents
your
you
know:
cuda
stream,
for
example,
and
we're
currently
kind
of
figuring
out
what
the
kernels
will
return,
but
it
will
probably
be
some
kind
of
handle
to
an
event,
but
it's
generic,
so
it
basically
kind
of
gives
you
what
you
say
right
you.
You
still
have
events,
but
it's
not
a
cuda
event
anymore.
D
D
Yeah,
so
I
mean
one:
the
one
of
the
motivating
use
cases
for
what
we're
developing
is
your
kind
of
communication
loop
right
where
you've
got
mpi
stuff
going
on,
and
you
want
to
be
dispatching
messages
as
kernels
are
finishing,
so
it's
not
something
that
we
have
working
right
now,
but
it's
certainly
it's
a
use
case.
That's
driving
this
development.
C
C
F
Sorry
I
was
half
paid
attention,
so
I
apologize
I
didn't
know
I
was
going
to
be
on
the
spot.
No
opennc
does
have
the
ability
to
do
asynchronous
communication
either
through
the
the
compute
kernels
can
be
launched.
Asynchronously
to
the
host
or
the
data
movement
can
be
launched
asynchronously.
So
the
host
continues
as
the
data
movement
progresses.
So
it's
all
fairly.
F
E
F
Yes,
some
implementations
will
use
a
stream
if
you're
doing
cuda
to
do
the
dependencies,
but
it's
really
an
async
number
and
you
can
apply
dependencies
via
the
you.
You
can
actually
do
an
async
weight
where
you
do
a
weight
or
to
me
a
acc
weight
on
one
str
once
a
sync
cue
to
another
and
you
can
create
a
whole
different
dependency
graphs.
Based
on
that,
so
you
could
have
a
one
compute
region
which
waits
on
strip
on
async
cube
one
and
two,
but
then
launched
as
three.
F
A
F
A
Thanks,
that's
nice!
So
let's
continue
so
we
have
about
eight
minutes
till
the
end
of
our
session,
so
lots
of
good
questions
and
some
of
the
questions
are
open-ended
and
and
I'd
like
to
remind
the
audience
that
we
have
breakout
rooms
and
hopefully
our
panelists
and
other
attendee
members
will
be
able
to
join
the
breakout
rooms
for
further
discussions.
A
But
before
we
end
this
session,
I'd
like
to
ask
the
final
question
to
all
of
our
panelists,
so
please
feel
free
to
take
turn
to
answer
each
of
these
parts
or
so
that
you
give
all
other
panelists
a
chance
to
answer.
So
my
question
is
so
what
are
the
lessons
you
as
a
panelist
as
an
expert
that
are
advocating
a
certain
programming
model,
have
learned
from
history
of
the
race
of
shared
memory?
Programming
models?
A
That's
the
first
question
so
to
give
you
a
hint.
So
there
is
a
learning
curve.
There
are
academia
and
community
at
large
that
are
taking
an
adoption
and
not
the
least
support
that
comes
with
it.
Finally,
there
are
domain
scientists
who
also
need
to
adapt,
take
adoption
of
those
programming
models.
For
example,
what
you
have
learned
from
feedback
that
you
obtained
via
downloads,
compiler
inclusions
publications?
A
C
I
can
chime
in
so
it's
a
very
good
question.
Thank
you
for
asking
this.
Having
worked
with
several
domain
scientists,
you
know
belonging
to
many
different
domains.
One
thing
we
have
learned,
as
part
of
my
you
know
group
here
in
ud,
is
profiling.
Profiling
helps
immensely
things
that
you
have.
You
know
thought
that
you
have
optimized
and
moved
beautifully
to.
C
This
is
with
respect
to
open
acc
directives
and
so
profiling
and
re
reprofiling.
You
know
going
back
and
fixing
the
optimizations
and
there
are
tools
like
pcast,
which
I
was
mentioning
in
the
call
yesterday,
which
allows
you
to
look
into
the
accuracy
or
verifications
of
the
ports
between
cpu
and
gpu.
That
also
has
helped
us
with
respect
to
you
know
important
domain
science
codes
where
accuracy
matters.
C
C
So
there
are
things.
Directives
cannot
do,
which
is
why
programming
models
like
cuda,
for
example,
or
you
know
even
base
languages-
are
doing
their
fair
share.
So
there
are
things,
directives
cannot
do
and-
and
you
know
instead
of
trying
to
fix
that-
I
think
it
would
be
ideal
to
you
know,
get
the
best
of
all
these
different
worlds
and
put
them
together
and
that
that
would
probably
be
the
base
language
going
forward,
and
probably
directives
won't
be
there.
Ten
years
from
now,
who
knows.
A
A
Okay,
so
the
question
general
question
was
that
what
are
the
lessons
learned
from
history
of
the
race
of
shared
memory,
programming
models
in
the
history?
So
most
probably
before,
advocating
something
you
have
looked
back
into
the
history
and
have
seen
this
race
for
shared
memory
programming
models?
There
were
things
that
were
introduced
as
it
is
happening
now,
and
what
are
the
lessons
that
you
have
learned
and
what
were
the
things
that
were
driving
you
to
advocate
the
things
or
the
specifically
the
programming
models
that
you
are
currently
trying
to
push
forward.
B
You
only
so
I
used
openmp
quite
extensively
before
I
started
using
cocoas
right
and-
and
I
agree
with
with
sunita
and
saying
that,
like
these
directive
based
programming,
models
have
a
have
a
limit
in
what
they
can.
They
allow
you
to
expose,
and
that's
one
of
the
reasons
why
I
I
I
like
cocos,
is
that
it
is
it
is.
It
allows
me
to
express
the
underlying
parallelism
in
in
a
much
finer
and
better
way,
along
with
allowing
me
to
to
use
its
it's.
B
Like
a
view
selected,
you
know
the
data
multi-dimensional
data
storage
classes
and
provide
layout
options,
and
things
like
that
which
which,
when
you
do
with
openmp,
you
would
actually
manually
do
this
this
kind
of
thing
and
and
then
you
have
to
change
it
every
time
you
you
actually
go
from
cpus
to
gpus
and
or
vice
versa,
but
using
it
some
sort
of
in
more
involved
frameworks
just
go
closer.
I'm
assuming
I've
never
worked
with
raja,
but
I'm
assuming.
B
It's
more
easy
to
start
programming
on
a
gpu
with
opencc,
as
against
nathan
mentioned
in
the
previous
question.
If
it's
a
three-month
project
that
you
want
to
do
then
open
acc,
your
opening
is
probably
the
way
to
go,
but
you
might
not
be
able
to
expose
everything
or
get
as
optimal
performance
as
you
can,
with
with
like
a
bit
more
involved
having
models
yeah.
Does
that
answer
your
question.
A
D
Yeah
sure
so
that
was
yeah
super
kind
of
in-depth
question.
I
mean
digging
back
into
my
memory
bank.
I
really
my
first
experience
with
this
kind
of
stuff
was
programming
opencl
right,
so
it's
portable,
that's
great,
but
the
one
thing
that
really
sticks
in
my
mind
is,
is
all
the
boilerplate
right
and
then
kind
of
I
moved
to
cuda
and
yeah.
You
get
less
boilerplate,
but
you
can
only
run
your
code
on
on
one
vendor's
hardware.
D
So
really,
I
think
that
it's
clear
that
some
kind
of
portable
approach,
whether
or
not
that's
you
know
some
c
plus
plus
base
thing
like
coco,
raja
or
a
directive
based
approach-
you
know-
is-
is
really
going
to
benefit
you
in
terms
of
the
the
life
time
of
your
application
and
first
for
a
more
kind
of
practical
recommendation.
One
thing
that
that
we've
seen
is
incredibly
useful
is
with
you
know,
with
something
like
raja
or
cocos.
D
You
can
write
your
code
and
then,
depending
on
the
back
end,
you
select
you
can
target
the
gpu
or
the
cpu.
So
what
a
lot
of
our
code
teams
have
done
is
if
they
run
into
a
bug
on
the
gpu
they'll,
take
exactly
the
same
code,
but
then
have
it
be
run
like
with
the
openmp
threaded
backend
on
the
cpu
and
then
that
lev
lets
you
leverage
all
of
the
kind
of
debugging.
D
You
know
and
correctness
tools
that
you
have
on
the
cpu
to
track
down
bugs,
and
then
you
can
go
ahead
and
rebuild
your
code
for
the
gpu
and
and
oftentimes.
You
know
the
the
tools
you
have
access
to
on
the
cpu
are
better
for
addressing
these
kinds
of
issues.
So
I
think
you
know
any
technology
you
choose
being
able
to
run
exactly
the
same
code
in
multiple
locations
can
really
help
in
terms
of
debugging.
Any
problems
that
you
run
into.