►
Description
2020 CESM Workshop Day 3 - Wednesday, 17 June 2020 - 13:00-16:30 - Software Engineering
The 25th Annual CESM Workshop will be a virtual workshop with a modified schedule on its already scheduled date. Specifically, the virtual Workshop will begin with a full-day schedule on 15 June 2020 with presentations on the state of the CESM. On 16-17 June 2020, working groups and cross working groups have half-day sessions, some with presentations and some that are discussion only.
More information about the CESM Software Engineering Working Group:
http://www.cesm.ucar.edu/working_groups/Software/
More information about the workshop:
http://www.cesm.ucar.edu/events/workshops/2020/
Talks included:
Update on Community Mediator for Earth Predictive Systems (CMEPS)
Coupling MOM6 in CESM: Software challenges and advances
CESM2 workflow capabilities
Fast Access of Irregular Grids and Meshes by Latitude and Longitude
E3SM update
Determining Best Practices for Archiving and Reproducibility for Computational Model Output
Discussion: Managing Model Output for CESM3
Discussion continued: Managing Model Output for CESM3
A
A
B
C
C
D
D
D
A
B
D
A
C
D
F
F
D
Is
some
behind
the
scenes
settings
that
have
to
be
in
place
and
you
have
to
have
a
youtube
account,
but
I
can
send
you
the
instructions.
We've
figured
that
out
and
we've
already
shared
that
as
well
with
some
Noah
local
operators.
You
know
that
was
our
workaround
for
anyone
with
Noah
that
wasn't
allowed
to
use
zoom
so
to
do
that
streaming
option
so
yeah.
What's
the
meeting
get
started,
I'll
send
that
to
you.
So
if
you
have
questions
you
can,
let
us
know
we're
happy
to
share
everything.
We've
learned
the
hard
way.
D
A
D
C
C
A
All
right,
well,
I,
guess
I'll
get
underway,
so
I'm
bill
sacks
I'm
in
the
CES
M
software
engineering
group,
at
NCAR
and
CJD
and
I,
and
the
other
co-chair
of
the
software
engineering
working
group
robbed
Jacob
who's.
A
Argonne
National
Lab
I
want
to
welcome
you
to
the
this
session.
Thanks
for
joining
us
digitally.
A
So
briefly,
as
you've
probably
seen
the
agenda
just
to
remind
you
will
have,
it
will
have
a
number
of
talks
in
the
first
half
of
the
session
and
then
we'll
up
until
about
2:30
and
then
we'll
have
a
little
break
and
reconvene
where
Gary
will
give
a
talk
on
model
output
and
then
that'll
lead
into
a
large
discussion
on
modded
managing
model
output
for
CES
m3.
We're
looking
forward
to
that
lively
discussion
later
this
afternoon.
A
A
C
C
So,
basically,
my
outline
is
going
to
be
I'm
going
to
cover
what
is
c-max.
What
are
its
benefits
to
see
if
I
mean
what
is
its
current
status
and
then
I'm
going
to
dealt
very
briefly
into
a
very
new
effort
that
I
think
is
going
to
be
really
beneficial
for
new
science
coming
into
CSM,
which
is
what
we
call
see:
ducts
cut
off
the
press
or
community
data
models
for
Earth
prediction
systems,
and
that's
just
it's
something
that
will
be
cemex
compliant.
C
So
what
is
cement
sand
before
we
dive
into
cemex
I'm,
just
going
to
give
a
two-second
overview
of
psmf
and
you
op
C,
which
is
the
language
that
C
MIPS
is
built
on
top
of
so
those
of
you
that
don't
know
psmf
where
the
earth
system
modeling
framework
is
an
open
source.
Software
for
building
earth
system
applications,
and
basically
it
provides
standard
component
interface
and
other
high
performance
utilities
for
River
mapping
online
and
parallel
communication.
C
Even
though
it
has
offline
capabilities,
it
is
commonly
used
as
a
common
coupling
infrastructure
layer
for
modeling
systems
and
that's
our
primary
usage
of
baton
seimetz.
But,
most
importantly,
it
is
the
de-facto
coupling
infrastructure
between
components
for
models
from
the
Navy
NOAA
NASA
and
upcoming.
Now
will
be
CSM,
so
when
you
op
C
is
but
stands
for
national
unified,
operational
prediction
capability,
that's
quite
a
mouthful.
C
It's
simply
a
software
layer
on
top
of
the
SMS,
the
pika
gonna,
do
it
as
SMM
plus,
and
what
it
defines
is
out
of
the
box
building
blocks
for
generating
coupled
models
and
conventions
for
them.
The
key
point
is
that
except
eyes,
the
ability
to
have
interoperability
of
model
components
across
multiple
coupled
systems
and
I'll
go
into
a
bit
more
detail
of
what
the
Ops
II
is
in
some
of
these
upcoming
slides.
So
let's
first
start
what
our
current
coupling
infrastructure
is.
C
Csn,
the
communication
between
components
is
called
a
hub-and-spoke
architecture,
so
the
components
don't
exchange
information
directly
with
each
other.
They
communicate
with
what
we
call
a
coupler
or
a
mediator
that
does
things
like
recruiting
or
merging
between
a
set
of
source
fields
and
then
sends
them
off
to
the
destination
component.
So
to
have
any
component
communicate
with
our
coupler.
C
The
system
and
Jude
will
be
Co
hierarchical,
mono
development,
which
is
selectively
turn
on
feedbacks
as
you
develop
your
component.
So
what
is
cement?
Well,
we're
going
to
take
the
central
circle
and
the
arrows,
and
that's
what
we
call
the
Siemens
mediator
and
what
you
need
then,
to
communicate
with
the
cement
mediator.
Is
you
need
to
have
what's
called
new
op
C
caps
in
your
component?
So
once
you
put
the
new
op,
C
caps
in
your
component
becomes
CEMEX
compliant
and
like
before.
C
C
C
So
what
is
new
capabilities
introduced
biopsy
for
the
mediator?
It's
going
to
enable
parallel
online
generation
of
many
ways.
Currently
the
way
MCT
and
our
coupler
7
does
it?
Is
you
need
to
generate
the
mapping
weights
between
the
components,
offline
and
then
store
them
in
an
XML
file
that
gets
bigger
and
bigger?
That
is
no
longer
going
to
be
needed
because
all
of
the
mapping
weights,
except
for
maybe
one
or
two
customized
ones,
like
River
runoff
to
ocean,
are
going
to
be
generated
at
runtime.
C
In
addition,
it's
going
to
enable
us
to
have
new
grid
capabilities,
such
as
the
exchange
production
in
the
mediator,
which
is
something
people
have
wanted
for
a
long
time
to
look
at
Mysterio,
ssin
flux,
calculation.
That
would
not
just
be
computed
on
the
ocean
route
is
an
option,
but
on
the
Union
of
the
atmosphere
and
ocean
grid,
what
about
the
driver?
That's
the
blue!
Currently
in
MCT
and
our
current
coupler
infrastructure,
the
driver
and
the
coupler
are
really
not
separate
there
to
the
conjoined
with
new.
Obviously
they
really
become
totally
separate
entities.
C
You
can
have
multiple
drivers
that
all
talk
to
one
mediator,
so
one
nice
thing
about
new
oxy
is
that
it
enables
you
to
have
a
very,
very
simple,
run:
sequence,
that
is
an
ASCII
file
and
that
it
doesn't
have
to
be
pre
compiled
code.
Another
really
nice
thing
says
these
connectors
in
green
are
auto-generated
from
you.
There
is
no
code
that
does
it
that
you
have
to
create
new
opsi,
doesn't
for
you
and
there's
a
lot
of
optimization
options,
including
reference
sharing
and
new
component
level
3
that
we've
been
putting
it.
C
C
A
real
priority
coming
up
for
CSM
model
developments
and
a
having
CMS
in
place
will
really
facilitate
that,
because
you'll
be
able
to
introduce
these
grids
and
then
do
the
regretting
between
components
on
the
fly
without
having
to
go
through
the
steps
of
storing
and
calculating
all
the
mapping
files
for
this
new
grid.
I
think
one
of
the
nicest
things
is
that
you
can
have
a
very
small
asking
run
sequence,
as
opposed
to
several
thousand
lines
of
complex
code.
C
So
what
does
this
run?
Sequence
really
need?
If
you
look
here
at
the
top
level,
you
have
must
multiple
times
frequencies
for
the
driver,
looping
structure.
These
are
the
coupling
frequencies
for
the
ocean,
which
in
this
case
the
ocean
couple
again
in
half-an-hour
or
1800
seconds,
and
the
inner
loop
is
coupling
in
at
900
seconds.
When
you
see
these
arrows
in
the
run
sequence,
that
means
send
data
via
a
connector
from
the
mediator
to
the
atmosphere
so
and
do
it
via
redistribution
of
the
sorry
redistribution
of
the
data.
C
The
nice
thing
is
that
new
opsi
creates
these
connectors
for
you
automatically
that
was
in
MCT.
There
was
a
ton
of
code
in
place
in
the
driver
code
to
create
these
connectors.
You
don't
need
that
once
you
put
these
connectors
in
your
run
sequence,
they
are
automatically
generated
and
there's
no
user
comments
needs
it.
Another
thing
is
that,
frankly,
here
with
the
mediator,
you
see
that
there's
different
mediator
phases
such
as
merging
to
the
ocean
and
doing
a
fast
accumulation.
C
C
So
what
is
the
current
state
of
C
Medicine
CSM?
At
this
point,
we
have
all
prognostic
and
data
components
that
have
the
caps,
so
we
can
now
do
hierarchical
model
development.
This
is
done
in
parallel
to
the
MCT
driver
we
have
and
the
new
oxy
driver
is
brought
in
to
the
scene
system
when
you
create
a
scene
check
out
right
now,
and
there
is
an
option
to
have
new
op
C
as
your
driver
and
mediator
for
any
component.
C
So
we
have
done
a
careful
of
validation
for
monsta
x,
using
core,
forcing
configuration
and
as
well
as
a
careful
validation
for
data
components
and
we're
going
to
start
with
an
extensive
validation
for
all
of
the
CSM
components.
A
summary.
So
all
the
plumbing
is
in
place.
We've
done
a
lot
of
tests
already,
but
we
have
not
finished
the
validation,
which
has
to
be
very
robust
before
we
say
this
is
something
that
is
ready
to
be
brought
as
a
default
way.
C
C
This
is
a
fact.
Yes
mm
is
that
you
can
bundle
each
of
these
frequencies
as
an
anchor
on
distributed
dimension
in
the
field,
bundle
that
you
switch
back
and
forth
so
rather
than
introducing
25
new
fields,
you're,
actually
wondering
all
of
these
frequencies
into
one
field
and
see.
Meps
knows
how
to
deal
with
that,
so
it
Maps
them
all
at
once.
C
My
final
slide
is
that
we
are
undergoing
a
major
rewrite
of
our
data
model
code
base
to
be
based
on
yes,
MF
new
op
C,
and
this
is
going
to
provide
much
needed
new
capabilities
in
the
system.
For
one
thing,
we
will
be
able
to
grid
online
between
the
streams
and
a
model
and
we'll
be
able
to
do
many
new
things
like
online
3d,
be
pretty
conservative
reprinting
the
ability
to
put
together
multiple
levels
and
recruit
them
between
two
grids
that
have
the
same
vertical
level.
C
C
All
of
this
new
sea
Ducks
data
model
is
compatible
with
C
maps
and
the
share
code
that
we're
creating
is
going
to
have
interfaces
that
can
be
called
directly
from
the
model
components
themselves,
as
is
currently
the
case
with
the
current
data
models,
and
these
are
all
now
being
placed
in
a
separate
repository
in
RTS
comp,
recognization
called
sea
ducks
and
that's
all
I
have
so
I
welcome
questions
and
I
hope.
This
was
not
too
much
to
throw
at
you
at
once
that
I'm
very
excited
about
all
of
these
developments.
A
All
right
thanks
thanks
very
much
Mariela
for
that
great
overview,
and
there
are.
There
are
a
few
questions
or
comments
in
the
in
the
chat,
so
I'll
start
by
reading,
what's
already
there
and
then,
if
others
have
questions
feel
free
to
either
raise
your
hand
via
zoom
or
post.
Your
question
in
the
chat
and
I
can
read
it.
Chuck
heck
eranin
says,
has
a
comment.
A
Thank
You
Keith
I'm.
Not
sure
do
you
feel,
like
your
question
has
been
answered
at
this
point,
Keith
asked
a
question
about
using
the
MCT
data
type
to
call
shares
stream
data
from
the
prognostic
components
and
Marianna
did
address
that
a
bit
well,
I'll
kind
of
ask
what
I
see
is
a
just
a
clarification
or
a
remaining
question
of
that.
Does
the
implication
that
the
prognostic
components
did
use
MCT
data
types
and
the
sharestream
data
will
have
to
have
some
modifications
with
this
new.
No.
C
So
we
have
very
carefully
built
this
so
that
it's
backwards
compatible,
so
your
prognostic
components,
if
they
use
MCT
data
types,
can
still
work
with
MCT
data
types.
Our
goal
is
to
encourage
you
to
move
to
nan
MCT
data
types,
because
it
will
have
many
new
capabilities
that
will
be
advantageous,
but
we're
not
telling
you
either
one
or
the
other,
so
it
is
backwards
compatible
with
the
original
MCT
share
code
base,
because
it's
in
a
totally
different
directory.
C
A
C
C
C
ducts
are
data
models
and
C.
Maps
is
a
mediator.
So
if
you
want
community,
there's
a
community
that
wants
to
use
Siemens
without
see
ducts,
but
my
hope
is
that
by
putting
C
ducks
out
there
and
making
it
really
a
total
bundled
entity
that
can
be
leveraged
by
multiple
efforts,
they
can
be
used
without
necessarily
saying
you
need
seam
or
you
need
C
maps,
I
mean.
Theoretically,
you
would
need
C
MAX
to
use
C
ducts,
but
you
could
also
think
of
calling
it
directly
from
your
code
base.
And
then
you
don't.
C
That
is
something
we
have.
The
hooks
in
place.
I
think
that
the
dart
never
had
a
hook
directly
in
it.
It
was
all
through
the
ESP
component
in
the
infrastructure
we
have
in
C,
so
we
just
have
to
have
the
ESP
component
be
cemex
compliant,
and
we
have
not
done
that
yet.
But
that's
that
is
something
that
is
planned
and
we
are
very
cognizant
still
has
to
be
done.
Okay,.
A
Eric
guests,
another
question:
Eric
I,
think
in
the
interest
of
time
I'm
going
to
go
onto
other
people,
and
your
second
question
is
a
good
one.
But
we'll
come
back
to
that
offline
about
the
details
of
how
streams
are
done
differently.
I
want
to
give
other
people
a
chance
to
see
from
Rob
Jacob
asks.
Have
you
done
any
performance
comparisons.
C
Initially,
yes,
we
are,
we
are
definitely
seeing
some
degradation
that
we
believe
we
can
get
rid
of.
You
know
this
is
the
first
thing
we
put
in
place.
If
we
saw
quite
a
bit
more,
we've
eliminated
a
lot
of
it,
so
I
think
we're
seeing
somebody's
still
in
the
order
of
10%,
and
we
need
to
eliminate
that
and
I'm
very
confident,
yeah,
okay.
A
C
Think
that
that
is
going
to
be
a
problem,
we're
going
to
work
really
hard
to
facilitate
that
to
happen
and
to
make
it
easy
to
do.
Currently
we
have
other
externals
that
need
to
be
installed
and
I
don't
see
this
as
a
huge
leap
to
insist
on
that
as
a
requirement,
given
all
the
benefits
that
you're
gonna
get.
A
Great
well,
thank
you
very
much
Mariana
for
for
that
overview
and
summary,
and
so
if
you
could
stop
sharing
your
screen
and
then
next
up,
we
have
al
pair
Altos,
who
is
going
to
be
telling
us
about
a
coupling
mom
in
CES
M
software
challenges
and
advances
so
out
there.
You
could
share
your
screen
when
you're
ready.
D
H
So
here
is
the
timeline
of
this
project.
We
started
about
three
years
ago,
and
these
are
some
of
the
major
activities
and
milestones.
I'll
just
mention
that
we
began
with
the
development
of
the
MCG
cap
and
then
we've
developed
the
superstructure
called
mom
interface.
Last
year,
we've
incorporated
the
new
ops
II
cap,
which
was
primarily
developed
by
Mariana,
and
we
also
develop
this
mom
RPS
module.
H
This
is
a
Python
module
within
mom
interface,
and
this
module
allows
us
to
define
and
create
out-of-the-box
matrix
configurations,
I'll
elaborate
on
that
a
little
more,
but
our
current
task
is
to
finalize
our
out
CSM
2.2
configuration
so
I'm
listening
here.
Some
of
these
software
challenges
that
we
encountered
during
this
project.
H
First
of
all,
this
has
been
a
collaborative
effort
across
multiple
institutions,
and
so,
as
one
would
expect,
there
has
been
some
occasional
differences
in
our
conventional
standards
priorities
and
another
challenge
has
been
that
we
have
been
working
with
actively
evolving
code
bases.
Mom
6
itself
is
rapidly
evolving.
H
We
get
new
science
feminization
functional
least
routinely,
and
we
get
about
1.5
documents
per
year.
Seem
is
pretty
much.
The
same
also
evolves
and
another
moving
part
has
been
this
new
coupling
technology
based
on
lop
skin,
see
maps.
One
other
major
challenge
I'll
mention
here
is
that
there
has
been
some
implementational
discrepancies
between
the
FMS
library
and
CSM.
Fms
is
the
underlying
library
that
was
developed
by
GFDL
and
gets
used
by
mom
sticks
extensively.
H
It
provides
functionalities
like
time
manager,
domain
definitions,
Diagnostics,
input-output
model,
constants,
communications
etc,
and
some
of
these
functionalities
overlap
with
what
CSM
or
scene
provides
and,
and
so
inevitably
there's.
Some
discrepancies
that
we've
encountered
between
these
two
that
had
to
be
addressed
and
another
challenge.
H
So
I'll
briefly
go
over
a
development
effort.
That's
representative
of
these
challenges
that
I've
just
listed
here.
We'd
roll
up
a
monstrous
runtime
parameter
manager
in
CSM,
and
the
goal
here
is
to
define
and
maintain
out-of-the-box
configurations
of
month
six
within
CSM,
and
to
do
that,
we
need
to
find
a
common
ground
between
conventional
one,
six
approach
of
defining
experiments
and
that
of
CSM,
and
we
also
need
to
address
some
complex
interdependencies
between
monster,
expanders
and
CSM
parameters,
and
the
approach
we
take
is
to
repurpose
some
of
the
conventional
mom.
H
H
It
gets
invoked
by
seam
when
a
user
creates
a
new
case
and
sets
up
that
new
case
and
I
don't
have
time
to
go
into
details,
but
one
functionality
that
I'll
just
highlight
is
that
this
module
supports
conditionals
and
formulas
that
use
arbitrary
Python
expressions
in
its
input
file
templates
and
also
note
that
CSEC
is
currently
exploring
two
adapters
by
other
components.
See
some
components
as
well
and
so
here's
an
example
of
an
input
parameter
sample
definition.
H
Similarly,
formula
is
the
pipe
is
a
Python
expression
that
incorporates
the
ccsm
variables
and,
let's
say
for
a
particular
case:
our
ocean
grid
is
txt
or
6061.
Coupling
base
period
is
day
and
we
couple
24
times
a
day
and
then
the
fee
theorem
then
gets
set
to
3600,
and
this
would
be
our
out-of-the-box
configuration
for
this
particular
case,
and
if
the
user
happens
to
change
the
coupling
interval,
then
this
month
six
parameter
also
gets
updated
automatically
and
will
be
make
sure
to
be
compatible
with
the
rest
of
the
coupled
system
and
just
to
summarize.
H
A
functional
mark,
6
release
will
be
not
coming
cos:
I'm
3.2.
We
have
this
online
user
manual,
that's
evolving
document
in
mr16
CSM,
it's
in
monitor,
face
repository
wiki
page,
and
then
we
have
this
ongoing
mom
six
webinars
a
webinar
series
where
the
monastic
community
members
provide
tutorials
science,
thoughts
and
case
studies,
and
then
here
are
our
own
activities.
We
are
working
on
improving
our
gmail
and
Bhima
configurations
and
my
clear
is
leading
the
efforts
to
incorporate
marble
into
mom
sticks
and
then
we
are
working
on
more
feminization.
A
H
So
these
are
practically
key
value
pairs
in
a
tektite
txt
file.
So
it's
a
bit
different
than
nameless
files,
but
some
of
them
also
had
their
more
specialized
syntaxes
and
that
mom
RPS
module
allows
us
to
output
in
those
syntaxes.
But
yeah
depends
on
the
file
parameter.
Files
are
basically
key
and
value
pairs,
but
the
Diagnostics
configuration
files
are
a
bit
more
complex.
A
A
I
I
I
I
Knows
that
the
system,
you
can
add,
pre-run
data
assimilation
and
post
run
scripts
to
the
the
case
that
run
itself.
Now
these
are
going
to
use
the
same
resources
as
the
model
executable
uses,
so
you
either
want
to
make
sure
that
they
are
very
fast
small
processes
or
that
they
actually
utilize
all
those
resources,
so
that
you're
not.
I
I
There
is
a
data
assimilation
script.
Obviously
that's
a
leading
name.
You
don't
have
to
use
it
for
data
assimilation,
but
that's
kind
of
what
was
what
we
had
in
mind
when
we
put
it
there
and
you
can,
and
so
that
will
run
after
the
model
executable
and
you
can
dictate
how
many
cycles
of
model
executable
and
data
assimilation
you're
going
to
run
before
you
complete
that
and
go
on
to
run
scripts.
I
Then
there's
also
the
resubmit
flag
that
you
probably
know
about,
and
it's
an
integer
flag.
It
tell
us
how
many
batch
system
jobs
you're
going
to
run
in
your
cycles
before
you
before
the
model
stops
and
we
resubmit
is
greater
than
zero.
The
completion
of
a
case
run
archive
site
will
start
another
case
run
an
archive
cycle
and
the
resubmit
flag
is,
is
decremented,
continue
around
and
sent
a
true.
F
I
So
a
basic
climate
model,
workflow
might
be
to
you,
know,
create
an
experimental
case,
build
the
model,
run
the
model
and
then
archive
some
model
output
and
then
and
and
all
this
is
in
the
default
CSM
to
distribution.
This
all
happens
right
out
of
the
box.
If
you
add
the
post-processing
tools
to
your
your
case,
then
you
can
also
implement
the
time
series
generation
and
model
Diagnostics
tools
that
are
available
in
that
post-processing
package.
I
So,
for
example,
if
you
want
to
add
series-
and
you
have
the
post
processing
tools
implemented
for
your
case,
you
can
just
add
when
you
create
the
new
case,
you
can
add
the
time
favorite
series
workflow
and
it
will
generate
code
to
run
that
time.
Series
script
along
with
your
other
aspects
of
the
model,
run.
I
So
the
case
control
system
provides
a
basic
workflow
generator.
It
uses
the
queuing
system
native
dependency
tools
to
scout
your
jobs
in
a
workflow,
so
that
has
some
limitations.
All
jobs
have
to
be
submitted
to
queues,
there's
no
clock
or
calendar
support
on
submission,
and
it's
limited
to
a
single
case,
but
you
can
use
preview
run
to
view
the
current
workflow.
Obviously,
this
this
kind
of
workflow
is
is
a
little
bit
limited
if
you're
doing,
for
example,
real-time
forecasting,
and
so
we
have
extended
our
workflow
to
allow
for
I'm.
I
I
I
I
Prereqs
so,
for
example,
for
case
town
st
arc,
I
have
to
run
the
variable.
The
xml
variable
DRS
has
to
be
equal
to
true,
so
that
would
be
a
prereq
listed
in
the
config
work
flow
file,
and
then
you
can
set
some
runtime
parameters
in
in
here.
The
tasks
count
the
number
of
tasks
per
node
in
the
wall
time
if
these
are
different
than
what
the
model
run
uses.
If
you
don't
set
these
they'll
use,
the
model
run
defaults.
I
Using
the
preview
run
script
in
the
case,
I
mentioned
resubmit
immediate
earlier.
If
you
want
all
of
your
jobs
to
be
submitted
at
once
and
wait
in
the
queue
and
there's
more
than
one
reason
to
do.
That
I
mentioned
that
you
know
some
systems
don't
allow
submit
from
compute
nodes.
That's
one
reason
to
do
it.
Another
reason
might
be
that
the
queuing
system
gives
an
advantage
to
jobs
that
have
been
waiting
in
the
queue
longer,
even
though
they
have
prerequisites
so
some
HPC
queueing
systems
will,
you
know,
run
your
jobs
faster.
D
I
I
We
have
tools
to
create
an
ensemble
workflow,
so
create
clone,
has
new
options
to
enable
ensemble.
So
this
example
of
create
foam
will
create
eight
ensemble
members
and
use
the
same
executable
for
all
into
those.
So
you
might
do
this.
For
example,
if
you're
using
cams
initial
condition
perturbation,
you
can
use
the
same
executable
and
just
have
a
bona
fide
painless.
I
And
there's
a
script
in
the
tools
directory
called
generate
cell
workflow,
and
it
translates
a
case.
Control
sets
workflow
to
a
soap
sweep
at
our
C
file
that
you
can
then
ingest
with
silk
and
used
to
create
a
more
sophisticated
workflow,
and
so
it
provides
support
for
ensembles,
where
the
the
internal
workflow
only
handles
a
single
case.
You
can
customize
workflows
with
all
the
extensive
features
of
silk.
I
I
I
I
My
goodness,
okay,
so
we've
had
several
experiments
using
these
tools.
I'll
mention
a
few
of
them.
Interglacial
land
ice
experiment.
They
were
doing
ten
year
model
runs
after
which
they
were
adjusting.
The
topography
sent
to
the
atmosphere
based
on
new
land
ice
and
they've
run
this
for
something
like
seventy
five
hundred
years.
Now,
with
this
with
this
workflow,
the
Noah's
em,
our
weather
app
came
out
of
the
box
with
a
workflow
that
includes
the
change
res
to
modify
initial
conditions
for
the
FE
3
dye
core
and
yeah
they're
they've
been
pretty
pleased
with
it.
I
I
A
Alright
thanks
so
much
Jim
for
that
I
will,
let's
see
I,
think
there's
a
couple
of
questions
already
in
the
chat.
So
I'll
start
with
that.
Well,
first,
as
a
comment,
just
following
on
from
your
final
slides
to
seal
honey,
also
says
that
she
has
used
this
ensemble
workflow
option
and
really
likes
it.
So
there
there's
a
testament
from
a
user
there
you
go
Frances
fit
asks
how
is
case
dot,
submit
desk
job
different
than
desk
only
job.
So.
I
A
I
The
ensemble
is
not
the
same
as
the
multiple
instance
feature
scene.
The
multiple
instance
feature
of
scene
creates
a
single
executable
with
multiple
instances
of
the
component
models,
including
the
coupler,
so
that
will
all
run
in
one
batch
job,
where
the
ensemble
feature
has
a
separate
batch
job
for
each
model
run
and
which,
which
method
you
use
depends
a
little
bit
on.
You
know
what
kind
of
experiment
you're
doing.
A
J
So,
a
couple
weeks
ago,
actually
about
a
month
ago,
I
was
given
an
assignment
by
Simone
films
to
take
the
output
of
CES
and
regionally
fine
grid
and
map
it
to
EPA
stations.
So
I
hadn't
seen
the
grid
before
and
I've
decided
to
plot
all
the
points.
The
points
are
in
columns
all
the
data's
and
columns
and
over
CONUS.
J
It
looked
like
the
image
on
the
left
here
and
over
zoomed
in
on
Colorado
and
I
found
that
it
wasn't
quite
what
I
expected
it
was
in
these
3x3
grid
cells
or
3x3
clusters
and
I've
since
learned
that
that's
that
those
are
the
spectral
elements,
so
I
found
this
to
be
fascinating
and
looked
at
it
larger,
and
you
can
see
here.
On
the
left
hand,
side,
this
is
zoomed
in
over
CONUS
and
then
these
clusters,
they
rotate
and
expand,
outwards
and
achieve
a
coarser
resolution
farther
away
from
the
continental
United
States.
J
So
I
have
some
numbers
here
on
the
right
in
the
CSM
regional
refined
model
that
I'm
using
it
has
hundred
seventy
four
thousand
grid
points
and
it
expands
out
this
way.
Their
rows
and
columns
over
CONUS
are
kind
of
latitudes
and
longitudes,
but,
as
you
can
see,
they
rotate
over
Africa,
so
they're
kind
of
the
latitude
and
longitude
are
almost
kind
of
reversed.
So
this
is
kind
of
the
idea
of
how
to
how
can
I
approach
this
and,
of
course,
I
know
this
is
for
computation
across
scales.
J
So
this
is
what
I
have
to
look
with
and
I
don't
know
if
anybody,
if
you've
all
looked
at
this
very
closely,
but
this
is
what
it
looks
like
when
you
plot
it
out.
So
here's
the
problem
as
Simone
presented
to
me.
We
have
a
thousand
DPA
stations.
We
want
to
look
up
the
value
of
CES
m
at
all
those
a
thousand
points
of
comparison
plotted
as
a
scatter
plot,
which
you
see
on
the
right
hand,
side
here,
and
if
all
the
points
are
clustered
along
the
diagonal,
then
we
know
we
have
good
treatment.
J
She
wanted
to
do
this
efficiently
and
she
remarked
that
she
had
written
some
IDL
code
that
goes
and
compares,
looks
up
every
EPA
station
with
all
174
K
points
and,
of
course
she
recognized
that
that
was
inefficient,
and
so
she
called
in
computer
scientists.
So
the
baseline
algorithm
is
to
look
up
all
p
epa
stations
across
NCES
m
grid
points
just
find
the
closest
one
and
report
that
oh,
that
would
be
a
algorithm
on
the
order
of
P
times
n.
J
That's
the
baseline.
Now
this
is
something
we're
gonna,
even
if
we
don't
do
regreting
as
Mariana
mentioned.
This
is
something
we're
gonna
do
a
lot.
We're
gonna,
have
flight
trash
curtain,
plots
all
sorts
of
things.
We're
gonna
be
accessing
these
things
by
latitude
and
longitude,
because
that's
a
natural
way
to
do
this,
so
we
certainly
want
something
more
efficient
than
that
and
I
came
up
with
an
so.
J
So
here
is
the
fast
access
algorithm
that
I
came
up
with
and
how
fast
it
is.
The
order
is
n
log,
n,
plus
P.
So
since
there's
a
plus
sign
in
the
middle
of
that
expression,
you
know
that
there's
gonna
be
two
phases
and
you're
correct
the
first
phase,
the
n
log
n
is
a
load
phase
and
there's
a
lookup
phase.
So
for
this
particular
EP,
a
problem
load
time
3.7
seconds
lookup
time
was
less
than
a
tenth
of
a
second,
so
depending
on
what
you
want
to
count
here.
Overall,
the
speed-up
is
40
X.
J
J
Let's
say
the
thousands
of
plots
that
I
produce
daily
you're
gonna
be
approaching
something
like
a
1500
factor
of
speed-up,
but
the
algorithm
works
by
taking
all
these
points,
all
hundred
seventy
four
thousand
points
and
really
ignoring
all
the
structure
to
them,
ignoring
all
the
regulatory
that
is
there
they're,
just
random
points
thrown
onto
the
globe
just
all
over
the
place,
and
the
first
thing
that
is
done.
Is
it
sorts
all
these
points
into
latitude
bands?
So
there's
these
bins
here
we
have
West
East
bands
by
latitude
here.
J
So
this
these
yellow
marks
are
a
picture
of
the
earth
globe.
Here,
second
step
once
you
have
them
in
latitude
bands,
you're
gonna
put
them
in
longitudinal
stripes
you're.
Setting
up
an
artificial
grid
by
latitude
and
longitude
and
you're
putting
all
the
points
into
these
grid
cells
now
I
have
a
sparseness
factor
because
I
don't
want
them
to
end
up
all
clustered
in
a
certain
place
and,
as
you
can
see
so
what
you're
gonna
expect.
Some
were
like
like
see
this
four
eight
and
five
here,
that's
CONUS
light
large
density.
J
There,
zero
one,
zero
down,
lowest
row,
that's
going
to
be
over
the
southern
ocean
where
there's
not
many
points
as
it
turns
out
the
maximum
of
points
in
any
one
cell
turned
out
to
be
nine
and
I
recognize
that
as
the
3x3
cluster.
So
in
all
this
mapping,
I
have
captured
one
cluster
in
one
of
my
grid
cells.
J
So
anytime
you
have
a
new
algorithm.
You
want
to
look
at
the
Diagnostics
and
make
sure
it
is
doing
what
you
think.
It's
doing
so
I
plotted
the
density
of
points,
namely
the
number
of
points
in
any
latitude
band
and,
as
you
can
see,
it's
high
over
CONUS,
which
is
good
because,
let's
say
Brownsville
or
Key
West
somewhere
it
starts
around
25
South
and
then
the
United
States
goes
to
about
49
north
and
the
points
tail.
But
I
expect
you
just
want
to
check
this
kind
of
thing,
there's
some
stuff
going
on
on
the
equator.
J
You
have
to
search
like,
let's
suppose,
a
is
an
EPA
station
and
we're
looking
for
the
closest
point
in
CES
M
we're
gonna
search
outward
in
these
squares
by
a
certain
radius
until
we
find
another
point
since
all
the
points
are
you
know,
Fault
within
some
grid
cell,
you
only
need
to
know
the
nearest
point,
the
nearest
portion
of
that
square,
and
if
that
is
outside,
let's
say
this
blue
circle
here,
then
you
know
that
nothing
in
there
can
be
closer.
So
in
this
case
we
have
an
illustration.
We
start
at
a
we're.
J
Looking
at
B,
these
circles
are
not
exactly
centered
on
a
vertex,
because
a
is
not
on
a
vertex.
So
this
is.
This
is
the
way
you're
looking
at
it
and
there's
this
like
I
mentioned,
there's
a
sparseness
factor
that
spreads
everything
out
so
you're.
Looking
at
these
things,
so
you
have
to
search
on
the
order
of
maybe
10
20
cells
instead
of
174,000,
which
is
a
big
advantage.
So
I
also
want
to
test
my
algorithm
and
I
found
that
the
maximum
search
radius,
the
maximum
distance
from
an
EPA
station
to
a
CES
endpoint,
was
radius.
J
4
and
I
said
that
must
be
a
mistake
too
far.
Looked
it
up
and
sure
enough.
It's
a
mistake.
It's
out
in
the
Northwest
Territories.
It
can't
be
right,
there's
nothing!
There!
There's
no
City!
There
is
no
nothing
until
I
turn
on
the
satellite
view
and
sure
enough.
There's
a
dam
and
there's
a
building
and
somewhere
in
that
building
people
are
measuring
quantities
and
reporting
them
to
us
and
okay.
So
the
point
here
is
that
the
accuracy
decreases
outside
of
CONUS,
as
you
expect.
J
In
other
words,
it's
farther
from
where
you
find
the
nearest
CSN
point,
but
the
algorithm
still
works.
That's
as
we
expect
when
you
go
way
down
north
or
away
from
the
grid.
If
you
get
you
have
to
get
farther
away.
So
in
terms
of
software
engineering,
this
algorithm
is
coded
in
about
600
lines
of
Python
code
and
I
made
it
as
a
Python
module,
and
we
have
unit
tests
and
diagnostics
there
for
the
future.
J
Here
we
are
at
a
workshop,
I,
propose
and
suggest
that
we
create
a
standard
Python
module
for
handing
handling
these
kind
of
grids,
just
like
the
Python
dwarf
module,
there's
something
in
there
called
interpol,
which
I
found
very
useful.
So,
let's
make
something
like
that
and
perhaps
add
it
to
the
psmf
framework
somewhere
else.
But
let's
use
this
thing,
so
every
person
who
encounters
the
same
problem
doesn't
have
to
spend
two
or
three
days
working
through
how
to
solve
it.
J
So
this
is
like
I
said
this
is
not
really
a
regrip,
but
it's
it's
a
lookup
thing.
A
fast
lookup
Oh
we'd
all
like
to
speed
up
code,
there's
a
lot
of
applications
for
this
and
just
beyond
post-processing
and
plotting.
But
a
lot
of
lookups
and
comparisons.
I
make
like
I,
said
thousands
of
plots
per
day,
so
I'm
really
interested
in
a
speed-up
factor
of
40
times
to
1,500
and
I'm
gonna
close
with
sort
of
the
look
at
the
parameters
we
have
here.
J
The
range
is
the
full
globe.
As
mentioned
the
there's,
an
aspect,
ratio
and
I
did
my
sparseness
factor.
The
latitude
stride
turns
out
to
be
three
tenths
of
a
degree
and
if
you
increase
this
person's
factor,
you'd
increase
that
and
perhaps
there's
a
way
to
tune
this
but
I
just
who
in
just
oh.
So
this
is
the
parameters
here
and
at
that
I
can
open
it
up
to
any
questions.
A
Hey,
thank
you
very
much
Carl
for
presenting
that
to
us.
There
are
a
lot
of
questions
already
pouring
in.
So
let
me,
let
me
start
with
less
here
and
others
should
feel
free
to
chime
in
with
additional
questions.
Chaka
Karan
and
asks
is
the
geographically
closest
grid
point
the
best
choice.
Are
you
investigating
other
techniques
that
take
into
account
the
modelling
values
for
the
EPA
relevant
parameters?
J
The
answer
is:
Simone
recognized
that
so
she
specified
the
closest
point.
So
that's
the
algorithm
following
I
didn't
think
about
land
or
sea
points.
What
I
thought
mostly
about
was
the
bilateral
interpolation.
So
if
so
in
expanding,
this
I
would
have
gone
towards
something
like
find.
The
nearest
enclosing,
quadrilateral
and
interpolate
across
that
I
didn't
think
about
the
land
sea
section,
so
that
could
that
would
be
an
improvement
here.
J
A
And
Jeff
Anderson
comments.
This
is
a
sub
problem
of
what
we
have
to
do
for
data
assimilation
in
dart
or
any
other
system.
We
have
a
number
of
tools
that
perform
similar
operations
and
are
based
on
quadtrees
searches
that
are
similar
to
your
ideas.
We
actually
do
much
more
than
finding
just
the
closest
point
we
find
a
set
of
neighbors
and
then
interpolate
in
various
ways.
A
J
A
Adam
Harrington
I
think
I.
Think
that's
who
a
herring
is
asks.
Can
you
can
you
think
of
how
to
extend
this
method
to
compute
zonal
means
using
your
storing
method.
This
is
relevant
to
data
storage
because
recruiting
to
a
high
resolution,
global
grid
dictated
by
the
smallest
DX
and
the
refined
reach
and
leads
to
massive
files
that
are
untenable.
J
B
Right
I
could
unmute
myself.
So
that's
when
you
did
these,
we
to
refine
runs,
are
using
kms
II
the
spectral
element,
so
you
can
just
specify
output
fields
to
be
the
closest
points
to
you,
give
it
a
collection
of
points,
and
you
say:
I
want
I,
want
the
output
from
the
closest
point
to
this,
and
it
will
actually
just
write
that
up
what
is
part
of
the
history
mechanism.
B
Also,
you
can
output
to
a
Latin
long
grid,
although
I
think
Adam
was
indicating
that
if
you
use
the
high
resolution,
you
end
up
with
a
large
lat/long
grid,
which
we
don't
want
to
do
because
you
end
up
with
like
this.
I
resolution
long
outfit
over
africa
think
you're
not
interested
in,
or
you
know
the
pacific
or
something
yeah.
J
J
Any
unstructured
grid
I've
done
storm
surge
modeling
and
what
people
do
is
they
put
a
bunch
of
points
in
New,
York,
Harbor
and
all
the
channels
there
in
the
East
River
then
out
in
the
Atlantic
Ocean?
That
gets
really
sparse,
because
it's
deep
and
and
it's
just
like
totally
put
where
we
feel
like
putting
points
this
will
work
fine
for
that.
Okay,.
J
A
So
this
is
something
I'll
just
chime
in
with
like
with
a
comment
for
related
to
this,
and
more
broadly
I
mean
I,
think
you
know,
we've
we've
had
discussion
in
a
number
of
sub
groups
about
about
kind
of
making
repositories
for
post-processing
in
CES
n
as
a
whole
or
individual
components
and
I
know.
The
land
modeling
group,
which
I'm
most
involved
with,
has
started
a
github
repository
for
post-processing
scripts
for
the
land
model.
A
C
TSM
I
know,
there's
a
lot
of
work
on
the
format
long
and
others
to
come
up
with
kind
of
tools
that
could
be
used
for
this
but
yeah.
This
is
maybe
something
where
we
could
use
some
more
more
coordination
and
I
think
that
would
be
useful
for
a
number
of
us
to
talk
about
and
come
up
with
a
plan
for
for
how,
where
to
host
these
useful
post
process
and
general-purpose
post-processing
scripts.
So
thanks
for
raising
that
point
of
discussion.
A
D
F
F
F
We
are
now
building
our
e3
SM
with
C
make.
This
was
process
started
last
year
by
a
gym
food
car.
What
happens
is
case?
Setup
now
builds
info
needed
for
C,
make
and
then
case
build
calls.
The
C
make
generated,
make
files-
that's
in
the
scene
case
control
system,
so
to
make
doesn't
handle
all
of
the
logic
in
the
giant
Perl
scripts
that
are
area
behind
the
atmosphere
in
gland.
Those
still
need
to
be
converted
to
Python
at
some
point,
but
C
make
building
is
now
the
default
within
3sm.
F
So
this
group
also
maintains
several
diagnostics
and
other
packages.
If
you're
some
dye
eggs
is
our
tool
for
doing
most
of
our
atmospheric
model.
Analysis
and
we've
been
adding
a
couple
of
components
for
and
so,
and
also
adding
a
key
bo
analysis
and
some
new
observational
data
sets
to
compare
against
as
well.
F
Ethier
M
supports
development
of
NCO,
along
with
other
agencies
and
I
know.
These
tools
are
well
known
to
people
who
work
with
this
ESM
data
they've
finalized,
released
improvements
to
sub-grid
scale,
B
gridding.
This
is
for,
if
you
have
a
Lam
model
file
with
data
on
by
vegetation
type
or
wear
time,
functional
type,
it
can
map
those
to
a
grid
for
for
visualization
and
analysis
and
there's
some
new
map
quality
checking
tools.
F
These
are
checking
the
offline
maps
that
are
generated
by
tools
like
psmf
or
tempest
female
and
making
sure
that
they
are
also
some
spatial
mean
options
for
the
instantly
mode
time,
series,
splitter
and
they're,
starting
to
work
on
porting
compute,
evie
parts,
an
MCO
to
GPU,
that's
available,
so
another
tool
that
we've
developed
is
called
c
stash.
And
what
this
does
is
it
kind
of
sits
between
the
short-term
archive
and
their
long-term
archive?
F
If
you
wanted
to
analysis
on
some
others,
so
looking
now
at
the
main
model,
repository
and
changes
in
the
past
year,
we've
had
Atomics
more
external
sub
modules.
We
just
get
so
much
of
a
lot
in
3sm
cost
to
the
cloud.
Simulator
is
now
some
module
tracking
the
development
of
the
main
simulator
in
the
CF
NIP
github
project
cv
mix
and
push
in
B,
G,
C
and
M
paths
are
now
sub
modules
now
I'm
passive.
This
is
itself
a
sub-module
nucleus
M,
so
III
o
set
now
has
to
be
cursin
sub
modules.
F
It
makes
the
code,
though,
mix
the
code
for
these
pieces
more
visible
and
removes
a
fragile
part
of
the
build
system,
and
but
you
can
still
clone
the
source
single
command.
It's
just
that
recursive,
which
we
were
already
telling
people
to
do
just
to
get
the
normal
some
modules
and
when
you
update
your
sub
module,
so
you
also
have
to
add
Kirsten.
F
We've
brought
that
back
to
master,
and
so
all
further
development
will
continue
there
and
there's
a
mate
1.2
branch
for
the
cryosphere
ones,
which
is
going
to
have
one
more
thing
at
it
to
it,
I
think
and
they're
going
to
have
to
and
they're
going
to
redo.
Some
of
the
grass
here
runs
than
that
we've
also
added
some
new
test.
Suites.
We
have
a
four
biogeochemistry
cases
running
nightly
on
the
main
one.
One
branch
and
we've
had
a
test
for
one
of
the
good
creation
with
grid
creation
tools
that
are
in
scene
yeah.
F
That's
the
part
of
the
scene
custody
so
and
within
the
main
bottle
of
the
development
v2
we
thought
was
going
to
be
minor
changes
compared
to
be
one,
but
that
seemingly
happen.
One
of
the
big
changes
is
that
we're
moving
to
a
tri
grid
configuration
so
the
land
and
River
components
are
now
on
a
single
half
degree
grid,
most
basically
whatever
grid.
That
means
we
have
to
make
a
new
atmosphere
to
land
Maps.
That's
no
longer
just
a
copy
operation.
That's
that's
yeah
interpolation!
F
F
We
also
experimenting
with
our
region,
where
you
find
meshes
not
only
in
the
atmosphere
but
also
in
the
ocean
and
the
sea
ice.
The
upper
right
there
shows,
where
the
grids
around
North,
America
I
think
we
now
have.
A
version
of
that
who
are
the
grid,
extends
all
the
way
across
the
Arctic
to
the
fine
part
and
then
there's
some
other
improvements
to
it.
I'm,
sorry
physics,
but
this
is
the
same
computational
approach,
v2,
it's
still
Fortran
MPI
and
open,
and
it's
scheduled
for
release
September
of
next
year.
F
The
feature
freeze
was
actually
July
of
last
year
and
we've
been
just
kind
of
tuning
it
and
testing
it
and
testing.
All
these
new
features
since
then
so
the
mode
based
coupler
that
I've
mentioned
before
we
had
a
lot
of
effort
in
the
past
year,
directed
to
an
offline
to
that's
long,
mb
tempest
this,
takes
the
tempest,
remap
capabilities
for
calculating
area
weights
and
and
combines
them
with
Moab's
data,
parallel
abilities
and
its
intersection
schemes.
F
F
This
is
like
some
part
over
Siberia
if
you
can
see
that
there's
parts
of
the
globe
where
there
are
no
mesh
and
there's
parts
of
the
globe
where
the
overlaps
and
that's
very
bad
for
us
for
doing
areally
calculations.
Basically,
you
have
to
just
correct
the
weights
afterwards
to
fix
that
and
we
were
able
to
get
Moab
coupler
to
send
from
two
different
atmospheric
grids,
but
now
we
have
to
kind
of
redo
that
work,
because,
with
the
introduction
of
the
physics
grid
and
the
tri
grid,
all
of
the
coupling
occurs
only
on
the
physics
grid.
F
A
F
Simple
GPU
systems,
basically
perlmutter
phase,
one
is
going
to
be
the
first
one
that
has
AMD
chips
with
nvidia,
then
Argonne
is
getting
an
Intel
system
with
Intel
CPUs
and
shortly
after
that,
frontier
is
getting
an
EMV
system
that
they
had
to
be
Radeon
GPUs,
and
these
are
will
be
the
first
exascale
systems
in
the
US,
and
what
you
can
see
here
is
first
along
at
4x
sko,
at
least
for
the
first
version.
Everything
is
GPU
and
there's
at
least
three
different
flavors
of
GPU.
So
what
we're
dealing
doing
to
handle
this
is
we're.
F
Looking
at
a
couple
of
different
approaches
for
programming
portability.
First
one
is
to
use
Fortran
with
OpenMP,
4.5
or
5.0.
4.5
has
GPU
target
offload
capability.
You
might
be
familiar
of
openmp
as
a
way
to
program
threads
on
a
cpu,
but
you
can
also
use
it
to
target
a
GPU
and
then
another
approach
is
C++
and
Koko's.
So
C++
Koko's
is
a
template
programming
model
that
has
a
back-end
for
various
GPUs
and
there
are
similar
approaches
to
this
and
he
uses
a
template
programming
model
in
C++
to
make
this
happen.
F
A
B
A
F
Some
of
the
things
that
are
Fortran.
We
don't
know
what
to
do
about
the
history
system.
I
can
has
a
lot
of
features,
and
it's
any
am
have
a
lot
of
history.
Teachers
in
its
history
system
and
we're
not
quite
they're,
not
quite
going
away
rewriting.
All
of
that
and
I
think
there
are.
There
are
various
parts
of
the
of
the
dynamics
driver
that
are
still
Fortran,
I,
think,
but
nothing
nothing
to
people
tense.
Okay,
okay,.
A
A
K
A
All
right
so
welcome
back
and
Gary.
Are
you
yeah?
Okay?
So
for
this
second
half
we
will.
The
second
half
of
the
meeting
this
afternoon
is
going
to
be
focused
on
model
output
and
gary
strand
will
get.
This
started
with
a
short
talk
on
determining
best
practices
for
archiving
and
reproducibility
for
computational
model
output
and
then
we'll
follow
from
that
into
a
large
discussion
for
the
rest
of
the
meeting
on
on
managing
model
output
for
CES
m3.
So
Gary
when
you
are
ready,
go
ahead
and
share
your
screen
and
go
ahead.
G
Indoor
met
near
Nick
at
the
in
car
library,
like
Shuster
of
the
who's,
the
head
of
the
essential
research
data
archive
here
at
NCAR,
talking
about
how
to
best
practices
for
archiving
of
reproducibility
of
model
data
and
if
you're
not
familiar
with
earth
cube.
It's
a
NSF
funded
project,
a
partnership
between
geo
and
the
division
of
advanced
cyber
infrastructure,
to
transform
Geoscience
research
by
developing
cyber
infrastructure
to
improve
access,
sharing,
visualization
and
analysis
of
all
forms
of
geosciences
data
and
related
resources.
G
G
So
this
is
a
topic
wretched
game
that
AMS
back
in
January,
which
seems
like
a
different
reality
compared
to
now.
But
so
what
we're
trying
to
try
to
do
is
get
as
much
community
collaboration
on
this
is
we
possibly
can
and
there
will
be
a
workshop
come
you
got
a
second
workshop
coming
up
in
August
I'll
talk
about
that
in
a
minute.
So
obviously
you
know
what's
the
problem,
what's
the
plan
to
tackle
problem
and
how
can
you
participate
so
on
the
atmospheric
sciences?
G
G
We
have
many
many
users
that
we
share
data
with
the
climate
data
portal.
Climate
data
gateway
here
at
NCAR
has
roughly
50,000
user
accounts
in
which
seven
hunter
have
just
been
registered
just
this
year.
So
far,
so
we're
getting.
You
know
several
a
day,
dozens
a
week
and
obviously
that
you
know
for
sharing
across
all
these
things.
G
We
need
to
come
up
with
good
data
centers
and
at
which
term
it
you
can
choose
from
and,
of
course,
push
for
open
access
and
reproducibility,
and
of
course
this
has
led
to
requirements
from
our
funding
agencies
and
publishers
and
others
to
actually
implement
all
this
stuff.
So
from
the
aju
publications.
Here
are
their
requirements
for
the
data
that
we
create,
so
all
data
necessary
to
understand
and
so
forth.
G
New
protocols
or
method
used
to
generate
the
data
in
the
paper,
any
software
or
code
and
as
well
as
derive
data
products
reported
or
described
in
a
paper
and
that's
fairly
daunting,
because,
even
though
we've
done
this
kind
of
thing
before
to
some
extent,
we've
never
had
a
requirement
from
publishers
to
do
this,
except
just
recently
I
mean
the
old.
You
know:
contact
the
author
for
access
to
the
data
was
the
old
tried-and-true
way
of
doing
things
and
that's
no
longer
acceptable.
But
I've
had
a
few
of
our
eyesight
just
come
to
be
saying.
G
I
put
that
on
my
paper,
the
reviewers
said
no
I
can't
do
that
anymore,
so
help
I
need
to
get
something
going
on
this.
So
the
aju
advices
keep
all
of
the
data.
Ams,
however,
has
a
different
set
of
guidelines
here,
so
they
had
all.
They
requires
that
your
data
are
car,
archived
and
cited
properly
and
so
forth,
and
so
they
follow
up.
They
ask
do
what
fair
sentence
and
what
fair
is,
is
findable,
accessible,
interoperable
and
reusable
data.
So
there's
a
couple
of
examples
of
fair
positing
issues.
G
Basically,
the
upshot,
edits,
AMS
says:
keep
somewhere
all
the
data
and
ask
someone
else
to
do
that.
For
you,
the
journals
will
archive
our
data
for
us.
The
funding
agencies.
Will,
you
know,
ask
for
work,
you
know,
provide
some
funding
for
archives,
but
we
don't
have
any.
You
know
NSF
master
archive
for
all
the
science
and
NSF
funds
and,
as
always,
you
know.
Typically,
we
have
so
much
more
data
than
what's
actually
shown
in
a
paper
that
you
know.
Maintaining
and
keeping
all
this
stuff
is
a
non-trivial
task.
G
G
Terabytes
of
archive
bottle
output
on
our
tape,
archive
system,
yeah
and
one
of
my
big
tasks
over
the
next
16
months
or
so
or
ever
it
is
until
October
of
2021
when
it
goes
away,
you
know
how
what
to
be
saving
that
old
stuff
onto
our
current
archive
system,
which
is
not
on
tape,
thinking
they
work
out
the
fact
that
we
have
a
brand
new
model
here
that
we
want
to
do
all
sorts
of
interesting
experiments
with
you
know
large
ensembles
see
that
six
being
and
another
example
and
so
forth.
So
how
do
we
balance?
G
G
G
Where
do
we
go
between
saving
all
the
output
or
saving
a
bunch
of
the
output
versus
just
saving
the
set
up,
and
you
know
for
TSM
specifically
I
mean
it's
use
across
many
research
disciplines,
it's
difficult
to
reproduce
and
it
has
high
computational
cost
and
it
requires
you
know
either.
You
know
like
Cheyenne
or
Cory,
or
some
of
it
or
one
of
these
you
know
HPC
systems,
so
we
tend
to
save
more
output
than
saving
the
set
up.
I
work
for
other
modeling
groups
for
modeling
other
modeling
communities.
G
It
may
be
more
important
to
just
say
the
initial
conditions
and
this
code
and
so
forth,
because
it's
relatively
easy
to
regenerate
the
data
and
those
not
always
the
case
with
us,
but
then,
of
course,
there's
other
aspects
of
for
example.
Reproducibility
you
know,
is
it
really
it
for
bit
reproducibility
or
is
it
you
know,
statistical
similarity
reproducibility?
G
In
the
old
days
you
used
to
think
that
as
a
model
became
obsolete,
and
then
you
know,
as
the
data
that
it
generated
aged
it
got
over
and
older
would
be
less
interesting,
therefore
not
needing
to
be
kept.
But
actually
people
are
now
interested
in
in
analyzing
these
generations
of
our
climate
models
to
see
how
they
improved
in
performance
over
time.
So,
in
fact,
you
know
we
have
data
from
ccsm
3
that
is
still
actively
being
used,
believe
it
or
not,
email.
G
G
So
the
next
thing
that
the
this
project
is
going
to
be
undertaking
is
some
use
case
use
case
samples,
idealized,
parameterization
studies,
so
I'm
going
to
do
that
seam.
If
I
talk
about
Leonard
comparative
project,
we
do
that,
of
course,
and
various
other
ensemble
modeling
systems
and
so
forth,
and
each
of
these
different
kinds
of
models
have
different
needs
in
terms
of
how
they
how
they
generate
and
how
they
need
to
keep
their
data.
G
G
G
What's
going
on,
and
the
stearic
committee
has
a
good
group
of
people
dokkan
that
csmp
scientist
is
also
on
the
steering
committee,
and
we've
also
got
a
couple
really
excellent
students,
there's
some
really
fantastic
ideas,
but
I'm
very
excited
that
they're
participating
and
just
my
last
slide.
If
you're
interested
in
invitation,
please
email,
Doug
and
while
you're
interested
in
participating
and
your
background,
there
is
the
github
location
for
the
on
the
documentation
and
the
about
and
so
forth
that
the
project
is
and
thank
you
for
attention.
I'll
be
happy
to
answer
any
questions.
A
Thanks
a
lot
Gary
so
yeah,
we
can
take
some
questions
if
you
have
questions
specific
to
Gary's
presentation,
of
course,
we'll
have
a
long
time
to
discuss
model
output
this
afternoon.
But
if
there's
questions
specific
for
for
this
for
Gary
in
this
presentation,
please
go
ahead:
either
raise
your
hand
in
the
participant
window
or
ask
something
via
the
chat.
G
That's
for
ever
considered
that
kind
of,
like
Cadillac
level
service.
That's
if
you
want
your
data
hosted
by
the
research
data
archive
which
is
a
you
know,
they
they
primarily
focus
on
observational
data,
but
they
they
will
host
data
for
us,
but
at
forty
five,
forty
five
dollars
per
terabyte
per
year,
that
gets
to
be
very
expensive
TSM.
G
A
Alright,
so
yeah,
so
we're
gonna,
we're
gonna
move
on
to
a
big
discussion
now
led
by
Shari,
Gary,
Jim
and
Jesse,
and
just
a
couple
of
points
there
so
feel
free
to
continue
to
use
the
the
chat
or
it
might
be,
might
be
most
convenient
to.
If
you
raise
your
hand
and
we
could
and
people
could
actually
speak,
please
feel
free
also
to
turn
your
video
on.
A
So
we
could
see
each
other,
especially
when
you're,
when
you're
speaking,
if
you're
comfortable
doing
so
and
I'm
going
to
be
putting
a
link
in
the
chat
for
for
a
Google
that
we
can
use
to
take
some
joint
notes
on
this
discussion.
I
will
be
taking
notes,
but
would
welcome
other
people
to
to
contribute
to
this
Google
Doc
as
well
and
I.
Believe
Rob
is
going
to
be
the
main
person
responsible
for
watching
the
chat
and
for
raised
hands
very
proud.
Yes
great!
So,
let's
see
sherry
are
you
gonna,
be
the
one
kicking
it
off.
L
L
L
If
you'd
like
to
contribute,
we
ask
that
you
use
the
raise
hand,
feature
where
you
use
the
chat
feature
and
for
those
of
you
who
missed
how
to
use
the
raise
hand
feature.
Basically,
you
click
that
participation
icon
at
the
bottom
and
then
that
pops
up
a
little
window
that
you
can
go
ahead
and
raise
your
hand
in.
L
So,
while
brainstorming,
we
came
up
with
four
topics
to
discuss
around
the
idea
of
standardization,
I
was
just
gonna,
go
ahead
and
walk
through
each
one
of
these
briefly
and
provide
some
talking
points
on
each
just
to
help
start
the
discussion
I
mean
no
way.
Are
we
limiting
the
discussion?
We
just
kind
of
wanted
to
provide
some
potential
starting
points
for
the
discussion,
so
appliance.
So
what
do
you
mean
by
mid
compliance?
L
Well,
it's
adapting
the
controlled
vocabulary
used
within
the
CAF
community,
so
this
includes,
like
variable
names,
specific
units
for
each
of
those
variables,
specific
descriptions
that
are
attached
to
each
one
of
these,
as
well
as
the
attributes
that
are
attached.
This
also
includes
the
benefit
of
more
uniform
output
coming
from
each
one
of
our
models
by
standardizing
all
these
different
attributes,
but
why
MIT
compliance
so
Gary
touched
upon
this
a
little
bit.
It
makes
our
data
and
work
there,
which
again
is
findable,
accessible,
interoperable
and
reusable.
L
It's
also
becoming
a
required
format
to
publish
to
yes
yep
the
MIT
compliance,
so
not
outputting
in
a
compliant
form
causes
other
issues
that
makes
our
default
data
hard
to
use.
Like
you
know,
for
example,
the
conversion
of
oceans,
CGS
space
to
an
end,
chaos
also,
the
finally,
the
the
translation
exercise,
it's
kind
of
a
black
art.
L
Not
many
people
know
how
to
do
this,
and
it
would
just
be
a
lot
easier
if
it
came
from
the
model
directly
as
an
option
and
if
you're
not
as
familiar
with
the
controlled
vocabulary,
first
seem
at
six
and
at
this
unit,
six
scatter
request
itself.
If
you
go
to
the
Google
Doc
on
that
bill
mentioned
I,
provided
some
exactly
what
the
the
syntax
is
there
in
case,
you
are
curious.
L
So
another
topic
for
discussion
is
if
we
should
standardize
the
model
output
across
all
the
components
or
in
other
words,
abstract
the
output
methods
from
each
of
the
components
and
then
put
them
into
a
shared
library,
I'm
also
up
for
discussion,
asynchronous
I/o.
This
would
allow
us
to
write
IO
while
having
the
model
continue
on.
Is
this
a
good
idea?
You
know
it
would
hide
a
lot
of
the
cost,
but
would
we
end
up
outputting
more
and
contribute
more
to
our
output
volumes
if
we
go
ahead
and
do
that
so
an
interesting
kind
of
perspective.
L
L
L
L
Should
we
be
setting
our
defaults
to
no
output
and
require
users
to
add
each
variable
by
hand
or
have
a
minimum
subset,
that's
smaller
than
what
we
currently
output
and
to
help
guide.
This
part
of
the
discussion
we've
added
some
sizes
and
for
some
of
the
collections
we
hold
for
those
totals
first
see
that
five
and
cement
six,
those
totals
are
what
we
provided
to
the
community
via
yes,
Jeff
there's.
Also
some
extra
data
behind
both
of
those
data
sets
as
well.
L
So
we
have
time
series
for
each
of
those
and
first
thing
that
v
we
also
have
the
time
slice
file
still
for
those
I
not
as
familiar
with
cmip5,
but
the
last
time
I,
checked,
I.
Think
cmips.
Six
was
at
one
point:
five
petabytes
compressed
sitting
behind
the
scenes
of
that
that
half
a
petabyte
that
we
have
listed
here
so
quite
a
bit
of
output.
L
And
finally,
we
also
hope
to
discuss
compression
and
the
benefits
of
both
lossless
netcdf
for
compression
and
lossy
compression
and
and
how
we
should
implement
this
within
our
workflow
to
make
it
easier
for
the
users
to
compress
their
data,
and
you
know
this
is
always
you
know
the
elephant
in
the
room,
but,
along
with
the
discussion
you
know
we
should
always
discuss
how
any
initiative
could
get
done.
You
know
we
always
have
these
great
ideas,
but
we
often
lack
the
manpower-
and
you
know
the
funding
to
get
some
of
this
stuff
done.
L
M
Well,
if,
if
I
can
yeah,
you
know,
Chuck
brings
up
an
interesting
plane
about
the
cost
of
data
storage
and
stuff
and
I
and
I
know
it's
not
as
simple
as
just
buying
more
disks
for
Cheyenne
or
whatever
machine.
Is
there
y-you
know
what
one
of
the
things
that's
puzzled
me
and
and
we've
talked
about
it
a
bit
in
our
data
task
force
meetings.
G
Well,
I
mean
you
know
if
Dave,
as
you
point
out,
I
mean
the
amount
of
space
that
we
have
available
to
us
will
always
be
filled
up
eventually
right,
no
matter
how
big
it
is,
we'll
always
manage
to
fill
it
out,
and
so
I
mean
that
the
way
to
get
around
out
of
me
compression
is
that
is
a
step
in
the
right
direction.
Direction.
Lossy
compression
is
a
bigger
step,
but
the
biggest
step
we
can
take
is
just
reduce
the
model
output.
G
I
mean
we
already
know
that
a
lot
of
our
data
isn't
used
by
that
many
people
I
mean
we've
got
a
you
know.
10%
of
the
data
is
used
by
90%
of
our
users,
and
you
know
the
vast
bulk
of
it
is
it's
often
used
by
maybe
one
or
two
people,
or
just
a
very
small
number,
so
we
will
have
to
make
some
tough
decisions
about
you
know.
Do
we
really
need
to
say
400
ocean
variables?
G
Every
you
know
by
default,
300
or
400
atmosphere,
variables
or
more
3
or
400
land
variables
or
more
I
mean
I
like
sea
ice.
It's
nice
and
small
and
plays
well
with
others,
but
you
know
we
we've
got
a
real
data
problem
and
so
it's
not
clear
that
simply
buying
more
and
more
stories.
That's
not
going
to
fix
it.
N
So,
just
to
come
back
to
completion.
So
far
we
have
been
doing
most
less
compression
and
that's
buying
us
some
space,
but
I
think
yeah
I
mean
probably
Allison
will
say
a
little
bit
more
I.
Think
if
you
are
going
to
essentially
address
this
problem
one-nothing,
it's
the
only
solution,
but
I
think
we
need
to
really
seriously
consider
the
most
compression
and
that's
going
to
be
essentially
transparent
to
the
user
as
far
as
I
can
tell,
because
the
end
product
will
be
essentially
reduced
size
netcdf
file.
N
So
the
second
thing
is
that
we
need
to
change
the
culture,
and
that
is
the
most
difficult
part
I
think
because
it
is
say
I
mean
we
are
a
community
project.
We
are
a
community
model
and
just
an
example
is
essentially
CSM
large
CSM,
large
ensemble
event
out
and
ask
people
what
variables
they
wanted
to
see
and
then
most
of
those
variables
to
the
extent
possible
are
now
included.
But
at
the
end
only
one
person
may
end
up
looking
at
that
particular
3-dimensional
ocean
variable.
N
So
I
think
we
need
to
be
more
judicious
in
our
choices
of
variables
because
we
can't
say
well,
if
I'm
going
to
look
at
it.
I
may
need
this
variable
five
years
down
the
road
and
that
has
been
going
on.
In
fact,
we
encountered
an
issue
with
the
new
CSF
two
simulations
for
spit
SEMA,
five
fields
or
forcing
fields,
and
in
that
case
the
default
was
essentially
give
again
as
many
variables
as
possible,
and
then
we
don't
have
the
place
to
store
those.
D
Yeah
well
I
have
many
questions,
and
especially
for
other
components,
but
because
of
a
particular
proposal.
I'm
working
on
by
now
I'm
facing
this
third
issue.
Right
now,
just
by
loss,
lossless
compression,
the
Sturge
adds
up
to
over
five
hundred
terabytes.
That's
a
lot
so
so.
My
first
question
is:
how
easy
is
it
to
actually
do
lossy
compression?
D
Do
you
need,
like
special,
you
know
special
skills
to
do.
This
is
like
easily
doable
by
anyone,
and
the
other
question
is
like
with
the
lossy
compression.
Do
you
lose
information
vital
for
your
analysis?
Have,
like
all
the
other
components
checked
whether
III
know
there
was
a
study
that
was
trying
to
look
into
this,
but
has
the
study
led
to
any
like
in
specific
variables
that
should
not
be
stored
as
lossy
in
in
the
university
compressed
files.
E
Sorry
this
is
Alison.
I
can
talk
a
little
bit,
lossy
compression,
there's
many
different
types,
there's
some
ones
that
are
directly
built
into
netcdf
that
you
can
use
like
bit.
Grooming.
There
are
some
more
advanced
ones
that
people
in
sizzle
are
working
on,
incorporating,
maybe
you're
familiar
with
the
CSM
lens
project,
where
we
applied
lossy
compression
to
two
of
the
ensemble
members
and
there
we
use
lossy
compression
as'
post-processing
tool
and
we're
hoping
to
make
this
tool
available
to
people
soon.
Given
the
interest
in
this
the
question
of,
do
you
lose
information?
That's
important!
E
If
you
are
compressing
data
for
your
own
use,
and
you
know
how
you're
going
to
use
it,
you
can
easily
pick
parameters
on
the
lossy
compression
to
make
sure
that
you
don't
lose
things
that
are
important
to
you.
What's
harder
is
when
you
have
a
data
set
like
CSM
lens
data
that
we're
putting
out
there,
that
hundreds
of
people
are
using
in
different
ways
and
we're
not
exactly
sure
how
they're
using
it.
E
So
it's
not
a
trivial
issue,
but
it's
something
that
I've
been
working
on
for
a
while
and
highing,
also
in
sizzle
and
others,
and
we
hope
in
the
future
to
have
automated
tools
to
suggest
the
right
amount
of
information
loss
and
also
we
have
tools
that
you
can
kind
of
analyze
the
data
yourself
to
see
what
the
differences
are.
So
that's,
hopefully,
a
quick
overview.
I'm.
A
E
It
can
I
mean
there's
two
issues:
how
much
you
reduce
the
data
and
then
after
you
do
so
what
the
quality
of
it
is
so
different
compression.
Our
approaches
are
better
for
different
types
of
variables.
If
a
variable
is
very
smooth,
it's
pretty
compress
if
a
variable
has
big
jumps
in
the
data
and
are
a
huge
dynamic
range,
it's
harder
to
compress.
So
it's
not
that
one
compression
method
is
always
better
for
every
variable.
It
depends
which
I
know
is
not
very
helpful,
but
that's
the
situation.
I
N
Thing
correct
me:
if
I'm
wrong
Allison,
most
of
the
so
far,
work
has
been
on
the
atmospheric
model
output
for
regarding
lossy
compression
and
if
you
want
the
Ascension
of
being
in
the
other
components,
such
as
ocean
and
sea
ice,
there
needs
to
be
some
preliminary
work
done
right
in
terms
of
the
how
to
approach
the
lossy
compression.
That's.
E
They
can
be
the
reason
that
the
atmosphere
is
easier
is
because,
basically,
there
aren't
fill
values
if
you
just
naively,
send
the
ocean
data
to
a
lossy
compressor
and
don't
tell
it
to
treat
the
ten
to
the
35th
fill
values
differently.
It's
going
to
do
a
horrible
job,
compressing
your
data,
so
there's
just
little
things
like
that
that
are
that
can
be
taken
care
of.
We
just
haven't
done
it
yet.
D
E
Well
again,
I
think
it
really
depends
on
which
approach
you're,
using
how
long
it
takes
and
I
think
at
some
point
you
have
to
think
of
compressing
data.
Maybe
if
you're
doing
a
lot
of
analysis,
you
would
uncompress
the
data
first
to
do
it
all,
and
this
would
sort
of
be
analogous
to
you
have
some
data
and
you
were
like
writing
it
to
tape,
and
that
takes
a
really
long
time
and
then,
when
you
want
to
do
something
with
it,
you
wouldn't
do
the
operations
to
it
on
tape.
M
Alison
I
wanted
to
ask
a
clarification
question.
You
said
it's
been.
The
compression
analysis
has
been
done
on
the
atmosphere,
so
does
that
mean
that
the
Pepsi
challenge
ensemble
members?
It
was
only
lossy
compression
on
atmospheric
history
files
and
that
the
other
components
were
untouched.
That's.
E
M
E
F
Like
to
bring
out
a
few
comments
from
the
chat
and
some
questions,
one
was
a
suggestion
from
Kevin.
Rader
could
part
of
the
data
volume
management
strategy
beta
to
find
some
kind
of
criteria
for
keeping
you
know
the
data
such
as
number
of
users
in
the
last
year
of
publications,
to
ride
from
it
something
to
life
times
can
be
defined
ahead
of
time
to
give
researchers
a
deadline,
and
that
way
you
can
tell
what
data
should
be
kept
in.
What
could
be
purged.
G
Those
are
good
ideas,
Kevin
that
I
like
those.
Why
I
want
to
keep
this
chat?
Those
are
good
ideas,
one
of
the
difficulties,
those
it's
since
people,
that
the
citation
of
data
isn't
quite
as
mature
a
citation
of
journals.
Sometimes
it's
hard
to
tell
if
the
datasets
been
used
in
a
publication
or
not
I
mean
I
can
get
download
metrics
easily,
but
that
doesn't
tell
me
tell
us
anything
about
whether
or
not
it
datasets
that
used
so
people
start
citing
datasets
regularly
and
making
that
part
of
their
process.
That's
kind
of
hard
to
tell.
F
O
And
you'll
hear
me
now:
yes,
sorry,
okay,
sorry,
sorry
I!
Yet,
like
my
microphone
I,
don't
know
it
was
not
working.
There
was
have
to
come
on
because
it's
I'm
reporting
in
with
for
all
the
cam
people,
because
we
had
a
discussion
last
week
about
this,
because
I
think
it's
really
important
that
we
try
to
reduce
the
output
that
we
generate
in
camera
and
another
component
too.
But
I
can
only
speak
for
cam,
especially
that
we
are
going
to
triple
the
vertical
resolution.
O
And
then
we
have
already
a
lot
of
issue
right
now
out
and
I
think
we
should.
We
should
really
try
to
reduce
the
output
and
they
were
several
idea
that
was
bring
it
right.
Now
we
can
control
the
output
of
cam
with
numbness
flaggin
and
it
would
be
nice
to
have
like
a
tier,
1
tier
2,
3
3
of
data
that
we
will
bring.
They
are
not
a
friend
that
we
output
a
lot
of
data
and
we
don't
really
need
it.
O
Then
it
would
be
nice
to
have
like
right
now
we
have
like
history,
atmospheric
cooking
group,
it's
the
default
flag
and
it
output
a
lot
of
variable.
Less
that
we
have.
Another
name
is
flag,
but
still
we
can
assume
reduce
this,
and
there
were
a
few
ID
that
we
can,
for
example,
output
on
the
lower
resolution,
great
balls
for
the
vertical
and
horizontal.
When
we
do
a
higher
resolution
in
the
horizontal
we
can
F
like,
for
example,
for
the
photo
degree
to
get
one
degree
greater
some
kind
of
grid
it
can.
O
We
might
need
to
do
internal
mapping
that
is
conservative
for
the
conserve
variable
and
also
for
all
the
3d
variable.
We
can
probably
output
on
less
level.
We
are
going
to
a
spa
biblia,
something
between
80
and
90
level,
vertical
level,
but
for
the
same
f6.
The
default
is
to
output
on
70
level
and
we
can
F
an
option
in
camel
that
would
just
say
Easter.
O
O
O
Think
it's
really
a
good
time
to
do
this,
as
as
we
are
going
to
move
to
come
seven,
especially
with
a
vertical
resolution,
and
then
the
last
thing
that
I
want
to
say,
I
really
like
the
idea
of
leaving
compassion
include
in
the
workflow,
but
it
would
be
also
nice
to
have
compression
script
and
I.
Think
Addison
mentioned
that.
O
Maybe
we
would
have
conspiration
script,
it
would
be
shared
because
right
now
we
have
I
got
some
compress
and
for
Alper
and
from
CAD,
but
I
feel
that
it
would
be
really
said
that
all
the
scientists
start
to
write
their
own
poorly
written
compression
script,
and
it
would
be
nice
if
we
can
have
something
that
we
share
a
CSM,
and
this
is
what
I
wanted
to
say.
It's
a
lot
of
saying,
but
yeah
and
I
can
send
I
yet
sent
note
to
Bill
about
this.
But
I
wanted
to
share
with
the
group
for
discussion.
A
Can
I
can
I
just
chime
in
there
for
a
second
to
kind
of
follow
up
on
that,
thanks
for
sharing
that
Cecile
and
I
I
guess
I'd
be
curious
to
hear
to
hear
from
other
other
components
of
people
that
are
on
online
now.
What
what
others
think
about
having
these
different
tiers
about,
but
and
especially
to
what
extent
components
should
make
their
own
decisions
about
what
works
best
for
that
component
and
to
what
extent
this
should
be
kind
of
a
shared
decision
that
we
try
to
do
something
uniform
across
all
of
the
CSM
components.
D
Hey
Bill,
this
is
Peter
Lauritzen
before
you
move
on
to
other
components.
I
just
wanted
to
second
what
Cecile
said
in
particularly
in
in
the
horizontal.
You
can
have
half
the
amount
of
data.
If
you
just
slightly
decreased
resolutions,
instead
of
operating
on
1
degree,
you
could
do
one
point
five
degree
and
it
kind
of
also
agrees
with
at
least
a
numerical
method,
science,
which
we
all
know
that
you
know
the
grid
scale
cannot
really
be
trusted.
So
nuts
engineer
is
not
that
you,
you
might
be
cutting
away
information.
I
I
would
like
to
say
in
the
development
of
the
next
kam
version.
I
think
that
the
the
cam
history
capability,
that
is
you
know,
averaging
keeping
mins
and
maxes
and-
and
all
that
is
is-
is
really
good.
But
it
should
be
pulled
out
into
a
into
a
shared
library
external
to
cam
that
the
other
components
can
use.
So
we
can
kind
of
standardize
across
the
phonons
right
now.
Each
component
kind
of
has
a
different
way
of
doing
I/o.
That
causes
confusion
and.
B
M
You
know,
and
so
that's
another
level,
I
think
that
we've
always
thought
about
for
the
sea-ice
and
then
there's
an
extra
level
beyond
that,
which
is
if
we
want
to
do
budget
analysis.
You
know
mass
budget
analysis
or
something
like
that.
Then
we
will
need
a
lot
more
data
potentially
daily
and
things
like
that,
but
that
may
only
be
for
a
few
years
of
the
run
or
something.
E
B
Just
want
to
back
with
him
on
six.
We
already
have
that
option.
So
there
is
a
XML
variable
ocean,
diag
mode,
which
currently
supports
three
different
options
of
production,
spinner
or
development,
and
then
the
amount
of
output
is
quite
different
between
those
a
for
example.
If
you,
if
you're
doing
something
like
what
God
was
saying
like
a
paleo
like
study
you,
you
would
initially
have
spin
up
as
an
option
and
then
saying
minimally,
and
then
you
can
change
later
on.
So
we
kind
of
already
have
that
option
between
mom
six.
L
B
L
B
N
B
L
An
extra
layer
on
top
of
that
you
know
the
the
data
request
itself
isn't
entirely
stable.
You
know,
different
things
are
changed,
you
know
variable
descriptions,
for
example.
Is
there
an
easy
way?
You
guys
keep
up
with
that
or
is
it
something
where
someone
manually
has
to
do
a
difference
between
you
know
versions
in
order
to
get
you
know
any
changes
and
then
it's
hand
coded
it
into
mom,
6
I'm.
L
It
can't
be
a
name
change
also
for
like
the
long
name
for
a
variable,
or
you
know
you
know
just
the
description
you
know.
Sometimes
even
units
are
change.
Do
you
guys
keep
up
with
that
at
all
or
is
it
just
kind
of
you
know?
You
know
putting
the
it's
hard-coded
in
and
it
would
take
someone
to
go
into
the
code
to
modify
something.
If
there's
an
update
to
the
data
request
itself,
so
yeah
yeah.
B
B
No
I'll
take
the
back.
We
do.
We
do
check
sums
on
the
Diagnostics,
so
the
it's
going
to
catch
if
it
changes
the
name
of
the
butcher,
but
the
issue
is
one:
could
adapt
for
a
particular
name.
Changing
one
is
hard-coded
with
Imam
six
and
the
second
one
is
what
we
call
bone
interface,
which,
as
the
name
says,
does
the
interface
between
mom
and
CSM
and
that's
a
tool
that
Alper
developed
and
within
mom
interface
we
can
like.
B
M
Yeah
I'm,
just
in
a
disturber
but
anyway
that
the
the
other
thing
I
was
thinking
of,
is
I.
Think
these
different
levels
of
output
is
a
great
idea
for
just
limiting
what
the
models
are
writing
out,
but
I
think
at
some
point
there's
only
so
much
we
can
do
with
those
different
tiers
and
there's
just
gonna
have
to
be
hard
decisions
made,
maybe
at
the
post-processing
step
where
some
of
those
variables
are
just
left
behind
and
just
step
removed
from
scratch
or
whatever,
but
or
not
post
processed
at
all.
M
But
the
other
point
I
was
going
to
make
and
and
sort
of
similar
to
what
share
you
were
asking
about.
Mom
6
is
that
one
of
my
worries
is
that
say,
seem
that
seven
comes
along
and
they
give
us
a
brand
new
list
of
variable
names.
You
know
and
and
and
so
I'm
a
little
worried
about
that,
making
me
sort
of
MIT
compliant
within
the
models,
because
I'm
gonna
have
to
redo
this
again:
four-seamer
7c
Miffy
or
maybe
we're
not
participating
in
those
but
I.
Don't.
H
G
Dave
I
I
wouldn't
worry
too
much
about
see
you
at
7.
Being
a
radical
departure
from
see
you
at
6
I
mean
seen
it.
6
was
an
enhanced
for
the
scene
at
5.
They
just
made
some
minor
changes
and
I
mean
I
asked
for
more.
It
feels
and
so
forth,
but
the
established
fields
there
were
no
changes
in
the
definitions
so
other
than
additional
fields.
The
request
that
I
don't
think
we're
going
to
see
you
know.
I
saw
a
change
from
a
chai
to
HQ,
or
you
know
some
some.
M
M
Encounter
in
his
pad,
you
know,
I
know
that,
because
this
assignment
body
was
formed,
they
completely
changed
the
sea-ice
variable
names.
So
I
had
a
lot
of
work
just
going
from
C
MEP
three,
the
CMM
five,
so
yeah.
So
it's
just
I
mean
maybe
maybe
you're
right,
maybe
seem
m7,
there's
not
a
big
wholesale
change,
but
it's
just
yeah.
So.
K
I'm,
going
back
to
compression
and
tying
into
the
statement
about
cmip5,
not
changing
much
going
into
cmips.
Six,
that's
of
course
true,
but
one
one
way
that
it
did
change
was
in
the
support
of
compression
that
had
long
avoided
and
so
now
seem
if
six
is
all
compressed
with
the
deflate,
which
is
the
only
algorithm
that
netcdf
four
has
historically
supported
by
default
and
there's
some
really
exciting
news
on
that
front.
That
I
wanted
to
make
workshop
members
available.
K
These
codecs
includes
e
standard
and
from
Facebook
and
snappy
from
Google
and
lz4,
and
what
that
will
allow
people
to
do
in
the
future
is
just
not
right.
Does
the
old
deflate
compress
theta,
which
is
the
lossy
compression
algorithm,
that
that
seemed
up
six
uses,
of
course,
and
some
of
these
algorithms,
like
snappy,
read
and
write
ten
times
faster?
So
when
Gunter
was
asking
about
compression
of
land
ice
data
and
how
fast
that
was
it's
not
that
fast.
K
But
that's
because
deflate
is,
if
I
understood
your
your
point
and
one
of
the
only
lossy
compression
that
we're
using
is
very
slow
and
it's
thirty
years
old.
So
with
is
with
these
new
codecs
that
Ed
and
I
are
putting
into
netcdf,
you
can
do
things
faster,
they'll,
compress,
more
efficiently
and
the
most
recent
netcdf
library,
which
four
point,
seven
four
I
think
or
maybe
four
point.
Seven.
Three
added
support
for
parallel
compression.
K
So
now
the
PIO
to
library,
ed,
is
working
on
putting
the
parallel
compression
into
the
PIO
two
library
and
the
models
themselves
will
be
able
to
hopefully
speed
up.
Throughput
I,
don't
know
what
the
status
of
that
is,
but
it's
in
the
library
support
now
from
netcdf
and
then
for
for
users.
So
for
post-processing
you
can
look
in
the
chat
if
you
want
and
get
an
example
of
how
these
these
new
codecs
will
allow
really
modern
compression
algorithms
to
be
accessed
and
I.
K
K
L
K
K
So
ad
Hartnett
is
working
on
that
for
the
Noah
models
actually
implementing
that,
and
then
the
functionality
applies
to
any
of
these
codecs.
So
it's
really
the
the
the
support
is
there
for
any
to
invoke
compression
through
hdf5
in
parallel
and
now
it's
being
exposed
through
net
CD.
App
is
probably
the
best
way
to
put
it,
and
so
yes
in
parallels
is
possible.
A
G
G
A
A
L
L
G
Guess,
I'll
start
no
thanks!
Everybody
for
the
excellent
discussion
so
far
and
the
the
chat
I've
saved
it
several
times
already,
thanks
Dave,
for
that
one
of
the
things
about
standardization,
well,
I,
think
my
reason
for
now
advocating
NIP
compliance
on
all
of
our
output
is
that
you
know
in
the
era
of
SEMA
3
and
Sina
5.
The
specification
for
those
was
relatively
modest
and
it
really
didn't
cover
that
much
of
what
we
typically
say,
whereas
now
with
cement
6,
that's
no
longer
true
I
think
you
know
what
we're
saving
for
cement
6.
Well.
G
Well,
you
know
be
useful
and
usable
for
a
very
long
time
because
it
really
covered
a
lot
of
bases.
You
know
I
think
back
to
the
heat
cmip5
for
the
CIA.
Only
after
like
six
fields
or
something
like
that
was
very
small,
but
for
this
time
around
they've
asked
for
considerably
more.
So
it's
much
more
useful
to
the
community
and
I've
gotten
a
number
of
requests.
Both
you
know
privately
and
through
the
bulletin
board
saying
you
know,
I've
completed
a
run
with
with
you
know
my
my
copy
of
CSM
where's
where's,
the
code.
G
G
G
Yeah,
just
just
those
things
and
I,
and
actually
on
top
of
that
I'd
also
like
to
see
better
metadata
coming
out
of
the
model,
because,
right
now
it's
it's
pretty
sparse,
but
that's
that's
the
secondary
issue.
Yeah
I
mean
just
needs
in
units.
Not
you
know
not
there,
the
file,
main
style
or
anything
like
that
just
needs
in
units
Thanks
yeah.
A
O
They
say
something
you
then
I
was
thinking
that
we
could.
There
are
people
I'm,
really
fine,
with
this
I
think
we
should
go
earth
and
move
with
this
I
know
that
that
people
that
are
very
reluctant
to
this,
because
it
they
will
have
to
have
the
update
or
the
script,
but
I
was
thinking
that
one
solution
could
be
that
we
keep
the
two
name
of
the
variable
and
they
would
have
the
possibility
to
do.
O
F
include
in
their
own
run
on
the
old
variable
the
day
if
it
would
not
be
like
if
we
do
run
that
like
lens
or
run
for
the
community,
but
at
least
they
would
have
the
option.
If
we
keep
the
name
the
old
name,
we
will
have
to
name
then
I,
don't
know,
what's
the
name
of
temperature
in
a
semi,
but
we
will
ft
like
the
owner
T
and
we
would
have
temperature
or
that's
it's
the
name
in
semi,
and
they
would
have
the
option
to
utter.
O
M
M
About
where
you're
able
to
write
one
of
the
other
you
know
like,
for
example,
I
went
looking
for
this
one
variable
in
the
atmosphere
and
the
C
map
name
was
CEO
WV
or
something
like
that.
Well,
that's
the
addition
of
the
total
ice
water
path
and
the
total
liquid
water
path
and
cam-
and
you
know
you
could
imagine
in
cam
you
want
to
look
at
those
two
things
separately,
so
so
I
think
it's
it's
not
just
as
simple
as
saying
we're.
Gonna
switch
over
to
the
cement
variables.
G
The
the
proposal
is
to
replace
CSM
variable
names
with
myth,
the
exact
myth,
equivalence,
where
appropriate
and
advocate
for
new
CF
standard
names
and
so
forth,
for
those
that
are
missing
and
also
units
a
match.
One
to
one,
for
example,
tam
dumps
out
precip
in
meters
per
second,
but
seemed
up
requires
kilograms
per
meter.
Squared
per
second
is
a
flux
unit
instead,
so
those
would
match.
Also
I
mean
it's
a
simple
conversion.
G
Some
others
are
not,
but
you
know
so
they
map
point
to
one
because
I've
been
through
this
exercise
of
translation
several
times
now,
and
it's
extremely
error-prone
and
tedious
and
difficult.
It's
it's
a
lot
of
work,
and
you
know
if
we
do
it
once
in
the
model,
then
we
don't
have
to
ever
have
to
do
that
again.
G
True,
but
for
I
mean
you
can
put,
you
can
put
you
know,
the
CF
standard
name
is
an
attribute
and
call
the
you
know,
call
the
field
whatever
you
want,
but
you
know,
we've
already
is
there's
already
a
pretty
much
a
de
facto
standard
coming
out
of
seeing
it
now,
for
you
know
a
number
of
iterations
of
what
the
short
name
for
a
variable
should
be.
You
know
kiosks
for
surface
temperature,
PA
ask
first,
you
know
temperature
at
two
meters
and
that
sort
of
thing.
G
F
D
K
F
I
was
referring
to
the
metadata.
So
if
you
had
open,
if
you
have
a
script,
that
only
has
the
CF
standard
names
in
it
and
you're
cracking
open
a
CSM
file
to
look
for
that
variable
you're
going
to
have
to
load
the
metadata
for
every
variable
to
find
the
real
variable
you're.
Looking
for,
because
you
can't
just
search
for
the
for
the
variable
name.
F
M
I
would
agree
with
that.
You
know.
The
thing
is
that
you
know
like
I
know
all
the
cement
sea-ice
variables
are
not
CF
compliant
names.
For
example,
sea-ice
concentration
is
s
icons
and
that's
not
the
CF
standard
name
for
sea-ice
concentration,
so
you
know
so
I
think
that
there's
there's
a
mixture
of
things
here
that
you
can't
just
assume
that
CF
standard
means
equal,
see,
map
names.
G
That's
interesting:
does
anybody
know
why
that
is
I
mean
why?
Why
are?
Why?
Are
there
differences
between
the
CF
standard
and
age?
The
standard
for
names
that
CF
doesn't
specify
like
Charlie,
said
yeah
CF
standard
doesn't
require
you
to
name
a
variable,
a
certain
short
name,
but
I
should
think
it
would
make
sense.
They
have
you
know.
Perhaps
EF
can
recommend
a
short
name,
but
we're
going
down
in
the
weeds
on
your
other
other
aspects
of
this
too.
L
Hey
Gary,
I
think
looking
at
the
seam
at
six
scatter
request,
I
think
only
a
handful
of
variables
actually
fell
into
that
category,
where
the
standard
name
didn't
match
the
short
name
so
for
the
most
part
they
do
match
and
the
story
behind
that
I
have
no
idea.
Maybe
someone
has
a
script
somewhere.
I
don't
know.
L
So
I'm
actually
curious
about
the
land
group.
You
know
we
didn't
really
hear
very
much
on
their
opinion
of
you
know,
switching
to
CF
standard
names
and
also
limiting
the
amount
of
output.
I
know
working
with
them
for
seem
f6.
You
know
they
actually
took
how
much
they
were
outputting
very
seriously
and
they
tried
to
reduce
it
as
much
as
possible,
including
you
know,
out
putting
things
on
just
one
grid
and
then
creating
functions
to
translate
between
those
grids.
G
B
I'm
not
too
worried
about
converting
our
scripts,
especially
if
there's
some
tool
to
help
and
we
have
gone
through
and
read,
reviews
our
output.
A
lot
part
of
it
is
going
towards
this.
The
thing
of
having
multiple
streams
of
output
not
exactly
by
default,
but
when
you
call
the
seam
it
outputs,
the
user-defined
mod,
which
is
still
a
bit
cumbersome.
B
But
when
we
do
use
that
we
can
get
all
the
output
and
particularly
there's
a
lot
of
things
that
are
only
required
it
like
annual
timescales,
which
allows
us
to
save
quite
at
this
space,
especially
for
3d
fields.
So
we're
all
laying
on
trying
to
keep
our
data
down
and
we've
done
most
of
what
we
can
do
that
we
can
always
circle
back
around
and
try
again
I.
B
G
Thanks
well,
I'm
gonna
jump
in
as
Gary
I
mean
we
also
have
just
released
to
the
high
res
nip
some
of
our
spectra
element,
data
which
we
have
never
released
before
so
we're
gonna
have
to
provide
some
tools
to
you
know:
do
interpolation
to
a
regular.
You
know
some
sort
of
Latin
language,
so
people
could
take
a
look
at
that
stuff,
and
so
that's
you
know
another
thing
that
could
go
into
our
cool
kid
that
you
know
aside
from
I
mean
an
CL
provides
some
of
that
functionality.
G
I
know,
but
I
think
you
know
we
should
be
able
to
provide
some
other
tools
that
you
know
maybe
are
a
little
more
flexible
or
faster,
or
something
like
that
for
people
who
want
to
look
at
these.
You
know
spectro
lawmakers,
especially
when
you
get
up
to
the
80/20
anyone
20
in
any
q40
I
mean
these
are
very
large
datasets.
So
we
need
some
really
good
tools
to
handle
this
I
think,
along
with
the
vector
output
from
Co
line.
That
is
that's.
G
M
I,
just
back
to
series
Queen
about
the
variable
request,
being
CF
standard
names,
I
guess
I'm
very
confused
here,
because
when
I
go
look
at
CF
standard
games,
it's
these
really
long.
Variable
names
like,
for
example,
sea-ice
area
fraction,
and
that's
not
what
I'm
providing
for
Seema.
But
it's
it's
s,
icons!
So
can
you
sort
of
explain
to
me
the
distinction
there,
her
I.
L
L
L
I
know
it
was
much
in
that
you
know.
Sometimes
it
makes
post-processing
a
little
bit
more
difficult
with
Python
tools.
You
know,
for
example,
like
the
ocean
you
know,
can
output
starting
here,
0
0,
0
0,
which
Python
complains
you
know
it
doesn't
exist
in
you
know,
isn't
valid
to
do
it.
This
is
a
problem
for
anybody
else.
L
D
D
Where
I'm
starting
up
as
a
well
actually,
not
even
that
any
run,
that
is
a
perpetual
year,
2000
run
where
I'm
starting
from
year,
zero
I'm
just
running
out,
so
that
will
open
up
unless
I
turn
that
off,
simply
because
I
forget
what
it
is,
but
x-ray
doesn't,
like
dates,
be
loved
beyond
hello,
1750
or
sixteen
I
figure.
What
the
date
is,
but
but
those
two
issues
have
one
of
two
issues
that
have
always
dealing
with
in
terms
of
tying
problems.
G
No
Danny
you're
exactly
right,
I
mean
this
is
one
of
the
issues
that
I
get
this
question.
A
lot
is
here,
I
open
up
an
8-0
file
and
it
says
it's
February
1st.
Why
does
it
say
that
it's
like?
No,
that's
the
January
average,
so
that's
yeah,
and
it's
this
off
by
one
thing.
Also,
that
I
think
needs
to
be
corrected
and
also
it
would
also
be
nice
to
have
all
the
component
models
standardized
on.
G
You
know
like
coordinate,
updated
data.
Like
you
know,
time
is
always
called
time
and
it
always
has
the
same
units
and
it's
its
metadata.
It's
always
the
same
across
components
is
in
some
components.
It's
like
time
underscore
bounds,
vou
NDS
other
ones.
It's
time,
others
4b
NBS.
Sometimes
it's
float.
Sometimes
it's
double
and
it's
you
know
that
should
be
something
I
think
that
across
all
the
components
should
be
exactly
the
same.
G
D
So
you
have
to
know
to
not
do
to
do
that
average
across
time
bounds
to
get
the
right
date
for
that
thing,
and
then
ask
it
to
do
the
seasonal
averaging
and
it
has
you
know
that,
and
that
can
automatically
be
paralyzed
very
very
quickly
because
you
can
just
open
up.
You
know
use
desk
to
do
a
lot
of
this
stuff,
but
you'll
get
the
wrong
answer
it.
If
you
don't
know
that
it,
you
know,
I
I
ran
into
this
momentum
meticulously.
D
He
sort
of
figure
I
calculated
it
by
hand
and
then
calculated
it
and
see.
It
was
wrong
and
then
had
to
shift
it.
So
people
will
get
will
be
publishing
the
wrong
figure
if
they
just
think
they're
grabbing
the
x-ray
stuff.
Oh
I'm
gonna
group
by
season
there's
my
DJ
fi
bridge
for
preset
and
it's
gonna
be
wrong.
A
D
D
That
that's,
where
the
debates
get
into,
if
you
have
a
colon
a
then
you
should
be
looking
at
time
balance.
You
shouldn't
be
looking
at
the
instantaneous
time
value,
but
it
throws
off
users
because
users
don't
appreciate
that
and
that's
why
Gary
gets
kinds
of
questions
over
the
years
and
in
Adam
and
others.
So.
L
A
question
might
be,
you
know,
I
mean
not
CDF
for
is
able
to
take.
You
know
multiple
time
dimensions,
but
obviously
we
use
currently
peanuts
CDF,
which
is
basically
not
cf3,
so
we
don't
have
that
available
to
us.
Could
those
just
be
outputted
it
as
a
different
stream
with
the
correct
timestamp
on
those
you
know?
Have
all
the
average
fields
in
one
stream
have
instantaneous
on
the
next
I
know
it
may
complicate
things
with
the
way
you
know
the
history
variables
are
specified.
L
Also
noticed,
too
that
we
only
have
about
five
minutes
left,
so
we
should
probably
you
know,
start
summarizing.
You
know,
and
also
you
know
briefly
mentioned-
may
be
passed
forward.
It
sounded
like
we
had
a
lot
of
traction
with
the
compression,
including
adding
you
know
it
within
a
workflow
script
itself.
You
know:
do
people
still
think
that's
a
good
idea.
L
G
Well,
wasn't
there
was
also
some
interest
in
you
know
toolkit,
for
example,
like
what
what
work
has
you
know
for
these
various
interesting
things
and
you
killing
things
in
utilities
and
people
I
could
do
with
CSM,
but
I
just
want
to
summarize
and
say:
I
thought
the
destruction
it
was
great.
I
mean
I,
went
off
in
directions,
I
wasn't
anticipating,
which
is
fantastic
and
I,
really
like
to
thank
everybody
for
for
chiming
in
and
letting
us
know
what
you
think.
Thanks.
L
G
And
everybody
that's
interested
in
the
next
workshop
for
the
model
data
earth
cube
thing.
Please
send
Doug
an
email
he's
interested
in
getting
interested
in
getting
as
many
people
as
possible
to
participate.
These
virtual
meetings
really
make
that
possible.
So
the
more
voices
we
hear
from
the
better
job
we'll
be
able
to
do
Thanks.
A
E
O
A
D
L
D
L
A
Right
you,
you
and
you
plural,
but
yeah
I-
think
actually
maybe
I'll
shoot
an
email
right
now
we
could
kind
of
discuss
next
steps
for
yeah,
where,
where
we
want
to
go
next,
to
make
sure
that
these
thoughts
aren't
just
dropped
in
the
momentum
is
kept
and
that
we
have
some
sort
of
action
items
that
come
out
of
this
and
people
assigned.
Even
if
I
say
no,
you.