►
From YouTube: 2020-03-06 - Zachar Ulissi - Intersections of AI/ML and Chemistry in Catalyst Design and Discovery
Description
NERSC Data Seminars: https://github.com/NERSC/data-seminars
Title: Intersections of AI/ML and Chemistry in Catalyst Design and Discovery
Abstract: Summary: Increasing computational sophistication and resources can enable a larger and more integrated role of theory in the discovery and understanding of new materials. This process has been slower to infiltrate surface science and catalysis than the field of bulk inorganic materials due to additional scientific complexity of modeling the interface. Most catalyst studies start in a data-poor regime where the material of interest is unrelated to previous to studies (new structure, composition etc) or the computational methods are incompatible with previous studies (different exchange-correlation functionals, methods, etc). Efficient methods to quickly define, schedule, and organize necessary simulations are thus important and enable the application of online design of experiments approaches. I will discuss on-going work and software development to enable data science methods in catalysis including open datasets for the community. These large datasets enable the use of graph convolutional models for surface properties and the uncertainty in these methods can be carefully calibrated. Finally, I will describe applications of our approach to ordered bimetallic alloy catalysts, with applications to several electrochemical catalyst discovery efforts including CO2 reduction, oxygen reduction, and water splitting chemistry.
A
Super
thanks
everyone
for
coming.
It's
a
pleasure
to
be
here,
as
some
of
you
I've
told
some
of
you,
my
wife's
in
the
so
it
works
in
the
Bay
Area.
So
we
come
back
every
few
months,
so
it's
been
awesome
being
able
to
work
with
Brendan
and
see
all
the
people
at
LBL
whenever
we
come
and
work
with
Mustafa.
A
Our
monthly
meetings
have
been
awesome
in
terms
of
suggestions,
so
I
just
wanted
to
roughly
introduce
some
of
the
things
that
we're
working
on
in
catalysis,
and
so
my
group
works
on
a
lot
of
different
application
areas,
mostly
on
various
types
of
chemistry
and
inorganic
materials,
but
also
some
other
materials
applications.
So
the
example
that
I
will
use
throughout
this
talk
will
be
selective.
A
Oxygen
evolution,
materials
and
oxide
chemistry,
selective
thermo,
catalysis
and
upgrading
water
desalination
remediation.
We
have
some
new
projects
on
looking
at
how
polymers
and
ionomer
setup
interfaces
and
how
we
study
effects
and
oxygen
diffusion
and
then
finally,
we've
been
working
with
some
collaborators
at
CMU
on
additive
manufacturing
for
high
entropy
alloys
and
also
high
entropy
alloys
for
catalysis,
I'm
sort
of
an
increasingly
interesting
area.
This
is
a
little
all
over
the
place,
but
what
ties
everything
together
is
how
we
think
about
the
problem.
As
Mustafa
said,
I
did
my
PhD
in
systems.
A
A
Maybe
an
order
in
a
metallic
that
you
get
from
the
materials
project
or
some
alloy
or
something
else,
the
things
that
we
often
care
about
the
properties
that
come
up
a
lot
are
similar
chemical
descriptors,
small
molecules
sitting
on
the
surfaces
tend
to
correlate
with
the
selectivity
and
the
activity.
These
thermochemical
descriptors
are
not
usually
available
in
the
inorganic
databases
that
are
currently
online
like
materials
project,
no
Tod
we're
also
interested
in
disability.
A
At
the
same
time-
and
it
becomes
a
little
hierarchical,
a
little
bit
how
those
things
fit
together.
So
an
ideal
scenario
is
we're
considering
different
materials,
and
we
want
to
find
all
of
the
bulk
materials
that
we
can
actually
synthesize
and
on.
All
of
those
I
want
to
say
what
are
the
actual
facets.
That
would
show
up
if
I
were
to
just
make
some
nanoparticles
and
then
I
want
those
facets
to
be
stable
under
reaction
conditions
and
I
want
them
to
be
active
and
selected
for
the
chemistry
and
then.
B
A
You
have
micro
kinetics.
You
can
also
make
some
predictions
on.
Maybe
what
is
the
optimal
temperature
or
pressure
to
be
running
out
at
the
same
time?
So
there's
a
lot
of
different
properties
that
we
need
to
be
able
to
calculate,
and
so
the
way
that
we
do
this
is
by
thinking
first
about
what
is
the
design
space
of
all
the
materials
were
interested
in
catalysis
is
super
broad,
there's
different
subsets
of
people
doing
metal
organic
frameworks,
oxides,
2d
materials
ordered
alloys,
like
intermetallics,
disordered,
solid
solutions,
each
of
those
sort
of
family.
A
You
have
to
work
very
closely
with
people
to
say
what
is
synthesizable
and
what
do
you
actually
need
to
worry
about?
As
a
systems
engineer,
we
first
developed
this
base
and
then
you
can
choose
anything
in
that
space
for
calculations
depending
on
the
properties
it
might
be
DFT
or
molecular
dynamics
or
whatever,
depending
on
how
much
entropy
we
need
to
consider,
and
we
wanted
things
like
adsorption
energies
or
stabilities
or
other
properties,
and
so
the
first
place
that
machines
I
mean
comes
into
this
as
I'm
sure.
A
lot.
A
A
The
interesting
problems
in
this
machine's
unenforceable
space
are
really
how
you
make
them
robust
and
usable
and
how
you
especially
start
them
with
small
data,
and
so
as
an
example.
This
is
some
work
that
one
of
my
students,
Mohammed
did.
This
is
a
little
toy
calculation.
We're
doing
of
CO
bouncing
around
on
a
copper
surface,
looks
pretty
good.
The
co
stays
together
the
top
particulars
around
a
little
bit.
A
A
Frequency
that
is
definitely
unphysical,
and
this
makes
it
really
really
complicated
these
things
and
an
online
framework,
because
you're
trying
to
sample
from
this
distribution
I
can
mean
I
can't
even
get
DFT
to
converge
in
those
places
right.
If
you
have
two
atoms
overlapping
you're,
not
even
gonna,
be
able
to
solve
the
electronic
structure
problem.
Even
if
you
do
solve
the
problem
and
you
had
the
data,
that's
probably
going
to
poison
your
data
set
and
those
aren't
things
that
you
actually
need
to
care
about.
A
A
Potential
doesn't
have
doesn't
have
a
continuous
increase
as
you
overlap,
and
so
this
is
what
and
what
we
found
is
incorporating
just
the
smallest
amount
of
physics
into
this,
adding
a
simple,
classical
potential.
At
the
very
beginning
that
says
there
are
some
things
like
vanderwaal's
or
Morse
potentials,
and
if
you
overlap
two
things,
you're
gonna
have
a
problem.
We
know
that
covalent
radii,
so
we
don't
even
have
to
like
really
fit
many
of
these
parameters.
A
Even
using
that
with
the
same
procedure,
we
get
much
better
convergence,
so
the
CEO
is
still
flying
off
the
surface,
but
the
copper
looks
very
reasonable
now
and
any
point
from
there.
I
could
actually
sample
and
add
back
to
the
training
dataset.
So
a
lot
of
the
work
here
is
not
in
building
new
potentials.
It
is
how
to
make
these
things
robust
and
stable,
and
the.
B
A
Instead
of
the
materials
that
I
was
originally
focusing
on,
and
so
the
second
obvious
place,
the
machine
then
it
comes
in
is,
if
I've
seen
enough
examples
of
things
in
the
design,
space
and
their
properties.
This
becomes
a
nonlinear
regression
problem
and
we
should
be
able
to
short-circuit
the
whole
thing
and
use
this
to
help
drive.
How
we're
selecting
the
next
data
points
that
sort
of
avoid
the
obvious
things.
There's
things
that
we
could
have
predicted
just
based
on
what
was
already
in
the
data
set,
so
as
an
example
for
co2
electrochemistry
at.
A
There's
a
lot
of
different
hydrocarbons
that
we
can
make.
This
is
also
hard
because
people
have
already
experimentally
brute
force,
the
problem
in
terms
of
single
cathode
composition,
and
so
there
was
an
awesome
set
of
furs
that
made
to
go
now
where
they
went
through
and
they
just
tried
making
the
cathodes
out
of
lots
of
different
possible
materials.
If
you
make
the
cathode
out
of
any
of
these,
you
bubble
off
a
lot
of
hydrogen,
which
is
good
for
making
hydrogen
but
bad
for
hydrogen
ends.
A
We
already
know
what
the
elements
are
going
to
do
and
an
obvious
place
to
start,
which
is
to
say
what
is
special
about
copper,
and
could
we
find
other
things
that
are
not
copper,
like
some
alloy
that
might
be
interesting
and
have
some
interesting
properties.
So
people
have
already
studied
co2
reduction
on
copper
surfaces.
This
is.
A
Going
to
paint
a
very
simple
pathway
may
see
a
few
hydrogenated
I
drug
name
again
use
water,
I've
gone
to
me.
Hydrogenate
eventually
get
a
methane
off,
but
let's
only
simple
pathway
when
you
do
the
kinetics.
What
you
see
is
that
this
conc,
a
chode
step,
is
usually
the
rate
limiting
one,
and
so
this.
A
These
two
intermediates
are
interesting,
and
if
you
really
examine
this
transition
state,
then
it
is
in
CH
o,
and
so
this
really
implicates
the
CEO
adsorption
energy
and
the
overall
kinetics
of
this
pathway.
So
an
obvious
thing
to
do
is
to
say:
does
Co
adsorption
correlate
with
what
people
know,
experimentally
on
co2
reductions,
so.
A
A
A
There
are
people
who
really
like
doing
synthesis
of
these
materials,
and
so
you
can
ask
someone
to
actually
gonna
make
this
specific
map
of
gallium
intermetallic.
It's
a
very
nice
space
of
compounds
to
play
around
with
and
again
because
the
materials
project,
people
and
opium
be
1/8
would
have
already
done
such
an
awesome
job
of
collecting
different
materials
and
databasing
em
and
saying
what
is
pretty
to
be
stable.
We
don't
have
to
solve
that
problem
right.
We
already
know
we
have
some
ideas
of
what
is
feasible.
A
So
what
is
different
from
the
bulk
material
side
is
just
how
complex
this
gets
from.
We
consider
surface
chemistry,
so
if
I
pull
all
of
the
stable
things
or
nuclei
materials
project,
those
elements
I
get
about.
Ten
thousand
different
ball
crystal
structures,
maybe
a
little
bit
more,
depending
on
what
your
limits
of
stability
are
and
if
I
consider
different
low
index
facets
a
lot
of
these
intermetallics
missing,
most
of
the
symmetry
that
we
think
about
in
solid
single
component
metals.
We
have
200,000
different
surfaces,
so
that's
already
up
by
about
factor
of
20.
A
And
I'll
talk
a
little
bit
about
that
at
the
end,
what
we
want
is
the
global
minima,
the
muscular
position
and
rotational
degrees
of
freedom
with
respect
to
the
energy
right.
We
wanted
to
optimize
that,
and
so
one
way
you
can
do
this
is
you
pick
a
surface
and
you
run
some
global
optimization
method
like
Basin
hopping
as
long
as
you
can,
until
you
find
the
absolute
best
one,
the
other
approach
is
you
consider
all
the
possibilities
and
make
your
prediction
for
all
of
them
and
then
find
the
minimum
from
those
I
am
reasonably
confident.
A
A
Okay,
so
the
approach
I've
taken
I
would
say,
is
I,
definitely
influenced
by
what
I've
seen
from
the
materials
project
I
knew
on.
Above
when
we
were,
on
the
same
view,
fellowship
I,
remember
seeing
all
of
his
presentations
on
the
workflows
that
he
was
building
to
make
those
things
possible.
So
I
will
just
talk
a
little
bit
about
how
we've
been
solving
some
of
these
things
for
surfer
science,
otherwise
I
think
the
stuff
we've
been
doing
is
very
similar.
A
One
difference
is
right
from
the
beginning.
We
started
with
the
idea
that
we
would
have
to
use
active
learning.
We
weren't
going
to
be
able
to
generate
this
huge
data
set.
Okay,
so
surface
chemistry
involves
a
lot
of
different
steps.
Just
like
material
properties.
We
need
to
do
a
lattice
relaxation.
If
our
calculation
settings
are
a
little
bit
different
from
the
materials
projects
to
get
the
lattice
constant
correct,
we
then
need
to
cut
the
surface
and
do
a
relaxation.
We
need
to
analyze
the
surface
and
say
where
can
I
put
in
that
zero
bait.
A
Structure
the
blank
one
for
over
the
hydrogen
is
so
relatively
straightforward.
So
the
way
that
we've
been
doing
this
is
with
another
layer
on
top
of
fireworks,
to
basically
orchestrate
these
dependencies,
the
one
we're
using
Spotify
it
has
some
nice
properties,
I.
Think
in
terms
of
how
you
define
these
individual
nodes,
but
also
a
lot
of
similarities
with
how
things
happen
with
automating
other
ones.
It's
a
little
bit
different
and
that
everything
works
backwards.
You
don't
have
to
have
a
fixed
workflow
system.
A
It
is
every
one
of
the
rules
you
define
as
a
set
of
three
Python
functions.
That
first
says:
if
I
want
to
run
this
task,
I
do
I'd
say
what
this
needs
and,
if
I
have
all
my
crew
as
prerequisites,
what
do
I
do
to
run
and
finally,
what
am
I
going
to
produce
that
someone
can
check
at
the
very
end
if
I
ran
or
not,
and
so
you
write
these
three
little
independent
steps
and
then
you
ask
something
at.
A
It
traverses
backwards
and
refined
the
calculations
that
need
to
run
but
again
in
terms
of
workload,
is
very
similar.
This
is
all
open
source
if
you
have
questions,
feel
free
to
ping
me
on
flank.
Okay.
So
when
we
started
this
there
weren't
really
large
surface
science.
Data
sets
available
for
these
descriptors.
So.
B
A
Thing
that
I
wanted
to
do
is
predict
CO
and
hydrogen
adsorption
energies
across
certain
organic
materials,
and
the
data
sets
were
just
really
really
really
small,
and
so
what
we've
done
is
across
different
materials
and
surfaces
and
structures.
We've
also
done
cleavage,
energy
or
stability
calculations
for
a
lot
of
these
inorganic
materials,
because
I
really
need
to
know
this
for
nanoparticles
and
the
materials
project
had
those
four
pure
elements,
but
it's
an
additional
challenge
to
do
it
for
by
metallics,
and
both
of
these
are
fully
automated.
A
If
you
want,
the
data
sets
they're
on
github,
alongside
all
the
papers
or
email
me
and
I'm
happy
to
give
them
to
you
or
email
Brandon,
he
has
most
of
them
as
well.
Okay,
so
we
have
surface
science
datasets.
Now
that
help
us
to
answer
these
questions
in
catalysis.
Those
didn't
exist
before
the
next
thing
is.
We
need
to
be
building
better
models
for
things
like
CO,
especially.
A
A
A
B
B
A
On
the
understanding
what's
happening,
so
other
people,
obviously
recognizing
this
is
a
problem,
and
there
was
a
great
sequence
of
papers
in
developing
coordination
based
approaches
and
things
like
the
generalized
coordination
number
to
explain
how
the
activity
changes
as
a
function
of
how
under
coordinated
the
surface,
Adams
work.
This
worked
awesome
if
you
have
a
single
metal.
So
if
you're
just
dealing
with
platinum
and
you're
trying
to
figure
out
what
is
the
activity
of
a
structured,
platinum
interface,
these
things
work
great.
A
A
Of
surface
descriptors
and
focus
on
those,
and
so
that's
the
approach
that
we've
been
taking
so
all
of
this
is
hinge
on
the
idea
that
you
can
take
a
atomic
structure
that
were
interested
in
and
turn
it
into
a
graph.
This
was
motivated
by
the
Grossmont
paper
that
came
out
two
years
ago:
the
CG
CNN
original
one.
That
was
the
first
one
for
energetic
materials.
We
saw
it,
we
thought
it
looked
interesting.
A
It
wasn't
clear
if
it
was
gonna
work
for
surface
chemistry,
so
in
der
graaf,
every
edge
is
a
bond
and
you
got
some
bonding
properties.
Every
node
is
an
atom
and
you
also
have
some
atomic
properties
that
you
think
are
going
to
be
correlated
like
atomic
radius
in
electronics
and
number
of
electrons,
and
whatever
else
you
think
is
interesting.
A
This
is
the
work
that
the
estep
program
is
helping
with
now
and
brandon
has
been
helping
with
he's,
been
working
on
building
optimization
tools
to
identify
how
we
build
these
models
and
tune
them,
which
I
think
is
going
to
be
really
helpful,
moving
forward.
Okay,
so
we
have
this
graph
and
we
need
to
turn
this
into
a
into
a
vector
problem,
standard,
non,
linear
regression,
one
and
so
way
that
you
can
do
that
is
with
graph
convolutions,
which
are
a
new
class
of
methods
in
machine
learning.
A
B
A
What
has
been
really
interesting
to
me
is
that
my
assumption
of
most
of
these
is
that
the
problem
is
usually
already
well
solved
on
the
computer
science
side,
because
usually
we're
about
ten
years
behind
them
in
terms
of
like
what
they
think
is
interesting.
These
graphs
convolution
models
are
very,
very
topical,
and
so
there's
an
entire
library.
Now
at
different
convolution
operations
and
pie,
charts
people
are
trying
all
sorts
of
different
things.
A
The
one
that
Grossman
tried
out
two
years
ago,
I
would
say,
is
embarrassingly
simple,
but
works
really
well,
there's
an
obvious
question
of
what
other
convolution
operation
should
we
be
trying,
and
how
else
should
we
be
optimizing?
This
architecture
uncertainty
in
these
models
is
not
clear.
If
you
talk
to
the
computer,
scientists
I
haven't
anyone
who
says
that
they
think
this
is
an
easy
problem
right
now.
Generative
models
for
grabs,
a
super,
topical
optimization
of
these
graph
structures
after
the
fact
to
try
and
improve
some
property,
also
a
very
topical
computer
science
problem,
and
so.
B
A
A
A
A
One
we
have
lost,
and
so
the
major
limitation
right
now
and
I
think
the
thing
that's
gonna
change
is
the
way
that
people
are
building
these
models
right
now
for
the
most
part,
but
not
all,
is
you
have
some
fun
features,
but
you
don't
really
have
anything
that
says
what
are
the
angles
or
where
exactly
are
those
atoms
nearby?
There's
one
one
method
called
Schneck
by
cross
rubber,
Mueller's
group
and
others
in
Germany.
A
That
looks
like
this:
it's
capable
of
getting
energies
and
forces,
but
the
atom
features
doesn't
scale
very
well
with
the
number
of
elements.
So,
if
I
ask
this
one,
what
is
the
derivative
of
the
energy
moving
one
of
these
little
atoms?
I?
Don't
think
it's
going
to
get
the
right
answer,
and
so,
if
I
had
to
guess,
the
major
thing
that
needs
to
come
is
incorporating
the
right
symmetry
functions
or
the
right
properties
or
radial
basis
sets
or
whatever
you
want
to
call
them
such
that.
A
Different
elements
interact
and
we
maintain
the
properties
that
allow
Co,
energies
and
forces
right
now
that
the
field
is
sort
of
bifurcated
and
machine
learning.
Force
field
said
don't
scale
with
elements
and
graph
convolutions
that
don't
do
energy
and
forces,
but
there's
no
fundamental
reason
why
those
won't
come
together.
I
think
that's
going
to
happen.
A
I
mean
for
surface
properties,
we've
been
keeping
things
small
just
because
DFT
is
slow,
but
a
very
typical
number
is
50
atoms,
and
this
is
a
sixth
graph.
Things
go
off
the
edge
went
back
around,
but
if
I
had
my
way
and
I
give
you
defects
or
grain
boundaries,
I
can
make
this
thing
as
because
I
want.
The
only
reason
we've
been
keeping
them
to
sixty
is
because
I
can
get
that
to
work
in
20
hours.
Harmonic
computer,
but
I
wish
I
could
do
more
yeah.
A
These
are
small
compared
to
the
computer
scientists
the
best.
Sometimes
they
talk
about
it
like
two
billion
people
on
Facebook
and
how
they
are
all
liking,
each
other
right.
That's
a
very
different
problem.
Okay,
so
this
is.
This
is
moving
really
quickly.
In
catalysis,
we
were
the
first
ones
to
do
this
for
absorption
energies,
but
a
lot
of
other
people
were
trying
different
variations.
Now
this
is
picking
up.
A
A
Atomic
contributions,
we
can
go
back
and
ask
what
is
the
contribution
of
each
atom
to
this
property,
and
so,
if
it's,
you
know
sitting
on
top
of
the
copper,
the
copper
lights
up,
they've
been
sitting
in
this
bridge.
If
you
light
up,
but
the
ones
nearby
also
seem
to
matter,
I
would
not
say
this
is
like
super
transparent,
but
at
least
we
can
tell
why
it
is
making
the
prediction
it
is
and
go
back
and
look
at
that
individual
atom.
A
Okay,
I
really
want
to
stress
that
the
accuracy
that
we're
showing
here
I
would
say
is
the
best
that
people
have
shown
across
such
a
wide
range
of
materials
and
surfaces,
but
I,
don't
actually
think
it
was
due
to
the
magic
of
the
model.
I.
Think
really
what
happened
was
we
were
the
first
ones
to
push
the
data
set
size
large
and
then
put
it
off
with
that.
So
this
was
a
people
from
the
paper.
A
Most
of
these
are
in
like
100
to
900
calculation
dataset
range,
and
these
are
PhD
students
sitting
down
and
doing
a
bunch
of
calculations
on
different
metal
surfaces.
Not
automated
I
do
put
this.
One
is
all
single
element
compositions
so
large,
but
not
particularly
diverse
ours
was
another
factor
of
40
over
those
and
in
the
past
year
there's
been
another
one
or
two
data
sets
that
have
come
out
larger
at
the
50
100,000
dataset
size,
I'm
Regine.
A
So
there
is
a
transition
happening
in
catalysis
right
now
from
really
small
data
problems
to
I,
don't
want
you
want
to
call
it
medium
sized
data.
Maybe
we
are
behind
the
small
molecule
people
in
the
inorganic
bulk
materials.
People
who
have
had
large
data
sets
to
train
on
for
much
longer
period
of
time.
So,
if
you
want
to
do
things
in
this
area,
you
either
have
to
wait
for
someone
to
generate
the
data
or
you
have
to
take
ownership
and
say
I'm
just
going
to
go
ahead
and
do
this
and
generate
the
data
I
need.
A
A
And
you
do
the
cut
you
end
up
with
an
asymmetric
top
and
bottom,
and
the
cleavage
energy
is
basically
the
average
of
those
two
surface
energies
if
I
am
interested
in
what
the
shape
of
the
nanoparticle
looks
like
really,
what
I
need
is
the
individual
surface
energy
of
each
size.
Some
of
these
things
are
so
unstable.
We
can
still
use
this.
B
A
Gain
idea
which
low
index
facets
might
be
interesting,
but
we
are
starting
to
get
into
the
point
where
this
is
really
a
material
scientists
problem
that
I
wish
someone
else
would
just
solve.
So
if
you
have
ideas,
please
please
let
me
know
we're
trying,
but
this
is
really
hard
anyways.
We
can
do
calculations.
We
can
collect
this,
these
cleavage
energies,
the
standard
procedure.
A
Is
you
extrapolate
back
to
zero
thickness
and
you
get
this
image
energy?
Basically,
the
same
see
you
CNN
model
that
worked
for
bulk,
inorganic
materials
also
seems
to
work
for
surfaces
with
just
a
little
bit
of
tweaking.
That
was
very
interesting
if
I
take
those
cleavage,
energies
and
I
just
blindly
assume
that
they
are
surface
energies
and
put
them
into
this
little
nanoparticle
construction
tool
deity.
These
are
three
different,
crystal
structures
that
I
had
cared
about.
Four
different
chemistry's
in
the
past.
A
Dfg
says
this
didn't
guy
in
was
all
one
one
now,
and
the
machine
learning
model
agrees
that
one
is
easy,
that
one's
actually
symmetric,
reclose
upper
gold,
we're
pretty
close,
is
just
an
additional
facet
here.
This
seems
to
be
good
enough
at
the
level
of
if
I'm
thinking
about
a
hypothetical
inorganic
surface
should
I
consider
for
catalyst
activity
or
not.
Is
there
some
reasonable
chance
that,
if
someone
mixes
these
things
together
in
form
to
nanoparticles,
am
I
going
to
get
any
surface?
It
looks
like
that.
A
Okay,
I
mentioned
that
we
can
predict
things,
but
I
haven't
talked
at
all
about
uncertainty
and
you
need
an
uncertainty
if
you're
going
to
do
anything
halfway
optimal
in
terms
of
how
you
choosing
the
next
data
point
any
sort
of
active
design
of
experiments
process,
and
so
it
is
clear
on
a
parity
plot.
We
want
things
to
be
accurate,
close
to
the
30
line
inaccurate
far
away
their
property
is
obvious.
A
This
is
just
accepted
this
money,
so
we
can
take
something
that
looks
like
one
of
these
simple
CG
CNN
models
and
we
can
predict
in
energy
and
the
approach
that
you
see
over
and
over
in
literature.
Is
you
train
the
same
model
five
times
against
different
subsets
of
the
data
on
bootstrap
and
look
at
the
end
on
pole
and
calculate
the
standard
deviation
and
use
that
to
give
you
some
idea
of
how
you
do
it?
And
so
that's
the
easiest
thing.
A
A
B
A
A
calibrated
model
that
does
not
actually
tell
you
anything
about
what
the
data
is
doing
by
just
making
by
fitting.
Basically,
this
special
standard
deviation,
and
so
you
need
to
incorporate
other
things
into
this
light.
What
is
the
dispersion
of
the
uncertainty?
The
dispersion
is
an
idea
of
how
wide
is
the
distribution
you
want
that
to
be
really
wide,
because
that's
telling
you
that
some
data
has
lower
uncertainty
than
others
and
I
want
to
know
which
ones
I
trust
and
don't
trust
this
one.
A
A
You
use
a
second
model
that
has
an
easier
fingerprints
and
use
a
credit
uncertainty.
You
use
these
easier
fingerprints
that
were
using
before
and
just
build
a
Gaussian
process
which
gives
you
uncertainty
by
default.
We
also
include
a
dropout
as
another
one,
the
one
that
ended
up
doing
the
best
for
this
adsorption
energy
dataset.
A
Is
this
thing
that
we
were
calling
emt-p
reach
basically
says
we're
gonna,
fit
the
model
to
energies
and
then
afterwards,
we're
gonna
go
and
the
final
evolution
there
and
feed
that
directly
into
a
Gaussian
process,
so
we're
basically
using
CGC
and
then
to
come
up
with
the
better
set
of
fingerprints
and
then
afterwards
we
do
a
very
standard,
Gaussian
process
approach
afterwards.
So
all
of
the
statistics
and
data
sets
are
in
there.
If
you're
interested
in
going
look
at
the
paper
or
shoot
me
an
email.
B
B
A
A
A
The
demands
for
that
the
time
scale
is
a
little
bit
shorter,
and
so
we
tend
to
get
a
model
that
works
and
uncertainty
method
that
works
and
run
with
it.
But
if
you
want
to
go
back
to
make
things
better,
I
think
you
have
to
ask
that
question:
yeah,
okay,
cool!
So
we
have
models
and
we
have
uncertainties,
and
so
the
last
thing
is
we
can
do
design
of
experiments.
So,
as
I
said,
this
people
with
uncertainties
just
came
out,
so
we
are
working
now
on
doing
more
rigorous
goe.
A
A
A
A
A
Don't
really
have
enough
trust
in
these
calculations
to
say
like
if
my
model
says
one
crystal
is
going
to
be
a
little
bit
better
than
the
other
there's
so
many
other
things
going
on
in
the
experiments
that
I,
don't
actually
know
what's
happened,
but
I
want
to
give
them
a
list
of
things
that
I
think
there's
something
interesting
will
happen
and
make
sure
that
I
exclude
everything.
That
is
obviously
really
bad,
and
so
this
is
a
level
set
estimation
problem.
A
So
this
is
what
we're
working
on
now
we
don't
have
results,
but
if
you
have
uncertainty,
you
can
combine
these
things
together
in
a
rigorous
fashion.
Okay,
we
published
a
paper
two
years
ago.
We
were
basically
addressed
this
problem
with
100%
heuristics,
just
randomly
trying
stuff
or
just
randomly
choosing
sites
that
were
the
best.
It
was
not
even
close
to
being
optimal
in
the
data,
but.
A
And
about
half
of
them
have
no
copper
whatsoever,
so
those
are
maybe
the
even
more
interesting
ones
to
follow
up
on.
We
had
different
materials
that
looked
good.
We
had
about
another
1,500
that
were
predicted
to
be
good,
with
the
models
that
we
had,
but
we
didn't
have
DFT
calculations
to
verify,
but
if
we
wanted
to
go
back
and
check,
hopefully
some
of
those
would
also
turn
out
to
be
interesting
and
again.
A
To
say,
I
don't
need
to
consider
this
entirely,
and
so
it's
important
to
sort
of
keep
your
eye
on
the
ball
in
terms
of
the
actual
engineering,
a
decision
that's
getting
made
with
choosing
these
materials,
so
we
can
go
and
analyze
different
combinations
and
the
upper
right.
These
are
things
that
we
confirm
with
DFT
and
the
bottom-left.
A
A
This
was
something
that
the
screen
that
I
spent
a
better
part
of
a
year
as
a
postdoc,
doing
calculations
by
hand
trying
to
figure
out
why
it
was
interesting
for
co2
production,
and
so
it
was
at
least
a
little
comforting
that
it
was
able
to
pick
up
on
that's
cific,
nickel,
gallium
one
or
no
facet.
That
I
was
able
to
find
as
a
student
do
anything
right
now.
A
A
These
islands,
for
example,
on
these
two
islands
that
are
all
copper
aluminum
and
you
want
things
to
be
in
the
firm,
already
arranged.
It's
opening
to
be
white.
It's
time
to
be
back,
and
so
these
islands
basically
say
no
matter
how
you
make
a
copper,
aluminum,
alloy,
no
matter
what
that's,
what
you
choose,
all
the
sites.
Look
interesting
right,
there's
no
site
on
there.
That
is
going
to
be
so
strong
that
a
poisonous
circus,
and
so,
if
I,
had
to
do
an
experiment
with
operator
and
say
why?
Don't
you
try
and
make
something?
A
A
We
took
this
list
to
Ted
sergeant's
group
at
the
University
of
Toronto,
and
they
tried
a
bunch
of
these
and
I
think
we
got
a
little
bit
lucky
on
this
copper
aluminum
turned
out
to
be
very
active
and
very
selective
for
making
ethylene
I
think
we
were
lucky
because
nothing
in
the
models
I
showed.
You
said
that
we
should
make
ethylene
all
I
can
say
is
that
it's
probably
going
to
make
some
c1
and
c2
product
and
not
make
hydrogen
I
wish
I
could
predict
that
selectivity.
B
A
About
80
percent,
it's
not
quite
where
you
need
for
an
industrial
process,
because
you
still
have
a
separations
problem
afterwards,
but
this
is
starting
to
get
close
to
the
point
where
you
can
think
about
economics,
and
it
might
not
be
too
unfavorable,
so
this
will
hopefully
be
out
this
year,
hopefully
soon
been
dragging
on
for
a
long
time.
Okay,
we
can
do
the
same
thing
for
oxides.
A
There's
another
set
of
challenge
for
these
oxide
materials
and
there's
many
different
polymorphs,
some
of
which
are
in
the
databases
like
oak
cumbia
materials
project,
some
of
which
have
not
been
considered
and
there's
an
additional
challenge
in
because
they're
oxides
and
we're
usually
considering
oxygen
chemistry
or
were
there's
a
bunch
of
water
around
there's
a
lot
of
different
terminations
that
you
can
have.
Also
it
can
be
there
with
a
bunch
of
coordinating
metals
exposed
metals.
You
can
be
opposite
in
terminating.
You
can
be.
B
A
Terminated
which
one
of
those
you
get
depends
on
where
you
aren't
under
reaction
conditions
and
which
one's
the
most
stable.
So
you
have
to
be
able
to
consider
this
additional
degree
of
complexity
if
you
want
to
move
to
oxide
materials
and
that's
on
top
of
all
the
complications
on,
how
do
you
even
model
oxide
electronic
structure,
so
this
gets
really
really
complicated.
This.
A
We
showed
that
similar
graphic
convolution
methods
can
help
us
predict
what
is
the
stable
determination
and
what
is
the
best
site
or
the
energy
of
oxygen
or
ovo
h,
I
different
places
and
we
were
able
to
rationalize
why
different
titanium
oxide
polymorphs
can
have
different
activities.
So
one
of
the
ones
that
we
found.
A
It
was
a
lot
like
one
of
the
ones
that
was
implicated
for
an
iro
3
system
after
you
leached
out
some
of
the
materials
that
was
an
experimental
paper
from
Stanford
again.
This
is
like
another
level
of
complexity
on
top
of
everything
else,
if
you're
rigorously
gonna
say
I
want
to
discover
mixed
metal
oxide
catalysts
or
mixed
metal
proselytes
or
whatever
else
you
want
to
do,
you
have
to
sort
thinking
about
these
coverages,
and
so
this
is
just
where
the
field
needs
to
move
we're
trying
to
take
things
one
step
at
a
time.
A
We
want
to
be
able
to
do
multi
objective,
optimization
at
the
same
time,
either
some
free
to
optimal
solutions
or
some
sort
of
qualitative
constraints
or
trade-offs
working
with
experimentalist
to
come
up
with
new
constraints.
There
are
some
people
at
CMU
who
have
high
Cooper
chemistry
systems
that
we
can
compare
against,
so
they're
doing
I
think
will
hopefully
be
interesting,
better
structural
property
relationships
and
the
datasets
that
feed
them.
Everything.
A
About
is
the
same
two
simple
co2
and
hydrogen
descriptors,
or
oxygen
or
whatever,
but
we
need
to
be
able
to
scale
across
chemistry.
I'm
gonna
predict
if
I
make
ethylene
ethanol,
and
so
that's
the
next
big
challenge.
And
finally,
we
want
to
incorporate
more
detailed
models
in
calculations
in
the
discovery
process
and
by
this
I
mean
I.
A
Don't
just
want
to
do
a
calculation
of
a
pair
Co
sitting
on
a
surface
I
want
to
have
solvent
effects
or
cation
effects,
or
something
else
that
comes
from
it,
because
it's
salvation
model
or
a
hybrid
model
or
whatever
you
want
to
call
it.
There
were
this
physics
that
we
know
we're
missing
and
we
need
to
add
those
things
in
okay,
so
with
that
I
just
need
to
thank
a
lot
of
people
who
have
helped
specifically
I
wanted
to
point
out
Brandon,
because
he's
here
and
he's
doing
awesome
was
the
first
postdoc
working
with
me.
A
A
A
summer
internship
for
summer
research
project
somewhere
in
the
field
of
Informatics
and
materials
or
catalysis,
or
something
so
we're
looking
for
position
for
her
just
graduated
from
CMU.
He
has
a
green
card.
He's
awesome,
he's
leading
a
project
on
additive
manufacturing
with
high
entropy
alloys
and
he's
looking
for
a
one
year
position.
This
sort
of
rigid
PhD
I
keep
on
telling
him.
He
should
just
do
a
PhD,
but
he
won't
listen
to
me
and
so
he's
interested
in
something
like
this
he's
been
applying
the
places
like
Oak
Ridge
in
Vienna.
A
We
do
is
only
possible
because
of
the
awesome
things
that
were
done
for
materials,
project
and
Pine
mention
the
pathogen
we're
closed
for
surface
chemistry
have
gotten
really
really
really
good
and
I'm
really
happy
with
all
those
work.
New
York
has
been
amazing.
All
of
the
workflows
we
use
run
at
nurse.
The
database
is
run
there.
If
Wahid
was
charging
me
they're
going
rate
for
commercial
rates
for
databases,
I
would
be
broke
two
years
ago.
That's
really
been
phenomenal.
Super
super
super
helpful
lots
of
different
funding
sources,
most
dearly,
but
also
other
people.
B
A
Are
asking
all
the
right
questions?
Okay,
so
let's
go
back
to
the
CEO
mechanism.
There
is
a
field,
a
subfield,
but
there's
maybe
three
different
ways
of
handling
this
problem.
One
is:
is
that
I
really
really
care
about
and
I
am
going
to
try
and
build
the
right
reaction.
Networking
come
with
the
right
mechanism
for
that
surface,
so
there
is
a
there
is
a
group
of
people
who
do
nothing
but
think
about
you,
enumerate
reactions
and
then
find
one
of
the
ones
are
rate-limiting
and
then
focus
your
calculations
and
bidding
on
those
things.
A
You
can
switch
between
mechanisms
that
is
like
the
intermediate
one
that
works
well
for
rationalizing
pure
metals,
but
is
super
complicated.
You
do
combined
metallics
right
now.
No
one
can
handle
the
question
you're
asking
across
anything
that's
more
complicated
at
a
funeral
circus
without
doing
a
lot
of
stuff
by
hand.
What
we
need
is.
We
need,
first
of
all,
structural
property
relationships
that
make
this
process
stocks
for
an
arbitrary
surface,
so
that
it's
not
a
three-year
PhD
to
come
up
with
the
mechanism,
but
it
is
like
an
automated
process.
A
There
is
a
group
at
Sandia,
National
Labs
that
is
trying
to
do
that.
This
is
open
source
code
called
orangey
cat
coming
from
the
homogeneous,
catalysis
literature
or
the
gas
base
combustion
literature.
It's
still
super
simple,
but
I
think
they're
the
right
people
looking
at
the
problem.
I
did
a
lot
of
work
on
reaction
mechanism
enumeration
for
selective
chemistry's
I
haven't
had
any
of
my
students
do
that
because
every
one
of
these
steps
is
like
three
weeks
for
students
to
run
and
I
don't
want.
B
A
That
cost
lots
of
different
possible
intermediates
in
structures
and
we
could
actually
build
the
structure,
property
relationship.
I
would
also
return
to
this
problem
right
now,
there's
not
a
price,
and
you
didn't
even
ask
the
real
question
of
I
made
it
something
here
on
which
transition
state
scaling.
You
choose
it's
on
a
step
surface.
It
is
more
active
or
higher
rates
than
on
a
flat
surface,
and
so
there's
not
even
anything
like.
If
you
have
a
structured
bimetallic,
it's
not
super
clear
what
what
scaling
relation
should
apply
to
each
site.
B
A
Are
so
many
interesting
questions
here
so,
if
you're
doing
a
chiral
surface
chemistry
where
your
molecules
are
chiral,
not
all
copper,
things
are
chiral,
but
copper,
a
really
nice
symmetric,
pure
metal.
You
can
actually
get
a
chiral
cut
so,
for
example,
6
4,
3
or
3
17,
1,
alright,
to
Miller
index
Miller
indices
for
upper
surfaces.
A
The
the
chirality
effects
are
strong
enough
that
the
copper
will
actually
rearrange
to
form
those
specific
chiral
cirrhosis.
I,
don't
know
how
you
come
up
with
371.
The
guy
who
works
next
to
me
makes
these
like
polish
crystals
and
does
these
experiments
and
yeah,
but
it
is
definitely
possible
for
most
of
these
simpler
things.
I,
don't
think
it's
a
problem
by
a
good
Naaman
yeah.
It's
super
interesting.
If
you
have
someone.
A
It
works,
usually
pretty
okay,
I
think
we're
wasting
cycles
doing
this
I
have
a
student
working
on
trying
to
predict
relaxed
structures
after
you've
done
enough
of
these
with
that
stations,
but
I
haven't
seen
any
more
like
Joe
a
model
that
looks
like
that
yeah.
But
when
I'm
like
dream
scenario,
someone
has
some
machine,
only
method
that
looks
at
the
initial
structure
and
gives
me
a
relaxed
structure.
That
is
like
three
single
points
away
from
the
real
thing.