►
From YouTube: DevoWorm (2023, Meeting #10): DevoLearn roadmap, GSoC, local/global ABMs, analyzing butterfly wings
Description
DevoLearn roadmap (semantic segmentation, instance segmentation, graphical representations, and source data). Implementing multiscale biological Agent-based Models (ABMs) using local vs. global criteria. Identifying butterfly wing spots using CNNs and implications for models of morphogenesis. Attendees: Richard Gordon, Sushmanth Reddy Mereddy, Jesse Parent, and Bradly Alicea
A
Wait
for
some
more
people
to
show
up,
so
anyone
have
any
any
updates
or
anything
they'd
like
to
give
or
hello
good
morning
morning.
How
are
you
not
sleepy
yeah.
A
All
right
so
do
we
have
any
updates
anything
before
we
get
started.
B
A
A
B
Okay,
things
like
our
k,
m,
identical:
okay,
yeah,
all
of
Usher's
equations
to
K,
okay,
okay
and
the
other
question
you
asked.
If
you
had
mixed
proportional
distributions
yeah-
and
you
said
you-
that's
unique
with
you,
but
I
thought
it
might
be
in
the
Radiology.
Oh.
B
No,
he
he
well,
he
fixed
them.
Okay,
he
fixed
them
because
somehow,
when
they
went
through
your
machine,
he
got
screwed
up,
I,
fixed
them,
and
then
he
double
checked.
Okay,
okay,
yeah.
A
All
right,
well,
I'll,
take
a
look
yeah,
we'll
talk
about
that.
If
you
want
to
talk
about
that
later,
we
can
yeah.
A
A
C
B
C
A
So
let
me
explain:
let's
see
if
I
can
share
my
screen,
I'll
just
draw
it
out
and
we'll
talk
about
it,
but.
A
So
we've
been
wanting
to
do
a
lot
of
different
things
here
with
the
software.
Eventually,
you
know,
one
thing
would
be
like
to
get
single
cells
and
segment.
Those
would
be
like
if
you
have
a
lineage
tree
like
we
have
in
say
elegans,
where
it's
well
defined
for
each
cell
to.
A
It
then
we
have,
you
know
like
tissues
and
other
types
of
things
that
you
can
extract
from
embryos
that
are
more
developed,
like
you
know
that
have
tissues
and
things
and
be
able
to
take
those
and
use
semantic
segmentation
to
segment
out
parts
of
the
tissue,
and
you
know
things
like
that,
so
we
want
to
do
things
in
a
number
of
different
levels,
but
one
thing
would
be
nice
if
we
could,
because
now
we're
kind
of
you
know
diversifying
how
we're
doing
the
evil
learn
aspect
is
to
have
different
modules.
A
So
we
have
semantic
segmentation-
and
this
is
part
of
this
pre-trained
model
yeah,
and
so
this
is
the
or
kind
of
the
origins
of
this,
and
so
that
comes
from
like
this
basic
need
for
segmenting
cells,
these
segment
images.
A
And
then
the
training
data
you
know
is
images
it
could
be,
it
could
be
text
or
I,
guess
we're
using
for
lineage
trees,
it'd
be
text
or
trees
like
tree
like
graphical
structures,
even
because
you
could
take
like
a
a
network
or
a
tree,
which
is
a
graphical
structure,
and
you
can
put
it
into
you
know
you
can
train
it
with
that.
So
you
know
a
lot
of
the
microscopy
data
sets
will
have
like
a
bunch
of
cells,
and
you
know
you
can
segment.
A
The
cells
like
this
is
just
an
area
just
a
generic
area
of
cells,
and
you
want
to
segment
out
these
cells.
You
know
as
independent
units
with
centroids,
but
sometimes
you
don't
have
the
identities
like
in
Mouse
embryo,
for
example,
each
cell
has
a
depending
on
the
mouse
depending
on
the
sample.
These
cells
are
different
identity.
B
A
Lineage
tree,
is
it
it
changes
depending
on
the
mouse,
so
this
has
been
a
problem
in
reconstructing
one
H3.
A
Really
want
to
connect
explicitly
some
of
the
like
graphical
data
or
some
of
the
label
data
to
the
cells
directly.
We
want
to
be
able
to
be
flexible,
so
one
thing
we
can
do,
of
course,
is
have
the
semantic
segmentation,
but
you
know
that's
not
always
going
to
help
us.
A
Then
we
want
to
have
maybe
instant
segmentation,
so
this
would
be
for
things,
embryos
or
images
where
we
don't
know
the
identity
of
the
components
which
is
sort
of
the
problem
that
you've
been
working
on,
and
you
notice
that,
like
you
know,
it's
not
going
to
give
you
a
great
resolution,
but.
A
Depends
on
what
algorithm
you
use?
It
depends
on
the
images,
so
we've
had
problems
in
the
past
with
different
types
of
images,
where
the
images
are
more
or
less
amountable
to
segmentation.
So
in
the
past
we've
had
you
know,
we've
tried
different
things
like
you
know:
you'll
have
a
bright
field
which
is
a
type
of
Imaging,
which
is
where
you
have
like
just
the
backwood
a.
C
A
A
These
things
like
image
J,
which
is
a
program
where
they
can
you
know,
have
to
transform
the
image
and
get
it
to
like
a
different
color
palette
to
augment
the
edges.
What
we've
done
for
machine
learning
is
we've
done.
Pre
we've
done
augment,
we've,
augmented
the
edges
using
different
augmentation
techniques.
A
So
in
the
past
years
we've
had
people
do
things
like
you
know,
highlight
the
edges
of
the
membranes,
with
contrast
or
done
it,
you
know
sometimes
manually.
That's
not
really
that
that's
not
a
really
tractable
method,
because
you
know
you
can't
imagine
I,
couldn't
imagine
someone
going
through
a
thousand
images
and
doing
that
you
could
do
it,
but
it's
not
not
efficient.
So
we
want
to
move
beyond
that
and
try
to
get.
You
know.
Other
types
of
images
we've
got
fluorescent
images
which
actually
have
the.
A
Images
so
the
benefit
there
is
you
have
those
kind
of
things
marked
with
the
fluorescent
markers?
So
sometimes
people
will
generate
data
with
like
a
membrane
as
some
sort
of
fluorescent
marker,
where
the
nucleus,
with
a
fluorescent
marker
and
the
problem
there,
is
that
you
know
well.
The
benefit
of
that
is
you
have
a
pretty
easy
hook
on
which
to
do
your
segmentation.
I
mean
it's.
A
A
Kind
of
problems
and
I
think
that
may
be
some
of
the
problem
in
the
data
that
we
have
with.
You
know
that
some
of
the
data
sets
like
the
Epic
data
set
where
yeah
you
have
this
kind
of
thing,
where
those
fluorescence
is
uneven
for
different
structures.
But
that's
you
know,
that's
a
problem
that
we
have
it's
a
common
thing:
we've
had
to
do
in
the
past,
so
that's
like
something
that
it's
hard
to
really
kind
of
find
the
killer
algorithm
to
fix.
A
Know
with
playing
around
with
the
color
palettes
in
the
how.
A
Before
you
actually
put
it
into
the
algorithm
that
can
be,
you
know
useful,
but
not
for
you
know,
not
fully
solve
the
problem,
and
then
we
have
types
of
images
we
have
spam
Imaging,
which
is
something
we've
been
using
with
C
elegans
they're.
These
high
resolution
microscopy
images
that
it's
a
different,
it's
a
type
of
microscopy
where
it's
very
just
a
very
high
resolution.
A
It
kind
of
looks
like
bright
field
Imaging,
but
it's
not
it's
a
different
method,
and
that
gives
you
probably
the
best
resolution
and
I
have
a
folder
of
different
images
that
we've
used
in
the
past
sort
of
our
stock
images,
as
you
will,
if
you
will,
and
those
have
a
lot
of
these
images
in
it,
I'll
probably
put
that
out
soon.
I
I
didn't
think
about
that.
A
When
we
were
talking
about
this
I
mean
we're
just
gonna
getting
started
on
it,
but
so
the
Epic
data
set
is
good
for
kind
of
like
bulk
training.
C
Etc,
etc,
but
I
mean
the
cell
tracking
filament
that
data
set
is
pretty
good
to
have
gone
through
like
some
of
the
files
from
it.
They
are
pretty
like.
We
can
train
our
model
with
that
and
it's
it's
look.
Good
I
mean
what
your
thoughts
on
instant
segmentation
I
mean.
You
are
okay
with
my
idea
right
like
that.
A
C
Just
have
small
doubt:
I
talked
with
mayuk
I
told
him
like
demograph
stage.
One
needs
centroids
and
volume
of
each
cell,
so
he
was
like
I've
shown
him.
The
segmentation
results,
I
I.
Think
you
think
you
like
you
want
to
join
today's
meeting,
but
due
to
some
other
meeting
classes
we
haven't
joined,
he
was
like
a
pretty
like
throwing
interest
like
we
can
make
a
segmentation
models.
You
can
expect
and
volume
of
each
in
actually
right.
Now.
C
A
B
A
But
what
my
point
was
in,
like
a
lot
of
the
stuff
we've
been
talking
about,
is
that
I
wanted
to
keep
them
kind
of
separate
into
separate
different
modules,
and
then
you
know
in
the
documentation
we
can
bring
them
together
so
like,
if
I
want
to
do
something
with
this
at
the
end
of
the
day,
what
do
I
want
to.
A
A
A
B
A
I
want
to
find
maybe
something
out
about
this
region,
so
we'll
use
the
semantic
segmentation
toolbox
and
then
you
know
I
don't
know
maybe
like
for
the
graph
neural
networks.
You
want
to
put
these
together
as
a
graph
where
you
can
connect
to
the
cells
or
connect
these
regions
and
according
to
different
criteria.
So
then
there's
this
graphical
representation,
and
this
would
also
work
for
like
a
lineage
tree
or
a
differentiation
tree
which
we've
talked
about
or
anything
else
where
there's
a
graphical
representation.
A
So
that's
that's,
maybe
how
we
want
to
do
it
because
I,
you
know,
I
mean
I
want
to
make
this
flexible,
and
there
are
a
lot
of
things
in
development.
You'll
need
these
different
methods,
for
so
I
mean
that's.
My
thinking
is
that
we
could
have
like
a
toolbox
for
each
of
these,
but
we've
got
a
lot
of
the
semantic
stuff.
A
We
have
the
instance
you're
gonna,
you
know
proposal
we
put
the
Instant
stuff
in
there
and
then
the
graphical
stuff
would
be
sort
of,
like
you
know
the
gnn's
work
and
so
just
kind
of
improving
upon
all
these
would
be
good.
A
B
Yeah
on,
as
you
may
recall,
my
mother
did
one
picture
of
drosophila
fixing
up
the
segmentation
and
I
think
it
took
her
somewhere
between
10
and
100
hours.
Yeah,
one
picture
yeah
yeah,
but
we
never
developed
any
tools
to
help
her.
B
Okay
and
I
was
looking
at
that
and
realized
that
that's
possible
like
a
tool
that
is
a
mixture
of
a
hand,
an
automatic
might
work
yeah,
okay,
okay!
Now
let
me
give
you
an
example:
your
sketch
that
says
epic
yeah.
B
You
could
determine
the
centroid
for
those
segments
all
right,
okay
and
then
from
the
centroid.
You
could
send
out
raise
in
all
directions,
and
you
could
you
could
indicate
when
they
hit
something
now
those
that
are
near
it
will
hit
it
sooner
and
goes
where
there's
a
gap
they'll
go
out
and
hit
something
else.
B
C
C
A
C
C
C
We
can
calculate
the
area
of
each
cell
after
that.
How
many
frames
like
video
frames
it
is
there
we
can
extract
the
volume
of
itself.
This
is
the
idea
of
extracting
it.
Actually
I
was
talking
with
I
had
a
meeting
with
them.
They
are
like,
okay,
this
these.
After
seeing
this
segmentation
results,
they
turn
like
yeah.
You
can
make
this
out.
They
have
given
a
green
signal
like
yeah,
you
can
do
it.
I
mean
these.
Are
the
segmentation
results
with
the
code?
C
A
A
Was
all
great,
so
yeah,
the
cell
tracking
challenge?
What
kind
of
data
is
that
like
what
organism
and
what
and
what
type
of
cell
is
it.
A
B
C
A
Yeah
I
see
that
yeah
I
remember
the
waters
and
my
Waters
and
lab
data.
We
use
we've
used
that
in
years
past
and
then
there
are
other
types
of
data
like
for
like
cancer
cells.
Those
are
usually
always
available
like
some
formula
yeah
there's
a
lot
of
good
stuff
in
here.
There
are
other
types
of
embryos
like
the
gas
trail
of
item
castania
embryo
there's,
yeah.
C
Sorts
of
different
types
of
cells-
here
you
can
train
the
model
our
model
and
we
can
extract
for
detailing
server
also
because
dmailing
staff
will
go
up
to
2500
cents.
Before
writing.
Code
I
have
gone
through
the
data
sets
actually
yeah
and
I
understood
like
what
happens
in
I.
Don't
I'm,
not
good
at
spellings
emailing,
sir
I
hope
that's
the
spelling
of
it.
C
At
that
time,
also,
whatever
the
instant
segmentation
I
have
done,
it
is
working
fine
and
it
is
working
fine
for
Mouse,
embryosals
and
even
for
sale
again.
So
we
can
implement
this
idea.
They
are
I'm.
Attaching
the
results
in
my
proposal
also
like
what
I
was
getting
and
the
the
main
work
would
be
like
segmenting,
more
like
driving
it
on
the
data
of
C
elegans,
first
and
after
that,
extracting
the
centroids
on
way
on
time,
series,
data
and
the
volume
of
itself.
So
that's
what
I
want
to
do
with
that?
A
B
Oh
yeah
yeah
brother
you're,
getting
back
to
yes
sketch
I
realized,
didn't
explain
it
properly.
The
centroid
can
only
will
work,
even
if
you
just
guessed
it,
but
touching
the
screen
that
okay.
B
B
Here,
what
you're
doing
is
taking
Rays
from
a
point
inside
that
group
of
line
segments
and
you're
characterizing
the
length
of
the
Rays
from
that
point,
from
distribution
of
them
and
saying
okay,
the
cell
has
got
to
be
amongst
the
closest
ones,
so
you
pick
the
closest
ones
and
then
you
can
close
them
off:
okay,
okay
and
since
they're
Arc
segments,
it's
easy
to
close
them
off
consecutively
and
get
get
in
proper
image
of
itself.
B
A
B
Yeah
yeah-
and
you
see,
all
you
have
to
do-
is
points
in
each
cell
that
looks
defective
and
then
the
algorithm
would
fix
it
right.
Okay,
that's
what
I
meant
by
next
human
machine
approach
right
and
that
would
be
much
faster
than
drawing
all
those
little
blind
seconds.
Yeah.
A
C
B
A
B
A
Drawing
that
was
yeah,
the
drawing
was
the
loops
were
closed
on
that,
but
then,
when
that
got
segmented,
you
know
there
was
some
work
that
needed
to
be
done.
Additionally,
yeah.
A
B
Yes,
it's
it's
I
think
it's
what's
called
filling
in
right
right
in
terms
of
vision,.
A
Yeah,
it's
it's
like
when
the
visual
system.
A
B
B
B
B
B
B
Okay,
all
right,
so
there
may
be
ways
to
do
it.
Do
the
whole
whole
thing
automatically,
but.
A
B
A
Digitized
basically
yeah
yeah,
so
yeah,
that's
that's
what
we
had
for
the
well.
We
have
a
lot
of
potential
microscopy
data
sets
whatever
we
want
to
use.
So
you
know,
and
then
there
are
things
out.
You
know
in
the
public
domain
that
we
can
use
that
you
know
I,
don't
know
what
the
most
recent
data
sets
are
that
people
are
using,
but
a.
A
Ones
are
a
bit
old,
you
know
I
think
they
started
really
doing
some
of
this
about
15
years
ago,
and
some
of
like
I
said
some
of
the
data
sets
are
better
than
others
for
certain
things.
So
it's
not
really
finding.
B
B
But
if
you
could
manually
identify
the
center
of
a
cell
and
put
in
the
information
on
the
maximum
width
of
the
Zone
maximum
length,
then
you
could
probably
segment
to
some
of
their
difficult
to
subscribe.
A
B
A
Like
using
it
as
a
training
module
like
a
training,
okay,
yeah,
so
it's
like
a
template,
then
you
would
apply
it
across
the
different
styles
to
say
this
is
basically
a
cell.
You
know:
where
do
you
see
the
end
of
the
cell?
The
sides
of
this
you.
A
For
the
most
part,
there
were
some
issues
with
that
I
mean
you
know,
you
would
get
things
where
it
wouldn't
fully
segment
and
then
there
was
some
refinement
that
needed
to
be
done
from
those.
So
but
it
was
based-
and
you
know
it
was
just
the
square,
so
it
wasn't
erected.
A
A
Yeah-
and
so
you
know
in
that
case,
we
didn't
really
care
about
the
division
of
the
cells
we
cared
about.
They
were
moving
against
one
another,
so
I
know
sushma,
probably
doesn't
know
this
work,
but
it's
where
we
had
these
colonies
of
cells
and
we
would
model
each
cell.
They
kind
of
slide
against
one
another
in
a
colony,
we're
interested
in
the
movement
of
the
cells
against
one
another,
but
we'd
had
to
model
the
cell
body
and
sort
of
the
boundaries
and
the
edges
relative
to
one
another.
A
So
you
know
we
just
drew
these
bounding
boxes
to
define
the
cell
length
Okay
and
the
width,
and
then
you
know
they're
adjacent
to
one
another,
and
we
observe
these
over
time.
So
we
could
track
them
and
it
was
actually
a
pretty
good
heuristic,
because
you
know
it.
You
basically
were
able
to
get
that
space.
You
know
those
two
things
together,
sort
of
next
to
one
another
and
you
could
get
the
entire
colony
and
just
characterize
the
movement
from
that
is
you
could
get
the
centroid?
You
could
get
the
displacement
and
other
variables.
A
A
A
Is
the
center
point
of
all
the
parts
of
the
edges
and
then
we
can
say
okay.
This
is
like
the
mean
Center
of
this
thing
and
then
there's
another
thing
over
here.
You
know
so
it
fits
into
the
not
only
the
segmentation
goal,
but
the
graphical
goal
of
like
finding
distances
or
you
know,
finding
an
edge
in
a
network
like
you
know,
yeah
yeah,.
B
C
B
A
Topological
data
analysis,
I
I,
didn't
it's
like
I
have
some
references
and
I
haven't
put
them
out,
but
I
mean
like
the
problem.
Is.
B
A
That,
like
it's
depending
on
the
problem,
you
know
there
are
a
lot
of
ways
you
can
approach
it.
So
basically,
topological
data
analysis
is
where
you
get
these
patterns
and
the
data
they're
spatial
patterns,
and
then
you
basically
want
to
find
different
patterns.
So,
for
example,
if
you
wanted
to
find
like
toroids
or
you
know,
different
types
of
shapes
in
the
data,
you
could
use
that
as
a
way
to
you
know
that
would
be
like
a
topological
data
analysis
where
you'd
find
like
a
shape
in
the
data
like
in
the
data
Cloud.
A
Basically,
you
know
it's
a
way
to
do
that.
It's
kind
of
a
mix
of
like
spatial
analysis
and
sort
of
you
know.
B
A
C
A
A
You
know
there
are
a
lot
of
ways.
People
do
this,
it's
not
there's
no
real
standard
way
of
doing
it.
So
that's
the
thing
we
have
to
keep
in
mind
there.
A
C
They
have
the
we
need
to
implement
all
this
on
sale,
tracking
challenge
data
set,
and
then
we
get
the
port
our
data
analysis.
We
want
like
the
centroids
and
the
boundaries
of
it
3D
image
segmentation
like
showed
in
this
video.
A
Just
yeah,
that
would
be
great
yes,
so
we
share
my
screen
here
with
people.
We
can
look
at
this,
so
this
is
a
a
resource
on
let's
see,
this
is
a
resource
on
this
software,
but
it
has
like
a
lot
of
things
for
topological
data
analysis,
so
you
can
do
things
like
manifold
learning,
which
is
where
you
find
shapes
within
the
data
so
you're.
Looking
at
your
point
cloud
and
you're,
looking
at
these
different
shapes
within
the
point
Cloud,
this
is
I
think
a
j,
a
general
example.
A
So
it's
not
really
like
you
know
it's
kind
of
a
toy
example.
It's
not
really
anything
challenging,
but
they
just
want
to
demonstrate
it.
You
know
there
are
other
types
of
things
like
different
instabilities
that
you
might
want
to
find
in
a
flow.
You
have
Contour
trees.
You
have,
let's
see,
merge
tree
clustering,
so
this
is
another
way
you
can
look
at
flows
and
things
like
that.
A
Persistence,
clustering,
which
is
again
or
persistence,
is
in
this
case,
but
I
think
it's
basically
things
that
exist
over
time
and
yeah.
So
you
have
this.
They
have
these
different
methods
for
that
persistence,
clustering,
Gallery.
So
you
can
see
some
of
these
shapes
that
they've
extracted.
So
it's
like
you
know,
you
have
a
point
cloud
and
you
don't
know
really
how
to
interpret
it.
You
can
find
different
shapes
in
there
in
different
ways,
so
they
have
algorithms.
A
A
B
A
Okay
yeah,
so
that's
great,
that's
great
stuff,
but
he
goes.
It
was
useful
to
go
through
that
because
I
I
every
year,
I
like
to
like
go
through
like
what
we
have
and
as
it'll
be
useful
for
people.
If
you're
applying
to
do
sock
for
this
project
in,
like
the
project
that's
listed
in
the
neurostars,
you
know
this
is
basically
a
project
that
is
sort
of
Maintenance
oriented.
We
want
to
upgrade
some
of
the
things
that
we
have
in
place.
A
So
you
know
the
solution
is
working
in
his
project
and
there
are
other
opportunities
as
well
to
do
other
things
in
the
project.
You
know,
there's
the
graph
neural
networks
pipeline.
There's
the
you
know
the
this
part
where
we're
doing
segmentation
to
get
to
the
graph
neural
networks.
There's
the
topological
data
analysis,
so
there
will
be
all
sorts
of
things.
People
can
propose
and
I.
A
You
know
if
you're
applying
to
the
project
or
you're
applying
to
this
project,
please
you
know,
let
me
see
your
I
offer
to
look
at
your
proposals
before
you
submit
them.
So
please
do
that
all
right!
There
are
no
other
things.
We
want
to
talk
about
I'll,
go
into
some
things
that
I
wanted
to
get
to
today.
So
let
me
share
my
screen
again.
A
So
this
is
actually
something
interesting.
I
found
a
paper
on
phenotype,
switching
and
agent-based
models
of
biological
tissue,
so
we've
talked
about
agent-based
models,
I.
Think
in
the
past
we've
talked
about
cellular
automata,
which
are
similar
types
of
models.
It
basically
allows
you
to
model
things
as
like:
a
set
of
cells
on
a
grid
or
in
in
some
sort
of
topology,
and
then
run
a
simulation.
A
Where
you
have.
You
know
neighboring
cells,
depending
on
the
state
of
those
cells.
You
can
you
know
you
see,
you
see,
changes
over
time,
so
in
a
cellular
automata,
for
example,
the
state
of
the
neighbor's
influences
the
state
of
the
central
cell
and,
depending
on
over
time
those
you
know
you
get
these
Global
changes
that
occur
over
large
areas.
Agent-Based
models
are
an
instance
of
this.
A
Where
you
have
these
discrete
models,
you
have
agents
as
the
cells,
and
then
you
have
the
Agents
of
Neighbors
in
the
state
of
the
neighbors
influences
the
actual
out
the
global
state
of
the
model.
Sometimes
the
locals.
Think
too,
in
this
case
they're
looking
at
something
called
phenotype
switching,
which
is
something
you
see
in
cells.
Sometimes
you
see
this
in
bacterial
cells,
where
you
have
different
types
of
phenotypic
states
and
depending
on
things
in
the
environment,
they'll
switch
this
their
state
from
one
state
to
another,
and
it's
usually
some
sort
of
trigger
a
trigger
event.
A
Like
stress
or
like
us,
you
know
a
stimulus
that
they
encounter,
and
so
this
is
an
important
thing.
Sometimes
you
know
you
see
this
as
well
in
development
with
different
cell
States,
but
this
is,
but
this
is
a
term
they
use
in
the
literature
phenotype
switching.
So
this
is
their
modeling,
this
using
agent-based
models
of
tissue.
B
A
These
grids
are
supposed
to
be
spatially
distinct,
so
you
know
one
end
of
the
grid
is
different
from
the
other
end
of
the
grid
spatially.
You
can
have
things
that
propagate
across
the
grid,
and
you
know
over
time,
and
you
can
actually
measure
that
as
spatial
diffusion
or
something
like
that,
so
the
resolution
of
modeler
can
achieve
in
these
regards
which
are
using
agent-based
models
is
unrivaled
by
other
approaches.
A
A
Involve
molecular
level
Dynamics,
especially
cell
level
Dynamics.
So
this
is
where
your
agent-based
models
have
molecular
level
resolution
and
they're
talking
about.
Well,
they
talk
about
cell
specific
Dynamics,
so
sometimes
they,
you
know
you
each
agent
or
each
cell
has
details
within
it,
and
so
you
have
Dynamics,
like
you
might
have
like
a
gene
regulatory
Network
within
an
agent
that
has
like
you
know,
has
some.
A
This
grid,
you
know,
if
there's
a
state,
that's
being
generated
by
that,
and
you
have
inputs
from
its
neighboring
cells,
then
that
can
be
a
little
bit.
You
know
that
can
govern
the
Dynamics
of.
B
A
Grid
and
it's
in
the
output
state,
but
it's
not
something
you
necessarily
directly
observe
so
this.
These
are
all
important
points
here.
We
have
developed
a
global
method
for
solving
these
computationally
expensive
Dynamics,
significantly
decreasing
the
computational
time
without
altering
behavior
of
the
system.
A
Here
we
extend
this
method,
the
case
where
cells
can
switch
phenotypes
in
response
to
signals
in
the
micro
environment-
and
this
is
this
case
of
like
in
bacterial
cells.
You'll
see
this
where
bacterial
cells
will
be
in
a
in
a
culture
dish
or
something-
and
you
have
a
you
know,
they
might
have
food
in
the
dish
and
they
might
not
and
they
can
switch
their
phenotype
to
like
a
starvation,
phenotype
or
some
sort
of
thing
where
they
they
conserve
their
metabolism.
B
A
Complex
molecular
mechanism
underneath
it
that
kicks
the
switch
back
and
forth.
We
find
that
the
global
method
in
this
context
preserves
the
temporal
population
Dynamics
and
the
spatial
Arrangements
of
the
cells,
while
requiring
markedly
less
simulation
time.
He
does
add
a
tool
for
efficiently
simulating
abms
that
captures
key
facets
of
the
molecular
and
cellular
Dynamics
and
heterogeneous
tissue.
So
this
is
using
a
tissue
model,
so
they're
able
to
do
this.
A
They
talk
about
the
limits
of
running
agent-based
models
on
a
desktop
workstation
to
around
a
million
cells,
and
they
talk
about
this
in
the
context
of
a
tissue.
A
lot
of
cubic
centimeter
of
tissue
contains
10
to
the
eight
cells,
so
10
to
the
sixth
is,
of
course
just
a
little
bit
smaller
in
10
to
the
eighths
or
actually
so
you
can't
necessarily
on
a
desktop
or
I
guess
a
laptop
or
something
of
that
nature
simulate
an
entire
cubic,
centimeter
tissue,
so
you're
just
short
of
that.
A
So
you
need
to
run
this
on,
like
a
some
sort
of
cluster
to
really
get
a
reasonable
simulation
of
tissue.
So
cubic
centimeter
isn't
a
small
amount
of
tissue,
but
it's
you
know.
If
we're
simulating
well
for
simulating
a
cleanser
might
be
able
to
do
something,
you
know
approximating
a
full
organism,
but
if
you're
talking
about
like
a.
C
A
A
So
some
of
these
techniques
are
simulating
a
small
but
relevant
region
of
the
tissue,
so
sampling
the
tissue
down
to
some
process.
That's
ongoing
at
any
one
time,
of
course
that
doesn't
give
us.
We
don't
really
know.
We
can't
really
fully
Define
those
processes,
and
there
are
a
lot
of
processes
in
the
tissue
that
are
important.
A
We've
talked
about
the
physics
of
the
full
tissue
in
previous
meetings,
so
that
you
know
that
would
sort
of
preclude
a
lot
of
those
processes,
there's
also
partitioning
the
micro
environment,
into
compartments
without
internal
spatial
resolution,
which
means
that
you,
basically
you
know,
give
up
on
your
spatial
resolution.
Very
fine
spatial
resolution-
and
you
know
with
you
know
in
the
trade-off-
would
be
that
you
can
basically
have
these
compartments
that
just
assume
homogeneity
in
you
know
certain
spatial
blocks,
so
that
might
be
useful
for
some
things,
but
again
like
for
other
things.
B
A
A
So
then
you
can
also
Target
specific
parameters
and
then
you
know
using
Bayesian
information
inference.
That's
a
good
method
in
this
citation
eight
we
developed
and
analyzed
what
we
refer
to
as
the
global
method
for
handling
molecular
dynamic.
A
So
you
know
you
can
do
Global
sorts
of
simulations
of
molecular
Dynamics.
Those
are
actually
good
for
speeding
up
the
simulation.
A
These
are
best
for
agent-based
models
that
require
simulating
reactions
at
the
cell's
surface
or
intracellular
signaling
Pathways,
so
we're
moving
down
from
the
whole
cell
down
to
like
signaling
Pathways,
and
things
like
that.
So
there
are
a
lot
of
ways
you
can
do
these
models,
so
the
traditional
way
of
doing
agent-based
models,
which
we
call
the
local
method,
is
to
solve
a
system
of
ordinary
differential
equations
for
every
cell
at
every
time,
step,
which
is,
of
course,
this
sort
of
Full
Resolution
method.
A
The
global
method
reduces
the
computational
time
compared
to
the
local
method,
by
averaging
the
molecular
State
variables
across
cells
in
a
region
of
the
microenvironment,
therefore,
solving
the
governing
differential
equations
and
applying
those
results
the
result
of
those
cells.
So
basically,
the
local
method
is
where
you
do
everything
you
do:
a
fulls
or
the
simulation
of
everything.
The
global
method
is
where
you
sort
of
average
across
different
parts
of
the
local.
A
You
know
the
localities
and
you
find
the
things
that
are
sort
of
you
know
the
most
important
features,
and
then
you
put
it
into
a
governing
set
of
differential
equations
and
apply
that
then
backwards
to
the
thing
you
know
the
the
different
regions
that
you're
interested
in.
So
these
are
two
ways
to
do
this:
it's
really
about
granularity,
I,
think,
and
so
it
just
depends
on
what
you're
interested
in
and
what
you
can
get
away
with.
So
for
some
processes,
this
works
and
some
it
doesn't.
A
So
you
know
there
are
different
types
of
things
you
might
be
interested
in
you
know
and
depending
on
what
those
are,
if
it's
apoptosis
or
proliferation
or
symmetric
division.
Some
of
those
things
you
can
model
using
the
Global
Effect
and
others.
You
may
need
the
local
method,
so
they
talk
about
your
agent-based
models.
They
give
a
set
of
parameters
in
their
Val,
their
default
values
here,
where
they're
doing
actually
something.
This
is
the
thing
they're
doing
in
phenotype.
A
Switching
so
they're
doing
this
of
a
cell-based,
ABM
they're
doing
two
types
of
Agents
proliferating
in
quiescent
cells,
so
proliferating
cells,
rules
that
divide
frequently
and
expand
their
population,
whereas
glass
and
cells
are
just
cells
that
don't
divide
and
just
sit
there
and
proliferating
cells.
Proliferate,
move
a
diet,
fixed
rates
given
in
table
this
table
here.
Glycine
cells
do
not
proliferate
and
their
death
in
movement
rates
are
orders
and
magnitude
slower
and
those
for
our
proliferating
cells.
So.
A
The
kinetics
are
much
less
prevalent,
and
so
you
can
see
right
away
that
you
have
one
set
of
cells
that
you
need
to
simulate.
You
know
do
these
local
simulations
on
and
another
population
where
you
could
get
away
with
global
parameters
and
be
okay.
This.
A
Advances
by
the
Westby
algorithm,
which
is
a
an
algorithm
that
people
use
in
molecular
simulations
it
simulates
stochastic
processes
without
getting
into
what
that.
Actually,
the
details
of
that
algorithm
know.
A
Well
validated
and
it's
actually
a
very
common
standard
algorithm,
but
it
advances
by
using
the
Gillespie
algorithm
randomly
deciding
on
the
next
time
step.
Given
the
sum
of
all
rates
in
the
model,
there's
one.
B
A
For
that
time,
step
is
chosen
randomly
with
the
weights
given
to
each
event
for
each
agent,
based
on
that
rate
of
the
event,
and
so
the
model
advances
forward
in
time
until
the
next
randomly
chosen
time
step
moves
the
simulation
past
the
predetermined
end
time,
in
which
case
no
cell
events
are
performed.
So
this
just
runs
down
to
where
you
stop
having
events
in
the
simulation.
A
So
then
they
simulate.
The
molecular
Dynamics
underlying
the
cellular.
Dynamics
saw
they're,
simulating
thermal
kinetic
substrate,
so
the
pharmacokinetic
parameters
involve
diffusion,
Cellular,
Exchange
and
intracellular
signaling,
which.
B
A
Basically,
a
scanning
for
cellular
metabolism,
if
you
were
to
model
that
you'd
have
like
several
parameters
that
you
need
to
locally
to
get
that
we
also
include
a
second
substrate
Quorum
factor
in
which
some
simulations
that
only
undergoes
diffusion
and
Cellular
Exchange.
So
there
there
these
different
levels
of
sort
of
representing
cellular
metabolism-
and
some
of
these
of
course
are
needed
for
the
proliferating
cells.
Others
are
sufficient
for
modeling
the
quiescent
cells,
and
so
they
kind
of
go
over
their
methods.
They
talk
about
their
different.
A
They
use
different
solvers
for
this,
so
in
these
type
of
models,
you're
using
equation
solvers
for
differential
equations-
and
there
are
different
methods
for
that.
So
you.
A
A
A
Cellular
Exchange,
you
can
use
a
matrix
exponential
for
both
local
and
Global
and
a
direct
Oiler
for
the
internal
OD
for
both
local
and
Global.
So
they.
C
A
The
average
of
the
molecular
State
variables
in
each
region
will
be
used
to
solve
the
differential
equations
rather
than
the
specific
concentrations
of
each
lattice
and
cell,
so
they're.
What
they're
doing
again
is
they're
doing
this
Global
method
of
averaging
out
the
state
variables
using
it
for
a
region
of
this,
this
agent-based
model
and
then
applying
it
to
the
cells
in
that
region.
A
Instead
of
calculating
the
specific
concentration
for
each
cell
Atlanta
site,
the
choice
of
regions
can
be
made
in
any
way,
which
means
that
you
know
this
is
a
it's
kind
of
guesswork
like
we
were
talking
about
with
cell
segmentation.
Sometimes.
B
B
A
A
A
That
the
vasculature
is
located
at
the
bottom
of
the
micro
environment.
So
this
is
where
you're
looking
at
two
and
3D
simulations
so
they're,
actually
looking
at
a
two-dimensional
model
and
a
three-dimensional
model
and
when
they
say
the
bottom
of
their
micro
environment
in
the
2D
simulations,
they
have
an
x
y
space
that
where
the
Y
axis
is
actually
the
depth,
and
so
the
Y
Min,
which
are
the
cells
at
the
bottom
of
the
y-axis,
is
the
bottom
of
the
micro
environment
and.
C
B
A
Z
that
the
bottom
turns
into
the
z-axis
and
the
minimal
values
are
the
z-axis.
So
it's
like
a
top-down
model
with
you
know.
If
you
have
a
cube,
you
have
a
depth
from
top
to
bottom,
the
X
Y.
It's
usually
some
gradient,
where
you
don't
really
have
a
top
to
bottom.
It's
I!
Guess
if
you
plot
it
plot
the
bivariagraph
x
and
y
y
is
at
the
bottom,
but.
B
A
Point
is:
is
that
you
have
more
spatial
resolution
on
the
3D
model,
so
you
basically
can
have
the
spatial
distribution,
the
spatial
segregation
of
things
and
they
can
grow
across
the
Grid
or
the
like,
the
cube
or
sphere
or
whatever
it
is
under
this
vascula,
which
are
assumption
regions
will
be
layers
stacked
on
top
of
one
another,
each
one's
so
thick.
A
So
this
is
where
you
have
this
vasculature
you're
building
regions
on
top
of
one
another
in
each
layer
is
this
layer
of
vasculature
or
this
layer
of
the
tissue,
and
so
we
shall
also
consider
the
case
which
vasculature
surrounds
the
growing
cells.
So
you
can
assume
that
there's
vascular
which
are
within-
or
you
know
around
these
units
and
these
these
squares.
But
then
the
model
in
this
case
regions
will
be
chosen
as
concentric
annually
around
the
center
of
the
micro
environment.
A
A
Types
of
you
know
shapes
to
to
form
boundaries,
Within
These
models,
and
these
are
usually
like.
These
are
often
used
for
diffusion
of
things
in
the
model.
So
each
cell
is
a
you
know,
each
square
is
a
cell
and
then
things
diffuse
across
those
cells,
and
so
you
have
to
have
diffusion
sort
of
a
limiting
distance
for
your
diffusion.
So
usually
people
will
use
a
center.
B
A
At
they'll
use
a
sphere
or
a
circle
with
a
center
of
diffusion,
and
then
things
Decay
linearly
from
that.
So
that's
what
they're
talking
about
here
then
they
go
through
the
thermokinetics
and
that's
something
that
I'm
going
to
leave
alone.
But
this
is
a
schematic
comparison
of
the
two
methods
where
you
basically
have
the
local
and
the
global.
You
have
the
PK,
the
pharmacokinetic
Dynamics
Chronicle
kinetics,
where
you
have
the
local
and
versus
the
global.
You
have
the
diffusion
Dynamics,
so
you
had
diffusion.
A
You
know
in
Maybe,
One
Direction
versus
multiple
directions,
so
the
global
again
is
averaging
these
things.
So
you
have
one
one
Arrow
coming
out
the
bottom
of
the
pharmacokinetic
Dynamics
model,
whereas
in
the
local
you
have
one
for
each
row,
so
you
can
refer
each
column
I'm.
Sorry,
so
you
have
the
arrows
multiple
arrows
coming
down.
So
there
are
multiple
States
here
versus
the
global,
where
you
have
one
average
State
diffusion.
A
The
same
story
where
you
have
a
lot
of
local
interactions
versus
one
set
of
global
interactions,
exchange
Dynamics,
is
where
you
have
exchange
between
the
cells.
In
this
case
for
the
local
method,
you
have
low,
very
local
exchange
for
the
global.
You
have
an
exchange
per
row,
so
it
goes
across
the
entire
row
and
just
averaged
across
that,
and
we
talked
about
how
these
are
just
levels
of
tissue
that
go.
You
know
from
bottom
to
top
and
then
intracellular
Dynamics,
you
have
you
know
each
row
is,
is
averaged
versus
having
each
cell.
A
You
know
things
going
on
in
each
cell
and
they
can
be
different.
You
know,
there's
massive,
you
know
maximal
heterogeneity
there.
So
that's,
basically
what
it
looks
like
in
this
model.
So
they
have
these
different
things.
They
have
different
strategies,
for
you
know
summarizing
things
with
parameters,
we're
having
you
know,
specific
things
for
specific
cells.
A
This
is
so
I
guess
the
results
come
out
as
my
Essence
as
an
intercellular
refreshold.
A
Some
of
these
things
are,
you
know
they
kind
of
agree
with
the
some
of
the
okay.
So
for
this
one,
the
global
method
agrees
with
the
local
method,
so
you
can
see
that
for
some
of
these
results
you
know
you
get
proliferating,
sales,
quests
and
cells.
You
get
things
that
agree
across
the
different
methods,
so
that's
good,
because
we
don't
want
Global
methods
averaging
to
deviate
too
much
from
our
local
methods.
A
The
point
of
using
a
global
method
is
to
have
like
averages
that,
are
you
know,
consistent
with
things
that
you
would
get
from
having
maximal
resolutions.
B
A
Good,
but
this
is
license,
isn't
interest,
so
your
threshold,
basically
to
show
the
accuracy
and
speed
of
the
global
method,
We
Begin,
using
both
a
local
and
Global
methods
to
simulate
a
growing
group
of
cells
at
a
100
by
100
lattice.
So
they
use
this
type
of
method
to
look
at
some
of
these.
They
use
they
look
at
quiescence
in
the
cells
and
they
want
to
see
if
these
things
are
consistent
and.
A
Okay
so
yeah,
so
then
they
talk
look
at
chemotaxis
so
to
test
the
validity
of
the
global
method
in
a
context
where
cells
can
move
along
gradients.
At
the
molecular
level,
we
allow
the
quiescent
cells
which
we
tested
up
here
to
chemotax,
which
means
to
follow
chemical
gradients
and
in
this
case
they're
letting
the
cells
follow
an
oxygen
gradient,
so
chemotax
means
to
follow
gradient
to
its
some
maximal
value.
A
So
you
know
when
you,
when
insects
follow
a
chemical
path
or
chemical
trail
of
pheromones,
that's
chemotaxis,
but
usually
we
talk
about
that
the
Single
Cell
level
across
all
of
our
metrics.
We
continue
to
see
agreement
between
the
two
methods.
This
includes
the
quiescent
compartment
migrating
and
mass
towards
the
blood
vessel
at
the
bottom,
and
that's
figure
four
F.
So
that's
actually
this
one.
C
A
Where
they
break
this
down
and
these
different
layers
and
so
they're
following
this
gradient
up
and
down
the
model
and
again
they,
you
know
they're
able
to
look
at
this
in
terms
of
compartments
and
they're
able
to
observe
this
kind
of
migration,
and
so
they
can
actually
disagree.
A
The
local
method
here
agrees
with
the
global
method
as
well,
and
so
they're
able
to
actually
do
this
again
for
tumor
micro
environments
using
this
Quorum
factor,
and
so
this
is
an
example
in
3D
of
this
agent-based
model,
where
you
have
the
red
and
the
blue,
so
the
micro
environment,
their
differences
in
the
micro
environment,.
B
A
Is
the
spherical
example,
but
it
basically
works
like
the
2D
example
here,
where
you
have
layers
that
go
up
and
down
and
there's
migration
within
the
model,
so
there
they
have
a
discussion
about
this.
So
we've
found
that
the
global
method
for
simulating
abms
is
sufficiently
robust,
handled
discrete
effects
of
the
molecular
skill
or
of
the
molecular
scale
on
the
cellular
scale.
So
we
want
to
simulate
the
molecular
Dynamics
and
look
at
how
what
the
effect
is
on
the
cells
and
switching
cells
state.
So
we
can
do
this.
A
We
can
do
this
for
This
Global
method
and
this
local
method.
We
can
capture
the
process
of
chemotaxis
along
a
gradient,
and
we
can
also
do
this
to
look
at
sulfate
decisions
which
are
these
switches.
These
phenotypes,
which
is
we
can
extend
that
to
how
a
cell
decides
to
make
a
change
in
Fate
In
other
words,
a
change
between,
like
a
stem
cell
and
a
differentiated
cell,
are
two
different
types
of
differentiated
cells,
so.
A
Some
of
the
and
we
talked
about
cell
segmentation,
so
in
cell
segmentation,
we're
taking
real
world
data,
we're
breaking
it
down
into
parts
that
we
can
analyze
and,
in
this
case,
we're
taking
those
parts,
we're
simulating
them
doing
sort
of
a
backwards
approach
to
understanding
what
they're
doing
so.
We
could
take
our
segmented
data.
Put
it
into
this
model,
use
it
to
calibrate
the
parameters
and
then
simulate
things
forward
to
observe
things
that
we
might
not
be
able
to
observe
under
a
microscope
or
get
like
more
General
Trends
in
biology.
A
This
paper
should
be
a
treat.
So
this
is
title
of
this
paper
is
detection
and
measurement
of
butterfly
eye
spot
and
spot
patterns
using
convolutional
neural
networks,
and
this
is
a
group
out
of
Portugal
and
Singapore,
so
this
is
published
recently
in
plus
one,
so
they
have
structures.
Butterflies
are
increasingly
becoming
model.
Insects
for
basic
questions
surrounding
the
diversity
of
their
color
patterns
are
being
investigated.
A
Some
of
these
color
patterns
consist
of
simple
spots
and
eye
spots,
so
you
can
see
that
there's
the
Wikipedia
stuff
on
I
spot
ice
spots
are
a
form
of
mimicry,
so
the
butterfly
wing
mimics
different.
You
know
eyes,
it
might
be
a
nature
mimicking
another
organism
like
a
predator
and
they.
A
General,
they
have
very
colorful
wings
and
these
wing
patterns,
sometimes
they're,
mimicry,
sometimes
they're
other
things,
and
so
this
is
attributed
to
a
pattern
forming
biological
process
of
morphogenesis.
So
this
is
controlled
by
a
number
of
genes
and
embryonic
development
such
as
ungrilled,
distillas,
Hedgehog
and
tenapedia
and
Notch.
So
these
are
all
signaling
Pathways
that
interact
in
the
formation
of
these.
So
this
is
something
that
you
see
early
on
in
the
organisms:
development
in
the
caterpillar.
A
You
have
these
imaginal
discs
that
have
these
these
sorts
of
genes
being
expressed
and
then
during
prepation,
these
Wings
form,
and
then
you
get
the
butterfly
with
the
colorful
wings.
A
B
A
Them
so
to
accelerate
the
pace
of
research
surrounding
these
discrete
and
circular
pattern
elements
we
train,
distinct,
convolutional,
neural
networks
or
cnns,
or
detection
and
measurement
of
the
butterfly
spots
and
eye
spots
on
digital
images
of
butterfly
wings.
We
compared
the
automatically
detected
and
segmented
Spa
eye
spot
areas.
What.
A
A
What
those
are
you
might
find
a
lot
of
alternate
variation
around
pattern
and
formation
might
find
things
that
are
spurious
patterns,
that
maybe
we
don't
see
with
our
eyes
and
indeed
the
patterns
themselves,
aren't
really
determined
by
any
cognitive
mechanism,
they're
determined
by
the
expression
of
these
genes
in
these
circuits,
during
development
and
especially
during
pupation.
A
So
you
get
this,
you
know.
Basically,
you
want
to
be
able
to
detect
things
that
may
be
interesting
in
terms
of
pattern
formation,
but
you
know
cnns
are
good
at
that,
but
sometimes
they're
not
trained
in
the
things
we
might
see
in
you
know
the
things
that
are
actually
patterns.
I,
guess
that's
the
way,
I
put
it.
A
In
any
case,
you
have
this
manual
annotation
of
things
that
we've
seen
in
other
butterflies
that
might
be
of
interest
as
opposed
to
just
something
that
maybe
repeats
or
that
that's
you
know
a
pattern
in
the
in
the
wing.
On
the
other
hand,
maybe
there
are
patterns
that
CNN's
can
uncover
that
human
observers
can't.
So
there
are
a
lot
of
opportunities
for
a
human
machine
cooperation
here.
A
We
then
compared
the
automatically
detected
and
segmented
Spa
eye
spot
areas
with
those
manually
annotated.
These
methods
were
able
to
identify
and
distinguish
marginalized
spots
from
spots
so
their
spots
and
then
their
spots
that
look
like
an
eye.
So
if
you
go
back
to
our
mimicry
example,
we
have
spots
here
things
that
are
of
a
different
color.
We
have
this
red
spot
here
versus,
say
like
an
eye
spot
here
with,
like
a
looks
like
an
eye,
it
looks
like
it
has
the
weight
of
an
eye
and
then
the
pupil.
A
A
These
methods
are
able
to
identify
and
distinguish
marginalized
spots
from
spots
as
well
as
distinguish
these
patterns
from
less
symmetrical
patches
of
color.
So
these
are
like
patches,
like
you
know.
They
have.
This
kind
of
you
know
this.
This
might
be
a
symmetrical
patch.
This
might
be
a
less
symmetrical
patch.
You
know
just.
A
If
you
zoom
in
you
can
find
them
in
any
case,
there's
a
lot
in
there
that
we
need
to
look
at
and
understand
and
quantify.
A
A
These
cnns,
after
Improvement
of
ice
spot
spot
detection
and
measurements
relative
to
previous
methods,
because
it
is
not
necessary
to
mathematically
Define
the
features
of
Interest,
so
you
don't
need
to
mathematically
Define.
These
features,
and
this
is
really
kind
of
stands.
In
contrast
to
some
of
the
differential
equations.
B
A
Use
the
model
reaction,
diffusion
and
other
types
of
pattern
formation.
In
that
you
know,
you
don't
need
to
mathematically
Define
the
features,
but
you
need
to
mathematically
Define
the
process,
and
so,
in
this
case
we're
defining
we're
just
looking
for
patterns
we're
using
a
statistical
method
to
find
patterns
that
are
common
across
the
images.
So.
A
Something
that
is
stands
in
contrast
not
just
to
other
types
of
measurement
methods,
but
to
other
types
of
ways
of
looking
at
morphogenesis,
although
all
that
is
needed
is
to
point
out
the
images
that
have
those
features
to
train
the
CNN.
So,
basically,
you
can
train
the
CNN
on
things
that
you
know
are
patterns
patches,
eye
spots
and
give
that
information
to
the
CNN,
and
it
can
extract
them
out
of
the
data
even.
A
They
are
used
by
a
variety
of
animals,
primarily
to
intimidate
or
start
on
Predators
or
deflect
Predator
attacks,
the
dispensable
areas
of
the
body,
so
this
is
just
a
way
to
like
full
Predators,
basically
and
keep
them
from
killing
the
butterfly.
So
it
increases
the
butterflies.
Fitness
eye
spots
have
been
studied
primarily
in
the
web.
Adoptera
or
different
modes
of
Defense
are
found
in
different
species
and
where
I
sponsor
All
Seasons
sexual
signaling.
So
we've
saw
examples
of
the
ice
spots
and
different
species
aside
from
butterflies-
and
you
know
this.
A
Been
studied
in
butterflies,
however,
I
spots
and
infallible
butterflies
in
a
single
origin,
this
evolutionary
origin
and
they
may
have
evolved
from
a
simpler
set
of
pattern,
element
spots.
So
you
know
there
are
different
types
of
patterns
that
came
together
in
morphogenesis
to
form
these
eye
spots,
and
this
was
of
course
you
know
there
was
a
fitness
imperative,
so
it
came
from
precursor
spots
that
are
common
on
butterfly
wings.
A
Spots
themselves
are
simple
patches
of
color,
contrasting
against
the
background
color
of
the
wing.
It
is
unclear
how
many
times
spots
of
evolved
independently
in
butterflies,
meaning
they
have
multiple
origins
in
phylogeny,
meaning
that
you
know
they're,
just
basically
artifacts
of
pattern.
Formation
spots
appear
because
that's
how
butterfly
wings
are
patterned.
These
gene
expression,
Networks
Express
things,
maybe
more
or
less
stochastically,
and
this
has
multiple
phylogenetic
Origins,
however
ice
mods
being
very
specialized
in
having
a
fitness
imperative,
maybe
evolved
once
in
a
single
winning
engine
and
diversified
it.
A
A
About
the
nice
spots
has
become
a
routine
task.
Researchers
different
size
eye
spots
are
found
in
males
and
females,
so
there's
sexual
dimorphism
there.
So
this
means
that
there's
probably
a
place
of
role
in
sexual
signaling
and
there's
also
differences
in
I
spot
size
between
the
dry
and
wet
Seasons.
A
So
there's
phenotypic
plasticity
in
the
same
species
they're
also
different
Predator
guilds
that
form
so
the
ice
spots
have
to
change,
and
so
these
ice
spots
aren't
constant
across
the
lifespan
they
change
seasonally,
but
they
also
are
sort
of
they
originate
in
occupation
and
even
further
in
development.
So
measuring
spots
and
eye
spots
is
also
become
important
for
researchers
who
explore
mechanistic
questions.
A
So
you
can
ask
all
sorts
of
ecologically
oriented
questions.
Then
they
decided.
You
know
that
you
can
actually
measure
these
things
with
calipers
or
with
some
other
quantitative
method,
but
you
need
to
classify
them
to
know
kind
of
you
know
if
it's
a
spot
or
a
nice
spot,
because
there's
a
lot
of
variation
in
the
data,
so
you
can
use.
You
know.
People
in
the
past
use
svm
classifiers
to
detect
sort
of
which
spots
are
in
which
category
and
then
they
build
a
circular
ring
and
they
measure
out
1D
whole
transformation
for
Circle
detection.
A
So
you
have.
These
circular
rings
that
you
can
derive
for
different
types
of
spot,
and
then
you
can
use
this
transform
to
detect
the
circle,
and
so
that
was
the
state
of
the
art
before
they
applied
this
convolutional
neural
network
approach,
and
so
this
is
a
very
common
approach
in
deep
learning
and
they're
able
to
do
this
and
of
course
you
need
a
lot
of
training
data
for
this,
so
they
were
able
to
use
a
lot
of
training
data
from
measurement
data
that
exists.
A
You
know
there
are
all
sorts
of
different
ways
to
do
this.
A
popular
approach
for
implementing
cnns
is
the
segment
the
images
into
two
classes
and,
in
this
case,
they're
using
the
nucleus
in
the
background
and
then
they're
able
to
so
since
semantic
segmentation,
which
we've
talked
about
earlier
in
the
meeting,
cannot
separate
touching
or
overlapping
nuclei.
Post-Processing
approaches
such
as
Watershed
or
H
Minima
transform
are
commonly
used.
So
this
is
an
interesting
point
with
respect
to
some
of
the
stuff
we
were
talking
about
earlier.
A
In
this
case,
we
have
these
classes,
and
sometimes
the
nuclei
are
touching
or
overlapping,
as
we
see
in
a
Cell
in
single
cells.
Sometimes,
if
they're
dividing
you
have
this
problem
and
using
something
like
instant
segmentation
over
semantic
segmentation
can
improve
performance.
However,
you
can
also
use
things
like
Watershed
or
age.
Minima
transform
to
do
the
same
thing
in
a
post-hoc
manner,
so
these.
A
To
the
data
collection,
so
you
can
actually,
you
know,
maybe
use
a
watershed,
algorithm
and
then
a
CNN.
So
you
get
like
you,
do
one
form
of
segmentation
and
then
plug
that
into
the
CNN
as
sort
of
a
supervisory
data
set,
or
some
combination
of
that
we've
done
this
with
the
basil
area,
where
we
did.
You
know
some
algorithms
on
that
data,
which
is
different
from
say
this
data
or
the
single
cell
data
that
we've
talked
about
in
in
embryos.
A
A
The
problem
you're
trying
to
address
so
they
get
into
some
of
these
things
about
convolutional
neural
Nets.
They
talk
about
fully
convolutional
networks,
which
are
a
type
of
CNN
composed
of
only
of
convolutional
layers
and
without
fully
connected
layers
can
be
used
to
look
at
nuclei
segmentation
maps
and
then
following
that
up
with
Watershed
post-processing.
So
in
this
case
they
use
nuclei
segmentation
using
the
fcn,
and
then
they
do
Watershed
post-processing.
A
In
this
paper
27,
we
use
fcn
to
predict
a
distance
map,
centroids
and
boundaries
of
nuclei,
so
they
use
this
with.
They
use
this
fcn
method
to
get
the
centroids
and
then
determine
the
boundaries
of
nuclei
as
a
distance
function
from
the
centroid.
So
there
are
different
ways:
you
can
do
these
kind
of
segmentation
procedures,
instant
segmentation
methods,
such
as
mask
rcnn,
jointly,
detect
and
segment
the
solving
the
splitting
problem,
which
is
where
you
have
splitting
of
cells
that
are
single
cells
and
so
forth.
So
there
there's
so
this
solves
this
problem.
A
It
could
solve
it
in
butterfly.
Spots
are
good
solvent
in
the
single
cells
in
an
embryo,
so
in
this
yeah
in
this
type
of
image
segmentation.
However,
the
images
of
the
cell
nuclei
and
cell
background
are
similar,
which
is
not
the
case
in
the
images
of
our
diverse
species
of
butterflies.
So
in
the
cases
where
they're
using
this
mask
are
CNN
they're
talking
about
you,
know
some
of
these
limitations.
But
in
this
specific
case
you
know
you
have
a
problem
where
you
can't
really
separate
out
the
nucleus.
A
In
the
background,
you
have
to
treat
them
sort
of
in
a
different
way,
because
you
can't
really
separate
them
out
very
easily,
so
they
use
CNN
for
both
spot
eye,
spot
detection
and
measurements.
They
first
use
a
data
set
of
images
of
different
species
of
butterfly
with
a
variety
of
spots
and
eye
spots
for
training
and
testing.
Three
state-of-the-art
object,
detection
methods.
A
Then
we
test
the
best
CNN
to
detect
ice
spots
in
a
different
image.
Data
set
insisting
of
images
of
many
individuals
of
single
species.
So
what
they
do
is
they
try
different
cnns
and
then
they
select
the
best
model
to
detect
ice
spots
across
all
types
of
images
of
these
eye
spots
and
then
in
the
second
image
data
set.
We
also
train
in
test
a
new
CNN,
just
recommend
to
measure
different
eye
spot
areas,
so
this
is
the
materials
and
methods
where
they
talk
about.
A
B
A
A
Input
data:
we
have
these
different
spots
and
eye
spots
that
are
sort
of
isolated
on
the
wing
and
then
they,
the
algorithm,
has
to
segment
out
these
parts
from
these
images,
and
so
this
is
an
example
figure
two
illustration
of
ground
Truth
and
then
this
the
way
are
the
regions
we
want
to
measure.
For
so
we
have
these
high
spots,
you
can
see
sometimes
they're
pretty
blurry,
sometimes
they're
hard
to
separate
from
the
background,
and
you
have
to
create
these
masks,
which
show
the
area
you
want
to
segment
out.
A
Then
they
use.
Let's
see
in
this
one.
They
use
two
two
methods,
these
eye:
spot
detection,
which
is
just
this
generalized
spot
detection.
Then
they
use
unit
as
a
form
of
ice
spot
measurements,
so
they
have
the
detection.
Then
they
have
this
extraction
and
resizing
of
each
detection.
Then
they
have
the
measurement
their
unit
and
then
this
resizes
it
to
the
original.
Using
these
masks.
A
And
they
go
through
a
lot
of
technical
detail
on
some
of
the
details
of
their
model,
but
basically
they're
using
CNN
and
then
they're
using
unit
for
these
different
things,
so
they
they
have
the
code
for
all
this
in
a
GitHub
repository
that
involves
YOLO,
which
is
you
only
look
once
written
in
that
efficient
debt
and
unit
cnns
and
we've
worked
with
a
number
of
these
models.
A
So
this
is
something
that
we
have
code
for
as
well
for
embryos
and
segmenting
cells
number
goes
so
we
implemented
two
versions
of
unit
that
can
provide
us
with
the
same
set
of
measurements,
namely
area
of
the
White
Center,
and
the
surrounding
color
brings
so
one
segment's.
Only
the
two
color
rings
from
the
rest
of
the
image.
That's
there,
two
class
Criterion
and
the
other
segment
Center
color
rings
and
background,
which
is
the
three
class
Criterion.
So
basically
you
have
the
color
Rings
the
background
and
the
center
are
always
different
classes
versus
the
background.
A
When
the
center
is
one
class
and
then
the
ring
is
the
second
class.
So
these
the
categorical
cross,
entropy
cost
function
to
determine
you
know
which,
which
parts
are
which
these
the
unwitted
version
of
this
cost
function,
which
is
more
commonly
used,
as
well
as
a
weighted
version
which
includes
class
weights.
So
there
are
all
the
code
is
in
this
GitHub
repository,
and
so
they
have
a.
A
That
they
use
for
quality
control,
they
look
at
the
pattern
element.
A
pattern
element
is
considered
a
true
positive
if.
A
Over
Union
between
its
bounding
box
and
any
ground
truth
bounding
box.
So
we
talk
about
bounding
boxes
with
respect
to
single
cells,
but
basically
a
bounding
box
is
something
you
define
as
an
element
around
you
find
like
the
edges,
and
then
you
sort
of
in
for
a
bounding
box
around
that
based
on
the
edges.
It
might
be
the
distance
from
a
centroid
or
whatever,
but
then
you
can
actually
use
that
against
the
ground
truth
and
the
ground.
A
Truth,
of
course,
is
the
mask
that
we
saw,
and
so
you
can
combine
those
two
looking
at
the
area
of
intersection
over
the
area
of
Union
and
so
intersection
is
where
the
two
sets
intersect
versus
the
union
where
they're
the
same,
and
so
you
get
this
measurement
of
IOU,
which
is
this
intersectional
reunion,
and
that
gives
you
an
indication
of
the
ground
truth.
So
you
know
we
couldn't
do
it
that
way.
We
can
get
a
score
for
its
reliability
and
it's
true
positive,
the
nature
of
which
are
positive.
So.
B
A
So
this
is
again
an
example
from
the
data
set.
You
know
the
different
methods
and
their
performance,
and
then
we
get
our
performance
here
on
icebox
marginalized
spots,.
B
A
A
This
is
an
example
of
the
masks
and
the
grown
truth
and
then
the
actual
segmentation
of
lie
spots,
and
so
this
is
I
think
this
stands
well
into
contrast
with
a
lot
of
the
segmentation
algorithms,
we
use
for
cells
and
embryo.
It's
a
different
type
of
system,
different
Dynamics,
but
it's
also,
you
know
it
does.
B
A
Some
interesting
parallels
to
consider
the
other
thing
is
is
that
it
stands
in
contrast
to
some
of
the
models
of
orphogenesis,
so
we've
talked
about
models
of
morphogenesis
using
differential
equations.
We've
talked
about
models
of
morphogenesis
using
physical
attributes
like
the
physics
of
cells
and
we've
talked
about
pattern
formation
using
a
genetic
regulatory
Network.
So
a
lot
of
these
genes
that
are
involved
in
pattern
formation
like
spot
formation
and
and
striping,
and
things
like
that
segmentation.
A
Those
are
all
you
know,
you
can
build
models
of
genetic
regulatory
networks
that
produce
these
patterns,
and
so
the
question
is:
is
you
know
what
is
the
relationship
here?
You
know
we
could
bottle
say,
for
example,
a
grn
or
a
differential
equation
with
noise
or
with
environmental
inputs
that
it
will
allow
us
to
model
some
of
these
ecological
variations.
A
A
You
know
you're,
basically
having
you
basically
are
asking
two
questions
there.
One
is
that
in
your
model
of
a
genetic
regulatory
Network
produce
things
that
can
be
recovered
by
convolutional
neural
network,
that's
trained
on
detecting
patterns
and
actual
biological
phenotypes,
and
then
two
can
your
model
of
the
phenotype
or
the
genetic
regulatory
Network
expressing
a
phenotype
can.
B
A
Be
sort
of
in
that
match
what
you
see
in
nature
and
that
match
the
variation
in
nature
and
that
match
the
patterns
that
are
produced
in
nature.
So
there's
some
really
interesting
things
that
you
put
together
from
this
again
I'll
post
the
paper
in
the
slack
I
think
that
you
know
if
you're
interested
in
CNN's,
if
you're
interested
in
pattern
formation.
This
would
be
a
really
interesting
thing
to
follow
up
on.
A
So
thank
you
for
your
attention
and
I
hope.
You
learned
something:
okay,
any
questions
or
comments
before
we
I
think
we'll
wrap
it
up
for
today.
Okay,
so
again,
if
you
have
any
questions
about
gsoc
or
any
other
projects,
we're
doing
or
any
anything
of
Interest,
let
me
know,
and.
C
C
A
It's
pretty
yeah
I,
look
forward
to
it.
Yeah
I
think
you're,
making
good
progress
on
yeah
make
sure
you
knew
that,
because
that
we
wanna
we're
kind
of
doing
this
road
map
on
the
Fly,
because
you
know
we'd
be
spend
time
on
the
project
in
the
Summers
and
then
it
kind
of
get
Earth
fallow
in
the
winter
time.
So
it's.
C
A
Time,
yeah
I
think
at
this
stage
the
proposal
is
most
ly,
I
mean
the
code
you'll
be
doing
in
in
the
summer.
You
know
Google
summer
code,
so
you
know
that
that's
where
that
would
come
in.
B
C
You
know
by
next
week,
after
that,
I
have
pretty
confused
week
like
exams
and
all
are
there
for
me
busy
with
that.
Before
that
I
will
complete.
My
proposal
and
I
will
submit
to
you
for
the
interview.
Okay,.