►
Description
Presented by: Calum Hall
GitHub has an obligation to the development community to ensure that they protect their employees and internal systems from threat actors across the globe. A core part of that is ensuring that GitHub is prepared to detect and respond to malicious behaviour targeting the company’s environment. This talk discusses the key detection principles that lead GitHub’s threat detection efforts, as well as how the company combats some of the toughest challenges GitHub, amongst many others, face in the industry today.
As always, feel free to leave us a comment below and don't forget to subscribe: http://bit.ly/subgithub
Thanks!
Connect with us.
Facebook: http://fb.com/github
Twitter: http://twitter.com/github
LinkedIn: http://linkedin.com/company/github
About GitHub
GitHub is the best place to share code with friends, co-workers, classmates, and complete strangers. Millions of people use GitHub to build amazing things together. For more info, go to http://github.com
A
A
I
was
a
former
offensive
operator,
I've,
always
focused
primarily
in
Mac
OS
security,
where
I've
been
fortunate
enough
to
speak
at
a
number
of
kind
of
high
level
security
conferences,
including
black
hat
and
Objective
C,
is
objective
by
the
Sea,
the
Apple
security
conference.
Now,
from
speaking
out
of
the
security
conferences,
there's
one
thing:
I
always
take
away,
and
that's
there's
always
a
lot
more
red
team
talks
and
offensive
operator
talks
than
there
are
blue
team
talks.
Now.
This
is
largely
for
obvious
reasons
by
the
very
nature
of
what
we
do.
A
Blue
teamers
are
always
kind
of
holding
their
cards
a
little
bit
closer
to
their
chest,
and
red
teamers
like
to
share
their
wins
a
little
bit
more,
and
so
we
believe
at
GitHub.
It's
important
to
start
talking
about
the
blue
team,
wins
and
sharing
what
we
believe
metal
help
might
help.
Others
in
the
industry
improve
their
detection
strategies
So.
Today
we're
going
to
be
talking
about
what
my
team
does
our
objectives
and
why
we're
employed
by
GitHub
and
we're
going
to
talk
about
our
detection
principles.
A
So
what
we
as
a
team
kind
of,
follow
and
adhere
to
to
make
sure
that
all
the
kind
of
team
members
that
I
work
with
that
we're
all
working
towards
the
same
goal
and
kind
of
have
the
same
strategies
in
mind.
We're
then
going
to
look
at
some
of
the
kind
of
fun
stuff
and
so
practicing
what
we
preach
so
taking
some
of
these
detection
principles
showing
how
they've
worked
for
us
and
kind
of
giving
you
some
examples
that
perhaps
you
can
take
away
and
Implement
yourselves
and
lastly,
as
much
as
I
wish,
we
were.
A
We
are
not
perfect.
At
GitHub,
we've
had
to
learn
a
lot
of
lessons
along
the
way,
and
hopefully,
by
kind
of
talking
about
these
lessons
today,
we
can
show
you
that
there
are
a
lot
of
commonalities
across
different
organizations
and
but,
whilst
things
might
be
difficult,
there's
always
a
solution
and
we
can
always
kind
of
better
our
detection
strategies.
A
So,
starting
with
our
objectives,
this
is
the
kind
of
generic
slate
and
the
kind
of
generic.
What
we
tell
people
we
do,
what
it
really
comes
down
to
is.
We
have
one
job.
We
are
there
to
protect,
GitHub
we're
there
to
protect
its
employees
and
where,
when
we
identify
threats,
it's
our
jobs
to
eradicate
those
threats
from
our
environment
and
ensure
the
GitHub
platform
is
safe
for
our
employees
and,
most
importantly,
for
our
users
and
customers.
A
So
that's
just
to
kind
of
break
up
exactly
what
my
team
does
and
I'm
sure.
Most
of
you
know
how
essential
GitHub
is
to
the
kind
of
Open
Source
community
and
the
GitHub
platform
itself,
and
the
content
on
that
platform
is
about
most
important
in
that
mission.
Now
we
have
teams
internally
whose
job
is
to
kind
of
Monitor
and
ensure
the
security
of
the
GitHub
application
itself
and
things
like
customers
and
and
those
kind
of
different
paths
there
and
my
team
is
our
job
to
support
and
secure
the
underlying
infrastructure
that
secures
the
GitHub
platform.
A
So
when
it
comes
to
kind
of
blue
teamwork,
I
believe
this
can
be
split
up
largely
into
two
categories.
This
is
the
proactive
work
which
is
our
kind
of
day-to-day.
What
we
do
and
when
everything
is
going
well
and
then
there's
the
reactive
work,
where
less
so,
the
proactive
stuff
and
my
day-to-day
is
primarily
focused
around
detection
engineering.
A
So
that's
performing
research
looking
at
different
detection
opportunities
within
our
environment
and
keeping
up
with
different
research
across
the
industry
and
how
we
might
be
able
to
kind
of
implement
those
into
detections
and
what
we
need
to
look
out
for
in
terms
of
kind
of
advancing
attacks.
That
may
come
our
way.
We
then
also
perform
a
lot
of
threat,
hunting
so
ensuring
that
our
environment
is
kept
secure,
looking
for
anything
that
may
have
bypass
any
detections
and
just
generally
looking
after
the
state
of
our
environment
and
lastly,
we
do
a
lot
of
testing.
A
The
general
idea
is
when
an
incident
happens
at
GitHub,
we
are
the
First
Responders
to
identify
what
that
activity
might
be,
whether
it
truly
is
malicious
or
whether
it's
kind
of
benign
behavior.
Now
it
makes
sense.
We
are
the
detection
Engineers
ourselves.
So
us
being
First
Responders
allows
us
to
have
full
context
around
detections
that
are
being
triggered
and
once
we've
kind
of
finished,
the
initial
triage.
A
A
And
lastly,
as
as
kind
of
any
good
blue
team,
should
we
kind
of
like
to
look
over
incidents
as
they
happen
and
see
how
we
might
be
able
to
take
new
detection
opportunities
and
cycle
that
back
in
to
the
proactive
work
we've
discussed
earlier.
Let's
take
an
example
when
we
can
talk
about
how
our
setup
looks
kind
of
behind
the
scenes
within
our
environment.
So
if
a
detection
triggers
in
GitHub
it
reaches
out
to
our
custom
soar
platform,
which
is
our
security,
orchestration,
Automation
and
response
platform.
A
Now
this
is
something
we've
custom
built
at
GitHub,
as
we
really
wanted
to
kind
of
kind
of
feel
extensibility
as
to
what
we
did
when
a
detection
triggered
in
our
environment
now
having
a
kind
of
custom
platform
allows
us
to
handle
our
detections
as
code.
It
allows
us
to
manipulate
kind
of
detections
and
the
data
that
they
sent
us
and
allows
us
to
perform
further
enrichment,
and
mainly
it
just
allows
us
in
the
future.
A
If
we
want
to
do
any
kind
of
automated
response
or
or
really
anything
we
want
with
this
data,
we
have
that
flexibility.
So
once
we've
hit
a
custom
store
platform,
we
then
create
an
issue
within
one
of
GitHub
repositories,
and
they
actually
looks
something
like
that
on
the
right
there.
Unfortunately
I've
taken
out
all
the
juicy
details,
but
the
general
idea
is,
you
can
see
what
has
happened
and
you
can
see
that
an
IM
user
has
been
created.
A
You
can
see
that
it
was
myself
Cal,
Hall,
that's
created
it
and
then
some
very
basic
details
about
myself
there
as
well.
Now,
once
that
issue
is
created,
it
triggers
a
GitHub
action.
Now
that
GitHub
action
we
used
for
enrichment
and
what
it
does
is
it
reaches
back
out
to
that
custom
soar
platform
now,
you're,
probably
wondering
why
we
do
this
in
two
separate
actions.
This
is
largely
from
a
timing
perspective,
so
we
want
to
create
our
alerts
as
quickly
as
possible
to
get
an
analyst's
eyes
on
it,
and
then
we
can
enrich
it
further.
A
So
once
that
alert
has
been
initially
created,
we
can
then
add
in
comments
to
the
original
alert,
like
you
can
see
here
on
the
right
which
provides
more
information,
and
it
really
makes
the
whole
process
as
efficient
as
possible.
We
don't
have
to
worry
about
enrichment,
taking
too
long
or
anything
along
those
lines.
We
can
really
work
in
a
kind
of
fastest
and
more
efficient
pipeline.
A
So
moving
on
to
the
detection
principles
now
I
mentioned
at
GitHub,
we
have
detection
principles
within
our
team
and
they
kind
of
keep
us
all
aligned
in
terms
of
how
we
Implement
detections,
what
those
depictions
look
like
and
just
the
kind
of
General
behaviors
that
our
team
follows
during
your
day-to-day
work.
So
the
first
principle
is
automate
all
the
things
and
something
very
dear
to
most
Engineers
Hearts
would
imagine
your
organizations
hire
incredibly
intelligent
people
so
that
you
can
make
the
most
of
their
time.
A
You
don't
want
them
spending
their
time
and
their
effort
on
kind
of
mundane
tasks
that
can
be
repeated,
and
so
we
like
to
follow
the
kind
of
if
something
can
be
automated,
make
sure
it's
automated
saves
us
time
and
as
we'll
see
a
little
bit
further
on
can
be
a
lot
better
for
users
as
well.
Now
next
is
focusing
on
detecting
Behavior,
not
iocs
or
indicators
of
compromise.
A
Now,
when
we
see
a
new
attack
in
the
wild
one
of
the
first
things
that
follows,
is
people
providing
different
detection
techniques,
specifically
looking
for
different
hashes
or
looking
for
different
file,
names
or
artifacts
that
are
generated
by
that
given
attack
now,
whilst
that
can
be
incredibly
useful
for
kind
of
immediate
response,
it
kind
of
throws
things
off
in
the
long
term.
You
can't
really
rely
on
artifacts
because
they're
very
easy
to
evade
from
an
attacker's
perspective.
A
So
what
we
push
at
GitHub
is
very
much
look
at
the
attacks
and
detections
that
people
were
suggesting
and
look
at
how
we
can
identify
the
behavior,
that's
generating
those
artifacts
in
the
first
place,
rather
than
the
file
names
of
the
hashes
themselves.
Next,
we
have
always
detect
the
never
events.
So
never
events
within
the
medical
community
are
known
as
events.
That
should
never
happen
if
proper
guidelines
and
safety
controls
are
always
full
of
now.
This
translates
pretty
nicely
over
to
Tech.
A
We
know
within
our
environments,
things
that
should
and
things
that
shouldn't
happen
so
at
which
point,
if
something
should
never
happen
due
to
legitimate
Behavior,
we
should
always
have
a
detection
in
place
to
identify
its
occurrence.
We
then
have
progress,
not
Perfection.
Now
this
is
one
very
dear
to
my
heart.
We
always
strive
to
have
100
Fidelity
when
we're
creating
detections
and
to
have
things
only
trigger
when
there's
a
hundred
percent
true
positive
rate.
A
The
reality
is
not
many
of
us
work
with
an
environments
anymore,
for
that
is
a
reasonable,
and
that
is
something
that
we
can
expect
and
as
such,
we
need
to
learn
how
to
detect
things
within
noisy
environments
as
we're
going
to
discuss
later
on.
In
this
talk-
and,
lastly,
is
the
credential
defense
now
the
Chrome
dual
defense
is
nothing
new
within
security.
We've
all
seen
it
talked
about
Lots.
The
general
idea
is
to
take
a
threat
model
and
work
backwards.
A
A
Let's
talk
about
automating,
all
the
things
and
kind
of
practicing
what
we
preach
so
going
back
to
the
alert
that
we
talked
about
at
the
start,
where
I've
created
an
IM
user
and
now
get
how
we
kind
of
push
our
hubbers
away
from
creating
IM
users
manually
within
AWS.
Now
we
provide
alternative
solutions
for
users
to
kind
of
get
this
pragmatic
access
or
programmatic
access,
and
so
this
theoretically
is
another
event.
This
is
theoretically
something
that
should
never
happen
now.
The
reality
is
that
isn't
how
it
works.
A
In
the
real
world,
we
have
lots
of
different
people
joining
GitHub
and
a
lot
of
those
people.
Who've
worked
in
AWS
environments
before
we're,
creating
IAM
users
was
part
of
their
day-to-day
work,
and
so
we
found
this
was
quite
a
noisy
alert
and
was
fairly
low
Fidelity,
but
yet
was
something
we
wanted
to
address
on
each
occurrence.
So
let's
walk
about
how
this
looks
traditionally
and
how
this
detection
triggering
and
how
it
would
normally
be
handled.
A
So
you
have
myself
as
the
little
Optical
up
in
the
left,
there
I'm
going
to
go
ahead
and
create
an
IM
user,
which
is
will
soon
triggers
a
detection.
Now,
once
that
detection
is
triggered,
it's
then
going
to
alert
an
analyst
whose
first
move
is
likely
going
to
to
be
to
reach
out
to
the
hover
and
say:
can
you
just
confirm
this
Behavior
as
yourself?
Why
are
you
doing
this?
This
shouldn't
be
your
kind
of
go-to
way
of
achieving
these
results,
and
they
have
that
conversation
and
based
on
the
results.
A
The
analysts
can
determine
they
need
to
investigate
further
because
something
seems
a
little
off
or
they
can
close
the
alert
which
we
see
happens
nine
times
out
of
ten.
Now
the
inefficient
part
of
that
whole
process
is
the
communication
between
the
analyst
and
the
hover.
Now
not
only
is
GitHub
a
global
company,
and
so
we
all
work
in
different
time
zones.
So
this
can
take
quite
a
bit
of
time.
There's
also
going
to
be
some
burnout
from
the
analyst
side.
A
If
this
is
happening
kind
of
frequently
now
that
burnout
is
going
to
directly
affect
that
hover
as
it's
a
lot
less
likely,
the
analyst
is
going
to
have
the
patience
to
handle
an
alert
appropriately
if
this
is
their
10th
of
the
day,
and
so
there
really
was
a
kind
of
disparity
here
that
we
needed
to
fix
and
we
needed
to
find
a
solution
as
to
how
we
can
improve
this
process.
So,
let's
take
this
whole
process
that
we
had
a
second
ago
and
let's
introduce
you
to
distribute
to
the
alerting.
A
So,
instead
of
going
straight
to
an
analyst
once
the
detection
is
triggered
once
an
IM
user
has
been
created,
detection
triggers,
let's
reach
out
to
our
distributed,
alert
and
tooling.
Now
our
distributed,
alerting
tooling,
has
a
bunch
of
predetermined
instructions
that
it
knows
to
action
in
the
event
of
Any
Given
detection
being
triggered
in
this
case
with
the
create
IM
user.
A
We
are
going
to
reach
out
to
the
user
and
say
something
along
the
lines
of
what
you
have
in
front
of
you
just
now
so
hi
there
with
no
tissue
performing
activity
X,
and
we
advise
against
that
for
for
these
list
of
reasons.
Whatever
your
organization
doesn't
like.
Instead,
you
can
follow
or
pave
paths
here,
and
we
can
kind
of
pervade
that
guidance
in
that
education
for
the
hover
and
how
they
can
do
it
properly.
A
Now
by
automating
this
you
dealt
with
kind
of
angry
and
aggravated
analysts
reaching
out
to
these
hubbers
and
kind
of
failing
to
give
them
consistent
information
and
consistent
instructions
on
how
to
better
themselves.
But
rather
we
can
fix
that
whole
process
by
saying
this
is
the
way
it
should
be
done
and
go
and
follow
these
paved
paths.
Next,
we
can
then
ask
the
hubber
to
confirm:
was
this
Behavior
itself?
If
they
say?
A
No,
then
obviously
that's
a
slight
security
concern,
something
we
want
to
look
at
and
we
can
go
and
alert
an
analyst
and
take
it
from
there
now.
If
they
say
yes-
and
they
say
this
Behavior
was
themselves
well,
then
we're
just
going
to
get
them
to
do
a
quick
physical
authentication
check,
so
they're
going
to
use
the
physical
security
key
to
say.
A
Next,
we're
going
to
talk
about
progress,
not
Perfection,
so
how
do
we
detect
enormous
Behavior
within
an
anomalous
environment?
Okay,
so
the
first
thing
that
I
want
to
kind
of
pushes
away
from
here
is
the
kind
of
stigma
we
have
in
the
industry
against
enormous
Behavior.
Now
enormous
behavior
is
often
seen
as
great
for
red
teams
and
terrible
for
blue
teams
offensive
operators
love
them
because
they
are
able
to
kind
of
hide
the
activity
that
they're
performing
and
for
blue
teamers.
It's
a
lot
harder
to
detect
because
there's
a
lot
more
Noise
Within
those
environments.
A
Now,
whilst
that
is
true,
we
also
need
to
kind
of
get
away
from
looking
at
it,
as
enormous
activity
is
bad.
Reality
is
if
you're
working
at
any
interesting
company
you're
going
to
be
facing
enormous
activity.
Anonymous
activity
is
the
result
of
innovation
as
a
result
of
your
developers
doing
their
job
and
building
new
things,
introducing
new
services
and
Technologies,
and
as
such,
we
must
learn
to
grow
with
that
noise
and
find
better
ways
of
detecting
these
attacks
and
achieving
the
same
kind
of
detection
results.
A
And
aside
from
just
High
Fidelity
alerting,
we
need
to
kind
of
Advance
ourselves,
past
low
noise
environments
and
High
Fidelity
alerts.
So
again
we
have
kind
of
three
pillars
and
three
strategies
we
like
to
take
on
with
this.
So
we
have
our
behavioral
detections
now
behavioral
detections
is
nothing
new
within
the
industry.
I'm
sure
a
lot
of
us
will
seen
this
before
and
it's
been
spoken
about
it
fairly
widely.
The
idea
is
being
able
to
correlate
events
from
a
single
entity
across
multiple
different
data
sets
and
across
your
entire
ecosystem.
A
Now
this
can
be
incredibly
effective
if
you
can
track
one
user
or
one
system
across
multiple
different
data,
sets
and
say
the
performed
action
X
over
here
action
Y
over
here
and
as
such,
the
pattern
of
the
behavior
reaches
such
a
threshold
that
we
want
to
get
human
eyes
to
investigate
what
is
going
on
now.
That's
incredibly
effective
and
we've
seen
this
work
in
the
past,
and
it's
almost
kind
of
proven
at
this
point.
A
As
we
have
seen,
it
provides
the
ability
to
kind
of
tune
down
the
noise
whilst
identifying
chains
of
attack,
rather
than
the
specific
events
themselves.
So
the
next
thing
that
we
like
to
implement
when
we're
attempting
to
identify
enormous
behaviors
enrichment
enrichment,
is
what
I
believe
to
be
one
of
the
most
important
techniques
to
identifying
this
behavior
is
the
process
of
kind
of
adding
context
to
events
within
your
environment.
A
Now,
when
we
we
talk
about
enrichment
at
GitHub,
we're
fortunate
enough
to
have
an
incredibly
smart
threat,
intelligence
team
and
we're
able
to
enrich
our
events
and
our
alerts
with
data
that
they
provide
us.
However,
not
everyone
has
that,
and
that
is
absolutely
fine.
We
can
still
enrich
our
events
and
our
alerts
to
the
point
in
which
we
can
kind
of
educate
the
decisions
that
we
make
so
using
data
sets
is
simple
as
HR
data
and
kind
of
work,
their
data.
Where
does
that
person
traditionally
work
from
what
errors
do
they
traditionally
work?
A
What
time
zones
do
they
operate
in
thin,
and
is
that
person
on
Parental
leave
for
the
next
two
months,
and
should
they
really
be
kind
of
deploying
things
in
production
and
by
being
able
to
add
all
this
context
into
the
events
and
alerts
that
we
have?
We
are
able
to
manually,
make
smarter
decisions
and
also
automate
a
lot
of
what
would
be
false,
positive
results,
and,
lastly,
is
the
criminal
defense.
Now
the
cranial
defense
in
this
situation.
A
A
So,
moving
on
to
some
lessons
learned
now,
we
have
what
I
believe
to
be
an
incredibly
efficient
detection
strategy
here
at
GitHub,
but
it
has
been
a
long
road
to
get
there
and
we've
had
to
learn
a
lot
of
lessons
along
the
way.
Now.
The
first
thing,
I
think
is
incredibly
important
to
talk
about
is
developing
Partnerships.
Now
from
someone
like
myself.
That
comes
from
a
purely
security
background,
and
this
is
something
that
you
might
not
think
to
be
on
the
top
of
your
list
when
implementing
a
detection
strategy.
A
Now
it
is
important
for
teams
like
ourselves
and
for
security
teams
as
a
whole
to
ensure
that
we
are
a
core
part
of
that
conversation
and
develop
Partnerships
with
the
people
in
these
teams
that
allows
us
to
influence
the
decisions
they
make
and
make
sure
they're
developing
secure
products
from
the
start
and
also
make
sure
that
we
have
the
necessary
Telemetry
moving
forward
to
ensure
that
we
are
securely
detecting
any
potential
attacks
within
those
new
products
that
we're
shipping.
Now
the
second
and
slightly
similar,
but
more
technical
note
is
rely
on
your
smes.
A
So
your
subject
matter
experts
now
my
chief
of
security
officer
Mike
Hanley
at
GitHub.
He
has
a
quote
that
he
used
to
see
when
we
all
can
have
joined
and
heard
it
lots
when
I
started,
which
was
raise
your
hand
if
you're
on
the
security
team
and
everyone,
regardless
of
your
security
engineering
and
whatever
you
maybe
would
raise
the
hand
now
I
thought
this
was
just
an
incredibly
good
way
of
building
morale
and
making
it
widely.
A
Knowing
that
security
is
everyone's
problem,
but
this
is
actually
a
lot
more
than
that
and
the
longer
I've
been
at
GitHub,
the
more
applicable
this
has
become.
When
we
see
everyone's
on
the
security
team
at
GitHub,
we
hire
some
of
the
best
people
in
their
fields.
They
know
their
services
better
than
anyone
else,
and
they
know
their
technology
is
a
lot
better
than
I
do,
and
anyone
in
my
team
will.
So
that
is
a
knowledge
Bank
of
information
that
is
essential
for
someone
like
myself
to
tap
into.
We
must
work
with
our
subject
matter.
A
Experts
ensure
that
we
have
those
conversations.
We
understand
the
Telemetry
available
for
their
Technologies
and
work
with
them.
They
know
what
a
bad
day
looks
like
for
their
services.
They
know
what
attackers
are
likely
to
go
after
and
so
discuss
things
like
detection
opportunities.
What
should
we
be
looking
for
and
kind
of
really
lean
on
the
people
within
your
organization
for
those
insights
into
very
specific
Technologies
across
your
organization?
A
Now?
The
last
point
is
something
I
can't
stress
enough
to
your
organizations,
and
that
is
that
good
security
requires
Great
Engineers.
Now
I
am
fortunate
enough
to
be
surrounded
by
what
I
believe
to
be
some
of
the
greatest
engineers
in
the
industry
and
our
security
Engineers
really
enable
us
to
deploy
a
really
effective
detection
pipeline
at
GitHub
the
custom
tools,
such
as
our
soar
platform
and
their
distributed,
alerting
that
we've
discussed
earlier.
A
These
are
a
huge
part
of
what
allows
us
to
really
spend
time
in
the
areas
that
matter
and
to
ensure
that
we
are
implementing
effective
detection
techniques
across
our
environment.
So
thank
you
very
much
for
your
time
here.
I
hope
you
learned
a
few
things.
I
hope
you
have
some
ideas
of
what
you
can
go
and
Implement
into
your
own
environments,
and
if
you
have
any
questions,
please
feel
free
to
reach
out
to
me.
Thank
you.