►
From YouTube: Customer Case Study Building DevSecOps into AWS EKS using RedHat ACS at Informatica OpenShiftCommon
Description
Customer Case Study: Building DevSecOps into AWS EKS using Red Hat ACS
Red Hat OpenShift Commons 2022 @ Kubecon/NA
Detroit, Michigan
October 25, 2022
Speaker: Pathik Patel (Informatica)
https://commons.openshift.org/gatherings/kubecon-22-oct-25/
A
B
Practices
into
AWS,
with
the
help
of
redhead
ACS
Advanced
cluster
security,
a
little
bit
about
myself,
I'll
start
with
that.
So
currently,
I
am
with
Informatica
I've,
been
here
for
past
six
years
and
few
months
before
that
I
worked
for
Netflix
and
Yahoo
currently
focused
on
kubernetes.
That's
that's
one
of
the
things
that
my
team
works
on.
Primarily
as
of
these
days.
We
are,
we
are
a
secret
engineering
team
mainly
focused
on
it.
All
of
you
can
reach
out
to
me
on
Twitter
at
pathikpa.
B
That's
my
handle
and
let's
now
take
a
little
bit
information
about
Informatica
as
well,
so
we
are
1.44
billion
in
Revenue,
as
of
today.
Informatica
has
been
around
for
a
long
time
about
27
years.
A
lot
of
you
folks
may
have
used
it
and
heard
about
it
already,
as
of
today.
Informatica
offers
intelligent
data
management
Cloud.
Our
customers
use
our
data
management
Cloud
to
do
API
integration,
data,
integration,
data,
quality
data,
governance,
Master
data
management
and
many
other
things
around
how
how
Enterprises
are
building
their
data
driven
organizations.
B
B
B
So,
from
my
point
of
view,
when
development
team
security
team
and
operations
team
they
are
coming
together
and
building
their
environment
in
a
singular
fashion
such
that
they
can,
they
are
moving
at
the
same
speed.
So
that's
that's
the
main
goal,
and
this
is
the
goal
that
my
security
engineering
team
also
follows.
As
a
security
engineering
team,
we
are
there
to
help
out
our
development
and
operations
team
to
ensure
that
we
are
taking
care
of
security
as
a
one
group
rather
than
operating
individually.
B
Also,
all
of
us
who
are
operating
in
the
cloud
we
have
heard
that
security
is
a
shared
responsibility,
so
on
that
devsecops
enables
a
shared
security
mindset
for
everybody.
This
is
where
a
development
team
understands
what
are
the
guard
rails,
how
those
guardrails
are
interpreted
in
the
development,
build
and
production
deployment
environment,
so
security
team
helps
out
defining
this
guardrails
and
then
helping
them
Implement
with
the
help
of
development
and
operations
team
on
the
shift
left.
B
So
these
are
the
gates
that
we
are
building
to
ensure
that
when
our
development
team
is
building
their
code,
operations,
team
is
pushing
that
code
out
from
the
build
to
production
environment.
They
are
given
a
continuous
feedback
on
how
their
software
piece
of
code
is
doing
and
what
are
the
checks
they
are
passing
and
failing
and
how
to
improve
upon
those
checks.
B
B
This
is
again
next
phase,
where
security
tools
are
integrated,
with
your
build
systems,
whether
it's
a
Jenkins,
Circle,
CI,
harness
or
any
other
build,
build
tools
that
you
are
using,
and
here
the
security
checks
are
extended
to
the
next
level,
where
we
are
also
making
sure
that
the
artifact
or
binary
that
is
being
built.
It
is
meeting
to
the
standard
of
our
compliance
and
security
requirements.
B
B
We
are
running
further
checks
that
we
Define
in
the
built-in
test,
but
this
is
more
of
a
validation
that
what
we
Define
in
the
build
environment,
is
it
the
same
in
the
production,
environment
or
not,
and
if
it
is
not,
how
do
we?
How
do
we
give
a
provide
a
feedback
to
our
deployment
team
on?
What's
what's
wrong?
What
is
the
drift
and
what
went
wrong
and
the
last
one
is
the
monitor.
B
B
B
What
is
built
and
what
is
running
in
the
production
also
make
sure
that
you
are
using
a
digitalized
container
images,
so
typically
tradition
or
traditionally
we
have
been
all
using
publicly
available
operating
systems
such
as
redhead,
centers,
Ubuntu
and
taking
that
in
and
on
top
of
it,
putting
our
binaries
or
artifacts
and
running
those
in
the
production
environment.
But
those
system
comes
with
a
lot
of
Burden,
a
lot
of
packages
that
are
not
necessary,
and
so
disturbless
images
like
Alpine
Linux,
is
very
helpful
here.
B
Also,
we
use
admission
controllers
heavily
to
ensure
that
we
are
securing
our
environment
from
the
get-go
from
the
first
entry
point
enable
your
audit
logs
for
kubernetes.
This
is
helping
out
both
in
the
security
and
debugging
point
of
view,
segregate
your
security
controls
from
the
build
time
and
runtime
perspective
also
adopt
service
mesh.
B
This
is
for
optimal
routing
and
encryption,
and
it
also
AIDS
in
your
security
visibility
as
well
like
which
ports
which
applications
are
talking
to
which
other
applications
and
pods
and
how
their
network
traffic
is
encrypted
and
segmented
and,
lastly,
Implement
CS
scans,
so
CS
scans.
There
are
three
different
verticals
into
it.
One
is
for
containers
your
Docker
container,
how
they
are
configured
from
the
Cs
perspective,
your
worker
nodes
and
your
clusters.
B
So
we
talked
about
segregating
your
repositories,
so
I'll
go
a
little
bit
deeper
into
what
I
mean
by
that.
Why
I'm
talking
about
segregation,
so
here
on
the
first
bucket,
we
have
the
development
environment.
So
this
is
where
a
developer
developer
is
writing
their
code
and
committing
that
into
a
container,
basically
building
out
a
container
image
that
can
then
be
used
further
in
the
development
cycle.
So
when
a
developer
is
putting
this
into
on
his
own
local
desktop,
we
enable
them
with
a
local
binary.
B
This
is
this.
This
is
actually
a
command
line
utility
that
is
provided
by
ACS
that
our
developers
are
using
to
scan
their
local
container
images
and
using
this
Scan
they
are
able
to
understand
what
are
the
dependencies
and
what
are
the
third
party,
libraries
or
various
libraries
that
are
used
and
what
type
of
vulnerabilities
exist
in
those
libraries.
So
this
is
the
early
feedback
system
where
they
are
able
to
look
at
their
own
code,
their
own
container,
and,
from
that
point
point
of
view
they
understand
that
here
are
what
here.
B
What
I
have
written
here
are
my
libraries,
and
this
is
what
I
need
to
fix,
so
once
they
are
happy,
they
have
patched
all
of
their
issues,
all
the
vulnerabilities
they
put
that
into
Dev
repository.
This
is
labeling
system
that
we
have
internally
implemented
so
that
developer
checks
in
their
containers
into
the
repository
after
that,
when
they
are
ready
to
go
into
QA
cycle,
it
goes
it
it.
Those
containers
are
promoted
to
the
queue
Repository
and
our
CI
CD
pipeline.
B
As
we
as
we
go
into
the
whole
life
cycle,
how
we
are
how
we
have
integrated
it
so,
to
put
it
into
perspective,
our
daily
container,
it
goes
into
awccr.
This
is
where
our
staging
or
QA
or
repository
comes
in
Jenkins,
which
the
tool
that
we
are
using
for
our
build
pipeline.
It
picks
it
picks
it
up
from
the
ECR
and
builds
its
whole
artifactory
binaries,
and
this
binaries
will
be
scanned
by
rated
ACS
for
validity
issues.
B
So
again,
going
over
the
real-time
security,
as
I
mentioned
previously,
like
digital
s,
images
are
a
very
important
factor
here.
At
Informatica
we
heavily
use
Alpine
Linux
as
our
digital
s
image,
and
even
some
of
the
Legacy
tools
that
we
have
or
Legacy
application
where
we
have
Mandate
of
red
hat
or
Centos
type
of
operating
system.
We
have
taken
that
operating
system
taken
that
image
and
reduced
it
down
to
whatever
the
minimum
things
that
we
need.
So
when
I
say
reduce,
we
are
removing
any
package
managers,
any
network
utilities
file
system,
modification
utilities.
B
B
So
this
is
again
like
we
previously
talked
about
it
in
our
in
our
pipeline
design,
and
all
of
this
is
only
possible
if
you
have
stronger
block
around
like
what,
where
your
build
system
accepts.
Your
image
is
from
so
we
are
locking
it
down
to
the
compliant
repositories
only-
and
this
is
achieved
by
specifying
specific
labels
that
are
accepted
during
the
build
time
now.
Moving
on
to
the
runtime
security,
this
is
a
very
similar
setup
using
Reddit
ACS.
B
So
whenever
container
images
that
they
go,
they
get
deployed
into
the
Amazon
eks
Reddit
ACS
is
deployed.
The
sensor
is
deployed
in
all
of
our
cluster
each
one
of
the
Clusters-
and
this
is
part
of
our
orchestration
layer
as
well
so
orchestration
layer
will
put
up,
will
install
a
Reddit,
ACS
sensors
on
all
the
Clusters
and
using
Those
sensors.
We
are
implementing
The
Continuous
scans.
This
continuous
scans
are
doing
multiple
things.
The
first
one
is
vulnerability
scans.
B
B
We
also
have
a
custom
policies
defined
in
our
admission
controller
as
well,
that
enforces
various
Arabic
related
privileges,
related
rules,
and
also
it
ensures
that
none
of
the
vulnerable
container
that
get
deployed
into
the
production
environment,
along
with
that
admission
controller,
is
also
making
sure
that
our
our
deployment
is
only
coming
from
the
broad
repositories
and
nobody
can
deploy
it
from
any
open
repositories
outside
of
informatica's
per
view.
So
we
have
a
lot
of
open
repositories
like
available
by
Docker,
Red
Hat.
B
Space.
Segregation
is
used
to
define
the
network
policies
and
our
service
mesh
policies
on
how
which
namespace,
which
parts
can
talk
to
which
other
namespaces
and
parts
along
with
that
admission
controller.
It
is
our
Central
Point
Central
enforcement
point,
which
is
enforcing
all
the
garteries
that
we
have
defined,
which
is
like
Port
security
policies.
B
Deployment
must
have
their
Network
policies
Associated
during
the
deployment.
How
API
servers
can
be
used?
Can
it
be
used
from
the
external
plans
how
it
is
used
from
internal
environment?
Who
can
query
the
API
servers,
so
all
of
this
is
put
into
the
admission
controller
and
in
the
last
monitoring
the
configuration
to
detect
CS
compliance
failures.
So
one
of
the
thing
that
we
we
may
assume
that
once
our
kubernetes
environment
is
deployed,
it
is
expected
that
kubernetes
system
itself
will
keep
monitoring
this
environment
and
do
the
self-healing.
B
B
B
But
even
with
all
of
that,
we
have
seen
that
people
the
deployments
they
don't
do
a
proper
cleanup
and
many
times
we
have
seen
this
Rogue
container,
which
may
be
used
for
testing
purposes
or
temporary
purposes,
and
then
they
never
got
cleaned
up.
So
can
this
configuration
monitoring
allows
us
to
detect
this
type
of
Road
containers.
B
Now
this
build
time
and
runtime
security
that
we
talked
about,
but
at
the
end
of
the
day,
we
have
to
figure
out
that
what
we
are
achieving
like,
how
do
we
get
a
single
pane
of
view
and
for
a
single
pin
of
view?
We
have
defined
this
wrist
lens
that
allows
us
to
figure
out
that
what
is
the
risk
profile
of
each
of
the
deployment
that
we
have
in
our
running
kubernetes
environment
and
to
Define
this
wrist
lens?
B
B
Are
there
any
Rogue
images
that
are
running
in
the
environment,
then
the
access
control
Access
Control
around
like
who
are
the
admins?
What
are
the
access
that
each
of
the
containers
has
in
the
kubernetes
environment?
The
next
one
is
Network
policies
who
can
talk
to
whom
also
for
the
power
security
policy
is
also
a
part
of
this,
and
then
the
cluster
security
itself,
like
how
clusters
are
configured.
What
are
the
CIS
parameters
configured
for
them
and
are
they
Meeting
those
CIS
parameters
or
not?
B
So
if
you
look
at
here
closely
here,
it
is
defining
all
the
policy
violations.
So
one
of
our
policies
says
that
if
there
is
a
container
that
has
any
vulnerability
that
has
cbss
score
greater
than
7,
then
that
should
not
get
deployed
or
if
it
is
deployed
and
a
vulnerability
is
detected
afterwards.
Then
it's
a
policy
violation
and
we
need
to
flag
it
out
similar
to
that
here's
the
process
with
uid
0..
So
this
is
the
example
of
Arabic
policies,
failure
and
similar
to
that
secret
mounted
as
environment
variable.
B
So
this
is
a
big
No-No
from
CIS,
so
it's
a
CIS
policy
failure
as
well.
So
these
are
the
combination
of
the
policies
that
we
have
identified
and
ultimately
this
policy
failures
are
calculated
under
the
risk,
and
that
is
the
priority.
Based
on
that,
we
have
defined
the
priorities
which
will
allow
our
developers
and
the
operations
team
to
prioritize
their
workload
and
figure
out
what
items
do
we
need
to
fix
first,
so
one
of
the
good
example
is
that
priority
will
go
higher
if
the
fixable
CVSs
score
greater
than
7
is.
B
So,
based
on
this
priority
based
decision,
we
are
feeding
this
data
back
into
our
development
team's
life
cycle
using
jira
tickets
and
in
the
last,
the
dev
shop
like
desec
Ops
takeaways.
That
I
want
to
mention
here.
So
the
first
one
is
Define
guardrails
and
document
them.
So
this
guard
rails
are
ultimately
helpful
for
everybody
to
be
on
the
same
page.
B
So
most
of
the
data
is
that
we
have
defined.
They
either
go
into
ACS
or
admission
controllers,
and
using
those
two
systems
we
are
able
to
ensure
that
all
of
our
kubernetes
deployment
will
fall
into
our
standards
and
the
requirements,
and
once
you
have
the
guard
rails,
one
of
the
one
of
the
main
advantages
is
using
this
guard
rails.
You
are
able
to
provide
feedback
to
developers,
so
this
is
also
very
important.
B
You
want
to
create
your
own
system,
using
whatever
technique
system
that
you
are
using,
so
that
the
issues
that
are
identified
by
admission
controllers
or
Reddit
ACS
goes
back
into
the
into
the
development
life
cycle.
This
will
allow
security
to
move
to
the
shift
left
and
reduce
your
secondary
issues
in
the
production
environment
and
when
I,
say
reduction.
B
One
of
the
major
thing
that
we
have
I
experience
is
kubernetes
amplifies
your
problems,
because
in
the
past,
when
you
were
running
like
1000
virtual
machines
and
running
your
products
or
application
using
that
now,
those
thousands
virtual
machines
are
multiplied
by
thousand
containers.
So
all
your
vulnerabilities,
all
your
issues
are
multiplied
by
thousands.
So
this
is
one
of
the
thing
that
is
important
to
keep
in
mind
and
to
to
control
this.
You
want
to
ensure
that
you
have
broader
mindset
work.
B
So
when
you
provide
your
early
feedback
to
developers,
they
will
be
sharing
your
load.
They
will
be
working
for
you
to
fix
the
security
issues
early
on
and,
at
the
end,
build
your
automations
to
provide
reporting.
It's
certainly
a
good
idea
to
put
your
issues
in
the
jira,
but
then
you
also
want
to
build
out
your
reporting
system,
your
dashboards
and
charts,
to
ensure
that
you
can
provide
a
routine
feedback
to
the
leaders
on
how
we
are
doing
as
a
security
organic
as
a
security-minded
organization.
A
B
B
B
So
those
innate
containers-
yes
it
so
during
the
run
time
it
probably
won't
get
scanned.
So
one
of
the
reason
is
we
have
a
schedule:
The
Continuous
scan
for
ACS
every
four
hours.
So
yes
during
the
runtime,
they
won't
get
scanned,
but
they
will
certainly
get
scanned
during
the
build
time.
So
we
know
what
like
dependencies,
what
are
the
configuration
from
the
docker
CS
point
of
view?
So
at
least
we
have
validation
during
the
development
time
and
build
time.
A
A
C
B
Yeah,
that's
a
great
question,
so
we
haven't
yet
found,
or
we
have
invested
a
lot
into
Port
security
policies
as
of
today,
so
the
network
policy.
That
is
something
that
we
are
looking
at,
but
as
of
right
now,
no
it
says
mostly
investment
issue
like
we
have
international
resources,
so
don't
have
time
to
go
into
the
next
one
right
now.