►
Description
This CD Foundation Meetup covers how Robert Krall from Rocket Software built out a test automation process using Jenkins and Pytest. Robert’s journey into continuous delivery started as a developer and shifted to CD automation when a better testing process was required. He will show us how to build out an automated testing pipeline that spans multiple operating systems, even connecting to z/OS and DB2 for big data applications. If you are striving to automate your test steps from your CD Pipeline, this meetup will be extremely helpful in seeing how to put it all together.
A
I
have
been
doing
a
lot
of
phone
calling
and
talking
to
people
about
our
open
source
project
artillius
and
through
that
experience
I
met
robert
and
we
started
chatting
and
he
started
chatting
about
his
background,
and
we
have
the
opportunity
here
here
today
to
learn
from
somebody
who's
been
a
software
developer.
He's
actually
got
a
he's.
A
I
have
the
it's
my
opinion
that,
while
orchestration
tools
are
really
important,
it's
what
we
orchestrate
them
to
do
that
creates
our
pipeline
and
makes
it
important.
So
I
bugged
him
and
said:
could
you
please
do
a
presentation
on
pie
test
and
how
you've
built
this
out
and
on
that
I
will
introduce
robert.
B
Sweet
thanks
tracy,
so
so
I
put
together
a
little
slideshow
before
we
get
into
the
real
fun
of
it.
So
the
agenda.
I
just
put
like
quick
overview
of
myself
high
level
background
on
the
product.
I'm
testing
the
current
tech
stack.
We
have
a
future
tech
stack,
a
quick
overview
of
pi
tests,
you're,
not
familiar
with
it.
B
So
excuse
me
a
little
bit
about
myself,
so
I
have
about
10
years
of
sql
development
experience
and
it
experience
using
both
microsoft,
sql
server
and
db2
for
z,
os
out
of
that
10
years.
Seven
of
it
I've
done
a
lot
of
etl
and
data
mining
data,
modeling
jobs,
so
I've
done
some
stuff
like
some
performance
stuff.
B
B
I
worked
in
various
all
kinds
of
different
industries
from
banking
to
paper,
to
consulting
to
law
firms
in
insurance
to
academia
and
the
state
government.
So
a
little
bit
of
my
education
background.
So
I
got
a
master's
degree
a
few
years
ago
in
data
science
at
the
university
of
st
thomas
in
st
paul
on
minnesota,
with
a
database
concentration.
B
Before
that
I
got
my
bachelor
of
science
in
management
information
systems
at
winona
state
in
well.
That's
supposed
to
be
winona
minnesota
on
saint
paul
minnesota
in
the
spring
of
2009..
So
I'm
working
on
an
automation
project
right
now
and
our
effort
is
to
convert
all
manual
tests
to
automated
tests.
We
started
this
project
back
in
like
january
of
2019
for
rocket
software.
B
Based
on
a
time
schedule,
so
a
quick
high
overview
of
the
product,
I'm
working
on,
so
I
work
for
rocket
software,
which
majority
of
you
probably
have
never
heard
of,
but
you've
probably
heard
of
ibm.
We
are
a
solutions
provider
for
ibm,
so
we
create
a
lot
of
software
for
ibm
and
one
of
the
software's
that
we
deal
with
is
the
ivf
security
guardian
product.
B
B
B
B
So
here's
some
examples
of
risk
commands
we
use.
We
can
stop
and
start
tasks
on
gls.
We
can
update
configuration
files
like
pdfs
files
on
the
mainframe.
We
can
install
and
uninstall
policies
with
the
appliance
we
can
run
jcl.
We
can
run
sql
statements,
we
can
copy
data
sets,
we
can
get
a
list
of
all
the
files
in
a
pdf.
B
We
can
stop
and
start
the
ims
control
region.
We
can
get
running
task
ids
if
we
want,
we
can
query
log
files
and
we
can
extract
the
lpar
or
the
system,
time
of
z
os,
so
here's
an
example
that
quickly
wrote
up
for
our
pipeline.
Basically,
we
have
several
pipelines
and
the
way
they
work
is
a
heartbeat
gets
kicked
off
before
the
smoke
is
kicked
off.
B
So
we
can
look
at
a
demo
here.
So
first,
let's
look
at
the
actual
pipeline
script,
so
we
do
everything
we
wrote
it
all
using
groovy,
groovy
scripting.
So
there's
a
slew
of
reasons.
We
did
this.
One
reason
is:
we
could
keep
all
the
jobs
and
get
and
have
version
control
over
them.
The
other
reason
was:
if
we
needed
a
new
jenkins
environment,
we
could
quickly
port
all
of
our
jobs
over
and
we
wouldn't
lose
anything.
B
So
I
don't
know
if
anyone's
familiar
with
groovyscript
or
not,
but
the
way
that
this
works
is
you
set
up
all
your
parameters,
so
you
can
have
different
types
of
parameters
like
strings
drop,
downs,
passwords
that
kind
of
stuff,
I'm
currently
only
using
strings,
and
so
we
have
different
types
of
parameters.
So
we
have
markers
what
git
branch
we're
going
to
pull
and
then
these
are
all
specific
on
the
actual
version
that
we're
testing.
So
we
test
a
couple
different
versions,
so
we're
passing
like
the
product
type.
B
If
it's
like
db2
data
sets
ims
the
task
name.
We
have
a
test,
that's
running
on
the
mainframe,
the
guardian
pulse,
which
is
the
appliance
version
numbers
some
zos
console
name
for
arrests,
the
lpar,
which
is
a
system
of
z,
os
some
configuration
names.
If
we
have
to
change
them,
the
database
name,
the
db2
port
and
then
the
next.
B
We
have
there's
different
stages,
so
the
first
one
is
to
download
the
test
packs
and
we
download
the
test
back
of
what
git
branch
we
specified
and
then
this
is
kind
of
where
the
real
meat
and
potatoes
come
into
play
so
for
pie
tests.
So
the
next
step
is
to
actually
run
it.
So
now
we
have
a
bash
script
and
we
do
some
echoes
just
so.
B
B
Excuse
me
so
that's
kicked
off
and
then,
after
that
we
keep.
You
know
all
the
results
in
jenkins
to
show
the
graph
and
then
a
post
on
successful.
We
go
on
to
the
next
one,
which
would
be
like
the
smoke.
So
if
we
look
at
our
our
pipeline
here
quick,
we
can
look.
We
can
review
this,
so
I
can.
Let
me
kick
this
off.
B
B
B
So
now
we
have
some
log.
So
what
happened?
Is
that
what
because
we
passed
a
marker,
so
we
passed
the
marker
called
db2
heartbeat,
so
the
heartbeat
only
tests
two
tests,
so
we
can
see
it.
It
collected
82,
it
deselected,
80
and
only
selected
two.
So
it
only
selects
two
tests
and
all
the
markers
are
dynamically
attached
to
test
cases.
I
can
show
you
how
we
do
that
in
a
little
bit
here
and
then
we
have
some
logging
here.
B
So
we
have
a
zeo
smf
class
which
deals
with
all
the
rest
commands
for
z
os.
So
it
tells
you
you
know
what
lpar
you're
testing
in
the
console
name
and
then
we
have
some
other
classes.
We
use
and
then
we're
downloading
the
configuration
file,
we're
updating
the
configuration
file
locally
and
now
we're
uploading
the
configuration
file
to
the
mainframe.
Oh,
this
is
all
done
through
rest.
Now,
what's
happening
is
stopping
the
task
on
the
mainframe
right
now.
B
B
Okay,
so
it
stopped
it
then
it
starts.
It
make
sure
it's
good
and
then
it's
gonna
set
up
the
test
case,
so
it
installs
the
policy
and
the
appliance.
And
then,
after
that
it's
gonna
run
some
jcl.
So
here's
some
jcl
that
ran
so
it
tells
you
what
lpar
system
it
ran
on
the
job
name,
the
job
id
and
who
ran.
A
B
And
then
it'll
output
some
time
stamps.
We
need
to
know
that
information
and
then
what
is
this?
This
is
oh,
so
then
this
is
we're
gonna
query
a
report
on
the
appliance,
so
these
are
all
the
values
that
were
passing
into
the
report.
So
if
you
had
to
like
troubleshoot
it,
we
could
like
log
into
the
appliance,
go
to
the
report
and
fill
in
the
variables
in
the
report,
and
then
this
test
case
passed
and
then
in
our
logging.
B
We
say
you
know,
here's
the
test
case,
name
what
we
expected,
what
the
actual
value
was
lpar
all.
This
is
like
helpful
for,
like
troubleshooting,
if
there's
an
issue,
job
names,
job
ids,
the
clients
that
it
ran
on
dashboards,
that
we
queried
db2
versions,
port
numbers,
that
kind
of
stuff-
and
then
it
goes
on
to
the
second
one
and
this
one
passed
as
well,
so
it
shows
either
expected
in
your
actual.
So
that's
good
and
then,
after
that
it
goes
on
to
the
the
smoke
job.
B
B
B
B
Hang
on
my
monitor
just
went
forward.
Okay,
so
so
here's
an
example
when
we're
setting
up
the
actual
session.
So
we
have
to
make
sure
that
the
like
we're
going
to
update
a
configuration
file.
So
we
update
the
configuration
file.
Then
we
stop
the
task.
Then
we
start
the
task.
Otherwise
the
changes
aren't
seen
in
the
task
if
we
update
the
configuration
file,
so
that's
that's
how
we
set
everything
up,
there's
the
first
run
and
then
there's
a
teardown
part
within
pi
tests.
B
B
Yep,
so
we're
not
we're
not
quite
there
yet,
so
we
would
like
to
get
it
to
like
once
a
developer
check
some
code
in
kick
off
some
tests,
but
we're
not
quite
there
yet.
This
whole
effort
started,
like
I
guess,
18
months
ago,
so
a
lot
of
work
has
been
done
in
18
months,
but
we
still
have
a
lot
more
to
do
so
currently
right
now,
it's
all
just
done
by
time.
B
A
B
Sure
so
this
tool,
specifically,
is
all
auditing.
So
all
we're
really
testing
is
data.
We
do
test
to
make
sure
that
some
configuration
files
are
done
right,
but
majority
of
all
these
tests
are
data
ensuring
that,
because
it's
for
auditing
purposes,
so
we
need
to
ensure
that
you
know
the
audit
trail
is.
Is
there
so.
A
B
A
B
A
A
B
B
You
can
pick
a
policy
here.
We
have
a
whole
bunch
of
them.
We
test
all
these
different
things
and
then
I
guess
you
can
select
more
than
one
and
then
you
just
say,
install
last
and
then
it'll
over
here's
what's
installed
currently
on
the
appliance
so
and
then
we
have
different
reports
and,
let's
see
if
we
go
to
like
now.
B
Of
course
I
haven't
logged
in
this
thing
in
a
long
time
and
apparently
I'm
forgetting
where
oh
dashboards,
so
we
have
different
reports
so
all
through
pi
tests
and
automation.
We
call
these
various
reports
all
through
rest
to
extract
the
data.
There's
no
data
on
any
of
these,
because
I
think
the
time
stamps
are
out
of
sync
now.
B
A
B
Yeah,
so
so
that's
a
good
question
I
mean
I
can
show
you
specifically
an
example
of
five
tests.
I
mean
it's
kind
of
to
be
honest
with
you,
there's,
probably
thousands
and
thousands
of
lines
of
python,
so
it
really
depends
on
what
you're
like
what
you
want
to
see.
One
big
if
you're
not
too
familiar
with
pi
test,
for
example,
everything
is
kind
of
driven
from
this
contest
file.
B
So
if
you
wanted
to
do
like
cli
parameters,
they
have
these
hooks
that
are
called
that
are
that
pi
test
automatically
knows
about.
So
one
of
them
is
pi
test,
underscore
ad
option,
and
so
you
can
add
whatever
cli
parameter
you
want,
so
you
can
see
I've
added
a
whole
slower
of
them
here
and
then
once
I
do
that,
then
I
can
call
pi
tester
the
cli,
and
it
knows
what
this
value
is
and
you
can
give
it
a
default
value
if
you
want
so.
A
B
Yeah,
and
so
it's
all
really
driven
a
lot
from
python,
so
we
have
libraries
and
so
here's
a
library,
so
kind
of
went
with
the
oop
design
of
programming,
so
we
have
classes
and
some
of
them
have
sub
classes
and
adherence,
and
so
here
we
have,
you
know
different
actions
you
can
take.
You
can
there's
a
function
called
like
start
task
right,
and
so,
if
you
want
to
start
a
task
first,
I
make
sure
that
the
task
isn't
running.
B
So
if
it
isn't
running,
then
I
can
run
it
and
so
like
this
is
a
rest
command
to
start.
The
task
on
z,
os
through
rest,
so
there's
different
rest
urls.
I
don't
know
if
people
are
familiar
with
rest
or
not,
but
rest
is
a
way
to
like
basically
communicate
with
whatever
service
you
want,
without
actually
going
through
a
gui
high
level.
So
there's
different
rest
urls
for
z,
zos,.
A
A
A
A
B
A
Like
it
feels
like
you
know,
you've
built
a
lot,
there's
a
lot
of
intelligence
around
what
jenkins
orchestrates.
Like
I
always
say,
and
in
this
case
it
looks
like
you're
almost
using
jenkins
as
a
as
a
centralized
job
scheduler,
and
basically
your
your
your
python
and
your
pi
test
is
doing
all
the
heavy
lifting.
B
A
B
Yeah,
I
think
you
could
the
hardest
part
of
the
entire
thing
is
like
the
setup
and
teardown
like
you
have
to
ensure
that
the
system
is
in
like
the
right
like
state
before
you
can
run
a
test.
Otherwise
the
test
will
automatically
it'll
fail.
You
know,
so
I
have
to
make
sure
that
certain,
like
policies
are
installed
on
the
appliance
and
if
they're
not,
then
it's
not
going
to
work.
A
B
Yeah
definitely
so
there's
actually
a
really
good
book
that
I
read
to
really
learn
pie
test.
Now,
if
I
remember
the
name
of
the
book,
it'll
probably
show
up
here,
I'm
almost
positive.
This
is
it.
This
is
actually
the
this.
Is
the
book
pie
test
appreciate
it
yeah.
This
is
the
book.
So
if
you
really
wanted
to
start
learning
about
pipes,
I
mean
pi
test,
I
think,
is
a
lot
of
a
great
functionality
and
I'm
just
like
touching
the
iceberg
of
it.
B
I
think,
but
this
book
is
extremely
helpful,
like
it
helped
me
a
lot.
I
never
read
it
cover
to
cover,
but
he
talks
about
a
lot
of
different
topics
that
are
very
good
and,
like
explains
it
really
well,
especially
for
someone
who's
never
really
used
it
before,
and
I've
really
used
it
and
drank
the
pie
test
kool-aid.
I
guess
so.
A
Stop
sharing
just
so
everybody
knows
I'm
going
to
quickly
share
my
screen.
We
are.
We
have
some
other
upcoming
events.
The
next
ones
will
be
if
you've
ever
wanted
to
understand
about
site,
reliability,
engineers
and
what
they're
up
to
this
should
be
a
pretty
interesting
presentation.
Shivagami
is
a.
She
runs.
The
sre
department
for
emirates
airlines
and
she's
got
quite
a
team
that
she
has
built.
A
Argo
is
a
cd
pipeline
that
is
cloud
native,
all
right,
everybody
from
the
cd
cd
foundation
online
meetup
and
the
new
mexico
devops
meetup
everybody
be
safe
and
have
a
fabulous
fourth
of
july,
and
we
will
see
you
july
15th
just
around
the
corner
and
thank
you,
robert
for
the
information.
It
was
super
helpful
and
I'm
sure
that
people
learned
a
lot
from
it
cool
thanks.