►
From YouTube: Ops Cross Stage Think Big - 2021-05-20
Description
Topic: Provide our users with a way to test their pipeline locally
A
Hi
everyone-
this
is
the
ops
craft
page-
think
big
today
is
may
20th
and
we're
here
today
to
discuss
across
a
topic
discussion
point
that
is
to
provide
our
users
with
a
way
to
test
their
pipeline
locally.
So
today's
facilitator.
Well
I'm
the
facilitator,
but
the
the
topic
today
is
going
to
be
posed
by
darren
so
there
and
why
don't
you
take
over
and
talk
to
us
about
the
problem
statement
or
the
job
to
be
done?
A
little
bit.
B
C
Yeah
yeah,
I
think
I
I
just
wanted.
The
first
bullet
was
just
like
the
the
the
high
level
what
users
are
asking
and
I
think
the
next
one.
Are
you
elaborating
on
like
why
so
I'm
just
going
to
read
it
like
the
user?
Would
our
users-
and
this
is
a
common
request
that
I'm
keep
getting?
They
asked
me
for
a
way
to
run.
Basically,
they
ask
for
tests,
but
they
actually
want
to
run
their
ci
pipeline
locally.
So
as
they
work
on
their
like
local
git,
they
are
doing
changes
to
the
pipeline.
C
B
C
Especially
especially
think
about,
like
you
have
like
really
large
pipeline
can
take
like
40
minutes
an
hour,
and
you
want
to
test.
You
know,
you
know
a
couple
of
the
jobs,
so
you
need
to
push.
You
need
to
wait
for
this
whole
thing
to
to
happen.
You
can
maybe
comment
out
a
few
jobs,
but
it's
it's
a
lot
of
it's
a
lot
of
overhead
for
for
our
users,
and
there
is
like
one
part
of
that
that
things
can
people
can
get
confused.
We
do
have
a
lintel
and,
like
the
lintel,
is
testing
the
validation.
C
It
is
validating
the
configuration
of
your
pipeline,
but
still
a
pipeline.
Our
pipeline
can
be
dynamic
and
and
when
you
test
the
pipeline,
you
test
like
the
whole
configuration,
but
when
you
run
the
pipeline
and
like
that,
if
I'm
on,
like
the
the
runner,
gets
like
the
job,
but
but
we
are
processing
the
pipeline
differently,
so
there
are
jobs
that
are
getting
created
and
some
jobs
will
not
get
created,
and
so
those
pipelines
can
be
dynamic.
C
And,
of
course,
if
you're
using
like
the
rule
syntax,
if
you
are
pushing
like
merge
requests,
if
you
are
on
different
branches,
so
pipelines
can
behave
differently.
So
even
if
it's
the
same
configuration
you
know,
the
same
pipeline
can
fail
and
the
same
pipeline
can
can
succeed
based
on
specific
conditions.
C
So
the
our
lintel
is
not
is
not
the
silver
bullet
that
will
solve
the
problem
to
our
users.
The
only
way
for
them
to
absolutely
know
if
the
pipeline
will
work
is
to
run
it
somewhat
somehow
and
the
only
way
they
can
do
it
today
is
running
to
our
gitlab
ci,
and
then
they
don't
want
it
because
of
what
that
just
mentioned.
C
B
A
B
Oh
I'll,
let
I'll
let
dob
answer
the
second
part,
because
he's
probably
had
the
most
recent
conversations
with
customers
about
kind
of
like
what
their
ideal
outcome
would
be.
I'll.
Just
talk
a
little
bit
about
the
first
piece,
which
is
like
the
example
solution,
so
for
everyone
that
doesn't
really
know
today.
There
is
this
capability
in
github
called
github
runner
exec,
which
was
a
feature
from
many
many
many
moons
by
right,
and
that
gitlab
runner
exact
feature
does
allow
you
to
do
some
type
of
pipeline
debugging
locally.
B
B
Personally,
I
wish
it
was
not
in
the
run
or
tight
feature,
and
so
today
there
are
a
couple
different
solutions,
but
this
specific
one
is
using
the
limited
capabilities
that
are
still
there
in
gitlab
runner,
exec
to
at
least
get
to
be
able
to
debug
pipelines
locally,
and
then
one
example.
I
share
this
one.
Article
as
you
as
you
scroll
through
it
and
the
end
of
the
article,
the
the
blog
post
individual
mentions.
Well
here
are
the
limitations,
because
gitlab
gonna
exec
doesn't
support
all
of
gitlab's.
B
You
know
very
cool
new
pipeline
features,
and
so
here's
how
you
kind
of
work
around
it,
so
it
so
summarize,
we've
got
this
thing
called
get
runner
exact.
It's
it's
kind
of
something
that
did
a
little
bit
of
this,
and
people
are
kind
of
working
around
it
and
there's
just
one
example
and
dove.
You
want
to
answer
the
question
about
the
ideal
outcome.
You
think
users
might
want.
C
Yeah,
I
I
I
I
can
only
guess,
okay,
because
we
didn't
go
to
do
any
solution,
validation,
and
this
is
like
one
of
the
hardest
problem
that
I
have
like,
because
I
have
like
no
idea
how
we
can
like
solve
this
problem
like
for
me.
I
think
the
ideal
solution
will
be
like
some
sort
of
a
cli
that
you
just
like
some
gitlab
one
or
some
some
basic
command.
Then
you
pass
some
parameters
and
like,
ideally,
you
can
like
omit
or
run
only
specific
jobs
within
a
pipeline.
C
Let's
say
you
want
to
test
like
job
number,
three
out
of
like
10
pipelines
or
whatever,
or
maybe
the
entire
pipeline,
but
it
will
run
it
will
land
the
pipeline
automa
locally
on
on
your
instance
and
will
not
need
to
trigger
this
whole
ci
makers
mechanism.
Maybe
it's
with
like
some
sort
of
a
local
runner
that
we
have
maybe
it's
an
extension
for
the
gitland,
the
gitlab
runner
exec.
I
like.
I
have
no
idea.
I
actually
didn't
know
about
this
walking
on
the
solution,
and
I
assume
that
our
users
don't
know
that
as.
D
C
And
but
yeah
I
mean
that
that's
that's
one
of
the
hardest
challenges
that
I've
faced,
because
when
I
ask
you,
okay,
what
would
what
do
you
want
like?
What's
the
ideal
solution?
It's,
it
was
very
hard
for
them
to
articulate
what
they
want
because
like
like,
yes,
we
have
a
linter
and,
yes,
we
can
do
different
things,
but
it
was
very
hard
to
to
understand
what
would
be
the
ideal
solution
for
them.
A
A
D
B
Yeah,
maybe
I
can
stop,
I
mean
on
a
high
level,
one
of
the
the
initial
challenges,
regardless
of
how
you
solve
this
like
one
of
the
the
challenge
is
challenges.
Challenges
is
that
we
have
a.
We
have
a
number
of
capabilities
in
github
ci
right
in
terms
of
what
we
can
do
at
a
pipeline,
and
the
question
is
depending
on
whatever
the
solution
is:
do
you
copy
and
do
you
replicate
all
of
those
capabilities?
B
So
that's
one
of
the
challenges
is
that
if
you
do
define
whether
it's
a
cli
approach
and-
and
we
can
get
to
this
in
your
other
question
on
high-end-
in
terms
of
what
what's
happening
in
the
market,
for
example-
github
actions-
there's
this
thing
for
github
actions
called
act,
which
is
basically
your
local
cli
I
was
just
reading
about.
But
what
they're
doing
with
act
is
not
every
single
feature
on
capability
in
gitlab,
app
and
github
actions
is
available
for
you
to
debug
or
test
and
act.
B
C
B
You
could
potentially
come
up
with
something
where
you
are
not
creating
a
cli
or
creating
yet
to
the
third
product,
but
then
the
the
question
of
where
is
the
the
code
being
executed
so
that
you
are
debunking
it
happening.
You
have
two
choices,
a
local
system
or
back
in
the
class.
I
think
yeah.
It's
that's
a
really
tough
problem
to
be.
A
A
E
Yeah,
I
was
quickly
scanning
through
an
article
I
read
by
circle
ci,
and
I
think
this
is
one
of
the
use
cases
for
their
cli
as
well
like
to
validate
the
config
file
syntax.
But,
like
I
said
one
of
the
use
cases,
the
cli
is
also
used
for
a
range
of
other
things.
So
that
just
makes
me
wonder,
like
just
for
the
sake
of
validation,
working
on
something
as
complex
as
a
cli,
how
physical
that
would
be.
E
Yeah,
so
what
I
see
is
that
I
mean
this
is
just
one
of
the
uses
for
a
cli
and,
of
course,
ccli
can
do
a
lot
more
right,
so
either
like.
If,
if
we
move
in
that
direction,
do
we
also
envision
expanding
the
expanding
the
range
of
things
that
users
could
do
with
the
cli?
I
mean
that
would
be
great
yeah.
That
would,
of
course,.
C
E
A
B
Well,
let's
see
well,
github
is
kind
of
new
to
the
game.
I
I
only
started
just
browsing
this
morning
in
terms
of
what
github
has
and
I
only
just
came
across
the
one
article.
I
don't
know
if
they
have
anything
official
which
is
again
this
kind
of
act
thing
I
think,
they're
new,
I
think
circus
civ,
I'm
sorry
the
civil
ci
cli
option.
Leviticus
point
is
one
pattern
and
again,
that's
still
kind
of
like
I
said
to
have
the
cli
installed
locally.
B
I
think
travis
has
the
ability
for
you
to
ssh
into
the
virtual
machine.
That's
executing
your
pipeline
jobs
on
their
system,
so
it's
kind
of
like
enough
and
they
may
use-
do
some
kind
of
testing
right.
So
yes,
but
you're
still,
of
course,
relying
on
executing
the
pipeline
and
logging
into
that.
You
know
what
I
mean
that
machine
to
debunk
it.
B
So
again,
I
haven't
really
done
that
type
of
analysis
in
terms
of
what
the
comparison
was
doing.
In
my
simplistic
mind,
I
was
kind
of
thinking
about
like
when
I'm
working
on
say
it's
really
a
bad
example,
because
again
this
is
happening
locally
right,
but
I
see
I'm
in
my
visual
studio
code,
ui
identifying
and
maybe
like
testing
out
something
with
go.
B
I
can
do
everything
within
that
id,
but
the
problem
with
that
example
is
obviously
the
ide
is
compiling
my
go
code
on
my
local
laptop,
but
from
a
user
perspective,
I'm
still
within
the
ide
interface
yeah.
I
think
for
me
it
would
be
like.
Can
I
do
it
within
the
interface
I'm
in
versus
having
to
go
to
somewhere
else
would
be
interesting.
That's
just
my
point
of
view
and
if
anyone
has
a
different
point
of.
B
B
C
C
C
B
A
Let's
take
a
step
back,
let's,
let's
start
thinking
small
right,
because
now
we're
like
going
the
direction
where
we
think
okay,
it
can
be
potentially
a
different
problem,
but
if
we
think
a
bit
smaller,
what
is
what
is
the
hardest
part
of
this
problem
that
we
have
to
solve
before?
We
can
even
consider
thinking
about
an
nbc
or
thinking
about
a
problem
that
we
want
to
to
validate
directly.
B
A
B
B
A
Ide
yeah:
no,
it's
not
I'm
not
thinking.
Where
would
this
leave,
but
what
will
be?
What
would
be
the
flow
for
people
or
users
to
get
to
the
point
where
they
are
debugging
there
and
how
that
interaction
would
overlap
with
what
we
offer
today
in
the
web
id
but
off
the
question,
but
it's
a
very,
very
blurry
line.
B
C
B
The
configuration
syntax
can
config
right
right,
so
we've
got
center,
second
fix.
So
what's
the
next
step?
Well
that
the
actual
workflows
stitch
together,
I
guess
you're
covering
that
with
config.
It's
like.
I
guess
that,
for
me,
is
what
is
definitively
bugging.
Are
we
trying
to
say
that
if
I
define
a
pipeline
with
three
jobs
in
them,
I
want
to
ensure
that
all
those
jobs
will
run
so.
C
So
I'll
explain
so
the
the
lintel.
If,
if
I'm
running
like
a
certain
set
of
commands
and
those
commands
failed,
the
lintel
won't
check
that,
because
the
letters
say
okay,
the
configuration
is
right.
There
is
like
a
bunch
of
skips
that
is
running.
They
don't
know
the
outcome
of
the
skate.
Okay,
they
don't
know
if
it
turns
out
like
exit,
zero
or
or
exit
exit
one,
because
it
is
not
executing
anything.
C
Just
like
I
guess
again
guess
we
have
some
rules
that
they
are
checking
some
dependencies
and
I'm
always
simplified.
Of
course
it's
very
complex,
but
it
is
not
like
debugging
and
checking
if
the
job
passed
or
succeeded.
It
is
just
saying:
hey,
yes,
the
job
can
run,
but
we
don't
know
like
what
would
be
like
the
outcome
of
the
job,
and
this
is
what
they
want
to
say.
C
They
want
to
see
that
the
job
build
finish
successfully
and
not
wait
for
like
to
install
to
set
up
and
do
all
those
things
in
in
in
our
in
in
our
cr.
They
won't.
They
want
to
run
it
locally.
They
want
to
run
like
only
I
don't
know,
only
test
jump.
Okay,
I
don't.
I
want
to
announce
this
job
to
see,
if,
like
those
three
commands
that
I
just
just
set
up
works
because
like
in
like
every
job
we
are,
we
can
we
can
like
build
things
from
from
sketch.
We
can.
C
We
can
run
like
like
npm
install
and
all
of
that
in
each
and
every
job
and
assuming
I
need
to
go
to,
like
I
don't
know
four
or
five
six
different
jobs,
then
I
guess
I
need
to
install
it
every
time
and
it
takes
a
lot
of
time
and
I
want
to
check
only
job
number
five
out
of
six,
for
instance.
So
something
like
that.
F
In
the
example
you
just
mentioned,
when
a
user
wants
to
test,
let's
say
a
build
job
or
a
test
job,
they
want
to
do
this
against
an
actual
commit
from
their
repo,
so
they
want
to
actually
run
the
test.
Script
against
their
code
or
something
like
a
dry
run
just
want
to
make
sure
the
tests
go
to
the
next
step.
C
F
Right
I
asked
I
asked
because
if
we're
talking
about
some
sort
of
local
environment
that
that
the
users
can
run
against
their
code,
then
if,
if
the
local
environment
that
runs
their
code
is
very
complex,
then
it
could
be
really
difficult
to
make
sure
all
users
were
properly
supported.
With
this,
I'm
thinking
like
huge
pearl
environments,
for
example,
they're
really
hard
to
run
on
a
local
machine.
C
E
A
Maybe
that's
our
next
step
right.
What
is
what
do
we
need
to
check?
What
do
we
need
to
validate
from
a
technical
point
of
view,
and
what
are
the
riskiest
assumptions
we've
made
from
what
we're
talking
about
here
that
it's
not
true
would
cause
this
to
fail.
So
we
know
that
it's.
It
will
be
very
difficult
to
support
this
execution
in
smaller
than
local
environments,
but
what
else
would
be
a
risk
for
this
for
the
solution
for
this
problem
in
general,.
B
I
think
what
daniel
said
earlier
is
also
a
risk,
which
is,
I
guess,
suggesting
it
does
said:
hey
simple,
simple
pipelines
are
easy,
but
then
daniel
suggesting
more
complex
workflows
are
very,
very
risky
because
then
we
get
into
the
situation.
If
you
had
a
solution,
do
you
really
support
all
these
more
complex,
workflows.
F
F
Darren
you
had
mentioned
earlier
about
possibly
supporting
a
subset
of
features
of
the
ci.
I
think
another
risk
is
choosing
what
the
subset
will
be
in
a
way
that
is
useful
for
for
specific
workflows
or
specific,
specific
personas,
because,
again,
depending
on
what
we
take,
we
might
not
be
properly
supporting
anyone,
or
at
least
not
well
enough.
E
C
Yeah,
I
think
like
supporting
this.
I
mean
it
is
if
this
is
something
that
the
gitlab
is
offer
is
offering,
then
we
need
to
support
it
and,
as
as
daniel
mentioned,
it
will
be
very
hard
to
make
sure
it
is
running
on
different
environment.
Number
of
permutation
can
be
like
it
can
be
a
lot,
so
it
will
be
hard
to
to
support
supported
with
like
the
different
variations
that
our
users
are
running.
I
mean
we
can
say,
hey
it's.
A
C
It's
a
complex
problem,
I
think,
like
we
need
engineers
insights.
I
think
I
think
that
the
mvc
can
be
like
writing.
This
whole
thing
that
we
just
on
an
issue
and
tag
like
specific
members
of
like
both
honors
and
see.
I
basically
all
verify
I
think
and
ask
for
their
opinion,
because
that's
a
problem
that
our
engineers
also
have
for
our
devops
also
had-
and
maybe
some
of
them
have
some
ideas
on
the
possible
solution.
C
A
A
A
C
A
A
A
So
I
think
an
obvious
next
step
would
be
to
create
an
issue
to
continue
this
conversation.
I
think
and
draft
write
and
define
this
problem,
define
the
scope
of
the
questions
that
needs
to
be
answered
and
identify
some
people
of
of
contact
in
the
teams
and
of
do
you.
Do
you
have
any
people
any
names
in
mind
or
just
in
general,
backing
engineers
from
verify.
A
I
know
that
there
is
gone,
but
usually,
though,
for
the
person
that
brings
the
the
topic
is
the
one
that
creates
this
issue.
So
I
will
assign
this
to
you,
so
you
can
also
well
you
have
other
technicalities,
the
ideas
in
your
head
and
you
can
create
that
issue
and
label.
It
label
the
issue
with
the
with
the
click,
the
big
big
label.
So
we
can
follow
the
conversation
and
we
know
the
outcome
from.
A
C
C
That's
a
hard
problem.
I
mean
I
I
I
didn't
thought
we'll
have
like
an
answer
here,
but
I
this
is
something
that
me
and
dan
discussed,
and
he
said
yes,
these
users
also
bring
this
a
lot
and
my
user
bring
it
a
lot,
and
we
this.
This
is
like
one
of
the
only
questions
that
we
don't
have
any
answer,
so
even
surfing
surfacing
it
and
knowing
that
you
know
it's,
okay,
that
we
don't
have
an
answer.
C
It's
something
that
again,
it's
like
a
step
in
the
right
direction
for
me,
because
it's
not
like
a
something
easy
to
to
build
and
yeah,
and
but
I'm
okay
with
the
action
item
and
like,
hopefully,
we
can
move
forward
with
an
asic
discussion
and
if
we
need
we
can,
we
can
do
like
another
thing
called
with
more
people
involved.
When
we
see
more
comments
in
the
issue.
A
F
Yeah,
it
was
just
a
quick
comment.
I
think
it
would
be
nice
to
think
of
opportunities
to
improve
perceived
performance
for
our
users,
perhaps
in
ways
that
we
don't
really
need
to
change
how
our
backend
works
or
create
a
lot
of
new
apis
or
cli's,
but
rather
just
breaking
down
small
pieces
that
make
it
seem
faster
right.
No,
don't
know
if
I
have
an
example
in
mind,
but
it's
just
a
thought.
C
I
I
I
have
example.
This
is
like
a
good.
This
is
a
good.
This
is
a
good
discussion.
I
think
it's
a
it's
a
bit.
It's
a
separate
conversation,
because
even
if
we
will
speed
the
pipeline,
still
user
would
like
first
of
all,
users
always
like
like
speedy
pipelines,
regardless
of
the
problem
or
any
other
problems.
So
and,
like
I
would
say,
is
the
smallest
things
we
can
do
is
to
provide,
in
our
documentation
best
practice
on
how
to
speed
your
pipeline.
C
For
instance,
using
a
cache.
You
can
speed
your
pipeline
using
images
like
locally
instead
of
pulling
them
from
docker
hub
can
speed
your
pipelines
and
what
else
I
think
like
if
you're,
using
like
the
default
script
and
after
escape
it
can
speed
up
your
pipelines
like
there
are
like
multiple
way.
You
want
parallel
jobs.
You
speed
your
pipeline,
so
we
can
probably-
and
as
I
mentioned
it's
a
topic
for
a
different
discussion,
but
it's
an
important
one.
C
We
can
probably
list
out,
like
all
the
best
practices
that
all
of
us
know
in
our
engineers,
in
even
like
one
page
in
our
documentation
and
then
provide
some
sort
of
a
link
from
the
product.
Hey,
you
want
a
speedy
pipeline,
it
even
as
the
pipeline
process,
because
when
the
pipeline
process
it's
it's
a
nightmare,
I
mean
nobody
sit
and
look
at
the
pipeline
processing.
You
hit
the
pipeline
and
you
go.
You
do
something
and
you
come
back.
C
You
see
it
keeps
spinning
and
keeps
spinning,
and
you,
like
you
know
you
want
to
throw
yourself
on
it
from
the
window,
but
I
mean,
if
you
have
like
a
link,
want
you
on.
Do
you
want
to
run
your
pipeline
faster
in
a
speedy
pipeline?
You
click
there.
You
see
like
best
practices,
how
to
do
different
things.
As
I
mentioned,
this
could
be
like
a
good,
a
good.