►
Description
James Ramsey and Marin Jankovski discuss the challenges of security development and use of confidential merge requests
A
B
A
We
kinda
have
to
do
two
things
and
in
one
go
because
now,
with
this
CD
process,
things
became
extremely
more
difficult
and
the
situation
was
already
complex
with
you
know
the
development
on
two
places-
I
am
sorry.
I
gave
you
these
documents
only
recently,
and
this
was
just
like
a
brain
dub
of
what
was
in
my
head,
so
that
we
have
something
you
know
to
talk
true,
but
but
basically
you
know
from
from
my
understanding
of
the
confidential
merge
requests.
It
is
a
very
simple
feature
right
like
it
goes
from
a
confidential
issue.
A
B
A
B
A
A
A
Used
the
wrong
word:
there
yeah,
it
made
it
harder
for
sure,
but
you
know
another
thing
that
makes
things
more
complicated
is
our
scaling
up?
Is
a
company
right.
So
now
we
have
even
more
engineers.
So
what
used
to
be
like
50m,
ours,
too
urgent
port?
Now
it
moves
250
200.
When
you
add
back
ports,
which
is
it's
it's
not
it's
not
scalable
like
we
cannot
scale
like
if
we
continue
doing
things
like
we
are
doing
right
now.
A
A
Let's,
let's
go
like
I
want
to
run
it
through,
just
from
the
developers
perspective,
how
this
looks
and
then
I
want
to
run
you
through
how
this
looks
from
our
perspective,
when
we
actually
need
to
cut
the
release.
So,
first
of
all,
you
know
that,
like
everyone
is
working
on
dot-com
publicly
and
when
you're
set
in
that
workflow
switching
contexts
to
do
security
releases,
even
if
it's
only
like
ok
now
you
need
to
work
on
dev.
It
is
extremely
complicated
because
you
have
this
automation
in
your
fingers.
A
Right
and
you're
gonna
push
a
branch
as
soon
as
you,
you
know,
get
an
issue
for
example,
so
that
automatically
has
a
chance
of
exposing
things.
So
that's
the
problem
number
one
problem
number
two:
is
you
go
through
the
regular
process
of
the
review
and
everything
that
goes
through
it,
but
the
difference
is
the
reviewer
doesn't
go
and
merge
your
merge
request
when
it's
ready
it
has
to
sit
until
the
release
me
can
be
ready
in
some
cases
which.
A
B
Just
clarify
I'm
one
of
these
myself
so
that
the
murder
quest
is
created,
targeting
master
and
not
merged
and
assigned
to
a
bot.
Is
that
correct,
correct
interesting?
So
then,
at
the
time
the
security
release
is
cut,
is
master
brought
up
to
previously
to
release,
and
then
everything
is
merged
and
I.
A
Absolutely
so
the
alternative.
The
alternative
to
this
is
release
as
soon
as
you
have
something
that
would
be
great,
but
we
don't
have
the
resources.
First
of
all,
we
don't
have
the
people,
we
don't
have
the
tooling.
We
don't
have
anything
to
do
something
like
this,
and
it
would
also
mean,
with
the
amount
of
security
reports
coming
in,
that
we
would
have
to
do
security
release
every
week
and
probably
a
couple
of
them,
because
there
is
no
sequence
like
this.
A
A
And
we
can
also
like
have
this
requirement
from
security,
that
we
cannot
mix
non
security
and
security
fixes
in
public
releases
so
which
do
need
to
bundle
them
into
one
go
in
order
to
release.
You
know
a
security
release
which
only
contains
security
fixes
so
for
comm
that
does
not
matter
so
CD.
There
is
fine
with
it,
but
then
we
cannot
do
snapshot
of
whatever
ship
on.com.
We
cannot
do
a
snapshot
of
that
and
say
this
is
self-managed
release
because
it
might
contain
security
fixes.
B
A
Seeing
the
problem
already
right,
imagine
that
now
that
the
developer
a
month
later
coming
in-
and
we
have
edge
cases
like
this,
where
the
backwards
merge
cleanly,
but
the
master
doesn't
merge
clearly
or
there
are
even
sillier
things
where
master
merge,
request
merges
cleanly,
but
one
of
the
backward
doesn't
now.
What
we
need
to
do
is
go
and
hunt
down.
A
Whoever
did
this
to
fix
this,
because
we
don't
have
the
time
or
well
the
knowledge
to
fix
all
of
those
which
makes
it
into
you
know
a
security
release
that
would
take
a
day
to
release
sometimes
can
be
a
week
during
that
time.
Everything
stops
because,
as
soon
as
we
merge
as
soon
as
we
merge,
that
means
publicly
public
repositories
diverge
from
our
private
repositories
and
things
accumulate.
B
B
A
I,
don't
I,
don't
I,
don't
have
a
love
around
ballpark,
but
let's,
let's
check
it
out.
We
had
security
release
just
recently,
so
I
can
find
out
like,
for
example,
how
many
security
little
security
leery.
So
we
do
backports,
so
three
releases
backbone
yeah.
That
means
three
releases
times
c
e
plus
e
e
six.
A
That
six
releases,
so
that's
six
times
the
merchant
west.
So
we
have
one
two:
three,
four:
five:
six:
seven,
eight
nine
10
11
12
13
14,
15,
16,
17,
20,
21,
22,
23,
merge
of
23
fixes
went
out
in
the
last
security
non-critical
security
releases
release
so
23
times.
6
is
the
amount
of
merge
request
that
we
had
to
handle
this
release.
That
is
so
it's
20
times
3
times
6,
but
then
also
+
23,
because
we
had
to
do
23
merges
into
master.
B
A
B
A
A
B
A
B
Because
yeah
so
I
I
remember,
we
discussed
this
a
little
bit
but
I
think
I
discussed
it
with
Nick
and
put
it
on
an
issue
it's
discussing
with
some
security
people
but
like
if
we
have
public
by
default.
There
is
no
way
to
prevent
stupidity
like
all
like
human
error,
don't
like
well
I
like
even
if
there
we
put
some
sort
of
confirm
on
every
Porsche.
It's
like
you're,
pushing
a
new
branch
like
you
have
to
use
this
special
push.
A
B
A
Have
to
name
your
branches
in
a
certain
way
so
say
everything
that
is
security
has
to
be
security.
/
everything,
that's
a
feature
feature
/
everything
that
is
a
bug
bug
/,
so
you
have
to
have
like
a
conscience
effort
of
writing
this
down.
Everything
else
gets
rejected.
Now
this
is
an
option
that
I
didn't
push
yet,
but
I'm
gonna
start
working
on
and
proposing
going
further
because
we
are
at
that
size
right
now.
Well,
we
have
to
enforce
something
like
it.
B
Is
possible
I
like
this
approach,
because
I
think
it's
it's
a
good
compromise
between
like
enforcement
and
it's
a
relatively
cheap
check
to
do
the
branch
name.
So
it's
not
gonna
have
like
negative
performance
impacts.
I,
don't
think
it's
particularly
I.
Don't
know
problematic.
I
I
use
this
kind
of
naming
scheme
myself
for
branches
so
that
I
can
find
things
like
III
prefixing
feature.
/J
Ramsay
slash
my
thing
so
like
I'm,
treating
them
like
folders
and
I've
always
thought
it
would
be
great.
B
A
Be
really
awesome,
but
we
need
to
take
up
think
about
Forks.
How
do
we
do
with
Forks
like?
That
is
my
main
concern,
because
we
need
to
make
sure
that
we
don't
block
any
I,
don't
know
any
action
from
community
community
there.
Theoretically,
there
is
nothing
to
go
wrong
there
because
they
are
pushing
to
their
branch
and
they're
submitting
their
branch
to
us,
but
we
need
to
kind
of
consider
the
edge
case
there.
If
there
is
one
we
didn't
go
that
far
yet,
but
that
is
solving
yes,
that
would.
B
B
Issue
I
can
think
of.
There
is
not
so
much
public
community
contributors,
but
internal
contributors
who
use
Forks,
I,
I,
encouraged
and
there's
a
number
of
people
in
my
team
that
act
please
for
King,
because
it's
critical
that
we
use
it
and
dog
food
for
King.
So
the
issue
there
would
be
that
those
internal
team
members
that
use
a
forking
workflow
would
then
bypass
all
the
checks.
Absolutely
and
that's
a
risk.
A
B
A
A
B
A
Have
this
situation
where
things
are
just
continuously
going
in,
and
we
now
have
this
timing
problem
of
everything
that
we
wanted
to
prep
for
the
security
release
and
backwards
is
ready.
We
need
to
merge
and
tag.
That
is
all
right
and
it
goes
through
the
process
of
QA
with
security
and
others.
But
now
we
need
to
ensure
that
we
dump
those
changes
in
master
so
that
we
can
pick
it
up
to
calm
and
we
need
to
ensure
that
those
changes
get
into
calm
before
we
are
ready
to
publish
the
release
right.
B
A
B
A
A
So
we
can
ahead
way
for
that,
one
which
delayed
the
security
lives,
which
then
delayed
as
deploying
all
of
those
things
in
calm.
So
we
have
this
cat-and-mouse
game
where
one
depends
on
another,
and
the
timing
is
everything
and
automating.
This
thing
is
is
nearly
impossible
at
the
moment,
at
least
so
that
is
the
overview
of
the
problem.
Let
me
move
this
item
further
up,
because
I
already
gave
an
overview
of
confidential
merger
requests
and
the.
B
Problem
with
the
deployments
is
that
it's
there's
not
really
single
source
of
truth
anymore.
All
right,
everything
like
if
you
have
a
single
source
of
truth
and
master
pastor
is
read
like
it's
there's
one
place
you
need
to
block
like
and
that's
like
a
tractable
problem,
but
then,
if
you've
got
like
oh
no
also,
this
other
private
fork
needs
to
be
blocked
and
it's
not
really
blocked.
It's
only
blocked
from
doing
this.
B
A
Are
only
deploying
from
a
private
repository,
but
public
and
private
repository
are
using
the
mirroring
feature,
which
means
that's,
why
I
told
you
that
we
we
diverge
so
as
soon
as
something
is
merged
into
the
private
repository
public
is
no
longer
the
source
of
truth.
Now
it
only
this,
so
it's
now
up
to
us
to
get
those
changes
that
are
in
the
private
repository
back
to
public
and
like
sync
them
up,
so
they
are
in
sync.
A
A
It
really
depends
on
what
gets
merged
into
into
master.
In
the
meantime,
in
90%
of
the
cases,
conflicts
are
manageable:
human,
manageable,
at
least,
if
nothing
like
we
need
to
find
someone
who's
gonna
help
us
automatically
resolve
all
unlikely,
like
I
haven't
seen
a
chance
where,
when
it
clashes
like
when
there
is
a
merge,
conflict,
I
haven't
seen
a
case
where
we
could
automatically
resolve
it.
Somehow,
yeah.
B
Like
so
I
think
I
proposed
on
a
number
of
occasions,
like
some
way
of
like
automatically
merging
the
changes
into
the
the
direct
look
at
the
private
repo
from
the
public
repo,
so
to
prevent
them
diverging
like
and
keeping
up
like
if
they
can
keep
up
keeping
them
in
sync,
it
I
wonder
if
it
would
be.
It
would
change
things
if
we
could
know
in
the
merge
request
in
the
public
repo
if
it
was
gonna
cause
a
conflict
with
the
private
one
like
if
you
could
say
that
this
is
like
a
confidential
pair
so
like.
B
A
You
want
me
to
tell
it
like
I
know
what
you're
aiming
at,
because
we
have
something
for
this.
So
we
have
this
project
called
merge
train,
it's
very
simple
and
naive.
What
it
does
is
it
clones
two
repositories,
so
we
use
it
for
syncing,
ce2
ee,
I
oh,
you
might
have
seen
it
in
action,
but
basically
what
it
does
is.
It
prefers
EE
as
a
source
of
truth
and
picks
all
the
conflicts
resolution
in
favor
of
a
submerged
strategy.
A
Ours
is
EE,
I
think
so
what
happens?
Is
we
have
process
in
place
to
say
that
if
you
have
a
CI
merge
request
that
has
an
E,
conflicting
change,
possibly
a
conflicting
change?
You
need
to
submit
an
e
counterpart,
merge
that
first
merge
te
den
and
merge
train
just
collects
two
changes
without
any
conflict.
So
the
idea
we
had
my
team
is
still
not
up
for
this,
but
we're
gonna.
A
We're
gonna
have
to
try
something
it's
what
if
we
say
that
instead
of
mirroring,
we
use
the
merge
train
project
and
it
will
take
public
repository
as
one
source
of
truth
and
private
as
another
private
would
always
win,
because
it
has
security
fixes
in
it
so
constantly
pull
from
there,
and
if
you
have
changes
that
conflict
in
public,
it's
all
manual
for
now.
If
you
have
changes
that
conflict
in
public,
you
need
to
submit
a
fix
into
the
private
repository
first
get
that
merged
and
then
merge
the
the
public
one.
A
So,
theoretically,
with
that,
we
would
be
able
to
have
this
not
really
sync,
because
it's
not
really
think
it's
actually
merge,
and
one
major
issue
we've
had
so
far
is
when
someone
does
not
follow
that
process.
Ie
wins,
but
then
there
is
a
chance
of
breakage
and
we
own
catch
it.
When
the
specs
run,
if
specs
cover
it,
we
only
catch
it.
If
the
QA
runs,
if
QA
covers
it,
we
only
see
it
in
production.
Basically,
when
when
there
is
like
a
larger,
larger
problem,
we
had
one
or
two
occasions,
I.
Think
of
that
happening.
A
A
B
A
B
B
A
A
Right,
but
with
security
releases
for
tagged
releases
so
for
self-managed
customers,
the
difference
is
that
we
can
kind
of
timing
there.
Like
is
time
there
is
running,
slow,
slower
than
a
master
in
master.
It's
like
right
then,
and
there
stable
branches
we
can
say
like
okay,
we
cannot
ship
anything
until
we
clear
out
the
stable
branch
right
like
that's,
not
a
problem,
we
can
get
it
done.
The
problem
is:
when
you
combine
these
two
things,
you
can
put
all
the
the
problems
on
the
developer.
B
B
So
like
at
the
point
that
you
run
a
security
lease,
you
want
to
replay
them
in
a
certain
order
and
then
every
time
you
hit
a
conflict,
then
it's
call
the
developer,
get
them
to
fix.
It
I
think
that
challenge
the
challenge
with
that
is
like
I
guess,
you've
probably
got
23
fixes
right.
So
well,
that's
still
a
lot,
so
let's
say
the
developer,
who
made
the
security
fix
you
just?
What
do
we
have
like
five
days
to
prepare
the
security
release,
something
like
that?
Theoretically,.
B
Well,
I'm,
just
thinking
like
if
you've
got
say,
twenty
five
merge
request
to
replay
and
onto
each
release
and
assuming
there's
conflicts
on
a
bunch
of
those
I
mean
the
odds
of
getting
that
done
in
a
couple
of
days
is
pretty
slim
because
every
time
someone
come
every
time,
there's
a
conflict.
Someone
has
to
go
and
resolve
it
run,
CI
pipelines
run
the
whole
regression,
I
mean
assuming
they
are
a
genius
and
they
fix
it.
First
go
you're
still
talking
like
a
minimum
is
like
two
to
three
hours
to
like.
Well,
no.
B
Than
that,
because
you've
got
to
like
run
the
pipeline,
which
is
like
Aaron,
1/2
or
whatever
then
you've
got
to,
then
then
it
fails
at
which
point
someone
knows
they're
to
fix
it.
Then
they
have
to
write
the
fix
say
they
do
that
in
an
hour
and
then
they
have
to
run
the
test
again
and
then
emerges
so
you're
talking
like
a
minimum
of
like
three
hours
of
CI,
chats
plus
an
hour
to
fix
they're
talking
like
a
minimum
of
four
hours.
If
anything
breaks,
I.
B
B
All
right,
gosh,
okay,
well,
I,
don't
know
that.
There's
any
practical
solution
to
that.
One
besides
me
pipelines
faster.
Thank
you
because
they,
if
you
replay
them
anything
in
anything
but
the
same
order.
Maybe
that's
not
true!
Maybe
you
could
get
lucky
and
have
commits
that
you
could
replay
in
different
order
like
imagine
if
we
went
to
master
and
we
replayed
them
and
well.
This
is
this
would
be
an
amazing
feature
for
someone
to
build.
B
Its
tries
to
replay
them
and
if
it
hits
a
merge
that
fails,
it
tries
the
next
thing
and
it
sort
of
works
out
like
which
ones
that
can
continue.
Maybe
even
just
a
naive
approach
where
it
just
skips
that
tries
the
next
one.
If
the
next
one
fails,
then
a
complete,
you
know
they
like.
If
you
have
one
failure
and
the
next
one
succeed,
you
just
keep
going
yeah.
A
That
will
be
amazing,
yeah,
and
it
would
also
have
to
then
enforce
that
everything
is
V
based
to
make
sure
that
we
don't
have
like
too
many
commits
coming
from.
You
know
like
America's
that
has
like
ten
commits,
for
example,
I,
don't
know
what
we
enforce
at
the
moment,
but
you
know
like
having
one
commit
and
like
just
skipping
that
would
skip
the
whole
bug
fix
or
a
feature
rather
than
like
within
that
one
bug
or
a
feature.
You
would
have
like
five
ten
community.
A
B
B
A
B
A
B
B
That
box
is
just
like
to
make
it
generic,
but
essentially
the
the
way
I
envisioned.
It
would
be
that
I
wouldn't
have
a
private
fork.
There
would
be
the
gitlab
or
private
Forks,
there's
like
a
group
and
then
the
private
group
and
everyone
we
have
access
to
both
and
so
any
developer
that
gets
added
to
the
gitlab
org
one.
We
then
inherit
access
to
the
private
one
and
then
any
confidential
merge
request
by
default.
I
would
see
the
drop-down
prefilled
with
the
private
fork
that
already
exists.
We.
A
Already
have
that
by
the
way,
because
that's
the
same
idea
I
just
didn't
think
of
it
that
way
so
I
don't
know
for
whatever
reason.
Well
actually,
I
know
I
didn't
think
of
it
that
way,
because
confidential
merge
request
didn't
exist.
At
that
moment,
we
created
that
group
and
we
have
projects
in
there.
So
if
we
recreate
those
projects
now
with
a
fork
relationship
and
set
them
to
be
dependent
on
one
another,
we
can
already
use
the
confidential
merger
class
dang.
A
A
Someone
needs
to
win
if
we
are
using
the
merge
strain
script
right
like
we
have
to
choose
the
source
of
truth.
If
we
choose
to
not
fail,
but
do
what
we
are
doing
right
now
with
C
and
E
merges,
there
is
always
a
chance
that
will
break
something
silently
alternative
option
is
that
we
fail
loudly,
but
then
this
is
equivalent
of
red
monster.
Stop
the
train.
This
needs
to
be
fixed
right
away
before
we
can
continue
with
red
monster.
A
B
The
red
main
red
master
in
the
security
release
of
red
master
and
say
in
general
in
general,
so
the
issue
would
be.
It
looks
green
in
this
the
check
in
the
CI
pipeline,
but
then,
when
an
action
manages,
it
becomes
red.
Yep
shouldn't
that
be
avoided
by
the
whole
feature
that
CI
a
built
where
it
actually
runs
like
the
test
on
a
simulated.
Merge
can
for.
B
A
B
A
A
B
It
would
be
very
interesting
from
my
perspective,
because
there
are
a
number
of
companies
that
I
know
and
customers
that
use
fast.
Fortnight,
requests
like
for
pretty
similar
reasons
that
they
just
they
have
a
lot
of
properties
around
making
sure
like
when
things
are
green,
they're,
actually
green,
and
that
that's
pretty
helpful,
but
there's
a
pretty
big
contention
problem
that
exists,
like
the
timing,
contention
and
I
know
that
I've
raised
this
with
the
CI
team
that
merge
trains
really
should
try
and
solve
this
contention.
B
Problem
like
we
built
them,
but
they
don't
actually
work
for
fast
board.
Merge,
I,
don't
think.
If
they
did,
then
the
problem
would
be
diminished
significantly,
because
you
could.
We
could
like
preemptively
rebase
into
the
queue,
and
so
you
can
re
based
in
your
position.
You've
essentially
got
like
master
proper
and
then
you've
got
like
pending
master,
and
so
you
can
like
do
like
a
rebase
run.
The
ten
paper
passes
else
like
rebased
fails
and
you're
like
kicked
out
of
the
queue,
but
at
least
that
way
like
you're,
like
it's
less
painful.
B
It
is
today
where
it
becomes
like
a
phrase
like
if
we
didn't
have
that
feature.
If
we
don't
have
merge,
trains
board
merge
like
this
whole
thing's
not
going
to
work,
because
we've
now
got.
However,
many
hundred
engineers
merging
as
fast
as
we
can,
and
it's
just
going
to
be
like
a
race
to
rebase
and
run
the
seei
pipelines
again.
B
B
A
That
in
December,
so
it
was
not
my
problem
or
my
team's
problem,
but
I
recognized.
It
is
one
of
the
big
problems
that
we
have
to
run
faster,
or
rather
to
get
cv,
and
we
did
that.
So
we
went
from
220
to
50,
parallel
specs,
I
think,
and
we
also
true
bigger
machines
in
some
of
the
pipelines
we
had
a
calculated
like
we
calculated
where
to
target
you
only
get
so
much
right.
That's
like
they
are
certain
like
now
made
with
dag.
A
There
is
a
possibility
that
things
could
be
speed
up
and
now
rainy
I
think
is
actually
working
on
this,
but
we
had
this
order
that
we
needed
to
follow
because
specs
depended
on
another
one
and
so
on
and
I
think
now.
Rainy
is
actually
like
implementing
bag
and
all
of
that
stuff,
like
all
of
the
new
features
that
we
are
using.
Hopefully
that
helps,
but
actually
just
paralyzing
and
throwing
more
money
at
the
problem
only
gets
you
so
far
and
I
think
we
exhausted
that
so
far
back
in
December,
ok,
so.
B
B
On
how
many,
how
long
a
test
can
take?
But
if
you've
got
a
chain
of
tests
that
have
to
be
run
in
a
certain
order
like
that,
change
should
be
no
longer
than
like.
Sixty
Seconds
execution
time
like
each
could
be
even
the
less
like
thirty
Seconds
execution.
Time
like
we
should
have
really
short
chains
like
test
should
be
really
focused,
and
it
is
like,
if
you're,
depending
on
specific
data
or
existing
just
like
declare
it
create.
B
B
A
A
B
A
A
B
A
Here's
the
thing
James,
because
we
like
our
primary
concern,
is
making
sure
that
security
releases,
development
of
security
releases
becomes
simpler
and
the
security
releases
becomes
simpler
to
manage
from
our
side
as
well.
However,
I
do
want
like
to
make
like
one
or
two
steps
with
custom
scripts,
but
then
work
closely
with
you
and
your
teams
to
build
this
back
in
with
the
knowledge
we
acquired.
So
it
means
that
we're
gonna
take
us
like
six
months
to
build
this.
A
It
doesn't
matter,
but
as
long
as
you
can
also
allocate
some
resources
time,
people
to
actually
help
out
when
we
need
to
sit
down
and
write
the
feature
that
would
be.
That
would
be
super
helpful,
our
device,
otherwise
we'll
we'll
have
to
do
scripts
and
move
on,
because
this
is
actually
we
are
at
our
and
with
scale-like.
We
can't
do
this
anymore
for
much
longer.
You
know
I
I,.
B
Think
I
don't
know
if
this
contradicts
the
conversation
you
had
this
but
I
think
like.
If,
if
there's
a
like
a
problem,
that's
obstructing
get
laughs
and
it's
going
to
be
faster
to
with
scripts
like
we
should
do
with
scripts
like
if
it's
obstructing
our
ability
to
execute
as
an
organization,
we
should
just
actually.
B
B
A
B
Yeah
and
I
think
the
other
thing
that's
good
about
this
is
I,
think
it
gives
us
a
chance
to
learn
like
I,
said
and
there's
a
degree
of
uncertainty
around
like
how
we're
actually
gonna
make
this
work
like
scripting.
Automation
is
an
API
is
all
this
was
a
great
way
to
just
test
things
cheaply
without
committing
and
like
getting
customers
relying
on
a
certain
behavior
that
we
decide
is
like
not
what
we
actually
want
right,
yes,
hard
to
make
those
breaking
changes,
that's
much
much
better
for
us
to
do
prototypes
experimentation
in
this
way.