►
From YouTube: Shell Code Libraries In Pure GitLab CI YAML
Description
You may wish to do away with the extra development of containers or you may want to use a variety of non-custom containers in your pipeline - but still need some shell code libraries available to all jobs in the pipeline - here is a way to accomplish that.
Working Example Code: https://gitlab.com/guided-explorations/gitlab-ci-yml-tips-tricks-and-hacks/shell-code-libraries-in-pure-gitlab-ci-yaml
Playlists:
- GitLab Guided Explorations (Working Examples) (https://youtube.com/playlist?list=PL05JrBw4t0KruBVtI-Ynb3mCHQTkh2eBY)
- GitLab CI/CD Tips Tricks and Hacks (https://youtube.com/playlist?list=PL05JrBw4t0Kr92rF9z0PEuOtQnH89VD7n)
- GitLab and Windows (https://youtube.com/playlist?list=PL05JrBw4t0KqOJeQ1y5wSBxArdTEtmnxb)
- GitLab and PowerShell (https://youtube.com/playlist?list=PL05JrBw4t0KrklJ2kDTl2qXNaGSlzKeoW)
- DarwinTheorizes (https://youtube.com/playlist?list=PL05JrBw4t0KoH74CcAtP0Ze7JXjX7zT6g)
A
Hey
everyone,
it's
darwin
here
from
git
lab.
I
just
wanted
to
go
through
some
information
with
you
about
handling
dependencies
of
code
in
your
gitlab
ci
pipelines,
dependency
management
of
actual
ci
code
itself
can
be
a
challenge,
and
you
can
see
all
through
our
auto
devops
that
what
we
do
in
general
is
we
have
some
gitlab
ci
yaml,
which
is
managed
to
be
nice
and
readable,
very
minimal,
and
then
we
put
the
dependencies
in
a
container
now.
This
is
super
important.
A
If
you
have
a
lot
of
run
time,
dependencies
like
part
of
your
pipeline
is
dependent
on
node,
but
another
pipe
part
is
dependent
on
python
and
you
don't
want
to
work
out
a
container
that
perfectly
holds
all
of
those
different
run
times,
so
it
can
be
really
advantageous.
There's
one
circumstance,
though,
in
which
it
can
be
unadvantageous,
and
I
want
to
show
you
kind
of
a
way
to
unwrap
that
pattern
to
work
for
that
situation
as
well.
A
So
we're
going
to
talk
about
the
most
efficiency
dependency
management
of
all,
so
with
get
lab
ci
we'll
have
a
pipeline
and
typically,
what
we'll
do
for
the
first
level
of
kind
of
efficiency
and
managing
dependencies
is
to
break
out
extensions.
There's
a
variety
of
ways.
We
do
this
in
autodevops,
for
instance,
our
sas
stage
is
a
separate,
includable
piece
of
the
pipeline.
A
You
can
do
this
yourself
with
includes
as
well,
like
you
see
here
on
the
screen
and
there's
a
whole
bunch
of
examples
out
and
guided
explorations
in
a
subgroup
that
talks
about
how
to
build
these
kind
of
extensions
now.
These
extensions
of
course
depend
on
another
bit
of
gitlab
ci
yaml,
that's
sitting
out
there
that
bit
of
gitlab
ci
yaml
has
to
be
written
in
a
way
where
it
does
not
have
any
hard
coding
when
it
comes
to,
including
in
another
project.
A
So
if
there
is
input
data
that
has
to
go
into
the
the
code,
then
it
has
to
be
done
in
variables
and
if
there's
anything
you
need
back
out,
then
you
have
to
collect
it
as
artifacts.
So
you
want
to
make
it
so
this
bit
of
code,
it's
not
really
a
template
that
you
grab
and
customize.
It's
actually
something
you
directly
depend
on
and
use.
A
A
For
example,
I
have
one
situation
where
I
need
to
do
a
lot
of
cloud
formation
stuff,
and
I
found
some
really
nice
libraries
that
are
written
in
shellcode,
but
there's
a
lot
of
code
and
so
to
stick
that
right.
Inside
of
my
regular
code,
makes
it
kind
of
messy
a
whole
bunch
of
functions
that
I
might
want
to
use
across
multiple
jobs,
and
so
the
temptation
is
to
very
quickly
well.
Let
me
put
all
those
in
one
file
put
them
in
an
image
and
then
they'll
be
self-contained
and
I'll
be
able
to
depend
on
them.
A
But
the
challenge
here
is
that
the
the
container
itself
isn't
the
only
dependency
once
you
encapsulate
something
in
it.
That
container
itself
depends
on
a
container
registry
and
that
container
registry
has
to
have
the
container
built
and
sent
to
it
by
some
other
kind
of
ci,
which
of
course
requires
some
sort
of
docker
file.
So
we
end
up
with
a
chain
of
dependencies
when
using
containers
very
advantageous
when
we've
got
to
tightly
run.
Those
runtimes,
but
potentially
a
challenge
if
all
you're
looking
to
do
is
especially
manage
some
shell
code,
libraries.
A
So
some
of
the
challenges
that
we
kind
of
inherit
are
that
the
bandwidth
and
time
of
that
container
being
pulled,
and
then
loaded,
and
that
happens
every
time
you
use,
actually
happens
every
job.
If
you're
on
gitlab.com,
we
have
the
availability
of
the
registry
from
a
devops
perspective
as
well
as
authentication
to
it.
If
it's
not
a
public
registry,
and
then
we
have
the
container
source
files
and
also
the
build
management.
So
how
often
do
you
build
that
container?
A
Is
it
once
in
a
while
when
you
update
it,
does
it
build
monthly,
and
then
we
also
have
the
container
contents
versioning.
So
if
you're
working
with
your
pipeline
and
you're
like
oh,
my
goodness,
I
need
to
update
one
of
these
routines,
I
depend
on
you-
can
no
longer
just
update
the
get
lab
ci
yml.
You
have
to
go
back
and
update
the
container
rerev
the
container
and
then
you're
able
to
use
it
forward
so
that
versioning
process
or
that
dependency
versioning
can
be
expensive
both
in
development
as
well
as
at
run
time.
A
So
what
if
we
could
just
completely
eliminate
this
whole
realm
here,
so
one
of
the
architectural
heuristics
that
I
want
to
expose
you
to
which
a
heuristic
is
sort
of
like
a
rule
and
a
lot
of
times.
It's
a
rule
that
helps
you
pick
something
on
a
sliding
spectrum.
So
it's
not
a
this
is
always
good,
or
this
is
always
bad
rule.
It
is
usually
dependent
on
other
rules
that
you
also
process.
So
in
this
case
the
architectural
heuristic
is
eliminating
dependencies
when
possible
is
the
most
efficient
dependency
management
strategy.
A
If
I
have
some
shell
code,
libraries
that
comprise
you
know
256k
of
code
or
maybe
a
lot
more
code
or
any
binary
dependencies,
then
all
of
a
sudden-
well,
the
container's,
probably
the
best
solution.
Looking
at
eliminating
dependencies
whenever
possible
is
a
high
level
order
thing
and
going
into
taking
a
dependency
on.
Something
is
something
you
want
to
do
with
caution
whenever
possible.
A
A
Let's
look
in
the
main
gitlab
ci
yaml
and
we're
going
to
just
scoot
right
down
to
the
bottom
right
away
and
see
that
we
have
included
the
shellcode
library,
so
this
is
called
shellcode
librariesgitlabci
yaml.
Now,
when
we
look
in
this
file,
you're
going
to
see
there's
not
much
in
there
and
we
could
have
actually
just
included
it
directly
here
so
there's
always
that
trade-off
as
well.
Do
you
want
another
include
dependency,
or
do
you
want
to
put
the
code
right
here
in
this
case?
A
So
let's
go
into
the
shell
code,
libraries-
and
this
is
where
we
have
some
shell
code
that
is
going
to
be
used
throughout
our
pipeline.
So
that's
the
other
aspect
is
we
want
to
use
it
throughout
our
pipelines
inside
of
the
scripts
that
our
pipelines
are
running,
so
you
can't
do
it
as
a
an
extends
as
easily,
and
you
also
can't
do
it
as
a
yaml
anchor
as
easily.
You
could
potentially
do
some
really
complicated
yaml
anchors,
but
I
appreciate
this
approach
a
little
bit
better
because
I
think
it's
a
little
bit
cleaner.
A
So
the
first
thing
that
we
do
is
we're
going
to
write
our
libraries
to
disk
on
in
one
of
these
stages.
Now
one
of
the
important
things
here
is
the
stage
is
pre,
so
anything.
That's
writing.
A
library
runs
in
the
pre-stage,
which
means
before
everything
else,
and
then
we
collect
that
as
an
artifact
and
pass
it
to
every
subsequent
container.
A
A
I
didn't
bother
with
worrying
about
the
sh
and
you'll,
see
why
here
right
now,
so
I
like
to
write
everything
I
write
to
enable
both
the
windows
devops
side
of
the
house,
as
well
as
the
linux
devops
side
of
the
house.
So
here
we
have
windows,
powershell
and
how
this
would
work
under
windows
powershell.
A
Now,
when
you're
using
windows,
powershell
powershell,
is
the
default
shell
for
gitlab
ci,
which
means
as
soon
as
we
hop
in
we're,
writing
powershell
right
away-
and
you
can
see
here
that
there
is
also
a
concept
of
a
here
doc
in
powershell,
and
this
is
how
it
works.
It's
writing
out
a
function,
writing
out
another
function
and
then
collecting
any
code
library
artifacts
that
we
created.
Of
course,
these
could
be
rather
large
if
necessary,
and
then
finally,
linux
powershell.
A
So
this
happens
a
little
bit
different,
because
most
linux
hosts
that
run
powershell,
even
the
ones
built
by
microsoft,
don't
run
it
by
default.
They
run
bash
by
default
and
you
call
powershell.
So
it's
just
slightly
different
in
that
we
go
ahead
and
cat
the
same
way
as
we
do
as
linux
to
build
the
libraries
we'll.
A
Also
see,
there's
a
little
bit
of
difference
when
we
write
the
libraries
notice
that
we're
following
a
convention
with
the
file
names
that
this
is
linux
code,
library
ps1,
which
lets
us
know
it's
linux
for
powershell
up
here
we
have
code
library
ps1,
which
lets
us
know
that
this
is
windows
powershell.
You
could
get
more
explicit
and
use
windows
powershell
and
then
anything,
that's
just
code
library,
shell,
is
linux
bash,
so
that
writes
our
files
out
and
once
they're
written
out,
we
can
then
start
using
them
in
our
main
code.
A
So
let's
go
take
a
look
at
that
inside
of
our
main
code.
Here
we
are
doing
bash
powershot
windows,
powershell
and
linux
powershell
testing.
So
in
many
cases
you
would
have
just
one
of
these
languages
and
of
course
you
just
grab
the
the
jobs
out
of
here
that
are
just
for
that
one
language.
If
you
have
that
situation,
we're
doing
two
jobs
that
use
two
different
functions.
It
doesn't
really
matter
where
those
functions
came
from
the
the
the
little
trick.
A
Right
here
is
to
use
this
little
loop
to
load
the
files
with
dot
sourcing,
so
dot
sourcing
means
load.
These
into
the
current
shell
context,
you
want
to
be
careful
about
trying
to
move
this
code
to
another
context
like
a
before
script,
because
if
it
does
not
run
in
the
exact
same
shell
context,
then
of
course
when
that
shell
unloads,
you
won't
have
that
code
anymore.
So
you
have
to
run
this
at
the
top
of
each
of
the
jobs.
A
That's
going
to
use
any
of
the
functions
in
the
function
library,
so
you
see
now
we're
very
simple.
We
can
just
call
these
functions
directly.
I'm
also
emitting
and
documenting
that
hey.
You
know
we
got
code,
libraries
here
just
so
that
it's
easy
for
the
next
person
through
to
see
what's
going
on
same
thing
happened
here,
we're
loading
the
exact
same
files
and
even
though
these
two
functions
are
in
separate
files,
we
load
them
all,
and
then
we
run
the
functions,
we're
not
going
to
accidentally
load
powershell
functions
here
because
of
the
file
naming
convention.
A
We
talked
about
windows
powershell
same
thing,
except
the
dot
sourcing
is
a
little
bit
different
for
those
of
you
don't
know.
Powershell
purposely
took
the
best
of
bash,
the
best
of
vms
and
all
other
kinds
of
shells
and
tried
to
implement
it.
So
consequently
there's
some
concepts
that
are
very
similar
to
bash
in
powershell
and
then
here's
another
windows
powershell
test
and
a
linux
powershell
test.
One
of
the
differences
with
linux
powershell
since
powershell
is
not
the
default
is
that
we
are
actually
doing
a
here
string
or
here
doc
to
pipe
the
code.
A
That's
in
here
the
powershell
code
that
triggers
our
other
powershell
code
directly
into
powershell
core,
and
so
that's
just
slightly
different
and
then
down
here
is
where
we
included.
All
of
that.
It's
probably
hard
to
tell
from
these
includes
and
the
different
stages,
how
this
processes
so
we'll
go.
Look
at
pipelines
that
have
completed
and
we'll
go
to
this
one.
That's
all
shiny
and
lit
up
as
successful.
A
So
we
see
here
that
the
pre-stage
runs
first
now.
One
thing
you
may
not
be
aware
of
is
that
there
are
built-in
stages
for
gitlab
ci,
so
you
notice,
I
didn't
have
stages
in
my
ci
yaml
and
that's
because
pre-build
test
and
a
few
others
are
built
into
gitlab
that
if
you
don't
have
to
declare
them
as
stages,
it
knows
how
to
use
those
ones.
If
I
did
declare
stages,
then
I
have
to
declare
all
of
my
stages
and
completely
override
that
default
behavior,
but
by
using
pre.
All
of
my
write
libs
happen.
A
A
So
I
hope
that
this
has
been
helpful,
helps
you
kind
of
understand
how
you
might
write
shell
code,
libraries
completely
in
bash
or
powershell,
without
having
to
resort
to
a
container.
Obviously,
if
you
have
a
lot
of
dependencies
or
depend
on
different
language
types,
then
containers
work
just
awesome.
I
just
wanted
to
show
you
a
way
to
encapsulate
shell
library.