DevOps and Docker Talk: Cloud Native Interviews and Tooling

The Docker Bake Build tool just went
general availability, and I'm excited

about what this means for creating
reproducible builds and automation

that can run anywhere CI locally.

I love it.

Really, I'm gonna break down some
of the features, the benefits

and walk through some examples.

For over six years, Docker has had this
little known tool that we could execute

with a Docker Buildex Bake command.

It's meant to provide even more
customization and flexibility than

what we're building with today.

But if Bake has been around all these
years, why isn't everyone using it?

To answer that, I'll need to
take you back in time to 2018.

Docker was on a streak of releases and
it seemed like they were creating new

tools every few months, experimenting
with various container tools on top of

Docker engine to see what would stick.

We had a release of the BuildKit
project and then Bake came out with

the release of Build X Command, even
us Docker captains were having a hard

time keeping up with all the ideas
coming out of Docker, Inc. But we also

started to see Docker deprecate products
causing some of their less popular tools

to silently stop receiving updates.

So I and others were hesitant to use
Bake in any significant way since we

weren't really confident in its future.

And we were kind of right at the
time, everyone was talking about

orchestration and Docker didn't mention
Bake again in a significant way.

So we just assumed that it would
quietly become a deprecated

idea like so many others.

Fast forward to today, and
Docker recently announced general

availability of Docker Bake.

They did this as a signal to us in the
community that after years of development

and heavy use internally, they're ready
to fully support Bake in the community.

And in fact, they're so committed to
this that they have announced that

they have a plan to change Compose,
to use, bake in the background for

all of its bills in a future release.

Now that I know I can trust in a
future for Bake, why do I care?

Like what can it do for me today?

Well, at its Core Bake is a simple
command that gets its instructions

from a file formatted in JSON,
Compose, or my preference, HCL?

Yes, that's the file format
we use with Terraform.

Bake Acts as a kind of
BuildKit native make file.

It provides a more structured
way to configure builds.

It still uses Docker files as the detailed
instructions, and you'll probably have

more stages in your Docker files if you
go fancy with it, but it replaces long

complex build commands and build scripts
with a more advanced and appropriate tool.

Bake is a superset of Docker
build and composed builds, but it

makes those things even easier.

Here's a partial list of things
that bake can do for you.

before I get to the list of benefits
and examples of how you can use Bake,

I wanna thank today's sponsor me.

I've been making Docker, Kubernetes,
and DevOps content since at least

2017, and I welcome you to my community
of cloud native DevOps engineers.

Besides this podcast, you can join
my 20,000 user Discord server.

Subscribe to my YouTube channel
where I'm posting content or

live streams You can also sign
up for my low traffic newsletter.

Or buy one of my courses with a coupon.

If you'd like to buy me a coffee
and support my work, you can become

a member on YouTube, or on Patreon.

And you'll get exclusive benefits
like videos on YouTube or

separate chat rooms on Discord.

I should also mention my High Fives
DevOps Guild, which is a monthly

virtual meetup on discord with other
DevOps professionals Where we talk

trends, what we're working on, and
anything we're really excited about.

I don't talk about that enough, but I've
been doing it for years and I always

have fun hosting that group every month.

So if you'd like to join that video call,
look for the high fives in the memberships

plans on YouTube, Patreon, or my website.

It's about six bucks a month.

Okay.

Let's talk about bakes benefits and some
examples on the YouTube version of this,

which you can see in the show notes with
everything else and all the details.

That is something where I go
through, I think it's four examples

in detail going through the files.

How it works with Compose.

The difference between the HCL
format that I prefer for using Bake,

which is the Terraform language
and how that compares to Compose.

Pretty much you can do 90% of this
stuff, or maybe 80 to 90% of all this

stuff Bake can do in a Compose file, but
I think I much prefer the HCL format.

Because it gives you a separate file
that's just focused on building images,

so it doesn't muddy the waters of
where the composed file is sort of

service centric and it has the volumes
and the services, and it's usually meant

for local dev and that sort of thing.

I like having this Docker
dash bake do HCL file in the

same route, director, my repo, where
the Dockerfile is and the Docker

ignore, and possibly a composed file.

It just feels right to have that separate
file sitting beside the Dockerfile.

But here's a short list of some
examples of things you can do with Bake.

A lot of these you can actually do
in BuildKit with just Docker build if

you wanna make a really long command.

And you can do a lot of it in Compose
if you wanna make the build section of

each composed service really long too.

Alright.

First.

Improve build speed by letting
BuildKit automate the caches and

parallelism essentially, because you
can give Bake a single command, it

will do potentially multiple builds.

You can have one command building
dozens of different images if you

set it up that way, it optimizes
the caching and parallelism across

all those different activities.

So that it only has to transfer your
local source code once into the BuildKit

engine, and it will parallelize all
the different parts of the build,

the different architectures in a
way that you can't really get in

advance cases with the Docker build
command or with composed builds.

Next you can import or export files
outside of the Docker context for

building images, which is a little,
I mean, honestly, I would feel like

it's a little bit of an anti-pattern.

'cause I always think of everything
in a Docker build is in that directory

or subdirectory that you build from.

That's, that's the context.

But there is an option where you can
add multiple context locations in the

file system that are outside that,
and when you finish the build, you

can choose to export everything in
the image to the host file system.

Maybe you were just building a go
or rust or see binary and it's a

single file and you want to put it on
GitHub releases for download, right?

So you can kind of do all this
at once, uh, except for maybe the

part where you actually upload
it to the releases in GitHub.

But you can have it not just in an image.

You can have the results
put out to the file system.

Next, there are more options for adding
variables to your builds and including

those in functions, which you can even
use Golang simplistic functions inside the

HCL format to have advanced programmatic
ish flow of how the image gets built.

next, like other ways you
can build images in BuildKit.

You can add SBOs and Providence
attestations to the images with

little options you can set in the HCL.

Next.

You can tag the image with multiple
names, which you could always do, but a

lot of people don't realize you can do
that in a single Docker build command,

but you can make a list of all the
different names you want to name it.

Maybe you want it to be colon latest, but
you also want it to be colon aversion or

a date timestamp, or you want to indicate
that it's approved for release in the tag.

Or maybe you have multiple registries,
which I see a lot, and you maybe

want to push these certain images.

For maybe an official release, you
wanna push 'em up to GitHub Container

Registry, also up to Docker Hub.

And if you're doing something
private in your company, you might

wanna push it to an ECR instance
as well as for developers and GHCR.

I mean, there's just all sorts of
scenarios, but you can do it all at

once essentially, and have a, a full
list with variables and a little bit

of advanced functions where you can put
in day timestamps and other methods to.

Inject dynamic tags into that file.

Next You can build for multiple
architectures concurrently, of course.

So you're always wanting to build these
days on arm and a MD. Next, just like you

can use Compose override files if you've
ever done that, I'm a big fan of that.

For Compose, for custom local individual
developer settings, you just create

a file called Docker Compose override
YAML, and then Compose will automatically

pick that up on your local system.

My technique there is I
usually put that particular.

File name in the dot get ignore so
it never gets back into the repo.

And then I create sort of a sample
override file for developers, and

then they will all create their
own override that they'll use for

development environment variables.

Well bake has that too.

Next you can set an option to
automatically push to registries after

the builds are finished, and that's
a part of something called the output

where you can have it, choose to load
the image into the Docker image cache,

or export to the file system, or
automatically push the registries or do

nothing and just have it leave itself
in the BuildKit cash, which I see used.

For building stages of images that
maybe aren't meant to go somewhere.

Maybe you have a stage that does a
CVE scan or a linter or some other

sort of testing, and you're really
just using, at this point, the bake

file and the Dockerfile as a way
to automate that stuff in the same

tooling during your build process.

But you don't necessarily need
those particular images pushed

anywhere or put anywhere.

You just need to run.

That build stage.

So just leave it in the build cache.

Right?

And without examples, this won't maybe
make a lot of sense, but inside the bake

file you have something called targets,
which is a particular build and that

build could technically build multiple
images or multiple architectures of

images, but it's the one thing that
you execute from the bake command.

You can also group multiple targets
into groups and build them all at once.

This is really handy for mono repo.

So if you decide that a certain file in
the mono repo changes, you wanna build

all images in that mono repo, you can
technically do that with one bake command

and just have multiple targets, and
then group those into a group and call

that group and it will automatically
cash and parallelize all that.

Next.

You can also build images remotely by
just pointing bake to a GitHub URL and

then it will build what it finds in there.

And you can do all of that in a single
bake command, which you can then throw

into your CI, you can run it locally.

And that's one of the wonderful
things of Bake is it is meant to run

everywhere In the true style of Docker,
it works on more than your machine.

And you know, all of our different cis
nowadays have abstracted the Docker and

BuildKit stuff into YAML or whatever their
format is, which is usually YAML nowadays.

And so you're usually doing YAML
work in their specific CI format.

And the power of this bake is that
you can take a lot of, at least

the build complexity, possibly even
more than the build complexity.

since you can make all these fancy
Dockerfile stages to do other things,

I. You could choose to use the bake
file and then in your CI, let's talk

about GitHub Actions for a second.

You really are just running a
Docker buildex bake command, and

you can easily switch cis or run
it locally in exactly the same way.

We don't today have that option in
GitHub Actions because as far as I

know, act is the only real way to try
to run GitHub Actions locally, but it's.

A non-official third party tool that
has limits and functionality problems.

So I would absolutely prefer
the bake file format there.

And of course, Docker has their own
official GitHub Actions, of which one

of them all along has been the bake
GitHub action that I haven't been

using because I was for reasons not
sure what its purpose or future was.

So I was always using and teaching
and showing off examples of using

the Docker build GitHub action.

Which I probably wouldn't be surprised
to find out that behind it is really

bake, just running in the background.

But there's now an official bake action,
so I'm going to personally be switching

a lot of my stuff over from the Docker
build action as I start writing some

of these HCL files for Docker Bake.

I'm gonna switch over to that GitHub
action to hopefully reduce the amount.

Of YAML, I have to type in order
to get things at GitHub to work.

And then with the advantage of being able
to run that exact same build locally.

And like I said earlier, you can watch
the YouTube video in in LinkedIn, the

show notes to see some of the examples
as well as a little bit of the video of

me showing how I baked cookies, which was
just a fun part of that video to make.

All right.

One of my final thoughts about Bake
is that to take full advantage of

it as a CI workflow engine, you
need to lean into Dockerfile stages.

If you like the idea of a Docker Buildex
Bake command being one of the only

things you need to run in your CI.

Then you need to lean into expanding
your Dockerfile with testing stages,

maybe CVE scanning stages, et cetera.

Now.

I've given demos at conferences where
I do that exact thing with multi-stage

builds, showing off how multi-stage
builds work and the things you can do

with them, and trying to shove every
possible CI action into a Dockerfile.

But the reality is, is that
Dockerfile stages aren't a

great place for a lot of that.

Stuff to live.

A lot of the CI tools actually
expect to run from outside

the image once it's built.

So for bigger teams and projects, it
kind of muddies the waters between

building your app image and all the other
activities you want to happen In CI, do we

want the single file format of Dockerfile
to be that single source of truth for

everything and then you end up with a
possible 300 or 400 line Dockerfile.

Also CI solutions don't
monitor BuildKit well.

So to them, this just looks like
one big pipeline step rather than

the detailed view you would get if
each action was its own CI step.

I've noticed that over the years as
I've tried to make advanced Docker

builds, and it's unfortunate that the
CI companies don't do more to see into

what BuildKit is doing to give you that
advanced workflow or pipeline view.

This really comes down to preference
of where you like to spend

your time creating automation.

Do you love the Dockerfile format
and want to use it to the max?

Great.

Use Bake for as much as you can.

Do you use GitHub Actions for
CI and prefer the YAML job

structure of that solution?

Great.

Do that.

I can tell you that if you like the
idea of bake, because you can run it

anywhere and you can make it declarative
and repeatable for more things than

just image builds, but maybe you're
not a fan of doing it all in Docker

files, then you might wanna look at
more advanced tools like Dagger that

allow you to run all these things.

With the language your app is written in.

More on that in another video.

Thanks for listening.

I'll see you in the next one.