DevOps and Docker Talk: Cloud Native Interviews and Tooling

You're listening to DevOps and Docker Talk, and I'm your host, Bret Fisher.

These are edited audio only versions of my YouTube
Live show that you can join every Thursday bret.live.

This podcast is sponsored by my Patreon members.

I'd like to thank all the paid supporters that make this show possible.

You can get more info and follow my updates on all the content
and open source I'm creating at patreon.com/bretfisher.

And as a reminder, all the links for this show, the topics we discuss.

As well as the links I've already mentioned are available
on the podcast website at podcast.bretfisher.com.

Bret: hello and welcome to the show.

My name is Bret.

I'm going to be here with you for at least the next hour.

And we're going to talk about a bunch of my favorite stuff today with a friend of mine.

We're going to be talking specifically about continuous integration, continuous deployment,
continuous delivery, GitOps, all the things between your code and servers like that whole segment.

And we're specifically gonna be talking about Argo.

So we'll get to that, but get those questions in early, get the
questions that you have around CIC D and get, uh, what is get ops.

I mean, if you have those questions, throw those in the
chat and then we're going to start working on them later.

We might even get into some demos.

This show is actually about my friend, Victor.

Who's all the way from the other side of the pond.

Victor is a principle DevOps architect, which makes it sound.

He's better than DevOps, which he totally is.

And let me just give you some credentials on his background, because he's not just a Docker captain.

Victor has been doing all sorts of stuff.

Now.

Not only does he write a bunch of books that all about DevOps and continuous integration
and all those things, and he also has a YouTube channel that you should check out.

, but he works at code fresh.

So he's a DevOps architect.

They're a principal DevOps architect.

He is a member of the Google developer experts program.

Cool, but that's not enough because he's also on continuous delivery foundation ambassadors.

And I mentioned he's got a bunch of books of which he did have me in one of those.

I was very lucky to be a part of one of those books.

And it was a, that was a fun time.

Welcome to the show, man.

Thank you.

Thank

Viktor: you for having me.

Bret: I just have to compliment you on your setup there.

You've got some nice new hardware that I like.

You got the lights where we're almost matching in our, our lights today

Viktor: exactly.

Bret: Let's back up a second.

Let's go through some announcements.

So you heard me talk about this last week and I've several weeks now every year for
October Digital Ocean has Hacktoberfest and I'm a big fan because one, it promotes
open source and it's designed to help you get involved with helping open source,

you don't have to be a developer by the way.

My first PR.

Which are the pull requests that you do and get hub typically,
that's how you change code and open source usually now.

And I started by just editing, read MES.

So all you have to know, no is some basic texts and maybe
a little mark down, which is a text formatting language.

So that's it.

And then it teaches you, you know, a lot of this stuff here teaches you how to use, get how
to use, get an open source, maybe how to open a poll request and maybe get some feedback.

It's a really great process.

And I highly recommend you do it.

Even if you don't think you're an expert on something, you probably use something in open source.

That's probably stored on GitHub or one of the other ones, but this is really focused on GitHub.

And I bet you you've found at least a spelling error or a grammar check
or something, and you can do those as part of Hacktoberfest and get a free
t-shirt . I think they're giving away 75,000 t-shirts, which is insane.

If you delivered 75,000 t-shirts to my house, could I fit them in my house?

Viktor: You have a house, it's like having an apartment, so you're better equipped than I am.

Bret: Well, you've been to a lot of vendor booths at conferences.

How many t-shirts are normally shipped?

Like, can you, is there a number that you can think, like, do they ship 500 T?

Like , How many t-shirts can you think of that?

You've seen in the real world at one time?

Viktor: Uh, let's say cube con, when we respond sitting, we would bring a bottle of 5,000.

. Bret: Okay.

Is that a room full of boxes?

Viktor: Oh yeah.

That's a storage space for conference, but you just
keep coming with more and more boxes to the booth.

Bret: Yeah.

5,000, even to me 5,000 is like ,hard to imagine I mean, I think I have a lot and I probably
have a hundred t-shirts in the house, but, you know, I can't fit them in drawers because
they're all conferenced t-shirts and you know, there's, they're stuck in boxes too, but

Viktor: my wife forced me to find that, uh, she said
that, Hey, all your t-shirts are walking commercials.

You don't care a single t-shirt.

That is not a commercial.

Right.

I said, that's not true.

Uh, here, she made the bed find me one and you can keep
them, if you don't find one, you need to throw them away.

I was left shirtless.

Bret: Yes.

Well, and I think that t-shirt is an advertisement for being a retro gamer.

Oh yeah.

. Which I'm also a fan of.

In fact I think actually I think I'm kinda missing stuff from my back wall.

I was looking over my shoulder real quick and realizing, I don't
think I have any game or stuff and that's a shame cause I've been.

I've always got it.

My favorite game, I've always got some game

Viktor: let me give you a recommendation by Pandora box.

That's a, life-changer like had them 10,000 games, something like that.

All the retro games you ever will need in your life.

You ever played in your life out there.

Bret: It's amazing.

Is that like a RetroPie?

Cause I have a RetroPie.

No,

Viktor: no.

It's kind of like the, you know, that you mentioned the top of the arcade joysticks and everything.

Bret: Okay.

Viktor: So it's like that, right?

It's a box.

Oh, it's the, at the top of the arcade.

Right?

So you get the joysticks and everything and you just plug it into your television.

Bret: Oh, okay.

Oh my goodness.

I'm going to look this up.

This isn't.

This is a new hobby now.

Cause I read this year, one of my projects, I took a, new raspberry
PI four and I turned it into a retro pie and now I have 40,000 games.

And I've been going through some of the classics, like.

And I think I went back, I went through super Metroid
and beat the whole thing again, to feel like a kid

Viktor: ghosts and

Bret: goblins, ghosts, and goblins.

That's a great one.

Okay.

Viktor: I thought you were going to say, I could never play death and then I would leave this show.

Bret: No, I I've.

I've played that this year.

I've played that game.

Cause you have the, you have like, you start out
with the plate mail and you have the, the javelin, right.

And the fire, it's hard.

That's a hard game, but

Viktor: yeah, arcade games psychotic because you're
not supposed to be playing more than a few minutes.

Bret: Yeah.

You don't have that much money and you're never supposed to
win because how could they ever get your quarters if you won?

Ghosts and goblins.

That's a great one.

That was like a precursor to Castlevania which is another great one.

We need to have a whole show just about retro.

All right.

So let's talk about what you got.

You got real quick, uh, your website technology conversations.

I'm going to put that out there.

Yeah.

He's got a bunch of books.

They're all focused on DevOps, so I highly recommend them.

And he, he keeps releasing new ones and there are, it's like a new version each time and I like it.

So he has this free YouTube channel.

Oh, you're not doing that anymore.

Viktor: No.

Now I have a new concept now.

It's kind of like a never ending catalog in Syncopedia here where it just keeps stuffing stuff.

It's like a garbage can, right.

Kind of play with this.

Let me put it inside, right?

Yeah,

Bret: yeah.

Yeah.

Well, that's what you're doing on your YouTube channel too, is on
the YouTube channel, you had a video recently where you you played
with Octant and didn't have a lot of positive things to say about it.

Viktor: No.

. But yeah.

So the thing is that, consider myself fortunate that people allow me
to do that, but I like giving kind of, this is what they really think
there is no fuss and no kind of political correctness and whatever.

I tried it, I used it.

I don't like it.

Bret: That's it?

Yeah, yeah.

I mean, first impressions matter, right?

We're usually not lacking for tooling.

Right.

It is, it is a tough thing.

And a lot of open source is like this UI design UX are
often like the last things, if ever to be put into a project.

So, you know, it's a challenge and I don't disagree , I'm always looking at it.

Like I have the dashboard, the official Kubernetes dashboard, how is this better?

So I liked how you approached it from the same method of like, why would I consider this?

But yeah, they've actually been on the show this year.

So if you want to know more about Octant and maybe see what features it has
they were on the show earlier this year, it's a VM-ware sponsored project.

, I'm assuming at some point it's going to tie into other things that
they're doing that will bring, maybe bring some more power out of it.

But I haven't seen that yet, but anyway let's get to the topic at hand, which.

Got some people excited in chat recently.

I got some retweets on it that people were interested in this.

GitOps stuff.

And specifically in Argo, because we've talked about, we've had we've on the show
in the last year, talking about flux and Argo is I wouldn't say it's a competitor.

They do have similar functionality, but they differ in the
way they're implemented and then scope of their features.

I'm not going to say they're competitors.

In fact, they're now seem to be friends because they're collaborating on this get ops
engine thing and they're merging a lot of their open source, so you're not following the

Viktor: news and then there's more, and then they
stopped, they got married and then they got divorced.

Bret: Oh, it a falling out.

This is the drama of open.

Yeah.

Viktor: Exactly, but they're not competitors because both are really open source.

Uh, especially Argo Argo has one of the things I like about Argo is that or dislike?

I'm not sure yet, uh, is that there is no really financial interest behind it.

It's more like a prestige because it's really a bank financial institution behind it.

Right.

Kind of money from it.

Uh, but anyways, it is not a competitor, but it is, it serves similar goals.

So let's put it that way.

That's probably nice to say them competitors.

Bret: Yeah.

And that is a good point.

And we're not here to bash on people, but there is a constant challenge
that I have because I love a lot of tools that have paid versions.

And the open, the free version is the limited version like Traefik, right.

I'm technically a Traefik ambassador, that's one of the challenges they have.

And the, one of the challenges I have as a user.

As open-source continues to mature their free product.

That's open source is fantastic, but there are things
I wanted to do that they will probably never put in it.

And they'll probably never accept a PR because it's in their paid version.

So I know that Weave provides a cloud and that the GitOps stuff is a part of it.

So I'm sure that they struggle with that challenge too, of
we still got to make money, but we also love open source.

Not like these people are, you know, there's not a lot of projects out there
that purely make open source, just so that they can get you to buy the product.

And you can kind of tell when you see those two,
because they usually are crappy open-source products.

I would not classify weave as one of those, but since we've had
get ops and flux on the show, do you want to back up a second?

What's the architecture of Argo and it looks like there's multiple projects, right?

So we're specifically talking about one.

Viktor: Yeah.

So in this case, I think , we're talking about Argo CD.

Yeah.

And to begin with, it's a very unfortunate name in my head, the way,
how I think about continuous delivery, this is not continuous delivery.

So let me get that out right away.

Right.

Uh, that does not mean that it's not the good tool.

It's a great tool.

I love it.

It's more one of my favorites, but it's not continuous delivery anyways.

It's relatively simple in what it does.

And what it does is that you have an agent in your cluster that listens or pulls
pulse periodically, or Git Repositories and make sure that the desired state defined
in your get repositories is the same as the actual state, which is your cluster.

If there is a drift between the two, it will.

Do certain actions to make sure that your actual state is the same as the desired state.

So it pulls manifests from get repositories and applies them to the cluster.

If there is a drift between the two states.

Okay.

When I say it's simple, it sounds like, oh, this is a simple,
I mean, there's a lot of complexity how to do all that.

Right.

But from a place from user perspective, it's a relatively simple and straightforward

Intro: tool.

Bret: Yeah.

So real quick, before we actually get into all the tools, it looks like they
have workflows, CD roll-outs and events, yes these all could probably be one
project or one product, but in open source world, we like things to are smaller.

Viktor: exactly.

That's one of the things I like about it.

It's different components that are very specialized in what they do, you know, relatively small
and potentially doing a very good job within a limited scope, then it's up to you to combine them.

I see many cases where people would use, let's say Argo CD
with a different tool for events or, you know, for pipelines.

That's the beauty of it was once you choose to use a huge something, kind of like in the era of many
years ago that we had fit, I'm not going to name right then, then you're kind of flucht yeah, I see.

It's a puzzle box, right?

Your system is a puzzle and then you need to figure out which pieces you're going.

Bret: Yeah.

And that's, you know, that's probably could be the better name for
this show is the DevOps puzzle box because, uh, you know, everybody,
I mean, I've, I'm on, I don't know, five different projects a year.

Maybe.

This year is a little higher than normal for some reason.

And there are no two companies do anything the same, like between their code getting committed
and their code running on a server, that stuff is different on every one of those customers.

Like it might be a different tool.

They might decide to do it.

You know, deployments based on images, they might decide to do deployments based on gigabits.

Like there's just so many variations there that it's hard.

It's hard when cause people show up at this show and they
ask questions and they're always wanting that one way.

Like what's the way.

. But real quick, before we get into the CD, I just want to, I want to throw
people a bone a little bit, at least on Argo so that they can understand it
looks like Argo workflows is the CI tool for testing and more of the traditional
creating a workflow about it, about a run, a bunch of different pipeline goals.

It looks like CD is all about the delivery or the deployment.

And that process in a modern GitOps way.

It looks like rollouts is specifically for Kubernetes and is a more advanced way to do roll
outs instead of a rolling update, which is what we all, when we all start out with Kubernetes,
we start out with rolling updates and that's, uh, that's one way to do it, but there's
all these other ones like Canary and blue green, and it looks like this is a new control.

That provides I'm sure some new CRDs and what I really liked about it is they highlighted for me.

Because I just read this a couple of weeks ago, but they highlighted
for me the reasons why you might want to get away from the rolling
updates standard out of the box deployment strategy of Kubernetes.

Not that it's bad because I know tons of people, they use it, but there are reasons
to evolve beyond that, to something more advanced, which there's lots of choices.

And I love this little list of five things because it's now like this is the kind
of list that I will tell clients as a consultant and make myself sound smart.

You know?

It's a pretty great little list.

So for people out there, uh, check out if you're looking
at more advanced stuff than this is something like that.

And then the last one, there was events, which I don't really get.

I mean, I know what events, the idea of events are in an, in a sense.

But it definitely looks like a Kubernetes specific thing.

Do you know anything about it?

Viktor: The one I use the least.

Uh, so I'm not really equipped to talk much about it,

Bret: to be honest.

Yeah.

But that's okay.

I'm guessing it's the newest one.

Cause I had not really heard of it until recently.

He's got some interesting ideas here on basically turning it into like a bot, like
turning your Kubernetes into instead of using like Zapier, which is what I use for normal.

Like one system talking to another one.

I don't have anything to when I don't want to have to develop it myself.

I'm going to guess that maybe this is that kind of idea that I can
plug into different end points and then automate things on Kubernetes.

But I don't actually know.

So let's get into the actual Argo CD, which was the when I asked
you to be back on the show or when I think you asked me actually.

Talking about all cause we could talk about, we've already covered
10 topics that we could cover talk about, but talking about CD.

So in this Argo CD, what does the CD stand for for you?

Or what does it officially?

Viktor: So within the context of Argo CD,

I would say the, if I, if my description of privacy, the, it is a way to deploy your applications,
following GitOps principles, meaning you start a definition of something, you get it.

It just kept us there.

Right?

It's a deployment mechanism calling the GitOps friendly now
for something to be CD, it from my perspective.

You know, for, for you to deploy something, you need to build it something you
need to create a release, you need to test something and so on and so forth.

So I see at least continuous delivery or city as a full life cycle
of publication starts with the push to get repository and ends.

And this is the tricky part ends with having a release that is
deployable to production or a release that is deployed to production.

And this is the slight difference between deployment and delivery
and deployment, continuous delivery, continuous deployment.

Right.

But the difference is in that button, do you push a button to deploy the production?

Uh, or it just gets

Bret: deployed?

Is it automated?

Yeah.

And for those that, oh, sorry, go ahead.

Viktor: No, no.

And then Argo is a piece of that puzzle.

At least I see it as a piece of this puzzle that, uh, when it, when you get to the point.

Le let's call it pipeline, right?

CD pipeline that you say, Hey, I want this to run in staging, or I want this to run in production.

Normally what the first instinct of every, but then that was
my instinct for a long time was, Hey this is the moment for us.

Should execute kubectl something, or pump something or SSH, right?

Your instinct tells you now I should execute the command, deploys that something, right.

While we dug our go, your job is not to actually execute that command.

Your job is to create a pull request or a push directly to get repository
that defines that production environment and say, Hey, the tug of this
application should now be that the one that I just built and tested,

Bret: right?

And , this is all based on.

GitOps principles.

So we had this question and this is maybe part of the challenge is we were not
necessarily the defining GitOps and distinguishing the difference between that
and what we used before, because a great question that we just got was from
method, man, the difference, but what is the difference between Argo and helm?

Viktor: The difference, the major difference would be that.

So Helm serves two purposes.

One is to define what your application is, right?

Oh, there will be the template for the deployment, for the service, for English, for this and that.

Right?

So it provides a templating solution.

Uh, and then you can execute commands, like Helm install or Margaret to convert those
templates into a request to Kubernetes, to create the objects based on those templates.

So two purposes, two main purposes.

Now Argo does care about the first purpose.

You can use Argo CD with helm definitions or with
customize or with pure Kubernetes, Jamo JSON and Watson.

But it, it takes control of the, of that here, Margaret command, right?

So instead of you executing command there, I want to kill me stall this.

You Argo CD is doing that for you.

And there are many benefits for, from that, first of all,
because it forces you to push that change into, into get right.

You're forced to do that right.

Second is that you do not need access to that cluster anymore.

And this is something that people don't talk much about when, when it took
all those concepts about we, that that might be the most important thing.

I do not need to have access to my production cluster.

And that is amazing.

Now in the past, when somebody says you don't need to have access to production cluster, I
w I hit desire to kill this person, because that meant that, Hey, I would need to wait three
weeks until something happens, because I would need to open 17 JIRA tickets and whatnot.

Um, but in this case you can still do everything you're doing.

It's just that not by directly communicating with your
destination, but by changing up the desire, this is, are stating

Bret: it, right?

Yeah.

I mean, you know, hub, sorry.

Tell him doesn't care how you deploy it, right?

It has its own template thing it's designed to, uh, use that templating when you want
something more than what Kubernetes YAML does out of the box and Argo CD doesn't have a form.

It's not a, it's not a CI standard or something like that.

It's, it's all about creating, get as that single source of truth for every release.

So that the goal really one of the big goals of GitOps is, um, these for
me, is that anyone in my team or anyone who has access to that, get repo,
can see every change we've made in production by going to the get log.

And that's a huge power.

If you've ever worked in a large team or had what I would say, management that loves to get
into the weeds with you and wants to be able to see things, they always want dashboards.

They always want, you know, they want to see what's changing.

And a lot of us, it's not really, it's just hard to even know what's changed.

Right?

What has changed in my production?

What's changed with my cluster infrastructure.

What's changed with my applications and that's really hard to do.

Obviously you can get a code repos, but it's really hard for people to
go, okay, well, Which one is in production, which version is in staging.

We'll get ops tries to solve all those problems by saying, get, is it,
that's the place where all of these things are doing so are happening.

So if you want to change something in production, or if you even want to change
your infrastructure by adding nodes or whatever, all of this can be done and
get, and then you need some sort of controller or some sort of program that's
watching get seeing the changes and understanding the difference between that.

And what's real, whether that's the infrastructure or the applications
that are running, whatever, and then it makes those changes on your behalf.

Right?

So, um, get ops is a relatively generic term and it doesn't have to be get right.

It could be some other form of versioning system, but you know, we're all using get, come on.

I could use, I could use HG ops, but that doesn't really re roll off the tongue very well.

So, I, I think that for those of us that are, are looking at.

Now another thing I'll say real quick is that get ops has primarily showed up.

It was a term coined by we've I don't know, three years ago and get ops has
largely only been implemented in the world of get versioning system, because
that's what most people are using, especially in open source now and to Kubernetes.

So it's possible to do it with other infrastructure.

There's nothing inherently unique.

So a lot of people will ask me, well, what about swarm?

Because swarm do get, GitOps.

Well, anything could do GitOps.

If you have a good ops controller, that's automating,
you don't even need to buy a tool or get a tool.

You could write something that's monitoring, GitOps for, get for changes and then implementing it.

So,

Viktor: oh, I would go even that's probably, it doesn't even have to monitor it for changes.

You can create a web cook from kit to notify the tool.

It can be the other direction.

The ultimate, the real goal is, Hey, you are defining the state and starting that thing.

Out to magically one way or another, by pushing by
pooling, by, uh, gremlins working in your cluster.

That is to become the extra state.

Bret: Yeah.

And I think, yeah.

Yeah.

Thank you so much for clearing that up.

And I think that the weave team that were on the show last year said it really great.

I think they say this in a blog post that if I can still kubectl apply on the server that I'm
not really doing GitOps because the idea is like you were saying, you said earlier, like GitOps.

The idea here is that I'm implementing change through, get not through server commands.

So when I, if I roll out something and it changes the server and it's
something that's problematic, I need to roll out a different version.

I'm doing that and get either I'm doing, I'm taking another branch and merging it,
or, you know, doing some sort of get operation that will make that latest commit.

Be pushed or whatever you need to do on the servers.

Right.

Isn't that?

Yeah.

And

Viktor: let's give you a one reason that you might not have heard off from
before, and you do all that because you're a team player because you know, when
you go to a server and then execute random command, so you go through some UIs
and accomplish certain things by clicking buttons, nobody knows who did what.

And I'm not saying who did what from, from the perspective of policemen, right?

Trying to catch you doing something bad, but we work as a team.

And the best way that I know of today is to work as a
team is to have, uh, everything available to everybody.

And we do that, the engineers do that through code and
pull requests and code reviews and all the good stuff.

Right.

So to me, that's, that's about working in a team.

Bret: Yeah.

And this kind of, sorry.

No.

I would say that this, this kind of evolved out of us all being able to operate
on consistent experiences like GitHub and Bitbucket, like, get lab and whatnot.

We, this, a lot of us were doing this before it was GitOps right.

We were continually trying to automate our tooling and re and cause there was a, uh, a decade
ago or something when infrastructure is code started happening and DevOps was starting to get
some traction that a lot of us that were in infrastructure were like, well, Hey, I'm just going
to start putting all of our infrastructure crap in, get like whatever other system we have.

Like we're not gonna use that.

Or like cloud formation, which back in the day, wasn't even YAML it was
JSON back before we were using Terraform, we were using CloudFormation.

It was like, okay, we're going to track the CloudFormation is, is crazy.

There's a ton of it.

We're going to put it in, get and track all changes and get like the developers do for their code.

And, and I remember we were actually training like new operators and sysadmins that were not.

Developers, right.

We were having to teach them get, because we wanted them to use this to version
instead of some ticketing system or, you know, you name it a Wiki, like just random
things out there that were all storing configuration data, or just screenshots.

Sometimes like if it was on windows, it was a screenshot that said, click this button.

This is what we did.

You know, there was a day where we were using not long ago, maybe
a decade, 20, uh, 12, 12 to 13 years ago, something like that.

2005, 2008, we were using SharePoint to document, writing
down what we changed in production and it never failed.

Right.

I don't know if you were doing that, but it never failed
that someone would forget someone would always forget.

And so we were like, great, great, well, you know, this whole thing is thrown
out the window because the human forgot to add that step to the process.

So this is the output of that is that humans suck.

We will.

Or even automated systems that we were tracking all the different
tooling, , there was some systems out there that would try to track,
all your shell history, any changes to any of the tools on the server.

And it would log all this stuff elsewhere so that you could look
at this log of change, but it's always going to forget a tool.

There's always going to be a tool that it missed or
some change that happened without it knowing yeah.

So this is like you said, this is the best I've seen and you had some thoughts there.

I know.

Let me make sure

We don't have any questions that are relative.

Would you recommend keeping everything and get, and set up C I C D
even for configuration of machines, for example, get lab runner doing
swarm deployments, but also running Ansible playbooks for note hosts.

Viktor: Yeah.

I mean, yes, yes, exactly.

Yes.

Short answer.

Yes.

If it's good.

Bret: It's good.

Yeah.

If and I would argue that every, just everything just put everything Git
everything that you can and Git, and let's relate it to infrastructure.

Yeah.

Viktor: If I can just clarify my previous statement when I said, if it's called it's to Git
when I say code, I mean, if it's something that is interpreted for machines, YAML still code.

Maybe script Fill code, mark down.

yes.

It's still code all the, all that is code

.
Bret: Yeah.

And I think that once you start creating a consistency around get, and people know
that they can start looking at the, get logs for infrastructure, for which applications
are running in which environments, each branch might be its own environment.

And so you can just swap branches and see what, you know, there are entire systems that I've
seen companies build simply to answer the question for the teams, what is running where, right?

What's in our dev, what's in our broad, what's in our staging.

What's in our, you know, we have customer a over here, customer B over here.

What versions are they running?

Git can do all of that without any additional extra functionality.

I wouldn't be surprised.

And then we're starting to see this over the next five, 10 years.

Git becomes this foundational repository for all the
workflows that all these tools are already using.

But what I'm noticing today is that it's still, we're very, we're still like in the
super basic beginnings of this, because for example, there are a lot of GUIs great GUIs.

Like, AWS has a GUI, right?

When I do cloud formation, what I really want out, or, whether it's Terraform
or whatever in front of it, but eventually it's converted into API APIs on AWS.

What I really want is I want the repo and I'm going to get to the question from six corners here.

That's related to, this is when I decided I'm going to do infrastructure as code.

I want that infrastructure GUI management system to be smart enough to know that and that
I can now edit things in the GUI and it will then provide me a workflow to change the code.

So if I, if I.

Editing CloudFormation on my command line or I'm
editing cloud formation through clicking in the GUI.

It's still the same workflow.

It imagine in AWS where if I say, I want to scale up the number of replicas
and then I click a button and then it takes me to basically a workflow.

It's like, okay, you just did that in a GUI.

Now we're going to make a PR for that.

Here's what the changes are going to look like.

We're going to commit it to the GitHub repo or whatever repo, you know what I mean?

Because there's so many tools that are, are touching this stuff from
different places and we're not getting consensus, but I expect that to change.

I really do think that once we realize how better, how much better this get, uh,
based, uh, you know, PR and commit approach is to all changes, not just code changes.

I think that a lot of these tools are gonna have that anyway.

That's my theory,

Terraform cloud, which is very basic, but it's essentially.

An automated way to apply your Terraform through get I
I've wanted them to have a GUI in front of it for a while.

Now that then we'll make the changes to get for me so that if I'm on my iPad and I need
to make a change to production and I don't want to have to go into a shell and edit a
YAML file, that I can just do it in the already existing GUIs that some of these apps
have, and then it just saves it to get, but nobody's really doing that a lot of that yet.

Viktor: But that's the thing I've been very negative towards UIs.

And the main reason that I'm negative is precisely because when I, I do something
in the UI and then it performs action and changes the state of something.

Right.

Yeah.

And I'm not really a case few eyes as UI.

So I'm a guest what they traditionally do, right.

If it would commit my change to kit and then perform the action.

What whichever action is doing right now, I would be fine.

Right.

I'm just against not knowing what's going on.

Bret: Right.

Cause it's out of band, right?

It's a change.

That's not tracked in the same system.

You're tracking all your other changes in.

And I think that's what every GUI right now that I use that I know of . Has
that foundational limitation of, I either have to change things in code and use
infrastructure as code, or I have to use the GUI, but if I do both, I'm screwing myself.

Like it's going to wreck my systems and ruin, ruin my day.

Viktor: Okay.

Because fundamentally using UI is not different from me using Visual Studio Code.

It's a UI.

Yeah.

It's just that this one is local on my laptop and it would be nice if it's remote, but

Bret: yeah.

Yeah, real good.

Better someday.

So six corners ask a related question.

Does Argo put any state back into get, will Argo update images if
the get repo doesn't change, but the tag changes in Docker hub?

Viktor: No.

Uh, Arco

Bret: Argosy questions.

Yes.

Two questions actually,

Viktor: Argo CD assumes that you, or some tool, something that is
not target CD is going to change the desired stating it, right?

Whether you build the images before that, whether you push it manually change to
kit thought through some CIC tool or this or that Argo doesn't care about that.

All it does is monitors your get repository.

And when there is a drift between the what is in kit and what is the actual state it for converged.

So it will not never put something together.

It's not changing the desired state.

It is converging the actual state into the desired state.

And you are in control of that.

These are state, right?

Bret: I think flux has a feature that if you tell it to be, image-based
not commit based and you can give it a parameter and say like anything
with SemVer two.one dot something like you would put in NPM or any SemVer.

And then I, so it technically in flux, it would, it would watch the kid for changes.

But for specifically, for application images, you wouldn't have to
change get, you could just have your CIA tool or whatever you're using
automatically deploy after the test update, it would push to your registry.

Yeah.

Then flux will watch that.

And then I believe it has a feature to re to change the game.

Yeah.

To re reflect the new image version is there is there's nothing like that with Argo.

Yeah, I know.

I know.

I know flux better than Argo

Viktor: and personally, I think it's conceptually wrong because I don't, I'm not sure
that I like the idea of changing the actual state and then saying, Hey, by the way,
that is the desired state kind of conceptually, I'm not sure I like that direction.

Yeah.

Cause it's, it's almost upside down.

It's kind of acknowledging that what you want is actually
what there is, instead of there should be what I want.

Bret: Yeah.

Well, it's not, I agree with what you're saying.

I get what you're saying.

It's not wait until after Kubernetes does anything.

Cause it's not talking to like only thing that talks to Kubernetes is flux.

I think really what it is on flux is that they have an extra feature and it's probably for other
tools, it's just, it would be a separate tool that would do this, but essentially it's watching.

For web hooks from new images that match a particular was it SemVer a
database or whatever your rejects is basically it's following a rejects.

And if it sees a new image with, uh, with the matches, the rejects based on a
rule you put in the git repo, it will then update the repo automatically as a bot.

So it's essentially automatically updating YAML and
then that YAML will, and then it was its own engine.

I'm hoping I'm getting this right.

It's own engine will then take that and do the cube control apply.

Essentially.

It'll do use the controller to talk to the Kubernetes API, but you're right.

If you get those out of order, then you, cause , your
state on the server may not have been successful.

So how you know, that's a lot of, and

Viktor: do you just mentioned one of my personal biggest pain points of GitOps is.

There are so many things that are changing the state of, so you mentioned that
earlier, when you were going through Argo roll-outs calorie deployments, right.

Kind of the permits could fail and then the system will roll back so far.

So good.

I wanted to roll back when there is something terribly wrong.

But it completely breaks the core concept of GitOps because the moment my, you
release throws back to the previous release, I have no idea what is the state?

Of course I know by doing cube cuttle and stuff like that, but right.

Every single from their own, my change is to get real produce unknown results.

It's

Bret: divergent.

Yeah, exactly.

And that's also true for auto-scaling right.

It's true for yeah.

Auto scaling failures.

Rolling, failures.

I'm trying to think what the other one was.

There was, I felt there was.

I know for me, it's a lot of it's auto scaling.

If you scale up to 10 and, but your YAML is still written to say two replicas
when you deploy a new release it'll scale, that, , deployment down to two
services when it's, when it actually needs 10, because the production servers.

So that's the, I guess that's the challenge of get ops is we probably are going to end up
needing multiple features, either built into the product or multiple ways to add extra plugins
or something, or our own automation that will consider these things like auto rollbacks,
auto scaling things that that, because Kubernetes is designed to do things automatically.

So if it does these various things that it can do without
get being aware, it's going to come back to haunt us later.

Does Argo, have anything like that that, you know,

Viktor: No really at the moment.

So city is just doing deployments, right?

It's not concerned about the other direction.

There is Argo rollouts, which does, progressive delivery.

And there is a discussion actually going on for a while, how to make that scale to
combine the two, because independently they're awesome, but they're not really compatible,

Bret: uh,

Viktor: because of that rollback, for example, or let's say if I do calorie deployments and
I do 20% rollout for a full day because I really want to see how my users really react right.

Then that's not an uncommon scenario.

What should be the state in my gate?

Is it the fully new release?

Is it those 20%?

Is it the old release?

I'm throwing you, I'm throwing things that I, I kept no answers.

I think that we are at the very beginning of trying to kind of figure out those things.

How do we ask those questions?

Bret: Yeah.

And specifically for, you know, this conversation is narrowed to Kubernetes, right?

Cause Argo and flux are both very Kubernetes specific.

Not that get ops is, but these tools that are coming out are all focused on Kubernetes.

So one would think that eventually this is going to turn into either
features in the product like flux has done with the watching of images
because maybe for example, I had a client, they actually implemented flux.

They're a solo developer, right?

They're only one person.

So they have, and they're already doing Kubernetes.

So they're already overworked.

They already got too much infrastructure, too much stuff.

They got, they, they got to develop the app.

Then they have to deploy the I'd have to manage the infrastructure.

They have to do all these.

, for reasons they had to stick with E I, yes, E E K S sorry.

Yeah, they had to, so at least EKS was provisioning.

They weren't able to do far gate yet, but they had EKS.

So that part is a little bit easier.

And then I said, well, let's look at automating with flux.

Let's see how easily we can get you to that.

And they're still using a different CA tool , they had their own separate CA tool for testing.

And I think actually in his case, he might just be because he's one person testing it locally.

Like he's not using an automated testing solution.

He's just doing NPM run tests on his local machine.

That's fine.

If he wants to do that, no one else is going to complain that
he's not running tests for the team because he's a team of one.

So he has a workflow where he can do the Docker Compose.

He can do the tasks and he can do like a doctor.

After the build or the Docker Compose bill Docker, Compose push kind of thing.

I think he does have now I think about, he might have Docker
hub building his tests or building his images for him.

Anyway, the point was is that once his tests were successful in image
went to Docker hub, what he didn't want to have to do is either go hand,
edit YAML for every new release that goes to production, which is fair.

And he also didn't want to have to cause he, he liked the idea of
flux and get ops, but he didn't want to have to hand edit YAML.

And so he was thinking, well, do I have my CIA tool now do a rejects or like a, a file parsing
of YAML to pull out the image tag and then change it and then commit it in an automated
fashion, which would basically be him writing, some Linux commands or some script to do that.

Yeah.

Set.

Yeah.

Set her off.

So he's he D he was like, should I do this.

But then I actually asked someone from the flux team and they
said, no, there's actually this feature built in that will do it.

So he was able to do it with a sort of adding one
line into his flux YAML or maybe a couple of lines.

And I thought that was way more elegant than him
having to create his own tool to edit YAML on the fly.

So I'm not sure, like that's the way flux does it.

I like it.

I'm not sure if there is a better way, but overall, the idea of automating it without having to,
who wants to write another command script, workflows step of editing ammo for every little thing.

I don't know.

I go back again.

So

Viktor: it's, you know, there is always, it's obvious.

There is always the answer that starts with depends right
now, the downside of that is that you need to use flexible.

And the offsite of not that is that you can use any format
like, you know, or JSON that, or customize or whatever you want.

And I think that's the major reason why something like that doesn't exist in Argo CD simply
because they did not try to force a certain format for storing the information about what is
running, where which from user perspective is, could be considered kind of like not friendly.

On the other hand, from transitioning to something like that, it could be friendly
because you already have a lot of investment in your helm or, JSON net or this or that.

Right,

Bret: right.

Viktor: Yeah.

Ultimately, and now I'm going back to the years for women.

I I'm lucky that simplicity that we got with Dr.

Wright kind of just all blend different

kind of, uh I'm yet to find a company that gets the depth, the level of user
experience that, Hey, you don't need to write 75 points of scriptures change a YAML.

Yeah.

Bret: Right.

Yeah.

Having some, at least some templates that would automate all of these things, so
that, at the end of the day, there are people that will never be comfortable with
fully automated, but for small projects, like personal projects, I would love without
using Heroku cause that's cheating because they've been doing it for 10 years.

I would love to be able to know that whatever I create, as
long as I got a Kubernetes cluster and a Kubernetes and.

Whatever I commit to it's going to be deployed there and it's going to be,
get ops based and I don't have to, and I don't have to edit multiple things.

I just edit my code.

I trust my test process.

I trust that's going to do all that.

And then I don't have to put together five different
tools and run them all on a separate infrastructure.

Cause I don't want him to be on my production infrastructure.

It just, yeah, it starts to get so it's why consultants love being.

That's why consultants exist is because no one can figure this crap out and no one tool does at all.

Or if they do the problem is they're not great at anything.

Cause they, they do everything.

And for those of us, I think that if we've drank that Kool-Aid
of containers and automation and DevOps, we kind of would really.

Like maybe this good ops will eventually result in these tools that do this
where I literally tell them here's my get ops repo or here's my GitHub repo.

Okay.

Where's your get off repo?

Isn't my Gatehouse repo.

Okay, great.

Here's the GUI do all the things you want in the GUI,
and we're going to get off to everything for you.

And, you don't even have to touch code.

You don't even have to open up them because you're going to be on
your phone, changing production with your phone browser or something,

Viktor: but that will never happen.

Yeah.

And let's explain why, because those specific cases, yes.

We're going to get there.

And it will be like that.

And what will happen after that?

The third expectations for change yet again, and then actually we will evolve more and
to get that more, we will kept to do the things that we just thought that we escaped.

Bret: Okay.

So we're checking, we're chasing a dragon, what we're doing.

Yeah.

Viktor: Because, you know, uh, do you remember the, how many promises we had when VMs appeared?

And actually those promises were fulfilled, but our expectations are not the same anymore.

Bret: Right?

It's a new problem.

Right?

Exactly.

Docker created the new problem of now.

It's really easy to make them reliable and PR and deployable.

Now we just need it to be automated, but that Docker is like, no, we're not going to do all that.

Like

Viktor: I am very excited about, I mean, there are many projects I'm excited
about, but one of the projects I'm excited to is excited about is Google cloud run.

Which is basically allowing you to run your applications without much fuss.

There is no connection to Kubernetes from user perspective.

Yet Kubernetes is running behind the scenes, but for you as a user.

You just, Hey, here's my image, scaly top scale it down,
depending on how much traffic I have and just do the thing.

Yeah.

And I think it's awesome right now.

And I'm using it to be honest for quite a few projects and I
don't get commission from them, so this is not a sales pitch.

But it's also right.

And then, but the expectations would change again and then this will not be enough for me.

But it does simplify things.

Bret: Yeah, I agree.

All right, so questions let's do this rapid fire real quick.

Cause we've already started talking an hour . And some of these questions
may have already been answered, but we're just going to put them on here for
the sake of the audio podcast Argo CD ensures the state of the cluster and
apps hosted on the cluster versus helm is only used for application templatey.

Yeah.

Does that sounds about right?

Viktor: Yes.

I would say it's Helm.

Primary purpose of Helm is to define a manifest of often application.

Well, the purpose of Argo CD is to apply that money.

Now cam can apply it as well, but that's not its primary purpose.

Bret: Right.

And I don't know that him approaches this from a get ops perspective.

Viktor: No, depends.

I mean, look if I still can chart in a good report and then I exit.

That chart from that executes Callum held him up.

Based on the chart from that report, doesn't matter whether it is from
CIA, from city, from a terminal, but I'm effectively doing gift ups, right?

Um, yeah.

I'm effectively applying something defined in kit.

So I would say yes, I think the tech, you have maybe a broader definition of what
Kitsap system is if it's in gate and then it gets applied to the cluster, it's GitOps.

Bret: Yeah.

And I think that's one of the guys was saying that, what do they call it?

They don't call that GitOps.

They call that infrastructure automation or infrastructure as code.

But if you're having to apply it to the server, because the problem is that let's
say you commit to get, and then you go to lunch and you didn't apply it to the.

So get is now out of sync from the server.

Yeah.

And there's no way for anyone to know that.

Whereas if it's fully automated from the moment you put it in, if you're going to put this
in the master branch and the get ops repo, let's just assume the repost called GitOps,
then everyone can, if it's fully automated, they can all look at that and go, I assume this
is what's in production, pending some problem with the deployment or some other variants.

But people can look at it and assume otherwise, if there's a human process involved,
then it's almost like, do I commit to get after I've deployed it before I deploy it?

Only within five minutes of my deployment.

Like it gets kind of weird on whether I can trust, get to be the actual thing that's in production.

Viktor: Let's put it this rather than trust humans to do any repetitive tasks period.

Yeah.

Kind of.

We're not good at it.

Yeah,

Bret: no, we aren't.

And.

Being someone who's, making a course, one thing that's one thing is about making a training course.

There's a ton of processes, like ton of human workflows.

We're always trying to automate them because no matter
how much we document them, we suck at getting them right.

Every time, the same way, every video needs to sound the same, look the same, have
the same volume, same intro, same outro, same colors, and the little pop-ups that
tell you things like there's all that stuff and it's same close captioning quality.

There's so many processes there.

And I can tell you from the last three years is one thing that is really
hard for us is if we can't automate it, it's probably going to not be
perfect because we're going to, we're going to forget it on some videos.

When you ship a course of a hundred videos, it's really hard to make all a hundred videos.

Exactly.

The same consistent experience.

And even when you automate it it's sometimes it's still really hard, right?

So more rapid fire questions.

Just want to get through these.

How to test an artifact and GitOps.

We have different software testing, methodologies like functional and non-functional,
I would say that's not, it has nothing to do with GitOps, GitOps is after that.

Would you agree?

Yeah.

Viktor: Yes.

I will assume that you keep your tests being good and that I would like
to distinguish there is life cycle of an application, right then that
life cycle, because many things building testing and stuff like that.

And this is just part of your idle life cycle of your obligation.

Yeah.

Bret: Yeah.

But it doesn't get ops doesn't re change the way you're testing.

I would say that the more you automate, the more you need to test,
but you can do automation without GitOps and you can do yeah.

Yeah.

Next.

At what point would you advise switching to Argo from example Azure
DevOps, assuming that there is no manual intervention in a cadence cluster?

Viktor: I don't think it would be switching.

So you have Azure DevOps.

That is your pipeline, right?

Basically it's set of tasks as you tip under certain conditions and you might just
want to modify it so that instead of Azure DevOps executing cube capital or whatever
you executing to apply that to the go back to kit, doesn't matter which tool you use.

It could be you using said what we commented before or
with flux or with this, or with that, that doesn't matter.

But the ultimate goal is that you don't deploy from your Azure DevOps.

You modify the state desired state in kids saying, I want this
release to be in production or in staging expression of a desire.

Bret: How do you recommend handling the secrets from get to keep the credentials hidden?

Well, first off I'll say that that is not a good ops related thing.

Um, well he said, get, sorry, Santiago.

Uh, you said, how do you handle secrets and get so slightly related question.

If you're new to putting your infrastructure in, get, this is definitely a topic.

There's some answers in chat.

A couple people in chat said sealed secrets.

I like sealed secrets.

I don't know if you've ever used that, for example.

Yeah.

Yeah.

, you can use the cloud secret provider.

You can use Kubernetes secret provider.

You can use vault.

Viktor: It does not necessarily have to be in, get really reference of it.

Like the same thing.

Like my application is nothing.

Yeah.

Just as my secrets or nothing, get references to my application, to the container image
aren't get, . The source of truth is I want to use this specific tag of this specific image.

The same thing can be for secrets, they can be encrypted one way or
another in, or it can be referenced, Hey, use this secret from vault.

That's perfectly

Bret: fine.

Right.

And yeah, references to secrets are fine because it's not the secret.

So it's not, it's fine.

You can have that in a public repo sealed secrets is a way to basically encrypt
the secret so that the only way to decrypt it is to put it on the cluster.

And that's a way in open source for us to now put secrets in there in public and not have to sweat
it not have to break the rules of you never should put secrets in and get repose, what's next up.

All right.

How do you let's see, so sealed secrets got that, or got the secrets thing for Santiago there.

Best practices for GitOps.

Well, I think we're talking about all those, so I'm
going to assume that we're answering that question.

If you're on Azure, look at Azure arc, it uses flux did not know that.

Thank you, Martin.

As your arc we have some disdain for Jenkins.

I don't know what it is.

Let's see what.

Oh, sorry.

Bikers answering secrets are really the domain of a orchestrator, like
case or swarm and external tools like vault encryption at the rest.

Yeah.

So get is not the right place to store secrets.

Yes.

That's a great response.

Thank you for that.

There's some kids like vault.

Yeah.

I think we, a Santiago answered his questions.

Yeah.

Sealed secrets.

That's the latest trend is the sealed secrets idea of basically using
standard PKI approach of things are encrypted with a secret so that the
thing can be in public, just like we'd use all TLS and SSL every day.

That's the same kind of PKI approach.

Oh boy, we're on a trend today.

Santiago's now it a super sticker.

You know, I always watch all those Twitter, those Twitch channels where they're
everybody's buying them coins or whatever, and I'm like, Hm, that must be nice.

So thank you for those breaking the ice today with some of the donations.

I really appreciate it.

And sadly Victor will get none of it.

So he's, he's, he's doing this for

Viktor: free.

I'm gonna bring some geographic kelp coffee with him.

Bret: Oh, nice.

Yes.

And of course, I'm sure over on the DevOps toolkit a channel, you can go
over there and if you're just joining us, I've got Victor on and we're where
he's got his own channel where he's doing some really cool discussions.

I like how he goes in depth.

He spends 20 minutes really talking about a topic, going deep on it.

And it's well done and he's got good quality hardware and his production is nice.

So, uh, in case you don't know what his books are, uh, you can go over to his website.

. He's got lots of them.

I mean, he's got so many that he doesn't even make a list on his website.

He's like, Okay,

Viktor: because I don't even know which ones I removed because some become tech
changes and then some become obsolete deprecated, and then I removed them anyway.

Bret: That of course is on Udemy.

You have courses on Udemy?.

Yeah, I

Viktor: do.

Bret: Nice.

Viktor: Yeah.

And the, the last one is actually, I was experimenting before, but last one is the one I want.

That's the one I've commented before the show is like, never ending.

I just keep adding stuff there.

Whatever I added there.

It's

Bret: DevOps.

Is this the DevOps catalog?

Yeah.

Oh man.

DevOps catalog patterns and blueprints.

This is perfect.

We were just talking about this.

All right.

Viktor: Well, the last one I did was Argo.

That's why I keep it in my head.

And the first thing is that people ask me, okay can you tell me what else will be there?

So I make a decision.

I give an idea,

Bret: look at all this stuff, 11 hours of videos, all right.

People, you know what to do, but no,

Viktor: I've watched your course there.

I I'm, I'm a very beginner style compared to yours, and I'm not trying to flatter.

I'm really impressed.

So if you're going to put something on a screen, you should put yours, man.

.
Bret: Well, I mean, everybody has their own style.

Everybody, I feel like the world's big enough that we can all have our own
little sandbox and, everybody's a friend of me to me, uh, you know, we all
might compete in some way for time and content, but as far as I'm concerned
the more people the better and the more courses, the better, it's all good.

As long as I can make a living and pay my bills.

It's all good.

All right.

, I don't think we need to do demos cause I think we covered
this and we've been going almost an hour and a half now.

, there are demos of Argo on Netflix or Netflix on YouTube, uh, and in
his course, so go by his course for the demos, they have walkthroughs
even on their website, you know, they have a little simple things.

They have stuff in the manual guides.

That's how I first tried to use it was through their user guides.

But he's got, he talks about on his YouTube channel.

He talks about it in his Udemy course..

So go do that for the demos.

I think that you probably got a good sense for what this is all about.

. Hey man.

Uh, can you

Viktor: do me a favor?

Yeah.

Can I mention my company then I did this on company time instead of free time.

Bret: Oh yes.

Yeah.

So let's just talk about codefresh for a second which is not why he's here, but
you know, Codefresh is another one of these platforms as adapting as a CI tool,
it's adapting to the DevOps workflows and specifically to the get ops workflows.

So you'll even see them talking about it on their website.

Anything you want to say?

Viktor: No, I think it's, I mean, try it out and you know, I can tell
you, I worked for corporate so I can tell you that it's awesome because
it's awesome and so on and support, but then that would be marketing.

I'm not going to do that.

So try it out.

You get unlimited builds for free.

Now there are.

Features like with everybody you pay later on if you, if we want them, but try it, talk to vault
cost anything, and then let me know what you think and ping me at any time if you need any help.

Bret: Yeah.

You know, I think there's right now in this we talked about this for the
last couple of years, especially like last year 2019, DockerCon is that the
CI and CD space is where a lot of the innovation is happening right now.

We kind of figured out containers, we've kind of at least
got the current iteration of what orchestration means.

And now we're trying to figure out that piece in the middle.

So, I would say that there's a lot of players in that
market, but they're all innovating in different ways.

, we all already have probably our own way of testing and doing on some automation
and workflows, but it probably is worth your time to at least try one out every
year, a new one every, you know, six months or something just to give it a shot.

Because , Code fresh is a newer option and it's sometimes takes newer
players to create that innovation so that the industry can move forward.

So, yeah.

Thank you.

Code fresh for lending Victor to us for the afternoon.

Well thank you so much for being on the show.

This is not your first time and it will not be your last on the show.

I always enjoy it and I, this will be a great podcast for everybody
listening in on more get ops topics on our podcast, which we get now
we're like up to like 15,000 downloads a month or something on that.

So there's definitely people that are listening to it
and they will be educated on Argo and what GitOps is.

So thanks to all the super chat out there.

Thanks to my Patreon members, my patrons.

I thank you so much for your support.

It is a big deal that you are consistently supporting
me every month with buying me a coffee, essentially.

And it helps the whole team here, which is not just.

It's not just Victor.

We've got other people behind the scenes making all this stuff happen.

So I really appreciate you.

And we will be back here next week like this video, if you want Victor to come
back because I make more stuff with the more videos that are liked, like the videos
that get the most traction or the videos that we keep doing over and over again.

So if you want to hear more theories on continuous integration and continuous
deployment and delivery please like this video and subscribe to Victor's channel.

We'll see you next week.

Chat.

Everybody shares.

Thanks so much for listening and I'll see you in the next episode.