K8s Maxxing with AI-Native Platform Engineering Stack with OpenChoreo
Download MP3Bret: Welcome to DevOps and Docker
Talk, and I am your host, Bret Fisher.
Technically, the full name of
this is Cloud Native DevOps and
Docker Talk with host Bret Fisher.
But, you know, DDT is what
we call it inside the team.
So DDT, we're now in our seventh year
of podcasting, and if you include
the livestreams I was doing before
we started the podcast, I think
it's over eight, maybe even nine at
this point, which is pretty amazing.
Uh, not a lot of podcasts get to
last a decade, so I'm anxious to
get there, and still have it called
DevOps and Docker Talk, which DevOps
never died, Docker never died.
Like, we didn't have to change the name
But on here, we talk about all cloud
native things, and another Kubernetes
project has begun and has now
showed up as a CNCF sandbox project
this year, and that is OpenChoreo.
Hence, open…
Which stands for Open Choreography,
and that's where Choreo… I was
actually mispronouncing it as choreo or
something with a hard H, but it's choreo.
And this project is all about adding on AI
agents and essentially a new abstraction
on Kubernetes itself that interfaces
between what we're all now using as our
primary interface to everything software,
which is agent harnesses, and what the
Kubernetes API and other APIs provide
that are typically on Kubernetes, like
your monitoring or your observability or
whatever else you might need in there.
There's a lot that goes on.
But this project is adding a new
abstraction layer, I'm calling it an
abstraction layer, because it provides
API, MCP, CLI, and web UI all to
front end everything else you typically
deal with, whether that's your GitOps
tools like Argo, your orchestration
Your storage, your observability,
Kubernetes itself, your
authentication, like all that stuff.
And we are now in the world where we're
all playing around with agent SDKs and
maybe trying to figure out what can we
automate in our infrastructure safely
with reliable LLM prompts like this.
These are all things we're
probably thinking about, right?
If you're not doing it, you're thinking
about it, you're waiting for someone
to tell you how to get it done.
And OpenChoreo is a new project that
came out of WSO2, the company, and
they were running it on their own SaaS
infrastructure and realized that it might
be useful to others, which by the way, was
also how the Docker project got started
by a SaaS company or a cloud company
realizing that their infrastructure
tooling was maybe the most useful part of
their product, and released it for free.
So OpenChoreo came out of a
business hosting a legitimate
tool for Kubernetes users and felt
like it needed to be open source.
So now they're gonna be running
that on their own platform
for customers eventually.
But for now, they're just focused
on getting it open source.
And now that it's officially a CNCF
sandbox project, that means they've met
the minimum level of sort of maturity
to explore whether this could succeed in
the marketplace, explore whether or not
those of us that are building open source
tools or running open source tools on
top of Kubernetes would be interested in
an agent platform on top of Kubernetes.
And specifically what this is about is
about us administrating and managing
these environments with agent harnesses
through the OpenChoreo abstraction,
so through their MCP API CLI.
This is not about running inference
or building custom models.
This isn't about being an AIOps
or MLOps person, and this isn't
about running custom agents that we
built in pods inside of a cluster,
although that's certainly possible.
You can do that.
But the main focus here is on exactly
what I talk about on this show, as well
as my other podcast, Agentic DevOps.
And quite frankly, this podcast
could probably be on both.
I'm trying to prevent that from
happening because they're, you
know, we've all got podcast players.
You can add both of my podcasts
to the, your player, right?
So you know that that's a thing.
Um, and this one I'm trying to keep
focused on CNCF projects, Kubernetes and
Docker projects, and the Agentics DevOps
podcast is really about the patterns,
behaviors, and tools for just being
a DevOps or platform engineer or SRE
who's using AI to get their job done.
That's what that podcast is about.
Technically, this could be on both.
But, uh, anyway, On the show this
time, I have Lakmal and Sameera from
the project team over at wso2 talking
about this, getting into the details
of what it does and doesn't do, and
also explaining to me how it's not
what I actually thought it was when
we first started talking about it.
And we even on the livestream, which
you can link in the show notes, there's
links to the actual demos that we tend to
cut out of the podcast version of this,
but the demos are from the livestream.
You can see how the product works, how
you can use agents to interface with
Kubernetes, which we, could use agents
before, but we didn't necessarily
have all of that infrastructure
in place besides just a raw API.
So this helps solve a lot of that
with a new level of abstraction
that we get into through this show.
So let's get into it Welcome to the show.
in the middle there, we got Lakmal, and
on the right, Samira, both from WSO2.
Lakmal, tell us about yourself.
Who are you?
Why are you here?
Lakmal: Yeah, I'm, working as a VP
and distinguished engineer at WS02,
also a Choreo, DM before Choreo BU.
yeah, I started my career as a
system engineer, in 2000, one,
and last 25 years, I engaging
with building many, many platform.
Yeah, here we are today.
Bret: Nice.
Sameera, how about you?
Sameera: Hey, Bret.
I'm good.
I'm Sameera Jaswal.
I'm one of the core maintainers
of OpenChoreo at WSO2, too.
I started my career in the middle as a,
you know, you can think of programming
language design, compile engineer,
platform architect, and then you'll
hear more about how abstractions and
compilation throughout this talk.
me.
Bret: a gist by the end of
what you really are into.
I'll get it.
Okay.
all right.
So y- you all reached out.
This is a new project.
I'm always excited about new CNCF
projects because it kind of… It--
Once they've accepted into the sandbox
process, we kind of get an idea.
I feel like it's reading
the tea leaves a little bit.
Not everything is a winner, not every
project makes it to graduation, but it's
an interesting way, I think, to, like,
separate sort of the hobby projects or
the experimentations from things that
teams are taking seriously, 'cause
you don't really go for sandbox status
unless you're taking things seriously.
That's where I sort of like wake
up and start paying attention to
the new projects, and I'm excited
to talk about this today because
I'm obsessed with AI agents.
I'm obsessed with, like, how we manage
infrastructure with agents involved.
How do we put MCP in our clusters safely?
how do we sit on our harness as our
primary mechanism for interacting
with, I don't know, 20 to 40 years of
complexity that we've all built up?
Like, the number of abstractions we
all have to deal with today is insane,
and it's amazing that anyone can
actually go to a conference and have
a common conversation, 'cause there's
a thousand tools we're all using.
So I'm excited to get into this.
how did the project get started?
who's gonna pick the first question?
Like, how did this project
come out of the company?
Lakmal: Yeah, we started
this, software offering called
Choreo SaaS, five years back.
it's, it start as a, the IPaaS initially,
like a integration platform as a service.
Then it evolved into the…
You can run all other services
within the platform itself.
So it's, it evolved into the internal
developer platform, couple of years later.
Name is also coming as a choreography.
It's, we shorten it to the Choreo.
It's like a choreography your
microservices in a certain way, like a
Kubernetes do the orchestration, we do
choreography for all your microservices.
That, that how the name also came.
so we were offering this SaaS offering
to our customers, enterprise customers.
It's fully managed by WS02.
we were running Choreo version
2, and two years back we started
Choreo version 3, project.
So next version of the,
our current offering.
Then eventually it's become the OpenChoreo
and in WS02, all our software open source.
So we thought of, "Okay, why not we
are running… we having this software
running as our SaaS. Why not open source
it?" That's how this OpenChoreo, started.
Then, this year, January, we have
donated into the CNCf, Foundation.
Now we are a sandbox project.
Bret: Yeah.
All happening so fast.
that's a similar path, by the way,
to how Docker was invented, 'cause it
was a platform company saying, "Hey,
this piece of our platform is pretty
interesting, but it's also c- at this
point maybe a utility. let's just give
it away." Turned out that that worked.
so it's not a bad idea to do this.
what was the unique problem we
were trying to solve with this?
Like, I say we.
I'm, I'm a part of the team now.
how is it, how does the team look
at it from, like, this is a unique
thing in a world where we've got
dozens of Kubernetes distros?
In fact, first off, I should say,
do you even call, do you call
this a, do you think of this as
like a distribution of Kubernetes?
Sameera: N-not really.
Bret: No?
Like, do you see it as like a vanilla
Kubernetes with some things bolted on?
Or like how is this different
from like a Rancher?
Sameera: okay, I'll,
I'll take that question.
in my opinion, so OpenChoreo
is something you, something we
have built on top of Kubernetes.
Bret: Mm-hmm.
Sameera: So you can use any of
Kubernetes distro, Rancher, EKS, AKS,
like any Kubernetes, and then you can
just install OpenChoreo on top of it.
So the idea behind is that, so
you have Kubernetes and all the
other tools, and then, so you have
a developer platform, I think.
what we have seen in the industry is,
so you ca- you have this developer
platform with Kubernetes and Argo
CD and all the things, glued on,
and then you expose that layer to
developers, and you build a developer
experience around it, portals, MCPs.
then we have figured out that,
that way you expose the platform
complexity to your developers.
So they'll have to learn Kubernetes,
they'll have to learn all the tools.
I think what's unique about here is, I
don't know whether this will work out, we
build a middle layer, abstraction layer
on top of Kubernetes and other tools.
And that's what Lakshman said earlier.
Through like six years, we have
figured out the abstraction layer.
That took some time for us to build.
And then that abstraction layer helps for
us to build a developer experience layer.
So developers, they're like kind of
abstracted away from Kubernetes, per se.
They can still see what's going on, but
they may not to learn a lot of things
about Kubernetes and other tools.
Bret: Yeah
Sameera: so that's about one of
the unique layers in the platform.
Bret: So when we talk specifically
about Open- OpenChoreo,
is it installing all the parts of this?
Okay, so we're looking at it
for the, for the audio audience.
We're looking at a visualization.
I love diagrams 'cause they really help
me I l- the conceptual-ness of like what a
thing is, is sometimes instead of a bullet
list, I love these, so, I brought that
up, and there's all sorts of com- It's…
To me, it looks like a full-fledged,
fully built out Kubernetes cluster
with all the extras, right?
Like, all the necessary things that a
sort of an enterprise cluster would have.
it's got these data components.
You know, you're mentioning
Keda for scaling.
You've got Cilium in the networking.
You've, you've got, you know, API
gateways in there, and then you
showed part of the diagram, there's
the workflow part where it's Argo
workflows and build packs in there.
And then you've got observability layers
where you've got probably , I don't
see Loki in there, but like Prometheus
and all the OpenTels, OTel stuff.
And then over to the side, there's
OpenChoreo as this control plane.
So it's almost like to me it's like the
control plane on top of the control plane.
so is this really just focused about those
things on the left and you're showing how
it interacts, or is this actually helping
us implement the rest of this puzzle?
I was a little confused by that.
Lakmal: Yeah.
I think the main idea behind the,
having the unified control plane
across all these, capabilities.
p-platform engineers or developers or
even now AI agents can interact with
control plane via our different protocols.
And then control plane do the, all the
orchestration across all these other
planes, like you, you mentioned the data
plane with the Cilium network policy,
with the S-Scale Zero, with the Keda.
All these thing can orchestrate using
a, a control plane with the governance
enforce through the control plane.
That's the main idea behind.
So what, what have seen as a problem,
even we expose all these services to
the end user, it can be a platform
engineer, it can developer, then
you can't enforce the governance.
But having this single control
layer, we can have the guardrails,
policy enforcement, and y-
single, pane of glass, we can see
everything within the control plane.
So that help to orchestrate
across entire different pl-planes.
That planes, built b- using the
modular architect in OpenChoreo.
So for example, i-if you take a API
gateway, it can be come from the, WSO2
gateway, K gateway, so, Co-Kong gateway.
It doesn't tie to a single vendor.
So it, it's building an
entire ecosystem around it.
it's not only the platform itself.
Users can pick and choose what they
want to use in their data plane.
so that, the abstraction layer
help to, os- orchestrate all these
different, vendor tooling that they
want to use in the, their planes
Bret: so, so is it-- If I'm a platform
engineer, I-- and I'm listening
to this show, that probably means
I have platforms today, right?
Like I've already got
Kubernetes in production.
I've already got a lot
of these components.
So is the idea h-here that this particular
project is for implementing the left
side of my screen of the specific
OpenChoreo control plane components,
and then including those system
components that would link or integrate
me with the APIs of those other things?
Is that how-- Is that sort of
where you're drawing your boundary
of where your project ends?
Sameera: Yeah.
yeah, I think that- that's
a good way to put it, Bret.
So our main component would
be the control plane, that you
see on the left of the screen.
And then at the same time, there
are certain system components
that are running in all the other
planes as well, data planes,
observability plane, workflow planes.
So if you're already running these
projects, then it's just a matter
of like installing our system
components and organizing your sys-
platform architecture in this way,
Bret: Yeah
Sameera: right?
and then I think what Lakmal said
about abstractions is that the…
When you think about these developer
platforms, primary users are platform
engineers, as you said, and then,
primary consumers are developers,
Bret: Right
Sameera: So, so they will perhaps,
they will use the experience plane,
UI, MCP server and things like that.
They will say, "I, I want a project, I
want a component. I want component to
de- deploy on dev environment," right?
And the control plane understands that
and executes and compile that into
Kubernetes and other projects, so in a
way that other projects understand, right?
So that's how these arrows and everything,
that's the idea behind these arrows.
Bret: Yeah.
And when I'm looking at this diagram,
I mean, from s- from sort of a
platform team perspective, I mean,
I recognize all the names, right?
It's a lot of terminology,
from different, CNCF projects.
But, are you thinking of this as AI
is the sort of main interaction point?
I see here that we've got, like there's,
I think there's a, a web console or
something, So, do you think about
this from the perspective of like AI
is going to probably be increasingly
the way we interact with Kubernetes?
So this thing, you know, we've got
these arrows in here of how things are
interacting with the Kubernetes cluster,
and it feels like it's all going through
the OpenChoreo control plane to get there.
'Cause you've got this list of like,
we could use the U- Backstage UI,
presumably that would be for humans only.
unless maybe there's some pretty
graphs for the AI to look at.
I'm not sure why it would use that.
But then you've got MCP, CLI, and API as
these other interaction endpoints, and
they're not… When I think, when I see
this diagram, I think, "Oh, this allows me
to put another layer of abstraction for my
AI to work through, and so I'm not running
my AI directly against a kubectl API,
or, you know, the Kubernetes native API."
I've sort of got this enhanced layer,
with additional functionality, presumably
maybe some… Yeah, I see you have
authorization and access control,
which is a big thing for me lately.
Like how do we minimize the blast radius
of giving agents access to different
tokens and like not giving them root, you
know, God rights to the cluster every day?
which I see a lot of people doing by just
giving them the kubectl admin abilities.
is that kind of the, the goal
of, of where you see agents being
responsible for all these clusters?
Lakmal: Yeah.
E-exactly that, you
describe is correctly, Bret.
So, what we want to have is, now in
the developer platform, we give the
golden path, the guardrails, the
policies to our human developer.
The same way we want to expose
this golden path to the AI agents
that interact within the platform.
So in, in OpenChoreo, the agents
are in first class citizens, so
they can, at the moment, current,
release, we exposing two MCP servers.
One is the OpenChoreo control
pane MCP server, other one is the
OpenChoreo observability MCP server.
So even external agents can
interact with this MCP as well
as internal agent, inbuilt agents
can interact with MCP C- servers.
Now, when they interact with MCP
servers, they have the permissions
and all the guardrails will apply same
as the developer, using the platform.
It's a unified experience, whether it's
a human developer, human SRE, or a agent,
working with the, SRE a- or as a developer
Bret: Very cool.
All right, so this shows up as a
whole bunch of MCP tools that I
could plug in locally to my local…
So if I've got Claude Code running,
I plug the MCP endpoints in.
do you know, like, the
estimated tool count?
Like, what are we dealing with here?
100, 100 tools?
20 tools?
Lakmal: yeah.
we have, depending on the user
persona interact with MCP.
So we have the developer
persona, we have the SRE persona.
They give different, tooling
based on the, what their role.
Yeah, so based on that they can
do different, different, activity.
Sameera: roughly 100.
Bret: Okay.
Okay.
Sameera: say 100 tools altogether
Bret: a not insignificant amount
of tools, but that's a good point.
So what you're saying is based on who I'm
au- authing as, which role I'm authing
as, I, I get a scoped set of MCP tools
based on my permissions essentially?
Yeah.
Sameera: exactly
Bret: So, all right.
so these agents are coming, you're
listing AI agent modules here.
You got SRE, FinOps, and architect.
are these things that are,
that are coming defined?
Like, and what does that agent mean to me?
Like, if I'm on my local harness, does
this mean that those are long-running
agents that are sitting in the cluster?
Like, how do I interact with those agents?
Sameera: like Lakmal said earlier, we
have internal agents and external agents.
They both use the same experience,
plain MCP servers, tools.
Claude Code is the external agent.
these FinOps, SRE, and architecture
agent, they're like platform
agents running in the cluster,
Bret: Yeah
Sameera: and they are reacting
to certain events in a way.
For example, if you, if there's an
alert, like lot of i00, errors are
going on, then our SRE agent will react,
and then it'll give you some report on
what's going on, why this is happening.
So you can configure that way.
You can configure the agent with
certain alerts, types of alerts,
memory, memory, like if you are
consuming high memory, certain limit
that SRE agent will again react.
Bret: Sure
Sameera: FinOps agent is
also similar in that way.
Yeah.
Bret: I see.
So each of those agents comes sort
of predefined, it sounds like.
Like, they have a scope to them.
And then presumably, I guess I'm
somehow through the control plane, I'm
plugging those into, like, my messaging
apps or other… Like, into Slack or
whatever, and that those are how those
agents reach out to me is through…
Or do they go through a middle
layer of, like, an alerting system
like Prometheus has for alerts?
Like, how does that path
look for the agents letting
us all know of the problems?
Lakmal: Yeah.
at the moment it's going a middle
layer, the alerting system.
we have the log-based alerting,
with the OpenSearch and other layer,
and the Prometheus for the metrics.
So agents also ke-going the same layer.
this alerting system can be,
configured with the PagerDuty or
Slack channel or the other channels.
That how the architecture
currently working.
Yeah
Bret: Yeah.
And so this feels like it's designed to be
sort of, I mean, we, I think the website
or something said batteries included.
I think I included that
in the, in the thumbnail.
It, 'cause it, a lot of the,
the conversations that I have,
I mean, I'm lucky to be in this,
this DevOps, Agentic DevOps guild
that I run, and we meet weekly.
So I get to talk with dozens of teams
around, like, what they're trying to
do right now in terms of everything
agents related to infrastructure
and this exact stuff, right?
So people are sort of in that
middle mode right now where
we've-- the local agent harness has
kind of become common knowledge.
Like, we've all experimented with
GUIs and TUIs and multi-agent
management and sessions and skills.
Like, we're all, you know, this is
the year of s- playing with skills.
And, but then the minute that I have
to start running agents, I'm, I just
call these things server agents b-
to try to define the difference.
These 20, these long-running, always
looping agents that are sitting there
checking something or, continually
polling for something or waiting for a
webhook, maybe if they're event-driven.
Like, these things are
probably gonna be everywhere.
We're gonna have them all over the place.
And y- you're providing th- a few of those
that are sort of out of the box defined
to do specific things, so I don't have to
go find an agent SDK and make up my ideas
of what an agent might do in my cluster.
'Cause that kind of feels like what
a lot of us are doing right now,
is we're sort of making up, we're
writing little a- agents everywhere.
We're making it up as we go along
'cause we don't really know what
parts that they, we would automate
or which parts they would be good at.
You know, what parts of summarization
and human judgment do they do well,
and what context do they need?
And that's… So, but it feels
like the rest of this system, is
the rest of this system to help
them with the context problem?
'Cause that always feels like a
really hard problem right now around
how, what context do I need to give
my agent to make it useful and not
hallucinate, and how do I get at
that stuff at the time it needs it?
Lakmal: Yeah, exactly.
So I think the main idea behind
we have this ecosystem, within
the ecosystem we have the agents.
So, in community or even, we, OpenChoreo
maintainers will release a agent, then
our users can pick and choose this
agent run in their OpenChoreo system.
So for example, for SRE agent, the
intention behind that, okay, when I say
when something happen, SRE agent can,
triggered and we feeding, all the logs,
matrices, config changes, code changes
to the SRE agent within that window.
Then, what happen, so it can generate the
RCA report, root cause analysis report,
and also alerting to the SRE, human SRE.
When they come into the system,
they already have the root cause
analysis report there, that
is providing to the SRE agent.
And also we are not stopping there.
this agent can provide the
re-remediation action as well.
Okay, this is a quick fix if
you want to, fix this issue.
So then at the moment it's a
human in the loop, SRE, human
SRE can apply these fixes.
But, I mean, we have the permission
model where we can give some of the,
this agent to automatically apply these
fixes, themself, rather waiting to SRE.
We can say, "Okay, is it only config
change? I will allow my SRE agent to
apply that change and fix the issue."
So eventually it can take the more,
more like activity, but it's again,
it's had the human control, in the loop.
So that's how the, we have designed it.
Like you said, the main i- intention
behind it, so we running a system
agent, you can, you say, survey agent.
We know what are the context we
need, feed into this agent to
provide a better, better result.
So that's how these
inbuilt agents are, acting.
Sameera: Yeah, and I also think,
like, if you're running something,
this is, these agents are a good
starting point for you to come up
with your own agents as well using.
And then you can replicate with, you can,
come up with your own agents because the
context problem is kinda solved i- in,
within the platform, what Lakmal said.
So you can… Because these are all
about not exposing Kubernetes details
into the agent, but exposing the
OpenChoreo abstraction layer, so that
way it's what we have figured out.
I mean, not sure whether it'll work.
Bret: that's the hard part.
Yeah.
Sameera: Yeah.
Figure out is this abstraction field
helped us to give the right context also
Bret: Yeah.
all right.
so I was writing this article over
the last week around what does an
AI Kubernetes platform even mean?
And what we're really talking about
here is something that I had to define
in this newsletter as what I'm calling
type three AI on Kubernetes . It sounds
really nerdy, but if you've been going
to KubeCons since the invention of
ChatGPT, for most of the history of
what we now call gen AI or whatever-- we
are not talking about AIOps and MLOps.
I mean, technically you could use this
thing to manage that too, but for those of
us that go to KubeCon, there's been-- I've
been ranting a little bit for a couple
of years now that there's been this weird
disconnect where when a lot of people
in the industry talk about Kubernetes
and AI, they're actually talking about
running AI or making AI, building
models, reinforcement, learning, running
inference, and that, that is not my world.
Like, I am not that person, right?
It's interesting from an architecture
and engineering perspective, but
I don't think I'll ever… that to
me, that's a speci-specialty and it
might get easier to do, and some of
us might have to have that new role.
But I feel like there's still a
significant, maybe even majority
of us that won't be doing that.
our new job is to build out these
agents and understand where the, you
know, the features and the edges of
models can be implemented so that
we can automate and build more.
Because I see this whole thing as just
like VMs, the cloud, the invention
of the PC, you… I-over my 30-year
career, we've had major pivoting points
in technology that essentially allowed
us to scale ourselves as a human.
We went from managing 10 servers in
the '90s to 100 servers with VMs to
a, a thousand servers with the cloud.
Maybe containers allowed us to run
10,000 pods per, per admin, and
then now this is the next level.
And to me, what you all are building is
exactly that layer of abstraction, which
is the agents and their context management
and permission management essentially.
that layer is now this new
level of abstraction that will
hide some of the complexity.
Like, we don't have to know every
kubectl command in the world anymore.
In fact, I don't even know how we all
get Kubernetes certified because if
we're not gonna be running kubectl
and have to know every option, what
is a Kubernetes admin test other
than just understanding architecture?
But my rant here is that like I had to
go through… This post took me a re-week
to write because I was trying to theorize
around we're really talking about agentic
operations, and that's where I'm--
that's the reason I have the new podcast.
That's the reason this project is
exciting, is this is that new layer.
This is the abstraction that
we're all kind of searching for
because I think we've understood
the local harness now after having
those for a little over a year.
I think a lot of us are like, "Yeah, I,
I get this." We have all sorts of neat
ways of managing these little local
agents, but this new nebulous layer that's
sitting in front of our infrastructure
is still not fully realized.
So I feel like you all are pretty early in
the game in terms of saying, we can help.
We have this defined component, and we can
shove it in there." And, I'm even trying
to draw out a diagram of what- If we're
all on a maturity path to, we started
with the agent harness locally, and the
end goal is we've maximized the agent
assistance that we're all gonna have.
Like the dozen agents we're gonna
have, two dozen agents, we've maximized
all the functions and features of
what models can do for us as DevOps
platform engineers, SREs, right?
all of us are managing this infrastructure
and there's a dozen tools and f- the
thousand features we all now need
to learn, and we're learning skills
and agent files and MCPs and where
all that can go awry and go crazy.
But eventually the idea is we're improving
our productivity, we're improving our
management, we're hopefully making
things more secure and reducing,
outages and everything's getting better.
But at the end of it, what are we doing?
And I think a year ago we were all kind
of nervous like, are our jobs going away?
that's been a big conversation.
are we-- Is there only gonna be
one DevOps person in the company?
But to me, this is the job, like
this is the new job, is you all built
this, now we've got to implement it.
Now we've got to understand it, we
got to understand the components.
Where do the agents help?
where do they not help?
where do we still need the human
in the loop, like you said?
and I just wondered if that's, if any
of that is anything that you would
agree with or that I'm… Am I crazy?
Does this all sound like
a good idea to talk about?
Lakmal: No, I think, this is great, Bret.
I think you're, you are correct.
So yeah, it's evolved last one year,
within the one year, we can run agent in
production, like helping to the different
personas engaging within the platform.
I believe the generating code problem
is almost solved now with the Claude
Code or other Cursor or Codex, right?
So people can generate code, write
application within minutes So we
call it as vibe coding, right?
Everyone call it as vibe coding.
When the moment they hit the
deployment, their vibe is ending.
Because now with vibe coded, application
has to be promoted into the production.
But still, the platform doesn't support--
they have to m- go away from their agent.
They have to either create a ticket
or like, go to the self-service
portal, and deploy the application.
But what we have seen, within
the, our, skills and MCP servers,
they can use the same, agent they
are using to develop, their code.
They can say, "Okay," I can say, "I
want to deploy, the, my application
into the development environment.
Just you figure it out how to do it."
So now it's, we call it vibe deployment.
So OpenChoreo try to fix, fill
the gap, vibe deployment part,
how the operations side of it.
So you can write, agent, you
can write your code using agent.
Now you use, you can use the same agent
to deploy your application into the
production and operation, with support
of OpenChoreo, section as well as the
tools supporting to the MCP servers.
that's, where we are, we want
to play, around this, platform
engineering side of it.
Sameera: Yeah.
And Bret, I think you mentioned,
some-something about how
abstractions help agents, right?
I think the way I think about that
is more abstractions, fewer tokens,
Bret: Hmm.
Sameera: so they don't have to learn
Kubernetes deep YAMLs and kubectl.
like, I think the same applies to
programming languages in a way,
writing assembly versus writing
in Java or C# or Go, right?
fewer tokens in that way.
So they, they have more
context with less tokens.
I think that is also, that's what
I'm, working on these days, see
whether we can establish that.
Bret: Yeah, that's a, I think we're all
very quickly, especially these last three
months when it comes to tokens, we're very
quickly understanding that this is finite,
and that, just kind of like Kubernetes
itself, we've all had that moment where,
early in our Kubernetes journey, we
would realize how much infrastructure
we actually need to just run the apps.
Like we, you know, the, when you
build out a full-scale Kubernetes
cluster, th- there's a s- there's a
non-insignificant amount of infrastructure
to manage the infrastructure.
And that, back when we first started
the journey on Kubernetes, I don't
think we all really understood, the
eventuality of dozens of different
controllers and dedicated nodes in the
control plane that we were gonna have
to have just to organize and Uh, to,
to herd the cats, as, they might say.
And, I feel like now the
same thing is with tokens.
It's like we got all excited, and we're
like, "We're gonna put it everywhere."
And now we're gonna very quickly end
up in a world where we've got budgets,
and now we've got to optimize our
agents, and we have to care about the
model, and we have to, we have to…
One of the conversations we're having
in the Agentic DevOps Guild is around
evals and how can we use evals, to
sort of figure out which model, if
I can apply a bunch of skills, if my
agents are really just behaving with a
bunch of skills and context management
stuff, how can I, sort of evaluate or
programmatically determine or test that
maybe I could use Kimi or, MiniMax or
Qwen or some, sort of open weight model
that's super cheap for my very specific
little, you know, FinOps niche agent.
Whereas it maybe doesn't need Opus
because maybe that's the one that
I need because I have dumb prompts
and I s- I type dumb things.
And maybe my agent in the cluster can
be running really cheap because none
of us wanna put these in production and
then find out a month later that it's
too expensive to run, and now I have
to shut it down and make the humans
do it again because I don't think
that's a world I, I wanna go back to.
Lakmal: That's true
Bret: I know a couple of companies
that are playing in this space.
We've had Anyshift on the show,
we've had, Mendral on the show,
and these are both AI startups that
are building agents to help with
infrastructure, CI/CD, sort of helping,
helping you with fault remediation.
and the pro- approach that I've
been seeing everybody take is,
like, it's just sort of the normal
pattern of engineering approach.
We walk before we run, you know, we r-
we run before we sprint, and that means
on day one, you might be read-only.
agents can't do things, they can
just go look up things, and that
they're just a little helper
assistant, and they're not gonna…
They have no control over anything.
They're just here to help you
find information in the sea of,
of infinite information that we
have about our infrastructure.
And then, then, like, with Mendral,
the team with Mendral that's been on
the show, they're now to the point
where, very specific use cases, they're
letting the agent have a tiny bit
of control because it's predictable.
They can test and make sure that
it's, that it works, that it-
it's as declarative as possible
so that it is predictable, and
they start to, to let that happen.
is that something we
can do with OpenChoreo?
Like, does this have sort of a default
stance of, "I'm only gonna be read-only"?
how does, how do the
permissions work that way?
Lakmal: Yeah, it's not only the read-only
at the moment, but you can control it.
we have the permission model
where, you can give the, different
tool to the UI, your agents.
It can be read-only tools, it can
be write operational tools as well.
Based on the, what we want to do, you
can allow this permission, because agent
have their own identity, and they, within
the identity they have the permission.
So based on the permission,
they can use different tools.
if you are confident enough that your
agent is acting, performing well, you
can give the right permission to some
s- some level of factory, for example.
you can give like, okay, if there are
configuration issue, agent can go and
fix it itself, the configuration, fixes.
So you can fine-tune how you, your
agent can interact with your platform
Bret: Yeah.
I've been wondering about a lot
of this around, like, how is per-
how are permissions gonna work?
so much of our permissions models today
feel like they're not designed for an
agentic world, and, how are we gonna
track the actions of these things?
Like, every little step they take,
is hopefully gonna be put into some
sort of observability platform.
So I'm, I am very curious to see
how, what your all's vision is.
Okay.
Yeah, I'm trying to envision, what ends
up… Like, how to correlate this to
the things we know today, like GitOps
repos with Argo CD definition files
in them that, that get, you know,
pulled in because Argo's watching
or Flux is watching a different repo
and it's looking for YAML changes.
And, I'm assuming, does
this change that workflow?
Does… I remember seeing GitOps on
the diagram, so I'm kinda thinking,
like, how does this correlate to
sort of that, that GitOps loop that
we've seen so many teams adopt?
Sameera: Yeah, I think
these, the CRDs, right?
so they, you can put all of these in
your, in your GitHub repo, and then you
can configure Argo CD, configure Flux.
the normal workflow will
work as it is, right?
Bret: Okay
Sameera: And so there
are two modes, right?
Here, I think we are using more ClickOps,
ClickOps approach, but you can configure
the same thing with, your GitOps.
Bret: Oh,
Sameera: will talk to… Yeah
Bret: I see.
S- so yeah, so in this case, what
we're saying is like, I mean, yeah,
we're looking at a web dashboard,
but we're not really… We're j-
I mean, for the audio listeners,
we're, we're looking at a dashboard.
We're not really, we're
vibe opsing, I guess.
Not ClickOpsing, 'cause
we're not clicking on the
Lakmal: Yeah
Bret: dashboard.
we're using the agent, but the
agent is causing… Just to be
clear, this is actually a question.
So the agent is causing OpenChoreo
to create new resources inside the
cluster for deploying this inf-
this, the Google, microservice demo.
And I, that is an alternative to having an
agent that I just say, "Hey, we're gonna
make a new PR in YAML, and you're gonna
push that to GitHub, and then Argo or Flux
is gonna pick that up later." because I
guess I could, I could tell these agents,
or I could give them skills or change
the skills somehow so that it knows that
that's my workflow, and that I'm just
using it as sort of a read-only partner.
But kind of just like we're not
supposed to go into the AWS console
and ClickOps away there, like, I don't
necessarily want OpenChoreo changing my
infrastructure outside of my GitOps loop.
I'm actually just curious, do you
think that the GitOps approach is less
important if we have this sort of chain
of events and… or do you, like, do
you see enterprise teams possibly mo-
shifting to more of this vibe approach
since we have something in the middle?
Or do you think that, you think
that GitOps is, like, here to
stay and that's still the s- sort
of more mature, safe approach?
Lakmal: Yeah.
I would say, when you come to the
lower environment, this a-agentic
directly creating this re-custom
resources will play a big role.
because, but maybe in, in production, when
you promote in the production, people will
use, GitOps or declarative way of defining
these things, in-into the production.
but eventually if the human will
confident enough with your agents
a-and, trust your, agent, eventually
I would say, people agent can directly
call to the MCPs and create the,
like, the create custom resources
directly within the class itself.
Bret: Yeah
Sameera: way I think, exactly.
Lakmal: in the this year,
but maybe in future.
Sameera: I mean, one, one adv-
the way I think about that is, I
don't know, one advantage of GitOps
is you have the tracing, right?
you know what, what happened
exactly over the years.
Probably, yeah, dev environment
probably you don't need GitOps,
like you can give developers full
capability, do whatever you want.
But for other environments, I don't know,
my view, like you would be… I would
still use GitOps to control the workflow.
Yeah
Bret: Yeah, I've started to wonder if,
if the agents will eventually sort of
like- We're interacting with the local
agent on our Claude Code, and we're using
these skills and, I think everyone's
already writing the YAML with agents.
Like, we're already at the point
where teams that have u- are using
Claude Code, they're not handwriting
Kubernetes YAML anymore, right?
they have the agent do that.
Why would, why would I
wanna do that anymore?
And, I wondered if there was m- 'cause
there is a mode in Argo where, like,
you can sorta do things retroactively,
where maybe you make changes,
but then you document them after
you're, after you're, you're done.
And I wondered if agents possibly created
a future where GitOps was more re- m-maybe
not the first thing we did, but maybe
more of a, a system of record, but done
after and if agents sort of were pushing
us into a faster evolution of this.
a constant conversation we're all
having is around sandboxing and,
specifically, our local agents.
one of the challenges is, I'm just
doing stuff on my local machine all
day, and I don't necessarily… And
every AI that I open up by default, even
if I use, like, Claude Code built-in
sandboxing, chances are it has access
to my AWS, my, my Docker CLI, my kubectl
CLI, my Terraform, all these things,
and those keys are all already authed,
and it could just deploy to things.
And so one of the, one of the things
that I'm trying to work on with
some friends is, where could we
draw a boundary around sandboxes?
And this almost feels like a very
perfect scenario of these MCP tools and
these skills are maybe all in a Docker
sandbox, which is more of like a VM with
a, with the harness running inside it.
And I only use that particular harness
because it has these particular,
permissions, and I only spin that
one up when I want to… It's almost
like per environment, configuration,
and each Docker sandbox tends
to have its own configuration.
the, it's… The way at least it works now
is it's very, things are very isolated,
including all your keys and everything.
It, it doesn't really bring a lot
into it, so you're sort of isolating
everything into different harnesses.
So I'd have, like, my prod harness, my,
my staging harness, and I can interact
with those environments, and keep them
safe away from, like, my normal day-to-day
harness that kind of has maybe too
many… a lot of small teams, too many
keys are on their local machines, right?
they know it.
They're out there.
if you're a team of three, chances are you
probably got the production Terraform key
on your machine, and you're just like, you
know not to type the Terraform commands.
you know that's the human thing.
But now that we have the agents, I
think people are getting, starting
to get a little more concerned.
They're like: What if it accidentally
picks the wrong environment?
Or if it-- I accidentally have the
wrong key and it just starts going?
so this feels like a very, very
interesting approach of Each environment,
or, I'm assuming this thing manages,
can manage multiple clusters in one
OpenChoreo, or is it a per cluster thing?
Lakmal: It's one multiple clusters
you can manage within OpenChoreo.
Bret: Yeah.
so yeah, so I, I almost treat it like an
environmental gateway t- for my agents
where I'm like, I call this production and
I have this special Docker sandbox or what
other sandbox I wanna apply, and the keys
to production, even if it's read-only,
are only accessible from that thing.
And maybe also I'm gonna have some GitHub
keys in there so that my agents i- in that
sandbox can also write to the GitOps repo.
Maybe they can make the PRs for me.
but I'm now realizing how these things
are coming together, where I can have
the one local agent that can see the
infrastructure through OpenChoreo,
but can also implement GitOps changes
based on the information that it's
getting out of the infrastructure.
So if the pods and, you know, a
pull back off loop, I can see that
the agent can determine that, write
me the GitOps in the same session.
It writes the GitOps b- diff.
We're pushing that to a PR, and that's how
I'm gonna be iterating on my cluster now.
Or maybe I'm doing it in Slack while I'm
at lunch, and it's all just through a
Slack bot, and I'm not even… I'll ask
you these questions to make sure that we
have them, have them at top of mind here.
"I like it's Claude Code that
can interpret or understand,
promote into staging. Can we use
a private LLM to interact with
OpenChoreo instead of Claude Code?"
Lakmal: yeah, so it's, it's
totally, in Claude Code, it's,
it's your, your model, right?
So it's a, Anthropic model.
You can use any Codex or any other agent.
It's just MCP servers
you are interacting with.
You can use any LLM, in your local agent.
And also in the inbuilt agent
also, you can configure your LLM.
you can either OpenAI or whatever
model you can configure it.
it's a configuration, for us
Bret: Right.
Yeah, picking the model for the agents,
is, is a good thing 'cause we're gonna,
we're, we're-- I got a feeling we're gonna
need to get cheaper models for the, for
the cluster, for the always-on activities.
yeah.
and then, does the agent have visibility
into inter-cell dependency traffic, or
is it context window completely blinded
to anything outside its assigned project?
Lakmal: i-it's depending on the different
agent interact with, different contexts.
So for example, say, in this case, SRE
agent, when that troubleshooting, it
can, go beyond the one project, because
some component will interact with the
other component within other project.
So, so if there are interaction,
this agent can look at, what
happening in the other, other
component, in the other project.
So it's b-based on the context that agent
are interacting with, they can go beyond
the single project or multiple project
Bret: Right.
Okay.
And just to be clear on, the scope of
this thing, it doesn't appear like you're
trying to boil the ocean here, where
OpenChoreo is the single control plane
for AWS, for GCP, for all the other
things that we might have to manage.
it, it's trying to remain
Kubernetes focused, right?
So, like, the lowest layer in our
infrastructure stack is, I guess m-
maybe it could possibly get some of the
technically OS level stuff underneath
Kubernetes, but that's, that feels like
that's the s- the dropping off point,
'cause I don't see, clouds listed in here.
Lakmal: Yeah.
we have, we are running
on top of Kubernetes.
We are not even managing the
Kubernetes with OpenChoreo.
we, or we build on top of the
Kubernetes, so you can run
OpenChoreo on E- EKS or EKS, but
we are not managing the EKS or EKS.
So that how we architect.
But, OpenChoreo has this resource
abstraction where you can manage the
cloud resources, within the OpenChoreo.
So we have integrated with,
Crossplane at the moment.
So OpenChoreo control plane can in-
talk with the Crossplane integration
and via the Crossplane, it can create
a, say, S3 bucket in AWS, and it can,
do the lifecycle manager for S3 bucket.
So it provide a single, unified
control plane to manage it, but,
it's up to the users if they want
to, use, manage in different way.
That's totally up to the users
Sameera: Yeah.
Just to add to that and then, Bret
you saw like multiple planes, right?
Control plane, data plane,
observability plane.
And you can run each plane in its
own Kubernetes cluster, or you
can run all planes in one cluster.
And you can run your control plane
locally, you can run data plane in AWS.
So it's, as Lakmal said, it's, the
way it's architected is so it's,
you can configure, the way you want.
And then like, like for example, if the
data plane is in AWS and control plane is
somewhere else, you don't have to expose
the Kubernetes API server to the internet.
There, there's a certain agent running
in the, in the data plane that creates an
outbound connection to the control plane.
that's sort of the standard these days
Bret: Right.
So yeah, basically, if you're gonna use
this to help manage something, it needs
to be something running in Kubernetes,
and it needs to be probably like a s-
something around the CNCF ecosystem
of tooling or something like that.
So this isn't trying to replace
Terraform or, you know, control my
cloud formation, and stuff like that.
Yeah, yeah, or run my, my,
my other infrastructure like
Vercel or something like that.
Yeah.
that gives me a nice boundary because
like when we talk about the future of
what it's m- means to be an SRE, DevOps
platform engineer, whatever, like where
do the edges of these tools exist?
Because, sure, I can manage everything
from Claude Code in theory at this point,
but, it probably, I probably need very
scoped things in order to keep it in line.
And having this thing w- well
defined is like it runs on top of
Kubernetes, so things that you can
do in Kubernetes, it can help with.
But don't-- you maybe can shoehorn
in a bunch of things like, for AWS or
whatever, but maybe it's not the right
tool to do everything all in one place.
'Cause obviously there's a ton of
context outside of Kubernetes that
it would probably need and, yeah.
So there's probably other tools for that.
At the end of the day, we're gonna have
dozens of agents, I feel like, and we're
just gonna, we're gonna have an agent
management plane where we're all just
staring at agent configs and skill files.
And that's like, that's our new job.
More markdown.
So how can people get started?
They go to openChoreo.dev
and or is this a Helm chart?
Like how do we get started
with implementing this?
We, well, step one, you gotta have
Kubernetes some- running somewhere, right?
Sameera: So yeah, if you, if you
go to OpenChoreo dev, documentation
page, there are m- couple of
ways for you to get started.
First thing, if you wanna just try
out OpenChoreo, so we have this quick
start guide that'll basically… The
quick start guide that'll basically…
You just need Docker for that.
So Docker and Kubernetes,
we will do the, we will…
There's one command that'll
install Kubernetes in Docker, and
then that'll install OpenChoreo.
So you get a… Once it is done, you
get a UI, you get MCP access, all that.
That's just 10 minutes.
won't, pollute your local
environment in that way.
You can just destroy
it after that is done.
The other option is you need to
have Kubernetes cluster on your
local machine, install OpenChoreo.
There are, then the third option is
install OpenChoreo in cloud environment.
That's so, that's sort of the way how
we have structured getting started.
Bret: Okay, so it's got
some persistence to it.
I'm assuming this thing, does
this thing have, like, databases
as a part of its deployment?
Does it have, like, a Redis or,
like, what-- When we talk about the
infrastructure inside of OpenChoreo,
I mean, we've got a web portal, we've
got some, obviously some MCP endpoints,
we've got some system components based
on other things you're installing.
what's that persistence layer look
like for us that are gonna have
to deploy this on our clusters?
Sameera: for the, I think for the whole
OpenChoreo control plan, I would say
right now the main persistent layer
is ETCD, whatever the Kubernetes use.
For Backstage, you can configure,
Postgres or any other database
that Backstage, recommends.
Right now it uses… Yeah.
Yeah.
I think apart from that,
there's no persistent,
Lakmal: Also, for a, like a, the
SRE agent, you can configure, it's
coming with SQLite, for the saving
the incident and providing the
context in the future incidents.
But you can configure a Post- a
Postgres or any other database.
Bret: Yeah.
I was actually wondering too, I was
trying to imagine, this thing is doing
things, these agents are doing things,
and we probably want a history of that.
Is this thing able to, log into
your existing… Like, is it logging
through Kubernetes to your existing…
So it's essentially creating more
m- monitoring and logging data in
your infrastructure by the act of it
just being there and doing things.
Is that, is that where I would
go to see, hey, what does my
agent do in the last 24 hours?
Is it something like I would
just look at my normal monitoring
and logging platform for that?
Lakmal: Yes.
it will do all the audit logs, to the, log
itself like we are using, depending on the
lo- log module, it can be, OpenSearch with
a different index with, for the, audit
logs, what agent do, whether human do.
Everything is tracked,
within the logs itself
Bret: Okay.
and all the data, about the
infrastructure, are the agents
just pulling that real time?
Or do we, do you have to, deal
with caching, you know, infrastruc-
infrastructure graphs or any… Do
you have to do anything like that and
to optimize in the OpenChoreo layer?
Lakmal: Yeah.
At the moment, it's a real time.
we are not using any, VectorDB
or something like that, for the
caching or something like that.
At the moment, it's, real time
pulling the data, but we give the
time interval, the context saying,
"Okay, within this time interval, you
have to pull the logs, matrices." it
will just look in that interval and,
get the, all the data to troubleshoot
Bret: Right.
So it's, so it, once it sees an event,
it already knows exactly the second
that event happened, so then it can
limit itself to th- so that you're not
burning a bunch of tokens by boiling
the ocean and scanning every log on
Lakmal: Yeah.
Bret: infinity.
Okay.
Lakmal: the idea
Bret: Yeah.
Perfect.
is it, does it need to look
at, like, Grafana graphs?
Like, or is it just simply doing
prom queries and just getting
that data real time as well?
Lakmal: It's the data to the,
observability MCP server.
It's, just going to the MC- MCP servers
Bret: Yeah.
RIP Grafana.
Sameera: Yeah.
Bret: Yeah.
Oh, sorry, what would you say?
I was talking over top of you
Sameera: No, this is directly from,
observability MPV service talk to
the OpenSearch and get the data.
Bret: Yeah.
Yeah, yeah.
Yeah, so that it's a little
bit simpler than it having to
constantly, figure out everything
and store it as some sort of cache.
Which, I mean, in theory, that would
make the agent faster so that it didn't
have to query everything itself when you
start to work with it, but it also adds
a lot of complexity and, and resource
utilization, which gets back to the
point of at some point, everybody has
had that infrastructure where they
realize they, they have a, a cluster
that has more resources used for the
infrastructure than the actual apps
themselves, which, you know, has happened.
sometimes when we have small clusters
and we're getting started, like we are
like, "Yeah, you know, the, the app
only costs $100 a month to run, but
the infrastructure costs 10,000 a day."
Lakmal: That's true.
Bret: yeah.
Well, this has been very cool.
we could honestly, I could talk to you
for another hour about this because
I'm really, I'm really interested in
the patterns and the, and the ar- the
architecture design that you've learned,
because I think these, these are things
that we're gonna have to do the same
thing for our cloud infrastructure.
We're gonna have to do the same
thing for our line of business apps.
We're gonna have to s- figure out how
to give these agents under- a deeper
understanding of that infrastructure
so that we can depend on them more and
rely less on the human, the human tribal
knowledge that we all have about how
these systems are put together and where
all the bodies are buried and where all
the, My, the friends, my friends of mine,
they call it the sins of the data center.
Where are the, where are the
sins of the data center at?
And this thing is gonna
have to know all that.
this thing is really, when I say that,
I mean like dozens of agents that all
have certain context and certain control.
but it all kind of feels like it's coming
back to this still, this, Viktor Farcic,
who runs the DevOps Toolkit channel,
a friend of mine, he talks about that,
like to me, to him, Claude Code is the
center of the universe and he manages
everything through it, and that anybody
who's creating a project that isn't
expected to be local harness first as my
interface is creating an outdated project.
So this kind of feels like it's, this,
is doing exactly what his, he was
predicting over the last year, which is,
you know, the harness is my gateway to
all this infrastructure, and everything
else needs to be there to help me
be like local agent harness first.
and it's pretty cool to see
that, to see that take place.
all right, so people get started
at the openChoreo.dev website, and
then I think you're on, LinkedIn.
I'm trying to think what the
project is on LinkedIn and X.
Lakmal: Yeah.
We have the X and LinkedIn
pages, and also we have the Slack
channel, CNCF Slack channel.
we were actively, engaging with there
Bret: Yes,
Sameera: And we use
GitHub discussions a lot.
If you want to engage
with us, we'll be there.
Bret: I love to, I love
GitHub discussions.
I d- wish they were used
more, r- rather than issues.
Like, I feel like a lot of things just
need to start in discussions before they
go to issues, and maybe some projects
do that now, where they not, they don't
even let you post things in issues.
You have to, you have to
start the discussions first.
yay, AI ruining open source for us.
at least for now.
We're figuring it out.
thank you both so much for being here.
you can follow the project, like s- we
said, on LinkedIn or on X or, join the
Slack and the… Is it the, is it the
Kubernetes Slack or the CNCF Slack?
Lakmal: CSCF lab
Bret: CNCF
Sameera: Slack OpenChoreo
channel, I would say.
Bret: Yeah.
Okay.
I'm in both, and I can never remember
which projects are in which and,
or if the Kubernetes… Yeah.
So it's in the CNCF Slack.
You can, probably just
Google that and find that.
tell your agent to sign you up.
Like, don't, you know,
don't go do it yourself.
thank you so much, Lakmal and Sameera,
for being here, I'm looking forward
to seeing what's next in the project.
last quick thing.
What's next?
What's, what big PRs are coming down the
pike since we're talking open source here?
Lakmal: More agents?
Bret: More agents.
Could-- I could have guessed it.
I could have guessed it.
Sameera: All about AI.
Bret: Yeah.
More, more of the AI overlords.
All right.
Thank you so much for being here.
thanks for everybody in chat
for being with us, and we'll
see you in the next one.
Bye, everybody.
Thanks so much for watching.
I'll see you in the next one.
Creators and Guests
