DevOps and Docker Talk: Cloud Native Interviews and Tooling

Bret Fisher: You're listening to DevOps and Docker talk.

And I'm your host, Bret Fisher.

These are edited audio only versions of my YouTube live
show that you can join every week on thursdays at bret.live.

This podcast is sponsored by my Patreon supporters.

I'd like to thank all of you paid supporters that made this show such a pleasure to do.

You can get more info and follow my updates for free at patreon.com/bretfisher.

This week, my guest is friend and Docker, captain Nirmal Mehta,
and we dive deep into using get Hub's newest beta product copilot.

I love this episode because we had so much fun trying
to misuse a machine, learning AI in visual studio code.

I've edited the recording down from a full demo video that we did on YouTube.

The link is in the show notes, but I left a lot of that in here because I felt even
in an audio format that describing how it's working real time is worth the listen.

So we go through long theories of where this AI is going, how it might be misused and what
we think the future holds for having a robot, complete your sentences and code all day long.

Now on what the show.

So I'm excited to have on the show today, a friend of mine that
has been here several times before Docker Captain Nirmal Mehta.

Hello!

Nirmal Mehta: Hello, Bret.

So we met years ago, Docker Captain program, those of you might've seen
him on the show before we talk together, the last two DockerCon's, right?

Yeah.

Yeah.

I miss seeing you in person, Bret, got to, hopefully 2022 is when we can see each other again.

Bret Fisher: What about your local Dev Ops Days?

Nirmal Mehta: DevOpsDays RDU.

You April 20, 22 at the McKinnon center, check it out on www.devopsdays.org.

There's likely another DevOpsDays conference near you.

If you're international there a ton of them around the world, they're all run by volunteers.

And they're a nonprofit kind of organization.

What are we exploring today?

Bret Fisher: yeah, we're going to talk about and demo and try to maybe not try to break.

I don't know if we can break it or not, but try to get some funny stuff out of GitHub Copilot
and I got, I was lucky enough to get in the beta a little over a week ago and a half ago.

I don't that beta technical preview, no one does betas anymore.

It's either preview or technical preview or early access

and then we, I mentioned this Copilot thing and I was using it with
Terraform, with Docker, Kubernetes, all these things that are DevOps
related, not specifically programming a lot of YAML and HCL stuff like that.

And I was finding it mostly useful and I put up a tweet, which I don't have, but it got
a little traction of people saying I didn't even think that you, that would be a benefit
of this, that it would, it was more of a code thing, but the reality is this thing will.

It'll recommend anything it'll finish any line, whether it's
a markdown file or no matter what language you're in it.

So anyway, for those of you that don't know it's exactly that your AI pair programmer probe.

Nirmal Mehta: How do you get it started with it?

Bret Fisher: So great question.

I started by clicking the sign up, you get on the
waiting list and you wait, I don't know how long I wait.

I can remember when I signed up, but I waited a while at least a week or two.

And then you get access to a repo of documentation, and then there is this.

Copilot thing that runs as a service.

So this is technically a service we're all using on the internet.

I think it's, they call it the OpenAI Codex, which
sounds something straight out of Matrix or Terminator.

I don't know what it is, but and then the only editor that you can use it in is VS Code.

So well, I'm sure that eventually others could probably build
into other IDEs, but the only one that works in today is VS Code.

And you can use it with your local VS Code or Codespaces, which I've talked about here before,
but it's an extension that you have add in that basically provides access from their AI service
to write in your screen or make suggestions and you ha you, and then you just start writing.

And if you, if I jump over and just show you.

It's this tiny little button.

I don't know if people can see it on the screen with this tiny little button down in the bottom.

And it started.

And once the, once you've added the extension just called copilot.

And of course, if you're not in the beta, sorry, technical preview
you won't, even though you can add the extension, it won't work.

It'll say you don't have access yet.

I tried to do that and it says you don't have access.

But once it's there, it, I think it starts deactivated.

So it'll look red.

Let me go back to an actual, wow.

I'm just going to create a simple file to see if at all.

Okay.

Nirmal Mehta: so maybe while you're doing that.

So fundamentally Copilot is a code completion suggestion tool.

That's been trained using machine learning off of a Corpus of all
public or open source or whatever GitHub repositories are out there.

Bret Fisher: Yeah.

So that's some great distinctions about, so there's all
these great questions on their FAQ which you can just use it.

You can see at copilot.Github.com, but yeah it's only on public source code they're there.

Their ML input is just open source code billions of lines as well as human language.

So it can complete human sentences as well as code sentences.

If that's a thing code statements it is not a search engine is a code synthesizer.

So only about 0.1, they say about 0.1% of the time.

It will give you the exact input.

Or output rather that it's got as an input.

So it is not just recreating the exact functions that it copied from open source code.

So a lot of people are like, there's a there's comments about, we're
not gonna get into the license, the licensing and legality thing of it.

People have said we're you're stealing other people's open source.

No, because it's not pasting in open source.

It's synthesizing based on actual, intelligent learning that it did.

And it gives you options.

You can use keyboard shortcuts to rotate through it's suggestions
because it's basically making these things up on the fly.

Nirmal Mehta: I just want to make sure we're very clear because obviously this
is a very new thing that tense was like, less than a month old at this point.

Or and we don't know what folks are used to that are on watching us right now,
but it is not, it's not the equivalent of searching on stack overflow for a coat.

And then copying and pasting that in based on what you were typing at the cursor.

It's not that, but it's that in the sense that it's generating.

What if thinks is the likely completion for what you're trying to type?

What you have typed already, correct?

Yeah.

Bret Fisher: Yeah.

And that's the thing is it's yeah, it's not uploading your code and analyzing it.

There's no, it's a one way Thing coming down.

It's not uploading your private code.

It's not learning on your project or other code.

It only is in the file you're in.

So the only file it can see and analyze for context, because
there's an important point we'll get into about context.

How does it know what to do?

It's all about context.

And it will only complete a line or a function.

It won't write a whole file.

It won't write a whole class.

It won't generate from scratch the entire manifest Kubernetes.

Mazar and chat says let's work on Kubernetes.

It'll work on any file that I so far I've given it a bunch of different files, Terraform, Docker,
Kubernetes, markdown there's lots of other channels out there that are showing coding languages.

So go node, so JavaScript and all those work.

So my theory is if it's an open source file in some
sort of code repo, it will help you in that file.

So yeah.

It doesn't think of it.

I don't think as this is Terraform, this is Kubernetes.

I don't actually, they don't have the engines, not public, at least that I'm aware of.

So I don't believe that it understands the F it's not it's looking at file type for me.

It's looking at what I'm doing in the file.

And then if I have nothing in the file, it's not very helpful yet,

Nirmal Mehta: So that word understand is very tricky.

When you talk about machine learning, because what
does a machine learning model actually understand?

So to your point it still needs some input.

So the context of the file that you're in, plus whatever
you've typed in that file is provided as input to this model.

And then it returns some kind of generated material with some kind of probability with that.

Bret Fisher: Yup.

Nirmal Mehta: And then the option to paste that in right to
to accept that output from that model into your file directly.

Bret Fisher: Yeah.

So it's always suggest suggesting.

And so if you don't hit tab, you can just ignore it.

And that's the tricky part for me over the last week is I have
all these other extensions that provide templates and stuff.

So it's, sometimes it's not easy to tell wait, was that Copilot
or was that my extension functionality for creating a deployment?

Cause I can do that with Kubernetes files and I can just
generate a template if I hit the right keyboard shortcuts.

But let's do one, let's do a couple and you can ask questions while we go.

Cause I think,

Nirmal Mehta: Yeah.

Just to close the loop on that, what's another IntelliSense, is something else that maybe
some of the folks will be used to using in an IDE and, IntelliSense was, is, pretty awesome.

Or that code completion kind of tooling that you see in
IDs, jet brains or VS Code or visual studio or whatnot.

So this is in that.

This seems an evolution of that kind of tooling, correct.

Or that kind of feature functionality of an IDE.

Bret Fisher: I would say that in some ways it's better than IntelliSense in some ways it's worse.

So that little red means it's not enabled.

I'm going to enable the globally, but you can enable it for specific file types or whatever.

And I'm in a Docker file.

So notice that the way this works is it's gray.

So if you see gray tech show up in front of my cursor, that is copilot.

If there's anything else you see, popups or squigglies, none of that's copilot,
that's all other things spellcheckers or things that are actually in language
specific intelligent reminders, because that's all a part of VS Code, right?

There's all this other stuff built in syntax things that, and copilot is not though.

So it's only going to show gray text in front of my Kerscher.

So if I just start typing, so notice what I maybe could do here is say, start from.

A boon to oops.

Nirmal Mehta: Okay.

Bret Fisher: I'm to,

Nirmal Mehta: It was already starting to, oh, there we go.

Bret Fisher: so look at that.

So what's interesting is I notice a lot of times when there's version
based things, it's old versions, it's never the latest version.

So is providing that line right,

Nirmal Mehta: Probably because older versions are statistically more

Bret Fisher: right.

More there's more of them, right?

Yeah, exactly.

It's so it doesn't have a concept of oh, that's an old version.

I might want to pick the newest version.

Nirmal Mehta: So it doesn't understand that.

Bret Fisher: Yeah.

I keep saying that word.

I don't know what that really means.

Yeah, there, isn't some sort of logic gate in there that says if it's
a number and it seems to be many numbers, pick the newest number.

Nirmal Mehta: So I want to, and this might not be true.

It might not be true now.

And it might not be true in the future, but it seems coprolite does not have a semantic
understanding of that languages structure, the semantics of a Docker file or any kind of language.

Bret Fisher: You would think so, but we're good.

I'm going to show some examples of a question

Nirmal Mehta: okay.

That's it?

Yeah.

There you go.

This

Bret Fisher: Yeah.

I don't know.

I don't know.

Nirmal Mehta: This is where the fuzzy line is.

With respect to all of this.

Go ahead.

Bret Fisher: I think that one of the big questions I wanted
to answer over the last week was will this two things.

One is this replace people and in no way, no and it doesn't make a junior person smarter.

Not really because the thing you have to do with every one of these examples is you
have to know whether what is suggesting will work or just try it and see if it'll work.

Because if I don't know what a Bluetooth version is that the latest one, it's not telling me
that now I can hit option bracket back and forth and it's not giving me any other suggestions.

So it's only suggesting that thing so I can type 2004 because I know better.

And then I can say, okay, so now I'm going to copy.

Now, here's the thing.

It only starts suggesting on either you adding a comment to give it directions.

So that's the context.

It needs to know what you want next, or you start align.

And it specifically says function, name, and function options in, and their documentation.

But again, we're not using it for programming here.

I might just say copy in source code and see.

Okay.

Yeah.

So because earlier I was typing in copy space and waiting for it to suggest what I want to put in.

But again, this is wrong because I don't have a code directory or I guess I could, I guess it
could be an opinionated way to say, yeah, I'm going to put the cop, the code in a code directory.

Maybe it's a problem of my kind of source.

Nirmal Mehta: you wrote copy

Bret Fisher: All

Nirmal Mehta: and where you were waiting, it was too broad.

Bret Fisher: Yeah.

Nirmal Mehta: There we go.

Bret Fisher: Yeah.

So if I give it better direction, it knows to have better output.

Nirmal Mehta: which in a way makes you, it forces you to make good comments.

Bret Fisher: Honestly, part of me thinks that this might actually yield
as better documentation and code in line, because if you're lazy and you
just don't want to have to top copy of the line or type the line in yeah.

How good at documentation do we have to create so that the AI can then generate my code for me?

So really I'm just a great describer of what I want.

Nirmal Mehta: I'm going to send us into the void, but I wonder if this
was hooked into a JIRA ticket, if it could just read the user story and

Bret Fisher: Yeah.

Nirmal Mehta: paste in the code.

Bret Fisher: So see it's yeah, every time if I said bill scrip at
Atlanta, it runs bill.sh if I say run NPM build, it knows in PM install.

Nirmal Mehta: But this is for a Docker file, which isn't.

Isn't necessarily the target functionality

Let me say this properly.

I think copilot was initially created mostly just for actual code, right?

Versus you're using it for infrastructure DevOps kind of stuff,
which it can do, it seems, but it might not be the target kind of

Bret Fisher: in my, exactly.

Yeah.

We may not be the exact, but the thing is they say that
it processes natural language as well as code files.

And to me what's the difference between, to me, I don't know.

I wonder if to a machine, the difference being a Docker file and a JavaScript file is that
JavaScript has functions and this doesn't, but other than that, it's still suggesting logic.

I don't know.

I, but I've found it very helpful, but it's also incredibly, there are bad examples.

I'd say 50% of the lines I have to edit that it gives me, and it's only an in my world.

It's only giving me one line at a time.

If I'm doing Yammel it's one line at a time.

So if we do

Nirmal Mehta: For if you're doing code though, it's it can like, I've seen
demos of it where it's giving you full, functioning blocks in code, correct?

Bret Fisher: Yes.

Yeah.

So in code, it will give you'd write the function name and then the function
options, and it will start and then it will give you an entire function to do that.

And they show that on their website.

Actually, we just go back to their little demo here,
which by the way, I realized that these are interactive.

Nirmal Mehta: That's some that's.

Mind-blowingly cool.

Bret Fisher: super cool graphics on a website, but yeah.

So if you can see them and you can say replay it, will you write a
function name with some and then it, yeah, that's what it gives you.

Nirmal Mehta: So it does understand the semantics of the
variables that you were using to describe certain things.

So it's in-between temple thing until, since an back
to with the training from a bunch of different repos.

Bret Fisher: You could argue that the value is continually declining
as I'm basically writing the exact thing in longer format in a comment.

I probably should just write the line.

So far for me, it's questionable value in a Docker file, but that's honestly,
because I live in Docker files all day and I know everything about them.

So it's not showing me anything I don't already know right.

Where it comes into valuable.

And to me, it became more valuable when I was doing Terraform this last week.

And I was using specific Terraform providers that I didn't have the
ma I didn't have all of the different values and options memorized.

And it.

And at least 50% of the cases it did.

So I was looking at documentation less because it was suggesting
something that I was like, oh yeah, that's the thing I want.

Yeah.

And we can do that demo next.

If anybody wants that one, or we can look at a deployment.

Cause that, one's also interesting, but

Nirmal Mehta: Let's do deployment and then let's do the Terraform.

Bret Fisher: So create a Kubernetes deployment of engine

Nirmal Mehta: deployment.

Let's do Terraform.

We can try to do a live share.

How about that after that?

Bret Fisher: Okay.

So here's an interesting thing.

The very first line is wrong so what I need to do is what I've found
with Kubernetes Jamel is because there's no other context in the file.

I have to start seeding it a little bit at the beginning.

So I have to say, I'm let me give that

Nirmal Mehta: So a comment, we'll cut it.

Bret Fisher: Write a comment on, cause I, again, I'm not even sure if it's reading the
file name or file type because I'm not actually sure that my see, this just knows yam.

It doesn't know that it's Kubernetes Yammel yet.

So it could be part of the problem of the API version.

Nirmal Mehta: And most people aren't putting a comment at the
top of their coronase emo saying this is a Kubernetes anvil

Bret Fisher: They are.

Which they will.

You totally

Nirmal Mehta: they had done.

Yeah.

But if they had this, might've picked that up already.

Bret Fisher: right now.

Notice that it is providing me a Kubernetes extension here.

It's the wrong one for a deployment, but apps, the one and then, okay, so there we go.

Now it's got me a right.

A correct line.

That is the correct line for deployment.

And then, okay.

That one is correct.

So all I'm doing is hitting tab and then enter and now, okay.

I'll call it engine X deployment.

That's because that's the correct format now.

Oh, wait shit.

Okay.

So notice this pop-up, which I've actually never seen it, do this before.

Nirmal Mehta: Open copilot.

Bret Fisher: Okay.

So the next of the previous are the keyboard shortcuts I've been using for
showing me the examples, except is the tab that I've been hitting open.

Copilot gives you this other window that will give you up to 10 solutions for that thing.

Nirmal Mehta: Wow.

Bret Fisher: So if I it's essentially what if I accepted every line of gave me?

So it goes through the spec pretty,

Nirmal Mehta: bubbles?

I think it's

Bret Fisher: oh, sorry.

Yeah.

Let me, I can just move that over here.

So this is the copilot suggestion, so it's giving me 10 of 10 suggestions.

Let me in there fill out the whole thing at once.

So I have to go line by line, but this is essentially what it would be giving me.

What I also noticed again, was versions were really out of date.

I think the first version it gave me when I first tried this, it was 1.9.

So it's a really old version of engine X, but then it's
also giving me a service, which is not what I asked for.

But also you would normally need, so what it's actually giving
me here is these dashes are the breaker between the solutions.

So a complete solution.

Nirmal Mehta: Yeah.

Machine learning models typically work on how close they are to
whatever survival or, fitness goal that they gave it right to.

Bret Fisher: And you're already exceeding my understanding of these

Nirmal Mehta: Yeah.

So In order to train it, they use some of the data and then they probably compared it to something.

And then the likelihood that it's close to, whatever that they
were training it off of is probably the order that this is in.

Bret Fisher: yeah.

Nirmal Mehta: But tried and true garbage in garbage out.

So if most of the Kubernetes deployment Yammel on GitHub, open source repos isn't written right.

Or has errors in it, then it will likely give you that error back.

Okay.

Bret Fisher: So it's I don't know.

I feel we're at the child development stages of this AI, right?

Nirmal Mehta: The thing like, this is just a, for lack of a better term brew machine learning model.

If this was used as a base and then there was some
IntelliSense or semantic, more classic kind of rules, engine

Bret Fisher: yep.

Nirmal Mehta: intelligence based then it becomes even more powerful.

If it was fed through there and then fed through some
constraints, then it could get really much more useful.

Bret Fisher: Of course, this could be total jibberish again, only about 50% of the time.

Is it giving me what I really wanted?

And that's on a per line estimate there and their official documentation.

They say that 45% of the time.

It's correct.

The first time.

Nirmal Mehta: Yeah.

Bret Fisher: So the project, this weekday, I took some screenshots from, I just want to show
some screenshots real quick and then we can dive into some questions that people have them.

I've been trying to keep up, but over here this was real-world stuff I was doing this week.

So this is a Terraform file outputs.

So this is me.

It's almost boiler plate stuff.

That's the inputs and outputs of Terraform.

A lot other things.

You're making a values file.

You're making an outputs file and it's all good, very repetitive.

But again this is not able yet to even look at my project or other files, much less the rest of
my own code, whether it's open-source or, other projects I can imagine, in years from now, because
they talked about that this is, and they're doing other things and I can see where you let it opt
in to maybe some sort of Areas is so some sort of secure area that they will analyze your code and

layer that opinion on top of their own generic opinion so that it can look at my other files, my
other repos, and understand the way that maybe I do things, my two, two spaces or four spaces and
my tabs, I can see how eventually it, I have my own actual personal one where this is very generic.

It's only to the file I'm in.

So I'm going to play this little video.

But what I'm doing is I'm just adding simple output variables.

And so I'm typing output, and then I'm just giving it a name.

Cause I have to start with this context of what am I wanting to name?

And so I type in the instance, I think this one's architecture
and then I gave it the curly brackets and then I hit enter.

And then it says correctly, the value from another file that it can't see.

And then it's giving me the correct description for the thing that I want.

And so I basically was able to go through this output file very quickly.

Without having

cause I had just created the other file stuff and I knew the names.

Now, what I believe is doing here is it's looking at the value names that I'm creating so far.

Yeah.

And determining based on that, but it was right way more than 50% of the time just because
I was creating a consistent value, but these are all variables that I'm just pulling in.

And I have a very consistent way of naming them.

And so it was able to fit you're that out and do that.

So that's a great example.

I thought of, I think what I was, I think the tweet I was talking about was about
this thing and I was saying 80% of this stuff that I'm typing in, I'm not typing.

And that was on day one.

That was the first day I was using it last week.

And I didn't know it, I think about it.

I didn't know.

I didn't know how it worked.

I didn't understand it.

I didn't know all the keyboards that's I just turned it on and this is what it was doing.

Nirmal Mehta: I think if there's any folks out there that are working on new open
source tooling or any kind of command line tool or configuration file and working
on the user experience of that or developer experience, I think it will be very
interesting to use copilot to see what the cause because of how it's probably trained.

There's a leaning toward what the majority of the content on the files that it's, that it uses.

Did whether those are right or wrong can be determined by
running through copilot and seeing what the typical errors occur.

And that could be very valuable to adapting your developer
experience for whatever tool that you're creating.

Because if you see that some majority of the repos out there are wrongly are incorrectly
using your Yamhill configuration file or your interface that probably highlights a
bad developer experience or something that goes against intuitive usage of your tool.

Does that make sense?

Bret Fisher: Yeah.

Nirmal Mehta: I'm getting a little bit too deep there, but there's, this opens up a lot of things in

Bret Fisher: I've started to wonder yeah.

Yeah.

And if this is an API that I'm assuming they're going to keep open so that other it's
not just vs code limited, that's an assumption, by the way, I don't actually know.

I can see people writing front ends for your CLI imagine typing Sudu space,
Docker, and then it just starts to try to complete auto complete the line.

Where does it end if the engine is looking at other behavior?

I don't know how it would complete command lines because yeah, because the thing about the command

Nirmal Mehta: right?

Or those.sh files that are just

Bret Fisher: Exactly.

Yeah.

There are those files are commands, right?

So if it can read those, then you would assume that it can help me at the command line,
which gets me to my next example of, I hate red jacks only because I don't understand it.

Obviously it's very powerful and we all need it.

But so in this example, I'm doing, I'm using a Yammel file in a workflow.

Forget actions, which I've talked nonstop about lately.

And what I'm trying to do five is the current branch I'm on inside of my action, which it
doesn't give you an output value that always tells you what current branch on, which is weird.

I find it weird.

So what you have to do is you have to take something
called I'm actually gonna use my screen painter.

So this is the part I'm talking about right here.

This is the line that I'm on and the X, so the get hub dot ref is the value
that they provide me inside of this machine runner that I want to get out.

And then I want to set it to get the part of that value that I want.

I was going to have to look.

I was going to go to a said site and then type in what I
know in practice my set again, the there to get my rejects.

And it was a relatively simple one, but I'm horrible at it.

Without any real, other context, other than maybe me
describing giving it a little bit, a little better.

Thank you.

It correctly got that line on the first try, which I thought was a fairly advanced line.

And the said, which I don't normally use says with hash, thanks.

I normally use a forward slash well, I had to look up that even that was
correct syntax, but it was, and it worked and yeah, so I thought that
was really interesting cause it taught me something I, and I'd see that.

And people commenting about it even on their website.

They've got a bunch of people saying things this is what people say about
Copilot and one of them was it's teaching me stuff about my language.

I didn't know, which I think is fascinating as a way to learn.

IntelliSense can often do that as well, but IntelliSense wouldn't ever give me this line,

Nirmal Mehta: so I think the combination of Intel sense with something
this is very powerful and I think that's probably where it's going.

So th that's one way to learn a library or a program idea, which is, it's just brute force, figure
out what's going on through, through that documentation through IntelliSense kind of tooling.

Bret Fisher: Yeah I can see an amalgamation of this stuff where it's at it's first
suggesting the logic and then analyzing the logic to see if that's even going to work.

Because I don't think that I'm just going to guess.

I don't know if the machine learning model actually
understands, said syntax and red jacks and get hub values.

I'm just guessing that this was a line that someone
else hasn't one of them or flows and they knew it.

Now, an example of how this fails is not that one yet
that one's a hole that, that one's a whole conversation.

But later in that file, I'm asking for an if statement.

So this is an F on if the value and get hub actions is this, then let this thing happen, right?

That value over here.

Isn't even a real thing.

That's not something that get hub provides.

So yeah, suggesting and then, and that, this is the part of the problem is
if you don't know your language or you don't know your file or whatever.

Sure.

It may be correct, but it also may be complete garbage and it won't work.

And it's not going to know how to tell you the difference.

So this one and I saw this and I went down a rabbit hole.

This actually wasted me time because I went down a rabbit hole of going wait, is github.pr a thing?

Let me go look at the documentation.

And I couldn't find it anywhere in the API.

Then I went back then I did some Google searches and didn't, barely found any results.

So I don't know where he got that from.

There's basically six hits on the internet that have that exact value.

So I don't know what that is or how I got that.

It is describing it correctly because that's exactly what I want.

I want the PR head ref, but that's not a thing I can get.

So it's sometimes it's super confusing and we'll take, we'll
waste you as much time as you want to save them something else.

Nirmal Mehta: Is it sending I'm assuming as part of the tech preview that
it's sending data back to Copilot whether you accepted it's solution or not.

Bret Fisher: Yeah, I think so.

I haven't seen documentation that says that.

Exactly.

But they do have this great little image that suggests that.

That's a great question.

Nirmal Mehta: That's the only way they would know whether

Bret Fisher: Yeah.

So

Nirmal Mehta: not.

Bret Fisher: Yeah, improve suggestions.

So my guess is what that is, because I don't know of any part in
the interface that, and I read their getting started guide it.

Didn't say thumbs up things you like, there wasn't anything that in there.

So my assumption is the improved suggestions, which is the arrow that's coming back.

So it is getting from you the editor context.

So I guess technically the thing you are sending it is either
the read me or the beginning of your function with the function.

So it is getting that, and then it's providing the suggestion and
then I'm assuming this is you're saying, I chose that suggestion.

So I'm giving it a thumbs up

Nirmal Mehta: Yeah.

And if you don't, if you keep typing and don't accept it,
it probably sends that as well saying that wasn't right

Bret Fisher: yeah.

It may be over time.

If I, for a thousand times, if I chose a Bluetooth 2004, maybe eventually we learn it.

That's the one I don't know.

But, and I also don't think these things are unique.

I don't think there's a, I know where have I read because it
suggested that there is some sort of profile on me to get mine.

There is no Bret learning.

It's not, it doesn't have a bred profile to, to know how I

So I'm assuming I'm going to keep getting the same old problems until a ton of people all together.

And I'm excited, imagine a million people doing this
thing, how much input and learning it's going to be doing.

Nirmal Mehta: Okay.

Bret Fisher: And changing on it.

What's weird is that you can start relying on it and then it keeps changing on you because it's
learning so much, you would hope that it's learning to be better, but you may not want that.

I don't know.

Nirmal Mehta: so there, so that's the second key word we're lying, reliability.

Yeah.

This isn't designed to completely be relied on in that sense.

Bret Fisher: Yeah.

They mentioned that in here that it does not replace you.

It's still totally up to you.

It is empowering you with either jibberish or knowledge.

You didn't have.

It's at least on stack overflow because Valentine is
mentioning this is, yeah, this is basically stack overflow.

At least a stack overflow, you can derive from the
thumbs up, what, that whether that's still working.

Yeah.

And if people, if it's no longer working, people will
start to comment on that and we get none of that here.

So it's a little blind one last weird, really odd result that kind of freaked me out.

So this gets me into the conversation around private data and things
that it should never suggest API keys or, one of the things you can do is

Nirmal Mehta: make a file with your, an AWS key

Bret Fisher: here.

Yeah.

So here we go.

If I go into variables and I say

Nirmal Mehta: yeah, there we go.

Yep.

Bret Fisher: Oh

Nirmal Mehta: there we go.

Bret Fisher: So that worked.

That's how it's supposed to do it, not pre-fill it, but I have

Nirmal Mehta: bet.

Can you try the other solutions?

You just go back.

Can you try to,

Bret Fisher: yeah.

So hold on.

Nirmal Mehta: This is fascinating.

Bret Fisher: oh, it's not see, it was totally filling it in three days ago and it wasn't my key.

It wasn't, my key was jibberish.

Nirmal Mehta: So that, that was I think you probably
were in a group of people figuring that out as well.

I guarantee you, people are figuring, trying to figure that out right now, too,

Bret Fisher: yeah, and if we do the.

Nirmal Mehta: Your database password or your.

What's that PHP file a, an it file, right?

Bret Fisher: the PHP.

I and I, yeah okay.

So yeah, it's only, so it's pretty smart here, right?

Per

Nirmal Mehta: I guarantee you though.

That's a, that's a bespoke rule.

They put into the engine.

Bret Fisher: Yeah, it was weird.

It's not giving me type here.

Okay.

So I think what I was doing was maybe something a little less common than that.

So valley let's see, get hub.

And of course this thing is constantly learning and this is already better.

This is already better.

So one of the things that they were saying, and one of the replies that
people that were asking about this was that it's so rare that it would ever
do that out of open source because of the way that they've trained it, that
what it's showing you is sample data, but you can't tell the difference.

How would I know?

So I can see someone creating something very quickly
that would just let it randomly generate that stuff.

Hoping it's one of the safest, random, and then just trying them basically a brute force attack.

Nirmal Mehta: Or there, and it's not every language.

So this is stuff that maybe is very common, but if we're using
some kind of library, that's not as common and put there.

Yeah.

Bret Fisher: it has.

Yeah, I think.

And you're right.

Yeah.

The nice thing is that if you're, as long as you're in, when you're an open
source, you're tend to be using things that a lot of other people are using.

So hopefully this is a very, they actually have an FAQ at the bottom, highly recommended
one, look at that, cause it's super informative and they talk about it happens very
rarely, but it does happen and it can happen that, but so that's why it's risky.

I think, for us to think what if I do it on the rest of my code
or the code that has access to my organization or whatever.

Cause then it has to understand security policy and con the context
of, do you have access to that data for it to learn on that?

That's just weird.

One more one.

This is the last one I got which is pretty hilarious.

So I'm writing a markdown file and it's actually
sometimes helpful in the markdown file I put in the.

Basically, this is a list of me putting in markdown
links to other projects for an open source project.

I created this week on something I did last week using Terraform.

So I create a get hub action runner from Terraform.

I have an open source repo.

We'll probably talk about it some other week.

And I'm pointing to other popular repos that provide ways for you to run, get hub runners.

I put in the first one, and then it provides me a suggestion.

So I hit tab and then enter.

And then it provides another suggestion and I hit tab and then enter and it keeps going.

And I'm like, I don't know if you can notice, but it's
going down a reared rabbit hole of the names on this thing.

It just keeps adding values.

Nirmal Mehta: are those legit links though?

No.

Bret Fisher: This person's name.

I don't know if they let me see if they actually exist.

I don't think they actually existed.

I know that the repo doesn't exist, Mike.

Nirmal Mehta: Sarah.

Bret Fisher: So then, okay, this is okay.

This person exists.

So it's making suggestions and this is what, this is a question.

Is that sensitive information?

Should it not be suggesting this real world persons?

Nirmal Mehta: But at the same time, how many places in the
AML files or other places are you actually referencing?

People's repos, right?

That's legit.

Bret Fisher: Yeah.

But the weird thing is they have no, they don't have
a single public repo with the word GitHub in it.

So that's a real person, real account.

None of these repos are real, but

Nirmal Mehta: now if you commit that and then open source that repo
and then the model gets trained, does this just become self-reinforcing

Bret Fisher: This is this the Twitter bots of 2017 or whatever, with Microsoft and yeah.

Nirmal Mehta: oh yeah.

We're not going to go down that path.

Bret Fisher: But that's, but this is the internet.

That's what people will do.

Let me generate a bunch of data to see if I can, if I haven't do I have
enough power to change the engine on my own, I guess as a question?

Or is it,

Nirmal Mehta: wild.

That's wild.

Bret Fisher: yeah, so I just kept hitting the answer,
accepting, and then at some point I was like, wait a minute.

Are these repos real?

Cause I thought they were real.

I was like, maybe this is suggesting me all the other
things that are this thing, maybe it's that smart.

And it wasn't, it was complete garbage.

Nirmal Mehta: It looks AWS auto scaling is very popular though.

Bret Fisher: it's using all the words that I said in the
document and then just giving me back what it wants me.

I think I want to hear it's it's lying to me.

And that's the weird part of this whole thing is you just.

Nirmal Mehta: Is it a lie?

Bret Fisher: It's fabricating URLs that don't exist.

That's exactly what it is.

It's fabricating URLs.

That's creepy,

Nirmal Mehta: That is very wild, but at the same time so much.

If you're doing this on Golang, you're going to have GitHub repos right.

In

Bret Fisher: right.

Nirmal Mehta: dependency graph.

And that's real,

Bret Fisher: yeah.

The weird thing is that it responds to comments.

So if you're doing a markdown file, what is a comment in a markdown file to it is a whole thing.

A comment crazy.

Nirmal Mehta: Nothing.

Bret Fisher: Yeah.

Links to other.

Crazy other copilot pages.

So remember everybody, if you see a gray, if you see any gray texts,
that is it suggesting inline in real time, there, there we go.

It's actually suggesting the copilot master read me.

Nirmal Mehta: There we go.

Bret Fisher: Is that a real link?

Let me check.

Oh, command click.

No, that is not a real file.

It is a 4 0 4.

Sorry.

People can't

Nirmal Mehta: But that might actually be an internal file.

Bret Fisher: yeah.

You know what you're right, but maybe Because cause that they do give
you, when you get access, they do give you access to something called
copilot it's under it's slash get hub slash Copilot deck preview.

But yeah, there's copilot itself.

Nirmal Mehta: PT internal read

Bret Fisher: go to github.com/copilot, oh, that's a person that's Erin Wagner.

Nirmal Mehta: Oh, that poor person.

Bret Fisher: yeah.

And they do not have a copilot named repo.

So now they're going to, and this is where it just starts to go off the rails.

And, but so if you go around on YouTube right now, this is th these people are doing all this stuff
all the time, but I think they, all the ones I've seen so far, they're largely doing it in code.

Maybe there, let me, I have to activate it again.

Bret Fisher Docker notes.

No, that's not a real repo.

So the, yeah it's basically broken on links and mark stuff, cause that you would
think that it would only suggest, but then again, that would be a suggestion engine
and it's not, it says it's not as, it's not a snippet suggestion engine or whatever.

It's an, it's a synthesizer,

Nirmal Mehta: You're actually proving that it is synthesizing things and not just

Bret Fisher: not giving me things that exist.

It's given me things that don't.

Nirmal Mehta: Correct.

So it is synthesizing.

Bret Fisher: exactly, yes, it works.

We've just used it the wrong way.

Yeah.

Nirmal Mehta: Okay.

What a fascinating, I hope for me with copilot.

I hope it's not a red herring, right?

I hope it's not no matter how much it gets trained,
it will never be, it will never speed up anything.

It will only be two X, what, whatever the time is for you to just do it anyway.

And the other thing is just for context when this was announced and people
started using it, there's a lot of questions around copyright legality.

You, what you were talking about, Bret private information or

Bret Fisher: stores stealing from others.

Yeah.

Nirmal Mehta: yeah or stealing code implementation.

And there, there's a very potentially legally gray area for all, a lot of this.

And then there's also originality, right?

If you're using it to create things, do you actually own who owns what's created?

And all that kind of stuff.

And we're not going to get into that.

I guess there's a thing on protecting originality.

But I think those are all really big open questions, but it's
interesting to me too, I'm excited to see where those discussions go.

The other part of this is I think it's, I think this highlights some of the.

Potential issues with the no code, low code movement as well.

Where I think fundamentally it's still true that the higher you go in abstraction, the more limited,
the scope of what problems you can actually solve with that without breaking that at, going deeper.

And that's still the same case for copilot, right?

As much as it's helping you, it can just be sending you in the completely wrong direction.

And at the same time, you'll have to spend the time to go deeper on anything that I suggest anyway.

Bret Fisher: I love that.

I love that you mentioned the low code thing too, have you ever
seen those videos where it's follow an expert programmer for a day.

And I, this was a year or two ago, at least that I watched this YouTube video
is fantastic because for those that are trying to get into development, not
specifically into DevOps, but just development in general it was someone recording.

Oh, I know it was actually a friend of mine on you to
me that basically recorded their screen for a whole day.

And then they were working on a client project and they showed as someone who's
been in the biz for 10 years, how much they spent searching and stack overflowing.

And that's just that it doesn't ever stop.

You don't get to a point in most careers nowadays where, we're
not all just writing the same COBOL language for 10 years.

That's just, and that the versions don't.

There was 30 or 40 years ago.

There was a time where the only place you had to look up was in
a book and you and I maybe maybe just me, I was old enough that I
there's a very short amount of time where that was what I was doing.

I had to go to Barnes and noble go buy a book.

And we're beyond that now.

So no one developer can be expected to know everything about their language.

And he sh he proved that by showing how much of his time on it day was
not actually writing, but actually searching for what he needed to write.

And what's interesting to me is that in this there's two aspects of it that I can see
specifically for developers, that it will be a very big advantage, is that it will at
least lower that that percentage of how much you have to go look for something when

you're not sure the right, the right part of that function that you need to add, or
how many values you can you put into that the options of that function or whatever.

There's just a certain amount of that you can.

Not have to spend Googling and it's right there in your car.

Oh, you don't have to leave your editor.

The other thing that I've seen it do that I haven't yet gotten into myself, but I've watched
other YouTube videos is that it's pretty good at writing tests, very simple small function tests.

So we should, one of these things is you could argue that it doesn't
really matter if your code isn't the way that someone else might write it.

If their test pass and the function correctly executes.

So if you can use it to help you write the function and then
you can write it, help it, write the test for that function.

And you're always writing tests.

Is there a, is there harm to this?

Is there, what, if it's not using the latest spec is
one of the problems I ran into is with my Terraform.

It was suggesting me things that were legacy that were deprecate.

And I didn't know that until I, because of one of the problems was all the examples
that was learning from were all using the old pre Terraform 1.0 way of doing things.

And I stumbled onto an example that was using the new way.

That was actually easier, but it wasn't suggesting that.

And I ran through the options.

Again, it never suggested the new way, because I guess that's so new that people aren't doing it
yet, but I was essentially using the new version of a method and it was not helping me with that.

It kept wanting to do the old way.

So that's a problem too, is it's going to likely give you old stuff,

Nirmal Mehta: Yeah.

And I wonder if there's some tweaking the sources, maybe only the top 1000 GitHub
repos or, and then only code that's w less than four years old or something that.

Maybe there's some way to tweak the model and get rid of the long tail of

Bret Fisher: Oh, that's an interesting, that's an interesting suggest.

So code that is dated this year, it gets more precedents

Nirmal Mehta: And then the other thing that would be interesting is
if you just write tests then let it write code to pass those tests,

Bret Fisher: yeah.

Nirmal Mehta: I think will be an interesting

Bret Fisher: so

Nirmal Mehta: reverse scenario

Bret Fisher: and then it writes the code and it's smart enough to look at the
test and that would all have to be in the same file at least currently, because

Nirmal Mehta: it would run it against right.

It would run it against the test.

And once it gets code that passes that test a genetic kind of way of coding right.

Evolutionary way of coding, that's going to be, those things
are going to be where this stuff gets really interesting.

So does that round out, your experience with copilot.

So far and where you think it's kinda going to go.

Bret Fisher: Yeah.

The thing is I guess it's a super early preview.

I don't know how many people have access, but it can't be more than the thousands
I would assume because they said that before the technical preview was only
hundreds that they were using it, I'm really interested in what does it look
in a year when tens of thousands, if not hundreds of thousands have used it.

And when they start, when it starts showing up in vs code is Hey, you
want, cause there's tons of people using vs code, but if it starts
promoting itself in vs code, cause there's an incentive here, right?

This is all Microsoft.

They're using their editor, it's their company.

And then on their read me, I just actually didn't realize this till today.

It says, will there be a paid version?

If the technical preview is accessible, our plan is to
build a commercial version of get up copilot for the future.

We want to use this preview to learn how people will use
ghetto copilot and what it takes to operate it at scale.

So I can imagine that maybe the paid version is what analyzes your own code.

Maybe this diversion we're seeing now is maybe the free version for everyone
that feeds their machine to, it feeds the data set that will be the paid version
that then we'll also analyze your personal code or your organizational code.

I'm going to, I'm totally speculating here, but I could see how that would be an
incentive for you to pay for get hub enterprise or get hub or orgs or whatever.

So someone sorry.

Got you.

You talked for a second.

I'm looking up the video, someone in chats asking for this video on a day in a division.

Nirmal Mehta: I agree by the way, I do want to see that video too.

And I think I also think that a path toward more combination with
the Intel sense and documentation to make it more of a training.

And an actual pair programming user exp or developer experience versus
maybe having a little bit more explanation of why it's it shows that code
snippet, or if you do accept that code snippet, what it is or link to the
documentation for that thing so that it would already anticipate your next step.

So if you do a code complete, you were saying, it's deprecated, Terraform, maybe it gives
you a link to the Terraform documentation so that you can validate what it just provided you.

Because that's the next thing you would probably want to do.

Bret Fisher: Yeah.

And if

Nirmal Mehta: you were being bit about it,

Bret Fisher: And the reality is what I think that, in my job as a
DevOps person, I write a lot of the same stuff over and over, right?

Infrastructure there's infrastructure, I'm re I'm, creating and creating
all the scaling groups I'm creating, Google, Kubernetes clusters.

And so that stuff is all in a repo that I have access to somewhere.

And then, so a lot of times what I'm in up doing, because I'm not going to stack
overflow, I'm going to my own old code and then copy and pasting it into my new stuff.

And this seems a lot more efficient way than some sort of advanced snippets add on, because I don't
know if you've been in the big, into your own personal snippets, but I've never, I just, I so rarely
do that stuff because whenever I try to implement it in myself, it's changed six months later.

I'm not doing that way anymore.

Now I have to maintain that the maintenance of my
snippets is more than the advantage of my snippets.

Nirmal Mehta: I didn't think about the snippets angle is like, that could be pretty powerful

Bret Fisher: Yeah.

So if you're

Nirmal Mehta: if you use

Bret Fisher: my stuff.

Yeah.

Nirmal Mehta: yeah.

So if copilot could auto-generate a point in time snippets library for you, that'd be

Bret Fisher: yeah.

Or if it's just simply suggesting based on what I've already written.

So if it knows my way, or this example of the Terraform of using the modern
Terraform 1.0 way, if it was looking at my other code, it would pile, I
would assume it would highlight or prefer those suggestions over its own.

And then, that's the thing is there's no, there's,
it's this endless, there's endless possibility here.

Nirmal Mehta: yes.

It's fascinating though.

Bret Fisher: Yeah.

Nirmal Mehta: Cool though.

Cool.

That they're trying it right.

This is really pushing the developer experience into new areas.

For sure.

I

Bret Fisher: Especially on, when GitHub tries something, it's like.

When Google tries something and in Gmail and they experiment you and
I were talking last week about how the GU the Gmail auto-complete,
this feels a lot that, where I start typing, I forget that it's there.

I start typing and then it auto fills and I go oh, that's right.

I have this thing in the background.

And then I read their statement and then I think about it.

Do I have what I want to say?

And then I tab, or I don't tab.

And that's very much the workflow of this.

Nirmal Mehta: And the difference there though, is that
the Gmail one for sure is it's my it's based off of that.

Person's writing style, I think, because the suggestions I've seen, I say y'all a lot and
my suggestions we'll have y'all in it and it will be the way I phrase things typically.

Whereas I don't think that's turned on for copilot yet.

And to your point where I think eventually once it's released likely you'll opt into a
thing where it will scan your repos and then it will weigh, it will it'll show a preference.

What you have written in the past and then fill it in with whatever is in the global model.

Bret Fisher: right?

Yeah.

It's there it's certainly early days compared to, I think to that Gmail one.

So real quick to the previous conversation on watching someone go throughout their
day as a high paid senior engineer this is over on Colt steel's YouTube channel.

I'll put the link in chat.

Cole steel is got a million Umi students and I got to
hang out with him at dummies conference a couple of times.

Walk around San Francisco, really cool guy.

And it makes great courses.

I own at least one or two of them.

And he did this thing where he paid for, he had a friend and
got permission from their client and them and everything.

And he has great intro graphics and argues against the idea that
somehow a senior engineer knows everything that they're writing.

And so he goes through this, he basically breaks down some steps that, of what
happened in this person's day and puts it into a nice visualization for 10 minutes.

You're not literally watching this person, but yeah.

And so the person's logging what they did all day.

And then he looks at the data of what they did, how many websites that they could do.

How many Google searches do they do they do, how much code did they write?

What were they writing?

Yeah.

So it's a really cool little breakdown.

a senior engineer, how much time they spent on the internet looking up answers.

So anyway, I put that jet cold scale steel, go

Nirmal Mehta: so they got the Google search history from that developer,

Bret Fisher: Yeah.

And I think and I think he just recorded a screen too, but
yeah, he also did some, I think some analytics on the history.

And I assume if I remember correctly, one of the days, it was a rabbit hole kind
of day where you just I've had several of those recently where you thought you were
gonna spend an hour on something and then eight hours or seven hours a whole day later
you're still banging your head against the wall, trying to get a thing to do the thing.

And it's a really good video.

Nirmal Mehta: Let's both ask each other this as a stocker captains.

How many times do you pull up the Docker file documentation?

Bret Fisher: Because they've

Nirmal Mehta: permanently tattooed on your arm?

Bret Fisher: lot.

Yeah.

I think for up until at least 2019 or 2020, it was permanent tab in my browser that in compose.

Nirmal Mehta: Yes.

Bret Fisher: say today, just because the change rate has
really gone to almost zero over the last two or three years.

And because I'm probably editing files every week that I've gotten super
comfortable, but then there are going to be those times where there's things,
I don't know everything about Docker felt I forgot about the maintainer line.

I saw someone using it the other day.

I was like, oh yeah, there's that maintainer line.

I'd never used that.

It's not I know it all.

It's just, I don't have to refer to it.

What about you?

Nirmal Mehta: right.

Same.

I get the source and destination port flipped all the time

Bret Fisher: Especially due to Kubernetes, right?

Dockers make sense.

Doesn't I don't know.

Nirmal Mehta: I have to look it up every freaking time, but
I'm also not in it as much as you're in it for your job.

Bret Fisher: It

Nirmal Mehta: but you go on a two week vacation.

You're going to have that tab back up, right?

Bret Fisher: sure.

And, but with Terraform, which I've used for years, but yeah, I will
absolutely have to have every page and every provider up every time I'm using

Nirmal Mehta: Yes.

Bret Fisher: because I don't do, I don't do it every week,

Nirmal Mehta: And it also changes.

That stuff just,

Bret Fisher: yeah.

Nirmal Mehta: it'd be interesting to watch the velocity of changes over time for
the interfaces and the API APIs that we typically use because that's very high.

I think.

Bret Fisher: Yeah, that's a separate conversation or kept separate episode.

We could talk about Kubernetes one dot two, two, which is dude
out next month and is going to have a lot of APIs go away.

And the one on one of the ways I know about how much Docker changes their spec and
how much Kubernetes changes or spec or any of these tools is if I put it in a course,
because the minute that it doesn't work anymore, I have a student that rightfully sows.

So they say this doesn't work anymore.

And I that's kinda how I ended up learning about deprecations and
deletions firsthand is I write something about it and then people use it.

Nirmal Mehta: So is there some kind of Murphy's law of of course creators and trainers yourself?

There's some kind of rule of the universe that the minute you put that into a
course, that's the minute some engineer goes, I think I need to deprecate this API.

Okay.

Bret Fisher: it only gets deprecated based on the number of people that put it in a course.

I would say that I definitely the number of versions in the software in
its entirety will, are directly correlated to how much their API changes.

Because it always slows down over time.

And Kubernetes was one of the reasons I had delayed so much of my Kubernetes
courses, putting them out was because I was, I knew that if I wrote a
course, it would have to be completely rewritten in those early days.

And it wasn't until 1 15, 1 14 that I started to
consider it because it started to slow down a little bit.

Nirmal Mehta: So if I was a consultant, I would use copilot to find out what the majority
versions of things are and create content to like, help migrate that to better versions of that.

Bret Fisher: Yeah.

Nirmal Mehta: Maybe.

That's interesting.

Bret Fisher: and getting back to copilot, the versions of things
and all that it makes me wonder, I'm now realizing we just did that
Docker example and I ha and the Docker plugin, which I haven't even.

It suggests in a different way.

It suggests public Republic, repos and versions of things by querying the API in the background.

And I haven't been, and I haven't been seeing that lately.

So now I'm wondering if there's somehow stepping on each other's toes because I actually use that.

If I type in from colon, cause I, one of things I about Ubuntu
is they will date base version, their releases, which I love.

And especially for production stuff, because I can rely
on them, not accidentally updating some, 8 20 0 4 version.

And it will provide it's clear as the API and then provides the options.

But I haven't been seeing that.

And now I'm wondering if copilot broke it, maybe there's something in there I got to look at.

Nirmal Mehta: That would be an interesting thing to turn that off and see what the differences are.

Bret Fisher: Yeah.

All right.

What else?

Anything else?

Anyone else got questions about copilot?

Mike's call it a Bret's breasts.

Nirmal Mehta: I it.

I'm going to start using it.

Bret Fisher: The number of courses, I put something,
which is the, my, my biggest one is Kubernetes one 18.

When they got rid of when they completely changed the run command.

Yeah.

It broke every course I had Kubernetes in.

Yeah.

So get on that, get in the beta.

If you want to try it out,

Nirmal Mehta: so I haven't gotten into the beta yet,
but now if I do get into the beta, I'll know what to do.

Oh, sorry.

Not the beta, the technical

Bret Fisher: technical preview.

So yeah, over at copilot, I get hub.com.

That's how you get started.

David's asking if you, did you try with Python?

It's the best way to use it?

I have not tried.

I have actually not done any programming since.

Since I got access to it about a week and a half ago, I had been entirely in configuration files.

So Docker files, Kubernetes files, Terraform files, read MES make files.

Yeah, I've been in that land, DevOps land, not in developer land.

I would say it's probably going to be the best insight side of programming languages,
no JS and Python and Ruby and go because they specifically list those on their site
and they, and if you go looking around YouTube, you will see all sorts of other great
developers for channels demoing those, but nobody would was demoing DevOps stuff.

So that's where I wanted to focus today's on is the non, the non
programming language is more of the, you can argue what Yamhill really
is that the data languages supposedly is really good at SQL syntax.

Nirmal Mehta: I can imagine that.

Yeah.

So David, what's your experience with Python?

Put it in the chat,

Bret Fisher: Yeah.

How does it

Nirmal Mehta: in terms.

Bret Fisher: Yeah.

Nirmal Mehta: Oh, die specter, just redacted their message.

In terms of creating attacks or being malicious.

I, it depends on if it's actually learning from the user base.

But at the same time, what Bret was showing with respect to API keys, I was feeling if there's
some common way or not so common ways, but just enough content around configuration files with
secrets in them that at some point it will probably leak that data when it's not the top 50,
typical ways of storing and configuration file, but it's still enough content in that way.

So maybe an esoteric library or a language that's not as popular.

Or a framework that's not as popular.

It's probably gonna start leaking those secrets I would imagine.

But the thing is that means that those secrets are
somewhere in the public open source GitHub, repo world.

Bret Fisher: Yeah and because it's not providing snippets that actually it
gets Pantano asked the major one made advantage over stack overflow is that
the suggestions are almost always from working code, but it's not a suggestion.

It seems it would be better, but the reality is the code it's generating
has never been tested, unlike a stack overflow answer, which presumably
the person would have run it or there wouldn't be so many thumbs up.

So it's weird that I actually started this conversation
with arguing for this and against that overflow.

But I could probably argue with myself back and forth on
this topic and say that it's not as good as stack overflow.

It's more convenient.

It's way more.

It's way easier than having us go through 20 different stack overflows
and realizing that 19 of them have nothing to do with your thing.

But the, we proved several times today, the, in the markdown and the fact that I got that example
that had to get hub value that doesn't even exist that it's clearly not producing workable code.

There's nothing that's testing this code.

It's up to you to determine is that even real code that would work.

So then you can't be a noob in your language and use it.

You just can't, you would only be correct 50% of the time.

Nirmal Mehta: Yeah, so now I can write my bugs even faster.

Bret Fisher: That's all right.

That's all right.

But as long as you're testing, so here would be the fun thing is if it creates a test
that's broken and fails, true to broken function that it wrote that doesn't work at all.

So it, so the test pass on broken code because it wrote
both is it possible how easy it is to do, I suppose as

Nirmal Mehta: That'd be interesting.

Test to try.

Bret Fisher: think there's tons of fun to be had with this.

I encourage you all we have, we don't do a lot in the DevOps fan right up here, DevOps fan.

This is our DevOps discord server.

Now with, I think we're got 5,000 or so, let me look at the
latest stats, how many people we got in here 5,700 members.

It's free.

You can come and hang out with us.

We've got channels on most DevOps topics, and we don't specifically talk
too much about code languages because there's a couple of other really
big and really great discord servers, including one called the coding den.

And then of course, if you're language specific, if your languages,
Google, or Microsoft languages, they have their own discord servers.

But if I just go to dev ups.fan, I can get in there.

We're gonna put the browser, so I encourage you all we actually have in here.

Oh, it's wanting me to sign up and I haven't logged in, so I'm not gonna
do that here, but that we have channels in there where if people are more
interested in this topic, we can actually create a whole channel on copilot.

And talk about it maybe just an AI channel but then again, I could see how Copilot
could be the entire topic of AI because AI in general is a little too general for this.

But as more of you get it into the beta, I'm sorry, technical preview.

I w I would love to see feedback and examples, people jumping into one of the
audio channels and sharing their screen and showing off what they're doing with
it, because I'm sure there are more useful ways to use this for DevOps workflows.

And I haven't quite figured them all out yet because I've only had it for a week.

Yeah.

Nirmal Mehta: I can't wait to be accepted for the tech preview and try it out for myself.

But I feel I'm neutral on it.

I think it's kinda too early to tell whether it's actually useful or not, but.

Bret Fisher: It, I would say in the week of me using it for primarily
Terraform markdown, Kubernetes and Docker and maybe some shell
scripting, so some bash stuff and some GitHub actions, GMO mark down.

So a lot of Yammer and a lot of HCL, I would say that it definitely saved me keystrokes.

I don't know if it saved me time.

Nirmal Mehta: There we go.

That's a tweet.

Bret Fisher: That's it definitely saves me keystrokes writing descriptions.

It's great at re in the AML and HTL writing descriptions, writing providing me default value things.

It's great at But providing, but when it provides me the entire yamble
of markdown for, sorry, not Mario Yammel of a manifest for Kubernetes.

I still have to look at that thing, understand the entire thing it gave me,
make sure that I think it's right and still have to like, validate it myself.

And that takes is just almost as much time as writing it sometimes.

So yeah that's a tweet.

I'll send that tweet out later.

Nirmal Mehta: There we go.

Bret Fisher: All right.

And if we don't have any other questions let me check real quick.

Nirmal Mehta: I think we've beat we've.

We've gone through this topic.

Bret Fisher: Yup.

Yup.

Now just everybody needs to get access so we can have another
conversation about it where I'm not the only one showing it off.

Nirmal Mehta: Yeah, for sure.

And next time we can try the live.

I don't know if that's going to work the

Bret Fisher: oh yeah.

Yeah.

The lives alive share,

Nirmal Mehta: Yeah.

I think that's, is that part of workspaces

Or workspace?

Bret Fisher: You mean code

Nirmal Mehta: Not that's not.

Oh yeah.

So that's not part of code spaces.

That's

Bret Fisher: separate features.

Yeah.

Live, share, and remoting as a different feature, which allows
you to work on other things that aren't on your machine.

So one of the things that, what I'm interested in is when you get access, if we're
both sharing a live share of the same repo, are we both seeing the same suggestions

Nirmal Mehta: Okay.

We'll definitely

Bret Fisher: think that's a way more interesting.

Yeah.

And maybe you're sharing your screen and I'm sharing my screen.

And are we seeing the same thing at the same time?

I would presume that it does because it's not user specific, but

Nirmal Mehta: So who's going to be the first person to
submit code copilot to do, to use copilot in a hackathon.

Bret Fisher: oh, you mean to make something new that didn't exist in the world from code

Nirmal Mehta: like

Bret Fisher: Yeah.

Cause it's, so it's pretty good on comments and it's directly, as
we've seen today as directly related to how good your comment is.

So if you're writing great comments, I don't how far could you get to a functioning app?

That's not just a hello world app, but a functioning app.

How far could you get there by never writing code and not writing a single function yourself?

You only are describing the functions and then
tabbing through the all options and then accepting it.

Can you create a Twitter clone by just using AI generated stuff?

Synthetic?

Nirmal Mehta: It.

I like

Bret Fisher: That's a good challenge.

That's a good challenge.

Nirmal Mehta: That a good challenge.

We'll put that out in the world

Bret Fisher: If one of you all want to try that challenge, once you get access, let us know.

And we may highlight your video in this in this live show.

All right.

Thank you Nirmal.

I'm not trying to kick you out, but I'm definitely gonna have you on again.

What are you doing next week?

Nirmal Mehta: I can be here again.

No problem.

Yeah.

Bret Fisher: All right.

Let's see if we can make it happen.

We'll talk about something else and it's always great to chat with you and I can't
wait to see you in the real world and real world hug high five and all those things.

And for all of those youth hanging around for this full hour and a half
show about copilot, I really appreciate you for hanging out and thumbs up.

Do the likes, follow, thumbs up, do all that stuff to help us with the algorithms.

So other people can check out DevOps.

I think this is the first time that we've had there's.

I haven't seen any other YouTube videos on DevOps and copilot.

So hopefully we're breaking new ground here and we will be
back here live next week, maybe with normal, if we're lucky

.
Nirmal Mehta: Thank you, Bret.

And I hope you never replaced me with copilot when
I'm Copiloting with you on your YouTube live show.

Bret Fisher: wait, hold on.

Nirmal Mehta: Robots will take it over.

Bret Fisher: yeah normal.

False is the name of this this robot that I'm spending time

Nirmal Mehta: I real?

Bret Fisher: Good jokes.

Good jokes.

All right.

Thanks man.

I'll see you soon.

Bye internet.

We'll see you next week.

Same time.