Building a Copilot Agent integrated with Azure DevOps

[Music] Hi everyone, I'm Todd and welcome to another edition of AI powered PM where we provide practical examples of how to use AI effectively to streamline your

job. One question that comes up a lot as

job. One question that comes up a lot as a PM is what's the status of a particular project. This information

particular project. This information often exists in Azure DevOps, but the people who ask this question want it in a more consumable and relatable format

than sharing a sprint board view or ADO query. Today, as promised, we're going

query. Today, as promised, we're going to build a co-pilot agent that will help in this area. I set out to build an agent that will answer two questions.

First, what are the items in a particular sprint? And second, how many

particular sprint? And second, how many total hours are there? How many are completed? and how many remain in that

completed? and how many remain in that sprint. So, enough talking. Let's get

sprint. So, enough talking. Let's get

into the demo. But first, let's talk about the prerequisites.

We're going to be working with Power Platform. And here's the big picture.

Platform. And here's the big picture.

It's made up of four different parts.

Copilot Studio where we'll build conversational AI agents. Power Automate

where you can build automated workflows.

PowerBI where you can analyze and visualize data. And then Power Apps

visualize data. And then Power Apps where you can build custom applications.

Specifically today, we'll be working with the data versse component of Power Apps. Now, this is just an extremely

Apps. Now, this is just an extremely high level. There's ton more tons more

high level. There's ton more tons more features and functionality to each of these, but let's just stay at the 50,000 ft view. And today, we're going to focus

ft view. And today, we're going to focus specifically on these baddies right here, Copilot Studio and Dataverse.

So, why this tech stack? First is

accessibility. 80 to 90% of enterprise users users already have access through an existing Microsoft 365 license. Uh

second is cost. If you don't, you can get Microsoft 365 Business Basic for $6 a month. So it's accessible to pretty

a month. So it's accessible to pretty much anybody. Um and then security,

much anybody. Um and then security, you know, your data stays within your organization's Microsoft tenant. There's

a lot of governance policies around it.

It's IT approved uh across enterprise customers that central user management and then integration. There's a seamless data flow between familiar business

applications with this product suite.

[Music] All right. So now we're in DevOps and

All right. So now we're in DevOps and we're looking at what's in the sprint board. And as you click in, you can see

board. And as you click in, you can see there's many templates built into ADO.

This one happens to have original estimate, uh, remaining, and completed hours, which will be important as we do our demo a little bit later in this in this presentation.

Now, let's jump on over to queries and we'll go into our sprint progress status. And you'll see there's just a

status. And you'll see there's just a simple query. It rep it returns back

simple query. It rep it returns back basically everything that we've got in ADL. And now we're going to export it to

ADL. And now we're going to export it to a CSV file.

In the CSV file, you can see that we have the same rows and data. Now we're

jumping into Copilot Studio. So, we'll

create a new agent and let's skip straight to configuration.

Let's name it Ktoso Air DevOps and I'll paste in a starter description and starter instructions.

Great. Now, let's check into the knowledge sources. They're very limited.

knowledge sources. They're very limited.

The only ones available at this point are public websites and SharePoint.

Everything else is grayed out, but that's okay. Let's go ahead and create

that's okay. Let's go ahead and create this agent.

Now, it's created. And if we come back to knowledge source, you'll see that we have uh data versse and many other options. So, let's go into data versse.

options. So, let's go into data versse.

All right. This isn't data versse. It

It's not quite that epic. Let me show you what the real data versse looks like.

All right, this is the real data versse.

Uh you can see we're going to create a table and we can start with our CSV file.

All right, so that's loaded. Let's

import it.

Now when it imports it's important that we go through and check all the columns. By default it won't necessarily have all the columns created.

All right let's find that table we created.

All right let's go down. Let's find work item record.

You notice it's a standard data uh standard type and which is what you need. And then you'll see that here's

need. And then you'll see that here's all the data that we have in our Excel.

It's now loaded into this table and data versse. Let's go into columns and we'll

versse. Let's go into columns and we'll create a new column. Let's call this one sprint and we'll add a description.

All right, let's select our data type which will be formula.

I'll give you this formula. I'll put it in the comments or link to it later.

For now, let's just see what what it did.

So, we'll go back into the table view.

Let's adjust it so we can show that column that we just created.

And we can see now that we took the iteration path column and instead of saying Ktoso air iteration 2, you'll get a value that's just two oro air

iteration three, you get a value that's just three, etc. All right, jumping back into Copilot Studio, let's add in the table we just

created as a knowledge source.

All right. So, we need work item record.

And let's go ahead and add that.

Great. Let's test this.

What are the available iteration paths?

All right, it's checking the knowledge source. Let's see what it comes back

source. Let's see what it comes back with.

All right, so it found Kontoso Air 2 iteration two, three, and one, which is correct. Let's ask it what are the rows

correct. Let's ask it what are the rows in Ktoso Air iteration 2.

Okay, good. It pulled back the titles.

By the way, I didn't mention this earlier, but title was the primary key.

So that's why it's pulling back those as the um default values.

So let's ask it show me the items in Ktoso air iteration 2 and include title assigned total

hours uh hours remaining and remaining cars.

All right, great. So, it pulled back what we were looking for. I didn't do a perfect job on that prompt, but it gave us the title, who it was assigned to, the total hours, and the hours remaining.

So, that's actually starting to look a little more useful.

But let's see by person. So, for each assigned to person, give me the total hours, completed hours, and

remaining hours.

and summarize by person.

All right. So, you can see we've got an all-star team over here at Ktoso Air.

We've got Johnny Cash and Finest Torbalis and Richard Feman. Uh, and they correctly pulled back total hours, completed hours, and remaining hours.

Now, let's see if we can uh reference the same data by saying sprint 2. Show

me the items in sprint two.

And it failed. So, all right, let's investigate and see um some different options we have.

So what are the values in sprint? We're

just asking what what's in that column.

Okay. So it pulls back the values.

All right. Let's try to ask it a more targeted question just to try to get things working.

All right. What are the total items in sprint? Quotation marks two.

sprint? Quotation marks two.

All right. And still it failed. So let's

dig in here and see what's available.

So we go into knowledge. Uh let's let's work with synonyms. Let's just try uh try this. Let's use iteration instead of

try this. Let's use iteration instead of iteration path. That's a pretty good

iteration path. That's a pretty good synonym.

So what are the available values in iteration?

Okay. So, so that worked. So, synonyms

are really just another way to call the column name a different name and have um copilot understand it. So, all right, let's try glossery. Let's still keep

working on this on this uh sprint problem.

So let's just try a hard-coded approach here. We're just trying to get the

here. We're just trying to get the basics working and we can always make it uh better once we get something simple

working. So Ktoso air iteration 2 and

working. So Ktoso air iteration 2 and let's describe what this is.

So this is the sprint iteration.

This example could also be re referenced referred to as sprint 2. All right. Very explicit in

our in our glossery here. So, let's

let's give it a try and see if it works.

Show me the items in sprint two.

All right, it got a little cut off there at the end, but when we queried for sprint, the items in sprint 2 and it it didn't end up working. Um, so to recap, we started and I wanted to answer two

questions with this agent. What are the items in the sprints and how many total hours are there? how many are completed and how many remain in sprint 2. So we

didn't quite get there. We got close. Um

we said we had to say what are the items in Ktoso web iteration 2. All right. And

we said and the same thing is true for the second question. Um but you know I think it's good enough for for MVP. um

if I was but I also think that I don't have any insider knowledge here but I would imagine that Azure DevOps will come as a knowledge source at some point in the product uh for copilot studio and

then I think then it's going to be a more appropriate time to build a co-pilot agent like this and then I think it'll be more more reliable more

robust right now uh end users have to be very familiar with the with the column names in order to use it effective actively and see the views. So, I don't

think it's like a nice MVP or proof of concept, but I don't quite think it's ready for prime time just yet. Um, if it was, my next steps would be I would show

you how to automate so you could always get uh the new items uh uploaded into data versse. Um, but we'll save that for

data versse. Um, but we'll save that for when things are a little farther along and a little bit more matured and then we can implement this scenario in a much better way. So, thanks for watching.

better way. So, thanks for watching.

That's how far I can get on Azure DevOps. I guess I know now why I

DevOps. I guess I know now why I couldn't find a video explaining how to build this up on YouTube. So, now

there's one. You see how far I can get and you see how how some of the shortcomings. So, all right. Thank you

shortcomings. So, all right. Thank you

very much. I'll see everyone next time.