Implementing and scaling AI agents in business

I think many companies have seen that if they put this off for a long time, it's time for them to revisit to revisit the legacy systems to revisit the old way of doing things to basically prepare um not just for the current benefit of being

able to use like AI in its current form, but to then be able to uh respond at all the the great innovations that are happening today.

>> Hello and welcome to Intelligence Squared, where [music] great minds meet.

This episode is sponsored by Box. I'm

Kamal Ahmed, executive editorial director of Fortune. And today we're back for part two of our series in partnership with Box, all about the world of AI in the workplace. Now, it's

not the hype, not the empty promises, but the practical steps that actually make a difference. Organizations are

experimenting with AI, but many struggle to turn ambition into measurable [music] business impact. The question isn't just

business impact. The question isn't just what AI can do. It's how companies get ready to do it safely, efficiently, and at scale. If you're listening to this

at scale. If you're listening to this episode, I know one thing about you. You

care about staying ahead. You want to understand not just what AI can do, but what it should do in your organization.

And if someone shared this episode with you, they think you're exactly the kind of leader who can take bold ideas, turn them into action, and help your team

thrive. And don't worry, we don't have

thrive. And don't worry, we don't have to do all this alone. To help us navigate, I'm joined by Ben Kus. Ben is

the chief technology officer of Box, the leading intelligent content management platform. He spent years helping

platform. He spent years helping organizations unlock the potential of their data [music] and build AI powered workflows that actually work. Today,

he's here to do the same for you. In

this episode, Ben will be walking us through the five essential steps every company needs to take [music] to get AI ready from auditing data to measuring real business [music] value. Welcome,

Ben, to the podcast.

>> Uh, happy to be here. Thanks for having me on.

>> Now, Ben, you have your own podcast.

This is going to be an absolute master class the next 45 minutes of our chat uh podcaster to podcaster. So it's lovely to have you on. Now as I said in the introduction five key steps we're going

to take you through this in the next 40 minutes or so around the issues of audit single source of truth starting small pushing beyond the chat and measuring

value. These seem to be the five key

value. These seem to be the five key issues that people who are considering how AI can work for them have to really

uh think through as they approach their new projects and 2026 and beyond. But

let's start at the beginning. I think

still for many many companies AI adoption feels a bit scary. It feels

something for the future but you have to do it now. Where should you start when you are thinking about AI, artificial intelligence and what it can do for your

organization?

>> Yeah. Yeah. This is a um very much uh something that is um a common question that many companies are dealing with and and um and uh I think it it does take a little bit of a difference for every

company that uh looks into it. But in

general um one of the things that we uh sort of see where people who are successful is when they're able to take things that they u different tasks or different processes they do internally

and they're able to start to have AI helping with them first on those tasks and then being able to then kind of uh have add AI more and more into the mix.

Um so a couple examples um so uh one of the big examples that that we've seen over the last year across u many industries is their ability to use AI as

sort of a uh as as a as a uh for for software engineering. So specifically

software engineering. So specifically like um it started out that AI was uh like a helper like an assistant for uh for developers and then it and it quickly moved into the world where you

started to have AI agents taking more and more of the coding tasks. So instead

of just having to complete a function for you, it would actually start to go and uh create more of the software, do more of the edits, and so instead of you uh when you're programming having like a

like a pair programmer, you started to go down the path of asking AI to do like uh more and more like almost like an assistant or almost like a uh like a of somebody who worked for you. And we

think that this sort of approach is like is is something where you start to see this in more parts of business that different people are able to not just have AI that helps them um but then also

turn around and have it do more and more tasks for them as AI agents become more and more complex.

>> Ben, let's start with that step one that actually we know that a lot of reasons for failure of artificial intelligence is nothing to do with actually the artificial intelligence project. is to

do with where you start and actually your data is where you should start. I

was fascinated not only listening to your podcast, the AI explainer podcast, but also reading into the work that Box does.

Audit your data architecture keeps coming up again and again. What do we mean by that?

>> Yeah. So, um like when we talk to many of our customers, uh we often hear them say things like um they don't actually have an AI problem. It's not like uh they're saying, "Oh, no. how do I get AI to do things for me? Instead, they have

a data problem. And in particular, um their data uh across their whole company, which is often a mixture of what you would call unstructured data or

things like uh uh uh files and and and emails and messages and so on. Um is uh in addition to structured data, which is like stuff in databases. Um so both of

these um in many cases are not sort of centralized in platforms that then make it easy to use with AI. And [snorts]

this is one of the key challenges that companies have is that they say, "Oh, I want to make I want to uh use AI to help me automate this process. I wanted to use it to help me my salespeople create better presentations. I wanted to uh

better presentations. I wanted to uh have my um uh uh my data analyst be able to to analyze the data databases and then and then relate it to the different uh other parts of this data in our

company." But then they say, "Oh, no,

company." But then they say, "Oh, no, but like that data lives over here in this silo. This this data lives in old

this silo. This this data lives in old system that we have. There's a legacy file server somewhere." Um, and so then this actually gets in the way of them being able to use the AI, not because AI models are not good enough or not

because AI agents couldn't do that process, but because they had no way to give AI agents access to this data. And

this is a huge problem that in many ways is like a common problem across many enterprises. Uh, historically for all

enterprises. Uh, historically for all sorts of reasons. Companies have always had to deal with this data silo problem, but it becomes more and more important as you start to have these new

technologies. Um, and remember AI uh

technologies. Um, and remember AI uh typically runs on GPUs in a cloud system somewhere and so requires this like uh access to your data which then bring up um some concerns. You have to make sure

that's secure, it's compliant and so on.

But then also you need to make sure that the AI has access to it in a way that is uh readily available for the AI models and and typically that's where a lot of people start as their first problem is

to say okay I'm ready to do AI but then they say well it doesn't have access to something that I want it to uh it to do.

And so the analogy here is um we like to think of uh uh in some way when you talk about AI agents it helps to think of them as almost like a new employee in the company. And it's like imagine you

the company. And it's like imagine you bring in the smartest employee you'd have in the company and you say uh like I want you to go and and then and and then start to work on something you and

and then they say uh okay like where's that data and you say I don't know like I I don't I don't won't give you access to it. I can't give you access to it. In

to it. I can't give you access to it. In

which case no matter how intelligent these AI models are or um in this analogy how smart the the new person who came in is they're going to struggle to deliver real business value if they

don't have access to the data. [sighs]

And Ben, how do you go about auditing your data architecture? Is that a CTO division operation? How how would you

division operation? How how would you start? I I remember speaking to many

start? I I remember speaking to many many CEOs in a previous sort of pre-AII era or pre the present generation of AI era and they would have computer systems

that did not speak to each other. One of

the big headaches when you were trying to do um takeovers and bring in new companies was that the computer system simply didn't speak to each other and it

was an absolute uh it was one of the most difficult things that chief executives and executives um had to deal with.

>> How do uh Ben you do a data architecture audit?

>> I I think one of the So there there's kind of two approaches to it and I think many companies are doing both at the same time. one is kind of a top down uh

same time. one is kind of a top down uh because many many systems uh many many companies they you understand in the most part like what is the approved systems that you're supposed to store data on um and then and again different

forms of data whether it's CRM data structured data unstructured data and so on um and so from a top level like what tools have you made available to your company from like a central IT level it is useful to do a top down audit so you

say where do I typically store my structured data is it in a is it in a tool like u a snowflake or data bricks and so on and and then similarly where's your CRM data. So that that makes

available your your uh employee data uh I mean uh your u customer data or your employee data with something like workday. Um like those kind of systems

workday. Um like those kind of systems are are uh uh are very important for you to be able to say like are we using a modern system that will be able to enable us to access it via via AI. And

one of the the key new areas is this world of unstructured data, right?

Because this is kind of what generative AI models were born on. uh they they do a really good job dealing with things like emails and and and documents and PowerPoint presentations and and Excel

files and they're able to understand all these things and and this is usually the area where uh there's a lot of potential for companies to get a lot of new value out of it because this was never before

possible before generative AI got good enough and before AI agents got good enough to actually uh uh start to to to to automate some things and help and help employees do their work faster. So

um when you're looking at like auditing these, you basically say what systems do I make available and are these platforms that we have internally able to uh be used with AI? Either of those tools come

with uh the ability to say we do AI ourselves to help you or uh they have access uh openly to another system that allows uh it to be accessed by AI agents. And you typically see that like

agents. And you typically see that like some of the more modern platforms have the ability to do to do both of these where they support things like AI agents themselves but then their AI agents also work together the MCP or ADA protocols

which are sort of a new emerging ecosystem of the way that these platforms work together. Um but then in addition to that centralized IT model u in many cases one of the key things is that when you're looking across your

company and you get either uh employees who are looking to say like I wish this this this sort of um thing that we do internally was faster or I wish it was uh automated or I could use some help in

this area with AI then just looking at okay well where is that data stored so um we talked to some customers and and um some of the challenges that they're facing are things that like uh uh like we're talking to some financial

organizations and they We still struggle with the idea that people are sending in scanned uh documents when they're doing things like loan origination or if you're doing like uh like insurance people have uh they fill out a lot of

forms and then they send them in. And it

used to require that that people would have to look through it, transcribe things themselves, be able to like go through understand and sort of uh like find the key aspects of whatever that person submitted. And these were the

person submitted. And these were the things that were taking a very long time like in in something like like a standard process that people care about like how long does it take me to get my loan approved. A lot of that might be

loan approved. A lot of that might be this manual time that somebody was doing kind of like trying to cross reference like what was um in this document versus what was in another one. And those kind of processes are are are right that AI

can help with a lot and usually in a way that the person who is who does it today would say I I I really wish AI could help me there. So things like data extraction and things like being able to

understand the intent of like uh uh some of these like uh more free form um uh uh uh uh uh different submissions that people uh would send in in all these different kind of use cases. Um and and

so in this world this is where AI can start to help you. So then again the very first question is well where is that data stored and does AI have access to it?

>> And Ben is that a relatively can that be a relatively straightforward process?

Obviously there is complexity embedded in lots of the companies that you work with, lots of the people you partner with. Is it quite straightforward? Is

with. Is it quite straightforward? Is

there a playbook that companies can use?

And can you just just ex explain that whether this is a simple exercise this audit or whether that is actually quite a demanding part of the process before

you start using actual AI products? I I

think what we hear from many of our customers is that it's conceptually quite simple especially the sort of the centralized approach. So you basically

centralized approach. So you basically say do I have a platform to store these different and critical data for my company and and if not get one then there's plenty of great options out there. Um at Box we we we sort of

there. Um at Box we we we sort of specialize in unstructured data. So um

happy to tell you about uh that if um uh for customers who are interested. Um but

uh at a at a high level um uh the challenge has been um that over many many years um this getting your data

into uh uh a centralized system uh any of these types of data um in making that uh and in enabling it for your employees has been a challenge and and so different companies for different reasons have approached it in the past.

They've done it for things like um uh for for uh to make make uh to make make your data available via mobile devices.

That was a major trend to make your data available in the cloud so you could access it anywhere. To make your data available um in in in in a system so that you can um uh sync it to your local computer in addition to your mobile device in addition to um other people

and collaborate in external collaboration. These kind of things have

collaboration. These kind of things have driven people to do this over time. But

not every company has has done it yet.

and they've said, "Well, maybe I'm too busy or maybe it's like too complic complicated as a lift uh to to make that work right now." And in in in these cases then um the uh they've kind of

postponed this idea of of this sort of uh uh upgrading their platforms of of of their of their company. And and I think the challenge is that for many companies is that they have to now go do that in a

way that sometimes it changes behavior of something where people were used to the old way of doing things. um or it it brings up some new security or compliance concerns that they sort of have to visit and and understand and make sure that when they're moving to

these new systems that they meet all their needs for the of of um you know many different stakeholders that are quite um uh important. Again, security

and compliance concerns are not something that you typically would just ignore. So this is usually where the

ignore. So this is usually where the challenge is is that uh it's like a new way of doing things and there's like a change management aspect to it and that is usually where you start to see the first set of challenges. So the playbook

is straightforward. um in terms of at a

is straightforward. um in terms of at a high level you say well let's get a better systems lots of people run these systems in many organizations but the challenge would be making it work in your organization and that's usually um

I I think many companies have seen that if they put this off for a long time it's time for them to revisit to revisit the legacy systems to revisit the old way of doing things to basically prepare um not just for the current benefit of

being able to use like AI in its current form but the expectation is as the AI gets as AI gets better as AI agents do more and more complex things then they're able to then um uh lay down the foundation to be able to uh respond at

with all of the the great innovations that are happening today.

>> I was at a very interesting um conference. I was speaking at a

conference. I was speaking at a fascinating conference Ben just before uh Christmas um where the company global company um deals with customer relations

enterprise products had put all of its um not only senior team but middle management teams through a process of AI

intelligence so that they understood the products that they were working with.

they had some degree of um AI education and that was speaking to one middle manager. She said it was so important

manager. She said it was so important for the cultural change that people felt they understood what they were being asked to do and this whole idea of lifelong learning within your business

was a really important part of adopting AI um uh in a way that was as positive as possible. Are there Ben for you any

as possible. Are there Ben for you any red flags when you maybe are working with a partner in this first stage which

as you say is sort of vital foundational principle where you would say you're not ready yet to move. Certainly the uh the

aspect of making sure that you have um your uh uh at least one sort of interesting target use case that you have available to uh that you want to start with. Um and in addition to what

start with. Um and in addition to what we talked about before which having your data like available in in a in a in a safe and secure uh way. Um but this is usually like kind of like where that you

start to say um do do you have something inside your company that would be interesting and usually just simple use cases in many uh is is a great start both from the person's perspective like

the people in your company as they get used to it in addition to the idea of um trying to have like a measurable value benefit there. So um like uh sometimes

benefit there. So um like uh sometimes when people hear about like oh AI is going to revolutionize things they shoot very high. they say like I'm going to

very high. they say like I'm going to pick the most complex the most valuable thing in my organization and I'm going to change it all and we're going to have it done soon. Um and then and then um usually that's um like with any project

AI or not like that's always a challenge is to do the most complex things. So um

what we've seen for for many um organizations and with many of of our customers is that if they start with AI on relatively uh straightforward things um uh they they get their people ready to use it because there there's like a

mechanism where as you work with AI people have kind of found this in their in their in their daily lives as they use things like chatbt or Gemini or claude or others is that there's like a way to use it um that is not completely

obvious before you use it but as you start to understand um and as you start to realize that you talk to these the these agent as you start to realize like the way that it has access to your data and then where it doesn't like those

kind of things people get used to so that they're able to then say ah I can see that like my original idea for how this worked is maybe like I can tweak it a little bit or I can realize that like

the limitations of the systems or the models today and then and then basically be able to build on top of that. So um

one of the sort of uh have big ambitions but start small is definitely one of the things that uh uh you is is typically a focus area for people that we see uh who

are more successful in their projects.

>> We'll go into that start small um part of our discussion uh just in a moment but I just wanted to finish off this sort of first foundational sort of chapter of a of a of a of a company or

an organization's journey into successful use of AI.

>> Yeah. this idea of the single source of truth. Yeah. That you have to have this

truth. Yeah. That you have to have this sort of foundational sought just just just talk me through how a how an organization gets itself to a single source of truth.

>> Yeah. So I mean so for for me like you start to look at uh what are the platforms of the data that you care about the most and and and and I'll sort of point to a few um of like the kind of common areas structured data

unstructured data sort of generic type of type in your companies. uh CRM data, HR data um and uh like these kind of things where you keep your customer data, your employee data, your just

generic things that people work on like u inside of company that's just your first question is uh where do we store that data um and then it is that data readily available to AI either because

those systems have AI that they have uh uh available built in which is common these days um or it is um it has uh open access via API via MCP server via other

things so that an agent can quickly uh be able to access it like starting there is is is is um usually the the the first question and then and sometimes it's actually even hard to answer often times many companies have many versions of

these things. So then the question is

these things. So then the question is okay uh does it mean that we should consolidate it into a single area for some of these or um uh like kind of in this in this this idea that we'll talk

about like starting small. Usually you

don't need to say okay well step one is to get rid of all old systems and move everything but just uh but but instead say let's focus on the things that are sort of the most important especially if

you have an idea of the early ways that you kind of uh something that that would be um approachable uh relatively small but valuable um so that you could then um use that as like your test case and

that often means moving to um data out of old systems into new systems and then that is sort of like one of the key first steps So you get to this at least at least in

part of your business this single source of truth so you can start. Now you've

touched on governance and access control. There's this great uh phrase

control. There's this great uh phrase you've used it at box or use it in your podcast. I've also heard it elsewhere.

podcast. I've also heard it elsewhere.

AI doesn't keep secrets. Yes.

>> If you ask AI something, it will tell you everything it knows about that thing whether or not you should have access to that data or not.

>> Yes. Now Ben, how on earth do we ensure as an organization that wants to be responsible as we do in these spaces that the governance structures are put

in place? So so that inappropriately

in place? So so that inappropriately data is not being shared between different teams that should not be seeing uh material that should be

protected. But we are still allowing

protected. But we are still allowing data search in unstructured places particularly where this is a particular problem to be fluid efficient and

actually improving the performance for the company for its customers.

>> Yes. But yeah, and then then the challenge you bring up is definitely something that is kind of um uh for people who start down the AI journey like a common way they're like okay I'm

going to take you know data oftentimes it's like a bunch of unstructured data um and then they say if I just make uh it available to AI um then it can start to have all these great benefits um it

uses things like retrieve augmented generation is one of these terms where the AI can kind of read a bunch of data pull out the key things and answer questions and start to do work for you on in this data and so this is like the

kind of the new and amazing thing that AI offers. Um but then your very first

AI offers. Um but then your very first challenge is what you said which is like well wait a minute like uh what is that AI going to tell somebody and and like in in any organization like almost no

two people have access to the same data.

this is kind of the fundamental thing that AI makes you kind of like everybody knows it but you kind of remember like wait a minute I when I'm working I have my own things that I'm working on or uh and then and then other people have access to special projects and that

other people don't have access to and so this is the key is that if you give AI access to all of this data and you're not careful about it the AI will happily look through it all and tell you uh

whatever it it um it knows and it is um uh the AI is designed to be very helpful and and and it has no sort of sense of your permission structure by itself like

AI models don't keep secrets like you say like if they if they have access to it and you ask them this question then they'll be like okay well I'm going to answer this question for you so it's absolutely critical that you do not give

AI access to what that person doesn't have access to AI as a system should have access to a lot of things because your employees have access to a lot of things but when that person is interacting with it you have to be very

aware of their role their access controls and everything about that user um and so that it only looks up things that they they have access to because you can imagine if you're um a manager

or an HR and you say I want to have um looking through employee salaries because maybe you're looking at ways to give you know like uh annual bonuses and things like that and you want AI to help you kind of like distribute it like it's

very different from somebody in else in the or who says show me all everybody's uh salaries like and so that kind of control typically is built into these underlying platforms that we me that we

talked about um the idea of user based access controls the idea of rolebased access controls the idea of like when you interact with most of these systems you it's they inherently have these permissions in this set of data access controls built in and this is usually

why one of the recommendations is for most organizations is to say don't try to start over with your own customuilt platform to then store all your data uh

of all these types to then apply um AI on top of it instead uh like many of these organizations have provided uh AI capabilities and then the if you're going to build something custom if

you're if you're going to be able to uh uh uh create these sort of um internal automations and systems like you can basically like piece together using all of your current um systems the ability

to have AI like reach into those but but with um a very strong awareness of those access controls and this is usually like one of the first things that after you get the data consolidated where you have

to solve this like like you there's there's um uh no organization or any then certainly no security or or anywhere would ever allow sort of unfettered access to data within an organization uh that doesn't have these

controls. So this is just kind of uh

controls. So this is just kind of uh like like uh you this this is the first starting thing before you can meaningfully use AI with your employees and and when accessing some of the more sensitive data which of course often

then is some of the more valuable things uh that AI can help you with? And are uh partners like Box able to plug in to the

systems of your clients and enable that type of work to be done? So the con the consolidation piece where necessary but also the governance piece can they come together?

>> Yes. Um and and I think um this is where I mean there's a lot of great like systems and platforms in the world that that and they sort of specialize in exactly this for enterprises um is the idea of being able to uh say that like

um they control the the access uh so you know you call it the system of of truth or the idea of of the system of record.

Um so or or just the idea of a uh internally to a company everybody has a set of things that are approved tools to work with that have met all of the security and compliance uh rules

internally and and then and then once you have those they they inherently are designed to basically control these kind of things. access controls uh uh being

of things. access controls uh uh being able to know that oh you when you're working on your data that's yours when you're collaborating it with a with a group that's uh uh everybody has access and then and then again controls for

things like oh uh maybe that person's uh done working on this so we remove access and and in all these cases what you want to happen is that the moment that that the system um like the way the people

work the the the same way that you when you add or remove access you want to to have that same capability with AI you want to be able to say like I want AI to then start to work on this either because I'm looking at it and I'm not

authorized to work on it or in some cases you actually can collaborate in AI agents to say I want the AI now to help me with a project. I want it to look through all of this project data so that it can help me create a proposal. I want

to look through all of my uh financial data so they can help me then create the review that I need for my annual planning. And then those kind of things

planning. And then those kind of things are where you need to give AI access to your data. Um, and again, you do it in a

your data. Um, and again, you do it in a way uh that's either uh because I'm using it um I'm I'm just asking AI to help me uh on my behalf with my privileges or you ask it to say like it's almost like you're inviting a

person, you're inviting an AI agent to then have access to these things. And so

these are the kind of techniques in that most uh platforms are putting together uh so that they can like better use AI and solve this fundamental problem.

>> Now we've we've said Ben, start small.

Don't try and boil the ocean at the start. You know, start on a little, you

start. You know, start on a little, you know, peer head that you've got an issue.

>> Maybe picking a bottleneck, something that you know is where you're, you know, maybe losing some time or losing some customer service.

>> Yeah.

>> Ben, can you take us through a why start small? Just remind us again about that.

small? Just remind us again about that.

But B, is there a case that comes to mind where you've supported uh an organization and you've been able to explain what small looks like?

>> Yeah. So, um I was I was talking to a customer recently and as a financial organization and they um and they do a lot of uh uh working with clients uh on

uh like uh their um uh uh portfolios of of of their of their client data, all the different accounts the clients have.

and they wanted to get to the point where they had uh AI helping their adviserss to basically give really good like financial recommendations like at a holistic level to their clients and they thought AI could help you know prepare

reports be able to bring the latest information to their clients and they said okay so they had this big goal and I'm always a fan of of big aspirations but then um they're like okay so how can

we then do this and then they they you start to look through it you start to realize that like that um there was really no way for um the uh even people in the organization to very quickly

understand what the clients were doing.

The client had submitted all this data, right? They had their um their financial

right? They had their um their financial uh uh uh statements and things that the their adviser had asked for. Um but but it it required the adviser actually reading all the data to then give them this recommendation. So in the star

this recommendation. So in the star small world, they said, "Well, why don't we start by uh taking that information and then sort of putting it into a standard uh sort of uh report format?"

And it turned out that this was actually like without um uh like before they had thought like the hard part was going to be to get it to create this report. But

it turned out that the hard part was actually starting with was to be able to um understand this client's situation given this variety of data you know again like financial statements, stock

statements and so on. And so the first step that they did, the sort of start small was to say, can we uh have AI help us understand this data first to be able

to put it into a a format so that we can then draw these conclusions from it. And

and this uh so this idea of data extraction is usually like a problem that many organizations have and then um and then uh uh that basically taking structured data from unstructured data and this is something where um often

times starting there uh is is is a great a great step. So um in this organization like uh you know depending on who you talk to they they would say wait a minute I thought our big goal was to get this thing available for our clients but

it turned out that most of the time and most of the challenges were actually at a step much lower and so this idea of starting small where they're saying okay our step one will be to structure some of this data is uh starts to expose some

of the questions like where's the data stored how can we handle this in a secure and compliant way are we making sure that we have the right protections in place and so on so that they get that and then from there they can then build on top of that all the other things that

they were looking to um achieve. But if

they just started by saying I my my number one and only goal will be to get it he this this thing created then often times they would they would essentially um not be able to achieve that because they haven't laid the proper foundation.

So starting small with a sort of the um a simple use case but then also one that's kind of lower in the in the uh stack to make sure that you sort of have the foundational capabilities like that is often something that we see helps and

and and along the way they gained a ton of experience right as they're starting to work through this they're starting to realize what the AI's limitations they're starting to realize they can do things that they didn't really quite realize is possible like look for risky sort of uh things that are happening uh

these things that typically require a lot of intelligence and so this is where the learning of the people doing it for that particular use case and that particular system at that particular company um is usually is is something

where um um we see people who approach it this way. They're getting some uh uh uh they're starting to have more successful projects than just the normal do something big and do it quickly kind

of approach. Listening to you, Ben, I'm

of approach. Listening to you, Ben, I'm I'm really struck. I had a conversation with a a CTO at a a big global um technology firm earlier today and he was

saying that lots of organizations outside um the technology world don't really know how to manage innovation within their business that they basically do what they did yesterday

hopefully more productiv more productively and therefore create better bottom lines and so again I'm I'm leaning back into the idea that culture also really matters when you're

considering how to launch successful uh projects of any description to be honest, but entrepreneurial projects

where AI will be in the lead. How do you value a pilot in terms of is it 3 months, is it 6 months, is it a year?

Obviously, there'll be lots and lots of different projects or different time scales to different projects, but what are you looking for? And how do you on

your side box help people understand whether they've actually got something you you don't want to keep sort of flogging a dying horse and say, "No, no, no, I think this is going to work. I think it's going to

And I think a lot of businesses start open-ended, a little vague, maybe haven't done the fin foundational work that we've been talking about, and then don't know when

to stop.

>> Yeah, I I I think um this is a a a common challenge uh like I I mean probably in general with people, but then also with with these AI projects that often have a lot of attention and

they have a lot of uh sort of um uh uh emphasis behind them. Um so a couple things I think uh well hopefully just in general best practices but but in particular for AI um we the idea of

being able to go in and say that you have very strong exit criteria and something that you believe to be achievable um and then you have a directly responsible person who's able to do that. So what we've seen works

quite well is um uh like when you like uh in our in in our organization at box and then and then similar with companies we've seen um like we did this approach where we said okay we want to give

access to to some of our new AI capabilities to people internally um and then we wanted to then uh uh do like like a hackathon to kind of see what's possible um in sort of a like a proof of

concept sort of approach um and then we had envisioned that we would then have like these more senior people come and say like okay like you know this this vice president director was going to be responsible for doing this like massive

thing. But what we learned in this

thing. But what we learned in this process was um actually some of the people who were closer to the problems some of the people who were sort of at the manager level um in different

departments in procurement and finance and HR and other areas they got very um uh uh familiar with the way the AI worked. They got they became the

worked. They got they became the champion of it. They became familiar with the details. And so interestingly we we quickly switched to the idea of instead of saying we're going to like you know drive this big efficiency number over here. We said actually we

know that if you kind of think about it a little bit differently from some of the sort of the proof of concept work is that automating this internal audit step or this procurement step or things like that from the person who's responsible

for it who's driving that project who who has become familiar with the details and who knows very well what it means to be successful or not. like let's have that be the initial goal with the

initial um set of exit criteria. So it

was and and then from and that person was very clear on it. They're like today I spend 10 hours a doing this thing. My

team hates it. This is the work that nobody wants to do. It often is delayed and I'm going to have AI start to do that. So then take it down from 10 hours

that. So then take it down from 10 hours to 20 minutes. Um and in that case maybe like reviewing a bunch of material for audit and preparing a report that kind of thing. um like that became then

of thing. um like that became then something that that that that uh person was responsible for. They had very clear exit criteria which was to get it working, get it to working at a a level of quality that that that they're um

that they could then use. Um and then and then that became the sort of the the uh way that they approached that project. Um and then that that was

project. Um and then that that was critical compared to let's say that we had done it the other way that we thought was the right way which was like okay like um like seeing this this metric at a high level that we use as a company internal metric that needs to

move in the short term because like like in reality what would happen is like some of the metrics that are high level they're they're kind of more trail uh you know you don't see their their change for a while. So it's almost like it requires 10 of these projects to go

through before you see some of the bigger benefit. But but um uh I'd say in

bigger benefit. But but um uh I'd say in general this is the key is to make sure that you have directly responsible people who have uh clear exit criteria and a clear goal that it will actually matter to them and then those add up

very quickly in an organization to becoming more and more uh useful especially as we're talking about like internal uh like like uh uh productivity style of getting more business work done

internally. So I this approach uh I

internally. So I this approach uh I think of like almost pushing down the problem to people who are closer is is is a big part of the challenge. Ben,

let's move on to step four, which we've described as pushing beyond chat. Um,

such was the sort of consumer revolution of chat GPT when it first became really publicly available and suddenly the general public for oh

my goodness there's this thing called artificial intelligence which of course had been running for decades in many various forms and this was simply one part of the artificial artificial

intelligence products suite but nevertheless everyone became hyperfocused on chat functional ity on bots on okay how can I

speak to my customers whoever they may be um via the chat function how do you help people that doesn't mean that that is not important by the way that that

could be what you want to do I I I I saw a presentation by a big jewelry firm and they wanted their um online uh chat function to feel very much like their

physical shop experience y >> and so they started training their chat chat bots on their actual uh shop um uh assistants, the retail people in their

shops who obviously had had a way of of describing the jewelry etc etc. And that seemed to me to be very very smart because then when you got online you had the same experience in digital as you would in real life.

>> So it's not to say that chat is not uh a very important part of uh the functionality of AI and how it may help.

But how do you help businesses push themselves beyond the oh yeah we can get a chatbot out of this?

>> Yeah. So I I think the there interesting part of this like chatbot question. So

so and like you said like um the chatting with an intelligent system is actually I think one of the interesting revolutions that has happened over the last few years. When it first started it

seemed quite weird to talk to um to like an AI system that was like that was that was different for people. But now it's become very common. and it's become like almost like a new uh interface. In fact,

um the uh people in the industry, they'll uh they'll often call it the AX, the uh agentic experience. And so like and this is, you know, this is not a term that that existed all that long ago, but now many companies including

Box has an AX, an agentic experience overall. So one of the keys is that

overall. So one of the keys is that talking to an intelligent system is actually becoming a more sort of uh uh common way that people are interacting in their personal lives in addition to to at work. And so it's not that the

chat itself like you were saying is is is problematic, but it is like the idea of simple chats is you got to kind of move beyond that. And so like in the early days of something like chat GPT like you would only talk to the model

about what it kind of had access to. Um

in in sorry what it was trained upon previously. Um and then and then and

previously. Um and then and then and then so in that world you just kind of ask it a general question and it would kind of give you a general answer and it was great but it sort of was quickly limited. And um and so like in the

limited. And um and so like in the example of this this company that uh you referred to like uh it it probably is very very valuable for them to be able to have like a model that knows about

their their um internal jewelry systems and so on or all all their products but like they um but one of the next steps would be that having it to be able to do things like um helping people who uh uh

take take orders um is so somebody says I want to potentially buy something can you start that transaction um or or going um in and and then moving out to like let's say like uh things inside of a company. Um, many times what you want

a company. Um, many times what you want is not just to have an AI respond to you, but you want to have AI start to do work for you. And this is usually where we get in the world of talking about things like AI agents. And so, um, uh,

uh, usually the chat experiences where you're talking to an AI agent is almost like talking to somebody in your organization who then has like a job, it have a role, it has a function, it has

access to certain systems. And that's um I think where uh being able to then have uh AI agents that you have inside of your organization started to either uh customize or develop or use from a

platform to that basically say I can do some specific work for you and then being able to think of it this way. So

you say like I wish that AI could help me with this task of um using the previous one like like uh going through this set of financial data so that I can understand my customer better. Well, if

you think of that like I wish I had an AI agent that could do that and then you can actually uh that's the kind of the the the capabilities that are available today. So you can make an AI agent who

today. So you can make an AI agent who has an objective who has understands how to go look at that data has access it can go under it understands the general output it understands the kind of format the person the adviser is looking for.

So making an AI agent to be able to do that. Uh you could then chat with it and

that. Uh you could then chat with it and then ask questions and say like okay I I want to know uh for I'm going to have this meeting with this this customer soon. Um so therefore I uh I had this I

soon. Um so therefore I uh I had this I can't remember this detail and the AI agent will answer it for you or I wanted to prepare this report. It'll do that for you. Um and and and so um the idea

for you. Um and and and so um the idea of these agents that have access to this is is one of the key things. But then um quickly as you go down this route you're like well wait a minute like if I'm going to do this every time like do I

really need to go always ask that agent something? Do I need always need to chat

something? Do I need always need to chat with it? Do I always need to make it do

with it? Do I always need to make it do things? And this is where the world of

things? And this is where the world of having AI agents not only available to talk to you, but also available in the background to do work for you. So why

not have it such that it knows that I'm going to have a meeting with this customer and then it basically will go pre-prepare the work and sort of a workflow. So it'll realize that like um

workflow. So it'll realize that like um uh based on integration with whatever system, it'll say like I see that like you know on Tuesday you have a uh one of these types of uh meetings. I will go

spend and it can take it a while maybe like minutes or or or um some cases hours of work to go through process all the data understand it all look at it sort of um the the newer agents kind of go on these loops where they're trying

to fill this objective and then they're able to say um here you go uh before your meeting I've done this work for you because asynchronously in the background using some sort of a workflow system like usually built on these platforms I

have done this work for you. So this is the world usually where it's not about the necessarily the the chat interface being um sort of uh bad but it's more about like why not have AI work more for you in this agentic form because it's

they're capable these days of doing things that are more complex. We move

past the world where the agent will give you an answer in 10 seconds or where the AI will respond in 10 seconds and we move much more towards the agents that think that reason that loop and then they're able to then have access to

tools to go and be able to accomplish these more complex things.

>> Finally, someone who can uh explain agentic in a way that anyone can understand. Thank you for that. Ben, um

understand. Thank you for that. Ben, um

give us an example uh pushing pushing a an organization on a bid. this idea that you might think it can help in this quite small way, but actually think 5x bigger,

>> think 10x bigger of what it could really do because it's that odd world. Um, a

friend of mine in in AI uh venture capital, you know, she says to me, one of the biggest problems is people understanding you don't know what you don't know and therefore you don't

always know what AI could do for you because it's a it's a whole different >> Yeah. paradigm at times. Yes.

>> Yeah. paradigm at times. Yes.

>> How do you support organizations? Have

you got an example of where AI can deliver 5x improvement, 10x improvement?

>> Yeah. Um so I I think um uh the the the phenomenon that you refer to is is is is quite uh not only challenging by itself uh but is actually extra challenging

because everything keeps changing like like uh I think in the last uh uh 12 months um there has been something like 15 major model releases across these

like different vendors they have these different names um and um uh you know open chatbt gem uh Google's gemini anthropic Claude and each of these

arguably was the the the most intelligent piece of software ever created in the history of of of of humanity, right? But like most people

humanity, right? But like most people don't even know the names of these or the versions of these like Opus 45, they kind of named in this funny way. Um and

so like uh it's hard to keep up with the changes. However, at some point the

changes. However, at some point the thing that you were familiar with a year ago or maybe 6 months ago or or two years ago, depending on on how much time you spend with it, it has dramatically changed as possible. So oftentimes

people would go back and they would say like, "Ah, yeah, I know about AI. I've

kind of tried it. I played with it." Not

realizing that it has dramatically changed. And it has this kind of almost

changed. And it has this kind of almost like a bold frog aspect where it's like getting better and better and better, but people aren't necessarily realizing it because um they you know, it's it's just a lot of uh information to consume in the world to see how well they're

doing things. And certainly it takes a

doing things. And certainly it takes a lot of time to go test all this stuff.

Um and so this um you usually what uh like uh in in the um one of the major areas that we've seen in in the last 12 months has been uh this mentioned before

in the idea of of software engineering is that there was like when people first started to see that the AI can write uh code for you. It was this revelation of like whoa that's crazy. I can't believe

if I write a comment at the top it'll be able to to decide based on this to be like I can write that for you. Then that

was like that's amazing. Um and that and that worked well and but it was it was somewhat limited. Um but then then when

somewhat limited. Um but then then when um the new coding tools came out with this idea of agentic coding where basically you cannot just say help me complete this function but help me write some major new aspect help me go find

this bug help me go figure out like the the way that we make this substantial change and the AI would it wouldn't return right away. It wouldn't help you inline. It would go think for a long

inline. It would go think for a long time. It would process, it would come

time. It would process, it would come back, sometimes taking minutes or or hours. Like this was a new paradigm of

hours. Like this was a new paradigm of people of like how to think about this overall. And this was like dramatically

overall. And this was like dramatically different. And like many many um

different. And like many many um software engineers, some of the better ones who are familiar with the system, they'll be able to come in the morning, they'll be able to kick off many of these different things. Do this work, do this work, do this work for me. And then

when AI agent comes back and does some great things, another AI agent comes back and it's able to uh maybe didn't do it quite well. So you kind of like ask it to try again. Um but this idea of managing agents going forward is actually one of the critical uh uh sort

of insights but it's very hard for people to understand it until you actually sit down and use it and realize that you you sort of have to get good about the way that you're managing these agents. And so I think this same qu this

agents. And so I think this same qu this same sort of approach is is going to be something that everybody's going to deal with. people are going to deal with

with. people are going to deal with their organiz and and in their jobs, organizations will have to deal with which is that you have to start to say that the key will be for me to sit down

and try things to experiment with it. Uh

like uh what is it what is possible now with the latest models with the latest platforms with way that companies are integrating their data into this into these AI systems. Um and and uh and then at some point you realize I can't

believe that it's able to do this stuff which is might be different from what you thought is what you want thought you wanted. So, so like I think this is the

wanted. So, so like I think this is the revelation that we're going to see in 2026 and beyond, which is that people basically saying, I wonder if AI can do this job for me and help it and then

getting back sometimes amazing results, sometimes not that that great, but then maybe again trying a little bit later as like, you know, the new model came out or a new capability or or like uh like the the software that you're most

familiar with and has an update that like uses AI better. like those kind of things I believe will be the way by which people will start to say I'm understanding better what I can do and I'm starting to use it more mostly

because I tried it and I'm familiar with this new paradigm I'm familiar with the agentic experience I'm familiar with the way you talk to these things the way that you that after you get it to work one time to then automate it to then

work in the background so this is the kind of uh the sort of evolution of using this technology that I think we'll start to see >> I love the idea of of of AI being a

constant process as well as a degree of decision-m. I think that's a really

decision-m. I think that's a really useful lesson to take. Let's just go on then finally to step five which is about measurement which I think you've touched on actually Ben regularly through this

great conversation about measuring what you're doing and then and then rethinking and testing again and and improvement with constant backwards and forwards with um the AI products you're

working with. But you've gone through

working with. But you've gone through these four great steps that we've outlined. and you're at this measurement

outlined. and you're at this measurement stage where you're trying to push and maintain momentum and as you say not forget that the world's improving out

there in this area every single day of the year. How do you then um embed

the year. How do you then um embed measurement and understanding within your organization?

So yeah, I think so so in general like it's always critical to make sure that you have an ROI on something that you're you're you're looking at like it is um because we talk so much across the

industry about technology and the capabilities associated with it sometimes like like um some people would just be like I'm using AI therefore good but that's probably not right probably

it's it's you want to make sure that you actually are delivering business value and then usually a mechanism to measure that is it is critical so that you can have these KPIs or these OKR style of like what is the result that you

actually are looking for. Um but but from what I've seen for people who are who are sort of good at this in addition to um the way that that I'd recommend is that rather than start with a very high

level KPI or or or sort of a company goal and say we're going to move this um instead say like um uh what is the most important business value but then what

is a sort of more immediate and direct metric that we can then measure that we know ladders up to that. Um so for instance let's say that you're you're interested in efficiency overall for your organization and like like uh one

of the things you you might have some efficiency measure at the top level but but that might be the very holistic big uh measurement that is hard to move or it's trailing it doesn't change for a long time and so instead you say well like some examples we talked about

earlier is you're like can I make this one process more efficient and then can I uh make it go from you know like 10 hours to one that kind of of of thing

and then and so um somebody who's maybe more sort of uh skeptical might say wait a minute that's that one process going down and being 10 times faster doesn't

change this high level metric immediately which I think is is a trap uh if you and and and you need to make sure that you have many different processes that are all becoming faster hopefully 10x faster so that they all

then add up and then this trailing metric that you might be looking at up here is something that is uh will improve over time and so basically starting smaller starting on things that matter but but um uh making sure that

you're moving those metrics so that they they then um and and then and then uh you believe over time that then that that uh if you do this repeatedly you'll be able to move these higher level metrics. That's sort of uh one of the

metrics. That's sort of uh one of the key sort of management and sort of uh approaches from people who are are successfully adopting this.

>> Thanks so much Ben. Look, we've gone through a fantastic list of how to prepare yourself for successful use of

artificial intelligence products. that's

all about the audit um the single source of truth getting the right governance and operational uh controls in place.

The idea I think really really important of starting small. I I love the the thought you had or the point you made Ben around you can still have a

transformational sort of topline um uh vision of where you want to be but you started in small places and actually you will learn a lot from that about whether that topline vision actually works. I

thought that was uh fascinating. This

idea of pushing beyond Q&A and I think um actually understanding Agentic now is much higher for me having listened to your explanation of how Aentic AI can uh

support that and I think measuring value and it's not just cost or productivity but it's actually true value for your business and for your business's customers which in the end is who you

want to serve in the best way possible.

just thinking about the sort of the next 3 to 6 months. I'm a I'm a CEO of of a company. I or I'm a chair of a of a

company. I or I'm a chair of a of a board or I'm on the leadership team.

What what decisions should I be making in the next 3 months that would really set me on the track to being successful as a company with

integrated AI in my business?

>> Yeah. Um so I I think that one of the keys for these organizations is um as we mentioned is are you do you have the foundation enabled for AI overall and and hopefully many organizations be like yes I've been looking at this for many

years we've started to put this in place or we have it in place now that our data the things that are very critical to our organization are available to AI. Um now

I know many organizations are still working on this. So it's like um you know for those who are haven't done this yet you know there's there's um you're not alone. Um but that so getting the

not alone. Um but that so getting the foundations so that data is available for AI and then one of the keys is then to say how can I have AI specifically and typically in the form of these AI agents that have these objectives and

these goals uh and then able to then often access multiple uh of these of these uh systems often times you have an AI agent who's going to try to accomplish something and just like a person you need to use different types

of data and so then the AI agents will then talk to other AI agents of these systems to be able to then go uh uh do something more complex. So this idea of

an ecosystem of AI agents that all sort of specialize in their area but then could come together to then accomplish a bigger task that I think is the world of

of um in 2026 and and beyond where you start to have uh AI agents that can do more and thus provide the more of this value. And so um sort of focusing on

value. And so um sort of focusing on that I've got my data, I've got it available AI, but now I need AI agents.

And then how do they cooperate then to then accomplish more things? That's kind

of the next generation of challenges that many platforms, many software companies like like Box, we work in this anic ecosystem are are um are are providing. Um and then now it's time for

providing. Um and then now it's time for organizations to start to see how they can use that to then solve all the kind of challenges that that they're interested in.

>> Ben, I've got a final question, of course. Uh, when will Agentic AI be a

course. Uh, when will Agentic AI be a better podcaster than either you or me?

>> Um, I uh uh I think uh well uh I I in some ways uh you know if if if we had AI agents that could help uh uh us be able to reach more people like and with some of the thoughts that we have. I think

that would be great. Um but yeah, it'll probably be a while before uh AI would replace that.

>> Look, Benus, that's the perfect place to wrap up today's discussion. And it's

clear that AI ready companies aren't the ones, as we've discussed, with the best models or the most expensive tools.

They're the ones with the best foundations. It's about understanding

foundations. It's about understanding your data, breaking down silos, experimenting with purpose, scaling intelligently, and measuring the value that really matters. Thank you so much,

Ben, for joining us for this fascinating conversation, part two of our podcast series with Box. Thanks to you, of course, uh, for listening. This podcast

was brought to you by Intelligence Squared in collaboration with Box.