Understanding reasoning models: Smarter AI for complex problems | Box AI Explainer Series EP 6

Today we're not just talking AI models, we're talking reasoning models. Welcome

to another episode of Enterprise AI explainer series. I'm here with Ben

explainer series. I'm here with Ben Coup, CTO of Box. My name is Mina Ganesh, senior product marketing manager for AI at Box. So Ben, let's start with the basics. What are reasoning models?

the basics. What are reasoning models?

Yeah, so reasoning models um you can think of as like a new class of models that have come out from multiple vendors. Um and they are sort of

vendors. Um and they are sort of characterized by the idea that they think more. they reason more about uh

think more. they reason more about uh what you're prompting it to to to do.

I thought yeah I was already smart.

Yeah. So the compared to like a traditional model like the a traditional model typically will respond to you very quickly no matter what and it has sort of a process in just like one shot. Um

it'll just it'll it'll respond. And so

these new reasoning models which you see things like like 03 mini 04 um mini from from OpenAI Gemini has the ability to do thinking models now um from Google

Anthropic as thinking models and others and and they have this uh sense of that you can ask them to think more about their answer. Mhm.

their answer. Mhm.

So in many ways this is kind of similar to I think the way that people work and it's so it's a one way to understand the thinking model difference is to kind of understand like how people operate in a normal sort of context and some people call this system one or system two

thinking if you you're into such things.

Um so here's an example. Here's a

question for you.

Um quiz.

Yes. Uh okay. Uh

ready? Uh you have to answer right away.

Um okay.

What was our last episode we did here and and tell me the key points about it.

Go. Uh last episode that we did here was about unstructured data and the key points where yeah uh quality is important and so is security and uh hallucinations are bad

and unstructured data is important and I need to refer to our episode.

Yeah. Okay. So um not not not bad. Uh

and so you you clearly knew your topic.

You clearly understood the different pieces. So is that a perfect answer

pieces. So is that a perfect answer though?

Um no. I would want to revise that at least two times and come back to you.

So if I prompted you to say, okay, let's focus on the value of unstructured data.

I want you to think about these things in here in the past and and maybe like you could even like try to understand, well, who's the audience that we're talking to and and and so on. Like so if I prompted you with those things and I asked you to try again, you probably give a better answer.

Yeah.

Or if I gave you more time, you might have just sat down and thought about it for a minute or That's true. I did when Yeah. You were

That's true. I did when Yeah. You were

like, "Right now, I I did. I panicked a little bit. I'll admit."

little bit. I'll admit."

Yeah. So, so this is a good example of the difference between a regular model which just has to answer immediately that it's just built in. It just you see it typing. It won't stop. Yeah. Compared

it typing. It won't stop. Yeah. Compared

to a thinking model which you'll see it actually will tell it'll you can watch it. It like makes a plan. It thinks

it. It like makes a plan. It thinks

about it. It does its own thought process and then revises it. And so

these thinking models are give you typically much more uh quality um answers when and and and well thought through. Mhm. So it sounds like

through. Mhm. So it sounds like especially when we talk about AI in the enterprise context, enterprises should always use reasoning models. Would you

say that that's a true statement?

Um I think that um there's a benefit to them. But remember, it comes at the cost

them. But remember, it comes at the cost of time. So for instance, if I go back

of time. So for instance, if I go back to our to our example here and I'm like, hey, um can you tell me about what the last episode is? And if you're like, well, I'm not going to answer. I'm going

to think about it for a long time. It's,

you know, it gets in the way of our conversation. It gets in the way of

conversation. It gets in the way of something that you want to do quickly.

And so uh typically in many of these models um they're starting to move towards this idea of your like you give it like a budget to say I want this quickly and I understand that I will have a limitation on the accuracy although many models are pretty good at

many tasks and so at some point when you ask them to think more sometimes it doesn't make them better um just I think like with humans too and then um you're able to then uh uh sort of tailor what

you want the to do. So in in our context at Box, we allow you to customize our AI agents. And so you can make one that is

agents. And so you can make one that is very much able to go through and give you quick answers. Like if you I have a question about this and it's in the documentation. It's like here's the

documentation. It's like here's the answer for like those quick summaries. It

doesn't take its time and think go through, we pull it out, it looks at it, sees this is the answer, I got it.

Yeah.

But compare that to something like I want you to analyze this contract to look through risks and to tell me about some of the like a nuanced aspect of this. you you actually are completely

this. you you actually are completely okay to wait a little bit of time to get a better answer. So those are kind of the differences in sort of how you would approach that different types of of generi on your content would require

different amounts of uh thinking and quality.

Got it. So it sounds like then in order to leverage AI reasoning models have a very specific place and and in terms of what we can get out of it and the context with which we use them

and uh can you give me one or two examples of like when I would use one when I would use another?

Yeah, for most enterprises this the simple example would be to try it um about in your in your content uh and in your use cases. The contract example is a good one where at some point you're looking for simple information that

comes you know just immediately and then you try you get great results. Um and

then other cases I think most people have had a moment with AI where they have said I am not happy with that result. The the historical model that

result. The the historical model that you might have tried at that moment might not have done been able to understand all your instructions might not been able to reason through it all well. And so this is a great time to try

well. And so this is a great time to try again with a thinking model to see if it the thinking model actually can get to better uh what you're looking for. And

in particular, when you're thinking about more complex things for AI to do, thinking models are usually the one that you would kind of want to apply to to that.

I see. Okay. Well, I want to go back to a couple things that you mentioned earlier in in one of your previous examples. You mentioned thinking models

examples. You mentioned thinking models being able to give better quality answers, which is so important for an enterprise to rely on AI for. Um, you

also mentioned that it reduces hallucinations, but then you also threw in rag in there. Can we connect those concepts? I thought in one of our

concepts? I thought in one of our previous episodes we talked about rag and how that can help reduce the risk of hallucinations. Where does the thinking

hallucinations. Where does the thinking model fit? Like how do these all come

model fit? Like how do these all come together?

Yeah, at some point an AI model or person if you just give them too much data then they're going to struggle with it. And so you're almost limited even

it. And so you're almost limited even with rag about how much you can hand to the model any moment to understand and and um and so uh and what they can focus on, what it can pay attention to and then how it can think through things.

And oftentimes you start to trade quality by giving it more and more and more. And so the idea of the thinking

more. And so the idea of the thinking model is when you're looking for nuance, when you're looking for processing a lot of info, like the thinking models can better plan out and be able to respond to you better with rag or not like um

and and and so therefore it goes back to the idea of of quality overall. I see.

Okay. So uh let's recap a little bit. We

talked about what is the reasoning model, how it's different, why that's important. I mean this episode and these

important. I mean this episode and these series are all about bringing AI into the enterprise. So for our viewers

the enterprise. So for our viewers watching, what would be the TLDDR here for them?

I think that the key is that um as a major new evolution of AI models uh are coming out, these thinking models, these reasoning models, um you can start to uh see this as a way to get higher quality

of responses and more complex tasks. And

this will be yet another tool that you can use as an enterprise or or the the companies and the platforms that that provide AI to you like Box can use to then do more and more of complex use cases in your environment. Yeah, that

was really insightful and I think that really helped put into the picture the difference of the you know the models that are coming out and reasoning and where that fits in with rag. So that was really insightful. Thanks Ben and thank

really insightful. Thanks Ben and thank you to our viewers for tuning in to yet another episode of Enterprise AI explainer series. See you next time.

explainer series. See you next time.