Full Walkthrough: Workflow for AI Coding from Planning to Production — Matt Pocock (@mattpocockuk )

Yeah, we good.

Okay, folks, we're at capacity. Let's

kick off. I don't want you waiting here for 25 more minutes before we some arbitrary deadline. So, welcome. My name

arbitrary deadline. So, welcome. My name

is Matt. Uh I'm a teacher and I suppose now I teach AI. Um

we have a link up here if you've not already been to this which is has the exercises for the um stuff we're going to do today. This is going to be around two hours. So we might just sort of kick

two hours. So we might just sort of kick off two hours from now. Is that right Mike?

Yeah. Perfect. Um, and the theory behind this talk or at least the thesis under which I've been operating for the last kind of six months or so is that

we all think that AI is a new paradigm, right? AI is obviously changing a lot of

right? AI is obviously changing a lot of things. You guys are obviously

things. You guys are obviously interested in this and that's why you've come to this talk. And

I feel that when we talk about AI being a new paradigm, we forget that actually software engineering fundamentals, the

stuff that's really crucial to working with humans, also works super well with AI. And this is what my keynote is on

AI. And this is what my keynote is on tomorrow. Really, I'm going to sort of

tomorrow. Really, I'm going to sort of be fleshing that out a lot more. And in

this workshop, I'm hopefully going to be able to direct your attention to those things and uh hopefully show you that I'm right, but we'll see. Um, can I get

a quick heads up first? How many of you guys um are coding have ever coded with AI? Raise your hand if you've ever coded

AI? Raise your hand if you've ever coded with AI. Perfect. Okay. Uh, keep your

with AI. Perfect. Okay. Uh, keep your hand raised.

Uh, let's all uh share those armpits with the world. Um,

how many of you code every day with AI?

Cool. Okay. Uh, ra keep your hand raised if you've ever been frustrated with AI.

Okay. Very good. You can put your hands down. Thank you for that show of

down. Thank you for that show of obedience. I really appreciate that. Um,

obedience. I really appreciate that. Um,

we are also being live streamed to the Gilgood room as well. I've not uh did we send someone up to the Gilgood room to just check they're okay? Don't know. But

I see you. Uh, and there is a way that you can participate which is we have the um a Q&A. We're going to be doing kind I have a sort of hatred of Q&As's because they're not very democratic. The mostly

the sort of um most talkative people get to um get to participate and share. And

so we're going to be going through this um QA here. So why do we have to wait till 3:45? The room is packed. The doors

till 3:45? The room is packed. The doors

are closed. 100% agree. And so if you want to uh ask a question, we're going to be I would like you to pile into this async and then we can vote on each other's questions and hopefully get the

best question surface so the for the entire room to enjoy.

So I want to talk about first the kind of weird constraints that LLMs have and those weird constraints are sort of what we have to base a lot of our work

around. Now,

around. Now, there's a guy called Dex Hy who runs a company called Human Layer, and he came up with this idea, which is that

when you're working with LLMs, they have a smart zone and a dumb zone. When

you're first kind of like working with an LM and it's like you just started a new conversation, you start from nothing. That's when the LLM is going to

nothing. That's when the LLM is going to do its best work because in that situation, the attention relationships are the least strained. Every time you add a token to an LLM, it's kind of like you're adding a team to a football

league. You think of the number of

league. You think of the number of matches that get added every time you add a team to a football league. It just

go scales quadratically. And that's

because you have attention relationships going from essentially each token to the other that are positional and the sort of meaning of the individual token. And

so this means that by around sort of 40% or around I would say around 100k is kind of my new marker for this because it doesn't matter whether you're using 1

million uh context window or 200k. It's

always going to be about this.

It starts to just get dumber. So as you continually keep adding stuff to the same context window, it just gets dumber and dumber until it's making kind of stupid decisions. Raise your hand if

stupid decisions. Raise your hand if that feels familiar to you. Yeah. Cool.

So this means that we kind of want to size our tasks in a way that sticks within the smart zone, right? We don't

want the AI to bite off more than it can chew. And this goes back to old advice

chew. And this goes back to old advice like Martin Fowler in refactoring uh like uh the pragmatic programmer talks about this. Don't bite off more than you

about this. Don't bite off more than you can chew. Keep your tasks small so that

can chew. Keep your tasks small so that you as a developer, a human developer don't freak out and don't start acting and going into the dumb zone.

But how do you tackle big tasks? How do

you take a large task like I don't know cloning a company or something or just doing something crazy? And how do you break it into small tasks so they all fit into the dumb zone? One way of

course you could do is I mean kind of what the AI companies maybe want you to do or the natural way of doing it is just keep going and going and going. You

end up in the dumb zone charging you tons of tokens per request. You then

compact back down. We'll talk about compacting properly in a minute. And you

keep going, keep going, keep going, compact back down, keep going, keep going, keep going. And I think that's doesn't really work very well because the more sediment, we'll talk about that

in a minute. So the theory here is then, and this is what I was doing for a while, is I would use these kind of multi-phase plans where I would say,

okay, we have this sort of number four thing here, this large large task. Let's

break it down into small sections so that we can then kind of chunk it up and do each little bit of work in the smart zone. Raise your hand if you've ever

zone. Raise your hand if you've ever used a multi-phase plan before. Yeah,

really common practice, right? This is

kind of how we've been doing it.

Certainly, this is how I was doing it up until December last year really.

And any developer worth their salt will look at this and go, "This is a loop, right? This is a loop. We've just got

right? This is a loop. We've just got phase one, phase two, phase three, phase four. Why don't we just have phase n,

four. Why don't we just have phase n, right?

Phase n where we essentially just say, okay, we have, let's say, a plan operating in the background and then we just loop over the top of it and we go through until it's complete. And this is

where um raise your hand if you've heard of Ralph Wiggum as a software practice.

Okay, cool. Raise your hand if you've not heard of Ralph Wigum as a software practice. Actually, that's more like it.

practice. Actually, that's more like it.

Okay. So there's this idea called Ralph Wigum uh which is kind of um sort of based on this which is essentially all you need to do is sort of specify

the end of the journey where you just say okay we create a PRD a product requirements document to say okay let's describe where we're going and then we just say to the AI just make a small

change make a small change that gets us closer and closer to there and Ralph works okay but I prefer a little bit more structure so that's kind where we got to in terms of thinking about the

smart zone. And that's kind of where I

smart zone. And that's kind of where I want you to first start thinking about here. Another weird constraint of LLM is

here. Another weird constraint of LLM is LLM are kind of like the guy from Momento, right? They just continually

Momento, right? They just continually forget. They could just keep resetting

forget. They could just keep resetting back to the base state. Let me pull up this diagram.

I sort of I I I really should use slides, but I just prefer just like randomly scrolling around a infinite uh TL draw canvas. Thank you, Steve.

Um, so let's say another concept I want you to have is that every session with an LLM kind of goes through the same stages. You have first of all the system

stages. You have first of all the system prompt here. This gray box here is

prompt here. This gray box here is essentially the stuff that's always in your context. You want this to be as

your context. You want this to be as small as possible because if you have a ton of stuff in here, if you have 250k tokens, like I have seen people put in there, then that you're just going to go

straight into the dumb zone without even being able to do anything. So you want this to be tiny. You then go into a kind of exploratory phase. This blue is sort of where the coding agent is going out

and exploring the codebase. Then you go into implementation and then you go into testing and kind of making sure that it works, running your feedback loops and things like this. Raise your hand if

that feels familiar based on what you've done. Yep. Sort of the like the the main

done. Yep. Sort of the like the the main cornerstones of any session. And when

you clear the context, you go right back to the system prompt. Bof, you go right back there. So you delete everything

back there. So you delete everything that's come before.

And raise your hand if you've heard of compacting as well. Yeah. Okay. There

are some people who've not heard of compacting. So let's just quickly show

compacting. So let's just quickly show what that means. For instance, I've just been having a little chat with my LLM.

Uh, I want to make sure we sort of, you know, just cover the basics so we're all sort of on the same wavelength here.

I've just been having a chat with my LLM. I've been talking about a thing

LLM. I've been talking about a thing that I want to build. How's the font size? Should I bump it up? Folks in the

size? Should I bump it up? Folks in the back. Bump bump bump bump bump.

back. Bump bump bump bump bump.

I'm using claw code for this session, but you don't need to use claw code. Uh,

in fact, it's often nice not to use claw code. Um, so I've been having a chat

code. Um, so I've been having a chat with the LM just sort of planning out what I'm going to do next. It's asking

me a bunch of questions and I can I highly recommend you do this. There's

this tiny little status line here that tells me how many tokens I'm using. The

exact number of tokens I'm using. Um I

have a article on my website AI Hero if you want to copy this. This is oh wow that is that shakes doesn't it? Um, this

is essential information on every coding session because you need to know exactly how many tokens you're using so that you know how close you are to the dump zone.

Absolutely essential. And so let's watch it. So I've got two options. I can

it. So I've got two options. I can

either clear and go back to nothing or I can compact.

And when I compact then it's going to squeeze all of that conversation which admittedly isn't very much into a much smaller space. And this in diagram terms

smaller space. And this in diagram terms kind of looks like this where you take all of the information from the session and you essentially create a history out of it, a written record of what

happened.

And devs love compacting for some reason, but I hate it. I much prefer my AI to behave like the guy from Momento because this state is always the same.

Always the same. Every time you do it, you clear and you go back to the beginning. And so if you're able to do

beginning. And so if you're able to do that and you're able to optimize for that, then you're in a great spot.

So that's kind of the two things I want you to think about with LLM, the two constraints that we're working with.

They have a smart zone and a dumb zone.

And they're like the guy from Momento.

So let's take a look at the first exercise. And I'm while I'm doing this,

exercise. And I'm while I'm doing this, the way I want this to work is I'm going to sort of show you how um I'm going to be sort of walking through it up here.

And I want you folks to be kind of like tapping away and doing things as well.

So that was just a little lecture bit.

Let's now actually get and do some coding. For anyone who arrived late or

coding. For anyone who arrived late or anyone in the Gilgood room, uh go to this link, this link up here

to see the exercises and clone the repo.

You absolutely do not have to. You can

just watch me do it if you fancy it. But

let's go there myself and let's see what exercises await us.

So essentially, I've built a um this is from my course. This is a uh a course management platform essentially a kind of CMS for instructors for students and

this is what we're going to be building a feature in. So I'm going to take you from essentially the idea for the feature all the way up to building a PRD for the feature all the way up to

implementing the feature and hopefully you can take inspiration from this process and use it in your own work. So

uh let's kick off. episode.

We're going to start by using a skill which is very close to my heart. It's

the grill me skill. And this grill me skill is wonderfully small, wonderfully tiny. And it helps prevent one of I

tiny. And it helps prevent one of I think the main issues when you're working with an AI, which is misalignment.

The uh the sort of silent idea that I'm talking against here, that I'm arguing against is the specs to code movement.

Has anyone heard of the specs to code movement? Raise your hand. It's not

movement? Raise your hand. It's not

really a movement. I suppose it's just sort of people saying specs to code. Um,

what it is is people say, okay, you can write a program or you want to build an app. The best way to build that app is

app. The best way to build that app is to take some specifications.

So to write some sort of like document and then turn that document into code.

So just turn it into code. How do you do that? You pass it to AI. if there's

that? You pass it to AI. if there's

something wrong with the resulting code.

You don't look at the code, you look back at the specs, you change the specs and you sort of just keep going like this. This is kind of like vibe coding

this. This is kind of like vibe coding by another name where you're essentially ignoring the code. You don't need to worry about the code. You just sort of keep editing the specs and eventually you just keep going. And I tried this. I

really tried it and it sucks. It doesn't

work because you need to keep a handle on the code. You need to understand what's in it. You need to shape it because the code is your battleground.

And so this again is where we're going. Let's

let's get some exercises. So what I'd like you to do is go to this page, the the grill me skill. And inside the repo here, we have a Slack message

from our pal. Where is it? It's in the root of the repo. And it's under where is it?

Clientbrief.mmd.

It's a Slack message from Sarah Chin.

For some reason, the Claude always chooses Sarah Chen as the name. I don't

know why. Um, it's saying that in Cadence, our um course platform, our retention numbers are not great.

Students sign up, do a few lessons, then they drop off. I'd love to add some gamification to the platform. And so,

when you're presented with an idea like this, you need to find some way of turning it into reality. Let's say Sarah Chen is your client. You're on a tight budget. You need to get this done fast.

budget. You need to get this done fast.

How do you go and do it? Um, raise your hand if you would. um enter plan mode when you're doing this. Anyone a big user of plan mode? Yep. Um let's

actually shout out quickly any other ideas about what you would do with this or raise your hand if you what would be your first port of call.

Yeah, sorry.

Yes, exactly. Let's imagine that Sarah Chen's gone on hold. You have no idea, right? Uh she's just posted this thing.

right? Uh she's just posted this thing.

You need to action it before you go.

Well, my first protocol is I go for this particular skill. I'm going to clear my

particular skill. I'm going to clear my context.

I'm going to uh get rid of you. You

don't need to be there. And I'm going to say um I'm going to invoke a skill, which is the grill me skill. Let's quickly check.

Raise your hands if you don't know what this is.

Cool. Oh, sorry. Sorry. Let me be more specific. Raise your hands if you don't

specific. Raise your hands if you don't know what I'm doing here when I uh do a forward slash and then type something.

Anyone everyone kind of understand what that is? I'm invoking a skill. I'm

that is? I'm invoking a skill. I'm

invoking the grill me skill. And what

I'm going to do is I'm going to say grill me and I'm going to pass in the client brief.

So now the LLM really has only a couple of things here. It just has the skill and it has the description of what I want to do.

And this is virtually how I start every piece of work with AI. And while it's exploring the codebase, I'm just going to show you what the grill me skill does. So this is inside

the repo so you can check it out. It's

extremely short. Interview me

relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of

understanding. Walk down each branch of the design tree, resolving dependencies one by one. For each question, provide your recommended answer. Ask the

questions one at a time. uh blah blah blah. What this does, and what I noticed

blah. What this does, and what I noticed when I was working with AI, especially in plan mode actually, is it would really eagerly try to produce a plan for

me. It would say, "Okay, I think I've

me. It would say, "Okay, I think I've got enough. I'm just goof plan."

got enough. I'm just goof plan."

And what I found was that I was really trying to find the words for this for for what I wanted instead of that. And Frederick P. Brooks in the

of that. And Frederick P. Brooks in the design of design he has a great quote uh talking about the design concept when you're working on something new with

someone when you're uh all trying to build something together then there's this shared idea that's shared between all participants and that is the design concept and that's what I

realized I needed with Claude I needed I needed to reach a shared understanding I didn't need an asset I didn't need a plan I needed to be on the same wavelength as the AI as my agent. And

this is an extremely effective way of doing it. So hopefully there we go.

doing it. So hopefully there we go.

Nice. It has done its exploration. First

of all, it's invoked a sub agent which spent uh 97 93.7K tokens on Opus.

Um and it's asked me the first question.

Cool. We can see that even though the sub agent burned a ton of tokens, I haven't actually um uh increased my token usage that much. Raise your hand if you don't know what sub aents are.

It's an important question. Everyone

kind of clear what sub aents are? Okay,

I'll give a brief definition which is that this this sub aents thing here, this explore sub agents, it has essentially gone and called another LLM

which has an isolated context window and then that LLM has reported a summary back. So a sub aent is kind of like a

back. So a sub aent is kind of like a delegation. You're delegating a task to

delegation. You're delegating a task to a sub agent. It goes eagerly does all the thing, explores a ton of stuff and then just drip feeds the important stuff back up to the orchestrator agent to the

parent agent. So, okay. So, hopefully

parent agent. So, okay. So, hopefully

you guys have seen the same thing. It's

done on explore. And we now have our first question. Points economy. What

first question. Points economy. What

actions earn points and how much? Okay.

At this point, you can ask it, by the way, questions to um deepen your understanding of the repo. I obviously

know this repo really well because I wrote it, but you might not um know what's going on. So, let's say my recommendation, keep it simple, twopoint sources to start. What's so nice about

this is that not only does it give us a question that kind of aligns us here, we get a recommendation, too. And often

what I'll find is the AI's recommendations are really good. And so

I'll just say skip video, watch events, they're noisy and gameable. I agree.

Sarah's asked while keep lessons in the bread and butter.

Yeah, looks good, pal.

Now, what I usually do is I usually dictate to the AI. I'm usually actually chatting to the AI instead of uh typing here, but uh this is a relatively new laptop and I couldn't get my dictation

software working on it um because Windows is crap. Um

so should points be retroactive? There

are existing lessons progress records.

We're completing out timestamps. This is

a really nasty question, right? Should

we actually go back and backfill all of the lesson progress events? This is a kind of question that you need to be aligned on if you're going to fulfill the feature properly. This is not something I considered and Sarah Chen

certainly didn't consider. Do I want it to be retroactive? H. Let's actually do a vote inside here. Should we go back and backfill all the records? Raise your

hand if you think we should backfill all the records.

Raise your hand if you think we shouldn't backfill all the records.

There are a lot of uh fence sitters in the room. I'm going to say,

the room. I'm going to say, you know, this is the kind of discussion you're sort of having with the AI.

You're getting further aligned. Yes, I'm

just going to go with this recommendation because I'm lazy.

Notice, too, how I'm able to keep in the loop here with AI. I'm not, you know, it's it's pinging me these questions pretty quickly.

I'm not having to go off and check Twitter or something. Levels. What's the

progression curve? Yeah, that looks about right, for instance. Yes. Okay. So

hopefully you should be able to go and um kind of work through this with the AI and essentially try to reach an alignment. And this grill me skill this

alignment. And this grill me skill this can last a long time. This can I've had it ask me 40 questions. I've had it ask me 80 questions. I've had some people it asks a hundred questions to literally

you're sat there for an hour chatting to the AI. And what you end up with is

the AI. And what you end up with is essentially this conversation history that works really nicely and works really nicely as an asset of the design concept that you're creating. This can

also function like this. You can uh have a meeting with someone who's a maybe a domain expert. Maybe I have a meeting

domain expert. Maybe I have a meeting with Sarah. I feed that meeting

with Sarah. I feed that meeting transcript into uh I don't know Gemini meetings or whatever you guys are using.

You take that, you feed it into a grilling session and you grill through the assumptions that you didn't have.

So, this ends up being a really nice kind of um a really nice way of just taking inputs from the world and then just turning and validating them. So,

okay, let's see. I really want to get to the

let's see. I really want to get to the end of this, but I also don't want to just like be sat here talking to the AI in front of you for uh a thousand days.

So, I'm just going to say yes.

Let's see what happens. So, I tell you what. Um, while you guys sort of have a

what. Um, while you guys sort of have a little fiddle with this locally, let's start a little Q&A session now. And

let's see how's this going to work. Can

we keep the door closed? I'll turn up the microphone. It's quite noisy. Uh,

the microphone. It's quite noisy. Uh,

let's see. Mike, can we uh Door closed?

Oh, it has been closed. Mark has

answered. Beautiful. So, what I'd like you to do is there any air con? Yeah,

there is some air con. I think there is some air con you guys aren't being lit here. I'm being I'm being fried alive

here. I'm being I'm being fried alive here. Uh so what I'd like you to do is

here. Uh so what I'd like you to do is go on to the slideo which you can join here. Have a if if you're not taking the

here. Have a if if you're not taking the exercise, go on to the slideo, have a little fiddle and vote on some good questions. I'm just going to chat to the

questions. I'm just going to chat to the AI for a second uh until we reach a stopping point. So do streaks earn

stopping point. So do streaks earn points?

Um, streaks are standalone.

Let's see what else it comes up with.

Where does gamification UI live? Let's

have it in the dashboard.

I'm just going to scan these and blast through them basically. So, how we doing with our slido?

Okay.

Have I tried specit open spec or taskmaster instead of the grill me skill? Do I find them more verbose or a

skill? Do I find them more verbose or a structured alternative? This is a great

structured alternative? This is a great question. So there are a ton of

question. So there are a ton of different frameworks out there that allow you to um sort of build up this planning process for you. I personally

believe you at at this stage when there's no clear winner, when there's no kind of like one true way and when things are changing all the time, you need to own as much of your planning

stack as you possibly can. What I've

noticed and a lot of my students is they tend to overuse a certain stack.

they get into trouble and they because they don't own the stack and they don't have observability over the whole thing, they just go, "This isn't working. This

sucks." Whereas if um if you have control over the whole thing, then at least you know how to fix it or potentially know how to fix it. So I'm

even though I'm sort of giving you uh a stack basically, I believe in inversion of control and you should be in control of the stack.

So, can I press zero, please?

Sorry.

Sorry, that was a lot of sort of mumbling. Can I

mumbling. Can I feedback? You have four options on the

feedback? You have four options on the bottom of you to hit dismiss.

Thank you.

I'm so sorry. Well, you didn't want to give Claude good feedback. Why? What's

wrong with you?

Okay cool.

Uh many of the questions asked by the grill me skill are not necessarily appropriate for a developer rather a PO in larger teams who should use it. Yeah.

Um raise your hand if um you've ever done pair programming. Anyone ever done pair programming? Right. Keep put your

pair programming? Right. Keep put your hands down and raise your hand again if you've ever done a pair programming session with an AI.

Right. How did it go? Was it good? You

enjoy it? I think pair programming sessions with AI is a great idea because you've got a third person in the room who will relentlessly quiz you and ask you questions. It should if you don't

you questions. It should if you don't know the answer, it should be you, the domain expert and the AI in the same room. If you have a question about

room. If you have a question about implementation, it should be you, a fellow developer and the AI in the same room. You know, you can be sort of

room. You know, you can be sort of working through these questions in your team. And I think actually we're going

team. And I think actually we're going to look at implementation in a bit and we're going to see how you can make implementation so much faster. And but I think the really crucial decisions, the

ones you need humans for, you actually need a lot of humans and it doesn't really matter how many humans are in there. You can actually throw a bunch

there. You can actually throw a bunch like a kind of like mob programming with AI essentially.

Uh what's my favorite metaprompting tool? I think I kind of answered that.

tool? I think I kind of answered that.

Uh there's no air con. Let's just live with it. Uh, how do I use the

with it. Uh, how do I use the conversation as an asset after the grill me session? Well, we're going to get

me session? Well, we're going to get there.

Um, okay. So, I really want to I want to speed this up sort of artificially.

Just what I This is the thing. So, someone just said, "Okay, Ralph loop this." But this is crucial because I can't loop over

this, right? I can't um I think of there

this, right? I can't um I think of there as being two types of tasks in the AI age where you have human in the loop tasks where a human needs to sit there

and do it which is this we are the human in the loop with multiple humans in the loop and there are AFK tasks there are tasks where the human can be away from the keyboard and it doesn't matter

implementation as we'll see can be turned into an AFK task but planning this alignment phase has to be human in the loop has to be.

So, I've got to do it, unfortunately.

Um, I don't know. Uh, give me a long list of all your recommendations.

I'm running a workshop right now, so I artificially need you to pull more weight.

So, let's see what it does. Uh, let's

answer a couple more questions while it's doing its thing.

What is my opinion on PMS or other non-dev rolls vibe coding task?

Um, I'm going to return to this later. I

think I'm going to leave this unanswered.

A bit of mystery.

I notice I'm not using the ask user questions UI for grill me. Why? Um,

there's a specific uh UI that you can bring up in claude code which I'll answer this just quickly. uh ask me a question using the ask user question

tool.

And this UI um is just sort of broken in Claude and I really hate it.

You notice I'm using Claude, but I don't like Claude very much. Like you you really are free with this method to choose any um system you like. And this

is what the UI looks like. It's very

pleasing when you first encounter it, but then you realize it is actually broken in a ton of different ways.

All right, what did it come back with?

Oh, blime me.

Oh no.

So, while this is doing its thing, let me do some teaching in the meantime. The plan

here is that we take our grill me skill and we need to essentially find some way of turning it into a destination.

We need to go down to the uh we essentially need to we're figuring out the shape of this. That's what we're doing. figuring out the shape of the

doing. figuring out the shape of the tasks during the grilling session. And

in order to turn it into a bunch of actionable actions for the AI, we essentially need to figure out the destination. We need to know where we're

destination. We need to know where we're going. We need to know the shape of this

going. We need to know the shape of this entire thing. So I think of there as

entire thing. So I think of there as being two essential documents that we need. We need a document that documents

need. We need a document that documents the destination.

Oh no, it's so not bright enough. There we go.

Still not bright enough. There we go. We

need something to document the destination and we need something to document the journey. In other words, we need something a document that's going to figure out what this even looks like

in all of its user stories and figure out a definition of done. And then we need to figure out what the split looks like. So that's where we're going to go

like. So that's where we're going to go to next. So once we finish with the

to next. So once we finish with the grilling session.

Yeah, it looks great. Fantastic. I love

it. It answered it answered 22 of its own questions. There you go. That's

own questions. There you go. That's

quite representative of what a grilling session looks like.

So at this point now I have used 25k tokens and all of that or loads of that stuff is gold. I want to keep that

around. I've I've got 25k great tokens

around. I've I've got 25k great tokens there. And what I want to do is kind of

there. And what I want to do is kind of summarize it in some kind of destination document. So this is um the next

document. So this is um the next exercise where we're going to uh we're going to write a product requirements document. And the product

requirements document. And the product requirements document or the PRD is essentially that's its function. It's

the destination document. And it sort of doesn't matter what shape it is. I've

got a shape that I prefer and that I quite like, but you can just choose your own shape or whatever your company uses.

And all we're really doing is too worried about that.

All we're really doing is summarizing the design concept that we have so far.

And the So let let's try this. So I'm going to initiate this. I'm going to say zoom all the way to the bottom. All I'm going

to do is just say write a PRD.

And we can take a look at that skill now.

Write a PRD.

So this skill, it does a few things. It first asks the user for a long detailed description of the problem. You can use write a PRD

the problem. You can use write a PRD without grilling first, but I just like to grill first and then write the PRD afterwards. Then you can um get it to

afterwards. Then you can um get it to explore the repo, which we've kind of already done. Then we get it to

already done. Then we get it to interview the user relentlessly. So have

a kind of grilling session again. And

then we start um putting together a PRD template. So this is available in the

template. So this is available in the repo if you want to check it out. And

essentially this is what it looks like.

We've got some problem statements, the problem the user is facing, the solution to the problem, and a set of user stories. And these user stories sort of

stories. And these user stories sort of define what this is. You know, as you you guys have probably seen things like this if you've been a developer at all.

um you know there are cucumber is a language you can use to write these in or we just sort of um uh write them ourselves essentially. Then we have a

ourselves essentially. Then we have a list of implementation decisions that were made and a list of crucially testing decisions too. So

I'm going to run this. Okay. And so it's finished its thing. Ah

Windows let me close the thing. Thank

you. I don't know why I bought a Windows laptop. I think I just I like the

laptop. I think I just I like the challenge. Um

challenge. Um so the first thing that it's going to give me are a set of proposed modules it wants to modify.

Now there's a deep reason why I'm thinking about this. So this is at this stage we have an idea. We have sort of speced out the idea. We've reached a

sort of understanding of what we're trying to do. And then we need to start thinking about the code because at this point we need to this is not specs to code. This is not where we're ignoring

code. This is not where we're ignoring the code. We actually keep the code in

the code. We actually keep the code in mind throughout the whole process. And

the way I like to do this is I like to just sort of think about a set of proposed modules to modify. We're going

to return to this this idea of continually designing your system and keeping your system in mind. So it's

it's saying recommend test for the gamification service is the only deep module with meaningful logic. These

modules look right. Yeah, that's good.

And it's going to ping out a PRD.

Now for ease of setup, I've got it so that it creates a set of issues locally.

So it's just going to create essentially a PD inside this issues directory. But

the way I usually do it, and you can check this out yourself, is you can go to my um essentially what I consider my work repo, which is github.com/mattpocco/course

github.com/mattpocco/course video manager up here. And in here, this is essentially a app that I create um that I use all the time to record my videos and things like this. I think

I've recorded like I pulled down the sets. I think I've recorded like a

sets. I think I've recorded like a thousand videos in here or something nuts. Um, and you can see here that it's

nuts. Um, and you can see here that it's got 744 closed issues. And this is essentially all of the uh PRDs and all of the implementation issues that I've

put into here. So, this is how I usually like to do it.

So, that's what I'm doing with the There we go. Yeah, I'm just going to say yes

we go. Yeah, I'm just going to say yes and uh and get that issue out. Let's see. It is

inside here. So, we got the problem statement. people sign up for courses,

statement. people sign up for courses, uh the solution, the user stories, uh 18 user stories, looks nice, some implementation decisions, level thresholds, etc. This is enough

information. We've kind of clarified

information. We've kind of clarified where we're going and what we're doing.

So that's what we do. We essentially

have a grilling session and we've created an asset out of it. Now, raise

your hand. Should I be reviewing this document? Raise your hand if you think I

document? Raise your hand if you think I should be reviewing the document.

Yeah, I don't I don't look at these. I

don't look at these. The reason I don't look at these is because what am I testing at this point? What am I like when I read it?

What am I testing? What am I what are the failure modes I'm trying to test for? I know that LLMs are great at

for? I know that LLMs are great at summarization because they are they're really good at summarization. I have

reached the same wavelength as the LLM, right? Using the grill meme skill, we

right? Using the grill meme skill, we have a shared design concept. So if I have a shared design concept, all I'm doing is I'm just essentially checking

the LLM's ability to summarize.

So I don't tend to read these.

Let's have let's have a Q&A because I can feel you guys are itching for it.

And then I think we might have like I don't know just a five minute comfort break just to rest my voice and so you can catch up with the exercises for a minute if that's all right. So let's

have a little Q&A sesh. Uh, if I don't like clawed code, which one do I actually like? Um,

actually like? Um, uh, have you ever heard the phrase, um, uh, democracy is the worst way to run a country apart from all the other ways?

That's how I feel about claw code.

Uh, we've answered that one.

Uh, what's your thoughts on developers needing to very deeply understand Typescript now that fix the TS make no mistakes exist? I don't understand the

mistakes exist? I don't understand the phrasing of this but I think I understand the meaning which is that I believe that code is very important and this is kind of going to feed

through the whole session and that bad code bases make bad agents. If you have a garbage codebase you're going to get garbage out of the agent that's working in that codebase. We'll talk more about

that in a bit. And so I think understanding these tools very deeply, understanding code deeply is going to make you a much much better developer and get more out of AI.

Uh, and that answers that question too.

Sweet.

Uh, get out of it. There you are.

Now that we have 1 million tokens available, do we ever actually want to take advantage of that?

I've noticed that the dumb zone has become less dumb lately. Okay, great

question. This goes back to our kind of initial idea on the dumb zone.

Uh I um I recorded my Claude Code course using a 200k context window and on the day that I launched the course, they announced the 1 million context window.

My take on this is that what Claude code did is they essentially just did this.

They shipped a lot more dumb zone to you essentially. Now, this is good for tasks

essentially. Now, this is good for tasks where you want to retrieve things from a large context window. If you want to pass five copies of War and Peace or something to it, and you want to find

out all the things that uh uh I can't remember a character from War and Peace. Uh why did I start with that?

and Peace. Uh why did I start with that?

It's good for retrieval. It's less good for coding. So, I consider that it is

for coding. So, I consider that it is about 100K at the moment is the smart zone. the smart zone will get bigger and

zone. the smart zone will get bigger and that will be a really nice improvement.

So folks, we're going to take like a five minute comfort break if that's all right just for my voice and so maybe you can have a little move around or something or grab a drink. I can just notice some sleepy eyes and I want to make sure that we're awake for the next

bit if that's all right. So we'll take five minutes and I'll see you back here then. All right.

then. All right.

So we have our PR which I'm not going to read a kind of destination document.

Let's quickly scan for any good questions before we zoom ahead.

And rediscovering the role of software engineer in today's world. Top three

disciplines you recommend. Um, taekwondo

is good. I've heard I' have no idea how to answer this question. Um, thank you for asking it though. Um, top three disciplines I recommend.

I mean, sorry, plumbing.

Plumbing is a good one. Yeah. Yeah.

Yeah. I don't know if that's a discipline. The plumbers I've hired are

discipline. The plumbers I've hired are not usually very disciplined. Um,

right.

So, okay, we now have our destination.

Okay. Um,

perfect.

So, how do we actually get to our destination? How do we We have a sort of

destination? How do we We have a sort of vague PRD. How do we split it so that we

vague PRD. How do we split it so that we don't put things into the dump zone? In

other words, we have our number four.

How do we split it into this kind of multi-phase plan? Well, probably what

multi-phase plan? Well, probably what you would do at this point is you would say, "Okay, Claude, give me a multi-phase plan that gets me to this destination." Right? That sort of makes

destination." Right? That sort of makes sense. This is what we've been doing

sense. This is what we've been doing before, but I have um a sort of better way of doing it now, which is that

I like creating a canban board out of this. Raise your hand if you don't know

this. Raise your hand if you don't know what a canban board is.

Cool. Okay. A camon board is essentially just a set of tickets that you put on the wall that have blocking relationships to each other. So, we're

going to see what it kind of looks like here. This is how we've worked um as

here. This is how we've worked um as developers for a long time, really since agile came around. And what it does, we can see it here. It has proposed that we

split this setup into um five different tasks. Here we have the first one which

tasks. Here we have the first one which is the schema and the gamification service. Yeah, that looks pretty good.

service. Yeah, that looks pretty good.

This is blocked by nothing. And we can even see here that it's a it's given it a type of AFK, too. Remember I talked about human in the loop and AFK earlier.

This is an AFK task. This is something we can just pass off to an agent to do its thing. Streak tracking. Okay, that

its thing. Streak tracking. Okay, that

looks good.

Uh then wire points and streaks into lessons quiz completion. This is blocked by one and two. Retroactive backfill.

This is blocked only by one. And then

this one here is blocked by all of the tasks. Cool.

tasks. Cool.

H. Now I consider this, you could say, why don't we just make this sort of generation of the issues? Why don't we just hand that over to the AI? Why do I need to be involved here? Right? Because

it's given us quite a good selection of tools here. Why do I need to review this

tools here. Why do I need to review this and sort of figure out what's next? Now,

my take here is that this is really cheap to do, like very quick to do once I've done the PR. And I can immediately see some issues here.

There's a really, really important technique when you're kind of figuring out what the shape of this journey should look like. And

it sort of comes to this very classic idea uh which comes from pragmatic programmer called tracer bullets or vertical slices.

and traceable. It's really transformed the way I think about actually getting AI to pick its own tasks.

Systems have layers, right? There are

layers in your system. These might be different deployable units. You might

have a database that lives somewhere.

You might have an API that lives maybe close to the database but in a separate bit. You might have a front end that

bit. You might have a front end that lives somewhere totally different like a CDN. Or within these deployable units,

CDN. Or within these deployable units, you might have different layers within those. In for instance the codebase that

those. In for instance the codebase that we're working in, we have a ton of different services servers. We have a quiz service, a team service, user service, coupon service, course service.

And these services have dependencies on each other. So they're kind of like

each other. So they're kind of like individual layers. Well, what I noticed

individual layers. Well, what I noticed is that AI loves to code horizontally.

So it loves to code layer by layer. So

in other words, in phase one, it will do all of the database stuff, all of the schema, all of the, you know, all the stuff related to that unit. Then it will go into phase two and do all of the API

stuff. Then it will add the front end on

stuff. Then it will add the front end on top of that.

Does can anyone tell me what's wrong with that picture? Why is that not a good thing to do? Raise your hand if you have an answer.

Yeah.

Have the whole feedback loop.

Exactly. You don't get feedback on your work until you've really started or completed phase three.

So what you really need to do is you're not until you get to phase three, you're not actually testing that all the layers work together.

You haven't got an integrated system that you can test against. And so

instead you need to think about vertical layers. You need to think about thin

layers. You need to think about thin slices of functionality that cross all of the layers that you need to. And this

is a much better way to work, much better way for the AI to work too because it means at the end of phase one or during phase one, it can get feedback on its entire flow. So what this means

to me is inside the PRD to issues skill up here I have got break a PRD into independently grabbable issues using

vertical slices traceable it's written as local markdown files we first locate the PRD uh again explore the codebase if this is a fresh session we draft vertical slices

so we break the PRD into tracer bullet issues a traceable bullet by the way is Uh, essentially when you're like an anti-aircraft gunner, it's quite a violent idea actually, uh, and you're

looking up in the sky and it's night, if you're just shooting normal bullets, you have no idea what you're firing at, right? You could just be, you know, you

right? You could just be, you know, you see the plane, but you don't see where your bullets are going. Tracer bullets

is they attach a tiny bit of phosphoresence or phosphor or something to make it glow as it goes. So, this

means that every sixth bullet or something, you actually see a line in the sky. So, you have feedback on where

the sky. So, you have feedback on where you're aiming. So this is what this is

you're aiming. So this is what this is the idea here is that we increase our level of feedback and we get near instant feedback on what we're building because without that the AI is kind of coding blind until it reaches the later

phases. We've got some vertical slice

phases. We've got some vertical slice rules. We quiz the user and then we

rules. We quiz the user and then we create the issue files. So what I see here is that even though I've I've told

it to do vertical slices, it's proposing to create the gamification service first on its own. That's just one slice there. And that to me feels like a

there. And that to me feels like a horizontal slice. What I want to see in

horizontal slice. What I want to see in the first vertical slice especially is I want to see the schema changes or some schema changes. I want to see some new

schema changes. I want to see some new service being created and I want a minimal representation of that on the front end. So I want it to go through

front end. So I want it to go through the vertical slices, not just the horizontal. Does that make sense? Okay.

horizontal. Does that make sense? Okay.

So I'm going to give the AI a rollicking.

Uh bad boy. No,

I'm not going to waste tokens just being just memeing. Um so the first slice is

just memeing. Um so the first slice is too horizontal. I'll just start with

too horizontal. I'll just start with that and see if it picks it up. Does

that make sense as a concept? And I

think having that um what I really like about going back to those old books is that we are really trying to in this day and age like get

uh verbalize best software practices in English. And these books, 20-year-old

English. And these books, 20-year-old books have already done that. And it's

an absolute gold mine if you want to throw that into prompts. But even with that, it's not going to um not going to do a perfect job each time. So, award

points for lesson completion visible on dashboard. Yes, that's a beautiful

dashboard. Yes, that's a beautiful vertical slice because it's definitely a big chunk of stuff. It's doing a lot of stories there, but we're going to see something visible at the end and the AI will then just be able to add to that.

You see why that's preferable to the first one. Cool. Uh, looks great.

first one. Cool. Uh, looks great.

So, we're getting closer now. And anyone

following at home as well, you're not at home, but you get the idea. um we'll

hopefully see the same thing too and start developing the same instincts.

Let's open up for questions just while I'm sort of creating these GitHub issues or not GitHub issues uh local issues.

When will I stop using Windows? Never.

What is your uh Okay, we'll get to that later.

How does AI um decide when to stop grilling? Because AI can ask

grilling? Because AI can ask incessantly. Can we have a smarter way

incessantly. Can we have a smarter way to decide the stop point? Yeah, it does tend to really um those grilling sessions can be super intense. And the

thing about these skills is you can tune them if you want to. If you feel like the AI is just absolutely hammering you, hammering you, hammering you, then you can just tell it to just pull back a little bit or get it to do, you know,

stop points and that kind of thing. So,

if that's a failure mode that you run into a lot, then you just, you know, change the skill.

Uh do I still use uh be extremely concise, sacrifice grammar for the sake of concision? Um there was a tip that I

of concision? Um there was a tip that I gave folks um five months ago which is that to basically increase the readability of your plans. So when

you're using plan mode then you can put it in your claw.md

and you can say okay yeah approve that.

Let's open up claw.md.

Do I have a claw.md? Maybe I don't. I I

really don't use clawd very much. I'm

just going to put a dummy inside here.

Um when no when talking to me uh sacrifice grammar for the sake of concision and this um prompt was uh really useful to me when I was reading the plans

because it meant that the plans would come out and they would be very concise, really nice, easy to read, often very uh concise. But I've since dropped this

concise. But I've since dropped this idea in preference to a grilling session because what I noticed was it just I didn't want to read the plans. I wanted

to get on the same wavelength as the LLM. I wanted it to ask aggressive

LLM. I wanted it to ask aggressive questions to me. And when I stopped reading the plans, I stopped needing them to be concise. So I think of the plans really in the destination document as uh the end state. And I don't need

that end state to be concise. Hopefully

that answers your question.

Uh, what do I think will be the outcome of the Mexican standoff of future roles of PMS and other roles converging? Uh, I

have no idea. I'm not a pundit. I have

no idea.

Uh, okay. So, we should uh after a couple of approvals uh end up with a set of issues. Now,

these issues that we're creating, they're designed to be independently grabbable, which means that this canon board ends up looking kind of like this

where you have essentially a set of tickets with a whole load of independent relationships.

So, this one needs to be done before this one. This one needs to be done

this one. This one needs to be done before this one. And this one, let's say we got another one over here. This one

needs to be done before this one. This

means that you can start to parallelize.

You can start to get agents working at the same time on these tasks because yeah, this one needs to be done first and then these two

can be grabbed at the same time by independent agents. Raise your hand if

independent agents. Raise your hand if you've done any kind of parallelization work with agents. Okay, cool. So this

allows you um to turn those plans into optimally kind of like into directed asyclic graphs essentially where you just are able to um essentially have

three phases here where you have phase one.

Let me grab move that uh above this line here you do this one.

Then phase two you do the two below it.

And then phase three you do this third one. and add it onto there. And when you

one. and add it onto there. And when you think about there could be this could this is a relatively simple plan but you could have many different plans operating all at once. It means that you can do really nice parallelization and

we'll talk more about that in a bit. But

that's why I prefer a canon board set up like this to a sequential plan because a sequential plan can really only be

picked up by one agent. So this where did it go? Over here.

Yeah, this plan here, this is really only one loop, right? Only one agent can work on these because we have numbered phases and they're not parallelizable.

Does that make sense? Cool.

So, we've got our issues. Ah, come on.

Stop asking me for Oh, no. It's creating

them on GitHub. I really don't want that.

Oh, no. You fool.

Create them in issues instead.

No, that's not precise enough. Uh, you

fool. Create them in local markdown files instead referencing the local version.

Sorry about this.

So, once we get to this point, we have a bunch of issues locally that we can start um looping over and implementing.

And it's at this point that the human leaves the loop. So, so far, let me pull up a a proper overview of this kind of flow that we're exploring

here.

So far, we have taken an idea, zoom this in a bit for the folks at the back, and we've grilled ourselves about the

idea.

We can skip over research and prototype, but we've turned that into a PRD into a destination document. We've then turned

destination document. We've then turned that PRD into a canon board and all of those steps are human reviewed. And now

the implementation stage, we step back and we let an agent um work through that camp board or multiple agents work through the camp board.

Now, what this means is that yeah, we've spent a lot of time planning here, but it means that we've queued up a lot of work for the agent. We can think of this as kind of like the day shift and the night shift. This is the day shift for

night shift. This is the day shift for the human, right? Planning everything,

getting all the uh all the stuff ready and then once we kick it over to the night shift, the AI can just work AFK.

But what does that look like?

Well, so I'm just going to Oh, yeah.

Just allow it. It's perfect.

So this looks like if we head to the next exercise which is uh in fact the last exercise here running your AFK agent.

Now I've called this uh Ralph really because it is a it is essentially a Ralph loop and this prompt here I want to walk through this really closely.

The first thing it's doing here is we're essentially going to run Claude and we're going to basically try to encourage it to work um completely AFK.

I'll show you what the sort of script for this looks like in a minute, but you say, okay, local issue files from issues are provided at the start of context.

The way we do that is if you look inside once.sh SH here inside the repo

once.sh SH here inside the repo we have uh it's essentially just a bash script where we grab all of the issues um which

are inside markdown files and we cap them into a local variable. So that

issues variable contains all of the issues that are in our entire backlog.

Then we grab the last five commits. I'll

explain why in a minute. And then we grab the prompt and we just run claude code with permission mode except edits and then just essentially just pass it

all of the information. This is what the implement looks like. So that's what a very very simple version of this sort of loop looks like. And of course this is not a loop. This is just running it

once.

The loop is in the AFK version up here which is uh a fair bit more complicated.

And the crucial part here is we're running it in Docker sandbox as well. So

I I don't want you to install Docker on your laptops because we're just going to be like you need to download a special image and we're going to tank the conference Wi-Fi if we do that. So I I am going to demo this to you, but you um

won't need to run this yourself. But

I'll talk through this in a minute. But

essentially this once loop here, we're just essentially running one version of the thing that we're going to loop again and again and again. So this

is kind of like the human in the loop version. And this is essential. Running

version. And this is essential. Running

this again and again is essential because you're going to see what the agent does and see how it ends up working. And any tuning that you need to

working. And any tuning that you need to add to the prompt, then you can do that.

Let's go to the prompt.

Um, so local issue files are being passed in. You're going to work on the AFK

in. You're going to work on the AFK issues only. That makes sense. If all

issues only. That makes sense. If all

AFK tasks are complete, output this no more tasks thing. And then the next thing, pick the next task. So

what we're doing here is we're essentially running a backlog or curating a backlog that our AFK agent is going to pick up. That's the purpose of

all of these um setups in the beginning in this uh all the way to this canon board here. We're just essentially

board here. We're just essentially creating a backlog of tasks for the night shift to pick up and the night shift this sort of Ralph prompt here.

It's got its own idea about what a good task looks like. So next pick up I'm I did talk about parallelization. I will

show you this later, but this is essentially a sequential loop here.

we're just going to run one coding agent at a time. This is a good way to just sort of um get your feet wet essentially.

So, it's prioritizing critical bug fixes, development infrastructure, then traceable bullets, then polishing quick wins and refactors. And then we just have a very simple kind of instruction

on how to complete the task. So, we

explore the repo, use TDD to complete the task. I'll get to that later.

the task. I'll get to that later.

And we then run some feedback loops. So

let's let's just try this and let's just see what happens. So good. It's created

the issue files. We should be good to go. I'm going to cancel out of this. I

go. I'm going to cancel out of this. I

clear and I'm going to run uh where is it? Ralph once.sh. And you

can feel free if you're following along to do the same thing.

So we can see it's just running Claude inside here with the prompt and with all of the issues that have been passed in.

And while it's doing its thing, you probably have some questions about this setup and about the decisions that I've made to essentially delegate all of

my coding to AI, right? So, let's let's do a quick Q&A while it's uh getting its feet under.

Uh okay.

I'm going to just remove those.

How do you retain negative decisions?

things that you decided against and ration when persisting the results from the Grommy session. A great question.

That's a very simple answer which is the in the PRD uh write a PRD section there is a stuff at the bottom a section of the things that are out of scope. So the

things we're not going to tackle in this PRD which is very important for giving a definition of done.

Feel free to ping on the slido if you've got any more questions.

Uh what's my front end workflow? Okay,

that's a great question. I'm gonna I'm gonna answer that in a minute, I think.

How to deal with agents producing more code that we can review? How to properly parallelize and use multiple agents in a separate way? Okay, that's um there's

separate way? Okay, that's um there's two questions there. Um raise your hand if you feel like you're doing more code

review now than you used to.

Yeah, definitely. Um I don't think there's a way to avoid this.

If we delegate all of our coding to agents, you notice that the implementation here is really the only AFK bit. We then also

need to QA the work and code review the work, right? And if we are running these

work, right? And if we are running these loops where it's essentially going to implement four issues in one, it's hard to pair that with the dictim that you

should keep pull requests small and self-contained, right? like small

self-contained, right? like small self-contained pull requests means you're needing to do fewer loops or shorter loops or something. Or maybe you do like a big stack of PRs, but that

seems horrible as well. That's still

just more separated code to review. I

don't honestly know what the answer to this yet. I think we just need to be

this yet. I think we just need to be ready to be doing more code review essentially, which is not fun. That's

not a fun thing to say. That's not like I don't know. I don't feel good saying that, but I do think it's probably the the way things are going. It's a great question.

Uh, can we grab a couple of questions from the room as well? Let's not we won't do the mic, but uh raise your hand if you've got a question for me immediately.

Yeah.

So, the approach looks very linear from an idea to QA.

Of course, the real world is a lot more messy. So you have all these ideas that

messy. So you have all these ideas that are in parallel and full picture and while you're working on something else comes in.

Yeah.

How do you deal with the messiness? How

do you feedback?

Great question. So the question was if this all looks great if you're a solo developer, but actually how do you implement this in a team? How do you gather team feedback on this? And my

answer to that is that if you have an idea up there and essentially the sort of journey from the idea to the destination is something you need to

figure out with the team, right? So all

of this stuff up here, this is kind of like team stuff, you know what I mean?

So if you have an idea and you do a grilling session on it and you have a question that you don't know how to answer, then you need to loop in your team as we described before. Then you

might need to go, okay, we just need to build a prototype of this. We need to actually hash this out. We need

something that the domain experts can fiddle with. Oh, okay. We might need to

fiddle with. Oh, okay. We might need to integrate a a third party library into this. We might need to do some research.

this. We might need to do some research.

We might need to actually kind of like um ping this back and forth and find a third party service that we can get the most out of. We might need to go back with the information that we gathered there to the idea phase. So all the way

up to the sort of PRD and the journey, that's something you need to involve your team with. That's something where these assets are going to be shared and argued over and you're going to have requests for comment on them and that

that loop is going to just keep grinding and grinding until you figure out where you're going. Once you figure out where

you're going. Once you figure out where you're going, then you can start doing the came on board the implementation.

But this is essentially super arguable and the you'll be bouncing back and forth between the phases. Does that make sense? Yeah.

sense? Yeah.

Would you not need a PR for your prototype?

Say it again. Sorry.

Would you not want to have a PR for your prototype? The question was, do you want

prototype? The question was, do you want to go through this whole session just to sort of create a prototype? Do you not need a PRD for your prototype as well?

Let's just quickly talk about prototypes for a second. Um, there was a question about how do you make this work for front end?

Like how do you because front end is like really sensitive to human eyes. You

need human eyes looking at the front end all the time to make sure that it looks good. AI doesn't really have any eyes.

good. AI doesn't really have any eyes.

It can look at code, but it front end is multimodal. And so my experiences with

multimodal. And so my experiences with trying to plug AI into um let's say agent browser or playright MCP to give it you can give it tools to allow it to

look through a front end and sort of look at images but in my experience the um it's not very good at that yet and it can't create a nice front end in a

mature codebase. It can sort of spit one

mature codebase. It can sort of spit one out. But what it can do is you say okay

out. But what it can do is you say okay uh I want some ideas on how uh this front end might look. give me three prototypes um that I can click between

in a throwaway uh throwaway route that I can decide which one looks best and you take the asset of that prototype and you then feed it back into the grilling session or you get feedback on it blah

blah blah blah blah answer your question kind of thing the prototype is just you know it's messy it's supposed to give you feedback early on in the process so that's a great way of working with front end code great way of looking at software architecture in general let's

go one more question yeah yes in your system How do you integrate respecting an architecture a design with API contracts and fitting with a larger system

security constraints? All kinds of

security constraints? All kinds of conraints like that.

Yeah, there's a lot in that question.

The question was how do you conform with existing architecture? How do you do um

existing architecture? How do you do um how do you make it conform to the code standards like of your codebase or Yeah. architecture design API security

Yeah. architecture design API security rules that constraints your designs.

Yeah.

I'm going to answer that in a bit if that's okay. So hopefully we have

that's okay. So hopefully we have started to get some stuff cooking.

It's just pinging on the explore phase here.

Tempted to just start running it AFK.

Maybe I will, maybe I won't. Um, what

it's essentially doing is it's exploring the repo. It's going to then start

the repo. It's going to then start implementing based on what we wanted.

Let's actually have one more question just while it's running. Yeah.

Yeah. So the question was why do you not get AI to QA?

AI to QA. I just got jargon overload for a second. Um why do you not get AI to uh

a second. Um why do you not get AI to uh test its own code? Now of course you absolutely can. And I think while it's

absolutely can. And I think while it's doing while it's cooking here, okay, it's got a clear picture of the codebase. It's assessing the issues.

codebase. It's assessing the issues.

It's doing issue O2 is the next task.

I'm again going to show you that in a bit. I think the sort of uh because you

bit. I think the sort of uh because you definitely should do an automated review step as part of implementation. So you

have your implementation. You should

then because tokens are pretty cheap and AI is actually really good at reviewing stuff. You should get it to review its

stuff. You should get it to review its own code before you then QA it. I found

that that catches a ton of different bugs. And

bugs. And the way that works is I will just do a little diagram is if you have let's say an implementation that's sort of like

used up a bunch of tokens in the smart zone. If you get it to sort of try to do

zone. If you get it to sort of try to do its reviewing, it's going to be doing the reviewing in the dumb zone. And so

the reviewer will be dumber than the thing that actually implemented it. If

we imagine this is the uh let's be consistent, that's the review. That's

the implementation.

Whereas, if you clear the context, then you're essentially going to be able to just review in the smart zone, which is where you want to be.

Let's see how our implementation is doing. Okay, good. It's generating a

doing. Okay, good. It's generating a migration. That looks pretty nice. We're

migration. That looks pretty nice. We're

getting some code spitting out.

And while I'm sort of like, aha, here we go. TDD.

go. TDD.

Let's talk about TDD and then I think we'll have a little another little break. TDD I found is absolutely

break. TDD I found is absolutely essential for getting the most out of agents. Uh raise your hand if uh you

agents. Uh raise your hand if uh you know what TDD is. Cool. Okay. TDD is

testdriven development. What it's

essentially doing is it's doing a something called red green refactor. And

if you look in the codebase, you'll be able to find a um a skill which really describes how to do red green refactor.

and teaches the AI how to do it. So what

it's doing is it's writing a failing test first. So it's saying, okay, I've

test first. So it's saying, okay, I've broken down the idea of what I'm doing and I'm just going to write a single test that fails and then I need to make

the implementation pass. I have found that first of all, this adds tests to the codebase and this this tends to add good tests to the codebase. And so we've got this kind of gamification service.

It looks like it's using some existing stuff to create a test database. Test

fails because the module doesn't exist yet. Okay, we've confirmed red. And then

yet. Okay, we've confirmed red. And then

it goes and hopefully runs it and it passes. I found that uh raise your hand

passes. I found that uh raise your hand if you've ever had AI write bad tests.

Yeah, it tends to try to cheat at the tests because it's sort of doing it in layers. it will do the entire

layers. it will do the entire implementation and then it will do the entire test layer just below it. Uh I'm

just going to say yes, you're allowed to use npxv text. And using this technique, it generally is a lot harder to cheat

because it's sort of instrumenting the code before it's then writing the code.

So I find that TDD is so so good for places where you can pull it off. And in

fact, it's so good that I sort of warp my whole uh technique around getting TDD to work better. I can see some drooping eyes. It is so hot in here. You can

eyes. It is so hot in here. You can

imagine how hot it is up here. Let's

take another five minute comfort break.

Let's come back at quarter two. I think

have a nice generous one. And we'll be back in about six, seven minutes and I'll talk about how uh I think about modules, think about constructing a codebase to make this possible. I've

just been sort of fiddling with the AI here and we have end up with some with a commit. So we have something to test.

commit. So we have something to test.

Issue number two is complete. Here's

what was done. This is kind of what it looks like when a Ralph loop completes is you end up with a little summary. Um

and we have now something we can QA because we did the feedback loops or because we did the tracer bullets because we were uh said okay give us something reviewable at the end of this we can immediately go and QA it. Now,

there's nothing uh less exciting than watching someone else QA something, but hopefully we can have a little play.

Let's just check that it uh works at all. In fact, before I go there, I just

all. In fact, before I go there, I just want to sort of work through what just happened, which is we see that it's created some stuff on the dashboard

and it then ran the feedback loops. So,

it then ran the tests and the types.

Now TDD is obviously really important and it's really important because these feedback loops are essential to AI essential to get AI to produce anything

reasonable because without this AI is totally coding blind right you have to have to um if if your codebase doesn't have feedback loops you're never ever

ever going to get decent AI decent output out of AI and often what you'll find is that the quality of your feedback back loops influences how good your AI can code. Essentially, that is

the ceiling. So, if you're getting bad

the ceiling. So, if you're getting bad outputs from your AI, you often need to increase the quality of your feedback loops. We'll talk about how to do that

loops. We'll talk about how to do that in a minute.

Now, so it ran uh npm run test, npm ran type check. It got one type error and it

type check. It got one type error and it needed to fix it with a nice bit of TypeScript magic. Very good. Yeah. Typo

TypeScript magic. Very good. Yeah. Typo

level thresholds number. Okay.

You see why I stopped teaching Typescript because just AI knows everything now. Um,

everything now. Um, so and it ran the tests and it passed and it's looking good. So we now end up with 284 tests in this repo. Pretty

good.

I I do find uh front end really hard to test here. We're essentially just

test here. We're essentially just testing the service. So we've created a gamification service if we look up here and then we have a test for that service. You can see the the service and

service. You can see the the service and the test itself. Now, if I was doing code review here, I would then go to re I would first go to review the tests, make sure the tests were testing

reasonable things and then go and kind of review the code itself just to make sure that it's it's not doing anything too crazy, right? The essential thing is I need to actually um look at the

dashboard. I'm going to log in as a

dashboard. I'm going to log in as a student. Oh, if it'll let me. Maybe it

student. Oh, if it'll let me. Maybe it

won't let me. Come on, son. There we go.

Let's log in as Emma Wilson. Head into

courses.

Uh, let's say I've got an introduction to TypeScript.

Continue learning.

Uh, yes, I completed this lesson.

Something went wrong. I imagine it's because I don't have uh SQLite error. I don't have the right table. So, I need a table point events.

table. So, I need a table point events.

Point events is a strange table name.

I'm not sure quite what it was thinking there. Uh, let's suspend. Let's run uh

there. Uh, let's suspend. Let's run uh npmdb migrate or push, I think.

Can't remember which one it was, but you kind of get the idea, right? I I'm not going to subject you to uh watching me do QA because it's so dull. Um but at this point, I would essentially go back

in. I would um let me open the project

in. I would um let me open the project back up.

Uh, and I would this this is a crucial moment. Um, and it's so important to um

moment. Um, and it's so important to um QA it manually here because QA Oh dear.

Oh dear. What's going wrong? There we

go. QA is how I then um impose my uh opinions back onto the codebase, how I impose my taste. What you'll often find is that um there are teams out

there who are trying to automate everything like every part of this process and they will tend to uh if you try to like automate the sort

of creation of the idea, automate uh the QA, automate the research, automate the prototype, you end up with uh apps that

I feel just lack taste and are bad.

maybe they just don't work or they they don't even work as intended or there's just no AI. You need a human touch when you're building this stuff because without that you just end up with slop and we are not producing slop here.

We're trying to produce high quality stuff and so that's what the QA is for.

So I'm going to do two things in this final section which is I'm going to first tell you how to there's probably a question in your mind here which is let's say I have a

codebase that I'm working on and it's a bad codebase. It's a codebase that's

bad codebase. It's a codebase that's like really complicated uh that AI just never does good work in and maybe actually most humans that go into that codebase don't do good work. How what

how do I improve that codebase? And the

second thing is I'll show you my setup for parallelization.

So let's go with um bad code first.

Now where is it? Where's the diagram?

Here it is.

In his book um the philosophy of software design, John Alistster talks about the ideal type of module.

And let's imagine that you have a codebase that looks like this. Each of

these uh blocks here are individual files. And these files export things

files. And these files export things from them. You know, they have um things

from them. You know, they have um things that you pull from the files that you then use in other things. And so you might have these weird dependencies where this file over here might rely on this file or might rely on that file for

instance. Now, if these files are small

instance. Now, if these files are small and they don't kind of ex like export many things, then John would call these shallow modules essentially where

they're not very um they kind of look like uh this. If I actually no I can't can't make a good diagram of it. They're

essentially lots and lots of small chunks. Now this is hard for the AI to

chunks. Now this is hard for the AI to navigate because it doesn't really understand the dependencies between everything. It can't work out where

everything. It can't work out where everything is. You know it has to sort

everything is. You know it has to sort of manually track through the entire graph and go okay this relies on this one relies on this one. This one relies on this one.

And it's then also hard to test this as well because where do you draw your test boundaries here? Do you test each module

boundaries here? Do you test each module individually?

Like just literally draw a test boundary. No, don't do that. Around this

boundary. No, don't do that. Around this

one and then maybe another test boundary around the next one and then the next one or should you sort of do big groups of it? Should you say, okay, we're going to

it? Should you say, okay, we're going to test all of these related modules together and just sort of, you know, hope and pray that they work.

Now this means that if I think that bad tests mostly look like that where the AI essentially tries to sort of wrap every tiny function in its own test boundary

and then just sort of test that those individually work. But what that does is

individually work. But what that does is it means that when let's say this module over here calls those two. So it depends on both of these. Then this module might

misorder the functions or there might be sort of stuff inside that poor module that's worth testing on its own. And if

you then wrap this in a test boundary, what do you do? Do you mock the other two modules? How does that work?

two modules? How does that work?

So actually figuring out how to um build a codebase that is easy to test is essential here because if our codebase is easy to test then our code our

feedback loops are going to be better and the AI is going to do better work in our codebase. Does that make sense? So

our codebase. Does that make sense? So

what does a good codebase looks like?

Look like well not like that. It looks

like this where you have what John Asterhout calls deep modules.

Modules that have a little interface on there that expose a small simple interface that have a lot of functionality inside them. Now

what this means is that these are easy to test because you just let's say that there's a dependency between this one and this one. My arrow working? Yeah,

there we go.

Then what you do is you just wrap a big test boundary around that one module around this one up here. And you're going to catch a lot of good stuff

because there's lots of functionality that you're testing and really the caller, the person calling the module is going to have a simple interface to work from. So it's not not too tricky. That

from. So it's not not too tricky. That

makes sense. Deep modules versus shallow modules. This is good. This shallow

modules. This is good. This shallow

version is bad. And what I find is that unaided um or if you don't uh if you don't watch AI carefully, it's going to produce a codebase that looks

like this. So you need to be really

like this. So you need to be really really careful when you're directing it.

And that's why too is that if we look inside the PD, uh where is the PR gone? It's inside the issues. It's inside the gamification

issues. It's inside the gamification system. Uh not found. Of course, it's

system. Uh not found. Of course, it's not. Here it is.

not. Here it is.

Then I have uh inside here data model the modules.

So it's specifically saying okay this gamification service is a new deep module which we're going to test around.

It's going to have this particular interface and it's going to have um okay we're modifying the progress service too. We're modifying the lesson route

too. We're modifying the lesson route modifying the dashboard roots etc. So, it's I'm being really specific about the modules that I'm editing and I'm making sure that I keep that module map in my mind at all times throughout the

planning and then throughout the implementation. That make sense? Very,

implementation. That make sense? Very,

very useful. It's useful for one other reason, too. Not only does it make your

reason, too. Not only does it make your app more testable, but you get to do a little mental trick.

And I'm going to refill my water while you wait for what that is.

Uh, let me Let me get a question from you guys. So,

raise your hands if you feel like.

Uh, if you feel like you're working harder than ever before with AI.

Yeah. Uh, raise your hands if you feel like you know your codebase less well than you used to.

Yeah.

This is a real thing. um because we're moving fast, because we're delegating more things, we end up losing a sense of our codebase. And if we lose the sense

our codebase. And if we lose the sense of our codebase, we're not going to be able to improve it. And we're

essentially delegating the shape of it to AI. I don't think that's good. But

to AI. I don't think that's good. But

then how do we how do we make it so that we can move fast while still keeping enough space in our brains? I think that this is a way

our brains? I think that this is a way to do it because what you're doing here is not only are you thinking about creating big shapes in your codebase,

big services.

What I think you should do is design the interface for these modules, but then delegate the implementation.

In other words, these modules can become like gray boxes where you just need to know the shape of them. You need to know what they do and sort of how they behave, but you can delegate the implementation of those modules. I found

this is really nice. I don't necessarily need to co-review everything inside that module. I don't necessarily need to know

module. I don't necessarily need to know everything of what it's doing. I just

need to know that it behaves a certain way under certain conditions and that it does its thing. So, it's kind of like, okay, I've got a big overview of my codebase and I understand kind of the

shapes inside it, understand what the interfaces all do, but I can delegate what's inside. I found that has been a

what's inside. I found that has been a really nice way to retain my sense of the codebase while preserving my sanity.

Make sense?

And so you might ask, how do I take a codebase that looks like this and then turn it into a codebase that looks like this? How do I deepen the modules? Well,

this? How do I deepen the modules? Well,

we have hopefully it's in here. Pretty

sure it is. We have a skill and that skill is called improve codebase architecture.

Nice and direct.

Uh let's run it. What this skill is going to do is it's essentially just going to do a scan of our codebase and looking for what's available here. And

feel free to run this yourself if you're um uh running the exercises. And it's

exploring the architecture, exploring um essentially how to work within this codebase. and it's going to attempt to

codebase. and it's going to attempt to uh find places to deepen the modules.

Pretty simple. One really cool um thing that it found here is part of my uh part of my course video manager app is a video editor. A video editor built in

video editor. A video editor built in the browser, which is really hardcore.

Uh it's a decent bit of engineering. And

I wanted a way that I could wrap the entire front end all the way to the back end in like a single big module so that I could test the fact that I press something on the front end and it goes

all the way to the back end. And so I found a way essentially by using a kind of discriminated union between the two types here by sort of I was able to use

this uh skill to essentially have a huge great big module that just tested from the outside or was testable from the outside this video editor infrastructure. And it meant that AI

infrastructure. And it meant that AI could see the entire flow, could act on the entire flow and test on the entire flow. And honestly, it was just night

flow. And honestly, it was just night and day in terms of the uh ability of AI to actually make changes because AI working on a video editor is pretty brutal if you don't give it good tests.

So that is honestly I if you take one thing away from today, just try running this skill on your repo and see what happens. Let's go to slider. Let's ask a

happens. Let's go to slider. Let's ask a uh check a couple of questions just while this is running.

So let's see. Have you tried claude's auto mode with claude enable auto mode?

Uh that way you can avoid many of the obvious permission checks. We'll talk

about permission checks in a second. Do

I keep the markdown plans and issues for later reference?

Okay, this is a great question. So

let's say that you uh have a great idea, you turn it into a PR raise and you then implement that PRD and the PRD is essentially done. Raise

your hand if you keep that information in the repo. So you turn it into a markdown file. Raise your hand if you

markdown file. Raise your hand if you want to keep that around.

Cool. Okay. And raise your hand if you if you don't want to keep it around. If

you want to get rid of it as soon as possible. Yeah. This is I think an

possible. Yeah. This is I think an a question that doesn't have a clear answer. What I'm really scared of

answer. What I'm really scared of with any documentation decision is that let's say that we have a PRD for this gamification system. We keep it in the

gamification system. We keep it in the repo. We go on, go on, go on. Let's say

repo. We go on, go on, go on. Let's say

a month later, we want some edits to the gamification system. And we go in with

gamification system. And we go in with Claude and it finds this old PR and says, "Yes, I found the original documentation for the PRD system." Well,

it turns out that the actual code has changed so much from the original PRD that it's almost unrecognizable. The

names of things have changed. The um

file structure has changed. Even the

requirements may have changed. We might

have actually tested it with users. This

is dock rot where the documentation for something is rotting away in your repo and influencing claude badly or claude agents badly. So I tend to not keep it

agents badly. So I tend to not keep it around. I tend to get rid of it. And for

around. I tend to get rid of it. And for

me because my setup uses GitHub issues, I just mark it as closed. It can fetch it if it wants to, but it's got a visual indicator that it's done. So I tend to prefer ditching these.

Thoughts on the beads framework from Steve? Uh I've not tested it, but it

Steve? Uh I've not tested it, but it seems like sort of um another way to manage Canvan boards and issues. Seems

uh very good, but I've not tried it.

Um uh let me just quickly check the uh setup here. Let's take a couple of

setup here. Let's take a couple of questions from the room. Anybody got any questions at this point about anything that we've covered so far, especially this last bit? Yes.

like code. How about migrations? Like

with migration files, we can also squash them off like database migrations.

Yeah, I don't know.

I hope that answers your question. I'm

so sorry. No, no, I think database migrations are a different thing because you have a sort of running record of exactly what changed and it's more deterministic and I think yeah, it's an interesting analogy. I'm

not sure. Let's talk about it afterwards.

That's a good way of saying I have no idea.

Yeah. Yeah.

Sorry guys. Um I'm just trying to listen to this guy's question.

Yeah. The question the question here is um should I um in the sort of early planning stage be trying to optimize the plan? This is something I actually see a

plan? This is something I actually see a lot of people doing and it's a really good um idea. So when you let's go back to the phases. So let's

say that you have all of these phases here and you uh you get to the point where you've sort of figured out everything with the LLM. you understand where you're going. You've created this sort

you're going. You've created this sort of journey destination document here.

How do you then uh like should you then try to optimize and optimize and optimize that PRD until it's the perfect PR you can possibly imagine? I don't

think there's a lot of value in that because I think the journey is really just sort of a hint of where you want to go and the place that you need to be putting the work is in QA and you can

sort of do that AFK I suppose but in my experience you're not going to get a lot of juice out of it like it's the the thing that really matters is getting alignment with the AI which is you do in

the grilling session initially.

Let's have one more question. You got

any more? Yeah. How do you get in your workflow to get it to code the way you want it to code? So by the time you get to code review, it's at least familiar, use the libraries you wanted to use.

Yeah. Um, we had this question before actually, which was like uh how do you uh enforce your coding standards on the agent? Essentially, how do you get it to

agent? Essentially, how do you get it to code how you want it to code? Now,

there's essentially two different ways of doing it. Um, you've got Come on. Push

Come on. Push and you've got pull.

What do I mean by push and pull?

Um, push is where you push instructions to the LLM. So you say, okay, if you put something in claw.md,

uh, talk like a pirate, that instruction is always going to be sent to the agent, right? So that is a push action. You're

right? So that is a push action. You're

pushing tokens to it. Pull is where you give the agent an opportunity to pull more information.

And that's for instance like skills. So a

skill is something that can sit in the repo and it has a little description header that says okay agent you may pull this when you want to.

My thinking my current thinking about code review and about coding standards looks like this. when you have an implement.

What's going on? There we go.

Implementer.

I'm going to make this less red in a second. Um, then you want the coding

second. Um, then you want the coding standards to be available via pull. If

it has a question, you want it to be able to sort of answer it. But if you then have an automated reviewer afterwards, then you want it to push.

You want to push that information to the reviewer. You want to say, "These are

reviewer. You want to say, "These are our coding standards." um make sure that this code um follows them. So if you have skills for instance, then you want to push that stuff to the reviewer so

the reviewer has both the code that's written and the coding standards to compare to.

Hopefully that answers your question. I

can show you an automated version of this as well. Actually, um yeah, let's do that now just while it's fresh in my mind. I recently um spent

mind. I recently um spent uh maybe a week or so uh building this thing called Sand Castle. And Sand

Castle is a I was sort of unhappy with the options out there for um running agents AFK. And what this does is it's essentially a TypeScript

library for running these loops. So you

have uh a run function that creates a work tree um sandboxes it in a docker container and then allows you to run a prompt inside there. And in that work

tree then it's just a git branch and you have that code and you can then merge it later. If I open up

later. If I open up um there are some really really nice ways of viewing this and it essentially allows you to run these kind of

automated loops and allows you to parallelize across multiple different agents really simply. So I'll go into my sand castle file go into main.ts here

and let's just walk through this.

So this is kind of like I showed you um a sort of version of the Ralph loop earlier. This is where we take it from

earlier. This is where we take it from sequential into parallel.

We have here first of all a planner that takes in it's has a plan prompt here that looks at the backlog and chooses a certain number of issues to work on in parallel. Remember I showed you that

parallel. Remember I showed you that canon board where it had all the blocking relationships. It works out all

blocking relationships. It works out all of the phases. So this one will say okay uh let's say we have uh you can ignore all this glue code here. This is

essentially just a set of issues, GitHub issues with a title and with a a branch for you to work on. And then for each

issue, we create a sandbox and then we run an implement in that sandbox passing in the issue number, issue title and the branch. This is like the loop that we ran just before.

Then if it created some commits, we then review those commits. This is

essentially the loop. What do we do with those commits? We pass those into a

those commits? We pass those into a merger agent which takes in a merge prompt, takes in the branches that were created, takes in the issues, and it just merges them in.

If there are any issues with the merge, you know, with the types and tests and that kind of thing, it solves them. And

this has been my uh flow for quite a while now for working on most projects.

It works super super well. And uh yeah, I recommend you check out sand castle if you want to sort of learn more. And to

answer your question properly is that in the reviewer uh I would push the coding standards in the implement I would allow it to pull.

And I'm actually using uh sonet for implementation and opus for um reviewing because I consider reviewing sort of I need I need the smarts. Then

any question? Actually, let let me uh before we do more questions, let's go back here. Okay, where are we at? Okay,

back here. Okay, where are we at? Okay,

we're sort of zooming everywhere in this uh talk because I'm kind of having to run things in parallel. So, let's go back to the improved codebase architecture. It has finally finished

architecture. It has finally finished running and it's found a bunch of architectural improvement candidates. So

it's got essentially a cluster of different modules that are all kind of related that could probably be tested as a unit. Got number one the quiz scoring

a unit. Got number one the quiz scoring service. There's some reordering logic

service. There's some reordering logic extraction as well. It has arguments for why they're coupled and it has a dependency category as well. So local

substitutable in SQLite within memory test DB quiz scoring service currently has zero test. This is the biggest gap. So this

test. This is the biggest gap. So this

is what it looks like when we come back of uh improved codebase architecture.

Okay.

So we have nominally kind of 17 minutes left. I don't know about you, but I'm

left. I don't know about you, but I'm knackered.

Um I want to let let me kind of sum up for you because I think we're sort of reaching the end of our stamina. I'm going to be available for the full time if you want to um come and ask me questions. Um, I

might do one more check of the slider, but let's kind of sum up where we've got to.

So, this is essentially the flow where throughout this whole process, we're bearing in mind the shape of our

codebase. This is not a specttocode

codebase. This is not a specttocode compiler. This is not an AI that's sort

compiler. This is not an AI that's sort of just like churning out code. We are

being very intentional with the kind of modules and the shape of the codebase that we want. We are making sure that we are as aligned as possible by using the grilling session by really hammering out

our idea. We're not overindexing into

our idea. We're not overindexing into the PRD. We're not trying to read every

the PRD. We're not trying to read every part of it. We're not thinking too much about it even. We're then just turning that into a set of parallelizable issues which can be worked on by agents in

parallel. We implement it and we QA and

parallel. We implement it and we QA and code review the hell out of it and then keep going back to that implementation.

One thing I didn't really mention is that in the QA phase, what the QA phase is for is creating more issues for that canon board. So while it's implementing

canon board. So while it's implementing even, you can be QAing the stuff and going back adding more issues. And the

canon board just allows you to add blocking issues kind of um sort of infinitely really. And then once that's

infinitely really. And then once that's all done, once you've got code that you're happy with, once you've got work that you're happy with, then you can share it with your team and you can get a full review. So this is kind of like once you get here, this is kind of one

developer or maybe a couple of developers sort of managing this and then it's kind of up to you to figure out how to merge it back in.

Of course, all of this can be customized by you. This is just something that I

by you. This is just something that I have found works. I'm not trying to like sell you on a kind of approach here.

What I recommend if you take one thing away from this session is that you should head back you should head to Amazon and just buy a ton of those old books because I mean I just found it so

enlightening reading them. Uh you know preai writing is always like a really fun to read anyway and I just on every single page I found that

there was something useful and something interesting to to read. So thank you so much. Thank you for putting up with the

much. Thank you for putting up with the heat. Um hopefully your body

heat. Um hopefully your body temperatures will reset soon. Uh thank

you very much.