How I Review AI Code - (Meta Senior Staff Engineer)

Well, hello there.

So are you even reading your code these days?

Or do you just send it off and just let your vibe code it, AI slop code, just into the wild without any review?

Or are you still on the camp of reading all your code?

In my opinion, this topic of code review is very split right now.

The community is very split.

There's a group of people who said, if you're reviewing your code still, then you're not moving fast enough.

The agents are probably already better at reviewing code than you are.

So why are you reading your code?

Those are the that camp.

The other camp is people who are saying, no, the AI agents are still not great yet.

It's not perfect, it makes dumb mistakes and you're creating a lot of AI slop and you really need to tone it down and review the stuff that you're writing.

So there's these two camps.

And in this video, we're gonna talk about what I think of this topic.

And I'm also going to give you my strategies on how I navigate and review my own code and land high quality code into the production.

So yeah, that's what we'll cover in this video today.

So if you're new to this channel, my name is John.

I'm a senior staff software engineer at Meta.

And on this channel, I talk about AI tech news, AI tech tutorials, anything to do with AI.

It's my fun side hobby to dig into all of this.

But yeah, a lot of you guys are not subscribed.

You guys watch a lot of my videos, but for some reason, don't subscribe.

So do a brother a favor and subscribe.

I get a ton of tokens if people subscribe.

I don't actually, but please subscribe.

So let's get right into this video.

So as usual, I made some slides, just some simple slides to have some talking points over all of this.

And for this video, I also created just a simple website that like you can swipe essentially to vote whether something is AI or Not.

And they will tell you like if it's AI or not.

It's just a simple demo app and it's really there so I could showcase some of the tools and like things that I'm thinking about for the video.

So let's start off with the obvious.

I think we can all safely say with the advent of AI coding, the amount of code that people are generating is going through the roof.

It's like so much more code.

People are writing a lot more features.

They're writing more tests.

They're just doing a lot more because one person can split up multiple agents and generate a ton of code.

And to be honest, it's almost impossible of a task for a single engineer to review even their own code, let alone do code reviews for other people.

So where do I stand right now personally for code reviews?

And the answer is, it depends.

I think like any good answer to any hard problem, answer is really, it depends because the reality of code reviews is a gradient.

It really depends on the code that you're reviewing.

So when you're writing code and when you're building a feature, the AI agents should be part of the entire code's journey to production from planning, building, reviewing, and launching.

Every single part of this, you should actually include your AI agents throughout the whole process, right?

Whether it's planning out a spec to build out, making sure things are feature complete and making sure you have like evaluations and like validation criterias that you're thinking about ahead of time.

And when you're building the code, you wanna have agentic validation so that the AI can self validate and self fix itself and then get the code right a lot more often.

And then we have the review, which is like what we're talking about today.

And we're gonna talk about some strategies on how you should do this.

And then when you're launching, I think this also should be part of it.

You wanna feature gate and run experiments on your code.

And all of these things I think tie together to answer the question, when and how much should you review a code?

So it really depends, right?

Now, the big concept that I wanna let you guys know about today is that I think of the code base like a tree.

And like a tree, there are different parts of the tree that is more important.

So think of like kind of the trunk, the root, the main part of the code, that's like your main entry points, like your App.js that has all of the different helpers that are like added to it.

And then your infrastructure code, like image processing, networking, and all these things that if something goes wrong in that code base, it can affect the entire app.

I think of that as like a trunk piece of code.

And then there's like leaf nodes.

And we're gonna get into more of this topic throughout this video, but start thinking about your code base as a trunk.

And as soon as you start thinking about this, then the obvious next thing that you naturally think about is that the review depth, the depth that your own effort level on reviewing a certain code, it should really be based on like the blast radius of that code change.

As an example, if you're touching something that's like core image rendering infrastructure that touches all the different image renderings, and there's a lot of downstream dependencies, then I would spend a lot of time reading that piece of code, for example.

And then for any like one off leaf node code, you could probably just have agents do simple validations like component testing or snapshot testing and just visually validate the PR as long as they have proof in the PR.

So this concept of like changing your effort level, depending on the importance of the code, I think really matters a lot.

And you should actually do this planning from the beginning.

So when you're planning the entire feature, you actually want to feature gate it.

Now, if you don't know what feature gating is, it's essentially adding a branching logic in your code base where you can say, hey, this feature is on or off.

So there's a concept called feature toggling, a really popular option is called LaunchDarkly, and you could have feature gating.

There's a lot of actually infrastructure that you need to do feature gating properly, but you can start out really simple by just adding a Boolean flag in your code base.

It's a gating code and you just turn things on and off.

So from the beginning, if you're planning a new feature, think in broad strokes and gate all of the things out, and then think a lot about the integration pieces.

So you want to separate out leaf nodes, things that doesn't touch any of the existing code bases, and then you want to think about integration separately.

You really want to like gate those integration layers.

So once you have like a good solid plan and a good like roadmap, and you have like good feature gating, like ready to go, then you can start coding.

And then this coding part is like, you know, just use agents to build a code out, and then you can just use the agents to like, you know, make atomic changes.

I have a ton of videos around using agents to code, but you know, we're not covering that particular topic, but once you're ready for a PR, what should go in that PR?

So let's think about that.

So in my opinion, for your diffs to be really like solid, your PRs to be like rock solid, you really want to anchor and push your agents to give you proof.

And there's many different types of proof that the agents could give you.

There can be just straight up unit tests.

So as an example, something like this, it's just a straight up unit test, very simple, a lot of tests.

And this is like just reading the test names and just skimming the tests and the asserts.

You could quickly get some like good sense of, oh, there's like good logic testing that's happening.

One side note is you want to be careful here when the agents are writing tests.

You should probably make your own skill around not writing useless tests.

That's like a little side tip.

And then you also want runtime evidence.

So let's say that you have a lot of like runtime logs.

And I talk about this in my agentic engineering video, where you want to really anchor on giving more visibility and more eyes for the agents.

So you can have runtime evidence and then you could have visual evidence where maybe it's a screenshot.

And then I also ask the agents to give a confidence level on their code and their gating and like different parts of the code.

So you really want to push your agents when you're ready to like push out a PR on these kinds of things.

It's like, do you have evidence?

Obviously you want to build these into your skills.

So let's take a look at like UI proof, right?

So one of the things I like to do every time I make changes with the AI, I always ask it to have a video or a UI screenshot, right?

So for this one, this lets the player save a card, right?

So you can see right here, there is a save button.

So I can see that, oh, okay, this was saved.

And here's some like simple code changes.

And then it looks roughly right.

But the point is I see a visual proof.

So immediately I'm a lot more believing of this AI's code output, right?

So if you have the visual proof and if you know that it's safe, then it's a lot safe to land.

So having proof and having more proof lets you move faster because you don't have to manually check these proofs.

And this is really around agentic validation.

And I talk about that in my agentic engineering video, but agentic validation is a core concept of being able to read less code.

Because if you have a lot of these proof upfront on the PR, then you don't have to read as much code.

Now the next big concept, and this is kind of in the tree concept is leaf code.

A leaf code is really code that is written in isolation or it's completely gated off in a feature gate.

So this can be like one off components that you can just validate with like a component test or a snapshot test, right?

Or maybe it's new infrastructure, new logic, maybe a new endpoint that you're integrating with.

And you could just validate that with unit tests or integration tests.

And as long as they're not integrated to the main production code base, you can move really quickly.

And then I'll cover later what you should be doing.

So for me, when I'm reviewing leaf like code, I spend a lot less time.

I'm almost just mainly skimming the code and making sure things are not like very off or like something is just obviously wrong.

And for example, what can be a trunk is like, you know, in the reducer.

Reducer has a lot of shared state.

So let's say you change some behavior here in this like game action advanced cards.

Maybe there's like a new card type or something like that.

And you're changing existing behavior.

And if that behavior is not gated out, then I will spend a lot more time reviewing it because that changes production right?

And then this has trickle down effects and side effects and all these things can really matter and affect production code, right?

So you're really anchoring on safety.

So we kind of covered like thinking about code as like this gradient and how you should anchor on changing your effort level right?

And in my opinion, everything should be anchored on safety.

What is the control experience that should not change?

And if this change went wrong, how bad can it be?

And then can I roll this change back?

And then that really just depends on feature gating, right?

So if you can answer these questions, especially the last one, then you can spend a lot less time reviewing each individual code.

Now, one other thing that is like not that important in my opinion these days that is part of like usually reviews is like nits and like little like different ways of writing.

And I would just leverage like static checking here to be able to do a lot of that work.

So like type checking, linting, and all those kinds of things.

And agents are really good at doing stuff like this.

So we have a Codex reviewer here and it's like seeing some linting issues.

So it's like saying we should fix it as an example.

And here's some like stylistic changes, right?

So one big thing about code review is that I think the old days of like having your nits and then like, oh, the code should look this way.

This is my favorite way of writing code.

I think that's kind of dead.

I think we write so much code and code is being a lot more fluid and refactors are happening a lot more often.

And it's a lot easier to do, especially once like there's good validations for the code that you wanna refactor.

I think these kinds of nits don't really belong in the conversation of code reviews anymore.

So going back to the original question is, do I read the code?

The simple answer is yes, I still read a lot of code.

I think I read more code than I did before, but I definitely have this like knob where I first asked myself, is this a core change, like a trunk change?

And is it like an infra?

Does it have a lot of like dependencies?

If so, I increase it.

And then I say, is this safe?

Is there gating?

If something went wrong, like what can happen?

So all of these things really change how much I review the code, right?

And then the more dangerous it is, the more I spend time.

More of that changes a one-way door, I spend more time reviewing, right?

Now let's get kind of to the fun parts of like looking at how you can use agents to kind of help you review your code.

Now, in my opinion, AI is really good at reviewing code already.

I actually do think it's better than most humans.

The issue is a lot of people are doing it wrong.

And in my opinion, there's a few things that you should really anchor on doing.

One, you should have a really good review skill.

I think both Claude Code and Codex have really good built reviews.

And then here is the review agent right here.

And then for Claude Code, it's slash code review.

So these are all both really good review agents in my opinion.

But one thing that's kind of important to do is to use a new sub-agent or a different agent, an agent that doesn't have the context of the code that it was used to create the code.

Because the agents will cheat.

You know, all the context that you gave it, it will cheat.

And they will really try to anchor on like a lot of the previous conversation.

So you really want to review the code with like a different sub-agent or like a separate agent that is called an adversarial review agent.

And when you have an agent write a diff or thing, just have like a good template that the agent can pull from.

So your PRs look the same, they feel the same.

They have the important bits.

If you don't give any guidance on what the agent should really have, it'll write you like a giant paragraph, like a story.

And honestly, that's my biggest pet peeve right now is when people submit PRs or diffs to review where the summary and test plan is longer than the code changes itself.

So don't be that guy.

Now I showed this off a little bit, but all the agent stuff that I've been talking about was during runtime, during the generation of the code.

And that's what you're running.

But then Codex actually offers this integration right here where you can add a Codex reviewer onto GitHub and it could run on new GitHub, like, you know, PR opens, or you could also like just add Codex here like that.

And essentially it's this chat to be the Codex connector.

And I think there's a separate model slightly that is custom trained just for like reviewing.

Now all of this costs money, right?

But yeah, that's an example.

Another thing you can do is actually use goals as an example.

So one of the common issues that people will run into when they start having these agentic systems that are reviewing their code on GitHub is that after you submit a PR, the agent will go do stuff.

And then it'll give you a bunch of comments that you need to fix.

And sometimes there'll be a lot of back and forth.

You fix it and then like you resubmit and then it'll be like back and forth.

So a thing you can do is like just use goals to babysit a PR.

So let's say we had this like random PR.

This is a draft, but you could just say for this PR, can you babysit it every hour, check it every hour and make sure that you address any changes requests from the Codex agent.

It misspelled it, but you get the point.

So what this will do is every hour it will check that PR and then it will fix any issues, rerun your local validations, agentic validations, update the PR notes and then resend it.

So these are the kinds of things that you should build into your own skill if you haven't done so.

But yeah, so those are kind of like the high level like useful tools in my opinion.

It's just really important to use adversarial agent in my opinion, have good review skills, actually just like the default like skill reviewers, have a template for your PRs and you could use my template if you want, but just have a template so it conforms your PRs to that template and then it adds like validation proof.

We talked about the different types of validation.

You really want to anchor on the agents giving you proof that will actually help your AI coding by the way.

And then you just submit all of that.

And then you make sure you anchor on safety and gating.

So these are kind of the high level things that I think about.

Before we end this video, I also wanna talk about what it means if you're coding like this, where you're reading some of the code deeply and you're reading like leaf nodes kind of like skimming it as you're going.

The end product of that entire feature at the end of your coding session, like you're like, oh, I'm done.

If you actually code in this way, you'll be only actually like 80% done.

So here I'm saying, even if it's merge ready, I don't think it's launch ready.

So what I like to do is I call this initial step of like getting everything gated and landed.

I call that like broad strokes.

I'm doing like broad strokes of the code changes.

And then this last like 20% or so is really getting something launch ready.

And this is where a lot of my human like effort comes in.

And then I do a lot of refactoring here and kind of like cleaning up of the code base right?

So I actually have a bunch of agents before I do any launch, I like audit the entire code that it was written.

And I think about it from high level.

And I think about like important things like performance, bugs, security, you know, and then whether the feature actually works to spec.

And then I'm really like detailing the animations or anything like that.

And so this last 20% actually takes as long as the first 80%, but it's actually the thing that is the most important.

And in my opinion, you're able to move so quickly in the beginning, but this last 20% still requires a lot of like, like human taste factor.

So you still need to look at it.

And this is where my code kind of becomes a lot more clean and production ready.

Now, finally, when you're ready to launch the feature, I highly encourage you to try to run experiments.

So you could like validate that the code is actually working.

Now, if you have a small user group, then you can't really run a sizable experiment.

You don't have statistical like power to even like run legitimate A/B testing, but I still recommend running A/B tests or doing like canary deployments, because what that helps you is if you trickle your traffic for your new feature through like feature gating, then you're able to very quickly see bugs or crashes or alerts that fire.

And that's the whole point.

You wanna anchor always on safety when you're using agents to code, because physically you won't be able to physically read everything in detail.

It's just impossible now.

So the idea of high level is you wanna create like an agentic safeguard, like an agentic garden, where you have a lot of tests, you have a lot of validation, and you have a lot of agents doing a lot of work to make sure your code is good.

But at the same time, you wanna anchor on safety and make sure you can take back bad decisions with gating.

So do I review code?

I think the simple answer is yes, but it really depends.

And it's a gradient, right?

But in my opinion, the models are getting better and better.

So you will probably review less code, but you should really anchor on validation, agentic validation.

And then the agent should like have a lot of validation and proof and evidence of the proof, like whether it's video recording or logs or like just screenshots, whatever it is.

And then you gate, and then you like start trusting the agents more.

And as long as you have like control to take things back with gating, then it should be safe to go.

So yeah, I hope you guys enjoyed this video on agentic code reviewing and like my thoughts on code reviewing.

The thing is this topic keeps changing for me.

Right now, I'm comfortable with the skimming of some code, depending on like where it is and the like tree concept that I talked about.

But maybe after another few generations of the model, I might say, you know what?

You don't need to read any of the code.

So who knows?

But right now I think you still need to like read some of the code and have like a fine tuned opinion of it.

Now, one last thing that I forgot to mention is that if you understand your code base, it's actually easier to review the code because you can very quickly spot what's touching like the dangerous zones of your code base and what's like really simple and leaf nodes, right?

So yeah, that wraps up this video.

I hope you guys enjoyed it.

I do a ton of these videos.

I did a video on like Codex and Claude Code, which I think would be very relevant for this video.

So feel free to check that out.

But until I see you on the next one, bye.