WritingOriginally published on FDE PodRead it there
Applied AI14 October 20267 min read

The AI Software Development Life Cycle

By Kevin Bai

I recently delivered a keynote and led a workshop on the AI software development life cycle for 50 CTOs and technology leaders from private equity backed companies, hosted by KKR and co-hosted by AWS. I have been giving a lot of these, and this room was engaged from start to finish.

Kevin Bai presenting the AI software development life cycle to a full room of CTOs and technology leaders at KKR

Private equity backed companies have a reputation for moving slowly. In 2026 that could not be more wrong. These companies are built around efficiency and value creation, and the clearest path to both is adopting AI in a way that makes employees smarter and more empowered to do their best work. That makes how you spend on AI a question they care about a great deal.

Most companies I talk to about building with AI are making one of two mistakes. Some build their first agent on a cheap model to save money and never find out what it could have done. Others got a great prototype working on the biggest model available and are now paying frontier prices to run it in production, a million times a day, on work a smaller model could do.

Kevin Bai at the KKR workshop

Both are treating model choice as a one-time decision. I think it is a life cycle, with two phases that want opposite things. In development you want the biggest, smartest model you can get. In production you want the right size. Most companies with no AI direction miss both. The ones that have a direction usually have only one of them.

Don't take it from me. Take it from Mikhail Parakhin, the CTO of Shopify, who has a name for it.

The barbell

Near the end of that conversation, Parakhin gives his advice to every other CTO. "Always use this barbell strategy," he says. For development, testing and research, use the largest model. For production, use the right model, often a cheaper one fine-tuned for the job. "Save money on production by using cheaper models, and invest that money in development."

Then the line that made me want to write this post. "I've seen repeatedly people doing the opposite," he says: saving money on coding tokens in development, while "wastefully running models larger than they need" in production. "I feel like I will outcompete anybody who is not on that strategy."

A barbell. The heavy plate on the left is development: the frontier model, maximum effort, no token budget. The plate on the right is production: the right-sized model, tuned effort, often fine-tuned. The bar between them is evals.

A barbell is heavy at both ends and thin in the middle. The mistake is the middle: one model, picked once, used everywhere. For any given job it is either too small to show you what is possible or too big to run cheaply, and usually you cannot tell which.

That does not mean the middle models are off the bar. The ends are roles, not model names, and a model like Sonnet can sit at either one depending on the use case. Build a coding agent on the largest model, and Sonnet may well be where its evals say it can run in production. Build a live voice agent that has to answer in under a second, and the largest model that meets the latency budget is your frontier for that job. That might be Sonnet too. What matters is that you choose each end on purpose, per use case, instead of picking one model for everything.

Develop on the frontier

When you are doing greenfield AI development, you do not know what you do not know. You do not yet know whether the task is possible, what the agent needs to see, or where it will break. A smaller model fails in ways you cannot tell apart from the task being too hard, so you give up on ideas that would have worked.

Parakhin puts it as floor and ceiling. AI raises the floor: work that used to need more diligence or more people now gets done. It also raises the ceiling: work that was not possible at all now is. "Raising the floor, you could have just been more diligent or thrown more people at it," he says. "Ceiling, there's no easy way to extend." Only the largest model shows you where the ceiling is. "You don't know unless you use the large model and compare the results. You don't know what you're missing."

The workshop room at KKR, tables of technology leaders working on laptops

Shopify acts on it. Everyone there has unlimited tokens and runs the heaviest models, with circuit breakers for runaway processes that start burning tokens on their own. The constraint in development is your imagination, not the bill.

Then make it cheaper

Once you have something that works, and works well, the job changes. Now you make it cheaper.

This is where most teams that got the first half right stall. The frontier model did more than answer the question. It helped you build the harness: the prompts, the tools, the context the agent reads, and the evals that tell you when it is right. A smaller model running inside that harness is a very different thing from a smaller model starting cold. The model you need in production is often not the model you needed in development, because the bigger model already did the hard part.

Kevin Bai and two others at the workshop

I call this rightsizing: picking the right model and the right effort level for each piece of the work. Not the biggest one you can afford, and not the cheapest one you can find. The smallest one that still satisfies the workflow, proven against your evals.

Reach for effort before you reach for a model

Most frontier models now let you set how hard they think: how long they reason, how many steps they take, how much they check their own work. That dial is the half of rightsizing most people skip.

Think about how you staff a team. There are tasks a junior employee can do well if you just give them more time. There are tasks you have to hand to a senior, no matter how long the junior gets. A good manager knows the difference, and does not send every hard task to the most expensive person on the floor.

Most people reach for a bigger model before they reach for more effort. Often the smaller model at high effort clears the bar, at a fraction of the cost of the bigger one. Sometimes it does not, and that is the task telling you it needs a senior. Either way you only know by testing.

A grid of three models, the senior to the junior, by three effort levels. Cells that pass the evals form a staircase: the large model passes at every effort, the medium model at medium and high, the small model only at high. The cheapest passing cell, the small model at high effort, is marked as the pick.

The best way is to build evals and test every combination of model and effort. If you cannot do all of them, test at least three effort levels per model: low, medium and high. The pass line usually forms a staircase, and the cheapest step on it is your production setting. Testing one effort level per model is how you end up paying for a senior to do a junior's job.

It is a cycle

The reason I call this a life cycle is that it never stops. A new frontier model ships and the ceiling moves. Go back to development on it, find what is newly possible, and build the harness for it. Then rightsize again, and you will often find that last year's frontier work now runs on this year's small model.

A two-by-two. Across, whether you develop on the frontier; up, whether you rightsize in production. No direction does neither. Penny-wise is cheap everywhere and never sees what the frontier could build. The prototype trap pays frontier prices for every call. The barbell does both, then goes back to the frontier.

So be honest about which square you are in. If you have never built with AI, start on the frontier and spend the tokens. If you have a prototype running on the biggest model, it is time to rightsize. And if you have done both, the next model is already on its way, and the cycle starts again.

Always on the frontier in development. Always rightsizing in production. You need both.

Kevin Bai and one other person in the workshop room at KKR

This framework is now part of every workshop I run. If you lead technology at a private equity backed company and want to be in the room next time, reach out.


These views do not represent Anthropic.

If you want to hear my thoughts —

I write and talk through forward deployed engineering, enterprise software, and geopolitics on FDE Pod. New pieces land there first.

Subscribe on Substack