Adam Jacob On Why Scaling Is ‘the Funnest Game’

“There’s nary funner crippled than nan at-scale exertion game. But if you play it, immoderate group will dislike you for it. That’s OK … that’s nan crippled you chose to play.” — Adam Jacob

At Monster Scale Summit 2025, Rachel Stephens, investigation head astatine RedMonk, said pinch Adam Jacob, co-founder of Chef and CEO of System Initiative, astir what it really intends to build and run package astatine scale.

Monster SCALE Summit 2026 will spell unrecorded March 11-12, featuring antirez, creator of Redis; Camille Fournier, writer of “The Manager’s Path” and “Platform Engineering”; Martin Kleppmann, writer of “Designing Data-Intensive Applications” and much than 50 others. The arena is free and virtual. Register and subordinate nan organization for immoderate lively chats. 

The Existential Question of Scale

Stephens opened pinch an existential question: “Does your package beryllium if your users can’t tally it?” Yes, your codification still exists successful GitHub moreover if us-east-1 goes down. But what if:

• Your strategy crawls nether load.
• Critical integrations perpetually break.
• You can’t spend nan infrastructure costs.

“Software astatine standard isn’t conscionable astir throughput,” Stephens said. “It’s astir making judge that your codification endures, adapts and remains accessible nary matter nan load and location of wherever you’re running. Because if your users can’t usage it, your package whitethorn arsenic good not exist.”

With that framing, Stephens brought successful personification who’s spent his profession dealing pinch standard firsthand: Adam Jacob.

Only Scale When It Hurts

Stephens asked Jacob really teams tin equilibrium quality, velocity and standard nether uncertainty. How do you debar some cutting corners and premature optimization?

Jacob argues that early on, it’s good not to interest overmuch astir scale. Most products neglect for different reasons earlier scalability ever becomes a problem.

He explained: “I deliberation of it fundamentally done nan lens of optionality. When you commencement building caller things, it’s bully not to interest excessively overmuch astir scale, because you whitethorn ne'er scope it. Most products don’t neglect because they neglect to scale. Think astir really severely Twitter grounded to standard … and yet present we are.”

The first privilege is to build a coagulated product. Once standard becomes a existent issue, that’s erstwhile it makes consciousness to refactor and region bottlenecks. But if you’ve been astir nan artifact a little, your acquisition helps you make early choices that salary disconnected later.

Jacob noted, “Premature optimization is real. But arsenic you summation experience, location are immoderate decisions you make early because you cognize that if things activity out, you’ll beryllium happier later — for illustration factoring your codification truthful it tin beryllium surgery isolated crossed web boundaries complete time, if you request to.”

Chef Scalability Horror Stories

Next, Stephens asked Jacob if he would stock a scaling scary communicative from his Chef days. Jacob obliged and offered 2 memorable ones.

“The champion was erstwhile we launched nan first type of Hosted Chef. The time earlier nan launch, we discovered it took astir a infinitesimal and a half to create a caller user. It didn’t return that agelong erstwhile we were moving it connected a laptop, but it did later … and we ne'er really tested it. So, successful nan last hours earlier launch, we changed it from ‘anyone tin motion up’ to a queue strategy pinch a small abstraction robot saying, ‘Demand is truthful high; we’ll get backmost to you.’ We conscionable papered complete nan scalability problem.”

“Another example: That aforesaid Chef server (the 1 that couldn’t create accounts quickly) yet had to activity astatine Facebook. The original type was written successful Rails, which was awesome to activity with, but not parallel enough. At Facebook scale, you mightiness person 40,000 aliases 50,000 things pointed astatine 1 Chef server. So we rebuilt it successful Erlang, which is awesome for that benignant of problem. I virtually brought nan Erlang type to Facebook connected a USB stick. When we installed it and bootstrapped a information center, we thought it was surgery because it was utilizing little compute and vanished almost instantly.”

Jacob explained that if they’d tried to build nan Chef server successful Erlang from nan start, nan task astir apt wouldn’t person gained traction. Starting successful Rails made it imaginable to get Chef retired into nan world and study what nan strategy really needed to do. Only later, erstwhile they understood really nan strategy really behaved, could they rebuild it pinch nan correct architecture and runtime for scale.

Growth aliases Efficiency: Know Which Game You’re Playing

At Chef, scaling was yet required to onshore customers for illustration Facebook and JPMorgan Chase, which run astatine monolithic scale.

Jacob advised, “Making it standard required awesome investment, but it worked. You can’t bargain your core. If it matters to customers, you person to build it yourself. People often hold excessively agelong to recognize they person a heavy architectural problem that’s besides a business problem. Rebuilding for standard takes months, truthful you person to commencement early.”

Your ain attack to standard should yet beryllium driven by what crippled you’re playing:

• In nan task superior game, maturation and traction travel first. You tin walk money to standard faster because you’re funded.
• In nan profitability game, ratio comes first. Overspending connected compute aliases mediocre architecture hits nan bottommost statement hard.

Why Scaling Is nan ‘Funnest’ Game

Stephens mentioned that “when package succeeds, it stops being yours — it becomes everyone’s.” She past asked Jacob what it’s for illustration erstwhile your tech scales to nan constituent that group person highly beardown opinions astir it. His response:

“It’s difficult to build things that group attraction about. If you’re fortunate capable to create thing you emotion and stock it pinch nan world and group emotion it back, that’s incredibly rewarding. Even erstwhile they don’t, that’s still a gift.

“Someone erstwhile tapped maine successful a java shop and said, ‘You wrote Chef? I dislike Chef.’ I said, ‘I’m sorry; I didn’t constitute it to wounded you.’ But astatine scale, that intends he utilized it. It mattered successful his life. And that’s what you want: for group to acquisition what you built.

“I emotion nan technology, nan problem, nan difficulty. Scaling adds much layers of complexity, much layers of fun. There’s nary funner crippled than nan at-scale exertion game. But if you play it, immoderate group will dislike you for it. That’s OK … that’s nan crippled you chose to play.”

You tin spot nan afloat talk below.

Group Created pinch Sketch.