I was giddy.
Not nervous-giddy. Confident giddy. I had an offer sitting in my inbox already, and I was pretty confident I knew why.
The interviewer asked me: Design a search system for every shopfront on our platform.
It’s info, it’s location, what it sells, I clarified.
The storage would be massive — every store every item in the world. Read traffic would be huge too — every query on the platform would hit this path, often many times.
I followed the formula: functional requirements, non-functional requirements, entities, API design. We got to the high level design, and in the database selection I locked in.
“I heard you guys use AWS, is that true?”
“Yes”
“I’d suggest using something like Aurora, if you’re familiar with it?”
They weren’t.
I went on, “The idea is that the storage layer is detached from the compute layer, so they can scale independently. Aurora partitions the storage layer — but we still write to one logical primary. Then we can add a bunch of replicas to handle the heavy read traffic.”
The interviewer was stone-faced.
“So you’d shard it?”
I’d given this exact answer before. Nearly word for word, two weeks earlier.
That time the interviewer wasn’t on AWS. I think the Azure equivalent is Hyperscale — they didn’t know it. I explained it anyways — the decoupling of the compute and storage layers, the single primary, the replicas.
They got curious.
We ended up just talking. About their actual database setup, about when sharding is worth the complexity trade-off, about picking SQL/NoSQL. It felt less like an interview, and more like a conversation. As if the clock on the interview had just paused.
I threw in a fun anecdote too — OpenAI published a blog post detailing how they “scaled PostgreSQL at OpenAI to support millions of queries per second for 800 million users” without sharding.1
The interviewer sat up a little. Loved that I read their blog, loved that I could point to a real system doing the exact thing I was proposing: “If OpenAI level workloads could run it, what better evidence could you ask for?”
That sealed it. I was negotiating the offer a week later.
“So you’d shard it?”
I had the same answer loaded from two weeks earlier, and I fired it without thinking.
“Sort of! We wouldn’t be the ones to shard it, certainly not for the actual compute on the writes. AWS would handle the partitioning for us on the storage layer. And since we don’t need strong consistency, we can rely on replicas to handle the heavy read traffic. The vertical scaling we get from larger instances can easily handle the write throughput that we require.”
“At that scale you wouldn’t shard?”
“For the workload we agreed, we don’t need sharding”.
I tried to call in backup — OpenAI didn’t need sharding. I gave the same story, told with the same energy. This time though, it didn’t land.
I offered to answer as if Aurora wasn’t on the table. The interviewer didn’t take it.
No facial reaction, no follow-up question. Just keys clacking.
I had a feeling how this was going to end.
We moved on, part of me hoping that maybe they did follow my proposal.
We talked about hot-keys, we talked about statelessness, something like that. I don’t remember. What I do remember, is them looking down at their laptop, then asking “What key did you say we’re sharding these queries on?”
My shoulders dropped.
By no means am I a database expert. But I learned the same database principles everyone else has, at uni, and at work. Whatever combination of sharded, unsharded, SQL, NoSQL, you want me to defend for your prod system, I got it. No problem.
Except that’s the thing, isn’t it? You’re not designing a system for production. You’re designing a system to match the answer someone wrote down late last night.
When I joined the call and we exchanged pleasantries the interviewer mentioned they hadn’t had lunch yet. They were tired. They’d probably heard five versions of “it depends” in the last few weeks. Somewhere on their laptop they had a rubric (whether mental or literal), with a checkbox that said shard key, and they just wanted to check it.
Most interviewers want you to pass. If it starts to feel like they’re guiding you somewhere, let them.
OpenAI’s unsharded setup only covers read-heavy workloads. Write-heavy, shardable workloads get migrated to Azure Cosmos DB. Via Scaling PostgreSQL to Power 800 Million ChatGPT Users.