Scalability is not an afterthought — it is a design principle. In this post we share some of the practices our engineering team relies on to ship reliable, scalable software.
Scale is rarely the thing that breaks a product first. What usually breaks is a decision made early, for good reasons, that quietly stops holding once the numbers change. A query that was instant against a thousand rows becomes a timeout at a million. A migration that ran in seconds on a laptop locks a table in production. None of these are exotic failures; they are ordinary choices that were never revisited.
Design the data model around the queries you will actually run
Most performance problems we are asked to fix are not really performance problems — they are data-access problems. The most common by far is the N+1 query: a page renders a list, and every row quietly triggers another query. It is invisible with ten records and fatal with ten thousand.
The habit that prevents it is unglamorous. Load the relationships you know you need, up front and explicitly. Index the columns you filter and sort on, not every column. Check the query count on any page that renders a list before it ships, because the difference between one query and three hundred is not something you notice on a development machine with seed data.
Make migrations additive and reversible
Schema changes are where scaling work most often turns into an incident, because they are the one part of a deployment that touches live data and cannot simply be rolled back by redeploying.
Two rules carry most of the weight:
- Add, do not rewrite. New columns are nullable or carry a default, so existing rows stay valid the moment the migration lands and old code keeps working during a rolling deploy.
- Write the reverse. A migration that moves data should restore it on the way back down. Writing that reversal is also the fastest way to discover that a change is lossy — if you cannot express the undo, you are destroying something.
Where a change has to backfill existing records, we scope the update to the rows that still hold the old value rather than rewriting the table wholesale. That way a value someone has already corrected by hand is never clobbered by a deployment.
Make operations safe to repeat
At scale, everything runs more than once. Jobs retry, webhooks arrive twice, someone re-runs a command after a timeout. Any operation that only produces the right result the first time will eventually produce the wrong one.
This matters most where money is involved. Payments work like AlecerPay is unforgiving about it: a retried request must settle to the same outcome, not a second charge. The same discipline applies well beyond payments — to imports, scheduled jobs and any process that writes data on a schedule.
Cache the expensive thing, not everything
Caching is often reached for too early and applied too broadly. Cached data is stale data, and a cache layered over an inefficient query hides the problem rather than solving it — usually until the moment the cache is cold and the underlying cost arrives all at once.
We prefer to make the underlying operation fast first, then cache what remains genuinely expensive and genuinely stable. Anything cached needs a clear answer to how it is invalidated. If that answer is not obvious, the cache is a bug waiting to be filed.
Build for the conditions the software will actually meet
Scale is not only about volume. Farmetix is used in the field as well as the office, where connectivity is intermittent and work does not pause because a request failed. Designing for that reality — handling failure as an expected state rather than an exception — produces more robust software than assuming ideal conditions and patching afterwards.
Knowing when not to scale
The most useful judgement is often restraint. Architecture chosen for imagined future load has a real present cost: more moving parts, slower delivery, and a system that is harder to reason about while you are still learning what the product needs to be.
Our default is to build the straightforward version well, instrument it so you can see where time is actually spent, and keep the structure clean enough to change when the evidence arrives. Measured problems are worth solving. Hypothetical ones usually are not.
If you are planning a build and want a candid conversation about which of these will matter for your project — and which are not worth your money yet — take a look at our software development and cloud and infrastructure services, or get in touch.
