Performance Testing Services in India

Load, Stress, Soak, Spike and Breakpoint: What Each One Actually Tells You

These get used interchangeably in requirements documents and they are not interchangeable at all. Each answers a different question, and running the wrong one is how a team ends up confident about the wrong thing. A useful engagement runs most of them, in an order, because they build on each other.

Load test: does it hold at the traffic you expect?

Drive the system at your realistic peak, hold it there for long enough to be meaningful, usually thirty to sixty minutes, and watch the latency distribution and error rate. This is the test everyone means when they say load testing, and on its own it tells you the least, because it only confirms that a level you already survive is survivable. Its real value is as the reference run: everything else gets compared against this baseline, and a regression is measured as movement away from it.

The mistake here is running it for four minutes. Four minutes is shorter than most autoscaling reaction times, shorter than a JIT warm-up on a JVM service, and far shorter than the interval at which a garbage collector does its expensive work. A four minute run measures your system in a state it is almost never in.

Stress test: what happens on the way past the limit?

Push beyond expected peak until behaviour changes, then keep watching. The question is not whether it breaks, because everything breaks at some level. The question is how. A system that sheds load cleanly, returns 503 quickly, keeps its health checks honest and recovers within a minute of the pressure lifting is in decent shape. A system that queues everything, holds connections open, times out at the load balancer, marks healthy instances unhealthy and then takes twenty minutes to come back after traffic normalises has a design problem that no amount of extra capacity will fix.

The recovery half of a stress test is the part most teams skip and the part we find most informative. Stop the load and time how long it takes to be well again. If the answer is longer than a few minutes, you have a system that turns a traffic spike into an outage.

Soak test: what leaks or drifts over hours?

Moderate, realistic load held for six, twelve or twenty-four hours. Nothing else finds this class of bug. Heap creeping upward across hours until a JVM spends its life in full garbage collections. A connection leak that only exhausts the pool after forty thousand requests. An unbounded in-memory cache with no eviction policy that is fine on Monday and out of memory by Wednesday. A log file filling a disk. A database table whose query plan flips once its size crosses a threshold the optimiser cares about.

Ask any team with a nightly restart cron why it exists and you will usually get a shrug. That cron is a soak test failure that somebody worked around instead of diagnosing. Soak runs are also the easiest part of this work to hand to a team in a different timezone, since they need a quiet environment and nobody watching, which is exactly what your night provides.

Spike test: how fast can it react?

Jump from low traffic to very high in seconds rather than ramping. This tests the reaction time of everything elastic in your stack, and the results are frequently unpleasant. Container autoscaling takes time to observe a metric, decide, schedule a pod and pass a readiness check. New instances start with cold caches, cold connection pools and, on the JVM, interpreted bytecode that has not been compiled yet, so they are slower than the ones already running while simultaneously being handed a share of the worst traffic of the day.

Our experience is that spike failures are usually warm-up failures rather than ceiling failures. The system could handle the volume if it had five minutes of notice. The fix is often pre-warming, keeping headroom instead of scaling on the edge, or a queue that absorbs the burst, and none of those get considered if the only test you ever ran was a gentle ramp.

Breakpoint test: where is the ceiling, precisely?

Ramp continuously until the system stops coping, and record the exact point at which throughput stops rising with offered load. That inflection is the number worth knowing. Below it, adding traffic adds throughput. Above it, adding traffic adds only latency and errors, because a queue somewhere has started growing faster than it drains.

Give this number to your product and marketing people. The useful framing is not the raw request rate, which nobody outside engineering can act on, but the headroom multiple: we currently take about three times our busiest hour before checkout latency crosses two seconds. That sentence changes planning conversations. A dashboard does not.

Where the front end fits

Protocol-level load testing measures your servers. It does not measure what a real browser does with the response: parsing, rendering, layout shifts, hydration cost, blocking third-party scripts. Both matter, and they are different measurements taken with different tools. Browser-level checks on Largest Contentful Paint and Interaction to Next Paint are worth running, and driving thousands of real browsers is an expensive way to load a server. We usually run protocol-level load for capacity work and a small number of browser sessions alongside it to catch the case where the API got faster and the page did not.

Workload Modelling: Using Your Real Traffic Instead of Inventing Numbers

Most disappointing load tests fail here, before a single script is written. Somebody decides the test should simulate a thousand concurrent users, a number that came from a meeting rather than from data, and the whole exercise inherits that fiction. The result passes, everyone relaxes, and the system falls over at a traffic pattern nobody modelled.

Start with the logs, not the whiteboard

Your access logs, load balancer logs or analytics already contain the answer. We pull the busiest hour of the last twelve months, then break it down: requests per second per endpoint, the ratio between read and write operations, the distribution of session lengths, how many requests a typical session makes, and the proportion of traffic that is authenticated. That last one matters more than it sounds, because anonymous traffic usually hits cache and logged-in traffic usually hits the database.

The output is a mix, not a total. A test that sends eighty percent of its traffic to the product listing page when production sends eighty percent to search is testing the wrong system. We have seen a rewritten test change nothing about the load level and move the bottleneck from the web tier to the database purely by correcting the endpoint mix.

Think time, and why omitting it produces fiction

Real people read. They pause between actions, get distracted, open a second tab, come back. A script with no think time hammers the next request the microsecond the last one returns, so a hundred virtual users generate the request rate of several thousand real ones, all of it arriving in a pattern no human population produces.

The damage runs in two directions. Without think time, a modest virtual user count looks like a catastrophic result and the team panics over a number that means nothing. Meanwhile the concurrency being simulated is far lower than the equivalent real traffic, so connection pool and session memory pressure are understated. You get a test that is simultaneously too harsh on throughput and too gentle on concurrency, which is an impressive way to be wrong twice.

Think time should also be distributed, not constant. Every user pausing exactly three seconds creates a synchronised wave that produces artificial peaks and troughs. A randomised pause across a realistic range, or a normal distribution around a mean, gives an arrival pattern that behaves like a crowd. Better still, stop thinking in virtual users at all and specify arrival rate directly, then let the tool work out how many users it needs to sustain it. That is the open model, and it is closer to how traffic actually reaches you.

Test data that does not lie for you

A script that requests the same product ID ten thousand times will report excellent numbers, because after the first request everything is in cache. All you have measured is the cache. Parameterising with a realistic spread of identifiers is not optional, and the spread has to match production: if ten percent of your catalogue receives ninety percent of the views, the test data should be skewed the same way, or you will either overstate the cache hit rate or understate it.

Write paths need more care again. A load test that creates orders needs enough distinct users and payment fixtures to avoid every virtual user updating the same row, which turns the test into a lock contention benchmark. It also needs a reset strategy, because a test that inflates a table by two million rows changes the behaviour of the next run and you lose your baseline. And if you are considering a copy of production data, the anonymisation and the legal basis for holding it are questions for your counsel and your data protection people before we touch it, not after.

Correlation, and the thing that breaks every recorded script

Recorded scripts almost never replay unmodified. Anti-CSRF tokens, session identifiers, dynamic form fields, signed URLs and one-time nonces are unique per session, so a replayed script sends yesterday's token and receives a polite 403 that the tool cheerfully records as a fast response. This is the single most common reason a load test reports beautiful numbers while proving nothing.

The defence is a response check on every step. Validate the status code, and also validate something in the body that only appears on success, then treat a failed check as a failed request in the results. It is dull to build and it is the difference between a result and a graph.

What a Performance Testing Engagement Includes

Scope varies with what you already have. A team with good observability and a stable staging environment needs something quite different from one testing for the first time. The following is the full set, and the assessment decides how much of it applies.

A workload model you can argue with

A written model of what we are simulating and why: peak hour derived from your traffic data, endpoint mix with percentages, session shape, think time distribution, data volumes, and the growth factor applied. You review this before any script runs. If it is wrong, everything downstream is wrong, and it is far cheaper to argue about it on a page than to discover it three weeks later.

Scripts that live in your repository

Test scripts committed to your version control, parameterised, with response validation on every step, written to be read and edited by your engineers rather than by us. If the engagement ends and the tests become unmaintainable, you have rented a report instead of building a capability.

Baseline, peak, stress, soak and breakpoint runs

The measured sequence described above, each run recorded with the full latency distribution, throughput achieved against offered, error breakdown, and the correlated system metrics captured over the same window. Raw results are kept so a later run can be compared against them properly rather than against a screenshot.

A bottleneck analysis that names components

This is the deliverable that matters. Not a statement that response time degrades above 400 requests per second, which you could have discovered yourself, but that response time degrades above 400 requests per second because the HikariCP pool is capped at 20 while the average query holds a connection for 90 milliseconds, so the pool saturates at roughly 220 queries per second and everything above that queues. Then the evidence: pool wait time, database active connections, the slow query, the thread dump. A finding without evidence is an opinion.

Prioritised recommendations with honest effort estimates

Each finding gets an expected improvement and a rough cost, split between configuration changes measured in hours, code changes measured in days, and architectural changes measured in months. We are explicit about which ones we can validate quickly and which need a redesign to prove, and about the ones where the right answer is to accept the limit and provision for it.

Retest and a regression harness

After fixes land, the same tests run again under identical conditions so the improvement is measured rather than assumed. The short version of the suite gets wired into your pipeline with thresholds, so the next regression is caught by a build rather than by a customer.

Handover documentation

How to run the tests, how to read the results, what the thresholds mean and why they are set where they are, what environment assumptions the numbers depend on, and the known limitations of the setup. If your team cannot run and interpret these tests without us, the engagement has not finished.

From a Slow Number to a Named Bottleneck

A load test on its own produces one fact: the system is slow above some level. Turning that into an action requires correlating the run against what every layer was doing at the same moment, which is why we ask for observability access before we ask for an environment. Without it we are outside the building describing the smoke.

Two frameworks organise the search well. Brendan Gregg's USE method walks every resource and checks utilisation, saturation and errors, which is the right lens for infrastructure. The RED method, popularised by Tom Wilkie, checks rate, errors and duration per service, which is the right lens for the application. Between them they cover most of what you need to look at, in an order, rather than by intuition.

CPU: usually innocent, occasionally guilty

High CPU is the bottleneck everyone assumes and rarely the one they have. When it is real, a profiler names it in minutes: async-profiler or Java Flight Recorder on the JVM, py-spy for Python, pprof for Go, perf underneath everything. A flame graph makes the answer visual, and the answer is usually less exotic than expected. Serialisation. Regular expression backtracking. A password hashing round count copied from a blog. Logging at debug level in production. Encryption on a path that did not need it.

Watch run queue length as well as utilisation. A machine at sixty percent CPU with a run queue consistently above its core count is saturated even though the percentage looks comfortable, and the percentage is what your dashboard is showing you.

IO and network: the wait nobody graphs

Disk saturation shows up as IO wait and rising device queue depth, most often from a database on undersized storage or a log volume that has run out of provisioned IOPS. Cloud block storage has a burst allowance that behaves like the CPU credit problem: fine for the first ten minutes, terrible afterwards, which makes short tests actively misleading.

Network problems are frequently about round trips rather than bandwidth. A service making four hundred sequential calls to a database three milliseconds away spends 1.2 seconds doing nothing but waiting, and no amount of extra capacity on either end changes that. This is why chatty code that was fine on a single machine becomes a crisis after a migration splits the tiers apart.

Locks and contention: the reason more cores did not help

The signature is throughput that flatlines while CPU stays moderate and latency climbs. Somewhere a shared resource is being serialised: a synchronised block, a global mutex, a distributed lock held across a network call, a database row that every transaction touches, or an inventory counter in a flash sale. Thread dumps taken during the plateau, three of them thirty seconds apart, usually make the culprit obvious because the same stack frame appears in every dump.

The distinctive property of a contention bottleneck is that scaling out makes it worse. Adding instances increases the number of contenders for the same lock, so throughput can actually fall as you add capacity, which is a genuinely confusing thing to watch on a dashboard if you have not seen it before.

Connection pools and queueing: Little's Law does the arithmetic

Little's Law says the average concurrency in a system equals throughput multiplied by average latency. That one line explains most pool exhaustion. A pool of 20 connections where each query holds a connection for 90 milliseconds sustains roughly 220 queries per second and not one more, no matter what the application servers are doing. Above that, requests wait for a connection, wait time counts as latency, and the whole thing tips over fast because slower queries hold connections longer, which starves the pool further.

The counter-intuitive part is that raising the pool size is often the wrong fix. Past the point where the database has enough cores to run the queries concurrently, more connections add contention inside the database and everything gets slower. The HikariCP documentation makes this argument well and the reasoning applies to any pool. Thread pools, HTTP client pools and worker queues all behave the same way. Measure the wait time inside the pool, not just its size.

Garbage collection and memory

On managed runtimes, look at allocation rate before pause times. A service allocating aggressively per request drives collection frequency, and the pauses are a symptom of the allocation, not an independent problem. On the JVM, G1 with a pause target is fine for most workloads, ZGC and Shenandoah exist for latency-sensitive ones, and switching collector before understanding the allocation profile is a common way to spend a week for nothing. In Go, the pauses are small but allocation still costs, and escape analysis in pprof tells you where. Node.js runs into a heap ceiling that has to be raised deliberately.

A soak test is the only reliable way to separate a leak from normal churn, because both look identical over twenty minutes and only one of them is still climbing after eight hours.

CDN and Caching: Where Results Are Most Easily Faked

Caching is the reason a performance test can report a system handling ten times its real capacity. If the test requests a small set of cacheable URLs, the CDN answers almost everything from an edge node close to the load generator, the origin sees a trickle, and the numbers are excellent. You have measured Cloudflare or Fastly, who were never in doubt.

So the first question in any test design is which layer we are aiming at. Testing through the CDN measures the real user journey and tells you whether your cache configuration works. Testing against the origin directly, bypassing the edge, measures the capacity that actually matters when the cache is cold or the content is personalised. Both are legitimate. Reporting one and describing it as the other is not, and it happens often.

Cache key design is where most real problems live. Query parameters usually form part of the key, so campaign tracking parameters can shatter the hit rate: the same page requested with fifty different UTM combinations becomes fifty separate cache objects, each needing its own origin fetch. Normalising or ignoring tracking parameters is often the single highest-value change we recommend on a content-heavy site. A careless Vary header does the same damage, splitting the cache by user agent or by a cookie that changes per visitor. Cookies are the classic culprit, since one session cookie on a static asset response can make an entire path uncacheable.

Then there is the stampede. When a popular cached object expires, every concurrent request for it misses simultaneously and they all reach the origin at once. A page served comfortably from cache at two thousand requests per second becomes two thousand simultaneous origin requests the instant its TTL elapses. The defences are well understood: request coalescing at the edge, stale-while-revalidate so users get slightly old content while one request refreshes it, jittered TTLs so a thousand objects do not expire together, and origin shielding or tiered caching so only one edge location talks to your origin. A spike test aimed at a cold cache is the test that finds whether any of this is configured, and it is rarely run.

Application-level caches deserve the same suspicion. A Redis instance is a shared resource with its own single-threaded limits, and a load test can find its ceiling as easily as the database. Watch for the same stampede pattern behind it, for keys with no expiry accumulating memory across a soak run, and for the case where the cache lookup plus deserialisation costs more than the query it replaced. That last one is more common than anyone admits, particularly when someone has cached a small, well-indexed query result.

One practical note before you run anything at scale: cloud providers and CDN vendors generally have acceptable-use terms covering generated load, and testing above certain volumes may need notification. We check the specifics for your providers with you before a run rather than discovering the policy afterwards.

Three Situations We Get Called For

These are composites drawn from the shape of the work rather than accounts of particular clients, and they are the patterns that repeat.

Checkout is fine at 200 requests per second and dead at 260

A retailer whose seasonal peak is a known date. Everything is comfortable through testing, then a narrow band where latency goes from 300 milliseconds to twelve seconds across a forty request per second increase, with application CPU at thirty percent throughout. The cliff shape is the clue: gradual degradation means a resource filling up, a cliff means a queue.

The investigation runs pool metrics alongside the load. Connection acquisition wait time, flat at zero, starts climbing at exactly the point latency does. The pool is capped at 20, the average query holds a connection for 90 milliseconds, and Little's Law puts the ceiling around 220 queries per second. What makes it a cliff rather than a slope is the feedback loop: once requests start queueing for connections they hold their threads longer, so the application thread pool fills too, and the load balancer starts failing health checks on instances that are alive but not answering.

The fix is rarely just a bigger pool. It is usually a combination: cut the number of queries per request so each one holds a connection for less time, add PgBouncer so the database is not asked for four hundred connections, size the pool against the database's actual core count rather than optimistically, and set an acquisition timeout so a saturated pool returns a fast error instead of holding a thread hostage. Then retest and confirm the cliff moved rather than assuming it did.

The service that has to be restarted every night

A B2B platform where somebody added a 3am restart eighteen months ago and nobody questions it any more. Latency is fine in the morning and noticeably worse by late afternoon, which everyone has learned to live with.

A twelve hour soak at moderate load reproduces it on the first attempt, which is usually the case with this class of problem. Heap climbs steadily, old generation occupancy never returns to its starting level after a full collection, and garbage collection time as a share of wall clock creeps from two percent to nineteen. A heap dump at the eight hour mark points at a map used as a cache, populated per unique customer identifier, with no eviction and no size bound. It was written when the platform had forty customers.

The change is small: a bounded cache with an eviction policy and an expiry, plus a gauge on its size exported to your metrics so the next version of this problem is visible before it needs a cron job. The valuable part of this engagement is not the fix, it is that a soak run in CI now fails if heap does not return to baseline, which is the thing that stops it recurring.

The lift-and-shift that got slower on better hardware

A team moves a monolith from a colocated data centre to a cloud provider, keeping the application unchanged and moving the database to a managed service. The new machines are objectively faster. Page loads are up by a factor of three, and everyone is baffled.

A load test with query-level tracing gets there quickly. In the old setup the application and database were on the same rack, with sub-millisecond latency between them, so a request making 380 sequential ORM queries paid almost nothing per query. In the new setup they sit in different availability zones, three milliseconds apart, and those same 380 queries now cost over a second of pure waiting. Nothing is slow. Everything is far away.

The remedies are ordinary once the cause is named: eager-load the relations causing the N+1, batch what can be batched, cache the reference data being re-fetched per request, and place the tiers to minimise cross-zone hops without giving up the availability you moved for. It is worth adding that this failure mode is a strong argument for testing before a migration rather than after, since the number that changed was one nobody thought to measure. Migration planning of this kind sits with our cloud architecture services in India.

How Does a Performance Testing Team in India Work Across Your Timezone?

Here are the real numbers, because vague reassurance about overlap is how offshore engagements start badly. A standard Indian working day of 09:30 to 18:30 IST maps onto your day as follows.

For a UK team that is 04:00 to 13:00 in winter and 05:00 to 14:00 during British Summer Time, giving roughly four hours of live overlap in winter and five in summer against a normal London day. Comfortable, and no shift changes needed. For Sydney, a standard IST day runs 15:00 to midnight local, leaving about two hours of overlap at the start of your afternoon; moving the Indian day earlier to 07:00 to 16:00 IST takes that to roughly four and a half hours. Auckland is harder: a standard IST day gives almost nothing, and an early Indian shift of 06:00 to 15:00 IST buys about three and a half hours against a normal New Zealand afternoon.

The United States is where honesty matters most. A standard IST day ends at roughly 08:00 US Eastern, which means zero overlap with a nine-to-five in New York. Three hours with the East Coast requires the Indian day to run to about 22:30 IST. Four hours with the West Coast requires it to run past midnight IST. That is a night shift. It is possible and people do it, and you should know what it costs: night work is harder to hire for, harder to retain, and the engineer doing it lives on a different sleep schedule from everyone else around them. Anyone selling you round-the-clock cover as though it were free is either running a rotation they have not mentioned or is not going to deliver it. We agree the working window with you before the engagement starts and write it down.

Why load testing suits the gap better than most work

This service fits a distributed model unusually well, and the reason is structural rather than a sales point. Long runs need a quiet environment and nobody using it, so the natural time to run a twelve hour soak or a full breakpoint sequence is your night. That is our day. The test executes while your engineers are asleep and your staging environment is idle, and the analysis is waiting when you get in.

The deliverable is also unambiguous, which removes most of the interpretation risk that makes offshore work hard. p99 at 400 requests per second either crossed your threshold or it did not. There is no stylistic disagreement about a latency histogram. Findings arrive as written analysis with evidence attached rather than as something that needs a live conversation to be understood.

How the days actually run

Written first, consistently. A daily written update in your Slack or Teams channel before your morning: what ran overnight, what it showed, what is blocked, and what needs a decision from you. Live calls happen during the agreed overlap, usually two or three a week rather than daily, because a daily call eats the overlap you wanted for working sessions.

Anything needing your input goes out as a written proposal with options and a recommendation, not as an open question, since an open question costs a full day of round trip. Blockers are raised before the Indian day ends rather than discovered at the start of yours. Runs that carry real risk, such as anything against production, are scheduled inside the overlap so both sides are awake, and we say in advance which ones those are.

Access, ownership and security

Work happens in your repositories, your cloud accounts and your observability tooling under named individual accounts with scoped roles, so your audit log shows who did what. Test scripts are committed to your version control from day one rather than developed privately and handed over at the end. If the engagement stops tomorrow, everything built is already yours because that is where it was built.

Terms covering IP assignment, confidentiality, data handling and access revocation belong in the agreement signed before work starts, and we settle those with you up front rather than quoting a policy from a marketing page. If your security team has a questionnaire, send it before the engagement rather than during it. Where the test environment needs a copy of production data, the anonymisation approach and the legal basis are a conversation for your data protection people and your counsel, and we build against their answer.

The talent question for this specific skill

India has a very large pool of engineers who can record a JMeter script, and a much smaller one of engineers who can read a flame graph, reason about queueing, tell a garbage collection pause from a lock wait, and argue about whether a percentile was computed correctly. We staff against the second group for this work, and the vetting reflects it: candidates are given a system that degrades under load along with its metrics and asked to explain what is happening, rather than being asked to define load testing. Written English is assessed by having them write the analysis, since that is the artefact you will actually read.

Engagement Models

Most engagements start as an assessment and become one of the other two, which is deliberate. We would rather scope a build against measurements than against a hypothesis. If you already know what needs doing and have the numbers behind it, we can start at the second model.

Frequently Asked Questions About Performance Testing in India