MLOps Services in India

What an MLOps Engagement Actually Covers

Not every engagement includes all of this. The assessment decides the order, and the order matters more than the list. Building a feature store before you have a working deployment pipeline is a common and expensive mistake. What follows is the full surface area, roughly in the sequence we usually build it.

Reproducible training pipelines

The first deliverable is almost always the same: a training run that a second person can reproduce on a second machine and get a metric within a defined tolerance. That means the code lives in a repository rather than a notebook, dependencies are pinned with a lockfile from uv, Poetry or pip-tools, the container is referenced by digest rather than by a floating tag, the data snapshot is addressed by version rather than by table name, and the seed is set.

Bitwise identical results on a GPU are usually not achievable, and we say so rather than pretend. Nondeterministic kernels, atomics in gradient accumulation and cuDNN autotuning all introduce small variation. The realistic target is metric reproducibility within a stated tolerance, plus a full record of what produced each run. If your evaluation metric moves more than that tolerance between two runs of the same code, that itself is a finding worth investigating.

Experiment tracking and a model registry

Every training run gets logged with its parameters, metrics, dataset version, git commit, container digest and output artifacts. That is the tracking half. The registry half is what turns a good run into a candidate: a named, versioned entry with a stage or alias, a link back to the run that produced it, and a record of who approved its promotion and against which evaluation set.

The registry is where a lot of teams get sloppy, and it costs them later. If you cannot answer the question of which model version scored a specific prediction six months ago, you cannot investigate a complaint, satisfy an auditor, or backfill a delayed label onto the model that produced it. We log the model version and a prediction ID with every inference from day one, because retrofitting that later means a gap in your history that never fills in.

Feature engineering and the training and serving contract

The single most common source of a model that scores well offline and does nothing online is training and serving skew: the same feature computed differently in two places. Offline it is a pandas expression written by a data scientist. Online it is a reimplementation in Go or Java by a backend engineer working from a description. Null handling differs, a timezone differs, a rounding rule differs, and the model is now being fed a distribution it never saw.

The fix is a contract rather than a document. Either both paths call the same transformation code, or the feature is computed once and read from the same store by both, or at minimum there is an automated test that computes a batch of features through both paths and fails the build when they disagree beyond a tolerance. We usually start with the test, because it can be added in a day and it tells you immediately whether you have the problem.

CI/CD built for models

A model pipeline needs the normal software gates plus four that do not exist in ordinary CI. Data validation runs first, against expectations defined in Great Expectations or Pandera, so a schema change or a null rate spike fails the run before any GPU time is spent. Then a training smoke test on a small sample, which catches the majority of code errors in two minutes rather than four hours.

After training, the model quality gate compares the candidate against the current champion on a frozen evaluation set and on the most recent data slice, because a model can beat the champion overall and be worse on the segment that actually matters. Then behavioural tests: invariance checks that an irrelevant change does not move the prediction, and directional checks that an increase in a feature moves the score the way domain logic says it should. Finally a latency and payload size test against the serving contract, because a model that is two points better and 300ms slower is often a rejection.

Serving infrastructure

Deployment splits into real time endpoints, batch scoring, streaming inference and embedded models, and they have almost nothing in common operationally. A real time endpoint needs autoscaling, health checks, request batching and a rollback path measured in seconds. A batch scoring job needs idempotency, partition level retries and a completion signal that downstream consumers can trust. Getting a batch job to behave like an endpoint, or the reverse, is a recurring source of waste.

We also push back on real time when it is not needed. A surprising share of production models are asked to serve real time predictions on features that only update daily. That is a batch job writing to a key value store, and it is an order of magnitude cheaper and simpler to operate than a live endpoint. The question we ask is how fresh the features actually are, not how fresh the prediction feels.

Monitoring, drift detection and alerting

Infrastructure monitoring tells you the endpoint is up. Model monitoring tells you it is still right. Those are different systems, and the second one is what teams skip. We instrument input distributions, prediction distributions, feature null and range violations, segment level performance, and delayed label backfill, then wire the ones that matter to alerts a human will act on. The section further down explains what we alert on and, more usefully, what we deliberately do not.

Retraining and continuous training

Retraining is where most automation projects overreach. A pipeline that retrains on a schedule and promotes automatically is a pipeline that will one day promote a model trained on a corrupted week of data. We build the retraining path in stages: automated training with a human approval gate first, then automated promotion once the quality gates have proven themselves over enough cycles that you trust them. Which triggers fire, and what has to pass before anything reaches production, is covered below.

Cost visibility and control

Every engagement includes tagging and a cost model, because ML spend is the easiest thing in a cloud bill to lose track of. Training runs, endpoint hours, feature store reads, artifact storage and data egress all get attributed to a model and a team. Two numbers go on the dashboard: cost per thousand predictions and cost per training run. Once those exist, cost conversations stop being arguments and become arithmetic.

Handover and runbooks

What we hand back is infrastructure as code, pipeline definitions, the registry with its history, dashboards, alert runbooks that name the first three things to check, and an architecture note explaining the decisions and their trade-offs. The test we apply is whether an engineer on your side who was not part of the build can take a 3am page, follow the runbook, and either fix it or roll back without calling India. If they cannot, the handover is not finished.

Choosing the Stack: Tools, Trade-offs and When Each Is Wrong

We do not have a house stack we push onto every client. What we have is a set of opinions about when each tool is the right answer and, more usefully, when it is the wrong one. The largest cost in this category is not licensing. It is standardising on a platform that fits a team four times your size.

Orchestration: Airflow, Dagster, Kubeflow Pipelines, Flyte, Prefect

Airflow is the safe answer when your data team already runs it. The operator ecosystem is enormous, the scheduler is well understood, and hiring for it is easy. Its weaknesses for ML are real though: passing artifacts between tasks through XCom is awkward, there is no native concept of a versioned data asset, and dynamic pipeline shapes fight the model. The TaskFlow API in Airflow 2.x softened this considerably. It did not remove it.

Dagster inverts the model around software defined assets, which fits ML better because you usually care about the dataset and the model rather than the task that produced them. Local development and testing are genuinely better. The cost is buy in: the asset abstraction is opinionated, and a team that thinks in DAGs will resist it for the first month. Flyte is the strongest option when you need strong typing and caching of expensive steps across runs, and the steepest to learn. Prefect suits Python first teams who want dynamic workflows without Kubernetes.

Kubeflow Pipelines is where we push back hardest. It is powerful, it is Kubernetes native, and it carries an operational burden that a team of four will spend more time servicing than using. It makes sense when you already run Kubernetes at scale with a platform team behind it, and when multi tenancy across several ML teams is a real requirement. If either of those is untrue, it is the wrong tool, and Vertex AI Pipelines gives you the same KFP authoring model without the cluster to babysit.

Experiment tracking: MLflow, Weights and Biases, Neptune, Comet

MLflow is the default we recommend for teams that want to self host and keep everything inside their own account. Tracking, packaging and the model registry are in one open source tool, the Python API is unobtrusive, and artifacts go to your own S3 or GCS bucket. Its weak point is access control. If you need per project permissions and a clean audit trail across several teams, you will be building that around MLflow rather than getting it from MLflow.

Weights and Biases has the better interface by a distance, and for teams doing genuine research iteration the comparison views and sweeps save real time. The trade offs are commercial and jurisdictional: it is SaaS by default, self hosting is a paid tier, and run metadata leaves your environment. For a client with data residency constraints, that conversation has to happen before anyone starts logging. Neptune and Comet occupy similar ground with different pricing shapes.

Model serving: BentoML, KServe, Seldon, Triton, Ray Serve, vLLM

BentoML is our usual starting point for a team without a Kubernetes platform. It packages the model, the preprocessing and the dependencies into one deployable unit, supports adaptive batching, and produces a container that runs anywhere. The abstraction is Python shaped, which suits the people writing the models. It gives you less once you need fine grained control over GPU sharing and pod placement.

KServe is the CNCF option for Kubernetes shops and gives you scale to zero through Knative, which matters for models with spiky or low traffic. Seldon Core covers similar ground with inference graphs for multi step pipelines, and its licensing changed in recent versions, so check the current terms against your policy before you standardise on it. NVIDIA Triton is the right answer when GPU efficiency is the binding constraint: multi framework support, dynamic batching, model ensembles and concurrent model execution on one GPU. It is also the hardest of these to operate, and it rewards teams who have someone willing to learn its configuration model properly.

For LLM inference specifically, vLLM has changed the economics through continuous batching and paged attention, and it is usually the first thing we try before anyone proposes buying more GPUs. Ray Serve is worth considering when inference is a composition of several Python steps rather than a single model call.

Managed platforms: SageMaker, Vertex AI, Azure ML, Databricks

Managed platforms buy you time and cost you optionality, and the trade is often worth it for a first production system. SageMaker has the broadest feature set, including Pipelines, Model Registry and Model Monitor, and its endpoints bill by instance hour whether or not anyone calls them, which is the line item that surprises people. Serverless inference removes that at the cost of cold starts. Vertex AI is cleaner if your data already sits in BigQuery, and its Pipelines are KFP underneath, so the authoring skills transfer.

Azure ML makes sense mostly when the organisation is already committed to Azure, and Databricks is the easy call when your feature engineering lives in Spark and you want MLflow, Unity Catalog lineage and the training compute in one place. The honest downside across all four is portability. Model artifacts move easily. Pipelines, monitoring configuration and registry history do not.

Feature stores: Feast, Tecton, and the case for neither

Start with the argument against. If you have one or two models, batch features, and a well governed warehouse, a set of dbt models plus a documented point in time join gives you correctness without a new piece of infrastructure to run. Most feature store projects we have been asked to rescue were started before the team had a deployment pipeline that worked, which is the wrong order.

Feast is the right first step when you do need one: it is lightweight, it does not try to be a transformation engine, and it sits over an online store such as Redis or DynamoDB and an offline store in your warehouse. Tecton adds managed streaming transformations and materialisation, which is genuinely hard to build yourself, at a price that only makes sense when features are shared across several teams and several models. The managed stores inside SageMaker and Vertex remove the operations at the cost of tying feature definitions to a cloud.

Monitoring and drift: Evidently, NannyML, Arize, WhyLabs, Prometheus

Evidently is our default for batch drift reporting and for the first version of a monitoring dashboard, because it is open source, it runs where your data is, and its reports are readable by people who are not statisticians. NannyML fills a specific and underrated gap: estimating model performance before labels arrive, which is the situation most teams are actually in. Arize, WhyLabs and Fiddler are the managed options, stronger on scale and on segment level analysis, and they involve sending prediction data outside your environment.

Underneath all of them sits ordinary observability. Prediction latency, throughput, error rate and saturation belong in Prometheus and Grafana with the rest of your services, and the inference path should emit OpenTelemetry traces like anything else. Splitting model telemetry into a separate universe that only the ML team looks at is how model incidents end up outside the incident process.

Drift: What We Alert On, and What We Deliberately Ignore

Drift monitoring has a reputation for being noisy, and it deserves it. Point a Kolmogorov-Smirnov test at a million rows a day and it will flag statistically significant change in almost every feature, almost every day, because at that sample size significance is guaranteed and meaningless. The discipline is in choosing what deserves a page, what deserves a ticket, and what deserves a line on a chart nobody looks at until something else goes wrong.

Data drift

Data drift is change in the inputs. The average transaction value rises, a new country starts appearing, a categorical feature gains a level that did not exist during training. It is measurable immediately, which is why it is where monitoring usually starts, and it is a leading indicator rather than a problem in itself. A model can tolerate substantial input drift if the relationship it learned still holds.

We prefer Population Stability Index and Jensen-Shannon divergence over raw hypothesis tests, precisely because they report effect size rather than significance. Thresholds get set per feature from the model's own importance ranking, not uniformly. A shift in the third most important feature matters. The same shift in a feature contributing almost nothing does not, and alerting on it teaches the team to ignore the channel.

Concept drift

Concept drift is change in the relationship between inputs and outcome. The inputs can look identical to training and the model can still be wrong, because what a given pattern means has changed. Fraud tactics adapt to your controls. Customer behaviour changes after a competitor's price cut. A policy change alters what counts as a valid claim. This is the drift that actually costs money, and it is invisible in the input distribution.

Detecting it requires outcomes, which is why label latency is the central design constraint in almost every monitoring build we do. Where labels come in days, we compute rolling performance and alert on decay against a baseline. Where they take weeks or months, we use estimation methods such as confidence based performance estimation, and treat the estimate as an early warning that gets validated when the truth lands rather than as a fact.

Prediction drift and the cheap signals

The output distribution is the single most valuable thing to watch, and the cheapest, because it needs no labels and no feature reconstruction. If your model's approval rate moves from 12 percent to 19 percent in a day with no release and no known campaign, something upstream has changed.

In practice this catches more real incidents than sophisticated statistical drift detection does, and it takes an afternoon to build.

Alongside it sit the boring checks that catch the majority of genuine breakages: schema conformance, null rate per feature, cardinality change in categoricals, out of range values, and volume against expectation. A feature that was 2 percent null and is suddenly 40 percent null is not a drift question. It is a broken pipeline, and it should page someone.

How the alerts are actually wired

Three tiers, and the discipline is in keeping the top tier small. Page a human for schema breaks, volume collapse, latency breach and a prediction distribution move beyond a hard threshold. Open a ticket for gradual drift on important features, estimated performance decay, and segment level regression. Chart everything else without notification. Every alert names an owner and links to a runbook, and any alert that fires three times without an action being taken gets deleted or retuned, because an alert nobody acts on is worse than no alert at all.

GPU and Inference Cost Control

ML spend goes wrong in a specific way. It starts small during experimentation, nobody tags anything, an endpoint gets provisioned on the instance type someone copied from a tutorial, and six months later there is a five figure monthly line item that no single person can explain. The work here is mostly measurement, and the savings mostly come from three or four unremarkable decisions.

Training cost

Spot and preemptible instances are the largest single lever for training, and they are only usable if your training loop checkpoints properly and resumes cleanly, which is a code change rather than a procurement decision. We make checkpointing work first, then move the workload. Beyond that: stop running full hyperparameter sweeps when a smaller search with early stopping via Optuna or Ray Tune finds the same region, cache expensive preprocessing steps so a rerun does not repeat them, and profile before scaling up, because a surprising number of training jobs are bound by the data loader rather than the GPU.

Inference cost

Most overspend on inference comes from three habits. Running a batch size of one when requests could be batched, which on a GPU wastes most of the hardware. Sitting on an oversized instance because that is what the model fit on during development, when an L4 or an A10G would serve the same traffic. And keeping an endpoint warm around the clock for traffic that arrives in two bursts a day, when scale to zero through KServe and Knative or a serverless endpoint would idle at nothing.

Quantisation and distillation come after those, deliberately. INT8 quantisation or an AWQ or GPTQ variant for a large language model can cut serving cost substantially, and it changes model behaviour, so it needs its own evaluation pass against the same gates as any other model change. Treating a quantised model as the same model is a mistake we have had to unpick more than once. And it is worth checking whether the model needs a GPU at all: gradient boosted tree models serving tabular predictions almost never do.

Making the number visible

Tag every resource by model and team, then publish two figures on the same dashboard as the model quality metrics: cost per thousand predictions and cost per training run. Once a product manager can see that a model costs a certain amount per thousand decisions, the conversation about whether a two point accuracy improvement is worth a fourfold cost increase becomes possible. Without those numbers, it never happens.

Four Situations We Get Called Into

The model nobody can rebuild

A risk scoring model had been maintained by one data scientist for two years, retrained by hand every quarter on a workstation under a desk. He left. The remaining team had the pickle file, a notebook that referenced three CSV paths that no longer existed, and no record of which version of the source data produced the model in production. The model still worked. It just could not be changed, which meant a known bug in one feature had to stay.

Recovery ran in a specific order. Reconstruct the training data from the warehouse using the model's own scoring history as a check, retrain, and compare the new model's predictions against the incumbent's on live traffic in shadow mode until the agreement rate was high enough to argue the reconstruction was faithful. Only then fix the bug, because otherwise you cannot separate the effect of the fix from the effect of the reconstruction. Total time was around seven weeks, and most of it was archaeology rather than engineering.

The silent unit change

A demand forecasting model fed by an upstream billing service started receiving one field in currency units where it had previously arrived in cents. No error was thrown. The values were valid numbers in a plausible range, so nothing rejected them. Forecasts degraded gradually, procurement decisions followed them, and because the real outcome data arrived on a 30 day lag, the problem ran for 11 days before a human noticed inventory looked wrong.

What would have caught it was not sophisticated. A distribution check on that single feature against its training range would have fired on day one, and a prediction distribution monitor would have fired on day two. We built both in under a week, added a data contract test between the two services, and put the null and range checks into the pipeline as a hard gate. The interesting part of the postmortem was that the upstream team had announced the change. It went to a mailing list nobody on the ML side read.

The endpoint that ate the budget

An LLM backed support assistant launched on a single large GPU endpoint, running one request at a time, kept warm 24 hours a day for traffic that arrived almost entirely in a nine hour window. The monthly bill quadrupled against the forecast in the first month, and the immediate proposal on the table was to reduce the feature's scope.

The fix was ordinary engineering. Continuous batching through vLLM raised throughput on the same hardware by a large multiple. Right sizing moved the workload off the instance type that had been chosen during a proof of concept. Off peak hours scaled down rather than sitting idle. A smaller model handled the classification step that was being sent to the large one for no good reason. Nothing about the product changed, and the quality evaluation was rerun after each step rather than assumed.

The model that worked offline and did nothing online

A propensity model validated at a strong offline AUC and delivered almost no measurable lift after launch. The team spent three weeks re examining the model. The cause was in the features: an aggregation computed over a 30 day window in the training pipeline was computed over a calendar month in the serving path, and a null was treated as zero in one and as missing in the other. Small differences, consistently applied, on the two features the model relied on most.

We found it by logging serving features alongside predictions for a week and comparing them against the offline pipeline's output for the same entities on the same dates. That comparison is now an automated test that fails the build. The general lesson holds beyond this case: when a model performs offline and not online, look at the features before you look at the model, because that is where the fault is the overwhelming majority of the time.

What Goes Wrong, and What We Do About It

Knowledge ends up in one head

The classic offshore failure is a single engineer who understands the whole pipeline, and the client discovering that when they resign. We pair from the start, rotate ownership across pipelines, and require that runbooks are written by whoever did not build the component. It is slower in month one. It is the difference between a team and a dependency in month twelve.

Data access blocks the first month

The most common delay in an MLOps engagement is not technical. It is waiting for access. Security review, VPN provisioning, IAM roles and a decision on whether an offshore engineer may see production data can consume three weeks if it starts after the contract is signed. We start it during contracting, and where production access is genuinely not available we work against a synthetic or masked dataset for the first sprint rather than sitting idle.

Attrition and replacement

Attrition in the Indian technology market is real and pretending otherwise is how clients get surprised. Notice periods and what happens on a resignation are written into the contract before work starts rather than improvised afterwards, and where we can we overlap the outgoing engineer with the incoming one rather than handing over a document and leaving. Because the work is in code and the runbooks exist, a replacement is a slowdown rather than a restart.

Scope creep into data science

MLOps engagements drift into model building, usually because we are the ones with hands on the data and it feels efficient. We resist it, and we say why. If we own both the model and the evaluation of the model, nobody is independently checking the thing that matters. Where you need modelling work as well, it gets scoped separately with a different person accountable for it.

The costs people forget

Onboarding is real and unpaid attention on your side: expect a senior person to spend meaningful time in weeks one and two, and less thereafter. Ramp up to full productivity on a complex estate is four to six weeks, not day one. Management overhead is a few hours a week from someone on your side who can make decisions. And the platform itself has a running cost in tooling and compute that should be estimated at the start rather than discovered in the first bill.

If it does not work out

Exit terms are in the contract from day one, not negotiated when relations are strained. Notice periods and the length and rate of the knowledge transfer period are all agreed up front; everything sits in your repositories and your cloud account throughout so there is nothing to hand back, and offboarding runs to a documented checklist for access revocation. You should never be in a position where leaving us is technically difficult. If you are, we built it wrong.

Where This Sits Alongside Our Other Work

MLOps sits downstream of data engineering and upstream of nothing. If your pipelines are unreliable, the model platform will inherit that unreliability, so a data engineering engagement often needs to come first or run alongside. Where the modelling work itself is the gap rather than the operations around it, our machine learning team covers that side.

On the platform side, the Kubernetes, IaC and CI foundations that this work sits on are covered by DevOps engineering, and the two engagements are frequently staffed together. For staffing rather than a scoped build, you can hire Python developers in India for the pipeline and tooling work, or hire AWS developers in India where the platform is SageMaker centred and the cloud side is the constraint.