Data Science Services in India

What Does a Data Science Engagement Actually Cover?

Not every engagement includes all of this, and the sequence matters more than the list. Feature engineering before target definition is wasted work. What follows is the full surface area, roughly in the order we build it.

Problem framing and the decision map

The first deliverable is a one page document that nobody enjoys writing and everybody refers back to. It names the decision being improved, who makes it, how often, what they do today, what a better decision is worth in money or time, and what the cost of a wrong call is in each direction. It also names the constraint: how many accounts can be contacted, how many parts can be inspected, how much inventory can be moved. That constraint decides whether you need a probability, a ranking or a threshold, and those are three different builds.

Data audit and feasibility read

Before modelling we go through what you actually have: row counts by period, null rates and when they changed, cardinality drift in categorical fields, duplicate keys, timezone handling, and the exact meaning of every timestamp. Half the surprises in a data science project are here. A field that was optional until last March, a status code that got reused for a new meaning, an ETL job that silently backfills. This phase ends with a written read on whether the question is answerable with the data in hand.

Target definition and label engineering

What exactly are you predicting, measured how, at what point in time, over what horizon. Churn is not one target. Churn within 30 days of the renewal date, defined as failing to renew, is a target. So is a support ticket predicting a downgrade. They lead to different models and different interventions. Where labels do not exist, this phase builds them, through annotation, weak supervision, or an operational proxy that gets validated against a hand labelled sample.

Baselines

Three of them, always, before any model is trained. The current process, measured on the same data as everything else. A trivial statistical rule such as the majority class, the seasonal naive forecast or the last known value. And a simple, interpretable model, usually a regularised regression or a shallow tree. Every subsequent result is reported as a gain over these, not as an absolute number, because an absolute number tells you nothing about whether the work was worth doing.

Feature engineering with point in time discipline

The features that matter are usually derived rather than raw: recency and frequency aggregates, ratios against a peer group, deltas against the customer's own history, time since last event, rolling windows at several horizons. Every one of them is built so it could only have used information available at prediction time. That constraint is what separates a feature set that transfers to production from one that quietly cheats.

Model development and selection

Gradient boosting on tabular data, because it is still the strongest default and it is honest about what it needs. Regularised linear and generalised additive models where the coefficients have to be explainable to a regulator or a credit committee. Sequence models where order genuinely carries information. Deep learning where the input is text, image or audio. Selection is by cross validated performance under the right splitting scheme, then by operational cost, then by explainability, in that order unless the domain reverses it.

Evaluation design

Chosen before the first model runs, not after the results arrive. Which metric, which slices must not regress, which threshold and why, what the confusion matrix costs at that threshold in your currency, and what result would make us recommend abandoning the project. Writing the abandon condition down in advance is the single cheapest form of intellectual honesty available in this work.

Causal and experimental analysis

Where the question is about effect rather than prediction, the toolkit changes completely. Difference in differences, synthetic control, instrumental variables, propensity weighting and uplift modelling. Where a live experiment is possible, the design work happens before launch: hypothesis, primary metric, minimum detectable effect, sample size, stopping rule, guardrail metrics.

Interpretation and handover

A model that the business does not understand does not get used. Handover includes global and local explanations, the segments where the model is weakest, a written statement of what it should not be used for, and code packaged so it can be trained again by someone else. Where the result goes into production, the boundary with the platform work is explicit and agreed rather than assumed.

Baselines First, and Sometimes the Baseline Wins

A model without a baseline is a number without a meaning. An AUC of 0.82 sounds respectable until you learn that the existing rule scores 0.80 and costs nothing to run. Baselines are cheap, they take a day or two, and they set the bar that every later decision is judged against.

The three baselines we always build

The first is the incumbent: whatever your business does today, whether that is a rule, a heuristic, an analyst's judgement or nothing at all. It has to be measured on the same rows and the same metric as the model, which is often the first time anyone has actually quantified it. The second is trivial: majority class for classification, last value or seasonal naive for forecasting, mean for regression. The third is a simple model with sensible regularisation and no exotic features.

That third baseline does more work than people expect. If a logistic regression on twelve features gets within a whisker of a tuned gradient boosting model on three hundred, you have learned that the signal is mostly linear and additive, that the extra features are noise, and that you should probably ship the simple one. If the gap is large, you have learned that interactions matter and it is worth investigating which.

When the simple answer is the right answer

We have recommended dropping the model more than once. A demand forecast where seasonal naive was within a few percent of every fancy method, because the series was dominated by a weekly pattern with almost no trend. A lead scoring problem where three business rules captured nearly everything the classifier found, and the rules could be explained to a sales director in one sentence. A fraud screen where the real gain came from fixing a data entry validation rather than from any model.

The reason to say this out loud is that a model carries permanent costs. Someone has to own it, monitor it, retrain it, explain it to auditors and debug it when a source system changes. If the gain over a rule is small, that overhead eats the gain within a year. The honest question is not whether the model is better, but whether it is better by enough to be worth operating.

Report gains, not absolutes

Every result we report is framed as a delta against the baseline, with an interval around it rather than a single number. Bootstrapped confidence intervals on the test set, or the spread across cross validation folds, whichever fits. A model that beats the baseline by two points with a five point spread has not beaten anything yet, and presenting it as a win is how teams lose credibility with their own executives.

Choosing Metrics That Match the Decision

Accuracy is almost never the right metric, and the default metric in whichever library you are using is a convenience, not a recommendation. The metric should follow from what a mistake costs you in each direction, and those costs are usually asymmetric by an order of magnitude.

Precision and recall priced in your currency

Precision is how many of your flagged cases are real. Recall is how many of the real cases you caught. You trade one against the other by moving a threshold, and the right place to put that threshold depends entirely on the relative cost of the two errors. Consider a fraud screen. A false positive blocks a genuine customer, costing you the margin on that order plus some goodwill. A false negative lets a fraudulent order through, costing you the full chargeback plus fees. If the second is ten times the first, the arithmetic says operate at high recall and accept the precision hit.

Now consider a manual review queue instead. Every alert costs an analyst fifteen minutes, and you have four analysts. Recall beyond what those analysts can process is worthless, so the objective becomes precision at a fixed alert volume. Same model, same data, opposite threshold. This is why we insist on the operational constraint being written down during framing: it determines the metric, and the metric determines everything after it.

The practical output we produce is a threshold table: for each candidate cut point, the expected count of true positives, false positives, false negatives, the workload implied, and the net value using your own cost figures. Business stakeholders engage with that table in a way they never engage with an ROC curve.

ROC AUC lies under heavy imbalance

ROC AUC is threshold independent and easy to quote, which is exactly why it gets misused. Under strong class imbalance it stays flattering, because the false positive rate has a huge denominator. Push the positive rate down to one in a thousand and thousands of false positives barely move the false positive rate at all, while precision collapses. Precision recall AUC, or average precision, is the honest summary in that regime because its baseline is the positive rate itself.

For ranking problems where only the top of the list gets acted on, neither is quite right. Precision at k, recall at k, and lift over random in the top decile speak directly to the operational reality of a team that can only work through so many cases a day. Where the ordering across a whole list matters, normalised discounted cumulative gain is the standard choice.

Calibration, and when it actually matters

A well ranked model is not necessarily a well calibrated one. If a downstream process multiplies the probability by an expected loss, or compares scores across segments, or feeds the number into a pricing formula, calibration is not optional. Gradient boosted trees are often over confident at the extremes, and the usual remedies are Platt scaling or isotonic regression fitted on a held out calibration set, checked with a reliability curve and expected calibration error rather than assumed to have worked.

If nothing downstream consumes the number as a probability, calibration is a nice property that does not need to be paid for. Knowing which situation you are in is the point.

Regression and forecast metrics have their own traps

Mean absolute error and root mean squared error disagree deliberately: RMSE punishes large misses harder, so choose it when one big error hurts more than several small ones, and MAE when it does not. MAPE is popular with business audiences and dangerous in practice, because it explodes near zero and is undefined at zero, which is exactly the situation in intermittent demand. Symmetric MAPE has its own asymmetry problems. For forecasting we prefer scaled errors such as MASE, which compares against the naive forecast on the same series and stays interpretable across series of very different magnitudes.

R squared deserves a caution of its own. It measures variance explained against the mean of the sample it is computed on, so it is not comparable across datasets with different variance, and a high value on a time series with a strong trend usually just means the model learned the trend.

Slices, not just the aggregate

An aggregate metric hides the failure that will actually cause a complaint. We report performance by segment as standard: new versus tenured customers, small versus large accounts, each region, each channel, each product line, and the newest cohort of data. A model that improves the overall metric while regressing on the enterprise segment is a model that should not ship, and you only find that out if the slice is on the report before the decision is made.

Correlation, Causation and When You Need Causal Inference

A predictive model tells you what tends to happen alongside what. It says nothing about what happens if you intervene. Confusing the two is expensive, and the mistake usually survives review because the model is genuinely accurate.

The classic trap, in three versions

Customers who use the mobile app churn less, so the growth team pushes app installs, and churn does not move, because engaged customers install the app rather than the app creating engagement. Accounts that received a retention call renewed at a higher rate, so the call is judged effective, but the calls went to accounts a human thought were saveable. Discount recipients spend more, so discounting is expanded, when discounts were being given to people who were already going to buy.

Every one of these is a confounded comparison. The treated and untreated groups differed before treatment, and the model has no way to know that. This is the point at which a different toolkit is required.

The methods we use and when each fits

A randomised experiment is the strongest evidence and the first thing we ask about. Where randomisation is impossible, difference in differences works when you have before and after data for a treated and an untreated group, and it depends on a parallel trends assumption that must be checked against pre period data rather than asserted. Synthetic control fits where a single unit was treated, such as one region or one store, by constructing a weighted combination of untreated units that tracks the treated one before the change.

Instrumental variables help where something shifts treatment without directly affecting the outcome, which is rare and demands a defensible story. Regression discontinuity applies where treatment is assigned by a cutoff, such as a credit score threshold, and comparisons just either side of it approximate an experiment. Propensity score methods balance observed covariates between groups, with the honest caveat that they can do nothing about confounders you did not measure.

Uplift modelling, because average effects hide the useful part

Knowing that a campaign lifts conversion by two points on average does not tell you who to send it to. Some customers convert either way and the discount is pure margin loss. Some are pushed over the line by it. A few react badly to being contacted. Uplift or conditional average treatment effect modelling targets the second group specifically, using two model approaches, transformed outcome methods, causal forests or the meta learners in libraries such as EconML and CausalML.

Uplift is harder to validate than accuracy because you never observe both outcomes for the same person. Qini and uplift curves on a randomised holdout are the standard, and the holdout has to be preserved rather than absorbed into the campaign, which is a conversation with marketing that is easier to have early.

Say out loud what you cannot rule out

Every observational estimate rests on assumptions that cannot be fully verified. We state them explicitly, run sensitivity analysis on how strong an unmeasured confounder would need to be to overturn the finding, and use placebo tests on periods or groups where no effect should exist. If the conclusion cannot survive that, it gets reported as suggestive rather than established. Directed acyclic graphs are useful here mainly as a communication device, because they force everyone in the room to state which arrows they believe in.

Forecasting: Seasonality, Holidays and Hierarchies

Demand planning, capacity, cash flow, headcount, inventory. Forecasting is where data science most often touches a decision with money attached, and it has its own failure modes that general modelling advice does not cover.

Start with seasonal naive and earn your way past it

The seasonal naive forecast, meaning last week's value for the same weekday or last year's value for the same month, is a surprisingly strong competitor. It is also the denominator in MASE, so it comes for free as a benchmark. A weekly retail series with a stable pattern and no trend is often forecast almost as well by that rule as by anything else, and knowing this before you spend six weeks on a model saves the six weeks.

Classical methods still hold up

Exponential smoothing in its ETS form handles level, trend and a single seasonality with very few parameters and is hard to beat on short, well behaved series. ARIMA and its seasonal variant suit series with autocorrelation structure that survives differencing, and the automatic order selection in statsmodels or the R heritage implementations removes most of the manual work. TBATS earns its place with multiple overlapping seasonalities, such as hourly data with a daily and a weekly cycle and a yearly pattern on top.

The general point is that classical methods need less data per series and fail more gracefully than machine learning approaches. With eighteen months of monthly history, they are usually the correct choice and a boosted tree is not.

Prophet and its trade-offs

Prophet is a decomposable model with a piecewise linear or logistic trend, Fourier terms for seasonality and a built in holiday mechanism. What it buys you is speed to a reasonable forecast, interpretable components you can show a planner, and easy handling of missing periods and outliers. It is a fair default for business series with strong seasonality and known event effects.

The trade-offs are real. It is a curve fitting approach rather than a generative time series model, so it does not use autocorrelation in residuals the way ARIMA does. Its changepoint detection can be over eager on noisy series, producing trend flips that a human would not draw. Its uncertainty intervals are frequently too narrow on the trend component. On short series or on data with strong autocorrelation, ETS often beats it. We use it, we do not default to it, and we always benchmark it against the naive and classical options on the same backtest.

Holidays, promotions and the calendar you forget

Holiday effects are usually the largest single source of forecast error in retail and consumer businesses, and they are not simple flags. Effects spread over a window before and after the day. Moving holidays such as Easter, Diwali, Ramadan and Chinese New Year shift the pattern between months and years, which wrecks a naive year on year comparison. Payday cycles matter in some markets. School terms matter in others. Promotions have to be treated as known future regressors, which means someone has to give you the promotional calendar for the forecast horizon, and that request often surfaces the fact that nobody keeps a clean history of past promotions.

Machine learning for forecasting, and when it pays

Gradient boosting on lag, rolling and calendar features is a strong global approach when you have many related series and want one model across all of them, since it can borrow strength from series with more history. LightGBM is our usual choice for this. Deep learning approaches such as DeepAR, N BEATS and the temporal fusion transformer are worth the effort when the series count is large, the histories are long, and covariates matter. Below a few hundred series, the overhead rarely pays.

Two hazards to name. Lag features must be constructed so nothing from the future leaks in, and multi step forecasts need either a recursive strategy that compounds error or a direct model per horizon that does not. And global models tend to underfit the unusual series, so the small number of items that matter most to your business can end up worse than they were under the old method.

Intermittent demand needs different tools

Spare parts, slow moving SKUs and long tail inventory produce series that are mostly zeros with occasional spikes. Standard error metrics are close to meaningless here and MAPE is undefined. Croston's method and its variants model the demand size and the interval between demands separately, which is the right decomposition. Often the useful output is not a point forecast at all but a distribution feeding a service level and reorder point calculation.

Hierarchies have to reconcile

Forecast by SKU, by store, by region and nationally, and the numbers will not add up. Finance will notice. Bottom up aggregation is noisy, top down disaggregation loses item level detail, and middle out is a compromise. Optimal reconciliation methods, MinT in particular, use the covariance of base forecast errors to produce a coherent set that is usually more accurate than any single direction. If your planning process consumes numbers at more than one level, reconciliation is not optional.

Backtest with a rolling origin and forecast a range

A single train and test split on a time series tells you how one period behaved. Rolling origin evaluation, refitting at successive cut points and scoring each horizon separately, tells you how the method behaves in general and how error grows with horizon. That growth curve is what a planner needs, because it tells them how far ahead the forecast can be trusted. And the output should be a prediction interval, not a point. A safety stock decision consumes the upper quantile. A cash flow plan consumes the lower one. Giving a planner a single number throws away the information they most need.

Three Situations We Are Usually Called Into

These are patterns rather than case studies, described the way they typically arrive. If one of them sounds like your quarter, the diagnosis section under each is the part worth reading.

The churn model that predicted the paperwork

A subscription business has a churn model with a strong offline score and no measurable effect on retention. The first thing we check is the target. In this pattern the label comes from a subscription status field that flips on the termination date, and the top features turn out to be things like a downgrade record or a final invoice adjustment, both of which are created as part of the cancellation process. The model has learned to recognise cancellations that have effectively already happened.

The rebuild starts with the intervention. If the retention team needs three weeks to run a save play, the target becomes non renewal within a window that starts three weeks before the renewal date, and every feature is rebuilt as of that observation point. The score drops, often sharply, because the leaked signal is gone. What replaces it is a model whose top ranked accounts can still be saved, which is the only version that has any value. The second half of the work is usually a holdout design so the retention programme itself can be measured rather than assumed.

The forecast that finance stopped trusting

A distributor forecasts demand at SKU and warehouse level. Planners have quietly built their own spreadsheet adjustments on top, which is the clearest signal that a forecast has lost credibility. Investigation usually finds three things: promotions were never included as a regressor even though they drive a large share of volume, holiday effects were modelled as single day flags rather than windows around moving dates, and the SKU, warehouse and national forecasts were produced independently so they never reconciled.

The fix is unglamorous. Reconstruct a clean promotional history from whatever systems hold it, model holiday windows properly including the moving ones, replace the single split evaluation with rolling origin backtests so error by horizon is visible, and reconcile the hierarchy with an optimal method rather than by rescaling. Then give planners intervals instead of points, and a short list of the series where the model knows it is weak, so their manual effort goes where it actually helps rather than everywhere.

The experiment programme producing wins that never show up in revenue

A product team runs a healthy number of tests, ships most of them as wins, and the aggregate metric refuses to move. Three causes usually combine. Tests are stopped when the dashboard first shows significance, which turns a five percent error rate into something far higher. Several metrics are checked per test with no correction. And on a marketplace or any shared inventory product, the treated group is partly taking supply from the control group, so a genuine looking lift is partly redistribution.

Rebuilding this is process work more than modelling work. Always valid confidence sequences so the team can look whenever it wants without inflating error, a pre registered analysis plan template that takes half an hour per test, one nominated primary metric with the rest treated as exploratory, cluster or switchback randomisation where interference is plausible, and CUPED to claw back the power lost by doing all of the above properly. The visible outcome is fewer declared wins, which is uncomfortable for a quarter and correct thereafter.

How We Run Data Science Delivery From India

The offshore part of this is a real question and it deserves a real answer rather than a claim about follow the sun coverage. Here is how the working week actually looks, including the parts that are inconvenient.

The overlap window, stated honestly

A standard Indian working day of roughly 09:30 to 18:30 IST gives you very different overlap depending on where you sit. With the UK you get around four hours in winter and five in summer, which is comfortable. With Australian eastern time you get roughly two to three hours in your afternoon, and with New Zealand closer to one or two. With US Eastern you get almost nothing, because the Indian day ends at roughly the hour your morning starts.

With US Pacific, on a standard schedule, there is none at all.

So for North American clients we shift the day rather than pretend. A team working a later Indian schedule can put several hours of live overlap into your morning, at the cost of an engineer's evening, which is a real cost and shows up in retention if it is imposed rather than staffed for. The workable version is a partial shift: one or two people on the later schedule for live collaboration while the rest work normal hours, with a written handover holding it together. Which shape applies to your engagement is something we agree with you before work starts rather than something we assert here.

Written first, because the meeting is not available

With a narrow overlap, anything that depends on synchronous conversation becomes a bottleneck. So decisions are written down. Every analysis lands as a short document with the question, the method, the result, the caveat and the recommendation, posted where you can read it when your day starts. Every model has a decision log recording what was tried and why it was rejected. Standups are written, not called. The reason this works better than it sounds is that data science conclusions need to survive being read six months later by someone who was not there, and a call leaves no trace.

Review that is specific to analytical work

Ordinary code review catches bugs and misses the mistakes that matter here. Our review checklist asks different questions. Is the split appropriate to the temporal and group structure. Could any feature have used information from after the observation point. Was any preprocessing fitted before the split. Does the metric match the decision. Are the reported intervals real. Has the model been checked on the slices that matter, not only in aggregate. Two people sign off any result that will inform a decision, and the second person is expected to try to break it.

Notebooks are treated as exploration, not as deliverables. Anything that produces a result someone will act on gets moved into a repository with pinned dependencies, a seeded and reproducible entry point, and tests on the feature transformations. If a second person cannot rerun it and land within a stated tolerance, it is not finished.

Access, residency and what actually crosses a border

The default arrangement we push for is that your data does not move. Analysts get named identities in your cloud account with scoped roles, work inside a hosted notebook environment or a browser accessible workspace in your chosen region, and do not pull raw records onto a laptop. Where the data is sensitive, most of the modelling can be done on masked, hashed or synthetically generated columns, with the final validation run against real data inside your environment. What crosses the border is access under audit rather than data. The specific terms covering confidentiality, IP assignment and data processing are set in the agreements before work starts, and they are yours to define with your counsel rather than ours to summarise on a web page.

The talent pool for this specific skill

India has depth in the mathematical and statistical end of this work, and the supply of people who can fit a model is very large. The scarcer profile, here as everywhere, is the person who will argue with the brief: who asks what the score will be used for, notices that the target is defined by an accounting event, and is willing to tell a client that the baseline already wins. That is what our screening is built around. Candidates work through a problem with a deliberately leaky feature and a target that does not match the stated decision, and what we are watching for is whether they spot it and how they say so.

Ways to Work With Us

Three shapes, chosen by what you actually need rather than by what is easiest to staff. Scope and terms are agreed with you before anything starts.

Frequently Asked Questions About Data Science Services in India