Statistics for Data Science Interviews — Hypothesis Testing, Distributions, and Bayes

Statistics for data science interviews is not a trivia contest about formulas. The real test is whether you can turn an ambiguous product or modeling question into assumptions, a measurable experiment, a defensible statistical method, and a plain-English recommendation. Hypothesis testing, distributions, and Bayes show up because they reveal how you reason under uncertainty, not because interviewers want a recited textbook chapter.

Statistics for data science interviews: what interviewers are really testing

A good statistics answer has four layers: the business question, the data-generating process, the statistical method, and the decision rule. Candidates often jump straight to a t-test or a p-value. Strong candidates first ask what is being measured, how observations are sampled, what could bias the sample, and what action the result will change.

| Interview signal | What the interviewer wants | Strong answer habit | |---|---|---| | Hypothesis testing | Can you compare alternatives without overclaiming? | State null, alternative, metric, alpha, power, and decision rule. | | Distributions | Can you model real processes with the right assumptions? | Match the distribution to the outcome and constraints. | | Bayes | Can you update beliefs with new evidence? | Define prior, likelihood, posterior, and base-rate effects. | | Experiment design | Can you protect decisions from noise and bias? | Discuss randomization, sample size, peeking, and guardrail metrics. | | Communication | Can a PM or executive act on your result? | Translate uncertainty into a recommendation and next step. |

If you remember only one framework, use this: "I would clarify the decision, define the metric, inspect the data-generating process, choose a method, state assumptions, then explain what result would change my recommendation." That sentence buys you time and shows mature thinking.

Hypothesis testing without sounding mechanical

Hypothesis testing answers should start with the business decision. Suppose a product manager asks whether a new onboarding flow improved activation. A weak answer says, "Run a t-test." A better answer says, "The null hypothesis is that activation is unchanged. The alternative is that the new flow increases activation. I would randomize users, compare activation rates after a fixed exposure window, estimate the lift and confidence interval, and decide in advance what minimum detectable effect is worth shipping."

The core ingredients are predictable:

• Null hypothesis: the baseline claim, usually no effect or no difference.
• Alternative hypothesis: the effect you care about, one-sided or two-sided.
• Test statistic: the standardized signal you calculate from the sample.
• P-value: the probability of seeing data this extreme, or more extreme, if the null were true.
• Alpha: the false-positive tolerance, commonly 0.05 but not sacred.
• Power: the probability of detecting a real effect of a chosen size.
• Effect size: the practical magnitude, which matters more than p-value alone.

Interviewers like to press on interpretation. A p-value of 0.03 does not mean there is a 97% chance the feature works. It means that, under the null hypothesis and the test assumptions, results at least this extreme would occur about 3% of the time. A confidence interval does not mean the true effect has a 95% chance of being inside this specific computed interval; it means the procedure would cover the true value in 95% of repeated samples. You do not need to deliver a philosophical lecture, but you should avoid the common wrong statements.

For proportion metrics, like conversion, activation, or retention, discuss a two-proportion z-test or a chi-square test for independence. For mean metrics, like order value or time spent, discuss a t-test if sample sizes are moderate and observations are independent. If the metric is skewed, heavy-tailed, or has many zeros, mention bootstrap confidence intervals, nonparametric tests, winsorization, or transforming the metric. The point is not to memorize every test. The point is to show that the method follows the metric.

Distribution questions: choose the model from the story

Distribution questions are usually disguised as product or operations questions. The interviewer may ask, "How would you model the number of support tickets per hour?" or "What distribution would you expect for whether a user clicks an ad?" Do not list distributions randomly. Identify the event, constraints, and independence assumptions.

| Situation | Useful distribution | Why it fits | Interview caveat | |---|---|---|---| | Click/no-click, churn/no-churn, fraud/not-fraud | Bernoulli | One binary outcome | Probability may vary by segment. | | Number of conversions in n trials | Binomial | Count of successes with fixed n | Assumes independent trials and same probability. | | Events per time interval | Poisson | Counts over time or space | Rate may not be constant; overdispersion is common. | | Time until next event | Exponential | Waiting time between Poisson events | Memoryless assumption may be unrealistic. | | Average of many small independent effects | Normal | Central limit behavior | Raw data may not be normal. | | Small-sample mean with unknown variance | t-distribution | Heavier tails than normal | Useful for confidence intervals and t-tests. | | Variance or goodness-of-fit problems | Chi-square | Sum of squared standard normals | Sensitive to low expected counts. |

A strong distribution answer often includes the sentence, "I would start with this distribution as a modeling assumption, then check residuals, calibration, and segment-level fit." That protects you from overclaiming. Real user behavior rarely follows clean textbook assumptions. Clicks cluster by user, fraud has adversarial patterns, support tickets spike after incidents, and revenue is famously heavy-tailed.

For example, if asked to model ride requests per minute in a city, Poisson is a reasonable starting point because you are counting arrivals in a time window. Then add nuance: the rate changes by neighborhood, time of day, weather, pricing, and events. A non-homogeneous Poisson process or a hierarchical model could be more realistic. In an interview, that progression is excellent: simple first, then explain the limitation and upgrade path.

Bayes: base rates, priors, and updating beliefs

Bayes questions test whether you can avoid base-rate neglect. The formula is compact: posterior is proportional to prior times likelihood. In interviews, the plain-English version matters more: start with what was believed before the evidence, ask how likely the evidence is under each hypothesis, then update.

A classic version: a fraud model flags a transaction. The model catches 95% of fraud and falsely flags 2% of legitimate transactions. Fraud is only 0.5% of all transactions. What is the chance a flagged transaction is actually fraud? Many candidates answer near 95%. The correct reasoning is lower because legitimate transactions are so common. In 100,000 transactions, about 500 are fraudulent; the model flags 475 of them. Of 99,500 legitimate transactions, 1,990 are falsely flagged. Among 2,465 flagged transactions, only 475 are fraud, or about 19%. The flag is useful, but it is not proof.

That example lets you say something practical: "I would not auto-block every flagged transaction. I would route high-risk cases to step-up verification, tune thresholds by cost of false positives and false negatives, and monitor calibration by segment." Now the statistics answer becomes a product answer.

Bayes also appears in A/B testing and machine learning. Priors can stabilize estimates for low-volume segments. Bayesian credible intervals can be more intuitive for stakeholders than frequentist confidence intervals, but they require explicit assumptions. Naive Bayes classifiers assume conditional independence among features, which is often false but can still work surprisingly well for text classification. The interview win is to explain the assumption and when it breaks.

A/B testing traps interviewers expect you to catch

Statistics interviews often become experiment-design interviews. The traps are consistent.

Peeking at results: If teams check the p-value every hour and stop when it crosses 0.05, the false-positive rate inflates. Say you would use a fixed sample size, sequential testing methods, or at least pre-defined stopping rules.

Underpowered tests: A non-significant result does not prove no effect. It may mean the experiment could not detect the effect size that matters. Discuss minimum detectable effect and power before launch.

Multiple comparisons: If you test twenty metrics or segments, one may look significant by chance. Mention correction methods, hierarchy of metrics, or treating segment findings as exploratory.

Metric mismatch: A feature can increase clicks while reducing revenue, trust, or retention. Pick a primary metric and guardrail metrics.

Interference: In marketplaces, social products, ads, or ranking systems, one user's treatment can affect another user's outcome. User-level randomization may not be enough; cluster randomization or geo experiments may be needed.

Novelty effects: A short-term lift may fade. For retention or habit features, include a follow-up window.

A crisp answer is: "I would decide the primary metric, guardrails, unit of randomization, sample size, duration, and stopping rule before looking at results. Afterward I would report effect size, uncertainty, and whether the result is practically meaningful."

How to answer a live statistics interview question

Use a repeatable structure instead of improvising from memory:

1. Clarify the decision. "What action will we take if the answer is yes?"
2. Define the metric. Avoid vague words like engagement; specify activation, retention, revenue, latency, or error rate.
3. State assumptions. Independence, stationarity, segment mix, sample representativeness, and measurement quality.
4. Choose the method. Tie the test or distribution to the data type.
5. Quantify uncertainty. P-value, confidence interval, credible interval, or prediction interval.
6. Explain limitations. Bias, confounding, underpowering, multiple testing, or model misfit.
7. Recommend an action. Ship, iterate, collect more data, segment analysis, or run a better experiment.

Here is a sample answer opening: "I would treat this as a comparison of two conversion rates. My null is no difference in activation between control and treatment. I would use a two-proportion test or bootstrap the lift, depending on sample size and distribution. Before running it, I would calculate the sample size needed for a lift we actually care about, freeze the stopping rule, and track guardrails like support tickets and day-seven retention."

That kind of answer is specific, business-aware, and statistically careful.

Practice checklist for statistics interview prep

Build a short drill set rather than reading endlessly. You should be able to do these without notes:

• Explain p-values, confidence intervals, Type I error, Type II error, and power in plain English.
• Choose between a t-test, z-test for proportions, chi-square test, bootstrap, and nonparametric alternative.
• Match Bernoulli, binomial, Poisson, exponential, normal, t, and beta distributions to realistic product examples.
• Solve a Bayes/base-rate problem with round numbers.
• Design an A/B test with a primary metric, guardrails, randomization unit, sample-size logic, and stopping rule.
• Discuss when observational analysis cannot prove causality.
• Explain why correlation is not enough and what confounders you would check.
• Translate a statistically significant but tiny lift into a product recommendation.

For resumes and interview stories, frame statistics as decision support. Instead of "Performed hypothesis testing," write or say, "Designed an activation experiment with pre-registered success metrics, 80% power for a 2-point lift, and guardrails for support contacts; recommended rollout after the treatment improved activation with no retention degradation." The second version shows method, judgment, and impact.

The best candidates make statistics feel like a practical operating system for uncertainty. They know the formulas, but they do not hide behind them. They ask what decision is at stake, choose assumptions transparently, quantify uncertainty, and explain the tradeoff in language a product, engineering, or finance partner can use.

Related guides

• North Star Metric in PM Interviews — Choosing, Defending, and Stress-Testing It — A practical PM interview guide for choosing a North Star metric, defending it with an input tree, and stress-testing it with guardrails so it does not become a vanity metric.
• A/B Testing Interview Questions in 2026 — Power Analysis, Peeking, and SRM — A tactical guide to A/B testing interview questions in 2026, with answer frameworks for power analysis, peeking, sample-ratio mismatch, guardrails, metrics, and experiment trade-offs. Built for product analysts, data scientists, PMs, and growth roles.
• Big-O Complexity Cheatsheet for Coding Interviews 2026 — A no-fluff Big-O reference card covering every complexity class, data structure, and algorithm pattern you'll face in coding interviews.
• Caching Strategies for System Design Interviews: Write-Through, Write-Back, and TTL Patterns — The caching section of a FAANG system design loop is where mediocre candidates blur together. Here's how to name tradeoffs, pick a pattern on purpose, and survive the hot-key follow-up.
• Circuit Breaker Pattern in Interviews: Fault Tolerance and Graceful Degradation — The circuit breaker is the pattern most candidates name and none can actually configure. Here is how to talk about states, thresholds, and graceful degradation at staff level.