Wheely: Техническое собеседование

The posterior is 50%. Among 10,000 people, about 100 are sick and 99 of them test positive; among 9,900 healthy people, 99 false positives occur, so positives split 99 sick vs 99 healthy.

Use Bayes theorem:

P(sick | positive) = P(positive | sick) P(sick) / P(positive).

Here P(sick) = 0.01, P(positive | sick) = 0.99 and P(positive | healthy) = 0.01. The denominator is:

0.99 * 0.01 + 0.01 * 0.99 = 0.0099 + 0.0099 = 0.0198.

The numerator is 0.99 * 0.01 = 0.0099, so the posterior is 0.0099 / 0.0198 = 0.5.

The intuitive count version is often safer in interviews. Out of 10,000 people, 100 are sick and 9,900 are healthy. The test catches about 99 sick people and falsely flags about 99 healthy people. A positive result is therefore equally likely to be a true positive or a false positive.

Precision is 90 / 120 = 75%. Recall is 90 / 100 = 90%.

Precision asks what fraction of predicted positives are correct. The model predicted 120 spam emails, and 90 of them are truly spam:

precision = TP / (TP + FP) = 90 / 120 = 0.75.

Recall asks what fraction of all real positives were found. There are 100 true spam emails, and 90 were predicted as spam:

recall = TP / (TP + FN) = 90 / 100 = 0.9.

The remaining confusion matrix values are FP = 30, FN = 10, TN = 870.

C(10, 6) / 2^10 = 210 / 1024, about 20.5%.

Each sequence of 10 fair tosses has probability (1/2)^10. To get exactly 6 heads, choose which 6 of the 10 positions are heads:

C(10, 6) = 10! / (6! 4!) = 210.

Therefore the probability is C(10, 6) (1/2)^6 (1/2)^4 = C(10, 6) / 2^10 = 210 / 1024.

The MLE is p_hat = k / n.

The likelihood of observing k heads and n-k tails is proportional to:

L(p) = p^k (1-p)^(n-k).

The log-likelihood is:

l(p) = k log p + (n-k) log(1-p).

Differentiate and set to zero:

k / p - (n-k) / (1-p) = 0.

Rearrange:

k(1-p) = (n-k)p, so k = np, and p_hat = k / n.

The second derivative is negative for p in (0, 1):

-k / p^2 - (n-k) / (1-p)^2 < 0.

So the stationary point is a maximum. Boundary cases are intuitive: if k=0, p_hat=0; if k=n, p_hat=1.

Define hypothesis, primary metric, unit, randomization, guardrails, alpha, power and MDE. Sample size depends on baseline variance/rate, desired MDE, alpha, power and traffic. A p-value is the probability, under the null, of observing a result at least this extreme.

A good A/B setup starts before data is collected. Define the product hypothesis, primary metric, guardrail metrics, experiment unit, randomization scheme, target population, exclusion rules, alpha/significance level, desired power, minimum detectable effect and stopping rule.

Sample size or duration depends on the baseline rate or metric variance, the minimum effect you care about, alpha, desired power and available traffic. Smaller effects, noisier metrics and lower traffic require longer experiments.

For a conversion metric, a two-proportion z-test or equivalent confidence interval is common when sample sizes are large enough. For continuous metrics, a t-test may be appropriate if the unit-level metric and independence assumptions are reasonable; heavy tails or user-level aggregation may require bootstrap, winsorization or a different metric design.

A p-value is not the probability that the null hypothesis is true. It is the probability of seeing data as extreme or more extreme than what you observed, assuming the null is true.