Geometric Median & Robust Estimation
Stanislav Minsker addresses the following question:
Problem: Given X1,…,Xk that are i.i.d. random variables, how can we estimate the mean of the distribution so that the confidence interval still has a coverage probability ≥1−O(e−t)? Note: the smaller the confidence interval, the better the estimator.
- Assuming the variables from normal distribution N(μ,σ2). If the estimator is constructed as ˆμn=ΣiXi/n, then:
For non-normal distribution, the confidence interval is much different:
Pr(|μn−ˆμn|≥σ√et/n)≤e−tSince the length of this interval depends on et, can we do better?
- If we split the sample into k=floor(t)+1 groups G1,…,Gk each of size approximate fracnt, then median-of-means estimator has the following confidence interval (result due to Nemirovskii and Yudin, 1983):
Pr(‖μn−ˆμn‖≥σC√tn)≤e−t
Where C is a constant. The length of confidence interval for median-of-means is much smaller than for the naive mean estimator.
- Minsker’s work extends this to multi-dimensional distributions in Banach space using geometric median to obtain confidence intervals of sub-exponential length with high coverage.
More in the paper here. See also, Fermat-Weber problem.