Frequentist and Bayesian Estimation

Since no other information is known about the possible values of the CEO’s salary, the reporter needs to estimate the unknown $\theta$ using the sample $X^n.$ For the uniform distribution the maximum likelihood estimator (MLE) for $\theta$ will be $${\hat{\theta }}_n=X_{(n)}=max(X_1,\cdots ,X_n).$$ Therefore, the reporter needs to ask for salary information from as many employees of bank X as possible and take the maximum of these values, which will serve as an estimate for the CEO’s salary.

Now suppose that the investigative reporter wants to get a Pulitzer prize for his reporting and remembers that he has a minor in Statistics. He reads economics literature and finds out that economists established that nationally the salaries of CEOs of banks follow the Pareto distribution $\theta \sim Pa({\theta }_0,a)$, because of the Pareto principle, which states that a large portion of wealth of CEOs is held by a small fraction of them. Using this prior distribution, a Bayesian estimator for the salary of CEO of bank X will be

$${\hat{\theta }}_n^B=\frac{a+n}{a+n-1}max({\theta }_0,X_1,\cdots ,X_n).$$ Here $a$ and ${\theta }_0$ are unknown as well and can be estimated using the available data of CEO salaries of other banks. Therefore, the investigative reporter decides to use the work of his colleagues - other investigative reporters - who conducted studies in other banks and reported estimates for salaries of CEOs. Denote this new sample of salaries of other CEOs as $${\vartheta }^m=({\vartheta }_1,\cdots ,{\vartheta }_m).$$ Since $\theta$ follows the Pareto distribution then the parameters ${\theta }_0$ and $a$ can be estimated as follows: $${\hat{\theta }}_0={\vartheta }_{(1)}=min({\vartheta }_1,\cdots ,{\vartheta }_m),\hat{a}=\frac{m}{\sum _{i=1}^m\ln \frac{{\vartheta }_i}{{\vartheta }_{(1)}}}.$$ Therefore, drawing on reports from other investigations and his own survey, the investigative reporter can obtain the following estimator of the salary of the CEO

$${\hat{\theta }}_n^B=\frac{\hat{a}+n}{\hat{a}+n-1}max({\vartheta }_{(1)},X_1,\cdots ,X_n).$$ Therefore, each time a new salary of some CEO is reported $({\vartheta }_{m+1}),$ this new data can be used by the investigative reporter to update the estimate of the salary of the CEO of bank X.

Pareto distributions

The random variable $\xi$ has a Pareto distribution with parameters $\theta >0$ and $a>0$, $\xi \sim Pa(\theta ,a)$ if the distribution function is given by the formula $$F(x)=1-{\left(\frac{\theta }{x}\right)}^a,x>\theta ,$$ and $F(x)=0,x\le \theta .$ The density function will have the form $$f(x)=a\frac{{\theta }^a}{x^{a+1}},x>\theta .$$ In the case of $a>1$ the expectation of a Pareto random variable is $$E\xi =\frac{a}{a-1}\theta .$$ If $a>2$ then the variance exists as well and equals to $$Var(\xi )=\frac{a}{(a-1{)}^2(a-2)}{\theta }^2.$$

Problems

1. Show that if $\theta$ follows the Pareto distribution $\theta \sim Pa({\theta }_0,a),$ then the parameters ${\theta }_0$ and $a$ can be estimated as follows: $${\hat{\theta }}_0={\vartheta }_{(1)}=min({\vartheta }_1,\cdots ,{\vartheta }_m),\hat{a}=\frac{m}{\sum _{i=1}^m\ln \frac{{\vartheta }_i}{{\vartheta }_{(1)}}},$$ where ${\vartheta }^m=({\vartheta }_1,\cdots ,{\vartheta }_m)$ is an i.i.d. sample from $Pa({\theta }_0,a).$

2. Suppose that an i.i.d. sample is observed $$X^n=(X_1,\cdots ,X_n),X_i\sim F(x,\theta ),\theta \in \mathrm{\Theta },x\in R.$$ Consider an estimator for $\theta$ based on the sample $X^n$ $${\hat{\theta }}_n(X^n)={\hat{\theta }}_n(X_1,\cdots ,X_n).$$ Mean Squared Error (MSE, $L(\theta ,\hat{\theta })$) for this estimator is defined as $$E_{\theta }({\hat{\theta }}_n-\theta {)}^2={\int }_{R^n}({\hat{\theta }}_n(x^n)-\theta {)}^{2}dF(x_1,\theta )\cdots dF(x_n,\theta ).$$ If the prior distribution of the unknown parameter $\theta$ is given $\theta \sim \pi (\theta ),\theta \in \mathrm{\Theta },$ then the Bayesian risk of the estimator ${\hat{\theta }}_n$ is defined as $$E_{\pi }L(\theta ,\hat{\theta })={\int }_{\mathrm{\Theta }}{E}_{\theta }({\hat{\theta }}_n-\theta {)}^{2}d\pi (\theta ).$$ Show that the Bayes estimator, defined as the one which minimizes the Bayesian risk, has the form $${\hat{\theta }}_n^B={\arg }_{{\hat{\theta }}_n}\underset{\theta \in \mathrm{\Theta }}{min}{E}_{\pi }L(\theta ,\hat{\theta })={\int }_{\mathrm{\Theta }}\theta f(\theta |x)d\theta ,$$ where $f(\theta |x)$ is the posterior distribution of $\theta .$

3. Suppose that $X_i\sim \mathbb{U}(0,\theta )$ and $\theta \sim Pa({\theta }_0,a)$. Show that the Bayes estimator has the form

$${\hat{\theta }}_n^B=\frac{a+n}{a+n-1}max({\theta }_0,X_1,\cdots ,X_n).$$