The Generalized Likelihood Ratio Test (GRLT) can be used when there is no intuitive test statistic and the [[uniformly most powerful test]] does not exist. However, calculating the GLRT is often also itself intractable. [[Wilks' Theorem]] allows for approximations of the GLRT for large sample sizes. Suppose that $X_1, X_2, \dots, X_n$ is a random sample from a distribution with [[probability density function|pdf]] $f(x; \theta)$. Let $\Theta$ be the parameter space. Assume that the parameter is one-dimensional. Consider testing the null hypothesis that the parameter $\theta$ is in some subset of the parameter space $\Theta_0$ versus the alternate that $\theta$ is not in that same subset of the parameter space. $\begin{align} H_0: \theta \in \Theta_0 && H_1: \theta \in \Theta \backslash \Theta_0 \end{align}$ > [!NOTE] Notation > $\theta \in \Theta \backslash \Theta_0$ is "set minus" notation that represents the complement of the set $\Theta_0$ in the parameter space $\Theta$. Let $\hat \theta$ be the [[maximum likelihood estimator]] (MLE) for $\theta$. Let $\hat \theta_0$ be the restricted MLE, which means that $\hat \theta_0$ is the MLE for $\theta$ in the parameter space where $H_0$ is true. Let $L(\theta)$ be a likelihood function. The generalized likelihood ratio (GLR) is a function of the data (the vector of $xs) given by the ratio of the likelihood function with the restricted MLE plugged in over the likelihood ratio with the unrestricted MLE plugged in. $\lambda(\vec X) = \frac{L(\hat \theta_0)}{L(\hat \theta)}$ The GRLT says to reject $H_0$, in favor of $H_1$, if $\lambda(\vec X) \le c$ # Example Suppose that $X_1, X_2, \dots, X_n$ is a random sample from the continuous [[Pareto distribution]] with pdf $f(x; \gamma) = \frac{\gamma}{(1+x)^{\gamma + 1}}$ for $x>0$ and $\gamma > 0$. Find the GRLT of size $\alpha$ for $\begin{align}H_0: \gamma = \gamma_0 && H_1: \gamma \ne \gamma_0 \end{align}$ A likelihood is $L(\gamma) = \frac{\gamma^n}{\Big[ \prod_{i=1}^n (1 + x_i) \Big ] ^ {\gamma+1}}$ The MLE for $\gamma$ is $\hat \gamma = \frac{n}{\sum_{i=1}^n ln\ (1+X_i)}$ The restricted MLE of $\gamma$ is simply $\hat \gamma_0 = \gamma_0$ The GLR is $\lambda(\vec X) = \frac{L(\hat \gamma_0)}{L(\hat \gamma)}$ The form of the test is to reject $H_0$, in favor of $H_1$, if $\lambda(\vec X) \le c$, where $c$ is determined by solving $\alpha = P(\lambda(\vec X) \le c; \gamma_0)$ In general, the goal would be to plug in the MLE and restricted MLE to the GLR then simplify by getting the $xs together and moving everything else over into the constant term $c$. If that is not possible use [[Wilks' Theorem]] to approximate the GLRT. Because the GLRT is calculated from the data, use [[R]] to conduct the calculations once the form of the test is established from the GLR. For the purposes of demonstration, let's draw a random sample from the Pareto distribution and assume that $\gamma_0=1.8$, $n=1000$ and $\alpha=0.05$. A potential issue in R is [[underflow error]], where the computed numbers are so small they are treated as zero. To avoid this, we will start by further simplifying the GLR to $\lambda(\vec X) =\Big ( \frac{\hat \gamma_0}{\hat \gamma} \Big ) ^n \Big [ \prod_{i=1}^n (1 + x_i) \Big ]^{\hat \gamma - \hat \gamma_0} $ ```R # Generate random sample from the Pareto distribution using the inverse cdf method set.seed(0) n <- 1000 gamma <- 1.8 u <- runif(n) mysample <- (1-u)**(-1/gamma)-1 # Specify the MLE and restricted MLE from the example above mle <- n/sum(log(1 + mysample)) mle0 <- gamma # Calculate the test statistic in Wilks' Theorem using the simplified GLR glr <- ((mle0/mle)**n) * (prod((1 + mysample)**(mle-mle0))) # Compute test statistic teststat <- (-2 * log(glr)) # Reject the null hypothesis if teststat > chisq(1- 0.05, 1) reject <- teststat > qchisq(1 - 0.05, 1) reject > FALSE ```