Properties of the coefﬁcient estimators for the linear regression model with heteroskedastic error term

. In this paperwe presentestimated generalizedleast squares(EGLS) estimatorfor the coefﬁcient vector β in the linear regression model y = βX + ε , where disturbance term can be heteroskedastic. For the heteroskedasticity of the changed segment type, using Monte-Carlo method, we investigate empirical propertiesof the proposedand ordinaryleast squares(OLS) estimators.The resultsshow thatthe empirical covariance of the EGLS estimatorsis smaller than that of OLS estimators.


Regression model coefficient estimators
Classical results in the econometric theory show that if coefficients of the linear regression model are estimated by the OLS method when error term is not homoskedastic, an estimator is obtained which, although remains unbiased, consistent and asymptotically normal, is no longer minimum variance unbiased estimator (MVUE).Thus diagnostic testing for heteroskedasticity has to be undertaken before further analysis starts.If the null hypothesis of homoskedasticity is rejected, a generalized estimator should be found.
We consider a linear regression model where β is a coefficient vector of length d and f : [0, 1] → R d is a given function, disturbances are ε j = g(j/n)u j , with i.i.d.random variables u j , Eu 1 = 0, Eu 2 1 = 1, and the function g: [0, 1] → R; T denotes transposition operation.The null hypothesis H 0 is specified by g(j/n) ≡ σ 2 .All other choices of g lead to heteroskedastic alternatives. Set And denote y = (y 1 , . . ., y n ) T .Let us assume now that the alternative hypothesis is true, i.e., Eε 2 j = g 2 (j/n) = σ 2 for some j = 1, . . ., n.It is well known (see [1], for example) that regression coefficients estimated by OLS method, β = (X T X) −1 X T y, (1) though are unbiased, have the covariance matrix which is no longer σ 2 (X T X) −1 .If we denote covariance matrix of ε = (ε 1 , . . ., ε n ) by we obtain generalized least squares (GLS) estimator Even more if we replace G by some of its estimator G we obtain estimated GLS estimator It is also known that β G and β EG under regularity conditions possess at least asymptotically consistency, normality and are more efficient than β.
In [2], for this type of alternative a class of test statistics is presented, which are obtained by taking a certain functional (defined in a Hölder space) of the polygonal line process constructed the partial sums of square residuals of the model.Without going much into detail assume that the residuals ε = ( ε 1 , . . ., ε n ) T are defined by with OLS estimator β defined in (1).For 0 α < 1/2, we introduce a class of test statistics Let (W t , t ∈ [0, 1]) be a standard Wiener process and let . The class of limiting test statistics then is (5) In this paper we will rely on theoretical results (two theorems) stated and proved in [2].The first theorem shows that a class of test statistics T n, α converges in distribution to T α and the second shows that this class is consistent for the heteroskedasticity alternatives of various types.Here as a special case we will deal only with the changed segment type alternative.
Define sample variance of the squared residuals Assume that H 0 holds and the function f is continuous and has finite p variation.Then if and only if The same is true for α = 0 provided Eε 4 1 < ∞.
THEOREM 2. Assume that H 1 is true and Eu 4 1 is finite.Also assume that σ is fixed and τ Under H 1 , if * , k * , τ and σ are known then G, as in (2), is a diagonal matrix with elements equal τ 2 when j ∈ {k * + 1, . . ., k * + * }, and the rest of elements equal σ 2 .It is natural approach to estimate the unknown G by taking estimates * , k * , τ and σ instead of the corresponding parameters.This approach leads to the estimated GLS estimator of β defined in (4), if the null hypothesis of the homoskedasticity is rejected.
Unfortunately we can not yet provide the reader with the results showing that these estimators converge to their true values, nevertheless we can give empirical results to justify our reasoning.

Monte Carlo experiment
Set d = 2 and f (t) = 1 + t, i.e., f 1 (i/n) ≡ 1 and f 2 (i/n) = i/n, i = 1, . . ., n and analyze a case with n = 128.We take u 1 ∼ N(0,1).Under the null hypothesis we fix σ = 1.0 and for the changed segment type of alternative we generate a changed segment of length * = 16 which begins at k * = 56 and is of size τ = 2.0.
In this experiment we are interested in the properties of OLS estimator β, GLS estimator β G and EGLS estimator β EG defined in ( 1)-(4).Knowing the configuration of the experiment we can compute true values of the parameters of interest.For example, E β = E β G = E β EG = (1, 1) T both under the null hypothesis and under the alternative.Let us denote covariance matrices of the estimators by V( 0.0316 -0.0472 -0.0472 0.0938 , and under the alternative H 1 it is easy to calculate that V( β) = σ * 2 (X T X) −1 , where Hence, V( β) = 0.0435 -0.0650 -0.0650 0.1289 and V( β G ) = 0.0325 -0.0473 -0.0473 0.0939 .
Thus, if we know the values of the change parameters, the GLS estimator reduces the variance.Next we will show that the EGLS estimator also does.
For the set {b 1 , . . ., b R } of realizations of any statistic b, let us denote empirical mean Mb = R −1 R i=1 b i .First for the model without the change we have generated R = 1000 replications of observations.For every replication we have computed β and V( β) and obtained MV( β) = (0.0315, −0.0471, −0.0471, 0.0934) (writing the elements of a matrix in a row).The results obtained are in agreement with the theoretical findings.Next with the changed segment in variance of disturbances we again ran R = 1000 replications.If despite of the presence of heteroskedasticity we use OLS estimator we obtain MV( β) = (0.0436, −0.0651, −0.0651, 0.1292), just according to the theory.Then, as was proposed, we test for a changed segment and, if H 0 is rejected, find β EG and V( β EG ).We deffer the results one paragraph ahead and first add one remark.
with normalization m −1/2 and Z i ∼ N(0,1), i = 1, . . ., m. Taking a large value of m for small sample sizes we obtain significantly biased approximation for the null distribution of the test statistics (especially for α close to 1/2).As a consequence wrong critical regions for the test statistics T n, α are obtained.In [2] the procedure is proposed to fix this problem.First we propose to take m = n.Second we suggest taking Z 2 i (and change normalization to (2m) −1/2 ) instead of Z i as for small sample sizes we can not expect CLT to work well.We use this procedure with α = i/16, i = 0, . . ., 7 and the significance level = 0.05.As a result we obtain the test p-values ranging from 0.046 to 0.053, i.e., close to correct.
In Table 1 the results of experiment under H 1 are presented.
For the significance level = 0.05, according to the critical regions obtained after the proposed procedure, for every α we compute the number when H 0 was rejected out of total number of replications R = 1000.In other words, we find empirical power (denoted pw in Table 3).The row indicated by β i represents sample mean of estimators for the variance of β i , i = 1, 2. And β 1 β 2 denotes sample mean of estimators for the covariance between β 1 and β 2 .Note however, that we have computed the sample means only from the realizations of estimators, when H 0 was rejected.Also note that all these estimators depend on α.We see that estimating β by β EG gives the most accurate results when α = 5/16.This confirms the authors' findings in other papers that detection of changed segments of various lengths crucially depends on the value of α.

Table 1 .
Empirical power of T n, α and mean of V( β EG )