Shrinkage estimators in inverse Gaussian regression model: Subject review.

In this paper, we presented a thorough review of literature regarding the biased estimators in inverse Gaussian regression model when the multicollinearity is existing. According to real data application, the two-parameter estimator has better performance than IGRM, IGRR, IGLE, and IGLT, in terms of MSE. In conclusion, the use of the two-parameter estimator is recommended when multicollinearity is present in the inverse Gaussians regression model.

1. Introduction

The inverse Gaussian regression model (IGRM) has been widely used in industrial engineering, life testing, reliability, marketing, and social sciences [1-7]. “Specifically, IGRM is used when the response variable under the study is positively skewed [8-10]. When the response variable is extremely skewness, the IGRM is preferable than gamma regression model [11]. In dealing with the IGRM, it is assumed that there is no correlation among the explanatory variables [12-32]. In practice, however, this assumption often not holds, which leads to the problem of multicollinearity. In the presence of multicollinearity, when estimating the regression coefficients for IGRM using the maximum likelihood (ML) method, the estimated coefficients are usually become unstable with a high variance, and therefore low statistical significance [33]. Numerous remedial methods have been proposed to overcome the problem of multicollinearity [34-38]. The ridge regression method [39] has been consistently demonstrated to be an attractive and alternative to the ML estimation method.

Ridge regression is a biased method that shrinks all regression coefficients toward zero to reduce the large variance [40]. This done by adding a positive amount to the diagonal of . As a result, the ridge estimator is biased but it guaranties a smaller mean squared error than the ML estimator. 

In linear regression, the ridge estimator is defined as

where  is an  vector of observations of the response variable,  is an  known design matrix of explanatory variables,  is a  vector of unknown regression coefficients,  is the identity matrix with dimension , and  represents the ridge parameter (shrinkage parameter). The ridge parameter, , controls the shrinkage of  toward zero. The OLS estimator can be considered as a special estimator from Eq. (1) with . For larger value of , the  estimator yields greater shrinkage approaching zero [39, 41].

1. Inverse Gaussian regression model

The inverse Gaussian distribution is a continuous distribution with two positive parameters: location parameter, , and scale parameter, , denoted as . Its probability density function is defined as

The mean and variance of this distribution are, respectively,  and .

Inverse Gaussian regression model is considered a member of the generalized linear models (GLM) family, extending the ideas of linear regression to the situation where the response variable is following the inverse Gaussian distribution. Following the GLM methodology, Eq. (1) can re-write in terms of exponential family function as

where  and  . Here,  represents the dispersion parameter and  represents the canonical link function.

 In GLM, a monotonic and differentiable link function connects the mean of the response variable with the linear predictor , where  is the i^th row of  and  is a  vector of unknown regression coefficients. Because depends on  and the mean of the response variable is a function of , then . Related to the IGR, the . Another possible link function for the IGRM is log link function,  .

The model estimation of the IGRM is based on the maximum likelihood method (ML). The log likelihood function of the IGRM under the canonical link function is defined as

The ML estimator is then obtained by computing the first derivative of the Eq. (3) and setting it equal to zero, as

Unfortunately, the first derivative cannot be solved analytically because Eq. (4) is nonlinear in . The iteratively weighted least squares (IWLS) algorithm or Fisher-scoring algorithm can be used to obtain the ML estimators of the IGRM parameters. In each iteration, the parameters are updated by

where  and  are  and evaluated at , respectively. The final step of the estimated coefficients is defined as

where , ,  is a vector where i^th element equals to , and . The covariance matrix of  equals

and the MSE equals

where  is the eigenvalue of the  matrix and the dispersion parameter, , is estimated by [42]

1. Ridge estimator

In the presence of multicollinearity, the matrix  becomes ill-conditioned leading to high variance and instability of the ML estimator of the IGRM parameters. As a remedy, Månsson and Shukur [43] proposed the IGR ridge estimator (IGRR) as

where . The ML estimator can be considered as a special estimator from Eq. (11) with . Regardless of value, the MSE of the is smaller than that of because the MSE of  is equal to [33]

 where  is defined as the j^th element of and  is the eigenvector of the  matrix. Comparing with the MSE of Eq. (9),  is always small for .

1. Liu estimator

Another popular biased estimator which is known as Liu estimator has been adopted in Poisson regression model. The inverse Gaussian Liu estimator (IGLE) is defined as

where .  Regardless of value, the MSE of the is smaller than that of because the MSE of  is equal to [33]

1. Liu-type estimator

Alternative to Liu estimator, the Liu-type estimator was proposed by Liu [44] to overcome the problem of severe multicollinearity. The inverse Gaussian Liu-type estimator (IGLT) is defined as

where  and .  In Eq. (15), the parameter can be used totally to control the conditioning of . After the reduction of  is reach a desirable level, then the expected bias that is generated can be corrected with the so-called bias correction parameter,  [45-49].

Liu [44] proved that, in terms of MSE, the Liu-type estimator has superior properties over ridge estimator. The MSE of   is defined as

1. Two-parameter estimator

Following Asar and Genç [50] and Huang and Yang [51] the two-parameter estimator in linear regression model is defined as:

where  and .  For IGRM, the two-parameter estimator (IGTP) is defined as:

It is obviously noted that the  is a combination of two different estimators IGRR and IGLE. Furthermore, if , Eq. (18) will be the  while if , Eq. (18) will be the . Besides, when , then Eq. (18) will equal .

In terms of MSE, the two-parameter estimator has superior properties over ML estimator. The MSE of   is defined as

1. Real application

To demonstrate the usefulness of the shrinkage estimators in real application, we present here a chemistry dataset with , where  represents the number of imidazo[4,5-b] pyridine derivatives, which are used as anticancer compounds. While  denotes the number of molecular descriptors, which are treated as explanatory variables [52]. The response of interest is the biological activities (IC₅₀). Quantitative structure-activity relationship (QSAR) study has become a great deal of importance in chemometrics. The principle of QSAR is to model several biological activities over a collection of chemical compounds in terms of their structural properties [53]. Consequently, using of regression model is one of the most important tools for constructing the QSAR model.

First, to check whether the response variable belongs to the inverse Gaussian distribution, a Chi-square test is used. The result of the test equals to 5.2762 with p-value equals to 0.2601. It is indicated form this result that the inverse Gaussian distribution fits very well to this response variable. That is, the following model is set

Second, to check whether there is a relationship among the explanatory variables or not, Figure 1 displays the correlation matrix among the 15 explanatory variables. It is obviously seen that there are correlations greater than 0.90 among MW, SpMaxA_D, and ATS8v ( ), between SpMax3_Bh(s) and ATS8v ( ), and between Mor21v with Mor21e ( ).

Third, to test the existence of multicollinearity after fitting the inverse Gaussian regression model using log link function and the estimated dispersion parameter is 0.00103, the eigenvalues of the matrix  are obtained as , , , , , , , , , , , , , , and . The determined condition number  of the data is 40383.035 indicating that the severe multicollinearity issue is exist.

The estimated inverse Gaussian regression coefficients and the estimated theoretical MSE values for the MLE, and the used estimators are listed in Table 1”. According to Table 1, it is clearly seen that the IGTP has MSE values less than the MSE of the IGRM, in general. Moreover, the MSE of the IGTP estimator is the lowest among all estimators. Specifically, it can be seen that the MSE of IGTP estimator was about 44.24%, 39.17%, 32.62%, and 12.11% lower than that of IGRM, IGRR, IGLE, and IGLT, respectively.

Figure 1. Correlation matrix among the 15 explanatory variables of the real data.

Table 1: The estimated coefficients and MSE values of the used estimators

1. Conclusions

In this paper, we presented a thorough review of literature regarding the biased estimators in inverse Gaussian regression model when the multicollinearity is existing. According to real data application, the two-parameter estimator has better performance than IGRM, IGRR, IGLE, and IGLT, in terms of MSE. In conclusion, the use of the two-parameter estimator is recommended when multicollinearity is present in the inverse Gaussians regression model.