Improving generalized ridge estimator for the gamma regression model.

In this study, a generalized ridge estimator was suggested as a solution to the gamma regression model's multicollinearity issue. The K matrix has been estimated using a variety of techniques. According to Monte Carlo simulation tests, the GGRRM estimator performs better than MLE and GRRM in terms of MSE regardless of the kind of estimating method used for thematrix. To further demonstrate the advantages of utilizing the GGRRM estimator in the context of gamma regression models, a real data application is also taken into consideration. It was determined that the GGRRM estimator is superior based on the resultant MSE, and it was further demonstrated that the outcomes are compatible with those of Monte Carlo simulations

Introduction

     The appearance of puberty in the female Many real data problems, such as automobile insurance claims, healthcare economics, and medical science, can be studied using the Gamma Regression Model (GRM) (1, 2, 3). A GRM is used particularly when a study's response variable is positively skewed or not normally distributed. As a result, gamma regression requires gamma distributions for the response variables (4, 5, 6).

The GRM presumes that there is no correlation between the regressors. However, in reality, this presumption frequently fails, which creates the multicollinearity issue. In the presence of multicollinearity, gamma regression coefficients are typically unstable with a large variance and poor statistical significance when estimated using the maximum likelihood (ML) approach (7, 8).  To solve the multicollinearity issue, many solutions have been presented out. It has been frequently shown that the ridge regression approach (9) is a desirable replacement for the ML estimation method.

The following relationship is typically used in classical linear regression models: 

where  is an  vector of response variable observations,  is a known design matrix of explanatory variables   is a  vector of unknown regression coefficients, and  is an  vector of random errors with mean 0 and variance . 

In order to decrease the high variance, the ridge regression shrinkage approach compresses all regression coefficients in the direction of zero (7, 10). The diagonal of  is raised in a positive direction to achieve this. The ridge estimator has a lower mean squared error than the ML estimator due to its bias.

The ridge estimator in linear regression is defined as:

With I as the identity matrix of size  and   as the ridge parameter (shrinkage parameter) which controls the shrinkage of  toward zero. A larger value of  yields greater shrinkage for the  estimator (9).

Statistical Methodology

Gamma Ridge Regression Model (GRRM)

       There are often positive skewed data used in studies in sociology, economics, and epidemiology, these kinds of data do not have any negative numbers, making the gamma distribution an ideal choice for these kinds of data (5). If  be the response variable and has a gamma distribution with nonnegative shape parameter  and nonnegative scale parameter , i.e. , then the probability density function is defined as  (6, 11):

with . When the parameter is known, it is shown that the response variable's variance is proportional to the square of its mean.

In a GRM,  is expressed as a linear combination of repressors . The  is called the log link function is what gives the relationship between the predictors and the response variable its linear shape. This log like function is alternatively used rather than the canonical link function (reciprocal link function, ) because it ensures that  

Using the Maximum likelihood technique is the most typical way to estimate the GRM coefficients. Considering that the observations are presumed to be independent and  the log-likelihood function is given by:

The first derivative of Eq. (4) is then calculated and set to zero to get the ML estimator, as:

      Unfortunately, the first derivative cannot be analytically calculated since Eq. (5) is nonlinear. The ML estimators of the gamma regression parameters may be obtained using either the iteratively weighted least squares (IWLS) technique or Newton-Raphson approach. In each iteration, the parameters are updated by:

Where . The estimated coefficients final step is defined as

              Where  and  is a vector where  element equals to . ML estimators are normally distributed with covariance matrices that are inverses of Hessian matrices.

Eq. (7)'s mean squared error (MSE) can be calculated as follows:

where  is the eigenvalue of the  matrix. The matrix  becomes ill-conditioned in the presence of multicollinearity, the ML estimator of the gamma regression parameters becomes unstable and has an excessive amount of variation. As a remedy, the gamma ridge regression model (GRRM) can be described as:

where . A specific estimator from Eq. (10) with  might be thought of as the ML estimator.

Generalized ridge estimator

The generalized ridge estimator (GRE), differs from the generalized ridge regression model (GRR) in that it takes values of into account (9).

where. Finding the optimal values of  while using GRE is advantageous because the MSE is smaller than when the ridge estimator and OLS are used.

The definition of the GRE for the gamma regression model (GRM) is:

 matrix selection must be carefully considered. Several approaches are modified to estimate  in this study, including (9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22). These approaches are listed below, in order.

where  is defined as the  element of  and  is the eigenvector of  the  and the dispersion parameter, , is estimated by

Modeling and simulation

      With the help of Monte Carlo simulations, the effectiveness of these approaches is examined using the GGRRM and different levels of multicollinearity.

The design of simulations

      The GRM's response variable for n observations is produced by (8, 11, 23, 24, 25, 26, 27, 28). where   with  and [29], and . Explanatory variables  have been generated from the following formula:

where  represents the correlation between explanatory variables and ’s are independent standard normal pseudo-random numbers. Three exemplary sample size values 50, 100, and 200 are taken into consideration since the sample size directly influences prediction accuracy. Additionally, the number of explanatory factors is taken into account as  and  because doing so might result in an increase in the MSE. Further, three values of the pairwise correlation are taken into consideration with  since we are interested in the influence of multicollinearity, in which the degrees of correlation are deemed more essential. The produced data is repeated 1,000 times for a combination of these various values of , and , the averaged mean squared errors (MSE) is determined as follows:

The results of simulations

There are six tables showing the averaged MSE for the combinations of , and . Throughout the table, the best MSE value is highlighted to emphasize its importance. The following are some possible observations: 

• GRRM frequently has a lower MSE than MLE.
• GGRRM achieved less MSE than GRRM, regardless of the estimating method of the matrix K.
• A comparison of the F method with other approaches revealed that the gamma generalized ridge estimator was significantly enhanced by Firinguetti (15) in Eq. (16)). HK and SB procedures consistently produced inadequate results, when compared with other approaches tested.
• MSE values increase as the degree of correlation increases with respect to p, regardless of the values of  and .
• In terms of the number of explanatory variables, it is easy to see that there is a negative impact on MSE, where their values rise as p increases.
• The MSE values decrease with increasing n, regardless of the values of , or
• As v increases, the MSE of all methods decreases for fixed n, p, and degree of multicollinearity.

Table 1: Average MSE values when  and

Table 2: Average MSE values when  and

Table 3: Average MSE values when  and

Table 4: Average MSE values when  and

Table 5: Average MSE values when  and

Table 6: Average MSE values when  and

Application of real data

 Here, we offer a chemical dataset with  where n denotes the quantity of antifungal drugs, to illustrate the applicability of the GGRRM estimator in practical applications. pMIC (the logarithm of reciprocal of MIC, where MIC is the lowest inhibitory concentration against C. albicans in mM/L) was used to quantify the antibacterial activity. As explanatory variables, molecular descriptors are represented by the integer (29, 30). In chemometrics, the quantitative structure-activity relationship (QSAR) investigation has gained significant attention. The fundamental idea behind QSAR is to simulate various biological functions across a group of chemical substances in terms of their structural characteristics. Regression modeling is therefore one of the most crucial techniques for building the QSAR model. Table 7 lists the explanatory variables that were employed. Every variable is a number.

The Chi-square test is performed first to determine if the answer variable is part of the gamma distribution. The test yielded a result of 10.0286 and a p-value of 0.9117. The gamma distribution closely matches this response variable, with an estimated dispersion parameter of 0.0153. Using the predicted dispersion parameter of 0.0153 and log link function to construct the gamma regression model, the test for multicollinearity, the eigenvalues of the matrix  are obtained as: The determined condition number   of the data is 35422.83 demonstrating the existence of the serious multicollinearity problem.

Table 8 lists the estimated MSE values for the MLE, GRRM, and GGRRM estimators using various estimating matrices. Table 8 makes it abundantly evident that the F approach effectively reduces the value of the calculated coefficients. The MSE has also been considerably decreased in favor of the F method. It is clear that the MSE of the F technique was around 64.97%, 60.63%, 59.33%, 47.98%, 44.34%, 46.79%, 45.37%, 42.43%, 48.26%, 65.96%, 44.34%, 44.47%, and 45.49% lower than that of MLE, GRRM, HK, N, TC, HSL, AH, D, SB, SV1, SV2, M, and AS estimators, respectively.

Table 7: Description of the used explanatory variables

Table 8: The estimated MSE values for the real data application