Analysis of Two Populations Dichotomous Data in Latent Variable Models Using Bayesian Approach

In the social and behavioural sciences, two populations nonlinear models with nonlinear effects, nonlinear covariates, and latent variables are quite prevalent. The initial goal of this investigation was to acquire all the estimated parameters using nonlinear LVMs for two populations, nonlinear covariates, and latent variables. The second goal is to use hidden continuous normal distribution (censored normal distribution and truncated normal distribution) to solve the problem of dichotomous variables and to use hidden continuous normal distribution (truncated normal distribution with known parameters) to solve the problem of dichotomous covariates. Two different thresholds (with equally and unequally spaced categories) were used in the suggested techniques. This presumption, nevertheless, is likely to be broken in many real world scenarios. Future study may focus on creating a nonparametric Bayesian technique to loosen the normality assumption in nonlinear NLVMs for two populations with missing data.

There are LVM cases where nonlinear elements of latent variables are included into equations. Bollen and Paxton (1998), Schumacker and Marcoulides (1998), and others have noted that the lack of applications is not attributable to the validity of the substantive arguments that suggest the existence of nonlinearity, but rather to the technical difficulty and lack of understanding of the current statistical methods. In this study, a Bayesian technique is suggested for analysing nonlinear models including two populations and dichotomous variables. Due to the complexity of the proposed model, we also offer statistical methods to estimate standard deviations and model comparison using the Deviance Information Criterion (DIC).  As we've shown, using various MCMC methods to supplement the data can help with issues brought on by the nonlinear causal links between the latent factors and the discrete nature dichotomous data manifest variables. More specifically, the fundamental concept of our work was motivated by adopting a popular strategy from recent statistical computing work (see Rubin, 1991) that formulated the underlying difficult problem in order that, when supplementing the actual observed data with the fictitious missing data, the analysis would be fairly simple with the full data. This approach is quite effective and may be used with other, more intricate models.

Introduction

Latent variable models (LVMs) are a statistical method for modeling a series of correlated data in order to assess the correlations between manifest and latent variables (Bollen and Paxton, 1998; Lee, 2007).

The Bayesian method starts by defining a prior distribution for the parameters that need to be estimated. Without referencing the dataset used to estimate the model, the prior represents the researcher's knowledge (Lee, 2007).

The rapid growth of LVMs is a result of the demand for complex models and the accompanying statistical approaches for tackling difficult research problems in a range of fields. The Bayesian technique is built using the Gibbs sampler algorithm (Geman and Geman, 1984), where latent variables in multiple populations and concealed continuous normal measurements are regarded as hypothetical missing data. Conjugate priors are employed for the structural parameters whereas non-informative priors are used for the thresholds (cut points with equally and unequally spaced distances). The primary objective of this study is to present a Bayesian approach for the analysis of two populations nonlinear LVMs with dichotomous variables and covariates.

Many academics have suggested models in recent years that incorporate nonlinear relationships between the manifest, and latent variables. Several of these publications were suggested by Lee and Song (2003), Lee and Song (2005), Lee (2006), Lee and Tang, (2006(, Cai et al. (2008(, Lee et al. (2009), Lee et al. (2010).

A specific methodology for using the Bayesian approach in factor analysis is provided by Song and Lee (2002). They developed an analytical model that uses joint Bayesian estimations for the component scores and structural parameters in relation to the established restrictions, enabling the simultaneous determination of numerous findings. This system has been shown to be effective in producing calculations of these estimations because it combines the Gibbs model and Metropolis-Hastings algorithm.

The maximum likelihood method was applied by Song and Lee (2006) to multi-sample nonlinear structural equation models with missing continuous and dichotomous data.

A Bayesian nonlinear structural equation model was created by Song and Lee (2006) using linear fixed covariate and latent variables in the measurement model and nonlinear fixed covariate and latent variables in the structural model. Mixed continuous and dichotomous data are used in this study, and a concealed continuous normal distribution (a truncated normal with unknown parameters) is presented to overcome the dichotomous data problem. In order to solve the issue,

Lee (2007) used an underlying latent continuous normal distribution (a truncated normal distribution with unknown parameters) in Bayesian multi-sample nonlinear structural equation models with dichotomous variables. Additionally, the Gibbs sampling method was employed to estimate the parameter. The ordered categorical variables were handled as a continuous normal distribution in Lu et al. (2012) Bayesian study of multi-sample nonlinear structural equation models with application to behavioural finance. The multi-sample analytic method is essential in many applications, such as cross-cultural research. Nonlinear effects, such as quadratic and interaction effects between the covariates and latent variables, are frequently essential for constructing the main theory.

The document has the following structure. The model is described in Section 2 of the document.  The Bayesian analysis is described in Section 3. The model comparison using DIC is described in Section 4. A case study may be found in Section 5. Section 6 summarizes the results and the discussion, while Section 7 offers conclusions and recommendations.

2. Model Description

The suggested latent variable model for this case include both latent variables and linear covariates in the measurement equation. The structural model also includes latent nonlinear variables and nonlinear covariates. This LVM is taken into account.

where  is a manifest variable with dichotomous data that has been established as a random vector, is a vector of linear covariates, and    is a vector of dichotomous covariates, since is a matrix with unknown parameters, it is frequently referred to as the factor loading matrix. A random vector of latent variables is shown in   , while a random vector of residuals is shown in .

This leads to the assumption that the outcome of  is independent, and that  is then distributed independently as . Additionally,  has an independent distribution as , where  is a diagonal matrix with  as its diagonal components.

Furthermore, it has been found that and  in this instance are both independent. A latent vector, , is separated into , where vectors  and are both present, in order to implement more complex mathematical situations.

The vectors of the exogenous latent variables are and the endogenous latent variables are , respectively.

The vector of dichotomous variables on is used to estimate the probable significant causal impact of . But if is non-normal, then must likewise be non-normal.

The following latent variable model gives the definition of the structural equation:

A matrix of unknown parameters is represented by , a vector-valued function with differentiable functions is represented by   , and an unknown parameter matrix is represented by , and . For a simple expression, (2) can be written as:

where , , and are error measurement vectors.

It is necessary to first suppose that is distributed as , then that is distributed as , and that  is a representation of a diagonal matrix with the entries and for which and  are independent of each other.

An illustration of two populations LVM defined in (2) that are connected to

and  is:

Here,  

and

where g = 1; 2. Further, and  are both quadratic terms of elements. As , may be drawn from the arbitrary distributions for covariates that are dichotomous data.

Furthermore, let and to stand for the k^th row for each A and .

So, let to be a partition of which corresponds with , which is also a partition. If follows that and , it follows from (1) that

But when employed in a practical application scenario,  is typically not complex, and as a result, it can be anticipated that  will likewise be very straightforward, making it easy to calculate .

It is also beneficial to investigate this indirect method for modeling covariates, similar to those illustrated above, by first adding with , and then by managing each component of the latter as if it were an exogenous latent variable that could be measured accurately using a single indicator.

 At most basic level, a dichotomous variable can be defined according to its underlying latent continuous random variable by:

In order that the set of threshold values that define the specified categories are also true, and the number of categories for the dichotomous variable is represented by .

We will utilize hidden continuous normal distribution (a truncated normal distribution with known parameters) to solve the issue of dichotomous data in covariates . Thus, it follows:

However, it should be noted that the number of thresholds (cut points) for each group is equal for every dichotomous variable. However, we employ both equally and unequally categorized distances as our criteria.

3. The Bayesian Analysis

Let serve as an unknown parameter vector in the previously mentioned model, and let serve as an unknown threshold vector for the dichotomous variables that belong to the g^th group.

This was chosen because a study of several populations typically identifies a certain type of parameter in as an invariant within group models. The following limitations, for instance, apply to limits on cut points:

The thresholds on the model are typically implemented as  and/or . Consequently, we may accept some common parameters while evaluating the data, . Allow  to be a vector that contains all of the unknown separate parameters , and allow to be a vector that contains all of the unknown thresholds.

The Gibbs sampler is used to create the Bayesian estimate of  and .

Let the dichotomous data that were observed be and . Let and Z, be the latent continuous measurements in and , respectively.

After that, add Y to the observed data in the posterior analysis. The problem will be easier to handle after Y has been defined since all the data is taken into consideration and is deemed continuous. Furthermore, assume that and represent the latent variable matrix.

Significantly reduced complications occur from the nonlinear connections between the latent variables. As a result, by enhancing the data, problems related to the model's more intricate components can be resolved

Through posterior analysis,  may be added to (Z), which stands for the collection of observed data.

We will also show how the joint posterior distribution looks. The Geman and Geman (1984) Gibbs sampler may be used to generate a set of observations from the associated joint posterior distribution.

As a consequence, using the created sample of data, a number of conventional inferences may be used to determine the Bayesian answer. Additionally, we may build a collection of sample observations from these conditional distributions ,  and using the Gibbs sampler and the iteration technique.

We may establish the non-informative prior to calculating in a method similar to how previous cut point issues were solved, so that the corresponding prior distribution is comparable to the constant.

According to the different group models, the conditional distribution  can also be divided into several parts that each comprise a variety of structural parameters.

Some examples of competing models are:

      No constraints            

As various theories are used or as competing approaches are explored, these components change. When placed under the various definitions of M_k as described above, the components of the conditional distribution, known as , and the condition applied to preceding distributions are considerably different, or variable.

The basic assumption is that the previous distributions for the unconstrained parameters would behave independently for each group. It is also necessary to determine the data that belongs in each group and to supply an accurate value of the prior distribution when creating an estimate for the unconstrained parameters in order for them to be fully implemented.

The Bayesian estimates and model comparison in the setting of two populations nonlinear LVMs with dichotomous variables are covered in this section. To complete the plan, the idea of data augmentation is merged with MCMC tools. Two populations nonlinear LVMs are theoretically a particular case of the two-level SEM, and the output may be utilized to produce different conditional distributions that are required by the Gibbs sampler.

Since there are clear restrictions on the parameters in different groups that must be satisfied, more attention needs to be paid to identifying the similar previous distributions. To employ the route sampling method for model comparison in two-level SEMs, similar information is required (Lee and Song, 2012).

This section explains how to use the Bayesian technique to examine the prior nonlinear LVMs in the setting of dichotomous variables. This method has various advantages for the whole application, including: (1) When it is included directly, applying past information can improve the total analysis. Particularly, it produces more precise parameter estimate. (2) Various scholars have shown that sampling-based Bayesian approaches are independent of asymptotic theory. (Lee, 2006; Lee and Shi, 2000; Shi and Lee, 2000; Lee et al., 2010; Lee et al., 2007; Lee and Song, 2002; Song et al., 2011; Yang and Dunson, 2010); (3) Both Bayesian and ML estimates feature similar optimal asymptotic properties. Through the posterior analysis, the observed data, as represented by [Z], is enhanced according to the latent data [Y, ]. Allow to represent the observed data set of Dichotomous variables and  to act as the vector having unknown parameters in order to construct the Bayesian method for the suggested LVMs.

By defining in such a way that  is treated as random variable with a prior distribution and prior probability density function, the Bayesian technique would be used to explain the situation. Thus, the related assumptions can be based on the observed data for Z and . So, allow Let represent the joint probability density function of both with reference to different M_k.

Based on a well-known identity in probability, , where  and are conditional probability  density functions. It follows that:

The posterior density function of the unknown parameters is the name given to the function .

The posterior density function , or unknown parameters, is what results from this. Additionally, the probability function  and the prior density function  make use of sample data and previous knowledge.

However, it should be emphasized that  depends on sample size, whereas (is not). Due to its greater similarity to the likelihood function , the posterior density function, is more pertinent for situations involving large samples than , which is less significant.

Therefore, keep in mind that  is important for the Bayesian technique when the sample size is less or when the data obtained from Z contains dichotomous information.

By treating yi as an unobserved variable in this situation, which corresponds to the manifest dichotomous variables as they are found in zi, MCMC techniques are used.

The Bayesian estimate for  and any standard error estimates may be obtained from the sample mean and variance matrices, respectively, if we can extract a sufficient number of observations (represented by ) from the joint posterior distribution defined by , which is then used to construct the joint posterior distribution.

This means that even if establishing the conditional distribution, as explained in Step (1), it is still required to expressly identify the previous distribution for the corresponding components in . The conjugate prior distributions have typically shown to be flexible and appropriate for the task during Bayesian analysis (Broemeling, 1985).

Many Bayesian analyses in structural equation models have used this form of prior distribution (see Lee and Song, 2004; Song and Lee, 2007). Consequently, the popular conjugate prior distributions listed below are employed:         

Given the definition that  is distributed according to,  , which is the k^th diagonal element of  and  are the k^th rows of  and , respectively. and    and  are assumed to be known, as prior information.

The dichotomous variables and covariates in this situation, however, can make the linked conditional distributions too complicated to readily extract or simulate data from them.

This encourages the additional escalation of Y, x the latent matrices, in the posterior analysis, and motivates attention to the joint posterior distribution . To garner observations of this posterior distribution, using the Gibbs sampler, it is essential to begin with the starting values . The following procedure is then implemented to simulate  and so on. More specifically at the m^th reiteration of the current values .

1. Generate from
2.  

• Generate from

3. Generate from                                                         The cycle will only produce after the m^th repeat, according to the earlier definition.

Therefore, it can be demonstrated that the joint distribution of the value of moves in the direction of the joint posterior distribution  as m gets closer to infinity (see Geman and Geman, (1984)).

4. Model Comparison

a measure of model comparability the Akaike Information Criterion (AIC; Akaike, (1973)) is an extension of the DIC (see Spiegelhalter et al., 2002). The DIC is calculated using a competitive model with a vector of unknown parameters as follows:

where measures the goodness of fit of the model, and is defined as

Here, is the effective number of parameters in , and is defined as

in which  is the Bayesian estimate of . Let  be a sample of observations simulated from the posterior distribution. The expectations in Equations (19) and (20) can be estimated as follows:

The model with the lower DIC value is chosen in Bayesian LVMs. We analyzed the same data using two populations of NLVMs using the same measurement model to demonstrate how to use DIC for model comparison. The OpenBUGS application generates the DIC values for two populations of NLVMs using actual data.

5. A Case Study

Let's have a look at the data that may be utilized to derive conclusions for various, independent samples that are chosen from the natural history based on the research of a rural drug discovered in Ohio (n=200) and Kentucky (n=200) in the USA between the years 2003 and 2005 (Booth et al., 2006).

The BSI-18 scale, which examined three categories of mental illnesses and took into account factors including somatization (SOM), depression (DEP), and anxiety (ANX), experienced several more modifications.

There are two covariates in each group of the sixteen variables that make up the data. Additionally, all of them were assessed using the following ordered categorical variables: (1, not at all; 2, a little bit; 3, moderately; 4, quite a bit; 5, extremely) are changed to only two categories to be dichotomous data (Wang & Wang, 2012).

This actual data analysis, in which 16 manifest variables are associated to two fundamental latent variables from two populations nonlinear LVMs defined in Equations 17 and 18, provides some insight into the empirical performance of the suggested Bayesian technique.

Because of this, a few quadratic and interaction effects of the latent variables are taken into account. We utilize a real data set connected to random vectors with G=1,2, to demonstrate the Bayesian approaches in analyzing nonlinear LVMs with dichotomous variables.

let  be the latent continuous random vector, which corresponds to the dichotomous variables where  are dichotomous variables that are related to three latent variables , , with the following values of the parameters in  and

where parameters with an asterisk are treated as fixed for identifying the model.

The true values of elements in  and  are given by: ; . The relationships of the latent variables in   are assessed by the nonlinear structural equation, which is described in the following equations.

Here,  

 and

and . The true values for . The true values for . The covariates  come from arbitrary distributions that give dichotomous data.

In the conjugate prior distributions of the parameters, the following precise prior inputs of the hyper-parameter values are taken into account:

Prior I: Elements in ,  and  in Equation (10) are set equal to the following values with initial values are equal to 1 for two groups of data;

    and are taken to be 0.25 times the identity matrices;  ,  , .

Prior II: Elements in ,  and  in Equation (10) are set equal to the following values with initial values are equal to 0.5 for two groups of data;

  and are taken to be 0.25 times the identity matrices;  ,  , .

The parameter estimates for a situation with a small sample size can be significantly impacted by the prior, which is informative.

Using Open BUGS, a data set (n1=200, n2=200) was analysed. The MCMC method for data analysis required more iterations to converge when compared to the Bayesian analyses of LVMs using data. Bayesian estimates for the truncated normal distribution and censored normal distribution in two populations of nonlinear LVMs were derived using T=10000 Iterations after discarding (1000) burn-in Iterations. The Open BUGS software (Spiegelhalter et al., 2007) can implement Bayesian estimates of the parameters in nonlinear LVMs. To demonstrate this, we apply Open BUGS to analyse the current aid data based on Equations (17) and (18) with different prior inputs.

6. Results and Discussion

This section's goal is to give the findings of a simulation research for NLVMs in order to demonstrate how well the DIC and Bayesian estimates work empirically when compared to other models. However, we have the following proposed four models for g=1,2:

This paper introduces the Bayesian technique for analysing two populations nonlinear LVMs for dichotomous variables and covariates. Using recently created powerful instruments and the completely free statistical program Open BUGS, the model selection statistic (DIC) and the Bayesian analysis of the unobserved parameters are both achieved. As a result, real data may easily be applied to our suggested strategy. The purpose of this analysis is to use Bayesian nonlinear two populations LVMs with Dichotomous variables and covariates. The analysis of dichotomous data in LVMs is subject to various limitations. First, data are typically originating from dichotomous variables and covariates due to the nature of discrete data in the behavioural, medical, and social sciences. It is highly important to discover an alternate approach to manage the problem of dichotomous variables and covariates because when analysing dichotomous data, the fundamental premise in LVMs that the data originate from a continuous normal distribution is plainly broken. Thus, it is obvious that drawing incorrect inferences from dichotomous variables when considering them consistently as normal may do so (see Lee et al., 1990; Olsson, 1979). Assessing these types of data more effectively involves treating them as observations from a concealed continuous normal distribution with unique threshold specifications.

Figure 2. Two chains of observation corresponding to (a) ; (b) (c) ; and (d) for two populations NLVMs with Dichotomous Variables using Truncated Normal Distribution

Figure 3. Two chains of observation corresponding to (a) ; (b) (c) ; and (d) for two populations NLVMs with Dichotomous Variables using Continuous Normal Distribution

TABLE 1. Bayesian Estimation of two populations NLVMs with Dichotomous Variables of First Group using Censored Normal Distribution

TABLE 2. Bayesian Estimation of two populations NLVMs with Dichotomous Variables of Second Group using Censored Normal Distribution

TABLE 3. Bayesian Estimation of two populations NLVMs with Dichotomous Variables of First Group using Truncated Normal Distribution

TABLE 4. Bayesian Estimation of  two populations NLVMs with Dichotomous Variables of Second Group using Truncated Normal Distribution

TABLE 5. Bayesian Estimation of  two populations NLVMs with Dichotomous  Variables of First Group using Continuous Normal Distribution

TABLE 6. Bayesian Estimation of  two populations NLVMs with Dichotomous  Variables of Second Group using Continuous Normal Distribution

Table 7.  Performance of Deviance Information Criterion DIC for two populations NLVMs with Dichotomous Variables Using Censoring, Truncation and Continuous Normal Distribution

Tables (1:2) contain the results for the first and second groups using Type I and Type II inputs, dichotomous variables, covariates, hidden continuous normal distributions (censored normal distributions) for variables, hidden continuous normal distributions (truncated normal distributions with known parameters), and two types of thresholds (with equally and unequally distances for categories). In the first and second groups, the SD values are noticeably low.

The results for the first and second groups under Type I and Type II inputs, dichotomous variables, covariates, hidden continuous normal distribution (truncated normal distribution with known parameters), as well as two types of thresholds (with equally and unequally distances for categories), are reported in Tables (3:4). We noticed that the first and second groups' SD values are rather low.

The parameter with the Highest posterior density (HPD) was determined. When adopting a censored normal distribution or a truncated normal distribution, we found that the HPD intervals work well for dichotomous variables.

We re-analysed the data sets using a nonlinear latent variable model (M4) with interaction term to show the efficacy of DIC for model comparison. The DIC values were contrasted with those obtained using the appropriate model. Tables 5 and 6 present the findings.

The DIC values of censored normal distribution, truncated normal distribution with equally distances of thresholds, are (18070.0) and (19310.0) respectively.

Using a censored normal distribution, the model that best fits the DIC of LVMs with dichotomous data is less accurate than using a truncated normal distribution. For dichotomous variables with censored normal distribution, it performs exceptionally well.

The DIC values of censored normal distribution, truncated normal distribution with equally distances of thresholds, are (17580.0) and (19350.0) respectively.

A model that fits the DIC of LVMs with dichotomous data using a censored normal distribution is less accurate than one that fits the DIC of LVMs with dichotomous data using a truncated normal distribution with uneven threshold distances. For dichotomous variables with censored normal distribution, it performs exceptionally well.

The censored normal distribution with unbalanced distances between thresholds (17580.0) is the best fitted model with the lowest DIC value. Additionally, the truncated normal distribution's DIC value with equally spaced thresholds is (19310.0). As a consequence, we discovered that the DIC's performance is unacceptable and would be even worse when used with dichotomous data and a truncated normal distribution with unbalanced threshold distances.

Plots of several simulated sequences of the individual parameters with varied beginning values are used to track the convergence of the Gibbs sampler and are shown in Figures (4 and 5, respectively). After eliminating (1000) burn-in rounds in two populations nonlinear NLVMs for censoring and truncation normal distribution, Bayesian estimates were obtained from T=10000 iterations for two groups.