| Title: | Likelihood-Based Inference for Joint Modeling of Correlated Count and Binary Outcomes with Extra Variability and Zeros |
|---|---|
| Description: | Inference approach for jointly modeling correlated count and binary outcomes. This formulation allows simultaneous modeling of zero inflation via the Bernoulli component while providing a more accurate assessment of the Hierarchical Zero-Inflated Poisson's parsimony (Lizandra C. Fabio, Jalmar M. F. Carrasco, Victor H. Lachos and Ming-Hui Chen, Likelihood-based inference for joint modeling of correlated count and binary outcomes with extra variability and zeros, 2026, under submission). |
| Authors: | Lizandra C. Fabio [aut], Jalmar M. F. Carrasco [aut, cre], Victor H. Lachos [aut], Ming-Hui Chen [aut] |
| Maintainer: | Jalmar M. F. Carrasco <[email protected]> |
| License: | GPL-3 |
| Version: | 0.1.4 |
| Built: | 2026-10-21 21:25:48 UTC |
| Source: | https://github.com/carrascojalmar/hzip |
Produces a normal Q-Q plot with a simulation envelope for
randomized quantile residuals from a Hierarchical Zero-Inflated
Poisson (HZIP) model fitted via hzip.
envelope(object, nsim = 21, method = "ghq", Q = 15, ...)envelope(object, nsim = 21, method = "ghq", Q = 15, ...)
object |
Either an object of class |
nsim |
Integer specifying the number of Monte Carlo simulations
used to construct the simulation envelope. Default is |
method |
Character string passed to |
Q |
Integer giving the number of Gauss-Hermite quadrature nodes
when |
... |
Additional arguments (currently ignored). |
The envelope is constructed using Monte Carlo simulation based on
nsim replications of independent standard normal samples.
For each order statistic, the 2.5% and 97.5% empirical quantiles
define the envelope limits.
If object is of class hzip, residuals are computed
internally via residuals.hzip before plotting.
A ggplot2 object containing the Q-Q plot with
simulation envelope.
fit.salamander <- hzip( y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders ) # Passing the fitted object directly envelope(fit.salamander) # Or passing residuals explicitly res <- residuals(fit.salamander) envelope(res)fit.salamander <- hzip( y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders ) # Passing the fitted object directly envelope(fit.salamander) # Or passing residuals explicitly res <- residuals(fit.salamander) envelope(res)
Produces a Q-Q plot of residuals from a Hierarchical Zero-Inflated Poisson (HZIP) Model fitted via hzip.
envelope.HZIP(object, nsim = 100, ...)envelope.HZIP(object, nsim = 100, ...)
object |
An object of class |
nsim |
Integer. Number of simulations used to construct the envelope. Default is |
... |
Additional arguments (currently ignored). |
A simulation envelope is added using Monte Carlo replications.
Envelope simulation plot.
## Not run: data(salamanders) fit.salamander <- hzip(y ~ mined|mined+spp,data = salamanders) res <- residuals(fit.salamander) envelope.HZIP(res, nsim = 5000) ## End(Not run)## Not run: data(salamanders) fit.salamander <- hzip(y ~ mined|mined+spp,data = salamanders) res <- residuals(fit.salamander) envelope.HZIP(res, nsim = 5000) ## End(Not run)
hzip() fits a longitudinal/clustered zero-inflated Poisson model with
subject-level random effects following a Generalized Log-Gamma (GLG)
distribution, by maximizing a marginal likelihood approximated via adaptive
Gauss-Hermite quadrature. The model uses a two-part Formula:
, where the count
intensity (Poisson mean) and the zero-inflation probability are linked to
(possibly different) sets of covariates. Initial values are obtained from
pscl::zeroinfl(..., dist = "poisson", link = "cloglog").
hzip( formula, data, hessian = TRUE, method = "BFGS", Q = 15, lower = -Inf, upper = Inf, control = NULL, start = NULL, refine = FALSE, ep_method = c("optim", "numDeriv"), n_starts = 1, verbose = FALSE, ... )hzip( formula, data, hessian = TRUE, method = "BFGS", Q = 15, lower = -Inf, upper = Inf, control = NULL, start = NULL, refine = FALSE, ep_method = c("optim", "numDeriv"), n_starts = 1, verbose = FALSE, ... )
formula |
A two-part Formula of the form
|
data |
A |
hessian |
Logical; if |
method |
Character string passed to |
Q |
Integer; number of Gauss-Hermite nodes for quadrature
(default |
lower |
Bounds on the variables for the |
upper |
Bounds on the variables for the |
control |
Optional |
start |
Optional numeric vector of initial values of length
|
refine |
Logical; if |
ep_method |
Character; method used to compute the Hessian for the
standard errors. Either |
n_starts |
Integer; if greater than |
verbose |
Logical; if |
... |
Further arguments passed to |
Let denote the count response for subject at occasion
. The HZIP model assumes
with linear predictors for the count and zero parts (links typically
log for the Poisson mean and cloglog for the zero-inflation).
Subject-specific random effects follow a GLG distribution and
induce within-subject dependence; the marginal likelihood is approximated by
adaptive Gauss-Hermite quadrature with Q nodes.
Optimization strategy. The default workflow is:
Generate initial values from pscl::zeroinfl (with scales set
to a neutral 0.5, ensuring reproducibility);
Run optim with the specified method
(default BFGS);
If refine = TRUE, refine the result with
nlminb (off by default; useful for
difficult problems);
If n_starts > 1, repeat steps 2-3 with perturbed initial
values and keep the best log-likelihood;
Compute standard errors from the inverse Hessian using the method
specified by ep_method.
An object of class "hzip", a list with elements:
call |
The matched call. |
formula |
The model |
coefficients_zero |
Estimated coefficients for the zero-inflation part. |
coefficients_count |
Estimated coefficients for the count part. |
scale_zero |
Estimated scale (zero part). |
scale_count |
Estimated scale (count part). |
loglik |
Maximized log-likelihood. |
AIC, BIC
|
Information criteria based on |
convergence |
Convergence code (0 = success). |
(loglikz, AICz, BICz and the AIC_HZIP/BIC_HZIP
contrasts are currently disabled/commented out in the code and are not
part of the returned object or printed output for now.)
n |
Number of observations. |
m |
Cluster sizes per subject (vector ordered by |
ep |
Approximate standard errors (square roots of the diagonal of the inverse Hessian). |
iter |
Number of optimization iterations. |
method |
Optimization method. |
refine |
Whether |
ep_method |
Method used to compute standard errors. |
n_starts |
Number of starting points used. |
optim |
Raw output from the final optimization stage. |
data |
The input |
The subject identifier must be named Ind. The internal parameter
vector follows the order
c(scale_zero, beta_zero, scale_count, beta_count).
For difficult problems (e.g., models with many interaction terms), if the
default fit produces large standard errors or the AIC suggests a worse fit
than a nested simpler model, try (i) n_starts = 10 for multi-start
optimization and (ii) supplying start = ... from a fit of the simpler
nested model.
Min, Y., & Agresti, A. (2005). Random effect models for repeated measures of zero-inflated count data. Statistical Modelling, 5(1), 1-19.
Jackman, S. (2020). pscl: Classes and Methods for R Developed in the Political Science Computational Laboratory. R package version 1.5.5.
Zeileis, A., & Croissant, Y. (2010). Extended model formulas in R: Journal of Statistical Software, 34(1), 1-13. (Formula)
fit.salamander <- hzip(y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders) summary(fit.salamander)fit.salamander <- hzip(y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders) summary(fit.salamander)
Computes randomized quantile residuals for fitted
hzip models.
## S3 method for class 'hzip' residuals(object, method = c("ghq", "laplace"), Q = 21, ...)## S3 method for class 'hzip' residuals(object, method = c("ghq", "laplace"), Q = 21, ...)
object |
An object of class |
method |
Character string indicating the approximation method used to estimate the posterior random effects. Possible values are:
|
Q |
Integer specifying the number of Gauss-Hermite quadrature
nodes when |
... |
Further arguments passed to or from other methods. |
The residuals are obtained by integrating out the latent random effects using either Gauss-Hermite quadrature or a Laplace approximation.
Let denote the response variable for the -th
observation of the -th subject. The residuals are computed
using the randomized quantile residual approach proposed by
Dunn and Smyth (1996).
For each observation, the conditional cumulative distribution function is evaluated as
where .
The residuals are then obtained as
where is the standard normal quantile function.
Random effects are approximated through either:
Gauss-Hermite quadrature (method = "ghq");
Laplace approximation (method = "laplace").
The Laplace approximation computes posterior modes using
optim, while the quadrature approach relies on
gauss.quad.
A numeric vector containing the randomized quantile residuals.
Dunn, P. K. and Smyth, G. K. (1996). Randomized quantile residuals. Journal of Computational and Graphical Statistics, 5(3), 236–244.
fit.salamander <- hzip(y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders) res1 <- residuals(fit.salamander, method = "ghq") res2 <- residuals(fit.salamander, method = "laplace") head(res1) head(res2)fit.salamander <- hzip(y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders) res1 <- residuals(fit.salamander, method = "ghq") res2 <- residuals(fit.salamander, method = "laplace") head(res1) head(res2)
This function calculates residuals for objects of class HZIP
usingrandomized quantile residuals. The computation is performed efficiently
using C++ functions for predicting random effects and calculating
residuals.
## S3 method for class 'HZIP' residuals(object, ...)## S3 method for class 'HZIP' residuals(object, ...)
object |
An object of class |
... |
Additional arguments (not used). |
The function internally groups the data by individual (Ind), constructs
model matrices for both zero-inflation and count parts of the model, and then
calls the C++ functions predict_HZIP_cpp_vec and r_ij_cpp_vec
to efficiently compute the residuals. Random effects are integrated using
adaptive quadrature based on the supplied nodes and weights.
A numeric vector of residuals with length equal to the total number of observations in the dataset.
## Not run: fit.salamander <- hzip(y ~ mined|mined+spp,data = salamanders) residuals(fit.salamander) ## End(Not run)## Not run: fit.salamander <- hzip(y ~ mined|mined+spp,data = salamanders) residuals(fit.salamander) ## End(Not run)
rhzip() generates panel/longitudinal data from a two–part
hierarchical zero-inflated Poisson model with subject-specific random
effects for both the zero-inflation and the count components. Random
effects are drawn from a generalized log-gamma (GLG) distribution via
rgengamma() (user must provide/attach a function with this name; see
Dependencies).
rhzip(n, m, para, x1, x2)rhzip(n, m, para, x1, x2)
n |
Integer. Number of subjects. |
m |
Integer vector of length |
para |
Numeric vector of parameters in the order
Internally, the linear predictors are
|
x1 |
Numeric matrix of covariates for the zero-inflation part
(dimension |
x2 |
Numeric matrix of covariates for the count part
(dimension |
For subject with observations, the model draws
two subject-level random effects and independently
from GLG distributions parameterized by lambda1 and lambda2.
Conditional on these effects, outcomes are generated as
where controls structural zeros and
controls the count size.
The returned data frame contains subject IDs, the response y, and the
supplied covariates. An attribute "propZeros" stores a small summary
with the percentage of structural zeros, additional Poisson zeros, and total
zeros.
A tibble with columns:
Ind |
Subject identifier (1..n), repeated according to |
y |
Simulated response. |
x*, w*
|
The covariates from |
The object has an attribute "propZeros": a 3×1 data.frame with
rows "Zeros" (structural zeros, %),
"Count" (extra zeros from the Poisson part, %), and
"Total" (overall zero percentage).
All columns of x1 are kept and renamed x1, x2, ..., xp1
(zero-inflation part). All columns of x2 are kept and renamed
w1, w2, ..., wp2 (count part). No column is dropped: if x1
and/or x2 include an intercept column (a column of 1's), it is
preserved in the output (as x1 and/or w1, respectively) and
can be referenced explicitly in the hzip formula (using
-1 on that side to avoid R adding a duplicate automatic intercept),
or simply omitted from the formula (letting R's automatic intercept play
the same role).
This function calls rgengamma() to draw GLG random effects. Ensure that
such a function is available on the search path (e.g., from a package that
provides a generalized log-gamma RNG) or provide your own implementation
with the signature rgengamma(n, mu, sigma, lambda). It also uses
dplyr and tibble.
hzip for model fitting.
set.seed(123) n <- 50 m <- rep(4, n) N <- sum(m) # design matrices (with intercepts) x1 <- cbind(1, rnorm(N)) x2 <- cbind(1, rnorm(N), rbinom(N, 1, 0.5)) p1 <- ncol(x1); p2 <- ncol(x2) lambda1 <- 0.7 beta1 <- c(-0.2, 0.6) lambda2 <- 0.9 beta2 <- c( 0.3, 0.5, -0.4) para <- c(lambda1, beta1, lambda2, beta2) sim <- rhzip(n, m, para, x1, x2) head(sim) attr(sim, "propZeros") # Fit it back with hzip(). With the new convention NO column is dropped, # so x1 (2 cols: intercept, slope) comes out as "x1,x2" and x2 (3 cols: # intercept, slope, slope) comes out as "w1,w2,w3". Referencing only the # real slopes ("x2" and "w2+w3") and letting R's formula add the # intercept automatically reproduces the true model exactly (equivalently, # one could write "x1+x2-1 | w1+w2+w3-1" using the explicit intercept # columns and disabling the automatic one). fit <- hzip(y ~ x2 | w2 + w3, data = sim) summary(fit) # Example WITHOUT an intercept in the zero part (e.g. simulation Scenario 1 # style): x1 has no column of 1's, so every column of x1 is a real slope # and must be fit with "-1" to avoid an automatic (spurious) intercept. n2 <- 200; m2 <- rep(4, n2); N2 <- sum(m2) w1b <- rep(rnorm(n2), times = m2) w2b <- rep(runif(n2), times = m2) x1b <- cbind(w1b, w2b) # zero part: NO intercept x2b <- cbind(1, w1b, w2b) # count part: WITH intercept parab <- c(0.8, 1, -1, 0.5, 1, 1, 1) # (lambda1,beta11,beta12,lambda2,beta20,beta21,beta22) simb <- rhzip(n2, m2, parab, x1b, x2b) head(simb) # columns: Ind, y, x1, x2 (zero part, both real slopes), w1, w2, w3 (count part) fitb <- hzip(y ~ x1 + x2 - 1 | w2 + w3, data = simb) summary(fitb)set.seed(123) n <- 50 m <- rep(4, n) N <- sum(m) # design matrices (with intercepts) x1 <- cbind(1, rnorm(N)) x2 <- cbind(1, rnorm(N), rbinom(N, 1, 0.5)) p1 <- ncol(x1); p2 <- ncol(x2) lambda1 <- 0.7 beta1 <- c(-0.2, 0.6) lambda2 <- 0.9 beta2 <- c( 0.3, 0.5, -0.4) para <- c(lambda1, beta1, lambda2, beta2) sim <- rhzip(n, m, para, x1, x2) head(sim) attr(sim, "propZeros") # Fit it back with hzip(). With the new convention NO column is dropped, # so x1 (2 cols: intercept, slope) comes out as "x1,x2" and x2 (3 cols: # intercept, slope, slope) comes out as "w1,w2,w3". Referencing only the # real slopes ("x2" and "w2+w3") and letting R's formula add the # intercept automatically reproduces the true model exactly (equivalently, # one could write "x1+x2-1 | w1+w2+w3-1" using the explicit intercept # columns and disabling the automatic one). fit <- hzip(y ~ x2 | w2 + w3, data = sim) summary(fit) # Example WITHOUT an intercept in the zero part (e.g. simulation Scenario 1 # style): x1 has no column of 1's, so every column of x1 is a real slope # and must be fit with "-1" to avoid an automatic (spurious) intercept. n2 <- 200; m2 <- rep(4, n2); N2 <- sum(m2) w1b <- rep(rnorm(n2), times = m2) w2b <- rep(runif(n2), times = m2) x1b <- cbind(w1b, w2b) # zero part: NO intercept x2b <- cbind(1, w1b, w2b) # count part: WITH intercept parab <- c(0.8, 1, -1, 0.5, 1, 1, 1) # (lambda1,beta11,beta12,lambda2,beta20,beta21,beta22) simb <- rhzip(n2, m2, parab, x1b, x2b) head(simb) # columns: Ind, y, x1, x2 (zero part, both real slopes), w1, w2, w3 (count part) fitb <- hzip(y ~ x1 + x2 - 1 | w2 + w3, data = simb) summary(fitb)
rHZIP() generates panel/longitudinal data from a two–part
hierarchical zero-inflated Poisson model with subject-specific random
effects for both the zero-inflation and the count components. Random
effects are drawn from a generalized log-gamma (GLG) distribution via
rgengamma() (user must provide/attach a function with this name; see
Dependencies).
rHZIP(n, m, para, x1, x2)rHZIP(n, m, para, x1, x2)
n |
Integer. Number of subjects. |
m |
Integer vector of length |
para |
Numeric vector of parameters in the order
Internally, the linear predictors are
|
x1 |
Numeric matrix of covariates for the zero-inflation part
(dimension |
x2 |
Numeric matrix of covariates for the count part
(dimension |
For subject with observations, the model draws
two subject-level random effects and independently
from GLG distributions parameterized by lambda1 and lambda2.
Conditional on these effects, outcomes are generated as
where controls structural zeros and
controls the count size.
The returned data frame contains subject IDs, the response y, and the
supplied covariates. An attribute "propZeros" stores a small summary
with the percentage of structural zeros, additional Poisson zeros, and total
zeros.
A tibble with columns:
Ind |
Subject identifier (1..n), repeated according to |
y |
Simulated response. |
x*, w*
|
The covariates from |
The object has an attribute "propZeros": a 3×1 data.frame with
rows "Zeros" (structural zeros, %),
"Count" (extra zeros from the Poisson part, %), and
"Total" (overall zero percentage).
Columns of x1 are renamed to x1, x2, ..., xp1.
Columns of x2 are copied except the first column is dropped and the
remaining are renamed w1, w2, ..., w_{p2-1}. (This mirrors the current
implementation that excludes the first x2 column from the output.)
This function calls rgengamma() to draw GLG random effects. Ensure that
such a function is available on the search path (e.g., from a package that
provides a generalized log-gamma RNG) or provide your own implementation
with the signature rgengamma(n, mu, sigma, lambda). It also uses
dplyr and tibble.
hzip for model fitting.
## Not run: set.seed(123) n <- 50 m <- rep(4, n) N <- sum(m) # design matrices (with intercepts) x1 <- cbind(1, rnorm(N)) x2 <- cbind(1, rnorm(N), rbinom(N, 1, 0.5)) p1 <- ncol(x1); p2 <- ncol(x2) lambda1 <- 0.7 beta1 <- c(-0.2, 0.6) lambda2 <- 0.9 beta2 <- c( 0.3, 0.5, -0.4) para <- c(lambda1, beta1, lambda2, beta2) sim <- rHZIP(n, m, para, x1, x2) head(sim) attr(sim, "propZeros") ## End(Not run)## Not run: set.seed(123) n <- 50 m <- rep(4, n) N <- sum(m) # design matrices (with intercepts) x1 <- cbind(1, rnorm(N)) x2 <- cbind(1, rnorm(N), rbinom(N, 1, 0.5)) p1 <- ncol(x1); p2 <- ncol(x2) lambda1 <- 0.7 beta1 <- c(-0.2, 0.6) lambda2 <- 0.9 beta2 <- c( 0.3, 0.5, -0.4) para <- c(lambda1, beta1, lambda2, beta2) sim <- rHZIP(n, m, para, x1, x2) head(sim) attr(sim, "propZeros") ## End(Not run)
This dataset is adapted from the glmmTMB package and contains
salamander counts with information on mining status and species.
It is intended for illustrating zero-inflated Poisson models with
random effects using the hzip() function.
salamanderssalamanders
A data frame with 644 rows and 5 variables:
Cluster identifier (integer, 1 to 161); each cluster groups the 4 repeated samples of one site-species combination.
Count response variable (integer).
Mining status: "yes" or "no".
Species factor with multiple levels (e.g., GP, PR, DM, etc.).
Repeated sampling occasion (factor with levels
"1" to "4").
The dataset was originally included in the glmmTMB package
(Brooks et al., 2017), and has been slightly modified for testing
the HZIP package: observations are ordered by site, the
variables count, mined, spp and sample
are kept, count is renamed to y, and the cluster
identifier Ind is added.
Adapted from the glmmTMB package.
data(salamanders, package = "HZIP") ## Fit zero-inflated Poisson with random effects fit.salamander <- hzip(y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders) summary(fit.salamander)data(salamanders, package = "HZIP") ## Fit zero-inflated Poisson with random effects fit.salamander <- hzip(y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders) summary(fit.salamander)
Compares the observed variance of the response with the distribution
of variances obtained from parametric bootstrap replicates simulated
under the fitted hzip model.
testDisp( object, nsim = 500, alternative = c("two.sided", "greater", "less"), alpha = 0.05, seed = NULL, plot = TRUE, ... )testDisp( object, nsim = 500, alternative = c("two.sided", "greater", "less"), alpha = 0.05, seed = NULL, plot = TRUE, ... )
object |
An object of class |
nsim |
A positive integer giving the number of simulated
replicates used to build the reference distribution.
Defaults to |
alternative |
Character string specifying the alternative
hypothesis. One of |
alpha |
Numeric value in |
seed |
An optional integer seed for reproducibility. |
plot |
Logical; if |
... |
Further arguments (currently ignored). |
An object of class "hzip_test" (invisibly), a list
with components:
statisticObserved variance of the response.
p.valueSimulation-based p-value.
alternativeThe alternative hypothesis used.
alphaSignificance level used for the decision.
simulatedNumeric vector of simulated variances.
methodDescription of the test.
h0Statement of the null hypothesis.
The result is printed automatically; to change the number of
significant digits shown, use print(res, digits = 6),
where res is the object returned by testDisp().
fit.salamander <- hzip( y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders ) testDisp(fit.salamander, nsim = 200, seed = 42)fit.salamander <- hzip( y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders ) testDisp(fit.salamander, nsim = 200, seed = 42)
Compares the observed proportion of zeros with the distribution of
zero proportions obtained from parametric bootstrap replicates
simulated under the fitted hzip model.
testZI( object, nsim = 500, alternative = c("two.sided", "greater", "less"), alpha = 0.05, seed = NULL, plot = TRUE, ... )testZI( object, nsim = 500, alternative = c("two.sided", "greater", "less"), alpha = 0.05, seed = NULL, plot = TRUE, ... )
object |
An object of class |
nsim |
A positive integer giving the number of simulated
replicates. Defaults to |
alternative |
Character string specifying the alternative
hypothesis. One of |
alpha |
Numeric value in |
seed |
An optional integer seed for reproducibility. |
plot |
Logical; if |
... |
Further arguments (currently ignored). |
An object of class "hzip_test" (invisibly), a list
with components:
statisticObserved proportion of zeros.
p.valueSimulation-based p-value.
alternativeThe alternative hypothesis used.
alphaSignificance level used for the decision.
simulatedNumeric vector of simulated zero proportions.
methodDescription of the test.
h0Statement of the null hypothesis.
The result is printed automatically; to change the number of
significant digits shown, use print(res, digits = 6),
where res is the object returned by testZI().
fit.salamander <- hzip( y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders ) testZI(fit.salamander, nsim = 200, seed = 42)fit.salamander <- hzip( y ~ mined+spp+mined:spp | mined+spp+sample, data = salamanders ) testZI(fit.salamander, nsim = 200, seed = 42)