Poisson regression
Part of a series on Statistics 
Regression analysis 

Models 



Estimation 

Background 

In statistics, Poisson regression is a generalized linear model form of regression analysis used to model count data and contingency tables. Poisson regression assumes the response variable Y has a Poisson distribution, and assumes the logarithm of its expected value can be modeled by a linear combination of unknown parameters. A Poisson regression model is sometimes known as a loglinear model, especially when used to model contingency tables.
Negative binomial regression is a popular generalization of Poisson regression because it loosens the highly restrictive assumption that the variance is equal to the mean made by the Poisson model. The traditional negative binomial regression model, commonly known as NB2, is based on the Poissongamma mixture distribution. This model is popular because it models the Poisson heterogeneity with a gamma distribution.
Poisson regression models are generalized linear models with the logarithm as the (canonical) link function, and the Poisson distribution function as the assumed probability distribution of the response.
Regression models
If is a vector of independent variables, then the model takes the form
where and . Sometimes this is written more compactly as
where x is now an (n + 1)dimensional vector consisting of n independent variables concatenated to a vector of ones. Here θ is simply α concatenated to β.
Thus, when given a Poisson regression model θ and an input vector x, the predicted mean of the associated Poisson distribution is given by
If Y_{i} are independent observations with corresponding values x_{i} of the predictor variables, then θ can be estimated by maximum likelihood. The maximumlikelihood estimates lack a closedform expression and must be found by numerical methods. The probability surface for maximumlikelihood Poisson regression is always concave, making Newton–Raphson or other gradientbased methods appropriate estimation techniques.
Maximum likelihoodbased parameter estimation
Given a set of parameters θ and an input vector x, the mean of the predicted Poisson distribution, as stated above, is given by
and thus, the Poisson distribution's probability mass function is given by
Now suppose we are given a data set consisting of m vectors , along with a set of m values . Then, for a given set of parameters θ, the probability of attaining this particular set of data is given by
By the method of maximum likelihood, we wish to find the set of parameters θ that makes this probability as large as possible. To do this, the equation is first rewritten as a likelihood function in terms of θ:
Note that the expression on the right hand side has not actually changed. A formula in this form is typically difficult to work with; instead, one uses the loglikelihood:
Notice that the parameters θ only appear in the first two terms of each term in the summation. Therefore, given that we are only interested in finding the best value for θ we may drop the y_{i}! and simply write
To find a maximum, we need to solve an equation which has no closedform solution. However, the negative loglikelihood, , is a convex function, and so standard convex optimization techniques such as gradient descent can be applied to find the optimal value of θ.
Poisson regression in practice
Poisson regression may be appropriate when the dependent variable is a count, for instance of events such as the arrival of a telephone call at a call centre.^{[1]} The events must be independent in the sense that the arrival of one call will not make another more or less likely, but the probability per unit time of events is understood to be related to covariates such as time of day.
"Exposure" and offset
Poisson regression may also be appropriate for rate data, where the rate is a count of events divided by some measure of that unit's exposure (a particular unit of observation). For example, biologists may count the number of tree species in a forest: events would be tree observations, exposure would be unit area, and rate would be the number of species per unit area. Demographers may model death rates in geographic areas as the count of deaths divided by person−years. More generally, event rates can be calculated as events per unit time, which allows the observation window to vary for each unit. In these examples, exposure is respectively unit area, person−years and unit time. In Poisson regression this is handled as an offset, where the exposure variable enters on the righthand side of the equation, but with a parameter estimate (for log(exposure)) constrained to 1.
which implies
Offset in the case of a GLM in R can be achieved using the offset() function:
glm(y ~ offset(log(exposure)) + x, family=poisson(link=log) )
Overdispersion and zero inflation
A characteristic of the Poisson distribution is that its mean is equal to its variance. In certain circumstances, it will be found that the observed variance is greater than the mean; this is known as overdispersion and indicates that the model is not appropriate. A common reason is the omission of relevant explanatory variables, or dependent observations. Under some circumstances, the problem of overdispersion can be solved by using quasilikelihood estimation or a negative binomial distribution instead.^{[2]}^{[3]}
Ver Hoef and Boveng described the difference between quasiPoisson (also called overdispersion with quasilikelihood) and negative binomial (equivalent to gammaPoisson) as follows: If E(Y) = μ, the quasiPoisson model assumes var(Y) = θμ while the gammaPoisson assumes var(Y) = μ(1 + κμ), where θ is the quasiPoisson overdispersion parameter, and κ is the shape parameter of the negative binomial distribution. For both models, parameters are estimated using Iteratively reweighted least squares. For quasiPoisson, the weights are μ/θ. For negative binomial, the weights are μ/(1 + κμ). With large μ and substantial extraPoisson variation, the negative binomial weights are capped at 1/κ. Ver Hoef and Boveng discussed an example where they selected between the two by plotting mean squared residuals vs. the mean.^{[4]}
Another common problem with Poisson regression is excess zeros: if there are two processes at work, one determining whether there are zero events or any events, and a Poisson process determining how many events there are, there will be more zeros than a Poisson regression would predict. An example would be the distribution of cigarettes smoked in an hour by members of a group where some individuals are nonsmokers.
Other generalized linear models such as the negative binomial model or zeroinflated model may function better in these cases.
Use in survival analysis
Poisson regression creates proportional hazards models, one class of survival analysis: see proportional hazards models for descriptions of Cox models.
Extensions
Regularized Poisson regression
When estimating the parameters for Poisson regression, one typically tries to find values for θ that maximize the likelihood of an expression of the form
where m is the number of examples in the data set, and is the probability mass function of the Poisson distribution with the mean set to . Regularization can be added to this optimization problem by instead maximizing
for some positive constant . This technique, similar to ridge regression, can reduce overfitting.
On a different vein, several researchers have recently considered much more potent, nonlinear alternatives, for modeling the mapping of independent to dependent variables; Gaussian processes ^{[5]} and deep latent variable networks ^{[6]} are characteristic such examples.
See also
References
 ↑ Greene, William H. (2003). Econometric Analysis (Fifth ed.). PrenticeHall. pp. 740–752. ISBN 0130661899.
 ↑ Paternoster R, Brame R (1997). "Multiple routes to delinquency? A test of developmental and general theories of crime". Criminology. 35: 45–84. doi:10.1111/j.17459125.1997.tb00870.x.
 ↑ Berk R, MacDonald J (2008). "Overdispersion and Poisson regression". Journal of Quantitative Criminology. 24 (3): 269–284. doi:10.1007/s1094000890484.
 ↑ Ver Hoef, JAY M.; Boveng, Peter L. (20070101). "QuasiPoisson vs. Negative Binomial Regression: How should we model overdispersed count data?". Ecology. 88 (11): 2766–2772. doi:10.1890/070043.1. Retrieved 20160901.
 ↑ Torrado, N., Wiper, M.P., Lillo, R.E., 2013. Software reliability modeling with software metrics data via Gaussian processes. IEEE Trans. Softw. Eng. 39 (8), 1179–1186.
 ↑ Andreas S. Andreou and Sotirios P. Chatzis, "Software Defect Prediction Using Doubly Stochastic Poisson Processes Driven by Stochastic Belief Networks," Journal of Systems and Software, vol. 122, pp. 72–82, Dec. 2016.
Further reading
 Cameron, A. C.; Trivedi, P. K. (1998). Regression analysis of count data. Cambridge University Press. ISBN 0521632013.
 Christensen, Ronald (1997). Loglinear models and logistic regression. Springer Texts in Statistics (Second ed.). New York: SpringerVerlag. ISBN 0387982477. MR 1633357.
 Gouriéroux, Christian (2000). "The Econometrics of Discrete Positive Variables: the Poisson Model". Econometrics of Qualitative Dependent Variables. New York: Cambridge University Press. pp. 270–83. ISBN 0521589851.
 Greene, William H. (2008). "Models for Event Counts and Duration". Econometric Analysis (8th ed.). Upper Saddle River: Prentice Hall. pp. 906–944. ISBN 9780136003830.
 Hilbe, J. M. (2007). Negative Binomial Regression. Cambridge University Press. ISBN 9780521857727.
 Jones, Andrew M.; et al. (2013). "Models for count data". Applied Health Economics. London: Routledge. pp. 295–341. ISBN 9780415676823.