Censoring and Truncation
The purpose of this session is to show
you how to use STATA's procedures for doing censored and truncated regression.
We also estimate Heckman's two-stage procedure for samples with selection bias
which is a form of incidential truncation.
/*
This file demonstrates some of STATA's procedures for doing censored and
truncated regression. In particular, we estimate a lower limit censored
regression (i.e., Tobit), Cragg's model that
assumes a heterogenous censoring process, Heckman's incidential truncation model for dealing with
sample selection
bias, and truncated regression.*/
/*We
will use some of the Mroz data on female labor force
participation and income for these examples. The first 428 observations of the Mroz
data contained
women who worked in 1975. The remaining 345 bservations
contained women who did not work. We will use only the first 50
observations from each of
these subsets of the data.*/
use
"c:\users\wood\documents\my teaching\maximum likelihood\data\tobit.dta",
clear
summarize
/*
Now let's estimate a Tobit model and also save the
log likelihood for later testing. The dependent variable (WHRS)is
the wife's hours worked in
1975.
The independent variables are a constant, number of children less than 6 years
old (KL6), number of children between 6 and 18 (KL618),
wife's age (WA), and wifes education (WE).
*/
tobit whrs kl6 k618 wa we, ll(0)
scalar ltobit=e(ll)
display ltobit
/*When
Using STATA, you must specify what is being censored. The command", ll(0)"
tells STATA that the lower limit is being set to zero*/
tobit whrs kl6 k618 wa we, ll(0)
fitstat /*Obtain
various fit statistics on the tobit regression*/
listcoef, help /*List the
coefficients and standardized coefficients*/
prvalue, x(kl6=2)
rest(mean) /*Compute
probabilities when kl6=2 and rest at mean */
/*
McDonald and Moffit suggest a useful decomposition of
the marginal effects associated with the censored regression model. They show
that a change
in the conditional mean due to
right side variables derives from two sources:
1)
It affects the conditional mean in the uncensored part of the distribution
2)
It affects the conditional mean by also affecting the probability that an
observation will
lie in the
uncensored part of the distribution.
Below
we calculate McDonald and Moffit's decomposition. */
gen one=1
mkmat whrs, matrix(y)
mkmat one,
matrix(ones)
mkmat one kl6 k618 wa we, matrix(X)
matrix Xb=inv(ones'*ones)*ones'*X
matrix b = e(b)
matrix
beta = b[1,5],b[1,1], b[1,2],b[1,3],b[1,4]
matrix BXoverS=Xb * beta'/b[1,6]
scalar
Z=el(BXoverS,1,1)
scalar Mu=normalden(Z)/normal(Z)
scalar P=normal(Z)
scalar
P1=P*(1-Z*Mu-Mu^2)
scalar P2=normalden(Z*Z+normalden(Z)*Mu)
display Mu
display P
display P1
display P2
*/
Note that the probability (P) associated with the marginal effects is now
decomposed into two parts. P1 is the probability
associated with X
affecting Y in the uncensored part of the distribution. P2 is the indirect
effect which determines the probability
of being in the uncensored part.
From this decomposition we can decompose the marginal effects into the two
parts as suggested above.
There
is also a procedure dtobit2 which can be used, but it does not do this
particular decomposition. It is executed below. */
dtobit2 whrs kl6 k618 wa we, ll(0)
/*
You can also calculate marginal effects after tobit using the either the margins or mfx procedure. The advantage here
is that you can do it at specific
values and you get standard errors.*/
/*
Margins */
tobit whrs i.kl6 i.k618 wa we, ll(0)
margins kl6, atmeans /* Predictions for kl6 at means */
margins kl6,
at(we=(7(1)17)) /* Predictions conditional on education */
marginsplot
margins kl6, predict(ystar(0,.)) /* Prediction conditional on work hours >0
*/
marginsplot
/*
Unconditional marginal effects using margins, Use these if you want the
marginal effect, whether censored
or uncensored. */
tobit whrs kl6 k618 wa we, ll(0)
margins, dydx(*)
/* Now, unconditional marginal effects
using mfx. */
tobit whrs kl6 k618 wa we, ll(0)
mfx
*/
Now, the marginal effects probabilities for being uncensored are below, where
the two numbers are the lower and upper censoring limits.
Note
that these numbers sum to the overall probability P computed above for McDonald
and Moffit's decomposition. */
mfx compute,
predict(p(0,4212))
*/
The marginal effects for the expected value of the dependent variable
conditional on being uncensored, E(y | a<y<b), are. Use this
if you are mainly interested in
the uncensored part of the distribution. */
mfx compute,
predict(e(0,4212))
*/
Finally, the marginal effects for the unconditional expected value of the
dependent variable, E(y*), where y* = max(a, min(y,b)), are */
mfx compute,
predict(ys(0,4212))
/*
Compare these results to the dtobit2 procedure above. */
/*
Cragg has suggested that assuming a censoring limit
that depends on the same distribution as the uncensored observations is often
incorrect.
He
suggests a two equation system in which the first equation estimates the
probability of being above the censoring limit and the second is a
truncated regression on
the uncensored observations. Below we estimate Cragg's
model using Probit and STATA's Truncated regression
procedure.
Also,
we do a likelihood ratio test of whether Cragg's
model is significantly different than the Tobit
model.
*/
probit lfp kl6 k618 wa we
scalar lprobit=e(ll)
display lprobit
keep if lfp==1
truncreg whrs kl6 k618 wa we, ll(0)
scalar ltrunc=e(ll)
display ltrunc
scalar lrtest=2*((lprobit+ltrunc)-ltobit)
display lrtest
/*
The restricted model is Tobit. The unrestricted model
is the two models estimated separately. The test statistic is 14.4, which is
chi-squared
with 5 degrees of
freedom for the number of additional parameters being estimated. The critical
value is 11.07, so we reject the null that the
restricted model is
true. The two equation approach is therefore more appropriate than Tobit. */
/*STATA
also allows estimating models where the data are censored within specific
intervals (intreg). The censoring can also vary
instead of being restricted to a single value.(cnreg). We will not illustrate those procedures here. */
/*
Now let's turn to estimating a model with sample selection bias. In these cases
the truncation is incidental, due to sample selection
on another variable that is
correlated with the truncation in the dependent variable. As discussed in
class, the standard model is Heckman's
two stage
procedure. Here is an example. First, read in the data again. */
use
"c:\users\wood\documents\my teaching\maximum
likelihood\data\tobit.dta", clear
/*
Now estimate the two stage sample selection model using Heckman's procedure,
reporting the first stage probit, and saving the
mills ratio in a variable called lambda. Note that omitting the twostep option returns the full information maximum
likelihood estimates, which jointly estimate the probit
and regression at the same time. This can present a difficult estimation
problem and may take some time to converge. */
heckman whrs kl6 k618 wa we, select(lfp=cit kl6) twostep
fir m(lambda)
list lambda
/*
The probit equation estimates an index, the inverse
Mills ratio that attempts to measure the omitted variable in the equation for
the incidentially
truncated variable in
the second equation. This variable is inserted as an additional variable in the
second equation. */