Censoring and Truncation

The purpose of this session is to show you how to use STATA's procedures for doing censored and truncated regression. We also estimate Heckman's two-stage procedure for samples with selection bias which is a form of incidential truncation.

/* This file demonstrates some of STATA's procedures for doing censored and truncated regression. In particular, we estimate a lower limit censored

regression (i.e., Tobit), Cragg's model that assumes a heterogenous censoring process, Heckman's incidential truncation model for dealing with

sample selection bias, and truncated regression.*/

/*We will use some of the Mroz data on female labor force participation and income for these examples. The first 428 observations of the Mroz

data contained women who worked in 1975. The remaining 345 bservations contained women who did not work. We will use only the first 50

observations from each of these subsets of the data.*/

use "c:\users\wood\documents\my teaching\maximum likelihood\data\tobit.dta", clear

summarize

/* Now let's estimate a Tobit model and also save the log likelihood for later testing. The dependent variable (WHRS)is the wife's hours worked in

1975. The independent variables are a constant, number of children less than 6 years old (KL6), number of children between 6 and 18 (KL618),

wife's age (WA), and wifes education (WE).  */

tobit whrs kl6 k618 wa we, ll(0)

scalar ltobit=e(ll)

display ltobit

/*When Using STATA, you must specify what is being censored. The command", ll(0)" tells STATA that the lower limit is being set to zero*/

tobit whrs kl6 k618 wa we, ll(0)

fitstat                                         /*Obtain various fit statistics on the tobit regression*/

listcoef, help                                  /*List the coefficients and standardized coefficients*/

prvalue, x(kl6=2) rest(mean)                    /*Compute probabilities when kl6=2 and rest at mean */

/* McDonald and Moffit suggest a useful decomposition of the marginal effects associated with the censored regression model. They show that a change

in the conditional mean due to right side variables derives from two sources:

1) It affects the conditional mean in the uncensored part of the distribution

2) It affects the conditional mean by also affecting the probability that an observation will

lie in the uncensored part of the distribution.

Below we calculate McDonald and Moffit's decomposition. */

gen one=1

mkmat whrs, matrix(y)

mkmat one, matrix(ones)

mkmat one kl6 k618 wa we, matrix(X)

matrix Xb=inv(ones'*ones)*ones'*X

matrix b = e(b)

matrix beta = b[1,5],b[1,1], b[1,2],b[1,3],b[1,4]

matrix BXoverS=Xb * beta'/b[1,6]

scalar Z=el(BXoverS,1,1)

scalar Mu=normalden(Z)/normal(Z)

scalar P=normal(Z)

scalar P1=P*(1-Z*Mu-Mu^2)

scalar P2=normalden(Z*Z+normalden(Z)*Mu)

display Mu

display P

display P1

display P2

*/ Note that the probability (P) associated with the marginal effects is now decomposed into two parts. P1 is the probability

associated with X affecting Y in the uncensored part of the distribution. P2 is the indirect effect which determines the probability

of being in the uncensored part. From this decomposition we can decompose the marginal effects into the two parts as suggested above.

There is also a procedure dtobit2 which can be used, but it does not do this particular decomposition. It is executed below. */

dtobit2 whrs kl6 k618 wa we, ll(0)

/* You can also calculate marginal effects after tobit using the either the margins or mfx procedure. The advantage here

is that you can do it at specific values and you get standard errors.*/

/* Margins */

tobit whrs i.kl6 i.k618 wa we, ll(0)

margins kl6, atmeans /* Predictions for kl6 at means */

margins kl6, at(we=(7(1)17)) /* Predictions conditional on education */

marginsplot

margins kl6, predict(ystar(0,.)) /* Prediction conditional on work hours >0 */

marginsplot

/* Unconditional marginal effects using margins, Use these if you want the marginal effect, whether censored

or uncensored.  */

tobit whrs kl6 k618 wa we, ll(0)

margins, dydx(*)

/* Now, unconditional marginal effects using mfx. */

tobit whrs kl6 k618 wa we, ll(0)

mfx

*/ Now, the marginal effects probabilities for being uncensored are below, where the two numbers are the lower and upper censoring limits.

Note that these numbers sum to the overall probability P computed above for McDonald and Moffit's decomposition. */

mfx compute, predict(p(0,4212))

*/ The marginal effects for the expected value of the dependent variable conditional on being uncensored, E(y | a<y<b), are. Use this

if you are mainly interested in the uncensored part of the distribution. */

mfx compute, predict(e(0,4212))

*/ Finally, the marginal effects for the unconditional expected value of the dependent variable, E(y*), where y* = max(a, min(y,b)), are */

mfx compute, predict(ys(0,4212))

/* Compare these results to the dtobit2 procedure above. */

/* Cragg has suggested that assuming a censoring limit that depends on the same distribution as the uncensored observations is often incorrect.

He suggests a two equation system in which the first equation estimates the probability of being above the censoring limit and the second is a

truncated regression on the uncensored observations. Below we estimate Cragg's model using Probit and STATA's Truncated regression procedure.

Also, we do a likelihood ratio test of whether Cragg's model is significantly different than the Tobit model.

*/

probit lfp kl6 k618 wa we

scalar lprobit=e(ll)

display lprobit

keep if lfp==1

truncreg whrs kl6 k618 wa we, ll(0)

scalar ltrunc=e(ll)

display ltrunc

scalar lrtest=2*((lprobit+ltrunc)-ltobit)

display lrtest

/* The restricted model is Tobit. The unrestricted model is the two models estimated separately. The test statistic is 14.4, which is chi-squared

with 5 degrees of freedom for the number of additional parameters being estimated. The critical value is 11.07, so we reject the null that the

restricted model is true. The two equation approach is therefore more appropriate than Tobit. */

/*STATA also allows estimating models where the data are censored within specific intervals (intreg). The censoring can also vary instead of being restricted to a single value.(cnreg). We will not illustrate those procedures here. */

/* Now let's turn to estimating a model with sample selection bias. In these cases the truncation is incidental, due to sample selection

on another variable that is correlated with the truncation in the dependent variable. As discussed in class, the standard model is Heckman's

two stage procedure. Here is an example. First, read in the data again. */

use "c:\users\wood\documents\my teaching\maximum likelihood\data\tobit.dta", clear

/* Now estimate the two stage sample selection model using Heckman's procedure, reporting the first stage probit, and saving the mills ratio in a variable called lambda. Note that omitting the twostep option returns the full information maximum likelihood estimates, which jointly estimate the probit and regression at the same time. This can present a difficult estimation problem and may take some time to converge. */

heckman whrs kl6 k618 wa we, select(lfp=cit kl6) twostep fir m(lambda)

list lambda

/* The probit equation estimates an index, the inverse Mills ratio that attempts to measure the omitted variable in the equation for the incidentially

truncated variable in the second equation. This variable is inserted as an additional variable in the second equation. */