Rense Nieuwenhuis » lmer

Influence.ME: don’t specify the intercept

Rense Nieuwenhuis — Thu, 18 Jun 2009 11:00:00 +0000

Just recently, I was contacted by a researcher who wanted to use influence.ME to obtain model estimates from which iteratively some data was deleted. In his case, observations were nested within an area, but there were very unequal numbers of observations in each area.

Unfortunately, he wasn’t able to use the influence.ME package on his models. He kindly sent me his data, so I could figure out what went wrong, and it showed to be a little problem with influence.ME.

The problem was with how the model was specified: the intercept was explicated, next to several (fixed) variables. It turned out, that such a model specification is not compatible with the internal changes made to the mixed model. Therefore, I advise users of influence.ME not to explicitly specify the intercept in their lme4 regression models.

I reproduced the problem with the school23 data, which is available in influence.ME. Compare the two model specifications below: in the first the intercept is specified, in the second it isn’t. The outcomes of both lmer models are identical. However, the first returns a convergence error when used with the estex() function, while the second doesn’t.

The input:
mod <- lmer(math ~ 1 + structure + (1 | school.ID), data=school23) estex.mod <- estex(mod, "school.ID")

mod <- lmer(math ~ structure + (1 | school.ID), data=school23) estex.mod <- estex(mod, "school.ID")

The output:
> mod <- lmer(math ~ 1 + structure + (1 | school.ID), data=school23) > estex.mod <- estex(mod, "school.ID") Error in mer_finalize(ans) : Downdated X'X is not positive definite, 3. > > mod <- lmer(math ~ structure + (1 | school.ID), data=school23) > estex.mod <- estex(mod, "school.ID")

I will surely investigate whether this can be resolved in a future update, but for now, simply leave the intercept out of your model specification: lmer will add it for you.

Influence.ME is an R package and provides tools for detecting influential data in mixed effects models. More information can be found here.

Introducing Influence.ME: Tools for detecting influential data in mixed models

Rense Nieuwenhuis — Wed, 29 Apr 2009 09:03:25 +0000

I’m highly excited to announce that influence.ME is now available. Influence.ME is a new software package for R, providing statistical tools for detecting influential data in mixed models. It has been developed by Rense Nieuwenhuis, Ben Pelzer, and Manfred te Grotenhuis. The basic rationale behind identifying influential data is that when iteratively single units are omitted from the data, models based on these data should not produce substantially different estimates. To standardize the assessment of how influential data is, several measures of influence are commonly used, such as DFBETAS and Cook’s Distance.

Mixed effects regression models tend to become common practice in the field of Social Sciences. However, diagnostic tools to evaluate these models lag behind. For instance there is no general applicable tool to check whether all units (or cases) roughly have the same influence on the regression parameters. It is however commonly accepted that tests for influential cases should be performed, especially when estimates are based on a relatively small number of cases. Testing for influence with mixed effects models is especially important in Social Science applications, for two reasons. First, models in the Social Sciences are frequently based on large numbers of individuals while the number of higher level units is often relatively small. Secondly, often the higher level units are remarkably similar, for instance in the case of neighboring countries. Influence.ME is a new package for R which provides two innovations for evaluating influential cases: it extends existing procedures for use with mixed effects models, and it allows to not only search for single influential cases, but for combinations of cases that as a combination exert too much influence.

I plan to use my blog to provide more information about influence.ME. For instance, you can expect some example analyses soon. Other developments, new features, or exciting applications in research papers will be discussed here as well in due time. A static page on influence.ME is available as well, where all important information is collected.

Questions, comments, thoughts, experiences, notes on bugs (and other vermin), feature requests, and what more: it is all highly appreciated. They can be sent by e-mail, or placed in the comments-section on this blog.

useR! 2009 acceptance: presenting influence.ME

Rense Nieuwenhuis — Thu, 23 Apr 2009 10:23:56 +0000

The organizing committee of the useR! 2009 conference just informed me, that my submission for presenting my extension package influence.ME, has been accepted! Influence.ME is a new R package that I’m currently developing, with the indispensable help of Ben Pelzer and Manfred te Grotenhuis. Although I did not yet introduce influence.ME on this blog, rest assured that I will do so within just a few weeks. Now is time for celebration!

Influence.ME is an R package that provides a collection of tools for detecting influential data in mixed effects models. Testing for inï¬‚uence with mixed effects models is especially important in Social Science applications, for two reasons. First, models in the Social Sciences are frequently based on large numbers of individuals while the number of higher level units is often relatively small. Secondly, often the higher level units are remarkably similar, for instance in the case of neighboring countries.

useR! is a yearly user conference on exciting applications in R. The useR! 2009 edition will be held in Rennes, France. A great variety of packages, applications, and other developments relating to R will be discussed. I’ve visited the useR! 2008 conference last year (in Dortmund, Germany), and found it a highly stimulating environment for those interested in exciting, practical applications in statistics using R.

Influence.ME is a project I’ve been working on for the last months, together with Ben Pelzer and Manfred te Grotenhuis. I’m still working – quite hard!- to iron out the last quirks, and we have tons of ideas for extending its functionality. I’m very happy to be able to present the result of this work to an R-minded audience this summer.

R-Sessions 31: Combining lmer output in a single table (UPDATED)

Rense Nieuwenhuis — Thu, 05 Feb 2009 11:00:38 +0000

There are various ways of getting your output from R to your publication draft. Most of them are highly efficient, but unfortunately I couldn’t find a function that combines the output from several (lmer) models and presents it in a single table. lmer is the mixed effects model function from the lme4 package. So, I wrote a simple function that does exactly that.

Using it for a specific purpose, it is not a general function or something, but it can easily be adapted for use in other settings. Here it goes:

require(lme4) require(mlmRev) require(lme4) require(mlmRev)


model.1 <- lmer(normexam ~ 1 + (1 | school), data=Exam)

model.2 <- lmer(normexam ~ standLRT + (1 | school), data=Exam)

model.3 <- lmer(normexam ~ standLRT + sex + (1 | school), data=Exam)

model.4 <- lmer(normexam ~ standLRT + sex + schavg + (1 | school), data=Exam)
model.a <- lmer(use ~ 1 + (1 | district), family=binomial, data=Contraception)

model.b <- lmer(use ~ livch + (1 | district), family=binomial, data=Contraception)

model.c <- lmer(use ~ age + (1 | district), family=binomial, data=Contraception)

model.d <- lmer(use ~ livch + age + (1 | district), family=binomial, data=Contraception)
m1 <- c(model.1, model.2, model.3, model.4)

m2 <- c(model.a, model.b, model.c, model.d)
combine.output.lmer <- function(models, labels=FALSE)

	{
	fix.coef <- lapply(models, function(x) summary(x)@coefs)

	var.coef <- lapply(models, function(x) summary(x)@REmat)

	n.par <- dim(summary(models[[1]])@coefs)[2]
	ifelse(labels==FALSE,

		fix.labels <- colnames(summary(models[[1]])@coefs),

		fix.labels <- labels)
	var.labels <- colnames(var.coef[[1]])
	# Creating table with fixed parameters

	output.coefs <- data.frame(Row.names=row.names(fix.coef[[1]]))

	for (i in 1:length(models))

		{
		a <- fix.coef[[i]]

		colnames(a) <- paste("Model", i, fix.labels)

		output.coefs <- merge(output.coefs, a, by.x=1, by.y=0, all=T, sort=FALSE)
		}

	output.coefs[,1] <- as.character(output.coefs[,1])

	output.coefs[dim(output.coefs)[1]+2, 1] <- "Loglikelihood"

	LL <- unlist(lapply(models, function(x) as.numeric(logLik(x))))

	output.coefs[dim(output.coefs)[1], 1:length(models)*n.par-n.par+2] <- LL
	# Creating table with random parameters

	output.vars <- data.frame(var.coef[[1]])[,1:2]

	for (i in 1:length(models))

		{
		a <- var.coef[[i]]

		colnames(a) <- paste("Model", i, var.labels)

		output.vars <- merge(output.vars, a, by.x=1:2, by.y=1:2, all=T, sort=FALSE)
		}
	# Combining output.coefs and output.vars

	n.cols <- dim(output.coefs)[2]

	n.coefs <- dim(output.coefs)[1]

	n.vars <- dim(output.vars)[1]
	output <- matrix(ncol=n.cols +1 , nrow=n.vars+n.coefs+2)
	output[1:n.coefs, -2] <- as.matrix(output.coefs)

	output[n.coefs+2, 1] <- "Variance Components"

	output[(n.coefs+3) : (n.coefs+n.vars+2), 1:2] <- as.matrix(output.vars[,1:2])

	output[

		(n.coefs+3) : (n.coefs+n.vars+2),

		which(rep(c(1,1,rep(0, n.par-2)),length(models))!=0)+2] <- as.matrix(output.vars[,c(-1,-2)])
	colnames(output) <- c("Parameter", "Random", colnames(output.coefs)[-1])
	return(output)

	}
combined <- combine.output.lmer(m1)

combined <- combine.output.lmer(m2)
combined <- combine.output.lmer(m1, labels=c("appel", "banaan", "grapefruit"))

combined <- combine.output.lmer(m2, labels=c("appel", "peer", "banaan", "grapefruit"))

write.csv(combined, "combined.csv", na=" ")

In this example I estimate four mixed effects models, which are concatenated in a single object 'm'. The function itself is called 'combine.output.lmer', and is used on the object 'm'. The output is a data.frame with the variable names in the first column. Not-estimated parameters in models are indicated by 'NA' in their respective columns. By writing the 'combined'-object to an external file, the NA's are lost and the file can be read into other software, such as Open Office Spreadsheet or Excell. Use the xtable-package to get it in your latex document.

UPDATE
I updated and improved the code somewhat, for I wasn't satisfied with the results. Now the code adapts to the number of parameters derived form the models' summary, allows to add your own names to the columns, and, most importantly, also reports the random slopes.

Please note: due to the internal matching procedure, errors may occur when the same variable is random 'within' more than one other variable. This is only the case when other variables are random within each nesting factor as well.

R-Sessions 16: Multilevel Model Specification (lme4)

Rense Nieuwenhuis — Wed, 27 Aug 2008 10:00:47 +0000

Multilevel models, or mixed effects models, can easily be estimated in R. Several packages are available. Here, the lmer() function from the lme4-package is described. The specification of several types of models will be shown, using a fictive example. A detailed description of the specification rules is given. Output of the specified models is given, but not described or interpreted.
Please note that this description is very closely related to the description of the specification of the lme() function of the nlme-package. The results are similar and here exactly the same possibilities are offered.

In this example, the dependent variable is the standardized result of a student on a specific exam. This variable is called “normexam”. In estimating the score on the exam, two levels will be discerned: student and school. On each level, one explanatory variable is present. On individual level, we are taking into account the standardized score of the student on a LR-test (“standLRT”). On the school-level, we take into account the average intake-score (“schavg”).

Preparation

Before analyses can be performed, preparation needs to take place. Using the library() command, two packages are loaded. The lme4-package contains functions for estimation of multilevel or hierarchical regression models. The mlmRev-package contains, amongst many other things, the data we are going to use here. In the output below, we see that R-Project automatically loads the Matrix- and the lattice-packages as well. These are needed for the lme4-package to work properly.
Finally, the names() command is used to examine which variables are contained in the ‘Exam’ data.frame.

library(lme4)
library(mlmRev)
names(Exam)

>library(lme4)
Loading required package: lme4
Loading required package: Matrix
Loading required package: lattice
[1] TRUE
>library(mlmRev)
Loading required package: mlmRev
[1] TRUE
>names(Exam)
 [1] "school"   "normexam" "schgend"  "schavg"   "vr"       "intake"
 [7] "standLRT" "sex"      "type"     "student"

null-model

The syntax below specifies the most simple multilevel regression model of all: the null-model. Only the levels are defined. Using the lmer-function, the first level (here: students) do not have to be specified. It is assumed that the dependent variable (here: normexam) is on the first level (which it should be).

The model is specified using standard R formulas: First the dependent variable is given, followed by a tilde ( ~ ). The ~ should be read as: “follows”, or: “is defined by”. Next, the predictors are defined. In this case, only the intercept is defined by entering a ‘1’. Next, the random elements are specified between brackets ( ). Inside these brackets we specify the random predictors, followed by a vertical stripe ( | ), after which the group-level is specified.

After the model specification, several parameters can be given to the model. Here, we specify the data that should be used by data=Exam. Another often used parameter indicates the estimation method. If left unspecified, restricted maximum likelihood (REML) is used. Another option would be: method=”ML”, which calls for full maximum likelihood estimation. All this leads to the following model specification:

lmer(normexam ~ 1 + (1 | school), data=Exam)

This leads to the following output:

> lmer(normexam ~ 1 + (1 | school), data=Exam)
Linear mixed-effects model fit by REML
Formula: normexam ~ 1 + (1 | school)
   Data: Exam
   AIC   BIC logLik MLdeviance REMLdeviance
 11019 11031  -5507      11011        11015
Random effects:
 Groups   Name        Variance Std.Dev.
 school   (Intercept) 0.17160  0.41425
 Residual             0.84776  0.92074
number of obs: 4059, groups: school, 65

Fixed effects:
            Estimate Std. Error t value
(Intercept) -0.01325    0.05405 -0.2452

random intercept, fixed predictor in individual level

For the next model, we add a predictor to the individual level. We do this, by replacing the ‘1’ of the previous model by the predictor (here: standLRT). An intercept is always assumed, so it is still estimated here. It only needs to be specified when no other predictors are specified. Since we don’t want the effect of the predictor to vary between groups, the specification of the random part of the model remains identical to the previous model. The same data is used, so we specify data=Exam again.

lmer(normexam ~ standLRT + (1 | school), data=Exam)

> lmer(normexam ~ standLRT + (1 | school), data=Exam)
Linear mixed-effects model fit by REML
Formula: normexam ~ standLRT + (1 | school)
   Data: Exam
  AIC  BIC logLik MLdeviance REMLdeviance
 9375 9394  -4684       9357         9369
Random effects:
 Groups   Name        Variance Std.Dev.
 school   (Intercept) 0.093839 0.30633
 Residual             0.565865 0.75224
number of obs: 4059, groups: school, 65

Fixed effects:
            Estimate Std. Error t value
(Intercept) 0.002323   0.040354    0.06
standLRT    0.563307   0.012468   45.18

Correlation of Fixed Effects:
         (Intr)
standLRT 0.008

random intercept, random slope

The next model that will be specified, is a model with a random intercept on individual level and a predictor that is allowed to vary between groups. In other words, the effect of doing homework on the score on a math-test varies between schools. In order to estimate this model, the ‘1’ that indicates the intercept in the random part of the model specification is replaced by the variable of which we want the effect to vary between the groups.

lmer(normexam ~ standLRT + (standLRT | school), data=Exam, method=”ML”)

> lmer(normexam ~ standLRT + (standLRT | school), data=Exam, method="ML")
Linear mixed-effects model fit by maximum likelihood
Formula: normexam ~ standLRT + (standLRT | school)
   Data: Exam
  AIC  BIC logLik MLdeviance REMLdeviance
 9327 9358  -4658       9317         9328
Random effects:
 Groups   Name        Variance Std.Dev. Corr
 school   (Intercept) 0.090406 0.30068
          standLRT    0.014548 0.12062  0.497
 Residual             0.553656 0.74408
number of obs: 4059, groups: school, 65

Fixed effects:
            Estimate Std. Error t value
(Intercept) -0.01151    0.03978  -0.289
standLRT     0.55673    0.01994  27.917

Correlation of Fixed Effects:
         (Intr)
standLRT 0.365

random intercept, individual and group level predictor

It is possible to enter variables on group level as well. Here, we will add a predictor that indicates the size of the school. The lmer-function needs this variable to be of the same length as variables on individual length. In other words: for every unit on the lowest level, the variable indicating the group level value (here: the average score on the intake-test for every school) should have a value. For this example, this implies that all respondents that attend the same school, have the same value on the variable “schavg”. We enter this variable to the model in the same way as individual level variables, leading to the following syntax:

lmer(normexam ~ standLRT + schavg + (1 + standLRT | school), data=Exam)

> lmer(normexam ~ standLRT + schavg + (1 + standLRT | school), data=Exam)
Linear mixed-effects model fit by REML
Formula: normexam ~ standLRT + schavg + (1 + standLRT | school)
   Data: Exam
  AIC  BIC logLik MLdeviance REMLdeviance
 9336 9374  -4662       9310         9324
Random effects:
 Groups   Name        Variance Std.Dev. Corr
 school   (Intercept) 0.077189 0.27783
          standLRT    0.015318 0.12377  0.373
 Residual             0.553604 0.74405
number of obs: 4059, groups: school, 65

Fixed effects:
             Estimate Std. Error t value
(Intercept) -0.001422   0.037253  -0.038
standLRT     0.552243   0.020352  27.135
schavg       0.294737   0.107262   2.748

Correlation of Fixed Effects:
         (Intr) stnLRT
standLRT  0.266
schavg    0.089 -0.085

random intercept, cross-level interaction

Finally, a cross-level interaction is specified. This basically works the same as any other interaction specified in R. In contrast with many other statistical packages, it is not necessary to calculate separate interaction variables (but you’re free to do so, of course).
In this example, the cross-level interaction between time spend on homework and size of the school can be specified by entering a model formula containing standLRT * schavg. This leads to the following syntax and output.

lmer(normexam ~ standLRT * schavg + (1 + standLRT | school), data=Exam)

> lmer(normexam ~ standLRT * schavg + (1 + standLRT | school), data=Exam)
Linear mixed-effects model fit by REML
Formula: normexam ~ standLRT * schavg + (1 + standLRT | school)
   Data: Exam
  AIC  BIC logLik MLdeviance REMLdeviance
 9334 9379  -4660       9303         9320
Random effects:
 Groups   Name        Variance Std.Dev. Corr
 school   (Intercept) 0.076326 0.27627
          standLRT    0.012240 0.11064  0.357
 Residual             0.553780 0.74416
number of obs: 4059, groups: school, 65

Fixed effects:
                Estimate Std. Error t value
(Intercept)     -0.00709    0.03713  -0.191
standLRT         0.55794    0.01915  29.134
schavg           0.37341    0.11094   3.366
standLRT:schavg  0.16182    0.05773   2.803

Correlation of Fixed Effects:
            (Intr) stnLRT schavg
standLRT     0.236
schavg       0.070 -0.064
stndLRT:sch -0.065  0.087  0.252

– – — — —– ——–

– – — — —– ——–
R-Sessions is a collection of manual chapters for R-Project, which are maintained on Curving Normality. All posts are linked to the chapters from the R-Project manual on this site. The manual is free to use, for it is paid by the advertisements, but please refer to it in your work inspired by it. Feedback and topic requests are highly appreciated.
——– —– — — – –