influence.ME depends on lme4. As the authors of lme4 have completely revised the inner workings of lme4 and are currently releasing version 1.0, influence.ME required an update to maintain forward compatibility with lme4. I just uploaded version 0.9.3 of influence.ME to CRAN, which will be available soon. This version should work with the new lme4, but if you happen to run into any problems please contact me.

Nevertheless, in the social sciences multilevel models are often used to analyze data on a limited number of groups with per group a large number of observations. A typical example would be the analysis of data on individuals nested within countries. By nature, only a limited number of countries exists. In practice, typical country-comparative analyses are based on about 25 countries. With such a small number of groups (e.g. countries), observations on a single group can easily be overly influential to the outcomes. This means that the conclusions based on the multilevel regression model could no longer hold when a single group is removed from the data.

In our recent publication in the R Journal, we introduce influence.ME, software that provides tools for detecting influential data in multilevel regression models (or: in mixed effects models, as these are commonly referred to in statistics). influence.ME is a publically available R package that evaluates multilevel regression models that were estimated with the lme4.0 package. It calculates standardized measures of influential data for the point estimates of generalized mixed effects models, such as DFBETAS, Cook’s distance, as well as percentile change and a test for changing levels of significance. influence.ME calculates these measures of influence while accounting for the nesting structure of the data. The package and measures of influential data are introduced, a practical example is given, and strategies for dealing with influential data are suggested.

With this publication, and of course with the software that was available for quite some time, we hope to contribute to a better usage of multilevel regression models. The provided example and guidelines were geared towards applications in the social sciences, but are applicable in all disciplines.

On a final note, the editorial of the R Journal describes how this journal is quickly ranking up in the degree of (academic) recognition it receives:

Thomson Reuters has informed us that The R Journal has been accepted for listing in the Science Citation Index-Expanded (SCIE), including the Web of Science, and the ISI Alerting Service, starting with volume 1, issue 1 (May 2009). This complements the current listings by EBSCO and the Directory of Open Access Journals (DOAJ), and completes a process started by Peter Dalgaard in 2010.

More information on our influence.ME software is available on this website.

Download the paper from the R Journal
Rense Nieuwenhuis, Manfred te Grotenhuis, & Ben Pelzer (2012). Influence.ME: tools for detecting influential data in mixed effects models R Journal, 4 (2), 38-47

This manual was written specifically as an introduction for the quantitative social scientist. To my opinion, R-Project is a magnificent statistical program, ready to be accepted and implemented in the social sciences. The flexibility of this program and the way data are handled gives the user a sense of closeness to and control over the data. I think this inspires users to analyze their data more creatively and sometimes in a more advanced way. At present, this manual has a strong focus on multilevel regression techniques. Reason for this is that in R-Project it is very easy to estimate these types of models, even the more complex variants. The more basic and fundamental aspects of R-Project are introduced as well. All this is done with the needs of the quantitative social scientist in mind.

Of course, this manual it provided without any warranty. Please realize that I wrote it almost four years ago.

I’d love to hear any feedback for (future) improvements!

Download:

Applied R for the quantitative social scientist

A .PDF manual containing many of the R-Sessions material is available here.

Introducing R

• Introducing the R-Sessions
• What is R?
• Why R-Project?
• Getting R-Project
• Getting Packages
• Getting Help

Data Manipultion

• Most Basic of All
• Data Structure
• Getting Data into R
• Data Manipulation
• Conditionals
• Tables

Graphics

• Basic Graphics
• Overlapping Data Points
• Multiple Graphs
• Intermediate Graphics

Mixed Models

• Multilevel Model Specification: LME4
• Generalized Multilevel: LME4
• Helper Functions
• Extractor Functions: NLME
• Multilevel Model Specification: NLME

Influence.ME: Tools for detecting influential cases in mixed models

• Introducing Influence.ME
• Influence.ME Overview
• Influence.ME Manual
• Influence.ME: Simple Analysis
• Specification of the intercept
• Influence.ME: Change-log

Some small functions I wrote

• Forward.lmer: Basic stepwise function for mixed effects in R
• Combining Output in a Single Table
• Select Nested Observations with Equal Number of Occurences
• Visualizing Missing Values

Books

• Book: Mixed Effects Models in S and S-Plus (Pinheiro & Bates, 2000)
• Book: An R and S-PLUS Companion to Applied Regression (Fox, 2002)
• Book: Data Analysis using Regression and Multilevel / Hierarchical Models (Gelman & Hill, 2007)
• Book: Introductory Statistics with R (Dalgaard, 2002)

Various

• Running R Twice on Apple Mac OS X
• Impressive R Speeds
• Text Editors for R: Textmate
• Text Editors for R: Internal Editor on OS X>

He asked me for a piece of code that creates a subset of the data that is balanced, i.e. all observations that are nested within equally sized groups. Or, as an alternative, all observations nested within groups with at least a minimum number of observations.

I solved it the quick and dirty way, and the solution involves creating additional variables, a new data.frame, and merging. It sure can be done much prettier, but it works.

So, I share it below:


id <- c("a", "b","b", "c","c","c", "d","d","d","d", "e","e","e")
y <- c(3,4,3,2,4,5,6,5,6,7,5,4,3)
df <- data.frame(id, y) # setting up original data.frame

subset(df.new, fre==3) # subsetting
subset(df.new, fre>3)


Intended to be a customized solution, it may have grown to be a little more. forward.lmer is an early installment of a full stepwise function for mixed effects regression models in R-Project. I may put in some work to extend it, or I may not. Nevertheless, in a ‘forward sense of stepwise’, I think it can be pretty useful as it is. Also, it has an interesting take on the stepwise concept, I think.

Most stepwise functions (as far as I know) take a base model and a bunch of variables, and then iteratively adds and/or subtracts some variables, according to various criteria, to come to the best fitting regression model. All very interesting, but how to deal with interaction variables? And moreover: most existing functions do not work with mixed effects models ((I use the term ‘mixed effects model’ to describe this stepwise function to refer to what is often referred to as hierarchical or multilevel regression models, as well)).

Built around the lme4 package in R, forward.lmer provides a forward stepwise procedure to mixed effects models. Also, it allows the user not only to enter single variables to models, but also to do the same with blocks of variables. This opens up many options: users can add the complete interactions at once (i.e. both the original and the multiplicative terms), or add these consequetively. Future development will focus on additional selection criteria for interactions, such as the criterium that at least the multiplicative term needs to be statistically significant.

The user provides a starting model and a set of variables to evaluate. The procedure then updates the starting model with the addition of every single variable (or block of variables). The models are ordered based on their LogLikelihood (other criteria, i.e. BIC and AIC following soon), after which the best fitting model is evaluated against one of two criteria. The first criterium is that at least one of the added parameters is statistically significant. The other criterium is that the addition of the parameters together is statistically significant.

There are several parameters to be specified:

• start.model: The starting model the procedure starts with. This can be a null-model, or a model already containing several variables. All lmer-models (i.e. logistic, poisson, linear) are supported.
• blocks: a vector of variable names (as character strings) to be added to a model. Several variables can a concatenated within the same character string, so that these are added as a block of variables, instead of a single variables at once.
• max.iter: The maximum number of variables that are evaluated. If max.iter is reached, the procedure stops without adding more variables.
• sig.level: This is the p-value against which it is tested whether the new model fits better than a base model. Either sig.level or zt needs to be specified, but not both at once.
• zt: This is either the T or Z value that is used to test whether (at least) one of the added variables is statistically significant. T values are used for linear regression, Z values for binary response models.
• print.log: Should a log be printed? The log contains information on which variables (and on which criteria) were added in each step.

The forward.lmer function returns the best fitting model (according to the given criteria). Of course, one can use this resulting model as a starting model for a new stepwise procedure.


forward.lmer <- function(
start.model, blocks,
max.iter=1, sig.level=FALSE,
zt=FALSE, print.log=TRUE)
{

As always, you're invited to use this function, or to adapt it and use that. However, it is required to make mention of this function and its author. Additionally, since I intend to continue working on this function (perhaps even evolve it to a 'package' on CRAN), I would love to hear about any experiences in using it.

There are various ways of getting your output from R to your publication draft. Most of them are highly efficient, but unfortunately I couldn’t find a function that combines the output from several (lmer) models and presents it in a single table. lmer is the mixed effects model function from the lme4 package. So, I wrote a simple function that does exactly that.

Using it for a specific purpose, it is not a general function or something, but it can easily be adapted for use in other settings. Here it goes:


require(lme4)
require(mlmRev)
require(lme4)
require(mlmRev)

write.csv(combined, "combined.csv", na=" ")


In this example I estimate four mixed effects models, which are concatenated in a single object 'm'. The function itself is called 'combine.output.lmer', and is used on the object 'm'. The output is a data.frame with the variable names in the first column. Not-estimated parameters in models are indicated by 'NA' in their respective columns. By writing the 'combined'-object to an external file, the NA's are lost and the file can be read into other software, such as Open Office Spreadsheet or Excell. Use the xtable-package to get it in your latex document.

UPDATE
I updated and improved the code somewhat, for I wasn't satisfied with the results. Now the code adapts to the number of parameters derived form the models' summary, allows to add your own names to the columns, and, most importantly, also reports the random slopes.

Please note: due to the internal matching procedure, errors may occur when the same variable is random 'within' more than one other variable. This is only the case when other variables are random within each nesting factor as well.

It always takes some time to get a grip on a new dataset, especially large ones. The code-books are often as indispensable as they are massive, and not always as clear as one would want. Routings, and resulting and strange patterns of missing values are at times difficult to find.

I found a nice way to plot missing values, using R. Basically, I thought it would be nice to calculate the percentage of missings on each variable, and do so for each year represented in the data. These numbers could be visualized using a levelplot(), which resulted in the graph below.

In this example I used a small subset of variables from the cumulative file of the General Social Survey, which is freely available from the web. I used this syntax:


testing.NA <- matrix(ncol=26, nrow=21)
for (i in 1:dim(GSS)[2])
{
testing.NA[i,] <- tapply(GSS[[i]], GSS$year, function(x) sum(is.na(x)) / length(x))
}

levelplot(testing.NA,
scales=list(x=list(rot=90)),
main="Percentage missing values on variables in GSS",
xlab="Variable",
ylab="Year")


First, I defined the testing.NA matrix, using the number of years and variables. Then, in a loop, I calculate the percentage missing values, basically using is.na() and length(). I assign dimnames to the matrix and use the levelplot() function from the lattice-library to plot the matrix. That's it, easy does it.

But: does it help? I think it does. Of course, all this information can be gained from the code-book, and needs to be verified. However, it does give us some immediate notes on the availability of these variables. For instance, we see that in the first few years, the abany variable is missing, whereas other variables on abortion don't. When creating scales this needs to be taken into account, not to lose the complete data on the first few years. The speduc-variable (spouse's educational level) has a high number of missings, as does the denom variable. This, however, makes sense: not everybody has a spouse and the denom-variable only applies to protestants. Finally, this graph gives some pointers on a change in survey-strategy from 1988 onwards regarding the items on induced abortion. The percentage missing values increased sharply at that point, and does so for all abortion-related variables.

This graph does not tell what exactly happened, but does provides nice pointers on what to look for when reading the code-book.

The procedure is very easy, and it works like a charm. Mind though that, obviously, it drains your computers’ resources heavily, so performance of each instance of R-Project decreases slightly at least. For that to change, we would need dual-hard disk laptops, and dual-RAM laptops and the such. Dual laptop-laptops basically.

Back to running R-Project twice. Just start R-Project as usual. Then go to your applications folder and secondary-click on the R-Project app. Select ‘duplicate’, and there you are: an app named R copy emerges. Start this as usual and start working.

in the image below you see two instances of R-Project running. The first is working on a heavy-weight function that results in some output every hour or so and runs 96 times. In other words: it takes ages. However, it stores the output in an external file, and since each little bit of output needs some post-estimation before being interpreted, I can use the second instance to load that data and examine it (not shown).

Although you don’t need to re-install packages, the only thing I did not (yet) find out how to do is to share resources between these two instances of R-Project. Being able to share variables, models, and such would be great. Ideas anyone?

– – — — —– ——–

• Discuss this article and pose additional questions in the R-Sessions Forum

– – — — —– ——–
R-Sessions is a collection of manual chapters for R-Project, which are maintained on Curving Normality. All posts are linked to the chapters from the R-Project manual on this site. The manual is free to use, for it is paid by the advertisements, but please refer to it in your work inspired by it. Feedback and topic requests are highly appreciated.
——– —– — — – –

Yesterday, I received my new Apple MacBook. It’s running a Core 2 Duo at 2.4 Ghz and it’s fast. Really fast!

Apparently, it’s very cool to show of the speed of R-Project on your system. Optimized .DLL files help to speed up your R on Windows systems (and possibly other systems as well) with respect to matrix transformations, which has led to enormous speed increases. So, let’s perform a speed-test of our own.

First of all, in the syntax below, the Matrix package is activated, using the require() command. Since we will be creating random data, we set the seed in order to receive the exact same data every time the test is run. This is done with set.seed(). The next line creates a matrix X, which in the last three lines is manipulated in different ways.

To test how long this takes, we enclose that matrix operations in the system.time() function, which clocks the operation.


require(Matrix)
set.seed(123)
X <- Matrix(rnorm(1e6), 1000)
system.time(for(i in 1:25) X%*%X)
system.time(for(i in 1:25) solve(X))
system.time(for(i in 1:10) svd(X))


This results in the following output:


> X <- Matrix(rnorm(1e6), 1000)
> system.time(for(i in 1:25) X%*%X)
user system elapsed
8.306 0.591 5.031
> system.time(for(i in 1:25) solve(X))
user system elapsed
8.933 1.331 6.684
> system.time(for(i in 1:10) svd(X))
user system elapsed
36.989 3.665 33.384


WOW! This is the fastest I've seen in real life, even faster than some of the desktops that I know people currently work with (i.e. my own). I'm however very sure that it is not the fastest possible, not to say compared with how fast future calculations will be.

Additionally, in the near future my MacBook will be configured with 4 Gb RAM, so I'm curious to find out whether or not this will result in an additional speed increase. I expect, however, most benefit from the additional RAM when doing binomial mixed effects models, so of course expect a comparative benchmark on that one as well as soon as the new RAM arrives.

So, in the meantime, you can use this code to do some benchmarks yourself, on various computers. Please post the results here, or discuss them in the R-Sessions Forum.

UPDATE:
I also tested my old Powerbook G4 (1.5 Ghz, 1.25 Gb RAM):

> set.seed(123)
> X <- Matrix(rnorm(1e6), 1000)
> system.time(for(i in 1:25) X%*%X)
user system elapsed
34.661 1.590 47.528
> system.time(for(i in 1:25) solve(X))
user system elapsed
37.184 1.656 51.516
> system.time(for(i in 1:10) svd(X))
user system elapsed
247.694 11.258 331.979


- - -- --- ----- --------

• Discuss this article and pose additional questions in the R-Sessions Forum

- - -- --- ----- --------
R-Sessions is a collection of manual chapters for R-Project, which are maintained on Curving Normality. All posts are linked to the chapters from the R-Project manual on this site. The manual is free to use, for it is paid by the advertisements, but please refer to it in your work inspired by it. Feedback and topic requests are highly appreciated.
-------- ----- --- -- - -