In a new publication in Acta Sociologica, myself, Mara Yerkes, Lovisa Backman and Jakob Striven reflected on what started out as a commissioned report by DG Employment, Social Affairs and Inclusion. We created a database of ECEC reform studies, in preparation of the revised Barcelona Targets (on childcare).

Five blindspots in reform studies of early childhood education and care (ECEC) policy.

At breakneck speed, with a very short deadline, we delivered the work, and a report (available online). The report had some interesting findings, we think, and demonstrated again how important the accessibility, affordability and quality of childcare policies are.

However, our commissioned focus was very much on reform studies, and in particular impact assessments with very strong research designs aim to isolate the causal effect. The value of such studies is indisputable, but a tradeoff is that these reforms studies tend to be empirically narrow. We kept wondering: “What are we missing?”.

So, we reflected on five blindspots:

Blindspot A: The context-specific nature of reform studies.

The research design of reform studies explicitly seeks to exclude the influence of ‘confounding’ or contextual factors. Consequently, such studies generally focus on a single country, and are less able to explain how to increase effectiveness depends on its interplay with other institutional and structural conditions.

Blindspot B: The limited focus on reducing inequality in the use and benefits of ECEC.

It is well established that parents with higher levels of education and income are more likely to enrol their children in ECEC. Yet, reform studies are silent on the degree of cross-country inequality in ECEC use. Consequently, which aspects of ECEC policy lead to an increase or decrease in inequality in ECEC use remains unknown.

Blindspot C: The focus on short-term outcomes.

A substantial share of studies in the ECEC reform database examined the immediate and short-term effects of policy changes. Only a few reform studies included in the database have examined how long it takes for implemented policy reforms to have an effect.

Blindspot D: The focus on individual-level rather than macro-level outcomes.

Reform studies focus on individual-level outcomes, which facilitates causal inference but overlooks higher-order outcomes and thus the relationship between ECEC and important societal developments.

Blindspot E: Various forms of publication bias.

Very few studies reported on ineffective reforms, and only a handful of studies examined fathers.

Countries known for extensive provision of ECEC were overrepresented. This means that the evidence base for the revision of the Barcelona targets might be the weakest for those countries that might be furthest away from achieving them.

In the conclusion, we critically reflect on our own role in taking on this commissioned work, the importance that policy makers involve academics already at the design stage of the tender/commissioned work, and the importance of methodological pluralism.