Revising Room to Read’s Adolescent Life Skills Assessment (ALSA)

By Fernanda Gándara
Director of Research, Monitoring and Reporting for Room to Read's Girls’ Education Program

The Girls’ Education Program at Room to Read equips adolescents with essential life skills that help them complete secondary school, meet day-to-day challenges and make informed, key life decisions in ways that enhance their wellbeing.

Room to Read's Adolescent Life Skills Assessment for Girls (ALSA) is a tool designed to measure girls' life skills in the context of Room to Read’s research and evaluation work. For example, in our evaluation work, the ALSA is used to determine whether students in Room to Read’s Girls’ Education Program are building key life skills at faster rates than their peers. Our Girls’ Education Program staff recently updated our curriculum and life skills framework to ensure alignment with the Big Five taxonomy. Accordingly, Room to Read’s Research, Monitoring and Evaluation team has revised the ALSA framework to meet the new demands of the program and keep up with the field’s best practices.

In addition, Room to Read’s Research, Monitoring and Evaluation team has embarked in a journey to redefine the ALSA development and validation practices in order to create assessment experiences that are contextually meaningful, culturally relevant and that can fully support the gender transformative goals of the program. In this blog entry I will describe the objectives and outcomes of the ALSA revision, and some of the characteristics that make our work unique.

Alignment to the Big Five Taxonomy

Room to Read’s ALSA framework draws from the best available evidence in the measurement of life skills. There is increasing consensus that the Big Five model can be used as an organizer for socio-emotional competencies (Casillas, Roberts, & Jones, 2023). The Big Five is a well-researched model which states that most personality characteristics can be categorized into a five-dimensional structure (Chernyshenko, Kankaras, & Drasgow, 2018). Virtually every social-emotional learning framework can be mapped onto the Big Five[1], and major organizations such as OECD have decided to adopt the Big Five as the basis for their Social and Emotional Skills Assessments. The consistency of results across different contexts (including non-Western cultures[2]) its strong empirical foundation, and the predictive power of each of its domains, explain its increasing adoption in the social-emotional learning measurement field. As some practitioners like to say: "whether you want it or not, you are always measuring the Big Five."

Room to Read's Girls' Education Program in Tanzania.

Therefore, the decision of Room to Read’s Girls’ Education Program staff to simplify our life skills framework was good news for our Research, Monitoring and Evaluation team. The Big Five model is built around five personality dimensions[3] and each of these dimensions has been found predictive of outcomes of interest to our program. For example, the area of conscientiousness — or the tendency for planned behavior and persistent effort in achieving personal goals (Chernyshenko, Kankaras, & Drasgow, 2018) — has been found positively associated with school performance, whereas low self-regulation has been found negatively associated with it (OECD, 2021). A closer and cleaner alignment to the Big Five enhances our ability to predict whether we are on track to improve students' outcomes.

[1] Including widely used frameworks like CASEL.
[2] For example, see McCrae, R. and P. Costa Jr. “Cross-cultural perspectives on adult personality trait development. In Mroczek & Little (2006).  Handbook of Personality Assessment.”
[3] Conscientiousness, Extraversion, Agreeableness, Neuroticism, Openness to Experience

Table 1. Correlations between Big Five domains and outcomes of interest

The updated ALSA Framework

The ALSA Framework was primarily revised to align with the updated goals of Room to Read’s Girls’ Education Program, including our recently revised curriculum. However, the process provided an opportunity for the team to improve the overall ALSA measurement strategy. Specifically, Room to Read’s Research, Monitoring and Evaluation team wanted to identify a framework that was more concise than our previous one, to focus in depth on key domains.

In addition, our team wanted to prioritize constructs that were sensitive to change within our evaluation timeframes, and that were relevant to our participants and to our staff. We also wanted to ensure that our updated ALSA framework was aligned with other prevailing social-emotional learning frameworks and measures in the sector, so that the ALSA could contribute with processes and evidence that were relevant beyond our work.

Room to Read's Girls' Education Program in Sri Lanka.

As a result, we engaged in an iterative process, with colleagues from our Girls’ Education Program around the world. We reviewed historical life skills data from our program and examined life skills frameworks in the countries where we work. We talked to girls during our field work, and we had several discussions with our colleagues and leadership. Ultimately, the ALSA Framework was simplified to focus on five domains: Decision-making, Emotional Resilience, Leadership, Collaboration and Gender Knowledge and Attitudes. While the latter is not a life skill in the traditional sense, it is central to Room to Read’s commitment to gender equality.

Table 2. Updated ALSA Framework

Incorporating girls' voices every step of the way

Beyond the ALSA Framework, Room to Read’s Research, Monitoring and Evaluation team redefined the ALSA development process. Changes were made to increase the contextual and cultural relevance of the ALSA. In particular, the team created processes to develop tools in participatory manners, in ways they reflected the lived experiences of students. The ALSA aims to be gender-transformative by supporting Room to Read’s Girls’ Education Program team to gain critical insight on girls’ skills, knowledge, and attitudes, and by creating reflection opportunities for test takers. Our goal is that taking the ALSA is a learning opportunity in it of itself, where students can take the time to think about themselves and their lives in meaningful ways.

This is one of the characteristics that sets the ALSA work apart: our instruments are now shaped directly from students’ voices. Using innovative techniques, Room to Read’s Research, Monitoring and Evaluation team develops items and scenarios that represent the lives and concerns of students. In addition, students in our program will increasingly play key roles in the piloting and validation of the tools. The feedback that we have received so far is encouraging, and we will continue to make strides to better assessments that support the transformations that we care about.  

Room to Read's Girls' Education Program in Laos. 

An exciting new chapter in Room to Read’s ALSA

Building on work done in recent years and driven by a sincere commitment to cultural relevance and excellence, Room to Read’s Research, Monitoring and Evaluation team will continue to expand and improve research and validation of our new ALSA. Gathering the right type of validity evidence and integrating sociocultural dimensions into all our assessment cycle remains a priority. We hope to share more on our ALSA in the months ahead.

References

Casillas, A., Roberts, B., & Jones, S. (2023). An integrative perspective on SEL frameworks. In J. Burrus, S. Rikoon, & M. Brenneman, "Assessing competencies for social and emotional learning: Conceptualization, development, and applications" (pp. 7-26). New York, NY: Routledge.

Chernyshenko, O., Kankaras, M., & Drasgow, F. (2018). "Social and emotional skills for student success and well-being: Conceptual framework for the OECD study on social and emotional skills." OECD Education Working Papers No. 173.

McRae, R. R., & Costa Jr., P. (2006). Cross-cultural perspectives on adult personality trait development. In D. Mroczek, & T. Little, "Handbook of Personality Development" (pp. 129-145).

OECD. (2021). "Beyond academic learning: First resutls from the survey of social and emotional skills." Paris: OECD.