Guest Post - Civic and Character Development: Good Data Starts with Good Measures

In June, CIRCLE launched a new blog series to connect research and practice and publish views from the field. We were thrilled by the response to our first call for guest posts about impact measures from researchers and practitioners. Below is the fourth post in this series, which provides and explains an instrument to capture the development of civic engagement during childhood and adolescence. Please also see the first, second, and third posts. Join us on Twitter and Facebook to discuss the content and implications, and keep an eye out for future posts and a culminating event for this series.

The fields of youth civic and character development are no longer in their infancy. Yet there remains an urgent cry for robust measures that tap the development of civic engagement and character strengths. These measures need to be reliable, valid, theoretically consistent, and appropriate for youth in middle childhood and adolescence. After all, good data starts with good measures.

The Roots of Engaged Citizenship Project (www.civicroots.org) was launched in 2012 to study the developmental roots of active participation in communities and society. In particular, we’re interested in understanding the role developmental competencies (e.g., perspective taking), character strengths, and ecological assets (e.g., relational support) in childhood and adolescence play in nurturing engaged citizenship. As a foundation of this effort, we developed and rigorously tested a set of developmental measures that tap young people’s civic engagement and character strengths in Grades 4-12. Our iterative measurement development process yielded a strong set of measures, while also revealing both challenges and opportunities.

Challenges

What do we ask of whom?

Deciding what to ask to youth of different ages is difficult, particularly when you’re most interested in assessing developmental change. The cognitive demands of a survey are greatest for younger youth, who are also typically slower readers and less experienced survey takers than older youth. Shorter, simpler surveys are essential to minimizing the amount of missing data from younger participants.

“Planned missingness” designs are increasingly common for surveys that are responsive to the developmental abilities of youth and efficient in maximizing the amount of information gathered. With these designs, a core set of items is asked of all youth but only two-thirds of youth are asked other items.

Yet even these designs require difficult decisions about what to include for whom. In our work, constructs that are widely assumed to develop later were cut entirely from the elementary and/or middle school versions of the survey (e.g., protesting). While this strategy proved effective in meeting the needs of our participants, it also limited our ability to empirically demonstrate the emergence of these civic and character constructs in our longitudinal design.

Is that clear to you? What about you?

Identifying and developing survey items that resonate equally with a 4th grader and a 12th grader pushed our team to carefully evaluate every word choice for readability and comprehension. This screen eliminated words like protest and thrift, as well as common abbreviations like “e.g.”. Separate rounds of semi-structured and cognitive interviews with youth pinpointed readability issues and reinforced the need for short items free of colloquial phrases (e.g., “take a stand”) and the limited use of introductory clauses. Integrating this early feedback, we attempted to write all items so they could be understood by our youngest participants. Items were further revised based on teacher feedback and questions asked during the first round of surveys.

What happens when complexity increases with age?

The motivating force and constant challenge of developmental research is figuring out how to capture change (and/or stability). Age brings complexity— the welcome product of cognitive growth and increasing independence—which may lead some civic and character measures to become more multifaceted, varied, and differentiated. At the same time, adolescents’ increased cognitive capacities allow for a comprehensive and coherent understanding of more complex and abstract civic concepts (e.g., civic values, critical consciousness).

For developmental phenomena where the idea becomes more complex, increased complexity can be handled by adding items to constructs for youth at older ages. There are statistical ways to appropriately compare constructs with different numbers of items over time (or across age groups). However, doing so is conceptually complicated, as the meaning of these constructs may change with the addition of items. For example, the addition of items measuring more confrontational forms of political engagement (e.g., protesting) to surveys of older youth changes the meaning of what constitutes political engagement.

Another approach is to administer items that tap the full range of developmental complexity to all participants. This approach may reveal variation in the dimensionality of the constructs across age groups or time, which can be substantively interesting. Of course, the biggest issue with this approach is that it’s hard to discern whether younger kids understand the items that are beyond their current developmental abilities and/or experiences or whether they simply don’t do the behavior. This often results in lower scale reliabilities for younger participants—even in our measures.

There is no simple answer; however, a combination of the above strategies might prove most fruitful. This is what we tried to do.

Opportunities

Struggling with each of these challenges has elevated our thinking about young people’s civic and character development and yielded a comprehensive set of measures that appear valid for a wide age range. While these measures will continue to be refined and built upon by ourselves and others, they are also an opportunity to contribute to two important advancements in the field.

Build a cohesive theory of civic engagement

Good developmental measures are a requirement of good longitudinal research. Strong longitudinal studies of civic engagement will collectively allow the field to test a cohesive theory of civic engagement. In this, we could test questions like:

• How do civic values, behaviors, and skills change across childhood and adolescence?
• Does civic engagement develop in tandem with other social, emotional, and cognitive developmental competencies?
• What is the association between civic and character development?
• How do the interactions kids have with adults and peers across contexts accumulate in promoting civic engagement? Does this vary by age? Race? Ethnicity? Gender? Socioeconomic status?
• Does civic engagement promote positive youth development by enhancing youth competencies and ecological assets?

It’s answers to these kinds of questions that will propel the field forward and more squarely situate civic engagement in the field of developmental science.

Inform practice

Parents and practitioners do the real work of nurturing kids’ civic identities and character strengths day in and day out. Efforts to mobilize and support adults in this work often leverage data to inform programming and identify best practices: What works? What doesn’t? What are they doing really well? Where could they use additional support?

Strong measures allow us to identify with confidence what works and for whom and the long-term developmental benefits of investing in small everyday practices at home, in the car, in the classroom, on the field—wherever —that set kids up for engaged civic life.

Measurement Toolkit

The measures that were developed as part of the ongoing Roots of Engaged Citizenship Project are organized around five domains: (a) civic beliefs and values, (b) civic behaviors, (c) civic skills, (d) civic socialization, and (e) character strengths. You can access detailed information on these measures and their psychometric properties through the Youth Civic and Character Measures Toolkit available here: http://www.search-institute.org/downloadable/Youth-Civic-Character-Measures-Toolkit.pdf