When Numbers Aren’t Everything: Defining Qualitative Success

The Journal of Youth Engagement is an online forum of ideas and dialogue for those committed to engaging youth in vibrant Jewish life and living. Join the discussion and become a contributor. By Michael Fuld Defining success is one of the key components to any organization. As a youth professional, when someone asks me, “How did the event go?” it’s almost always followed by, “How many people were there?” As we continue to innovate and provide new and unique entry points for Jewish teens, the number of teens shouldn’t be the only benchmark by which we judge success. In our field, the number of participants that we serve is something that is constantly being scrutinized. We’ve all heard the numbers. 80% of teens leave our movement after b’nai mitzvah. The Campaign for Youth Engagement began with a goal that defines the quantity - that we see a four-fold increase in the number of Jewish youth engaging in Jewish life by 2020. When we look at our best youth-friendly congregations, we often refer first to the number of participants that are coming through our doors. Quantity is an incredibly important factor in determining whether or not a program is successful. However, as we continue to innovate and provide new and unique entry points for Jewish teens, it’s important to also remember that the number of teens doesn’t always tell the full story, and shouldn’t be the only benchmark by which we judge success. On the flip side, quality is something that’s incredibly hard for us to measure. Often times we determine whether something was of high quality based on whether it “felt good,” and whether or not we saw teens looking “engaged.” In my own experience, these words are at best only vaguely defined and are anecdotal, varying from event to event. However, as we look to continue to professionalize our field, what our “gut” says about our qualitative successes isn’t enough. We have to find a way to evaluate in a way that measures both the quantity and quality of our work. Drs. David Bryfman and Joseph Reimer, in their article “What We Know About Experiential Education” lay out a framework that might be a good starting place for us to think about qualitative success. Bryfman and Reimer lay out three different components of Jewish experiential education. In their research of successful experiential educational models, they define a high quality experience as one that

• provides its participants with a social comfort, fun and belonging in a Jewish context (Recreation)
• provides the knowledge skills and attitudes to be an active member of the Jewish community (Socialization)
• encourages participants to undertake the challenge of stretching themselves towards a more complex participation in one’s Jewish life (Challenge)

A basic disclaimer here - while ideally we want all three of these components to make up our experiential educational programs, ensuring that all three fit into every program that you run isn’t realistic either. In the beginning of the group formation, a youth community often needs to spend significant time in pure recreation. This is okay because it lays the groundwork for the other stages. Keeping all of this in mind, I’d like to propose some ideas for collecting and measuring qualitative data. In the NFTY Region that I advise, the Regional Board hands out a one-page survey at the end of every regional event. One of the questions that we ask is, “What’s something new that you learned?” Based on those responses, we can begin to determine if the participants met our challenge goals for the event. We also look for responses that talk about meeting and engaging with new people (recreation), and how Judaism played a role in their experience over the weekend (socialization). These satisfaction surveys can provide some of the qualitative data we are looking for. But surveys only tell a piece of the story. Another way to collect data is through what I’ve come to call reflective learning. A few years ago when I was a congregational youth advisor, our youth group of about 20-25 teens in grades 8-12 met every Wednesday night. At the end of every session, we invited each teen to share one word that summed up their experience that evening. Just from listening to the words that they chose to share, I could gain important information about whether that session was successful. Certain words were indicators of different successes. If the program that evening was largely social, I looked for words that related to group bonding. If the program had more of an educational focus, words that were directly related to the content helped me to determine if the teens found it to be meaningful, and also gave me information as to whether I should directly follow up with an individual teen about the evening’s activity. While it was initially designed to be an evaluative exercise, this end-of-session activity has become a tradition within that community - the activity is something over which the teens now feel complete ownership and is a meaningful component to their youth group experience. Occasionally, I would also follow up with individual conversations with teens after the program to help me get a sense if the goals and expectations I had for the activities met my own qualitative goals. As you seek to enhance your youth programs, I encourage you to set goals not only from a quantitative perspective, but also from the qualitative. This may involve thinking critically about what questions you need answered in order to assess the effectiveness of your program and better engage the young people in your communities. In the end, you’ll see the quantitative benefits as well. Michael Fuld is the Regional Director of Youth Engagement for the New York Area at the Union for Reform Judaism. Mike holds a B.A. and M.P.A. from New York University and is an alum of HUC-JIR’s Certificate in Jewish Education specializing in Adolescents and Emerging Adults. Prior to becoming a Regional Director, Mike was the Senior Assistant Director at the URJ Kutz Camp, and the Senior Youth Director at the Reform Temple of Forest Hills. [[{"type":"media","view_mode":"media_large","fid":"56971","attributes":{"class":"media-image alignright size-full wp-image-41668","typeof":"foaf:Image","style":"","width":"568","height":"96","alt":"Journal of Youth Engagement"}}]]