The Narrative to the Numbers: Focus Groups

2015-03-14_OhNoLogo22-abby3Assessment may use many sources from which to collect data (e.g., surveys, pre/post-tests, etc.) – especially when you’re working with assessing learning (due to its the blessed messiness). Conducting a focus group can be a good way to collect the narrative that complements or explains the quantitative data; focus groups breathe life into an otherwise typically survey-based methodology (read: fancy way to say “the ways I plan to collect data”).

Not to mention, focus groups also tend to provide a flare in the data that surveys and other written methods have a harder time conveying: inspiration. I can collect all the survey data I want and find great trends and solid longitudinal results. But the minute I actually hear the same thing out of one student’s mouth, WHOA – talk about a call to action!

venus gif - 3 loop
Ok, for those who don’t know, I used to be a pretty…um…dedicated (read: obsessed 13 year old) Sailor Moon fan. But anyway…

Focus groups are great (hooray for actually hearing the student voice!) but my goodness can they be a BEAR to plan, coordinate, collect the information, synthesize the information, follow up with participants, assign compensation (when applicable), triangulate the findings with other data, etc. Again, WHOA – talk about exhausting.

moon - 3 loop

I think some of this comes from the dichotomy you’re trying to achieve with a focus group: structured, yet open. You want to construct an environment beforehand that gets your participants in the appropriate mindset to give you the feedback about the specific topic you’re wanting (because you don’t want their feedback on anything and everything; you are seeking their feedback about a certain kind of thing), but while also allowing the environment enough openness to get actual feedback (because you don’t want to structure it so much that their feedback is just a regurgitation of what you -the facilitator- already knows/thinks). Finding the balance between structured yet open with a focus group seems more like an ideal that you’re always seeking to achieve.

I recently conducted a focus group to gauge student perspectives our office learning outcomes. I still have a few steps to go in the process, but here are a few early reflections that I have about preparing for the focus group and creating the environment with the group, and then, questions I need to think about for next time:

Things I’m glad I considered beforehand:

  1. Clarify the aim of the focus group.
    • For me, I wanted student feedback on: [1] their usage and engagement with our learning outcomes, [2] the benefit of the learning outcomes to students, and [3] clarifying #1 and communicating #2 to students and the College community.
  2. Understand more precisely what I hoped to get out of their feedback.
    • I had eight discussion questions that I had them work through that came from the intended aim of the group.
  3. Anticipate what will get the group off track, and account for that.
    • I thought they might want to talk about lots of amazing ideas that, with our time and resources, could never be done. So I addressed that with them prior to starting.
  4. Decide the amount of context to give without overly directing their feedback.
    • I erred on the side of giving them little background because I wanted to hear as many new and different ideas as possible. So I shared our larger office vision for these student learning outcomes, but did not explain HOW (in detail) we hoped students would achieve them.
  5. Use different methods to engage everyone.
    • I had the students provide feedback to me in the large group, and then break into smaller groups to accommodate different learning and communication styles to discuss the discussion questions.

But next time, I need to ask myself:

  • What visuals should I provide to convey our aim with the focus group and the topic they’re focusing on?
  • How should I use facilitation to encourage innovation and mitigate tangents?
  • How should I handle the conversation-dominators? How to better engage introverts?
  • Which other facilitators should I bring into the focus group to provide greater perspective on the participant feedback?
  • And, just overall, how can I do this better next time?

I’m still at the early stages with this focus group. Next up for me will be the fun part – analyzing the feedback! Woot woot! But for now, I want to hear from you. Does any of this resonate with your experiences with setting up focus groups? What were some of your successes? What are things that you would change next time? I want to hear from you! Pretty please comment below.

Advertisements

3 thoughts on “The Narrative to the Numbers: Focus Groups

  1. I’m just wrapping up my “Year of the Focus Group” — 16 groups for one topic and 13 for another. One of the topics was of interest to us but not necessarily to the students, so we had trouble getting people to sign up. The other dealt with a “hot topic” on campus, so we had no trouble filling the groups. I learned a lot from the two parallel experiences, and there are definitely things I would do a bit differently the next time around. One of the big challenges in the “hot topic” groups was that many students came looking to get information/answers rather than to share their own experiences. Because of the nature of the topic and the fact that some people explicitly said they came to listen more than to talk, I didn’t try to draw out the quiet folks as much as I would have otherwise. A bigger challenge for me was figuring out how to handle questions that students asked of the facilitators. I know that the best approach would have been to ask them to hold their questions until the group was over, or to make a note of the questions and answer them at the end. Unfortunately, that just didn’t feel good in the moment, and it seemed like the discussion could have been as inappropriately influenced by our not answering the questions, especially when the students’ concerns were about lack of transparency from the administration. I ultimately decided that it was okay to answer brief factual questions just to keep the discussion from getting hung up on a minor point of fact, but to deflect questions of a more complex or opinion-based nature.

    Another lesson learned was the value of having consistency in note-taking and facilitating. We had several different facilitators and note-takers, each with their own style. Although I’d given them some broad guidance about how to fill those roles, I learned that I should have been a lot more specific in order to cut down on the variation. Given that we had a lot of groups to conduct in a limited span of time, we opted not to let the perfect be the enemy of the good. Even with the weaknesses of our process, we gleaned valuable information that will be very helpful to our efforts moving forward. It wouldn’t hold up under the scrutiny of a doctoral defense, but that wasn’t our aim. Sometimes we just have to do the best we can with what we’ve got. I think interviews (individual or group) are an incredibly important component of a good assessment strategy, and everyone learns to do them better over time, with practice. As with anything, it’s important to reflect on what worked and what didn’t so that we can apply the lessons to the next opportunity.

    Like

    • Thank you so much for your quick and thoughtful reflection. Wow – that definitely was the year of the focus group for you. I took many great points from your comment, specifically: [1] “don’t let perfect be the enemy of good” (so easy to do when working with assessment! great point!), [2] the importance of preparing facilitators and having consistent note taking (I never would have thought of that but you’re so right!), and [3] discerning if students are there to contribute or get answers (you pointing this out has made a few things from the group make new sense to me now). Thank you again for contributing – I learned a lot!

      Liked by 1 person

  2. Pingback: Focus Groups: How Do We Get Students to Participate? |

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s