So you’ve designed a new workshop, which you’ve guaranteed will bring up your student’s grades by a full letter! You spent weeks preparing the workshop, you gave the workshop, and now the grades are coming in. Did your students improve by a letter grade? That’s an easy calculation using descriptive statistics. Simply average last term’s GPAs among the students in your group and compare that to this term’s average.
But did your workshop really make a difference? Let’s say you want to know if your workshop can really be said to bring up student grades (or if you just got lucky). This is where inferential statistics come in! Remembering Abby’s post from last week, the sample in this case is the students in our workshop and the population is all of the students at the university.*
T-tests can tell you if a given experience (e.g., a workshop) impacts the mean score (e.g., grades) for a given student. When you hear folks describing “pre” and “post” tests, this is likely a scenario where t-tests are helpful.
Regression can tell you if (and the extent to which) two variables are connected. For example, if you want to know if a students grade in calculus can predict their grade in physics. A regression analysis will tell you is there’s a relationship and how strong that relationship is. This test is appropriate when both variables are quantitative.
Writing this post, it occurs to me that 1) trying to explain this stuff gets complicated FAST. 2) I’ve lost most of the details I learned in my statistics courses.
The main idea here is that we have mathematical tools which can calculate for us how likely it is that a given experience or situation can predict another experience or situation. Want to know if career counseling helps students find a career? Statistics can answer that.
The downside here is that not every statistic-based conclusion can be trusted. Much like Harry Potter’s wand,** this only works when a person knows (or at least sort-of knows) what they’re doing. I’ve noticed that it gets cold a few weeks after students arrive on campus — it’s the STUDENTS who cause winter!!!!
In most cases, assessment doesn’t require statistics (beyond mean, median, etc.). As intelligent people with a limited amount of time on our hands, it’s okay to look at some numbers, make conclusions, and update our office processes. That said, if you happen to have someone on your staff with the time and the background, you’re in luck — you can start making conclusions about the effectiveness of your department practices. This allows you to identify the practices making a difference. In a time when resources are tight, the ability to carefully prune our student affairs bonsai trees (you’re welcome for that metaphor) will become more and more important.
*This assumes your workshop was attended by a random group of students among the university. If, for example, the workshop was only advertised to engineering students, then your population would be engineering students. In short (and probably under-sufficient), your population is the group from which the sample comes.
**This is just an assumption. I haven’t read any Harry Potter, but I assume he doesn’t want other people messing around with his wand.