Where do I begin?

 

2015-03-14_OhNoLogo22-mark3It starts when you notice the local construction projects winding down. Then you’re cut off by a Ford Escape with New Jersey plates and a back full of clothes, books, a pink rug, and one of those chairs made solely of bungee cords. That’s right, it’s back to school season.

Abby and I met to for a pre-season re-vamp of Oh No and you can look forward to two posts a week this year — Mondays and Thursdays. We’re trimming a bit because those Friday posts came upon us awful fast and we want to keep this thing valuable. So without further adieu…

I met with a colleague from across campus this week. She works for a newer (and smaller) office just starting to wrap its mind around how to capture its value to the students it serves. The office focuses on developing an entrepreneurial mindset in our students and supporting student ideas from conceptualization to implementation. To further complicate its assessment process, the office is not yet on permanent funding, thus is under pressure to justify its existence.

I’ve already covered starting over in my Zero to Assessment post, however this conversation yielded a few new questions I wanted to chew on a bit.

What if I don’t know what students are learning from this experience? In the old, dusty textbooks of assessment you’ll find a flow chart looking something like this…

Presentation1

oh, well hello smart art…

This flow chart is helpful if you have clear and measurable learning outcomes, but leaves out instructions for when your outcomes are a bit cloudy. My colleague proposed measuring this through a series of qualitative questions — which, despite my aversion to the labor intensive nature of properly analyzing qualitative questions, seemed appropriate given the situation. And you know what, old dusty textbook I made up to illustrate my point, if an office centered around innovation can’t build a plane while they’re flying it, can any office? That is, if we can’t get an initiative started until we have every detail (e.g., assessment) ironed out, we’ll be missing out on a good number of valuable initiatives.

While I’m complaining about the rigidity of fictitious textbooks, it’s worth acknowledging that neither her nor I was all too sure of how she would analyze the data she’s collecting. It would be great if she had the labor to code each response, but that doesn’t seem likely. I think this is okay. It takes a few cycles to get an assessment process ironed out. Even by simply reading through the responses, she’ll get a feel of what her students are learning and how to better support them.

How do I get students to reply to my surveys? If I ever figure this out, I’m leaving the profession of higher education to hang out with the inventor of stick it notes on an island covered in boats, flat screen TVs, and Tesla convertibles. And, I guess, charging stations for the convertibles.

Very few people know how to do this well, however I’ve come across a few strategies which seem to be working.

-Make it personal. More than half of my job is forming relationships with students. Surveys are one of the times I leverage those relationships. I’ll often send the survey link in an e-mail noting (very briefly) the importance of the data collected in this survey, letting them know that every response is a favor to me (for all of the time I spend e-mailing them with answers to questions I’ve already answered in previous e-mails, this is the least they could do). If you’re sending a survey out to thousands, you can expect a very low return rate.

-Get ‘em while they’re captive. Do you have advising meetings with students at the start of these programs? is there an application to get into your program? Can you (easily) tie in the survey as a requirement for completing the program? I don’t mean to hint that surveys are the only means of collecting assessment data — but they’re direct, effective, and tend to be less labor intensive than other means.

Countdown to College football: 3 DAYS!

Advertisements

Top Tier vs Lower Tier Engineering Programs

2015-03-14_OhNoLogo22-mark3Greetings everyone! In the past few weeks, I purchased a house. The buying process took up much of my free time (though didn’t seem to diminish my blog-ambition). Anyway, with summer in the home-stretch it’s time to get back into it. Abby and I will have our regular weekly posts returning soon. In the meantime, my brain is in the depths of “big thinking” mode. As a former engineer, and an advisor of first-year engineering students, I often find myself thinking about how we educate them.

Modern society holds complex problems. No longer is it enough for engineers to create a widget, present it to the world, and say “go forth, use this widget to make your life easier!” Beyond the question of making things faster or more-efficient, we want to know if a widget intended for a developing nation can be manufactured with local materials at low cost. Will the locals even want to use the widget?

In addition, we’re increasingly reliant on our engineers’ moral decision making. So you want to build a car that drives itself? How much will one of these cost? Can it share the road with human-driven automobiles? If the car finds itself in a certain-impact situation, should it rear-end the car in front of it or veer off onto the sidewalk? Our engineers should have the technical toolbelt to needed to design the widget as well as development required to decide how to use those tools.

This takes me to the state of today’s engineering education. Each engineering degree has a set of technical content expected of a person with that degree. The technical content takes up most of what can be covered in four years. This leaves little room for personal/moral/ethical development in the curricular portion of a student’s experience, and has us (student affairs folks) urging students to get involved outside of the classroom.

Students arrive to college with varying levels of abilities and backgrounds. The most desirable students tend to have credit from Advanced Placement courses or college courses they took while in high school. Assuming they plan to stay for four years, completing college requirements while in high school leaves students with more “wiggle room” in their schedule to pursue interests in addition to the bachelor’s degree. This may mean a minor or even taking fewer credits each term to allow for more extra-curricular time.

But what about the students who do not arrive with AP credit — the students who have the grit and determination, but have not yet studied calculus and may not have stellar standardized test scores? These students are often overlooked by the more competitive institutions. When you’re looking at thousands of applications, you need a means of sorting through them quickly.

We’re left with two tiers of institutions. Students accepted into the lower tier of institutions are not coming in with much college-level coursework completed. Since those students will be expected to have a certain toolbelt upon graduation, those programs are forced to put their resources into the classroom experience. Students accepted into the upper tier of institutions – already having college credit – have the room to take a minor in philosophy or spend time working on research in a professor’s lab.

What’s the difference? Top-tier students have the time for extra-curriculars which force them to juggle the more complex questions, often lead to the personal/moral/ethical development required of today’s engineers. Lower tier students are forced to put their time into learning technical content. They’re acquiring the tools but are not challenged to think about how to use them.

Both of these groups finish with bachelor’s degrees, the signal to employers that they’re ready for the workforce. What happens next? My gut tells me that top-tier students, equipped with experience in how to use the tools, are moving into leadership positions at a higher rate than their lower tier colleagues.

The questions I keep coming back to: Is the personal/moral/ethical development part of what is (or should be) a part of the bachelor’s degree? With the high cost of higher education, are we limiting the potential of some students by not allowing them the opportunity for that development? Does our current educational system, which nudges affluent students toward leadership, exacerbate the gap between the upper and lower socioeconomic classes?

The Dog Days of Summer Orientation

2015-03-14_OhNoLogo22-mark3Somewhere around the time I mentioned our college’s honor code, I looked up at the students sitting in a “U” around the room. Several of them sat with their hands to their cheeks supporting their heads. One was leaning so far back he was nearly sleeping. Our Peer Advisors (undergraduate student staff) were trying their best to stay awake. I thought to myself “self, this is painful.”

I’m a fairly energetic presenter. One of my favorite crowd tricks is to ask them a question: By a round of applause, how many of you are excited to be here? The initial response varies, but it doesn’t matter. I then place both arms out, palms up, waist height; raise my eyebrows and slowly lift my hands. Even the comatose crowds tend to get a respectable clap going. If it’s a good crowd, I’ll even lower one hand while I keep another one up — about half the crowds make it that far.

I’ll then transition into one of my favorite energizers — I avoid the term “ice breakers” because of their inherent negative connotation. I explain the rules of the rock paper scissors tournament. You know the one, where if you win, you accumulate the person you beat and all of their fans (the people they beat) as your fans. By the end of the ice breaker you have two groups raucously cheering on their respective representative. The best energizers are the ones that are easy to understand, and hard to do while looking cool.

Then we break into smaller groups. Everyone leaves cheery and riled up. We get to a classroom, sit down, I turn on the projector, *WHAM — I’ve lost them.

*That was someone’s sleepy head bouncing off the desk

Sometimes I sympathize with Bill Murray in Groundhog Day. Only the difference is that he gets to see the same people every day. The people I meet are there for one day only before I get a completely new crowd. I try the same jokes — every day. Sometimes they work. When they don’t, I’ll say something like “well… it’s pretty clear I need to work on my jokes… or maybe my audiences!” That one works half the time.

For inspiration, I weave into my presentation little tidbits of my story. That is, the experiences I’m comfortable sharing with my advisees about my undergraduate years. You know, to give them something to look up to (yes, that was simultaneously sarcastic and completely serious). I’ll even get our Peer Advisors involved by having them discuss their experiences.

Later on I meet with each of my advisees one on one for a few minutes. We chat for a few minutes about their interests and come up with a set of classes. This is the time when they’re most alive. It seems a fair number of them are too careful to fully engage when we’re with the group of nine.

When we’re preparing for orientation, we spend SO MUCH TIME discussing the same of orientation. How much time do we devote to energizers and ice breakers and how much to presentations. How much to organized time and how much to free flowing conversations.

And the worst part is, when you ask them later on why they joined a particular extra-curricular, or how they knew about our tutoring program, some of them will say “I remembered it from orientation.” It’s as though they all get together and agree one which parts they will each individually remember. Come on guys, we can do this. Frank, you remember the first slide of the presentation. Tina, you pretend to be asleep for the first half, then at the very end ask a question that clearly indicated you were paying attention the whole time. Thomas, you pretend to sleep for the whole thing — only, actually be asleep.

When I make up names, I almost always go with Frank and Tina. I’m not sure where Thomas came from.

It’s orientation season folks. No matter how you lay out the time, just about everything you do will be well-received by a portion of the group, but not everyone. Some students are worried about making friends. Others about if they can handle college. Some wonder if they’ve picked the right one. While I don’t think there’s a perfect way to do orientation. It seems to me that orientation should be a time of meeting people (both students and staff), thinking deeply about their college experience, and learning just enough to make do for the first semester — they’ll pick up the rest.

Campus Climate Survey on Sexual Assault

2015-03-14_OhNoLogo22-mark3In Case You Missed It:

This week, the University of Michigan released the results of its 2015 campus climate survey regarding sexual assault.

Message from the President
Complete Survey Results

The University sent the survey out to a sample of 3,000 students. 67% of the students responded to the survey. 75% of the sample living on campus completed the survey.

Assessing Advisor Conversations

2015-03-14_OhNoLogo22-mark3A few weeks ago I was eating in my favorite Ann Arbor pizza joint (New York Pizza Depot), when a friend of mine walked by the front window. Elaine peered in and decided to have some pizza herself. We started chatting about work and, despite trying to avoid the details, I got talking about assessment.

As I finished my elevator pitch of what we do for assessment, she says “ugh, so you must all be evaluated based on the results of the assessment?” The feeling I had must have been how Jimi Hendrix felt when he realized that, as a lefty, he could just string a righty guitar upside down. Until that point I had shied away from connecting our assessment effort with individual advisors under the idea that our assessment results could not accurately reflect on the efforts of an individual advisor. But in that moment, I simply could not muster any justification to avoid separating certain assessment results by advisor.

Pausing for a moment, I don’t want to get into a discussion on appropriate evaluation advisors. My short response to that is; no, advisor performance should not solely be measured by a few surveys with which we’re ecstatic if we receive a 30% return rate. The evaluation ought balance a handful of factors (and perhaps use more reliable data).

It hit me that we’re using our assessment to inform our delivery of information. Are students unsure what the Bulletin is? Let’s discuss it at orientation a bit more, or perhaps we’ll refer to it when we’re in our advising appointments or responding to e-mail. But there’s more to higher education than supplying information. Much of the value of advising comes out of our conversations with students. Since we each have our own style for advising, wouldn’t it be helpful to know if Steve’s students are getting involved at a higher rate, or if they’re more likely to find a position in the first few months after graduation? And isn’t that a great opportunity for us to have a conversation with Steve about how he approaches his student conversations, allowing us to reflect on our own practice?

My sense is that many of us, on some level, are afraid that our conversations with students are not always helpful and certainly not transformative. Helping a student sculpt a meaningful path for the near and far future is not at all like tutoring him for a math exam. It’s not uncommon for a student to leave my office with me thinking “Did I help that student at all?” Couple that insecurity with the fact that the outcomes we shoot for are often complicated and you’ve created an environment pushing assessment to the periphery.

If you’re reading this post (especially if you made it this far), you probably believe higher education is more than a set of information loaded into the minds of our students — that higher education can help individuals see themselves and the world in a more complex way. Central to this transformation are the conversations students have with faculty and staff. Though difficult to assess, it’s important that we improve the effectiveness of those conversations. While the assessment might not spit out a quantitative number of our conversation quality, it does give us a good opportunity to reflect and re-tool.

Friday: Family Income and College Chances

2015-03-14_OhNoLogo22-mark3It’s Friday!

I hope this starts a trend. Last week, the New York Times asked readers to draw a line predicting how likely a student is to attend college versus their income. FASCINATING.

My line was okay, but before I give away how I did, I want to give you a chance to try it out.

Go ahead, click here.

I love this because it challenges you to really lay out what you think is going on, and then it tells you how you did. No more of the easy “I don’t know what I think… why don’t you tell me what’s right” way of interacting with graphs.

Okay,
.
..

This segment is just filler so you don’t see the results on accident.
.
..

I suppose it’s a good place for a joke:
“I haven’t slept for 10 days… because that would be too long.” -Mitch Hedberg
.
..

I don’t even feel bad about that joke.
.
..

Here’s how I did:

forblogGood news, I’m not the worst!

Though I’m amazed at how straight that line is.

How did you do?

In Case You Missed It: Title IX and Northwestern University

2015-03-14_OhNoLogo22-mark3Back in February, Northwestern University professor, Laura Kipnis, wrote an article titled Sexual Paranoia Strikes Academe.

The article touches on a few sensitive topics — revolving around the impact that the current environment has on professors and their students. Sensitive enough that I’m going to let the article speak for itself.

Shortly thereafter, she received a notice from the university that she was under investigation for two Title IX complaints resulting from the article. She wrote about this experience in My Title IX Inquisition. Which is currently available on the Chronicle of Higher Education’s website, though it looks like it may soon be locked as premium content.

If you’ve been following this at all, you’ve also seen that she has been cleared from these complaints.

Seeing this, I couldn’t help but remember my post two weeks about on the Kennesaw State Advisor Incident. As Bob Dylan would say, the times they are a changin’. Ten years ago, the incident at KSU would not have made it to the internet. Without the video, that story loses much of it’s impact. Before 2011’s Dear Colleague letter, Kipnis’ article would not have prompted in investigation.

As someone who believes a core purpose of college is to challenge ideas, it’s important to me that professors feel comfortable voicing their thoughts and opinions — and if they choose, exploring them in a public forum.

It is also important to me that we as higher education professionals (and as a country) do better when it comes to sexual misconduct.

I think Northwestern dropped the ball on this one. Kipnis wrote an article that may offend some readers. Upon reading the article, a few students at that university felt attacked, and they (rightfully) submitted a complaint. As lack of reporting is a common issue with sexual misconduct, it’s a good thing that these individuals felt comfortable submitting the complaint. But then, according to Kipnis’ article, the university decided it was ill equipped to investigate this case — essentially freeing themselves of the consequences of the decision.

Anything related to sexual misconduct seems to get a lot of press and is a topic that should be taken seriously. My concern here, as other sexual misconduct cases come out, universities will choose to hire outside investigators as a means of mitigating the risk of damaging their image. This effectively works around the whole purpose of the Title IX investigator (a role required of all institutions receiving federal aid — a stand alone position at larger institutions), creating an under-utilized administrator at a time when administration bloat is a concern among law makers, the people who get to decide how — and the extent to which — higher education is funded.

As colleges and universities are held responsible for more, I sense that we’re in for situations where institutional values are put in tension of one another. In this case, it’s the want to fulfill their Title IX obligations with the urge to supporting academic freedom for professors.

Is Your Assessment Any Good?

2015-03-14_OhNoLogo22-mark3This Friday, I had my annual performance review. When discussing my goals for the coming year, I mentioned that I’d like to change the way my we measure my performance. Over the five years I’ve been with the Engineering Advising Center, I’ve pulled in a number responsibilities. And that felt good. I felt important seeing an inbox with e-mails from across the college and university, all pertaining to different hats I wear in my role.

But who cares if you can juggle six tennis balls when the guy next to you is juggling three chainsaws! What I mean is, I’m not doing anyone (my students or myself) any favors by trying to pull in as many responsibilities as possible. This year, I want to be measured by the quality of my two largest commitments — assessment and our Peer Advisor program. When I mentioned this to my supervisor, he asked me how we would know if our assessment is any good. What a good idea for a blog post.

So how do you know if your assessment is any good? My initial thoughts start to resemble the movie Inception.

blogjpeg

I hope that image was from Inception and not one of the Batman movies.

How do you know if your assessment is any good? Here are a few thoughts:

Does it measure outcomes you care about?
Does it accurately measure those outcomes?
Do you ever share the results with others?
Does the data collected inform changes to office processes?

But that list feels a bit empty. You could chase after that second criteria forever — will any set of questions completely capture some of the more complicated learning outcomes? Are there other criteria not critical, yet still of value? For example:

Do students care about the results?
Could the assessment be completed by someone else? (the ol’ “if you get hit by a bus” situation)
Do you ask high-quality, non-leading questions?
Can your assessment plan be explained in only a few minutes?

If you’re not careful, you can create a monster (as I did with the first assessment plan I designed). Recently, I’ve focused on assessment that captures the learning outcomes with as little excess as possible. In a quote often attributed to Einstein “…everything should be made as simple as possible, but not one bit simpler.”

Thus far, I’ve been writing mostly about assessment at the office level. What if we take a step back and look at assessment at the college or institutional level? How powerful would assessment be if office-level assessment were just a part of a larger institutional effort? At the office level, we’re limited to fairly simple learning outcomes; mostly because we tend to have limited interactions with students. But as an institution, we have a great impact. Our students should be learning and growing in ways not captured by simply adding up the assessment efforts of individual offices. Shouldn’t we capture that impact? Shouldn’t this effort include more than simply employment data?

At the institutional level, this hints at the importance of assessment informed by institutional mission. Shouldn’t we try to capture the extent to which we’re meeting our mission?

I may have posed more questions in this post than answers. What attributes of good assessment have I missed?

Friday Standup

It’s Friday! and this bit is from one of my favorites, Steve Martin. Enjoy!

The Kennesaw State Advisor Incident

exterior_image2015-03-14_OhNoLogo22-mark3Every once in a while I come across a story that buzzes so close to my job or my values that I feel compelled to talk about it. A few weeks ago articles blaming over-sized administration for tuition increases got me thinking about where that rise might come from. This week, it’s an article about academic advising.

Last week, Kennesaw State student, Kevin Bruce, paid a visit to his advising office and was asked to leave — well, he was actually threatened with campus security. Rather than recap the story, I’ll pause for a minute to give you a chance to read it for yourself here or here. If you want to just skip to the video, see below.

Once this student posts the video, it takes off a little; makes its way onto the Huffington Post and other sites. The twitter-verse bounces it around. This is where I start to lose track of what’s going on. Reading through his twitter feed, the #ItsBiggerThanKSU hashtag gets some steam. Is race a factor here? Given the racism systemic to our nation, race often plays a factor at some level. But it seems to me the point he’s trying to make is that many students at Kennesaw are not graduating in 4 years and he’s implying that student affairs offices are core part of a student’s ability to make that achievement.

I’m with you Kevin, advising is important. And we need more (and apparently better) of it.

If you dig a little — and I mean just a little — you’ll come across very mean words directed at both Abby Dawson and at KSU. Are they are fault here? To some extent, sure. But let’s not forget a few core values of higher education (get ready, I’m going to speak for an entire industry here): learning and development. If every doofus-moment led to fired employees, we’d end up in a world filled with doofuses and nobody actually working. If her supervisor and the human resources department are not in the process of a performance improvement plan with Ms. Dawson, now’s probably a good time to make that happen. If incidents like this continue to happen, it might be time to look into termination of employment.

For anyone who’s not an advisor, the end of the term is when we are bombarded with student questions. This makes sense; courses fill and students need help finding resolved schedules which move them toward graduation. Some students are persistent with their questions. Often, students will reach out their advisor without consulting online documents at all. I hope that this was just a bad day for her. I hope that in the vast majority of her conversations with students, she’s pleasant. I hope. Regardless of whether the criticisms are justified, you can bet she’s had an awful week. Social media leaves little time for due process.

Fellow higher ed professionals, and everyone else, this is a reality of our lives now. In a moment’s notice, someone might be recording us at work. Is this inconvenient? Yes. Let’s try to remember that everyone we’re interacting with is also a person (and that our students are paying thousands to attend our institutions). Even though when we’re having a bad day, we want to ensure everyone around us is also having a bad day, and even though we sometimes try to pretend our busy times are “learning opportunities” for our students, it’s worth taking a second, minding our tone, and saying: “Hey man. I am so sorry, I can’t help you right now. Can you e-mail this to me and I’ll get back to you as soon as I can?”