myU OneStop

What's Inside

Related Links

Creating a data-driven culture

The challenge with data is in the sharing of it

By Adam Overland

Survey Wordle 165

October 25, 2011

"The University is committed to becoming not only more efficient, but also more effective in everything it does. To do this, we need to know how well what we're doing is working," says Peter Radcliffe about the new resources on the website of the U's Office of Institutional Research (OIR).

Radcliffe is executive director of planning and analysis, and works to help the University connect decisions and data. For example, Radcliffe and members of the Office of Institutional Research have been working with the State of the Student Committee, a group convened by Laura Coffin Koch, associate vice provost for undergraduate education, to examine ways to educate faculty and staff about the demographics, characteristics, attitudes, and behaviors of undergraduate students. The group asked OIR to find a way to make a sea of survey data gathered by OIR, colleges, and departments across the U more navigable and useful.

The challenge with data is that somewhere, someone has the answer, but without a way to get to it, it can't be shared. Think of the OIR website as a boat. And faculty and staff can move the information to where it needs to go—be it the high seas, or the beach.

"What we are trying to do is to make data available to people who need it," says Koch.

Survey says
Some of the information that can be found on the site include reports on the relationship of freshman seminars, study abroad programs, student engagement in the community, diversity trends, of need-based scholarships and financial aid, and more with student outcomes. There's also information, beyond student data, such as the Pulse survey of faculty and staff satisfaction, a survey titled "Staff work satisfaction: An analysis of the unexamined majority in Academia," other human resource data, and tuition related data. Making that data available for use in making policy changes and progress is the ultimate goal of the site.

Freshman seminars, for example, says Ron Huesman, associate director of OIR and the author of a number of its reports, are expensive to conduct given their format—senior faculty working with a small number (around 15) of students. So is that money being spent wisely and effectively?

Huesman says research shows it is. For example, compared to freshmen who did not participate in freshmen seminars, those who did participate in freshmen seminars report higher frequencies of having a class in which a professor knew their name and reported higher levels of a sense of a belonging.

"The undergrad experience has improved dramatically in the past several years…the Welcome Week program, student engagement, freshman seminars—there is a whole list of things that academic and student affairs offices have done to make the student experience better here. Those outcomes are again reflected later with retention and graduation rates," says Huesman.

As the undergraduate experience continues to improve the U is getting better performing students, in ACT scores and graduation rates, and in turn, better performing students are seeking out the U.

Steering the U's mission
"Better" might not just mean smart kids, but also engaged kids—kids who give back to the community. For example, survey data show that one-third of undergraduates performed 1-10 hours of community service in 2009–10, 23 percent did 11-20 hours, another 23 percent did 21-50 hours, 13 percent spent 51-100 hours, and 8 percent performed more than 100 hours.

With more than 30,000 undergrads, says Huesman, this volunteer work has a significant impact on the community. Beyond that community benefit is a benefit to the students themselves; another study identified a relationship between participation in community based learning activities and first-year achievement in college (Serving to Learn…).

"One part of the mission of the institution is to promote public engagement," says Huesman. "So, are we doing it or not? Are students finding it on their own"? (52 percent are.) Are they getting it from programs that we're offering? (Yes, 34 percent through another organization on campus, and 26 percent through a related class.) We need to know where we are to know where we're going," says Huesman (Community Engagement Results & Analysis).

Some results may seem surprising. For example, data about student loans.

One report, "Priced Out?: Does Financial Aid Affect Student Success?" concludes that students who borrow during their first semester are more likely to transfer or drop out. "This is extremely important as it directly undermines institutional retention goals," says the report. "To overcome this challenge, the University could potentially benefit from first-year programs aimed to help student borrowers make more informed financial decisions."

Sharing data
Koch also envisions the site as being a place where faculty and staff conducting related-research can upload or link to research that they have done and that would be valuable to the U community.

Huesman agrees that such sharing could be very beneficial on a number of levels. One is a potential consolidation of surveys, as happened when the U replaced four homegrown surveys with the Student Experience at the Research University (SERU) survey, a pdf of this cornerstone web-based survey is available for review on the site.

"Doing this can save money and make surveys more effective. A group like CSOM (Carlson School of Management) might be conducting a survey we could use but don't know about," says Huesman.
Behind every survey is a lot of staff time—and money. Not only that, but the U is caught up in a trend of falling survey efficacy, he says.

"That's the big $64,000 question with survey research. You need a good response rate, in part, to make it work, but because there are so many surveys that go out and they're not coordinated, it's hard [for recipients/students] to distinguish what's important. And survey response rates are declining not only at the U, but nationally."

"To the extent that we can offer fewer surveys, better illustrate their importance to our students, and make the data collected more accessible and useful to the University community, hopefully in the end more students will respond to our surveys," Huesman says.

And the U, in turn, will be better able to respond to the students.

For more information, see the OIR website.