Mistaken Goal: Where Higher Education & Technology Meet

I’m not shilling for Google; I’m just very pleased with what we’ve been able to do with Google Docs and want to share.

Me and several of my classmates are beginning our qualifying exam on Friday. In our PhD program, we are given a question on Friday morning and we turn in our answer – maximum 20 pages – by Monday afternoon. Then we do it again the next Friday with a different question written by our respective advisers. We can’t work together on our answers but we have worked together to collect and summarize (what we hope to be) helpful resources on Google Docs.

Google recently added two features to Google Docs that have made this possible:

• Shared folders: We have created folders in Google Docs to organize our materials (Faculty, History, Finance, etc.) and shared those folders with one another.  So we didn’t have to create our own set of materials and organize them; we created one set of folders, organized them, and shared them with everyone.
• Upload any file: Google now allows you to upload any kind of file to Google docs, including pdfs and Word documents.  This has allowed us to not only create summaries of resources for each topic but also upload a copy of the article, chapter, etc.

It’s not perfect.  Sharing folders with 7 other people requires you to either (a) obsessively reorganize and rename every file so everything is consistent or (b) let it go.  We have also stepped on one another’s toes at least once in editing a document but it was easy to roll back to an older version of the document.

Our colleagues who took their quals last semester also did something like this and those materials are shared with us.  We also uploaded to Google Docs older collections of materials from colleagues that took quals a few years ago.  So at this point we have a sizable collection of materials that should be helpful in preparing for quals and writing our answers.

But it’s not just about those of us taking quals this week.  We’re also thinking about the future and those who will come after us.  In that vein, we’ve already shared all of the materials (they’re all in one folder with many subfolders so sharing is easy!) with classmates who will be taking quals in the future (next year, 2 years from now, etc.) so they have access to all of these resources, too.  We hope that they do the same thing for future generations of IU HESA doctoral students.  We’re paying it forward and Google Docs has made it easy for us to do so.

(There is one additional step in my personal process of collecting these materials and organizing them: I’ve uploaded all of them to my personal bibliography.  I’m trying to keep my materials centralized there where I can access them from anywhere and share them with anyone.  It was also a helpful part of the review process because as I was uploading them I reviewed every document that my colleagues contributed.  Finally, my bibliographic tool creates the correctly-formatted APA citation – with some known exceptions and quirks – for each source so I don’t have to worry about doing that myself or relying on the citations created by my classsmates.)

Comments(4)

A few weeks ago, I attended the 2010 ResNet Symposium in Bellingham, Washington where I was invited to present a preconference session on assessment.  I presented two identical sessions, one in the morning and one in the afternoon.  In this post I’ll reflect on what we discussed in these sessions and my perceptions of assessment in IT in American colleges and universities.

ResNet preconference session

I was invited to present these sessions by one of the conference organizers who has a strong student affairs background.  As a profession, student affairs has tried to embrace outcomes assessment so this person is familiar with the issues.  We both share a perception that IT professionals and organizations in American higher education have not yet begun to understand and perform outcomes assessment so an introductory session at the ResNet Symposium would be beneficial for attendees.  I didn’t know how well it would be received but I was pleased with the turnout: 15-16 attendees were in the two sessions, a good representation of the 101 attendees of this small conference.

At the beginning of the session, I asked the attendees to write on the whiteboard the words they associate with “assessment.”  I wanted to gather a bit of information about the attendees and their preconceived notions and I also wanted them to begin thinking about the topic.  The words they wrote most often were analysis/analyze, data, measure(ment), and evaluation.  Not a bad start.

In the first half of the session, we talked about assessment in broad, general terms.  I began by trying to provide some context for the importance of assessment, concentrating particularly on the political context and how academic and student affairs have reacted.  Next, I tried my best to introduce topics that I believe are important to understand or least know exist such as direct vs. indirect assessment and formative vs. summative assessment.  I also tried to get attendees thinking about issues and collaborating with one another by having them brainstorm in small groups to generate a list of sources of data already available on their campuses.

In the second half of the session I focused on surveys and survey development.  Not only are surveys (unfortunately) one of the most common ways of gathering data, they are also a topic in which I have some expertise.  After discussing some survey methodology concepts, primarily sources of error as identified by Dillman in many of his publications, we looked at a survey instrument I recently put into the field.  More specifically, we looked at different iterations of the survey and discussed how and why the survey changed throughout the development process.  I closed with a brief list of survey tips.

I think the session was successful in introducing some of the important concepts in assessment.  It was hard to figure out what to concentrate on during this brief session (the Assessment Framework developed by NASPA’s Assessment, Evaluation, and Research Knowledge Community was very helpful!)  and I’m still not sure that I struck the right balance between introducing important ideas and engaging the participants and meeting their expectations.  It would have been easier, I think, if I had titled the session “Outcomes Assessment” and used that phrase throughout the session; that would have provided some needed focus and better described the topic I intended to introduce.

Outcomes assessment in IT

As mentioned above, this preconference session was developed because of a shared concern about the lack of outcomes assessment in higher education IT.  We’re doing a very poor job of not only establishing how we contribute to the bottom line of our institutions (and the bottom line, of course, is the production and dissemination of knowledge) but also if we’re actually succeeding in meeting those objectives.  I believe this is fundamentally important in justifying the resources expended on in-house IT operations.  You should know why you’re doing what it is you’re doing and you should know if you’re succeeding.

Student affairs professionals realized this a decade or two ago and began emphasizing assessment both in practice and in their graduate programs.  I think that was a very smart move in that it tries to move student affairs from the periphery of the academic enterprise to a place much closer to the center, making student affairs more visible and important in many ways.  Much of IT is in the same boat that student affairs was in a few decades ago where there is an implicit belief that their services are necessary but it’s hard to explain exactly why they’re necessary and should be supplied by the institution itself.  Simply arguing that the services are “important” or even that they’re in demand doesn’t give us a license for incorporating them into our colleges and universities.  Many services are important and desirable but we’re content to contract them, outsource them, or just rely on the outside world to provide them.

We have to prove that what we do significantly contributes to the mission of our institutions and that we do it better – more effectively, more efficiently, cheaper, etc. – than anyone else.  I know that it’s hard to do that; the rest of the campus has been trying to do that for some time and they’re still struggling!  But IT has to get on board and move beyond mere measures of satisfaction and internal metrics that are uncoupled from the mission of the institution.  It’s not even about self-preservation (although that should be a motive!).  It’s about know what you’re doing, why, and if you’re getting it done.

Comments(0)

I’m a few weeks away from my qualifying exam so I took an opportunity to update my dissertation synopsis so my advisor can use it to write my second question.  It includes my final survey instrument and a few notes related to the pilot administration.

In the meantime, I wait for surveys to be administered and returned.  I’ve mailed about 5,500 surveys to 11 institutions.  I was very fortunate in being able to get a fairly diverse group of institutions to participate.  However, I’m a bit nervous that a handful of institutions that have not yet committed to participating in NSSE in 2011 may decide (or be forced) to not participate.  That wouldn’t kill my study but it might require significant changes.

Comments(0)

(Sorry, not much technology in this post.  My blog, my rules.  And I can break those rules when I want to.)

It is common in student affairs Master’s programs for faculty to require their students reflect on and document their professional philosophies, documents that are analogous to faculty teaching philosophies.  A recent thread on CSPTALK, a listserv for student affairs faculty, focused on these professional philosophies and it inspired me to look back at the document that I wrote as a Master’s student nearly 7 years ago.

Of course, it’s interesting to see how I’ve grown as a writer and as a professional.  I cringe at some of what I wrote, especially my naive citations of “the literature.”  It’s hard to be impressed with a paper that quotes Monty Python in the second paragraph.

It greatly pleases me, however, that the fundamental ideas expressed in this 7-year old document remain sound and close to my heart:

• We are all individuals: I continue to fervently believe this and I cite as evidence my distaste of generational stereotyping.  As someone who often conducts large-scale quantitative research, I struggle to balance this belief with the (natural) desire and need to generalize.  This belief may stem from a keen awareness of the importance of context, an awareness that some of my colleagues seem to lack or disregard at times (e.g. students who are “minorities” in some situations are not in others).  And this belief in and awareness of context is one of the primary drivers of my dissertation.
• We are all students: I’m back in graduate school so this one is a gimme.  But what I really meant – and still mean – is that in nearly all situations we have much to learn from one another, even as experienced professionals working with undergraduates.  I think that this belief contributes to my seemingly-misplaced admiration for well-conducted qualitative research, research where the reader and researcher find themselves moving with and learning from the research subjects as the boundaries between them blur.
• We are all teachers: This belief complements the previous one.  More specifically, I think that this belief speaks to a deeper belief in altruism, empathy, and perhaps even love.

If I had to write this all over again, I don’t think the big ideas would change, just the details (I might want to add something about empiricism or a belief in evidence but I’m not sure). Despite the immature writing, numerous APA errors, and poor grasp of (outdated) literature, I remain proud of the ideas expressed in this document.

Comments(0)