Mistaken Goal: Where Higher Education & Technology Meet

In a recent blog post releasing a (very nice!) infographic about “Best Practices in Using Twitter in the Classroom Infographic,” Rey Junco writes:

I’d like to point out that I’m a real stickler about using the term “best practices.” It’s a concept we toss around a lot in higher education. To me, a “best practice” is only something that has been supported by research. Alas, most of the time that we talk about “best practices” in higher ed, we’re focusing on what someone thinks is a “good idea.”

I agree and I’m even more of a stickler. There have been several specific situations in which I have been asked or encouraged to write a set of best practices for different things but I always got stuck asking myself: What makes this particular set of practices the “best?” I share Rey’s dislike of “good things I’ve done” being presented as best practices. But my (relatively minor) frustration extends a bit further because to me the adjective “best” implies comparison between different practices i.e. there is a (large) set of practices and this particular subset has been proven to be better than the rest.

I’d be perfectly happy if people were to stop telling us about best practices and just tell us about “good” practices until we have a large enough set of practices and data to judge which ones really are the best. If you’ve done good work, don’t distort or dishonor it by trying to make it bigger than it is. After all, even Chickering and Gamson (1987) presented their (now-classic and heavily-cited) ideas as “Seven Principles for Good Practice in Undergraduate Education” and not “Seven Best Practices in Undergraduate Education.”

Comments(4)

In a comment to my previous post sharing some of my thoughts about #sachat in advance of their “State of #SAchat” discussion tomorrow, Gary Honickel asked about demographics of #sachat participants.  In our forthcoming chapter (I’m not trying to advertise it – honest! Just trying to explain why I have all of this information. I’m a researcher, not a stalker!), Laura Pasquini and I analyze #sachat and we include some information about the participants.  We didn’t include the specific information Gary asked about: gender and geographic location of participants.  But I did collect that data and although it’s for three sessions that occurred last year maybe this is still useful or helpful.  My sense is that these things haven’t changed much in the past year.

Keep in mind that these data come from three 2011 chat sessions:

The thing that jumps out at me in the table above are the average number of messages per participant and the standard deviation of that number.  There is immense variance in the number of messages posted by each participant and that makes me wonder about the pattern(s) of participation for each session.  The histogram below showing how many people posted a particular number of messages in each chat helps us understand these numbers (click on it to view a larger version).

This histogram is a classic “long tail” distribution, showing us that most participants in these three #sachat sessions posted very few messages and only a handful of participants posted many messages; the participant with the most messages is, of course, the moderator.  This is a very typical situation and an unsurprising finding.

This gives us a broad understanding of #sachat participation but let’s look a bit deeper and explore two different ways of classifying participants: gender and geography. First, a few words of caution: these data were inferred from the Twitter profiles and messages posted by these participants.  Geography was the easier datum to capture for each participant as most participants associated themselves with a particular college or university, either in their profile or in their introduction during one or more #sachat sessions.  Gender was much more difficult and I present these data with trepidation because there was a significant amount of guesswork involved in classifying participants as male or female.  If this were anything more than a one-off blog post or if gender were a central concern for this or any other analysis, I wouldn’t even share or use these data because inferring gender from name and photo obviously lacks rigor.

This chart shows the geographic locations of the participants in these three #sachat sessions (I used the U.S. Census geographic regions to aggregate the data).  Nothing surprising here.  #SAchat is indeed U.S.-dominated but even that isn’t a surprise.  Nothing particularly interesting is discovered if you look at the number of messages posted by participants from each region; the numbers get very small very quickly when slicing the data this many ways so it’s not worth trying to display.

What about gender?  For at least these three sessions, the gender breakdown seems to be about even.  Like geographic region, nothing terribly interesting happens if you slice these numbers in different ways.

So what do we make of all of this?  I think it shows that – for these three sessions – there was considerable diversity among #SAchat participants, at least in two ways we can measure. Of course, these are coarse (and in the case of gender, potentially problematic) measures and there are many other ways in which we might examine the makeup and diversity of this population.  Functional area and role (student, entry-level professional, faculty, etc.) are two measures that jump to mind as interesting and useful.  (Incidentally, I tried to classify participants using those two measures in a previous study; it was difficult, time-consuming, and very incomplete since those data are not spontaneously volunteered by all participants.)

Are #sachat participants diverse enough?  I don’t know.  How do we define “diverse enough?”  Should we be concerned about how well the #sachat population matches the larger student affairs population?  A quick glance shows some alignment between these populations but I have not done any definitive work in this area, partially because it’s very hard to obtain data about the larger student affairs population.

Of course, all of this does not and can not include anything about lurkers.  I agree that there is value in #sachat even for those who do not directly or visibly participate but we’d have to make a concerted effort to identify those people if we want to know anything about them.

I hope this is helpful or interesting!  I wish I had more up-to-date data but I don’t.  I’m job searching, working, and trying to finish a dissertation so I don’t have time or plans to gather additional data right now.  This is data that I had at hand and I am happy to share it in the hopes that it’s useful for someone.

Comments(1)

Tomorrow, the members of the #sachat community will be engaging in introspection and discussing “The State of #SAchat” instead of their usual weekly discussion of topical student affairs topics.  I have been conducting research on the #sachat community for a couple of years now so I thought it might be helpful for the community if I could organize and share some of my thoughts.

I won’t spend time describing the basics of #sachat; if you are interested in this particular conversation, I assume that you are familiar with the community and its tools.  If I wrong and you are not familiar with #sachat, the official overview is here.  An annotated visualization of one chat session – a February 10, 2011 discussion about job searching – is below (my original blog post discussing this visualization has some of its background details).

The chart below shows Twitter message traffic from six hashtags – #highered, #sachat, #sadoc, #sagrad, #sajobs, and #studentaffairs – during the week of June 27, 2011.  This illustrates how #sachat differs in that it not only has consistent traffic everyday (although not as much as #highered) but it spikes during the scheduled chat session on Thursday afternoon.

In a book chapter Laura Pasquini and I have in press, we examine #sachat as a case study of informal learning using technology.  One of our conclusions is that #sachat is doing several things right to overcome the significant limitations of Twitter by:

• Allowing participants to direct the discussions as much as practical.  For example, potential participants vote on each week’s topic and do not have to register to participate (in the voting or the actual discussion).
• Using other tools to supplement the core use of Twitter.  Most of these tools reside on the SA Collaborative website.  One of the most important may be the chat archives that give the chats a sense of continuity and history beyond the typically ephemeral nature of Twitter.
• Employing a well-prepared and clearly identifiable moderator in each discussion.  This account helps impose order on the Twitter chat, allowing conversation to run for a bit before drawing it back to the core topic by using clearly marked, pre-prepared questions.

We also identify several specific concerns and challenges:

• Can the participants continue to overcome the inherent limitations of Twitter, especially its (a) short message length, (b) lack of threading, and (c) ephemerality?  Although some participants attempt to overcome the first limitation using multipart messages, this is not very successful; the 140 character limit of Twitter is one of its core features and unlikely to be overcome.  The second limitation has been addressed with some success with the use of MOD messages and Q# replies.  The third limitation has been partially overcome by regularly making transcripts of chats publicly available.
• Is the small community of volunteers that run the chats – those who use the moderator account and the SA Collaborative website – sustainable?  These volunteers and the tools they provide and maintain are essential to the success of the community.  For how long will these volunteers sustain their energy and will there be a smooth transition as members come and go?
• How representative of the larger student affairs community is the #sachat community?  Is that important?
• How diverse are the members of the #sachat community?  In what ways are they diverse and in what important areas is diversity lacking?

Comments(2)

It’s taken me about a year to reach this point but I finally completed and submitted to my chair a draft of chapter 2, the literature review.  It’s pretty solid but it’s still a draft.

1. There are a handful of places that I know I could expand but I’ll wait until I hear back from my chair before doing that.
2. I need to add a couple of new sources to the digital divide section but nothing significant and nothing worth delaying this draft any longer.
3. I think that I need to add a brief section – to this chapter or another one – summarizing the assumptions of the study.  Several of them are spread throughout the lit review where I discuss and justify them but it seems that it would be more organized if I would summarize them all in one place.

Why did this take so long?  I don’t know.  It’s certainly not because I struggle with this kind of writing.  It’s definitely not because I don’t know what I want or need to write.  I imagine that it has to be some kind of emotional block, some sort of fear of failure perhaps.  That is completely uncharacteristic of me but it’s the only thing that makes sense.

Why did I finally buckle down and get this draft completed?  The shame of not having done this yet became overwhelming and I simply had to finish this so I could look my chair in the eye.  I am also running out of time; I’m already behind where I wanted to be right now and hitting the job market ABD.  I’m not very pleased to be on the market as an ABD but now my energy needs to be focused on making as much progress as possible before I land a job because the more progress I have the better the odds that I’ll finish.

Comments(2)

Researchers at Elon University and the Pew Internet & American Life Project have released a report describing some opinions about the future of the Internet and its impact on society, particularly the younger generations.  One of the themes is that educational practices must change to address information literacy.  Although I agree with some of its content I don’t know if this report will have any impact on practice or policy; I am becoming a bit jaded and burnt out by academics discussing how education should be reformed when so little seems to actually happen (which doesn’t stop us from writing books, publishing articles, and giving talks – we must feed our egos and sustain the systems that have grown rich on this fare so they in turn will continue to sustain us!).

I am a little bit amused by The Atlantic’s description of the sampling strategy of the survey that underlies this report: “[The report is] based on surveys with more than 1,000 thought leaders.”  I was asked to participate in this survey.  That means that I must be a thought leader!  I don’t know what that means but it sounds suspiciously like “someone who doesn’t actually do anything but is really good at it!”  Now if you’ll excuse me I need to figure out where “Thought Leader” goes on my CV…

Comments(0)