First, let’s see how common U.S. college and university Wikipedia articles are.  To do this, I need to figure out how many institutions have a Wikipedia article.  I randomly selected 10% (732 units) of the 2008 IPEDS universe, a listing of every Title-IV-participating institution (e.g. virtually every accredited institution in the United States and its territories).  I then checked to see if these units have Wikipedia articles.  Broken down by sector and control and ignoring the handful of system offices and unclassified institutions pulled into the sample, here is what I found:

Considering that most people in the U.S. think of 4-year institutions when they think of “college” or “university,” Table 1 shows us that it’s fair to say that college and university Wikipedia articles are very common.  Not only are they ubiquitous for public 4-year institutions, they’re very common for private 4-year institutions and community colleges.  The primary types of institutions for which they are uncommon are private 2-year institutions and all types of less than 2-year institutions, institutions typically associated with specialized technical training and usually omitted when talking about colleges and universities.

Next, we need to figure out the popularity of U.S. college and university Wikipedia articles.  In this context, I am defining “popular” by examining where the top three search engines – Google, Yahoo!, and Bing – place U.S. college and university Wikipedia articles.  To do this, I selected a random sample of these Wikipedia articles; the sample is also stratified, including 12 articles from each major quality classification assigned by the Wikiproject Universities (Featured, Good/A, B, C, Start, and Stub).

As shown in Table 2, when you search for these institutions in each of the three leading search engines, Wikipedia articles are not only among the very first results but they’re usually the first result that isn’t controlled by the institution.  Google seemed to struggle with providing accurate results for the institutions who do not have unique names (i.e. Southwestern College, Sierra College), listing several other similarly-named institutions above the Wikipedia article.  Yahoo! and Bing did not have this problem, almost always listing the Wikipedia article immediately after the institution’s official website or immediately after the institution’s official website and the official athletics website (of course, Yahoo! and Bing provided the same results since they use the same search technology).

Based on a random sample of the accredited colleges and universities in the United States, Wikipedia has articles for the majority of institutions.  This is particularly true when considering 2- and 4-year institutions, especially public ones.  Further, those Wikipedia articles are placed very highly in search results, usually immediately proceeding the institution’s official website.  Not only are U.S. college and university Wikipedia articles very common, they’re extremely popular.

(The data are available here:

• IPEDS population
• IPEDS sample
• Wikipedia population
• Wikipedia sample

A few of the spreadsheets are rather large for Google spreadsheets so they’re a bit sluggish.  Sorry!)

I haven’t touched it for a while but for a few years I’ve been working on historical research focused on entertainment and communications technologies in American college and university residence halls. As is often the case, I began this research as it was a topic of interest to me; I placed only superficial thought on practical applications and implications. In other words, I did it only because I liked it and it interested me. But that won’t convince others to care about this research, to listen to me discuss it, or allow me to publish it.

This morning I finally found my hook. This will be the first time I’ve written it down so let’s see how it looks in print:

Understanding the history of entertainment and communications technologies in residence halls provides us with a means for understanding the tapestry of forces that have shaped not only residence halls but academia in the United States. These technologies provide rich examples of innovations motivated by economic competitiveness, cultural expectations, and academic experimentation.

Not only does this provide me with a much-needed organizational framework for this work but it also provides others with a motivation for understanding and supporting this historical research.

In the paper Allison BrckaLorenz and I presented earlier this week, we used data collected with the 2009 administrations of the National Survey of Student Engagement (NSSE) and Faculty Survey of Student Engagement (FSSE) to examine how often these two populations – students and faculty – use academic technologies.  We added several questions about technology to the surveys administered to some institutions.  In this paper, we examined the responses to those additional questions from senior undergraduate students and faculty who teach them at 18 institutions who participated in both NSSE and FSSE.  Specifically, we (a) compared student and faculty responses and (b) explored responses across academic disciplines.  However, to keep this blog post a manageable and readable length, I will omit most of the discussion of disciplinary differences; I encourage you to read the full paper if you are interested in those findings.

The survey question on which we focused was multi-part and asked respondents how frequently (Very often, Often, Sometimes, or Never) they used some academic technologies:

1. Course management systems (WebCT, Blackboard, Desire2Learn, Sakai, etc.)
2. Student response systems (clickers, wireless learning calculator systems, etc.)
3. Online portfolios
4. Blogs
5. Collaborative editing software (Wikis, Google Docs, etc.)
6. Online student video projects (using YouTube, Google Video, etc.)
7. Video games, simulations, or virtual worlds (Ayiti, EleMental, Second Life, Civilization, etc.)
8. Online survey tools (SurveyMonkey, Zoomerang, etc.)
9. Videoconferencing or Internet phone chat (Skype, TeamSpeak, etc.)
10. Plagiarism detection tools (Turnitin, DOC Cop, etc.)

The average responses to this question are shown in the figure below.

A few things are apparent from this figure and the responses that it displays.  First, the only technology that students and faculty really use is course management systems (CMSs); most respondents never used the other technologies.  Second, except for Plagiarism detection tools, students are reporting more frequent use of these technologies than faculty.  This is particularly noticeable for collaborative editing software, a technology that students probably use outside of class to collaborate much more often than they use it during class or when specifically assigned to use it.

Another way to make sense of these survey responses is to use cluster analysis to group respondents together.  For students, a 4-means cluster analysis made the most sense:

The students in the High and Medium Use clusters used multiple technologies with relatively high or medium frequency (remember that most students never used most of these technologies).  Students in the Low Use cluster only used CMSs.  And students in the No Use category didn’t really use any of these academic technologies.

A 3-means cluster analysis was most appropriate for the faculty respondents:

As with the student clusters, faculty in the High Use cluster used multiple technologies with some frequency.  Faculty in the Low Use cluster only used CMSs and faculty in the No Use category didn’t really use any of these academic technologies.

From these figures, it is clear that most students and faculty are making little use of academic technologies except for course management systems like Blackboard and Sakai.  Given the resources campus have invested in these particular technologies, it is probably good that faculty and students are making frequent use of them.  However, we stop short of making a subjective judgment based on these responses as there are certainly many instances in which technology is neither helpful nor appropriate in classwork and assignments.  A better approach – one that may be impossible using self-administered surveys – would be to understand not just how often students and faculty use technologies but how often they use them appropriately and well.

More interestingly, students are reporting a higher usage of these academic technologies than faculty.  Most likely, these technologies are not be required by faculty but used by students on their own initiative to complete their work and collaborate and communicate with one another.  The differences between student and faculty responses might be a result of the methodology of this study.  But these differences are probably real and point to genuine differences in how frequently students and faculty use these and other technologies, differences that may result in tension and other differences between these two groups.

Here are my current and upcoming projects, listed in no particular order…

• Continue editing and submit for publication (EDUCAUSE Quarterly?) the paper (A Comparison of Student and Faculty Academic Technology Use Across Disciplines) I just presented with Allison BrckaLorenz at the AIR Forum.
• Finish preparing for my ResNet 2010 assessment preconference session.
• Continue working with the ResNet 2010 hosts to schedule and conduct attendee focus groups to supplement the survey data we recently collected regarding the current state and future direction of the ResNet organization.
• Two potential AERA proposals:
  • Discourse analysis of #sachat.  I wrote a solid paper for the discourse analysis class I took in the spring but Rey Junco will be helping me to redo some of the analysis and edit the paper.
  • Historical analysis of student affairs and technology.  I have a solid draft of this paper already done (another class paper) but it’s very long and needs to be edited down to a more manageable, readable length.  Additionally, I’ve recently discovered that we have in the library stacks at Indiana University proceedings from NASPA and ACPA meetings held during the first half of the twentieth century.  I need to spend time in the library with those proceedings as I haven’t yet incorporated them into my study (I didn’t know where I could find them; I certainly didn’t expect to find them at my home institution!).
• Begin a new project analyzing the demographics of student affairs professionals.  I wanted to use these data in my Twitter research but no one has done this work in 15 years so I’ll have to do it (I hope that I’m wrong and that I simply haven’t found a current or recent source!).
• Wait to hear back from ASHE to know if our Wikipedia proposal has been accepted.  If so, then we need to do more work on it to update it and get it into shape for the conference later this year.

Of course, I have other things going on and coming up: quals in 2 months, ongoing projects at work, and beginning data collection for my dissertation.  I thought that summer – especially the summer after you finish coursework – was supposed to be quiet and relaxing?

I hope this paper is accepted (otherwise I wouldn’t have submitted it!) but I know it’s a bit “out there.”  Despite my apprehension, I firmly believe that we must be mindful of how the public perceives higher education and the explosion of information available on the Internet provides an incredibly rich source of information if we can figure out how to harness it (In this vein, I am extraordinarily happy and grateful to have had the opportunity to study web content analysis and computer-mediated discourse analysis, giving me some of the necessary background and tools to study these data!).  And given that (almost?) every significant college and university in the United States has a Wikipedia article that (theoretically) lies largely outside the control of the institutions, these articles are a rich source of public opinion.

I know what some of you are thinking: Wikipedia editors don’t represent the general public!  I’m not entirely convinced that is true – especially without data – but I’ll concede the point anyway.  Even if those editing the articles are not representative of the general public, surely we can agree that the information placed in these articles clearly indicates what the general public is learning about these institutions from Wikipedia.  So it’s still important to know what’s going on in these articles.

Since we submitted our paper, Wikipedia articles have gotten another boost in visibility and importance: Facebook is making heavy use of Wikipedia articles in Community pages.  This has already raised a discussion within Wikipedia (full disclosure: I’m one of the participants in the discussion) about the role (or lack thereof) Wikipedia should play given that articles are being displayed in Facebook.  More specifically, at least one institution has objected to the graphic that is being displayed in Facebook.  The topmost graphic in nearly all of these Wikipedia articles is the official seal or crest of the institution.  But most institutions have graphic identity standards that mandate the use of another set of graphics (their “wordmark”) and limit the use of the official seal or crest.  Of course, Wikipedia is not required to honor those standards and it’s pretty clear that fair use allows Wikipedia to use official seals and crests without the permission of the institutions.  This is the kind of interesting complexity about which higher education administrators and scholars should know and in which they should appropriately participate.

Love it or hate it, Wikipedia is an immense force in today’s information societies.  We don’t yet know exactly what role it plays in the college choice process but we can be certain that many people are learning about our institutions via Wikipedia.  We can not and should not control the information in Wikipedia but we should be aware of it and the communities that create, edit, and even vandalize that information.  And we should be eager to use that information to develop a better understanding of how the public views higher education and our institutions.

[August update: The proposal has been accepted.  I look forward to sharing the final paper here and at ASHE this fall.]

For those who are unfamiliar with #sachat, here is how I described it in my first paper for this class:

Beginning in the fall of 2009, a group of American higher education administrators began using the micro-blogging tool Twitter to communicate, collaborate, and connect with one another.  Each week for at least one hour, these professionals employ Twitter as a public synchronous mass communication medium by marking each of their messages with the #sachat hashtag and discussing a predetermined topic of professional interest.

Each Wednesday, student affairs professionals use Twitter to vote on a topic of discussion.  On Thursday, these same professionals discuss this topic (and others) for at least one hour.  These discussions are loosely coordinated and moderated by one user associated with the TheSABloggers.org website.  Although the participants are highly-educated professionals and many of the topics are related to their professional interests, the tone of the discussions is informal and often playful.

Using Twitter for these conversations imposes particular properties and restrictions.  First, Twitter is nominally an asynchronous medium; by collectively participating at a prearranged time, these users are effectively using Twitter as if it were synchronous.  Second, to coordinate all of their discussions, including the voting and discussion outside of the established hours, participants must include in their messages the phrase “#sachat.”  This phrase – a Twitter “hashtag” – allows Twitter users to search for and categorize these messages.  Third, Twitter restricts messages to 140 characters.  Finally, although Twitter users can address particular users in their messages there is no threading or other advanced addressing functionality.

Since this class is focused on computer-mediated discourse, I’ll be analyzing patterns in these online conversations in terms of features such as participation, message complexity, speech acts, topic development, and politeness. I’m initially focusing on the discussion that occurred on January 21 so I can learn and begin to understand these methods used in discourse analysis.  Later in the semester, I’ll expand my analysis to also include January 14 and January 28 (daytime only; I can’t seem to locate an archive of the evening conversation) for my final paper in this class.  Eventually I would like to expand the analysis to include more discussions and to include content analysis in addition to discourse analysis so I can write a fully-formed paper for publication or presentation (I’m thinking maybe AERA 2011 if I can meet their submission deadline in late summer).

I am interested in conducting this research not because it focuses on Twitter but because it focuses on a grassroots community that has found a unique way to connect and communicate with one another.  It’s especially interesting because their method of communication is free and this is a time of financial stress with reductions to or eliminations of professional development budgets prominent at many institutions.

Many of the methods I’ll be using have been pioneered or extensively used by Susan Herring.  It’s terribly exciting to learn from and with her as she is probably the world’s foremost expert in these methods!  This is the second class I’ve taken with her and it’s a lot of fun to learn from someone who not only intimately knows the topic but is also still really excited about it and super supportive of new, young researchers.

If any #sachat participants have questions, concerns, or suggestions, please share them with me!  Although the data are all publicly-available, I will be using pseudonyms in all of my public presentations and papers so hopefully that will allay any privacy concerns.  Additionally, I imagine that I’ll eventually file an IRB amendment so I can officially talk to you about your experiences and opinions (because a study on this topic seems incomplete without actually talking to the participants).  But in the meantime I’m definitely open to informal discussion, especially if you have concerns about this research.

(And can someone throw a link to this post out there in Twitter and tag it with #sachat?  I would do so myself but I am trying to retain some distance as I study this phenomenon.  More importantly, I just don’t have time right now to jump into Twitter, at least not this month as I prepare for quals and begin preliminary work on my dissertation.  There are only so many hours in the day…)

The relationship between student engagement and technology is a hot topic right now.  The current issue of EDUCAUSE Quarterly focuses on this relationship.  Both the National Survey of Student Engagement (NSSE) and the Community College Survey of Student Engagement (CCSSE) have focused on technology.  NSSE most recently published technology-related findings in 2008 and 2009 Annual Results (CCSSE followed suit in 2009) but we’ve poked at this topic several times in the past ten years.

In general, every time we’ve examined this relationship we find it to be positive.  The relationship isn’t always terribly strong but it’s positive and significant*.  More importantly, this relationship appears to persist no matter what we throw into the mix.  We’ve tried many different things (“controls”) to see if there is something tricky going on, such as a complex relationship with other variables.  For example, it’s possible that students from more affluent backgrounds both use technology more often and score higher on our measurements of engagement because they had better schooling.  But that doesn’t appear to be the case.  At the moment, however, it appears simply that “technology is good.”

That conclusion is neither satisfying nor likely.  It’s not satisfying because it seems very shallow and not at all explanatory (e.g. it doesn’t tell us what it is about “technology” that encourages more engagement and better learning).  It’s not likely because several decades of research has told us that it doesn’t matter which medium we use to deliver education (Clark, Yates, Early, & Moulton (2009), available as a pre-print, is an excellent overview of this body of research).

So if we don’t accept the overly-simple statement that “technology is good,” what do we do?  We did two things.  First, we focused on some specific technologies so we could move beyond broad conceptions of technology and look at some tools currently in use.  Despite the excellent research that tells us that technology itself should not have an impact, we must keep an open mind and explore that possibility, especially as technology advances and becomes more complex and ubiquitous.  Second, we asked faculty participating in the Faculty Survey of Student Engagement (FSSE) a nearly identical set of questions as we were asking the students participating in NSSE in the spring of 2009.  We even convinced 18 institutions to administer both sets of questions!  We wanted to draw faculty directly into the mix because the most likely explanation for our repeated finding of “technology is good” is that use of technology is associated with good teaching.  (That hypothesis also seemed to tentatively arise from one of our studies of distance learners, a study that didn’t seem to do much to cut through the clutter despite using sophisticated methodology.)

We presented some of our results at POD’s 2009 conference in Houston.  As mentioned above, we’ll be presenting some more at ELI’s 2010 conference in Austin.  And we’ll be presenting again at AIR in Chicago in a few months.  These are all different presentations focusing on different aspects of our data.  And there is still data we haven’t yet analyzed and presented!

I’m sorry that I haven’t give you any answers in this blog post.  We’re still working to find them and so far it’s been devilishly difficult.  It’s probably hard for us because our tools – voluntary, self-administered surveys administered to massively large groups of students and faculty – are blunt objects with limited capabilities.  And every answer we find raises more questions.  But it’s clear that there is positive relationship between student use of technology and student engagement, even if the relationship is more complex than it appears on the surface.

* – Statistical significance is tricky for us.  Our data sets are enormously large and since significance is sensitive to sample size a whole lot of things are significant.  So we often turn to other measures such as effect size and other contextual indicators to make sense of our data.

• The ECAR Study of Undergraduate Students and Information Technology, 2009 is the latest report from EDUCAUSE’s research arm focusing on undergraduate students and their use and perceptions of technology.  It’s always a well-done study and EDUCAUSE makes the full study (2.7 MB pdf) freely available to everyone so you should take a few minutes to glance over at least the Key Findings (330 KB pdf).
• The EDUCAUSE Core Data Service Fiscal Year 2008 Summary Report is another report released by EDUCAUSE this week.  As the name implies, it’s a summary of results from the last round of data collection in the Core Data Service, EDUCAUSE’s database of educational technology information.  This document is one of the best (and often the only) publicly-available empirical source of information on technology in higher education, particularly if you’re looking for campus-based statistics such as how much money is spent on technology, how many people are employed to support it, and what kinds of practices and technologies are being used.
• The Department of Education has released its final rules (2.12 MB pdf; search for “copyright” to find the specific areas of interest) specifying how to interpret the laws passed this summer requiring (Title IV-participating) colleges and universities to actively combat online copyright infringement.  At first glance, the final rules do not appear to differ from the proposed rules.

I hope to find time to dig into all three of these documents in the next couple of days.  I recommend that you do the same.

Discussion about the possible relationship between college students’ use of Facebook and lower grades continued this week with the publication of a First Monday article addressing this topic.  This article follows up on previous discussions that followed the widespread publicity surrounding a poster session presented at AERA that found a correlation between Facebook usage and lower grades. Unfortunately, I’m not sure that the research and related discussions have shed more light on this topic.  But it sure has been exciting to watch how quickly it’s all happened!

The discussions have followed two general threads: (a) the AERA research was poorly done and (b) the media got the story wrong. I’ll address the first thread in detail below.  The second thread has been relatively short-lived as there isn’t any real disagreement that many reporters and editors leaped (without looking, thinking, or corroberating) from “there appears to be a link between Facebook usage and low grades in this small sample of this very limited study” to “Facebook causes bad grades!!!”  That’s irresponsible and everyone agrees on that point.  There is also a third thread that focuses on “I can’t believe this is true but I don’t have any evidence!” but it’s not worth wasting any time on those ill-informed opinions.

In general, most of the current research into Facebook usage seems to lack sophistication (and much lacks rigor; how many of the articles based on surveys discuss or even hint at validity or reliability?). The researchers behind the poster session and this First Monday article both rightly acknowledge that they are discussing correlation but there is a whole lot going on that they don’t acknowledge or can’t account for with their selected (or mandated) methodologies and data.  In trying to understand college students, we go to great lengths at the shop where I work to isolate and separate the influence of different variables and we struggle with this mightily.  In many instances, we have to employ relatively-sophisticated analyses such as multilevel modeling to adequately control for different variables, particularly the institution-level and student-level variables.  In fact, I don’t recall seeing any mention of institution-level influences in any of the currently-available research even beyond this poster session and article (of course, one can’t do anything about this if one’s sample is only drawn from a handful of institutions, another significant limitation of nearly all Facebook research). I acknowledge that institution-level influences only account for a small proportion of the variance among most of the things we measure but omitting measurement and discussion of institutional characteristiscs altogether seems to indicate a lack of theoretical and methodological sophistication. To put it bluntly, this is the kind of thing that many non-higher education researchers often miss as it simply isn’t their area of expertise and why higher ed scholars desparately need to be actively contributing to this conversation.

What most people want to see is not correlation but causation.  In other words, we want to be able to say that (the use of ) Facebook causes lower grades.  That’s a damn hard claim to make.  Even under the best circumstances, establishing causation is fiendishly difficult.  It would require sophisticated measures and analyses. Given the previously-mentioned lack of sophistication in most of these studies I don’t know that these researchers collected the right kinds of data to even begin to do the work necessary to establish causation.  Frankly, I think it’s so complicated and the analysis would be so fragile and fraught with assumptions and caveats that it’s a fool’s errand.

Let me illustrate this with an example drawn from the work done by folks with whom I work.  We know, from several years of repeated data collection and analysis by different researchers, that more frequent use of technology is strongly associated with higher levels of student engagement.* But even with all of the data we have collected, the rigor of our data collection methods, and the sophistication of our analyses, we haven’t yet figured out what exactly causes these measures to be correlated.  In other words, although we know that students who frequently use technology do better in many different ways we don’t know why that happens.  There are many different possibilities but even after 10 years of poking at this we don’t have any explanations upon which we can hang our hat and say, “That’s it – that’s why!”

It’s interesting and instructive to read not only the First Monday article, the response from the AERA poster session author, and the response from the FM authors.  I am hopeful that we will see more sophisticated and better planned research and I am more hopeful that this will occur if those who are most knowledgeable of college students and American higher education continue working and contributing to this discussion.

* In the context of this discussion I must emphasize that although we do ask students about their grades our focus is almost always much wider than just that one measure; in fact, we see broadening discussions of educational quality beyond simple measures such as grades or rankings as one of our primary missions.  I also add that we typically don’t specifically ask in any of our surveys about SNS use.  We do have a set of experimental questions out right now that asks about this but if I recall correctly the question is limited to communication about academic issues as we’re exploring how students and faculty communicate and collaborate.  Our colleagues at UCLA have explored this general issue, however, and it’s worth looking at their work if you haven’t already done so.

If I understand Frank’s letter correctly, he is pointing out some resources from the 1960s, 1970s, and 1980s that  describe student affairs’ use of technology during those years. Rather than in any way refuting my main point, these documents provide further evidence supporting my main point: student affairs staff made regular and often innovative use of technology throughout the 20th century. The documents and events described by Frank are valuable additions to our collective bank of resources and knowledge and it’s wonderful that he has described them for us!

I add two additional comments, one in response to Frank’s letter and one more general in nature. First, it’s not at all surprising or unusual that these particular sources were not included in my original article. Logistically, it sounds as if some of these documents are a bit off-the-beaten-path, particularly for research that was physically conducted in the Midwest (I’m at Indiana University and the bulk of this research was conducted at the National Student Affairs Archive in Bowling Green, Ohio). In addition, all researchers must place realistic and workable limits on their research. In historical research, this means that we specify from which documents and sources we are going to pull information when we tell our story. In this instance, I am satisfied with the sources selected to best tell this story (ACPA and NASPA conference proceedings and journal articles); there are certainly additional sources that could be added (I would particularly like to get into the conference proceedings for the Association Personnel and Guidance Association (APGA), the umbrella organization to which ACPA belonged for several decades) but I have to place limitations on the sources used if I am to make sense of them. In other words, I can only ready and synthesize so much and I must select my sources and use my time wisely. Of course, if there were sources that could really change, shape, or inform the story then I would be remiss to not include them even if they were not originally on my radar!  Based on the available evidence, I don’t think this is the case here as these documents seem to fully support the story as I already understand it.

Second, it’s also useful to know a little bit about the impetus for my article and the context in which it was published. For scholars, it’s often useful for us to “put our mark” on topics on which we are actively working to let others know what we’re doing and in what topics we consider ourselves to be knowledgeable. One of the ways in which we do this is by publishing shorter pieces when we’re not quite ready to publish longer, intensive pieces. In this instance, I was ready to make public that I’m doing this historical work while I continue to work on the longer detailed pieces in which I present my full arguments and supporting evidence. Student Affairs On-Line is not, in my opinion, the right place to publish a fully-developed and lengthy scholarly article but it’s a great place to publish shorter, more informal pieces. And one consequence of this being a shorter less formal piece is that I did not present all of my arguments and evidence; it’s a careful balance to present enough to be interesting, engaging, and accurate without going too far and making the piece too intricate and detailed for the medium and the stage at which I’m at with the research.