I Don’t Trust This Article – And Here’s Why

On Friday, a colleague pointed out a new article on Mashable that is titled “Why Tablet Publishing Is Poised To Revolutionize Higher Education.” I don’t trust the claims made in this article. I’m going to explain why I don’t trust the claims, not to convince you that my opinion is correct but to give you an understanding of how I evaluate claims like the ones made in the article. I’ll lay out my thoughts in chronological order.

  1. The article is published at Mashable. I removed Mashable from my RSS reader over a year ago because I got tired of their poorly-written articles that make ridiculously overwrought and unprovable claims. This certainly isn’t enough for me to condemn this particular article but it certainly makes me cautious right from the beginning.
  2. The title makes a very bold claim. Many people have attempted to “revolutionize” education; few have succeeded. And even fewer have been able to explicitly predict revolutions before they occur or even recognize them as they are occurring. The author has a helluva case to make and he better bring remarkable evidence to support his claim(s).
  3. After reading the title, a quick glance through the article indicates that it’s a utopian piece largely based on the idea of technological determinism. In other words, it’s not only wildly optimistic but it also relies on the idea that we can predict and control how people use technologies by the way in which those technologies are designed. Both of these ideas – utopia and technological determinism – have a bit of history in the field of social informatics. The history is mostly negative; these ideas simply don’t work most of the time. So my skepticism continues to increase.
  4. The author of the article is an executive at Adobe. In fact, he’s the “director of worldwide education.” That doesn’t mean that his opinions are necessarily biased but it’s another reason for me to be skeptical.
  5. The article claims that “[There are] better study habits and performance with tablets.” Only one study is cited to support these sweeping claims: a Pearson Foundation “Survey on Student and Tablets.” For example, the author states that “86% of college students who own a tablet say the device helps them study more efficiently, and 76% report that tablets help them perform better in their classes” and a few other claims. Even if this study were flawless, the author needs a whole lot more evidence to support such a broad claim.
    1. To their credit, Pearson offers to share methodological details about and data from their survey if you just ask them; I haven’t asked so I don’t have any more detail than what is provided in that 2-page overview. But we do know that the survey was conducted online. Given that about 20% of people in the U.S. do not have access to the Internet (Dept of Education estimates 18.6% and the Pew Internet & American Life Project estimates 21%), it seems unlikely that an online survey can produce data that is representative of the entire population. It seems particularly problematic to omit non-Internet users when asking about technology since the results will almost certainly be skewed.
    2. Even if we accept that the Pearson numbers are accurate or in the right ballpark, I’m still not sure if they’re very informative. I guess it’s interesting that many young people think that tablets will help them study more efficiently and that they will replace textbooks in the next five years. I just don’t think that we can use these data to make any predictions.
    3. Let’s ignore the validity issues for some of Pearson’s data (e.g. people are notoriously bad at distinguishing between “what I like” and “what is most efficient/effective) so we can move on.
  6. The authors correctly assert that digital textbooks can include more features than printed textbooks, including “video, audio, animation, interactive simulations and even 360-degree rotations and panoramas.” However, the author does not say how we’ll produce all of that additional material. I don’t expect the author to solve every challenge associated with his predicted revolution but it would be nice to at least acknowledge them instead of glossing them over or ignoring them entirely.
  7. In the next section of the article, the author claims that “interactive learning leads to better retention.” The only evidence cited is this news article about a study of elementary and high school students using 3D technology in science and math classes. Of course, since I’m an academic snob I think it would be much better to cite a primary source, preferably one that has been peer-reviewed, than to rely on a popular press article. Once again, even if we accept that this study is perfect it’s not even close to being enough to support such a broad claim.
  8. Next, the author claims that digital publishing can help us better “[understand] learning effectiveness” using “integrated analytical tools.” I have no issue with this claim as a broad theoretical claim. But it seems to completely bypass the fact that U.S. higher education is in complete disarray in terms of even settling on broad learning objectives much less specific objectives and associated assessment tools or indicators. (Look into the “tuning project,” especially the “Tuning USA” project, to get an accurate view of these issues.)
  9. The next claim the author makes is that “digital publishing makes knowledge more accessible.”
    1. The author must be using “accessible” in a different way than I commonly use it because it’s hard to take that claim seriously given the (a) lingering digital divide, participation gap, and similar inequities in the U.S. and (b) the immense resistance many digital publishers have exhibited to making their content accessible to the visually impaired.
    2. Once again, the author focuses solely on a possibility offered by the technology without giving any thought to the cultures in which the technology is embedded. He writes that “digital publishing allows professors or subject matter experts to self-publish their own educational materials or research findings and distribute the information on tablet devices” without offering even the barest hint about how this will occur without adjusting or overturning the systems that would need to support this. In other words, why would faculty do this? What is the incentive?
    3. Similarly, the author claims that “by harnessing interactive technologies, educators can explain even the most complex scholarly or scientific concepts in compelling and intelligible ways.” Once again, I accept this broad claim (ignoring the “even most complex” qualifier because it’s just silly) in theory but balk at it in practice. It takes complex skills to create effective interactive content, skills that are different from those possessed and valued by faculty in many disciplines.
  10. At this point I’m just tired of reading these grand claims supported by flimsy or no evidence…

I’m not a Debbie Downer or a Luddite. I agree with the broad proposition that digital publishing has potential to make a huge impact on U.S. higher education. And I agree that tablets are super cool and very useful in some circumstances; I purchased an ASUS Transformer a few months ago to replace an ailing netbook and I’m very happy with my purchase! Fundamentally, I distrust the claims made in this article because the author fails to support them. Even when the author provides cherry-picked examples and studies, they are often of poor quality and always insufficient to support those claims. This is quite disappointing since the author could have easily drawn upon the large and rapidly-growing body of evidence in this area. I expect very little from an article published by Mashable and this article delivered.

Impact of Anti-Social Networking Website Legislation on Higher Education

We’ve seen and are continuing to see attempts by state and federal legislators to restrict the use of social networking sites like Facebook and MySpace on computers with Internet access funded by the (state or federal) government. Although DOPA was not passed into law last year, Sen. Stevens introduced a similar law earlier this year. State legislators in Connecticut and Illinois have introduced similar legislation.

I am neither a lawyer nor an experienced policy junkie so my understanding of the detailed specifics of these proposed laws is likely incomplete and possibly outright wrong in some areas. As best as I can tell from reading the bills and the media reports surrounding them, the federal bill, the Protecting Children in the 21st Century Act, requires that minors not be allowed to use computers with Internet access funded by the federal government (e-rate) to access “social networking websites” and “chat rooms” without parental permission. Connecticut’s law levies a $5,000 fine on social networking websites that fail to verify the age of participants and require parental permission for minors to participate. Illinois’ proposed law is the most stringent; it requires that “each public library must prohibit access to social networking websites on all computers made available to the public in the library [and] each public school must prohibit access to social networking websites on all computers made available to students in the school.”

As a university administrator, I wonder if we have paid enough attention to these and similar proposed laws to gauge their impact on our pre-matriculation programs. In other words, would these proposed laws have an impact on online orientation or similar programs that are aimed at applicants, interested high school students, and other minors? The answer seems to be a clear “yes” for those institutions whose programs have taken on characteristics of social networking. However, the precise definition of “social networking website” has yet to be crafted; for the federal law, it will take into account if the service:

  • is offered by a commercial entity
  • permits registered users to create an on-line profile that includes detailed personal information
  • permits registered users to create an on-line journal and share such a journal with other users
  • elicits highly-personalized information from users
  • enables communication among users

For some of the proposed bills, requiring that the users supply proof of their age and secure parental permission for minors to participate would satisfy the legislative requirements. That seems like a low bar for colleges and universities, particularly if the users are those who have already applied to the institution and thus already supplied proof of age; it’s only one more bullet point on the application form signed by the applicant and, if necessary, mom or dad. Of course, this completely dodges the question of how the laws would actually be enforced in libraries and schools, how easy it will be to overcome the necessary technological filters and restrictions, what evidence would be necessary for librarians or teachers to allow minors to access social networking sites, etc.

The primary concern of many who perform research into youths’ use of social networking sites is the disparate impact this legislation would have on youths whose primary Internet access occurs at school or the library. That concern should hold true for college and university administrators as this proposed legislation would have a negative and disproportionate impact on prospective students with a low SES. It seems to me that this legislation may strengthen continued concerns about the widening SES gap in America between those attend college and those who do not. Further, this proposed legislation may harm efforts to attract students with low SES and help them fit into the college environment in that crucial first year.

Colleges and universities must monitor this area of legislation. Not only does it impact current and developing programs such as online orientation programs and cutting edge recruiting efforts, it may intertwine with the continued debates about widening SES gaps and efforts to shrink those gaps.

2006 ECAR Study of Undergrads & IT

ECAR, EDUCAUSE‘s research arm, recently released the results of their 2006 Study of Undergraduate Students and Information Technology. While most ECAR documents are only available to ECAR subscribers and those who specifically purchase them, ECAR released this study to the public “because of the topic’s critical importance.” While I recommend everyone read through at least the Key Findings, let’s take a look at some of the findings, place them in context with other research, and try to extract some additional meaning from them.

First, without downplaying its strengths and essential validity, we must note the limitations of this study. Freshmen and seniors at 96 institutions were invited to participate (and each institution had to seek IRB approval – 96 separate IRB approvals for one study). The response rate was about 11% for a total of nearly 29,000 students. While that is a large number of respondents the researchers correctly state that their findings may only be directly applicable to the participating institutions and generalizing these results even to those institutions should be done with extreme caution. As with most surveys, the survey also suffered from self-selection bias. However, ECAR also conducted focus groups at 5 participating institutions to gather qualitative data which may have helped to offset some of these limitations. But enough about the methodology – let’s get on to some of the results.

One of the most striking findings of this survey is that computer ownership among respondents is nearly ubiquitous: 97.8% own at least one computer with over one-third (37.2%) of respondents owning both desktop and laptop computers. However, just as we’re finding in American society at large, there is also a very small minority of students who avoid or choose not to use technology. This is a separate group from those who can not afford technology or at least the level of technology they would like. Both of these groups not only present some difficulties for technical support personnel (who must support aging computers, users with uncommonly low technical skills or knowledge, etc.) but they also reveal a segment of American society who may never cross the Participation Gap, never mind the Digital Divide.

Another finding relevant for college and university administrators is that “overwhelmingly…students prefer e-mail [for institutional communication].” This finding should not be surprising. While we know that young people prefer to use Instant Messaging and other media such as social networking sites to communicate with their friends, they view e-mail as something for “old people” and a medium to be used to communicate with “institutions.” Without discussing whether the choice of medium is appropriate (there are very strong arguments that it is), we must admit to ourselves that we are indeed “old people” who work for “institutions.” Thus we can conclude that e-mail is most likely the correct medium for communicating most information to students.

Throughout the study, differences between male and female respondents are reported. For example, when discussing self-reported skill levels, the researchers note that “gender…is an influential factor in explaining perceived differences in skill levels: being male is associated with higher reported levels of skills.” Female respondents (as well as younger respondents) indicated a preference for less technology in their courses. Not surprisingly, “male [respondents] are more likely to be gamers, reporting higher usage of computer and online games.” While there is some evidence that many of these differences can be explained by factors other than gender (personal interests, academic major, economic status, etc.), this study provides evidence supporting the common sense notion that males and females use technology differently. (This is a fascinating area of scholarship)

ResNet professionals may be interested to learn that more than one-third (36.1%) of respondents reported owning a wireless “hub” (quotation marks are necessary as hub is a technical term often misused and likely incorrect in this context). While the report does not break down the different levels of ownership among on- and off-campus residents it does show that the level appears to correlate with age – the older a respondent the more likely he or she is to own a wireless “hub.” Based on that, I suspect the level of ownership may be higher among off-campus residents. But that may be wishful thinking. We know that despite students’ desire for ubiquitous wireless is far from being a reality in residence halls. We also know that wireless is perceived by many ResNet professionals as one of their top challenges. These issues are all summed up by the ECAR researchers who state that “the 1990s battle cry of a ‘port per pillow’ may be getting supplemented this century with ‘a router for every room, or at least a hub for every home!'” We’ll have to discuss the security ramifications of this later.

One surprising item in this study is that “more than 70% [of respondents] use social networking sites such as MySpace and Facebook.” Although there is significant qualitative data in this study supporting the assertion that usage is very high, this survey’s quantitative data regarding reported usage of social networking sites is much lower than reported in most other studies of this specific topic of which I am aware. Why the disparity? Perhaps the data is “stale” (i.e. too old to reflect current trends). Or, more likely, the demographics of this group of respondents differ very significantly from those in the other studies with which I am familiar. For example, this is the only study of which I am aware that included students in 2-year institutions. Given the differences between “typical” students at 2-year institutions and 4-year institutions, many illuminated and discussed in this study, this may explain some or much of the difference. Although I am initially inclined to lend more weight to the larger body of evidence presented by those who specifically study this phenomenon, the sample size of the ECAR study is much larger than most other studies and surveys which lends it considerable weight. In any case, whether it’s “more than 70%” or closer to 90% or 100%, usage is still very high.

Here’s something that will make many student affairs professionals nod their head and smile: When discussing self-reported skill levels, the researchers noted that “students who report learning a skill for employment or personal interest also report higher levels of learning.” Preceding this comment is a brief discussion of students who possess skills not learned or taught in their coursework but acquired through employment, personal interests, or other means. The researchers even included a quote from a student who talks about skills learned through volunteer work with Big Brothers/Big Sisters. Score one for internships, co-ops, volunteer opportunities, (reasonable) employment, and other experiential learning opportunities and those who support and encourage students in finding those opportunities.

Finally, when asked to select where institutions should invest more money in IT, if money were available, nearly 28% of all respondents selected “Music (Napster subscription, etc.).” Moreover, there is a very clear trend that younger respondents selected this response much more frequently than older respondents. While this is an interesting finding, I assert that the methodology significantly weakens this finding as the respondents were asked to select three responses from a pre-defined list of ten possible responses. Nevertheless, this is an interesting finding as the research into entertainment services has thus far been very limited and found mixed results.

There’s a lot more in the full report and I’m sure there any many interesting and important findings that I could not or did not discuss here. It’s a good study and the report is well-written so read it when you have the time.

Divisions and Gaps

Jakob Nielsen‘s latest “Alertbox” article is entitled “Digital Divide: The Three Stages.” In this (very brief) article, Nielsen posits three types or stages of divides that “alienate huge population groups who miss out on the Internet’s potential:”

  1. Economic Divide
  2. Usability Divide
  3. Empowerment Divide

Nielsen argues that the Economic Divide is largely a non-issue in modern America. His other two divides are very similar to Jenkins’ Participatory Divide. In short, both of these researchers believe that significant divides still exist between (a) Internet users and non-users and (b) different groups of Internet users. The two researchers differ in some ways on the exact form and causes of those differences but those differences appear to be more in point of view than significant and substantive differences. As a usability researcher, Nielsen concentrates largely on the user experience and how users interact with particular tools, suites of tools, and technologies. Thus his focus is often on how someone can or cannot use something to perform a particular task. Jenkins, on the other hand, is a communications researcher whose focus lies more on the sociological impact of technologies and societal changes or influences caused, aided, or disrupted by technologies.

One subtlety that is masked by the label “divide” is that these divides are more like continuums than binary, black-and-white issues. Whether one speaks of Nielsen’s Usability Divide or Empowerment Divide or Jenkins’ Participatory Divide, these are areas in which one can have more or less (understanding, power, or rates of participation). Even the seemingly-black-and-white issue of access is a continuum wherein no possible access and high-speed, always-available, unfiltered and uncensored access lie at the endpoints of a continuum with different levels of access in between (borrowed access, slow access, filtered or censored, etc.).

For us, it’s very important to remember that our students will come from both sides of these divides and all places in between. I don’t have good data at my fingertips but I have no doubt that traditional measures of diversity such as race, ethnicity, SES, age, gender, sexual orientation, etc. play huge roles in where one lies in these issues. The role of SES should be obvious. In his exploration of fan culture, Jenkins has noted the role of gender and the differences in how men and women interact with and use technology to interact and collaborate. danah boyd, herself a noted researcher in these areas, speaks of the role that communications technologies (specifically, IRC) played in her experiences as a young queer woman, technologies that may not been available to her had circumstances (location, finances, experiences, etc.) been different. It’s very important for us to begin to understand how (a) access or lack of access to, (b) understanding or lack of understanding of, and (c) use or non-use of these technologies – technologies ubiquitous and essential for many young people – is shaping and influencing youths in America (see the recent debate about DOPA for a great discussion of these issues). I assert that we don’t understand this right now as it’s complex and changing very rapidly. I further believe that applying what we know about young people (both through our experiences and through our research i.e. student development theory) we can uniquely contribute to this discussion and begin to understand its importance.