Mistaken Goal: Where Higher Education & Technology Meet

I don’t often write about what I do in my day-to-day job.  But I’ve recently spent quite a bit of time working on survey item non-response and survey abandonment and I want to save you some time if you’re working on those issues, too.

One of the projects on which I’ve worked over the last couple of years is the development of an updated version of the National Survey of Student Engagement (NSSE) survey instrument. We’ve done a lot – a LOT – of work on this.  As part of this work we’ve pilot tested the draft versions of the new survey.  Some of the many things we’ve analyzed in the pilot data are item non-response and survey abandonment.  I worked on this last year with the first pilot and when I worked on this again with this year’s pilot I got smarter.  Specifically, I wrote an Excel macro that generates the SPSS syntax necessary to analyze item non-response and survey abandonment.

As described in the Excel file, this macro takes a list of survey variable names and creates SPSS syntax that will add several new variables to your SPSS file:

• A “Abandoned” variable indicating the last question the respondent answered if he or she abandoned the survey. If the respondent didn’t abandon the survey, this variable will be left empty (“SYSMIS”).
• For every variable, a “SkippedItem__” variable indicating if the survey item was answered, skipped, or left blank because the survey was abandoned.
• A “SkippedItems” variable indicating the total number of questions the respondent skipped.
• A “SkippedPercentage” variable indicating the percentage of questions the respondent skipped.
• A “AbandonedPercentage” variable indicating the percentage of questions the respondent did not answer because he or she abandoned the survey.

I created this macro because there were several versions of the pilot instrument.  Because you have to “work backward” through each question to identify respondents who abandoned the survey, each version of the instrument required a different set of SPSS syntax because each version had a different set of survey questions.  So it was much easier for me to write a program that generates the appropriate syntax then to do it by hand multiple times.  Laziness is a virtue.

Warning: This macro generates a lot of syntax.  The sample input has only four variables but it creates code with 105 lines (including blank lines and comments).  The surveys with which I was working had 130-160 variables and I worked with 11 different versions of the survey instrument.  In the end, I had an SPSS syntax file with tens of thousands of lines of code.  The SPSS syntax editor got very grumpy and slow, probably because of the large number of DO IF conditionals and the syntax highlighting it applies to those blocks of code.  I ended up working mostly in Notepad as I was troubleshooting the syntax and pasting the resulting text into the SPSS syntax editor only when I was ready to run it.  The good news is that the syntax is actually very straight-forward and arithmetically simple so it ran fairly quickly.

I know that this fills a very, very small niche.  But maybe someone will find this helpful or useful.  I spent a few days working on this so there’s no reason why someone else should have to redo this work.

Comments(2)

 This post will be less organized than most posts; some of these thoughts and ideas are still a little raw.

Backward design – the method by which one begins with the desired end result(s) of an educational program, determines acceptable evidence showing that the result(s) has been achieved, and then creates a plan to teach the skills and content that will lead students to provide that evidence – has been on my mind lately.  It’s one of the core concepts of a college teaching and learning course I co-teach but that’s not why I’ve been thinking about it.

For me, backward design is a “threshold concept;” it’s an idea that changed how I think about teaching and I can’t go back to how I thought prior to this change.  So although I learned and most often use and teach backward design in the context of designing or redesigning a single college course, I’ve been thinking about the role of backward design in different contexts.  For example:

• I know that backward design has been and is used to develop curricula and not just individual courses.  Today was the first time I got to see firsthand how that plays out with a group of faculty to develop a full 4-year curriculum for this discipline.  I was most struck by how difficult it was to keep true to the backward design philosophy and not get mired down in content coverage and the limitations imposed by the current curriculum.  It was difficult even for me to remain on course as I tried to help facilitate one of the groups of faculty engaged in this process.  I underestimated the increased complexities involved in scaling up the process from a single course to an entire curriculum; it’s not a linear function.
• There has been quite a bit of discussion lately among student affairs professionals regarding their conference presentations (e.g. this Inside Higher Ed blog post with 30 comments).  Put bluntly, many people are unsatisfied with the current state of these presentations.  Just as backward design can scale up from a class to a curriculum, it can also scale down to a single class session.  And shouldn’t a good 50 minute conference presentation resemble a good 50 minute class session?  So why not systematically apply backward design to conference presentations?  Many conferences seem to try to push presenters in that direction by requiring them to have learning outcomes for their sessions but that isn’t enough.
• Unfortunately, pedagogy and good teaching practices are not formally taught and emphasized in most student affairs programs so I expect that most student affairs professionals have not been exposed to backward design as a formal process.  That’s a shame because it seems like such a good fit for what student affairs professionals do!  And it fits in so well with the ongoing assessment movement because it so firmly anchors design in measurable outcomes and evidence-based teaching!

Would any student affairs professionals out there want to learn more about backward design and try to apply it to some of your programs?  Please let me know because I’d love to help!  I’m positive this would work out well and I’d love to test these ideas!

Comments(7)

At a presentation I attended at this year’s ACPA conference, the presenters discussed technology as a competency for student affairs professionals.  It’s a discussion that’s been going on for many years but I don’t know if many people – particularly younger professionals – know just how long it’s been going on.  The presenters of this particular session asserted that formal discussion of technology as a competency began in 2002.  Maybe they’re right but informally and on different levels this conversation has been ongoing for decades. To provide historical context for this discussion (and to substantiate some glib comments I made to those sitting next to me in the presentation), I skimmed through my historical documents to find the earliest occurrences of this discussion.

Although there is foreshadowing in the middle of the 20th century of calls for technology competency in student affairs professionals, the first explicit calls I found begin in the middle of the 1970s.  In “Dealing with the Computer,” Penn (1975) asserts that “If the modern student personnel administrator expects to provide leadership and to have an impact on his or her campus, it will be necessary to understand computers and to communicate with computer technicians” (p. 56).  He goes on to write that “the functioning of computers is still a mysterious process to many individuals” (p. 56) before going on to define and briefly discuss topics such as “hardware” and “software.”  Similarly, Peterson’s 1975 NASPA Journal article “Implications of the New Management Technology” recommends that student affairs professionals not only “familiarize [themselves] with [their] institution’s data base, its automated technology, the major administrative analytic offices, and the major reports they generate” (p. 169) but they also “develop [their] own capacity to assess, analyze, and/or use some of the more basic data sources at your disposal” (p. 169).

By the 1980s, technology as a competency was a clear concern for student affairs professionals in the U.S. In the mid 80s, several student affairs departments were engaged or interested in increasing the computer literacy and comfort of their staff (e.g. Barrow & Karris, 1985; Bogal-Allbritten & Allbritten, 1985).  In a 1983 survey of 350 student affairs departments at 2-year colleges (with 141 respondents), the second need most frequently expressed by chief student affairs officers (CSAOs) was “information about basic computer functions, computer literacy, and how to write microprograms” (Floyd, 1985, p. 258).  In 1987, Whyte described the results of a similar survey of 750 colleges and universities (with 273 respondents):

Many student affairs professionals have expressed mixed emotions regarding computerization in the educational realm. There seems to be a need for direction regarding how to coordinate computerized management, instruction, and evaluation capabilities into a meaningful, comprehensive package to assist students….Coordination of the fragmented computerization efforts of most student affairs offices into a comprehensive plan is the next logical step. (p. 85)

In describing the “Three Rs” of recruitment, referral, and retention, Erwin and Miller (1985) wrote that “to meet the changing times and increased demands for excellence, student service professionals must look for new tools to assist in problem solving. Administrators will find management information systems particularly useful…” (p. 50).  Finally, MacLean (1986) explicitly calls for computer technology (then referred to as “management information systems”) to become “integral parts of all student affairs offices and departments” (p. 5).

Calls for student affairs professionals to develop and increase their knowledge of and comfort with computer technology are decades old.  Even a quick glance through my limited resources shows implicit and explicit calls beginning in the 1970s and blossoming in the 1980s as (micro-)computers became widely available and mainstream.  The discussion has changed tenor and intensity as technology has become more intertwined with our lives but the discussion itself is not new and dates back at least 35-40 years.

References

Barrow, B. R., & Karris, P. M. (1985). A hands-on workshop for reducing computer anxiety. Journal of College Student Personnel, 26(2), 167–168.

Bogal-Allbritten, R., & Allbritten, B. (1985). A computer literacy course for students and professionals in human services. Journal of College Student Personnel, 26(2), 170–171.

Erwin, T. D., & Miller, S. W. (1985). Technology and the three rs. NASPA Journal, 22(4), 47–51.

Floyd, D. L. (1985). Use of computers by student affairs offices in small 2·year colleges. Journal of College Student Personnel, 26(3), 257–258.

MacLean, L. S. (1986). Developing MIS in student affairs. NASPA Journal, 23(3), 2–7.

Penn, J. R. (1976). Dealing with the computer. NASPA Journal, 14(2), 56–58.

Peterson, M. (1975). Implications of the new management technology. NASPA Journal, 12(3), 158–170.

Whyte, C. B. (1987). Coordination of computer use in student affairs offices: a national update. Journal of College Student Personnel, 28(1), 84–86.

Comments(7)