Completed surveys from two of the eleven institutions participating in the first wave of data collection have arrived. Now I’m working with my colleagues in IU’s Center for Survey Research (CSR) to transform these from a stack of completed surveys into an SPSS data file. One of my colleagues in CSR likened this process to alchemy and I think he’s right!
One of the final steps in creating my survey instrument was to send it down to CSR for them to review and reformat it so their scanners can read it. The main part of that process involved setting up their scanning program to read this instrument. Not only did they have to indicate where to look for responses but also what the responses mean (i.e. a mark in this specific area is response number 3 to question 1). This also involves telling the program how to record the responses (i.e. response number 3 to question 1 generates a value of “4” for the “compuse” variable). As can be surmised from the previous example, this setup process also includes naming and defining all of the variables that will eventually end up in the SPSS file.
Just as interesting and important as the automated processes are the manual processes that must be created, documented, and enacted. Most of these are quality assurance or error checking processes. For example, after a batch of surveys is scanned someone must manually review the places where the program is unsure (i.e. a large checkmark that spans multiple response boxes) or the response was too faint for the scanner to properly record (all “missing” values are checked to ensure they are actually missing and not a scanning error). There are also a few points in the process where results are manually double-checked to provide quality assurance.
When the instruments are scanned, the data are inserted into a database. Then the data have to be extracted from the database and inserted into an SPSS file. Once the SPSS template is created (and checked and double-checked), inserting the data is fairly trivial. It can get a bit tricky, however, if you’re merging in data from other sources. In this instance, we’re merging the results from this survey with the results from these students’ BCSSE surveys but I’ll do that on the back end using SPSS instead of doing it on the front end with a database query; that will make it easier for me to merge these data into the institution-specific data files we return to participating institutions. It’s also something I can do myself which gives me more control over and understanding of things (I don’t touch the database; that is all CSR).
There are a lot of small details not described in the above overview and I’m really enjoying learning about this entire process. It’s nice that my survey is a relatively small one: ~1600 one-page instruments. That allows me to be very hands-on which (a) ensures that I understand the whole process and (b) saves me money because I don’t have to pay someone else to do these things.
There are still some unanswered questions, mostly those surrounding what to return to participating institutions and when to do so. I wish I had an answer to some of those questions but I don’t. Part of this is caused by the fact that I spend almost all of my time working on NSSE or occasionally FSSE where data are collected and reports generated at pre-determined and coordinated times. BCSSE, on the other hand, uses a rolling schedule where we generate reports and return data to institutions as we receive their data. That might not sound like a big difference but it’s not just a different process but a different mindset, one I had not fully anticipated or appreciated.
Finally, it’s tremendously exciting to finally see data! We’re going through several test runs to ensure everything is set up properly and I understand how everything works. I’ve been able to glance at a handful of surveys during testing but it was finally real to me when I received the first (test) SPSS file with MY data from MY survey instrument. It sounds silly to admit that a screenful of numbers is exciting and even exhilarating but it’s true. I have quite a ways to go but through the haze I’ve glimpsed the light at the end of the tunnel reflecting off something shiny in the far, far distance.