obsessing about assessment
The last few days when I have been at school working, I have been trying to figure out how to get testing information for my students to evaluate the tests we use and the ones we have created this summer. The problem is we do not have an assessment center with a person knowledgeable about testing and measurement. We no longer get detailed reports on scantron tests since we no longer have a mainframe computer. We used to be able to get a lot of information including student performance on different types of items on a test such as all the main idea items. This capability is no longer available. We use a scantron in our department to give us the score on a test and how many items are correct out of the the total number of items.
This clearly is not enough information for teachers or students. So I have started investigating another scantron scanning source available to faculty. This source requires me to learn how to learn how to run the program that reads the scantron results and gathers the data. However, there is no handbook nor directions except for a cheat sheet that a math instructor worked out. I am going to try to figure out how to use the program by trial and error to determine what kind of data I can get. I then will need to analyze the data, so I have been trying to find out more about item analysis. I downloaded a spreadsheet for a simple analysis today and tried it on our first reading comprehension test. It provided some basic analytical data. Our results weren’t so good, but we have found the scores going up on the tests as the students become comfortable with them. I will try the spreadsheet out a couple more times and see what kind of results we get. However, I still have a problem and that is figuring out how to match up the item types to the different items on the test.
There must be some more sophisticated applications available to do this, but I still want to know what we have at school. My source should be back from a conference soon, and maybe he can tell me. Since I think this is going to come down on us more seriously in the future with the increased emphasis on measuring learning outcomes from SACS (our accrediting organization), I hope to get a little jump on that. More importantly, since there are tools available for giving us an idea of how well our tests are working, it seems like we should be using them. Since I am the only full time person in my program, it is pretty much up to me.
While I remain skeptical about the use of quantitative data to make major decisions about people, I think we should use the tools we can to make our teaching better. I must admit I have been sitting back waiting for people to do this and because I have struggled to get a lot of things done, but waiting hasn’t worked.