Using apps in revising a reading curriculum
One of my goals this summer is to revise the curriculum for our intermediate reading class. We have a few problems with the class that make it more difficult for the students than it needs to be. First, we have a final exam that has been very difficult to pass with a one third passing rate the norm. Also, we have reading comprehension tests that are outdated. We have a difficult textbook and uneven support for the textbook, and we do not have enough vocabulary being cycled and recycled in the class.
At the end of the first summer session, I took the final exam, which I had made attempts at revising before, and looked at the data on questions that caused difficulty on the test, that is less than 60% of the test takers were correctly responding to the item. Out of 40 items, 17 were in my category of too difficult. I revised the test by revising many of the items unless the item seemed to be a case of students not being able to answer correctly because they did not have the proficiency, which sometimes seems to occur with main idea items. The revisions sometimes involved using more frequently used words in the items and distractors, revising distractors to remove those too closely related to the correct answer. The results seemed to accurately reflect proficiency; that is, instead of about one third of the class passing the test, a good majority of the students passed.
The data I used for the revision of the test came from results of previous that that I entered by hand in Excel and calculated the success or failure on different items. Recently, I downloaded and used an item analysis application, jMetrik. Included in the information on the site is
jMetrik is a pure Java application that runs on Windows, Mac OSX, and Linux platforms. System requirements include 256MB of available memory and Java 6 (i.e. JRE 1.6) or higher. Almost all computers created since 2008 meet these requirements.
So I can run it on my Mac at home or my PC at the office. I had a problem with the application and wrote to the professor who wrote the application, J. Patrick Meyer, an Assistant Professor at the University of Virginia, who responded on the same day and provided the information I needed to finish an analysis.
I used the application to analyze the results of the first new reading comprehension test. From the analysis, I found two items that seem to need revision because few students were able to correctly answer them. We had a similar experience the previous summer in writing reading comprehension tests for the upper level reading comprehension tests. Some items look good until they are tested; then we wondered “what were we thinking?”
When I contracted a teacher to write the comprehension tests, I gave her guidelines for the type of items for each of the four tests. The guidelines consisted of the types of skills to be tested on each test. I also gave her some suggestions on where to look for texts such as Discover Magazine and English Articles. After deciding on an article, she used Vocab Profile on the Lexical Tutor site to analyze the vocabulary in the text. I asked her to take the words that are Off Types, that is, not on the more frequent words or Academic Word List, and use EasyDefine to help construct a pre-reading vocabulary list. I am piloting the reading comprehension tests during the second summer session.
Vocabulary has been a problem because the textbook includes words for vocabulary study that are not appropriate our the students. For example, 23 of the 28 words for vocabulary study in chapters 3 and 4 are off list words including words like homage, sociopathic, and charisma. In the fall, we are adding Academic Vocabulary in Use by Michael McCarthy and Felicity O’Dell for the class. The book is primarily a self-study book, so I will work on some ideas for teachers to incorporate the book into the curriculum. That was one of my goals for the second summer term before I was pressed into service in teaching a second class. I will still do it, but I fear it won’t be as well thought out.
With the revisions to our upper level reading class appearing to work from what the teachers have told me and my own experience teaching the class for a semester, I look forward to finding out if the changes in the mid-level class will work out also.