Greg Wilson and I talked about the possibility that I could work on developing assessments for Software Carpentry (so we can easily test how effective new pedagogical techniques are, like pair programming). Here are ideas that I am looking into:
(1) Define the learning goals and objectives (LO)
Using the existing LOs on the SWC website, those from Jorge Aranda’s and Julie Libarkin’s assessments, and some of my own ideas, create a list of the top 30(ish) LOs for SWC. Survey the SWC tutor community to see what they believe to be the most important LOs are, and what LOs we might be missing. Narrow it to 20(ish) LOs. Iteratively survey the tutor community until we have a commonly accepted top 5-10 LOs for SWC. These should be relatively stable objectives, which are only redefined and surveyed on a 6-month or yearly basis.
(2) Modify the Likert-scale survey
The existing Likert-scale surveys for SWC assess student perception of what they have learned. This information is very useful, because it can speak to the student comfort and confidence levels with material. We do, however, need to modify this survey so that it aligns exactly with the LOs identified in (1) – once we have them solidified. We also need to break the scaling items into basic subjects, so that if an instructor doesn’t cover a topic (like, we didn’t do database stuff at Scripps), then the assessment can be easily adapted for just the topics they covered.
(3) Develop an observation assessment
Julie has been teaching me about different types of assessments. One is using observation. I want to try this at the Chicago workshop.
(4) Develop an assessment of actual student learning
Currently, SWC is not testing if students actually learned material; we are only assessing student perception of what they learned. We need to create this second type of assessment as well. I think we will probably need to start with open-ended questions, which we may be able to develop into (easier to process) multiple choice questions over time. This process must be iterative.
(5) Develop an interview assessment
Possibly build this assessment off of Jorge Aranda’s work. Julie is teaching about this in the class of hers that I am taking next semester, so I’ll know more in the next few months. There should be plenty to do between now and then. But I do want interview people approximately 6 months after their participation in SWC.