Quantitative Analysis

Description of Analysis Procedures (Method)
            NEH focus group participants’ and the contrast groups’ answers to essay questions were scored according to a rubric adapted from the Ideas and Content section of the Six Trait Writing Analysis (Spandel, 1996), an assessment tool created by the Northwest Regional Laboratory.  Answers to the essay questions were scored on a five point scale.  Scores ranged from 0 to 5 and were graduated into units of .25. Scores of 0 were given only when there was no attempt to answer the essay question. Total scores ranged from 0 to 65.
            The scoring process was conducted by the two graduate students both of whom had participated in graduate level courses that included instruction and utilization of this scoring method prior to the study.  The scoring procedure was adapted from recommendations made by Jentzsch et al. (1991). To increase inter-rater reliability, the raters scored all of the essay responses independently.  The following steps were utilized:  (1) Raters became familiar with the criteria for the rubric. (2) Raters read the entire selection of writing samples to become familiar with the range of writing context. (3) Raters scored essay questions, referring to the rubric criteria frequently to ensure that the ratings remained constant throughout the scoring process. Upon completion of independent scoring, the raters’ scores were compared.  The rating results showed scores that matched or stayed within a .5 range per essay question.  When scores were not matched, researchers used the mean of the two individual numbers as the final score.

Results

           Total Scores for the Self- Efficacy Expectations Survey and the Content Mastery Essay Questions are presented in Tables 1 and 2, revealing that the majority of scores for the contrast group were lower than those of the treatment group on both measures.

Individual scores for the contrast and participant groups were compared using the Mann-Whitney U Test.  The Mann-Whitney U Test is a non-parametric statistical test that can be used to determine whether there are significant differences between two populations.  Both tests indicated statistical significance, with a U value of 0 for the content mastery and a U value of 20 for the self-efficacy scores (critical U value=25).  This finding indicates that the range and rank of scores was not due to chance.
            Further, the Spearman Rank-Difference Method for calculating the correlation between two variables indicated a strong correlation between self-efficacy and content mastery scores across groups at .01 (R_s=.629). These findings
suggest that a correlation exists between measures across groups.
             The difference between groups on the self-reported efficacy measure is more clearly illustrated in Figure 1, where modal scores for each of the fifteen self-efficacy questions comparatively show the differences between groups.  For most questions, the mode score was lower for the contrast group.  This indicates that teachers who have not completed 36 or more hours of Holocaust or genocide related workshops self-report lower efficacy.  The range of modal scores for the participant group was 4-5 and for the contrast group, the range was 2-5.

Figure 1: Modal Scores on Self-Efficacy Test for Questions 1 through 15 

            The data was further analyzed by a t test, in order to compare the values of the participant group against those of the contrast group.  On all but two of the fifteen questions we found a significant difference.

Discussion of Quantitative Results   

              Thus far, analysis indicates the NEH workshop participants have significantly higher scores in self-reported efficacy.  They believe that they can do a good job of teaching students about key issues in Holocaust/genocide studies and their self-reported efficacy is closely correlated with their knowledge of the content. 
            The correlation between the self-efficacy and content mastery scores across groups holds profound implications for those who fund and/or provide study opportunities on Holocaust/genocide topics.  The data suggests that educators like those in the contrast group, who for reasons similar to their peers teach or wish to teach about the Holocaust and genocide, may not be equipped with adequate knowledge and understanding of the subject without a comparable NEH study opportunity.   This is reflected by both the correlation of their scores from the Spearman Rank-Difference Method and the overall lower scores indicated by the ranking Mann-Whitney U Test (U=0).  The distinction between groups was quite clear, as the highest 14 content mastery scores were all from workshop participants.  We hypothesize that the correlation between self-efficacy scores and content mastery scores found within both groups implies that the teacher’s improved confidence is grounded in real knowledge of Holocaust/ genocide studies.
            The modal self-efficacy scores further suggest that educators in the NEH focus group believed more strongly that they could do a good job teaching Holocaust/genocide topics, knew the necessary step to teach Holocaust concepts effectively, and understood Holocaust/ genocide studies concepts well enough to be effective in teaching this subject to their students. 
            As mentioned earlier, all but two of the fifteen questions yielded significant difference (p = .05).  One of the two questions was question 8 (p= .077), which read, “Given the opportunity, I believe that I could effectively work with a colleague to develop interdisciplinary approaches to teaching about the Holocaust and related genocide issues.”  The other question was number 12 (p= .12), “I believe I can do a good job of teaching students about hate crimes (past and present.)”  It is speculated that this is due to sampling error. It is possible that question 8 elicited high responses from both groups because it reflects an answer that is socially desirable. It is hypothesized that the wording of the question did not only address self-efficacy of the individual but also the ability of participants to work with colleagues. In the case of question 12, it is hypothesized that the difference between groups was less pronounced due to lower self-reporting by the NEH focus group participants.  It is possible that their scores reflect the NEH focus groups’ recognition of present day opposition to controversial issues, such as homosexuality. We suspect that through discussion at the seminars, participants may have developed a reluctance to cover this sensitive topic because of potential risk for creating controversy amongst fellow colleagues, administrators and/ or parents.