May 8, 2024
Hamdollah Ravand

Hamdollah Ravand

Academic rank: Assistant professor
Address:
Education: PhD. in ٍEnglish Language Teaching
Phone: 9132484429
Faculty:

Research

Title
Investigating Factors Affecting EFL Writing Assessment Scores: A Generalizability Approach
Type Thesis
Keywords
Writing Assessment, Generalizability theory, Classical test theory, Decision- making study,
Researchers ALI KHODI, Hamdollah Ravand

Abstract

The present study attempted to explore factors affecting EFL writing assessment scores with generalizability theory (G-theory) to investigate improvement of assessment methods. To this purpose, one hundred and twenty students participated in both independent and integrated writing tests. Then their performances were scored by six different raters namely: self, three peers, and two instructors using the TOFEL scoring rubric. The relative and absolute contributions of different facets such as student(S), rater(R), task (T), method of scoring (M), and background of education (B) were investigated by implementation of G-theory using EduG, a G-theory software package. The results indicated three major sources of variance: (a) the student by task by method of scoring (nested in background of education) interaction (STM: B) with 31.8 % contribution to total variance. (b) The student by rater by task by method of scoring (nested in background of education) interaction (SRTM: B) with 26.5% of contribution to total variance and (c) the student by rater by method of scoring (nested in background of education) interaction (SRM: B) with 17.6% of contribution . With regard to the G-coefficients in G-study (relative G -Coefficient ≥ 0.86), it was also found that the result of assessment was highly valid and reliable. The sources of error variance were detected as student by rater (nested in background of education) (SR: B) and rater by background of education with 99.2% and 0.8 % contribution to error variance respectively. Additionally, ten separate G-studies were conducted to investigate the contribution of different facets across rater, task, and methods of scoring as differentiation facet. These studies suggested that peer rating, analytical scoring method and integrated writing tasks were the most reliable and generalizable designs of the writing assessments. Finally, five decision-making studies (D-study) in optimization level were conducted and it was indicated that at leas