基于效度整体观的中国高校大学英语分级考试效度研究—以上海理工大学2013年分级试卷为例

VIP免费

3.0 侯斌 2024-11-19 4 4 941.89KB 107 页 15积分

侵权投诉

基于效度整体观的中国高校大学英语

分级考试效度研究

—以上海理工大学 2013 年分级试卷为

例

摘要

语言测试是教育心理测量界的重要概念，其中的一种重要形式即为分级考试。

分级考试可以把不同语言水平的学生编入相应的教学环境学习以期达到合理分配

教学资源、提高教学效果的目的。国内大学通常要求大学新生参加大学英语分级考

试，了解其英语水平，根据英语能力将新生分入不同的班级，配置与级别要求一

致的教师，学习相应的教材，提供多样的教学计划等，从而最大程度上发挥学生

的语言能力。由此，分级测试对大学生的四年英语能力的提高至关重要。因此，在

设计大学英语分级考试时，设计者更须关注该类考试是否具有效度。然而，国内

缺乏对该类考试具体实施效果的研究，更鲜见基于理论对分级考试进行的效度验

证。

上海理工大学（USST）于 2012 年开始实施新生大学英语分级考试

（CEPT），以为该校大学英语分级教学提供参照标准。为提高该校分级考试的设

计，作者对 USST CEPT 2012 进行效度验证。结果表明该分级考试整体上具有效度

但是仍有提高之处。借鉴 USST CEPT 2012 效度验证经验，作者更全面地对 USST

CEPT 2013 进行效度验证，以探究修正后的 USST CEPT 2013 是否在效度方面与

USST CEPT 2012 保持一贯性。

本文采用 Messick (1989) 提出的效度整体观作为研究的主要理论基础。众所周

知，效度整体观在语言测试界产生了重要影响，获得众多语言测试学者的认同。

正如 Bachman (2004: 259) 所赞：“当代教育心理测量界所遵循的效度理论概念要

得益于 Messick 的贡献，其思想对北美及以外的国家的测试学皆产生了重要影

响。”此外，本文还借用 Bachman 和 Palmer (2010) 任务特征框架以及 Weir (2005)

提出的基于证据的效度验证，通过将该两个理论的部分内容融合进效度整体观，

作者提出 USST CEPT 2013 效度验证框架。该理论框架由五部分构成，即构念效度

内容效度、校标关联效度、表面效度和后效效度。

研究结果表明，USST CEPT 2013 具有很高的构念效度，一定的内容效度、表

面效度和后效效度，但是校标关联效度比较低，总体来说可以保证根据分级考试

成绩学生可以被分入相应的级别。同时在研究的过程中，作者也发现了该考试存

在一些不足，并提出了改进的建议，以提高 USST CEPT 的效果进而促进该校大

学英语教学。

关键词：大学英语分级考试效度验证效度整体观

iii

ABSTRACT

Assessment is a crucial concept in fields of educational and psychological

measurement. One of its types, placement test, is frequently used in language teaching

to group students with various proficiencies into different classes. Chinese college

freshmen are generally asked to attend College English placement test (CEPT) with the

aim to be distributed into appropriate learning classes, equipped with particular teachers

and materials, provided with diverse teaching plans according to actual language levels

so as to optimize their learning abilities. There is no doubt that CEPT can influence their

English learning in four years. Thus, test developers should pay attention to its proper

difficulty level and representativeness. However, it was found that researches on

implementation results of CEPT are rarely carried out in China, let alone validation

study based on a theory in this field.

In 2012, University of Shanghai for Science and Technology (USST) began to

implement CEPT in order to provide reference for English placement teaching. To

improve its design, the validity of USST CEPT 2012 was investigated. Results show

that it is valid as a whole, but some parts can be developed better. Referring to

experience in validating USST CEPT 2012, the validity of USST CEPT 2013 is

analyzed more comprehensively to observe whether the test has consistent validity.

Unified concept of validity of Messick (1989) is mainly served as the theoretical

foundation. Messick’s idea about validity is influential in language assessment and

obtains agreement of many scholars later. As Bachman (2004: 259) eulogizes, “the

theoretical conceptualization of validity that conducts current practice in educational

and psychological testing in many parts of the world is largely due to the idea of Samuel

Messick. It has a strong influence on current measurement practice in North America, as

well as in countries outside of North America.” Besides, the framework of task

characteristics by Bachman & Palmer (2010) and Weir’s evidence-based approach for

validation (2005) are also used to some extent. Some of their theoretical content is

integrated in the unified concept of validity. And finally, a new research framework is

proposed to investigate the validity of USST CEPT 2013 from five aspects including

construct validity, content validity, criterion-related validity, face validity and washback

validity.

Through analysis, the author concludes that the validity of USST CEPT 2013 can

be ensured to some extent in construct, content, face and washback but not promised in

criterion-related aspect. Thus, it can be said generally to differentiate students of

different English levels so that they can be distributed into appropriate learning

environment. Finally, the author puts forward some weakness in the test and provides

suggestions for improving the design of future USST CEPT so as to further advance

College English teaching of USST.

Key words: College English; placement test; validation study; unified

concept of validity

List of Acronyms

AZELLA Arizona English Language Learners Assessment

CECR College English Curriculum Requirements

CEPT College English Placement Test

CET-4/6 College English Test Band 4/6

CETR College English Teaching Requirements

ESL English as Second Language

EPT English Placement Test

GPT Global Placement Test

NCEE National College Entrance Examination

SPSS Statistical Package of Social Sciences

TEM-4/8 Test for English Majors Band 4/8

TOEFL Test of English as a Foreign Language

USST University of Shanghai for Science & Technology

WPH Words per Hour

WPM Words per Minute

List of Tables

Table 3.1 Unified Validity Framework······································17

Table 3.2 College English Curriculum Requirements (2007)····················20

Table 3.3 Composition of CET-4··········································22

Table 4.1 Original Framework for Validation on USST CEPT 2013···············25

Table 4.2 Research Participants···········································36

Table 4.3 Flesch Reading Ease Standard····································38

Table 5.1 Descriptive Statistics of USST CEPT 2013··························42

Table 5.2 Reliability of Each Component and the Whole Test····················43

Table 5.3 Correlation Matrix of USST CEPT 2013····························44

Table 5.4 Instructions of CET-4···········································46

Table 5.5 Instructions of USST CEPT 2013··································47

Table 5.6 Structures of CET-4, CECR and USST CETR························48

Table 5.7 Structure of USST CEPT 2013····································48

Table 5.8 Characteristics of the Input in CET-4 and CECR······················49

Table 5.9 Topics of USST CEPT 2013······································50

Table 5.10 Text Length and Speededness of USST CEPT 2013··················50

Table 5.11 Expected Response of USST CEPT 2013··························52

vii

Table 5.12 Correlation between USST CEPT 2013 and CET-4···················54

Table 5.13 Correlations between the corresponding sections of Two Tests··········54

Table 5.14 Results for General Information of Students Surveyed················56

Table 5.15 Results for Questions of Students Surveyed························57

Table 5.16 Results for General Information of Teachers Surveyed················60

Table 5.17 Results for Questions of Teachers Surveyed························60

List of Figures

Figure 4.1 Modified Framework for Validation on USST CEPT 2013·············26

Figure 4.2 Framework for Content Validity of USST CEPT 2013················28

Figure 5.1 Normal Distribution of USST CEPT 2013··························42

viii

Contents

Acknowledgements·····················································i

摘要··································································ii

ABSTRACT··························································iv

List of Acronyms······················································vi

List of Tables·························································vii

List of Figures·······················································viii

Chapter One Introduction···············································1

1.1 Background of the Study··········································1

1.2 Objectives of the Study···········································4

1.3 Significance of the Study··········································5

1.4 Configuration of the Thesis········································6

1.5 Summary ······················································6

Chapter Two Literature Review··········································7

2.1 Theoretical Concerns on Validity····································7

2.1.1 Definition of Validity··········································7

2.1.2 Development of Validity·······································9

2.1.2.1 Single Concept of Validity·································9

2.1.2.2 Categorical Concept of Validity····························10

2.1.2.3 Unified Concept of Validity·······························11

2.1.2.4 An Evidence-based Approach for Validation··················12

2.1.2.5 A Framework of Task Characteristics························12

2.2 English Placement Test···········································13

2.2.1 Researches abroad···········································13

2.2.2 Researches at Home··········································15

2.3 Summary·····················································16

Chapter Three Research Bases··········································17

3.1 Theoretical Basis···············································17

3.2 Empirical Basis·················································19

3.2.1 College English Curriculum Requirements ······················19

3.2.2 Syllabus for College English Test Band 4·························21

3.2.3 USST College English Teaching Requirements····················23

3.3 Summary·····················································24

Chapter Four Research Implementation··································25

4.1 Research Framework············································25

4.1.1 Construct Validity···········································26

4.1.2 Content Validity·············································27

4.1.3 Criterion-related Validity······································32

4.1.4 Face Validity···············································32

4.1.5 Washback Validity···········································33

4.2 Research Subject and Participants··································35

4.2.1 Research Subject············································35

4.2.2 Research Participants········································35

4.3 Research Instruments············································36

4.3.1 Qualitative Instruments·······································36

4.3.1.1 Interview···············································36

4.3.1.2 Questionnaire Survey·····································37

4.3.2 Quantitative Instruments······································38

4.3.3 Statistical Package of Social Sciences 19.0························39

4.4 Research Procedure··············································39

4.5 Summary······················································40

Chapter Five Results and Discussion·····································42

5.1 Basic Considerations·············································42

5.1.1 Descriptive Analysis·········································42

5.1.2 Reliability·················································43

5.2 Main Considerations··············································44

5.2.1 Construct Validity············································44

5.2.2 Content Validity··············································46

5.2.2.1 Characteristics of the Rubric·······························46

5.2.2.2 Characteristics of the Input·································49

5.2.2.3 Characteristics of Expected Response························52

5.2.2.4 Executive Resources······································52

5.2.3 Criterion-related Validity·······································53

5.2.4 Face Validity ···············································55

5.2.5 Washback Validity············································56

5.2.5.1 Questionnaire Survey Results for Students·····················56

5.2.5.2 Questionnaire Survey Results for Teachers·····················59

5.3 Summary······················································62

Chapter Six Conclusion················································63

6.1 Answers to Research Questions·····································63

6.2 Recommendations to Future USST CEPT·····························65

6.3 Limitations of the Research········································66

6.4 Suggestions for Future Research····································67

Bibliography·························································68

Appendices··························································75

Appendix I College English Curriculum Requirements (2007)················75

Appendix II Syllabus for College English Test Band 4 (2006 revised)··········79

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

15 积分 4人已下载

立即下载 VIP免费下载

摘要：

i基于效度整体观的中国高校大学英语分级考试效度研究—以上海理工大学2013年分级试卷为例摘要语言测试是教育心理测量界的重要概念，其中的一种重要形式即为分级考试。分级考试可以把不同语言水平的学生编入相应的教学环境学习以期达到合理分配教学资源、提高教学效果的目的。国内大学通常要求大学新生参加大学英语分级考试，了解其英语水平，根据英语能力将新生分入不同的班级，配置与级别要求一致的教师，学习相应的教材，提供多样的教学计划等，从而最大程度上发挥学生的语言能力。由此，分级测试对大学生的四年英语能力的提高至关重要。因此，在设计大学英语分级考试时，设计者更须关注该类考试是否具有效度。然而，国内缺乏对该类考试具...

展开>> 收起<<

基于效度整体观的中国高校大学英语分级考试效度研究—以上海理工大学2013年分级试卷为例.doc

共107页,预览10页

还剩页未读，继续阅读

基于效度整体观的中国高校大学英语分级考试效度研究—以上海理工大学2013年分级试卷为例

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

推荐作者

热门标签

举报选择: