diachroniccontentvaliditystudyonreadingcomprehensiontestsofnmetinshandongprovince(2007-2

1、0或幾璘丈摩本科生畢業(yè)設(shè)計(jì)(論文)題目：Diachronic Content Validity Study on Reading Comprehension Tests of NMET in Shandong Province(2007-2011).教學(xué)單位 外國(guó)語(yǔ)學(xué)院 _姓 名_學(xué) 號(hào) 200830701021_年 級(jí)_2008級(jí) _專 業(yè)英語(yǔ) _指導(dǎo)教師 _職 稱 _2012年4月30日.6ContentsAbstract. IIIChapter 1lntroduction . . . . .1Chapter 2 Content Validity . . .12.1 Defin iti o

2、n of Validity . . 12.2 Types of Validity . .22.2.1 Face Validity . .22.2.2 Corte nt Validity . .22.2.3 Criterio n Validity. .22.2.4 Con struct Validity . .22.3 Natio nal Matriculatio n En glish Tests. .2Chapter 3 Theoretical Foundationand Research Framework . . .33.1 Theoretical Foundation. .33.2Res

3、earchFramework . . .33.2.1 Data Collectio n .33.2.2 DataAnalysis . . . . .3Chapter 4 Findings and Discussi ons . .44.1 Len gth . . .44.2 Speed . . . . .54.3 New Words4.4 Readabilities.74.5 Topics .4.6 Genres .84.7 Read ing Skills .8Chapter 5 Conclusions and Suggestions . . .95.1 Conclusions . . . .9

4、5.2 Suggestions . . . .105.2.1 Suggestions for the Testing Service Centre . 105.2.2 Recommendations and Implications for English Teaching andLearning . . .115.3 Limitation of the Study. . .11References. 12Ack no wledgeme nt. .13Abstract: This paper aims to explore the content validity of the Shandon

5、g NMET(National Matriculation English Tests)reading comprehension component based on the framework of task characteristics and related content validity theories, the senior high English teaching syllabus and the Syllabus for NMET test. The results show that reading comprehension parts of 2007-2011 S

6、handong NMET have well met the requirements of the two syllabi and have got a well content validity in many aspects such as topics, new words and reading skills. However, the author also finds some weaknesses. Based On the findings, this author puts forward some suggestions in order to improve langu

7、age testing as well as teaching and learning English reading in high schools.摘要：本文以山東省20072011年高考英語(yǔ)閱讀理解測(cè)試為研究對(duì)象，結(jié)合任 務(wù)特征框架、內(nèi)容效度的相關(guān)研究 , 再根據(jù)全日制高級(jí)中學(xué)英語(yǔ)教學(xué)大綱 ( 試 驗(yàn)修訂版 ) 和2007- 2011 年的高考英語(yǔ)考試大綱 , 對(duì)其閱讀理解內(nèi)容效度進(jìn)行 研究分析。 結(jié)果表明 : 這五年高考英語(yǔ)山東省英語(yǔ)閱讀理解試題在選材，生詞 量，以及考察能力等方面符合考試大綱和教學(xué)大綱的要求 , 具有較好的內(nèi)容效 度。但同時(shí)也發(fā)現(xiàn)還存在一些不足，文章對(duì)此提出幾點(diǎn)建議

8、，希望能對(duì)高考英語(yǔ)閱讀測(cè)試的命題質(zhì)量的提高以及今后的高中英語(yǔ)教學(xué)有所促進(jìn)。Key words: National Matriculation English Tests; Reading Comprehension Tests;Content Validity關(guān)鍵詞 ：高考英語(yǔ)；閱讀理解測(cè)試；內(nèi)容效度Diachronic Content Validity Study on Reading Comprehension Tests of NMET in Shangdong Province(2007-2011) .Chapter 1 IntroductionNow, the National Un

9、ified Enrollment Examinations (NUEE) is one of the largest- scale tests in China. Due to that National Matriculation English Tests (NMET) is one of the three major subjects, it is necessary to make well analyses about it. As a high- stakes test, NMET, is used as agents to change or attempt to change

10、 the classroom teaching(Qi Luxia,2005：142) It is widely acknowledged that the central problem of foreign language testing is validity. Reading comprehension is a main language skill. What s more, it has a significantpercentage in NMET. Therefore, the content validity of reading comprehension has a d

11、irectly influence on the realization of NUEE Function. Currently, scholars do many researches on validation study about TOFE，L IELTS ，TEM and CET. But, few are related to the NMET, let alone the English test paper of Shandong province.Owing to the significant role reading plays, this paper has chose

12、n to analyze the content validity of reading comprehension component of 2007-2011 Shandong NMET to make a clear analysis of it and throw some light on teaching and learning English reading in high schools as well as on the NMET.Chapter 2 Content Validity2.1 Definition of ValidityMany scholars give m

13、any kinds of definitions about validity. Theproblem of validity is that of whether a test really measures what it purports to measures . Kelly(1927 ： 14)defines. Henning(2001： 89)notes， validity usually refers to a given test s fitness or any of its component parts as a measure of what it is purport

14、ed to measure Henning states： tests are more or less valid for their purposes； validity is not an all or-nothing matter (Alde，rsConlapham&Wall ，2000：1 70)In sum, the validity of a test is the extent to which it measures what it is supposed to measure and no thi ng else Every test, whether it is a sh

15、ort, in formal classroom test or a public exam in ati on, should be as valid as the con structor call make it The test must aim to provide a true measure of the particular skills which it is in ten ded to measure to the extent that it measures external knowledge and other skills at the same time. It

16、 will not be a valid test.2.2 Types of Validity2.2.1 Face ValidityFace validity is a measure of how representative a research project is at face value, and whether it appears to be a good project. Ingram(1 97:7 1 8)notes face validity as “ thtests surface credibility or public acceptability. In gen

17、erally, if a test seems to cater for other testers, teachers, moderators, and testees, it can be described as having at least face validity. measures what it should measures we can be said that it has face validity . Therefore, directly tests have high face validity, while in directly tests have low

18、 face validity.2.2.2 Content ValidityContent validity is the estimate of how much a measure representsevery single eleme nt of a con struct. Kerlin ger(1973:458)states that content validity is a tool to measure the represe ntative ness of the content and the adequacy of the sampli ngs. In other word

19、s, it refers to whether the tests the requireme nts of the syllabus or not. Content validity is an important issue for both Ianguage testing and Ianguage teachi ng. Hughes(200Q 27)states that the greater a test s contene validity is more likely it is to accurately measure what it is supposed to meas

20、ure2.2.3Criterion ValidityCriterion Validity assesses whether a test reflects a certain set of abilities. Usually, there are two kinds of Criterion Validity - Concurrent validity and Predictive validity . Concurrent validity measures the test aga inst a ben chmark testa nd high correlati on indicate

21、s that the test has strong criterion validity. While, Predictive validity is a measure of how well a test predicts abilities. It invo Ives test ing a group of subjects for a certa in con struct and the n compari ng them with results obta ined at some point in the future.2.2.4 Construct ValidityCon s

22、truct validity defi nes how well a test or experime nt measures up to its claims. A test desig ned to measure depressi on must only measure that particular con struct, not closely related ideals such as anxiety or stress. It means that if a test has con struct validity, it is capable of measuri ng c

23、erta in specific characteristics in accorda nee with a theory of a Ian guage behavior and lear ning.2.3 National Matriculation English TestsThe Nati onal Un ified En rollme nt Exam in ati ons (NUEE) is one of the largest- scale tests in China.Due to National Matriculation English Tests (NMET) is one

24、 of the three major subjects, many authors pay attention on it. There are many articles about it. A list of them is followed.? Validation Study on 2008-2009 National Matriculation English Test Papers of Three Provi nces is writte n by Yan Ji ng in 2010 .? An Investigation of Washback Effect of the N

25、ational Matriculation English Test is writte n by Hong Xiaoxia ng in 2008.? Content Validity Study on Reading Comprehension Tests of NMET of Chongqing is written by Qiu Jin in 2010.? A study on the Backwash Effect of the Nmet (sha ndong Provi nce)readi ng and Expressi on is written by Zuo Peilan in

26、2010.? An Empirical Study on the Washback of the Grammar Part in the National Matriculation English Test(Shanghai) is written by Zhao Meijuan in 2009.? Diachronic Study on NMET Reading Comprehension Discourse Characteristics and Question Design is written by Zhang Quan in 2010.? A Study of Character

27、istics and Content Validity of Nmet Reading Tests in Shaan Xi Province (2006-2009) is written by Bai Haiyu in 2011? A Study of Content Validity on Read ing Comprehe nsion of Matriculati on En glish Test in Guangdong Province (2004-2010) is written by Zhou Yiqiong in 2011.There are also many other ar

28、ticles about NMET. However, by carefully analysis author finds that a majority of them regard the NMET as whole to analyze. Of course, there are some articles are about reading comprehension. But few concerns Shan gdo ng province let alone the diachr onic research about it. Therefore this paper has

29、chosen to analyze the content validity of reading comprehension component of 2007-2011 Shandong NMET to make a clear analysis of it and throw some light on teach ing and lear ning En glish readi ng in high schools as well as on the NMET.Chapter 3 Theoretical Foundations and Research Framework3.1 The

30、oretical FoundationThe author combines the framework of task characteristics and relied content validity study theories proposed by Bachma n and Palmer ( 1999 ) , the senior high English teaching syllabus and the syllabus for NMET test.3.2 Research Framework3.2.1 Data CollectionThe author collects r

31、eading comprehension parts of 2007-2011 Shandong NMET, the n makes a careful an alyses both qualitative and qua ntitative data which are about len gth, readability, readi ng speed, new words, topics and gen res. Computers have also bee n adopted in the data collect ing.322 Data AnalysisBy comparing

32、the collected data with the requirements of the senior high English teaching syllabus and the syllabus for NMET test. The author also invited a classmates and a lecturer to help me when it is difficult for the author to analyze some questi ons.Chapter 4 Findings and DiscussionsThe author combines th

33、e framework of task characteristics and relied content validity study theories proposed by Bachman and Palmer (1999), the senior high En glish teach ing syllabus and the Syllabus for NMET test to put forward the objects for testing the content validity (Table1).Table 1 Objects for testing the conten

34、t validityObjectsLengthSpeedNew WordsReadabilitiesTopicsInputGenresExpected ResponseReading Skills4.1 LengthThe syllabus for NMET test stipulates that the total le ngth of the passages should be above 1000 words (Dong 2008). Careful cou nting is show n in the table 2.Table 2: Le ngth of readi ng pas

35、sages from 2007 to 2011PassageWordsYearABCDPTQT07291296292291117074919190826926432422910865801666093193023022821205570177510320317294290122169219131126628731729011606421802(PT: Passage Total; Q: Question; T: Total)First, it is obviously the total length of the reading tests is above 1000 words. Alde

36、rs on (2000:256) recomme nds that the len gth of each test passageis suitable when it is between 150350 words. Liu and Han also mentioned that reading material should be con trolled at about 200-500 words(Do ng 2008).The table 2 shows that the length of each reading test is between 250-320 words. Th

37、erefore, it meets the requireme nt of scholars.As for the layout of the articles, it should follow the gen eral prin ciple that fromshort to long and from easy to difficult. However, this study found that some years the articles layout is not very reas on able, such as the 2009 and 2010, which begi

38、n with the Ion gest passage of the whole read ing test. This may give can didates slap feeli ng, which will affect stude nts no rmal performa nee.4.2 SpeedAs for readi ng speed of NMET, two ways can be con sidered. First is the speed of reading articles, and the second is finishing the reading tests

39、. The syllabus for high school stude nts has specific dema nds. Senior middle school goals are divided into two levels, and the first level is 5070 words per minute, then, the second level is 7080 words per minu te. Yang Huizho ng and Weir(1998)submit that the proporti on of reading articles and ans

40、wering questions is 1 to 0.75. Syllabus requires that readi ng comprehe nsion secti on should be fini shed by 35 minu tes. Accord ing to this, reading the article takes up 20 minutes, and answering the questions takes 15 minutes. Based on these, the author calculates the reading speed according to t

41、he proporti on of twenty to fiftee n minu tes (WPM1) .In additi on, the writer also adopts the usual way to calculate t he read ing speed, or the total in put amount is divided by reading time (WPM2). The two calculations are shown in t he table below:Table 3 Readi ng speed from 2007 to 201107080910

42、11WPM158.554.360.461.058.0WPM254.747.650.754.651.4Through the first calculation, the read speed of the five-year is from54.3-61.0 wpm. Basically , the speed fully meets the curriculum requirements. And the reading speed is well balaneed, except for 2008 which is a slightly slower.As for the sec ond

43、calculati on method, gen erally speak ing, read ing speed for every paper almost meets the requireme nt of the teach ing syllabus, except for the speed of 2008 which is lower that 50 wpm.By analyzing the Table 3, the author finds out that they just reach the first level requirement of the first grad

44、e ( 50- 70 wpm) , not reach the second level ( 70-80 wpm ) . NMET is a top level test which is used to select the qualified can didates for university. Therefore, reading speed should sometime rise to second level. Based on the analysis, a suggestion may be made that reading speed of NMET needs to b

45、e slightly raised.4.3 New WordsThe teaching and testing syllab(判決理由書的要旨 )stipulate clearly that the new words of each reading passage should be less than 3%. The new words and vocabulary bey ond the teachi ng syllabus of each year are show n in the table 4.Table 4 New words and vocabulary bey ond th

46、e teach ing syllabus0708091011AN1.421.341.031.451.231.29(N: New Words A: Average)From the Table 4, the author finds that the new words rates of 2007-2011 are lower than 3%. What s more, the new words of each year are stable except the year of 2009. Readi ng comprehe nsion tests of NMET in Shangdong

47、province gets better control of the amount of new words which matches well in line with the requireme nts of the syllabus.4.4 ReadabilityThe readability of the article refers to the level of read ing material that is easy to read and understand. The readability of reading comprehension should be app

48、ropriate because it can not reflect the actual reading abilities if it is too difficult or too easy.Nowadays widely used formula was the Flesh, which is first used in 1948 and still in use today. The formula is like this: RE = 206. 835-(0. 846 x NSYLL) - (1.015 x W/S) (qt d in Alderson 2000: 71). NS

49、YLL refers to the number of syllables per 100 words, and W/S refers to the average nu mber of words per sentence Although, the formula is desig ned f or the En glish n ative lear ners not for the foreig ners, Yang and Weir (1998) put forward that it can be regarded as a refere nee for the test desig

50、 ners and provide us a basis to build our comparis on with a uniform criteri on. The formula describes the readability with seven levels from easy to difficult.Table 5 The references of Flesh readability (Yang Huizhong &W e IR, 1998)R0-3030- 5050- 6060-7070- 8080- 9090-100DVery difficultdifficultFai

51、rly difficultstandardFairly easyeasyVery easy(R: Readability; D: Description of difficulty)Based on the Flesh formula, the readability of reading passagesof NMET inShangdong province is described accord ing to readability yardstick as follows:Table 6 Readability classificati ons of read ing passages

52、 from 2007 to 2011YearPassage ReadabilityARABCD0755.256.758.257.256.80856.359.358.354.457.10957.558.961.464.560.11060.259.763.262.661.51159.357.861.459.659.5(AR: Average Readability)By analyzing the table shown above the readability of reading comprehension tests of NMET in Shangdong province is fro

53、m 50-70. According to the Flesh reference scale, the readability of the reading materials is from fairly difficult to standard. So, a conclusion can be made that it well meets the requirement of the two syllabi. What s more, it is characterized with stationarity.4.5 TopicsAlders on (2000) no tes tha

54、t usually good read ing tests and good assessme nt procedures will make sure that readers have been assessed for their ability to make a well known of texts in a range of differe nt topics. The senior high En glish teach ing syllabus and the syllabus for NMET test require that senior middle school g

55、raduates can read and un dersta nd articles on differe nt topics in clud ing scie nee and tech no logy, biography, story, social life and culture, literature and history. Based on these, the topics will be divided in to six parts and the collected data will show in the form 7.Table 7 Topics of readi

56、 ng passages from 2007-2011S&CSES&TH&GPE200701110120081111002009111001 -20100202002011021100274502(S&C: Social life and Culture; S: Story; E: Education; S&T: Science and Technology; H&G: History and Geography; PE: Personal Experience.)By analyzing the table 7, the author concludes some characteristi

57、cs about the topics of read ing materials as follows:First, the selection of topics is extensive. The table 7 shows that almost the six topics are in cluded. By this, it can en sure to test the can didates read ing abilities more scientifically, objectively and thoroughly.Second, the selection of to

58、pics is comfort to the times. There are some heated topics which can stimulate sudents interest in reading and help to guide the students to focus on social and curre nt affairs.Third, the selection of topics is realization. Many topics are concerned with the students daily life which they are so fa

59、miliar with. This can make sure the fairness of the test. Just as Bachma n (1990: 138) proposes that if the subject matter of in put is familiar to some test takers and not others, these individuals may have an unfair adva ntage, result ing in better performa nee.Fourth, the selection of topics is i

60、deology. The table 7 shows that different topics acco unt for differe nt proporti ons. Among of them story occupies a large proporti on. Every story has bee n carefully selected, and every of them has its ideology, which can make sig nifica nt in flue nee on the stude nts.In short, all kinds of topi