中小學(xué)生的學(xué)習(xí)評(píng)價(jià)-2015-7-23-27.ppt

Assessment of Student English Learning in Secondary Education,Xiongying Tang Email: July 23 2015,Workshop,Part One: Lecture Part Two: Hands-on Activities,Course Plan,D1：Shift, Types + Portfolio D2: Questioning + Mediation D3: Test types + Qualities D4: Test development + abilities tests D 5: Tests,Assessment,Attendance Learning logs, Action plan Tests: a cover letter + 6 questions/problem-solving (200wds Chinese/120wds English),Background,Shift of assessment culture New ways of assessment,Stories heard about assessment,Teaching driven by external high-stakes tests as NMET, etc. Poor self-made achievement tests Low proficiency/motivation of ss,Why do we assess ss learning?,Assessment of learning Assessment for learning,Or,Summative assessment Formative assessment,more terms indicating the contrast,Static assessment Convergent teacher assessment Assessment while teaching ,Dynamic assessment Divergent teacher assessment Assessment by teaching ,Anyway, all assessments need to support and help with students learning,An assessment activity can help learning if it provides information to be used as feedback by teachers, and by their students in assessing themselves and each other, to modify teaching and learning activities in which they are engaged.” (Black et al. 2003: 2),To assess is to,Provide feedback, and Promote change,Ultimate goal of educational assessment is to,Develop autonomous/self-regulated learners,Key elements in formative assessment,Setting up learning goals Identifying learning problems Providing Mediation,Assessment tools,Tests What else?,New forms of assessment,Presentation Interview Checklist Questionnaire Logs Portfolios Research report ,Teachers can be creative enough to develop different forms of assessment to serve their own assessment purposes (Brown 1998),Assessment could take place,“Minute by minute”, whenever it is possible for teachers to see, interpret and support students in learning “assessment by teaching”,Types of assessment in practice,Curriculum-embedded Assessment Planned-for Interaction On-the-Fly Assessment Heritage (2007),In Australian context,In-class contingent formative assessment-while-teaching More formal mock or trial assessments modeled on summative assessments but used for formative purposes More planned integrated formative assessment Prescribed summative assessments, but results also used formatively to guide future teaching/learning Davison (2008, in Davison & Leung 2009),In China Tang (2015),External Tests,External tests in your school?,NMET only? TOEFL IELTS,School-based Assessment?,Mid-term tests Final tests,Washback of the tests,The effect of tests on teaching and learning Positive & negative,Positive washback,Madaus: tests have very explicit goals of teachingthe public has concrete information on how well the schools are doing. Gipps: unless tests have high stakes, teachers will not make use of the test results.,Views on Washback Effect,Alderson & Wall (1993): 15 hypothesis Bailey (1996: 264): Basic model of washback Watanabe (1996): stakeholders,15 Hypothesis: A test will influence,1 Teaching 2 Learning 3 What teachers teach 4 How teachers teach 5 What students learn 6 How students learn 7 The rate and sequence of teaching 8 The rate and sequence of learning,9 The degree and depth of teaching 10 The degree and depth of learning 11 Attitudes to the content, methods, etc. of teaching and learning 12 Tests that have important consequences will have a washback 13 Tests that do not have important consequences will have no washback 14 Tests will have washback on all students and teachers 15 Tests will have washback effects for some students and some teachers, but not for others.,Alderson & Hamp-Lyons (1996),A comparison between non TOEFL preparation class and TOEFL preparation class by the same teachers: Test will influence both what and how teachers teach, but the effect is not the same in degree or in kind from teacher to teacher.,Tests affect,Students Teachers Material writers (Xinguang bookstore in Macau) Curriculum designers Researchers Bailey (1996:264),Bailey (1996: 264),Tests affect stakeholders as,Students Teachers Parents (other sponsors) Educational administrators and others Watanabe (1996),Formative use of external tests,Use for goal-setting Feed the information back to ss Assessment as learning,Assessment as learning,Teaching and supporting the underlined abilities or strategies,Classroom-based Assessment,Curriculum-based assessment: aligned to goals at different stages of teaching, planned in advance at the time curriculum design e.g. Voc fluencyspeaking logs, Research ability-project,Design of curriculum-embedded assessmentSetting Goals,Making reference to goals set in teaching syllabus Using relevant language criteria (e.g. Common European Framework of Reference for Languages, CEFR） Developing your own criteria (drawing on teaching experience and together with your ss),CEFR Levels,Basic users: A1, A2 Independent users: B1, B2 Proficient users: C1, C2,Description of A1 Level of CEFR,Description of A2 Level of CEFR,Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g. very basic personal and family information, shopping, local geography, employment).,continued,Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters. Can describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need.,Description of C1 Level of CEFR,Can understand a wide range of demanding, longer texts, and recognize implicit meaning. Can express ideas fluently and spontaneously without much obvious searching for expressions. Can use language flexibly and effectively for social, academic and professional purposes. Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organizational patterns, connectors and cohesive devices.,Planning assessment tools,mock or trial assessments modeled on summative assessments Integrated formative assessment e.g. portfolios, learning logs,Curriculum-embedded assessment can not be expected to produce necessarily good effect. Monitor the assessing process and make the process interactive and informative,Using assessment information,Identifying problems Acting on the information: teaching strategies, giving affective supports, etc.,Lesson-based Assessment,Plan the questions to ask Plan interactive assessment activities: self-assessment, peer assessment Observe ss performance Acting on the information,Planned Questions,Cognitive Levels of Questions Display Questions and Referential Questions Closed Questions and Open Questions,Cognitive Levels of Questions,Lower cognitive questions are those which ask the student merely to recall and the teacher knows the answer and they are fact, closed, direct, and knowledge questions. Higher cognitive questions are those which ask the student to create an answer with logically reasoned evidence using information previously learned.,Types of Questions,Display question is “not a real question, i.e. which does not seek information unknown to the teacher, but which serves to elicit language practice” (Richards, Platt & Platt, 2000:142). Referential question is “a question which asks for information unknown to the teacher” (Richards, Platt & Platt, 2000: 390).,Display Questions,T: What function does the word does play here? S: Emphasize. T: The phrase to be addressed, to address the problem means what? S: Solve the problem. T: In this sentence, glare at means what? S: To look at somebody angrily.,Referential Questions,T: Do you have any habit to protect our environment? Anyone? S: Yeah, I have the habit to use the plastic big bag that I got when I bought something in grocery. Use the bag as my trash bag, just reuse it not to throw it into the rubbish can.,Referential Questions,T: In your opinion, why do the old people like to talk about their past life instead of looking forward? Whats your opinion? S: Because they may have no future. Their past life is the most important part of their life. And there were many wonderful things that they want to remember. And their current life may be boring.,On-the-fly assessment,Noticing ss performance in contingent, unplanned interaction Interpreting their learning (gains and problems) Acting on the noticing,e.g.（Thompson 1997:103),T: What does your father do SI: Teacher T: Hes a teacher/good/what does your father do S2: My father dead T: Good/and what about your father,T: What does your father do SI: Teacher T: Hes a teacher/good/what does your father do S2: My father dead T: I am sorry. What did he do?,Model of formative A (Bell & Cowie 2001, p. 91),Procedures of classroom-based assessment,Feuersteins Mediation Theory,Direct learning Mediated learning,Mediated Learning Experience,Sociocultural theory ZDP “the others” Optimism about success of learning Strategies,Features of MLE,Intentionality & Reciprocity Transcendence Meaning (Kozulin & Presseisen 1995: 70),These features also apply to assessment, which aims at the development of individual learners.,Expanded MLE-12 criteria,1 Significance: The teacher makes students realize the importance of a assessing activity so that they can look at the significance of the activity to their own and in a broader cultural context. Listening log: they might want to understand radio program, films and conversation at work place,2 Purpose beyond the here and now: explains to learners how conducting a assessing activity will help them in the future beyond the moment and situation at present only. e.g. Listening log: they can learn to monitor their own learning, and become more independent learners.,3 Shared intention: In presenting an assessment, the teacher must make instructions clear and ensure the intention is understood and reciprocated by learners. Listening log: students need to be told about what they are supposed to do, and they are willing to participate,4 A sense of competence: Fosters learners feelings of competence and capability of learning. Listening log: give positive feedback and tell them about the progress they have made,5 Control of their own behavior: Encourages students to be autonomous by self-controlling their learning procedure. Listening log: record the total time spent on listening, and check how much the original plans are met.,6 Goal-setting: teaches learners how to establish achievable targets and to locate approaches for the purpose of realizing them. Listening log: refer them to an goal achievable by minimum effort, for example, a minimum time length to be spent in listening in a week, and set their own goal based on their own facilities,7 Challenge: Helps learners to develop internal needs to face challenges and to seek for new challenges in life. Listening log: ask them to try new, more authentic recordings, or standard English radio programs,8 Awareness of change: Stimulates learners to monitor changes in themselves and to understand the fact that humans are changeable all the time. Listening log: help them to see in their own eye the progress theyve made, and the weakness they need to deal with,9 A belief in positive outcome: Urges learners to assume there is always the possibility of finding a solution, even when faced with an apparently intractable problem. “It always seems impossible until its done” -Nelson Mandela,10 Sharing: Invites learners to share behaviors and to perceive that it is advisable for some problems to be addressed collaboratively. Listening log: have them discuss in groups, and share the materials and listening experience,11 Individuality: Helps learners realize their individual characteristics in terms of their unique aspects Listening log: Believe each learners have their own path to master listening abilities, and give them more freedom to decide what to listen, and how to listen, etc.,12 A sense of belonging: Aids learners to establish a consciousness of pertaining to the whole class community in the process of the completion of the task collaboratively. Listening log: Make them understand he is not alone because the whole class are making efforts, facing similar problems, but are making progress.,Quality of assessment,Traditional tests Classroom-based assessment,Qualities of a test,Validity Reliability Washback Authenticity Interactiveness Practicality,Authenticity,Refers to the extent how the test task represent the real-life task and thus better predict ss performance in future task in real context,Interactiveness,refers to the extent to which a test involves ss personal response (reporting personal life experience, for example),Practicality,How much the cost of test could be met in terms of money, human resources and facilities.,Validity,means the extent to which the test has tested the targeted abilities. Content Construct Concurrent Predictive,Content Validity,Definition: the extent to which the test content constitutes a representative sample of the language skills, structures, etc. which it is meant to be concerned. Relevance & representativeness Validation,Construct Validity,Alderson (2003: 118) “ A construct is a psychological concept, which derives from a theory of the ability to be tested.” Every test is to some degree measuring the ability theorized,Criterion-related Validity,Some independent and highly dependable test-Current Validity Some test of ss language in the futurePredictive Validity,Reliability,means the extent to which the tests enable the test takers to perform consistently on different times of testing and on tests of different versions. Split-half Alternative forms Test-retest,Methods to ensure reliability,Take enough samples of behavior Write unambiguous items Provide clear and explicit instructions Ensure that tests are well laid out and perfectly legible Make candidates familiar with format and testing techniques,Provide a detailed scoring key Agree acceptable responses and appropriate scores at outset of scoring Identify candidates by number, not name Employ multiple, independent scoring ,Formative use of tests,Be Aware of the existence of test washback Make good use of the test: setting goals, giving feedback and follow-up activities,Quality of classroom-based assessment,Fitness of purpose Effectiveness for promoting learning,Part TwoHand-on activities,A1: Group-discussion and presentation: Types of assessment used in your own teaching context A2: Hand-on activities: An assessment plan to address one of the problems you see,Possibly, we want testing for,Placement Achievement Diagnosis Goal setting Motivation Curriculum development ,Language Testing,Types of test Test construction,Types of Language Tests,achievement tests & proficiency tests direct & indirect tests subjective & objective tests norm & criterion-referenced tests discrete point & integrative tests,Achievement Tests,Mid-term tests Final tests Weekly/Monthly tests ,Proficiency tests,NMET CET 4-6 TEM 4-8 TOEFL TOEIC IELTS BEC ,Direct tests,Writing test involving ss to write Speaking test involving ss to speak ,Indirect tests,Writing tests involving ss to do proofreading Speaking tests involving ss to identify difference in pronunciation of words, e.g. A. look B. stood C. took D. loop,Subjective & objective tests,The difference lies in the rating of ss responses, i.e. whether they involve the personal judgment of the raters,Norm & criterion-referenced tests,The difference lies in the interpretation of test results,Norm-referenced Tests,Norm refers to the test performance of a target group of ss Norm-referenced tests report performance according to the position of a test taker within a group.,Criterion-referenced tests,Criteria refers to the targeted standards of language proficiency Criterion-referenced tests report test performance based on the extent of how a test taker accomplishes the requirements stated in the standards e.g. Dialang developed in U of Lancaster,Criteria,Blooms Taxonomy of Educational Objectives in the US Common European Framework of Reference for Languages ACTFL Proficiency guidelines,Blooms Taxonomy (Revised 2005),One of the earliest taxonomies developed to model cognitive development Useful when creating assessments to diagnose the complexity of students learning,CEFR Levels,Basic users: A1, A2 Independent users: B1, B2 Proficient users: C1, C2,ACTFL Levels,NL/NM/NH Novice Low/Mid/High IL/IM/IH Intermediate Low/Mid/High AL/AM/AH Advanced Low/Mid/High S Superior D “Distinguished“,Order of the levels,NL_NM_A1_NH_A2/IL_IM_B1_IH_B2 _AL_ AM_C1_AH_C2_S_ (Baztn, 2008 ),Merits of Criterion-referenced Testing,Criterion-referenced test is significant in that it sets up the goals of learning. the result o