Interesting Powerpoint Slides I Created

Validity & Reliability

Validity & reliability an interesting powerpoint slide i created from Sze Kai

  Which do you prefer - Holistic or Analytic marking?

      Honestly speaking, I personally very much prefer the holistic marking as compared to the analytic. To me, the former is deemed more practical, i.e. time efficient especially when a teacher already has so much to do. In addition, it also highlights the students’ strengths in the subject matter as opposed to the latter which is sometimes claimed to look more on both student’s weaknesses and strengths. This I believe is an important aspect when it comes to marking.

          

Stress???

Nevertheless, the purpose of the assessment do also play an important role in the selection of the above two mentioned marking scales. No doubt that when it comes to marking a huge load of examination papers on essays, i.e. the summative exam, the holistic marking would be a better suited one. However, during class lesson when a teacher is trying to teach students on writing a good essay by focusing on each criterion of the essay separately (content, organization, language, vocabulary, voice and mechanics), the analytic marking seems to play its role better. Reason being that the latter breaks down every aspect of the essay into smaller components and thus allows the possibility of looking at each aspect separately. Hence, teachers could actually now teach students to master one aspect of the writing skill at a time. Besides that, analytic marking also helps to identify the students’ strengths and weaknesses. This allows the teacher to be better equipped with information on the performance of the students. Moreover, with the scale given to students as the rubrics, the students would know what and how they are expected to perform both during the normal in-class ongoing formative assessment as well as during the summative assessment.

Rubrics- Holistic or Analytic

           To add on to that, reliability does play a role in selecting the marking scales too. An experienced teacher may appear to have no problem with using the holistic marking. However, an inexperience teacher or a fresh graduate, on the other hand, may not be so. They do not fully understand and could not distinguish between the set of key words used in the rubrics given yet. Hence, it is deemed more appropriate and easier for an inexperience teacher to actually apply the later marking scale.

          In conclusion, though I prefer the former marking over the latter, at the end of the day, the purpose of the task and assessment does have a say in the choice of the marking scales. When the practicality issue, such as when a teacher is loaded with a lot of marking to be done in a very short amount of time, is concerned, it would be wiser and more appropriate to apply the holistic marking. On the other hand, during normal days when a teacher is helping her students to master each aspect of the skills individually, the analytic marking would seem to be a better choice. 

Want to know more about holistic and analytic marking when it comes to writing evaluation for placement purposes? Click here.

Tutorial Task 2b (Week 3)

UNIVERSITI MALAYSIA SABAH

B.Ed TESL (Hons)

TE 30503 LANGUAGE TESTING AND ASSESSMENT

Criteria for Evaluating Language Tests

Work in groups of four and look carefully at the following test questions. Try to do the questions. What observations do you have to make about them relating to issues of validity, reliability, practicability, and washback effect.

1.         Complete the following statements by selecting one answer from the four provided:

i.                    Jakarta is a large city_________ on the shores of the Java Sea.

a.         is situated                                b.         situated

c.         situates                                     d.         which situated

Ans: B

ii.                  Tokyo, which is Japan’s capital, is also ________ largest city.

a.         this                              b.         its

c.         the                               d.         that

Ans: C

Issues: Construct Validity -Uncertain of the theories behind the construction of these questions (though it seems like it is a grammar test- but which aspect is it testing on?)

 2.         Look at the sentences which follow and rearrange them to form a dialogue.

John     Oh, well I’m in a bit of a hurry myself. I don’t think that I’ve got time to walk all that way and back again.

John     Oh I see. Actually, I am going to the city centre as well. Mind if I join you?

John     Yes, I suppose so.

Mary    Well, if you don’t mind I think I’d rather walk. It’s such a lovely day and it seems a pity not to enjoy the weather.

John     Yes. Bye

John     Good morning, Mary!

Mary    see you around sometime

Mary    Good morning, John!

Mary    Goodbye, John.

Mary    Oh well, we’ll have to go our separate ways then!

Mary    Oh I’m just going down to the bank to get a new cheque book.

Mary    Sure. I was planning to walk – is that OK with you?

Issues: Reliability -Question is ambiguous in that it does not mentioned the answering method (Are the students required to rewrite the whole text? Are they just to numbers them, if so, how?) -The text itself does not seem to even form a correct discourse. Hence, attempts to rearrange the text would be futile.

3.                  The sentences which follow should be examined with great care and, if any verb forms are found to be incorrect, they should be corrected. Answers should be written in the answer booking such a way that, in each case, the correct form precedes the incorrect one.

i           John is a teacher

ii          Mary were a doctor   (Ans: was/is)

iii         Sometimes it snow in the Atlas Mountains.   (Ans: snows)

iv         The teacher went to London last week.

Issues: Reliability & Content Validity Reliability -The main issue here would be reliability. The question is ambiguous, with no clear and explicit instructions as to what should be done is the statements/options given above are correct. -There could be more than one (1) answer for some statements. Content Validity -Some of the statements do not even use proper punctuation mark. Hence the issue of validity comes in as to what this question is really trying to test (Is it to test subject-verb agreement, tenses or punctuation marks).

4.                  Vocabulary test

Choose one answer from those provided to complete the sentences:

I           Mary was angry whilst John was positively ____________

            A.        anorexic                       B.        apoplectic

            C.        artichoke                     D.        artesian

II         This beautiful little animal is called a ____________

                        A.        marmoset                     B.        margrave

                        C.        marimba                      D.        martinet

            III        Animals such as cows generally ______________

                        A.        rummage                     B.        rusticate

                        C.        roseate                         D.        ruminate

Issues: Construct Validity & Washback Construct Validity -What language theory is this test based on? What is the reason and justification behind testing students such difficult and extremely rare vocabulary? At the end of the day, the fact being that the students are not going to use these vocabularies in their daily life. Washback -Are the students going to benefit from doing these kind of test? How are they going to relate or use what they have learned here to communicate in their daily life? Students who are doing this sort of test would more probably be demotivated to learn English after this since it is too difficult and irrelevant to ordinary school students, unless these sort of questions is being posted to students taking semantics or hoping to graduate as a dictionary writer.

5.                  Listening Test

Look at the following passage and then listen to the questions from the tape recorder. The questions will be repeated once. Write the answers to the recorded questions in the answer books. Do not write the answers in this test booklet.

Princess Diana has announced that she will be retiring from public life in the near future. She has indicated that this decision has been caused because the pressure of press attention has become unbearable for her. She intends to spend more time with her sons, Prince William and Prince Henry. Today’s newspapers have devoted many pages to discussions of the crisis.

[Tape recorded questions]

1.                  What are the names of Princess Diana’s sons?

2.                  Why has the Princess decided to retire from public life?

3.                  How has the press reacted to the Princess’ decision?

Issues: Content Validity & Face Validity Content Validity -This test does not test what it claims to test. Instead of asking students to listen to the text before answering the questions, students are required to read the text. This is more like a reading test. Face Validity -On the first look, this paper seriously looks like a reading test rather than a listening test.

6.                  Test of Communicative Competence

Put all the verbs in the following passage into the correct form.

Once upon a time there (be) a teacher (call) John.  John (live) in a city in the North of England. John (travel) a lot for his job; in one year he (visit) nine different countries. All John’s friends (think) that John (be) very lucky because he (can) travel so much. However, John actually (hate) aeroplanes, and so every overseas visit (be) torture for him! Yet he never actually (admit) his problem to his colleagues because he (worry) that he (lose) his job.

Issues: Face Validity & Reliability Face Validity -This test looks more like a reading test than a grammar test. Reliability -The instructions used are ambiguous and misleading. How is one suppose to “put all the verbs…in the correct form” when there is nowhere to take it from and nothing to choose from. The proper word to be used could be “change” instead of “put” -It is ambiguous and full of uncertainty as to how one ought to answer the question. Are the students suppose to write the answer in the answer booklet or simply to erase of the wrong form in the brackets and re-write the correct forms?

  

7.                  Elementary Level Test

Write a letter to your local councilor complaining about some aspect of local government services – for example, rubbish removal.

Issues: Reliability -Again the test item has ambiguous instructions. For instance, the number of words that a student should write is not mentioned. -Secondly, the students are granted with too much freedom in terms of the area and scope of government services. The complain  letter could range from the subject of rubbish removal to the failure of possibly the Prime Minister’s office in entertaining the people.

8.                  Test of Spoken English

Look at the timetable below and tell your teacher about it.

Malaysia Rail Timetable

            Ipoh                              Kuala Lumpur

07 00                                       10 05

09 00                                       12 05

11 00                                       14 05

1400                                        17 05

               

Issues: Reliability -As mentioned above, the area or topic of discussion is not mentioned. The problem would come if the students just tell the teacher what they see in the test paper and ends with “I do not understand”. How is the teacher going to mark the student?  After all, the topic is not mentioned, hence anything is accountable.

Tutorial Task 2a (Week 3)

SCHOOL OF EDUCATION & SOCIAL DEVELOPMENT

UNIVERSITI MALAYSIA SABAH

B. ED. TESL  

TE 30503: Language Testing & Assessment

The Purposes of Testing

Question:

Answers: 

Types of Tests & Purposes

When Test is Administered	Types of Tests	Purpose of Test
Beginning	Placement	Used to enable teachers to sort students into groups/ classes according to their language ability at the beginning of the course.
	Diagnostic	Used to help check students’ progress for specific weaknesses and problems they may have encountered so as to help teachers to teach effectively.
	Aptitude	Used to predict the success or failure of students prospective in a language-learning programme.  It is Usually used as a screening instrument in langange testing
During (Formative)	Achievement: Progress (Formative)	Used to find out how well the students have mastered the language areas and skills which have just been taught.
End (Summative)	Achievement : Attainment (Summative)	Like progress test but it is usually designed to cover a longer period of learning than a progress test and is usually given at the end of the school year/the course.
	Proficiency	Used to measure how suitable candidates will be for performing a certain task or follow a specific course.

Google Docs

Comments on Youtube Video

TUTORIAL TASK (Week 1):

Give your thoughts on what was significant to you in the assessment used in the Youtube video.

As much as I am amazed at how much alternative assessments are being carried out overseas and of how effective the results are (and would like to share on alternative assessment as well), I would like to just highlight on one or two points which I think have not been touched on so far. (trying to spice things up here… ;) ). What I am interested to say here is that besides showing examples of how alternative assessments are being carried out, the video also points out that all these assessments should be done with at least some clear objectives, standards and/or criteria. In other words, we do not just do it anyhow we like to, just do it because we feel we like it or because this is the government’s new policy. Instead we should be looking at the rubrics of the assessment as how Anderson (1998) put it. And these rubrics or these learning objectives have to fit in, if possible nicely, with the curriculum set by the Ministry of Education.  

Secondly, we have to document the evidences on what that the students have done throughout the year and even years as in the video. This is to show that “you [the students] learn something substantial and significant that relates to school curriculum” (Dr. Wiggins, Grant). I think that multimedia portfolios like the ones done in the video are one good way to go. I believe these are some of the extra points that we need to pay attention to as well seeing that most of us would probably be those who are to implement these alternative assessment at school in a few years time.

Thirdly, I wish to post a question that we could perhaps discuss about as well. Seeing that we all have agreed on alternative assessment thus far, and hence against traditional assessment, let me just ask “Is it a choice only between traditional and alternative assessment or could the two actually complement each other?” Because at the end of the day, a high quality local assessment [alternative assessment] does indeed prove to support a higher test score (Dr. Wiggins, Grant).

Comment on Anderson (1998)

Why Talk About Different Ways to Grade? 

The Shift from Traditional Assessment to Alternative Assessment

by Rebecca S. Anderson

TUTORIAL TASK (Week 1 & 2):

Having read Anderson's article, what's your take on her argument 'from traditional assessment to alternative assessment in HE'. Give your reasons for the stand taken. 

After having read Anderson’s (1998) article, I found the arguments very true as this is the real situation that is happening out there right now. First and foremost, I agreed with Anderson’s point that it is high time for a shift from traditional assessment to alternative assessment. There are three points that I would like to briefly comment on here, i.e. (1) the considerations of each learner’s needs, (2) how ready would alternative assessment equip learners to face the real world demands and challenges and, (3) the role of the teacher.

Based on Anderson (1998), I found out that alternative assessment takes a more thorough consideration of the needs of every individual, especially on the part of the learners, in terms of their learning, improving, interests and goals. As compared to traditional assessment, learners are given more opportunity and empowerment in deciding what they are interested to learn, how they wish to learn a subject, and how they hope to be evaluated. This time round, they are not only providing suggestions but instead have their suggestion heard and be carried out, if not in full, to a certain degree. This allows students to generate a more self-regulatory learning and to approach the topic from their respective styles of learning and multiple intelligence.

In other words, alternative assessment promises a better approach in terms of equipping learners to face the real world challenges, needs and demands, perhaps even immediately, upon graduation from school- something which traditional assessment has never seen to be able to do in full up the this moment in time. The reasons as to why the former fulfills this role is that it takes into consideration of both the product and the process while only the product is emphasize in the latter. Thus, the how’s and why’s of learners’ efforts in achieving and producing the product is being valued and assessment, allowing them to score in the process as well as the product. Hence, they will also feel that their credentials and hard work are acknowledged.

Furthermore, learning is also viewed as an active collaborative process when it comes to alternative assessment. Therefore, learners are ‘forced’ to learn about teamwork, team management, organization, task delegation and decision-making. These are all the ‘on-the-look-out’ points of major companies today.

It reads 150cm.

As for the teacher, her role here is only as a facilitator to learning instead of being a dictator to learning. In other words, more empowerment would be given to learners and the teacher should not have full say in everything (activities, syllabi, assessments) done in class anymore. Thus, a teacher is also now a learner, learning from the ideas and development of their students.

** To read on Anderson (1998), please click here **