UNIVERSITI MALAYSIA SABAH
B.Ed TESL (Hons)
TE
30503 LANGUAGE TESTING AND ASSESSMENT
Criteria for Evaluating Language Tests
Work in groups of four and
look carefully at the following test questions. Try to do the questions. What
observations do you have to make about them relating to issues of validity,
reliability, practicability, and washback effect.
1. Complete the following statements by
selecting one answer from the four provided:
i.
Jakarta is a large city_________ on the shores of
the Java Sea
a. is situated b. situated
c. situates d. which situated
Ans: B
|
ii.
Tokyo, which is Japan
a. this b. its
c. the d. that
Ans: C
|
Issues: Construct
Validity
-Uncertain of the theories behind the construction of
these questions (though it seems like it is a grammar test- but which aspect
is it testing on?)
|
2. Look at the sentences
which follow and rearrange them to form a dialogue.
John Oh, well I’m in a bit of a hurry myself. I
don’t think that I’ve got time to walk all that way and back again.
John Oh I see. Actually, I am going to the city
centre as well. Mind if I join you?
John Yes, I suppose so.
Mary Well, if you don’t mind I think I’d rather walk.
It’s such a lovely day and it seems a pity not to enjoy the weather.
John Yes. Bye
John Good morning, Mary!
Mary see you around sometime
Mary Good morning, John!
Mary Goodbye, John.
Mary Oh well, we’ll have to go our separate ways
then!
Mary Oh I’m just going down to the bank to get a
new cheque book.
Mary Sure. I was planning to walk – is that OK
with you?
Issues: Reliability
-Question is ambiguous in that it does not mentioned the
answering method (Are the students required to rewrite the whole text? Are
they just to numbers them, if so, how?)
-The text itself does not seem to even form a correct
discourse. Hence, attempts to rearrange the text would be futile.
|
3.
The sentences which follow should be examined with
great care and, if any verb forms are found to be incorrect, they should be
corrected. Answers should be written in the answer booking such a way that, in
each case, the correct form precedes the incorrect one.
i John is a teacher
ii Mary were a doctor (Ans: was/is)
iii Sometimes it snow in the Atlas Mountains. (Ans: snows)
iv The teacher went to London
Issues: Reliability
& Content Validity
Reliability
-The main issue here would be reliability. The question
is ambiguous, with no clear and explicit instructions as to what should be
done is the statements/options given above are correct.
-There could be more than one (1) answer for some
statements.
Content Validity
-Some of the statements do not even use proper
punctuation mark. Hence the issue of validity comes in as to what this
question is really trying to test (Is it to test subject-verb agreement,
tenses or punctuation marks).
|
4.
Vocabulary test
Choose one answer
from those provided to complete the sentences:
I Mary was angry whilst John was
positively ____________
A. anorexic B. apoplectic
C. artichoke D. artesian
II This beautiful little animal is called
a ____________
A. marmoset B. margrave
C. marimba D. martinet
III Animals such as cows generally
______________
A. rummage B. rusticate
C. roseate D. ruminate
Issues: Construct
Validity & Washback
Construct Validity
-What language theory is this test based on? What is
the reason and justification behind testing students such difficult and
extremely rare vocabulary? At the end of the day, the fact being that the
students are not going to use these vocabularies in their daily life.
Washback
-Are the students going to benefit from doing these
kind of test? How are they going to relate or use what they have learned here
to communicate in their daily life? Students who are doing this sort of test
would more probably be demotivated to learn English after this since it is
too difficult and irrelevant to ordinary school students, unless these sort
of questions is being posted to students taking semantics or hoping to
graduate as a dictionary writer.
|
5.
Listening Test
Look at the following
passage and then listen to the questions from the tape recorder. The questions
will be repeated once. Write the answers to the recorded questions in the
answer books. Do not write the answers in this test booklet.
Princess Diana has announced
that she will be retiring from public life in the near future. She has indicated
that this decision has been caused because the pressure of press attention has
become unbearable for her. She intends to spend more time with her sons, Prince
William and Prince Henry. Today’s newspapers have devoted many pages to
discussions of the crisis.
[Tape recorded questions]
1.
What are the names of Princess Diana’s sons?
2.
Why has the Princess decided to retire from public
life?
3.
How has the press reacted to the Princess’ decision?
Issues: Content
Validity & Face Validity
Content Validity
-This test does not test what it claims to test. Instead
of asking students to listen to the text before answering the questions,
students are required to read the text. This is more like a reading test.
Face Validity
-On the first look, this paper seriously looks like a reading
test rather than a listening test.
|
6.
Test of Communicative Competence
Put all the verbs in the
following passage into the correct form.
Once upon a time there (be)
a teacher (call) John. John (live) in a
city in the North of England. John (travel) a lot for his job; in one year he
(visit) nine different countries. All John’s friends (think) that John (be) very
lucky because he (can) travel so much. However, John actually (hate)
aeroplanes, and so every overseas visit (be) torture for him! Yet he never
actually (admit) his problem to his colleagues because he (worry) that he
(lose) his job.
Issues: Face
Validity & Reliability
Face Validity
-This test looks more like a reading test than a
grammar test.
Reliability
-The instructions used are ambiguous and misleading.
How is one suppose to “put all the verbs…in the correct form” when there is
nowhere to take it from and nothing to choose from. The proper word to be
used could be “change” instead of “put”
-It is ambiguous and full of uncertainty as to how one
ought to answer the question. Are the students suppose to write the answer in
the answer booklet or simply to erase of the wrong form in the brackets and
re-write the correct forms?
|
7.
Elementary Level Test
Write a letter to your local
councilor complaining about some aspect of local government services – for
example, rubbish removal.
Issues: Reliability
-Again the test item has ambiguous instructions. For
instance, the number of words that a student should write is not mentioned.
-Secondly, the students are granted with too much
freedom in terms of the area and scope of government services. The
complain letter could range from the
subject of rubbish removal to the failure of possibly the Prime Minister’s
office in entertaining the people.
|
8.
Test of Spoken English
Look at the timetable below
and tell your teacher about it.
Malaysia
Ipoh Kuala
Lumpur
07 00 10 05
09 00 12 05
11 00 14 05
1400 17 05
Issues: Reliability
-As mentioned above, the area or topic of discussion is
not mentioned. The problem would come if the students just tell the teacher
what they see in the test paper and ends with “I do not understand”. How is
the teacher going to mark the student?
After all, the topic is not mentioned, hence anything is accountable.
|
No comments:
Post a Comment