Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 12 No. 2. April 2008. (p. 42, 43) [ISSN 1881-5537]

Part I: Open Questions

1. A person with a hearing disability is asked to take an EFL placement test. Half of the test consists of reading questions which that person can undertake. However, half of the test consists of listening exercises. The examinee is able to read lips to some extent and also read closed-caption headlines, but has no ability to hear. What would be the most ethical way to rate this individual?
   
   
2. A person who may have a stuttering disorder is receiving a foreign language oral proficiency rating. Two of the evaluation criteria include pronunciation and the ability to naturally respond to questions and answers. You are unsure whether the examinee's stuttering is due to nervousness from this specific examination or a persistent speech disorder. According to the interview guidelines, only the target language can be spoken during the interview. How would you rate this candidate?
   
   
3. In an attempt to promote community service, one EFL teacher awards credit towards his students' final grades if they indicate that they have recently donated blood to a local blood bank. According to the school syllabus, this is a basic "English communication" class designed to foster "overall proficiency" in the target language. Any content validity issues here?
   
   
4. What information should test-takers receive before taking a test? Consider this example: one online company provides only the following details to potential examinees: (1) a claim that the test measures "correct pronunciation in [American] English", (2) the duration of the given test, (3) the number of test questions and general test format, (4) test fee information, and (5) an assurance of client confidentiality. To ethically market this EFL/ESL test, what additional information should be provided to end users?
   
   Source: ExpertRating. (n.d.) English Speaking Test. Retrieved March 11, 2008 from http://www.expertrating.com/english-speaking-test.asp.

- SUGGESTED ANSWERS -

[ p. 42 ]

Part II: Multiple Choice Questions

1. Increasingly, qualitative methods are being used to validate tests. One method is Conversation Analysis (CA). Which of the following is not a feature of traditional conversation analysis?
   
   (A) using authentic, recorded data which is fully transcribed (B) analyzing single cases or deviant cases (C) using turns as units of analysis (D) codifying and quantifying the data
   
2. If a person takes a multiple-choice reading test and selects an answer simply because the other choices don't seem correct, it is a _____________ strategy.
   
   (A) test-wiseness (B) test-management (C) language learner
   
3. Which of the following statements is/are true about p-values?
   
   (A) They indicate the likelihood of a correlation between two or more variables. (B) They indicate the direction of a correlation between two or more variables. (C) They indicate the strength of a correlation between two or more variables. (D) Actually, none of these statements.
   
   
4. Which of the following usually does not lead to score inflation?
   
   (A) Coaching effect from teachers who know what a given test will probably cover. (B) Exempting low-performing students from taking the test. (C) Random marking errors by those marking the test. (D) Narrowing the test focus: having the test focus on just a few aspects of the target curriculum.
   
   
5. Which of the following statements is true about power in a statistical sense?
   
   (A) It ranges from -1 to 1. (B) It should be used post-hoc and tailored to the data configuration. (C) It depends in part on effect size. (D) It reveals the likelihood of a Type I error.
   
   

- SUGGESTED ANSWERS -

---

NEWSLETTER: Topic IndexAuthor IndexTitle IndexDate Index
TEVAL SIG: Main Page Background Links Network Join

---

www.jalt.org/test/SSQ5.htm

ASSESSMENT LITERACY ONLINE SELF-STUDY QUIZES:

Quiz 1: Qs   / As    *    Quiz 2: Qs   / As    *    Quiz 3: Qs   / As     *    Quiz 4: Qs   / As    *    Quiz 5: Qs   / As   *   

---

[ p. 43 ]