Instructions 1) Select one standardized language test used in your school for the critique. You can refer to a copy of a test available in your school for the K-12 tests. I am trying to get examples from K-12 and University tests both in the US and international.
2) Choose a specific language skill in the test that you want to focus on e.g. speaking, writing, structure etc. Take a closer look at the test samples, scoring systems, criterion etc. 3) Evaluate the test by using, but not limited to, the following elements of the test below. The questions are provided as guidelines for your critique.
a. Test purpose and Use
What are the intended uses of test? Who is the test designed for? What is the basis for considering whether the test is applicable for the target group?
b. Content Validity : The relevance of the test content to the goal of the test
Are the language skills constructed for the test relevant to the purpose of the test? To what extent does the test task relate to purposes of the test? Does the test sufficiently predict test takers' desired performance? What characteristics of the test rubric (if any) are likely to cause different test takers to perform differently?
c. Reliability : The consistency of measurement
Is the test free from errors such as ambiguity of test instructions, misleading samples, etc? Are there any ambiguous items? Are the test scores likely to be consistent across different testing situations? d. Scoring Method and Score Reporting
What is the scoring method used : criterion-referenced or norm-referenced? How are the test results reported to test-takers? Are they clear and consistent with the intended use of the test? What materials and resources are available to aid in interpreting test results?
e. Impact of the test
To what extent does the feedback that test takers receive from the test relate to their actual language use? How relevant and appropriate are the test scores to the decisions to be made? Is the purpose of the test consistent with instructional goals of teachers and curriculum? What are the potential consequences, both positive and negative, for society and educational system of using this test?
* Guideline questions above are adapted from Test Evaluation by Lawrence M. Rudner, ERIC/AE 12/1993 and Language Testing in Practice by Lyle F. Bachman & Adrian S. Palmer
