News | Connections | E-Bridges

High-Stakes Assessments: A Review of the Chinese Civil Service Exam

By Suzanne Wayne (December 2006)

As college admissions becomes more and more competitive, standardized assessment exams like the SAT are viewed by prospective students and their families with more importance, and in many cases, with increasing stress and anxiety. Furthermore, assessment tests are now used to determine the success of schools and individual teachers, with exam outcomes determining real rewards or punishments. In both cases, the stakes are high.

Professor of educational psychology, Hoi Suen, has termed assessments such as these as “high-stakes” assessments. In his recent paper, “Chronic Consequences of High-Stakes Testing? Lessons from the Chinese Civil Service Exam,” which appeared in Comparative Education Review in 2006, he especially looks at the long history of the Chinese Civil Service Exam to identify some of the negative consequences of such tests, and whether these consequences can be avoided.

The Keju examination was a government-orchestrated system of high-stakes employment testing, education testing, and test-driven education. It began in the year 606 and officially ended in 1905. At its height of implementation, millions of examinees were tested. Keju exam results determined government position appointees from grand councilors (similar to prime minister) down to local positions, as well as educational opportunities, and their resulting financial rewards.

Suen explains, “With approximately 1,300 years of history and extensive official and unofficial records that were kept throughout this period, China’s is the only examination system that can provide us with a glimpse of what might be some long-lasting chronic problems of high-stakes, large-scale testing programs as well as of the efficacy of attempts to remove unintended negative consequences.”

Through his research of this exam, Suen determines that a number of negative consequences developed from this high-stakes assessment, including memorizing, teaching to the test, focusing on test-taking skills, cheating, and psychopathological effects.

Memorization

The earliest form of the Keju involved writing philosophical essays. This quickly led to memorization of successful essays by test-takers and producing them word for word on the exam. Attempts to curtail this by adding a poetry section did not alleviate the problem, as test takers merely moved into memorizing poetic combinations.

Suen concludes, “We can surmise that as long as there are high stakes associated with a test, examinees will attempt to improve their own performance by memorizing by rote and by imitating surface features of past successful performances.”

Teaching to the Test

The rote memorization of essays, and the subsequent introduction of poetry to the exam, led to the development of “test-coaching” books. These books taught students “tricks” to writing essays and poems in the required styles. For example, one book contained pairs of words, phrases, and even complete sentences that were matches for one another under the style rules, and, thus, were necessary for composing matching couplets of sentences. Books like this became supplemental textbooks used in many schools.

However, the art of couplet writing was never the intended skill to be assessed by the Keju exams. Rather, the ultimate purpose of essays and poetry was to demonstrate an understanding of the classics.

Says Suen, “The couplets, as a format, were no more than a surface feature of the exams—no more meaningful than, say, the art of answering multiple-choice questions is to the intended constructs measured by the SAT. Focusing on the art of couplet writing by encouraging the memorization of contents of these rhyming primers was a form of test coaching, serving only to improve test-taking skills.”

Cheating

Cheating on the Keju was rampant throughout its history. It took many forms, including hiring of substitutes, hiding and using cheat notes, and bribery of exam officials. One form of cheating included bribing examiners to provide high scores to anonymous essays with certain agreed-upon key phrases at agreed-upon points in the essays.

Officials took many measures to limit cheaters, including sequestering examinees in large exam compounds, searching examinees, anonymous exam forms, isolated examiners, and severe physical punishment for known cheaters. Yet, in spite of these measures, cheating continued. As new measures were introduced, examinees found new ways to sidestep them.

Psychological and Behavioral Problems

The anxiety from preparing for the Keju exams and the disappointment of doing poorly on them led to incidents of pathological behavior, ranging from mild cases of self-doubt to suicide or physical violence directed at others. In fact, the prevalence of these psychological problems led physicians to identify a particular disorder known as the “Keju syndrome.”

Today, China continues to experience exam-induced psychological and pathological problems, including suicides of high school students who did not perform well in China’s current college entrance exam (called the Gaokao). Exam-induced suicides are also found in Korea, Taiwan, Hong Kong, Japan, Vietnam, Cambodia, and Singapore today.

Conclusion

A review of the history of the Keju exams, has identified a number of negative consequences with no evidence that these problems can be avoided or eliminated. Thus, these negative consequences could be found to varying degrees in most, if not all, of our current high-stakes testing programs.

Suen concludes. “We need to move beyond finding ways to eliminate these consequences from high stakes testing, as they would most likely be met with new forms and manifestations. Rather, test theorists and psychometricians need to develop appropriate and realistic models of scoring or of the use of assessment results in which such chronic consequential factors are accepted as inevitable components of the score or of the system.”

He adds, “From the perspective of policy decisions, the solution will be found not in designing procedures to stop such chronic problems, but from taking measures to minimize the stakes of tests.”

###

The Penn State College of Education serves approximately 2,800 undergraduate and 1,200 graduate students each year. The College prepares administrators, counselors, psychologists and researchers, as well as K-12 teachers in 21 different specialty areas. All of the College of Education graduate programs, that are ranked by the U.S. News & World Report, appear at least in the top 15, with six programs in the top ten.The College's Higher Education Administration program is ranked 1st and the Workforce Education and Development program is ranked 2nd. The College is known nationally for its education research and outreach, housing such centers as the Center for the Study of Higher Education, the Center for Science and the Schools, the Mid-Atlantic Center for Mathematics Teaching and Learning, and the Regional Education Laboratory--Mid-Atlantic.

For more information on Penn State’s College of Education, contact EdRelations@psu.edu, call 814-863-1192, or visit www.ed.psu.edu.