Press Releases

College Board President Delivers Testimony about October 2005 SAT Scoring Issues to New York State Senate Higher Education Committee

The following testimony was presented by College Board President Gaston Caperton at 10:00 a.m. on Tuesday, May 2, 2006, to the New York State Senate Higher Education Committee in Albany, New York.

05/02/06

Testimony of Gaston Caperton, President, The College Board, New York, New York

New York State Senate Higher Education Committee
May 2, 2006
Albany, New York

Good morning, Senator LaValle and members of the Committee. I am Gaston Caperton, President of the College Board. On behalf of the Trustees and members of the College Board, I thank you for this opportunity to appear before you this morning.

For more than 100 years, the College Board has played a prominent role in American education. The college admissions process is at the heart of our services, of course; so when any issue arises regarding the SAT, it is of paramount importance to us. Knowing of the Committee's keen interest in the scanning problem on the October 2005 SAT, I want to review what happened and how the College Board responded.

We had three priorities in getting the information out:

Promptly, accurately, and responsibly providing corrected scores to students and parents.
Promptly, accurately, and responsibly providing corrected scores to colleges and universities.
Promptly, accurately, and responsibly providing information to the public.

Some have said that we should have made a public announcement as early as January, when we first suspected there might be some sort of problem, but before we knew its nature, scope, or effect. We respectfully disagree. Based upon our long experience in testing and admissions-related work, it was clear to us that it would have been irresponsible to make an announcement at that time. Putting out incomplete and unreliable information would have caused the October SAT test-takers who were not affected by the scanning problem—more than 99 percent of the total—to have worried needlessly about the accuracy of their scores. Doing so could also have severely disrupted the 2006 college admissions process by leaving colleges, universities, and schools uncertain about the accuracy of any of the information they had.

As soon as we understood the nature of the problem and who had been affected by it, we fully disclosed it—not just to students and schools but also to the public and the press. When, after getting the word out as quickly as we could, we learned that a small additional percentage of students might have been affected, we continued without pause to complete the job we had set out to do. We determined the facts, re-reported several hundred additional test scores, and publicly disclosed our actions as promptly, as widely, and as candidly as possible.

I come before you today to say that the SAT is our test. We are proud of it, we believe in it, and we are responsible for it, in good times and bad, and we take that responsibility very seriously. Our focus is, and has been all along, on doing everything we can for the students who take our tests and the schools that use our test scores as part of their admissions process. We are committed to giving them the very best service and to making every effort to prevent this kind of occurrence in the future. The staff of the College Board has been honest, straightforward, and clear with students, our members, and the general public and we will continue to be. This is an organization with a proud history of service to our members and to the students and families of this nation. Our hard work over that time has been rewarded with the trust and confidence of students and their families, colleges, universities, and schools. We will continue to do everything we can to continue to earn their trust and confidence.

That is why we are taking concrete actions intended to make sure this never happens again. Beginning with the April 1, 2006, administration of the SAT, our scanning vendor, Pearson Educational Measurement, has:

Scanned each answer sheet twice to ensure accuracy.
Ensured that all answer sheets are acclimatized before scanning to eliminate any possibility of a humidity impact.
Used new software to identify any answer sheet expansion that might occur due to humidity.

We are exploring a number of additional procedures that could prevent future problems, including further communications with students concerning the importance of marking answer sheet bubbles darkly and completely. Finally, the College Board has engaged the consulting firm Booz Allen Hamilton to perform a comprehensive review of SAT scoring, with particular emphasis on the scanning process.

If additional measures are warranted, we will take them.

Before I address the October SAT scanning problem in greater detail, let me provide some background on the College Board.

The College Board

Founded in 1900, the College Board is a national not-for-profit association of more than 5,000 member schools, colleges, and universities—including more than 400 member institutions in the state of New York—with a challenging mission: To connect students to college success and opportunity, with a commitment to excellence and equity.

Many of you may know the College Board best as the sponsor of the SAT, PSAT/NMSQT and Advanced Placement Program, which together add up to more than 6 million exams taken each year worldwide. The College Board is also an organization that provides hundreds of other programs and services to its members, and directly to students and parents, with the aim of opening the door to college, and ensuring successful completion of college, for all of our nation's young people. Our programs and services include the sponsorship and operation of scholarship programs, extensive teacher professional development, curriculum development, direct services to students and adults, and research on the most pressing issues related to education.

Among the most significant programs and services of the College Board are:

Advanced Placement Program. Through its AP Program, the College Board is supporting the efforts of a growing number of states and school districts to promote high academic standards, increase student academic performance, and prepare all students for enrollment and success in college. Because research consistently demonstrates that participation in AP dramatically impacts traditionally underserved minority students' likelihood of obtaining college degrees, the College Board has fostered consistent increases in both the participation and the exam scores of minority students over the past decade. As states seek to raise standards and ensure that all students take rigorous courses that prepare them for college and careers, AP is recognized as a powerful tool for increasing academic rigor, improving teacher quality, and creating a culture of excellence in high schools.

SpringBoard. This integrated, Web-supported program provides students in grades 6-12 with the critical thinking, reasoning, mathematics, and reading and writing skills they need to succeed in college. SpringBoard is designed around the rigorous College Board Standards for College Success, which identify the skills and competencies students need to be prepared for college. SpringBoard has back-mapped these competencies from grade 12 down to grade 6. This enables school systems to provide a systematic, seamless, and challenging curriculum throughout middle and high school to better prepare students—especially low-income students—for college success.

The SAT. The SAT measures critical thinking, reasoning, and writing skills that students have developed over time and skills they need to be successful academically. The SAT is the oldest, most thoroughly researched standardized measure of a student's college readiness. High school grades are a very useful indicator of how students perform in college, and colleges consider them carefully in the admissions process. However, grading standards and course rigor vary greatly within and across high schools. As a standardized measure of college readiness, the SAT also plays an important role in college admissions. Each of the SAT's three main sections is scored on a scale of 200-800. The SAT is typically taken by high school juniors and seniors.

PSAT/NMSQT (Preliminary SAT/National Merit Scholarship Qualifying Test). For many students, the PSAT/NMSQT, with three parts—critical reading, math, and writing skills—is the first official step on the road to college. The PSAT/NMSQT also provides personalized feedback on test performance and suggestions for improving specific academic skills.

CollegeEd. CollegeEd provides students and families, particularly from underserved communities, with information—to make self-directed, informed decisions about course work and other activities. CollegeEd instills in middle and high school students the belief that college is a possibility for everyone; it also provides students and their parents with access to information about financial aid and scholarships.

New York Headquarters

The College Board is a national organization, but New York is our home. The College Board's national headquarters is located in New York City and our proximity to New York State's schools and students has enabled us to form a strong, long-term relationship with the New York State K-12 and higher education communities.

In 2005, New York State's college-bound seniors led the nation in SAT participation at a rate of 92 percent, representing a 15 percent increase since 1995. Over the last 10 years New York State's students also increased their average SAT scores—in verbal by 2 points to 497, and in mathematics by 13 points to 511. In 2005, New York State's students also led the nation in both the proportion of students who took an AP Exam in high school (34.8 percent of all graduates) and the proportion of students who scored a 3, 4, or 5 on that exam (22.8 percent of all graduates). Many students receive course credits or advanced placement in college based on AP scores of 3 or higher.

The College Board has long-term, constructive relationships with the New York State Education Department and the New York City Board of Education. In New York City, this collaboration has led to the creation of several College Board public schools, serving primarily low-income and minority students in grades 6-12, with support from Gates Foundation and Dell Foundation grants. There are already five such schools in operation, with plans to establish at least 11 more in New York City and elsewhere in the State.

College Board partnerships reach students throughout the State, including Yonkers, Syracuse, Rochester, and the Bronx, where our aforementioned programs, services, and support systems have been used to help districts meet their educational goals. The College Board is an official partner in the New York State Higher Education Services Corporation (NYS HESC) New York State Gear Up grant, collaborating with the City University of New York, the State University of New York, and independent colleges to provide services to disadvantaged students across the State.

New York students benefit substantially from their participation in College Board programs and services. We are working diligently to expand our partnerships with the State and its schools, colleges, and universities, with the aim of preparing all students for enrollment and success in college, and in the world of work.

The October 2005 SAT Scanning Problem

Mr. Chairman, accuracy, accountability, and integrity always have been and always will be the foundation of the College Board. The students who participate in our programs and receive our services expect us to operate with the highest standards, and we expect that of ourselves and of the firms that provide goods and services to us. We recognize that errors occur from time to time in all fields of human endeavor, but we continue to strive for perfection. That means diligently working to minimize errors, investigating and correcting them should they occur, and disclosing them forthrightly to test-takers, test users, and the public. We are therefore devoting all necessary resources to addressing the October 2005 scanning problem, preventing a recurrence of problems of this kind, and keeping the public fully informed.

We believe our actions have shown that the College Board has responded appropriately to the October 2005 scanning problem. However, we believe some of the reactions just don't represent the facts. As an example, one law firm has posted the following on its Internet Web site: "Nearly 4,000 high school students who took the October SAT test received the wrong score. The scores were off by nearly 400 points." As I will discuss, the fact is that 5/6 of the underreported scores were too low by 40 points or less (on a 2400 point scale), and only 10 scores were off by 300 points or more. About half of the affected test-takers had higher scores on file, so it is unlikely that their admissions prospects would have been affected by the October 2005 scanning problem. Finally, as discussed below, the College Board re-reported all of the scores that had initially been underreported before the end of the spring admissions season, and colleges and universities have said that they took them into account before making final decisions.

Now, please allow me to discuss the October 2005 scanning problem. Because of the large numbers of students who take the SAT, it is necessary to use optical scanning technology as part of the multiple-choice scoring process. The College Board is not a scanning company. Accordingly, we have contracted with Pearson Educational Measurement, a leader in the optical scanning industry, to scan SAT answer sheets. Pearson scanned the answer sheets for the October 2005 administration of the SAT.

Over the past several months, the College Board has asked Pearson to help us determine what happened with the October 2005 SAT scanning. In sum, Pearson has explained to us that an unusual amount of moisture in some answer sheets led to the expansion of the paper on which the answer sheets were printed. This expansion, in combination with light or incomplete marks by some test-takers, caused some responses not to be recorded by Pearson's scanning equipment.

The following is a timeline of what transpired once we began to suspect that there might be a problem:

The SAT was administered on Saturday, October 8, 2005, and the College Board reported scores during the week of October 24. Score Verification (hand scoring) requests are typically received soon after scores are reported and take 3 to 5 weeks to complete, which is consistent with practice in the standardized testing industry. Hand scoring usually results in no score change. However, scores may change when, for example, the test-taker erased one answer incompletely and marked a second answer lightly; a scanning machine may "read" such marks as two answers to the same question while a human reader may reach a different conclusion about the student's intent. The College Board received hand-scoring requests from two students on December 14 and December 22, 2005.
Hand scores on the SAT are conducted for the College Board by Educational Testing Service (ETS). The hand scores for these two students were completed by ETS in late January. On January 31, the College Board determined that the differences between the hand-score results and the original scores based on scanning for these two students could not be easily explained. The College Board asked Pearson to immediately investigate and report back on the reasons for the differences.
Pearson began its investigation during the first week of February.
Pearson told us that the problem appeared to go beyond the two hand-scored answer sheets. The College Board therefore asked Pearson to rescan all October 2005 SAT answer sheets. Consideration was, of course, given to what we could or should publicly announce at this point. In our professional judgment, we decided it would be irresponsible to make a public announcement before we had more information about the nature of the problem, how many students were affected by it, who those students were, and how they were affected. I concurred in that decision based on my experience at the College Board and, before that, as a public servant. To have issued a public statement at that point would have caused unnecessary alarm and confusion with no offsetting benefit. Instead, we resolved to learn as much about the problem as we could as quickly as possible so that we would be able to announce our findings before the close of the college admissions season.
To be sure that the scanning problem was correctly diagnosed, the College Board decided that Pearson must also rescan all answer sheets from subsequent SAT administrations. In all, more than 1.4 million answer sheets were rescanned. Pearson reported to us in late February that there was no evidence of the October 2005 scanning problem in the other administrations. Pearson's analysis also showed that only a small number of the approximately 495,000 SAT test-takers in October 2005 were affected. Those test-takers' answer sheets were then rescored by March 2.
During the week of March 6, the College Board re-reported scores for 4,018 students (approximately 0.81% of October 2005 SAT test-takers) who received higher scores as a result of the rescanning process.
A massive communications and outreach effort was launched on that date. Among the steps we took were the following:
- College Board staff contacted affected students by e-mail and also sent revised official score reports and an official letter from the SAT Program. Revised scores were also made available to students online. The College Board told affected students that we were refunding their October test registration fees (including late and standby fees) and fees associated with sending scores from the October test administration to score recipients.
- College Board staff telephoned the affected students who had the most significant changes in their scores.
- College Board staff contacted all of the colleges and scholarship agencies to which any affected students had sent SAT scores and sent revised score reports and a roster of affected students. In addition to re-reporting scores and explaining what had happened, we urged all of those colleges and scholarship agencies to take this new information into account, even if it meant re-reviewing admission and scholarship applications.
- The College Board set up a special hotline for students whose exams had been rescored, and College Board staff has worked directly with students and parents on their particular situations. The College Board offered to contact colleges on behalf of affected students to explain their situation; we were happy to assist any student who requested that we do this and we were pleased with the response we received from admissions officers.
- The College Board also created a special Web site for affected students (http://www.collegeboard.com/student/testing/sat/scores/oct_2005_sat.html) that has given students another way to communicate with us and for us to provide as much guidance and assistance as we can.
- The College Board sent revised score reports to the high schools attended by affected students, along with corrected October labels.
- In addition, during the week of March 6-10, the College Board express-mailed informational letters to admissions officers, deans and directors of admission, high school principals, and high school directors of counseling. We e-mailed admissions officers who receive electronic score reports, students and counselors directly, and posted an informational message on the e-list forum of the National Association for College Admission Counseling (NACAC) and on our own Web site. We made ourselves available to major print media outlets, such as the New York Times, USA Today, the Los Angeles Times, the Wall Street Journal, Time, Newsweek, and the Associated Press, and major television networks and smaller print and broadcast outlets throughout the country. We responded to hundreds of press inquiries, answered their questions, and provided information to them.
On March 15, the College Board announced that approximately 1,600 answer sheets from the October 2005 administration had not been rescanned. The 1,600 had been held separately from the rest of the October 2005 answer sheets because they were on administrative hold at ETS for test security reviews having nothing to do with the scanning problem. As I stated above, during the month of February more than 1.4 million answer sheets were rescanned. This involved a tremendous amount of work in an extremely short period. Quite frankly, the fact that the 1,600 answer sheets were not rescanned at the same time as all the rest was the result of human error. When the College Board recognized the error, we disclosed it publicly and worked vigorously to address it.
Also on March 15, the College Board asked Pearson to once again confirm that it had fully rescanned and reprocessed all answer sheets from the October 2005 administration of the SAT.
During the week of March 13-17, we sent informational update e-mails to counselors and admissions officers, and representatives of College Board member institutions. We mailed informational update letters to college presidents and deans of admission via overnight service, and again posted a message to counselors on the NACAC e-list and on our own Web site. We continued to reach out to major print media outlets as well as large and small television networks and outlets throughout the country, and we continued to respond to daily press inquiries.
On March 19, Pearson informed the College Board that approximately 27,000 answer sheets in its possession had not been fully evaluated, and that Pearson would promptly complete the rescanning process for those tests.
On March 22, the College Board reported that 393 test scores—18 out of the 1,600 on administrative hold and 375 out of the 27,000 that had not been fully evaluated by Pearson—were being re-reported. All such scores were re-reported within the next 48 hours.
The College Board launched a second wave of communication and outreach on that same date:
- College Board staff contacted the students in this second affected group by e-mail and also sent revised official score reports and an official letter from the SAT Program. Revised scores were also made available to students online. The College Board informed students in this second affected group that we were refunding their October test registration fees (including late and standby fees) and fees associated with sending scores from the October test administration to score recipients.
- College Board staff again telephoned students with the most significant score changes.
- College Board staff contacted all of the colleges and scholarship agencies to which any students in this second affected group had sent SAT scores. We sent revised score reports and a roster of affected students to each institution. In addition to re-reporting scores and explaining what had happened, we urged all of those colleges and scholarship agencies to take this new information into account, even if it meant re-reviewing admission and scholarship applications.
- The College Board sent revised score reports to the high schools attended by affected students, along with corrected October labels.
- During the week of March 22, we e-mailed this updated information to counselors and admission officers and express-mailed explanatory letters and revised score reports and rosters to admissions officers, directors of counseling, principals, and students. We also continued to work with the media outlets described above.
On March 22, Pearson certified to the College Board that it had completed the reevaluation of all answer sheets received from the October 2005 administration of the SAT Reasoning Test. The College Board disseminated a press release nationwide with a chronology and full details of the situation and responded to all media inquiries.
Finally, during the week of March 27, we mailed letters of regret to college presidents, admissions deans, College Board member delegates in colleges and schools, and also to counselors.

As has been reported, the rescanning process revealed that 613 students (approximately 0.12% of October test-takers) received scores that were somewhat higher as a result of the scanning problem. All but one of those score differences involved a total of 40 points or less across all three tests on the SAT's 2400 point scale; the remaining student had a 50 point differential.

The College Board has decided as a matter of fairness not to re-report these students' scores. We believe it would be unfair to penalize students for test-taking factors beyond their control and for which there was no readily available remedy (for instance, no ability to re-test before submission and consideration of their application). In light of the very modest amount by which these students' scores would have changed, we doubt our decision disadvantaged any student.

On March 22, the College Board announced that we had directed Pearson to implement a series of new procedures and policies to minimize the chance of scanning problems in the future. Beginning with the April 1, 2006, administration of the SAT, Pearson has:

Scanned each answer sheet twice to ensure accuracy.
Ensured that all answer sheets are acclimatized before scanning to eliminate any possibility of a humidity impact.
Used new software to identify any answer sheet expansion that might occur due to humidity.

The Impact on Students

Mr. Chairman, from the outset our primary concern has been the impact of the scanning problem on students and their families. If only one student was affected by the scanning problem, that is one student too many.

Although the SAT plays an important role in the admissions process at many colleges and universities, the College Board encourages its members to consider a variety of other factors and strongly discourages the use of cut-off scores in admissions decisions. Virtually every college and university conducts its admissions process in this manner, thus minimizing any impact of the scanning problem.

At the urging of the College Board, colleges and universities report the SAT scores of their entering students in ranges, from the 25th to the 75th percentile. This is done specifically to reinforce the point that SAT scores are only one measure colleges use in evaluating applicants, and that students with a wide range of SAT scores are admitted to every college. SAT score ranges for the "middle 50 percent" of entering students average about 100 points on each section, or about 300 points across the SAT as a whole. Of the 4,411 students whose scores were re-reported, there were only 10 whose scores increased by that much or more. The vast majority had score increases of less than 100 points, and 83 percent of the score increases were less than 50 points across all three sections.

In addition, half of all SAT takers test more than once, and the re-reported October 2005 scores of approximately half of the affected students were not their highest scores. This is important because 90 percent of colleges state that they consider a student's highest SAT scores for admissions purposes when the student tests more than once.

As I mentioned previously, we have been in close contact with admissions officers at colleges and universities across the nation and have offered every assistance we can throughout the admissions season. We have appreciated the good will of these institutions in reconsidering students. In most cases, because the score changes were modest, there was no change in decisions already made or being made. In a few cases, we have been told that the new, higher scores did make a positive difference and that students were being reconsidered for admission and for scholarships.

All of this taken together—the fact that the SAT is only one factor in admissions, the fact that so few students experienced a significant score change, the fact that many students have higher scores on record, and the good and hard work of admission officers—leads us to believe that we have done everything we could have done to minimize the effect of the scanning problem on October 2005 SAT test-takers. But let me say again: if only one student was negatively impacted, we regret it deeply.

Let me close by stating that the College Board's mission is to serve students, schools, and colleges and to increase the possibility of college success for an ever-greater number of our nation's young people. We have done our best to stay true to that mission throughout our long history and through these trying recent times. We are determined to learn from this experience and to press on more vigorously than ever with our important educational mission.

Thank you again for the opportunity to share this update with this Committee.

< Previous Back to All Next >