The Rankings Czar
Bob Morse
Photo by Ron Aira
Tropical Storm Allison blasted through Houston killing 22 people, flattening homes and drenching the University of Houston Law Center in mud and floods. That was 2001, closing out Nancy Rapoport’s first year as law school dean.
“Our computers survived only because our IT director risked his life to move classroom PCs to higher floors through the night of the storm,” Rapoport says.
The faculty and staff—some of whom lost their own homes—removed debris from buildings, scrounged supplies and aided students displaced by the storm. Despite the stress, Rapoport was exhilarated by the camaraderie. She never shed one tear of frustration.
Maybe that’s why her tears at a meeting with students and faculty six years later became a law school legend. The school had learned, just a week earlier, that it had fallen five spots (to 70th) in U.S. News & World Report’s annual law school rankings.
Distraught faculty and students wrote scathing critiques of her performance on blogs and computer bulletin boards, noting the school had plummeted nearly 20 spots during her six-year tenure.
Rapoport resigned.
Nancy Rapoport
Photo by Jamie Rector
Now a law professor at the University of Nevada-Las Vegas, Rapoport is still amazed to see that her tears that day seem always near the top of any story that mentions her. The Rice University and Stanford Law alum is a nationally respected bankruptcy expert. She co-edited a book about corporate fiascoes and was interviewed onscreen in the Oscar-nominated Enron: The Smartest Guys in the Room. There are many facets to Rapoport’s career beyond those tears. Still …
“Am I the poster child for why the U.S. News & World Report rankings are bad?” Rapoport wondered last year in her blog.
It’s perhaps more accurate to say Rapoport’s Houston ordeal scares the bejesus out of deans. Her tears are emblematic of the nerve-shredding power U.S. News rankings hold over deans, faculty and students.
“Everyone is well aware of U.S. News [rankings] and their effect in the real world,” says Frank Wu, who is stepping down this year as Wayne State University Law School dean. He describes the weeks just prior to the new rankings issue as excruciating. “There is always a sense of anxiety, foreboding, even dread.”
“It would be hard to overestimate the effect law school rankings have on everyday campus life,” says Paul Caron, dean of faculty at the University of Cincinnati College of Law and a vocal critic of the magazine’s methodology. “Students worry about how a ranking will affect their chances of future employment. Faculty worry about whether a drop in rankings will make it more difficult to be published in law reviews. Deans feel that they will be judged by how many places up or down the school moves in the rankings.”
Perhaps not surprisingly, behind all this angst is an equally broad-based resentment. Since it began the rankings in 1987, the magazine is often attacked as wielding too much power; its methodology is denounced as easily manipulated and too subjective to carry such inordinate weight.
No one understands this more than Robert Morse, the man who created the law school rankings for U.S. News. As the magazine’s data research director, Morse says he, too, feels a high level of anxiety each year when the law school rankings are revealed.
“It’s very nerve-racking. I always lose a lot of weight then,” Morse says. “It’s definitely anxiety-driven supermetabolism, because I could drink five milkshakes a day and still lose the weight and I can’t do that any other time of the year.
“We’re the only mass media organization doing the comprehensive rankings and I feel the responsibility,” Morse says.
He also feels the heat from those who resent their enduring influence. For a ratings czar, he is a very reluctant despot. Far from being impervious to complaint, he maintains a blog where he explains his rankings and encourages constructive criticism. He’s been known to show up unannounced at gatherings likely to denounce him.
Surprisingly, he thinks it is others who invest the rankings with greater importance than they deserve.
He defends them, neither as calculus nor social science, but as an honestly derived “guide.”
MEAGER ALTERNATIVES
Critics of U.S. News have long called for alternatives. Some have been suggested and created, but with little popular success. In 1998, law school deans even implored prospective students to pay no attention to rankings at all.
Deans from 164 accredited U.S. law schools (including highly ranked Columbia, Cornell, New York University, UCLA, Virginia and Yale) signed a letter sent to 93,000 potential applicants saying the rankings are flawed and urging them to rely on their own observations and individual investigation.
“Rankings generate huge hype, which is far more likely to serve the publisher’s purpose than the reader’s,” the letter read.
Ten years later, the U.S. News rankings are still the guide most applicants use when pondering law schools to attend. And since the rankings are impossible to ignore, some law school deans, even longtime critics, believe it is time to parley with U.S. News and try to make them fairer.
“I think rankings need to be changed, and the only way that will happen is if law school deans sit down with Bob Morse for honest discussion,” Rapoport says. “I would attend a meeting like that without hesitation.”
Morse says he understands and agrees that the rankings are not perfect, and he would like nothing more than to discuss with law school deans ways to improve them.
“Deans are welcome to call me or come by my office in Washington,” Morse says. “I want to work with them to improve the rankings.”
But to discuss the rankings would be to validate them in the public eye, and the deans and other critics seem to remain reluctant to do so.
U.S. News law school rankings have gone through a series of changes since 15 top law schools were ranked in 1990. Last year, the magazine ranked 100 of 184 ABA-accredited institutions as “Top Law Schools” for 2008, combining what used to be the top two tiers of a four-tier listing.
Specifically, according to Morse, the schools are rated on the basis of about 18 factors that fall into four general categories: selectivity, placement success, resources and reputation. The top school is given a score of 100 percent, with descending scores for all others.
Raymond Nimmer
Photo by Nash Baker
Below the top 100, other law schools are tiered but not ranked. Tier III holds schools whose scores fall between the 45th percentile and the 26th percentile. Tier IV holds those that score in the bottom quarter. In both tiers, the schools are listed in alphabetical order.
Critics of the U.S. News rankings say the magazine exercises too little control over the quality of the information submitted; several of the self-reporting factors utilized in the methodology, they say, actually reward those law schools willing to cheat.
Selectivity, for instance, includes the median student LSAT scores, undergraduate GPAs and acceptance rate, and it accounts for 20 percent of the overall rating. The Association of American Law Schools has reported that some schools increase rejection rates—and boost selectivity scores—by encouraging students with no chance of admission to apply.
Placement success, which accounts for 25 percent of the overall rating, combines bar passage rates with self-reported job placement rates—student employment at time of graduation and those employed within nine months of graduation.
University of Texas professor Brian Leiter, who compiles his own independent law school rankings in competition with U.S. News, calls the job placement data “essentially fiction.”
“It may have elements of truth, but basically it’s a work of the imagination,” Leiter says on his blog.
Rapoport says reporting students as “employed” if they have any kind of job—whether at a federal courthouse or a fast-food cash register—is commonplace, but she refused to do so.
“There are deans who will hire a student to [photocopy] papers, work that has nothing to do with the law, and they count those students as successfully placed,” Rapoport says. “My school was punished with a lower ranking because I wouldn’t fudge placement figures.”
Resources is a category that counts for 15 percent of a school’s score and includes such statistics as school expenditures per student, financial aid, number of library volumes and support services.
Still, deans complain that schools turn in slipshod or misleading stats to lift their resource ranking. In 2005, for instance, the New York Times revealed that the University of Illinois College of Law in Urbana-Champaign reported $8.78 million spent for LexisNexis and Westlaw database subscriptions, about 80 times what was actually charged. The university had, in effect, inflated its per-student expenditures by calculating the fair market value of the services, not the deeply discounted amount the school actually forked out.
But it is the dreaded Reputation category that draws the most suspicion and ire. Weighted at an enormous 40 percent of a school’s total rating, it is the one category that can make or break a law school’s ranking.
To measure reputation, according to Morse, U.S. News surveys 1,300 practicing lawyers, judges, deans and hiring partners, who rate the schools on a scale of 1 (marginal) to 5 (outstanding). Respondents remain anonymous.
But deans complain there is no way to know whether the anonymous people rating their schools have ever set foot on their campus, visited their websites or know anything about any of the school’s programs or faculty.
Some schools work to enhance their chances the old-fashioned way: marketing.
“We easily spend $100,000 on glossy marketing materials to send to the people we think will be filling out the surveys,” says Wu, the Wayne State law dean. “There are schools spending more than twice that. That’s money we could be spending on faculty and students. It feels like a trap none of us can escape on our own.”
Low placement numbers pushed Wayne State, which is located in Detroit, into the fourth and lowest tier last year. Wu took responsibility, saying an employment survey sent to recent graduates was long and confusing.
Wu’s alumni and faculty stuck by him. The junior faculty even offered to take him out drinking to commiserate. One alumnus promptly mailed Wu a $5,000 contribution for the school. Just weeks before the announcement, the school had received two of its largest gifts ever, totaling more than $6 million.
But there was still fallout among current and potential students, Wu says. The school’s yield rate dropped, meaning more students who had been accepted were declining to attend in greater numbers than before.
While he attended Wayne State, James Britton was president of the Student Bar Association board of governors. He was among dozens of students who attended a meeting where Wu explained the problematic career placement questionnaire.
“Students were upset and tempers were running high,” Britton recalls. “Their language was strong, but I think it needed to be strong.
“Rankings count for students, not just school applicants. Law firms in Michigan know and respect the school’s reputation. Outside the state, the rankings probably are a quick guide a lot of recruiters use to form an opinion. There was also a feeling among students considering a transfer to another law school that the drop to fourth tier gave them less mobility,” says Britton, who now works at a Detroit-area labor law firm.
VENTING IN THE BLOGOSPHERE
Caron of the University of Cincinnati edits an online publication known as TaxProf Blog that, despite being exactly what it sounds like, often becomes a forum for venting about law school rankings.
“How does a school leap 21 slots or drop 20 slots in one year?” one professor asks in a typical entry. “Did faculty leave en masse from one to the other? Any ranking with that much volatility seems questionable.”
Caron says his own university does not publicize its place in the U.S. News rankings (57th for 2008) or whether it rises or falls. He is the publisher of Brian Leiter’s Law School Rankings, an alternative to the newsmagazine’s list.
“But the U.S. News rankings are the most famous and influential among potential students; there is no close second,” Caron says.
Caron has studied U.S. News rankings and written law review articles arguing that they lack transparency. He finds it frustrating that any school’s ranking depends so heavily on the most subjective of categories—reputation.
“There is no way of knowing whether the magazine has geographical diversity among the lawyers and judges it surveys or even how many bother to respond to the surveys,” Caron explains. (A U.S. News webpage says 71 percent of university deans and law school faculty and only 29 percent of lawyers and judges responded to the reputation survey in 2008. Morse said response rates are similar for all the magazine’s graduate school rankings.)
Caron offers this solution for law schools: Instead of castigating or ignoring the rankings, sit down with U.S. News researchers and hammer out improvements.
“Students want accountability—some way of knowing that the huge loans they are acquiring will help fund a good, solid education,” Caron says. “I believe in accountability. Right now, the rankings we have don’t measure accountability. But I think we can make them better and more precise.”
Caron believes a big splash of glasnost can help scour away the animosity between law schools and U.S. News. He wants the magazine’s website to post the details of its methodology, including the names or institutions that respond to the surveys. He believes the magazine owes readers, as well as students, more explanation of the difference between the 100th-ranked school and those one tier lower.
In any other ranking system, the difference would seem slight. But here, descending one slot is a soul-annihilating freefall into the lower pits.
“Sunshine is the best disinfectant,” Caron quips. “Most of the statistics the magazine uses are the same stats schools report to the American Bar Association. Reputation is the one category that is completely the responsibility of U.S. News.”
A numerical ranking is not subtle. It cannot illuminate the nuances of a school’s mission. Caron realizes schools below the 50th place find it harder to get the word out about achievements like impressive new faculty hires or even endowments. Yet, he points out, urban law schools like his often mesh better with student goals than a school ranked in the top 20.
“We have several high-powered law firms and Procter & Gamble headquartered here in Cincinnati and they recruit heavily from our school,” Caron says. “Not every young lawyer is happy in a huge, often impersonal Northeastern law firm. Yet there is valid data indicating the U.S. News rankings are skewed in favor of Northeastern universities partly because of the way reputation is evaluated.”
RANKER RESPONDS
Bob Morse has his own blog that invites comments and criticisms. He’s shown up uninvited to university symposiums dedicated to fighting the U.S. News rankings he created. He wants to hear what the critics have to say.
“We recently lowered the weight of at-grad employment and increased the weight of 9-month employment as a result of many career services offices speaking to us as a group,” Morse says. “We also added legal writing as a specialty as a result of faculty who teach in that field talking to us.”
He says efforts to manipulate the rankings are usually unsuccessful. Though the rankings can generate campuswide pain and obsession, Morse urges deans and faculty to keep them in perspective. He does not want schools wasting money or emotion on factors beyond their control.
“We think that there is no proof that doing U.S. News marketing helps,” Morse says. “In fact, studies have shown that U.S. News marketing does not work.”
When law schools refuse to file data, for instance, the magazine pulls the statistics they’ve filed in the annual ABA/LSAC Official Guide to ABA-Approved Law Schools. The guide is a collaboration between the Law School Admission Council and the ABA’s Section on Legal Education and Admissions to the Bar. It presents information about 194 ABA-accredited law schools gathered via an ABA questionnaire. Data is certified as fair and accurate by the law school dean, and the guide includes admission profile data and school descriptions collected by the council.
A disclaimer says the data is not intended for use in such rankings, and no form of law school rankings are sanctioned by either group.
But Morse says schools are supposed to submit the same data to U.S. News regarding library size and student-teacher ratios that they submit to the ABA. To complain that U.S. News does not fact-check the data sufficiently is to misconstrue what the rankings are. They are simply a guide, not a statistical analysis or a journalistic enterprise.
“The rankings are not social science,” Morse says. “The role of fact-checkers is for stories, not for data analysis and data collection.”
If the rankings team spots any statistical anomalies or big changes that don’t appear to make sense, Morse asks the law dean to verify the data and even explain the fluctuation. Each law school signs a data verification form.
And while Morse can empathize with complaints about how easily such data as placement figures can be fudged, he doesn’t feel it is appropriate for his researchers to define what constitutes statistics like employment.
“We are asking schools to report to U.S. News what they have reported on their ABA accreditation questionnaire [except for employment at graduation],” Morse says. “If the ABA and National Association for Law Placement think the definition should be narrower, we would use another one that they had agreed on. We are open to change.”
Rapoport says her own experience with Morse proves that he is flexible. When the Houston Law Center library lost 300,000 books to Tropical Storm Allison, Rapoport appealed to Morse, arguing it would be unfair to penalize the school on the basis of a natural disaster. He agreed to use the previous year’s figure.
It turned out, however, that she didn’t ask for enough. With administrative offices in shambles, the school’s job placement efforts had to be relocated.
“Our student career services office was the refreshment stand,” Rapoport says. “I didn’t ask Bob to give us a pass on the placement figures. I should have. That was my mistake. It cost us.”
Flexible or not, there is one change Morse cannot envision: releasing the names of the legal professionals answering the surveys pertaining to reputation.
“We have promised anonymity,” Morse says. “We also don’t know who actually returns a survey. We just know who it was sent to. We think giving out the names of those who rated schools or programs would hurt the quality of the results, not help them.”
Morse says an indiscreet emphasis on the rankings suggests a deep misunderstanding both of what they are and how they should be used.
“Any student or parent who uses the rankings as the No. 1 reason to go to a school, well, that’s exactly the wrong way to use them,” Morse says. “The rankings are a tool. They give deans and students useful statistics for comparisons. But there are a lot of factors that go into evaluating whether a school is the right fit.”
Morse says his own daughter graduated from New York University law school last year and was a member of the prestigious Order of the Coif honor society.
He says he does not want to speak for her, but he is certain she didn’t rely solely on the rankings when she made her choice. That she loved New York City and could find a fun, interesting job there while she was in law school were also key factors, he says.
LIFE AFTER RANKINGS
Avi Soifer, the law school dean at the University of Hawaii, can identify with that. He says his William S. Richardson School of Law doesn’t spend a cent on marketing materials to mail to anyone rating Hawaii’s reputation. Every year, his campus —on the palm-shaded, ocean-lapped island of Oahu—draws far more applicants than the law school can ever admit.
“We’ll always have a high selectivity score no matter what our ranking,” Soifer says, chuckling. “And our students are always in demand in Tokyo, San Francisco and Pacific Rim law firms. Many of our students have linguistic as well as cultural fluency.”
The law school does not highlight its rankings in materials aimed at prospective students or alumni. (It was tied for 91st in the last survey, for those who care). And Soifer cannot recall any faculty or student uproar over them.
“We don’t get every student we accept,” says Soifer. “Some choose to go elsewhere, maybe to schools with higher spots on the rankings.”
Because of his wife’s problematic health, Frank Wu has resigned from Wayne State. He plans to teach at Howard University School of Law and write a book.
Nancy Rapoport says she is much happier with her job and life in Las Vegas. She and her husband love the city’s blend of lavish color and pioneer spirit, and the beauty of the desert landscape. She confides she already had a job offer from the University of Nevada when she resigned as dean in Houston.
“Truthfully, the Houston job just wasn’t the right fit for my personality, and office politics had made my relationship with the faculty strained,” she says. “The rankings exacerbated tension that already existed.”
Her successor, acting dean Raymond Nimmer, is now using technology to reach prospective students and publicize his law school’s accomplishments. The school is developing an online social network similar to MySpace or Facebook for alumni. The site will be a window into the school’s culture for potential applicants.
Nimmer also hopes to find ways to use text messaging and satellite radio to raise the school’s profile.
In the past year, Nimmer says, the school has “reduced faculty-student ratio with some outstanding new hires.” The school’s Center for Children, Law & Policy; Criminal Justice Institute; Institute for Intellectual Property & Information Law and other centers are hosting topical conference events.
And when asked what the school’s U.S. News ranking was last year, he can’t remember and asks his assistant. The University of Houston had jumped 10 slots to 60th place.
“I guess I’ve been so busy with work,” Nimmer says, “the number just slipped my mind.”
Web extras:
ABAJournal.com: “Hour-Long Chat with Bob Morse”
ABAJournal.com: U.S. News Law School Rankings Leaked; NYU Down, Columbia Up
U.S. News & World Report’s 2009 law school rankings.
Morse Code: Inside the College Rankings blog by Bob Morse, director of data research for U.S. News & World Report.
