Lawyer ratings site not without objections

Avvo's plan is ambitious: to award a numeric score to every attorney in the United States, along with a profile, client recommendations and peer endorsements.

"It's the most critical piece of guidance that we provide," Mark Britton, a former vice president of Expedia who is Avvo's chief executive, said about the numeric score. "It's our assessment of how good a job that lawyer is going to do for you." Avvo says it has received $14 million in funding, including money from Benchmark Capital and Ignition Partners, co-founded by Microsoft alum and Avvo board member Brad Silverberg.

In tests, however, Avvo's pages seemed to be riddled with bizarre errors, profiles of attorneys who have been dead for more than a century and inexplicable scores in which some felons received better ratings than law school deans and internationally renowned litigators.

According to Avvo's profiles of "licensed attorneys," president Abraham Lincoln, once a lawyer who traveled on horseback between county courthouses, and Scopes defense attorney Clarence Darrow, who died in 1938, have no disciplinary sanctions pending and are encouraged to update their profiles by personalizing them with "professional experience" and achievements. Supreme Court Justices Ruth Bader Ginsburg and Samuel Alito each receive hardly flattering "experience" and "trustworthiness" ratings of three out of five stars.

U.S. Solicitor General Paul Clement, the magna cum laude graduate of Harvard Law School who has argued more than 25 cases before the Supreme Court, receives a mere 6.1 overall score out of 10. Barry Scheck, the famous member of O.J. Simpson's "dream team" (6.3), and Stanford Law professor Larry Lessig (6.3) don't fare much better.

But lawyers who have been convicted of serious crimes--including disbarred attorney Lynne Stewart, currently in prison for conspiracy to defraud the federal government--boast 6.5 ratings. Atlanta attorney Ulysses Ware, convicted of securities fraud (PDF) in April, gets a 6.3 rating and is listed as an "active member in good standing" of the Georgia bar.

Britton, an attorney who boasts a personal Avvo rating of 8, defends the results by saying, "You're talking about a very complex mathematical model...We take all the public records, and that provides the basis for the entire system. We layer on top of that the information from other publicly available sources that we find with our search technology."

Britton said the Avvo score was developed "with input from hundreds of lawyers, thousands of consumers and some of the best legal minds in the business," including Robert Hirshon, the past president of the American Bar Association. He also said not all attorney profiles have been updated with information gleaned from their Web sites, for instance, and scores may be reflected upward or downward as a result.

But he would not, however, specify why legal superstars and sitting Supreme Court justices receive poor ratings relative to lawyers who list no awards and only one published article in their bios (and garner a 9.8 of 10). "You can't handicap--in a golf sense--these people, or give them a certain level just because of who they are," Britton said. When asked about Justice Ginsburg's lackluster rating, he replied, "Arguably, her rating is a bit less efficient."

Avvo acknowledges that it does not currently collect criminal records but otherwise "cannot disclose the contents of the Avvo rating." Because CNET News.com agreed not to publish the story until Tuesday, we were not able to contact some of the attorneys who received poor scores for comment.

In response to a follow-up set of questions from News.com, Britton replied, "We cannot disclose the elements of the Avvo Rating; however, we take into account multiple factors that extend beyond those you have listed. In reviewing (the list of examples given), we find the rating system to be working as designed."

Modern society is rife with rating systems, of course. Movies are rated for violence and nudity. Google assigns a PageRank score to every Web site it indexes. Standard & Poor's provides ratings of the creditworthiness of bond issuers.

But rating attorneys is a far trickier and more subjective task: Should one additional published article boost a score? If so, by how much? Is Yale University's law school more prestigious than Stanford University's? Should a partner at a law firm be given credit for work largely done by unacknowledged junior associates? And what if a counselor's sage advice saves a company billions of dollars--and is never publicly acknowledged?

One way around that problem is to veer in a more democratic direction and rely primarily on customer-generated ratings, which sites like Yelp use for restaurant scores and Digg does, in a different way, for news articles.

Bar associations in many states, including California, offer online lawyer searches that can say whether an attorney has been disciplined. But they don't include third-party ratings.

LexisNexis Martindale-Hubbell's site, Lawyers.com, relies on the relatively safe procedure of collecting peer review rankings. Health Grades ranks physicians based on patient feedback, and DrScore.com claims to do the same.

Avvo's Britton thinks Lawyers.com and similar sites are not as useful. "We essentially turn the concept of a legal directory on its head," he said. "There is no established brand in legal. If you need a book online, there's Amazon. If you need travel, there's Expedia."

But neither Amazon nor Expedia started out by trying to build a massive database of scores of the professional accomplishments of hundreds of thousands of individuals. Some other excerpts from News.com's testing:

• You might expect attorneys that Law.com highlights at the top of its "40 under 40" list to receive top marks. But of the lawyers who had scores on Avvo, the average score was a disappointing 6.85.

• Superlawyers David Boies (once named by Time magazine as "Lawyer of the Year") received a 6.5, and Larry Sonsini (Silicon Valley's favorite lawyer, according to BusinessWeek) was rated just an 8.2. And then there's John J. Conroy, chairman of the executive committee of the largest U.S. law firm, Baker and McKenzie (6.5).

• Former Attorney General John Ashcroft gets a 5.3 overall rating, and former presidential candidate Bob Dole gets just a 5.6. Onetime Supreme Court nominee Harriet Miers is erroneously listed twice; one profile has a score of 6.1 and the other a 6.5.

• Bobby Keith Moser, a former Little Rock attorney, was sentenced in May 2005 to 15 years in prison for tax evasion. He receives an Avvo score of 5.8, which the company says is average. The site also says: "We have not found any disciplinary sanctions for this lawyer."