As the world's top figure skaters converge in South Korea this week to jump, spin, and twizzle their way toward Olympic glory, the quality of their triple axels may not be the only factor that determines their success. At the highest levels of the sport, a BuzzFeed News investigation shows, scoring is often slanted in favor of judges' home countries.

In an exclusive analysis developed in consultation with three prominent statisticians, BuzzFeed News crunched more than a year’s worth of scoring data and found that the sport is rife with judges who give their own country’s skaters a boost. Collectively, judges from the sport’s powerhouse countries, such as the United States and Russia, consistently up-score their own skaters. Sixteen of the 48 judges for the Winter Olympics in South Korea show a pattern of home-country preference so strikingly consistent that the odds of it occurring by random chance are less than 1 in 100,000. These judges include all three sent by Russia, three from China, and two each from Canada and the United States.

Home-country preference has long been rumored, but BuzzFeed News’ statistical analysis for the first time provides overwhelming evidence that home-country preference plagues the sport. In close contests, the analysis showed, a single judge’s score could make a difference in where skaters stand in the final rankings.

Concern about judges up-scoring their own skaters is widespread within the sport, interviews with more than 20 current and former judges, coaches, and skaters revealed. “We see clear tendencies of national bias,” the Royal Dutch Skating Association declared last fall in a proposal to the International Skating Union, or ISU — the group that runs world figure skating — to more sharply define and combat home-country bias. The ISU is expected to consider the proposal at its annual congress in June.

National bias is “not checked well enough,” said Jeroen Prins, a Dutch judge who crafted the proposal. “I think many people share the concern because we need credibility in a judged sport.”

Higher home-country scores do not in and of themselves show a judge is deliberately trying to raise a compatriot’s standing; the scores could reflect a preference for a regional style of skating, for example, or an inclination toward skaters the judge has taken special note of, or even just patriotism. Judges might not even be aware that they consistently up-score their compatriots. And the vast majority of skating officials who spoke with BuzzFeed News said that the quality of judging has improved since the 2002 Winter Olympic Games in Salt Lake City, when a judging scandal prompted the ISU to take action.

But a major problem remains built into the system, current and former officials said: Each country’s skating program chooses its own judges. The US Figure Skating Association chooses the American judges who will go to the Olympics, the Russian association chooses the Russian judges, and so forth. This process, the officials said, creates incentives for bias: National federations want judges who will give their skaters the best scores. If judges “do something that’s not liked, then their country will just remove them and appoint a different judge,” said Tim Gerber, a former US national-level skater.

The ISU declined to comment on BuzzFeed News’ analysis or to answer detailed questions about exactly how the organization evaluates and disciplines judges. In a brief statement, the ISU said it “closely monitors the judging of all ISU Figure Skating Events and has a robust evaluation and reporting procedure in place.” It added, “Judges who make mistakes and/or are over marking skaters receive a warning and can be penalized by the ISU.” This season, the ISU asked officials to look for “bias” when evaluating judges, according to current officials.

One key element in the ISU’s procedure is an algorithm that flags judges whose scores fall outside what is known as the “corridor” — a buffer zone above or below the average of the other scores on their panel. But two former high-level ISU officials, whose duties included overseeing judges and deciding whether to sanction them, told BuzzFeed News that the algorithm was ineffective because the corridor was so wide that it caught only the most extreme outliers. To test this claim, BuzzFeed News recreated the ISU’s algorithm and found that it would very rarely flag any scores at all.

What’s more, the ISU’s system looks at only one performance at a time; as a result, it wouldn’t detect judges who consistently, across many performances, give their home-country skaters a modest boost.

Except in severe cases — such as in 2011, when an Italian judge was caught on video appearing to crib scores from his colleagues — the ISU does not publicize the warnings or sanctions it hands down. The ISU is not known to have sanctioned for bias any of the judges flagged by BuzzFeed News as showing a consistent pattern of marking skaters from their own countries above the average score of the other judges.

A half a dozen former top-level ISU officials who have served in the organization over decades acknowledged that managing judges’ home-country preference has always been a challenge. “If we don’t fix this thing,” said one former high-level ISU official, “it can get out of hand.”

“Everyone wants to see their country on the podium,” added a former member of a committee that monitors international judges. “When I was a judge, I used to place my skaters one place higher.”

For more than a decade, the ISU listed scores without identifying which judges awarded them. But that changed at the beginning of the 2016–17 season, when the ISU returned to the earlier practice of listing which judge gave each score. “The cancellation of anonymity,” the ISU said in its statement, “makes it possible to trace not only the quality of judging, but also any bias.”

We collected all of the scoring data from every major international competition, starting when that change was implemented and running through December 2017. We looked at the difference between a judge’s score and the average of the scores given by the other judges on that panel. (You can read more about our analysis here.)

Not every judge showed a pattern of preferring skaters from their home country. But the analysis showed that, across the sport, nationalistic preference is prevalent. Across more than 1,600 performances, figure skating judges on average gave their home-country skaters a 3.4-point advantage. That's a small percentage of the total score in a sport where, depending on the program, final scores in the last Olympics ran from the 40s to above 220. But our analysis found that even such small differences could alter the final results.

A home-country preference was found in every skating discipline: men’s, women’s, pairs, and ice dance. The phenomenon is particularly visible in countries that dominate the sport. China's judges on average gave Chinese skaters a 4.6-point boost, the largest of any country with at least 50 home-country judgments in the dataset BuzzFeed News analyzed. Italy, Russia, the United States, and Canada, all gave their skaters a boost of more than 3.4 points.

Our statistical analysis cannot determine how technically proficient a judge is, or what the correct score might be for any given performance. Instead, our algorithm reveals whether judges consistently score skaters from their home countries higher than the average of the other judges officiating the same performance.

We deliberately set a very high bar: To make it onto our list, individual judges had to show such a consistent and distinct pattern of up-marking their home-country skaters that the odds it occurred by random chance were less than 1 in 100,000. That means judges who scored their compatriots only a few times might not show up, even if they strongly favored their home-country skaters each one of those times.

Of course, statistics alone can’t explain why a judge gave their compatriots higher scores. According to interviews with current and former judges, many factors could be at play. Cultural differences might make a judge more inclined to favor styles of skating common in his or her region of the world. Judges often know skaters from their home countries, having watched them grow up in the sport, judged them frequently in national competitions, and become familiar with their routines — all factors that could lead a judge to boost their scores. Some judges, one former official said, know what their home-country skaters “ate for breakfast.”

And judging is hard, requiring officials to evaluate many different technical and artistic aspects of a performance in real time, without the benefit of slow-motion replays. Even where a judge sits can affect their view of the performance and thus their score. Judges who show a home-country preference might well be superb at evaluating the elements of a double lutz or a toe loop. Officials we interviewed said some of the 27 judges had a reputation for both technical acumen and fairness.

But more sinister forces could also be at work. At least three current and former ISU officials said judges sometimes collude to down-vote certain countries and up-vote others. Coaches and judges often lobby — sometimes subtly, sometimes blatantly — for skaters from their countries, these officials said. And in some cases, they said, scores for a particular skater are all but predetermined.

“Sometimes you’re on a panel where five or six people have agreed to what they’re going to do and what range they’re going to mark and how they’re going to go after a team,” explained another judge. In those cases, she said, someone who does not participate in that collusion, and whose scores therefore do not align with the others on the panel, “can look like they’re biased when in fact they are not.”

Our statistical analysis can’t determine why judges gave any particular score. But for whatever reason, among more than 200 judges in our analysis, 27 stood out. These judges consistently gave higher scores to their home-country skaters than the average of the scores given by the other judges. For each of these 27 judges, the odds that such a pattern was a fluke were less than 1 in 100,000.

These 27 judges span 10 different countries. Some are just a few years into their international judging careers, while others have been at it for decades.

Of these 27 judges, 16 have been selected for the Olympics in Pyeongchang. (See chart, above.)

BuzzFeed News reached out to all 27 judges for comment through their national federations, as well as to the individual judges themselves by phone, email, Facebook, or messages left at work. The US and Chinese skating federations declined to comment.

In an email, a spokesperson for Skate Canada told BuzzFeed News that “all of the Canadian judges submitted to the ISU for the Pyeongchang 2018 Olympic Winter Games meet the eligibility criteria, and were approved as judges by the ISU, according to its rules and regulations.” A representative of the Figure Skating Federation of Russia declined to answer questions and said, “None of our judges want to comment.”

Most of the judges either did not respond to messages or chose not to speak on the record. The only judge slated for the Pyeongchang Olympics who commented was Walter Toigo of Italy. Toigo was suspended for two years over an unrelated matter: He was caught apparently copying the marks of another judge. But in our analysis, he stood out for a different reason. He gave skaters from his home country a 7.5-point advantage on average, the largest of any official on our list. “Everybody has a different opinion and we judge what we can see,” Toigo told BuzzFeed News. “We are human beings, not machines. I judge what I think. I try to be consistent with all the skaters.”

Other than Toigo’s suspension (which he said he “did not agree with”), the ISU has not publicly sanctioned any of the 27 judges.

The ISU suffered a blow to its credibility in 2002, when a vote-trading scheme at the Salt Lake City Olympics became a public scandal. In response, the ISU overhauled its judging system.

To make scoring less subjective, the ISU put more emphasis on the technical elements of a program. Formerly, judges awarded competitors just two scores on a scale of 0 to 6.0; now, judges evaluate each jump, spin, or step sequence individually on a scale from –3 to +3, which is then adjusted for difficulty. They also grade five different artistic components of the program. A skater’s final score is a complex calculation based on the scale of difficulty of the program and an average of the judging panel’s marks. The highest and lowest scores for each aspect are tossed out to lessen the influence of outliers.

Since the 2002 scandal and the introduction of the new judging system, the sport has been largely cleaned up, said Charles Cyr, the ISU’s sports director for figure skating. “It’s a new breed of judges. Let me tell you they’re not wilted lilies,” he said. “The old guard of judges from 15 or 20 years ago have retired and gone,” he added, taking with them “the old adage of ‘I have to do what my country says.’”

But with all the fixes that the new scoring system promised, there also came less transparency. In the new scoring system, the ISU made the judges’ individual scores anonymous, on the theory that secrecy would shield them from pressure from their home country federations.

“Anonymous judging was the worst mistake ever made by the ISU,” said Sonia Bianchetti, an ISU hall of famer who used to serve as a judge and technical committee chair. The change made it almost impossible to see if judges were making errors or regularly scoring their own skaters higher, she said. It was a stark change from the 1970s, when she led the successful effort to suspend all Soviet judges for an entire season because they had demonstrated repeated national bias.

When the Russian skater Adelina Sotnikova beat out Korea’s Yuna Kim for the women’s gold medal at the Sochi Olympics in 2014, one of the Russian judges sparked an uproar when she was seen hugging Sotnikova backstage after the win. The Korean skating federation accused the Russian federation of an ethics violation for appointing that judge, who was married to the former head of the Russian skating federation. The ISU dismissed the complaint.

Two years later, at the 2016 ISU Congress in Dubrovnik, Croatia, skating federations voted almost unanimously to end anonymous judging. “I heard judge after judge going up to the microphone and say, ‘I want to be accountable for my marks,’” said John Coughlin, a 2012 US national champion who currently serves on one of the ISU’s technical committees. “The good judges have nothing to hide.”

The ISU has its own algorithm for catching scoring errors or potential bias — but some insiders say it is not effective.

“The corridor is so big that it’s almost impossible to get flagged,” said a former ISU technical committee member.

“The judges really know how do it — how to play within that band,” said another former ISU technical committee member.

“It’s quite a huge margin that the ISU allows,” said a former high-level member of a national federation.

To see if these claims were valid, BuzzFeed News recreated the ISU system that highlights scores far above or below the average — or outside the corridor, in skating parlance. Public ISU documents describe the system in detail, and BuzzFeed News translated that description into a computer algorithm. We consulted with Prins, the Dutch judge, and academic literature to verify our interpretation. We also shared a draft of our methodology with the ISU, which declined to confirm or correct it. We found that the ISU’s system would have flagged barely 1% of all scores for technical elements and an even smaller fraction of scores for artistic components.

Unlike our analysis, the ISU’s system only looks at one performance at a time, flagging any scores that fall far enough from the average. So if, during a particular competition, one judge gave a skater from her home country a much higher score than any of the other judges on the panel, the algorithm would detect that anomaly, and members of the technical committee would follow up to determine if the outlying score was the result of error or bias. But the algorithm does not track patterns across time. So it wouldn’t flag a judge who consistently gave skaters from her home country a less noticeable boost across many different performances or even many years.

A review of the ISU’s online disciplinary records shows that there have been no major punitive actions over the last decade explicitly related to judges scoring their home countries’ skaters higher. However, the ISU does not release information when a judge receives more minor sanctions — such as a “letter of criticism” or an “assessment.”

Cyr defended the ISU’s decision to keep these evaluations private, comparing them to a company’s internal disciplinary records of its employees, but other current officials said greater transparency would improve the sport. “If the ISU did make these people public, they’d get better,” said one former high-level ISU official.

After the 2014–15 season, the ISU stopped publishing even the total number of assessments it hands down.

Within the ISU, responsibility for evaluating judges falls mainly to two technical committees, each a panel of three international judges, an athlete, a coach, and a chairperson. These committees wield considerable power: Not only can they recommend sanctioning a judge, but they can also set new rules in the sport and determine the list of qualified judges for each season.

At every international competition, referees help oversee judges. Referees score each skating performance independently from the judges, and they can recommend that a technical committee give additional scrutiny to a judge.

High-level ISU events, such as the World Championships or the Olympics, include an extra layer of oversight for judges. A pair of observers, known as an Officials Assessment Commission, reviews any outlier scores flagged by the ISU’s algorithm. The technical committee can then recommend a letter of criticism or assessment to the judge. That recommendation must be approved by the sports director. Repeated assessments can lead to a suspension or a demotion.

Some officials said that the system for overseeing judges had enough checks and balances to protect the integrity of competitions. “The judges know that even though the event is done, their marks are going to be scrutinized,” the ISU’s Cyr said.

But others officials refuted that claim. In interviews, officials said they couldn’t recall a judge who the ISU had booted out for good. “They’re not policing it,” said a former skating federation official. “It’s all a bit farcical.”

Our analysis shows that just one judge can influence the final results. One example: At the men's competition at the Progressive Skate America in October 2016, Russian judge Maira Abasova scored her compatriot Sergei Voronov higher than any judge except for one. Abasova’s score helped boost Voronov into fourth place overall, just 0.20 points ahead of Boyang Jin, a Chinese skater, in the final standings. It’s impossible to know why Abasova gave the scores she did, but replacing her marks with the average of the other judges would have dropped Voronov into fifth — behind Jin.

Abasova could not be reached, but BuzzFeed News sent a letter with detailed questions to her through the Figure Skating Federation of Russia. A federation representative declined to comment or make the judge available for an interview but said, “Abasova is aware of the letter and has no comment.”

A former high-level ISU official said that it was common to push one’s own skaters, explaining, “If you don’t do it, if you don’t join in the game, then you get left behind.” ●