August 2021 – Roger Pielke Jr.

This post is a more detailed follow -up to my Substack post on the remarkable correction to Bermon and Garnier 2017 (BG17) published by BJSM yesterday. If you are not caught up, please have a read and then come back here.

This post is written in reponse to an apparent effort by World Athletics to rewrite history. Their chief spokesperson, Jackie Brock-Doyle has taken to Twitter in multiple posts (such as the below) claiming that BG17 was corrected in 2018 and not really that important in the female eligibility regulations. Both claims are false as I’ll show here.

World Athletics’ regulations were based on a substantive body of research carried out over a decade not on a single research paper that was corrected in 2018 and discussed at length during the CAS hearing in 2019 which resulted in the regulations being upheld.
— Jackie Brock-Doyle (@jbd01) August 18, 2021

Rewriting history?

The first claim — that BG17 was corrected in 2018 — is easy to show to be false. There was no correction published in 2018. There was a follow-up not-peer-reviewed letter hastily published re-doing some of the analysis based on errors that we found, but there was no admission that the findings were “potentially misleading” or overstated. That letter was also deeply flawed. We had called for BG17 to be retracted, and IAAF stated clearly to the New York Times in July 2018 that they stood behind the 2017 study and the 2018 letter:

“The I.A.A.F. will not be seeking a retraction of the 2017 study. The conclusions remain the same.”

It is hard to square the claim made now that a correction was issued in 2018 when in 2018 IAAF said that it stood behind its conclusions. The only correction that you can find at BJSM is the one published yesterday, and that doesn’t even scratch the surface on the issues with BG17.

The second claim — that BG17 was never really that important — is also false. BG17 was absolutely essential to IAAF pursuit of regulations in two important ways. First, it brought the IAAF regulations off of life support following the 2015 judgement of the Court of Arbitration for Sport in the Chand case. That case resulted in the regulations being suspended by CAS due to the inability of IAAF to provide evidence from actual competition showing a performance advantage by female with high testosterone.

CAS explained in the Chand judgment:

However, in order to justify excluding an individual from competing in a particular category on the basis of a naturally occurring characteristic such as endogenous testosterone, it is not enough simply to establish that the characteristic has some performance enhancing effect. Instead, the IAAF needs to establish that the characteristic in question confers such a significant performance advantage over other members of the category that allowing individuals with that characteristic to compete would subvert the very basis for having the separate category and thereby prevent a level playing field. The degree or magnitude of the advantage is therefore critical.
CAS 2015

CAS thus suspended the regulations and gave IAAF two years to come up with the data.

In these circumstances, the Panel is unable to uphold the validity of the Regulations. The Panel therefore suspends the Hyperandrogenism Regulations for a period of two years, subject to the following provisos. At any time during that two-year period, the IAAF may submit further written evidence to the CAS concerning the magnitude of the performance advantage that hyperandrogenic females enjoy over other females as a result of their abnormally high androgen levels.
CAS 2015

Flash forward two years. IAAF announced that they had the goods! They came up with evidence in the form of BG17. IAAF announced the good news in a press release.

The IAAF explained in that release:

The article published today, “Serum androgen levels and their relation to performance in track and field: mass spectrometry results from 2127 observations in male and female elite athletes”, is part of the evidence that the IAAF is preparing for its return to CAS.

It turns out that the paper represented the only evidence provided responsive to the CAS request for actual data from athletic performances. The other evidence referred to was the same evidence present in the Chand case deemed to be inadequate (e.g., such focused on the effects of testosterone on bodily characteristics).

Without BG17 it is likely that the IAAF female eligibility regulations would have been terminated by CAS in 2017 due to IAAF being non-responsive to the CAS request for evidence from actual performances. What BG17 did instead was to provide IAAF a lifeline to craft revised regulations, which CAS allowed.

When the revised regulations were released in 2018, they had this curious feature of regulating athletes only at the 400m to one mile distance. (Coincidentally, Caster Semenya’s distances, but I digress.) The basis given for the focus on a limited number of events was … BG17. This can be observed unambiguously in the regulations. In fact, even after the Semenya CAS judgment, the IAAF reliance on BG17 as the basis for the regulations remains clearly stated in the regulations, as shown below.

Further, if you go to the IAAF website right now, you will find that they refer to the analysis of BG17 as the basis for the restricted events, as shown below. Obviously, BG17 is not irrelevant to the regulations, it is central.

To sum, there is simply no doubt that BG17 offered a lifeline bridging the Chand CAS case to the subsequently revised 2018 regulations. Had CAS known in 2017 that the research provided to them by IAAF was “potentially misleading” and overstated, it is likely that CAS would have ended the effort to regulate right there.

Further, when IAAF came up with the revised regulations in 2018, they relied on BG17 as the sole empirical, peer-reviewed basis for the limited range of restricted events. And following the CAS judgment in Semenya, IAAF continued to rely on BG17 as the sole empirical, peer-reviewed basis for the regulations. At any point along the way, IAAF could have removed the references to BG17, if indeed it was so inconsequential or deemed to be in error. They did not.

Efforts to revise history will be difficult because the historical record is clear.

Bottom line: BG17 has been foundational to the IAAF — now World Athletics — regulations of female eligibility. Yesterday’s World Athletics admission that BG17 is “potentially misleading” and overstated is thus, as I have said, massive.