The subtitle for this should be "How Cass Weaponized Science".
Now that the Cass Review has published their final report, it is possible to look beyond the apparent issues in how the report was assembled, and examine the report, its recommendations, and see just how bad it really is.
The Cass Review
The Cass Review consists of several publications that need to be considered in their entirety in order to arrive at any kind of reasonable understanding of the effort and the conclusions stemming from that.
Publications Associated With The Cass Review:
Systematic Reviews Which Underpin The Final Report (There are 9 documents here)
Unsurprisingly, this is a lot of material to read. At the time of writing, I have actually read the full contents of the Review and its supporting evidence.
Critiquing The Cass Review
I could spend a pile of time going through the Cass Review and laying out my observations and thoughts about it. But that's a lot of material to analyze and distill into a reasonable form, and given my tendency to go deep, would take a very long time to wade through. That's the bad news - the good news is that we now have 3 papers which provide significant expert commentary on the problems with the Cass Review.
The Cass Review: Cis-supremacy in the UK's Approach to Healthcare for Trans Children
Biological and Psychosocial Evidence In the Cass Review - A Critical Commentary
Critically Appraising the Cass Report: Methodological Flaws and Unsupported Claims (pre-publication early release)
An Evidence Based Critique of the Cass Review (From Yale) [Added July 27, 2024]
These papers largely align with many of my own thoughts as I have read through the Cass Review and attempted to assess just how seriously it should be taken.
Broadly speaking, the first paper focuses primarily on the Interim Report and the terms of engagement that provide direction to the Cass Review itself. The second and third paper examine the final report and the supporting systematic reviews that are (theoretically) providing the basis on which the report's recommendations rest.
Issues of Language and Terms of Engagement
In Horton (2024), we get a deep dive into the language used in the framing of the Cass Review project. Horton asserts, and I largely agree, that the framework around the Cass Review itself is rooted in language which disrespects the experiences of those who are transgender. It takes a perspective that treats being transgender as "a problem to be solved", rather than as a valid human experience to be understood.
Further, as Horton (2024) points out, the perspective taken emphasizes a pathologization perspective that transgender people have fought against for decades, and the treatment community has been working hard to step away from for several decades now. By treating being transgender as "pathological", the Cass Review's perspective is fundamentally out of step with the current consensus approaches.
That framing sets the direction and perspective that the Cass Review team would take as they undertook their investigations. As we will see, this will lead the Cass Review to somewhat arbitrarily exclude some evidence, and include other evidence in ways that lead to questionable conclusions.
Issues of Evidence
For example, in Grijseels (2024), the Cass Review draws some major conclusions about comorbid (concurrent) mental health diagnoses:
This section leads to a number of key recommendations (30-42). Notably, recommendation 31 (Cass, Citation2024, p. 26) states that “[a]mong referrals there is a greater complexity of presentation with high levels of neurodiversity and/or co-occurring mental health issues and a higher prevalence than in the general population of adverse childhood experiences and looked after children. The increase in referrals and change in case-mix is also being seen internationally”. As demonstrated above, this claim is based on weak trends at best, with no clear support for increase in case-mix or psychiatric disorders among the population (Karvonen et al., Citation2022).
(Grijseels, 2024)
But the studies those conclusions are drawn from state something that should have given the Cass Review team pause:
They also explicitly say that these disorders were not reported prior to gender dysphoria, and as such no causality can be inferred. Neither study supports the claim made in the Cass Report that more than three-quarters were referred for psychiatric issues other than gender dysphoria, or that the majority of these were severe and preceded gender dysphoria onset.
(Grijseels, 2024)
Another interesting point that Grijseels (2024) brings up is related to the biology of sex and sex differentiation.
The Review aims to find a biological basis of gender dysphoria, starting this section by stating that “[b]iological sex is determined by sex chromosomes” (Cass, Citation2024, p. 98). This is an oversimplification, as in the scientific literature there is no clear consensus about the exact meaning of ‘biological sex’ (Velocci, Citation2024).
(Grijseels, 2024)
Again, the Cass Review makes recommendations with scant evidence, but those recommendations are long on implications:
In summary, although puberty and adolescence are key times for brain development, claims about brain maturation are not as black-and-white as the Review posits. Additionally, the question as to how brain maturation should advise legal policies is a far-reaching one, beyond the scope of both this commentary and arguably the Cass Review, and should be considered with extreme caution (Somerville, Citation2016; Steinberg, Citation2009a, Citation2009b). The Cass Review provides insufficient evidence to claim that brain maturation should be a reason to prohibit or restrict gender care for children and adolescents, especially in the face of these far-reaching legal consequences for such a claim.
(Grijseels, 2024)
All of this is to say that where the Cass Review engages with data, it often does so erroneously and draws conclusions that the data does not adequately support.
Issues of Methodology
Broadly speaking, methodology is a discussion of how a particular review or research effort is conducted. It is here that the Cass Review is the most problematic, because the approach to methodology led them to use tools that were either inappropriate or to apply those tools in an inappropriate manner. For this particular part of things, I will turn mostly to the paper by Noone and colleagues (2024, June 11).
The Cass Review makes myriad errors of method which end up excluding potentially important evidence. For example:
The exclusion of qualitative research from the overall set of systematic reviews is questionable, especially because qualitative studies were part of the inclusion criteria in the protocol and several of the research questions for the reviews have been investigated using rigorous qualitative methods 14–17 . Given that the Cass Report itself presents qualitative data (anecdotes, quotations, community claims) as evidence and purports to value stakeholder input, the fact that qualitative studies were not reviewed impedes the comprehensiveness of the overall project.
(Noone, et al., 2024, June 11)
When reviewing treatment guidelines, the Cass Review team used a tool called AGREE-II to assess the various clinical guidelines. Intriguingly, the Cass Review arrived at very different conclusions compared to other treatment guideline reviews that also used AGREE-II.
In the systematic review of clinical guidelines, which was not described in the pro- tocol, an appropriate study appraisal tool—the AGREE-II—was used. However, its reliability in this context may be questionable since several other systematic reviews have applied this tool to some of the same studies and arrived at quite different conclu- sions, with some also criticising the usefulness of AGREE-II for GAC guidelines.22–24
(Noone, et al., 2024, June 11)
Data quality assessment was performed using a modified Newcastle-Ottawa Quality Assessment Scale (NOS).
The NOS has been highly criticised 27 and the use of an adapted scoring negates any previous attempts to validate the NOS. In fact, one of the systematic reviews7 cites this critical paper by Stang and col- leagues (2010) 27 to support their use of the NOS—a practice its authors that have called out as a major quotation error. 28 The systematic review authors provide no rationale for the threshold scores used to categorise the quality of studies. It is generally accepted that the use of a single score (as used in these systematic reviews) is unacceptable in the assessment of risk of bias of individual studies within systematic reviews.29,30
(Noone, et al., 2024, June 11)
Further, the protocol laid out for the Cass Review included using the Mixed Methods Appraisal Tool instrument, and yet, the NOS was used instead.
It is notable that the combination of using the NOS instead of the MMAT, altering how it is scored, and then excluding evidence on the basis of this altered score only applied to the systematic re- views on what could be considered the three most controversial topics that the Cass Report addressed—puberty blockers, hormone therapy, and social transition. The fact that these decisions were deviations from the protocol and that justifications for them were not provided raises concerns about cherry-picking.
(Noone, et al., 2024, June 11)
At this point, I think it's really important to understand that the matters being discussed here may seem like "squabbles between academics", but they are in fact central to understanding the issues that undermine the credibility and validity of the Cass Review Final Report. The biases introduced by the manner in which these were being handled are serious enough to make conclusions drawn from them problematic at best.
Similarly, when we come to matters of established best practices when conducting reviews like Cass Review, domain expertise is important, and the Cass Review is decidedly short on relevant domain expertise.
Another deviation from best practice concerns the composition of the review team. Both the Cochrane Handbook11 and the Institute of Medicine41 recommend including content area experts on the review team. Initially, the Cass team specifically excluded content experts, but they later added Dr Trilby Langton who is ”a former Clinical Psy- chologist at the Tavistock Gender Identity Development Service”.9 Despite this, there is still a distinct lack of content expertise among the authors on many of the issues examined by the systematic reviews.
(Noone, et al., 2024, June 11)
This is a big deal, because it substantially undermines the credibility of the Cass Review. Expertise in a domain matters. You don't evaluate the safety of nuclear reactors without talking about it with experts. We should expect no less from a project like the Cass Review when its implications are so far reaching.
On the matter of the Cass Review's insistence on using Randomly Controlled Trials (RCTs), Noone and colleagues make the following point that summarizes the failings of the Cass Review nicely:
Finally, and perhaps more fundamentally, evaluating the efficacy of GAC based on psychosocial well-being is misguided. The primary goal of GAC is to prevent or induce the appearance of certain physical characteristics, and their physiological ef- ficacy is undisputed. Mental health benefits are a logical consequence of living au- thentically.57 Advocating for RCTs with mental health outcomes frames transness as a quasi-psychiatric condition, a distress whose suffering must be alleviated by the most evidenced-based methods, which contradicts the depathologisation of transness and its recognition as an issue of bodily autonomy and human rights.64,65 Improvement in well-being does not come from a physiological action of hormones, which could be adequately isolated by an RCT, but from a combination of factors contributing to increased congruence. Proposing RCTs with a mental health outcome thus shows, at best, a profound misunderstanding of transness.
(Noone, et al., 2024, June 11)
Conclusions
This is but a sampling of the problems in the Cass Review summarized in the works of Grijseels (2024), Horton (2024), and Noone and colleagues (2024, June 11). It should be enough to make any policy maker thinking of relying on the Cass Review Final Report take a moment to pause and consider the implications of their actions.
Using questionable, or downright bad, science to form public policy in an area which is going to affect people for their entire lives should never be done. While the study's proponents are going to swear up and down about its validity and the importance of its conclusions, the politically charged environment in which the Cass Review was commissioned, and the fact that domain experts are criticizing it for serious flaws, renders its objective usefulness virtually non-existent.
References
No comments:
Post a Comment