I love this substack. I have had all these thoughts over the past two years but could not articulate them as well. I have shared your posts with brainwashed people but alas, to no avail. Maybe because they’re midway sophists. They are also very pretentious, which seems to go hand in hand.
"A problem with anecdotes is that they are often just a few data points, and they are not the result of controlled experiments. This causes some people to discount them altogether, and relatedly, claim that RCTs (randomized controlled trials) are the “gold standard of evidence."
Funny thing is the same deep pocket corps who insist only RCT evidence has value also pay vast numbers of social media influencers because most ordinary individuals tend to put more weight on anecdotes related by a trusted individual.
Every playground has more useful information from other parents than official sources. For more than a decade doctors were surgically implanting tubes in the ears of kids with chronic ear infections and continued long after parents knew cutting dairy in the diets solved the problem.
Of course there are no sponsors for trials that make folks healthier w dietary balance.
I always think of the career of Judah Folkman, and how his theories on angiogenesis and cancer were so mocked by the scientific establishment until they turned out to be correct.
P-values be damned, indeed, agreed. I think of large standard errors as denoting imprecision, not inaccuracy. Especially in instances where other datasets show a similar beta effect.
The term they like to fling around when trying to claim their own superiority (appeal to their own authority) is Dunning-Kruger. It's amazing to have arguments with some of these people...
In one argument I got told:
1. Don't appeal to authority, then
2. You suffer from Dunning-Kruger effect (strange... I thought the appeal to authority would have voided that claim) then
3. Stop mansplaining because the female I was talking to apparently had so much more expertise than I could ever have thanks to all her hours of lab-work.... this was again quite strange since I didn't even know the person was female (she had a generic nickname on the forum) and it seemed to me to just be an appeal to authority which I was apparently not allowed to use...
When I pointed out that statistically the most likely outcome from the data provided was still a 40% RR irrespective of the p value, they of course just kept saying the results were statistically not significant and kept saying that this means there was no benefit...
Eventually when they could not explain away the fact that they were making absolute statements on massively underpowered RCTs without even being able to tell me the likelihood of a Type-II error and that the statistically most likely outcome was still a pretty big benefit I got blocked with a final Dunning Kruger good-bye.
2. it makes sense to me that something will be more likely correct after several adequately qualified* pairs of eyes have looked at something and not found errors. Bias in the negative direction should not be so bad if one doesn't just accept someone saying "uh thats so wrong" -> study garbage, but instead examines the explanation for what is wrong and how? Reviewers _not_ finding fault slips through more easily by a 3rd party who also just glances over, because of missing contrast, I guess.
* With something new, radical, or/and requiring a unique mix of interdisciplinary knowledge, perhaps difficult. Still can scan for basic errors -> better than no review?
Then of course, if a number of reviewers, coincidentally receiving research funds by a certain "Foundation", break loose a shitstorm threatening to quit unless papers with "wrong" conclusions get retracted, and it works... makes it seem questionable how well peer review currently works.
But something that _does_ seem to be important and have been introduced for good reasons, currently being compromised, does not imply: "oh it's not working, hence all not reviewed stuff is fine, no worries"?
I mean, our car safety testing standards of today were not present at all, not even in a badly working form, in the, say, 1940s. Would you not, still, prefer sitting in a car tested according to today's standards, even if yes, 1940 had working cars and not everyone died...?
That said, I found it somewhat amusing when I saw someone dismissing a study preprint referenced by R.Malone, stating "Malone? That guy who doesn't understand the difference between preprints and peer-reviewed?" because M. was indicating that thing probably had significance.
He was kinda missing the fact that for the subject areas of the study, Malone *is a* _peer_, and just publicly reviewed it. Lol. Just because he's not on the official review team of a particular journal... mere organizatorial fact (or worse).
3. An experienced doctor is not the collection of a mess of vague tales of different people often not even understanding the question properly or/and being subject to (auto-)suggestion. It is one mind, sorting out experiences with prior expert knowledge, and connecting it more in practise, so his abstract understanding becomes ever more intuitive and solidified. This process is guided by feedback, i.e. if not a total dummy, realizes what worked better in what circumstances and vice versa. I don't recognize the analogy there in "a bunch of people saying things".
The experienced doctor _tested_ his ideas, repeatedly, in substantial numbers and differently shaped cases. It's not as systematic as studies (depends on the doc I guess). But not anecdotes either (aka hearsay, in its worst form, with stages of indirection, with a lack of clarity and different ideas in different minds about what the question is) .
My idea of the use for anectoes was: show you where to do proper research next. No?
Not as its replacement, unless it's really urgent and they're the best you got.
4. "But doesn’t this fly in the face of “The vaccines save lives” trope? If those results were not a “red flag,” I don’t know what is."
Funny doctor that. So MORE people DYING in the treated group than placebo, for a product that is supposed to REDUCE DYING, isn't an important fact because the more-dying is not statistically significant? Maybe it's because "I'm not a doctor", but it would seem that conversely, the frickin product should demonstrate signficantly _fewer_ deaths in the treated group if it were worth any praise, huh? We're not dealing with a too small bad signal, we're dealing with the entire lack of a clear good one that should be expected, or what am I missing?
As for sophists and midwits aligning, well. There seem to be master sophists, who know exactly what they are doing, training the midwit ones how to say the right things. In some cases more, in others less formally.
One of the elephants in the room, on this topic, is that there's no credential that can ever represent Agency, by which I mean actively pitting yourself at the question, with all reasonable methods at your disposal.
Credentials are, at root, social approval. Letters of introduction, proof you've been "vetted" by your "alleged" betters in a field.
But Agency, itself, waxes and wanes, due to EVERY circumstance in life. We have all been "on point" or "in the zone" at times, and other times, we have all "run on empty" or "rested on our laurels".
It seems that the Credentialist Faithful fail to recognize that a credential is, at root, merely a social mechanism to screen applicants to the specific group to which the credential pertains, but it's not proof of present or future merit in that group or field.
Oppositely, and I admit, more perilously, a lack of a credentials does NOT necessirily mean someone is totally ignorant of a field, or doesn't understand enough to question a particular group or feild.
I love this substack. I have had all these thoughts over the past two years but could not articulate them as well. I have shared your posts with brainwashed people but alas, to no avail. Maybe because they’re midway sophists. They are also very pretentious, which seems to go hand in hand.
"A problem with anecdotes is that they are often just a few data points, and they are not the result of controlled experiments. This causes some people to discount them altogether, and relatedly, claim that RCTs (randomized controlled trials) are the “gold standard of evidence."
Funny thing is the same deep pocket corps who insist only RCT evidence has value also pay vast numbers of social media influencers because most ordinary individuals tend to put more weight on anecdotes related by a trusted individual.
Every playground has more useful information from other parents than official sources. For more than a decade doctors were surgically implanting tubes in the ears of kids with chronic ear infections and continued long after parents knew cutting dairy in the diets solved the problem.
Of course there are no sponsors for trials that make folks healthier w dietary balance.
"Ignorance is Strength" It is amazing how many people are willing to outsource their thinking and reasoning to "experts".
I always think of the career of Judah Folkman, and how his theories on angiogenesis and cancer were so mocked by the scientific establishment until they turned out to be correct.
Well put. Describes a multitude of Climate Bloggers with remarkable accuracy.
Thank you.
6. “The truth is somewhere in the middle”
I hear this one from people who are actually very intelligent; people who are neither sophist nor midwit. It’s an easy one to fall for.
-------
Made me think of this:
"My pledge to you is that I will do my best to try to balance out these more controversial viewpoints with other people’s perspectives.
So we can maybe find a better point of view."
--------
Joe Rogan's Instagram hostage statement apology of sorts and Spotify Hub plan to rectify having on Dr. Robert Malone
Great post!
P-values be damned, indeed, agreed. I think of large standard errors as denoting imprecision, not inaccuracy. Especially in instances where other datasets show a similar beta effect.
What's the derogatory name-calling term for folks who mock others with derogatory name calling terms?
Fritwits?
In any case stop adding to that same stream of perdition.
A better title for this article would be, "How to take apart medical disinformation bullshit."
Good article. Thank you.
The term they like to fling around when trying to claim their own superiority (appeal to their own authority) is Dunning-Kruger. It's amazing to have arguments with some of these people...
In one argument I got told:
1. Don't appeal to authority, then
2. You suffer from Dunning-Kruger effect (strange... I thought the appeal to authority would have voided that claim) then
3. Stop mansplaining because the female I was talking to apparently had so much more expertise than I could ever have thanks to all her hours of lab-work.... this was again quite strange since I didn't even know the person was female (she had a generic nickname on the forum) and it seemed to me to just be an appeal to authority which I was apparently not allowed to use...
When I pointed out that statistically the most likely outcome from the data provided was still a 40% RR irrespective of the p value, they of course just kept saying the results were statistically not significant and kept saying that this means there was no benefit...
Eventually when they could not explain away the fact that they were making absolute statements on massively underpowered RCTs without even being able to tell me the likelihood of a Type-II error and that the statistically most likely outcome was still a pretty big benefit I got blocked with a final Dunning Kruger good-bye.
When it comes to peer-review, the absolute best example showing how BS it is, is the Corman Drosten paper that was peer reviewed in what time frame?
Fun thread about it here:
https://www.goddek.com/review-process-of-the-corman-drosten-paper/
2. it makes sense to me that something will be more likely correct after several adequately qualified* pairs of eyes have looked at something and not found errors. Bias in the negative direction should not be so bad if one doesn't just accept someone saying "uh thats so wrong" -> study garbage, but instead examines the explanation for what is wrong and how? Reviewers _not_ finding fault slips through more easily by a 3rd party who also just glances over, because of missing contrast, I guess.
* With something new, radical, or/and requiring a unique mix of interdisciplinary knowledge, perhaps difficult. Still can scan for basic errors -> better than no review?
Then of course, if a number of reviewers, coincidentally receiving research funds by a certain "Foundation", break loose a shitstorm threatening to quit unless papers with "wrong" conclusions get retracted, and it works... makes it seem questionable how well peer review currently works.
But something that _does_ seem to be important and have been introduced for good reasons, currently being compromised, does not imply: "oh it's not working, hence all not reviewed stuff is fine, no worries"?
I mean, our car safety testing standards of today were not present at all, not even in a badly working form, in the, say, 1940s. Would you not, still, prefer sitting in a car tested according to today's standards, even if yes, 1940 had working cars and not everyone died...?
That said, I found it somewhat amusing when I saw someone dismissing a study preprint referenced by R.Malone, stating "Malone? That guy who doesn't understand the difference between preprints and peer-reviewed?" because M. was indicating that thing probably had significance.
He was kinda missing the fact that for the subject areas of the study, Malone *is a* _peer_, and just publicly reviewed it. Lol. Just because he's not on the official review team of a particular journal... mere organizatorial fact (or worse).
3. An experienced doctor is not the collection of a mess of vague tales of different people often not even understanding the question properly or/and being subject to (auto-)suggestion. It is one mind, sorting out experiences with prior expert knowledge, and connecting it more in practise, so his abstract understanding becomes ever more intuitive and solidified. This process is guided by feedback, i.e. if not a total dummy, realizes what worked better in what circumstances and vice versa. I don't recognize the analogy there in "a bunch of people saying things".
The experienced doctor _tested_ his ideas, repeatedly, in substantial numbers and differently shaped cases. It's not as systematic as studies (depends on the doc I guess). But not anecdotes either (aka hearsay, in its worst form, with stages of indirection, with a lack of clarity and different ideas in different minds about what the question is) .
My idea of the use for anectoes was: show you where to do proper research next. No?
Not as its replacement, unless it's really urgent and they're the best you got.
4. "But doesn’t this fly in the face of “The vaccines save lives” trope? If those results were not a “red flag,” I don’t know what is."
Funny doctor that. So MORE people DYING in the treated group than placebo, for a product that is supposed to REDUCE DYING, isn't an important fact because the more-dying is not statistically significant? Maybe it's because "I'm not a doctor", but it would seem that conversely, the frickin product should demonstrate signficantly _fewer_ deaths in the treated group if it were worth any praise, huh? We're not dealing with a too small bad signal, we're dealing with the entire lack of a clear good one that should be expected, or what am I missing?
As for sophists and midwits aligning, well. There seem to be master sophists, who know exactly what they are doing, training the midwit ones how to say the right things. In some cases more, in others less formally.
One of the elephants in the room, on this topic, is that there's no credential that can ever represent Agency, by which I mean actively pitting yourself at the question, with all reasonable methods at your disposal.
Credentials are, at root, social approval. Letters of introduction, proof you've been "vetted" by your "alleged" betters in a field.
But Agency, itself, waxes and wanes, due to EVERY circumstance in life. We have all been "on point" or "in the zone" at times, and other times, we have all "run on empty" or "rested on our laurels".
It seems that the Credentialist Faithful fail to recognize that a credential is, at root, merely a social mechanism to screen applicants to the specific group to which the credential pertains, but it's not proof of present or future merit in that group or field.
Oppositely, and I admit, more perilously, a lack of a credentials does NOT necessirily mean someone is totally ignorant of a field, or doesn't understand enough to question a particular group or feild.
Applied science has no such problem, in that its one and only focus is real world solutions. Sophists and midwits are quickly shown the door.