Tag Archives: science

Warning: relying on magic and anecdotes is bad for your health

In my post on evidence-based medicine, I said that “when valid scientific principles are not followed, no valid conclusion can be reached.”

Why not? Why don’t anecdotal claims (such as personal experience) count as evidence? Here is an example of what I mean:

A young woman who was a Christian Scientist told me that various relatives of hers have been healed by prayer. For example, her sister was revived from death by the prayer of her family. I have two comments about her story:

  • The observations that she told me about (that her sister was physiological dead, that her family prayed for her, and that she was revived) are probably true.
  • There was no dishonesty on her part: she sincerely believed her story

Despite accepting her observation, I disagreed with her conclusion about the causal relationship between prayer and health. How could I know that interpretation is wrong? For one, I was not there. Furthermore, after I expressed skepticism at this story, the young woman gave me many more examples, all from her direct personal experience of various friends and family being healed by prayer. On what basis could I reject them all without any personal experience on my own?

This was the essence of my reply:

I cannot object to the events you observed, as I was not there. But this does not mean that I must accept your causal explanation for those events. I have three reasons for this:

  • I have a certain understanding about the nature of the universe and of the means by which things happen. We call this cause and effect. In my experience, cause and effect happens according to certain rules, which we formally call the “laws of nature.” If someone presents an explanation that is inconsistent with my basic understanding of the laws of nature, they should have overwhelming evidence. Otherwise, I must conclude that their understanding of causality is wrong, even if their observations are true. This is especially true in observations regarding human health.
  • There are good reasons for scientists to reject personal experience and informal observations as sufficient basis for conclusions. There are many forms of cognitive bias can we can honestly make unless we follow strict rules to eliminate errors. In many fields (such as fixing a car) trial and error is good enough. There are no lives depending on a car working perfectly. In others, such as human physiology anecdotal evidence cannot lead to correct conclusions no matter how honest or smart you are. The forms of error take many forms: availability bias, post hoc ergo propter hoc, hasty generalization, placebo effects, selection bias, regression to the mean, bias by prior beliefs, social influence, etc. Even if you are aware of the biases, you cannot fully escape their effects; only try to structure your research to minimize them. In other words, when trial and error is not good enough, there is no substitute for proper science.
  • I categorize causal explanations into three kinds: true, false, and arbitrary. True explanations correspond with what I know. False explanations contradict what I know, but with further evidence may be proven true. Arbitrary explanations are neither true nor false because they do not refer to anything. They are “magical theories” because there is nothing that we can point to as the causal mechanism. How does prayer work? It just does – no mechanism is possible because it by definition exists outside of causality. Because arbitrary claims they cannot be proven or disproved, once identified as such, we can only dismiss them from consideration. We should be extremely skeptical of anyone who makes causal claims based on arbitrary/magical explanations. In the rare cause they point out a true causal connection, it is only by accident, and has no value to anyone as we have no more basis to believe that idea than any of their other claims.

If personal anecdotes are not acceptable as evidence, what is? A good theory:

  • is supported by many different kinds of observations
  • is consistent with existing knowledge
  • is possible to confirm by repeating the observations
  • has high predictive value: it should predict what will happen as well as what will not happen

“A theory is a good theory if it satisfies two requirements: It must accurately describe a large class of observations on the basis of a model that has only a few arbitrary elements, and it must make definite predictions about the results of future observations.” (Stephen Hawking)

Further reading:

 

Leave a Comment

Filed under General, Philosophy

On government-funded science

When tax dollars rather than private investment directs research, political ideology by scientific amateurs (politicians) determines which direction the research heads. The inevitable result is that political connections determines who gets research funds, while the unpopular and risky yet more ultimately world-changing prospects are ignored.

For example, AIDS kills very few Americans versus heart disease or cancer, yet gets significantly higher research funds than the two major killers. The majority of government research funds is directed toward better ways to kill (via the Defense Dept) and heal (via the NIH) people. Yet the government-funded research is good at neither, since breakthroughs consistently come from private search, which composes over 60% of science funding in the developed world.

What standard are politicians supposed to use to decide which scientific and medical projects show the most promise? Popularity is not a suitable standard, since popular scientists are the champions of the big discoveries of the past, not the future. Unfortunately, when your one’s own investment money is not at stake, the only remaining standard to guide research dollars is political pull, which is exactly what happens with government-funded science.

More:

Leave a Comment

Filed under General

The case for evidence-based medicine

Few people would openly admit that they prefer irrational treatments and doctors. But most people do in fact advocate irrational health practices – using pseudonyms for “irrational” as “holistic,” “alternative,” “homeopathic” and the deadly “natural.”

Medicine requires reason

The human body operates according to certain causal principles. If we wish to make a change in our health, we must understand some of those causal principles and act according to our understanding. To act without a rational basis is to disconnect our goals from their achievement. Irrationality does not guarantee failure — it just means that success, to the extent that it happens, will be due to other factors that our goals.

The study of human health is especially difficult

In the field of health, especially rigorous rationality is necessary for at least five reasons:

  1. The human body will solve, or at least try to solve most problems on its own. This makes establishing causality due external factors quite difficult and introduces biases such as the placebo effect and the regression fallacy.
  2. The body is very complex! Because it evolved over billions of years, the causal relationships in the body are extremely complex and interdependent.
  3. For example, even if we know that the body has too little of a certain substance, taking that substance may: a: not do anything b: cause the body to produce even less of the substance or c: cause an unpredictable side effect. On the other hand, if the body has too much of something, then the solution may be to a: consume less of that substance b: consume more of that substance or c: the consumption has no relationship at all to the level of that substance.
  4. It can be difficult to measure the extent to which medical problems are solved. While some things can be measured, many things, such as pain levels are very difficult to quantify.
  5. It is difficult to isolate causal factors in human beings since changes in health take time to develop and we can’t control every factor during an experiment or dissect human subjects when it is over.
  6. Humans tend to be irrational when it comes to their own mortality! We fear death, leading us to irrational over or under spending on health as well as being especially vulnerable to all the logical fallacies.

In medicine, rationality requires quality science research

There is a name for the field that applies rigor to the discovery of facts about nature: science. Science has been so successful in improving the state of human knowledge that many irrational, anti-scientific quacks have begun to use the term “scientific” to describe anti-scientific practices and ideas. In response to this, the medical community has come up with a term which identifiers the distinguishing aspect of rationality: “evidence based medicine.” This phrase is a necessary redundancy that identifies the essential characteristic of science: that it is based on sensory evidence. The alternative to non-evidence based science is not science at all, but emotionalism – “I feel it is true, so it must be.”

In the last hundred years, we have discovered certain practices for ensuring the conclusions of our medical experiments are valid. We know experimentally that observing these practices leads to more accurate conclusions. Let me emphasize that: the truth of medical claims is strongly correlated with the degree to which experiments follow accepted scientific standards. There are a number of objective scales for measuring the quality of an experiment.

Five characteristics of quality medical studies

Here is the essence of quality medical studies:

  1. The experiment and its results are fully described in enough detail to reproduce and compare the results
  2. There is a randomized control group
  3. The selection of control subjects is double blind
  4. The methods of randomization and blinding are accurately described and appropriate
  5. There is a description of withdrawals and dropouts.

How to judge health claims

Unfortunately, it is difficult to design good experiments and more difficult still to reach firm conclusions from most experiments. In the legitimate (“evidence-based”) medical community, the degree to which practitioners adhere to the principles varies greatly – but they at least try. Fortunately, in the “quack community,” while there is sometimes the pretense of evidence, basic scientific principles are so grossly violated and ignored that is becomes easy to distinguish fraud from legitimate science.

Here is an important point: it is difficult to make firm conclusions in medicine. But when valid scientific principles are not followed, it is easy to conclude that no valid conclusion can be reached. In other words, you can’t always be sure what’s good for you, but you can be sure when someone is talking nonsense.

Another important point: When someone makes irrational health claims, it does not mean that those claims are false. It just means those claims were not derived by rational (scientific) principles, and so we cannot say anything about their truth – we can only ignore them as arbitrary. It is as if someone claimed an invisible, undetectable pink unicorn in the sky – that which cannot be proven or disproved can only be dismissed.

How can we apply these ideas? It so happens that most health claims in the non-scientific media and many health “practitioners” are unscientific. This does not mean that they are wrong, or that people don’t feel helped by them. It means that their claims have no connection to reality.

 

In some cases, the practices that quacks suggest are helpful — but not for the reasons they identify. More importantly, in all cases following rational, scientific principles leads increases the likely hood of successful outcomes over quackery (aka emotionalism).

To conclude, to judge whether a medical claim is legitimate or arbitrary nonsense, check whether:

  1. It is based on quality experiments
  2. It is consistent with medical consensus (of evidence-based medicine)
  3. The certainty of the claim is well-established (by numerous studies, systematic reviews, etc

 

More:

3 Comments

Filed under Essays

Are philosophical claims scientifically provable?

This question makes the logical fallacy of the stolen concept.  The question of what is “scientifically provable” is derived from our metaphysics and epistemology.  We use our basic philosophy to derive the epistemological standard by which to investigate the specific aspects of reality (e.g. physics, chemistry, mathematics, and economics).  To demand that philosophical statements be scientifically validated is to demand that a derivative which depends on philosophy be used to prove philosophy.  This is like trying to build a house by assembling the roof, walls, and windows before the foundation.  It is fine to examine the whole structure of knowledge to verify that it is internal consistent and sound.  But we cannot use a higher-level deduction to prove the premise that it depends on.   The only way to validate philosophical claims is to use reason: to use logic to validate abstract ideas by reducing them to sensory evidence.

What is the difference between science and philosophy?

Science is distinguished from philosophy by subject matter: science studies the specific nature of the universe, and philosophy (of which religion is a primitive form) studies the fundamental and universal of the universe and man’s relationship to it.  Both are concerned with facts, but they differ in subject matter and the standard of evidence.  In the field of philosophy, we must be logically rigorous, but we cannot, and need not measure the physical evidence quantitatively as in the subject-specific sciences.

Science is made possible by the acceptance of certain philosophical axioms in metaphysics and epistemology. In metaphysics, science requires recognizing that all entities behave in a causal manner according to their nature. In epistemology, it recognizes that man is capable of perceiving and understanding reality by the use of his senses, and because his consciousness is fallible and not automatic, he needs to actively adhere to reason and logic to reach the right conclusions.  Science requires a systematic method to collect evidence and correctly interpret it because knowledge of how nature works is not self-evident.

Science is different in degree from informal empirical methods such as “trial and error” and in kind from non-empirical methods such as revelation, astrology, or emotionalism.   But the basic method – of rational investigation based on the evidence of reality must be used in all fields, whether philosophy, law, chemistry, mathematic, or cooking.

More:

The One Minute Case for Science.

1 Comment

Filed under Essays, Philosophy