Chester Clinic

Lies, damned lies, and medical science - magazine - the atlantic

Lies, Damned Lies, and Medical Science - Magazine - The Atlantic Lies, Damned Lies, and Medical
Science
MUCH OF WHAT MEDICAL RESEARCHERS CONCLUDE IN THEIR STUDIES IS MISLEADING, EXAGGERATED, OR FLAT -
OUT WRONG. SO WHY ARE DOCTORS—TO A STRIKING EXTENT—STILL DRAWING UPON MISINFORMATION IN THEIR EVERYDAY PRACTICE ? DR. JOHN IOANNIDIS HAS SPENT HIS CAREER CHALLENGING HIS PEERS BY EXPOSING THEIR BAD SCIENCE.
By David H. Freedman IMAGE CREDIT: ROBYN TWOMEY/REDUX IN 2001, RUMORS were circulating in Greek hospitals that surgery residents, eager to rack up scalpeltime, were falsely diagnosing hapless Albanian immigrants with appendicitis. At the University ofIoannina medical school's teaching hospital, a newly minted doctor named Athina Tatsioni wasdiscussing the rumors with colleagues when a professor who had overheard asked her if she'd like totry to prove whether they were true—he seemed to be almost daring her. She accepted the challenge Lies, Damned Lies, and Medical Science - Magazine - The Atlantic and, with the professor's and other colleagues' help, eventually produced a formal study showing that,for whatever reason, the appendices removed from patients with Albanian names in six Greekhospitals were more than three times as likely to be perfectly healthy as those removed from patientswith Greek names. "It was hard to find a journal willing to publish it, but we did," recalls Tatsioni. "Ialso discovered that I really liked research." Good thing, because the study had actually been a sort ofaudition. The professor, it turned out, had been putting together a team of exceptionally brash andcurious young clinicians and Ph.D.s to join him in tackling an unusual and controversial agenda.
Last spring, I sat in on one of the team's weekly meetings on the medical school's campus, which isplunked crazily across a series of sharp hills. The building in which we met, like most at the school,had the look of a barracks and was festooned with political graffiti. But the group convened in aspacious conference room that would have been at home at a Silicon Valley start-up. Sprawledaround a large table were Tatsioni and eight other youngish Greek researchers and physicians who, incontrast to the pasty younger staff frequently seen in U.S. hospitals, looked like the casuallyglamorous cast of a television medical drama. The professor, a dapper and soft-spoken man namedJohn Ioannidis, loosely presided.
One of the researchers, a biostatistician named Georgia Salanti, fired up a laptop and projector andstarted to take the group through a study she and a few colleagues were completing that asked thisquestion: were drug companies manipulating published research to make their drugs look good?Salanti ticked off data that seemed to indicate they were, but the other team members almostimmediately started interrupting. One noted that Salanti's study didn't address the fact that drug-company research wasn't measuring critically important "hard" outcomes for patients, such assurvival versus death, and instead tended to measure "softer" outcomes, such as self-reportedsymptoms ("my chest doesn't hurt as much today"). Another pointed out that Salanti's study ignoredthe fact that when drug-company data seemed to show patients' health improving, the data oftenfailed to show that the drug was responsible, or that the improvement was more than marginal.
Salanti remained poised, as if the grilling were par for the course, and gamely acknowledged that thesuggestions were all good—but a single study can't prove everything, she said. Just as I was gettingthe sense that the data in drug studies were endlessly malleable, Ioannidis, who had mostly beenlistening, delivered what felt like a coup de grâce: wasn't it possible, he asked, that drug companieswere carefully selecting the topics of their studies—for example, comparing their new drugs againstthose already known to be inferior to others on the market—so that they were ahead of the game evenbefore the data juggling began? "Maybe sometimes it's the questions that are biased, not theanswers," he said, flashing a friendly smile. Everyone nodded. Though the results of drug studiesoften make newspaper headlines, you have to wonder whether they prove anything at all. Indeed,given the breadth of the potential problems raised at the meeting, can any medical-research studiesbe trusted? That question has been central to Ioannidis's career. He's what's known as a meta-researcher, andhe's become one of the world's foremost experts on the credibility of medical research. He and his Lies, Damned Lies, and Medical Science - Magazine - The Atlantic team have shown, again and again, and in many different ways, that much of what biomedicalresearchers conclude in published studies—conclusions that doctors keep in mind when theyprescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber orless meat, or when they recommend surgery for heart disease or back pain—is misleading,exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the publishedmedical information that doctors rely on is flawed. His work has been widely accepted by the medicalcommunity; it has been published in the field's top journals, where it is heavily cited; and he is a bigdraw at conferences. Given this exposure, and the fact that his work broadly targets everyone else'swork in medicine, as well as everything that physicians do and all the health advice we get, Ioannidismay be one of the most influential scientists alive. Yet for all his influence, he worries that the field ofmedical research is so pervasively flawed, and so riddled with conflicts of interest, that it might bechronically resistant to change—or even to publicly admitting that there's a problem.
THE CITY OF IOANNINA is a big college town a short drive from the ruins of a 20,000-seatamphitheater and a Zeusian sanctuary built at the site of the Dodona oracle. The oracle was said tohave issued pronouncements to priests through the rustling of a sacred oak tree. Today, a differentoak tree at the site provides visitors with a chance to try their own hands at extracting a prophecy. "Itake all the researchers who visit me here, and almost every single one of them asks the tree the samequestion," Ioannidis tells me, as we contemplate the tree the day after the team's meeting. "‘Will myresearch grant be approved?'" He chuckles, but Ioannidis (pronounced yo-NEE-dees) tends to laughnot so much in mirth as to soften the sting of his attack. And sure enough, he goes on to suggest thatan obsession with winning funding has gone a long way toward weakening the reliability of medicalresearch.
He first stumbled on the sorts of problems plaguing the field, he explains, as a young physician-researcher in the early 1990s at Harvard. At the time, he was interested in diagnosing rare diseases,for which a lack of case data can leave doctors with little to go on other than intuition and rules ofthumb. But he noticed that doctors seemed to proceed in much the same manner even when it cameto cancer, heart disease, and other common ailments. Where were the hard data that would back uptheir treatment decisions? There was plenty of published research, but much of it was remarkablyunscientific, based largely on observations of a small number of cases. A new "evidence-basedmedicine" movement was just starting to gather force, and Ioannidis decided to throw himself into it,working first with prominent researchers at Tufts University and then taking positions at JohnsHopkins University and the National Institutes of Health. He was unusually well armed: he had beena math prodigy of near-celebrity status in high school in Greece, and had followed his parents, whowere both physician-researchers, into medicine. Now he'd have a chance to combine math andmedicine by applying rigorous statistical analysis to what seemed a surprisingly sloppy field. "Iassumed that everything we physicians did was basically right, but now I was going to help verify it,"he says. "All we'd have to do was systematically review the evidence, trust what it told us, and theneverything would be perfect." Lies, Damned Lies, and Medical Science - Magazine - The Atlantic It didn't turn out that way. In poring over medical journals, he was struck by how many findings of alltypes were refuted by later findings. Of course, medical-science "never minds" are hardly secret. Andthey sometimes make headlines, as when in recent years large studies or growing consensuses ofresearchers concluded that mammograms, colonoscopies, and PSA tests are far less useful cancer-detection tools than we had been told; or when widely prescribed antidepressants such as Prozac,Zoloft, and Paxil were revealed to be no more effective than a placebo for most cases of depression; orwhen we learned that staying out of the sun entirely can actually increase cancer risks; or when wewere told that the advice to drink lots of water during intense exercise was potentially fatal; or when,last April, we were informed that taking fish oil, exercising, and doing puzzles doesn't really help fendoff Alzheimer's disease, as long claimed. Peer-reviewed studies have come to opposite conclusions onwhether using cell phones can cause brain cancer, whether sleeping more than eight hours a night ishealthful or dangerous, whether taking aspirin every day is more likely to save your life or cut itshort, and whether routine angioplasty works better than pills to unclog heart arteries.
But beyond the headlines, Ioannidis was shocked at the range and reach of the reversals he was seeingin everyday medical research. "Randomized controlled trials," which compare how one groupresponds to a treatment against how an identical group fares without the treatment, had long beenconsidered nearly unshakable evidence, but they, too, ended up being wrong some of the time. "Irealized even our gold-standard research had a lot of problems," he says. Baffled, he started lookingfor the specific ways in which studies were going wrong. And before long he discovered that the rangeof errors being committed was astonishing: from what questions researchers posed, to how they setup the studies, to which patients they recruited for the studies, to which measurements they took, tohow they analyzed the data, to how they presented their results, to how particular studies came to bepublished in medical journals.
This array suggested a bigger, underlying dysfunction, and Ioannidis thought he knew what it was.
"The studies were biased," he says. "Sometimes they were overtly biased. Sometimes it was difficult tosee the bias, but it was there." Researchers headed into their studies wanting certain results—and, loand behold, they were getting them. We think of the scientific process as being objective, rigorous,and even ruthless in separating out what is true from what we merely wish to be true, but in fact it'seasy to manipulate results, even unintentionally or unconsciously. "At every step in the process, thereis room to distort results, a way to make a stronger claim or to select what is going to be concluded,"says Ioannidis. "There is an intellectual conflict of interest that pressures researchers to find whateverit is that is most likely to get them funded." Perhaps only a minority of researchers were succumbing to this bias, but their distorted findings werehaving an outsize effect on published research. To get funding and tenured positions, and oftenmerely to stay afloat, researchers have to get their work published in well-regarded journals, whererejection rates can climb above 90 percent. Not surprisingly, the studies that tend to make the gradeare those with eye-catching findings. But while coming up with eye-catching theories is relativelyeasy, getting reality to bear them out is another matter. The great majority collapse under the weight Lies, Damned Lies, and Medical Science - Magazine - The Atlantic of contradictory data when studied rigorously. Imagine, though, that five different research teams testan interesting theory that's making the rounds, and four of the groups correctly prove the idea false,while the one less cautious group incorrectly "proves" it true through some combination of error,fluke, and clever selection of data. Guess whose findings your doctor ends up reading about in thejournal, and you end up hearing about on the evening news? Researchers can sometimes winattention by refuting a prominent finding, which can help to at least raise doubts about results, but ingeneral it is far more rewarding to add a new insight or exciting-sounding twist to existing researchthan to retest its basic premises—after all, simply re-proving someone else's results is unlikely to getyou published, and attempting to undermine the work of respected colleagues can have uglyprofessional repercussions.
In the late 1990s, Ioannidis set up a base at the University of Ioannina. He pulled together his team,which remains largely intact today, and started chipping away at the problem in a series of papersthat pointed out specific ways certain studies were getting misleading results. Other meta-researcherswere also starting to spotlight disturbingly high rates of error in the medical literature. But Ioannidiswanted to get the big picture across, and to do so with solid data, clear reasoning, and good statisticalanalysis. The project dragged on, until finally he retreated to the tiny island of Sikinos in the AegeanSea, where he drew inspiration from the relatively primitive surroundings and the intellectualtraditions they recalled. "A pervasive theme of ancient Greek literature is that you need to pursue thetruth, no matter what the truth might be," he says. In 2005, he unleashed two papers that challengedthe foundations of medical research.
He chose to publish one paper, fittingly, in the online journal PLoS Medicine, which is committed torunning any methodologically sound article without regard to how "interesting" the results may be. Inthe paper, Ioannidis laid out a detailed mathematical proof that, assuming modest levels of researcherbias, typically imperfect research techniques, and the well-known tendency to focus on exciting ratherthan highly plausible theories, researchers will come up with wrong findings most of the time. Simplyput, if you're attracted to ideas that have a good chance of being wrong, and if you're motivated toprove them right, and if you have a little wiggle room in how you assemble the evidence, you'llprobably succeed in proving wrong theories right. His model predicted, in different fields of medicalresearch, rates of wrongness roughly corresponding to the observed rates at which findings were laterconvincingly refuted: 80 percent of non-randomized studies (by far the most common type) turn outto be wrong, as do 25 percent of supposedly gold-standard randomized trials, and as much as 10percent of the platinum-standard large randomized trials. The article spelled out his belief thatresearchers were frequently manipulating data analyses, chasing career-advancing findings ratherthan good science, and even using the peer-review process—in which journals ask researchers to helpdecide which studies to publish—to suppress opposing views. "You can question some of the details ofJohn's calculations, but it's hard to argue that the essential ideas aren't absolutely correct," says DougAltman, an Oxford University researcher who directs the Centre for Statistics in Medicine.
Still, Ioannidis anticipated that the community might shrug off his findings: sure, a lot of dubious Lies, Damned Lies, and Medical Science - Magazine - The Atlantic research makes it into journals, but we researchers and physicians know to ignore it and focus on thegood stuff, so what's the big deal? The other paper headed off that claim. He zoomed in on 49 of themost highly regarded research findings in medicine over the previous 13 years, as judged by thescience community's two standard measures: the papers had appeared in the journals most widelycited in research articles, and the 49 articles themselves were the most widely cited articles in thesejournals. These were articles that helped lead to the widespread popularity of treatments such as theuse of hormone-replacement therapy for menopausal women, vitamin E to reduce the risk of heartdisease, coronary stents to ward off heart attacks, and daily low-dose aspirin to control bloodpressure and prevent heart attacks and strokes. Ioannidis was putting his contentions to the test notagainst run-of-the-mill research, or even merely well-accepted research, but against the absolute tipof the research pyramid. Of the 49 articles, 45 claimed to have uncovered effective interventions.
Thirty-four of these claims had been retested, and 14 of these, or 41 percent, had been convincinglyshown to be wrong or significantly exaggerated. If between a third and a half of the most acclaimedresearch in medicine was proving untrustworthy, the scope and impact of the problem wereundeniable. That article was published in the Journal of the American Medical Association.
DRIVING ME BACK to campus in his smallish SUV—after insisting, as he apparently does with allhis visitors, on showing me a nearby lake and the six monasteries situated on an islet within it—Ioannidis apologized profusely for running a yellow light, explaining with a laugh that he didn't trustthe truck behind him to stop. Considering his willingness, even eagerness, to slap the face of themedical-research community, Ioannidis comes off as thoughtful, upbeat, and deeply civil. He's acareful listener, and his frequent grin and semi-apologetic chuckle can make the sharp prodding of hisarguments seem almost good-natured. He is as quick, if not quicker, to question his own motives andcompetence as anyone else's. A neat and compact 45-year-old with a trim mustache, he presents as asort of dashing nerd—Giancarlo Giannini with a bit of Mr. Bean.
The humility and graciousness seem to serve him well in getting across a message that is not easy todigest or, for that matter, believe: that even highly regarded researchers at prestigious institutionssometimes churn out attention-grabbing findings rather than findings likely to be right. But Ioannidispoints out that obviously questionable findings cram the pages of top medical journals, not tomention the morning headlines. Consider, he says, the endless stream of results from nutritionalstudies in which researchers follow thousands of people for some number of years, tracking what theyeat and what supplements they take, and how their health changes over the course of the study. "Thenthe researchers start asking, ‘What did vitamin E do? What did vitamin C or D or A do? Whatchanged with calorie intake, or protein or fat intake? What happened to cholesterol levels? Who gotwhat type of cancer?'" he says. "They run everything through the mill, one at a time, and they startfinding associations, and eventually conclude that vitamin X lowers the risk of cancer Y, or this foodhelps with the risk of that disease." In a single week this fall, Google's news page offered theseheadlines: "More Omega-3 Fats Didn't Aid Heart Patients"; "Fruits, Vegetables Cut Cancer Risk forSmokers"; "Soy May Ease Sleep Problems in Older Women"; and dozens of similar stories.
Lies, Damned Lies, and Medical Science - Magazine - The Atlantic When a five-year study of 10,000 people finds that those who take more vitamin X are less likely toget cancer Y, you'd think you have pretty good reason to take more vitamin X, and physiciansroutinely pass these recommendations on to patients. But these studies often sharply conflict withone another. Studies have gone back and forth on the cancer-preventing powers of vitamins A, D, andE; on the heart-health benefits of eating fat and carbs; and even on the question of whether beingoverweight is more likely to extend or shorten your life. How should we choose among these dueling,high-profile nutritional findings? Ioannidis suggests a simple approach: ignore them all.
For starters, he explains, the odds are that in any large database of many nutritional and healthfactors, there will be a few apparent connections that are in fact merely flukes, not real health effects—it's a bit like combing through long, random strings of letters and claiming there's an importantmessage in any words that happen to turn up. But even if a study managed to highlight a genuinehealth connection to some nutrient, you're unlikely to benefit much from taking more of it, becausewe consume thousands of nutrients that act together as a sort of network, and changing intake of justone of them is bound to cause ripples throughout the network that are far too complex for thesestudies to detect, and that may be as likely to harm you as help you. Even if changing that one factordoes bring on the claimed improvement, there's still a good chance that it won't do you much good inthe long run, because these studies rarely go on long enough to track the decades-long course ofdisease and ultimately death. Instead, they track easily measurable health "markers" such ascholesterol levels, blood pressure, and blood-sugar levels, and meta-experts have shown that changesin these markers often don't correlate as well with long-term health as we have been led to believe.
On the relatively rare occasions when a study does go on long enough to track mortality, the findingsfrequently upend those of the shorter studies. (For example, though the vast majority of studies ofoverweight individuals link excess weight to ill health, the longest of them haven't convincingly shownthat overweight people are likely to die sooner, and a few of them have seemingly demonstrated thatmoderately overweight people are likely to live longer.) And these problems are aside from ubiquitousmeasurement errors (for example, people habitually misreport their diets in studies), routinemisanalysis (researchers rely on complex software capable of juggling results in ways they don'talways understand), and the less common, but serious, problem of outright fraud (which has beenrevealed, in confidential surveys, to be much more widespread than scientists like to acknowledge).
If a study somehow avoids every one of these problems and finds a real connection to long-termchanges in health, you're still not guaranteed to benefit, because studies report average results thattypically represent a vast range of individual outcomes. Should you be among the lucky minority thatstands to benefit, don't expect a noticeable improvement in your health, because studies usuallydetect only modest effects that merely tend to whittle your chances of succumbing to a particulardisease from small to somewhat smaller. "The odds that anything useful will survive from any of thesestudies are poor," says Ioannidis—dismissing in a breath a good chunk of the research into which wesink about $100 billion a year in the United States alone.
And so it goes for all medical studies, he says. Indeed, nutritional studies aren't the worst. Drug Lies, Damned Lies, and Medical Science - Magazine - The Atlantic studies have the added corruptive force of financial conflict of interest. The exciting links betweengenes and various diseases and traits that are relentlessly hyped in the press for heralding miraculousaround-the-corner treatments for everything from colon cancer to schizophrenia have in the pastproved so vulnerable to error and distortion, Ioannidis has found, that in some cases you'd have doneabout as well by throwing darts at a chart of the genome. (These studies seem to have improvedsomewhat in recent years, but whether they will hold up or be useful in treatment are still openquestions.) Vioxx, Zelnorm, and Baycol were among the widely prescribed drugs found to be safe andeffective in large randomized controlled trials before the drugs were yanked from the market asunsafe or not so effective, or both.
"Often the claims made by studies are so extravagant that you can immediately cross them outwithout needing to know much about the specific problems with the studies," Ioannidis says. But ofcourse it's that very extravagance of claim (one large randomized controlled trial even proved thatsecret prayer by unknown parties can save the lives of heart-surgery patients, while another provedthat secret prayer can harm them) that helps gets these findings into journals and then into ourtreatments and lifestyles, especially when the claim builds on impressive-sounding evidence. "Evenwhen the evidence shows that a particular research idea is wrong, if you have thousands of scientistswho have invested their careers in it, they'll continue to publish papers on it," he says. "It's like anepidemic, in the sense that they're infected with these wrong ideas, and they're spreading it to otherresearchers through journals." THOUGH SCIENTISTS AND science journalists are constantly talking up the value of the peer-review process, researchers admit among themselves that biased, erroneous, and even blatantlyfraudulent studies easily slip through it. Nature, the grande dame of science journals, stated in a2006 editorial, "Scientists understand that peer review per se provides only a minimal assurance ofquality, and that the public conception of peer review as a stamp of authentication is far from thetruth." What's more, the peer-review process often pressures researchers to shy away from strikingout in genuinely new directions, and instead to build on the findings of their colleagues (that is, theirpotential reviewers) in ways that only seem like breakthroughs—as with the exciting-sounding genelinkages (autism genes identified!) and nutritional findings (olive oil lowers blood pressure!) that arereally just dubious and conflicting variations on a theme.
Most journal editors don't even claim to protect against the problems that plague these studies.
University and government research overseers rarely step in to directly enforce research quality, andwhen they do, the science community goes ballistic over the outside interference. The ultimateprotection against research error and bias is supposed to come from the way scientists constantlyretest each other's results—except they don't. Only the most prominent findings are likely to be put tothe test, because there's likely to be publication payoff in firming up the proof, or contradicting it.
But even for medicine's most influential studies, the evidence sometimes remains surprisinglynarrow. Of those 45 super-cited studies that Ioannidis focused on, 11 had never been retested.
Perhaps worse, Ioannidis found that even when a research error is outed, it typically persists for years Lies, Damned Lies, and Medical Science - Magazine - The Atlantic or even decades. He looked at three prominent health studies from the 1980s and 1990s that wereeach later soundly refuted, and discovered that researchers continued to cite the original results ascorrect more often than as flawed—in one case for at least 12 years after the results were discredited.
Doctors may notice that their patients don't seem to fare as well with certain treatments as theliterature would lead them to expect, but the field is appropriately conditioned to subjugate suchanecdotal evidence to study findings. Yet much, perhaps even most, of what doctors do has neverbeen formally put to the test in credible studies, given that the need to do so became obvious to thefield only in the 1990s, leaving it playing catch-up with a century or more of non-evidence-basedmedicine, and contributing to Ioannidis's shockingly high estimate of the degree to which medicalknowledge is flawed. That we're not routinely made seriously ill by this shortfall, he argues, is duelargely to the fact that most medical interventions and advice don't address life-and-death situations,but rather aim to leave us marginally healthier or less unhealthy, so we usually neither gain nor riskall that much.
Medical research is not especially plagued with wrongness. Other meta-research experts haveconfirmed that similar issues distort research in all fields of science, from physics to economics(where the highly regarded economists J. Bradford DeLong and Kevin Lang once showed how aremarkably consistent paucity of strong evidence in published economics studies made it unlikelythat any of them were right). And needless to say, things only get worse when it comes to the popexpertise that endlessly spews at us from diet, relationship, investment, and parenting gurus andpundits. But we expect more of scientists, and especially of medical scientists, given that we believewe are staking our lives on their results. The public hardly recognizes how bad a bet this is. Themedical community itself might still be largely oblivious to the scope of the problem, if Ioannidishadn't forced a confrontation when he published his studies in 2005.
Ioannidis initially thought the community might come out fighting. Instead, it seemed relieved, as if ithad been guiltily waiting for someone to blow the whistle, and eager to hear more. David Gorski, asurgeon and researcher at Detroit's Barbara Ann Karmanos Cancer Institute, noted in his prominentmedical blog that when he presented Ioannidis's paper on highly cited research at a professionalmeeting, "not a single one of my surgical colleagues was the least bit surprised or disturbed by itsfindings." Ioannidis offers a theory for the relatively calm reception. "I think that people didn't feel Iwas only trying to provoke them, because I showed that it was a community problem, instead ofpointing fingers at individual examples of bad research," he says. In a sense, he gave scientists anopportunity to cluck about the wrongness without having to acknowledge that they themselvessuccumb to it—it was something everyone else did.
To say that Ioannidis's work has been embraced would be an understatement. His PLoS Medicinepaper is the most downloaded in the journal's history, and it's not even Ioannidis's most-cited work—that would be a paper he published in Nature Genetics on the problems with gene-link studies. Otherresearchers are eager to work with him: he has published papers with 1,328 different co-authors at538 institutions in 43 countries, he says. Last year he received, by his estimate, invitations to speak at Lies, Damned Lies, and Medical Science - Magazine - The Atlantic 1,000 conferences and institutions around the world, and he was accepting an average of about fiveinvitations a month until a case last year of excessive-travel-induced vertigo led him to cut back.
Even so, in the weeks before I visited him he had addressed an AIDS conference in San Francisco, theEuropean Society for Clinical Investigation, Harvard's School of Public Health, and the medicalschools at Stanford and Tufts.
The irony of his having achieved this sort of success by accusing the medical-research community ofchasing after success is not lost on him, and he notes that it ought to raise the question of whether hehimself might be pumping up his findings. "If I did a study and the results showed that in fact therewasn't really much bias in research, would I be willing to publish it?" he asks. "That would create areal psychological conflict for me." But his bigger worry, he says, is that while his fellow researchersseem to be getting the message, he hasn't necessarily forced anyone to do a better job. He fears hewon't in the end have done much to improve anyone's health. "There may not be fierce objections towhat I'm saying," he explains. "But it's difficult to change the way that everyday doctors, patients, andhealthy people think and behave." AS HELTER-SKELTER as the University of Ioannina Medical School campus looks, the hospitalabutting it looks reassuringly stolid. Athina Tatsioni has offered to take me on a tour of the facility,but we make it only as far as the entrance when she is greeted—accosted, really—by a worried-lookingolder woman. Tatsioni, normally a bit reserved, is warm and animated with the woman, and the twohave a brief but intense conversation before embracing and saying goodbye. Tatsioni explains to methat the woman and her husband were patients of hers years ago; now the husband has beenadmitted to the hospital with abdominal pains, and Tatsioni has promised she'll stop by his room laterto say hello. Recalling the appendicitis story, I prod a bit, and she confesses she plans to do her ownexam. She needs to be circumspect, though, so she won't appear to be second-guessing the otherdoctors.
Tatsioni doesn't so much fear that someone will carve out the man's healthy appendix. Rather, she'sconcerned that, like many patients, he'll end up with prescriptions for multiple drugs that will dolittle to help him, and may well harm him. "Usually what happens is that the doctor will ask for asuite of biochemical tests—liver fat, pancreas function, and so on," she tells me. "The tests could turnup something, but they're probably irrelevant. Just having a good talk with the patient and getting aclose history is much more likely to tell me what's wrong." Of course, the doctors have all beentrained to order these tests, she notes, and doing so is a lot quicker than a long bedside chat. They'realso trained to ply the patient with whatever drugs might help whack any errant test numbers backinto line. What they're not trained to do is to go back and look at the research papers that helpedmake these drugs the standard of care. "When you look the papers up, you often find the drugs didn'teven work better than a placebo. And no one tested how they worked in combination with the otherdrugs," she says. "Just taking the patient off everything can improve their health right away." But notonly is checking out the research another time-consuming task, patients often don't even like it whenthey're taken off their drugs, she explains; they find their prescriptions reassuring.
Lies, Damned Lies, and Medical Science - Magazine - The Atlantic Later, Ioannidis tells me he makes a point of having several clinicians on his team. "Researchers andphysicians often don't understand each other; they speak different languages," he says. Knowing thatsome of his researchers are spending more than half their time seeing patients makes him feel theteam is better positioned to bridge that gap; their experience informs the team's research withfirsthand knowledge, and helps the team shape its papers in a way more likely to hit home withphysicians. It's not that he envisions doctors making all their decisions based solely on solid evidence—there's simply too much complexity in patient treatment to pin down every situation with a greatstudy. "Doctors need to rely on instinct and judgment to make choices," he says. "But these choicesshould be as informed as possible by the evidence. And if the evidence isn't good, doctors shouldknow that, too. And so should patients." In fact, the question of whether the problems with medical research should be broadcast to the publicis a sticky one in the meta-research community. Already feeling that they're fighting to keep patientsfrom turning to alternative medical treatments such as homeopathy, or misdiagnosing themselves onthe Internet, or simply neglecting medical treatment altogether, many researchers and physiciansaren't eager to provide even more reason to be skeptical of what doctors do—not to mention howpublic disenchantment with medicine could affect research funding. Ioannidis dismisses theseconcerns. "If we don't tell the public about these problems, then we're no better than nonscientistswho falsely claim they can heal," he says. "If the drugs don't work and we're not sure how to treatsomething, why should we claim differently? Some fear that there may be less funding because westop claiming we can prove we have miraculous treatments. But if we can't really provide thosemiracles, how long will we be able to fool the public anyway? The scientific enterprise is probably themost fantastic achievement in human history, but that doesn't mean we have a right to overstatewhat we're accomplishing." We could solve much of the wrongness problem, Ioannidis says, if the world simply stopped expectingscientists to be right. That's because being wrong in science is fine, and even necessary—as long asscientists recognize that they blew it, report their mistake openly instead of disguising it as a success,and then move on to the next thing, until they come up with the very occasional genuinebreakthrough. But as long as careers remain contingent on producing a stream of research that'sdressed up to seem more right than it is, scientists will keep delivering exactly that.
"Science is a noble endeavor, but it's also a low-yield endeavor," he says. "I'm not sure that more thana very small percentage of medical research is ever likely to lead to major improvements in clinicaloutcomes and quality of life. We should be very comfortable with that fact." This article available online at: Copyright 2010 by The Atlantic Monthly Group. All Rights Reserved.

Source: http://www.isepp.org/pdf%20and%20doc%20files/Lies,%20Damned%20Lies,%20and%20Medical%20Science%20-%20Magazine%20-%20The%20Atlantic.pdf