Why So Many of Us Become Detectives in Our Own Illness

I was very surprised when I looked at the numbers to see just how common diagnostic error is. Specifically, in 2015, the National Academies of Sciences, Engineering, and Medicine took on a really big report where they focused on diagnostic error and tried to figure out the scope. They ended up concluding that every American will experience at least one diagnostic error in their lifetime, sometimes with devastating consequences. That was shocking to me.

They defined diagnostic error as getting either an incorrect diagnosis, a delayed diagnosis, or a missed diagnosis, meaning the person is still undiagnosed. I did think this happened frequently, but I did not realize how frequently and how common it is. The scope was very surprising to me.

The definition of a diagnosis is basically a label that explains the symptoms you have. It’s an answer to what is ailing you, but it’s really so much more than that, it’s this gateway to care and community.

Without a diagnosis, it’s hard to get effective treatment. It’s hard to get insurance coverage for that treatment. It’s hard to become eligible for clinical trials. And there’s this whole other emotional element where you also don’t have the benefit of community. Even for something very rare, if you don’t know what it is, you don’t know how to access other people who have it.

The other thing I found really interesting in the reporting is this concept that sociologist Sarah Nettleton calls “permission to be ill,” and basically a diagnosis provides that. [So many people I interviewed] talked about the doubt you experience from the medical community, from family and friends, and even from yourself if you don’t have a diagnosis. Without a diagnosis, you don’t have the ability to tell somebody, “I have this specific thing, I am sick.” And so, people are less likely to fully understand you.

Dr. Cejas had this experience where early in medical school she had a lump in her neck. She kept going in to seek treatment, and over years she was told again and again that it was nothing, maybe a cyst, and not to worry about it. She says the first time she went in, she actually thought that made sense. She doesn’t blame the physicians who saw her initially for reaching that conclusion because of this saying: rare things are rare. You don’t expect, and maybe shouldn’t expect, that a young person has cancer the first time you see them. She didn’t necessarily want someone to biopsy her unnecessarily at the first complaint.

However, she went in multiple times to complain about the same thing. The lump was clearly getting larger. It was creating pain she didn’t have before. She felt that at that point, maybe even by the second visit, someone should have started putting together that this young woman had been complaining about this lump for a very long time, and if you look at her medical history, it was changing. That should have prompted someone to think, “Maybe it is zebras. Maybe we should pay a little more attention.”

Instead, she basically had to advocate for herself to even get a scan done of her neck. What was really interesting to me was that the sheer timeline didn’t seem to be taken into account.

There were basically two things physicians recommended when I asked them for advice on getting a diagnosis, and I think these work for both rare and common conditions.

One was to come in with a very clear articulation of your chief complaint, that’s the number one thing bothering you. Because that’s what they have to narrow in on when reaching a diagnosis. But if you’re experiencing something rare, it might not just be “this lump is bothering me.” Technically that’s the chief complaint, but you could also say, “This lump is bothering me, and I am very worried that it is not benign and that it is something serious.” Make it clear that this is your primary concern and that they need to actively explain their reasoning to you.

The other piece of advice doctors gave was to ask, “What else do you think it could be?” Even if they’ve given you a diagnosis or said, “I think you have X,” asking whether it could be something else helps prompt physicians either to consider other possibilities or to articulate why they are so convinced of the diagnosis.

I want to say: I don’t think this is ideal [as a dynamic]. Patients are already so overburdened in the diagnostic process. But these two pieces of advice have been helpful for me, and I use these questions in my own doctor’s appointments now.

One of the biggest issues and, unfortunately, this is a pretty major systemic issue, is the fact that a physician rarely finds out whether they got a diagnosis right or wrong. Sometimes they do. If you have an ongoing relationship with your primary care doctor and they diagnose you with something, but your symptoms don’t go away and you return to them, then yes, they find out. But often, if you get referred to a specialist and that person doesn’t reach a diagnosis, you go somewhere else. The original specialist may never find out you got a different answer elsewhere.

A lot of clinicians also work in urgent care or emergency room settings and have no mechanism to find out what happened to the patients they saw. People come in briefly, then go back out and maybe see somebody else.

Many of the physicians I interviewed, especially those interested in improving their diagnostic skills, talked about this as a major problem and gap. Without tracking, a doctor doesn’t even know there’s an area where they could improve. So many of the physicians particularly interested in combating diagnostic error create systems for themselves. They might look back at all the patients they saw last week and follow up through charts or other means to see whether they got it right or wrong.

There are researchers developing systems hospitals could adopt to do this kind of tracking, but they struggle to get buy-in because healthcare systems worry that tracking diagnostic error could open them up to malpractice suits. We have such a litigious society. But I think it would be great if there were some way for doctors once a month to review their cases. Without that kind of feedback, it’s hard to know you need to change.

I also think if hospitals were ranked for diagnostic accuracy, the same way they’re ranked for cancer care or diabetes care, people would be very inclined to go to a place known as one of the best diagnostic centers. There are groups like Leapfrog trying to develop something like this. Some combination of feedback and incentives for healthcare systems to care about diagnostic accuracy would be really helpful.

I do think there’s a shift happening. In conversations with younger physicians and medical students, I think there is a greater awareness of these dynamics, and that’s super encouraging.

I also think there’s more willingness to engage openly with the uncertainties of medicine. There used to be a pervasive culture where physicians felt they needed to be all-knowing, and it was hard to admit uncertainty to peers or anyone else. I think that’s changing, and that will be really helpful. Medical students talk now about learning to navigate uncertainty, whereas that really wasn’t the case even not that long ago.
Diagnostic error still doesn’t feel like it has really broken through the public consciousness. There is a movement to improve diagnosis, but it’s relatively small and made up mostly of researchers and some doctors. This is a patient safety issue deserving of major public attention. If we collectively realized how pervasive this is, it feels like something the public could and should be pretty angry about, and could collectively demand change around.

When I talked to both patients and clinicians about the challenges around diagnosis, there was one area where both sides aligned.

Patients would say things like, “It took me months to get an appointment. Once I finally got one, it was incredibly short, and I felt like the physician wasn’t listening to me. They were typing on a computer the entire time and barely looking at me.”

Then I would ask physicians the same question, and they would say, “I have to see 20 patients a day. I’m only allowed 15 to 20 minutes at most for each appointment. I have to type all this data into the electronic health record, and I don’t have enough time with the patient.”

So physicians were also complaining about the lack of time and the inability to really listen. From the doctors’ perspective, they repeatedly said they wanted more time with patients to actually listen, or more time to go through charts and dig into medical histories. They also wanted more time simply to think: to take in what they’d heard from a patient, especially in a complicated case, and really digest it before reaching a diagnosis.

Everybody was complaining about the same aspects of the healthcare system that make deep listening difficult, and that translates into so many other problems in the diagnostic process.

When people have a negative experience with the healthcare system, it’s only logical that they seek care elsewhere. Sometimes that’s okay, you find another clinician or another practice that’s helpful. But it can also be dangerous because if you’re dealing with a serious medical issue, the healthcare system is still the system best set up to help you.

When people have bad experiences, they begin to distrust the system, often for very legitimate reasons. [But] you don’t know whether the alternative sources they seek out are actually helpful or potentially harmful.

I know many people with complicated diagnoses that took forever to identify who tried therapies that ended up being dangerous or caused awful side effects. It’s devastating because people are looking for relief and looking for someone to listen to them. When that trust breaks down, it’s dangerous on a personal level, but also on a societal level because it can lead to an embrace of treatments or ideas that may ultimately be counterproductive to care.

The Undiagnosed Diseases Network consists of clinics run through the National Institutes of Health that focus on people with rare genetic disorders and some of the most baffling medical mysteries. Patients come in, and a multidisciplinary group of doctors and scientists—cardiologists, neurologists, oncologists, dermatologists, and others—all review the same patient together and try to crack the case.

These appointments are long. Patients go through extensive testing. Even before they arrive, the team has already reviewed all their medical records. I don’t think that level of evaluation can happen with every appointment because it’s very time-consuming and costly. But I do think a version of that should exist at every major medical center in the United States.

Physicians quickly recognize that [certain] cases are more complicated than they can fully address in a short appointment. Instead of referring patients to just one specialist who will view the issue narrowly, there could be somewhere patients are seen holistically and given the time and care they need. I’m hopeful that model could become more widely embraced throughout medicine.

I’m optimistic about AI’s ability to help in the rare disease sector specifically because, as Dr. Kohane implies, it’s not possible for a single physician to know everything or have seen everything, or even to know they’re seeing something unusual.

People with rare diseases who eventually got diagnosed described it as almost serendipitous. After seeing many physicians, they finally ended up with someone who had seen the condition before or remembered a colleague discussing a similar case, and suddenly connected the dots. That’s not an ideal diagnostic system: relying on eventually finding the one doctor with encyclopedic knowledge.

The benefit of AI is that it can look across everything. Even the Undiagnosed Diseases Network has experimented with AI by inputting applications into chatbots and finding that they were fairly good at suggesting potential diagnoses.

At the same time, there are concerns about deskilling, which is a concern across industries. Will doctors become so reliant on these tools that aspects of diagnosis or physical examination become weaker skills over time? Still, if research continues to show that AI is genuinely helpful for rare disease diagnosis, then I think it absolutely should be embraced.

Also, one thing that came to mind: when I interviewed patients who went through the Undiagnosed Diseases Network and eventually got a diagnosis, what stood out most to them often wasn’t the sophisticated diagnostic technology itself. Nine times out of ten, what they talked about was how the physicians made them feel. A lot of their relief came from finally feeling like they didn’t have to be diagnostic nomads anymore. They had found a place where people were committed to figuring out what was wrong and helping them create a plan going forward.

Even for people with devastating diagnoses, this mattered deeply because they finally felt someone was willing to sit with their questions and actively go on the journey with them.

That really stood out to me. I do think AI tools will hopefully be very helpful, but I don’t want us to lose sight of the fact that what so many people are craving is simply more humanity from the healthcare system.