Deep Medicine
Hippocrates said, “It is more important to know what sort of person has [a] disease than to know what sort of disease a person has.”
As Peabody said years ago, the secret of caring for patients is in caring for the patient.
In medicine, we often rely on changes in the frequency of so-called surrogate endpoints instead of the frequency of endpoints that really matter. So
In 2017 the American Heart Association and American College of Cardiology changed the definition of high blood pressure, for example, leading to the diagnosis of more than 30 million more Americans with hypertension despite the lack of any solid evidence to back up this guideline.7
This is where we are today: patients exist in a world of insufficient data, insufficient time, insufficient context, and insufficient presence. Or, as I say, a world of shallow medicine.
Lastly, AliveCor’s successes show that, in the era of AI in medicine, David can definitely still beat Goliath.
For medicine, a landmark Nature paper in 2017 on the diagnosis of skin cancer using DNN, matching the accuracy of dermatologists, signified AI’s impact on our area of interest.20
Deep learning AI is remarkably different from and complementary to human learning. Take child development: Yann LeCun, the AI pioneer at Facebook, weighed in on this key issue: “Human children are very quick at learning human dialogue and learning common sense about the world. We think there is something we haven’t uncovered yet—some learning paradigm that we haven’t figured out. I personally think being able to crack this nut is one of the main obstacles to making real progress in AI.”3
Harari, in Homo Deus, projected that “twentieth century medicine aimed to heal the sick, but twenty-first century medicine is increasingly aiming to upgrade the healthy.”40
home. A Mayo Clinic team showed that the texture of brain MRI images could predict a particular genomic anomaly, specifically 1p/19q co-deletion, that’s relevant to surviving certain types of brain cancer.17
Theoretically, with further refinements it will be possible to offer what are known as ultra-low dose CT scans, reducing the radiation by orders of magnitude and even lowering the cost of the CT machines themselves by eliminating the need for very high powered components. What an unexpected twist: machines disrupting machines instead of disrupting humans.
The need for a human to integrate and explain medical results will become even more pronounced.
A team led by Australian researchers used neural network analysis of 15,957 CT scans from individuals over the age of sixty to develop and validate a five-year survival plot, partitioning patients by their risk of death, ranging from groups in which 7 percent were expected to die, to groups in which 87 percent were expected to die (Figure 6.2).43 While today these algorithms are confined to research papers and haven’t entered the realm of clinical care, it’s just a matter of time until they are at least available for clinical care, even if they are not always applied. Among today’s medical specialties, it will be the radiologist who, having a deep understanding of the nuances of such image-based diagnostic algorithms, is best positioned to communicate results to patients and provide guidance for how to respond to them.
Nevertheless, although some have asserted that “radiologists of the future will be essential data scientists of medicine,” I don’t think that’s necessarily the direction we’re headed.44 Instead, they likely will be connecting far more with patients, acting as real doctors.
In 2015, Richard Levenson and colleagues tested whether pigeons could be trained to read radiology and pathology images.45 The team placed twelve pigeons in operant conditioning chambers to learn and then to be tested on the detection of micro-calcifications and malignant masses that indicate breast cancer in mammograms and pathology slides, at four-, ten-, and twenty-times levels of magnification.
skin cancer is the most frequent human malignancy, with its highest incidence in Australia and New Zealand (about 50 per 100,000 population) and 30 per 100,000 in the United States.
Google study of more than 300,000 patients, retinal images can predict a patient’s age, gender, blood pressure, smoking status, diabetes control (via hemoglobin A1c), and risk of major cardiovascular events, all without knowledge of clinical factors (Figure 7.2).18 The predictive accuracy was quite good for age and gender, and moderate for smoking status, blood pressure, and adverse outcomes. Such a study suggests the potential of a far greater role for eyes as a window into the body for monitoring patients. If such approaches are prospectively validated, we might see wide use of periodic retina self-exams with a smartphone in the future.
Tempus provides “digital twin” information with their report generated two to three weeks after samples are received. This consists of treatment and outcomes information from the de-identified patients most similar with respect to demographics and biologic information.
When I asked Lefkofsky why more people in the medical community don’t know about Tempus and why the company itself remains on the quiet side, he responded they had no interest in being the next Theranos. The company is intent on total transparency and publishing its data in peer-reviewed journals. That’s laudable.*
These metrics can be applied to a range of problems. Researchers at the University of Southern California developed software that was able to use seventy-four acoustic features, including voice quality, shimmer, pitch, volume, jitter, and prosody to predict marital discord as well as or better than therapists.10
Even the way people use a smartphone’s keyboard can be a useful marker. The company Mindstrong has broken this behavior down to forty-five patterns, including scrolling and latency time between space and character types. Their data correlated with gold-standard measurements for cognitive function and mood (Figure 8.1) in initial studies. Computer scientists at the University of Illinois took this concept further with deep learning and a custom keyboard, loaded with an accelerometer. Using an algorithm they built called DeepMood, they predicted depression with very high accuracy in a pilot study, providing some independent proof of concept for passive mood tracking via an individual’s keyboard activity.14
Sandy’s Human Dynamics lab has studied “honest signals,” the ways we unconsciously and nonverbally communicate truths about ourselves, for decades. Some examples of honest signals include our tone, fluidity, conversational engagement, and energy while we speak.
Cogito used deep learning algorithms and honest signals to build an app called Companion that is used by psychologists, nurses, and social workers to monitor the mental health of their patients.
In 2017, Andrew Reece and Christopher Danforth used deep learning to ingest 43,950 Instagram photos from 166 people (who digitally consented to share their social media history), of whom seventy-one had a history of depression.17 As many photo features as you can imagine, and more, were analyzed for psychological insight: whether people were present; whether the setting was indoors or outdoors, night or day; color and brightness by pixels; comments and likes of the photos; and the posting frequency of the user.
Depression accounts for more than 10 percent of the total global burden of disease; worldwide more than 76 million human-years are lost to disability each year, which far outstrips heart disease, cancer, and all other medical diagnoses.21
Each year 7 percent of Americans (16 million adults) will be clinically diagnosed with depression, and the lifetime risk of a mental disorder is approximately 30 percent.
Using diffusion tensor MRI measures of brain white matter and machine learning, major depression disorder was shown to be quite distinct from healthy controls.24 Conor Liston and colleagues at Weill Cornell Medicine analyzed scans from nearly 1,200 people, of whom 40 percent were diagnosed with depression.25
A mammoth review of fifty years of 365 suicide research studies from 2,542 unique papers looking at more than 3,400 different metrics found that, at best, those thousands of risk factors are very weak predictors of suicidal ideation, suicide attempts, or completion—only slightly better than random guessing.35 With no category or subcategory accurately predicting above chance levels, Joseph Franklin and colleagues concluded, “These findings suggest the need for a shift in focus from risk factors to machine learning algorithms.”36
In 2017, a team of researchers at Vanderbilt and Florida State Universities did just that. After reviewing 2 million de-identified electronic medical records from Tennessee hospitalized patients, the researchers found more than 3,000 patients with suicide attempts. Applying an unsupervised learning algorithm to the data accurately predicted suicide attempts nearly 80 percent of the time (up to a six-month window), which compares quite favorably to the 60 percent from logistic regression of traditional risk factors.37
A machine learning algorithm developed at Cincinnati Children’s Hospital by John Pestian was reported, in 479 patients, to achieve 93 percent accuracy for predicting serious risk of suicide.39 It incorporated data on real-world interactions such as laughter, sighing, and expression of anger.
Researchers at Carnegie Mellon did a very small but provocative study with functional MRI brain images of seventeen suicidal ideators and seventeen controls.40 Machine learning algorithms could accurately detect “neurosemantic” signatures associated with suicide attempts. Each individual, while undergoing the MRI, was presented with three sets of ten words (like “death” or “gloom”). Six words and five brain locations determined a differentiating pattern. Machine learning classified the brain image response correctly in fifteen of the seventeen patients in the suicide group and sixteen of the seventeen healthy controls. This study is interesting from an academic perspective but of limited practical use because it is not likely we will ever be doing MRI scans to find people with suicide risk.
Multiple studies have taken on the challenge of predicting whether a hospitalized patient will need to be readmitted in the month following discharge from a hospital, particularly finding features that are not captured by doctors. For example, a study conducted by Mount Sinai in New York City used electronic health records, medications, labs, procedures, and vital signs, and demonstrated 83 percent accuracy in a relatively small cohort.15
Manual, human scheduling for operating rooms or staffing all the inpatient and outpatient units in a hospital leads to remarkable inefficiencies. Much of the work that attends to patients calling in to schedule appointments could be accomplished with natural-language processing, using human interface as a backup.
Algorithms are already being used at some health systems to predict no-shows for clinic appointments, a significant source of inefficiency because missed appointments create so many idle personnel.
in the marginAirlines do this right
How AI can ease medical workflow is exemplified by a program that MedStar Health, the largest health system in the Washington, DC, region, has initiated in its emergency rooms. The typical ER patient has about sixty documents in his or her medical history, which takes considerable time for clinicians to review and ingest. MedStar developed a machine learning system that rapidly scans the complete patient record and provides recommendations regarding the patient’s presenting symptoms, freeing doctors and nurses to render care for their patients.31
sepsis accounts for 20 to 30 percent of all deaths among hospitalized patients in the United States. Timely diagnosis is essential since patients can deteriorate very quickly, often before appropriate antibiotics can be selected, let alone be administered and take effect.
That’s a critical goal: we lose about 2 million brain cells for every minute a clot obstructs the blood supply.42 Even earlier in the diagnosis of stroke, paramedics can apply the Lucid Robotic System, FDA approved in 2018, which is a device put on the patient’s head that transmits ultrasound waves (via the ear) to the brain, and by AI pattern recognition it helps diagnose stroke to alert the receiving hospital for potential clot removal.43
Mercy Hospital’s Virtual Care Center in St. Louis gives a glimpse of the future.46 There are nurses and doctors; they’re talking to patients, looking at monitors with graphs of all the data from each patient and responding to alarms. But there are no beds. This is the first virtual hospital in the United States, opened in 2015 at a cost of $300 million to build.
in the marginFascinating
Although being observed from a distance may sound cold, in practice it hasn’t been; a concept of engendering “touchless warmth” has taken hold. Nurses at the Virtual Care Center have regular, individualized interactions with many patients over extended periods, and patients say about the nurses that they feel like they “have fifty grandparents now.”47
Although systems to monitor all vital signs automatically, such as the Visi device of Sotera Wireless, are approved and currently being used by many health systems, there is no FDA-approved device for home use yet. Until we have FDA devices approved for home use that are automatic, accurate, inexpensive, and integrate with remote monitoring facilities, we’ve got an obstacle.
in the marginLook into
Looking overseas, one relatively small insurer that has been gaining some experience with more comprehensive data is Discovery Limited, which originated in South Africa but is also now available in Australia, China, Singapore, and the UK. Its Vitality program uses a Big Data approach to capture and analyze physical activity, nutrition, labs, blood pressure, and, more recently, whole genome sequences for some individuals. There have yet to be any publications regarding the betterment of health outcomes with this added layer of data, but it may represent a trend for insurers in the future.
Jean-Louis Reymond, at the University of Bern in Switzerland, has put together a database known as GDB-17 of 166 billion compounds, representing all chemically feasible molecules made up of seventeen or fewer atoms. Nearest-neighbor algorithmic analysis can sift through the whole database in just a few minutes to find new molecules that have effects similar to those of known drugs. Many compounds in Reymond’s database have turned out to be very hard to synthesize, so he whittled it down to a “short list” of 10 million easy-to-make compounds. Just 10 million!
in the marginFascinating
One of the most impressive papers in AI drug discovery to date comes from Marwin Segler, an organic chemist at BenevolentAI.33 He and his colleagues at the University of Munster designed a deep learning algorithm to learn on its own how reactions proceed, from millions of examples. It was used to create small organic molecules from more than the 12 million known single-step organic chemistry reactions.34
One, known as ATOM, for Accelerating Therapeutics for Opportunities in Medicine, brings together multiple academic centers (Duke and Tulane Universities) and pharmaceutical companies (including Merck, AbbVie, Monsanto) to “develop, test, and validate a multidisciplinary approach to cancer drug discovery in which modern science, technology and engineering, supercomputing simulations, data science, and artificial intelligence are highly integrated into a single drug-discovery platform that can ultimately be shared with the drug development community at large.”43
in the marginExctly what i want to do
First, there are the place cells, which fire when we are at a particular position. Second, there are the head-direction cells, which signal the head’s orientation. Third, and perhaps most remarkable, there are the grid cells, which are arranged in a perfectly hexagonal shape in the hippocampus. The hippocampus is often referred to as the brain’s GPS, and the grid cells make clear why. They fire when we are at a set of points forming a hexagonal grid pattern, like a map inside our head that our brains impose upon our perception of the environment.51
A 2018 study of more than 95,000 people in eighteen countries, while verifying a modest increase of blood pressure (with increasing sodium ingested as reflected by urine measurement), showed that the bad outcomes only occurred when sodium intake exceeded 5 grams per day.17 The average American takes in about 3.5 grams of sodium per day.18 In fact, for less than 5 grams per day of sodium, there was an inverse correlation between sodium intake and heart attack and death!
The food constituents weren’t the driver for glucose response. The bacterial species in the gut microbiome proved to be the key determinant of each person’s glucose response to eating.
the Salk Institute by Satchin Panda, who used a smartphone app to monitor daily eating patterns. Panda’s work showed people completely lack a three-meals-a-day structure but instead eat for a median of 14.75 hours a day!36
New technology, like an ingestible electronic capsule that monitors our gut microbiome by sensing different gases, may someday prove useful for one key dimension of data input.40
Some have speculated iCarbonX will need 10 million, not 1 million, people and far more capital than $600 million to execute this far-reaching mission. Nonetheless, it indicates that there is at least one major pursuit of the broader AI human health coach.
But this narrower approach would likely introduce human bias as to what data are useful as inputs, not taking advantage of the hypothesis-free discovery capabilities of the neural network. Nonetheless, it will probably be the default, a compromising way of moving forward in a non-holistic fashion. While this path for specific-condition coaching may accelerate success and validation, we shouldn’t take our eye off the ball—the goal of overall health preservation.
in the marginSpecified AI Problem
The EHR is a narrow, incomplete, error-laden view of an individual’s health. This represents the quintessential bottleneck for the virtual medical assistant of the future.
It’s your body. You paid for it. It is worth more than any other type of data. It’s being widely sold, stolen, and hacked. And you don’t know it. It’s full of mistakes that keep getting copied and pasted, and that you can’t edit. You are/will be generating more of it, but it’s homeless. Your medical privacy is precious. The only way it can be made secure is to be decentralized. It is legally owned by doctors and hospitals. Hospitals won’t or can’t share your data (“information blocking”). Your doctor (>65 percent) won’t give you a copy of your office notes. You are far more apt to share your data than your doctor is. You’d like to share it for medical research, but you can’t get it. You have seen many providers in your life; no health system/insurer has all your data. Essentially no one (in the United States) has all their medical data from birth throughout their life. Your electronic health record was designed to maximize billing, not to help your health. You are more engaged and have better outcomes when you have your data. Doctors who have given full access to their patients’ data make this their routine. It requires comprehensive, continuous, seamless updating. Access to or “control” of your data is not adequate. ~10 percent of medical scans are unnecessarily duplicated due to inaccessibility. You can handle the truth. You need to own your data; it should be a civil right. It could save your life.
in the marginIDEA FOR A DEVICE
Take the little post-Soviet nation Estonia, profiled in the New Yorker, as “the digital republic:” “A tenet of the Estonian system, which uses a blockchain platform to protect data privacy and security, is that an individual owns all information recorded about him or her.”44 No one can even glance at a person’s medical data without a call from the system overseers inquiring why it is necessary. The efficiencies of Estonia’s health informatics system, in contrast to that of the United States, are striking, including an app for paramedics that provides information about patients before reaching their home and advanced telemedicine capabilities with real-time vital sign monitoring (with AI algorithms for interpretation) setting up doctoring from a distance and the avoidance of adverse drug-drug interactions.
Today the self-driving car is viewed as the singular most advanced form of AI. I think a pinnacle of the future of healthcare will be building the virtual medical coach to promote self-driving healthy humans.
A clinic appointment for a new patient was slotted for one hour minimum and return visits for thirty minutes.
Amazingly, ninety years ago, Francis Peabody predicted this would happen: “Hospitals... are apt to deteriorate into dehumanized machines.”3 (Parenthetically, if you read one paper cited in this chapter, this would be the one.)
In recent decades, it has lost its way from taking true care of patients. A new patient appointment averages twelve minutes, a return visit seven.
The National Bureau of Economic Research published a paper in 2018 by Elena Andreyeva and her colleagues at the University of Pennsylvania that studied the effect of the length of home health visits for patients who had been discharged from hospitals after treatment for acute conditions. Based on more than 60,000 visits by nurses, physical therapists, and other clinicians, they found that for every extra minute that a visit lasts, there was a reduction in risk of readmission of 8 percent.6
The increased happiness derived from purchasing time was across the board, independent of income or socioeconomic status, defying the old adage that money can’t buy happiness.10
The ongoing Time Bank project at Stanford University’s medical school shows how this works. The Time Bank is set up to reward doctors for their time spent on underappreciated work like mentoring, serving on committees, and covering for colleagues. In return, doctors get vouchers for time-saving services like housecleaning or meal delivery, leading to better job satisfaction, work-life balance, and retention rates.11
This time it will be vital for doctors to take on the role of activists.
AI experts admit there will always be a gap, the inability to “imbue such a machine with humanness”—that ineffable presence the Japanese call sonzai-kan.21
An important clause in the Hippocratic Oath holds that “sympathy and understanding may outweigh the surgeon’s knife or the chemist’s drug.”
Empathy is crucial to our ability to witness others who are suffering.23 Ironically, as doctors, we are trained to avoid the s-word because it isn’t actionable.
in the marginKEY
Perhaps it’s not surprising, then, that there are billing codes, reimbursement rates, and pills for treating anxiety but none for alleviating suffering. There’s also no machine that will do it; alleviating suffering relies on human-to-human bonding; it requires time, and its basis is trust.
The term alone conveys a coldness even worse than labeling people as having “heart failure”—a coldness that needs to be replaced with something warmer. The way we speak about our patients’ suffering becomes the important words that affected people have to live with and think about every day of their lives.
in the marginLanguage matters
It was the father of modern medicine, William Osler, who said, “Just listen to your patient; he is telling you the diagnosis.”
in the marginKEY
As doctors we are trained to take the history. But that’s clearly the wrong concept; it preempts conversation, which is both giving and taking.33 That’s how the deepest and intimate feelings come out, and if there’s one thing doctors wish, it’s that “they had time to talk with their patients, knowing the value of such contact.”34
A new trend in some medical centers is for doctors to give their patients a card with their picture and details about their family, where they live, and their hobbies and nonmedical interests.36 While this lies in direct opposition to how doctors have historically been trained, it represents the right future path of humanistic medicine.
I was struck two decades ago when Yale’s medical school announced a required course for students to learn the art of observation by spending time in an art museum.41 Verghese gets that, too, writing in a “Narrative Matters” piece that “my tool is the medical gaze, the desire to look for pathology and connection, and it would seem there was no opportunity for that within a pigmented square of uniform color or a rectangle of haphazard paint splashes. But in me a profound and inward sort of observations was taking form.” Abraham takes medical students to the art museum at Stanford to foster observational skills.42
David Epstein and Malcolm Gladwell wrote an editorial to accompany the paper, which they called “The Temin Effect,” after the Nobel laureate Howard Temin,
Sarah Parker, a neurologist, wrote about an extraordinary example of human connection, empathy, and keen observation, without a single word uttered, in the face of tragedy: The doctor walked out of one of his clinic rooms and told his nurse he thought he was having a stroke. By the time I saw him, he was nonverbal, globally aphasic, unable to move his right side, his brain filling with rapidly expanding hemorrhage. He didn’t understand what I was asking him to do. He couldn’t tell me what he was feeling, but he recognized my white coat. He recognized the tone in my voice. He recognized the expression on my face. He took my hand in his left hand and repeatedly squeezed it and looked me right in the eyes. There was a moment of connection. A moment where two people know what the other is thinking and feeling without a word passing between them. He knew this was bad. He knew I knew this was bad. He knew I was trying to help, but he knew that there wasn’t much I could do. He was scared but also strong and courageous. He knew the situation and knew the likely outcome, and he was telling me that it was OK if it ended that way. That he knew I cared. It was a moment of peace. A man facing death, both afraid and aware. A man who was looking for that human connection. A man who had cared for others and comforted others his whole life was trying to comfort me while I tried to care for and comfort him.45
Rituals are about transformation, the crossing of a threshold, and in the case of the bedside exam, the transformation is the cementing of the doctor-patient relationship, a way of saying: ‘I will see you through this illness.
in the marginCLASSICS
The fundamentals—empathy, presence, listening, communication, the laying of hands, and the physical exam—are the building blocks for a cherished relationship between patient and doctor.
patients confront doctors who are trained in medical school to keep an emotional distance from their patients. That’s simply wrong. Without trust, why would people reveal their most intimate and sensitive concerns to a doctor? Or agree to undergo a major procedure or surgery, putting their lives in the doctor’s hands?
Verghese describes this dichotomy eloquently: We are perhaps in search of something more than a cure—call it healing. If you were robbed one day, and if by the next day the robber was caught and all your goods returned to you, you would only feel partly restored; you would be “cured” but not “healed”; your sense of psychic violation would remain. Similarly, with illness, a cure is good, but we want the healing as well, we want the magic that good physicians provide with their personality, their empathy and their reassurance. Perhaps these were qualities that existed in abundance in the prepenicillin days when there was little else to do. But in these days of gene therapy, increasing specialization, managed care and major time constraints, there is a tendency to focus on the illness, the cure, the magic of saving a life.56
It makes me think of the recent claim in China that an AI-powered robot—Xiaoyi—passed the national medical licensing examination for the first time. Are we selecting future doctors on a basis that can be simulated or exceeded by an AI bot?
we need to modernize the physical exam if physicians are to routinely incorporate new tools like smartphone ultrasound. Virtual telemedicine will, in many routine circumstances, replace physical visits, and that requires training in “webside manner,” which highlights different skills. There’s still a face-to-face connection, but, much as skipping a physical exam interferes in the practice of medicine, these physicians will be impaired by an inability to truly connect, lay on hands, and examine the individual, even when better sensors and tools are routinely transferring data remotely.