Physician turnover is disruptive to patients and costly to healthcare providers and physicians. In a new study, Yale researchers used machine learning to uncover factors that may increase the risk of such turnover, including physician tenure, age and complexity of cases.
And, by evaluating nearly three years of data from a large U.S. healthcare system, they were able to predict the likelihood of a physician leaving with 97 percent accuracy.
The insights provided by the findings could help healthcare systems intervene to reduce turnover before doctors decide to leave, the researchers said.
The study was published in PLOS ONE on February 1.
While healthcare organizations typically use surveys to track physician burnout and job satisfaction, the new study used data from electronic health records (EHRs), which most U.S. physicians use to track and manage patient information.
The problem with the survey is that doctors often feel burdened to respond, said Ted Melnick, associate professor of emergency medicine and co-senior author of the new study. As a result, response rates are often low. “Investigations can tell you what happened in that moment,” he added, “but they can’t tell you what happened the next day, or the next month, or the next year.”
However, electronic health records are constantly generating job-related data in addition to collecting clinical patient data, thereby providing the opportunity to observe physician behavior patterns moment by moment and over time.
In the new study, researchers used three years of de-identified EHR and physician data from a large New England healthcare system to see if they could take three-month data and predict physician turnover within Possibility six months.
“We wanted something that was useful on a personalized level,” said Andrew Loza, a lecturer and researcher in clinical informatics at Yale School of Medicine and the study’s co-senior author. The likelihood of a job leaving and the variable that contributed most to the estimate at that time, and intervened where possible. “
Specifically, data were collected monthly from 319 physicians representing 26 medical specialties over a 34-month period. Data included time spent by physicians using the EHR; clinical productivity measures, such as patient volume and physician demand; and physician characteristics, including age and years of employment. Different parts of the data are used to train, validate, and test machine learning models.
When tested, the model was able to predict with 97 percent accuracy whether the doctor would leave, the researchers found. The sensitivity and specificity of the model showed that the proportion of correctly classified departure and non-departure months was 64% and 79%, respectively. The model was also able to determine the extent to which different variables affected turnover risk, how the variables interacted, and which variables changed when physicians moved from low to high turnover risk.
The details about what drives the predictions are what make the method particularly useful, the researchers say.
“There has been an effort to make machine learning models not be black boxes, where you get predictions, but it’s not clear how the model arrives at it,” Loza said. The details will identify the issues that could cause the doctor to leave. “
Through their method, the researchers identified several variables that contribute to turnover risk; they found that the top four factors were physicians’ length of employment, age, case complexity, and demand for their services.
Whereas previous work could only analyze linear relationships, the machine learning model allowed the researchers to take a more granular look at the challenges doctors face. For example, recently hired physicians and physicians with longer tenure were at the highest risk of turnover, while physicians with mid-term tenure were at lower risk of turnover. Likewise, those younger than 44 were at higher risk of leaving, doctors aged 45 to 64 were at lower risk, and those 65 or older were at higher risk.
There are also interactions between variables. For example, the more time spent on EHR activities, the lower the risk of turnover for physicians with fewer than 10 years of employment. But for those doctors who work longer hours, it increases the risk of turnover.
“The findings underscore that there is no one-size-fits-all solution,” Loza said.
The risk of physician turnover changed over the entire study period, which covered the 34-month period from 2018 to 2021 (a period that includes both the pandemic and the pre-pandemic world), the researchers said. They also identified specific variables that changed as physicians moved from low to high turnover risk; EHR inboxes responded by team members rather than physicians, physician needs, and patient volume as physicians transitioned from low to high risk of turnover The proportion of messages is the most variable variable. The COVID-19 wave has also been associated with changes in turnover risk.
“I think this research is an important step in identifying the factors that drive clinician turnover, with the ultimate goal of creating a sustainable and thriving workplace for our clinicians,” said author Brian Williams, Medical Informatics Officer at Northeast Medical Group. Research.
To achieve this, the researchers created a dashboard that can display this information. Healthcare leaders see value in the type of analysis this approach can provide.
“As physician burnout is an increasingly recognized problem, healthcare systems, hospitals and large groups need to figure out what they need to do to ensure the physical and mental health and well-being of physicians and other clinicians who provide actual care to patients,” Northeastern Medical Group “Many health care systems already have health officials and health boards who can be tasked with collecting and analyzing this data and drawing conclusions, and then developing an implementation plan to make changes and hopefully improve,” said Robert McLean, New Haven Regional Medical Director. “
Melnick added, “We’re excited to see what this might look like in practice. We’ll continue to work on ethical implementation because this is really about how best to promote physician well-being and a thriving workforce.”
