Primary care clinicians’ perspectives about quality measurements in safety-net clinics and non-safety-net clinics

Abstract

Background

Quality metrics, pay for performance (P4P), and value-based payments are prominent aspects of the current and future American healthcare system. However, linking clinic payment to clinic quality measures may financially disadvantage safety-net clinics and their patient population because safety-net clinics often have worse quality metric scores than non-safety net clinics. The Minnesota Safety Net Coalition’s Quality Measurement Enhancement Project sought to collect data from primary care providers’ (PCPs) experiences, which could assist Minnesota policymakers and state agencies as they create a new P4P system. Our research study aims are to identify PCPs’ perspectives about 1) quality metrics at safety net clinics and non-safety net clinics, 2) how clinic quality measures affect patients and patient care, and 3) how payment for quality measures may influence healthcare.

Methods

Qualitative interviews with 14 PCPs (4 individual interviews and 3 focus groups) who had worked at both safety net and non-safety net primary care clinics in Minneapolis-St Paul Minnesota USA metropolitan area. Qualitative analyses identified major themes.

Results

Three themes with sub-themes emerged. Theme #1: Minnesota’s current clinic quality scores are influenced more by patients and clinic systems than by clinicians. Theme #2: Collecting data for a set of specific quality measures is not the same as measuring quality healthcare. Subtheme #2.1: Current quality measures are not aligned with how patients and clinicians define quality healthcare. Theme #3: Current quality measures are a product of and embedded in social and structural inequities in the American health care system. Subtheme #3.1: The current inequitable healthcare system should not be reinforced with financial payments. Subtheme #3.2: Health equity requires new metrics and a new healthcare system. Overall, PCPs felt that the current inequitable quality metrics should be replaced by different metrics along with major changes to the healthcare system that could produce greater health equity.

Conclusion

Aligning payment with the current quality metrics could perpetuate and exacerbate social inequities and health disparities. Policymakers should consider PCPs’ perspectives and create a quality-payment framework that does not disadvantage patients who are affected by social and structural inequities as well as the clinics and providers who serve them.

Background

Quality metrics, pay for performance (P4P), and value-based payments (VBP) are prominent aspects of the current and future American healthcare system, which may impact health disparities [1]. It is recognized that variability in clinics’ quality scores used to compare quality between clinics and between providers can be attributed in part to patient population factors beyond the scope or control of traditional care delivery and patient behaviors, such as poverty, housing, education, and employment [2, 3]. Health care systems and providers serving patients with lower socioeconomic status (SES) or higher burdens of poor structural determinants of health (SDOH) may be disproportionately impacted by P4P, thus widening already evident quality disparities [1, 4].

There are many examples of significant differences in provider quality scores for those serving high and low SES patient populations. In the outpatient setting, primary care providers serving a higher proportion of disadvantaged patients have lower quality scores [5]. Providers in accountable care organizations (ACOs) under Medicare contracts who serve a high proportion of patients with low SES have worse quality scores compared to other ACOs, despite similar practice characteristics and capabilities [6]. Disadvantaged patients [7] and subsequently safety-net hospitals [8] have higher readmission rates. Safety-net hospitals have been disproportionately financially penalized by Medicare’s value-based purchasing and Hospital Readmission Reductions Program [9]. Models have also indicated that Medicare’s Merit Based Incentive Payment System (MIPS) may exacerbate existing disparities due to its focus on specific clinical outcomes with failure to measure other aspects of healthcare quality such as access to care or patient experience [10]. These currently unmeasured aspects of healthcare are often more important to minority and low-income patients [11] as healthcare quality perception differs across race, ethnicity, and language preference [12]. Indeed, Medicare adjustment with a VBP Modifier could lead to exacerbation of racial and ethnic health care disparities due to inequitable payment differences to systems that serve higher-risk and lower- risk patient populations [13].

Current quality metrics do not typically take into account the SDOH factors that can contribute to quality score disparities, and providers caring for disadvantaged populations have greater difficulty meeting quality measures in P4P [14]. Because of this disparity in quality scores, outcomes, and financial penalties, the question of whether or not to risk-adjust quality metrics for socio-economic status (SES) of patients or SDOH risk factors has persisted [15]. P4P appears to have an overall mild positive effect on quality, especially process measures, but the unintended consequences regarding health disparities remain a concern [16]. Although there has been some evidence suggesting P4P may actually narrow disparities for low SES patients [17] and minority patients [18], several studies have indicated that health inequities related to sex, age, ethnicity, and practice type may be exacerbated [16].

While some qualitative studies have explored providers’ viewpoints on quality metrics and P4P [19,20,21,22,23,24,25,26], no studies have included primary care clinicians who have worked in both low-resourced clinics (such as federally-qualified health care systems with generally lower quality scores) and high-resourced clinics (such as private insurance systems of ACOs with generally higher quality scores) in order to understand disparate quality scores in the United States. Our study aims were to identify primary care providers’ (PCPs) perspectives about 1) quality metrics at both safety net clinics (SNCs) and non-safety net clinics (NSNCs), 2) how clinic quality measures affect patients and patient care, and 3) how payment for quality measures may influence health care. These PCPs’ perspectives could be useful to improving quality metric approaches and creating a state level P4P or VBP system.

Methods

Setting

Minnesota law requires the Minnesota Department of Health (MDH) to administer a statewide quality reporting and measurement system (SQRMS), and requires Minnesota providers to submit data on these quality measures. Contracting with a private nonprofit organization, Minnesota Community Measurement [27], to gather, report, and publicly publish the quality data, the state uses the data for multiple purposes, including P4P and VBP programs. With the increased use of quality scores for payment and accountability, organizations such as the Minnesota Heath Care Safety Net Coalition became concerned about the substantial influence of non-clinical patient and community factors on provider scores. In 2014, the Safety Net Coalition, the Minnesota Association of Community Health Centers and other organizations formed the Quality Measurement Enhancement Project (QMEP) to conduct research projects in order to account for the influence of SDOH on patients’ health, treatment outcomes, and provider quality scores and in order to influence MDH’s creation of a new P4P system¹.

This QMEP research project involved obtaining the perspectives of PCPs who had experience working in both SNCs and NSNCs. SNCs were defined as federally-qualified health care centers or Indian Health Services, which serve disadvantaged or uninsured populations and NSNCs were defined as large health care systems or privately-owned clinics which do not routinely offer sliding-fee discount programs for uninsured patients. This QMEP research team included two family physicians (KACP from a FQHC and DJS from University of Minnesota) and three researchers (SLP, LMO and MST) from SoLaHmo Partnership for Health and Wellness, a community-based participatory action research group. The research team created the research design and interview questions, with input from a QMEP Technical Work Group made up of members with expertise in clinical care, quality measurement, research and data. In addition, one of the key informants (LSO a family physician who has worked at both a NSNC and two SNCs) joined the analysis team, in community-based participatory action research fashion [28, 29].

Design

To identify PCPs perspectives based on their experiences, we chose a qualitative research design with interviews, including in-depth face-to-face key informant (KI) interviews with 4 family physicians to begin our process followed by 3 focus group (FG) discussions with 10 PCPs.

Recruitment

We recruited all 14 PCP participants by word-of-mouth, emails and snowball sampling; the 4 KIs were identified by QMEP committee members and invited by email; the 10 focus group participants were identified by the KIs, QMEP committee members, SNC medical directors, and invited by email. The two inclusion criteria were (1) primary care clinicians (2) who had worked at both safety net and non-safety net primary care clinics. These were chosen to obtain a diversity of opinions based on PCPs’ experience in two significantly different primary health care settings. Additional 19 PCPs were invited but did not participate (8 were not interested; 8 were interested but could not attend; and 3 did not meet criteria).

Data collection

Two researcher dyads (KACP and LMO or KACP and MST) interviewed each key informant for 1.5 h, and led three 2-h focus group discussions following the same open-ended question guide supplemented by spontaneous follow-up questions (Additional file 1). In addition, participants completed a demographic questionnaire about their age, gender, profession, race/ethnicity, and work history. We concluded data collection after completing the planned processes that fit our timeline and funds (4 KIs and 3 focus groups), and which coincided with saturation of thematic content and exhaustion of potential participants as identified by our recruitment technique. The University of Minnesota Institutional Review Board determined the study was exempt from IRB overview. Participants were instructed in emails and at interviews that the study was IRB review exempt; were given written information about the study; and were encouraged to keep the information confidential.

Qualitative and participatory analysis

The audio recorded key informant interviews and focus groups were transcribed verbatim. The 3 interviewers (KACP, LMO, MST) agreed upon a basic organizational coding structure created from the interview questions, forming a framework with which each person then inductively coded each transcript, and then wrote summaries of the main organizational categories. One interviewer (KACP) placed the summaries onto a spreadsheet, following the coding structure. The complete five member research team (KACP, MST, LMO, SLP and LSO) read the transcripts, reviewed the codes and the summaries, discussed codes, reconciled differences, inductively identified the main themes, completed the overall analysis and reached the final interpretation of the data [30, 31]. Three additional QMEP team members joined the writing team (AMP, MS and DJS). The three interviewers (KACP, LSO and MST) selected illustrative quotes to include in the presentation of findings. The participants received copies of the draft report and the article to review; all who responded affirmed the findings and none made suggestions for changes.

Results

Characteristics of the 14 PCPs are in Table 1. Generally, there are more women than men, mostly older people, mostly family physicians and mostly European-Americans. Results are presented by three themes and three subthemes. Illustrative quotes are in Table 2.

Table 1 Characteristics of Primary Care Clinicians

Table 2 Representative Quotes for Themes

Theme #1: Minnesota’s current quality scores are influenced more by patients and clinic systems than by clinicians

Participants view disparate scores at NSNCs and SNCs as being due to differences in the patient populations who attend these clinics and due to variations in the clinic systems that support clinicians. Differences in patient populations lead to disparate quality metrics between NSNCs and SNCs. Minnesotans who attend NSNCs are seen as being more able to act in concert with the quality measures because they generally have low burden of SDOH, have health insurance, and have literacy levels, education, and cultural backgrounds that are generally congruent with mainstream medical culture. Thus, they are more capable of engaging with the clinic-based efforts to respond to quality metrics, especially the bio-medically defined self-management processes that are necessary to improve quality scores of chronic diseases.

Generally, Minnesotans who attend SNCs have higher burdens of SDOH, which pulls their energies and resources away from health and health care. Many are uninsured or under-insured, have low English proficiency, have low medical literacy, are immigrants or refugees, and come from diverse cultural backgrounds that are less in concert with, or even in conflict with, mainstream healthcare culture. Cultural issues due to differences in language, expectations of the role of healthcare systems, health care’s focus on individuals rather than on families, and cultural concepts of health, healing, and decision-making, as well as historical distrust and discrimination, influence the incongruence.

Differences in clinic systems lead to disparate quality metrics between NSNCs and SNCs. NSNC health care systems use their higher financial resources to create clinic teams, clinic workflows, electronic medical record (EMR) processes, and adjunct patient education approaches that specifically address the metrics, as administrators view the metrics as promoting cost effective care.

In addition, NSNCs have financial and social incentive programs to influence clinicians to act on the metrics in order to increase their quality scores, so that clinicians will address the quality metrics in their interactions with patients and the EMR. In contrast, SNCs do not have the financial resources to develop teams, workflow processes, and systems specifically aimed at metrics. Most have not created the quality report cards or implemented financial incentives aimed at improving clinicians’ quality scores. Also, clinician and staff energies are diverted from quality metrics to deal with other aspects of patient care (language, health literacy, medical-legal-social issues, etc.).

There are specific clinics that serve low-income or immigrant populations within NSNC systems. Generally, these clinics have lower scores than other clinics in their systems because the clinics serve patients with high SDOH, and have higher scores than SNCs because they have more system resources. It is in these clinics that individual providers feel the punitive nature of linking quality scores with job performance and financial remuneration, since their colleagues in other clinics with patients with low SDOH burdens have better scores.

Overall, these PCPs do not see the differences in quality scores between NSNC and SNC as being due to clinicians’ having variable knowledge, skills, and abilities.

Theme #2: Collecting data for a set of specific quality measures is not the same as measuring quality healthcare

These PCPs see value in quality measures, but assert that healthcare quality measures should not be conflated with measuring quality healthcare. Quality measures are valuable when they are consistent with professionals’ mission to improve people’s health, when they provide clinicians with population-based perspective, when they are based on evidence-based medicine, and when staff and systems assist the PCP in providing care, which can prevent patients from “falling through the cracks”.

However, quality measures are not valuable when they result in clinicians’ taking empty actions that are “just clicking boxes”, when measures are impossible for their patients to meet, when they take clinic visit time away from connecting with patients’ focus on health problems, and when their actions improve scores but do not improve patients’ health.

In addition, quality measures can harm care, as clinicians focus their attention on things that are measured rather than things that are not measured, as they “cut corners” in order to avoid being overworked, work more hours to “click more boxes” (which has contributed to professional dissatisfaction and burnout), shuttle low-scoring (or “non-compliant”) patients to other providers or other clinics, take financial hits to their base salaries, or reduce their clinic hours (thus decreasing access), or adjust their practices in order to keep “high-performing patients” so their scores are good.

Specific clinic processes that focus on increasing quality scores include public displays of clinician specific data within clinics and clinic specific data within large healthcare systems. A few participants feel the positive nature of data displays and competition between individual providers, teams, and clinics. Most participants express discontent with the negative consequences of publicly displayed data and tying compensation, performance review, and even termination to quality scores, calling these tactics “shaming”, “unfair”, and “punitive”. These processes feel unfair because of the inequalities of providers’ patient populations, which lead to inequities of provider workloads and ability to meet measures. Finally, some participants argue that the process to improve quality measures and the money used to build the processes are not being used wisely to improve health.

Subtheme #2.1: Current quality measures are not aligned with how patients and clinicians define quality healthcare

Participants sense that most patients are unaware of current quality measures, their scores, their doctors’ scores, or their clinics’ scores. They speculate that some patients from high socio-economic class backgrounds who attend NSNCs may be aware, but not those who attend SNCs. They sense that for most patients, the current metrics do not measure the important aspects of quality healthcare, as patients would not define quality healthcare by numbers, cut-off values, or percentages of numeric goals; for most patients, these are too remote from their lived experiences. Rather, patients would define quality healthcare by their subjective sense of well-being, feeling respected by their doctors, nurses, and clinics, and being in trusting relationships with clinic staff and PCPs.

Clinicians define quality health care in broader ways than in the narrow, specific and well-defined quality measures. Their definitions include philosophical aspects of relationship-based care, from listening to and caring about patients, educating and empowering patients, having a therapeutic relationship with people, and developing a patient-provider partnership that helps people to attain their personal health goals. In addition, they stressed that patients should define their own health goals in whatever way is important to them, such as quality of life and satisfaction with their lives.

Theme #3: Current quality measures are a product of and embedded in the social and structural inequities of the American health care system

Participants express the view that the current quality metrics are based on social inequities. Many of the measures are grounded in inequitable research that gave rise to evidence-based medicine, whose data was generated from population studies done on majority white Americans. As such, they are not based on data collected from other specific patient populations.

Some people further identify the quality measures as unequitable tools that historically were selected by inequitable social processes. Initially, metrics were chosen by corporate executive officers who were purchasing health insurance as a technique to evaluate the quality of the product in balance with the cost in order to make wise financial decisions. Then, healthcare organizations and insurance plans adopted them to improve quality health care while curtailing costs, and government officials chose which measures to use to compare clinic and clinician performance in order to reduce costs. Current traditional quality measures have not been selected with input from patients or from communities living with the highest levels of disparities in health.

The quality metrics measure unequal processes. The disparate results between SNCs and NSNCs reflect the privilege of the insured, educated, middle and high social-economic class white Minnesotans whose lower SDOH burden and congruence with biomedical systems contribute to their higher quality scores. The current quality measurement system quantifies the biomedical view and the hierarchical American society into a “quality score” that shows lower class people at the bottom and higher-class people at the top, in congruence with social inequities.

Sub-theme #3.1: The current inequitable healthcare system should not be reinforced with financial payments

While acknowledging that the healthcare system is changing to P4P and VBP processes and concrete metrics are a necessary component to that process, participants express concern about the inequity of a clinic-based financial payment system that is tied to quality scores. Inequality in healthcare and healthcare quality measures mean that the neediest clinics serving the neediest patients will receive the least amount of money, when, in reality, they need more money to respond to patient population health needs.

Sub-theme #3.2: Health equity requires new metrics and a new healthcare system

These PCPs emphasize the need to: 1) operationalize patient-centric definitions of quality, beyond patient satisfaction, which are based on patients’ health goals, patients’ healthcare experiences, and patients’ assessments of the quality of their relationships with clinicians; 2) implement clinic processes to expand the team, hire staff from diverse communities, and have adequate time and resources to develop trusting relationships and deliver culturally and linguistically appropriate patient-centered care; 3) choose metrics based on evidence-based medicine for the populations and not just on measures that will save money; 4) utilize risk adjustment mechanisms to take into account the challenges that clinics and providers face whose patients have high SDOH burden; and 5) reward relative improvement in quality scores rather than attainment of an absolute threshold number.

Finally, true improvements in healthcare quality measures cannot be achieved solely with medical actions inside clinics. Quality health requires a universal health care plan, or innovative or inclusive processes and payment mechanisms so that basic medical care is available to everyone. Quality health requires societal and community actions outside of the clinics, as that is where the societal inequities are influencing health. The government needs to recognize this and work to address health at the societal level, and not attempt to hold clinics solely responsible for societal needs.

Discussion

This qualitative study of individual interviews and focus groups with primary care providers (PCPs) who had worked both in safety net clinics (SNCs) and non-safety net clinics (NSNCs) in metropolitan Minneapolis-St Paul Minnesota reveals PCPs’ critique of the current quality metric system and processes to align payment with scores on quality metrics. Participating PCPs see that quality measurements do not fairly identify which clinics provide superior care. They assert that current quality metrics, developed from inequitable evidence and selection biases, reflect and intensify social disparities. This view illustrates one way in which the American medicine system is influenced by structural racism.

Participants perceive the measures as more influenced by patient and clinic factors than by clinician factors. (Theme #1). SNCs have worse quality scores given that they serve patients with high SDOH burdens and have low resources to respond to multiple patients’ needs. In contrast, NSNCs have higher scores because they serve patients with low SDOH burdens and have more resources to create processes to prioritize what is measured. In their opinions, the current system of quality measurement does not truly measure quality health care as patients and clinicians define quality health care (Theme #2). They are not alone in these assessments. Significant disparities in health care have been correlated to patients’ social-structural determinants of health, which are out of the control of clinicians [32]. One qualitative study of PCPs’ early reactions to accountable care organizations (ACOs) processes found a similar concern about the challenges of being held responsible for quality measure results that are affected by societal factors that are beyond their control (i.e., patients’ SDOH) [24]. Likewise, SNCs serving patients with high burdens of SDOH are more likely to be financially punished by quality-based payment systems [33], which would leave them with even fewer resources to focus on both quality metrics and their patients’ social complexities. Furthermore, the inequitable selection biases that have designed and chosen these measures have resulted in metrics that may not be as important to vulnerable populations served by SNCs, [11, 12]; other health issues as defined by communities themselves may be more valuable, such as mental health and substance use [1].

Overall, these PCPs criticize the current quality metric systems as being a product of, and embedded in, social and structural inequities of the American health care system, and warn that tying financial payments to these inequitable processes would exacerbate current health disparities (Theme #3). Similar PCP perspectives about aligning quality metrics and payment have been described (24]. Evidence for these concerns is seen in large studies indicating that physician participation in ACOs (one of the most popular delivery system reforms utilizing quality metrics) is less prevalent in disadvantaged communities [34] and that those organizations serving minority populations perform worse on quality metrics [6]. Similar concerns that aligning payment with quality measures will not redress societal inequities and health care disparities have been documented in interviews with a broad range of health care professionals, including clinicians [25, 26], medical directors [35,36,37], health care administrators [25, 35], and hospital executives [36,37,38,39] as well as in a systematic review about financial reimbursement for hospitals [40]. If ACOs and value based purchasing, aimed at improving quality and reducing spending, are less effective in diverse, vulnerable, or socio-economically disadvantaged populations, they will not be successful in achieving the desired results and there is potential to actually exacerbate existing disparities by connecting payment with current quality metrics [6, 34]. These published studies corroborate the PCP participants’ perspectives that the prior quality metric design and implementation approaches have historically not supported, and will not lead to, improved quality health care in all populations, and ultimately should not be reinforced with an inequitable P4P system.

These clinicians want a just and equitable health care system, echoing calls for health equity from around the world [41], the United States [42, 43], and Minnesota [44]. Hardeman et al. [45], from Minnesota, argue that achieving health equity requires that healthcare professionals use their power to explore, understand and respond to the underlying structural racism that is undergirding the inequitable system. They build on Jones [46] to state: “Structural racism — a confluence of institutions, culture, history, ideology, and codified practices that generate and perpetuate inequity among racial and ethnic groups — is the common denominator of the violence that is cutting lives short in the United States.” (45, page 2113) While several participants use the term “SDOH” to describe the challenges that patients from disparate neighborhoods and communities face in obtaining quality health care, they acknowledge the structural racism that underlies these SDOH [45,46,47,48]. This recognition leads to expanding the term from SDOH to Structural/Social Determinants of Health Inequities (S-SDOH) in order to directly acknowledge the structural racism that underlies all of these inequitable social determinants of health [49].

Limitations

As with all qualitative research, the limited number of people interviewed limits the generalizability of the results to other populations or locations. In addition, the participants’ experience was focused on the metropolitan area of Minneapolis-Saint Paul, but did not represent all of the major NSNCs or SNCs in the metropolitan area in Minnesota, or all systems throughout Minnesota. Also, participants’ temporal experiences of SNCs and NSNCs mean that they were not comparing current quality practices within these clinical systems. Because more participants were currently working in SNCs may indicate underlying biases towards SNCs over NSNCs. A quantitative survey with open-ended questions of clinicians currently working in NSNCs and SNCs may address some of these limitations. Nonetheless, participants’ assessments of the difference between SNCs and NSNCs are similar, and hence are summarizable, and provide a cohesive view based on their experiences.

Conclusion

These PCPs in Minnesota, USA who have worked in both safety net (SNC) and non-safety net clinics (NSNCs) perceive that 1) current clinic quality scores are influenced more by patients and clinic systems than by clinicians; 2) current quality measures are not measuring quality healthcare; and 3) current quality measures are a product of and embedded in social and structural inequities in the American health care system, which should not be reinforced with financial payments such as current P4P or VBP, as aligning payment with the current quality metrics could perpetuate and exacerbate the existing social inequities and health disparities. They recommend that a new comprehensive approach to measuring and reimbursing quality needs to be designed that truly measures quality healthcare, is equitable and fair, and does not exacerbate the current inequities in the American healthcare system.

The National Quality Forum’s Roadmap for Promoting Health Equity and Eliminating Disparities, which was published after these interviews, is an approach that is consistent with the PCPs perspectives and recommendations, as it illustrates how quality metrics could be used to improve health equity [50]. One main NQF proposed strategy to achieve equity (“Incentivize the Reduction of Health Disparities and Achievement of Health Equity”) is directly relevant to PCP’s concerns for P4P and VBP. Several final recommendations: “Redesign payment models to support health equity”; “Support closing disparities by providing additional payments to providers who care for patients with social risk factors”; “Ensure organizations disproportionately serving individuals with social risk can compete in value-based purchasing programs”; and “Fund care delivery and payment reform demonstration projects to reduce disparities” [50, page 3] are consistent with the study PCPs recommendations. Policymakers should listen to PCP’s perspectives and work with PCPs to create a fairer quality metric system with an equitable quality-payment approach that does not perpetuate the inequitable system.