When Angela Walker first heard of the Supreme Court’s decision to outlaw affirmative action in college and university admissions, she found it disheartening. Now a medical director in Georgia, Walker credits part of her educational success to affirmative action and the opportunities made to uplift Black medical students who were not as privileged as other classmates.

“I know how important it was for me and because my field of medicine is gynecology, I know how important it is for African American maternal morbidity,” says Walker, who co-wrote the book The Game Plan: A Woman’s Guide to Becoming a Doctor and Living a Life in Medicine.

Disheartening is the same term Walker now uses when contemplating the future of medicine. Affirmative action was a policy intended to ensure equal opportunity for minority groups historically underrepresented in undergraduate and graduate programs across the US. Medical schools are expected to see less enrollment of Black and other minority students if there is a less diverse pool of students to choose from. This could potentially trickle down to having fewer healthcare providers of color practicing medicine.

Multiple studies have shown that patients prefer their doctor to be the same race or ethnicity as them. People of color in the US have had a long history of mistrust in medical institutions—and for good reason. The Tuskegee Experiment, where Black men were intentionally infected with syphilis, left a scar 50 years ago. Members of the Havasupai Tribe in Arizona who donated blood samples for diabetes research had their DNA tested for other health factors without their consent. The US used Puerto Rico women for birth control trials they did not consent to and forced sterilizations. Black inmates at a Philadelphia prison were forcibly exposed to viruses, fungus, and other chemical agents for skin care research.

Having someone who looks like you, shares a similar cultural background, or knows your native language can ease some of the anxiety about going to the doctor, Walker says. It also helps with building rapport and trust. When providers are the same race or ethnicity, people are more likely to listen to advice, take their medication, and feel like their doctor is looking out for their best interests than dismissing their symptoms. “Increasing representation and diversity in people who are traditionally underrepresented in medicine saves lives,” Walker says.

[Related: Dialysis is riskier for Black and Latino patients, CDC says]

Choosing a medical professional that matches your own needs can be a challenge for anyone who’s marginalized. Only 5.7 percent of US doctors are Black—less than double the amount needed to treat 12.1 percent of Black people living in the country. While a little higher, Latinos make up 7 percent of US doctors. The shortage of Spanish-speaking doctors can make it harder to communicate with patients and have them feel comfortable enough to express their health concerns. Finally, Native Americans have one of the lowest rates of representation in medicine at just 0.4 percent of the physician workforce

Research suggests affirmative action directly contributed to more diversity in healthcare, including increasing the number of medical practices in underserved areas. And affirmative action bans are directly doing the opposite. In states like Florida and California with affirmative action bans, there has been a 17 percent decline in people of color enrolling in medical schools. The authors of the review suggest this is likely creating less culturally competent doctors—and further increasing health disparities among racial minority groups.

One popular argument critics of affirmative action use is that it allows students with lower grades into competitive programs. Responses like this completely ignore the layers of barriers that people of color face to get to medical school, says Christine Eady Mann, a family practitioner in Texas and cofounder of Doctors In Politics. “It’s not about being smart—success in life doesn’t equate to being intelligent.” She counters that not everyone has the privilege to have access to premed prep courses or mentors growing up. These gripes also fail to account for other factors that could speak to an applicant’s merits, like volunteer work and extracurriculars.

The overturning of affirmative action is a setback, but Mann is hopeful it’s a temporary one. “There are so many barriers to equity [in healthcare], and this is just one more piece that is going to have to be worked around.”

Even if affirmative action was reinstated in the future, the lack of diversity in medical schools right now can have decades-long effects on the entire healthcare system. According to one business psychologist, it reduces the opportunity for different perspectives and innovation in treating patients of many backgrounds. What’s more, the absence of minority doctors can worsen patient outcomes if there is no cultural competence training or experience. “We all rise when we lift the people who are not getting the best care,” says Mann. “This ruling will make the entirety of the healthcare system more difficult for everybody.”

[Related: Edward A. Bouchet paved a path for generations of Black students]

Colleges and universities may no longer be able to use affirmative action in their decision making, but there are other ways to support students who are historically underrepresented in medicine. One way is voting and supporting elected officials with ideas on how to close the health equity gap. Another is mentoring. Walker says she always tries to guide young women from minority backgrounds who are interested in medicine—creating this opportunity encourages these prospective doctors to see what it’s like on the job and ask questions about navigating medical school. 

The burden of equalizing medical education now falls on the community, including on doctors who might have benefited from affirmative action in the past 60 years. Walker has one piece of advice for those on the front line: “We need to continue supporting diversity in medicine even if that means having to do it ourselves.”