We use cookies on this site to improve our service, perform analytics, and remember website preferences. By using this site, you consent to these cookies. For more information visit our Privacy Policy

Investigating Systemic Discrimination in Human Factors Recruitment Practices

Systemic discrimination in participant recruitment for usability studies can result in medical devices that do not adequately meet the needs of all users and patients. With the desire to achieve design for equity as one of our core values, HHF investigated potential bias in our participant recruitment practices to ensure that the studies we do here are representative of real-world user populations.

Project Type

Research

Services

Research, Usability Testing

Challenge

Human Factors (HF) usability studies assess whether novel medical technologies are safe, usable, and meet intended users’ needs. Because the recruitment of study participants that represent intended user populations is an important part of any evaluation, it is critical to safeguard against discrimination in participant sampling techniques. Otherwise, study results may not reflect the behaviour and opinions of true user populations. Discrimination in participant recruitment can lead to the development, validation, and commercialization of biased medical technology, thereby disadvantaging minority groups and catalyzing health inequity.

As one of the largest healthcare-focused human factors teams internationally, HHF is in a unique position to influence the equity of medical device design globally.

Knowing how crucial it is to ensure equity in recruitment, we developed a methodology to investigate whether there was discrimination in our own recruitment practices. The methodology and results of this investigation can focus attention on the issue and how it can be investigated within HF organizations and HF recruitment agencies.

Methodology

Data collection for this evaluation began in 2020 when HHF added a voluntary Racial and Social Equity (RSE) section to our participant recruitment surveys. HHF uses recruitment surveys to procure and screen healthcare professionals and laypeople for usability studies. The RSE posed questions about ethnic and cultural origins, racial background, religious and spiritual affiliation, and gender identity. Collection of this data followed The Government of Ontario’s Anti-Racism Data Standards. Answering these questions was not mandatory, and the information provided did not influence respondents’ eligibility to participate.

Using one-proportioned, two-tailed z-tests, with a p < 0.05, data collected from the RSE was statistically compared with data from the Canadian Institute of Health Information (CIHI)’s 2022 Health Workforce Database to answer two questions:

  • Is the gender distribution of HHF’s participant pool representative of the provincial healthcare practitioner (HCP) population?
  • Is the gender and racial distribution of HHF study participants representative of the HHF participant pool?

 

After the initial evaluation, the results of which were published in the International Journal of Industrial Ergonomics, we were able to use data from Statistics Canada to continue our evaluation of Medical Doctor (MD)’s race ratios. We compared the racial distribution of HHF’s participant pool with the racial distribution of MD’s across Ontario, and were able to evaluate an additional user group (Respiratory Therapists).

unnamed image

Results

Collectively, there was no evidence of systemic discrimination in HHF’s recruiting practices. Statistical analyses indicated that there were no significant differences regarding gender between either the HCPs at the provincial level and the HHF participant pool or between the HCPs in the HHF participant pool and study participants. Additionally, there was no significant difference between the racial composition of the HCPs in the HHF participant pool and study participants.

Based on the results of this evaluation, there is no indication of systemic discrimination in the way HHF recruits individuals for the HHF participant pool, or in the way HHF selects participants for studies from its participant pool when compared to the CIHI and Statistics Canada healthcare workforce data. However, there may still be systemic discrimination within the greater population of healthcare workers in Ontario which has yet to be addressed. We will continue to audit our study participant selection to ensure that study participants are representative of individuals in the greater healthcare worker population based on these demographic characteristics.

Read the published results