Medical students interested in obtaining a radiation oncology residency position often use the internet as a resource to gather information about residency programs. Although there have been no surveys specific to radiation oncology applicants, studies in other medical specialties have confirmed the importance of online program information.1-8 Therefore, it is important for residency programs to maintain informative and comprehensive websites for prospective radiation oncology applicants.
Prior studies in other medical specialties have demonstrated that residency program websites are often suboptimal and that missing information can be crucial for applicants to determine which programs are a better “fit” for them.9,10 Given that program websites may be the only novel program-specific resource medical students have before applying, completeness of program information may be a significant factor in allowing residency programs to remain competitive for applicants, particularly with the recent increase in unmatched radiation oncology program spots in 2019.11
Previous studies in other medical specialties have found deficiencies in online content on residency program websites; however, similar information regarding the availability and quality of current radiation oncology residency website content is not available. We sought to evaluate the accessibility of educational and recruitment content of radiation oncology residency websites in the US.
Methods and Materials
A list of US radiation oncology residency programs was obtained from the Electronic Residency Application Service (ERAS). All websites were publicly available. This study did not require institutional review board approval per the SUNY Downstate Medical Center.
The program websites were accessed through the link provided by ERAS or through an online search. Websites were evaluated for the presence of 20 unique features related to categories of application process, work incentives, educational instruction, research, clinical training, and program leadership introduction (Table 1). The 20 features were derived from published studies in other specialties evaluating residency websites and were considered to be relevant to the field of radiation oncology by the authors.4,5,12 Inclusion of information required its presence directly on the radiation oncology residency or department website. However, information on salary, benefits, parking, and faculty listing was considered present if it was accessible by a direct link from the residency website.
| Feature | Percent of Programs with Feature Included on Website |
|---|---|
| Application Process | |
| Contact e-mail | 98 |
| Link to ERAS | 54 |
| Number of spots for match | 33 |
| Selection criteria | 32 |
| Work Incentives | |
| Benefits | 41 |
| Information on surrounding area | 41 |
| Salary | 36 |
| Parking information | 9 |
| Educational Instruction | |
| Description of didactics | 63 |
| Research | |
| Research rotations/opportunities | 86 |
| Active/past research projects | 58 |
| Clinical Training | |
| Comprehensive faculty listing | 82 |
| Equipment description | 76 |
| Current residents | 73 |
| Rotation schedule | 51 |
| Medical student electives | 47 |
| Alumni job placement | 42 |
| Call schedule | 15 |
| Introduction to Program Leadership | |
| Message from program director | 36 |
| Message from chairperson | 24 |
Programs were organized by geographic location and residency size for further analysis. Programs were divided into institutions based in the Northeast (Maryland, Pennsylvania, Delaware, New Jersey, New York, Connecticut, Massachusetts, Vermont, Rhode Island, New Hampshire, Maine, and the District of Columbia), South (Virginia, Kentucky, Arkansas, Oklahoma, Texas, North Carolina, South Carolina, Georgia, Florida, Louisiana, Mississippi, Alabama, Tennessee), West (New Mexico, Colorado, Washington, Oregon, Nevada, Utah, Arizona, California, Hawaii), and Midwest (Nebraska, Kansas, Minnesota, Iowa, Missouri, Wisconsin, Illinois, Michigan, Indiana, Ohio, West Virginia). States/territories without programs included Alaska, Wyoming, Montana, Idaho, North Dakota, Puerto Rico, and South Dakota. Program size was determined by the median number of residents in the programs, with ≤ 7 residents considered to be small and ≥ 8 considered large. Programs were then categorized by the ranking of the cancer program per US News and World Report as a “Top 25” vs a “Not Top 25” program and further categorized using the National Resident Matching Program data as having filled or not filled in the 2019 cycle. Chi-square and Mann-Whitney/Kruskal-Wallis tests were used to compare categorical and continuous variables, respectively. Univariate logistic regression was performed to assess predictors of matching in the 2019 cycle. SPSS version 21.0 (IBM Inc., Armonk, New York) was used for statistical analysis.
Results
A total of 94 radiation oncology residency programs was obtained from ERAS. Of the 94 programs, 92 websites were evaluated (2 websites were inaccessible or nonexistent). Individual program websites contained a mean (SD) of 9.75 (3.8) of the 20 factors sought (49%).
Only 10 (11%) of the program websites had at least 75% of the 20 features. In addition, 37 (40%) programs had < 50% of the features listed on their websites. Websites ranged from having 15% of the features to as much as 90%. Most sites had contact e-mail (98%), research opportunities (86%), and a comprehensive faculty listing (82%), while fewer than a quarter of programs had a message from the chairperson (24%), call schedule (15%), or parking information (9%) (Table 1).
After organizing by geographical location, the Northeast contained 29% of programs, the South 29%, the West 19%, and the Midwest 23%. There were no differences in the amount of information available on each website based on geographic region (p = 0.102). There were 45 large programs and 47 small programs, and there was no significant difference in mean number of features reported by large or small programs (10.90 vs 9.15, p = 0.114). Comparison of Top 25 vs Not Top 25 programs by the US News and World Report found no difference in mean number of features reported (9.96 vs. 9.68, p = 0.760) (Table 2).
| Feature | Number of Features on Programs’ Websites, Mean (SD) | p-value |
|---|---|---|
| Overall | 9.75 (3.8) | |
| Size of program | 0.114 | |
| Small | 9.15 (3.7) | |
| Large | 10.80 (3.2) | |
| Location of program | 0.102 | |
| Midwest | 11.32 (3.2) | |
| West | 10.83 (3.3) | |
| Northeast | 8.93 (3.9) | |
| South | 8.56 (4.0) | |
| Ranking | 0.76 | |
| Not Top 25* | 9.68 (4.05) | |
| Top 25 | 9.96 (2.96) |
There were 88 programs that entered the NRMP in 2019 of which 22 (25%) did not fill all positions. On univariate logistic regression, large programs were more likely to fill their spots in the 2019 match (OR 3.85, p = 0.013) and there was a nonsignificant trend in increased likelihood of matching with 6 to 15 features on program websites (OR 2.07-2.14)(Table 3).
| Feature | OR (95% CI) | p-value |
|---|---|---|
| Number of website features | ||
| 0-5 | 1 | |
| 6-10 | 2.07 (0.41-10.36) | 0.378 |
| 11-15 | 2.14 (0.41-11.26) | 0.368 |
| 16-20 | 1.00 (0.13-7.57) | 1 |
| Size of program | ||
| Small | 1 | |
| Large | 3.85 (1.34-11.11) | 0.013* |
| Location of program | ||
| Midwest | 1 | |
| West | 1.77 (0.28-11.04) | 0.544 |
| Northeast | 0.38 (0.10-1.45) | 0.155 |
| South | 0.71 (0.17-2.95) | 0.633 |
| Ranking | ||
| Not Top 25* | 1 | |
| Top 25 | 2.56 (0.68-9.67) | 0.166 |
Discussion
The 2019 Match Day results, with 22 programs (25%) going unfilled for the first time in many years, was a surprising development after years of a competitive match in radiation oncology. The decline in applications is likely multifactorial, with causes including an anticipated future oversupply of radiation oncologists and a much higher-than-usual failure rate on the 2018 radiation biology and physics qualifying board examinations.13,14
As future generations of medical trainees undoubtedly will continue to use the internet as a resource for investigating residency specialties and individual programs, having comprehensive program websites will continue to grow in importance. In this study, we evaluated current program websites based on 20 criteria and note that on average, programs met about half of these predefined criteria. Furthermore, the geographical location and program size were not related to website completeness, implying that the issue is widespread and overlooked among a variety of programs.
While the deficiencies in online content available for other medical specialties has been reported extensively,1-8 information on availability and quality of online information regarding radiation oncology programs is limited.12 We found that important information for prospective residents, including the number of match spots, selection criteria, and alumni job placement, is omitted in the majority of websites. Previous studies have noted that websites influence prospective applicants’ decisions5,7 and that an easily navigable site may be an important factor in deciding where to apply.3 The lack of information on radiation oncology residency websites may leave applicants with insufficient information with which to gauge their interest in a particular program.
It is likely that another resource programs use is social media, which may be used in recruitment. A survey study of prospective anesthesia residents showed that the majority (52.8%) felt a residency-based social media account impacted their evaluation of programs. Specifically, the most popular platforms included Doximity and Facebook.15 With an increasing Twitter presence in oncology,16 its utilization by programs may also be an emerging trend in resident recruitment.
This study has several limitations. First, the choice of program website features by the study team was completed through extensive literature review of desired features in other specialties as well as consensus on factors specific to radiation oncology; however, additional factors of interest to medical students may not have been included. Second, due to website variability, available features may have been overlooked despite thorough review. There was also no official way to verify the accuracy of the information posted on the websites. Furthermore, intangible factors such as website design and ease of use were not assessed in this study. Nonetheless, these results highlight several areas for potential improvement.
Conclusion
The recent match results indicate that individual residency programs, and even our field as a whole, cannot be complacent when it comes to attracting the best medical students. We demonstrate that residency program websites, a medical student’s first and sometimes final look at a program, often lack completeness. Enhancing the quality and completeness of residency program websites may be a very high-yield first step toward optimizing future matches and reversing the recent concerning increase in unfilled radiation oncology residency spots.
