The Selection Committee continues to review and refine the RANZCOG Trainee Selection process, which has become increasingly competitive as medical school placements and postgraduate numbers have increased. For the past few years, the College has received a minimum of three-times the number of applicants than available training positions. Inevitably, this means that very good candidates miss out on selection causing not only disappointment to the candidates themselves but also to those Fellows and members who support them in their endeavours to obtain a place on the FRANZCOG Training Program.

Finding the right method or tools to ensure the right candidates are selected for training is challenging; all medical schools and specialist colleges grapple with this issue. RANZCOG has recently trialled and/or implemented several initiatives to assist in this process and to address training needs and workforce shortages in regional and rural Australia.

In 2015 and 2016, the College introduced changes to some of the processes including:

• measures to address the issue of high transfer requests from some locations
• limiting the number of regions/states for which candidates can apply
• limiting the overall number of times candidates can apply
• the approval of three Provincial Integrated Training Programs (PITPs)
• the introduction of institutional references, including hospital ranking and recommendation, to assist the shortlisting process
• piloting of situational judgement tests (SJTs) as a potential shortlisting tool.At the November 2016 meeting, the Selection Committee reviewed this year’s processes, including feedback received, before making their final recommendations to the Board for the 2017 process.

Applications

The RANZCOG Trainee Selection Process is applicable in both New Zealand and Australia. The New Zealand selection process commences in February and is completed by the end of May, to allow trainees to begin their training year at the start of December. The Australian selection process opens in April and is completed in August, ready for the training year which starts in February of the following year.

In New Zealand, all shortlisted candidates are interviewed in Wellington over two days for all positions in all regions. In Australia, all shortlisted candidates are interviewed on the one day in the state of their first preference. In 2015 and 2016, interviews were held in Brisbane, Sydney, Melbourne, Hobart, Adelaide and Perth.

In recognition of increasing numbers of Integrated Training Program (ITP) transfer requests in previous years from certain locations, the Selection Committee introduced a separate preference phase for the newly approved Provincial ITPs as well as for Canberra, Newcastle and Tasmania. Shortlisted applicants for these ITPs were interviewed on the same day by the same panels with additional relevant questions for those applying for the PITPs. Applicants successful in being offered one of these ITPs, accept the offer with full knowledge that transfer requests to other ITPs will not be considered in subsequent years.

In addition to the individual ITP preferences outlined above, applicants are able to preference up to three states in which they would be happy to train. Applicants who list only one state because they are not willing and/or able to move interstate, can limit their selection chances if they are not highly ranked; however, they are less likely to spend the next few years trying to return to their state of first preference. In 2016, all but two candidates were offered their state of first choice in Australia.

The rule restricting the number of applications has also been introduced. From 2016, applicants are permitted to apply for the FRANZCOG Training Program a maximum of three times. This cap has not been retrospectively applied, so that 2016 marks the ‘first’ application by all candidates.

Establishment of a provincial training pathway

The College recently established a PITP for RANZCOG Core Training in an effort to attract those medical practitioners who have a demonstrated commitment to rural health, to be able to undertake their specialist O&G training and pursue a career in a regional area. The first PITP trainee commenced in February 2015 at Dubbo, the second at Orange in 2016, and one trainee will commence at Dubbo and one at Mackay in 2017. Trainees undertaking a provincial pathway are required to spend three out of their four years of core training at the accredited provincial site(s) and their remaining year in a major metropolitan teaching hospital. The selection process follows the standard process with the addition of a supplementary application that uses additional selection criteria which are shown to increase the likelihood of long-term practice in provincial areas.

Shortlisting

The selection process involves ranking of all applicants. Shortlisting for interview has traditionally been based on the scores received for the CV/Application and Referee Reports (equally weighted) and final selection has been based on the scores received for the CV/ Application, Referee Reports and the interview (weighted as 25 per cent, 25 per cent and 50 per cent, respectively).

It has been acknowledged that the current process of shortlisting for interview, using only the CV and the applicant-nominated referees, has failed to adequately discriminate, leading to applicants who may have been considered highly suitable for interview and training missing out by very small margins. This is understandable given the number of applications received, the high calibre of applicants, and that scoring and weightings are published on the website as required by the Australian Medical Council.

While publication of a scoring system increases transparency, it also means that applicants work hard to gain the maximum marks in categories such as presentations, publications, attending relevant courses, experience as an unaccredited O&G registrar and so forth, which further reduces discrimination on the CV/application component. The individual referee reports have also failed to discriminate well. In 2015 and 2016, approximately 80 per cent of applicants received 85 per cent or greater for their total score for this component.

With an aim to better discriminate and assist the shortlisting process, the Selection Committee introduced Hospital Ranking or Institutional References (IR) as an additional component to the 2015 and 2016 selection process. As past performance is considered to be a good predictor of future performance, hospitals are well placed to distinguish between applicants on the basis of how they actually perform in the workplace. To this end, hospitals were asked to recommend and rank applicants on their comparative relative merit, knowing the applicants’ surgical aptitude and professional attributes and behaviours. The IRs were used in conjunction with the ranking obtained from the CV/application and applicant-nominated references to assist with the shortlisting process.

All hospitals where an applicant had completed 10 weeks or more in a prevocational O&G position at their hospital in the previous two years, were asked to rank the applicant in terms of their suitability for the FRANZCOG Training Program. The College asked that the consensus ranking be completed by the relevant hospital ITP Coordinator/Training Supervisor/Head of O&G, after discussion with and input from a range of O&G consultants and/ or senior registrars. Although high ranking by a hospital translated into an increased chance of being shortlisted for many preferred candidates, it did not guarantee selection onto the training program as scoring from the interview, CV/application and individual referees all contribute to the published process.

Interviews

Regional Training Accreditation Committees increased their panel numbers for 2015 and 2016, which enabled the College to shortlist and interview approximately two-thirds of the total number of applicants in both years. However, as the numbers of postgraduate doctors move through the system and compete for places on the training program, the College is unlikely to be able to accommodate any higher numbers at interview; hence the need to investigate other tools, methods or processes that will assist in selecting the best candidates with the resources available.

SJTs

The College piloted SJTs with all shortlisted applicants in New Zealand and Australia in 2015, and with all applicants in 2016. SJTs are designed to assess an individual’s judgement regarding situations encountered in the workplace. Applicants are presented with a set of hypothetical basic work-based scenarios and asked to make judgements on possible responses. Applicant responses are evaluated against a predetermined scoring key to provide a picture of their situational judgment in that particular context. An individual SJT question can assess several attributes per scenario, such as interpersonal and communication skills, problem solving and teamwork, empathy and professional integrity, clinical reasoning and coping with pressure. In longitudinal studies, SJTs have been shown to have a high correlation with OSCE performance and are therefore used for their predictive ability.

The College engaged the World Psychology Group (WPG) to oversee both pilots as they have extensive experience in the field of selection and evaluation, and in particular in the design and evaluation of SJTs in high-stake settings, both here and overseas. The results from the pilots did not contribute to the scoring for the 2015 or 2016 selection process. However, the psychometric analysis that was conducted following both allocation processes will be considered by the Selection Committee at the November 2016 meeting to help inform any recommendations regarding their future use.

We are very cognisant of, and grateful to, the many College Fellows and senior trainees who give their time and expertise to various aspects of the selection process – from assisting applicants with their CV/applications, providing guidance for mini research projects, holding practice interview sessions, completing references, contributing to IRs, writing and/or reviewing scenarios for the SJT questions, volunteering to sit the SJT under test conditions to ensure concordance with the question reviewers, interviewing shortlisted applicants in state-based panels, and finally in counselling applicants who may miss out because of the highly competitive field. Almost every hospital has excellent potential trainees, so it is inevitable and unfortunate that some will miss out despite the efforts, refinements and tools adopted. The College is considering how hospital knowledge and feedback about potential new trainees can be further incorporated into the selection process. Feedback is always welcome.