Interviews and Observation
Administrator/IT interview
Optimal Performance:
- How many of the new software modules would need to be utilized for you to feel that it was a successful rollout?
- What statistics would you collect in order to determine which parts of the application process could be standardized?
- Do you feel that each committee member should be required to review a certain number of candidates in order to stay on committee? Why or why not?
- What are the major problems with the current system?
- Since the majority of the pre-admissions screeners are voluntary what is the back-up plan if they are unable to complete the process?
- In the past has screening of the applicants been done in a timely and equitable manner, if not why not?
- Is the current screening process defensible if outside constituents would evaluate it? Why or why not?
- How much pushback do you anticipate from the committee members on using the new software?
- What are some ways that you feel would facilitate IT personnel and volunteer clinician committee members working together on this problem?
- Why is the new software necessary?
- Why are software changes being considered?
- What have been the complaints and the reasons you have lost volunteers in the past?
- What studies have been done looking at the efficiency and the reliability of the current screening process?
- Have the systems utilized by other medical schools been explored?
- Have there been any outside consultants brought in to evaluate the process?
- What improvements are you specifically looking for in the pre-admissions screening process?
Committee member interview
Please list your questions as relevant next to each assignment requirement:
Optimal Performance:
Optimal Performance:
- How do you think the ideal preadmissions screening process should be done?
- Is it possible to develop a standardized screening process that maintains the diversity of the class?
- Could you take me through the step-by-step process on how you screen an applicant?
- Which parts of the application are most fruitful in your decision-making process?
- What is your comfort level with new technology when it is being implemented?
- As a volunteer committee member what requirements on your time do you consider fair?
- Do you think there should be a requirement for each committee member to do a certain number of screens in order to stay on committee?
- When data is presented showing correlation or lack of correlation to committee scores do you believe it or trust in its value?
- Each season there are a number of complaints about equity in the preadmissions screening process, how would you address some of these complaints?
- Committee members in the past used to screen applicants and will no longer do so, why do you think this is the case?
- If a few members of the committee were chosen to pilot the new software, would you volunteer to do so? Why or why not?
- What are important factors for you, in determining whether a candidate should be admitted to medical school?
- How should these factors be weighted? Why?
Observation Protocol
- What is being observed? The output from individual members of the committee is being observed. Data is being collected on each pre-admission screener including: how many people they screen, how they rate them and ultimately how many are chosen for an interview. In addition, correlation studies are being done between individual components of the application and ultimate acceptance decisions in order to determine if there is a correlation and if so is it positive or negative. During the workshop each new member of the committee will be observed rating an applicant and be given immediate feedback by a more senior member of the committee.
- When and for how long observation will occur? The observation will continue throughout the current season and data added as each applicant is screened as well as after each committee meeting.
- Purpose of observation? The purpose of the observation is to collect data to see if there are parts of the applications that could use technology to score while still maintaining diversity. The first step is to gather reactions from the committee members on the data that is collected and to get committee buy in that if there is no correlation and it has been determined that a piece of data contributes X amount of variability to the committee score then this portion of the evaluation could be standardized. In addition, any time a new software system is being implemented that requires the time of busy clinician volunteers care must be taken that the process is as painless as possible and that committee members feel that their concerns are being heard.
- Specifically, what is being looked for?
- Feelings about the current process as well as solutions
- Evaluation of data and the correlation to outcome
- Feelings about the use of technology
- Integration of new committee members with old
- How well the IT people will work with the volunteer committee members
- Whether the admissions coordinator will be able to use the new software
- Whether there is adequate staffing in the admissions office.
- Benefits of observation?
- I am new in my position. The position has had fairly high turnover and I think there is a need to evaluate how things are currently done before making any suggestions for changes.
- There are many constituents with different focuses and conflicts can develop.
- The members of the admissions are all volunteers and if the process feels difficult or cumbersome they may choose to walk away.