The College commissioned additional external analyses of the results of the 2015 Registrant Competence Assessment (RCA) in order to look into three issues about which registrants have requested more information or have expressed concern. The results are summarized below and include next steps.
1. Were there topics on the RCA that registrants (as a group) performed significantly better or worse on than others?
Topics on the RCA correspond to the roles and key competencies in the Essential Competency Profile for Physiotherapists in Canada. For this analysis, registrants’ performance on questions corresponding to the same key competency was analyzed.
- For five of the Essential Competency roles (Expert, Communicator, Collaborator, Manager, and Professional), as a group, registrants performed above the standard of minimal competence, and there was no significant difference in performance in any of the key competencies. For example, there are 8 key competencies within the Expert role, and as a group, registrants performed well on the items blueprinted to those key competencies, with no significant differences seen. Similarly, registrants performed well on the 3 key competencies of the Communicator role, etc.
- Two of the Essential Competency roles- Advocate and Scholarly Practitioner- had too few items in the 2015 RCA to allow reliable analysis.
Next steps: Given that there were no significant differences in the group performance by topic, nothing further will be done on this question right now.
2. Was there a significant difference in registrants’ performance in cases having different practice settings?
Within one exam, there are cases from each of the three settings (facility, clinic, community). There were mixed results from this analysis but there was nothing to cause the Quality Assurance Committee (QAC) to make changes before the 2018 RCA administration.
a. Consider comparing registrants’ performance on items within the same practice setting as their workplace and outside of that practice setting. For example, for registrants who report working in facilities, was their performance on items set in facilities better, worse or the same as their performance on items taking place in the community and clinic settings?
b. Repeat the original analysis after the 2018 RCA administration; combining data from both 2015 and 2018 should improve the reliability of the analysis.
3. Was there a significant difference in performance by clinical practice area in the Child and Youth practice context?
The Child and Youth practice context is different from the three adult/older adult contexts in that it includes all three clinical areas: cardiorespiratory, musculoskeletal, and neuromuscular. The results of this analysis were also mixed, and again there was nothing to cause the QAC to make changes before the 2018 RCA administration.
a. Repeat the original analysis after the 2018 RCA administration; combining data from both 2015 and 2018 should improve the reliability of the analysis. This is particularly the case as only 82 registrants wrote the Child and Youth exam context in 2015, and thus the sample size of registrants and items (cases and questions) is relatively small.
b. Review and revise the Quality Assurance Program Blueprint as indicated by current practice. This is a lengthy process that needs to happen regularly; any changes would come into effect for the second RCA cycle, which will begin in 2021.