Phone: (416) 234-8800
Fax: (416) 234-8820

What's New

May 2, 2008
Client Services Coordinator

Physiotherapy Competency Examination FAQs

What purpose does the examination serve?

  1. What does the examination aim to accomplish?
  2. What role does the exam play for provincial physiotherapy regulators?
  3. What roles do the Clinical and Written Components each play?

Who takes the examination?

How are passing scores set?

  1. What is the passing score for the Written Component?
  2. What is the passing score for the Clinical Component?
  3. Total score
  4. Number of stations
  5. Critical incidents
  6. How are station scores calculated?
  7. How is the total score for all stations calculated?
  8. How is the passing score determined for the total score criterion?
  9. How are the passing scores determined for each station?
  10. How are the passing scores confirmed?
  11. What information does the Board of Examiners consider when making decisions about passing scores?
  12. How are critical incidents considered in scoring?
  13. Why does the number of stations passed matter when a candidate has achieved a passing score on the total score?
  14. Why are scores reported on a standard score scale?
  15. How is the standard score determined?
  16. I have heard that sometimes a station is deleted from scoring. Why does that happen? How is the decision made?

What are the psychometric features of the examination?

  1. What is validity?
  2. What is the evidence for the validity of the results of the PCE?
    1. Content aspect of validity:
    2. Construct aspect of validity:
    3. Structural aspect of validity:
    4. Generalizability aspect of validity:
    5. External aspect of validity:
    6. Consequential aspect of validity:
  3. What is the reliability of the physiotherapy competency examination?
  4. How can the scoring of the written stations be consistent when so many different people are doing it? What quality control checks are put in place to ensure consistency of scoring written test sheets?

How do candidates tend to perform on the examination?

  1. Why do the pass rates on the Clinical Component vary from examination to examination?
  2. Why doesn't the alliance adjust the pass rate so the same percentage pass each time?
  3. How and why do graduates of different Canadian physiotherapy programs perform differently in the examination?
  4. Why is the pass rate lower for non Canadian-educated physiotherapists?

Who does what for the examination program?

  1. Does The Alliance decide that a candidate is not eligible to be registered/licensed?

How is the physiotherapy competency examination developed?

  1. Who decides what goes in the examination?
  2. What is the examination blueprint?
  3. Who writes the questions?
  4. Who approves the questions?

How is the quality of the examination maintained?

  1. What is the Monitoring and Evaluation program?
  2. What research is being done related to the examination?
  3. What are the quality assurance processes related to the examination?
  4. Who oversees the Monitoring and Evaluation program?

How are examination materials translated?

  1. Who participates in the translation process?
  2. What are the steps to translate and check exam questions?
  3. Are examination scores for French and English candidates reviewed?
  4. How is the Written Component reviewed?
  5. How is the Clinical Component reviewed?
  6. What translation reference material does The Alliance use?

Glossary

  1. Angoff method
  2. Board of Examiners
  3. Content domain
  4. Exam Consultants
  5. Inferences
  6. Question
  7. Reliability
  8. Standard score
  9. Validity

What purpose does the examination serve?

The Short Answer: The Physiotherapy Competency Examination (PCE) has been developed to help provincial and territorial physiotherapy regulators determine a candidate's readiness for independent practice.

  1. What does the examination aim to accomplish?

    The PCE aims to ensure the safety of the public in interactions with physiotherapists. It does so by effectively determining whether or not a physiotherapist candidate has demonstrated the minimal standard of physiotherapy practice.

    The exam is a fair and accurate assessment of candidate readiness for safe, effective, independent physiotherapy practice. It is designed to assess the knowledge, skills and abilities required for practice. These include history taking, physical examination, data interpretation, clinical problem solving, intervention skills, ethics, safety and communication. The examination also covers the core clinical practice areas of musculoskeletal, neuromuscular and cardiorespiratory.

    A pass result on the examination confirms that the candidate has demonstrated the minimal standard of knowledge, skills and abilities. A fail result indicates that the candidate has not yet demonstrated the minimal standard of knowledge, skills and abilities.

  2. What role does the exam play for provincial physiotherapy regulators?

    The provincial/territorial regulators have a mandate to protect the public. They can do this best by ensuring that a consistent and defensible standard is applied to all candidates for registration, regardless of where they have completed their accredited physiotherapy program. The PCE is a reliable and valid assessment tool that generates an unbiased assessment on a wide-ranging set of competencies.

  3. What roles do the Clinical and Written components each play?

    The Written and Clinical Components provide different information about a candidate's knowledge, skills and abilities.

    The Written Component assesses a broad base of physiotherapy knowledge in the areas of musculoskeletal, neuromuscular and cardiorespiratory physiotherapy. The skills assessed by the Written Component include application and integration of clinical knowledge and clinical problem solving through the use of clinical scenarios. A candidate must achieve a minimum overall score in order to pass the Written Component.

    The Clinical Component assesses a candidate's understanding and demonstration of safe, effective application of the principles and processes of physiotherapy practice. The knowledge, skills and abilities assessed include communication skills and professional behaviours.


Who takes the examination?

The Physiotherapy Competency Examination (PCE) is taken by Canadian-educated and non Canadian-educated graduates of physiotherapy programs, regardless of their experience. Successful completion of the examination is required by most provinces/territories before a license to practice physiotherapy is granted by a provincial/territorial regulator.

Non Canadian-educated graduates of physiotherapy programs must have their credentials assessed before they are eligible to take the PCE. Credentialling ensures that the applicant's education is substantially equivalent to the minimal educational requirements for a Canadian-educated physiotherapist.


How are passing scores set?

The Short Answer: The process of setting passing scores for the examination is quite detailed. Each component has distinct passing scores.

  1. What is the passing score for the Written Component?

    The passing score for the Written Component is a specific score on the Standard Score scale. The Board of Examiners sets the passing score for the Written Component.

    The Written Component Test Construction Committee completes a periodic review of the passing score and recommends adjustment to the passing score if necessary.

  2. What is the passing score for the Clinical Component?

    Candidates must meet three criteria to pass the Clinical Component:

    • Achieve or exceed the minimum total score required;
    • Achieve a passing score on a minimum number of stations;
    • Demonstrate an overall level of safe, professional practice appropriate to physiotherapy.
  3. Total Score

    The Total Score criterion provides an indication that a candidate has demonstrated an overall minimal standard over a range of knowledge, skills and abilities. The Total Score does not provide information about specific areas or functions in which a candidate may have gaps in knowledge, skills or abilities. To pass on Total Score the candidate must achieve or exceed a minimal score.

  4. Number of Stations

    The Number of Stations criterion provides an indication that gaps in knowledge, skills and abilities are not so frequent or systematic as to cause concern that the minimal knowledge, skills and abilities are not met. It ensures that candidates demonstrate reasonably consistent knowledge, skills and abilities from station to station by eliminating the possibility of passing the Clinical Component by doing well in some stations and badly in others. The Number of Stations criterion thus provides a balance to the Total Score criterion. To pass on Number of Stations the candidate must have a passing performance on a minimum number of stations.

  5. Critical Incidents

    The Critical Incidents criterion documents issues that would cause an examiner to have concern about the safety and/or professional knowledge, skills and abilities of a candidate. This criterion provides a way for examiners to identify potentially unsafe or unprofessional actions or behaviours. To pass on Critical Incidents the candidate can have no more than 2 minor safety or professionalism violations, and can have no major safety or professionalism violations. This criterion has recently been reviewed through the Safe Professional Practice Criterion Study¹ and was found to add value to the overall process of the Clinical Component.


    ¹ Canadian Alliance of Physiotherapy Regulators. (2002). Report on the use of the safe professional practice criterion in the Clinical Component of the Physiotherapy Competency Examination. Toronto: Author.

  6. How are station scores calculated?

    For ten-minute stations, the station score is calculated from the checklist items done correctly (80% of the station score) and the global and communication ratings (20% of the station score).

    For the five-minute (couplet) stations, the station score is calculated from the checklist items done correctly (40% of the station score), the global ratings (10% of the station score) and the written station score (50% of the station score).

  7. How is the total score for all stations calculated?

    The total score is the average of the station scores. Candidates must achieve or exceed a minimum score to pass on this criterion.

    The total score criterion provides an indication that a candidate has demonstrated an overall minimal standard over a range of knowledge, skills and abilities. The total score does not provide information about specific areas of practice or functions in which a candidate may have gaps in knowledge, skills or abilities. It is therefore considered to be a "compensatory" score, because a candidate can compensate for poor scores in some station with good scores in other station.

  8. How is the passing score determined for the total score criterion?

    The passing score for the total score criterion is the average of all the station passing scores plus one standard error of measurement.

    The standard error of measurement (SEM) is a statistical term. It is calculated for each examination. The SEM is an estimate of the amount of random error in the measurement process. Adding one SEM to the scores decreases the chance that a candidate will pass the exam when he or she should not pass. This is the same method used by the Medical Council of Canada for its Objective Structured Clinical Examination (MCC QE Part II).

  9. How are the passing scores determined for each station?

    For each of the 16 stations, the examiners assign an over-all rating for each candidate on a 6-point scale (Satisfactory - excellent, satisfactory -good, satisfactory - borderline, unsatisfactory - borderline, unsatisfactory - poor, unsatisfactory - unacceptable). Examiners are required to provide reasons for rating a candidate's performance as unsatisfactory.

    The passing score for each station is the mean score of all candidates who received a Satisfactory-Borderline or an Unsatisfactory-Borderline overall rating on that particular station.

  10. How are the passing scores confirmed?

    The Board of Examiners reviews the consultant's Technical Report after each administration of the examination and sets pass/fail levels for total score, number of stations, and Critical Incidents, according to the Board of Examiners' Decision Rules.

  11. What information does the Board of Examiners consider when making decisions about passing scores?

    The Board of Examiners (BOE) considers:

    • Descriptive statistics for each examination item and the overall examination (number of candidates, mean, median, standard deviation).
    • Correlations between each exam item (i.e., station) and total exam score. The correlation coefficients indicate the extent to which a station is contributing useful information to the overall score. Correlation coefficients theoretically range from -1.0 to +1.0. The BOE considers a correlation coefficient of +0.20 or higher as acceptable (i.e., contributing useful information to the total score).
    • Reliability and reproducibility coefficients.
    • A comparison of the degree of difficulty estimates (Angoff estimates) for each station with the actual performance (mean score) for that station;
    • Trends in mean station scores for repeated stations (i.e., mean scores for stations used on this administration compared to previous uses of the same stations). This information helps to determine the consistency of candidate performance from one exam to the next;
    • The performance of Canadian-educated physiotherapists completing the examination for the first time in this administration and in comparison to past administrations.
    • The comparison of key indicators between the examination being scored and previous examinations. These indicators include the mean of Angoff estimates, the mean candidate scores, and BOE decisions on past examinations. The impact of various passing scores on the passing rates is also informative in understanding the consequences of different passing scores.
    • A report on any administrative or statistical issues affecting scoring (i.e., if any items should be eliminated.)
  12. How are Critical Incidents considered in scoring?

    The Board of Examiners reviews all Critical Incidents identified by examiners and makes a final decision about whether each identified incident is a safety or professional violation. The decision takes into consideration multiple factors including accuracy and completeness of the documentation by the examiner, planned portrayal by the standardized client, relevant literature, past decisions on similar critical incidents, and the professional judgement of the Board of Examiners members.

  13. Why does the number of stations passed matter when a candidate has achieved a passing score on the total score?

    The number of stations criterion is the number of stations in which the candidate achieved or exceeded the passing score for the station. Candidates must achieve or exceed the passing score in a minimum number of stations.

    The number of stations criterion ensures that frequent or systematic gaps in knowledge, skills and abilities are identified. The number of stations criterion provides a balance to the total score criterion. It ensures that candidates demonstrate reasonably consistent knowledge, skills and abilities from station to station through elimination of the possibility of passing the Clinical Component by doing well in some stations and badly in others. The number of stations criterion is therefore a "non-compensatory" score. It provides a check on candidates who achieve a passing continuous score, but who do not demonstrate the minimal knowledge, skills and abilities in a number of stations.

    Candidates who fail on this criterion often state that they failed the exam "by one station". In fact, these candidates have failed several stations, indicating that there are demonstrated gaps in their knowledge, skills and abilities.

  14. Why are scores reported on a standard score scale?

    Scores are converted statistically to a standard score before examination results are reported to candidates. This conversion ensures comparability between examination administrations.

  15. How is the standard score determined?

    The standard score is determined by calculating a Z score for each candidate. The Z score is then linearly transformed to a scale with a mean of 500 and a standard deviation of 100.

  16. I have heard that sometimes a station is deleted from scoring. Why does that happen? How is the decision made?

    Elimination of a station from scoring is very uncommon. Reasons for eliminating a station include poor statistical performance of the station, problems with the administration of the station or evidence of a breach of the security of the station prior to the examination.

    After each administration of the examination, the Board of Examiners (BOE) reviews the preliminary technical report on the examination. This report provides information on the performance of the stations on the examination and the performance of the examination in general. If there are problems with the statistical performance of a station, or if there were operational or security problems with a station, the BOE may direct that the station be eliminated from scoring.


What are the psychometric features of the examination?

The Short Answer: Each step in the development and ongoing maintenance of the Physiotherapy Competency Examination (PCE) has been guided by the best available evidence in measurement and evaluation research literature. The PCE is a model of a well-developed high-stakes licensure examination.

  1. What is validity?

    The validity of test results is the extent to which the scores measure the attribute(s) of interest. In the assessments used in making licensure decisions for health care professions, we intend to measure the competence of individuals practitioners entering practice. In other words, do candidates who successfully complete the PCE have the competence to perform at the desired level of proficiency? And secondly, do candidates who are not successful lack or fall short of the necessary level of proficiency?

  2. What is the evidence for the validity of the results of the PCE?

    Validity has to be built into an examination program. In licensure testing, this is generally done by conducting a rigorous review of the competencies required for safe and competent practice (The Analysis of Practice¹). These competencies define the content domain from which an examination is built. When an examination is built by sampling from the specified content domain according to a pre-determined formula (the blueprint), the decisions made based on examination scores can be considered valid.

    In 1998, the College of Physiotherapists of Ontario independently commissioned Dr. John Norcini, an established expert in licensure examinations, to evaluate the national examination program. In his review of the program², Dr. Norcini (1998, p.3) commented on the development of the PCE:

    "This is a textbook example of the way to develop test specifications. It is thorough, convincing, and sophisticated. Moreover, the list of the entry level activities includes certain aspects of competence that can only be assessed in an examination format like the Objective Structured Clinical Examination."

    Establishing the validity of examination inferences is a process of accumulating evidence over time. Recent evidence supporting the validity of the decisions made based on PCE results includes:

    1. Content aspect of validity:

      • the Cost and Structure Study³ (1998).
      • the Report on the 2000 Analysis of Physiotherapy Practice in Canada† (2001).
      • the Blueprint for the PCE (2001) (included in the Analysis of Practice).
      • extensive local and national consultation process involved in development and review of examination items/questions.
      • processes for translation and verification of translation for all examination items/questions.
      • standardized quality control of examination administration.
      • standardized training of exam site coordinators, of the standardized clients, and of the physiotherapist examiners.
      • examiner feedback on station content.
    2. Construct aspect of validity:

      • the Competency Profile for the Entry Level Physiotherapist in Canada* (1998).
      • the Report on the Use of the Safe Professional Practice Criterion in the Clinical Component of the Physiotherapy Competency Examination (2002).
      • the Candidate Demographic Study (unpublished).
    3. Structural aspect of validity:

      • support by external testing and measurement experts. Specifically, Dr. Dwight Harley of the University of Alberta and Dr. Arthur Rothman of the University of Toronto provide services for the development and administration of the Written Component and the Clinical Component, respectively.
      • standardized quality control of examination scoring and results processing.
      • verification of the quality of the multiple choice questions through statistical analysis.
      • the development and consistent use of decision rules for the Board of Examiners.
      • verification of borderline results.
      • standardized data collection/evaluation sheets.
      • documentation of all procedures.
      • standardized communication of exam information to candidates.
      • selection of testing sites according to pre-determined criteria.
    4. Generalizability aspect of validity:

      See What is the reliability of the Physiotherapy Competency Examination? below.

    5. External aspect of validity:

      A study is planned for 2005 on this aspect of validity of the PCE results.

    6. Consequential aspect of validity:

      The development and consistent use of decision rules for the Board of Examiners.


    ¹ Canadian Alliance of Physiotherapy Regulators. (2001). Report on the 2000 Analysis of Physiotherapy Practice in Canada. Toronto: Author.

    ² Norcici, J. (1998). Review of the Clinical Component of the Physiotherapy National Examination. Unpublished report.

    ³ Canadian Alliance of Physiotherapy Regulators. (1998). A study to examine the structure and cost of the Physiotherapy National Examination. Toronto: Author.

    † Canadian Alliance of Physiotherapy Regulators. (2001). Report on the 2000 Analysis of Physiotherapy Practice in Canada. Toronto: Author.

    * Canadian Alliance of Physiotherapy Regulators, Canadian Physiotherapy Association, Canadian University Physical Therapy Academic Council. (1998). Competency profile for the entry-level physiotherapist in Canada. Toronto: Canadian Physiotherapy Association.

  3. What is the reliability of the Physiotherapy Competency Examination?

    Reliability is extent to which the scores would be reproducible on repeated administrations of the examination. Since we cannot administer the same examination to the same individuals, statistical methods are used to estimate the reliability based on the results of a single examination administered on a single group.

    The bodies that develop standards for educational and psychological tests (the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education) do not set numerical thresholds for reliability, even for use in specific types of decision-making. The reason for this absence is that a "one size fits all" approach to reliability is not consistent with the context-specific nature of psychometrics. Different uses to which test results will be put demands that different standards be used for different examination results.

    In the context of licensure decisions to determine clinical competence, it is more important that a test consistently classify candidates as passing or failing relative to a standard. The most important reliability for licensing examinations is the consistency of classification - would the same candidates be classified as passing and failing the examination on a repeated administration.

    On both the total score and the number of stations criteria, we have chosen to use the Subkoviak¹ approach, which requires identification of the pass score (in standard deviation units) and the reliability of the test results. With this approach a simple coefficient of consistency of classification can be defined. The Clinical Component of the Physiotherapy Competency Examination (PCE) consistently achieves acceptable values for criterion-reference consistency of classification at the passing score for both the total score and the number of stations passed.

    Finally, it is more important that the reliability be comparable to other examination programs than that it compare favourably to an arbitrary external standard. In this respect, Norcini² found that the "reproducibility of the [total binary score] is not equivalent to most written examinations, but it is comparable to other Objective Structured Clinical Examination/oral examination formats."


    ¹ Subkoviak MJ, A practitioner's guide to computation and interpretation of reliability indices for mastery tests. JEM 1988;25:47-55.

    ² Norcici, J. (1998). Review of the Clinical Component of the Physiotherapy National Examination. Unpublished report.

  4. How can the scoring of the written stations be consistent when so many different people are doing it? What quality control checks are put in place to ensure consistency of scoring written test sheets?

    Written station answers are marked by Physiotherapist Examiners at the examination site. The Physiotherapist Examiners use answer keys to determine whether the candidate has provided an acceptable answer to the question. A local written station coordinator oversees the marking process to ensure that the markers accept only the answers on the answer key. Markers are provided with detailed instructions to help them perform their task.

    Unexpected answers are reviewed by the Physiotherapist Examiner and the local coordinator at the site. The Physiotherapist Examiner has the authority to accept an answer that is not on the answer key if the answer is clearly the same in intent as the answers on the answer key.

    If an unexpected answer is not clearly the same in meaning as the answer key, but the Physiotherapist Examiner and the local written station coordinator believe it should be considered, it is referred to a national coordinator who makes a determination of the acceptability of the answer. Any answers that are deemed acceptable through this process are communicated to all sites for marking. In this way, consistency of marking is maintained across the country.

    For unsuccessful candidates the scoring of written station answers is reviewed by The Alliance office staff before the release of results. This final check ensures that all candidates receive proper credit for Written Station answers.


How do candidates tend to perform on the examination?

Generally speaking, candidates perform very well on both components of the examination. Canadian-educated candidates tend to do a little better than those who did not receive their physiotherapy education in Canada. Examination results have been quite stable over time.

  1. Why do the pass rates on the Clinical Component vary from examination to examination?

    The passing rate on the Clinical Component has varied annually from 79.9% to 92.8%. Fluctuations in passing rate are more pronounced for candidates educated outside Canada (42.1% to 67.7%). This can be attributed to the increased variance in education and clinical practice of candidates educated outside Canada.

    Exam Year

    Pass Rate Total Candidate Pool

    Pass Rate:
    Canadian-Educated

    Pass Rate:
    Non Canadian-Educated

    1995

    92.8% (n=297)

    95.5% (n=276)

    67.7% (n=21)

    1996

    80.9% (n=372)

    84.4% (n=356)

    42.1% (n=16)

    1997

    89.4% (n=387)

    95.2% (n=358)

    50.9% (n=29)

    1998

    88.5% (n=524)

    93.7% (n=492)

    47.76% (n=32)

    1999

    91.9% (n=553)

    96.0% (n=497)

    66.7% (n=56)

    2000

    89.5% (n=522)

    94.1% (n=462)

    65.2% (n=60)

    2001

    79.9% (n=529)

    85.4% (n=469)

    52.6% (n=60)

    2002

    85.0% (n=595)

    91.8% (n=504)

    60.3% (n=91)

  2. Why doesn't The Alliance adjust the pass rate so the same percentage pass each time?

    The leading psychometric standard-setting body, the American Educational Research Association (AERA) specifically recommends against credentialling tests using a norm-reference approach - that is, adjusting the passing rate to pass a specified percentage of candidates. Standard 14.17 provides:

    "The level of performance required for passing a credentialing test should depend on the knowledge and skills necessary for acceptable performance in the occupation or profession and should not be adjusted to regulate the number or proportion of persons passing the test."¹


    ¹ American Educational Research Association, American Psychological Association, National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington DC: American Educational Research Association.

  3. How and why do graduates of different Canadian physiotherapy programs perform differently in the examination?

    The performance of graduates of different Canadian physiotherapy programs varies from examination to examination. No one program consistently does better or worse than others.

    There are many possible factors that could explain variation in the performance of graduates of a particular university program from administration to administration.

  4. Why is the pass rate lower for non Canadian-educated physiotherapists?

    The non Canadian-educated physiotherapists represent a very diverse group. Many factors affect their success on the examination including language of education and language of clinical practice, years since graduation (reflecting knowledge decay) and practice patterns in the country of education (similar to Canada or different).

    There may also be a practice effect related to familiarity with the Objective Structured Clinical Examination (OSCE) format from undergraduate education; that is, candidates who have experienced OSCE format examinations during their physiotherapy education may perform better on the Clinical Component than candidates who have not experienced this type of examination. The OSCE format is used in physiotherapy education programs in countries besides Canada.


Who does what for the examination program?

The Short Answer: There are many individuals and groups involved in the examination program. Most of them are volunteers, who draw on their clinical, management and measurement skills to provide advice to The Alliance. Some of the groups are:

  • Candidate Advisory Group

    Provides feedback on communications related to the examination program.

  • Written Test Construction Committee &
    Written Item Subcommittees

    Develop and revise questions for the Written Component.

  • Clinical Test Construction Committee &
    Clinical Item Generation Subcommittees

    Develop and revise stations for the Clinical Component.

  • Exam Steering Group

    Provides advice and guidance on the operation of the examination program.

  • Board of Examiners

    Reviews Critical Incidents and set passing scores for the examinations.

  • Exam Advisory Group

    Provides input into the research and quality monitoring related to the examination.

  1. Does The Alliance decide that a candidate is not eligible to be registered/licensed?

    The Alliance is not a regulator of physiotherapy. The Alliance is a national federation of the provincial/territorial regulators of physiotherapy. Incorporated in 1992, the Alliance is committed to providing leadership and support to the provincial/territorial regulators of physiotherapy in fulfilling their public interest mandate. The Alliance has identified Core Activities in three areas: Evaluation Services; Policy Development; and Communication, Technology and Data Collection.

    The Alliance does not make the decision about who is licensed and who is not. The Alliance provides information on examination results to the physiotherapy regulators. It is up to the regulators to make decisions about who is and is not registered to practice.

    The regulator's role is protection of the public. Regulators do this through various processes, including registration requirements. Some regulators in Canada require successful completion of the PCE as a condition of registration. Some regulators allow candidates to work under temporary certificates of registration while they are waiting to complete the PCE.

    The regulators have different mechanisms which apply when a candidate has failed the examination or a portion of the examination. Each regulator determines the process and criteria in these situations.


How is the physiotherapy competency examination developed?

The Short Answer: The Physiotherapy Competency Examination (PCE) is developed through a comprehensive process of Analysis of Practice, blueprinting and item writing.

  1. Who decides what goes in the examination?

    The content of the examination is determined through research into the activities that physiotherapists in Canada perform as part of their practice. The Report on the 2000 Analysis of Physiotherapy Practice in Canada¹ studied the acquisition, frequency and consequences of activities performed by physiotherapists in Canada. The data from this study was used to revise the examination blueprint.


    ¹ Canadian Alliance of Physiotherapy Regulators. (2001). Report on the 2000 Analysis of Physiotherapy Practice in Canada. Toronto: Author.

  2. What is the examination blueprint?

    The examination blueprint is a guide to the content of each written and clinical examination. The blueprint provides the weighting of the Areas of Practice and Functions for each examination. The blueprint also guides the test construction committees during item development.

  3. Who writes the questions?

    Items for the examination are written by local Item Generation Subcommittees. These subcommittees are located across the country. Members of the subcommittees are physiotherapists with experience in all areas of physiotherapy practice. A number of the subcommittee members are recent graduates of Canadian physiotherapy programs. Written Item Generation Subcommittees write multiple choice questions for the Written Component, and Clinical Item Generation Subcommittees develop stations for the Clinical Component, including the station checklists answer keys.

  4. Who approves the questions?

    The national test construction committees approve the items. The Written Component Test Construction Committee (WCTCC) approves the multiple choice questions, and the Clinical Component Test Construction Committee (CCTCC) approves the clinical stations. The members of both the WCTCC and CCTCC are physiotherapists from across the country, with experience in all areas of physiotherapy practice. Many members have academic appointments in Canadian physiotherapy programs.

    All examination questions and stations are reviewed regularly and necessary changes are made based on statistical and administrative feedback. The review of clinical stations includes review of the task, the Standardized Client instructions, the checklist and the answer key.


How is the quality of the examination maintained?

The Short Answer: The quality of the Physiotherapy Competency Examination (PCE) is continuously reviewed and maintained through the Monitoring and Evaluation Program.

  1. What is the Monitoring and Evaluation Program?

    The Monitoring and Evaluation Program is a comprehensive research and quality assurance program for the examination. It provides a mechanism for overseeing research and quality assurance activities related to the examination, and for implementing improvements based on the results of these activities.

  2. What research is being done related to the examination?

    Some examples of research projects are:

    • Inter-rater Reliability Study
    • 2008 Analysis of Physiotherapy Practice in Canada
    • Report on the Use of the Safe Professional Practice Criterion in the Clinical Component of the Physiotherapy Competency Examination
  3. What are the quality assurance processes related to the examination?

    Examiner feedback, Standardized Client feedback, candidate feedback, incident reports and site feedback reports are all part of the regular quality assurance.

    Feedback is provided to the examination sites (about organization, catering and staff), to the item development committee (about station content and scoring) and to The Alliance staff (about materials and procedures), to continuously improve examination processes.

  4. Who oversees the Monitoring and Evaluation Program?

    The Evaluation Services Committee is responsible for providing advice to the Board of Directors of The Alliance about research and quality assurance for the examination. The Evaluation Services Committee reviews planned research, research reports, and quality monitoring reports and makes recommendations to the Board of Directors. Members of the Evaluation Services Committee include physiotherapists in clinical practice, physiotherapy program faculty, regulators, examination consultants.


How are examination materials translated?

The Short Answer: The Canadian Alliance of Physiotherapy Regulators (The Alliance) is strongly committed to ensuring the accuracy of the French version of the Physiotherapy Competency Examination (PCE). The Alliance has a rigorous, multi-staged translation process to ensure that each exam question translated into French is accurate, checked and approved before the exam material is printed and distributed.

  1. Who participates in the translation process?

    French-speaking and/or bilingual physiotherapists and physiotherapy faculty members across the country participate in the translation process. Here is a brief explanation of the key contributors involved and their primary responsibilities.

    Name of group

    Members

    Responsibility

    Translation Working Group

    French-speaking and bilingual physiotherapists from Quebec, Ontario and New Brunswick.

    Meets every 2-3 years to review translation processes, standards and terminology.

    Written Item Generation Subcommittees (WIGS) and Clinical Item Generation Subcommittees (CIGS)

    Each subcommittee consists of physiotherapists who have clinical practice and/or educational experience.

    Develop new exam questions using approved terminology.

    Written Component Test Construction Committee (WCTCC) and Clinical Component Test Construction Committee (CCTCC)

    Physiotherapists from across Canada with expertise or interest in exams and education, including at least one French-speaking or bilingual physiotherapist.

    Review and approve new or changing exam questions.

    Exam Steering Group

    • Chair of WCTCC
    • Chair of CCTCC
    • The Alliance Director of Examination Operations

    Oversees translation requirements for the exam.

    Exam Advisory Group

    • Recent and future PCE candidates.
    • Representatives from physical therapy educational programs and provincial/territorial regulators.
    • Test experts.

    Reviews Quality Assurance reports after each exam. Makes recommendations to the Board of Directors as needed.

    External Translators

    Individual translators or translation services contracted by The Alliance.

    Prepare and check translations.

    Alliance Administrative Staff

    Bilingual Exam Materials Officer

    Assists with preparing of translation and checking translated material.

  2. What are the steps to translate and check exam questions?

    New questions for both the Written and Clinical Components of the PCE are translated in several stages until they receive final approval by a francophone bilingual physiotherapist. Here is a brief explanation of the major activities during each stage.

    Step 1: Identify new or changed question to be translated

    Request a new or changed question to be translated (WCTCC or CCTCC).

    Remove changed questions from active question bank so they won't be used for exams until new question is translated and checked (Exam Consultant).

    Step 2: Translate the question(s)

    Translate question (External Translator).

    Step 3: Check the translation

    Check each translated question for accuracy and completeness by comparing to English version (Bilingual Francophone Physiotherapist).

    Proofread translation (Bilingual Alliance Staff Member).

    Step 4: Update exam question bank with new translations

    Import updated questions in exam software. Questions are now ready to be selected for upcoming exams.

    Step 5: Prepare exam administration

    Select exam questions.

    Approve English exam (WCTCC Chair or CCTCC Chair).

    Confirm that French translation exists for all questions selected.

    Check translation of all exam questions (Bilingual Alliance Staff Member).

    Review corrections (The Alliance's Manager of Examination Operations).

    Approve translation of all exam questions (Bilingual Francophone Therapist).

    Import corrections in exam software.

    Final proof of exam materials (The Alliance's Manager of Examination Operations).

    Print exam materials.

  3. Are examination scores for French and English candidates reviewed?

    After exams are administered, we also review the French and English exam scores to confirm the fairness and accuracy of both English and French exams. Here is a brief description of that process:

    1. Analyze the scores from the French and English Written and Clinical Components of the PCE (completed by exam consultants).
    2. Identify any significant differences between the average scores for the English and French exams.
    3. Identify any questions that have an unexpected response pattern (completed by exam consultants).

    We also review the Written and Clinical Components of the exam. In addition, identified questions receive careful scrutiny. More details are provided below.

  4. How is the Written Component reviewed?

    1. Review identified exam questions and check for accuracy (francophone bilingual physiotherapist, the Chair of WCTCC and the Manager of Examination Operations).
    2. Meet to discuss identified questions. This may result in:
      • deleting the identified question from the exam score OR
      • accepting other correct responses.
  5. How is the Clinical Component reviewed?

    1. Review identified exam questions and check for accuracy (francophone bilingual physiotherapist, the Chair of CCTCC and Manager of Examination Operations).
    2. Adjust scoring for identified questions if necessary to ensure fairness.
    3. Review all identified questions and other issues affecting final scores (Board of Examiners).
    4. Determine total score and number of stations and other factors required to pass the Clinical Component of the exam (Board of Examiners). The Board of Examiners considers many factors in making these decisions.
  6. What translation reference material does the alliance use?

    Our translators rely on a variety of translation reference material when translating and checking exam questions. In addition to the leading French translation resources, we also use leading medical translation material. Here are just a few examples:

    • Canadian Alliance of Physiotherapy Regulators, 2001, Lexicon of Terms/Lexique de termes, Toronto
    • Lussier, A. and Dionne, S (Eds.), 1990, Vocabulaire de sémiologie de l'appareil locomoteur Volume I: signes cliniques/Vocabulary of Signs and Symptons of the Musculoskeletal System Volume I: Clinical Findings. Canadian Government Publishing Centre, Ottawa
    • Lussier, A., Beauregard, G. and Dionne, S. (Eds.), 1992, Vocabulaire de sémiologie de l'appareil locomoteur Volume II: signes d'imagerie médicale/Vocabulary of Signs and Symptons of the Musculoskeletal System Volume II: Medical Imaging Signs.
    • Canada Communication Group, Ottawa.

Glossary

  1. Angoff method

    A method of standard setting based on the expected performance of candidates. It requires a panel of judges to estimate the likelihood of borderline (or minimally competent) candidates answering each item correctly. The cutscore is determined as the sum of the likelihoods expressed as a percentage.

  2. Board of Examiners

    A group of physiotherapists from across Canada that sets the standard for passing the Physiotherapy Competency Examination. Includes one or more bilingual physiotherapists.

  3. Content domain

    All entry-level tasks and activities performed by physiotherapists in Canada.

  4. Exam Consultants

    Testing and measurement experts (psychometricians) under contract to provide advice and support to The Alliance.

  5. Inferences

    The conclusions that are drawn from examination scores.

  6. Question

    one multiple-choice item for the Written Component or one station for the Clinical Component of the Physiotherapy Competency Exam.

  7. Reliability

    The consistency of scores over multiple measures. It can also be thought of as the proportion of total score variance that is error free.

  8. Standard score

    An examination score on a scale with a predetermined mean and standard deviation. Examination scores examinations administered by the Canadian Alliance of Physiotherapy Regulators are reported on a score scale with a mean of 500 and a standard deviation of 100.

  9. Validity

    An integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy, meaningfulness and appropriateness of inferences and actions based on test scores (Messick, 1989, p.13). The inferences refer to a construct or constructs (e.g., problem-solving, empathy, competence) that the test is intended to measure.


The current version of the Candidate Handbook is the official version of Examination Policies and Procedures.