INTRODUCTION:

Choosing an assessment method which is matching the course needs and the existing norms is extremely important in an education system. Also it should optimize learner’s abilities¹.

The scope of traditional clinical examination (TCE) in nursing is limited in terms of the uniformity of skills assessed and the agreement between the examiners. The marks of the students largely depended on the type of patients and examiners’ satisfaction². Also, the system considers the marks of global performance of students for pass/fail decisions and not the specific skills separately³.

Published evidences suggest that OSCE balances most of the above mentioned assessment issues. It tests cognitive, practical as well as the communication skills under direct observation and facilitates core competency evaluation in a reliable and valid way. A team of examiners are involved, each looking for specific learning. Tasks are chosen according to the need, but are uniform for all students⁴. OSCE follows a transparent, unambiguous and simple marking system with clear instructions. Besides, the trained/simulated patients reduce variability in task assignment. Incorporating OSCE to evaluation system facilitates more comprehensive assessment⁵ with standardized problems^6,7. Defined period of examination and specific criteria for assessment are additional strengths³.

OSCE is the choice of examination method in many of the reputed institutions across the world² and also for entry-to-practice licensing examinations⁵. It is said to have better psychometric properties and helps to maintain the teaching standards of the institution. In the current study, the researchers describe the process of introducing OSCE for assessment of post graduates at an Indian college of nursing.

Many of the Indian Universities are refining their clinical examination system for nursing and many have established their own satisfactory system of assessment. In the selected setting, the examination is routinely conducted in the hospital wards in which a long case for nearly two hours and a short case for 20 minutes are assigned. Two examiners test 15-20 students a day. Since nursing is a skill oriented profession, competence, communication, decision making and overall patient management need to be comprehensively assessed. Our institute used its own evaluation format in which the knowledge, attitude and skill domains are divided into smaller components and each were given score. But as it followed the global pass/ fail system, lack in one of the domains was easily compensated by score in the other domains and thus it did not differentiate between students of specific strength or weakness. In terms of patients, they are selected conveniently, preferably the difficult cases. Difficulty index was not calculated during the assignment and patient problems are never found to be equivalent. Lack of task uniformity leads to wide variations in marks between the students. Students often complained of lack of time to complete all the tasks in the specified time. Interference by the ward chores is common, sometimes patients are withdrawn for investigations or consultations, since they are not isolated for nursing exams. Above all, safety of the examination environment remained a question, as it is not controlled.

Both the examiners take the evaluation rounds together once the procedure session begins. With many students to be evaluated in a short period, most of the time, examiners reach the students after they have completed the basic tasks. This forced the assessors to test the ‘knows and knows how’, leaving the ‘shows how’ part of Miller’s pyramid⁸. Some examiners try to tackle this issue by taking evaluation rounds individually, but this challenged the reliability of the scores. Moreover, evaluating for tasks which are already completed by the student built the examiner stress. Some examiners felt that the evaluation form is too intrusive that they could not concentrate on the student’s performance.

The aim of the study was to assess the feasibility and effectiveness of OSCE as an assessment method with an initial exploration of faculty perception on TCE followed by comparing the feedback on OSCE and TCE among post graduate nursing students in a tertiary care teaching hospital in South India.

MATERIALS AND METHODS:

In order to implement a proven clinical evaluation system in the current scenario, we needed to know the perception of stakeholders. Therefore, the study adopted a mixed methods approach and supportive evidence was collected in two phases- the needs assessment among faculty members and feedback from students. The first phase utilized the qualitative technique of focus group discussion (FGD) of faculty members about the existing TCE system and the second phase adopted quantitative approach of single group post-test only assessment among a cluster of post graduate nursing students after a trial session of OSCE.

During the qualitative phase, the focus group discussions were conducted by the authors, who had experience in conducting qualitative research. There was prolonged engagement between the interviewer and participants and there was detachment by the interviewers to elicit unbiased data. 38 faculty members contributed five focus groups based on their specialties. Each group consisted teachers of Professor, Associate, Assistant and Lecturer grades with teaching experience varying from 7-20 years. The existing examination system was broadly explored using an interview guide which was validated by the subject expert. Peer debriefing and member checking was also done to generate quality data.

Following the qualitative phase, an experimental one group post-test only design was adopted to assess the effectiveness of OSCE compared to traditional method. Choosing the student group for the trial OSCE needed significant workup- we preferred those in their advanced learning levels who can well-sense the ‘goods and bads’ of the new method. Thus, it was consensually agreed by the faculty members to implement it among post-graduates. Since the students belonged to different specialties, their common clinical subject-‘Advanced nursing practice’ was selected for OSCE. The essential tasks were finalized with the help of senior faculty members of the institution.

There were 30 post graduate students in the institution and all of them were included in the study. They were basic-trained from various nursing institutions of the state who followed the TCE method by default. We provided them with an OSCE session for their advanced clinical practice. Five basic clinical skills were chosen namely blood pressure measurement, patient identification, instrument identification, drug calculation and intra-muscular injection. The students were sensitized about OSCE before the session by the OSCE team. OSCE was conducted at the end of second term of the academic year so that students had completed one practical internal assessment session in traditional way during first term. This was also a feasibility check to conduct OSCE in the college using available resources or minimum resource modifications.

As an initial step, a Team OSCE was constituted with the faculty members who either had undergone OSCE training as part of faculty development programme or had successfully practiced OSCE in their previous institutions. The team comprised the roles of assessor, coordinator and reliever. A brush up session including a brief introduction about stations, tasks, steps and guidelines was held among the team members before conducting the OSCE.

Stations were set in the practice lab of the institution with necessary amenities. Mannequins or junior undergraduate students as simulated patients were utilised for the performance stations and real instruments were used for the non-performance station. The time allotted for each station was 10 minutes with transit and preparation time of 3 minutes. The marks awarded for each OSCE station was 10, and the steps were categorized into must know and could know. The pass criterion was full score in the must know areas. Unsatisfactory performance in more than one task was considered fail in the session. Student who failed in any one of the tasks was supposed to repeat it. A modified, validated version of Examination Feedback Questionnaire was utilized⁹, which is a 22 item five-point likert scale with questions in three sections namely, Overall exam evaluation (10 items), Quality of performance testing (8 items) and Validity- reliability (4 items). The responses were self-reported. The modified questionnaire was validated in the new settings for content and the obtained Cronbach alpha value was .68 for the English version.

After OSCE was completed, students were provided with the examination feedback questionnaire and were asked to register their responses on OSCE and the TCE in the space provided. The study received permission from the ethics committee of the institution and all the students voluntarily consented to register their feedback on the examination methods.

The FGD sessions with faculty members were audio-taped, transcribed and verbatim was analyzed using a thematic approach to identify themes depicting their perception on the TCE.

Student feedback data were analysed using descriptive and inferential statistics. The mean and standard deviation of the overall scores and that of the three components of the questionnaire were calculated. The feedback scores on the two examination systems were compared using paired ‘t’ test. The analysis was done using SPSS version 27.

This study has been approved by Institutional Ethics Committee of Sri Ramachandra University, Porur, Chennai, Tamil Nadu, India (No. IEC-N1/16/AUG/55/58).

RESULTS:

The qualitative analysis revealed the themes as ambiguous task assignment, non-standardized patients, task overload, examiner burden and time constraints. Each focus group involved at least one faculty from the higher grades and two from the lecturer grade. The average teaching years for the faculty members was 12.7. All were females with the mean age of the group being 42.2 years.

Among the students, all were females of age range 25-33 yrs. 50% of them had been working in teaching institutions before their admission to the course and the rest were from clinical service. The overall and component-wise feedback scores of students were higher for OSCE. The overall mean feedback score on OSCE and on TCE differed significantly (t= 4.576, p<0.001) (Table1).

Table 1. Comparison of Overall feedback of students on OSCE and TCE (N=30)

Overall score		t value (P value)
OSCE	94.20 (7.25)	t= 4.576, p< 0.001
TCE	81.63 (13.53)	t= 4.576, p< 0.001

Feedback scores on OSCE were significantly higher for all three components of the tool as well (overall, quality of performance and reliability/validity) (Table 2).

Table 2. Comparison of component specific scores for OSCE and TCE (N=30)

Component	OSCE	TCE	t value
Overall Evaluation	41.70 (3.82)	36.86 (5.57)	3.92 (p<0.001)
Quality of performance	34.40 (2.87)	26.46 (4.81)	8.14 (p< 0.001)
Validity and reliability	17.83 (2.30)	14.26 (4.05)	5.15 (p< 0.001)

Interestingly, the item-wise comparison revealed that the scores for the component stress level were favouring the TCE (97 vs. 116) and that of time adequacy were similar for both (119 vs. 118) (Table 3). In the current study, all members of the sample scored full marks in all the tasks.

Table 3. Item-wise scores of OSCE and TCE (N=30)

Item	OSCE	TCE
Overall exam evaluation
Exam was fair.	127	112
Wide knowledge area covered.	131	110
Needed more time at stations	119	118
Exams very stressful.	97	116
Exam well structured & sequenced.	135	112
Exam minimized chance of failing.	117	94
Allowed student to compensate in some areas	131	99
Highlighted areas of weakness.	135	116
Student aware of level of information needed.	131	116
Wide range of clinical skills covered.	128	117
Quality of performance testing
Fully aware of nature of exam.	135	116
Tasks reflected those taught.	135	117
Time at each station was adequate	124	112
Setting and context at each station felt authentic.	116	111
Instructions were clear and unambiguous.	131	115
Tasks asked to perform were fair.	123	118
Sequence of stations logical and appropriate.	131	105
Exam provided opportunities to learn.	137	105
Reliability and validity
Exam scores provide true measure of essential clinical skills.	132	103
Scores are standardized	138	108
Practical and useful experience	134	118
Personality, ethnicity and gender will not affect scores.	131	111

DISCUSSION:

The current study discussed the process of introduction of OSCE in an Indian Nursing College. The need for this was ascertained using a two-stage process. Initially FGDs were conducted among faculty members with an aim to collect their feedback on the existing traditional system of clinical examination. They expressed concerns of lack of task uniformity, non-standardised patients, task overload, examiner burden and time constraints. Similar were the reports from faculty members of India¹⁰ and other countries earlier, who continue to face an increasing demand to ensure high standards of nursing education¹¹. The faculty members play facilitator role in learning and thus the expressions and concerns put forward by them are deemed real and practical. The clarity, comfort and ease of the evaluators are invariably important in an evaluation system.

Secondly, OSCE was introduced among a group of post graduate nursing students who were pre-exposed to the TCE. The feedback on both OSCE and TCE were collected from the students and compared. The overall and component-wise feedbacks of students were found more favourable for OSCE. This was consistent with the reports of various OSCE studies in India^12-14 and different parts of the globe across the disciplines^15,16.

The feedback on examination was collected from the students using a three-component examination feedback questionnaire with questions in the aspects of overall examination performance, quality of performance and reliability/ validity. OSCE was reported to have higher reliability/validity and better quality of performance testing when compared to TCE. The higher reliability has most likely arised from the task uniformity and examiner permanence. Though the total score of the component of overall exam evaluation was higher for OSCE, interestingly, two items under the overall evaluation component namely examination stress and time adequacy were scored differently.

One of the items rated low for OSCE was ‘less stressful’ under the component of overall exam evaluation. Though reported worldwide the same way predominantly among undergraduates^5,17 It was rather an unexpected feedback from a sample of post-graduates. The investigators attribute this finding to the student’s first-time exposure to OSCE. It is surmised that exposure to a new assessment method can be stressful irrespective of the level of the examinee.

Another item whose scores did not differ between OSCE and TCE was ‘time adequacy’. Similar reports on time constraints during OSCE were also located in the literature¹⁸. Students in the current study felt that they needed more time per stations to complete the tasks. Also, possibly, the time allotted per station for higher level learners needs to be tailored since the performance demand of postgraduates is higher. Modifying the task selection or fragmenting broader tasks to suite time allotment without compromising the completeness of the task may tackle this issue.

Considering integration of stress reduction strategies into assessment sessions may be a good solution, especially during entry-to-practice and licensing tests. Sine OSCE is time bound and demanding, the room for ice-breaking and air thinning is limited compared to TCE. The pressure of performance is more for OSCE, though it is inevitable, as most of the tasks of medical field are complex. However, allowing undue longer performance time per stations will dilute the seriousness of the assignment.

Though collected, the assessment scores of the two examination methods were not compared, as the purpose of the study was primarily to test the feasibility of OSCE and thus to form the base for its introduction into the regular examination system. The assessment of effectiveness of the new method in larger scales was planned to be ascertained in further studies after getting the administrative nod.

The results of the current study ie. the faculty opinion from the FGD and the examination feedback from students were presented to the academic administrators of the university. A wait of a short while was there before the Board of Studies meeting was scheduled and the findings were positively accepted. It was approved to include OSCE in the syllabus as a regular assessment method.

Since demands of the specific settings may vary widely, we recommend good background work before introducing OSCE in any institution. Selection of tasks and type of stations are crucial and both are to be conveniently modified according to the level of the examinee. Fragmenting the tasks without compromising the completeness will retain the purpose of the test. The students’ feedback conveys that OSCE may not a suitable choice for situations which demanded complex performance abilities. Integrating OSCE along with other examination methods is recommended as it will be complementary in the assessment process. Course objectives and examination blue prints can be relied on while planning. The students’ concerns of examination stress and time adequacy need to be addressed in further sessions.

The student participants scored on OSCE after experience of a single session. One of them had an exposure to OSCE in an in-service education programme from her previous employer. The number of OSCE stations was less compared to the widely accepted standard of 10-20. Validation of quantitative data by member checking probably by a personal interview to extract some additional background thoughts of the students associated with various items of the feedback questionnaire might have been useful.

CONCLUSIONS:

Faculty members of the selected institution had real concerns on the way the traditional clinical examinations were conducted and they expressed the preference for a more objective practical examination system during FGDs. Students’ feedback on OSCE was promising. The researchers could successfully mobilize the resources of faculty, materials and time slot of the institution to implement OSCE. Moreover, it is concluded that OSCE is a preferred and feasible assessment method in the selected set-up, inclusive of advanced learners. Thus the study was suggestive, though done in a minor perspective, as it successfully effected the integration of OSCE into the assessment system of the institution. It is also inferred that strong and evocative research evidence makes the implementation of an innovative curricular change enduring and meaningful.

CONFLICT OF INTEREST:

Authors declare that there is no conflict of interest

ACKNOWLEDGEMENT:

Authors acknowledge Prof. Dr. Asma Rahim, Govt. Medical College, Kozhikode, Kerala, Indiaand Prof. Dr. Thomas V Chacko, Dean of Medical Education, Believers ChurchMedical College, Thiruvalla, Kerala, India.

REFERENCES:

1. Bhowate R, Panchbhai A, Tankhiwale S, Vagha S. Introduction of objective structured clinical examination (OSCE) in dental education in India in the subject of oral medicine and radiology. J Educ Ethics Dent. 2014; 4(1): 23. Available from: http://www.jeed.in/text.asp?2014/4/1/23/143169

2. Zayyan M. Objective structured clinical examination: the assessment of choice. Oman Med J. 2011; 26(4): 219–22. Available from: http://www.ncbi.nlm.nih.gov/pubmed/22043423

3. Vishwakarma K, Sharma M, Matreja PS, Giri VP. Introducing objective structured practical examination as a method of learning and evaluation for undergraduate pharmacology. Indian J Pharmacol. 2016; 48(Suppl 1): S47–51. Available from: http://www.ncbi.nlm.nih.gov/pubmed/28031608

4. Choudhari S, TN U. Objective structured Clinical Examination (OSCE): A Review. Res J Pharm Tech. 2016; 9(8): 1–3.

5. Shirwaikar A. Objective structured clinical examination (OSCE) in pharmacy education - a trend. Pharm Pract (Granada). 2015; 13(4): 627. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26759616

6. Krishnavenimurugesh R. OSCE the Curriculum tells the staff what to teach. The OSCEs tells the Students what to Learn. Int J Adv Nurs Manag. 2018; 6(2): 135–9.

7. Brannick MT, Erol-Korkmaz HT, Prewett M. A systematic review of the reliability of objective structured clinical examination scores. Med Educ. 2011; 45(12): 1181–9. Available from: http://doi.wiley.com/10.1111/j.1365-2923.2011.04075.x

8. Miller GE. The assessment of clinical skills/ competence/ performance: Academic Medicine. Acad Med. 1990; 65(9): 563–7. Available from: https://journals.lww.com/academicmedicine/abstract/1990/09000/the_assessment_of_clinical.45.aspx

9. Lisle J De. OSCE student evaluation form. MountHope, Trinidad; 2001.

10. Bhalerao P. Perception of medical teachers and students regarding objective structured long examination record as an assessment tool- a prospective study. 2017; 5(4): 1679–82.

11. Al-Saqri S, Wardeh YSA, Balawi A. Nursing field teaching staff perception toward using objective structured clinical evaluation to evaluate nurse’s students. Int J Adv Appl Sci. 2018; 5(5): 71–8. Available from: http://science-gate.com/ IJAAS/Articles/2018/2018-5-5/09 2018-5-5-pp.71-78.pdf

12. Vijayalakshmi K, Revathi S. Performance of Nursing Students in Psychiatric Nursing using Objective Structured Clinical Examination (OSCE) Versus Traditional Practical Examination (TPE)- A Comparative Approach. Asian J Nurs Educ Res. 2017; 7(4): 561–8.

13. Smitha GS. Students Evaluation of an Observed Structured Clinical Evaluation (OSCE) in Community Health Nursing Clinical Practice. Asian J Nurs Educ Res. 2018; 2(8): 278–81.

14. Kshirsagar SS. A Study to Assess Perception of Nursing Students about the Effectiveness of Objective Structured Clinical Examination (OSCE) during Perineal Care Procedure in the selected College of Metropolitan City. Int J Nurs Educ Res. 2021; 9(4): 425–8.

15. Ameh N, Abdul MA, Adesiyun GA, Avidime S. Objective structured clinical examination vs traditional clinical examination: An evaluation of students’ perception and preference in a Nigerian medical school. Niger Med J. 2014; 55(4): 310–3. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25114366

16. Eldarir SA, Abd NA. Objective Structured Clinical Evaluation (OSCE) versus Traditional Clinical Students Achievement at Maternity Nursing: A Comparative Approach. IOSR J Dent Med Sci. 2013; 4(3): 63–8.

17. Bdair IAA, Abuzaineh HF, Burqan HMR. Advantages and Disadvantages of the Objective Structured Clinical Examination OSCE in Nursing Education: A Literature Review. Int J Trend Sci Res Dev. 2019; 3(2): 270–4.

18. Ataro G, Worku S, Asaminew T. Ataro, G., Worku, S., & Asaminew, T. Experience and Challenges of Objective Structured Clinical Examination (OSCE): Perspective of Students and Examiners in a Clinical Department of Ethiopian University. Ethiopian Journal of Health Sciences. 2020; 30(3).

Received on 20.07.2023 Modified on 13.11.2023

Int. J. Nur. Edu. and Research. 2024; 12(1):27-32.

DOI: 10.52711/2454-2660.2024.00006