Towards online assessments

On November 24, 2010, the last batch of Common Admissions Test (CAT) takers for admissions to the prestigious IIMs clicked the submit button for their exams, bringing to close a mammoth exercise that saw 2,04,267 candidates taking the computer-based exams over a 20-day period! While the decision by the CAT committee to go online was in the right direction, it was marred by technical glitches and virus attacks in the 2009 edition. Despite criticism and mounting pressure to go back to the traditional pen and paper method, it was praiseworthy that the CAT committee decided to continue with the online mode.

What changed between the 2009 and 2010 CAT administrations?

*MeritTrac and Everonn were roped in as delivery partners for administering CAT 2010

*The testing window was extended from 10 days to 20 days, thereby reducing the number of concurrent terminals required to be managed

*Rigorous project management and dry-runs were conducted to stress-test the infrastructure

The success in conducting the first error-free online CAT is attributed to the experience and expertise of the delivery partner in conducting online exams in India.

After CAT, CBSE declared that the All-India Engineering/Architecture Entrance Examination (AIEEE) would go online from this year. One lakh candidates will be allowed to take the exam online on a first-come-first-served basis in 20 cities with a capacity of 5,000 per city, the CBSE said in a release.

Several exams have successfully migrated to the online mode. Manipal University and BITS Pilani have been conducting online entrance exams since 2005, for around 1,00,000 candidates.

MeritTrac and Edge ValueNotes recently conducted a survey on ‘Examinations and the Role of Technology: Emerging Directions’  that gives a big thumbs up for technology in exams from both students and universities. This comprehensive survey covered over 75 university VCs/Registrars/Controllers of Examination (CoE) and 400-odd students.

This report, the first of its kind in India, endeavours to look at the issues with traditional paper-pencil examinations and the potential benefits of leveraging technology to deliver such exams. The report provides insights into the awareness, perception, readiness and experience of key stakeholders in Indian higher education with respect to technology solutions to assessment, primarily online examinations.

The study also includes a student perception survey - covering 400 students across several Tier I, II and III cities.

Here are some interesting insights from the survey of university officials:

*52% of all VCs/Registrars/CoEs interviewed indicated their interest in the concept of using technology in exams.

*70% of all VCs/Registrars/CoEs indicated they were considering introducing online examinations and 68% of these respondents indicated that they would do this in the next 6-36 months.

*Decision makers largely believe that technology solutions can mitigate some of the challenges we face and the level of awareness on the potential of technology is quite high.

*The top three priorities in exam management are faster results, greater transparency, and student authentication/higher security. Respondents said these would have the greatest impact on their decision to go online.

*Infrastructure, suitability of the test format and equating between multiple batches of exams emerged as the top three perceived challenges with almost 60% of the respondents referring to them.

*Some respondents were also worried about the students from rural areas, who may not have access to computers at all, and may face difficulty in adopting this.

On the perceptions of the students:

*An overwhelming majority of students (95%) are aware of online assessments, and this is so even in Tier III cities

*The majority of students (69%) felt that universities that conduct online exams are more technically advanced - especially students originating from Tier III cities

*A majority of students consider the online process to be more accurate and secure, compared to traditional methods

*Transparency of processes and convenience are the perceived benefits, though students are concerned about the difficulty of usage of online exams, and lack of support from exam authorities.

Any large scale high stake online exam's success rides on the following factors:

Online testing engine: It is critical to have a platform built for the Indian context.  Can the test be completed even if there are power outages, weak networks, poor internet connectivity? Are the test screens easy to comprehend?

Infrastructure readiness: It has to work through extensive full load dry runs (terminals, power, backup servers). Centres have to be sanitised before exams. The trick is to iron out all issues well before D-day.

Trained proctors: Test administrators have to be candidate-friendly, trained on the system and the candidate interface, and be able to handle any eventuality.

Candidate support services:  Accessible help-desks with multi-modal support like telephone, mails, chat and SMS will go a long way in reducing stress levels when the exam process is new.

Project management: Finally, an examination project must have a good project management team that can seamlessly integrate all aspects of the exam, identify risks and mitigation plans.

Online exams allow the conduct of a secure, accurate exam with instant result processing minus the logistic hassles of transporting questions and answer sheets.

(The writer is CEO and Co-founder, MeritTrac)