Computerized CAT 2009 - What went wrong?

Any serious B-School aspirant in India will know how revered CAT (Common Admission Test administered by IIMs) is. It is rather aptly considered to be the mother of all competitive exams in India. Not that all competitive tests originated from CAT but that it is by far the most rigorous, time intensive and an outlandishly competitive examination. To me, only IIT's JEE is as competitive, if not more. That said, both these examinations are poles apart. JEE is far too technical whereas CAT is all about precise Math and mind boggling English in a single package.

And now, IIMs have gone a step ahead and wanted to have CAT administered in an online format. And ironically, what transpired makes a very good case study. The CAT committee has stressed that its role in administering the examination at the 105 centres in 32 cities was restricted to preparing questions. That was the objective. IIMs may have wanted to get rid of the test administration part and focus on their institutes and respective programs, which by the way is a great idea. These institutes can focus more on their programs and worry less about the test delivery. And then they chose Prometric as the test delivery partner.

To start with, it is a bold and forward step to plan administer CAT online. I see nothing wrong in this intention. What should take centerstage is to determine if this idea is feasible taking into consideration the limitations of our infrastructure, number of aspirants planning to take the test and above all, the nuances involved with implementing the plans. Do we have the infrastructure to successfully hold an online examination of this magnitude. The answer is evident from this week's fiasco. Not yet!

BITS Pilani administers its admission test online. It has been successful so far. There are a couple of institutes that either conduct their admission test online or outsource the testing process. All this while, an online test is not a test administered on the internet. Computers in a test center are pre-loaded with the examination software and students log on to take the examination. It is pretty much similar to GRE/GMAT with a significant exception that the CAT (pun intended) is not computer adaptive at this point. It is a computer based test as opposed to GRE/GMAT which are computer adaptive. So, what worked for BITS and not for IIMs? Foremost reason would be the number of applicants. About 250,000 (2.5 lakhs) students took CAT in 2008. This year 240,000 students were scheduled to take CAT over a period of 10 days in 105 centers. That is a huge number and it appears bigger when seen in the perspective of our current infrastructure. To pull of a test of this magnitude, Prometric should have had some fool-proof practices in place. It could have organized some mock-CATs (atleast for a price) to see if the computer centers can handle the load. Students would have flocked to avail this offer. In a nation taunted to be a computing superpower, we shouldn't have dearth of anti-virus experts or software. Yet we blame virus attack as a reason for this debacle. A brainstorming session amongst IIMs would have yielded couple of other good fool-proof measures.

Now that CAT is mid-way and the problems are still rampant it is a tricky situation for IIMs. It is an opportunity for IIMs to implement their management practices and clear this mess. But in this debacle, spare a thought for the applicants. Many of them may have been preparing for months and all this stress in the last minute is completely unwanted and unnecessary. I have been going through lot of user comments online and it is a mix of frustration and helplessness. A user complained that his restless neighbor at the test was trying to peep into his computer screen as the neighbor's test hasn't yet begun. Some of them had frozen screens. Some of them were made to run from one center to the other through SMSes. The biggest doubt in the minds of students will be integrity of the test. If the test delivery is vulnerable to viruses, as is claimed, is the scoring vulnerable too? Bringing 'virus' into the discussion opens up a bag of worms (pun again!). Viruses can let outsiders handle a system. If these examination centers have incompetent anti-virus, can outsiders control the software and get to know the questions? I know, it is all a mess!

It will be interesting to see what the future course of action on part of IIMs is. It may be a better idea to start from a clean slate and administer a paper-based test probably 2-3 months down the lane and work on reducing post-processing timelines. One expert suggested administering CAT via the cloud. It could be a good idea for next year. Atleast the crashing part can be taken care of. That said, the main objective for this years CAT should be to ensure that no test taker receives any unfair advantage whatsoever. Only then will this CAT prevail.