Tuesday, March 2, 2010

IIMs' faulty scoring system, and why I'm pissed off

In this blog article, I’d only like to talk about the IIMs’ scoring system for boards examinations and graduation marks. The fact that they consider a heavy weightage for the marks obtained in boards examination as compared to the CAT score is another contentious issue which I’d like to discuss some other time.
A rough pattern of weightage of various marks for any IIMs’ scoring system looks something like this:
CAT marks: 60%
Xth: 10%
XIIth: 10%
Graduation: 10%
Work Experience: 10%
The score given for marks obtained in boards and graduation is a step function defined as: Score = 10 for %age=< 100 and >= 95, Score = 8 for %=<95 and >90 , and so on)
Here, I’d like to compare two candidates A and B. The differences between their percentages in boards and graduation is small and A has performed better than B in the CAT exam.
Suppose candidate A has secured 264/450 (A percentile of 99.85 overall) in CAT, 89.6% in Xth, 79.4% in XIIth, 84.2% in graduation and no work experience. On the other hand, candidate B has secured 253/450 (A percentile of 99.54 overall) in CAT, 90.4% in Xth, 80.8% in XIIth, 85.6% in graduation and no work experience.
In this case, IIMs calculate the score from the CAT exam by linearising it through the following equation:
Score = (Cat score / 450) * 60
This gets A 35.2 + 6 + 2 + 4 + 0 = 47.2/100, while it gets candidate B 33.73 + 8 + 4 + 6 + 0 = 51.73/100.
Instead of this, had the IIMs linearized the scoring system for boards as follows:
Score = (percentage – 75) / 2.5,
A would’ve had a total of 35.2 + 5.84 + 1.76 + 3.68 + 0 = 46.48/100, while candidate B would’ve got 33.73 + 6.16 + 2.32 + 4.24 + 0 = 46.45/100.
The candidate B is helped by the fact that his percentages are on the right side of the percentage range while candidate’s A percentages fall on the wrong side. As we see, unlike in the first case with the final score difference between A and B is huge, the first case makes their score comaparable. Hence, when we linearize both the CAT scores and boards, graduation percentages before adding them up, it gives us far more just results.
Also, it is a known fact that boards across the country are far from uniform in their scoring. So, it makes very difficult for a person from, say Rajasthan board to compete against a person from Andhra Pradesh State Board.
This faulty and unjust system has surely robbed a lot of candidates who did extremely well in CAT the chance of getting into and IIM. Since my marks in boards and graduation are similar in manner to candidate A and the fact that I can't go back to my past and write the board exams again or work harder for my cgpa, I don’t think I am going to write CAT again, knowing the fact that the top IIMs are always going to elude me.

[Certain data used in this blog post is based on the scoring criteria released by IIM C and IIM L.]

13 comments:

Anonymous said...

If your case is closer to Candidate B, don't you end up in a better position that Candidate A in the present* scoring system?

present* - I am not entirely sure if the approach you have mentioned is how the IIMs do it, but that's my ignorance as I haven't yet read the criteria publisized by IIM C and IIM L.

kay gee said...

@anon
error rectified. It's closer to A, not B.
the point is that you shouldn't take a step function for giving scores for boards/graduation percentages

Anonymous said...

IIM Calcutta has shortlisted for its PGDM programme only based on CAT scores, and nothing else. PGDM is the flagship course of the institute with close to 400 seats. Only for the smaller PGDCM programme have they considered academic scores. IIM Calcutta has always maintained that they wouldn't like to penalise students for their past academic performance, and that they believe in an 'absolute' meritocracy. Hence the reason for IIM-C shortlisting only on CAT scores for its flagship programme, as always. But then you need to have very high scores in CAT for this. There has to be at least one area where you are exceptional.

Similarly, IIM Lucknow takes into account acad scores and work-ex because they feel that a brilliant person might lose out on a seat just because he did not perform well on the day of CAT. Same with IIM Bangalore-they feel that if you're good at sports, you would add diversity to the classroom. Hence their preference for good profiles in the interview stage. IIM Ahmedabad follows a different criteria each year, depending upon what the profs think is important that year.

If you think about it, there are valid arguments on both sides and different IIMs follow different criteria, thus making the IIMs accessible to all. You'll have to work depending upon your strengths and weaknesses, much like CAT itself. If your acads are bad, instead of ensuring just a good score in CAT, you ought to ensure an excellent one. The IIM-C cutoff for PGDM was 99.5 percentile overall, with around 96 percentile in each section. It was the only shortlisting criteria.

It's unfair to blame the system all the time. There is a sense of fairness, and opportunities for all

kay gee said...

@ Anonymous
I understand that it is the IIMs' discretion to choose the kind of candidates they want, and that if one has poor acads one must do very well in CAT with good sectionals.
Also, IIM A selects candidates for the GD/PI round only on the basis of the CAT scores which is good. very good.
however, my grouse (in this post) is against the manner in which scores are given for the board exams and graduation. if you give 8 points to a candidate securing 90.6% and 6 points to a candidate securing 89.6%, it creates a difference of a score of 2 which is so huge that it cannot be mend by their differences in CAT scores.
And, every IIM (to the best of knowledge) gives away scores in 10 - 8 -6 -4 -2 -0 manner only, at some stage of selection or the other.
Why can they not linearize it by say,
score = (percentage - 75)/2.5
because they give a score of 0 of percentage under 75 anyway.
But thanks for an informative comment.

Ashtung said...

Must say I do not know the nitty-gritties of the scoring system...
However, I do feel that it's unfair to even consider 10th board marks... C'mon what does a kid know back then... While Kipal Sibal is gung-ho about reducing the pressure on JEE aspirants (hence the cases of students with even 5 marks in one subject getting in), nobody is looking at this...

Also, does your degree or college carry any weightage? Please answer

kay gee said...

@ Ashtung
Yeah, as I said taking into account your class X marks is a contentious issue, but sadly IIMs believe that people who have had a good past academic record tend to do better in IIMs. It might be an erroneous argument, but it's a policy followed by them. And I doubt we can debate it with them.
Suppose, for the sake of humour, tomorrow IIMs want footballers as IIM students and want to consider the number of goals scored in school tournaments because they've somehow found out that good footballers tend to make good management students. We can question the study, but if the study is correct we can't question it because it's the way they want to do it.
Having said that, this system shouldn't be in place until there is a uniformity in the way various boards award marks.
Coming to you question, your graduation marks are given weightage by some IIMs. But, that again is a step function. And, not linear. The kind of college of the nature of degree doesnt carry any weightage, to the best of my knowledge.

jimmy said...

Think about it. They use this discretisation policy everywhere. Your grades in IIT are given like that. Income tax to be paid is calculated based on salary scales.

Problem with linearising, as u called it is- it is more just alright; but it carries with it that sense of enormous competition. Guy with 83% will forever detest the guy with 85%. Scaling/grading system is an attempt to get away with this kind of foolish competition. The price that you have to pay though is at the edges of the steps, as your example pointed out. And ya people in the delta margin of the step are statistically speaking smaller than people in the bulk regime. So greater good, go with it!

kay gee said...

@ jimmy
we don't crib about the grading system because it's an assumption that the errors get evened out through so many courses in so many years. in the case of a curriculum, it is relatively fair and preferred because it does reduce competition.
but, if it were just exam that would decide your face, you will obviously crib.
And, when the IIMs are using linearization for calculation of CAT score, does adding a discrete function score for your boards score reduce any competition?

jimmy said...

well, as u rightly pointed out there are two things that iims are looking at:

a.cat score, which is supposed to indicate the seriousness of the candidate to get into iim- which rightly so needs to be competitive and hence linearised like jee

b. continuous performance of a candidate, which apparently is given by 10th 12th and degree grades- in which it makes no sense to distinguish an 8.2 pointer with an 8.3 pointer- or an 81% with 84 %, right. i mean, slight fluctuations should not matter when you are assessing somebody over a long run (as you said too)

kay gee said...

@ jimmy
i am saying it should be justified. the fluctuations - whether slight or huge - must be represented as they are.

jimmy said...

ok saar, read the hindu- saw that u r on the receiving end. sorry for that. hope things get better :(

but still, argument wise- i stick to mine

kay gee said...

@ kay gee
and, i to mine.
btw, i contacted them. :)

amaneesh said...

I really appreciate the blog. Would also like to state that i really don't understand the point of giving weightage to 10th and 12th marks.most of the students who aspire to make it to the IIM's are not even serious during their school times.That doesn't determine their caliber.Students literally slog themselves for doing well in CAT and what they get is mere 20-30% overall weightage of the score they gain.Isn't that an injustice.I am doing my graduation for now and before starting to prepare for the same i am scared to be a prey of such a fatal system of evaluation.something's really fishy.

Share

Share |
Creative Commons License
blog page by aneesh bangia is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 India License.
Permissions beyond the scope of this license may be available at www.somethingisfishy.blogspot.com.