Monday, May 18, 2015

The problems with CBSE Class XII examinations

18 comments :
India's largest educational board, the Central Board of Secondary Education (CBSE) is expected to release Class XII results (equivalent to GCE A-levels or the AP examination) later this week. Students' performance in this exam is critical for college admissions as well as getting government jobs.

Given how critical the examination is, however, it seems to be poorly designed.

Diving into the CBSE 2014 Class XII results data

I was curious about what the CBSE Class XII results looked like, and decided to snoop around the results website to see if there was any way I could scrape data. It turned out to be worryingly easy to do. A simple look at the source code of the page, and a few lines of python, were all it took to get the results (as well as personal information such as name and father's/mother's name) of all students who took the test. I selected ~17,000 out of the million students who took the test for my analysis. This analysis is only reflective of some regions (most likely in the north), and not of India as a whole.

People smarter than me have highlighted similar data before, but I tried to explore the problem from a different angle. Here is what I found.

Some subjects are much easier to score in than others

Amongst subjects that had atleast 300 students (out of the 17,000 samples), Music and Painting seemed to have the highest average score. Popular subjects like Informatics, Physical Education, and Computer Science also had reasonably high average scores.



On the other hand, Political Science, Economics, Mathematics, and Physics have the lowest average score. This disparity in scores is a cause for concern, as it puts students that take 'harder' subjects at a disadvantage when applying for jobs or universities.

At the same time, inequality in scores is another major cause for concern. This is especially stark for Mathematics, where an 80th percentile student scores 89 marks compared to a 20th percentile student who scores 33 marks.


Papers are not well designed, and "grace marks" are far too prevalent

Can you guess what the most common score in the English, Mathematics, Biology, Chemistry and Accountancy exams is?

Most people I asked thought it would be between 70/100 and 80/100. The actual answer? 95/100.

Yes, you read that right. A well designed test would ideally have an average score of between 70-80 marks range, and will have relatively few students below 50 and above 90. CBSE's tests, however, yield very different results.

Mathematics is an extreme example of this. Many students barely pass the subject. Yet, a very large number of them score 95 marks. A similar trend can be observed in English and Economics.



This can largely be attributed to the vast majority of questions being 'reasonably easy' for someone who knows the material well, and only a couple of questions differentiating the top students. This is a major problem for university admissions, where cutoffs have consistently reached ~97%.

It is also interesting to note that no one scored between 25 and 32 marks for the above 3 subjects. This trend is explained by the fact that 33 is the cutoff for passing marks, and that examiners are likely artificially inflating students' grades (where feasible) in order to make them pass.

A similar (albeit less extreme) scenario is observed for Physics, Chemistry, Biology and Accountancy.

Caste still matters

While many Indians feel that students' performance should be independent of their caste in a world where everyone is supposedly given equal opportunity, caste still seems to matter. Surnames commonly used by people in the SC/ST perform worse (as a group) than average. Conversely, groups with the surname Gupta, Goel and Arora tend to perform better than average.


This disparity particularly noticeable for subjects like Mathematics.


Concluding thoughts

CBSE and the Human Resource Development (HRD) ministry needs to get their act together. Disparities in scores between subjects, within subjects, and between different castes are extremely obvious. It might serve the government well to devote fewer resources to opening more IITs and IIMs, and instead ensure that the quality of primary and secondary education rises in an equitable manner.

18 comments :

  1. >I selected ~17,000 out of the million students who took the test for my analysis.

    Why not use all ~million records if they are available? Are you certain that the ~17k records you chose represent a random sample? If they don't, then that could significantly alter the results.

    Also, isn't it possible for the same caste to have multiple surnames?

    ReplyDelete
    Replies
    1. > Why not use all ~million records if they are available?

      Because they were taking ages to download. I do eventually aim to scrape all records, but it will take a while. Given that the results for the 2015 exam are coming out soon, I thought it would be better to scrape those instead.

      The 17k records do not represent a random sample. As noted in the post, this analysis is only reflective of some regions (most likely in the north), and not of India as a whole. I'm not entirely sure of what those regions are.

      You're right - it is entirely possible for a caste to have multiple surnames. In general though, some surnames are associated more with certain castes (like Rana and Kumar).

      Thanks for leaving this comment and for pointing out the flaws in this analysis; much appreciated! :) In hindsight, I might have published this prematurely. Will try to be more thorough in the future.

      Delete
    2. >The 17k records do not represent a random sample. As noted in the post, this analysis is only reflective of some regions (most likely in the north), and not of India as a whole. I'm not entirely sure of what those regions are.

      Given that fact, I would be very wary of sharing visualizations from the article with the title, e.g., "Caste still matters in India when it comes to math scores" (paraphrasing your reddit post title) without making it clear that this is not a representative sample of test scores in India. I would even go so far as to put this note in your visualizations (perhaps at the bottom?) so readers know to take your findings with a grain of salt at this point until you complete the analysis.

      Interesting idea anyway - I hope you continue with this project.

      Delete
    3. True. My bad for not noting making it clearer in the visualization, should have included a note at the bottom.

      Thanks again for the feedback!

      Delete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Hi Rishabh,

    A friend and I also did something similar, including comments by insiders. Do have a look:

    http://www.firstpost.com/india/grade-distortion-in-indian-education-should-be-seen-as-a-serious-public-policy-problem-2887650.html

    Prabhat

    ReplyDelete
  4. View lots of cool education-related articles and exam tips at https://essayclick.net/exam-tips.html

    ReplyDelete
  5. Paper writing is a challenging task. Especially when it comes to exam tests. Be sure to click here for some tips.

    ReplyDelete
  6. Understudies should take after this timetable for their matric examination 2018. Understudies can likewise download the official date sheet Telangana Board 10th Time Table 2018
    Tripura Board 10th Date Sheet

    ReplyDelete
  7. Applicants those are sitting tight for most recent Government work opening 2018 they can apply online.Candidates who are scanning for most recent February Government Job 2018 they all can check here most recent Government jobs of February month and Upcoming all Govt Job 2018.
    Mahadiscom JAA Eligibility Criteria 2017
    SBI PO Category Wise Vacancy Details

    ReplyDelete
  8. I read this article. I think You put a lot of effort to create this article. I appreciate your work.
    thesis Writing Service

    ReplyDelete
  9. This comment has been removed by the author.

    ReplyDelete
  10. I hope there will be more of this type of publication.

    Matilda

    ReplyDelete