The Numerate Indian

Monday, May 18, 2015

The problems with CBSE Class XII examinations

12 comments :
India's largest educational board, the Central Board of Secondary Education (CBSE) is expected to release Class XII results (equivalent to GCE A-levels or the AP examination) later this week. Students' performance in this exam is critical for college admissions as well as getting government jobs.

Given how critical the examination is, however, it seems to be poorly designed.

Diving into the CBSE 2014 Class XII results data

I was curious about what the CBSE Class XII results looked like, and decided to snoop around the results website to see if there was any way I could scrape data. It turned out to be worryingly easy to do. A simple look at the source code of the page, and a few lines of python, were all it took to get the results (as well as personal information such as name and father's/mother's name) of all students who took the test. I selected ~17,000 out of the million students who took the test for my analysis. This analysis is only reflective of some regions (most likely in the north), and not of India as a whole.

People smarter than me have highlighted similar data before, but I tried to explore the problem from a different angle. Here is what I found.

Some subjects are much easier to score in than others

Amongst subjects that had atleast 300 students (out of the 17,000 samples), Music and Painting seemed to have the highest average score. Popular subjects like Informatics, Physical Education, and Computer Science also had reasonably high average scores.



On the other hand, Political Science, Economics, Mathematics, and Physics have the lowest average score. This disparity in scores is a cause for concern, as it puts students that take 'harder' subjects at a disadvantage when applying for jobs or universities.

At the same time, inequality in scores is another major cause for concern. This is especially stark for Mathematics, where an 80th percentile student scores 89 marks compared to a 20th percentile student who scores 33 marks.


Papers are not well designed, and "grace marks" are far too prevalent

Can you guess what the most common score in the English, Mathematics, Biology, Chemistry and Accountancy exams is?

Most people I asked thought it would be between 70/100 and 80/100. The actual answer? 95/100.

Yes, you read that right. A well designed test would ideally have an average score of between 70-80 marks range, and will have relatively few students below 50 and above 90. CBSE's tests, however, yield very different results.

Mathematics is an extreme example of this. Many students barely pass the subject. Yet, a very large number of them score 95 marks. A similar trend can be observed in English and Economics.



This can largely be attributed to the vast majority of questions being 'reasonably easy' for someone who knows the material well, and only a couple of questions differentiating the top students. This is a major problem for university admissions, where cutoffs have consistently reached ~97%.

It is also interesting to note that no one scored between 25 and 32 marks for the above 3 subjects. This trend is explained by the fact that 33 is the cutoff for passing marks, and that examiners are likely artificially inflating students' grades (where feasible) in order to make them pass.

A similar (albeit less extreme) scenario is observed for Physics, Chemistry, Biology and Accountancy.

Caste still matters

While many Indians feel that students' performance should be independent of their caste in a world where everyone is supposedly given equal opportunity, caste still seems to matter. Surnames commonly used by people in the SC/ST perform worse (as a group) than average. Conversely, groups with the surname Gupta, Goel and Arora tend to perform better than average.


This disparity particularly noticeable for subjects like Mathematics.


Concluding thoughts

CBSE and the Human Resource Development (HRD) ministry needs to get their act together. Disparities in scores between subjects, within subjects, and between different castes are extremely obvious. It might serve the government well to devote fewer resources to opening more IITs and IIMs, and instead ensure that the quality of primary and secondary education rises in an equitable manner.

Sunday, March 8, 2015

The railways freight sector would be India's highest profit company if it was a standalone business

5 comments :
The Indian Railways recently released a White Paper to the parliament, and it made for an interesting read about the industry. The most fascinating aspect of the paper was the dichotomy between the freight and passenger business.


If the railways freight business was a stand-alone entity, it would have the highest profit among all listed companies in India (for FY2012-13) with Rs. 33 thousand crore in profit before tax, just ahead of ONGC. Yet, the profit made by the freight business has been largely offset by the losses in the passenger business, which is hemorrhaging money.

Looking at the railways estimates for cost vis-a-vis yield, the reason for this becomes clear. The freight business operates at an operating margin of 39%, while the passenger business operates at -103%, spending almost double of what it earns. The table below was copied from page 20 of the aforementioned white paper.

(PKM stands for Passenger Kilometers, while NTKM stands for Net Tonne Kilometers)

Benchmarking with global standards, India has one of the highest rates in the world for freight transport, while it's passengers fares seem unsustainably low, according to a World Bank presentation to a government committee (I would have liked to see Japan, Germany and the US represented in both categories, but could not find the corresponding data for them).


Clearly, this situation is unsustainable. The high cost of rail-freight (which, incidentally, was increased further in the recent rail budget), is one of the biggest reasons why the annual Net-Tonne Kilometers of freight transported per wagon in India is only 2.73 million, compared to 4.31 million in China and 5.52 million in Russia. The high cost of rail-freight also contributes (although it's not the only factor) to most freight being transported to roads through trucks, which cause significant traffic congestion on roads. At the same time, it contributes to high prices of food, fertilizer and other commodities across the country.

While the recent rail budget was rightly lauded for focusing on the right things (network decongestion and growth instead of newer trains), I was disappointed by the ministry's refusal to make hard choices. Freight-prices were increased instead of passenger prices in order to ensure that the railways was able to overcome operational costs. This was a populist move that was frequently adopted by the UPA government. While any part that sanctions an drastic increase in passenger prices will draw flak, gradual but consistent increases would help make it a more viable business.

At the same time, the government needs to curb pension spending in the railways. While this does seem like a cruel policy, the pension fund of the railways is exceptionally large. In 2014-15, the budgeted railways pension fund was more than the entire budgeted spending on Science and Technology in Union Budget 2015-16. The pension fund accounts for 18% of the entire railway expenditure.

The BJP government came in with a mandate for reform, and they have being doing a very good job so far. While increasing passenger fairs and reforming the railway pension system might be one political battle too many right now (especially in face of the controversy around the Land Acquisition Bill), this is something that I hope will appear on the roadmap over the next few years.

Wednesday, March 4, 2015

Breaking down Budget 2015

1 comment :
The Union Budget was declared recently to much fanfare on social media and in the corporate sector. Yet, as I tried to make sense of where exactly we spend our tax money, I could not get a clear picture. The best that I could get was the screenshot below at the Finance Ministry site.



We can do a lot better than that when it comes to showing citizens how their money is spent, so I decided to do a bit of digging around. The government of India has put up detailed expenditure statements at the Indian Budget site, and while it is not entirely comprehensible to folks not familiar with public finance (like yours truly), it can give a more holistic picture of how the budget is being spent. I primarily used these two documents to visualize our spending. Because I used my discretion to categorize the spending as Infrastructure-based or Food and Agriculture-based, this might have some minor errors (although it is fairly representative of the big picture).

Now that the disclaimers are done, here is how India plans to spend its funds in FY 2015-16.

A dark orange rectangle indicates increased spending compared to the previous year, while a blue rectangle indicates decreased spending from the previous year.

Some quick insights from this analysis:

1. Debt is strangling India's budget

The UPA government's reign saw India's external debt increase more than four-fold from $100 billion in 2004 to $450 billion in 2014. Subsequently, some analysts have cautioned the country about the possibility of falling into a debt trap - a situation where it needs to take on more debt in order to pay off its current loans.

To overcome this problem, the government needs to bring in higher revenues while reigning in spending. Yet, doing so at this juncture would be detrimental to the country's overall growth, especially as it needs greater investment in infrastructure.

I feel that the Modi government's approach to solving this conundrum is to invest heavily in infrastructure that will drive growth for now, while at the same time cutting down on subsidies. At the same time, it is trying to rake in more taxes through Goods and Services Taxes and Value Added Taxes, which ensure that it gains more revenue streams. Lastly, they are aiming to reduce India's tax rate from 30% to 25% while at the same time abolishing tax exemptions for corporations. This is aimed at making India a more attractive place for business while not reducing tax revenues.

Most importantly, it is very heartening to see India embrace the Public Private Partnership model to finance its infrastructure. By utilizing capital from private companies, the government's burden of financing is reduced. At the same time, more stringent oversight resulting in more efficient use of capital can be expected with private partners on board.

It's too early to see if their efforts will yield fruit, but they do seem like a step in the right direction.

2. The abolition of energy subsidies has had a massive impact - even in the face of low oil prices

The budgeted petroleum expenditure fell from Rs. 62743.22 Crore in 2014/15 to Rs. Rs. 30125.55 Crore for 2015/16. This are significant savings, and were largely achieved by the government's abolition of fuel subsidies. The impact of this will be even more significant once oil prices start to rise again. However, a significant subsidy on LPG and Kerosene still exists. It is significantly more difficult to get rid of these subsidies as doing so would be a very thorny political issue.

3. Food and Fertilizer Subsidies are still a huge problem

The Agriculture sector in India is a bit of a black hole for government money. It accounts for 15% of the country's GDP, but around 50% of employment. The sector is highly fragmented, and has unfavorable yields compared to other Asian countries - despite superior soil quality.

Because a very large chunk of Indian population depends on agriculture for livelihood, and because many in India are too poor to afford food, the government cannot cut down on food and fertilizer subsidies until a) the agriculture space gets consolidated and b) the output of food in India increases significantly, and its price decreases correspondingly.

I would like to see the government do more to improve the scene for agriculture in India. The Land Acquisition Bill is a good start. If passed, it would lead to a more rapid transition from agricultural work to manufacturing labour in the economy. But it would do little to solve the problem of the country's notoriously agricultural low productivity.

On the whole, I think that the budget is a monumental step in the right direction. I hope that the policies that follow will be in the same vein, too.

Thursday, February 12, 2015

UP's problem with rape is more serious than what the numbers indicate

1 comment :
As the world turns its lens on brutal rapes and murders committed against women in India, many have used statistics to 'prove' that India is not as unsafe as the media would have one believe. According to the official numbers, incidence of rape cases in India (adjusted for population) were only 7% of that in the US in 2012. Clearly, the official numbers don't match the popular narrative.

Yet, frequent reports of brutal gang-rapes that have emerged in recent years make it hard to believe that India is indeed more unsafe for women than America.

As someone who lives in India, I'm concerned that many politicians have been turning a blind eye to the problem by hiding behind numbers, particularly in Uttar Pradesh (UP). Chief minister Akhilesh Yadav said that
"When such incidents occur, it is publicised a lot. Such incidents don't occur only in UP and if I give you statistics across the country you will again ask me questions. It's Google era. If you go online and check you will see where else such incidents occur."
His remarks had been lambasted by many in the country, where UP is believed to be one of the least safe states for women. A brutal gang-rape and murder happened inside a school in the capital of the country, and a female judge was raped at her residence in the state. Yet, the official figures indicate that UP has one of the lowest rates of rape (adjusted for population) in the country at 3.1 cases of rape reported per 100,000 women (in 2013). In comparison, India as a whole had 5.5 reported cases of rape per 100,000 women in 2013.

Is UP really more safe for women than Kerala or Goa? Most in the country would find it hard to believe. For instance, the district of Badaun is "officially" one of the more safe places for women in India with an incidence of 3.04 cases of rape per 100,000 women (compared to 5.47 cases per 100,000 women for the national average). Yet, the district has become notorious in recent months for the absymal state of girls and women. In 2014, two police constables allegedly raped a minor inside a police station and later absconded (yes, you read that right).

At the same time (and contrary to popular belief), UP had one of the highest rates of conviction for rape in 2013 at 53.7%. In comparison, the national conviction rate for the crime stands at 26.4%.

Amongst the many hypotheses that could be used to explain the situation, the following are the most common:
  1. Crimes against women are massively under-recorded in UP, either because victims under-report crimes, or because the police refuses to record them (or a combination of the two). Only crimes that are extremely difficult to refute are recorded regularly. This might explain the exceptionally high conviction rate.
  2. The media really does have a bias against UP and reports crimes that happen their more regularly.
I feel that hypothesis 1 is more likely to be true than hypothesis 2 because:
  • UP Districts that border Madhya Pradesh (MP) tend to have significantly fewer recorded rapes than their MP neighbours, despite similar demographic characteristics and culture.
  • Registered cases of sexual harassment have absolutely plummeted since 2010.


    This cannot be explained by anything else except for police's refusal to register them since other influential factors (like actual instances of the crime and proclivity of victims to register the crimes) cannot change by three orders of magnitude within a year.
  • Anecdotally, the complicity of the police in not recording crime cases in order to keep figures low is well understood (although there is not hard data to prove this claim). The National reported how the police often does not record crimes for political reasons. It quoted Mr. SR Darapuri, a retired inspector-general of the UP police: “Every chief minister here wants to keep the crime figures low, and this may be directly or indirectly communicated to the police. The performance of police here is evaluated through statistics, so the police has an incentive to keep those statistics low.”
In general then, we need to understand that sensationalist headlines by the media about increasing numbers of crimes against women might be counter-productive. For instance, the significant increase in reported crimes in the aftermath of the 2012 Delhi Gang Rape might have meant that more women were reporting crime, rather than crimes actually being committed more often. Headlines stating that 'Crimes against women increased', without acknowledging that they are still under-reported do more harm than good, in my opinion.

At the same time, we need the government and the media to step up and encourage men and women in undeveloped areas to stand up for women's issues. A commendable job has been done to spread this awareness in rich and urban areas, but much needs to be done in rural areas.

Most importantly, more oversight on the police is needed. Aberrations like a 99+% year-on-year drop in the number of harassment cases can and should be identified early and relevant police officials should be held accountable for their actions, instead of letting the years go by without any action. The UP government's initiative to allow women to lodge complaints online (instead of going to a police station) is commendable, but is unlikely to benefit the poor and uneducated.

Until police officers at the lowest rung are sensitized (and punished heavily for their crimes), the most populous Indian state is unlikely to become a save haven for its female population.