Deciphering UP

Results of the elections to the 16th Lok Sabha will be declared tomorrow.  It has been a long and arduous battle, culminating in a crescendo with the marquee contest for Varanasi in the last round of polling. The voting is all done now and as we move towards counting day, Uttar Pradesh, is the center of all the attention. Without even going into the efficacy and accuracy of exit polls all over again, it is almost certain that once again the result in UP will determine who rules in Delhi. This state that elects 80 of the country's 543 legislators has given it 9 of its 13 prime ministers. Its small wonder that the latest aspirant to that position rushes to UP to get the right credentials. But India's most electorally significant state is also its most enigmatic one. This post is about trying to understand Uttar Pradesh better to see if there are any pointers to what is going to happen tomorrow.

To start with it is perhaps good to get an idea of the scale we are dealing with here. UP is the biggest state of India by population and 4th biggest by Area. With a population of ~200M if UP were a country it would be the sixth most populous in the world. Secondly, even though it seems to be a single large swathe of riverine plains extending across the northern part of India, it is pretty diverse in terms of the demographic. Essentially we are dealing with a reasonably complex creature here, and it needs to be broken into smaller, more manageable regions to do any meaningful analysis. The following graphic shows something I have done in a hurry. It is not based on any official source, but rather something rough to give some idea of bearings.

Learn About Tableau

Learn About Tableau

The Doab is the name given to the narrow strip of land between the rivers Ganga and Yamuna the two most prominent features of UP's political geography. This extends all the way from the northern border of the state with Uttarakhand where the two rivers originate to Alahabad where they meet. This is split further into what is called the Upper Doab, or the Jat heartland of Western UP and the Lower Doab (or Doab here) extending from Agra to Allahabad. Rohilkhand is the region wedged between the Ganga and the border with Nepal and gets its name from being hitorically ruled by the Rohilla Afghans of Bareily. Similarly the region below the Yamuna is called Bundelkhand and stretches into northern part of the adjoining Madhya Pradesh. The eastern part of the state is often referred as Purvanchal together with a large part of western Bihar. Awadh is the area around the capital city of Lucknow between Rohilkhand and Purvanchal, this was the area hitorically part of the empire of the Nawabs of Awadh (Oudh). Besides the historical, geographical and demographic context to the way UP is devided into regions, there is also a strong distinction in economic aspects. The western parts of Up for example benefitted from the green revlution much more than the eastern ones and that has led to a significant economic divide between these regions. Needless to say these regions exhibit significantly different voting behavior and thats where we go nex

The graphic below shows the results of elections in UP from 1977 to 2013. Use the < and> buttons to scroll across years or the dropdown to go to a particular year.

Learn About Tableau

As one can see, UP is not a state averse to whitewashes. While in 1977 the BLD wiped out the Congress from UP on the back of a strong anti emergency sentiment, The Congress did the same in 1984, winning all but two in 1984 (Etah and Baghpat which to this day remain agrarian/socialist strongholds). What seems to be interesting though is that since 1989 the contest has got more and more quadrangular, with the BJP and then the BSP emerging as strong forces in the state. Consequently we can see this in a four way split of the seats between the 4 main parties. Also 1998 was the closest that UP has come to a one sided result in a while and in the last 4 elections there has been a convergence in the vote shares and seats won by various parties.(notice funnel shape in vote share graphic below).

Learn About Tableau

Also worthwhile to note is where we ended in 2009. The 2009 election saw as clear a 4 way split in terms of seats as UP has ever seen. Even though the BSP was not the largest party in terms of seats, it did end up with the highest vote share (rather comfortably). At 27% the BSP's voteshare was a good 10% higher than the BJP and Congress and 4% higher than the SP. However even more interesting is the gradual consolidation across 5 general elections that this culminates in.  This looks like a base that has been gradually built over time something that might not be very easy to blow away with a wave. The vote base of the Samjwadi Party seems a little more fickle, but only a little. If the BJP has to get to levels anywhere near to where it was in 1998 and what the opinion polls are predicting, it has to increase its voteshare by a whooping 20%. As in much of the country the BJP would be hoping that at least a part of it would come from the Congress voteshare (say 10%), but that itself is not going to be enough. The BJP will have to make a significant dent in the well established caste coalitions of the SP and BSP. There has been some debate around 'wave' and 'tsunami', if the BJP has to get back to 40% voteshare and 50+ seats, it better be a tsunami because a mere wave wont do. I will try to explore a bit more about where the remaining 10% is going to come from.

Learn About Tableau

The graphic above shows the regional distribution of seats between various parties in 2009. We can see that in 2009 the Congress put up a very credible performance to emerge as a force in Awadh whereas in the Doab region the SP continued to maintain its stranglehold winning 6 of the 9 seats (this includes traditional SP bastions like Mainpuri, Etawah and Kannauj) . The BSP turned out with its best performance in western UP and Purvanchal (where the spoils were almost equally split in 3 and 4 cornered contests, many of the quiet close- Graphic below)

Learn About Tableau

One more thing that I tried to look at was what are called bastions. Or seats that some parties seem to win with reasonable frequency. One could follow with the argument that if a party Wins often from a seat and if the margin was particularly large in the last election, chances are that they would retain it. To look at this i looked at a 3 election horizon (and tried to find seats where one party had won twice or more) and a 5 election horizon (and tried to find seats where one party had won thrice or more) The results are shown below. 

Learn About Tableau

In the 5 election horizon - the BJP shows up with 20 seats where it has won more than 3 times. Interestingly a majority of these seats are concentrated in western UP and Purvanchal apart from a few urban centers like Lucknow and Bareily elsewhere. Lucknow and Gorakhpur are 2 seats that the BJP has won each time in the last 5 elections, true bastions. Not even Banaras qualifies for that distinction.
In the case of SP there are 17 such seats. Their strength in the doab is reflected in the strongholds like Mainpuri, Kannauj and Etawah. Kaiserganj, Mohanlalganj (in Awadh) ,Jalaun (Rohilkhand), Balia and Phulpur are others that are less widely known.
In the case of Congress its the usual Amethi, Rai Bareily, Pratapgarh, Kanpur, whereas in the case of the BSP they have a couple of strongholds in Purvanchal (notably Akbarpur) and few in Awadh. To note here is the fact that the BSP is a late entrant and hence seems to be under represented in the 5 election horizon.

Learn About Tableau

The case of BSP was the reason for looking at the 3 election horizon separately  (which has a higher recency effect and in the case of the BJP a higher dose of reality). The big change between the 5 election and 3 election horizons is the viibility of the BSP as a force in northern Purvanchal of late, as well as emergence of SP as a force in southern purvanchal. The good news for the BJP is that they still seem to be retaining their strength in western UP, even in the more recent 3 election horizon.

So where does all of this leave us in the context of our original question on what it would take for the BJP to make it to the 40% vote share, 50 seats mark in UP.

Lets start with Western UP. It is the only region in UP where the BJP was still number 2 in terms of voteshare (having been relegated to 3 and 4 in other regions). Also to keep in mind is that if we included a part of the RLD (part of NDA in 2009) to the BJP vote share it gets even better (though still 3-4% behind the leader BSP). To start with western UP seems to be the BJP's best bet to get back in the game in UP.  If the BJP can take away 6-8% from the Congress and supplement it with a base among new first time voters (which will be a significant factor this time) they have a good chance of winning western UP without making a significant dent in the base of the BSP and the SP. The recent riots and resulting polarization will also play a part here.
In Rohilkhand - a similar scenario presents itself here, with the difference that the difference between the BJP and the leaders SP, BSP is lower at about 5%.. So if the BJP could increase their vote share by about 10% points taking from the Congress and gaining supplementing with new voters, it still barely gets it slightly in front the leaders. To get back to levels where it was in 1998 (37%), it will have to make a severe dent in the bases of BSP and SP.
Moving on to the Doab, here the BJP is more than 22% behind the leader SP. One wonders whether even a tsunami can make a dent for the BJP in this Yadav heartland, where seats like Mainpuri, Etawah and Kannauj elect the SP with heavy margins in election after election. 

Awadh is a slightly different case. Here even though the BJP is behind the leader by 20% (like in the Doab) that leader happens to be the Congress in this case. After Western UP this seems to be another area where the BJP would be looking to make some serious inroads. It needs to be noted though that even if the Congress seats and voteshare seems to be up for grabs the same cannot be said about SP and BSP which have some notable bastions in this area (Mohanlalganj and Kaiserganj for SP and Sitapur, Misrikh for the BSP)

Lastly to the largest and most important one, Purvanchal. Here the BSP has established itself as a significant force in the last two elections, With highest vote share (still only 27%) and joint highest number of seats. Most of Purvanchal saw 3 and 4 cornered contests some of them very close. The other interesting  point is that the BSP seems to be the only party which is strong across Purvanchal (in 2009). The SP seems to be stronger in Southern Purvanchal, while the Congress was stronger in the northern part and parts bordering Awadh.

To sum it up, If the BJP has to get to 50 seats (36-38%) vote share in UP, it has to sweep Western UP. To add to it it has to literally replace the congress in Awadh and make severe dents in the BSP's vote base in Purvanchal, to get ~2/3 of the seats in both regions. BJP will have its biggest challenge, and a worthy challenger in the form of Mayavati and BSP. It will be difficult, bordering on impossible and if they can pull it off, nothing less than a tectonic shift in the politics of UP.

The juggernaut is rolling!

The Banaras visit is behind me now, and am back to the humdrum of daily life. Meanwhile the bandwagon of elections for the 16th Lok Sabha was set rolling yesterday. Six parliamentary constituency went to the polls in the first phase of polling. There is a sense of occasion - the biggest democratic exercise in human history is on its way. On average the turnout in the first phase of elections  (75% in 5 seats in Assam and 84% in one seat in Tripura) is a few percentage points higher than 2009 . Though an early indicator, if this (higher turnout) is a precursor to what will happen in the rest of the country, then together with the 100 million incremental registered voters in 2014,  it makes for a very interesting prospect. Suffice to say that this election is ripe with possibilities. In the effort to try to understand what 2014 has to offer, understanding the political geography of the country and what happened the last time when the nation went to vote are possibly good starting points. Also 2009 is the only election we have had since the last round of delimitation in 2007, hence some merit in starting with a quick look at the political geography before following it up with what happened in 2009.

The following graphic takes a quick look at the political geography of India, with the 35 states/union territories and 543 constituencies - the geographical and demographic diversity of the country is on display in all its grandness.A quick glance tells us about the distribution on population of the countries. The density of constituencies in the doaba region(between the Ganga and Yamuna)  of UP and along the Ganga in Bihar and West Bengal contrasts sharply with the sparser distribution in the mountains to the north, the deserts in the west and the forests in the center . The largest constituencies (Ladakh, Kachchh, Barmer are orders of magnitude bigger than the smallest ones (North Kolkata and South Mumbai).

Learn About Tableau

 Worth noting is the natural hierarchy of states in terms of electoral importance. Just the top 6 states (UP, Maharashtra, West Bengal, Andhra Pradesh, Bihar and Tamil Nadu) constitute well over 50% of the total seats. It would be fair to say that no national party/coalition can hope to form a government in the center without a strong performance in at least 3 (ideally UP+2) of these states. With 80 seats, the result in UP alone has often had a strong direct import on who forms the government in the center.

Similarly the top 12 states (top 6 + MP, Karnataka, Gujarat, Rajasthan, Orissa,Kerela) constitute 420 or ~ 80% of the total seats, for all practical purposes National elections are won or lost in these states. 

Out of the 543 seats of the Lok Sabha  seats are reserved for the scheduled castes and  seats are reserved for the scheduled tribes. The below graphic shows the distribution of seats for scheduled castes and tribes across the country. As we can see, the scheduled caste reserved seats are almost uniformly spread across the country while the scheduled tribes seats are concentrated in the forests of central India and the hilly regions of the north east.

Learn About Tableau

 

Learn About Tableau

Lastly worth mentioning here is that looking at a large states like Uttar Pradesh  as a monolith may be seen as analytical oversimplification, and rightly so. Ideally the larger states should be split into smaller political regions for a more nuanced analysis. For example UP can be split into Rohilkhand, Awadh, Purvanchal, Bundelkhand and the Doab (region between Ganga and Yamuna), similarly Andhra Pradesh into Telengana, Rayalseema and Coastal Andhra and Maharashtra into metropolitan Mumbai, Konkan and Vidharba. 

Just a little more work for the data wrangler!

A heroic and flawed compact with democracy

We landed in Banaras yesterday and were promptly greeted by the hot summer sun and huge hoardings proclaiming ‘Abki Baar…’ - just to remind us that we are out of cozy Bangalore and bang in the epicenter of elections 2014.  Modi, the challenger to the throne will be facing off with Kejriwal the wildcard in these elections in Banaras. Mau strongman  Mukhtar Ansari’s wife is contesting and like much of the country it seems like the Congress has not shown up for the elections yet. 

Signs the impending contest are everywhere. In the car the driver slips in an opinion unobtrusively, the owner of my hotel is tallying electoral rolls as we check in. In the mutt the brahmins deliberate quietly, a kid known to them comes by and one of them bounces him in the air regaling him with ‘har har modi’, perhaps unaware of the latest party line, or maybe he couldn’t care less. In the corner shops people are talking about elections over their chai,chat, lassi and paan. 

Elections are serious business here and opinions are professed with a sense of deliberation one does not find in metropolitan folks.  The usual cynicism that big city people have for politics is there but it is not used as a shield to insulate themselves from the political process. Most of them are incredibly well aware about the political context, they know about the candidates and even though some might not have made up their mind, they will vote. In many ways Banaras represents much that is wrong with our country today and still in its narrow and dingy lanes ‘Indias heroic and flawed compact with democracy’ that author and historian Ram Guha talks about, almost comes to life. And with it the thought that perhaps a quick look at our journey as a young democracy, through data on its most significant milestones, the 15 general elections, is a good place to start.

Worth mentioning at this time is that the much maligned task of data cleaning is not quite as thankless as it is made out to be. Along the way one discovers little gems that make it worth the effort. There were quite a few about our democracy that I discovered in the course of the last few months.

For example I did not know that in the first 2 elections (1951 and 1957), we had 86 two-member constituencies (each returning 2 members of parliament) and even one three -member constituency (North Bengal). The purpose of multi member constituencies was to give a stronger voice to minority communities. This practice was discontinued in 1962 elections.

Likewise gathering and cleaning data pushes one to find out more about things that one would not have otherwise encountered, like delimitation. Though most of us heard of the delimitation in 2006, this was the fourth time delimitation happened (before that in 1956, 1966 and 1976). The purpose of this periodic exercise being to ensure that the way constituencies where organized (and reserved) was in sync with changing demographic (due to natural population increase, migration etc). The constitution was specifically amended in 2002 to freeze the allocation of seats by state for 25 years, to ensure that states that did well on social welfare measures and family planning were not unfairly penalized. 

It also tells the story of the gradual evolution of the Union of states. We had 17 after the first reorganization based on linguistic states and now we have 35 states and union territories. Meanwhile, new states were formed, old ones broken up, and names changed to reflect changing sensibilities

Learn About Tableau

As an aside, apart from the sheer magnitude of of it (15 elections with thousands of candidates every time) Indian elections are also a analysts nightmare for other reasons. There are three pairs of constituencies in the country that have the same name- even the spelling.  Hamirpur ( Himachal Pradesh, Uttar Pradesh),  Aurangabad (Maharashtra and Bihar) and Maharajganj (Uttar Praadesh and Bihar) make up the trio (or sextet) that will be an irritant to many an analyst. Not to mention constituencies with very similar names – like  Ballia (UP) and Balia (Bihar), Godda (Bihar) and Gonda(UP) , Raigad(Maharashra) and Raigarh (Chhattisgarh). 

The data also tells another story far more important than these factoids, that of the gradual maturing of Indian democracy to reach where it is today. 

Learn About Tableau

To start with the number of votes cast was little over 105 million in the first general election in 1951. It was more than 417 million in the last general election in 2009, that’s more than the four fold increase over 6 years, higher than the average growth rate in population, symptomatic of a young democracy gradually taking root. Though the increase in the number of voters has been steady there have been outliers. Every time the electorate was called on to cast its vote, before the scheduled 5 year interval (example in 1991, 2 years after elections in 1989 and in 1999, barely a year after 1998) the enthusiasm has slacked and the number of votes cast has gone down. On the other hand, the biggest increase in the number of votes polled (compared to the previous election) is quiet visually in the 1977, 1984 and 1989 elections. In each of these instances the electorate was stirred by extraordinary issues and sentiments)

In the same time the number of political parties contesting the elections has gone up from 54 to 366 and the number of candidates has gone up from 2000 to about 8000 (actually consolidated of late from a high of 12500 in 1996)

In the first general elections as many as 10 candidates were elected unopposed. That number has fallen over the years to almost zero is yet another testament of the democratic process gaining maturity.

However merely increased participation is not a strong enough indicator of a democratic system taking root. How well democracy has served its weakest constituents is perhaps a better indicator. In that sense, bringing about a degree of empowerment among some of the historically oppressed sections of society is probably one of the qualified successes of our nation and the democratic process has played a part, credit is due to our founding fathers who envisaged our republic thus. Even the first Lok Sabha elections had 8 seats reserved for the scheduled tribes. Seats were reserved for scheduled castes starting 1962 and as of today, of the 543 constituencies of the Lok sabha-84 are reserved for Scheduled castes, 47  for scheduled tribes and the remainder (412) in the general category.

Lastly to end on a somber note, the data also reminds us is that this compact of ours, with nationhood and democracy, heroic though it might be is far from perfect.  As late as just two decades ago, democracy could not be freely practiced throughout the country. 

Learn About Tableau

There were pockets of unrest - Jammu and Kashmir (1991), Assam (1980 and 1989) and Punjab (1984) where elections were not held with the rest of the country because the state could not guarantee free and fair polls. Just a reminder that democracy, so central to the idea of India, still needs a lot of nurturing.

Heres to new beginnings..

It has been a while since I have been here. The Karnataka elections are far behind us, and the big one is looming. Actually the big one is almost on us now, the dates have been out for a while, the biggest democracy in the world goes to polls in a couple of weeks. In the next month 800 Million voters will have the chance to exercise their democratic privilege as the country waits for May 16th - the day of reckoning.

I must confess that I restart this journey with a certain wariness. Part of it perhaps comes from a natural inertia at the start of something new, but there is the other part which is about the magnitude of the subject at hand.  It is a bit like how you feel when you get late, late because you are not sure if you should be there in the first place. The intent of this blog has always been to provide a historical perspective of elections, driven by data, and in the process try to understand the workings of democracy.  I am no expert and this is more like a tool to document learnings but even then Lok Sabha 2014 has proved to be daunting subject.

To start with, this is a "datablog" and hence the essential and most important ingredient of it is the data. To say the least getting to the data necessary to do any meaningful analysis of India's history of parliamentary elections is not a trivial task. We have had 15 elections in the last 60 odd years and even though the election commission of india provides all records of past elections - most of it is in pdf files and step 1 is to convert it into a usable machine readable form like excel. That by itself is a time taking, manual task done over months. Finding GIS data on constituency maps is even more difficult.

To add to this General Elections in India is a big deal, 2014 in particular, is possibly the most hotly contested and divisive one in my memory. An incumbent who has led the country into despondency of corruption and a low growth, high inflation economy, a challenger to the throne who is by most accounts a highly divisive figure,  and a wildcard who has thrown the gauntlet at traditional politics but has little experience of governance make for an interesting cast of protagonists. Then there is the big set of character actors and regional chieftains, who see this as their big chance for a coup.  It is safe to say that this election is a magnum opus that has captured the imagination of this country. All this is going to be played over a battlefield spanning 35 states and union territories with 543 constituencies between them. Candidly, I have had much of the data for months now but have been vacillating on where to start.

However two events have acted as a proverbial kick on the back and put me to work again. Firstly the data camp over the weekend was an inspiration like last time - plus we managed to crack the post delimitation shape file nut with the help of GIS guru Devdutta. Secondly circumstances take me to Varanasi tomorrow. I will be spending the next week in the nerve center of India's political heartland. I had to get into the act, now.

So heres the plan - In the month or so to the lead up to the results - I will try to sift through the mounds of data to try to better understand the political geography and history of this country. Throughout the process looking for nuggets from the past that might have bearings on the future. Given the broad canvas, the plan is to look at the country at a macro level and then dive deeper into the states that matter - namely UP, Maharashtra, West Bengal, Andhra Pradesh, Bihar and Tamil Nadu, others depending on time. 

Lastly, to close with all the caveats expressed in this previous post still apply. The data is gleaned through a very manual process, so please do feel free to point out any inconsistencies in it. The opinions and analyses in the blog are open to suggestions and critique.