Friday, May 16, 2014

Deciphering UP

Results of the elections to the 16th Lok Sabha will be declared tomorrow.  It has been a long and arduous battle, culminating in a crescendo with the marquee contest for Varanasi in the last round of polling. The voting is all done now and as we move towards counting day, Uttar Pradesh, is the center of all the attention. Without even going into the efficacy and accuracy of exit polls all over again, it is almost certain that once again the result in UP will determine who rules in Delhi. This state that elects 80 of the country's 543 legislators has given it 9 of its 13 prime ministers. Its small wonder that the latest aspirant to that position rushes to UP to get the right credentials. But India's most electorally significant state is also its most enigmatic one. This post is about trying to understand Uttar Pradesh better to see if there are any pointers to what is going to happen tomorrow.

To start with it is perhaps good to get an idea of the scale we are dealing with here. UP is the biggest state of India by population and 4th biggest by Area. With a population of ~200M if UP were a country it would be the sixth most populous in the world. Secondly, even though it seems to be a single large swathe of riverine plains extending across the northern part of India, it is pretty diverse in terms of the demographic. Essentially we are dealing with a reasonably complex creature here, and it needs to be broken into smaller, more manageable regions to do any meaningful analysis. The following graphic shows something I have done in a hurry. It is not based on any official source, but rather something rough to give some idea of bearings.



The Doab is the name given to the narrow strip of land between the rivers Ganga and Yamuna the two most prominent features of UP's political geography. This extends all the way from the northern border of the state with Uttarakhand where the two rivers originate to Alahabad where they meet. This is split further into what is called the Upper Doab, or the Jat heartland of Western UP and the Lower Doab (or Doab here) extending from Agra to Allahabad. Rohilkhand is the region wedged between the Ganga and the border with Nepal and gets its name from being hitorically ruled by the Rohilla Afghans of Bareily. Similarly the region below the Yamuna is called Bundelkhand and stretches into northern part of the adjoining Madhya Pradesh. The eastern part of the state is often referred as Purvanchal together with a large part of western Bihar. Awadh is the area around the capital city of Lucknow between Rohilkhand and Purvanchal, this was the area hitorically part of the empire of the Nawabs of Awadh (Oudh). Besides the historical, geographical and demographic context to the way UP is devided into regions, there is also a strong distinction in economic aspects. The western parts of Up for example benefitted from the green revlution much more than the eastern ones and that has led to a significant economic divide between these regions. Needless to say these regions exhibit significantly different voting behavior and thats where we go nex

The graphic below shows the results of elections in UP from 1977 to 2013. Use the < and> buttons to scroll across years or the dropdown to go to a particular year.





As one can see, UP is not a state averse to whitewashes. While in 1977 the BLD wiped out the Congress from UP on the back of a strong anti emergency sentiment, The Congress did the same in 1984, winning all but two in 1984 (Etah and Baghpat which to this day remain agrarian/socialist strongholds). What seems to be interesting though is that since 1989 the contest has got more and more quadrangular, with the BJP and then the BSP emerging as strong forces in the state. Consequently we can see this in a four way split of the seats between the 4 main parties. Also 1998 was the closest that UP has come to a one sided result in a while and in the last 4 elections there has been a convergence in the vote shares and seats won by various parties.(notice funnel shape in vote share graphic below).




Also worthwhile to note is where we ended in 2009. The 2009 election saw as clear a 4 way split in terms of seats as UP has ever seen. Even though the BSP was not the largest party in terms of seats, it did end up with the highest vote share (rather comfortably). At 27% the BSP's voteshare was a good 10% higher than the BJP and Congress and 4% higher than the SP. However even more interesting is the gradual consolidation across 5 general elections that this culminates in.  This looks like a base that has been gradually built over time something that might not be very easy to blow away with a wave. The vote base of the Samjwadi Party seems a little more fickle, but only a little. If the BJP has to get to levels anywhere near to where it was in 1998 and what the opinion polls are predicting, it has to increase its voteshare by a whooping 20%. As in much of the country the BJP would be hoping that at least a part of it would come from the Congress voteshare (say 10%), but that itself is not going to be enough. The BJP will have to make a significant dent in the well established caste coalitions of the SP and BSP. There has been some debate around 'wave' and 'tsunami', if the BJP has to get back to 40% voteshare and 50+ seats, it better be a tsunami because a mere wave wont do. I will try to explore a bit more about where the remaining 10% is going to come from.




The graphic above shows the regional distribution of seats between various parties in 2009. We can see that in 2009 the Congress put up a very credible performance to emerge as a force in Awadh whereas in the Doab region the SP continued to maintain its stranglehold winning 6 of the 9 seats (this includes traditional SP bastions like Mainpuri, Etawah and Kannauj) . The BSP turned out with its best performance in western UP and Purvanchal (where the spoils were almost equally split in 3 and 4 cornered contests, many of the quiet close- Graphic below)




One more thing that I tried to look at was what are called bastions. Or seats that some parties seem to win with reasonable frequency. One could follow with the argument that if a party Wins often from a seat and if the margin was particularly large in the last election, chances are that they would retain it. To look at this i looked at a 3 election horizon (and tried to find seats where one party had won twice or more) and a 5 election horizon (and tried to find seats where one party had won thrice or more) The results are shown below. 




In the 5 election horizon - the BJP shows up with 20 seats where it has won more than 3 times. Interestingly a majority of these seats are concentrated in western UP and Purvanchal apart from a few urban centers like Lucknow and Bareily elsewhere. Lucknow and Gorakhpur are 2 seats that the BJP has won each time in the last 5 elections, true bastions. Not even Banaras qualifies for that distinction.
In the case of SP there are 17 such seats. Their strength in the doab is reflected in the strongholds like Mainpuri, Kannauj and Etawah. Kaiserganj, Mohanlalganj (in Awadh) ,Jalaun (Rohilkhand), Balia and Phulpur are others that are less widely known.
In the case of Congress its the usual Amethi, Rai Bareily, Pratapgarh, Kanpur, whereas in the case of the BSP they have a couple of strongholds in Purvanchal (notably Akbarpur) and few in Awadh. To note here is the fact that the BSP is a late entrant and hence seems to be under represented in the 5 election horizon.





The case of BSP was the reason for looking at the 3 election horizon separately  (which has a higher recency effect and in the case of the BJP a higher dose of reality). The big change between the 5 election and 3 election horizons is the viibility of the BSP as a force in northern Purvanchal of late, as well as emergence of SP as a force in southern purvanchal. The good news for the BJP is that they still seem to be retaining their strength in western UP, even in the more recent 3 election horizon.

So where does all of this leave us in the context of our original question on what it would take for the BJP to make it to the 40% vote share, 50 seats mark in UP.

Lets start with Western UP. It is the only region in UP where the BJP was still number 2 in terms of voteshare (having been relegated to 3 and 4 in other regions). Also to keep in mind is that if we included a part of the RLD (part of NDA in 2009) to the BJP vote share it gets even better (though still 3-4% behind the leader BSP). To start with western UP seems to be the BJP's best bet to get back in the game in UP.  If the BJP can take away 6-8% from the Congress and supplement it with a base among new first time voters (which will be a significant factor this time) they have a good chance of winning western UP without making a significant dent in the base of the BSP and the SP. The recent riots and resulting polarization will also play a part here.
In Rohilkhand - a similar scenario presents itself here, with the difference that the difference between the BJP and the leaders SP, BSP is lower at about 5%.. So if the BJP could increase their vote share by about 10% points taking from the Congress and gaining supplementing with new voters, it still barely gets it slightly in front the leaders. To get back to levels where it was in 1998 (37%), it will have to make a severe dent in the bases of BSP and SP.
Moving on to the Doab, here the BJP is more than 22% behind the leader SP. One wonders whether even a tsunami can make a dent for the BJP in this Yadav heartland, where seats like Mainpuri, Etawah and Kannauj elect the SP with heavy margins in election after election. 

Awadh is a slightly different case. Here even though the BJP is behind the leader by 20% (like in the Doab) that leader happens to be the Congress in this case. After Western UP this seems to be another area where the BJP would be looking to make some serious inroads. It needs to be noted though that even if the Congress seats and voteshare seems to be up for grabs the same cannot be said about SP and BSP which have some notable bastions in this area (Mohanlalganj and Kaiserganj for SP and Sitapur, Misrikh for the BSP)

Lastly to the largest and most important one, Purvanchal. Here the BSP has established itself as a significant force in the last two elections, With highest vote share (still only 27%) and joint highest number of seats. Most of Purvanchal saw 3 and 4 cornered contests some of them very close. The other interesting  point is that the BSP seems to be the only party which is strong across Purvanchal (in 2009). The SP seems to be stronger in Southern Purvanchal, while the Congress was stronger in the northern part and parts bordering Awadh.

To sum it up, If the BJP has to get to 50 seats (36-38%) vote share in UP, it has to sweep Western UP. To add to it it has to literally replace the congress in Awadh and make severe dents in the BSP's vote base in Purvanchal, to get ~2/3 of the seats in both regions. BJP will have its biggest challenge, and a worthy challenger in the form of Mayavati and BSP. It will be difficult, bordering on impossible and if they can pull it off, nothing less than a tectonic shift in the politics of UP.




Wednesday, May 7, 2014

The legend of 'votekataua'

I heard this term for the first time in Benaras a month ago, at a lassi shop, in the context of Arvind Kejriwal. Almost a month has passed since, and from what I hear Arvind and AAP have since risen much in Varanasi's esteem - he is considered a serious contender now. Though used in a negative connotation, in reality, the phenomenon of 'votekataua' is a very natural consequence of the voting system that we used. In the first past the post system (or simple plurality system the 'winner takes all'.  The candidate with the maximum votes (not the majority of votes) wins the seat. Let us consider a hypothetical constituency with party A and party B engaged in a tight two cornered contest. In a FPPS system getting more vote for oneself is not the only way for party A to win an election. The other way obviously is to cause a division in the votes of party B such that both parts are now less than party A. The new entity that wins enough votes to severely damage the chances of the challenger but not quite enough for itself to win, advertently or inadvertently plays the role of a vote-cutter or 'votekataua'. Come to think of it, if we followed a proportionate representation system, this word would probably not exist.

The word is new but the phenomenon is not. I have talked about this in detail in the context of the Karnataka assembly election (Will Yeddyurappa do a Bangarappa?).  We went on to see how Yeddyurappa spoiled the prospects of the BJP in that election (Karnataka 2013). The Congress romped home with a simple majority without increasing its vote share by much.

In this post I want to look at 2 more parties who earned this title in the last election. Specifically we will look at the case of estranged nephew Raj Thakray's Maharashtra Navanirman Sena (MNS) in urban Maharashtra and Matinee idol Chiranjeevi's Praja Rajyam Party (PRP) in Andhra.

The Maharashtra Navanirman Sena was born out of a fratricidal war of succession within the Shiv Sena. When Bal Thackray retired from active politics in 2006 he appointed his son Uddhav as leader of the party resulting in Raj Thackray breaking away to form a party of his own. The MNS in many ways was an ideological clone of the Shiv Sena, with similar 'marathi manus' DNA and hindu nationalist leanings. In the initial years of its formation the MNS adopted a particularly virulent  xenophobic stance against north Indians in Mumbai, almost as a way to differentiate itself from its parent, leading to several bouts of violence in the city. All the while amidst clamor for action against the MNS to contain the violence the Congress/NCP state government stayed away, letting MNS build its own constituency by cutting into Shiv Senas base. It turned out to be a masterstroke. As we can see in the figure below, even though the MNS was not able to win a single seat in Maharashtra, it still managed to put up a reasonable contest. The result was that what was expected to be primarily a 2 cornered fight across much of the state did not quiet pan out that way. Much of Maharashtra saw three cornered contest, with the MNS coming up as a strong third force in urban centers and the BSP consolidating its position in much of Vidharba to emerge as a contender there (Graphic below)




Between - 2004 and 2009 - the Congress +NCP increased its number of seats from 22 to 25 inspite of its vote share falling by almost 4%, primarily because the BJP-Shiv Sena combine lost even more (7.5%). 4% of that drop is a straight shift of Sena voters to the MNS.




 The effect of the MNS was particularly severe in the Urban centers. A little quick math shows how in all the 10 of the 11 constituencies that the MNS  contested from  it severely dented the prospects of the BJP-SHS alliance while not managing a single seat for itself. 



Had it not been for the MNS the results from Maharashtra would have read 30-15 in favor of the BJP-SHS instead of the final 25-20 in favor of the Congress-NCP. As far as 'Votekatauas' go, the MNS in 2009 was a textbook example.

Moving on to Chiranjeevi and his Praja Rajyam Party. The PRAP was the other impressive debut of 2009. In their first election The Praja Rajyam Party won a whooping 15% of the vote share in Andhra, but did not have a single Lok Sabha seat to show for their efforts. The PRAP however did manage to turn to contest in much of Andhra into a 3 cornered affair. 




With the breakup of the NDA and the emergence of the PRAP the the divided opposition was no match to the strong Congress. As per a lot of observers - because of the emergence of the PRAP the anti incumbency vote against the Congress got split in half, making it possible for YSR Reddy to deliver Andhra for the Congress for the second time in a row, with a higher number of seats in spite of a fall in vote share.




And so it happens every time, it is something like an adoption barrier in marketing parlance. New parties have to always face the challenge of being seen as nothing but a vote cutter. Its kind of a self fulfilling prophesy, this thing about being a 'votekataua', if people think that's what a party is, that is what it is likely to end up being being (tactical voting in FPPS).  But then again there are parties that are able to cut through this, communicate their message to the voters and convince them of a possibility of victory, the AAP campaign in Delhi is an excellent example of this. New parties are the needed in democracy, they keep it fresh and bring in new ideas and leaders. While it is important to understand the phenomenon of the votekataua as a tactical weapon in democracy, it is also important to get rid of some of this negative connotation with wasted votes.






Saturday, May 3, 2014

No country for cubism.

The brief study of the types of democracy and voting systems was interesting as well as threw up a couple of insights for me. Still, mulching Wikipedia is really not my favorite pastime, this is a datablog and yours truly is a data-wrangler, its time to go back to the roots. The plan in the next couple of posts is to go deeper into the consequences of the 'first past the post' voting system that India uses for its elections with the lens of data.  In this post I also want to make an attempt to understand the twisted relationship between vote shares and seats won by different parties. In the process I also want to take a closer look at polygonal contests in this voting system.

There is a tendency in large parts of the media to simplify the narrative of this election. Straight contests lend easily to tv debates and montages, Congress vs. BJP,  Rahul Gandhi on one side, Narendra Modi on the other and the parliament house in between. A rather simplistic picture, with undertones of a wave of voters moving from one camp to another that needs to be exploded further. The graphic below shows the vote shares and seats of the primary Parties/agglomerations over the years as also combined vote share of the two main parties- the Congress and the BJP in the last few elections.






Two observations are notable here. Firstly, from (b) and (d) the strength of the two remaining national parties in terms of the votes they poll seems grossly overrated. Even if we were to add up the vote shares secured by the BJP and Congress that number often doesnt even cross 50% and if anything seems to be going down in the last couple of elections. More than half of the electorate votes for neither the BJP nor the Congress. This remaining 50% is highly fragmented among hundreds of parties (mostly regional) and this makes elections across large parts of India multi-cornered (polygonal) affairs.

Secondly from (a) and (c) the correlation between the vote shares and seats won is not straightforward.  However It is well known that the first past the post voting system accentuates differences in vote share and tends to reward parties with higher vote shares disproportionately- case in point being the 1984 election when the congress with 49% vote share swept the election winning almost 80% of the seats. Again this is not all that straightforward. In the three elections between 1996 and 1999 the BJP won more seats than the congress even though it had lesser vote share nationally.   A more nuanced region by region view, with an understanding of the 'polygonality' (how many corners) of the contest is needed to get to better answers and that's where I intend to go to in this post. Also there are empirical observations to correlate the the vote share and seats won - for example the the "cube law" which  states that in a two cornered contest the ratio of seats won is proportional to the cube of the ratio of  vote shares. I want to round up this post with a quick empirical test of the law of cubes in the Indian context.

To understand 'polygonaility' by region i.e which seats/regions in the country have straight, triangular, quadrangular and even pentagonal contest, some definitions are needed first. Given that most seats have 10 or more contestants, how do we define whether a contest is straight/triangular/quadrangular? The graphic below shows the frequency distribution of the vote share of the top 2 candidates in all seats in the 2009 election  



In about 22 constituencies, the top two candidates together secured between 95% to 100% of the vote share. In the context of Indian democracy this is as direct a contest as it gets. A quick look at the cumulative numbers reveals that the top two contestants together constitute more that 80% of the vote share in 272 constituencies out of the 543 (roughly half). 80% is the number I chose as the threshold for whether a contest is classified as 2 cornered, 3 cornered or 4/5 cornered. Essentially, if the top 2 make up more that 80% of the vote share it is classified as  straight or  2 cornered contest. If that is not the case but the top 3 constitute more than 80% of the vote share then it is classified as a triangular or 3 cornered contest and similarly for 4 cornered (quadrilateral) and 5 cornered (pentagonal) contests.

The following graphic shows the geographical distribution constituencies based on the polygonality of the contest in 2009.




It is interesting to see that the contest is essentially 2 cornered (by the given definition) in large parts of Punjab, Himachal, Rajasthan, Madhya Pradesh, Chhattisgarh, Gujarat, Northern Karnataka, Kerala and West Bengal. Meanwhile  large parts of  Tamil Nadu, Andhra Pradesh, Orissa, Assam and Maharashtra  witnessed three cornered contests where as other parts of UP, Bihar and Jharkhand witnessed triangular/quadrangular and in some rare cases even pentagonal contests. In Nawada, Bihar the top 4 condidates secured just 62% votes between them, with the winner getting away with merely 22% vote.

The following graphic shows different states characterized by the polygonality of the contest in its seats.





Looking at the major states one by one, in UP we find the usual suspects(BJP, BSP, SP, Congress) engaged in a 3 or 4 cornered contest in most seats. The notable exceptions to this are the usual pocket boroughs of various parties where one predominant contestant gives the appearance of a straight contest. For example Mainpuri, Etawah (SP strongholds) and Amethi, Raebareily (for INC).

 In the case of Maharashtra one would be expecting it to be a straight contest between the NDA (BJP+SHS) and UPA (INC+NCP) but it is not quiet. In a large part of Vidharba the BSP among others offers a third alternative making it a triangular contest and similarly in the Urban areas of Mumbai, Pune and Nasik, the MNS was able to eat into significant vote share of the Shiv Sena to make it a 3 cornered contest.

AP sees quiet a number of 3-4 corner contests with Congress, Telegu Desam, Telangana Rashtra Samiti, Chiranjeevi's Praja Rajyam party and in a few case the BJP fighting for the spoils.

Rajasthan, Gujarat, Madhya Pradesh, Chhattisgarh, Punjab, West Bengal and Kerala witness primarily straight contests. The only nuance being that while in the case of the first 5 it is primarily between the Congress and the BJP, the latter 2 were primary between Congress vs Left/others.

In both Bihar and Jharkhand , multiple regional parties (some of whom fight elections in both the contiguous states) lead to 4 cornered and 5 cornered contests.

To wrap up I want to take a look at the applicability of the law of cubes in smaller subsets of seats, where there is a direct contest. Of the 272 seats where there seams to be a direct contest , The NDA and the UPA were locked in direct contest in only about 150 seats. The states of Gujarat, Madhya Pradesh, Maharashtra,  Rajasthan, Chhattisgarh,  Delhi, Punjab constituted a bulk of these seats.  Of the remaining 122 direct contest seats - the UPA was competing with a regional party/Third front in 107 seats (primarily West Bengal, Kerala and Tamil Nadu), the NDA was competing with regional parties/3rd front in 8 seats and the contest was between 2 regional parties in the remaining 7. Must be said here that in West Bengal and Tamil Nadu, the UPA had significant allies in 2009 which it has since lost. The contour of elections in West Bengal will be more and more 3/4 cornered and the same would go for Tamil Nadu where the old Congress alliance with DMK no longer exists.


 The graphic on the side shows a quick analysis shows that the law of cubes is only roughly applicable to even this set of constituencies where we have as close to a direct contest as possible in the Indian context. What the law can do for us is that it can give us a rough idea of what a change in vote shares would do to the number of seats won (on the 150 seats where there was a direct contest). The above calculation also illustrates the effect of a 10% change in vote share. As for trying to predict the number of seats at a national level based on a shift in vote shares (from poorly opinion polls), well thats a job better left to the professional psephologist.