Friday, 24 January 2014

Looking at Governors and parties in US

PoliticoMagazine has ranked all 50 states in the United States.  The method is pretty objective:

"With this in mind, Politico Magazine rounded up 14 different state rankings from reputable sources like the Census Bureau, the Centers for Disease Control and Prevention, and the FBI, and on important factors such as high school graduation rates, per capita income, life expectancy and crime rate. Then we averaged out each state’s 14 rankings to come up with a master list—atop which sits none other than New Hampshire."
The result is graphed here, showing all 50 states, Blue bars are a state with a Democratic governors, red is a state with a Republican governor. 

Big Brothers big failure

Many people in Ukraine have received txt messages indicating that the phone company, and thus the government, has them registered as protestors.  People have responded to this with concerned statements that it was 'Orwellian' which means they are again linking it to the idea of Big Brother in Orwell's most read book 1984.

And what was the response?  The protests in Ukraine have expanded, the government has been forced for now in to a retreat.

Seems Big Brother has failed.

In fact if you re-read 1984 you will notice that Big Brother failed at the end of the book.  Everyone takes the sufferings of Winston and his final destruction by the party as evidence that Orwell believed that the technology of surveillance and psychology along with ruthless ideology could crush the individual and reduce humanity to a machine.

Assange and Snowden, along with many people in the EFF and all over the web are fond of pointing out that the current technology of agencies like the NSA dwarf anything Orwell could have imagined back in 1948.  And on a level this is true, were are being watched all the time and the state and private companies, along with criminal groups, have massive amounts of data about us, they have been collecting it for a long time.

But does the collection of data mean effective execution of power?

The final line of 1984 is written in lovely, complex, subversive and critical English:

It was chiefly in order to allow time for the preliminary work of translation that the final adoption of Newspeak had been fixed for so late a date as 2050.

But obviously the existence of the sentence above proves that in the end Newspeak and the project of Big Brother failed.  The novel closes with discussion of the Declaration of Independence, of Milton, Shakespeare, Byron and a discussion of freedom.

In fact Orwell closes the entire peace with 'The Principles of Newspeak' where he opens:
Newspeak was the official language of Oceania and had been devised to meet the ideological needs of Ingsco, or English Socialism.
Not only is the state clearly set in the past but it is clearly political, it was the kind of statement Newspeak was suppose to make impossible, but somehow in the future Ingsoc and Big Brother and Newspeak vanished and English and Shakespeare and the Declaration of Independence survived.

Clearly Orwell didn't think much of Big Brother's long term chances.

And neither should we.  Just possession of data about subjects in a society does not give power.  Every parent knows this.  They know so much about their children, and this knowledge can come to such little effect.  Anyone who has worked as a school teacher, a social worker, a home care giver or a police officer can tell you that though information is critical, it is more often than not of limited value in controlling the subject.

Foucault was certainly brilliant for his insight that modern prison created a regime of surveillance.  But Foucault also understood that power always created its own opposition, that these regimes of watching resulting in regimes of hiding and deviance.

Yes the United States has a historically unprecedented ability to collect, store and manage information about not only its people but the people of the world.  And yet how much has this ability given it?  Did it enable the US to win the war in Iraq?  Could reading people's mobile phones or emails enabled it to end the conflict in Syria?  To stop protests in Europe?  Was the US able to stop massive leaks of its data by Manning and Snowden?

If I was given control of the NSA tomorrow the first thing I would look at, before abuses, is value for money.  How much reduction in crime, terrorism and radicalism did all this work get the US?

I think we should remember Gödel who proved that mathematics could not be fully reduced to logic, that statements could be true but could never be proven.  If mathematics, the most formal activity humanity undertakes has pieces of illogic installed in it then what for people.

The reality is that humans are not formal systems, that if you know every fact about a person they can still surprise you.  Humans are very good at hiding their true intentions, changing their minds before anyone can learn about it, and engaging in irrationalism.

As I write this it is being report that the 2012 Philpott murders in Derby England, where parents set fire to a house with their own children inside killing six children, could not have been predicted.  The UK is the ultimate surveillance state. The Philpotts would have been monitored not only by police, school and social workers.  The Philpotts may have been one of the most watched families in the world.

And yet after the murders of the children the Council had to conclude that there was no way that anyone could have predicted the outcome.

I was in Cairo at the start of 2011.  Even though it was pretty obvious the Regime was not popular, there was no reason to think that in 2 months a revolution would break out.  I travelled through Spain in 2007 and what reasons were there to conclude the booming economy was about to collapse.

Perhaps the most obvious example is the 2008 stock market collapse.  Not only did some of the worlds most advanced computers and analysis software fail to see it coming, to a large extent the existence of this software promoted and may have caused the collapse.

My reading of 1984 is a bit different than most.  I see O'Brian in a desperate effort to remove free thinking and descent, his vicious confidence is a defence mechanism from the fact that no matter how much the Party cracks down on humanity, humanity remains time and time corrupt, irrational, selfish and free.  I see the massive data collections by Big Business, Big Brother and Criminal Groups as much a desperate effort to try and find something of value, an effort that has failed again and again, and always will fail.



Tuesday, 21 January 2014

Tweets coming from #BenefitsStreet

With enough people talking about them on twitter what are the people who live on the now famous Benefits Street tweeting about? Most ordinary stuff, some humour about the show, some anger.

Monday, 13 January 2014

Twitter map of #KellyThomas verdict

Social Media and retail Christmas in London

The analysis I did below shows me the limitations of extracting data from social media activity.  From November to December the UK retain sector saw strong growth, that I could not see in the Twitter and Foursquare data.  Either I missed something or the information was not captured.  But trend line analysis after analysis showed no significant rise in tweeting from shopping areas in London.

This chart shows hourly tweeting for Oxford Street from through November, December, and early January.
Analysis of the levels of tweets coming each hour from a KM around Oxford Street through November and December fail to give an evidence of a major surge in foot traffic on London's main high Street in the run up to Christmas or even on Boxing Day.  This does not establish casual evidence that the recovery is in any trouble, but there is a lack of social media data coming from London.

Beyond the lack of a bump in Tweets coming from Oxford Street there is a dramatic lack of Foursqaure  checkins to major retailers in London, with only Selfridges & Co, Apple Store and Liberty showing significant foursquare checkins.

Most Foursquare activity were at public spaces and events, leading me to wonder if people were looking for cheaper days out.  Tate Modern, Covent Garden and public Markets were major locations for Foursquare.

It is very hard to draw conclusions from Twitter and Foursquare public data, but these numbers do not indicate a large surge in Christmas shopping at the end of 2013 in London.