Good to monitor and correct but not necessarily bad overall. Wall Street and others have always used questionable data at many levels. In a neighborhood that I lived in for a long time, it was very obvious that once-every-10-years census data was almost completely controlling development. The existing credit laws should force companies to expose what factors they used in making a decision. That should either expose or limit the use of new types of big data. One thing we need more of is communities comparing themselves to others in comprehensive ways to realize that they have problems that they should address. The open data movement, open government data especially, is extremely helpful for this. This is a really good example: http://data.cityofpaloalto.org/home One good result of open data and democratized big data processing might be that we become more efficient in good ways. For instance, it would be good if we could become more effective at meritocratic commerce and innovation while drastically lowering the amounts skimmed by the Wall Street-driven finance industry. There are too many cases where Finance absorbs much if not all of the profit generated by some companies and industries. I don't mind them getting paid for their value, but often they take it all and then some. Leveraged buyouts and pension raiding are the prime examples of this, but there are others. sdw On 9/10/16 8:20 AM, Razer wrote:
There's are a number of reasons why 'teh gubmint wants ur dataz', and it's not ONLY about killing 'terrorists' or suppressing potential insurrection. It's about ripping you off for every penny they and their corporate BFFs can shake out of you.
Rr
Big Data Isn’t Just Watching You—It’s Making You Poorer
Cathy O’Neil’s new book, Weapons of Math Destruction, shows mathematical models aren’t free of ideology.
BY Pankaj Mehta, In These Times, Sept 6 2016
By some estimates, humanity now produces 2.5 quintillion bytes of data every day—more than a hundred times the amount of data in the entire Library of Congress. This data ranges from Facebook posts to military-grade satellite photos. Increasingly, this data is analyzed by complex mathematical models that determine more and more aspects of our lives, from the advertisements we see to whether we have access to private insurance. Yet despite their growing importance, these models often remain hidden.
Advocates of such mathematical modeling, in both the public and private sectors, portray it as a neutral and efficient alternative to fallible and biased human decision-making. Mathematician, data scientist and popular blogger Cathy O’Neil, author of Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, doesn’t agree. She argues that many mathematical models are ideological tools that exacerbate oppression and inequality. Her examples range from the crime models used by police departments to determine which neighborhoods to patrol, to the recidivism models used by judges to hand out prison sentences.
O’Neil is passionate about exposing the harmful effects of Big Data–driven mathematical models (what she calls WMDs), and she’s uniquely qualified for the task. She earned a Ph.D. in math from Harvard and landed a tenure-track at Barnard. But she became bored with the pace and insularity of academia, and left to work as a quantitative analyst at the hedge fund D.E. Shaw. There, she had a front-row seat for the 2008 financial crisis.
This experience fundamentally changed O’Neil’s relationship to mathematics. She realized that far from being a neutral object of study, mathematics was not only “deeply entangled in the world’s problems but also fueling many of them.” People in power were “deliberately [wielding] formulas to impress rather than clarify.” This disillusionment led O’Neil to get involved with Occupy Wall Street and start educating the public about the dangers of WMDs through her blog, MathBabe.
She is careful to point out that there is nothing inherently destructive about mathematical modeling. Sophisticated data modeling enables much of modern technology, from wireless communication to drug discovery. How can one distinguish a destructive math model from an ordinary, or even helpful, one? O’Neil identifies three key features of WMDs: lack of transparency, lack of fairness and, most importantly, operation on a massive scale.
O’Neil grounds her argument in case studies of WMDs in a variety of settings: finance, higher education, the criminal justice system, online advertising, employment decisions and scheduling, and credit and insurance provision. The “value-added” model for teacher evaluation, which looks at improvements in individual students’ test scores—a favorite of the so-called “educational reform” movement—is touted as an objective measure of a teacher’s worth. Yet this is far from the truth. O’Neil cites an analysis by blogger and educator Gary Rubinstein of New York’s 2010 value-added scores for public school teachers. O’Neil explains that “Of teachers who taught the same subject in consecutive years, one in four registered a 40-point difference. [This] suggests that the evaluation data is practically random.” O’Neil argues that this is because the value-added model, which relies on predictions of student performance, suffers from a built-in logical flaw: No statistical model can accurately make predictions about a class of 25 or 30 students—the sample size is too small. Yet the high-stakes testing regime continues to wreak havoc on the trajectories of students and teachers alike.
Other WMDs, known as e-scores, use data such as ZIP codes, web-surfing patterns and recent purchases to evaluate a person’s credit-worthiness. Unlike the more familiar FICO credit scores that are freely available and regulated by the government, these secretive e-scores are “unaccountable, unregulated and often unfair.” Whereas FICO scores are based on your own financial history, e-scores compare you to other people with similar profiles. This may seem benign, but it can result in feedback loops that reinforce existing social inequities. If you live in a poor ZIP code, then your e-score will drop, meaning less credit and higher interest rates—essentially, an algorithmic redlining of the poor and working class.
More: http://inthesetimes.com/article/19364/the-numbers-do-lie
sdw