Pencil Icon

Big Data is Not the New Oil

After reading this article by Jer Thorp, I’m still trying to think about how The Arts could transfigure the way we think about data. Will be on the lookout for examples.

- Laurie


Big Data Is Not the New Oil

by Jer Thorp, HBR Blog Network, 8:01 AM November 30, 2012

Every 14 minutes, somewhere in the world, an ad exec strides on stage with the same breathless declaration:

"Data is the new oil!"

It’s exciting stuff for marketing types, and it’s an easy equation: big data equals big oil, equals big profits. It must be a helpful metaphor to frame something that is not very well understood; I’ve heard it over and over and over again in the last two years.

The comparison, at the level it’s usually made, is vapid. Information is the ultimate renewable resource. Any kind of data reserve that exists has not been lying in wait beneath the surface; data are being created, in vast quantities, every day. Finding value from data is much more a process of cultivation than it is one of extraction or refinement.

Still, there are some ways in which the metaphor might be useful.

Perhaps the “data as oil” idea can foster some much-needed criticality. Our experience with oil has been fraught; fortunes made have been balanced with dwindling resources, bloody mercenary conflicts, and a terrifying climate crisis. If we are indeed making the first steps into economic terrain that will be as transformative (and possibly as risky) as that of the petroleum industry, foresight will be key. We have already seen “data spills” happen (when large amounts of personal data are inadvertently leaked). Will it be much longer until we see dangerous data drilling practices? Or until we start to see long term effects from “data pollution”?

One of the places where we’ll have to tread most carefully — another place where our data/oil model can be useful — is in the realm of personal data. A great deal of the profit that is being made right now in the data world is being made through the use of human-generated information. Our browsing habits, our conversations with friends, our movements and location — all of these things are being monetized. This is deeply human data, though very often it is not treated as such. Here, perhaps we can invoke a comparison to fossil fuel in a useful way: where oil is composed of the compressed bodies of long-dead micro-organisms, this personal data is made from the compressed fragments of our personal lives. It is a dense condensate of our human experience. This re-framing of data into a human context is crucial. I believe there are three things we can do to make data more human, and in doing so generate much more than short-term business value:

First, people need to understand and experience data ownership. While everyone in our society is producing vast quantities of data, individuals rarely see or interact with any of it. When people are given tools to store, visualize, and explore their own data, they gain an understanding of the worth and utility of this information. Applied on a broad scale, this improved understanding of data could lead to better decisions by individuals — both in cases where there data is being misused and in cases where data can be applied to solve important problems like disaster response, cancer diagnosis or disease spread.

Second, we need to have a more open conversation about data and ethics. Of the dozens of start-ups who have approached me for advice on their personal data-centered ventures over the last year, not a single one has mentioned the rights of the people from whom the data is being extracted. This needs to change. I suspect there will be tremendous benefit for companies who position themselves as “data humane” and that ultimately this could be the standard practice for business as consumers become more data-savvy.

Finally, we need to change the way that we collectively think about data, so that it is not a new oil, but instead a new kind of resource entirely. For this to occur we need to foster a deep understanding of data in society. As it happens, humanity has a mechanism for this kind of broad cultural change: the arts. As we proceed towards profit and progress with data, let us encourage artists, novelists, performers and poets to take an active role in the conversation. In doing so we may avoid some of the mistakes that we made with the old oil.


Images are from my watercolor of the crowd at a Giants baseball game, run through Illustrator’s mosaic filter.

Pencil Icon

Raptor Identification on Hawk Hill


When the ocean winds hit the shear cliffs just north of the Golden Gate Bridge, the updrafts can lift migrating raptors high enough to soar for miles. Hardcore birders with spotting scopes come out every year to count the fall migration, calling out identifications on bird-like flecks so distant they seem like floaters in my eyes.


The ability to make those ID’s seems like divination to me, but the Golden Gate Raptor Observatory is trying to let ordinary humans in on the game by handing out a flyer comparing raptor silhouettes. 


That’s great, but what really excites the information designer in me are the lines someone painted on the pavement, comparing the lengths of different raptors’ wingspans. I keep coming back to look at it, even though it doesn’t much help me identify those distant flying specks. (It also doesn’t help that it reminds me of those plastic coin sorters where the penny rolls down until it falls into the right slot, and then I start wondering how stiff a turkey vulture would need to be to fit into one of those slots…)


There is something really satisfying about this simple chart. I love that it is actual-size—that I could lie down on the ground and measure those wingspans with my earthbound body, spreading my arms to find my own wingspan. But that’s an altogether different kind of identification.

Pencil Icon

An app that filters transience

The Tourist Eraser app for Android smartphone, developed at a recent hackathon in Berlin, removes the people from your photos of buildings by filtering out all the transient elements. You take multiple photos and it erases anything that has moved, leaving only the static parts of the images.

This begs the question: If you took enough photos, over a long enough time, would you end up with an empty frame?

Pencil Icon

Maps of the mind

From Bento’s Sketchbook by John Berger

When I’m drawing…I have the impression of participating in something like a visceral function, such as digestion or sweating, a function that is independent of the conscious will…something prototypical and anterior to logical reasoning.

Thanks to the recent work of neurobiologists like Antonio Damasio, it’s now known that the messages which pass from cell to cell in a living body do so in the form of charts and maps. They are spatial arrangements. They have a geometry.

It is through these ‘maps’ that the body communicates with the brain and the brain with the body…. In the act of drawing there’s perhaps an obscure memory of such map-reading.

Drawing is anyway an exercise in orientation and as such may be compared with other processes of orientation which take place in nature.

When I’m drawing I feel a little closer to the way birds navigate when flying, or to hares finding shelter if pursued, or to fish knowing where to spawn, or trees finding a way to the light, or bees constructing their cells.

Maybe this explains why I feel so closely connected to the thing I’ve just sketched, like it has become a part of me that I will never lose. Maybe the act of sketching has somehow drawn that thing onto the larger map in my brain that structures my experience of the world.

Pencil Icon

Voting shifts in 2008 and 2012

This animated data visualization from the New York Times does an elegant job of representing a complex dataset with two simple images.

The power in these images lies in the animation: mousing over the date causes the animation to spring to lifes with many lines, waving to the left or right like a field of red and blue sea anemones, to show how voters shifted political orientation in each election. The length of the line represents the magnitude of the shift. The difference between 2008 and and 2012 is startling.

Like all good data visualizations, it invites questions about the story behind the numbers. What caused all the blue shifts in the South in 2012? Why is the area below the Great Lakes so volatile, with big shifts in each election, in opposite directions? Why so little shift in Arizona?

Click here to view the animation, and to see more graphics showing how different demographic groups shifted.

Pencil Icon

San Francisco’s Mission District, through many filters

Mission Possible: A Neighborhood Atlas is a collaboration between Mission Loc@l and cartography students at UC Berkeley. “The map is conceived as a narrative of place, using data visualization techniques, cartographic symbology and graphic art and design concepts to tell different stories.” Those mapped stories include babies, dogs, demographic changes and water underground.

Here are some some of the maps. (Note that the conventional map compass positions have been rotated: WEST is on top.)

This one shows the number of Parking meters (darker orange areas have more) and people per parking space (darker blue areas have more people per space)

Here are locations of light manufacturing.

Locations of coffee shops and the decrease in Latino residents (darker green areas have highest loss of Hispanic population).

For $25 you can get all 22 of them printed out as tabloid-sized posters for your wall.

Pencil Icon

Subverting the TOC: the anti-filter

Standing in line at the supermarket a few weeks ago, bored and looking for something to distract me from those bright yellow bags of Peanut M&M’s by the cash register, my eyes landed on a a Vanity Fair cover with an intriguing headline: “How Microsoft Lost its Mojo.” Intellectual stimulation won out over glucose stimulation, and I carried the magazine out of the store, looking forward to finding out how such a big and powerful company could make such seriously awful software.

But when I got home and tried to find the table of contents, I was frustrated by page after page of ads. I flipped back to the beginning, sure I must have missed it, then tried opening it in the middle and working backwards. I gave up and tried again later that night, going through 20 pages of ads to find the first TOC page. I was seriously annoyed, but forgot about it once I started reading the article, which was engaging and informative.

But the same problem came up again the next Sunday, when I picked up ”T”, The New York Times Style Magazine, to check out the cover article on Emma Watson. I had to thumb through 54 pages of ads to find the TOC and the Emma Watson article wasn’t even listed on that page: I eventually discovered it on the second TOC page, after another 5 pages of ads. When I finally managed to read the article it turned out to be pretty much devoid of content, and I vowed to never open that idiot excuse for a magazine ever again. (Hermione, I’m sure, would have been outraged too.)

The table of contents is an important kind of filter. It may not have the surgical efficiency of a well-designed search engine, but it still gives you random access to the content of a publication so you can quickly find the stuff that interests you without having to thumb through every page. Stuffing all those ads in the front subverts the filtering function of the TOC, forcing you to pass through each one, because without the TOC’s help you’d have to sift through most of the magazine to find the piece you wanted. Because this is so frustrating for anyone who wants to read that content, I’m guessing that only a magazine which doesn’t expect anyone to read it can get away with this. Maybe the publisher figures their readers are only interested in the ads, and any editorial content is just a framework to vary the rhythm of those ads.

Testing this hypothesis, I compiled a visual index of the signal-to-noise ratio of the four magazines I found lying around my house (OK, not exactly a representative sample). There’s a photo of every spread up through the first page of editorial content. The TOC pages are highlighted in yellow, with a pink circle around the cover article I was trying to locate.

Is there always an inverse correlation between the quantity of ads before the TOC and the quality of the content? I’d say so, except for that Vanity Fair article about Microsoft: it actually was worth reading, even if I had to sort through an awful lot of junk to find it. 

T score:
54 pages of ads before TOC 1 + 5 pages of ads before TOC 2. 
70 pages before first page of editorial content.

Vanity Fair score:
20 pages of ads before TOC 1 + 5 pages of ads before TOC 2 
32 pages before first page of editorial content

New Yorker score:
2 pages of ads before TOC + 1 ad page before TOC 2.
8 pages before first page of editorial content. 

The New York Time Magazine score:
1 ad page before TOC 1 and 2.
7 pages before first page of editorial content. 

Pencil Icon

Who is poor in America?

This is an clean and simple example of visual filtering of data. It’s a guided tour, walking the viewer through different ways of looking at who is poor in America but leaving some room for individual exploration of the data. 

It starts out with the simplest possible pie chart, adding complexity each time the viewer clicks the NEXT button.

Keep clicking the NEXT button to step through the data, filtering by race, gender, education, age and family type. On each screen you can mouse over a single pie to get more detail about that group (see the detailed purple pie chart for AMERICAN INDIAN following this).

You can also see how the situation has changed over time by clicking on “change year”. 

Drag the orange slider forward and back to see how the picture changes from 1967 to 2010.  

I like having control of the speed: dragging fast generates movement like an animation, giving a sense of the magnitude and direction of change, but it’s also possible to slow down and pause to think about individual numbers.

The downside of this presentation is that there is a minimum of interpretation, not telling any particular story and not linking to any of the longer explanatory documents on this website or others. I love the simplicity of the way this is put together, but I’d like the option to dig deeper and follow up interesting stories.

Nevertheless, in this election season when no politician will talk about class in any way except to mention how a policy might help or hurt the Middle Class, it’s good just to be reminded that poverty is real, and to look at where it exists.

Source: Demos -  Ideas and Action to Promote the Common Good 

Quote IconIBM estimates we generate 2.5 quintillion bytes of information each day. That’s the equivalent of the entire printed collection of the Library of Congress about three times per second.

Source:  The New York Times, September 9, 2012

We are drowning in a sea of data, with more information available to us than at any other time in human history. We make sense of it by applying filters that organize and simplify it into meaningful shapes we can think about.

I want to use this blog to look at all the ways humans filter experience, from the patterns of perception embedded in the deepest, most primitive parts of our brains to the new wave of data visualization techniques. I want to think about how filters can be used to confuse and obfuscate, how politicians can filter the same set of facts to create parallel realities with no points of agreement, and how artists can discover meaning and beauty in chaos and disorder.

I’m hoping that the juxtaposition all these different pieces will tell a bigger story or reveal some unexpected pattern, but for starters I’m just going to toss in any shiny thing that catches my eye. We’ll see where it gets me.

– Laurie Wigham

Red tile roofs in a sea of trees

Trying to sketch the view out the window on the train to Katoomba through Sydney’s outer Western suburbs, my vision blurred and the red tile roofs became goldfish swimming in a sea of green trees. It occurred to me that all sketching transposes or organizes reality in some way, whether it’s visual, conceptual or both.