Visualizing Twitter

Visualizing Twitter

This week marks the 5th birthday of Twitter, the micro-blogging platform whose meteoric rise it's almost impossible to exaggerate. Since its creation in March 2006 and launch in June '06, Twitter has gained an estimated 190 million users, who collectively generate some 65 million tweets and roughly 800,000 search queries per day. Whether it's the routine of eating breakfast or the play-by-play account of cooling a nuclear reactor, Twitter has added a new dimension to how, what, when, and why we communicate. We've watched as celebrities have gamed the stockmarket, journalists have impersonated mayors, and poets have experimented with new forms of 140-character haiku. We've seen Twitter take on geopolitical dimensions too, first in Iran and now across the Middle East, as long-repressed citizens have capitalized on the tweet to organize grassroots protests, talk both to one another and the outside world, and stoke pro-democracy agitations into a fervor.

It's an open question, of course, as to whether Twitter — or any technology, for that matter — actually spurs social change, or merely serves as a vehicle for a fed-up and disenfranchised people. Many scholars believe that demographics, more than any single factor, primed the Egyptian revolt: more than two-thirds of the population is under 30, and they are over-educated and unemployed. This past Saturday, as Egyptians took to the polls for the first free election in decades, a young man told this NY Times cameraman, "The days of online activism and Facebook activism are over — it's only reaching a small segment. So, gotta hit the streets."

And before we all get too excited about Twitter Revolutions, it's also worth recalling that the media and political protest have a long history. Just think of Thomas Paine's famously inflammatory Common Sense monographs, fueling rancor against the British crown. Or the anti-czarist pamphlets circulated by factory workers in the run-up to the Bolshevik revolution. If we've largely lost our press muckrackers, our voices that, like Paine, spoke truth to power in pithy prose, it can only be because our media has largely lost its independence, consolidated into Clear Channels and News Corps and Time Warners, who are in turn legally responsible to their shareholders to maximize profits. It may be too much to ask of Twitter to topple the Fox New Empire, but if it can recoup even a modicum of independent press, it will be worth preserving on that count alone.

As for the debate over "Twitter Revolutions" (are they for real?), it's likely that historians of the future will look back on these recent Iranian and Egyptian upheavals and decide that we've witnessed a catalytic combination of modern technology and old-fashioned activism: anger at authoritarian regimes spurs tweets; tweets not only draw new attention to the problem, but brings previously isolated individuals into contact. In a phenomenon well-known by both complexity scientists and crowd psychologists, the network has self-reinforcing bent: call it peer pressure, or call it emergent behavior, but through corroboration and validation, an idea grows until, eventually, you get a Tahrir Square, a Pearl Square, a Benghazi — the Tiananmens of the social media age. In other words, Twitter doesn't create revolutions; people do. But Twitter can certainly help.

Meanwhile, for those whose job it is to stay abreast of events like Twitter Revolutions or disaster trifectas happening on the other side of the planet — i.e. the media — the advent of microblogging has unveiled a dizzying array of possibilities. On-the-ground sources (for example, we've been following myriad @'s in Japan, and have just picked up @ShababLibya) are now providing first-hand accounts of events as they unfold, and ramping up the imperative of round-the-clock reporting. It has also uncorked a firehose of new research potential, as each tweet inevitably leads to another web link, and as the Twitter conversation itself becomes fodder for reporting.

For journalists, however, the real game-changer hasn't just been the kinds of information we aggregate, assimilate, and convey, or the rate at which we do so. It's the "we" itself that is changing via Twitter. Time was that for many events, especially those unfolding in remote or perilous regions, large media organizations served as the gatekeepers of information. Now, who is reporting and writing today's news is transforming. Not only are we seeing mainstream media correspondents providing live Tweet coverage, they too are relying on twitterfeeds of locals on the ground. Those locals, in turn, don't just serve as sources to Al Jazeera, The Economist, and The Times, they also broadcast directly to an audience of followers, circumventing — and throwing into question — the role of the "article" altogether". For sure, a tweet provides little room for context, history, even a full-stop quote. But it is an incisive jab to the press, as crowdsourced content continues to create not only new opportunities, but also epistemological questions, for the arbiters of today's information.

No doubt, these are volatile times. But they are wonderfully exciting times too. We can be sure that social scientists have a heyday deconstructing the impact of Twitter on the 21st century zeitgeist, analyzing is effects on everything from reading patters to the democratic discourse at-large. Was it was a flash-in-the-pan symptom of a distracted culture, or the birth of brilliantly terse communications? How will Twitter change to reflect the needs of society, even as we change with it? And will these scholars be typing their thoughts into literary texts...or will they be Tweeting them? We probably won't know answers to these kinds of questions for years to come. What we can know for certain, however, is that Twitter has already generated an abundance of data. And while "too much information" may be a thorn in the side of a time-pressed journalist, it's almost a contradiction in terms for the data maven, and the visualizer. For the past several months, since we launched Visualizing.org, we've begun to see Twitter data leveraged in remarkably innovative ways, to draw not only captivating images, but new insights into human culture.

For most of us, a tweet is a thought expressed in 140 characters. But for those with the right kinds of data-parsing skills, a tweet is much more. It can be searched for particular words, hashtags (#), @mentions, and URL's. It can be categorized according to author, language, timestamp and original tweet or "re-tweet." Word frequencies can be run through sophisticated semantic programs to generate idea-based nuggets called "memes." And all of this can be rendered into a visual form that, at its best, sheds light on something previously unseen.

In celebration of this week's Twitter birthday, we take a look at a few tweet-based visualizations that have recently impressed us. From mapping moods across the US to calibrating the strengths of democratic discourse in Iran and Egypt, they are just a small slice of future possibilities.

If you have a twitter-based visualization you'd like to send our way, please do so! Either send us a tweet at @VisualizingOrg or, if it's your own original work, upload it here and join our growing community.


Truthy:

Truthy is a system to analyze and visualize the diffusion of information on Twitter. Developed by a team of network scientists at the Indian University School of Informatics and Computing, the Truthy system evaluates thousands of tweets an hour to identify new and emerging bursts of activity around memes of various flavors. The data and statistics provided by Truthy are designed to aid in the study of social epidemics: How do memes propagate through the Twittersphere? What causes a burst of popularity? The above meme is @sarahpalinusa, the Twitter account of Sarah Palin. According to lead researcher Fil Menczer, "This is an example of the diffusion pattern we observe around celebrities and other popular, influential people. Users in the orange cluster mention @sarahpalinusa (in the center) or address tweets to her. Some of these are critical (away from the blue). Others are supporters, and we observe that those users also retweet @sarahpalinusa (blue edges)."

Egypt Influence Network:

Experts say Egypt is the crystal ball in which the Arab world sees its future. Now that Mubarak has stepped down, Kovas Boguta shares the work that he has done making that metaphor tangible, and visualizing the pro-democracy movement in Egypt and across the Middle East. It is based on their Twitter activity, capturing the freedom of expression and association that is possible in that medium, and which is representative of a new collective consciousness taking form. Read full article: Visualizing an Arab Revolution.


Pulse of the Nation:

U.S. Mood Throughout the Day inferred from Twitter

Click for high-resolution PDF version (11MB)

One afternoon back in 2009, a team of network researchers from Northeastern University and Harvard were gathered around, ogling the New York Times’ amazing Super Bowl visualization. The Times had collected all Super Bowl-related tweets they could find, mined them for time and location, and generated a riveting movie that revealed not just the game — from kick-off to end — but also the story of the nation watching.

This got the researchers thinking: what if they were somehow able to extract the emotional context of tweets? Could they effectively plot millions of twitter messages to create a real-time map of mood-swings across the US? What if the president could give a speech and immediately view the effect on his computer? Is it true the the east coast is more stressed out than the west coast? Are people happier in cities? Twittermood is a result of that discussion.

Video

A time-lapse video of the maps, cycled twice, is available below (best viewed at 720p):

Mood Variations

A number of interesting trends can be observed in the data. First, overall daily variations can be seen (first graph), with the early morning and late evening having the highest level of happy tweets. Second, geographic variations can be observed (second graph), with the west coast showing happier tweets in a pattern that is consistently three hours behind the east coast. Similar variations were discovered independently by Michael Macy and Scott Golder, and first reported in the talk "Answers in Search of a Question" at the New Directions in Text Analysis Conference in May 2010.

Weekly Variations

Weekly trends can be observed as well, with weekends happier than weekdays. The peak in the overall tweet mood score is observed on Sunday mornings, and the trough occurs on Thursday evenings.

About the Data and Visualization

The plots were calculated using over 300 million tweets (Sep 2006 - Aug 2009) collected by MPI-SWS researchers, represented as density-preserving cartograms. This visualization includes both weekdays and weekends; in the future, separate maps will be created for each. The mood of each tweet was inferred using ANEW word list using the same basic methodology as previous work. County area data were taken from the U.S. Census Bureau, and the base U.S. map was taken from Wikimedia Commons. User locations were inferred using the Google Maps API, and mapped into counties using PostGIS and U.S. county maps from the U.S. National Atlas. Mood colors were selected using Color Brewer 2.

About Cartograms

A cartogram is a map in which the mapping variable (in this case, the number of tweets) is substituted for the true land area. Thus, the geometry of the actual map is altered so that the shape of each region is maintained as much as possible, but the area is scaled in order to be proportional to the number of tweets that originate in that region. The result is a density-equalizing map. The cartograms in this work were generated using the cart software by Mark E. J. Newman, available at http://www-personal.umich.edu/~mejn/cart.


Again, please send us your tweet-based visualizations, and we'll feature them here over the course of the week. Happy Birthday, Twitter!

Add a Comment

Login or register to post comments