Visualization Sprint

World Bank Global Development Sprint

Visualization Sprint:
an experiment in collaborative
data visualization
Visualization Sprint Process Outline

World Bank Global Development Sprint

For International Open Data Day — a global event where citizen coders and designers work to demonstrate the benefit of open data policies — we've teamed up with the World Bank to launch the Global Development Sprint, a collaborative data visualization project where anyone can fork code to visualize the same open data set.

We’re using the World Bank’s Major Contract Awards dataset, which serves as a guide on the distribution of major contract commitments among the Bank’s member countries. Together with fellow coders and designers, help governments, development agencies, and NGOs better understand and communicate the impact of global development.  

Mahir Yavuz got things started with an initial sketch using d3. Now it’s your turn. Over the next few days, add layers of comparison by exploring the relationships between and among countries or the funding across sectors. Are there trends over time? Don’t use d3? No problem. Head over to the World Bank Data Channel to download the full set and design your own visualization.

Data: For the initial sketch, Mahir used data from 2013 but you can get the full data set here (available in .csv and an API). Like our challenges, don’t feel restricted by the data. Mash it up with other open data to add context or a deeper level of analysis. We’re excited to see what you uncover.

Explore Versions

Comments

jamesmalvi's picture

In case if you want to try xml visualizer try http://codebeautify.org/xmlviewer :) it supports image view if there are any images in XML data

artzub's picture

@sam-lee I think what i can do for that. Now I've made so these contracts appear near the capital of countries and stay there.

> anything interesting or patterns you are noticing as you have been working through this dataset?
Could you be a bit more explicit, please? I don't understand =)

Sam Lee's picture

@artzub one more quick question- how does the visualization distinguish between contracts that are awarded to firms registered in the beneficiary country as opposed to those that are awarded to firms registered outside the country. If there was a way to see that distinction, it would be fantastic.

Sam Lee's picture

@artzub- thanks. very cool. anything interesting or patterns you are noticing as you have been working through this dataset?

artzub's picture

@sam-lee You can checked the option "show all edges". And if you mouseover to a small white point, which display a position capital of country, you can see relationship between a selected country and other countries.

Sam Lee's picture

@artzub- very neat! Could the tails stay instead of fading? That way it may be possible to draw out some patterns about regional interactions, etc.

artzub's picture

If you have problem with performance, then you can try to uncheck a option 'fading tail'.

Sam Lee's picture

@DataGraf - interesting hypotheses. The Bank does lend more during times of economic crisis- but it would be interesting to see if during crisis there was an increase in contracts (by total amounts and numbers) in specific sectors or types of procurement. If correlations can be shown, we can start down the path of unpacking causation... :)

Sam Lee's picture

@artzub- super neat! any progress on grouping the countries loosely by geography? that might help draw out additional patterns/trends.

on a slightly unrelated not- this article on visualization being a process as opposed to an output (http://blogs.hbr.org/cs/2013/04/visualization_as_process.html) reminded me a lot of the collaborative iterative visualization happening right here!

artzub's picture

I try make particles trakes and have some result http://img171.imageshack.us/img171/9245/e5643835106e4d5aa9b7542.png

De Datagraaf's picture

@all
First of all: sorry for being incommunicado for a couple of days. I have been very busy, that is why. (And I still am.)

@artzub
That is one pretty visualization, artz! Really beautiful to watch!
I love the idea of the money floating from supplier to borrower. And the growing histogram is a really nice addition.
But the fact that in a lot of the cases these are one and the same spoils the effect a bit. And I wonder whether it is possible to make the flows more explicit and visible (at the moment the money seems to appear out of thin air).
Which brings me to Sam's question: what patterns emerge from the data?

Oh, and you are right about the tree: it is nice.

@Sam
To be honest: I do not know.
There is the one I mentioned before: the growing role of China in Africa's economies, and the diminishing part Europe plays on this continent.
I do think the data shows this happening. But I haven't managed to make this visible. You can see China's current contributions to Africa being of major importance, but I haven't had the time to visualize this developing over time.

To track down other patterns in the data, it would help a lot to have someone around who can word a couple of well-informed hypothesis about this data. If any World Bank expert is reading this...

Coming from journalism, I can not help but wondering whether the data contains any biases or indications of wrong doing by anyone. Maybe that is why I am interested in who all these people are that are mentioned in the dataset.
I am curious how important these people are to the whole process. How powerful are they?
But, finding all this out takes a lot of expertise, and a lot of time.

What I am curious about, is whether the economic crisis has had an effect or effects on the sort of contracts that have been awarded over the years (major sectors, procurement category/type), and maybe even on the way they have been awarded (procurement method). But, again, not being a specialist in this type of data, I have no clue as to what might have happened.
But, just to illustrate what I mean: the financial crisis started out in the real estate sector, a lot of the contracts awarded deal with real estate being build. So I wonder: what happened to the number these kind of contracts over the years?

Anyway, I hope I can find some more time in the coming days to add a little more to this project, but I can't make any promises...

artzub's picture

So beautiful tree of version we made =))

artzub's picture

Dear friends!
In order to see other years need use this link https://dl-web.dropbox.com/spa/6x4vg7uwuzglgh3/wbgds/public/index.html (if you use google chrome you should allow executing of script in omnibox). I hope what it'll work in other browers =)
Enjoy the spectacle =)

Sam Lee's picture

@Datagraaf- the idea of replacing the bubbles with (or incorporating) some type of image is fascinating and puts more of a face (no pun intended) on the visualization- that seems like it would be an extremely time-consuming endeavor, but one that would make this much more readily understandable. Did you have a particular format in mind? Maybe this is something that can be done for just a few (Google image search) just to see what it might look like. Intriguing!

Sam Lee's picture

@Datagraaf - no problem. It's hard to say much about the motives or drivers of individual suppliers. I suspect they are as varied and complex as most any human/corporate transaction or interaction.

I have some questions for you- so all of these visualizations are extremely interesting and stimulating. What types of relationships or patterns do each of them represent? Has playing with the data surfaced anything interesting for you? I am impressed and fascinated by this process and would love to better understand what it is these visualizations are saying.

De Datagraaf's picture

@Sam, sorry, that should of course read: "who are these people", not "how. are these people".

De Datagraaf's picture

Woops, there appears to be an error in my handling of the toottips to the circles. Guess it has something to do with the way i use d3.event.
I'll look into it asap.

De Datagraaf's picture

@Sam
Thanks, but that much I had gathered myself. What I ment was: how are these people?! What do they look like? Are they in it for reasons of charity, or for the money? Are they under some sort of democratic control, or are they maybe deriving some sort of power from being awarded these contracts?
But, unless we can replace the bubbles by photographs, we should maybe leave these kind of questions to the journalists. :)

De Datagraaf's picture

@artzub Thanks man! The main thing I have been going for is trying to keep it simple, and concentrate on the possibilities for eploration.
Looking for a storyline/communication goal, so to speak.
Try grouping by region and supplier country, colored by supplier country...
A week or so ago I saw a documentary on how China has moved a lot of resources into Africa over the last decade or so, while Europe and the US have been lacking in this respect. The data confirms this, i think.

But you are not doing so bad there yourself, I must add. You manage to control the data in ways I can only dream of - it will take me at least another decade or so to become such a skilfull programmer - if at all.
The resulting exploratory tool is great fun to play with. Moving about those streams is almost hypnotising.
Ps: my next version will be a cosmetical upgrade - no new graphs for now.

artzub's picture

@de-datagraaf Really stunningly! I've played with v22 and got interesting results =) Great work... my respect =)

Sam Lee's picture

@Datagraaf - Ah, yes. Those are the companies (in some cases consultants) who have been awarded the contract to complete the work described in the "contract description" field. Sorry for taking so long to respond. Any other questions?

De Datagraaf's picture

@Sam
Yes, I those are the ones i mean. I haven't got a clue as to waht their role is in the whole process. Do you?

Sam Lee's picture

@DataGraaf - people mentioned in the dataset? Are you referring to the "supplier: field?

De Datagraaf's picture

So, I did a quick light version of the pack viewer I made yesterday. What I am trying to do is make some sort of tool to break down the data.
Basically it shows the data per region and country, and you can then pick the next packing variable with the left radio buttons, and you can color the bubbles by the same variables on the right.
It's a start. Transitions would be a nice addition; and it would be even nicer to be able to show developments over time.
Still no real storyline in sight though...
And could someone maybe find out who all tjhese people are that are mentioned in the dataset ? :) The faces behind the data, so to speak.
Regards,
wim

Edward Lee's picture

@zenodux - Yeah, I think it's more that d3.json() is pretty basic and World Bank's API requiers CORS/JSONP. See the answer on this page for details: http://stackoverflow.com/questions/10627703/d3-json-d3-xhr-and-cross-dom...

Sam Lee's picture

Any of you folks around in DC this weekend? Would be fun to push on this some more in person! http://datakindworldbank.eventbrite.com/

zenodux's picture

@artzub @wdward @dedatagraaf So poking around, here is what I have found:
https://finances.worldbank.org/api/views/kdui-wcs3/rows.json (JSON download, metadata, can filter down dataset if desired through https://finances.worldbank.org/Procurement/Major-Contract-Awards-FY2007-...)

https://finances.worldbank.org/resource/kdui-wcs3.json (API, no metadata, no CORS, supports JSONP)

@edward I was not aware JQuery got around the cross domain restrictions - interesting.

Also, I've been thinking of setting up a proxy to allow us to just use D3 as if the API supported CORS.

Of course, for testing, we can always run Chrome with the --disable-web-security switch.

@artzub Great work on the new viz! I can't wait to play with the code!

De Datagraaf's picture

@Edward
It was slow, but I managed to run it from my own website.
As for the other options: I would have to learn how to do that first. I will try the WB API, I guess.
Meanwhile, here is the version on my own website (there is no progress indicator, just yet; your patience will be requiered during loading):
http://www.datagraaf.nl/lib/D3/index_pack.html

Edward Lee's picture

@De Datagraaf - I don't think anyone's connection or browser will sustain a 34 MB json file very happily. This was why Mahir reduced the data set to one year to begin with. However, maybe you could switch to using World Bank's API? One difficulty we had encountered was that the API's security restrictions cause d3.json() to fail. Using jQuery's $.ajax() should work though, and jQuery is loaded by default on the Sprint scripts.

artzub's picture

The version 17 need look on fullscreen =) Use for that link Open in a new window.

De Datagraaf's picture

Let's put it another way: I have a 34.5 megabyte json file i would like to use as a source. How can i make this available for a graph on here?

De Datagraaf's picture

It would be nice to have one single (json) version of the data set on visualizing.org (instead of having to serve it ourselves, I mean).

De Datagraaf's picture

Just have to ask again: can I do a json call to the whole dataset on the site instead of just 2013?

De Datagraaf's picture

@artzub, you might be right about that competition thing (it sure helped evolution in biology). So I decided to have another go at another perspective. Results to follow shortly.
But I do hope we somehow manage to end up with one coherent visualization in the end - or at least a couple that go together well.
It is funny that you mention the organizations involved; I just had the same idea.
Regards,
wim

De Datagraaf's picture

Heya Sam, thanks for answering those questions.
Just one more for good measure: is there a ready made json call I can use to address the whole dataset?
Regards,
wim

Sam Lee's picture

@artzub - even in cases when they are the same, supplier and borrower country should remain the country listed. the world bank is just the original source of funds. hope i didn't confuse you earlier!

yes, the dataset can be just by year- i've filtered them for you quickly here:

07 - https://finances.worldbank.org/Procurement/Major-Contracts-FY-07/4scn-axex

08 - https://finances.worldbank.org/Procurement/Major-Contracts-FY-08/ix4e-wtr3

09 - https://finances.worldbank.org/Procurement/Major-Contracts-FY-09/j53w-wy6a

10 - https://finances.worldbank.org/Procurement/Major-Contracts-FY-10/sx62-vab4

11 - https://finances.worldbank.org/Procurement/Major-Contracts-FY-11/5uuz-kv7b

12 - https://finances.worldbank.org/Procurement/Major-Contracts-FY-12/59cu-pnxk

13 - https://finances.worldbank.org/Procurement/Major-Contracts-FY-13/dggm-jp3d

They also exist in a single dataset- https://finances.worldbank.org/Procurement/Major-Contract-Awards-FY2007-...

all of these can be pulled from APIs as well.

Hope this helps!

artzub's picture

@sam-lee If I understand you correctly, we can make World Bank as supplier in those cases when the borrower country is the same as the supplier country? But seems it's not correct... Maybe make a chord graph based on firms not countries... I'll think about that.

Can you help to get the same data only for other years in one dataset, or each year in a separate dataset? I want to try to run visualization into the developments over time on big data.

artzub's picture

@de-datagraaf You will not believe, I had also many affairs in the family =)
If after work I'll not be tired, you see my new version.
In my mind in a dispute born truth, but we can cooperate for getting the best result =)

Sam Lee's picture

Really neat work folks. Impressed and amazed by this entire process- saw that there were a couple questions about the data, and I wanted to offer my take.

One thing to note is that these are contract awards and not grants. There was also a question about the source of the money. For what's contained in this dataset, the World Bank provides development loans/credits/grants to countries who in turn issue development contracts that are tied to that financing stream.

De Datagraff, as you note in many cases the borrower country is the same as the supplier country. The original source of the money is still the World Bank, it's that the firm or individual that has been awarded the contract is registered in the same country as the original borrower country.

The Twitter functionality mentioned earlier is really interesting- there are lat/long coordinates available for the projects that these contracts are tied (not the contracts themselves) to which would be helpful in plotting some of this on a corresponding/complementary map.

Hopefully this is helpful! Very impressed, and happy to answer any more questions!

De Datagraaf's picture

Hey artz, been busy building a website this weekend. And lots of family stuff on the agenda this past weekend.
How are you doing with the visualization? Did you already get an insight into the developments over time?
I haven't set up anything major for lack of time, but also because I do not want to spend a lot of effort on competing versions - this time around I am more into cooperation.

artzub's picture

@de-datagraaf Yeah, <15 is arbitrary and we need remove this limit, but it's not problem.
I've looked your 14-15 versions and now i'd like merge all types in one version with a switcher by types. But the first i finish my new version which shows relation between countries - sectors - projects.

artzub's picture

@moira-hill if you look chord graph at #12 you can see more interactive mode with filtering by category and tooltip on chord when you hover the item.

Moira Hill's picture

Hello - I saw #13 promoted on Twitter by Visualizing.org, and was asked to provide feedback on it.

For any project, I ask: what is the communication goal, who is the audience, and what is the viewing platform? Is this more effective or less effective in meeting those goals?

I appreciate the amount of work that went into this. #13 is aesthetically pleasing. However for a 2d graphic on small viewing screens it is visually challenging, with extremely poor readability. If it's on a poster (30" x 36" or greater) with type point sizes in the 36 pt or greater range, then it could work as is.

If it is meant to be an interactive graphic, then these problems could be ameliorated by: when you hover the pointer over one From/To country set, the rest of the From/To nodes gray out and the selected From/To nodes pop visually - e.g., more saturated colors, jump out of the view plane slightly, fonts have more weight, etc.

Hope this is helpful.

De Datagraaf's picture

hey artz, sorry for not getting in touch with you yesterday (I tried earlier today; noonish here now).
I spent all day yesterday trying to get my topojohnson out, but I got as far as installing node.js and npm - sort of - but I never managed to install topojson.js itself (on ubuntu that is, having only recently switched from windows).
My goal was a world map using topojson, I ended up using old fashioned geojson, and wasting most of my day on installing and other stuff I don't really care about.
I have skype up and running though (de.datagraaf).

The other thing that kept me from calling, is that I don't really know what to do with this dataset just yet. (But the obvious that is.)
The problem is that we don't have any hypotheses to test. So we can only do exploratory work. And while getting in more data might show some interesting correlations, at the same time it might just complicate things more.
Maybe we should look into what sort of reductions of the data we could do? The <15 grants filter in the current data does reduce the hair ball somewhat when making a chord, but it seems somewhat arbitrary.
Have reduced the data in any other way already maybe (like into quantiles)?

artzub's picture

@de-datagraaf I live in Krasnoyarks in Russia. My timezone is UTC+8. I usually work at home after 11 p.m. on my timezone. You can find me in skype at this time. Though my skype allways on-line, so you can try to send message for me, when i'll be at the computer i'll answer you.

De Datagraaf's picture

Hey artzub, what hours would you be available for a skype chat? Tomorrow would be alright for me (but I have to set up Skype, I have to admit).

artzub's picture

@de-datagraaf Thank you very much =) And i hope that you're understood because it's very important for the further development of this visualization.

I'm now think about detalization projects and viewing data by years. but if you give me data about GDP, i'll glad to connect data about GDP with data about money's stream from country to country. And also very interesting for me connect data about population in each country with data about money and maybe view this information on map how you're shown me in some visualization.

Could have we chat in skype? my nick: artzubalex.

De Datagraaf's picture

Btw: in a lot of cases the borrower country is the same as the supplier country.
I have no further info on the workings of these grands. The database description mentions this possibility, but gives no further explanation.
So, what do we make of this? Where is this money coming from?
How do we work this into the visualizaton?

De Datagraaf's picture

Btw: in a lot of cases the borrower country is the same as the supplier country.
I have no further info on the workings of these grands. The database description mentions this possibility, but gives no further explanation.
So, what do we make of this? Where is this money coming from?
How do we work this into the visualizaton?

De Datagraaf's picture

Meanwhile I have had a look at extra data to add. There is one measure that is potentially interesting, were it not for the fact that the datasets available are very incomplete. I am talking about income differences within nations.
So, I guess something like GDP is the way to go instead.
Maybe color scaling the groups or the countries in the chord graph depending on GDP would be a good idea.

Add comment

Login or register to post comments

How It Works

A "visualization sprint" is a new collaborative design and coding experiment where contributors work together to find the best solution, or solutions, for visualizing a data set. Taking cues from citizen science, open source culture and code sprints, Visualizing.org has conceived of this format in order to promote community dialog, learning, and feedback about visualization techniques.

Starting from an initial sketch written in a popular scripting language, anyone is invited to "fork" the code — adding to or modifying an existing version and then posting it back to the sprint. Any version can be forked, and new "alpha" sketches can also be submitted to start a new branch from scratch.

Check out the project prompt and data set at the top of the page, then look at the initial sketch. What are we trying to visualize here?
Explore other contributions: what are the branches the design has taken? where does the visualization still need work?
Add your voice to the discussion by voting versions up or down and commenting as you explore.
Choose a version to modify and click the "Fork This Version" button.
Work on the code, testing as you go along. When you're ready, commit your contribution along with a short description.
Repeat from step 2!

More Info

Is this project only for programmers?

Sort of. The purpose of this sprint is to explore and discuss techniques in interactive visualization, and it uses simple scripting languages so that we can easily see and modify the code. If you're not a coder we encourage you to participate by voting on the versions, adding your voice to the comments, analyzing the data set, or even sketching visual design directions that others could help realize.

What resources are available for my version?

When you fork a version, the top of the edit form will have a list of libraries and data sets you can use. If you need help figuring this out, just send us an email.

Can I transform the data set?

Yes! You are encouraged to transform and analyze the data set. If you modify the data in a useful way, send us the updated file so we can share it with everyone working on the sprint.

How does the sprint end?

The focus here is on process, but we also want to have a nice visualization (or several) to present. Hopefully, the consensus of all participants will bring the most promising branches to a state polished enough to be called "final."

Is there a prize?

No, there is no prize.