Scraping Canabalt scores off Twitter, ramming them into Mongo and processing in R. The results are not vastly revelatory, but it's a nice account of the process of storing, processing, and representing big data.

"...don’t confuse this kind of data exploration, where the goal is to size up the data, with building proper data plumbing, where you want robustness and maintainability. Perl and bash scripts are nice for the former, but can be a nightmare for building data pipelines." Lots of good stuff in this article; this was a highlight.

Learning datamining, using the WoW Armory as a data set.

Very simple; very effective.

Hmn. Visualisation of tweets about the word "FIFA" (do the maths there) and all games played of FIFA 10 - so you can see both which teams are doing well, and which countries have good FIFA gamers in them. There's little bits of stats-fluff, but it doesn't go nearly deep enough. It's lovely EA are doing this... but it could be, you know, useful, rather than just shiny? Bungie's statistics crown is still a long way off.

"Someone at work recently asked how he should go about studying machine learning on his own. So I’m putting together a little guide." Ooh, useful. Lots of starting points for machine learning in R.

"Statisticians’ sex appeal has little to do with their lascivious leanings ... and more with the scarcity of their skills. I believe that the folks to whom Hal Varian is referring are not statisticians in the narrow sense, but rather people who possess skills in three key, yet independent areas: statistics, data munging, and data visualization. (In parentheses next to each, I’ve put the salient character trait needed to acquire it)."

"You are a web programmer. You have users. Your users rate stuff on your site. You want to put the highest-rated stuff at the top and lowest-rated at the bottom. You need some sort of "score" to sort by."

"On Tuesday we shipped an update that added a bunch of features / bugfixes / balancing tweaks that came out of the community's feedback. In particular, it made some changes to the underlying TF damage system, and as part of that, it modified the way critical hits are determined. We thought it might be interesting to dig a little into the change, and hopefully give you some insight into our thinking." Another cracking example of explaining game mechanics clearly and directly, to an engaged community.

"We’ve generated what we call the Personal Annual Report for all our users. It’s a unique-to-you PDF of data, visualisations and factoids about your travel in 2008, that we’re delivering over the next week via email to every Dopplr user who travelled in 2008. To give you an example, we thought we’d show you the Personal Annual Report of someone who’s had a very busy 2008 - President Elect Barack Obama." This is super-awesome. Can't wait for mine, no matter how small it is.

"...it’s become apparent to me that social software is a medium turns all communication into a self-representation game whose ultimate goal is popularity."

"A dude by the name of Phoebus has posted a collection of his research on Left 4 Dead's infected and weapon damage statistics over on the official Steam forums. I think that it'll be of great interest for any serious player of the game to delve into this information." There's a question over their accuracy, but there's still a decent amount of detail here, and the details on tail-off of weapon damage is useful to know. Also a relief to have the hellacious friendly-fire damage on Expert confirmed.

"If you need to perform data analysis, provide graphics for your users in your webapp, or produce high quality plots I encourage you to investigate the combination of ruby, GSL and GNUPlot." Looks good. I should probably give this a poke some time; could come in handy.

"I spent 10 weeks last Summer as an intern on the strategy team of Transport for London's (TfL) London Rail division.... My general task was to help London Rail start to make use of the oceans of data spewing out of the Oyster smartcard ticketing system, but I spent the bulk of my time working on a project that came to be titled Oyster-Based Performance Metrics for the London Overground. I've posted my final report and slides and outline for the presentation I gave to TfL executive management." Some interesting data and information here.

Visualising the heights of people's towers by importing their savegame. Lovely.

This might come in handy sometime.

"Some people believe that there's no correlation between quality and sales, and thus think that the way to make money is to make things that are easily marketable (read: licenses). Game developers themselves usually argue that sales above a certain level require a game to be sufficient quality. I decided to see which of these perspectives was correct for the Playstation 2 era." Datanalysismachinego!

"The winner is Tim Graham who took manual personal data collection to another level. From email spam, to beverage consumption, to aches and pains, Tim embraced the spirit of self-surveillance. He even made his personal data available in the forums." Dataviz overload!

"After all, what is science? It's a technique for uncovering the hidden rules that govern the world. And videogames are simulated worlds that kids are constantly trying to master. Lineage and World of Warcraft aren't "real" world, of course, but they are consistent -- the behavior of the environment and the creatures in it are governed by hidden and generally unchanging rules, encoded by the game designers. In the process of learning a game, gamers try to deduce those rules. This leads them, without them even realizing it, to the scientific method."

"UbiGraph is a tool for visualizing dynamic graphs. The basic version is free, and talks to Python, Ruby, PHP, Java, C, C++, C#, Haskell, and OCaml."

"The easier way to monitor servers and web applications." Single-server plan is free; looks like it could be very handy indeed.

Remarkable series of images by Chris Jordan depicting rates of consumption in America through exhaustive diagrams. Staggering when you see them in close-up.

Wow. Bookmooch has some seriously comprehensive data available from it, if you fancing munging their entire dataset (security-sanitised, obviously).

There’s nothing wrong with a service that appeals to tens of thousands of people, but in a billion-person internet, that population is also a rounding error. If most of the people who try Second Life bail (and they do), we should adopt a considerably mo

Good Edge piece on XBLA

Crazy Egg is, basically, low-budget heatmapping for the web. Powered by Rails - it'd be interesting to look into this more when certain things go live...

Good analysis of the situation, and ev's right: pageviews pretty much *are* obsolete...

Full transcript of Reggie Fils-Aime's speech at TGS 2005. Great statistics.

Useful statistics, and the Nolan Bushell link is great.