Wet streets cause rain.

Quote: "In God we trust; all others must bring data."

Some interesting ideas around scheduled tasks / pipelining.

Fun stuff in here. via @joshua schachter

Cool infoviz.

Great blog post. Quote: "... People who develop broad skills also tend to develop a good mental model of how different layers of a system behave. This turns out to be especially valuable for performance & optimization work. No one can know everything about everything, but you should be able to visualize what happens up and down the stack as an application does its thing. An application is shaped by the requirements of its data, and performance is shaped by how quickly hardware can throw data around."

Cool tool for working with large, unrefined data sets.

"After we realised that we just went onto the site and deleted that field - overnight there was a step function [change], resulting in $12m of profit a year, simply by deleting a field. We have found 50 or 60 of these kinds of things by using analytics and paying attention to the customer."

Some inside baseball looks at bit.ly data / flow.

A breakdown of about 15 different browser-based Excel-like jQuery modules.

Today, Facebook counts 29% of its employees (and growing!) as Hive users. More than half (51%) of those users are outside of Engineering. They come from distinct groups like User Operations, Sales, Human Resources, and Finance. Many of them had never used a database before working here. Thanks to Hive, they are now all data ninjas who are able to move fast and make great decisions with data.

Big data is being democratised, but there's a lot of unmet need in businesses around data warehousing. The typical solution is to build a data warehouse team around a product like Oracle, but I've heard plenty of business people grizzling about the result. They want answers, they don't want the headaches and lag that a data warehouse involve. Big Data (or Cloud Analytics or whatever) may be the opportunity to figure out a new minimum viable product for these folks, and offer it without the "data warehouse" baggage. This might be back end, might be UIs, might be visualisation, but all of these have a lot of room for improvement.

Good stuff.

AdMob notes that Apple advanced its lead in smartphone traffic share from 43% last month to an even 50%. Symbian slipped from 29% to 25%, while third place Android grew from 10% to 11%. RIM's share fell slightly from 8% to 7%, Windows Mobile dropped from 5% to 3%.

Twitter + terabytes of information + datamining = pig.

Quote: "[Facebook] ... found that activity at three months was predicted by variables related to three classes of behavior: (i) how often a user was reached out to by others, (ii) frequency of third party application use, and (iii) what Itamar termed “receptiveness” — related to how forthcoming a user was on the site."

Palantir Government is a platform for information analysis. Palantir was designed for environments where the fragments of data that an analyst combines to tell the larger story are spread across a vast set of starting material. Palantir provides flexible tools to import and model data, intuitive constructs to search against this data, and powerful techniques to iteratively define and test hypotheses. With Palantir, an analyst can go all the way from an initial tasking to a final product in hours or days, rather than weeks.

Another business built around access to real-time data.

Quote: "The gist of the idea is that people change their behavior—often for the better—when they are being observed (which is why it's sometimes called the observer effect). Those workers at Western Electric didn't build more relays because there was more or less light or because they had more or fewer breaks. The Hawthorne effect posits that they built more relays simply because they knew someone was keeping track of how many relays they built."

Quote: ".. But if inboxes don’t fundamentally change in order to adapt to their new role as the keeper of myriad transactions across the entire web, they’ll be obsolete."

Ann Winblad: The unstructured data deluge creates the next great information leaders. Every click, message, tweet is rich data amassed at exponential rates. Gartner predicts enterprise data growth of 650 percent in five years; 80 percent of that data will be unstructured.

Quote: "My suspicion is that there’s a market in making each and every one of those queues smaller, if not making them disappear entirely." Bigger market: being able to prioritize across queues.

Making things talk, mostly with twitter, in 1st person.

Quote: "The best way to achieve data portability is to just do it." Couldn't agree more.

A list of some of the seemingly crazy things you have to do if you're going to use BigTable.

JS based charting including treemaps, hyperbolic trees and space trees.

Free weather data in RSS format for "... your own web site or client application" but then the fine print says only personal / non-commercial. Bollucks.

Description: SIOC provides a Semantic Web ontology for representing rich data from the Social Web in RDF.

Great list (and comments) of freely available data sources. When is someone going to make the weather forecaster datasource free so that I don't have to pay accuweather or weather.com for forecasts / current weather?

References to some other cool Java viz tools.

Quote: "... adding more, independent data usually beats out designing ever-better algorithms to analyze an existing data set."

Feed corpus: 124,460 unique RSS/Atom feeds

Exploring Hyperlinks, Contents and Usage Data

"... a site for large data sets and the people who love them"

Quote: "As the applications become apparent, the data will be valuable in new ways, and the company with the most data wins."

Quote: "... just like using more methods and pushing information into the headers gives more information to the network, by denormalizing you are implicitly giving more information to the database, and that 'extra information' makes things run faster."

Quote: "... we need a new kind of data store, a new kind of SQL, something that does for storing and querying large amounts of data what SQL did for normalized data."

Very cool combination of infoviz and browsing.

Great rant on pie chart and 3D graphs. Quote: "... if it’s a simple dataset, boil it down to the essentials. If there are two numbers which add up to 100%, you don’t need to tell me both, and you certainly don’t need to draw me a picture of it."

JSON is a better fit for Web services that power Web mashups and AJAX widgets due to the fact that it is essentially serialized Javascript objects which makes it fit better client side scripting which is primarily done in Javascript.

Graph / infoviz p0rn. I could look at these all day long.

Very cool FireFox plugin that basically gives you Excel graphing / spreadsheet abilities in your browser.