infovore + data   110

Reshaping JSON with jq | Programming Historian
I love jq at the command line for even the simplest tasks; I need to go over this at some point.
json  jq  unix  tools  programming  data 
december 2017 by infovore Pipe Viewer
"pv - Pipe Viewer - is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion." Looks very handy.
unix  software  cli  utility  data  pipeline 
june 2016 by infovore
"xsv is a command line program for indexing, slicing, analyzing, splitting and joining CSV files. Commands should be simple, fast and composable." iiinteresting.
csv  rust  data  cli  unix  tools 
february 2015 by infovore
Tangle: a JavaScript library for reactive documents
OK, this is great: Bret Victor's library for exploring interactive documents. Tidy - thanks to its use of data-attributes - but super-clear. Really nice to have a web-based library, too, and one focused on text. Now thinking about this conceit again.
data  interactive  bretvictor  javascript  exploration  spelunking 
october 2014 by infovore
One CSV, Thirty Stories - Whatfettle - Paul Downey (psd)
Rather looking forward to seeing this play out: thirty days of processing and spelunking CSV, from Paul Downey. Lots of new tools and tricks emerging already.
data  cli  unix  commandline  psd  pauldowney  csv 
october 2014 by infovore
7 command-line tools for data science
Really nice exploration of a small stack for poking data at the commandline. I'm a fan of jq and its ilk already, so this extends some of those techniques.
cli  data  json  tools  unix 
october 2014 by infovore
"Sheetsee.js is a JavaScript library, or box of goodies, if you will, that makes it easy to use a Google Spreadsheet as the database feeding the tables, charts and maps on a website. Once set up, any changes to the spreadsheet will auto-saved by Google and be live on your site when a visitor refreshes the page." This is good.
javascript  spreadsheets  data  publishing  web  software 
august 2013 by infovore
(the teeming void): Figuring Data (Datascape Catalog Essay)
"Here we get a glimpse of an alternative figuration of data itself. Rather than some kind of precious (but immaterial) stuff, or fuel for market speculation, data here is a relationship, a link between one part of the world with another, and a trace that can be endlessly reshaped."
data  art  visualisation  mitchellwhitelaw 
june 2013 by infovore
"DataFart lets you easily graph data from the command line." So it does.
data  graphing  cli  bash  visualization 
march 2013 by infovore
Under the Stairs (with vintage Apple hardware porn)
"The moment that stopped me in my tracks was when I checked to see if there was anything in the external disk drive." I really want to find out what's on it. Lovely, simple storytelling from Aanand.
aanandprasad  programming  history  computers  data  stories 
september 2012 by infovore
Ethernet data transfer stalling or failing
"I've been working on a sketch wherein some data is downloaded from an HTTP server and is then processed on the Arduino (printed, as it happens, but I don't think that's important). In my original sketches, I was occasionally seeing transfers fail midway through." James is running into issues that might be relevant to me.
ethernet  arduino  data  http 
march 2012 by infovore
The Digital Humanities and Interpretation -
"When another scholar worries that if one begins with data, one can “go anywhere,” Ramsay makes it clear that going anywhere is exactly what he wants to encourage. The critical acts he values are not directed at achieving closure by arriving at a meaning; they are, he says, “ludic” and they are “distinguished … by a refusal to declare meaning in any form.” The right question to propose “is not ‘What does the text mean?’ but, rather, ‘How do we ensure that it keeps on meaning’ — how … can we ensure that our engagement with the text is deep, multifaceted, and prolonged?”" Which is interesting, as is the whole article - the author is not convinced by the 'digital humanities', but he still links to some very interesting stuff about algorithmic criticism.
humanities  literature  criticism  literarycriticism  algorithms  data  datamining 
january 2012 by infovore
a sorted tale of data over time | BEN PURDY
"As amazing as it was to find the disk, the file was corrupt and couldn’t be read; all attempts to view the now 20 year old animation failed. It was part one of a science fiction saga titled “Porth” that our friend Cory had made by stretching the animation tool to the absolute limits. To say the least it was worth putting some effort into saving this file." Data archaeology.
data  animation  history  archaeology 
january 2012 by infovore
jqPlot Charts and Graphs for jQuery
"jqPlot is a plotting and charting plugin for the jQuery Javascript framework." Ooh, nice; another one for the collection.
charts  javascript  jquery  data 
january 2012 by infovore
daniel sinker • Hacker-Journalism 2011: A year of "show your work"
An Impressive list of notable examples of programmatic journalism from Dan Sinker; something I must return to.
data  programming  journalism  code  software 
december 2011 by infovore
Hacking Carbon Emissions into Minecraft
"When you burn some wood in a furnace, the mod calls out to AMEEconnect to do a calculation, and adds the result to a tracker in-game. As the carbon ticks up, the environment gets more and more polluted as the skies go dark and the clouds come down. OK, not entirely accurate, but an effective visual indicator!" Fun.
amee  minecraft  modding  data  games 
november 2011 by infovore
Bringing the London Bus Network home – Blog – BERG
"A service involving 8,500 GPS enabled busses and many servers is very impressive, but it really comes into it’s own when it doesn’t show off." Modest devices again. (This is very nice).
transport  visualisation  data  jamesdarling  quietdevices 
september 2011 by infovore
Browser as a weapon in a guerilla war – Techbelly
"It’s a bit like augmented reality, a layer inserted between what leaves the server and what hits your brain." Yes. Also: see Ben's comment about the browser as weapon.
browser  chrome  journalism  data  augmentedreality 
august 2011 by infovore
"Designers get handed a tool kit that has as many tools as a good swiss army knife, and the maps reflect these tools. Millions of people use them to make appointments across town, find restaurants, and drive home for the holidays.

But what if, instead of a swiss army knife, we used a box of crayons? Or charcoal and newsprint? Or play-doh? What would those maps look like? What could they tell us about the world?"
data  design  maps  mapping  stamen  whynotmakeitpretty 
july 2011 by infovore
onyxfish/csvkit - GitHub
"csvkit is a library of utilities for working with CSV, the king of tabular file formats." Ooh.
data  csv  shell  commandline 
june 2011 by infovore
All Watched Over: On FOO, Cybernetics, and Big Data | Ideas For Dozens
"On my way home from FOO I sat staring out the car window, all of these impressions, ideas, and seeming contradictions bouncing around in my head. And then something occurred to me. O’Reilly’s human-centered approach is still a kind of systems thinking. O’Reilly is still building a model of what the geek world is working on. They’re just doing it through the social relationships that their employees form with other geeks. The “data” they gather is stored in their employees heads and hearts and in those of the wider community of geeks they bring to events like FOO. Instead of trying to live in the model, O’Reilly tries to live in the community."
data  oreilly  community  systems  networks  cybernetics  gregborenstein 
june 2011 by infovore
Best practice: Import mySQL file in PHP; split queries - Stack Overflow
"Here is a memory-friendly function that should be able to split a big file in individual queries without needing to open the whole file at once:" Yep, that'll do.
php  mysql  data  import  utility  script 
april 2011 by infovore A Few good men
"The recent generation of young turks is doubtless having fun with data scrawling but at some point it will pass people by unless there is a purpose or utility to it. They've got the engagement sorted. These things are mostly usable. What they are not is useful.

That is where people like Few come in. They work in analytics - using data for decision making. They are ideal real-life mentors, solving real life problems. They can point the way to thinking of these apps as tools for whatever outcomes." Max is right - it's a great blog. Good spot.
data  visualisation  informatics  information  stephenfew  blog 
april 2011 by infovore
In Bloom « Bloom Blog
"The ways in which people interact with computation are changing swiftly as we move into more casual relationships with our digital services on tablets, big screens, and across social networks. We believe we have some compelling answers about how digital experiences will evolve into these new contexts. Please, follow along with us and explore these playful, dynamic instruments of discovery together." These guys are going to be worth keeping a very beady eye on; what a team.
bloom  visualisation  information  data  design  friends 
february 2011 by infovore
"...Compares the magnitude of stuff. Like one guided missile destroyer costs as much as three million fluffy kittens."
comparison  data  language  english 
february 2011 by infovore
PeteSearch: Data is snake oil
"Next time somebody's trying to sell you on the awesomeness of their new data technique, ask to see a prototype. If they haven't got that far, it's snake oil." Everything in this article is, basically, true. It's a really good run-down of all the issues that emerge in the reality of dealing with data-driven products at any scale."
data  prototyping  materialexploration  software 
december 2010 by infovore
Romance has lived too long upon this river: A London Companion |
"I’ve been playing with glanceables and synecdoches for a while now, until I came up with something that had to be got out of my head, and into the world. So here it is: Romance has lived too long upon this river; a single-serving web page that tells you how high the tide is at London Bridge: explicitly close up, but also, roughly, at a glance". This is great. Also: James has an ear for domain names.
thames  riverthames  glanceable  synecdoche  data  representation 
november 2010 by infovore
dataists » Blog Archive » The Iraq War Diary – An Initial Grep
"Please don’t believe any of this. Go instead to the data and have a look for yourself." Which is, for this audience, a very good way of putting it.
data  iraqwar  belief  trust 
october 2010 by infovore
Music from Saharan Cellphones. This is amazing.... | intercourse with biscuits
"Sahel Sounds rounded up music salvaged from the discarded mobile phone memory chips in West Africa." Wow; the after-life of dead electronic media made real.
music  culture  media  data  storage  africa 
october 2010 by infovore
dataists » Blog Archive » A Taxonomy of Data Science
"Both within the academy and within tech startups, we’ve been hearing some similar questions lately: Where can I find a good data scientist? What do I need to learn to become a data scientist? Or more succinctly: What is data science?" Great starting point; looking forward to more from the blog.
data  machinelearning  datascience  blog 
september 2010 by infovore
scraplab — This Trail
"The GPS looks forward for me, projecting all my future successes and failings. Every bit of information helps to optimise my path. Contour maps spring out of the hills surrounding, and round the corner ahead. It took a space shuttle and an army of volunteers to help me shift down a gear, and hopefully the data exhaust I leave behind will help someone do it better next time." Tom is brilliant. I miss him.
tomtaylor  poetry  data  geodata  cycling 
september 2010 by infovore
[this is aaronland] that's how the light gets in
"This is the part that interests me: What happens to a person's experience of prettymaps when the echoes of their own life start to make up the map itself? What happens when the only streets on a map are those you and your friends have traveled?"
aaronstraupcope  maps  friends  geo  data 
september 2010 by infovore
The Seven Secrets of Successful Data Scientists : Dataspora Blog
"...don’t confuse this kind of data exploration, where the goal is to size up the data, with building proper data plumbing, where you want robustness and maintainability. Perl and bash scripts are nice for the former, but can be a nightmare for building data pipelines." Lots of good stuff in this article; this was a highlight.
bigdata  data  datamining  statistics  machinelearning 
september 2010 by infovore
Chris Heathcote: anti-mega: griotism
"I thought this was a fascinating take on the need within companies for stories... Companies spend a lot of money looking for these stories. Traditional product companies had to ask people and users to tell their stories, normally through market research. Web companies are at a huge advantage: they have rivers of usage data flowing through their servers, and the problem inverses – how to make sense and tease out meaning and interest from such a torrent." This is very good; I'm looking forward to future installments.
data  visualisation  grindr  griot  stories  chrisheathcote 
july 2010 by infovore
Open Library Ore « The Open Library Blog
"Ben Gimpert is a friend of the Open Library. He and I got together over lunch a few months ago to talk about big data, statistical natural language processing, and extracting meaning from Open Library programmatically. His efforts are beginning to bear some really interesting fruit, and while we work out how we might be able to present it online, we thought you might be interested to hear what he’s been up to." Answer: good things. Ben is awesome, and this work sounds great. (I can't quote a suitable passage, so George's intro will have to do).
bengimpert  openlibrary  data  bigdata  books  categorisation  textextraction 
april 2010 by infovore
Making the Physical from the Digital « Random Hacks
"Ben O'Steen's talk from OKFN; lots of nice little things in here about preserving data and opening it up into a variety of forms."
papernet  paper  printing  information  data  archiving 
april 2010 by infovore
ASBOrometer - Measure UK anti-social behaviour on iPhone and Android
"ASBOrometer is a mobile application that measures levels of anti-social behaviour at your current location (within England and Wales) and gives you access to key local ASB statistics... This app was created by Jeff Gilfelt and made possible by the initiative, which is opening up UK government data for public reuse." What sensationalist rot; no number of pretty visualisations make this kind of fearmongering acceptable. It's nice that the data is open; it's a shame this is the best thing people can think to do with it. Whether you like it or not, this information is very, very loaded.
data  government  society  culture  fearmongering  infononsense 
february 2010 by infovore
The 4 Big Myths of Profile Pictures « OkTrends
More brilliant data-analysis and writing from OkTrends - perhaps my favourite data-blog out there, and one of my favourite discoveries in 2010 so far.
dating  data  analysis  okcupid  photos 
january 2010 by infovore
E-Books – The Bigger Problem, Part Two of Three. | Dangerous Precedent
"With every piece of metadata that you don’t throw away, you gain a factor more potential ways of slicing through your content and delivering it as a separate product, simply as a result of a database lookup. In the case of Vogue today, say, commissioning an editorial product that simply shows every dress designed by Christian Dior that appears in the archive would involve weeks of intern-work, instantly making it unprofitable or too late. A metadata-complete archive in the future would give you that with a single line of code." Hammersley on the value to journalism of sensible datastorage. Data-driven journalism in the sense that it is not *about* data, but in that it is *treated as data* - and from this more stories can flow.
publishing  journalism  benhammersley  data  content  metadata  storage  intertwingly 
december 2009 by infovore
Andy Huntington Interaction & Sound » Datadecs
"For Christmas 2009 the Really Interesting Group wanted to create a a gift comprising a series of 4 unique decorations based on each recipient’s use of the Flickr, Dopplr, and Twitter. Having used a couple of the software APIs they were thinking about using (flickr and dopplr) and with experience of rapid prototyping we worked together to turn the data into something physical." Can't believe I haven't linked this already. Ours were wonderful; many thanks to RIG and Andy.
datadecs  data  christmas  design  rapidprototyping  rig  andyhuntingdon  awesome 
december 2009 by infovore
Natural Earth
"Natural Earth is a public domain map dataset available at 1:10m, 1:50m, and 1:110m scales. Featuring tightly integrated vector and raster data, with Natural Earth you can make a variety of visually pleasing, well-crafted maps with cartography or GIS software." Oooh.
gis  data  maps  cartography  vector  free 
december 2009 by infovore
Four Walks
"I did a set of four walks in Austria; two long ones, two short ones. I did some "daystreaming" where using bits of technology I was updating my location, status and pictures as I walked." Ambient information gathering, whilst taking in the outdoors, and all for charity. Lovely.
walking  streaming  ambient  data  hiking  christhorpe 
november 2009 by infovore
SPARQL By Example (1)
Really excellent presentation on the basics of SPARQL - lots of good examples, lots of hands-on stuff, and clear. Worth going back to.
tutorial  presentation  data  semanticweb  web  programming  rdf  sparql  semantic 
october 2009 by infovore
GameSetWatch - Backup Your Files Before Playing Lose/Lose
"Lose/Lose is a simple vertical-scrolling shoot'em up with a twist -- each alien appearing on your screen represents a random file on your computer. Thus, each time you kill an alien, the game will delete that sprite's associated file. If the aliens manage to destroy your ship, the Lose/Lose application is deleted." Way to make a point, but, you know, *blimey*.
games  security  loselose  data  value 
september 2009 by infovore
Graphic Presentation - a set on Flickr
"Some pages from Willard Cope Brinton's second book (1939)". Very, very lovely.
graphics  charts  design  diagrams  books  data  information  infoviz 
september 2009 by infovore
World of Warcraft helps fight crime in LA | Technology | The Guardian
'"We studied these online gangs at the same time I was looking at the offline gangs and it turned out the model we were developing to explain the behaviour of the online guilds began to coincide with the offline gangs," says Johnson. "We could explain the data using the same mathematical ideas."' Which all makes sense, you know, but it's still interesting to see this stuff being done and taken seriously.
data  groups  community  games  wow  gangs  social  analysis 
july 2009 by infovore
YouTube - Giant White Glove
That performance of Billie Jean. But with a Giant White Glove. Brilliant.
videos  processing  manipulation  michaeljackson  data  motioncapture 
july 2009 by infovore
White Glove Tracking
"On May 4th, 2007, we asked internet users to help isolate Michael Jackson's white glove in all 10,060 frames of his nationally televised landmark performance of Billy Jean. 72 hours later 125,000 gloves had been located. wgt_data_v1.txt (listed below) is the culmination of data collected. It is released here for all to download and use as an input into any digital system. Just as the data was gathered collectively it is our hope that it will be visualized collectively." This is amazing. And what it leads to is even better.
michaeljackson  motiontracking  video  art  data  crowdsourcing  visualisation 
july 2009 by infovore
Hiding data, content and technology in real world games
Some jolly good stuff from Chris, notably "And I Saw..". I mainly like it, though, because he went and made a thing, and it definitely worked, and it's so, so simple.
data  games  play  sms  christhorpe  andisaw  simplicity  making 
july 2009 by infovore
Michael Jackson, Tracks played per hour after his death on Flickr - Photo Sharing!
"Fresh data straight out of our uber warehouse: As the news breaks, scrobbles soar as people go to pay tribute to one of the greatest pop artists of all times." I really didn't want to talk about this story at all - but at least there's some interesting data about it. So have some data.
michaeljackson  data  lastfm  currentevents  kneejerk 
june 2009 by infovore
The Three Sexy Skills of Data Geeks : Dataspora Blog
"Statisticians’ sex appeal has little to do with their lascivious leanings ... and more with the scarcity of their skills. I believe that the folks to whom Hal Varian is referring are not statisticians in the narrow sense, but rather people who possess skills in three key, yet independent areas: statistics, data munging, and data visualization. (In parentheses next to each, I’ve put the salient character trait needed to acquire it)."
data  analytics  visualization  statistics  datamining  maths  analysis  trends 
june 2009 by infovore
Just Landed: Processing, Twitter, MetaCarta & Hidden Data | blprnt.blg
Mapping where people are leaving and arriving based on nothing more than what they said on Twitter. Pretty, and perhaps the beginnings of something quite useful.
data  informatics  twitter  visualisation  processing  mapping  socialmedia 
may 2009 by infovore
Left 4 Dead Blog - Surviving the L4D Survival Pack
A thoughful post (as ever) from the L4D team detailing some of the balancing and planning that's gone into the Survival Mode experience. Looking forward to firing this up next week...
games  left4dead  valve  mechanics  balance  data 
april 2009 by infovore
Above 49: The Importance of Readability in Games
"Players need to understand all the inputs and all the outputs to make interesting, informed decisions. These are the mechanisms through which we express our will in the game. This is the machinery that transforms our medium from passive to interactive... This is a multifaceted (and as far as I'm aware, relatively unexplored) issue, but we can begin making inroads. Making games more readable begins with two things- empathy and data." Nels on Don Norman and readability, amongst other things.
games  donnorman  nelsanderson  interaction  data  design  readability 
april 2009 by infovore
Internet records to be stored for a year - Telegraph
This is not good. And the worst part: "Hundreds of public bodies and quangos, including local councils, will also be able to access the data to investigate flytipping and other less serious crimes." It's not the police having this that's the big worry; it's the incompetent lower echelons of civil service. who shouldn't need this.
security  privacy  uk  europe  internet  data 
april 2009 by infovore
Commissioning for Attention Part 1 - Read Me! « TEST
"...these ideas have been massively influenced by friends working in game design, agile website design or service design. Narrative media is still (outside of gaming) light-years behind the curve compared to the work going on in these disciplines, so a lot of the time I’m trying to act as a translator - taking concepts and ideas from more functional design disciplines into narrative/editorial contexts. When I speak to indies or producers, there’s a set of blogs/presentations that I tend to refer them to, so I thought i’d start by sharing this reading list." This looks like it's going to be an excellent series from Matt Locke.
mattlocke  crossplatform  media  design  comissioning  planning  trends  advice  data  platform 
march 2009 by infovore
Beautiful Data | O'Reilly Media
"With this unique book, programmers, administrators, and others who handle data can learn by example from the best data practitioners in the history of the field. Modeled after O'Reilly's highly-acclaimed book, Beautiful Code, Beautiful Data lets readers look over the shoulders of prominent data designers, managers, and handlers for a glimpse into some of the most interesting projects involving data. In an engaging narrative format, the authors think aloud as they explain their work, highlighting the simple and elegant solutions to problems they encountered along the way." Oh. This could be lovely.
book  publishing  data  visualisation  informatics  oreilly 
march 2009 by infovore
The Guardian Open Platform |
The Guardian Open Platform launches, with their Content API, their Data Store, and a selection of client libraries for the API (one of which I did a smidge of work on). This is not just a good thing, it's a good thing Done Right, and I'm looking forward to what's next from the Open Platform team.
platform  web  guardian  data  journalism  api  content  openplatform 
march 2009 by infovore
Matthew Bloch - accidents
"A collection of accidents that happened while working on maps and other graphics." Bloopers from interactive infographics. Delightful; the patina and happy accidents of the 21st century.
infographics  maps  data  visualisation  error  happyaccident  bloopers 
march 2009 by infovore
Animation: Target's Spread Across The U.S.
"Not to be outdone (well, maybe a little outdone), we've combed through hours of imaginary data here at Consumerist HQ and put together a similar animation that illustrates Starbucks' explosive growth over the past 20 years. Enjoy." I did.
animation  information  infographics  data  dataviz  starbucks  lol 
february 2009 by infovore
Lubing the Edges of the Internet - Jan Chipchase - Future Perfect
"...the biggest consequence [of a universal micro-USB adaptor] will be the ease of transferring data/content from street service provider to consumer, and consumer to consumer... There is a place at the edges of the internet where the level of friction makes content and data grind to a halt. It's largely unregulated. And it just got seriously lubed."
mobile  distribution  interface  data  friction  connectivity  phone  standards  edges  microusb  telephony 
february 2009 by infovore
Warcraft guild achievements as RSS -
"...once WoLK came out and half the guild went completely insane and started chasing the really silly achievements, it was clear we were going to need an RSS feed of the things. So I built one. It’s based on the Armory, like most WoW tools, and is a complete kludge, like most of my tools. But here are my notes anyway." Hurrah! Tom wrote his magic tool up. It's great, it's daft, and I love the Armory's crazy XML. Alas, my achievements are few and far between...
games  feeds  wow  worldofwarcraft  data  scripting  python  unassignedvariable  armory 
february 2009 by infovore
Energy Information
"Google PowerMeter, now in prototype, will receive information from utility smart meters and energy management devices and provide anyone who signs up access to her home electricity consumption right on her iGoogle homepage."
data  visualisation  google  sustainability  energy  power  energyconsumption 
february 2009 by infovore
See the data underlying our tax database | Business |
"Our team of investigative journalists has compiled a database from four years' worth of company accounts to show how much the FTSE 100 companies make in pre-tax profits, and how much they pay in tax. We have published this data as a user-friendly interactive guide at" But, as well as the user-friendly guide, there's also all the data. Bravo.
information  guardian  data  journalism  tax  datajournalism 
february 2009 by infovore
Slow data and the pleasure of automated nostalgia « TEST
"I’m much more interested in automated nostalgia than automated presence - data feeds that gradually acrue in your wake, rather than constantly dragging your focus on to the next five minutes." Yes.
information  narrative  history  data  visualisation  slow  pace 
january 2009 by infovore
Fitting curves to data using Ruby and the GNU Scientific Library
"If you need to perform data analysis, provide graphics for your users in your webapp, or produce high quality plots I encourage you to investigate the combination of ruby, GSL and GNUPlot." Looks good. I should probably give this a poke some time; could come in handy.
gsl  graphing  plotting  data  analysis  statistics  ruby  visualisation 
december 2008 by infovore
Down the Rabbit Hole of the Pentagon Graphics Machine. | The Art of Information
"I won’t rant about how our tax dollars pay for these images and how we deserve better. But what I do find alarming is that these documents are used to brief major decision makers. These decision makers may know a thing or two about policy and politics, but if decoding and understanding the armed forces budget is the goal of these documents, then there is a huge failure here." Datafail and slidecrime, all under one roof.
military  infographics  visualisation  data  information  charts 
november 2008 by infovore
Databases, Lists, Maps, Rankings - Index - Data Desk - Los Angeles Times
"Maps, databases and other resources that help you dig deeper." A shame the raw data isn't available, but great they're collating this stuff and seeing it as another channel of news they provide.
data  visualisation  resource  latimes  newspaper  journalism  stats 
november 2008 by infovore
The Screens Issue - If You Liked This, Sure to Love That - Winning the Netflix Prize -
"Mathematically speaking, “Napoleon Dynamite” is a very significant problem for the Netflix Prize. Amazingly, Bertoni has deduced that this single movie is causing 15 percent of his remaining error rate; or to put it another way, if Bertoni could anticipate whether you’d like “Napoleon Dynamite” as accurately as he can for other movies, this feat alone would bring him 15 percent of the way to winning the $1 million prize."
data  prediction  movies  netflix  modelling  napoleondynamite 
november 2008 by infovore
Nodalities » Blog Archive » A data-centric view
"The point here, is that the flickr team did not wake up one morning and think: “You know, if we captured THIS kind of data, we could create this mashup; so let’s create an application.” Instead, they re-used data they were already capturing, and brought out something very interesting indeed. By creating tools which match their data (and could be used with other data of the same kinds), flickr is able to expose layers of value from the rich-pickings of their own data-cloud. The good stuff is where the data are." Yes, it is.
data  flickr  reuse  information  tools  geodata  mapping 
november 2008 by infovore
You Know What I Did Last Summer? (Frumination)
"I spent 10 weeks last Summer as an intern on the strategy team of Transport for London's (TfL) London Rail division.... My general task was to help London Rail start to make use of the oceans of data spewing out of the Oyster smartcard ticketing system, but I spent the bulk of my time working on a project that came to be titled Oyster-Based Performance Metrics for the London Overground. I've posted my final report and slides and outline for the presentation I gave to TfL executive management." Some interesting data and information here.
travel  tfl  statistics  oyster  overground  data  graphs 
november 2008 by infovore
"Faker, a port of Data::Faker from Perl, is used to easily generate fake data: names, addresses, phone numbers, etc."
tool  testing  data  ruby  gem  programming  rails 
october 2008 by infovore
russell davies: design engaged the second
"The dataspace of the well-tempered environment will soon be invaded by logos, credits, banners and offers. The financial temptations will, I suspect, be too hard to resist." Loads of excellent stuff in here besides this, though. Can't recommend enough.
ubicomp  spimes  design  spam  cities  totalexperiencedesign  data  visualisation  information  advertising 
october 2008 by infovore
Brendan O’Connor’s Blog - AI and Social Science » conplot - a console plotter
"This has to be the most quick-and-dirty data visualizer out there: I wrote an ascii art plotter script that takes a column of numbers on stdin and throws out a plot on your console." Oh, that's going to come in handy.
graphing  data  visualisation  plotting  console  cli  shell  linux 
october 2008 by infovore
Geo Spidering » Blog »
"The technology will probably improve, but in lieu of the promised emergent web AI, we need to build more small tools, more games to bootstrap datasets, and more simple ways of encouraging people to play their part in the semantic web without ever having to explain what it is." tt++.
geo  location  scraping  semantics  tools  small  little  data  parsing  tomtaylor 
october 2008 by infovore
post-ONA conference (tecznotes)
"For one, there's an undercurrent of a siege mentality in journalism right now, with newsrooms cutting staff and print operations frozen stiff in the headlights of the internet. The focus on narrative and story gives a softer edge and an escape valve, though - this group is not primarily a tech-driven community, but they catch on to new developments quickly and bend them into the service of storytelling." Interesting round-up from Mike, particularly with respect to the NYT's election coverage.
journalism  conference  data  programming  development  media  datadriven 
september 2008 by infovore
Near Future Laboratory » Blog Archive » *-computing
"There's a weird conceit in here, that the activities and practices of normal human beings will involve data processing and algorithms of some sort, which is an awfully big assumption. So big, in fact, that it has distilled down to a way of seeing the world as consisting of bits of data that can be processed into information that then will naturally yield some value to people... Design for people, practices and interaction rituals before the assumptions about computation, data structures and algorithms get bolted onto normal human interaction rituals."
computing  data  ubicomp  julianbleecker  social  software  socialsoftware  design 
september 2008 by infovore
Chris' Survival Horror Quest
"Some people believe that there's no correlation between quality and sales, and thus think that the way to make money is to make things that are easily marketable (read: licenses). Game developers themselves usually argue that sales above a certain level require a game to be sufficient quality. I decided to see which of these perspectives was correct for the Playstation 2 era." Datanalysismachinego!
data  visualization  statistics  sales  games  quality  analysis 
september 2008 by infovore
« earlier      
per page:    204080120160

related tags

3d  aanandprasad  aaronstraupcope  accumulation  addiction  adrianholovaty  advertising  advice  africa  algorithms  ambient  amee  analysis  analytics  andisaw  andyhuntingdon  animation  anonymisation  api  application  archaeology  archiving  arduino  armory  art  augmentedreality  awesome  balance  bash  belief  bengimpert  benhammersley  bigdata  biology  blog  bloom  bloopers  book  bookmooch  books  bretvictor  browser  bumpmapping  cartography  categorisation  charts  chrisheathcote  christhorpe  christmas  chrome  cities  ckan  cli  code  collection  collectiveintelligence  comissioning  commandline  community  comparison  computers  computervision  computing  conference  connectivity  console  content  credibility  criticism  crossplatform  crowdsourcing  csv  culture  currentevents  cybernetics  cycling  data  database  datadecs  datadriven  datajournalism  datamining  datascience  datasets  datavisualisation  dataviz  dating  dconstruct08  design  development  diagrams  distributed  distribution  donnorman  edges  effects  energy  energyconsumption  english  error  ethernet  europe  exploration  facebook  fearmongering  feeds  fisualisation  flickr  free  friction  friends  games  gangs  gem  genetics  geo  geocoding  geodata  gis  glanceable  google  government  graphics  graphing  graphs  greebling  gregborenstein  grindr  griot  groups  gsl  guardian  happyaccident  hardware  hiking  hiphop  history  hoes  http  humanities  IA  import  infographics  infononsense  informatics  information  informationdesign  informationoverload  infoviz  infovore  input  intelligence  interaction  interactive  interface  internet  intertwingly  iphone  ipod  iraqwar  itunes  jamesdarling  javascript  journalism  jq  jquery  json  julianbleecker  karsalfrink  kneejerk  knowledge  language  lastfm  latimes  learning  left4dead  linux  literarycriticism  literature  little  location  lol  loselose  ludacris  machinelearning  making  management  manipulation  map  mapping  maps  marketing  markup  materialexploration  maths  mattlocke  mechanics  media  medicine  mentalmodel  metadata  metaphor  metrics  michaeljackson  microusb  mikemigurski  military  minecraft  misogyny  mitchellwhitelaw  ml  mmorpg  mobile  modding  modelling  motioncapture  motiontracking  movies  music  mysql  napoleondynamite  narrative  navigation  nelsanderson  netflix  networking  networks  newspaper  nike  nikeplus  observation  obsolescence  okcupid  open  openlibrary  openplatform  oreilly  output  outsidein  overground  ownership  oyster  pace  package  paper  papernet  parsing  pauldowney  perl  personal  phone  photography  photos  php  pipeline  planning  platform  play  plotting  plugin  poetry  portability  power  prediction  presentation  printing  privacy  processing  programming  protection  prototyping  psd  psychology  publishing  python  quality  quietdevices  radiohead  rails  rapidprototyping  raw  rdf  readability  recommendation  regex  reporting  repository  representation  research  resource  reuse  rig  riverthames  ruby  rubyonrails  running  rust  sales  scale  science  scraping  script  scripting  security  semantic  semantics  semanticweb  sensor  shell  simplicity  slow  small  sms  social  socialmedia  socialsoftware  society  sociology  software  source  spam  sparql  spelunking  spimes  spreadsheet  spreadsheets  stamen  standards  starbucks  statistics  stats  stephenfew  stevenjohnson  storage  stories  streaming  structure  surface  surveillance  survey  sustainability  synecdoche  systems  tables  talking  tax  technology  telephony  testing  textextraction  tfl  thames  tomtaylor  tool  tools  totalexperiencedesign  towerbridge  transport  travel  trends  trust  tutorial  twitter  ubicomp  uk  unassignedvariable  unix  usability  utility  uxweek  value  valve  vector  via:blackbeltjones  video  videos  visualisation  visualization  walking  web  webofdata  whynotmakeitpretty  workflow  worldofwarcraft  wow  xml 

Copy this bookmark: