jerryking + open_data   65

How 5 Data Dynamos Do Their Jobs
June 12, 2019 | The New York Times | By Lindsey Rogers Cook.
[Times Insider explains who we are and what we do, and delivers behind-the-scenes insights into how our journalism comes together.]
Reporters from across the newsroom describe the many ways in which they increasingly rely on datasets and spreadsheets to create groundbreaking work.

Data journalism is not new. It predates our biggest investigations of the last few decades. It predates computers. Indeed, reporters have used data to hold power to account for centuries, as a data-driven investigation that uncovered overspending by politicians, including then-congressman Abraham Lincoln, attests.

But the vast amount of data available now is new. The federal government’s data repository contains nearly 250,000 public datasets. New York City’s data portal contains more than 2,500. Millions more are collected by companies, tracked by think tanks and academics, and obtained by reporters through Freedom of Information Act requests (though not always without a battle). No matter where they come from, these datasets are largely more organized than ever before and more easily analyzed by our reporters.

(1) Karen Zraick, Express reporter.
NYC's Buildings Department said it was merely responding to a sudden spike in 311 complaints about store signs. But who complains about store signs? was hard to get a sense of the scale of the problem just by collecting anecdotes. So I turned to NYC Open Data, a vast trove of information that includes records about 311 complaints. By sorting and calculating the data, we learned that many of the calls were targeting stores in just a few Brooklyn neighborhoods.
(2) John Ismay, At War reporter
He has multiple spreadsheets for almost every article he works on......Spreadsheets helped him organize all the characters involved and the timeline of what happened as the situation went out of control 50 years ago......saves all the relevant location data he later used in Google Earth to analyze the terrain, which allowed him to ask more informed questions.
(3) Eliza Shapiro, education reporter for Metro
After she found out in March that only seven black students won seats at Stuyvesant, New York City’s most elite public high school, she kept coming back to one big question: How did this happen? I had a vague sense that the city’s so-called specialized schools once looked more like the rest of the city school system, which is mostly black and Hispanic.

With my colleague K.K. Rebecca Lai from The Times’s graphics department, I started to dig into a huge spreadsheet that listed the racial breakdown of each of the specialized schools dating to the mid-1970s.
analyzed changes in the city’s immigration patterns to better understand why some immigrant groups were overrepresented at the schools and others were underrepresented. We mapped out where the city’s accelerated academic programs are, and found that mostly black and Hispanic neighborhoods have lost them. And we tracked the rise of the local test preparation industry, which has exploded in part to meet the demand of parents eager to prepare their children for the specialized schools’ entrance exam.

To put a human face to the data points we gathered, I collected yearbooks from black and Hispanic alumni and spent hours on the phone with them, listening to their recollections of the schools in the 1970s through the 1990s. The final result was a data-driven article that combined Rebecca’s remarkable graphics, yearbook photos, and alumni reflections.

(4) Reed Abelson, Health and Science reporter
the most compelling stories take powerful anecdotes about patients and pair them with eye-opening data.....Being comfortable with data and spreadsheets allows me to ask better questions about researchers’ studies. Spreadsheets also provide a way of organizing sources, articles and research, as well as creating a timeline of events. By putting information in a spreadsheet, you can quickly access it, and share it with other reporters.

(5) Maggie Astor, Politics reporter
a political reporter dealing with more than 20 presidential candidates, she uses spreadsheets to track polling, fund-raising, policy positions and so much more. Without them, there’s just no way she could stay on top of such a huge field......The climate reporter Lisa Friedman and she used another spreadsheet to track the candidates’ positions on several climate policies.
311  5_W’s  behind-the-scenes  Communicating_&_Connecting  data  datasets  data_journalism  data_scientists  FOIA  groundbreaking  hidden  information_overload  information_sources  journalism  mapping  massive_data_sets  New_York_City  NYT  open_data  organizing_data  reporters  self-organization  systematic_approaches  spreadsheets  storytelling  timelines  tools 
june 2019 by jerryking
Spinning raw government datasets into gold - The Globe and Mail
Special to The Globe and Mail
Published Monday, Feb. 02 2015
massive_data_sets  open_data  data  datasets 
february 2015 by jerryking
Let me see
Posted by Seth Godin on July 08, 2008.

Passive contributions of public behaviour information to traditionally-sorted data
data  ideas  information  inspiration  Seth_Godin  social_data  datasets  open_data  social_physics  massive_data_sets  wisdom_of_crowds  thick_data  public_behavior  sorting  value_creation 
january 2015 by jerryking
How Consumers Are Using Big Data - WSJ
March 23, 2014

An app called Neighborland, created by social entrepreneurs Candy Chang and Dan Parham, aims to help community groups and government offices work well together. The app combines photos, data and APIs from sources including Twitter, Google Maps and Instagram, agencies that report on real-estate parcels, transit systems, and "311" complaints about nuisances like noise, broken lights and garbage.

In 2012, the New Orleans Food Truck Coalition used Neighborland to collect community ideas, map "food deserts," which are areas lacking easy access to groceries and healthy food, and show what the economic and health impact could be if coalition members were permitted to work in more areas.
311  massive_data_sets  APIs  data  analytics  Amazon  Pandora  Netflix  Nike  Jawbone  fitness  CDC  infertility  travel  Skyscanner  Routehappy  open_data  mobile_applications  consumers  hyperlocal  neighbourhoods 
november 2014 by jerryking
They’re Tracking When You Turn Off the Lights - WSJ - WSJ
Oct. 20, 2014

Tech companies have used the technologies and techniques collectively known as big data to make business decisions and shape their customers’ experience. Now researchers are bringing big data into the public sphere, aiming to improve quality of life, save money, and understand cities in ways that weren’t possible only a few years ago....Municipal sensor networks offer big opportunities, but they also carry risks. In turning personal habits into digital contrails, the technology may tempt authorities to misuse it. While academics aim to promote privacy and transparency, some worry that the benefits of big data could be lost if the public grows wary of being monitored... Anthony Townsend, author of the book “Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia.”...The goal isn’t to sell products or spy on people, the academics say, but to bolster quality of life and knowledge of how cities function
cities  massive_data_sets  sensors  urban  privacy  smart_cities  predictive_analytics  books  quality_of_life  customer_experience  open_data  community_collaboration  white_hats 
october 2014 by jerryking
How the big-data revolution can help design ideal cities - The Globe and Mail
The Globe and Mail
Published Wednesday, Sep. 24 2014

The big-data revolution faces two key challenges, both concerning the collection of information.

First, as is always the case when it comes to monitoring individuals and collecting details about their lives, is privacy. Second, there is the issue of using that data responsibly....Once municipalities have that consent, there is then the issue of harmonizing data sets in order to gain a fuller picture of issues. For instance, if a municipality wants to understand water-consumption levels, it helps to know how they track weather patterns.

Many cities are still struggling to understand how to use big data, but it promises to be a hugely important urban-planning tool.
algorithms  IBM  real-time  urban  sensors  municipalities  massive_data_sets  cities  data  decision_making  privacy  urban_planning  open_data 
september 2014 by jerryking
Andreessen Horowitz Bets on a Government Software Start-Up -

OpenGov, a start-up that sells software to help local governments manage data, announced on Thursday that it had received a roughly $15 million financing round led by Andreessen Horowitz. It is the first time that Andreessen Horowitz, one of Silicon Valley’s leading investment shops, has backed a company in the business of “govtech,” to use the industry parlance.

Local governments have suffered in the aftermath of the financial crisis, with many forced to lay off workers and cut back on services. But Andreessen Horowitz sees an opportunity in that market.

Government software is “kind of old,” Balaji Srinivasan, a partner at the venture capital firm, said. “So we think we can do a lot there.”
Andreessen_Horowitz  open_data  start_ups  government  OpenGov  government_2.0  gov_2.0 
june 2014 by jerryking
Why the Ontario election campaign is a mystery, even to those involved - The Globe and Mail
The Globe and Mail
Published Friday, May. 23 2014

reports from candidates and their teams who are out knocking on doors indicate that even fewer voters than usual are aware there is an election on, let alone have strong impressions of how it’s playing out.

That’s especially the case in the suburban ridings of the Greater Toronto Area, generally considered Ontario’s most important electoral battleground, where the commuter-heavy population is particularly difficult to make contact with. ...Then there are the uncertainties about what campaign Ontarians will see the rest of the way. Just as the pollsters are trying to adjust to the difficulty of reaching people the way they used to, so too are the parties. Amid experimentation with online and other less traditional forms of advertising, nobody is quite sure what will break through; neither is it obvious whose efforts to use data to micro-target voters in ground campaigns will work.
Ontario  elections  data  political_campaigns  GTA  microtargeting  open_data 
may 2014 by jerryking
How to Make a Map Go Viral
MAY 2 2014 | Atlantic Monthly |ROBINSON MEYERMAY 2 2014,

What kind of data do you look for, and how do you find it?

I don't have a particular type of data that I look for beyond my subjective ...
mapping  howto  virality  massive_data_sets  open_data  data  data_scientists  from notes
may 2014 by jerryking
Disruptive entrepreneurs: An interview with Eric Ries
April 2014 | McKinsey & Company |

All of our process diagrams [in major corporations] are linear, boxed diagrams that go one way. But entrepreneurship is fundamentally iterative. So our diagrams need to be in circles. We have to be willing to be wrong and to fail. But modern management says, “Failure means you get dinged.”

For example, one of things I’ve tried to do is to tell companies, “Put on your employees’ performance evaluation a concept we call productive failure: ‘How many productive failures did you have this year?’ If someone comes to you and claims that they didn’t fail this year, you know one of two things: they’re either lying to your face or they were incredibly, unbelievably conservative.”

In both cases, it’s actually not a positive attribute. You want to say, “Show me a time when you failed but learned something really valuable, or were able to pivot from something that didn’t work to something that did.” I have a lot of examples now where it’s possible to say: “You saved the company an incredible amount of money, because instead of spending $10 million on something, we spent $100,000 and did an experiment that proves conclusively there ain’t no business here.”
Eric_Ries  lean  open_data  innovation  experimentation  entrepreneurship  McKinsey  disruption  failure  linearity  iterations  performance_reviews 
april 2014 by jerryking
Making dollars and sense of the open data economy - O'Reilly Radar
by Alex Howard | @digiphile | +Alex Howard | Comment | December 11, 2012.

Any post-mortems that picked up on the broad challenges, problems. difficulties of monetizing open-data?
monetization  open_data  commercialization 
january 2014 by jerryking
Kensho, a startup doing Siri (or Watson) for financial markets, has raised $10M — Tech News and Analysis
By Derrick Harris
Jan. 22, 2014

It looks like a smart product from a smart team, especially if the UI and visualizations are as good as the algorithms.... Warren (as in Warren Buffett, I presume), is a natural-language search engine for data on financial markets. You (assuming you’re a banker or very sophisticated day trader) type in a question — an example from the company’s website is “Which aerospace companies rally following major breakthroughs in drone technology?” — and it returns results in the form of data.
start_ups  open_data  value_chains  fin-tech  finance  Kensho  search  search_engines  financial_services  Siri  IBM_Watson 
january 2014 by jerryking
Accessing Open Data via APIs: Never Mind the App, Is There a Market for That?
Mark Boyd, September 4th, 2013

But is the market ready to monetize? In Big Data: A Revolution That Will Transform How We Live, Work, and Think, authors Viktor Mayer-Schönberger and Kenneth Cukier argue that at present, those with “the most value in the big data value chain” are those businesses and entrepreneurs with an innovative mindset attuned to the potential of big and open data. While still in its nascence, “the ideas and the skills seem to hold the greatest worth”, they say. However, they expect:

“…eventually most value will be in the data itself. This is because we’ll be able to do more with the information, and also because the data holders will better appreciate the potential value of the asset they possess. As a result, they’ll probably hold it more tightly than ever, and charge outsiders a high price for access.”
data_scientists  open_data  massive_data_sets  entrepreneurship  start_ups  InfoChimps  Junar  mindsets  commercialization  monetization 
january 2014 by jerryking
Opening up open data: An interview with Tim O’Reilly | McKinsey & Company
Opening up open data: An interview with Tim O’Reilly
The tech entrepreneur, author, and investor looks at how open data is becoming a critical tool for business and government, as well as what needs to be done for it to be more effective.
January 2014
open_data  Tim_O’Reilly  McKinsey 
january 2014 by jerryking
What executives should know about open data
anuary 2014 | McKinsey & Company | by Michael Chui, James Manyika, and Steve Van Kuiken.
open_data  McKinsey  executive_management  data  MyData 
january 2014 by jerryking
New York, the Silicon City -

What lessons does this have for the new mayor? New York’s gains came, in part, from the aggressive efforts of the Bloomberg administration to stimulate the technology and information sector. These included funding tech incubators; the “Made in NY” marketing campaign to support small tech companies; the rapid extension of broadband access across the city; the city’s broad-reaching Open Data initiative, which makes city data available to the public and software developers; and the selection of Cornell and Technion, the Israel Institute of Technology, to open a huge new campus on Roosevelt Island.
Cornell  New_York_City  Silicon_Alley  Bill_de_Blasio  digital_economy  open_data  geographic_ingredient_branding 
january 2014 by jerryking
Open Data a Boon for Entrepreneurs -
Jan. 8, 2014 | WSJ | By Angus Loten.

A poster child for the movement might be 34-year-old Matt Ehrlichman of Seattle, who last year built an online business in part using Seattle work permits, professional licenses and other home-construction information gathered up by the city's Department of Planning and Development.

While his website is free, his business, called, has more than 80 employees and charges a $35 monthly fee to industry professionals who want to boost the visibility of their projects on the site.

The site gathers raw public data—such as addresses for homes under renovation, what they are doing, who is doing the work and how much they are charging—and combines it with photos and other information from industry professionals and homeowners. It then creates a searchable database for users to compare ideas and costs for projects near their own neighborhood.

Mr. Ehrlichman raised $6.25 million from angel investors in October 2012, and expects to hire nearly 80 more workers by the end of 2014, as he continues to expand the online service nationally.
open_data  entrepreneur  entrepreneurship  home_renovations  Seattle  home-center_industry  home-improvement 
january 2014 by jerryking
Open Data Movement
July 2012 | Public Management | ALISSA BLACK.Director, California Civic Innovation Project New America Foundation Washington, D.C.
open_data  local  public_sector  massive_data_sets  DataCouch  open_government 
january 2014 by jerryking
The open data dilemma
May 19, 2010 | NOW Magazine | By Joshua Errett.

David Eaves, Canada's open government advocate, has a few ideas why a movement with an impressive wingspan has yet to fly.

According to Eaves, there are three main areas on inaction: government, media and the developer community.
(A) Firstly, the city. It simply hasn't opened up enough data. The above mentioned TTC data, for instance, is not out in the open yet. "Cities still think of this as a cost.
(B)The media – an industry that survives on information – could be the single largest benefactor in the open data movement. For one, investigations would become easier and efficient. All the digging into council minutes or agendas could be simplified in a single web application. "I want an ecosystem where the smallest media shops to be able to grab feed from city hall and tell citizens what's going on there. Media can then mash it up to make only data relevant to, say, the west end," argues Eaves. For proof, see what the Guardian does with London's enviable open data program. "If we had all the information, local media could deliver valuable services. But I've seen none of the media stand up and say, we would like this. Because they don't get it either."
(C) local web developers haven't done their part.

"I'm not sure the developer community here has pushed as aggressively as say San Francisco or Washington, DC. The biggest question I have for Toronto is: What have you done? How many people have taken the whatever small amount of data here and mashed it up in some way?"

Developers are the essential part of the open data equation.
open_data  Toronto  dilemmas 
december 2013 by jerryking
Open data: Is there a business case? | ZDNet
By David Meyer for Communication Breakdown | September 18, 2012
open_data  monetization 
december 2013 by jerryking
Monetizing open data
September 21, 2012| Strata| by Jenn Webb

One of the big questions on everyone’s mind at this year’s Open Knowledge Festival in Helsinki, according to a report by David Meyer at ZDNet, is: Where’s the money in open data?

Ville Peltola, IBM’s innovation chief in Finland, told Meyer the situation is becoming frustrating, that he doesn’t understand why it’s so hard to properly open up data, or even just some of it. “You could have bronze, silver and gold APIs, where more data costs more,” Peltola said to Meyer. “It’s like a drug dealer. Maybe you have to solve this chicken-and-egg problem by giving samples of raw data.”

Meyer points out the real issue inherent in what Peltola is saying: “that large amounts of data are very valuable, and the companies that create them tend not to know how to realise the greatest value from them.” Peltola had an interesting idea to address this: “What if you have an internal start-up in your company tasked only with monetising your data?”

Chris Taggart, co-founder of OpenCorporates, made a more competitive argument for opening up your company’s data: it “exposes your competitors’ internal contradictions” and might inspire disruption, he told Meyer — “Most big, fat secure companies don’t have the confidence to disrupt themselves,” he said.
open_data  monetization  massive_data_sets  problems  challenges  intrapreneurship  chicken-and-egg  commercialization  APIs  disruption  complacency  contradictions 
december 2013 by jerryking
The messy reality of open data and politics | Public Leaders Network
8 April 2013 | | Guardian Professional | Tim Davies, Guardian Professional.

In practice, datasets themselves are political objects, and policies to open up datasets are the product of politics. If you look beyond the binary fight over whether government data should be open or not, then you will find a far more subtle set of political questions over the what and the how of opening data.

Datasets are built from the categories and relationships that the database designer (or their political masters) decide are important. In their book, Sorting Things Out: Classification and its Consequences, Geoffrey Bowker and Susan Leigh Star describe how the international classification of disease, the basis for worldwide mortality statistics, has historically under-represented tropical diseases in its code lists. The result is that global health policy has been less able to see, distinguish and target certain conditions....Local authority spending data has never existed as a single dataset before – but a central edict that this should be published, itself a decision with a political edge, has generated new standards for representing local spend, that have to decide what sort of information about spend is important.

Should the data contain company identifiers to let us see which firms get public money? And should spend data be linked to results and categorisation of public services? These decisions can have big impacts on how data can be used, what it can tell us, and what impacts open data will have.
datasets  data  open_data  cities  municipalities  politics  political_campaigns  sorting  messiness 
december 2013 by jerryking
Open Data: Empowering the Empowered or Effective Data Use for Everyone? | Gurstein's Community Informatics
The overall intention is to make local, regional and national data (and particularly publicly acquired data) available in a form that allows for direct manipulation using software tools as for example, for the purposes of cross-tabulation, visualization, mapping and so on.
open_data  definitions 
december 2013 by jerryking
Data analysis tools target non-experts - Strata
by Ben Lorica | @bigdata | Comments: 4 | August 25, 2013
tools  open_data 
december 2013 by jerryking
A Modern Approach to Open Data | Make government better, together.
October 01, 2013 by Ben Balter, GitHub.

Traditionally, consuming open government data required building and curating many custom tools and wrappers to convert the data from the form it’s exposed in to something more immediately consumable by civic hackers, watchdog groups, and the general public. Developers haphazardly wrote small scripts as one-off efforts and threw them away, or left their solutions buried inside larger infrastructure, reinventing the wheel with each new transparency initiative.

Developers from the Sunlight Foundation, GovTrack, and the New York Times, however, decided to join forces and break from tradition when they reached out to other civic-minded developers and “decided to stop each building the same basic tools over and over, and start building a foundation [they] could share.”
open_data  open_source  tools  self-organization  sharing_economy  reinventing_the_wheel  organizing_data 
december 2013 by jerryking
Can I build a company on open data?
September 27, 2013 | MaRS Data Catalyst | By Joe Greenwood.
MaRS  open_data  start_ups  analytics  entrepreneurship  presentations 
december 2013 by jerryking
About Data Catalyst - MaRS Data Catalyst
Helen Kula, Manager, Data Product
Adam Jacobs, Data Analyst
Sameer Vasta, Manager, Public Affairs
MaRS  open_data  entrepreneurship  Ontario  Toronto 
december 2013 by jerryking
Open data is not a panacea | mathbabe
December 29, 2012 Cathy O'Neil,
And it’s not just about speed. You can have hugely important, rich, and large data sets sitting in a lump on a publicly available website like wikipedia, and if you don’t have fancy parsing tools and algorithms you’re not going to be able to make use of it.

When important data goes public, the edge goes to the most sophisticated data engineer, not the general public. The Goldman Sachs’s of the world will always know how to make use of “freely available to everyone” data before the average guy.

Which brings me to my second point about open data. It’s general wisdom that we should hope for the best but prepare for the worst. My feeling is that as we move towards open data we are doing plenty of the hoping part but not enough of the preparing part.

If there’s one thing I learned working in finance, it’s not to be naive about how information will be used. You’ve got to learn to think like an asshole to really see what to worry about. It’s a skill which I don’t regret having.

So, if you’re giving me information on where public schools need help, I’m going to imagine using that information to cut off credit for people who live nearby. If you tell me where environmental complaints are being served, I’m going to draw a map and see where they aren’t being served so I can take my questionable business practices there.
open_data  unintended_consequences  preparation  skepticism  naivete  no_regrets  Goldman_Sachs  tools  algorithms  Cathy_O’Neil  thinking_tragically  slight_edge  sophisticated  unfair_advantages  smart_people  data_scientists  gaming_the_system  dark_side 
december 2013 by jerryking
Sizing Up Big Data, Broadening Beyond the Internet -
June 19, 2013 | NYT | By STEVE LOHR.

The story is the same in one field after another, in science, politics, crime prevention, public health, sports and industries as varied as energy and advertising. All are being transformed by data-driven discovery and decision-making. The pioneering consumer Internet companies, like Google, Facebook and Amazon, were just the start, experts say. Today, data tools and techniques are used for tasks as varied as predicting neighborhood blocks where crimes are most likely to occur and injecting intelligence into hulking industrial machines, like electrical power generators.

Big Data is the shorthand label for the phenomenon, which embraces technology, decision-making and public policy. Supplying the technology is a fast-growing market, increasing at more than 30 percent a year and likely to reach $24 billion by 2016, according to a forecast by IDC, a research firm. All the major technology companies, and a host of start-ups, are aggressively pursuing the business.

Demand is brisk for people with data skills. The McKinsey Global Institute, the research arm of the consulting firm, projects that the United States needs 140,000 to 190,000 more workers with “deep analytical” expertise and 1.5 million more data-literate managers, whether retrained or hired, by 2020.
massive_data_sets  Steve_Lohr  data_scientists  data_driven  open_data  neighbourhoods  decision_making  public  McKinsey 
june 2013 by jerryking
Push to exploit an ocean of information
Richard Waters Source: The Financial Times. (Dec. 10, 2012): News: p19

Like anticipating film demand and judging the effectiveness of window displays, much of the effort in the field of big data analytics is aimed at making existing companies more effective. Designing products, setting optimal prices and reaching the best prospects among potential customers are turning into data-driven exercises.

But it is also throwing up disruptive new businesses. Companies set up from scratch have the chance to draw on public streams of digital data to enter markets that were once closed to incumbents with long-established customer relationships and proprietary information. And such businesses come without the legacy technology platforms, entrenched business processes and cultural norms that make it hard for big groups to change.

"Even if you're not a bank or a healthcare company, you can play in banking or healthcare," says James Manyika, director at McKinsey's research arm.
massive_data_sets  Quantifind  Hollywood  Climate_Corporation  sensors  Euclid_Analytics  Kabbage  Factual  disruption  start_ups  McKinsey  data_driven  new_businesses  large_companies  open_data  legacy_players  digital_disruption  customer_relationships  legacy_tech  cultural_norms  Richard_Waters  from notes
february 2013 by jerryking
Jake Porway, Data Scientist Information, Facts, News, Photos -- National Geographic
Data scientist Jake Porway (Ph.D.) is a matchmaker. He sees social change organizations working to make the world a better place, collecting mountains of data, but lacking skills and resources to use that information to advance their mission. He sees data scientists with amazing skills and cutting-edge tools, eager to use their talent to accomplish something meaningful, yet cut off from channels that allow them to do so. He sees governments ready to make data open and available, but disconnected from people who need it. For Porway, it's a match waiting to happen and the reason he founded DataKind (formerly Data Without Borders). It connects nonprofits, NGOs and other data-rich social change organizations with data scientists willing to donate time and knowledge to solve social, environmental and community problems. Ultimately, he wants to build a globally connected network of dedicated experts who can be deployed at a moment's notice to tackle any big data science task worldwide
data_scientists  DataKind  data  match-making  haystacks  PhDs  open_data  nonprofit  NGOs  volunteering 
july 2012 by jerryking
Open Data in Agriculture and Why It Matters
July 16, 2010 | Food+Tech Connect | By Elizabeth Mcvay Greene.

With capabilities like social media that offers instantaneous mini-reports, remote sensing that announces field-level conditions, and user-generated mapping that offers an on-the-ground view of production, merchandising, and consumption activity, we are beginning to get the tools at our fingertips to optimize decision-making with connected, real-time information, not just intuition...don't want farmers’ wisdom to evaporate in the face of technology. Quite the contrary, we want that specialized knowledge of acre, crop, and herd to be augmented and preserved....If a farmer needs to decide how much to irrigate during a drought. It’s a decision that affects just his farm in the short run, but has systemic costs and benefits. If the farmer could connect historical commodity prices, weather charts, financial and environmental costs, and soil conditions to assess the trade-offs in the choice he makes, he could complement his highly refined intuition with the long-term effects that his decision has on his farm and beyond. The more widely information and tools like this are available, the more optimal decisions participants can make throughout the food system.
open_source  agriculture  farming  data  distribution_channels  tools  supply_chains  massive_data_sets  open_data  wisdom  intuition  real-time 
april 2012 by jerryking
How crowd-sourcing will spark a data revolution
March 22, 2012 |Globe and Mail Blog | by frances woolley.

Yet all of these initiatives are geared towards government data sets and professional researchers. Important private records – diaries of early settlers, for example – can find a home in Canada’s National Archives. But the Archives do not have sufficient resources to process and document records of snowdrops or goldfinches. Moreover, the Archives keep records, not data sets – it is fascinating to look at census records from 120 years ago, but they aren’t much use for statistical analysis.

There is a solution: crowd-sourcing. Across the country there are students, amateur and professional historians, policy analysts, bloggers and data nerds. I’m one of them. I’ve taken data collected by a notable Ottawa record keeper, Mr. Harry Thomson, and posted it on Worthwhile Canadian Initiative. Mr. Thomson’s records go back to the 1960s, long before Environment Canada began collecting comparable hydrometric data. An analysis of the data shows a significant decline in peak water levels during the spring flood – with this year being no exception.

Yet Worthwhile Canadian Initiative is just one blog in the vast expanse of the World Wide Web, and might not even be there in five or ten year’s time. We need a permanent site for all of this data, through which the collective power of the internet can be unleashed – editing, compiling, analyzing, telling stories and, above all, building understanding.
analog  archives  Canadian  cannabis  census  crowdsourcing  data  data_driven  datasets  massive_data_sets  nerds  open_data  record-keeping  Statistics_Canada  unstructured_data 
march 2012 by jerryking
Open and shut
March/April 2012 | THIS | Stephanie O'Hanley.

Montréal Ouvert is a citizens’ initiative to obtain a formal open data policy for the city of Montréal, Canada. Launched by four Montrealers in August 2010 to mobilize public and political support for the adoption of an open data policy for the city of Montreal, it has had considerable success. The online presence includes 567 Facebook Fans, 743 Twitter followers and tens of thousands of visits to its website. Over 1 year, Montréal Ouvert organised three public meetings, two hackathons, and presented at over 8 conferences – not to mention blogging, tweeting, report writing, media interviews and general communication in both official languages – no easy task! Also:
Montreal  open_source  open_government  open_data  hackathons 
march 2012 by jerryking
Agriculture And Big Data
11/24/2010 | Forbes | Written by Michael Ferrari.

So, after one panel session comprised of investors looking for opportunities in both hemispheres of the Americas, I asked about the “non-tangible” innovations that often fly under the radar: those that require access to large databases, data manipulation creativity, and computational resources. The panel agreed that these are major focal points for the next generation of agricultural investments. Nearly every discussion that followed seemed to touch upon this theme.

The nice thing about quantifiable data for this community is that it can come from subjective sources as well as those repeatedly tested in a laboratory. A grower’s logbook for instance — containing such information as how a particular crop might respond to a specific weather pattern, the amount and type of pest-fighting application used in a given season, and local market offers — can all be assembled into an index, which is another quantifiable data stream that users may have at their disposal. And while upon first glance one might suppose that data streams are closely-guarded secrets, growers are probably among the most supportive advocates of open access and data sharing. What wiped out your neighbor’s crop a decade ago may be the very thing that hits you this year.
agriculture  massive_data_sets  food_crops  weather  data  farming  tacit_data  unstructured_data  open_data  under_the_radar 
february 2012 by jerryking
Toronto website gives deep look at neighbourhood statistics - The Globe and Mail
, Jun. 29, 2011
Want to know what neighbourhood has the highest graduation rates, the
most trees or the greatest number of car accidents?

The answers are now a click away with a new hub on the city website.
Wellbeing Toronto lets users map an array of services and demographic
information and compare the results across 140 neighbourhoods. Users can
see basic information about a neighbourhood, such as average family
income, education level and age of population by sliding their cursor
over an interactive city map. They also can delve deeper to plot
services such as daycare centres, transit stops, and even convenience
stores and supermarkets in a specific area and see how they stack up
with other parts of the city.
Toronto  websites  community  statistics  neighbourhoods  demographic_information  Elizabeth_Church  municipalities  mapping  open_data  crowdsourcing 
july 2011 by jerryking
Can Todd Park Revolutionize the Health Care Industry? - Technology
June 2011 - The Atlantic By Simon Owens.

The potential benefits of open government initiatives are immense. .. In
the 1970s, the National Oceanic and Atmospheric Administration began
releasing its daily weather data to the public, and today that data is
used by hundreds of companies, from to a variety of
smartphone apps. The govt. also opened up its GPS data in the '80s, a
move that gave birth to an entire industry of companies that use the
data across millions of devices. A recent report from the McKinsey
Global Institute found that, as the NYT put it, "the value [of open
data] to the health care system in the United States could be $300
billion a year, and that American retailers could increase their
operating profit margins by 60 %." Given that U.S. health care costs
billions of dollars a year and makes up 17 % of GDP, companies have more
than enough incentive to create applications and tools that can cut
costs and drive economic activity within this sector.
data  open_government  healthcare  health_informatics  government_2.0  CTO  HHS  pattern_recognition  patterns  open_data  open_source 
june 2011 by jerryking

related tags

5_W’s  actionable_information  agriculture  algorithms  Amazon  analog  analysis  analytics  Andreessen_Horowitz  APIs  archives  behind-the-scenes  Bill_de_Blasio  blogs  books  BuzzData  Canada  Canadian  cannabis  Cathy_O’Neil  CDC  census  challenges  chicken-and-egg  cities  Climate_Corporation  collaboration  Colleges_&_Universities  commercialization  Communicating_&_Connecting  community  community_collaboration  competitive_landscape  complacency  consumers  contradictions  Cornell  crowdsourcing  CTO  cultural_norms  customer_experience  customer_relationships  dark_data  dark_side  data  DataCouch  DataKind  DataMarket  datasets  DataSift  data_driven  data_journalism  data_marketplaces  data_mining  data_scientists  decision_making  definitions  demographic_information  digital_disruption  digital_economy  dilemmas  disruption  distribution_channels  Don_Tapscott  economic_data  elections  Elizabeth_Church  emerging_markets  entrepreneur  entrepreneurship  Eric_Ries  Euclid_Analytics  executive_management  experimentation  Facebook  Factual  failure  farming  fin-tech  finance  financial_innovation  financial_services  fitness  FOIA  food_crops  forecasting  Freebase  gaming_the_system  geographic_ingredient_branding  GoC  Goldman_Sachs  government  government_2.0  gov_2.0  groundbreaking  GTA  hackathons  haystacks  healthcare  health_informatics  HHS  hidden  history  Hollywood  home-center_industry  home-improvement  home_renovations  howto  hyperlocal  IBM  IBM_Watson  ideas  ignorance  infertility  Infochimps  InfoChimps  information  information_gaps  information_overload  information_sources  innovation  inspiration  insurance  intrapreneurship  intuition  iterations  Jawbone  John_Lorinc  journalism  Junar  Kabbage  Kasabi  Kensho  large_companies  lean  legacy_players  legacy_tech  LendUp  linearity  local  mapping  MaRS  massive_data_sets  match-making  mayoral  McKinsey  messiness  Michael_Bloomberg  microtargeting  mindsets  mobile_applications  monetization  Montreal  MOOCs  municipalities  MyData  naivete  navigation  neighbourhoods  nerds  Netflix  networking  new_businesses  New_York_City  NGOs  Nike  nonprofit  no_regrets  NYT  Ontario  OpenGov  open_data  open_government  open_source  organizing_data  Ottawa  Pandora  patterns  pattern_recognition  Paypal  performance_reviews  PhDs  philanthropy  policy  policymaking  political_campaigns  politics  predictive_analytics  preparation  presentations  privacy  privacy_chill  problems  problem_framing  Progressive  public  public_behavior  public_sector  public_works  quality_of_life  Quantifind  real-time  record-keeping  reinventing_the_wheel  reporters  research  Richard_Waters  Roger_Ehrenberg  Routehappy  scaffolding  search  search_engines  Seattle  self-organization  sensors  Seth_Godin  sharing_economy  Silicon_Alley  silo_mentality  Siri  skepticism  Skyscanner  slight_edge  smart_cities  smart_people  social_data  social_physics  Socrata  software_developers  sophisticated  sorting  spreadsheets  start_ups  statistics  Statistics_Canada  Steve_Lohr  storytelling  supply_chains  systematic_approaches  tacit_data  Tavia_Grant  thick_data  thinking_tragically  timelines  Timetric  Tim_O’Reilly  tools  Toronto  transparency  travel  under_the_radar  unfair_advantages  unintended_consequences  unstructured_data  urban  urban_planning  value_chains  value_creation  virality  visualization  volunteering  weather  webinars  websites  white_hats  wisdom  wisdom_of_crowds  World_Bank 

Copy this bookmark: