massive_data_sets   407

« earlier    

The Man Who Solved the Market — how Jim Simons built a moneymaking machine
November 1, 2019 | | Financial Times | Robin Wigglesworth

The Man Who Solved the Market: How Jim Simons Launched the Quant Revolution, by Gregory Zuckerman, Portfolio, RRP$30/£20, 384 pages

Jim Simons looked to math and computers as ways to eliminate the emotional ups and downs of investing. “I don’t want to have to worry about the market every minute. I want models that will make money while I sleep.”
algorithms  books  finance  hedge_funds  James_Simons  massive_data_sets  mathematics  moguls  quantitative  Renaissance_Technologies  talent_spotting  winner-take-all 
4 weeks ago by jerryking
Investment managers need to become coders, says former CPPIB CEO - The Globe and Mail
CLARE O’HARAWEALTH MANAGEMENT REPORTER
DAVID MILSTEADINSTITUTIONAL INVESTMENT REPORTER

Mark Wiseman is learning Python, one of the world’s top computer programming languages.

The former chief executive officer of the Canada Pension Plan Investment Board is not trying to become a master coder, but instead believes investment managers must become proficient in manipulating large data sets to beat the market.

“If you are waiting to get a company’s quarterly or annual report and you think that is how you’re going to make an investment, you are dead meat,”........

“Sources of information are completely different than they were even 10 years ago for investors,” he says.

Today, BlackRock has already begun using “alternative data sources” to gain more in-depth information on companies such as sales predictions, customer traffic and inventory......“As we look at data in industry and how fast it’s moving, there is going to be an increasing bifurcation between proprietary and non-proprietary data."

Non-proprietary data is information that is readily available on the internet and can easily be used by competitors. Now, money managers are increasingly looking for proprietary data to win a competitive advantage.

For BlackRock’s equities business alone, Mr. Wiseman says the firm has tripled the budget for data over the past two years and holds between 400 and 500 proprietary data sets at a time.......learning Python is a more important skill for a young investment manager than learning foreign languages, or even some of the curriculum taught to chartered financial analysts.

“But this is what investing is about today,” he said. “So those of you who are spending your time on your CFA Level III, that is really nice to have the letters after your name on the business card. But you probably would have been better off spending your time learning how to code Python.”
alternative_data  BlackRock  coding  commoditization_of_information  CPPIB  information_sources  investment_management  Mark_Wiseman  massive_data_sets  proprietary  software_developers  software_development 
6 weeks ago by jerryking
The Mystery of the Miserable Employees: How to Win in the Winner-Take-All Economy -
June 15, 2019 | The New York Times | By Neil Irwin.
Neil Irwin is a senior economics correspondent for The Upshot. He is the author of “How to Win in a Winner-Take-All-World,” a guide to navigating a career in the modern economy.......
What Mr. Ostrum and the analytics team did wasn’t a one-time dive into the numbers. It was part of a continuing process, a way of thinking that enabled them to change and adapt along with the business environment. The key is to listen to what data has to say — and develop the openness and interpretive skills to understand what it is telling us.......Neil Irwin was at Microsoft’s headquarters researching a book that aims to answer one simple question: How can a person design a thriving career today? The old advice (show up early, work hard) is no longer enough....In nearly every sector of the economy, people who seek well-paying, professional-track success face the same set of challenges: the rise of a handful of dominant “superstar” firms; a digital reinvention of business models; and a rapidly changing understanding about loyalty in the employer-employee relationship. It’s true in manufacturing and retail, in banking and law, in health care and education — and certainly in tech......superstar companies — and the smaller firms seeking to upend them — are where pragmatic capitalists can best develop their abilities and be well compensated for them over a long and durable career.....the obvious disadvantages of bureaucracy have been outweighed by some not-so-obvious advantages of scale......the ability to collect and analyze vast amounts of data about how people work, and what makes a manager effective (jk: organizing data) .... is essential for even those who aren’t managers of huge organizations, but are just trying to make themselves more valuable players on their own corporate team.......inside Microsoft’s human resources division, a former actuary named Dawn Klinghoffer ....was trying to figure out if the company could use data about its employees — which ones thrived, which ones quit, and the differences between those groups — to operate better......Klinghoffer was frustrated that ....insights came mostly from looking through survey results. She was convinced she could take the analytical approach further. After all, Microsoft was one of the biggest makers of email and calendar software — programs that produce a “digital exhaust” of metadata about how employees use their time. In September 2015, she advised Microsoft on the acquisition of a Seattle start-up, VoloMetrix, that could help it identify and act on the patterns in that vapor......One of VoloMetrix's foundational data sets, for example, was private emails sent by top Enron executives before the company’s 2001 collapse — a rich look at how an organization’s elite behave when they don’t think anyone is watching.
analytics  books  data  datasets  data_driven  exhaust_data  Fitbit  gut_feelings  human_resources  interpretative  Managing_Your_Career  massive_data_sets  meetings  metadata  Microsoft  Moneyball  organizational_analytics  organizing_data  people_analytics  quantitative  quantified_self  superstars  unhappiness  VoloMetrix  winner-take-all  work_life_balance 
june 2019 by jerryking
How 5 Data Dynamos Do Their Jobs
June 12, 2019 | The New York Times | By Lindsey Rogers Cook.
[Times Insider explains who we are and what we do, and delivers behind-the-scenes insights into how our journalism comes together.]
Reporters from across the newsroom describe the many ways in which they increasingly rely on datasets and spreadsheets to create groundbreaking work.

Data journalism is not new. It predates our biggest investigations of the last few decades. It predates computers. Indeed, reporters have used data to hold power to account for centuries, as a data-driven investigation that uncovered overspending by politicians, including then-congressman Abraham Lincoln, attests.

But the vast amount of data available now is new. The federal government’s data repository contains nearly 250,000 public datasets. New York City’s data portal contains more than 2,500. Millions more are collected by companies, tracked by think tanks and academics, and obtained by reporters through Freedom of Information Act requests (though not always without a battle). No matter where they come from, these datasets are largely more organized than ever before and more easily analyzed by our reporters.

(1) Karen Zraick, Express reporter.
NYC's Buildings Department said it was merely responding to a sudden spike in 311 complaints about store signs. But who complains about store signs?....it was hard to get a sense of the scale of the problem just by collecting anecdotes. So I turned to NYC Open Data, a vast trove of information that includes records about 311 complaints. By sorting and calculating the data, we learned that many of the calls were targeting stores in just a few Brooklyn neighborhoods.
(2) John Ismay, At War reporter
He has multiple spreadsheets for almost every article he works on......Spreadsheets helped him organize all the characters involved and the timeline of what happened as the situation went out of control 50 years ago......saves all the relevant location data he later used in Google Earth to analyze the terrain, which allowed him to ask more informed questions.
(3) Eliza Shapiro, education reporter for Metro
After she found out in March that only seven black students won seats at Stuyvesant, New York City’s most elite public high school, she kept coming back to one big question: How did this happen? I had a vague sense that the city’s so-called specialized schools once looked more like the rest of the city school system, which is mostly black and Hispanic.

With my colleague K.K. Rebecca Lai from The Times’s graphics department, I started to dig into a huge spreadsheet that listed the racial breakdown of each of the specialized schools dating to the mid-1970s.
analyzed changes in the city’s immigration patterns to better understand why some immigrant groups were overrepresented at the schools and others were underrepresented. We mapped out where the city’s accelerated academic programs are, and found that mostly black and Hispanic neighborhoods have lost them. And we tracked the rise of the local test preparation industry, which has exploded in part to meet the demand of parents eager to prepare their children for the specialized schools’ entrance exam.

To put a human face to the data points we gathered, I collected yearbooks from black and Hispanic alumni and spent hours on the phone with them, listening to their recollections of the schools in the 1970s through the 1990s. The final result was a data-driven article that combined Rebecca’s remarkable graphics, yearbook photos, and alumni reflections.

(4) Reed Abelson, Health and Science reporter
the most compelling stories take powerful anecdotes about patients and pair them with eye-opening data.....Being comfortable with data and spreadsheets allows me to ask better questions about researchers’ studies. Spreadsheets also provide a way of organizing sources, articles and research, as well as creating a timeline of events. By putting information in a spreadsheet, you can quickly access it, and share it with other reporters.

(5) Maggie Astor, Politics reporter
a political reporter dealing with more than 20 presidential candidates, she uses spreadsheets to track polling, fund-raising, policy positions and so much more. Without them, there’s just no way she could stay on top of such a huge field......The climate reporter Lisa Friedman and she used another spreadsheet to track the candidates’ positions on several climate policies.
311  5_W’s  behind-the-scenes  Communicating_&_Connecting  data  datasets  data_journalism  data_scientists  FOIA  groundbreaking  hidden  information_overload  information_sources  journalism  mapping  massive_data_sets  New_York_City  NYT  open_data  organizing_data  reporters  self-organization  systematic_approaches  spreadsheets  storytelling  timelines  tools 
june 2019 by jerryking
The Art of Statistics by David Spiegelhalter
May 6, 2019 | Financial Times | Review by Alan Smith.

The Art of Statistics, by Sir David Spiegelhalter, former president of the UK’s Royal Statistical Society and current Winton professor of the public understanding of risk at the University of Cambridge.

The comparison with Rosling is easy to make, not least because Spiegelhalter is humorously critical of his own field which, by his reckoning, has spent too much time arguing with itself over “the mechanical application of a bag of statistical tools, many named after eccentric and argumentative statisticians”.

His latest book, its title,
books  book_reviews  charts  Communicating_&_Connecting  data  data_journalism  data_scientists  Hans_Rosling  listening  massive_data_sets  mathematics  statistics  visualization 
may 2019 by jerryking
Spy tactics can spot consumer trends
MARCH 22, 2016 | Financial Times | John Reed.
Israel’s military spies are skilled at sifting through large amounts of information — emails, phone calls, location data — to find the proverbial needle in a haystack: a suspicious event or anomalous pattern that could be the warning of a security threat.....So it is no surprise that many companies ask Israeli start-ups for help in data analysis. The start-ups, often founded by former military intelligence officers, are using the methods of crunching data deployed in spycraft to help commercial clients. These might range from businesses tracking customer behaviour to financial institutions trying to root out online fraud......Mamram is the Israel Defense Forces’ elite computing unit.
analytics  consumer_behavior  cyber_security  data  e-mail  haystacks  hedge_funds  IDF  insights  intelligence_analysts  Israel  Israeli  Mamram  maritime  massive_data_sets  security_&_intelligence  shipping  spycraft  start_ups  tracking  traffic_analysis  trends 
april 2019 by jerryking
Every Company Wants to Become a Tech Company–Even if It Kills Them
March 8, 2019 | WSJ | By John D. Stoll.

Wall Street loves a good reinvention story. The tough part is finding a happy ending.

All the plots seem to go something like this: Every company wants to convince us it’s becoming a tech company–even if it kills them..... an increasing number of companies are at least dabbling in new tech ventures to improve operations......The boom in vendors offering affordable ways to crunch data or utilize cloud computing, for instance, unlocks new strategies for companies across a wide variety of industries........Planet Fitness Inc. is one of the interested companies. The gym boasts 12 million members but CEO Chris Rondeau admits the company knows relatively little about them.

“Besides checking in the front door, we don’t know what members do,”.....The company is spending millions to retool certain treadmills and cardio equipment to better collect data as people exercise, commissioning a new smartphone app, and wants to tie into its customers’ wearable technology....many other CEOs aren’t convinced they have the luxury (of time to take things slowly). Even if it is hard to figure out what to do with all the data gathered and tools employed in the course of regular business, paralysis is not an option. Like a shark, they feel they need to keep swimming or die....... Nokia Corp., the Finnish company, started as a pulp mill in the 19th century and then branched off into various industries, including a successful venture into rubber boot making, ditched its failed mobile handset unit in 2013 to focus on a networks business that was thriving under the radar. Today, it’s locked in a high-stakes race to deploy 5G technology......In 2000, Major League Baseball owners committed $120 million to fund MLB Advanced Media. It aimed to infuse technology into the game and resulted in initiatives like online ticket sales and expanded radio coverage. The gem of that initiative, however, was a streaming television network launched in 2002...... it has attracted outside clients, such as ESPN, the WWE Network, Playstation Vue and HBO. The Walt Disney Co. acquired control recently for nearly $3 billion.... Dunnhumby Ltd., the data and analytics consultancy owned by European grocery chain Tesco PLC. Tesco bought Dunnhumby after it created the chain’s loyalty-card program. Dunnhumby ballooned into a storehouse of information and amassed clients and partners...Searching for the next BAMTech or Dunnhumby is now a religion at many companies......Walmart Inc., which has already heavily invested in e-commerce, wants to take its technology, marry it with everything the world’s largest retailer knows about us and use it to get into the advertising business......“Everyone’s thinking ‘we’ve got a ton of this stuff (data), how do we use it,’” Executives are trying to answer that question by hiring outside firms to analyze trends or setting up in-house units for experimentation.

Walmart is dumping digital-marketing agency Triad, a unit of WPP PLC, and will try its hand at selling advertising space. Armed with a trove of shopper data and connected to a chain of suppliers wanting to place ads in stores and on websites, Walmart hopes to challenge Amazon.com Inc. on this new front......At Ford Motor Co. , CEOJim Hackett envisions a day when automobiles roam streets collecting data from the occupants and the cars’ behavior like rolling smartphones. This is part of that “mobility as a service” vision car makers peddle.......“Corporations tend to reward action over thinking,”“But the truth is…you’ll find the companies that didn’t do the deep thinking and acted quickly have to redo things.
BAMTech  digital_savvy  Dunnhumby  experimentation  Ford  in-house  Jim_Hackett  massive_data_sets  MLB  Planet_Fitness  reinvention  Wal-Mart  mobility_as_a_service  technology  under_the_radar 
march 2019 by jerryking
Big data: legal firms play ‘Moneyball’
February 6, 2019 | Financial Times | Barney Thompson.

Is the hunt for data-driven justice a gimmick or a powerful tool to give lawyers an advantage and predict court outcomes?

In Philip K Dick’s short story The Minority Report, a trio of “precogs” plugged into a machine are used to foretell all crimes so potential felons could be arrested before they were able to strike. In real life, a growing number of legal experts and computer scientists are developing tools they believe will give lawyers an edge in lawsuits and trials. 

Having made an impact in patent cases these legal analytics companies are now expanding into a broad range of areas of commercial law. This is not about replacing judges,” says Daniel Lewis, co-founder of Ravel Law, a San Francisco lawtech company that built the database of judicial behaviour. “It is about showing how they make decisions, what they find persuasive and the patterns of how they rule.” 
analytics  data_driven  judges  law  law_firms  lawtech  lawyers  Lex_Machina  massive_data_sets  Moneyball  predictive_modeling  quantitative  tools 
february 2019 by jerryking
Opinion | Warning! Everything Is Going Deep: ‘The Age of Surveillance Capitalism’
Jan. 29, 2019 | The New York Times | By Thomas L. Friedman, Opinion Columnist.

Recent advances in the speed and scope of digitization, connectivity, big data and artificial intelligence are now taking us “deep” into places and into powers that we’ve never experienced before — and that governments have never had to regulate before. I’m talking about deep learning, deep insights, deep surveillance, deep facial recognition, deep voice recognition, deep automation and deep artificial minds.

Some of these technologies offer unprecedented promise and some unprecedented peril — but they’re all now part of our lives. Everything is going deep........how did we get so deep down where the sharks live?

The short answer: Technology moves up in steps, and each step, each new platform, is usually biased toward a new set of capabilities. Around the year 2000 we took a huge step up that was biased toward connectivity, because of the explosion of fiber-optic cable, wireless and satellites.

Suddenly connectivity became so fast, cheap, easy for you and ubiquitous that it felt like you could touch someone whom you could never touch before and that you could be touched by someone who could never touch you before.

Around 2007, we took another big step up. The iPhone, sensors, digitization, big data, the internet of things, artificial intelligence and cloud computing melded together and created a new platform that was biased toward abstracting complexity at a speed, scope and scale we’d never experienced before.....as big data got really big, as broadband got really fast, as algorithms got really smart, as 5G got actually deployed, artificial intelligence got really intelligent. So now, with no touch — but just a voice command or machines acting autonomously — we can go so much deeper in so many areas....DeepMind, the artificial intelligence arm of Google’s parent, developed an A.I. program, AlphaGo, that has now defeated the world’s top human players of the ancient strategy game Go — which is much more complex than chess — by learning from human play......Today “virtual agents” — using conversational interfaces powered by artificial intelligence — can increasingly understand your intent... just by hearing your voice.....The percentage of calls a chatbot, or virtual agent, is able to handle without turning the caller over to a person is called its “containment rate,” and these rates are steadily soaring. ....But bad guys, who are always early adopters, also see the same potential to go deep in wholly new ways.....On Jan. 20, The London Observer looked at Harvard Business School professor Shoshana Zuboff’s new book, the title of which perfectly describes the deep dark waters we’ve entered: “The Age of Surveillance Capital.”....“Surveillance capitalism,” Zuboff wrote, “unilaterally claims human experience as free raw material for translation into behavioral data. Although some of these data are applied to service improvement, the rest are declared as a proprietary behavioral surplus, fed into advanced manufacturing processes known as ‘machine intelligence,’ and fabricated into prediction products that anticipate what you will do now, soon and later. Finally, these prediction products are traded in a new kind of marketplace that I call behavioral futures markets. Surveillance capitalists have grown immensely wealthy from these trading operations, for many companies are willing to lay bets on our future behavior.”
5G  algorithms  AlphaGo  artificial_intelligence  automation  books  complexity  connectivity  dark_side  DeepMind  digitalization  gaming_the_system  human_experience  massive_data_sets  patterns  rogue_actors  Tom_Friedman  trustworthiness  virtual_agents 
january 2019 by jerryking
Piecing Together Narratives From the 0′s and 1′s: Storytelling in the Age of Big Data - CIO Journal. - WSJ
Feb 16, 2018 | WSJ | By Irving Wladawsky-Berger.

Probabilities are inherently hard to grasp, especially for an individual event like a war or an election, ......Why is it so hard for people to deal with probabilities in everyday life? “I think part of the answer lies with Kahneman’s insight: Human beings need a story,”....Mr. Kahneman explained their research in his 2011 bestseller Thinking, Fast and Slow. Its central thesis is that our mind is composed of two very different systems of thinking. System 1 is the intuitive, fast and emotional part of our mind. Thoughts come automatically and very quickly to System 1, without us doing anything to make them happen. System 2, on the other hand, is the slower, logical, more deliberate part of the mind. It’s where we evaluate and choose between multiple options, because only System 2 can think of multiple things at once and shift its attention between them.

System 1 typically works by developing a coherent story based on the observations and facts at its disposal. Research has shown that the intuitive System 1 is actually more influential in our decisions, choices and judgements than we generally realize. But, while enabling us to act quickly, System 1 is prone to mistakes. It tends to be overconfident, creating the impression that we live in a world that’s more coherent and simpler than the actual real world. It suppresses complexity and information that might contradict its coherent story.

Making sense of probabilities, numbers and graphs requires us to engage System 2, which, for most everyone, takes quite a bit of focus, time and energy. Thus, most people will try to evaluate the information using a System 1 simple story: who will win the election? who will win the football game?.....Storytelling has played a central role in human communications since times immemorial. Over the centuries, the nature of storytelling has significantly evolved with the advent of writing and the emergence of new technologies that enabled stories to be embodied in a variety of media, including books, films, and TV. Everything else being equal, stories are our preferred way of absorbing information.

“It’s not enough to say an event has a 10 percent probability,” wrote Mr. Leonhardt. “People need a story that forces them to visualize the unlikely event – so they don’t round 10 to zero.”.....
in_the_real_world  storytelling  massive_data_sets  probabilities  Irving_Wladawsky-Berger  Communicating_&_Connecting  Daniel_Kahneman  complexity  uncertainty  decision_making  metacognition  data_journalism  sense-making  thinking_deliberatively 
february 2018 by jerryking
Daring rather than data will save advertising
John Hegarty JANUARY 2, 2017

Algorithms are killing creativity, writes John Hegarty

Ultimately, brands are built by talking to a broad audience. Even if part of that audience never buys your product. Remember, a brand is made not just by the people who buy it, but also by the people who know about it. Fame adds value to a brand, but to build it involves saying something that captures the public’s imagination. It needs to broadcast.

Now, data are fundamentally important in the building of a market. “Big data” can provide intelligence, gather information, identify buying patterns and determine certain outcomes. But what it cannot do is create an emotional bond with the consumer. Data do not make magic. That is the job of persuasion. And it is what makes brands valuable...... Steve Jobs or James Dyson did not build brilliant companies by waiting for a set of algorithms to tell them what to do.

Persuasion and promotion.

In today’s advertising world, creativity has taken a back seat. Creativity creates value and with it difference. And difference is vital for giving a brand a competitive edge. But the growing belief in “data-only solutions” means we drive it out of the marketplace.

If everything ends up looking the same and feeling the same, markets stagnate.
advertising  Steve_Jobs  creativity  human_ingenuity  data  massive_data_sets  data_driven  brands  emotional_connections  persuasion  ingenuity  daring  algorithms 
february 2018 by jerryking
Can big data revolutionise policymaking by governments?
January 30, 2018 | FT | Robin Wigglesworth in New York.

Alberto Cavallo is today a professor of applied economics at MIT, where he runs the Billion Prices Project with Roberto Rigobon, another MIT professor. The project started in 2006, during a period when the then-Argentine government was accused of manipulating its inflation data. Professors Cavallo and Rigobon realised that by compiling the online prices listed by Argentine retailers they could build a more accurate and contemporaneous measure of the true inflation rate....The project’s commercial arm, PriceStats, now collects enough data to provide daily inflation updates for 22 economies. “It was kind of an accident. But we quickly realised that it had applications elsewhere,” Prof Cavallo says....Quandl, an alternative data provider

The project is just one example of a broader trend of trawling the swelling sea of big data for clues on how companies, industries or entire economies are performing.
alternative_data_provider  informational_advantages  massive_data_sets  MIT  Quandl  policymaking 
february 2018 by jerryking
When biased data holds a potentially deadly flaw
SEPTEMBER 27, 2017 | FT | Madhumita Murgia.

Researchers at scientific journal Nature said findings from its own investigation on the diversity of these data sets “prompted warnings that a much broader range of populations should be investigated to avoid genomic medicine being of benefit merely to ‘a privileged few’ ”.

This insidious data prejudice made me curious about other unintended biases in the tech world. Several new consumer technologies — often conceived by, built by and tested overwhelmingly on Caucasian males — are flawed due to biases in their design.
massive_data_sets  biases  data  data_driven  unintended_consequences  racial_disparities  algorithms  value_judgements 
january 2018 by jerryking
Algos know more about us than we do about ourselves
NOVEMBER 24, 2017 | Financial Time | John Dizard.

When intelligence collectors and analysts take an interest in you, they usually start not by monitoring the content of your calls or messages, but by looking at the patterns of your communications. Who are you calling, how often and in what sequence? What topics do you comment on in social media?

This is called traffic analysis, and it can give a pretty good notion of what you and the people you know are thinking and what you are preparing to do. Traffic analysis started as a military intelligence methodology, and became systematic around the first world war. Without even knowing the content of encrypted messages, traffic analysts could map out an enemy “order of battle” or disposition of forces, and make inferences about commanders’ intentions.

Traffic analysis techniques can also cut through the petabytes of redundant babble and chatter in the financial and political worlds. Even with state secrecy and the forests of non-disclosure agreements around “proprietary” investment or trading algorithms, crowds can be remarkably revealing in their open-source posts on social media.

Predata, a three-year-old New York and Washington-based predictive data analytics provider, has a Princeton-intensive crew of engineers and international affairs graduates working on early “signals” of market and political events. Predata trawls the open metadata for users of Twitter, Wikipedia, YouTube, Reddit and other social media, and analyses it to find indicators of future price moves or official actions.

I have been following their signals for a while and find them to be useful indicators. Predata started by creating political risk indicators, such as Iran-Saudi antagonism, Italian or Chilean labour unrest, or the relative enthusiasm for French political parties. Since the beginning of this year, they have been developing signals for financial and commodities markets.

The 1-9-90 rule
1 per cent of internet users initiate discussions or content, 9 per cent transmit content or participate occasionally and 90 per cent are consumers or ‘lurkers’

Using the example of the company’s BoJ signal. For this, Predata collects the metadata from 300 sources, such as Twitter users, contested Wikipedia edits or YouTube items created by Japanese monetary policy geeks. Of those, at any time perhaps 100 are important, and 8 to 10 turn out to be predictive....This is where you need some domain knowledge [domain expertise = industry expertise]. It turns out that Twitter is pretty important for monetary policy, along with the Japanese-language Wiki page for the Bank of Japan, or, say, a YouTube video of [BoJ governor] Haruhiko Kuroda’s cross-examination before a Diet parliamentary committee.

“Then you build a network of candidate discussions [JK: training beds] and look for the pattern those took before historical moves. The machine-learning algorithm goes back and picks the leads and lags between traffic and monetary policy events.” [Jk: Large data sets with known correct answers serve as a training bed and then new data serves as a test bed]

Typically, Predata’s algos seem to be able to signal changes in policy or big price moves [jk: inflection points] somewhere between 2 days and 2 weeks in advance. Unlike some academic Twitter scholars, Predata does not do systematic sentiment analysis of tweets or Wikipedia edits. “We only look for how many people there are in the conversation and comments, and how many people disagreed with each other. We call the latter the coefficient of contestation,” Mr Shinn says.

The lead time for Twitter, Wiki or other social media signals varies from one market to another. Foreign exchange markets typically move within days, bond yields within a few days to a week, and commodities prices within a week to two weeks. “If nothing happens within 30 days,” says Mr Lee, “then we say we are wrong.”
algorithms  alternative_data  Bank_of_Japan  commodities  economics  economic_data  financial_markets  industry_expertise  inflection_points  intelligence_analysts  lead_time  machine_learning  massive_data_sets  metadata  non-traditional  Predata  predictive_analytics  political_risk  signals  social_media  spycraft  traffic_analysis  training_beds  Twitter  unconventional 
november 2017 by jerryking
Novartis’s new chief sets sights on ‘productivity revolution’
SEPTEMBER 25, 2017 | Financial Times | Sarah Neville and Ralph Atkins.

The incoming chief executive of Novartis, Vas Narasimhan, has vowed to slash drug development costs, eyeing savings of up to 25 per cent on multibillion-dollar clinical trials as part of a “productivity revolution” at the Swiss drugmaker.

The time and cost of taking a medicine from discovery to market has long been seen as the biggest drag on the pharmaceutical industry’s performance, with the process typically taking up to 14 years and costing at least $2.5bn.

In his first interview as CEO-designate, Dr Narasimhan says analysts have estimated between 10 and 25 per cent could be cut from the cost of trials if digital technology were used to carry them out more efficiently. The company has 200 drug development projects under way and is running 500 trials, so “that will have a big effect if we can do it at scale”.......Dr Narasimhan plans to partner with, or acquire, artificial intelligence and data analytics companies, to supplement Novartis’s strong but “scattered” data science capability.....“I really think of our future as a medicines and data science company, centred on innovation and access.”

He must now decide where Novartis has the capability “to really create unique value . . . and where is the adjacency too far?”.....Does he need the cash pile that would be generated by selling off these parts of the business to realise his big data vision? He says: “Right now, on data science, I feel like it’s much more about building a culture and a talent base . . . ...Novartis has “a huge database of prior clinical trials and we know exactly where we have been successful in terms of centres around the world recruiting certain types of patients, and we’re able to now use advanced analytics to help us better predict where to go . . . to find specific types of patients.

“We’re finding that we’re able to significantly reduce the amount of time that it takes to execute a clinical trial and that’s huge . . . You could take huge cost out.”...Dr Narasimhan cites one inspiration as a visit to Disney World with his young children where he saw how efficiently people were moved around the park, constantly monitored by “an army of [Massachusetts Institute of Technology-]trained data scientists”.
He has now harnessed similar technology to overhaul the way Novartis conducts its global drug trials. His clinical operations teams no longer rely on Excel spreadsheets and PowerPoint slides, but instead “bring up a screen that has a predictive algorithm that in real time is recalculating what is the likelihood our trials enrol, what is the quality of our clinical trials”.

“For our industry I think this is pretty far ahead,” he adds.

More broadly, he is realistic about the likely attrition rate. “We will fail at many of these experiments, but if we hit on a couple of big ones that are transformative, I think you can see a step change in productivity.”
adjacencies  algorithms  analytics  artificial_intelligence  attrition_rates  CEOs  data_driven  data_scientists  drug_development  failure  Indian-Americans  kill_rates  massive_data_sets  multiple_targets  Novartis  pharmaceutical_industry  predictive_analytics  productivity  productivity_payoffs  product_development  real-time  scaling  spreadsheets  Vas_Narasimhan 
november 2017 by jerryking

« earlier    

related tags

311  5_w’s  5g  a&r  academia  adjacencies  advertising  affirm  aftermath  agribusiness  agriculture  aimia  airbnb  airline_industry  airports  albert_einstein  algorithms  alpha  alphago  alternative_data  alternative_data_provider  alumni  amazon  analog  analysis  analytics  anecdotal  anita_elberse  anticipating  apis  apparel  apple  appliances  arcane_knowledge  art  artificial_intelligence  artists  assessments_&_evaluations  associations  attrition_rates  automated_reasoning  automation  bae  bamtech  bank_of_japan  banking  banks  bay_street  behavioural_targeting  behind-the-scenes  biases  biometrics  blackrock  blockbusters  book_reviews  books  boom-to-bust  boyd_erman  brands  brian_scudamore  business  business_models  campaign_2012  campaign_2016  career_paths  cdc  ceos  charts  chicago_school  china  chinese  chocolate  churn  cia  cios  cisco  cities  clarity  clothing  clothing_labels  cloud_computing  codebreaking  coding  colleges_&_universities  commodities  commoditization_of_information  communicating_&_connecting  communicating_risks  community_collaboration  complexity  conferences  connectivity  consistency  consumer_behavior  consumer_finance  consumers  contextual  control_systems  cord-cutting  core_competencies  corporate_investors  correlations  cppib  creating_demand  creative_class  creative_economy  creative_types  creativity  credit_scoring  cross-disciplinary  crossborder  cryptography  customer_experience  customer_insights  cyber_security  cyber_warfare  cyberattacks  cyberphysical  cyberthreats  daniel_kahneman  daring  dark_data  dark_side  dashboards  data  data_breaches  data_collection  data_driven  data_journalism  data_marketplaces  data_mining  data_quality  data_scientists  databases  dataminr  datasets  david_sanger  decision_making  deep_learning  deepmind  defensive_tactics  delighting_customers  design  digital_economy  digital_media  digital_savvy  digitalization  disruption  distributed_data  distrust  diversity  donald_trump  drug_development  dunnhumby  e-mail  earnest  easy-to-measure  economic_data  economics  economies_of_scale  economists  edward_snowden  efficiencies  emergencies  emotional_connections  emotional_intelligence  empiricism  encryption  engineering  enlitic  entertainment_industry  eq  espionage  event-driven  events  exhaust_data  experiential_marketing  experimentation  expertise  extreme_weather_events  facebook  failure  false_confidence  farming  fashion  feedback  fin-tech  finance  financial_markets  financial_services  financiers  fitbit  fitness  five_eyes  floods  foia  food  food_genius  foodservice  foot_traffic  ford  forecasting  forward-thinking  forward_looking  fresh_produce  future  futurists  gaming_the_system  gchq  ge  ge_capital  genomics  geolocation  gilder's_law  google  got_junk?  gps  granularity  graze  greg_ip  groundbreaking  growth  gut_feelings  hacks  hadoop  haier  hal_varian  hans_rosling  harvard  haystacks  hedge_funds  heterogeneity  hidden  high-energy  hiring-a-product-to-do-a-specific-job  history  hospitality  howto  huawei  human_experience  human_factor  human_ingenuity  human_resources  humint  hustle  hyperlocal  ibm  ibm_watson  ideas  idf  in-house  in-store  in_the_real_world  indian-americans  industrial_internet  industry_expertise  inexpensive  infertility  inflection_points  infographics  infoliteracy  information  information_gaps  information_overload  information_sources  information_vacuum  informational_advantages  informed_consent  infrastructure  ingenuity  innovation  insights  inspiration  instrumentation_monitoring  insurance  intelligence_analysts  interactivity  interpretation  interpretative  interviews  intuition  investment_management  investment_research  investors  ipos  irving_wladawsky-berger  israel  israeli  ivy_league  james_simons  jawbone  jetblue  jevons_paradox  jim_hackett  jim_manzi  john_lorinc  john_tory  jonathan_knee  journalism  judges  judgment  jumpmath  kaggle  keynesian  kill_rates  konrad_yakabuski  kroger  l._gordon_crovtiz  large_companies  latent  law  law_firms  lawtech  lawyers  layer_mastery  lbma  lead_time  leaders  lex_machina  lgbt  liberal_arts  linearity  listening  loblaws  location_based_services  lou_eccleston  loyalty_management  machine_learning  mamram  managing_your_career  mapping  maritime  mark_wiseman  marketing  mass_media  mathematics  max_levchin  mckinsey  measurements  meetings  metacognition  metadata  metrics  michael_bloomberg  microeconomics  microsoft  microtargeting  middle_class  middlemen  missteps  mit  mlb  mobile  mobile_applications  mobile_phones  mobility_as_a_service  moguls  monetization  moneyball  movingonup  multiple_targets  municipalities  museums  music  music_industry  nate_silver  natural_calamities  natural_selection  neighbourhoods  nestlé  netflix  network_effects  network_risk  new_york_city  niches  nielsen  nike  noise  non-traditional  novartis  nps  nsa  nyt  octothorpe_software  offensive_tactics  okcupid  open_data  operational_tempo  organizational_analytics  organizing_data  overlooked  overlooked_opportunities  overquantification  overreaction  p&g  pandora  panics  paradoxes  partnerships  patriot_act  pattern_recognition  patterns  pbs  people_analytics  perishables  personalization  persuasion  pharmaceutical_industry  phds  philanthropy  physical_world  physicists  physics  planet_fitness  platforms  poaching  policymaking  political_campaigns  political_risk  politicians  politics  predata  predictions  predictive_analytics  predictive_modeling  premise  preparation  privacy  probabilities  product_development  productivity  productivity_payoffs  proprietary  public_behavior  public_relations  quality_of_life  quandl  quantified_self  quantitative  quants  quantum_computing  quentin_hardy  questions  racial_disparities  rankings  real-time  recruiting  reinvention  renaissance_technologies  reporters  retailers  retention  risks  rogue_actors  routehappy  sales  sales_per_square_foot  satellites  scaling  security_&_intelligence  selection_processes  self-organization  self-protection  sense-making  sensors  seth_godin  sharing_economy  shifting_tastes  shipping  sigint  signals  silicon_valley  singapore  skyscanner  slight_edge  small_business  small_data  smart_cities  smartphones  social_data  social_impact  social_media  social_physics  soft_skills  softbank  software  software_developers  software_development  sorting  special_sauce  specialization  speed  sports  spreadsheets  spycraft  spymasters  stanford  start_ups  statistics  steam_engine  steve_jobs  steve_lohr  storytelling  strategy  sublinearity  superstars  supply_chains  surprises  susan_krashinsky  sustainability  systematic_approaches  tacit_data  talent  talent_management  talent_scouting  talent_spotting  teams  technological_change  technology  test_beds  testing  the_long_tail  the_right_people  thick_data  think_differently  thinking_deliberatively  timelines  tmx  tom_friedman  tools  toronto  tracking  traffic_analysis  traffic_congestion  training_beds  transportation  travel  trends  trucking  trustworthiness  twitter  uchicago  uncertainty  unconventional  under_the_radar  underestimation  unevenly_distributed  unhappiness  unintended_consequences  united_kingdom  unknowns  unstructured_data  ups  urban  urban_planning  uwindsor  valuations  value_creation  value_judgements  vas_narasimhan  venture_capital  vetting  virtual_agents  virtual_reality  visualization  vivendi  volometrix  wal-mart  wall_street  war_for_talent  warfare  warning_signs  weak_links  wealth_creation  wealth_destruction  weather  whirlpool  white_hats  william_jevons  winner-take-all  winton_capital  wisdom_of_crowds  work_life_balance  worldquant  zestfinance 

Copy this bookmark:



description:


tags: