jerryking + algorithms   96

Opinion | How Artificial Intelligence Can Save Your Life
June 24, 2019 | The New York Times | By David Brooks.
Opinion Columnist

In his book “Deep Medicine,” which is about how A.I. is changing medicine across all fields, Eric Topol describes a study in which a learning algorithm was given medical records to predict who was likely to attempt suicide. It accurately predicted attempts nearly 80 percent of the time. By incorporating data of real-world interactions such as laughter and anger, an algorithm in a similar study was able to reach 93 percent accuracy.....
algorithms  artificial_intelligence  books  David_Brooks  depression  diagnostic  doctors  medical  mens'_health  mental_health  op-ed  pattern_recognition  predictive_analytics  tools  visual_cues 
11 weeks ago by jerryking
How Spotify’s algorithms are ruining music
May 2, 2019 | Financial Times | Michael Hann.

(1) FINAL DAYS OF EMI, By Eamonn Forde, Omnibus, RRP£20, 320 pages
(2) SPOTIFY TEARDOWN, By Maria Eriksson, Rasmus Fleischer, Anna Johansson, Pelle Snickars and Patrick Vonderau, The MIT Press, RRP£14.99, 288 pages
(3) WAYS OF HEARING, By Damon Krukowski, The MIT Press, RRP£14.99, 136 pages

In April, the IFPI — the global body of the recording industry — released its latest annual Global Music Report. For the fourth consecutive year, revenues were up, to a total of $19.1bn, from a low of $14.3bn in 2014. Nearly half those revenues came from music streaming, driven by a 33 per cent rise in paid subscriptions to services such as Spotify, Apple Music and Tidal...... It is worth remembering that 20 years ago, the IFPI reported global music revenues of $38.6bn. Today’s “booming” recording industry is less than half the size it was at the turn of the century.....The nadir for the recording industry coincided with the first shoots of its regrowth. ....In August 2007, the British record company EMI — the fourth of the majors, alongside Universal, Sony and Warner — was bought by private equity firm Terra Firma (Guy Hands, the fund’s founder and chairman) for $4.7bn; a year later, a Swedish company called Spotify took its music streaming service public. The former was, perhaps, the last gasp of the old way of doing things — less than four years after buying EMI, Terra Firma was unable to meet its debts, and ceded control of the company to its main lender, Citigroup. Before 2011 was out, the process of breaking up EMI had begun...EMI’s demise was foreshadowed before Hands arrived, with a blaze of hubris in the early 2000s. Forde, a longtime observer and chronicler of the music business recounts the “disastrous and expensive” signings of that era......Handspreached the need to use data when signing artists, not just the “golden ears” of talent scouts; data are now a key part of the talent-spotting process.

* to qualify as having been listened to on Spotify, a song has to have been played for 30 seconds.
* hit songs have become increasingly predictable, offering up all their pleasures in the opening half-minute. Their makers dare not risk scaring off listeners.
* for all the money that the streaming services have generated for the music industry, very little of it flows back to any musicians except the select few who dominate the streaming statistics,

.......On Spotify, music consumption has been reorganised around “behaviours, feelings and moods” channelled through curated playlists and motivational messages......The data Spotify collects enable the industry to work out who its market is, where it lives, what else they like, how often they listen to music — almost anything, really. It’s the greatest assemblage of information about music listeners in history, and it has profoundly altered the industry: it has made Spotify music’s kingmaker......when an artist travels abroad to promote a new album, the meeting with the local Spotify office is more important than the TV appearances or the newspaper interviews. ...Spotify enables artists to plan their band’s set lists so they can play the most popular song in any given city.............So what? What does it matter if one model of music distribution has been replaced by another.....It matters because Spotify has profoundly changed the listener’s relationship with music....Older musicians often wax about how, when you had to buy your own music as a kid, you listened to it until you liked it, because you wouldn’t be able to afford a new album for another month. Now you simply skip to the next one, and probably don’t give it your full attention. Without ownership, there’s no incentive to study...........Faced with the impossibly wide choice of Spotify, it becomes easier to return to old favourites — easier than when flicking through your vinyl or CDs, because the act of looking through your own music makes things you had not thought of in years leap out at you. Spotify actually makes people into more conservative listeners, a process aided by its algorithms, which steer you towards music similar to your most frequent listening.....The theme of Krukowski’s book is that the changes in the way the music industry works have been about controlling and eliminating excess noise. That’s in a literal sense and in a metaphorical one, too. Streaming has stripped music of context, pared it back to being just about the song and the moment....but noise is the context of life. Without noise, the signal becomes meaningless......The world of the old EMI was one of both signal and noise; where myths and legends could be created: The Beatles! Queen! The Beach Boys! Pink Floyd! It was never all about the signal. The world of Spotify is one of signal only, and if you don’t appreciate that signal within the first 30 seconds of the song...all may be lost
abundance  algorithms  Apple_Music  books  book_reviews  business_models  curation  cultural_transmission  data  decontextualization  EMI  gatekeepers  Guy_Hands  hits  indoctrination  iTunes  legacy_artists  music  music_catalogues  music_labels  music_industry  music_publishing  noise  piracy  platforms  playlists  royalties  ruination  securitization  signals  songs  Spotify  streaming  subscriptions  talent  talent_scouting  talent_spotting  Terra_Firma  Tidal  transformational 
may 2019 by jerryking
US declining interest in history presents risk to democracy
May 2, 2019 | Financial Times | by Edward Luce.

America today has found a less bloodthirsty way of erasing its memory by losing interest in its past. From an already low base, the number of American students majoring in history has dropped by more than a third since 2008. Barely one in two hundred American undergraduates now specialise in history......Donald Trump is a fitting leader for such times. He had to be told who Andrew Jackson was.....He also seems to think that Frederick Douglass, the escaped slave and 19th century abolitionist, is among us still.....But America’s 45th president can hardly be blamed for history’s unpopularity. Culpability for that precedes Mr Trump and is spread evenly between liberals, conservatives, faculty and parents........Courses on intellectual, diplomatic and political history are being replaced at some of America’s best universities by culture studies that highlight grievances at the expense of breadth.......Then there is the drumbeat of STEM — science, technology, engineering and mathematics. Most US states now mandate tests only in maths and English, at the expense of history and civic education...... In a recent survey, only 26 per cent of Americans could identify all three branches of government. More than half could not name a single justice on the US Supreme Court.....
the biggest culprit is the widespread belief that “soft skills” — such as philosophy and English, which are both in similar decline to history — do not lead to well-paid jobs.....folk prejudice against history is hard to shake. In an ever more algorithmic world, people believe that humanities are irrelevant. The spread of automation should put a greater premium on qualities that computers lack, such as intuitive intelligence, management skills and critical reasoning. Properly taught that is what a humanities education provides.......People ought to be able to grasp the basic features of their democracy. [Abiding] Faith in a historic theory only fuels a false sense of certainty....What may work for individual careers poses a collective risk to US democracy. The demise of strong civics coincides with waning voter turnout, a decline in joining associations, fewer citizen’s initiatives — and other qualities once associated with American vigour......There is no scientific metric for gullibility. Nor can we quantitatively prove that civic ignorance imposes a political cost on society. These are questions of judgment. But if America’s origins tell us anything it is that a well-informed citizenry creates a stronger society.
=============================================
here is what robots can't do -- create art, deep meaning, move our souls, help us to understand and thus operate in the world, inspire deeper thought, care for one another, help the environment where we live.......The role of the human is not to be dispassionate, depersonalized or neutral. It is precisely the emotive traits that are rewarded: the voracious lust for understanding, the enthusiasm for work, the ability to grasp the gist, the empathetic sensitivity to what will attract attention and linger in the mind. Unable to compete when it comes to calculation, the best workers will come with heart in hand.
========================================================================
algorithms  automation  citizen_engagement  civics  Colleges_&_Universities  critical_thinking  democracy  Donald_Trump  Edward_Luce  empathy  engaged_citizenry  foundational  historians  history  historical_amnesia  humanities  ignorance  political_literacy  sense-making  soft_skills  STEM  threats  U.S.  vulnerabilities 
may 2019 by jerryking
Always seek out novelty — even at home
April 26, 2019 | Financial Times | by Tim Harford.

* The search for new experiences should not just be for our holidays.
* Japan: 10 days in a far-off land produces a richer treasury of detailed memories than 10 weeks back home. But why?
* Actively searching for new experiences --whether on holiday abroad or within your daily routine at home!!
* Novelty isn't just about mental stimulation. It also exposes you to opportunity.....Variation also reshapes the mental categorisation of experiences, so that freshness can be found within routine activities.
+++++++++++++++++++++++++++++++++++++++++++++++++++
While on an adventurous holiday, many people experience that strange sense of time having slowed down in the most pleasurable way, and of conversations that begin, “Was it really only yesterday that we . . . ?”

Ten days in a far-off land produces a richer treasury of detailed memories than 10 weeks back home. But what is behind this phenomenon?

Claude Shannon,in 1948, published one of his two profound contributions, A Mathematical Theory of Communication,.....a message can be compressed to the extent that it is predictable. ....(e.g. Ritualised conversations (“How are you?” “Very well, thank you. How are you?”) can be heavily compressed.....A movie can be compressed because, between cuts, each frame tends to resemble the previous one....Although the parallel is not exact, much the same thing seems to be going on with our memories of life. The brain is not a video recorder; we recall the gist. Sometimes the gist is very brief. If I get up in the morning at the usual time, eat my customary breakfast and catch my usual train to the office, why should my brain trouble itself to remember this day two weeks after the fact? The diffs are barely worth bothering with. In contrast, fresh experiences defy compression: the diffs are too big........Brian Christian, author of The Most Human Human, a book about conversations between humans and computers, speculates that if we’re seeking advice we should ask the person of whose answer we are least certain. If we want to understand a person, we should ask them the question to which we are least sure of their answer.
algorithms  compression  creativity  creative_renewal  economists  experience_economy  fresh_eyes  habits  holidays  insta-bae  Japan  mybestlife  novelty  non-routine  Slow_Movement  Tim_Harford  travel  unpredictability  vacations 
april 2019 by jerryking
What You Need to Know to Pick an IPO
April 7, 2019 | WSJ | By Andy Kessler.
Dig up dirt on the competition and board members, and buy to hold long-term.......How do you know which IPOs to buy? No, not to trade—you’d never get it right. Lyft priced at $72, traded at $85 on its first day, then closed at $78, only to fall to $67 on its second day. It’s now $74. I’m talking about buying and holding for a few years. Yes I know, how quaint.

The trick is to read the prospectus. What are you, crazy? That’s a couple hundred pages. Well, not the whole thing. But remember, where the stock trades on its first day is noise....... So understanding long-term prospects are critical. Here are a few shortcuts.

(1) First, glance at the underwriters along the bottom of the cover. On the top line are the banks putting their reputation on the line. If the one on the far left is Goldman Sachs , Morgan Stanley or JPMorgan , you’re probably OK.
(2) open the management section and study the directors. Forget the venture capitalists or strategic partners with board seats—they have their own agendas. Non-employee directors are the ones who are supposed to be representing you, the public investor. And their value depends on their experience.
(3) OK, now figure out what the company does. You can watch the roadshow video, look at prospectus pictures, and skim the offering’s Business section. Now ignore most of that. Underwriters are often terrible at positioning companies to the market.......when positioning companies, only three things matter: a monster market; an unfair competitive advantage like patents, algorithms or a network effect; and a business model to leverage that advantage. Look for those. If you can’t find them, pass. Commodities crumble........read the Management’s Discussion and Analysis. Companies are forced to give detailed descriptions of each of their sectors and products or services. Then flip back and forth to the Financials, looking at the items on the income statement and matching them up with the operations being discussed. Figure out what the company might look like in five years. And use my “10x” rule: Lyft is worth $25 billion—can they make $2.5 billion after-tax someday? Finally there’s the Risk section, which is mostly boilerplate but can contain good dirt on competition.
(4) Put the prospectus away and save it as a souvenir. Try to figure out the real story of the company. Do some digging.
(5) My final advice: Never, ever put in a market order for shares on the first day of an IPO.
10x  advice  algorithms  Andy_Kessler  boards_&_directors_&_governance  business_models  competitive_advantage  deception  due_diligence  howto  IPOs  large_markets  long-term  Lyft  network_effects  noise  patents  positioning  prospectuses  risks  stock_picking  think_threes  Uber  underwriting  unfair_advantages 
april 2019 by jerryking
DE Shaw: inside Manhattan’s ‘Silicon Valley’ hedge fund
March 25, 2019 | Financial Times Robin Wigglesworth in New York.

for a wider investment industry desperately trying to reinvent itself for the 21st century, DE Shaw has evolved dramatically from the algorithmic, computer-driven “quantitative” trading it helped pioneer in the 1980s.

It is now a leader in combining quantitative investing with traditional “fundamental” strategies driven by humans, such as stockpicking. This symbiosis has been dubbed “quantamental” by asset managers now attempting to do the same. Many in the industry believe this is the future, and are rushing to hire computer scientists to help realise the benefits of big data and artificial intelligence in their strategies........DE Shaw runs some quant strategies so complex or quick that they are in practice almost beyond human understanding — something that many quantitative analysts are reluctant to concede.

The goal is to find patterns on the fuzzy edge of observability in financial markets, so faint that they haven’t already been exploited by other quants. They then hoard as many of these signals as possible and systematically mine them until they run dry — and repeat the process. These can range from tiny, fleeting arbitrage opportunities between closely-linked stocks that only machines can detect, to using new alternative data sets such as satellite imagery and mobile phone data to get a better understanding of a company’s results...... DE Shaw is also ramping up its investment in the bleeding edge of computer science, setting up a machine learning research group led by Pedro Domingos, a professor of computer science and engineering and author of The Master Algorithm, and investing in a quantum computing start-up.

It is early days, but Cedo Crnkovic, a managing director at DE Shaw, says a fully-functioning quantum computer could potentially prove revolutionary. “Computing power drives everything, and sets a limit to what we can do, so exponentially more computing power would be transformative,” he says.
algorithms  alternative_data  artificial_intelligence  books  D.E._Shaw  financial_markets  hedge_funds  investment_management  Manhattan  New_York_City  quantitative  quantum_computing  systematic_approaches 
march 2019 by jerryking
The robot-proof skills that give women an edge in the age of AI
February 11, 2019 | Financial Times |by Sarah O’Connor.

in a world of algorithms and artificial intelligence, communication skills and emotional intelligence — traditionally seen as female strengths — could prove key.

The latest panic about artificial intelligence is that it will deal a blow to women in the workplace..... The concerns are legitimate enough, but they fail to appreciate the big ways in which the world of work is going to change. In fact, it is quite possible the age of AI will belong to women. Men are the ones in danger of being left behind....Some AI tools may be biased against women — a risk for any group that has been historically under-represented in the workplace. Because machine learning tends to learn from historical data, it can perpetuate patterns from the past into the future......It is right to pay attention to these problems and work on solutions. Algorithms shouldn’t be given power without transparency, accountability, and human checks and balances. Top AI jobs should be held by a more diverse set of smart people.....As machines become better at many cognitive tasks, it is likely that the skills they are relatively bad at will become more valuable. This list includes creative problem-solving, empathy, negotiation and persuasion. As Andy Haldane, chief economist at the Bank of England, has put it, “the high-skill, high-pay jobs of the future may involve skills better measured by EQs (a measure of emotional intelligence) than IQs”..... increasing demand in these jobs for supplementary skills such as emotional intelligence, which has given women an edge.....as the AI era dawns, it is the right moment to overhaul the way we value these skills, and the way we teach them. With an eye on the demands of the future, we are trying to persuade girls that coding is not just for boys. So why aren’t we also trying to persuade boys that empathy is not just for girls?

We could start by changing the language we use. For too long we have talked about “soft skills”, with connotations of femininity and a lack of rigour. Let’s call them what they are: “robot-proof skills” that neither men nor women can afford to face the 21st century
21st._century  algorithms  artificial_intelligence  biases  checks_and_balances  dark_side  emotional_intelligence  EQ  future-proofing  gender_gap  machine_learning  soft_skills  smart_people  under-representation  women  workplaces  pay_attention 
february 2019 by jerryking
Opinion | Warning! Everything Is Going Deep: ‘The Age of Surveillance Capitalism’
Jan. 29, 2019 | The New York Times | By Thomas L. Friedman, Opinion Columnist.

Recent advances in the speed and scope of digitization, connectivity, big data and artificial intelligence are now taking us “deep” into places and into powers that we’ve never experienced before — and that governments have never had to regulate before. I’m talking about deep learning, deep insights, deep surveillance, deep facial recognition, deep voice recognition, deep automation and deep artificial minds.

Some of these technologies offer unprecedented promise and some unprecedented peril — but they’re all now part of our lives. Everything is going deep........how did we get so deep down where the sharks live?

The short answer: Technology moves up in steps, and each step, each new platform, is usually biased toward a new set of capabilities. Around the year 2000 we took a huge step up that was biased toward connectivity, because of the explosion of fiber-optic cable, wireless and satellites.

Suddenly connectivity became so fast, cheap, easy for you and ubiquitous that it felt like you could touch someone whom you could never touch before and that you could be touched by someone who could never touch you before.

Around 2007, we took another big step up. The iPhone, sensors, digitization, big data, the internet of things, artificial intelligence and cloud computing melded together and created a new platform that was biased toward abstracting complexity at a speed, scope and scale we’d never experienced before.....as big data got really big, as broadband got really fast, as algorithms got really smart, as 5G got actually deployed, artificial intelligence got really intelligent. So now, with no touch — but just a voice command or machines acting autonomously — we can go so much deeper in so many areas....DeepMind, the artificial intelligence arm of Google’s parent, developed an A.I. program, AlphaGo, that has now defeated the world’s top human players of the ancient strategy game Go — which is much more complex than chess — by learning from human play......Today “virtual agents” — using conversational interfaces powered by artificial intelligence — can increasingly understand your intent... just by hearing your voice.....The percentage of calls a chatbot, or virtual agent, is able to handle without turning the caller over to a person is called its “containment rate,” and these rates are steadily soaring. ....But bad guys, who are always early adopters, also see the same potential to go deep in wholly new ways.....On Jan. 20, The London Observer looked at Harvard Business School professor Shoshana Zuboff’s new book, the title of which perfectly describes the deep dark waters we’ve entered: “The Age of Surveillance Capital.”....“Surveillance capitalism,” Zuboff wrote, “unilaterally claims human experience as free raw material for translation into behavioral data. Although some of these data are applied to service improvement, the rest are declared as a proprietary behavioral surplus, fed into advanced manufacturing processes known as ‘machine intelligence,’ and fabricated into prediction products that anticipate what you will do now, soon and later. Finally, these prediction products are traded in a new kind of marketplace that I call behavioral futures markets. Surveillance capitalists have grown immensely wealthy from these trading operations, for many companies are willing to lay bets on our future behavior.”
5G  algorithms  AlphaGo  artificial_intelligence  automation  books  complexity  connectivity  dark_side  DeepMind  digitalization  gaming_the_system  human_experience  massive_data_sets  patterns  rogue_actors  Tom_Friedman  trustworthiness  virtual_agents 
january 2019 by jerryking
Amazon offers cautionary tale of AI-assisted hiring
January 23, 2019 | Financial Times | by Andrew Hill.

the task of working out how to get the right people on the bus has got harder since 2001 when Jim Collins first framed it, as it has become clearer — and more research has underlined — that diverse teams are better at innovation. For good reasons of equity and fairness, the quest for greater balance in business has focused on gender, race and background. But these are merely proxies for a more useful measure of difference that is much harder to assess, let alone hire for: cognitive diversity. Might this knotty problem be solved with the help of AI and machine learning? Ming is sceptical. As she points out, most problems with technology are not technology problems, but human problems. Since humans inevitably inherit cultural biases, it is impossible to build an “unbiased AI” for hiring. “You simply have to recognise that the biases exist and put in the effort to do more than those default systems point you towards,” she says...........What Amazon’s experience suggests is that instead of sending bots to crawl over candidates’ past achievements, companies should be exploring ways in which computers can help them to assess and develop the long term potential of the people they invite to board the bus. Recruiters should ask, in Ming’s words, “Who will [these prospective candidates] be three years from now when they’re at their peak productivity inside the company? And that might be a very different story than who will deliver peak productivity the moment they walk in the door.”
heterogeneity  Amazon  artificial_intelligence  hiring  Jim_Collins  machine_learning  recruiting  teams  Vivienne_Ming  cautionary_tales  biases  diversity  intellectual_diversity  algorithms  questions  the_right_people 
january 2019 by jerryking
Computer vision: how Israel’s secret soldiers drive its tech success
November 20, 2018 | Financial Times | Mehul Srivastava in Tel Aviv.
.... those experiences that have helped such a tiny country become a leader in one of the most promising frontiers in the technology world: computer vision. Despite the unwieldy name it is an area that has come of age in the past few years, covering applications across dozens of industries that have one thing in common: the need for computers to figure out what their cameras are seeing, and for those computers to tell them what to do next.........Computer vision has become the connecting thread between some of Israel’s most valuable and promising tech companies. And unlike Israel’s traditional strengths— cyber security and mapping — computer vision slides into a broad range of different civilian industries, spawning companies in agriculture, medicine, sports, self-driving cars, the diamond industry and even shopping. 

In Israel, this lucrative field has benefited from a large pool of engineers and entrepreneurs trained for that very task in an elite, little-known group in the military — Unit 9900 — where they fine-tuned computer algorithms to digest millions of surveillance photos and sift out actionable intelligence. .........The full name for Unit 9900 — the Terrain Analysis, Accurate Mapping, Visual Collection and Interpretation Agency — hints at how it has created a critical mass of engineers indispensable for the future of this industry. The secretive unit has only recently allowed limited discussion of its work. But with an estimated 25,000 graduates, it has created a deep pool of talent that the tech sector has snapped up. 

Soldiers in Unit 9900 are assigned to strip out nuggets of intelligence from the images provided by Israel’s drones and satellites — from surveilling the crowded, chaotic streets of the Gaza Strip to the unending swaths of desert in Syria and the Sinai. 

With so much data to pour over, Unit 9900 came up with solutions, including recruiting Israelis on the autistic spectrum for their analytical and visual skills. In recent years, says Shir Agassi, who served in Unit 9900 for more than seven years, it learned to automate much of the process, teaching algorithms to spot nuances, slight variations in landscapes and how their targets moved and behaved.....“We had to take all these photos, all this film, all this geospatial evidence and break it down: how do you know what you’re seeing, what’s behind it, how will it impact your intelligence decisions?” .....“You’re asking yourself — if you were the enemy, where would you hide? Where are the tall buildings, where’s the element of surprise? Can you drive there, what will be the impact of weather on all this analysis?”

Computer vision was essential to this task....Teaching computers to look for variations allowed the unit to quickly scan thousands of kilometres of background to find actionable intelligence. “You have to find ways not just to make yourself more efficient, but also to find things that the regular eye can’t,” she says. “You need computer vision to answer these questions.”.....The development of massive databases — from close-ups of farm insects to medical scans to traffic data — has given Israeli companies a valuable headstart over rivals. And in an industry where every new image teaches the algorithm something useful, that has made catching up difficult.......“Computer vision is absolutely the thread that ties us to other Israeli companies,” he says. “I need people with the same unique DNA — smart PhDs in mathematics, neural network analysis — to tell a player in the NBA how to improve his jump shot.”
Israel  cyber_security  hackers  cyber_warfare  dual-use  Israeli  security_&_intelligence  IDF  computer_vision  machine_learning  Unit_9900  start_ups  gene_pool  imagery  algorithms  actionable_information  geospatial  mapping  internal_systems  PhDs  drones  satellites  surveillance  autism 
november 2018 by jerryking
JPMorgan Invests in Startup Tech That Analyzes Encrypted Data - CIO Journal. - WSJ
By Sara Castellanos
Nov 13, 2018
(possible assistance to Robert Lewis)

JPMorgan Chase & Co. has invested in a startup whose technology can analyze an encrypted dataset without revealing its contents, which could be “materially useful” for the company and its clients, said Samik Chandarana, head of data analytics for the Corporate and Investment Bank division.

The banking giant recently led a $10 million Series A funding round in the data security and analytics startup, Inpher Inc., headquartered in New York and Switzerland. JPMorgan could use the ‘secret computing’ technology to analyze a customer’s proprietary data on their behalf, using artificial intelligence algorithms without sacrificing privacy.......One of the technological methods Inpher uses, called fully homomorphic encryption, allows for computations to be conducted on encrypted data, said Jordan Brandt, co-founder and CEO of the company. It’s the ability to perform analytics and machine learning on cipher text, which is plain, readable text that has been encrypted using a specific algorithm, or a cipher, so that it becomes unintelligible.

Analyzing encrypted information without revealing any secret information is known as zero-knowledge computing and it means that organizations could share confidential information to gather more useful insights on larger datasets.
algorithms  artificial_intelligence  encryption  JPMorgan_Chase  start_ups 
november 2018 by jerryking
Anti-Algorithm Fashion
Sept. 10, 2018 | The New York Times | By Vanessa Friedman.

Some fashion brands are displaying an increasingly confident adherence to their own ideas about what the world should look like now.

They make what they want, in the way they want. If that means getting rained on, so be it. If that means they lose audience members to shelter, well, O.K. It sounds like a small thing, but it’s getting harder and harder to find. The industry bends toward compromise. There’s a lot of pressure these days to design by algorithm. We know too much about buying habits and likes, and the result is an insidious bias toward giving people what they have already indicated they want. It may be safe, and easier to sell, but it’s antithetical to the whole point of fashion, which should be about giving people what they never knew they wanted — what they couldn’t imagine they wanted — until they saw it. There’s a clarity to such commitment that keeps people in their seats, a ruthlessness toward pandering to the prevailing winds (or rain) that is itself desirable.
====================================================
Excerpt from 'A whole new mind: why right-brainers will rule
the future' By Daniel H. Pink. "Indeed, one of design's most potent
economic effects is this very capacity to create new markets... The
forces of Abundance, Asia, and Automation turn goods and services into
commodities so quickly that the only way to survive is by constantly
developing new innovations, inventing new categories, and (in Paola
Antonelli's lovely phrase) giving the world something it didn't know it
was missing.
analog  fashion  messiness  inspiration  algorithms  apparel  brands  clothing_labels 
september 2018 by jerryking
Platform companies have to learn to share
August 19, 2018 | Financial Times | Rana Foroohar.

Algorithmic management places dramatically more power in the hands of platform companies. Not only can they monitor workers 24/7, they benefit from enormous information asymmetries that allow them to suddenly deactivate drivers with low user ratings, or take a higher profit margin from riders willing to pay more for speedier service, without giving drivers a cut. This is not a properly functioning market. It is a data-driven oligopoly that will further shift power from labour to capital at a scale we have never seen before......Rather than wait for more regulatory pushback, platform tech companies should take responsibility now for the changes they have wreaked — and not just the positive ones. That requires an attitude adjustment. Many tech titans have a libertarian bent that makes them dismissive of the public sector as a whole.......Yet the potential benefits of ride-hailing and sharing — from less traffic to less pollution — cannot actually be realised unless the tech companies work with the public sector. One can imagine companies like Uber co-operating with city officials to phase in vehicles slowly, rolling out in underserved areas first, rather than flooding the most congested markets and creating a race to the bottom......Airbnb...often touts its ability to open up new neighbourhoods to tourism, but research shows that in cities like New York, most of its business is done in a handful of high end areas — and the largest chunk by commercial operators with multiple listings, with the effect of raising rents and increasing the strains caused by gentrification. On the labour side, too, the platform companies must take responsibility for the human cost of disruption. NYU professor Arun Sundararajan, has proposed allowing companies to create a “safe harbour” training fund that provides benefits and insurance for drivers and other on-demand workers without triggering labour laws that would categorise such workers as full-time employees (which is what companies want to avoid).
Airbnb  algorithms  dark_side  data_driven  gig_economy  information_asymmetry  New_York_City  oligopolies  on-demand  platforms  public_sector  Rana_Foroohar  ride_sharing  sharing_economy  safe_harbour  training  Uber 
august 2018 by jerryking
Inside FreshDirect’s Big Bet to Win the Home-Delivery Fight - WSJ
By Jennifer Smith
July 18, 2018 5:30 a.m

Designed to keep food fresh longer and move it faster, FreshDirect’s 400,000 square-foot distribution centre is the online grocer’s multimillion-dollar bet on the fastest-growing sector in the grocery business, home-delivery. FreshDirect pioneered the e-commerce home-delivery market, and now with Amazon and big grocery chains like Kroger Co. piling on investments, companies are jockeying for position in a business that some believe is the future of supermarket sales.....FreshDirect's trucks now provide next-day delivery to customers across the New York-New Jersey, Philadelphia and Washington, D.C., metropolitan areas, with plans to expand into Boston next. The private company says it generated between $600 million and $700 million in annual revenue in 2017.

It declined to disclose the cost of the new facility, which was financed with the help of a $189 million investment round in 2016 led by J.P. Morgan Asset Management, direct funding and incentives from state and local governments......Amazon, Target Corp. and other large companies have invested hundreds of millions of dollars to expand food delivery and build out their grocery e-commerce operations. Supermarket chain owner Koninklijke Ahold Delhaize NV’s Peapod unit, the longest-running online grocery service in the U.S., has expanded to 24 markets and is investing in technology to cut its handling and delivery costs.

Walmart Inc. said this month that Jet.com, the online retailer it bought two years ago, will open a fulfillment center in the Bronx this fall to help roll out same- and next-day grocery deliveries in New York City.

The grocers are trying to solve one of the toughest problems in home delivery: Getting food to doorsteps in the same condition consumers would expect if they went to the store themselves. Delivering perishables is trickier than dropping off paper towels or dogfood. Fruit bruises, meat spoils, eggs break. ........FreshDirect’s logistic hurdles start well before delivery. It must get products from its suppliers to the building, process the food, then pick, pack and ship orders before the quality degrades.

That is why the new distribution centre has 15 different temperature zones. Tomatoes do best at about 55 degrees, but “chicken and meat like it to be just at 32 degrees... it gives more of shelf life to it,"....Software determines the most efficient route for each order, and tells workers which items to pick.....A big part of the facility [distribution centre] is ripping out tons and tons of operating costs out of the business.....The stakes in getting the technology right are high. FreshDirect is competing with grocery chains that often fill online orders through their stores, using a mix of staff and third-party services like Instacart Inc. So-called click-and-collect services, where consumers swing by to pick up their own orders, tend to have better margins because the retailer isn’t paying for last-mile delivery.....Online-only operations with centralized warehouses tend to be more efficient than logistics run out of stores, because they use fewer workers and can position goods for faster fulfillment.
algorithms  Amazon  big_bets  cold_storage  distribution_centres  distribution  e-commerce  food  FreshDirect  grocery  home-delivery  infrastructure  Kroger  logistics  perishables  retailers  software  supermarkets  Target  Wal-Mart  warehouses  fulfillment  same-day  piling_on  last_mile 
july 2018 by jerryking
Commodity trading enters the age of digitisation
July 9, 2018 | Financial Times | by Emiko Terazono.

Commodity houses are on the hunt for data experts to help them gain an edge after seeing their margins squeezed by rivals......commodity traders are seeking ways of exploiting their information to help them profit from price swings.

“It is really a combination of knowing what to look for and using the right mathematical tools for it,” ........“We want to be able to extract data and put it into algorithms,” .......“We then plan to move on to machine learning in order to improve decision-making in trading and, as a result, our profitability.” The French trading arm is investing in people, processes and systems to centralize its data — and it is not alone.

“Everybody [in the commodity world] is waking up to the fact that the age of digitisation is upon us,” said Damian Stewart at headhunters Human Capital.

In an industry where traders with proprietary knowledge, from outages at west African oilfields to crop conditions in Russia, vied to gain an upper hand over rivals, the democratisation of information over the past two decades has been a challenge......the ABCDs — Archer Daniels Midland, Bunge, Cargill and Louis Dreyfus Company — all recording single-digit ROE in their latest results. As a consequence, an increasing number of traders are hoping to increase their competitiveness by feeding computer programs with mountains of information they have accumulated from years of trading physical raw materials to try and detect patterns that could form the basis for trading ideas.......Despite this new enthusiasm, the road to electronification may not come easily for some traders. Compared to other financial and industrial sectors, “they are coming from way behind,” said one consultant.

One issue is that some of the larger commodities traders face internal resistance in centralising information on one platform.

With each desk in a trading house in charge of its profit-and-loss account, data are closely guarded even from colleagues, said Antti Belt, head of digital commodity trading at Boston Consulting Group. “The move to ‘share all our data with each other’ is a very, very big cultural shift,” he added.

Another problem is that in some trading houses, staff operate on multiple technology platforms, with different units using separate systems.

Rather than focusing on analytics, some data scientists and engineers are having to focus on harmonising the platforms before bringing on the data from different parts of the company.
ADM  agribusiness  agriculture  algorithms  artificial_intelligence  Bunge  Cargill  commodities  data_scientists  digitalization  machine_learning  traders  food_crops  Louis_Dreyfus  grains  informational_advantages 
july 2018 by jerryking
Music’s ‘Moneyball’ moment: why data is the new talent scout | Financial Times
JULY 5, 2018 | FT | Michael Hann.

The music industry loves to self-mythologise. It especially loves to mythologise about taking young scrappers from the streets and turning them into stars. It celebrates the men and women — but usually the men — with “golden ears” almost as much as the people making the music....A&R, or “artists and repertoire”, are the people who look for new talent, convince that talent to sign to the record label and then nurture it: advising on songs, on producers, on how to go about the job of being a pop star. It’s the R&D arm of the music industry......What the music business doesn’t like to shout about is how inefficient its R&D process is. The annual global spend on A&R is $2.8bn....and all that buys is the probability of failure: “Some labels estimate the ratio of commercial success to failure as 1 in 4; others consider the chances to be much lower — less than 1 in 10,” observes its 2017 report. Or as Mixmag magazine’s columnist The Secret DJ put it: “Major labels call themselves a business but are insanely unprofitable, utterly uncertain, totally rudderless and completely ignorant.”......The rise of digital music brought with it a huge amount of data which, industry executives realized, could be turned to their advantage. ....“All our business units must now leverage data and analytics in innovative ways to dig deeper than ever for new talent. The modern day talent-spotter must have both an artistic ear and analytical eyes.”

Earlier this year, in the same week as Warner announced its acquisition of Sodatone, a company that has developed a tool for talent-spotting via data, another data company, Instrumental, secured $4.2m of funding. The industry appeared to have reached a tipping point — what the website Music Ally called “A&R’s data moment”. Which is why, wherever the music industry’s great and good gather, the word “moneyball” has become increasingly prevalent.
........YouTube, Spotify, Instagram were born and changed the way talent begins its journey. All the barriers came down. Suddenly you’ve got tens of thousands of pieces of music content being uploaded.......Home computing’s democratization of recording removed the barriers to making high-quality music. No longer did you need access to a studio and an experienced producer, plus the money to pay for them. But the music industry had no way to keep abreast of these new creators. “....The way A&R people have discovered talent has barely changed since the music industry began, and it’s fundamentally the same for indie labels, who put artistry above sales, as it is for major labels who have to answer to shareholders. It’s always been about information.....“We find them by listening to new music constantly, by people giving us tips, by going out and seeing things that sound interesting,”.....“The most useful people to talk to are concert promoters and booking agents. They are least inclined to bullshit; they’ll tell you how many people an act is drawing,”...like labels, publishers also have an A&R function, signing up songwriters, many of whom will also be in bands)....“Journalists and radio producers are [also] very useful people to give you information. If you know you’ve got particular DJs or particular writers who are going to pick up something, that’s really good.”
.......Instrumental’s selling point is a dashboard called Talent AI, which scrapes data from Spotify playlists with more than 10,000 followers.....“We took a view that to build momentum on Spotify, you need to be on playlists,”....“If no one knows who you are, no one’s going to suddenly start streaming a track you’ve just put up. It happens when you start getting included on playlists.”......To make it workable, the Talent AI dashboard enables users to apply a series of filters to either tracks or artists: to sort by nationality, by genre, by number of playlists they appear on, by the number of playlist subscribers, by their industry standing — are they signed to a major? To an independent label? Are they unsigned?
.......What A&R people are looking for, though, is not totals, it’s evidence of momentum. No one wants to sign the artist who has reached maximum popularity. They want the artist on the way up....“It’s the direction. Is it going in the right direction?”....when it comes to assessing what an artist can offer, the data isn’t even always about the numbers. “The one I look at the most is Instagram, because that’s the easiest way for an artist to express themselves in a way other than the music — how they look, what they’re into,” she says. “That gives a real snapshot into [them] and whether they really have formulated a world for themselves or not.”......not everyone is delighted with the drive to data. “[the advent of] Spotify...became the driving force for signings...“A&Rs were using their eyes rather than their ears — watching numbers change rather than listening to music, and then jumping on acts....they saw something happening and got it out quickly without having to invest in the traditional A&R process.”... online heat tends to be generated by transient teenage audiences who are likely to move on rather than stick around for a decade: online presence is a big thing in electronic dance music, or some branches of urban music, in which an artist might only be good for a single song. In short, data does not measure quality; it does not tell you whether an artist has 20 good songs that can be turned into their first two albums; it does not tell you whether they can command a crowd in live performance..........The music industry, of course, has always had an issue with short-termism/short-sightedness: [tension] between the people who sign the cheques and those who go to bat for the artists is built into the way it works..........The problem is that without career artists, the music industry just becomes even more of a lottery. It is being made harder, not just by short-termism, but by the fact that music has become less culturally central. “It’s so much harder to connect with an audience or grow an audience, because there’s so much noise,”
.......Today the A&R...agree that the new data has its uses, but insist it still takes second place to the evidence of their own eyes and ears.......As for Withey, he is not about to tell the old-school scouts their days are done....Instrumental can tell A&R people which artists are hot, but not which are good. Also, there will be amazing acts who simply don’t get the traction on the internet to register on the Talent AI dashboard.....All of which will come as a relief to the people running those A&R departments. .....when asked if data will become the single most important factor in scouting talent: “I hope not. Otherwise we may as well have robots.” For now, at least, the golden ears are safe.
A&R  algorithms  analytics  data  dashboards  tips  discoveries  filters  hits  Instagram  inefficiencies  momentum  music  music_industry  music_labels  music_publishing  Moneyball  myths  playlists  self-mythologize  songwriters  Spotify  SXSW  success_rates  talent  talent_spotting  tipping_points  tracking  YouTube  talent_scouting  high-quality  the_single_most_important 
july 2018 by jerryking
The quant factories producing the fund managers of tomorrow
Jennifer Thompson in London JUNE 2, 2018

The wealth of nations and individuals is ever more likely to be influenced by computer algorithms as investors look to computer-powered quantitative trading strategies to generate returns. But underpinning those machines and algorithms are real people, namely the world’s sharpest mathematicians and data scientists.

Though not hard to identify, virtually every industry — and especially Big Tech — is competing with the financial world for their skills....Competition for talent means the campuses of elite universities have become a favoured hunting ground for many groups, and that the very best students and early career academics can command staggering starting salaries should they join the investment world......The links asset managers foster with universities vary. In the UK, Oxford and Cambridge are home to dedicated institutes established and funded by investment managers. Although these were set up with a genuine desire to foster research in the field, with a nod to philanthropy, they are also proving to be an effective way to spotting future talent.

Connections between hedge funds and investment managers are less formalised on US campuses but are treated with no less importance.

Personal relationships are important,
mathematics  data_scientists  quants  quantitative  hedge_funds  algorithms  war_for_talent  asset_management  PhDs  WorldQuant  Big_Tech 
june 2018 by jerryking
The digital economy is disrupting our old models
Diane Coyle 14 HOURS AGO

To put it in economic jargon, we are in the territory of externalities and public goods. Information once shared cannot be unshared.

The digital economy is one of externalities and public goods to a far greater degree than in the past. We have not begun to get to grips with how to analyse it, still less to develop policies for the common good. There are two questions at the heart of the challenge: what norms and laws about property rights over intangibles such as data or ideas or algorithms are going to be needed? And what will the best balance between collective and individual actions be or, to put it another way, between government and market?
mydata  personal_data  digital_economy  Facebook  externalities  knowledge_economy  public_goods  algorithms  data  ideas  intangibles  property_rights  protocols 
april 2018 by jerryking
Cry revolution if you like, Alexa is not listening
FEBRUARY 16, 2018 | FT | Henry Mance.

We know that a revolt against Big Tech is coming. All the ingredients are there: unaccountable elites, wealth disparities, popular discontent......We should be drawing the opposite lesson. We should be grateful for these moments when technology fails: they remind us that we are relying too much on algorithms.

Silicon Valley has created such gloriously useful products that we mostly overlook their limitations. We don’t notice that Google inevitably has a bias towards certain sources of information, or that Amazon directs us towards certain products. We forget that messaging apps draw us away from other forms of interaction. Already Snapchat has over 100m users who use it for more than 30 minutes a day on average. Already you can have Alexa listen attentively to everything you say at home, which is more than any member of your family will. 

Occasionally, however, we are confronted with the imperfections of technology. We are shown online ads for products we have already bought or for which we are biologically ineligible. We are invited to connect on LinkedIn with people we’ve never met, but who have the same name as our first line manager.....It is these moments which allow us to see that the emperor has no clothes. They demonstrate that the software is only as clever as the humans who have designed it. They remind us that the real revolutionary act is to switch off.
backlash  platforms  Snapchat  imperfections  algorithms  biases  limitations  Big_Tech 
february 2018 by jerryking
When algorithms reinforce inequality
FEBRUARY 9, 2018 | FT | Gillian Tett.

Virginia Eubanks, a political science professor in New York, undertakes academic research was focused on digital innovation and welfare claims. ......Last month, she published Automating Inequality, a book that explores how computers are changing the provision of welfare services in three US regions: Indiana, Los Angeles and Pittsburgh. It focuses on public sector services, rather than private healthcare insurance, but the message is the same: as institutions increasingly rely on predictive algorithms to make decisions, peculiar — and often unjust — outcomes are being produced. And while well-educated, middle-class people will often fight back, most poor or less educated people cannot; nor will they necessarily be aware of the hidden biases that penalise them....Eubanks concludes, is that digital innovation is reinforcing, rather than improving, inequality. ...What made the suffering doubly painful when the computer programs got it wrong was that the victims found it almost impossible to work out why the algorithms had gone against them, or to find a human caseworker to override the decision — and much of this could be attributed to a lack of resources....a similar pattern is described by the mathematician Cathy O’Neil in her book Weapons of Math Destruction. “Ill-conceived mathematical models now micromanage the economy, from advertising to prisons,” she writes. “They’re opaque, unquestioned and unaccountable and they ‘sort’, target or optimise millions of people . . . exacerbating inequality and hurting the poor.”...Is there any solution? O’Neil and Eubanks suggest that one option would be to require technologists to sign something equivalent to the Hippocratic oath, to “first do no harm”. A second — more costly — idea would be to force institutions using algorithms to hire plenty of human caseworkers to supplement the digital decision-making.

A third idea would be to ensure that the people who are creating and running the computer programs are forced to think about culture, in its broadest sense.....until now digital nerds at university have often had relatively little to do with social science nerds — and vice versa.

Computing has long been perceived to be a culture-free zone — this needs to change. But change will only occur when policymakers and voters understand the true scale of the problem. This is hard when we live in an era that likes to celebrate digitisation — and where the elites are usually shielded from the consequences of those algorithms.
Gillian_Tett  Cathy_O’Neil  algorithms  inequality  biases  books  dark_side  Pittsburgh  poverty  low-income 
february 2018 by jerryking
Daring rather than data will save advertising
John Hegarty JANUARY 2, 2017

Algorithms are killing creativity, writes John Hegarty

Ultimately, brands are built by talking to a broad audience. Even if part of that audience never buys your product. Remember, a brand is made not just by the people who buy it, but also by the people who know about it. Fame adds value to a brand, but to build it involves saying something that captures the public’s imagination. It needs to broadcast.

Now, data are fundamentally important in the building of a market. “Big data” can provide intelligence, gather information, identify buying patterns and determine certain outcomes. But what it cannot do is create an emotional bond with the consumer. Data do not make magic. That is the job of persuasion. And it is what makes brands valuable...... Steve Jobs or James Dyson did not build brilliant companies by waiting for a set of algorithms to tell them what to do.

Persuasion and promotion.

In today’s advertising world, creativity has taken a back seat. Creativity creates value and with it difference. And difference is vital for giving a brand a competitive edge. But the growing belief in “data-only solutions” means we drive it out of the marketplace.

If everything ends up looking the same and feeling the same, markets stagnate.
advertising  Steve_Jobs  creativity  human_ingenuity  data  massive_data_sets  data_driven  brands  emotional_connections  persuasion  ingenuity  daring  algorithms 
february 2018 by jerryking
When biased data holds a potentially deadly flaw
SEPTEMBER 27, 2017 | FT | Madhumita Murgia.

Researchers at scientific journal Nature said findings from its own investigation on the diversity of these data sets “prompted warnings that a much broader range of populations should be investigated to avoid genomic medicine being of benefit merely to ‘a privileged few’ ”.

This insidious data prejudice made me curious about other unintended biases in the tech world. Several new consumer technologies — often conceived by, built by and tested overwhelmingly on Caucasian males — are flawed due to biases in their design.
massive_data_sets  biases  data  data_driven  unintended_consequences  racial_disparities  algorithms  value_judgements 
january 2018 by jerryking
BlackRock bets on Aladdin as genie of growth
MAY 18, 2017 | FT | Attracta Mooney.

Aladdin, a technology system developed by BlackRock, the world’s largest asset manager, is also clever. It analyses the risks of investing in particular stocks, figures out where to sell bonds to get the best prices, and tracks those trades. And it is wily too, combing through huge data sets to find vital pieces of information for investors.....Launched in in 1988, when it was developed as an internal risk tool for BlackRock employees, Aladdin has become bigger, better and far more influential. It is now one of the best-known pieces of technology in the fund industry and is widely used by BlackRock’s rivals, including Deutsche Asset Management, the $733bn investment house, and Schroders, the UK’s largest listed fund manager.

But as Aladdin — which stands for Asset Liability and Debt and Derivatives Investment Network — has grown, concerns have mounted about its influence on markets. There are also questions about whether Aladdin can maintain or increase its hold on the asset management industry as rival technologies emerge.....with more and more investors using Aladdin, there are concerns about its impact on markets. The argument is that if trillions of dollars are being managed by people using the same risk system, those individuals may be more likely to make the same mistakes. i.e. Aladdin may increase systemic risk!!...Aladdin has a 9 per cent share of the 250 largest asset managers and a 15 per cent share of the insurance market, according to Credit Suisse, the Swiss bank. .......Many asset managers have recently begun the slow process of overhauling their technology systems after years of neglect. Previously, fund houses often had hundreds of different systems, but Aladdin and similar enterprise platforms allow businesses to cut out huge chunks of IT, reducing costs and jobs in the process.

At the same time, running money has become more complex and there is more regulatory scrutiny of investment decisions. This has meant that fund houses have been forced to assess how technology can help their investment processes.

“Money management is very tricky these days. Any tool that can help you with decisions is going to be highly in demand,”
........Under plans by Larry Fink, BlackRock’s chief executive, Aladdin will become an even more important source of cash for the fund giant. Mr Fink recently said that his goal is for Aladdin and the wider BlackRock solutions business to account for about 30 per cent of revenues in five years, compared with 7 per cent currently.......Even if there is a stumble in demand, BlackRock is already eyeing up other avenues for Aladdin.

In the past two years, it began promoting Aladdin, which comprises 25m lines of code, in the retail investment space, targeting wealth managers and brokers.

Last week, UBS Wealth Management Americas became the first wealth manager to say it will use Aladdin for risk management and portfolio construction......“Technology has always been a key differentiator for BlackRock. It is more essential to our business than ever before. We believe technology can transform our industry,” he said.

.......
Aladdin  asset_management  BlackRock  institutional_investors  Laurence_Fink  wealth_management  systemic_risks  order_management_system  algorithms  platforms 
january 2018 by jerryking
Algos know more about us than we do about ourselves
NOVEMBER 24, 2017 | Financial Time | John Dizard.

When intelligence collectors and analysts take an interest in you, they usually start not by monitoring the content of your calls or messages, but by looking at the patterns of your communications. Who are you calling, how often and in what sequence? What topics do you comment on in social media?

This is called traffic analysis, and it can give a pretty good notion of what you and the people you know are thinking and what you are preparing to do. Traffic analysis started as a military intelligence methodology, and became systematic around the first world war. Without even knowing the content of encrypted messages, traffic analysts could map out an enemy “order of battle” or disposition of forces, and make inferences about commanders’ intentions.

Traffic analysis techniques can also cut through the petabytes of redundant babble and chatter in the financial and political worlds. Even with state secrecy and the forests of non-disclosure agreements around “proprietary” investment or trading algorithms, crowds can be remarkably revealing in their open-source posts on social media.

Predata, a three-year-old New York and Washington-based predictive data analytics provider, has a Princeton-intensive crew of engineers and international affairs graduates working on early “signals” of market and political events. Predata trawls the open metadata for users of Twitter, Wikipedia, YouTube, Reddit and other social media, and analyses it to find indicators of future price moves or official actions.

I have been following their signals for a while and find them to be useful indicators. Predata started by creating political risk indicators, such as Iran-Saudi antagonism, Italian or Chilean labour unrest, or the relative enthusiasm for French political parties. Since the beginning of this year, they have been developing signals for financial and commodities markets.

The 1-9-90 rule
1 per cent of internet users initiate discussions or content, 9 per cent transmit content or participate occasionally and 90 per cent are consumers or ‘lurkers’

Using the example of the company’s BoJ signal. For this, Predata collects the metadata from 300 sources, such as Twitter users, contested Wikipedia edits or YouTube items created by Japanese monetary policy geeks. Of those, at any time perhaps 100 are important, and 8 to 10 turn out to be predictive....This is where you need some domain knowledge [domain expertise = industry expertise]. It turns out that Twitter is pretty important for monetary policy, along with the Japanese-language Wiki page for the Bank of Japan, or, say, a YouTube video of [BoJ governor] Haruhiko Kuroda’s cross-examination before a Diet parliamentary committee.

“Then you build a network of candidate discussions [JK: training beds] and look for the pattern those took before historical moves. The machine-learning algorithm goes back and picks the leads and lags between traffic and monetary policy events.” [Jk: Large data sets with known correct answers serve as a training bed and then new data serves as a test bed]

Typically, Predata’s algos seem to be able to signal changes in policy or big price moves [jk: inflection points] somewhere between 2 days and 2 weeks in advance. Unlike some academic Twitter scholars, Predata does not do systematic sentiment analysis of tweets or Wikipedia edits. “We only look for how many people there are in the conversation and comments, and how many people disagreed with each other. We call the latter the coefficient of contestation,” Mr Shinn says.

The lead time for Twitter, Wiki or other social media signals varies from one market to another. Foreign exchange markets typically move within days, bond yields within a few days to a week, and commodities prices within a week to two weeks. “If nothing happens within 30 days,” says Mr Lee, “then we say we are wrong.”
algorithms  alternative_data  Bank_of_Japan  commodities  economics  economic_data  financial_markets  industry_expertise  inflection_points  intelligence_analysts  lead_time  machine_learning  massive_data_sets  metadata  non-traditional  Predata  predictive_analytics  political_risk  signals  social_media  spycraft  traffic_analysis  training_beds  Twitter  unconventional 
november 2017 by jerryking
Novartis’s new chief sets sights on ‘productivity revolution’
SEPTEMBER 25, 2017 | Financial Times | Sarah Neville and Ralph Atkins.

The incoming chief executive of Novartis, Vas Narasimhan, has vowed to slash drug development costs, eyeing savings of up to 25 per cent on multibillion-dollar clinical trials as part of a “productivity revolution” at the Swiss drugmaker.

The time and cost of taking a medicine from discovery to market has long been seen as the biggest drag on the pharmaceutical industry’s performance, with the process typically taking up to 14 years and costing at least $2.5bn.

In his first interview as CEO-designate, Dr Narasimhan says analysts have estimated between 10 and 25 per cent could be cut from the cost of trials if digital technology were used to carry them out more efficiently. The company has 200 drug development projects under way and is running 500 trials, so “that will have a big effect if we can do it at scale”.......Dr Narasimhan plans to partner with, or acquire, artificial intelligence and data analytics companies, to supplement Novartis’s strong but “scattered” data science capability.....“I really think of our future as a medicines and data science company, centred on innovation and access.”

He must now decide where Novartis has the capability “to really create unique value . . . and where is the adjacency too far?”.....Does he need the cash pile that would be generated by selling off these parts of the business to realise his big data vision? He says: “Right now, on data science, I feel like it’s much more about building a culture and a talent base . . . ...Novartis has “a huge database of prior clinical trials and we know exactly where we have been successful in terms of centres around the world recruiting certain types of patients, and we’re able to now use advanced analytics to help us better predict where to go . . . to find specific types of patients.

“We’re finding that we’re able to significantly reduce the amount of time that it takes to execute a clinical trial and that’s huge . . . You could take huge cost out.”...Dr Narasimhan cites one inspiration as a visit to Disney World with his young children where he saw how efficiently people were moved around the park, constantly monitored by “an army of [Massachusetts Institute of Technology-]trained data scientists”.
He has now harnessed similar technology to overhaul the way Novartis conducts its global drug trials. His clinical operations teams no longer rely on Excel spreadsheets and PowerPoint slides, but instead “bring up a screen that has a predictive algorithm that in real time is recalculating what is the likelihood our trials enrol, what is the quality of our clinical trials”.

“For our industry I think this is pretty far ahead,” he adds.

More broadly, he is realistic about the likely attrition rate. “We will fail at many of these experiments, but if we hit on a couple of big ones that are transformative, I think you can see a step change in productivity.”
algorithms  analytics  artificial_intelligence  attrition_rates  CEOs  data_driven  data_scientists  drug_development  failure  Indian-Americans  multiple_targets  Novartis  pharmaceutical_industry  predictive_analytics  productivity  productivity_payoffs  product_development  real-time  scaling  spreadsheets  Vas_Narasimhan 
november 2017 by jerryking
The Ivory Tower Can’t Keep Ignoring Tech
NOV. 14, 2017 | The New York Times | By Cathy O’Neil is a data scientist and author of the book “Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Follow her on Twitter at @mathbabedotorg.

We urgently need an academic institute focused on algorithmic accountability.

First, it should provide a comprehensive ethical training for future engineers and data scientists at the undergraduate and graduate levels, with case studies taken from real-world algorithms that are choosing the winners from the losers. Lecturers from humanities, social sciences and philosophy departments should weigh in.

Second, this academic institute should offer a series of workshops, conferences and clinics focused on the intersection of different industries with the world of A.I. and algorithms. These should include experts in the content areas, lawyers, policymakers, ethicists, journalists and data scientists, and they should be tasked with poking holes in our current regulatory framework — and imagine a more relevant one.

Third, the institute should convene a committee charged with reimagining the standards and ethics of human experimentation in the age of big data, in ways that can be adopted by the tech industry.

There’s a lot at stake when it comes to the growing role of algorithms in our lives. The good news is that a lot could be explained and clarified by professional and uncompromised thinkers who are protected within the walls of academia with freedom of academic inquiry and expression. If only they would scrutinize the big tech firms rather than stand by waiting to be hired.
algorithms  accountability  Cathy_O’Neil  Colleges_&_Universities  data_scientists  ethics  inequality  think_tanks  Big_Tech 
november 2017 by jerryking
Disney’s Big Bet on Streaming Relies on Little-Known Tech Company
OCT. 8, 2017 | The New York Times | By BROOKS BARNES and JOHN KOBLIN.

For two days in June 2017, Disney’s board of directors wrestled with one topic: how technology was disrupting the company’s traditional movie, television and theme park businesses, and what to do about it?.....Cord cutting was accelerating much faster than expected. Live viewing for some children’s programming was in free fall......Robert A. Iger, Disney’s chief executive and chairman, proposed a legacy-defining move. It was time for Disney to double down on streaming..... bet the entertainment giant’s future on a wonky, little-known technology company housed in a former cookie factory: BamTech.....Based in Manhattan’s Chelsea Market, the 850-employee company has a strong track record — no serious glitches, even when delivering tens of millions of live streams at a time. BamTech also has impressive advertising technology (inserting ads in video based on viewer location) and a strong reputation for attracting and keeping viewers, not to mention billing them.....BamTech grew out of Major League Baseball Advanced Media, or Bam for short, which was founded in 2000 as a way to help teams create websites. By 2002, Bam was experimenting with streaming video as a way for out-of-town fans to watch games.

Soon, Bam developed technology that attracted outside clients, including the WWE, Fox Sports, PlayStation Vue and Hulu. HBO went to Bam in 2014 after failing to create a reliable stand-alone streaming service on its own. Could Bam get HBO up and running — in just a few months? Bam built HBO Now for roughly $50 million, delivering it just in time for the Season 5 premiere of “Game of Thrones,” which went off flawlessly. “They were nothing short of herculean for us,” said Richard Plepler, HBO’s chief executive.

In 2015, Bam decided to spin off its streaming division, calling it BamTech. With an eye toward its own direct-to-consumer future, particularly with ESPN, Disney paid $1 billion in 2016 for a 33 percent stake and an option to buy a controlling interest in 2020. To run the stand-alone company, M.L.B. and Disney recruited Michael Paull, 46, from Amazon, where he oversaw Prime Video and the introduction of Amazon Channels.....Disney contends that a big part of BamTech’s value has been overlooked. Down the road, as other media companies move toward streaming, BamTech intends to sign them up as clients.....Though BamTech has proved its streaming bona fides, it still lacks the algorithms and the personalization skills that have helped propel Netflix to success. To fill that gap, Mr. Paull recently hired the former chief technology officer of the F.B.I. to be the head of analytics.....The level of engineering required for that enormous volume of content is no small matter. Each bit of streamable content has to be made to fit a dizzying number of requirements. Start with web browsers, ranging from Safari to Chrome or Explorer, all of which have slightly different demands. It also has to fit every iPhone and Android phone. And then there are connected living room devices like Apple TV.
algorithms  BamTech  big_bets  boards_&_directors_&_governance  CEOs  cord-cutting  digital_savvy  digital_strategies  Disney  disruption  entertainment  game_changers  personalization  Quickplay  sports  sportscasting  streaming  theme_parks  direct-to-consumer 
october 2017 by jerryking
We Survived Spreadsheets, and We’ll Survive AI - WSJ
By Greg Ip
Updated Aug. 2, 2017

History and economics show that when an input such as energy, communication or calculation becomes cheaper, we find many more uses for it. Some jobs become superfluous, but others more valuable, and brand new ones spring into existence. Why should AI be different?

Back in the 1860s, the British economist William Stanley Jevons noticed that when more-efficient steam engines reduced the coal needed to generate power, steam power became more widespread and coal consumption rose. More recently, a Massachusetts Institute of Technology-led study found that as semiconductor manufacturers squeezed more computing power out of each unit of silicon, the demand for computing power shot up, and silicon consumption rose.

The “Jevons paradox” is true of information-based inputs, not just materials like coal and silicon......Just as spreadsheets drove costs down and demand up for calculations, machine learning—the application of AI to large data sets—will do the same for predictions, argue Ajay Agrawal, Joshua Gans and Avi Goldfarb, who teach at the University of Toronto’s Rotman School of Management. “Prediction about uncertain states of the world is an input into decision making,” they wrote in a recent paper. .....Unlike spreadsheets, machine learning doesn’t yield exact answers. But it reduces the uncertainty around different risks. For example, AI makes mammograms more accurate, the authors note, so doctors can better judge when to conduct invasive biopsies. That makes the doctor’s judgment more valuable......Machine learning is statistics on steroids: It uses powerful algorithms and computers to analyze far more inputs, such as the millions of pixels in a digital picture, and not just numbers but images and sounds. It turns combinations of variables into yet more variables, until it maximizes its success on questions such as “is this a picture of a dog” or at tasks such as “persuade the viewer to click on this link.”.....Yet as AI gets cheaper, so its potential applications will grow. Just as better weather forecasting makes us more willing to go out without an umbrella, Mr. Manzi says, AI emboldens companies to test more products, strategies and hunches: “Theories become lightweight and disposable.” They need people who know how to use it, and how to act on the results.
artificial_intelligence  Greg_Ip  spreadsheets  machine_learning  predictions  paradoxes  Jim_Manzi  experimentation  testing  massive_data_sets  judgment  uncertainty  economists  algorithms  MIT  Gilder's_Law  speed  steam_engine  operational_tempo  Jevons_paradox  decision_making 
august 2017 by jerryking
Meet the People’s Quant, an Ex-Marine Who Champions Value Investing - WSJ
By Chris Dieterich
June 2, 2017

Wesley Gray’s value-focused fund of overseas stocks is beating all its rivals over the past year. For him, it’s almost beside the point.

Mr. Gray, chief executive of asset manager Alpha Architect LP outside of Philadelphia, says watching short-term market moves doesn’t pay off. Instead, his firm focuses on the benefits of finding and buying a small number of very cheap stocks, and holding them through thick and thin.

Alpha Architect is an upstart active investment manager that tripled its assets last year, a noteworthy performance at a time when traditional stock pickers are struggling with lackluster performance and investor withdrawals. The firm, with $522 million in assets, is among a growing crop of money managers using academic financial and behavioral research, and algorithms, to identify stock bets likely to beat the market.

So-called quantitative investment strategies pulled from academic research have been around for years, popularized by the likes of Dimensional Fund Advisors and AQR Capital Management. Mr. Gray and Alpha Architect aim to deliver highly potent iterations to smaller investors.

Mr. Gray is a former captain in the U.S. Marine Corps who served a tour in Iraq, and later earned a Ph.D. in finance from the University of Chicago Booth School of Business. He says extreme discipline is a crucial component of his concentrated, algorithmic adaptations of classic value investing, popularized by Benjamin Graham and Warren Buffett.

Last year Mr. Gray put out a report, “Even God Would Get Fired as an Active Investor,” concluding that stock-picking foresight alone wouldn’t equip investors to conquer perhaps their most formidable foe: the fear-driven urge to cut losses.....the market is littered with winning strategies that lose their potency over time, and smart-sounding theories that fail outright when put into practice. Moreover, success in investing often leaves market-beating managers awash in fund inflows that quickly outstrip their capacity to generate ideas.

Mr. Gray responds that the research upon which his strategies are based have proved their resilience for years, and that they can be explained by investor behavior. He admits that he has considered the implications of getting too big, a state that he says isn’t imminent but could force unhappy changes on his firm.
alpha  investors  quants  USMC  PhDs  value_investing/investors  asset_management  algorithms  behavioural_economics  quantitative  idea_generation  finance  active_investing  stock_picking  investment_strategies  beat_the_market 
june 2017 by jerryking
Art market ripe for disruption by algorithms
MAY 26, 2017 | Financial Times | by John Dizard.

Art consultants and dealers are convinced that theirs is a high-touch, rather than a high-tech business, and they have arcane skills that are difficult, if not impossible, to replicate..... better-informed collectors [are musing about] how to compress those transaction costs and get that price discovery done more efficiently.....The art world already has transaction databases and competing price indices. The databases tend to be incomplete, since a high proportion of fine art objects are sold privately rather than at public auctions. The price indices also have their issues, given the (arguably) unique nature of the objects being traded. Sotheby’s Mei Moses index attempts to get around that by compiling repeat-sales data, which, given the slow turnover of particular works of art, is challenging.....Other indices, or value estimations, are based on hedonic regression, which is less amusing than it sounds. It is a form of linear regression used, in this case, to determine the weight of different components in the pricing of a work of art, such as the artist’s name, the work’s size, the year of creation and so on. Those weights in turn are used to create time-series data to describe “the art market”. It is better than nothing, but not quite enough to replace the auctioneers and dealers.....the algos are already on the hunt....people are watching the auctions and art fairs and doing empirics....gathering data at a very micro level, looking for patterns, just to gather information on the process.....the art world and its auction markets are increasingly intriguing to applied mathematicians and computer scientists. Recognising, let alone analysing, a work of art is a conceptually and computationally challenging problem. But computing power is very cheap now, which makes it easier to try new methods.....Computer scientists have been scanning, or “crawling”, published art catalogues and art reviews to create semantic data for art works based on natural-language descriptions. As one 2015 Polish paper says, “well-structured data may pave the way towards usage of methods from graph theory, topic labelling, or even employment of machine learning”.

Machine-learning techniques, such as software programs for deep recurrent neural networks, have already been used to analyse and predict other auction processes.
algorithms  disruption  art  art_finance  auctions  collectors  linear_regression  data_scientists  machine_learning  Sotheby’s  high-touch  pricing  quantitative  analytics  arcane_knowledge  art_market 
june 2017 by jerryking
With 125 Ph.D.s in 15 Countries, a Quant ‘Alpha Factory’ Hunts for Investing Edge - WSJ
By BRADLEY HOPE
Updated April 6, 2017

The firm is part of the forefront of a new quantitative renaissance in investing, where the ability to make sense of billions of bits of data in real time is more sought after than old-school financial analysis.

“Brilliance is very equally distributed across the world, but opportunity is not,” said Mr. Tulchinsky, a 50-year-old Belarusian. “We provide the opportunity.”

To do this, WorldQuant developed a model where it employs hundreds of scientists, including 125 Ph.D.s, around the world and hundreds more part-time workers to scour the noise of the economy and markets for hidden patterns. This is the heart of the firm. Mr. Tulchinsky calls it the “Alpha Factory.”....Quantitative hedge funds have been around for decades but they are becoming dominant players in the markets for their ability to parse massive data sets and trade rapidly. Amid huge outflows, traditional hedge funds are bringing aboard chief data scientists and trying to mimic quant techniques to keep up, fund executives say.

Some critics of quants believe their strategies are overhyped and are highly susceptible to finding false patterns in the noise of data. David Leinweber, a data scientist, famously found that the data set with the highest correlation with the S&P 500 over a 10-year period in the 1990s was butter production in Bangladesh.
quantitative  Wall_Street  PhDs  alpha  investors  slight_edge  massive_data_sets  signals  noise  data_scientists  real-time  algorithms  patterns  sense-making  quants  unevenly_distributed  WorldQuant 
april 2017 by jerryking
The High-Speed Trading Behind Your Amazon Purchase - WSJ
By CHRISTOPHER MIMS
Updated March 27, 2017

Beneath the placid surface of product pages lies an unseen world of bots, algorithms, flash crashes and fierce competition......Just beneath the placid surface of a typical product page on Amazon lies an unseen world, a system where third-party vendors can sell products alongside Amazon’s own goods. It’s like a stock market, complete with day traders, code-slinging quants, artificial-intelligence algorithms and, yes, flash crashes.

Amazon gave people and companies the ability to sell on Amazon.com in 2000, and it has since grown into a juggernaut, representing 49% of the goods Amazon ships. Amazon doesn’t break out numbers for the portion of its business driven by independent sellers, but that translates to tens of billions in revenue a year. Out of more than 2 million registered sellers, 100,000 each sold more than $100,000 in goods in the past year....It’s clear, after talking to sellers and the software companies that empower them, that the biggest of these vendors are growing into sophisticated retailers in their own right. The top few hundred use pricing algorithms to battle with one another for the coveted “Buy Box,” which designates the default seller of an item. It’s the Amazon equivalent of a No. 1 ranking on Google search, and a tremendous driver of sales.
fulfillment  Amazon  pricing  back-office  third-party  bots  algorithms  flash_crashes  competition  retailers  e-commerce  product_category  private_labels  stockmarkets  eBay  Wal-Mart  Jet  Christopher_Mims 
march 2017 by jerryking
Bumble founder Whitney Wolfe on millennial matchmaking | Evernote Web
18 March / 19 March 2017 | FT| by Alice Fishburn.

. . . Half a year in tech-app time, it’s like a normal-world five years.” What’s the solution? “You just have to run faster than it does.’’
Wolfe has surfed an extraordinary sea-change in how we approach relationships. Our phones now allow us to identify potential life companions through location, ethnic origin or hatred of the same thing and reject them just as quickly. Such opportunities come with a healthy serving of ethical and personal dilemmas.....Bumble’s USP — “truly not a gimmick”, Wolfe stresses, and timely for a feminist age — is that the woman has all the power (while both sexes swipe to show interest, only she can start a conversation). Wolfe may be firmly on-brand but she laughs wickedly at the ambitions of many tech evangelists. “So many entrepreneurs approach me and say, ‘I want to start the next big thing’, and I say, ‘Well, what are you solving?’ And oftentimes they say, ‘Oh, I’m not sure. I want to start something big.’ ” Sigh. “You can never start something big without solving something small, right? And for me, that was not being allowed to text guys first.”.....What has all this time with the data taught her about humans? “You understand when people are the happiest, the most busy, the most detached, most involved.” Sunday nights and Mondays are the busiest times on the site: “I think that’s probably really telling because that’s usually people’s downtime, when they are relaxing or when they’re feeling bummed out . . . a little bit lonely.”
Our view on the idea of technology running our love lives unsurprisingly depends on our culture. One transatlantic dater tells me that, in the US, Bumble is strongly associated with empowered women. In the UK, some moan that it just caters to lazy men.
women  entrepreneur  Tinder  Austin  dating  mobile_applications  relationships  feminism  millennials  match-making  sexism  Silicon_Valley  accelerated_lifecycles  algorithms  gestures  online_dating  downtime 
march 2017 by jerryking
Oxford Diary
4 March / 5 March | Financial Times | Madhumita Murgia.

The goals is to build a conversation around change, to make technological change less scary, to make sure people don't feel left behind because of technology---do this within 26 hrs.....In the Cotswolds, too, senior British media executive tells me his own experience of working with YouTubers "was more like a one-night stand than a marriage". "We use each other for numbers and legitimacy, but the question is will they ever understand the subtler issues of traditional programming? Rules? Political correctness?.....A government adviser tells me that they are afraid that AI will change the relationship between state and citizen....Algorithms helping governments make important social decisions. Algorithms are a kind of black box and that government many not be able to explain its choices when questioned.
Google  future  conferences  change  handpicked  entrepreneur  ISIS  civil_servants  algorithms  YouTube  mass_media  digital_media  artificial_intelligence  biases  value_judgements  large_companies  print_journalism  technological_change  cultural_clash 
march 2017 by jerryking
Machine learning, algorithms drive this advertising company’s growth - The Globe and Mail
MARK BUNTING
Special to The Globe and Mail
Published Wednesday, Mar. 08, 2017

What is programmatic advertising?

Canadian company AcuityAds Holdings Inc. (AT-X) is at the forefront of that transformation. It specializes in what’s called programmatic advertising where algorithms are used to allow advertisers to target, connect with, and accumulate data about their campaigns and their audiences. One of AcuityAds’ co-founders has a PhD in machine learning and algorithms. It’s one of the reasons the company believes its patented technology stands out from its peers.....
A happy advertiser spends more money

“The whole idea was build the algorithm in a way that delivers a positive ROI for clients,” Mr. Hayek says. “As long as they get a positive ROI, they’re going to spend more with us. And that’s proven itself to be a very good concept because we deal with advertisers. When they make money using our system, they’re very happy and they spend more money on our systems.”

Risks

Is Mr. Hayek concerned that in the fast-growing, rapidly changing sector in which AcuityAds operates, a new technology or unforeseen competitor could emerge to disrupt its model?

“Digital advertising is an $83-billion (U.S.) market place. $51-billion out of that is already programmatic,” Mr. Hayek explains. “All the pipes are already built, it was a fundamental shift that this is how we do this kind of business.”
machine_learning  algorithms  ad-tech  advertising  programmatic  risks 
march 2017 by jerryking
As Goldman Embraces Automation, Even the Masters of the Universe Are Threatened
February 7, 2017 | MIT Technology Review | by Nanette Byrnes.

Automated trading programs have taken over cash equities trading function at Goldman Sachs. A job that once employed 600 people in 2000, is now in 2017 being done by 2 people, with the rest of the work, supported by 200 computer engineers. Marty Chavez, the company’s deputy chief financial officer and former chief information officer, explained all this to attendees at a symposium on computing’s impact on economic activity held by Harvard’s Institute for Applied Computational Science last month.....Chavez, who will become chief financial officer in April, says areas of trading like currencies and even parts of business lines like investment banking are moving in the same automated direction that equities have already traveled.....Complex trading algorithms, some with machine-learning capabilities, first replaced trades where the price of what’s being sold was easy to determine on the market, including the stocks traded by Goldman’s old 600.

Now areas of trading like currencies and futures, which are not traded on a stock exchange like the New York Stock Exchange but rather have prices that fluctuate, are coming in for more automation as well. To execute these trades, algorithms are being designed to emulate as closely as possible what a human trader would do,.....Goldman’s new consumer lending platform, Marcus, aimed at consolidation of credit card balances, is entirely run by software, with no human intervention, Chavez said. It was nurtured like a small startup within the firm and launched in just 12 months,
automation  Goldman_Sachs  Martin_Chavez  CFOs  CIOs  risk-assessment  platforms  human_intervention  Marcus  software  algorithms  machine_learning  job_displacement 
february 2017 by jerryking
Buy, buy, baby
Sep 13th 2014 | The Economist

The advertising industry is going through something akin to the automation of the financial markets in the 1980s. This has helped to make advertising much more precise and personalised. Some advertising agencies and media companies have told their executives to read “Flash Boys” by Michael Lewis, a book about Wall Street’s high-speed traders, to make quite sure they get the message......Real-time bidding sounds high-tech but straightforward. When a consumer visits a website, his browser communicates with an ad server. The server sends a message to an exchange to provide data about that user, such as his IP address, his location and the website he is visiting. Potential ad buyers send their bids to the exchange. The highest one wins and an ad is served when the website loads. All this typically takes about 150 milliseconds.

In reality, though, the ad-tech ecosystem is stupefyingly complex. Luma Partners, an investment bank, has put together the "Lumascape", a bafflingly crowded organisational chart showing several hundred firms competing in this market. Sellers of advertising space often go through technology firms: a "supply-side platform" (SSP) helps publishers sell their inventory, and a "demand-side platform" (DSP) gives access to buyers. Many choose a data-management platform (DMP) to store and buy information about users.

Advanced behavioural targeting, which uses technology to reach specific users with the desired characteristics, helped advertisers increase their return on investment by 30-50%. One popular tactic is "retargeting", which allows advertisers to look for people who have visited their website before and show them an ad related to an item they were looking for but did not buy.
online_advertising  programmatic  advertising  advertising_agencies  LBMA  behavioural_targeting  location_based_services  automation  real-time  algorithms  ad-tech  auctions  ROI 
february 2017 by jerryking
Algorithms Aren’t Biased, But the People Who Write Them May Be - WSJ
By JO CRAVEN MCGINTY
Oct. 14, 2016

A provocative new book called “Weapons of Math Destruction” has inspired some charged headlines. “Math Is Racist,” one asserts. “ Math Is Biased Against Women and the Poor,” declares another.

But author Cathy O’Neil’s message is more subtle: Math isn’t biased. People are biased.

Dr. O’Neil, who received her Ph.D in mathematics from Harvard, is a former Wall Street quant who quit after the housing crash, joined the Occupy Wall Street movement and now publishes the mathbabe blog.
algorithms  mathematics  biases  books  Cathy_O’Neil  Wall_Street  PhDs  quants  Occupy_Wall_Street  Harvard  value_judgements 
october 2016 by jerryking
Make Algorithms Accountable
AUG. 1, 2016 | The New York Times | By JULIA ANGWIN.

An algorithm is a procedure or set of instructions often used by a computer to solve a problem. Many algorithms are secret. ....Algorithms are ubiquitous in our lives. They map out the best route to our destination and help us find new music based on what we listen to now. But they are also being employed to inform fundamental decisions about our lives:
résumés sorting, credit scoring, prediction of a defendant’s future criminality.....as we rapidly enter the era of automated decision making, we should demand more than warning labels [about the algorithms that are being used].

A better goal would be to try to at least meet, if not exceed, the accountability standard set by a president not otherwise known for his commitment to transparency, Richard Nixon: the right to examine and challenge the data used to make algorithmic decisions about us.

Algorithms should come with warning labels. Obama White House called for automated decision-making tools to be tested for fairness, and for the development of “algorithmic auditing.”
tools  automation  decision_making  algorithms  data_driven  transparency  fairness  Richard_Nixon  proprietary  accountability  biases 
august 2016 by jerryking
Steven A. Cohen’s Newest Bet: Do-It-Yourself Computer Traders - WSJ
By BRADLEY HOPE
July 27, 2016

Steven A. Cohen is betting as much as as $250 million that mechanical engineers and nuclear scientists can come up with market-beating mathematical models in their spare time. He's investing in a hedge fund launched by Boston investment firm Quantopian that provides money to do-it-yourself traders who come up with the best computerized investing methods, giving a share of any profits to the creators.

Mr. Cohen, chief executive officer of Point72 Asset Management LP, is also making an undisclosed investment in Quantopian itself through his family-office venture arm Point72 Ventures.

The billionaire’s new commitments are part of a broader push in the money- management world to embrace quantitative investing, which relies mainly on math-based models to bet on statistical relationships or patterns in stocks, bonds options, futures or currencies......Point72 Asset Management oversees the personal wealth of Mr. Cohen, his family and employees. It already has an internal team devoted to computer-driven trading strategies......Quantopian says it has 85,000 users signed up from 180 countries who have created more than 400,000 algorithms on the company’s free web-based platform. So far, the firm has only selected 10 of those to trade a few hundred thousand dollars on behalf of Quantopian. The platform is only for U.S. equities trading so far, but Quantopian plans to expand to other asset classes.
algorithms  quantitative  Wall_Street  Steven_Cohen  beat_the_market  hedge_funds  DIY  SAC_Capital  money_management  investing  Point72  asset_classes  family_office 
july 2016 by jerryking
Algorithms Need Managers, Too
January/February 2016 | HBR | by michael Luca, Jon Kleinberg and Sendhil Mullainathan.
algorithms  HBR  tools  social_media 
may 2016 by jerryking
Uber’s algorithm and the mirage of the marketplace.
July 2015 | Slate | By Tim Hwang and Madeleine Clare Elish
Uber  algorithms  sharing_economy 
july 2015 by jerryking
GE, Cisco flex major muscle in trend toward 'Industrial Internet' - The Globe and Mail
DAVID MILSTEAD
The Globe and Mail
Published Friday, Jun. 05, 2015

What GE did, says William Blair & Co. analyst Nicholas Heymann, is write software to collect data from its equipment – from locomotives to jet engines – and develop algorithms that help its customers make better plans, like a railway predicting where to add capacity based on port traffic, or where an airline should develop a hub for travel in 2020....Cisco, the global leader in the routers that allow computer networks to communicate, has spent $1-billion setting up six global “Internet of Everything” data centres and committed $100-million to an innovation fund. It’s promoting app development in developer communities and is working to create technical standards for the industry. It’s deployed Internet of Things offerings at several major customers, including Shell and Harley-Davidson,
sensors  Industrial_Internet  GE  Cisco  algorithms  predictive_analytics 
june 2015 by jerryking
Feeling uncertain, CEO? Better go on the attack - The Globe and Mail
HARVEY SCHACHTER
Special to The Globe and Mail
Published Tuesday, May. 05 2015

Taking control of uncertainty is the fundamental leadership challenge of our time … ” he writes in The Attacker’s Advantage. “The advantage now goes to those who create change, not just learn to live with it. Instead of waiting and reacting, such leaders immerse themselves in the ambiguities of the external environment, sort through them before things are settled and known, set a path, and steer the organization decisively onto it.”
Harvey_Schachter  Ram_Charan  uncertainty  algorithms  mathematics  data  management_consulting  anomalies  change  Jack_Welch  books  gurus  offense  data_driven  leadership  ambiguities  offensive_tactics 
may 2015 by jerryking
Venture: The (musical) schlock stops with Jingle Punks - The Globe and Mail
DAVE MORRIS
The Globe and Mail
Published Thursday, Sep. 25 2014

Jared Gutstadt had been playing in struggling bands by night and working as a video editor at MTV by day, choosing tracks from “production music” libraries to soundtrack the action in the likes of Chappelle’s Show.

The music industry boasts dozens of libraries, the largest of which are affiliated with the major record labels, and millions of songs are available for licensing, from no-name tracks to cover songs to huge, prohibitively expensive hits. The Rolling Stones famously charged Microsoft a reported $3-million (U.S.) to license Start Me Up for an ad campaign for Windows 95.

Ready-made production music normally costs a fraction of that figure. The filmmaker or TV company licenses the publishing rights (the lyrics and structure of a song, as opposed to the actual recording), paying what’s known as a “synchronization” fee. In 2013, according to the IFPI, synchronization fees worldwide totalled $337-million. In addition, whenever the TV show or movie featuring the track is broadcast or reproduced on DVDs, the owner of the recording itself is usually entitled to another sum, producing a revenue stream that can be small, but potentially steady.

Gutstadt and a partner saw an opportunity to be the suppliers of the music for the shows he and his MTV co-workers were editing, and Jingle Punks was born. The opportunity to become more than a niche player emerged not long after.

“There wasn’t enough production music that was easily accessible for the tidal wave of content that was going to occur,” Gutstadt says on the phone from his office in Los Angeles. That wave was unscripted reality shows.

Jingle Punks’ technical innovation, spearheaded by co-founder and software developer Dan Demole, was to offer a curated selection of license-able songs organized by what Gutstadt describes as a “relational search algorithm.” Users can search for music using non-musical terms such as the names of movies, and select and pay for the use of those songs, all through the company’s website.
music  free  start_ups  MTV  digital_media  algorithms  licensing  licensing_rights  musicians  music_catalogues  music_labels  music_publishing  Dave_Chappelle 
september 2014 by jerryking
How the big-data revolution can help design ideal cities - The Globe and Mail
DAVE MCGINN
The Globe and Mail
Published Wednesday, Sep. 24 2014

The big-data revolution faces two key challenges, both concerning the collection of information.

First, as is always the case when it comes to monitoring individuals and collecting details about their lives, is privacy. Second, there is the issue of using that data responsibly....Once municipalities have that consent, there is then the issue of harmonizing data sets in order to gain a fuller picture of issues. For instance, if a municipality wants to understand water-consumption levels, it helps to know how they track weather patterns.

Many cities are still struggling to understand how to use big data, but it promises to be a hugely important urban-planning tool.
algorithms  IBM  real-time  urban  sensors  municipalities  massive_data_sets  cities  data  decision_making  privacy  urban_planning  open_data 
september 2014 by jerryking
BlackRock’s Aladdin: genie not included - FT.com
July 11, 2014 | FT |By Tracy Alloway.
(Risk management technology is no substitute for investor instinct)
Aladdin is BlackRock's current, state of the art risk and order management system. Aladdin has been described as BlackRock’s “central nervous system” but what is less well-known is that the operating platform also acts as the brains at some 60 other financial firms which altogether handle a whopping $14tn worth of assets.

At banks, investment managers and trading outfits around the world, Aladdin’s genie is hard at work analysing portfolios, running stress test scenarios and generally employing BlackRock’s “collective intelligence” to perform a whole host of financial functions....the increasingly significant role that Aladdin and its 25m lines of code plays in the wider financial markets has, with notable exceptions, largely been overlooked....The role of these formulas or programs tends to go unnoticed but they often play two key roles in the build-ups to financial crises. Firstly they give investors and traders a potentially dangerous sense of control over risk. Second, as their use proliferates, they also encourage a build-up of “one-way” bets as investors increasingly come to rely on similar data and analysis.
BlackRock  Laurence_Fink  asset_management  pretense_of_knowledge  long-term  risk-management  Wall_Street  collective_intelligence  systemic_risks  order_management_system  algorithms  platforms  Aladdin  stress-tests  overconfidence  overlooked  false_confidence  scenario-planning  financial_crises 
july 2014 by jerryking
A 25-Question Twitter Quiz to Predict Retweets - NYTimes.com
JULY 1, 2014 | NYT | Sendhil Mullainathan.

how “smart” algorithms are created from big data: Large data sets with known correct answers serve as a training bed and then new data serves as a test bed — not too differently from how we might learn what our co-workers find funny....one of the miracles of big data: Algorithms find information in unexpected places, uncovering “signal” in places we thought contained only “noise.”... the Achilles’ heel of prediction algorithms--being good at prediction often does not mean being better at creation. (1) One barrier is the oldest of statistical problems: Correlation is not causation.(2) an inherent paradox lies in predicting what is interesting. Rarity and novelty often contribute to interestingness — or at the least to drawing attention. But once an algorithm finds those things that draw attention and starts exploiting them, their value erodes. (3) Finally, and perhaps most perversely, some of the most predictive variables are circular....The new big-data tools, amazing as they are, are not magic. Like every great invention before them — whether antibiotics, electricity or even the computer itself — they have boundaries in which they excel and beyond which they can do little.
predictive_analytics  massive_data_sets  limitations  algorithms  Twitter  analytics  data  data_driven  Albert_Gore  Achilles’_heel  boundary_conditions  noise  signals  paradoxes  correlations  causality  counterintuitive  training_beds  test_beds  rarity  novelty  interestingness  hard_to_find 
july 2014 by jerryking
Promoting Health With Enticing Photos of Fruits and Vegetables
FEB. 19, 2014 |NYT| By STEPHANIE STROM.

Bolthouse Farms, which produces juices, smoothies and other items, has developed an exceptionally playful website, FoodPornIndex.com, that calls attention to such food inequities. The company, owned by Campbell’s, wants to generate more clicks highlighting the plight of those unpopular beets and other less trendy but nutritious fruits and vegetables.

It has devised an algorithm to track hashtags on Twitter and elsewhere on the Internet and other mentions of 24 keywords for different vegetables, fruits and all those fatty, sugary favorites. Then, using alluring photographs, humor and music, the website lets visitors click on the Pomegranate Piñata, the Pizzabot or the Guac-a-Mole to get a sense of the numbers behind the item’s popularity on the web in real time....The Bolthouse algorithm checks for references to the keywords every 15 minutes. Of the 171 million posts picked up by the algorithm shortly before the site went live on Wednesday evening, 72 percent featured less healthy foods, while roughly 28 percent were accompanied by photos and posts of fruits or vegetables.

For example, the algorithm had spotted almost 13 million hashtags linked to posts with photos of pies by the time the website went live, compared to just 318,000 attached to posts featuring beets.... as more and more consumers make the connection between what they eat and how they feel and seek information about the ingredients n the foods they consume, food companies are increasingly trying to promote the healthiness and purity of the foods they sell.
fruits  vegetables  fresh_produce  diets  healthy_lifestyles  visualization  Bolthouse_Farms  social_media  algorithms  Twitter 
february 2014 by jerryking
How Beats' New Music Service Plans to Crush Spotify | Gadget Lab | Wired.com
By Roberto Baldwin
01.16.14

Beats Music won’t be joining the most-tracks arms race when it launches Tuesday. Instead, the new subscription service brought to you by Jimmy Iovine and Dr. Dre will win converts through a potent mix of smarter algorithms and human curation....Beats Music is different. The service is betting on smarts instead of sheer depth. While it will have enough songs to compete — anybody entering the game at this point has to — with a library millions of tracks deep, it hopes its unique approach to music discovery tools will give it an edge.

Setting up your Music DNA. Photo: Beats Music

As soon as you begin using the streaming service, Beats starts logging your “music DNA.” This serves as a personal profile used to determine which albums and tracks would be most relevant to you. To start generating your DNA, the service asks rudimentary questions, like which bands and genres you love.

But it takes other things into account. Your age,Your sex ,Who do you play quietly?Which artists do you crank up? ...But the system doesn’t solely rely on algorithms. It’s also backstopped by a small army of curators and behavioral scientists. This human element is there to help present music that doesn’t simply sound like the music you might enjoy, but also feels like it. Just because you listen to Mumford & Sons doesn’t mean you’d want to listen to a bunch of songs featuring banjos, for instance. You’d probably be more at home listening to Arcade Fire than Earl Scruggs. Humans can help make that determination. Algorithms can’t.
Beats  streaming  music  algorithms  curation  Spotify  Rdio  Rhapsody 
january 2014 by jerryking
Using 'remarkable' source of data, startup builds rich customer profiles - The Globe and Mail
Ivor Tossell

Special to The Globe and Mail

Published Monday, Jan. 06 2014

RetailGenius, a product from a Toronto startup called Viasense, promises to algorithmically generate customer profiles based on a remarkable source of data: Anonymous location data that’s collected by big mobile carriers, from the passive pings that every single cellphone sends out as it goes through the day.

The data that RetailGenius uses is anonymized – it doesn’t have any way of knowing whose cellphone belongs to who; it simply has a gigantic plot of where thousands of cellphones were at any given time.

“We create a unique identifier between those signals, and we can see those signals move throughout the city,” says Mossab Basir, RetailGenius’ founder. “We can see those changes in your location but we never really know who it is.”

What the product does next is intriguing: Based on some 50 million pieces of location data a day, RetailGenius crunches the numbers to make inferences from where each cellphone spends its time, and generates customer profiles by the thousands.

For instance, if a given cellphone spends the hours between 7 p.m. and 6 a.m. in a single area, it’s a good bet that its owner lives there. If that cellphone spends its working hours downtown five days a week, its owner is probably a daily commuter. And if it visits a given retail store once a week, a picture of its owner’s habits living and shopping habits starts to emerge.

By lumping these inferred profiles together, RetailGenius can give retailers a picture of who walks through their doors. For instance: What are the top 50 postal codes that are represented in their customers? What kind of volumes of customers are arriving at the store? How long do they stay?
data  start_ups  customer_insights  customer_profiling  RetailGenius  location_based_services  massive_data_sets  data_marketplaces  algorithms  Viasense  metadata  postal_codes  inferences  information_sources  anonymized  shopping_habits 
january 2014 by jerryking
An Ode to Joyful Music Streaming
Jan. 3, 2014 | WSJ.com | By John Jurgensen..

With more Spotify-like services on the horizon, a bounty of unexplored music beckons. But will they do a better job of helping you figure out what to listen to next?

As more people switch from the known confines of their personal music collections to the endless offerings of music-rental services like Spotify or Rdio, they're likely to suffer similar bouts of search-bar paralysis. As with the endless smorgasbord of gummy bears, Froot Loops and other toppings at those frozen-yogurt chains, what starts as a tantalizing abundance of music can suddenly seem overwhelming. That's one reason why we fall back on the same stuff we've been listening to since senior year in high school.....There's no shortage of guides designed to lead us through the wilds of digital music, but they all have drawbacks. .......Automated algorithms are OK for interpreting my personal listening patterns, but a music service should also show some humanity by reacting to what's happening in the zeitgeist. In the way that a cable-TV channel will program Will Ferrell movies when "Anchorman 2" is hitting theaters, why not play off the moment when everyone was talking about Beyoncé's surprise album by suggesting singers that influenced her or opened the door for her career? (Then again, I don't need an excuse to load up some classic Tina Turner.)

People complain that MP3s triggered the demise of extensive liner notes. While I'm not one to slavishly pore over the fine print in my LP collection, I want to click on a digital track as it plays to find out who wrote it, identify any samples it includes or—dare to dream—see how it connects to work by other artists......With every digital music service offering more or less the same stock—give or take a Led Zeppelin, which recently made an exclusive deal to stream its catalog on Spotify—my money will go to the one who can best guide me through the aisles.
algorithms  concierge_services  humanity  music  playlists  Rdio  Songza  Spotify  streaming  zeitgeist 
january 2014 by jerryking
Listen to Pandora, and It Listens Back - NYTimes.com
By NATASHA SINGER
Published: January 4, 2014

After years of customizing playlists to individual listeners by analyzing components of the songs they like, then playing them tracks with similar traits, the company has started data-mining users’ musical tastes for clues about the kinds of ads most likely to engage them. ... some companies are trying to differentiate themselves by using their proprietary data sets to make deeper inferences about individuals and try to influence their behavior... Pandora, for one, has a political ad-targeting system that has been used in presidential and congressional campaigns, and even a few for governor. It can deconstruct users’ song preferences to predict their political party of choice.
Pandora  music  massive_data_sets  algorithms  behavioural_targeting  data  voting  elections  online_advertising  streaming 
january 2014 by jerryking
Open data is not a panacea | mathbabe
December 29, 2012 Cathy O'Neil,
And it’s not just about speed. You can have hugely important, rich, and large data sets sitting in a lump on a publicly available website like wikipedia, and if you don’t have fancy parsing tools and algorithms you’re not going to be able to make use of it.

When important data goes public, the edge goes to the most sophisticated data engineer, not the general public. The Goldman Sachs’s of the world will always know how to make use of “freely available to everyone” data before the average guy.

Which brings me to my second point about open data. It’s general wisdom that we should hope for the best but prepare for the worst. My feeling is that as we move towards open data we are doing plenty of the hoping part but not enough of the preparing part.

If there’s one thing I learned working in finance, it’s not to be naive about how information will be used. You’ve got to learn to think like an asshole to really see what to worry about. It’s a skill which I don’t regret having.

So, if you’re giving me information on where public schools need help, I’m going to imagine using that information to cut off credit for people who live nearby. If you tell me where environmental complaints are being served, I’m going to draw a map and see where they aren’t being served so I can take my questionable business practices there.
open_data  unintended_consequences  preparation  skepticism  naivete  no_regrets  Goldman_Sachs  tools  algorithms  Cathy_O’Neil  thinking_tragically  slight_edge  sophisticated  unfair_advantages  smart_people  data_scientists  gaming_the_system  dark_side 
december 2013 by jerryking
Education: Minding the gap
Nov 16th 2013 | | The Economist |

The University of Arizona has a system called eAdvisor. This keeps track of each student’s progress towards his degree, and can make sure that courses which are critical but difficult—such as maths or statistics—are taken early on. Thanks to this system, which came online in 2007, the proportion of students (of all races) who move up to the next year each year has risen from 77% to 84%.

New findings from four Tennessee colleges support the idea that eAdvisors work. Software called the Degree Compass (developed by Tristan Denley, a mathematician) makes course suggestions for students in much the same way that Netflix recommends films to watch and Amazon offers goods to buy. The program ranks courses by their usefulness to a student for the degree he is taking, and also predicts those in which he is likely to get the best grade.
Colleges_&_Universities  African-Americans  achievement_gaps  students  personalization  algorithms  recommendations 
november 2013 by jerryking
How Technology Wrecks the Middle Class - NYTimes.com
August 24, 2013, 2:35 pm 30 Comments
How Technology Wrecks the Middle Class
By DAVID H. AUTOR AND DAVID DORN

In the four years since the Great Recession officially ended, the productivity of American workers — those lucky enough to have jobs — has risen smartly. But the United States still has two million fewer jobs than before the downturn, the unemployment rate is stuck at levels not seen since the early 1990s and the proportion of adults who are working is four percentage points off its peak in 2000…. Have we mechanized and computerized ourselves into obsolescence?... Economists have historically rejected what we call the “lump of labor” fallacy: the supposition that an increase in labor productivity inevitably reduces employment because there is only a finite amount of work to do. While intuitively appealing, this idea is demonstrably false. In 1900, for example, 41 percent of the United States work force was in agriculture. By 2000, that share had fallen to 2 percent, after the Green Revolution transformed crop yields…. Fast-forward to the present. The multi-trillionfold decline in the cost of computing since the 1970s has created enormous incentives for employers to substitute increasingly cheap and capable computers for expensive labor. These rapid advances — which confront us daily as we check in at airports, order books online, pay bills on our banks’ Web sites or consult our smartphones for driving directions — have reawakened fears that workers will be displaced by machinery. Will this time be different?
A starting point for discussion is the observation that although computers are ubiquitous, they cannot do everything. … Logically, computerization has reduced the demand for these jobs, but it has boosted demand for workers who perform “nonroutine” tasks that complement the automated activities. Those tasks happen to lie on opposite ends of the occupational skill distribution.
At one end are so-called abstract tasks that require problem-solving, intuition, persuasion and creativity. These tasks are characteristic of professional, managerial, technical and creative occupations, like law, medicine, science, engineering, advertising and design. People in these jobs typically have high levels of education and analytical capability, and they benefit from computers that facilitate the transmission, organization and processing of information.
On the other end are so-called manual tasks, which require situational adaptability, visual and language recognition, and in-person interaction….. Computerization has therefore fostered a polarization of employment, with job growth concentrated in both the highest- and lowest-paid occupations, while jobs in the middle have declined. Surprisingly, overall employment rates have largely been unaffected in states and cities undergoing this rapid polarization. Rather, as employment in routine jobs has ebbed, employment has risen both in high-wage managerial, professional and technical occupations and in low-wage, in-person service occupations….…workers [can] ride the wave of technological change rather than be swamped by it [by] investing more in their education.…The good news, however, is that middle-education, middle-wage jobs are not slated to disappear completely. While many middle-skill jobs are susceptible to automation, others demand a mixture of tasks that take advantage of human flexibility.…we predict that the middle-skill jobs that survive will combine routine technical tasks with abstract and manual tasks in which workers have a comparative advantage — interpersonal interaction, adaptability and problem-solving….The outlook for workers who haven’t finished college is uncertain, but not devoid of hope. There will be job opportunities in middle-skill jobs, but not in the traditional blue-collar production and white-collar office jobs of the past. Rather, we expect to see growing employment among the ranks of the “new artisans”: licensed practical nurses and medical assistants; teachers, tutors and learning guides at all educational levels; kitchen designers, construction supervisors and skilled tradespeople of every variety; expert repair and support technicians; and the many people who offer personal training and assistance, like physical therapists, personal trainers, coaches and guides. These workers will adeptly combine technical skills with interpersonal interaction, flexibility and adaptability to offer services that are uniquely human.
productivity  middle_class  automation  algorithms  interpersonal_interactions  downward_mobility  hollowing_out  MIT  Erik_Brynjolfsson  Andrew_McAfee  Luddites  problem_solving  job_destruction  job_displacement  barbell_effect  technological_change  blue-collar  white-collar  artisan_hobbies_&_crafts 
august 2013 by jerryking
The Secret Life of Data in the Year 2020
July-August 2012 | World Future Society Vol. 46, No. 4 |By Brian David Johnson.

A futurist for Intel shows how geotags, sensor outputs, and big data are changing the future. He argues that we need a better understanding of our relationship with the data we produce in order to build the future we want....Data is only useful and indeed powerful when it comes into contact with people.

This brings up some interesting questions and fascinating problems to be solved from an engineering standpoint. When we are architecting these algorithms, when we are designing these systems, how do we make sure they have an understanding of what it means to be human? The people writing these algorithms must have an understanding of what people will do with that data. How will it fit into their lives? How will it affect their daily routine? How will it make their lives better?...the only way to make sense of all this complex information—by viewing data, massive data sets, and the algorithms that really utilize big data as being human. Data doesn’t spring full formed from nowhere. Data is created, generated, and recorded. And the unifying principle behind all of this data is that it was all created by humans. We create the data, so essentially our data is an extension of ourselves, an extension of our humanity.
future  data  algorithms  Intel  sensors  massive_data_sets  storytelling  ethnography  questions  sense-making 
july 2013 by jerryking
LeadSift | Social Media Lead Generation and Identification Platform
LeadSift specializes in automobile, consumer electronics, higher education, insurance, telecom, travel & tourism.
start_ups  Mesh  Twitter  lead_generation  information_overload  algorithms 
may 2013 by jerryking
How a Toronto startup plans to transform ‘massive’ vitamin industry - The Globe and Mail
Ivor Tossell

Special to The Globe and Mail

Published Monday, Apr. 29 2013

drawing from a roster of about 100 different vitamins and supplements – everything from Indian ginseng to green tea extracts to selenium to grape seed extracts – Køge recommends a regime. If a customer accepts, Køge will fulfill the custom order, packaged in a month’s supply of thirty little sachets, each containing a day’s worth of pills. (Mr. Lenjosek points out that pills in sachets last longer than pills in bottles.)

As for the vitamins themselves, quality control is one of the company’s central pitches; in the vitamin world, it’s all too possible for products to be offered in insufficient concentrations to make a difference, or to be constituted of too much inert placebo material. Køge is sourcing its products through an established, Health Canada-certified (though as-yet unnamed) manufacturer in Montreal with which Mr. Lenjosek found a simpatico in values – especially, an earnest belief in the value of vitamins.

And Mr. Lenjosek is confident that that attitude is widespread. He pegs the vitamin market at $28-billion a year, with 140-million people regularly taking “lifestyle” vitamins.

“We’re aiming at people who already take vitamins. They’re in the 20s, their 30s, their 40s. It’s a massive market.”
start_ups  algorithms  vitamins  Køge 
april 2013 by jerryking
Working With Big Data: The New Math - WSJ.com
March 8, 2013| WSJ | By DEBORAH GAGE.

Researchers turn to esoteric mathematics to help make sense of it all.

New views [of old data are arriving] came courtesy of software that uses topology, a branch of math that compresses relationships in complex data into shapes researchers can manipulate and probe....

Better Tools
Seeking better tools than traditional statistical methods to analyze the vast amounts of data newly available to companies and organizations, researchers increasingly are scouring scientific papers and esoteric branches of mathematics like topology to make sense of complex data sets. The developer of the software used by Dr. Lum, Ayasdi, is just one of a small but growing number of companies working in this field.

So much data is now available, in such vast scope and minute detail, it is no longer useful to look at numbers neatly laid out in two-dimensional columns and rows,.....The research that inspired Ayasdi was funded by the Defense Advanced Research Projects Agency, or Darpa, and the National Science Foundation.......Data is so complex that using the same old methods, asking the same old questions, doesn't make sense....What is useful, he says, is to look at data arranged in shapes, using topology.

Topology is a form of geometry that relies on the way humans perceive shapes. We can see that an A is an A even when the letters are squashed or written in different fonts. Topology helps researchers look at a set of data and think about its similarities, even when some of the underlying details may be different....But topology is just one of the new methods being explored. Chris Kemp, former chief technology officer for IT at the National Aeronautics and Space Administration and now the chief executive of cloud computing company Nebula Inc., says he expects to see a renaissance in advanced mathematics and algorithms as companies increasingly realize how valuable data is and how cheaply they can store it.......Using graph theory, a tool similar to topology, IBM is mapping interactions of people on social networks, including its own. In diagrams based on the communications traffic, each person is a node, and communications between people are links. Graph-theory algorithms help discover the shortest path between the nodes, and thus reveal social cliques—or subcommunities—which show up because the cliques are more tightly interconnected than the community around them.......Tellagence's algorithms, for example, predicts how information will travel as it moves through social networks, but assumes that the network will change constantly, like the weather, and that what's most important about the data is the context in which it appears.

These techniques helped Tellagence do a bit of detective work for a Silicon Valley company that wanted to track down the source of some influential ideas being discussed online about the kind of integrated circuits it makes, known as field programmable gate arrays. Tellagence identified a group of more than 100 Japanese engineers involved in online discussions about the circuits. It then pinpointed two or three people whom traffic patterns showed were at the center of the conversation.

Tellagence's customer then devised a strategy to approach the engineers and potentially benefit from their ideas.

Says Tellagence CEO Matt Hixson, "We love to talk about people who have followers or friends, but these engineers were none of that—they had the right set of relationships because the right people listened to them."
algorithms  Ayasdi  DARPA  esoteric  IBM  infographics  massive_data_sets  mapping  mathematics  Nebula  networks  patterns  sense-making  Tellagence  the_right_people  tools  topology  visualization 
march 2013 by jerryking
Big Data should inspire humility, not hype
Mar. 04 2013| The Globe and Mail |Konrad Yakabuski.

" mathematical models have their limits.

The Great Recession should have made that clear. The forecasters and risk managers who relied on supposedly foolproof algorithms all failed to see the crash coming. The historical economic data they fed into their computers did not go back far enough. Their models were not built to account for rare events. Yet, policy makers bought their rosy forecasts hook, line and sinker.

You might think that Nate Silver, the whiz-kid statistician who correctly predicted the winner of the 2012 U.S. presidential election in all 50 states, would be Big Data’s biggest apologist. Instead, he warns against putting our faith in the predictive power of machines.

“Our predictions may be more prone to failure in the era of Big Data,” The New York Times blogger writes in his recent book, The Signal and the Noise. “As there is an exponential increase in the amount of available information, there is likewise an exponential increase in the number of hypotheses to investigate … [But] most of the data is just noise, as most of the universe is filled with empty space.”

Perhaps the biggest risk we run in the era of Big Data is confusing correlation with causation – or rather, being duped by so-called “data scientists” who tell us one thing leads to another. The old admonition about “lies, damn lies and statistics” is more appropriate than ever."
massive_data_sets  data_driven  McKinsey  skepticism  contrarians  data_scientists  Konrad_Yakabuski  modelling  Nate_Silver  humility  risks  books  correlations  causality  algorithms  infoliteracy  noise  signals  hype 
march 2013 by jerryking
For ‘House of Cards,’ Using Big Data to Guarantee Its Popularity - NYTimes.com
February 24, 2013 | NYT | By DAVID CARR

Rick Smolan wrote “The Human Face of Big Data.” “
Netflix, which has 27 million subscribers in the nation and 33 million worldwide, ran the numbers. It already knew that a healthy share had streamed the work of Mr. Fincher, the director of “The Social Network,” from beginning to end. And films featuring Mr. Spacey had always done well, as had the British version of “House of Cards.” With those three circles of interest, Netflix was able to find a Venn diagram intersection that suggested that buying the series would be a very good bet on original programming.

Big bets are now being informed by Big Data, and no one knows more about audiences than Netflix....But there are contrarian opinions, "“Data can only tell you what people have liked before, not what they don’t know they are going to like in the future,” he said. “A good high-end programmer’s job is to find the white spaces in our collective psyche that aren’t filled by an existing television show,” adding, those choices were made “in a black box that data can never penetrate.” "...The rise of the quants has some worried about the impact on quality and diversity of programming. Writing in Salon, Andrew Leonard wonders “how a reliance on Big Data might funnel craftsmanship in particular directions. What happens when directors approach the editing room armed with the knowledge that a certain subset of subscribers are opposed to jump cuts or get off on gruesome torture scenes” or are just interested in sexual romps?

Netflix insists that actual creative decisions will remain in the hands of the creators. “We don’t get super-involved on the creative side,” Mr. Evers said. “We hire the right people and give the freedom and budget to do good work.” That means that when Seth Rogen and Kristen Wiig are announced as special guests on coming episodes of “Arrested Development,” it is not because a statistical analysis told Netflix to do so.

But there are potential conflicts. Given that Netflix is in the business of recommending shows or movies, might its algorithms tilt in favor of the work it commissions as it goes deeper into original programming? It brings to mind how Google got crossed up when it began developing more products, and those began showing up in searches.

And there are concerns that the same thing that makes Netflix so valuable — it knows everything about us — could create problems if it is not careful with our data and our privacy.
David_Carr  Netflix  data_driven  massive_data_sets  streaming  data  television  digital_humanities  Asha_Isaacs  quantitative  big_bets  white_spaces  original_programming  human_psyche  craftsmanship  Venn_diagrams  content_creators  algorithms  biases  the_right_people 
february 2013 by jerryking
Max Levchin talks about data, sensors and the plan for his new startup(s) — Tech News and Analysis
Jan. 30, 2013 | GigaOm |By Om Malik.

“The world of real things is very inefficient: slack resources are abundant, so are the companies trying to rationalize their use. Über, AirBnB, Exec, GetAround, PostMates, ZipCar, Cherry, Housefed, Skyara, ToolSpinner, Snapgoods, Vayable, Swifto…it’s an explosion! What enabled this? Why now? It’s not like we suddenly have a larger surplus of black cars than ever before.

Examine the DNA of these businesses: resource availability and demand requests — highly analog, as this is about cars, drivers, and passengers — is captured at the edge, automatically where possible, then transmitted and stored, then processed centrally. Requests are queued at the smart center, and a marketplace/auction is used to allocate them, matches are made and feedback is given in real time.

A key revolutionary insight here is not that the market-based distribution of resources is a great idea — it is the digitalization of analog data, and its management in a centralized queue to create amazing new efficiencies.”
massive_data_sets  data  Max_Levchin  radical_ideas  sensors  start_ups  incubators  San_Francisco  sharing_economy  analog  efficiencies  meat_space  data_coordination  match-making  platforms  Om_Malik  resource_management  underutilization  resource_allocation  auctions  SMAC_stack  algorithms  digitalization 
february 2013 by jerryking
« earlier      
per page:    204080120160

related tags

5G  10x  21st._century  A&R  abundance  accelerated_lifecycles  accountability  achievement_gaps  Achilles’_heel  actionable_information  active_investing  ad-tech  ADM  advertising  advertising_agencies  advertising_networks  advice  African-Americans  agribusiness  agriculture  Airbnb  Aladdin  Albert_Gore  algorithms  alpha  AlphaGo  alternative_data  Amazon  ambiguities  analog  analytics  Andrew_McAfee  Andy_Kessler  anomalies  anonymized  apparel  Apple_Music  applications  arcane_knowledge  arms_race  art  artifical_intelligence  artificial_intelligence  artisan_hobbies_&_crafts  art_finance  art_market  Asha_Isaacs  asset_classes  asset_management  attrition_rates  auctions  augmented_reality  Austin  autism  automation  Ayasdi  back-office  backlash  BamTech  Bank_of_Japan  barbell_effect  Beats  beat_the_market  behavioural_economics  behavioural_targeting  beverages  biases  big_bets  Big_Tech  BlackRock  blindsided  blue-collar  boards_&_directors_&_governance  Bolthouse_Farms  books  book_reviews  bots  boundary_conditions  brands  Bunge  business  business_models  Capital_One  Cargill  Cathy_O’Neil  causality  cautionary_tales  CBC  CEOs  CFOs  change  checks_and_balances  Christopher_Mims  CIOs  Cisco  cities  citizen_engagement  citrus  civics  civil_servants  Claire_Cain_Miller  climate_change  Climate_Corporation  clothing_labels  cloud_computing  Coca-Cola  coding  cold_storage  collective_intelligence  collectors  college-educated  Colleges_&_Universities  collusion  commodities  competingonanalytics  competition  competitive_advantage  complaints  complexity  compression  computer_vision  concierge_services  conferences  connectivity  consent  content_creators  contests  contrarians  cord-cutting  core_businesses  corporate_investors  correlations  counterintuitive  CPI  craftsmanship  creative_renewal  creativity  critical_thinking  cross-disciplinary  cultural_clash  cultural_transmission  curation  customer_experience  customer_insights  customer_profiling  customer_service  cyber_security  cyber_warfare  D.E._Shaw  DaaS  daring  dark_side  DARPA  dashboards  data  Dataminr  data_coordination  data_driven  data_marketplaces  data_mining  data_quality  data_scientists  dating  Dave_Chappelle  David_Brooks  David_Carr  deception  decision_making  decontextualization  DeepMind  democracy  depression  diagnostic  diets  difficult_conversations  digitalization  digital_economy  digital_humanities  digital_media  digital_savvy  digital_strategies  direct-to-consumer  discoveries  Disney  disruption  distribution  distribution_centres  diversity  DIY  doctors  Donald_Trump  downtime  downward_mobility  drones  drug_development  dual-use  due_diligence  e-commerce  eBay  economics  economic_data  economists  Edward_Luce  efficiencies  elections  EMI  emotional_connections  emotional_intelligence  empathy  encryption  engaged_citizenry  entertainment  entrepreneur  entrepreneurship  EQ  Erik_Brynjolfsson  esoteric  ethics  ethnography  exhaust_data  experience_economy  experimentation  externalities  Facebook  factors_of_production  failure  fairness  false_confidence  family_office  farming  fashion  feminism  filters  fin-tech  finance  financial_crises  financial_markets  financial_risk  flash_crashes  Florida  food  food_crops  foundational  free  frequency_and_severity  FreshDirect  fresh_eyes  fresh_produce  fruits  FTC  fulfillment  funding  future  future-proofing  game_changers  game_theory  gaming_the_system  gatekeepers  GE  gender_gap  gene_pool  geo-tracking  geolocation  geospatial  gestures  gig_economy  Gilder's_Law  Gillian_Tett  globalization  Gnip  Goldman_Sachs  google  Google_Ventures  grains  Greg_Ip  grocery  gurus  Guy_Hands  habits  hackers  Hal_Varian  handpicked  hard_to_find  Harvard  Harvey_Schachter  HBR  healthy_lifestyles  hedge_funds  heterogeneity  high-quality  high-touch  hiring  historians  historical_amnesia  history  hits  holidays  hollowing_out  Holman_Jenkins  home-delivery  hospitals  howto  humanities  humanity  human_experience  human_ingenuity  human_intervention  human_psyche  human_resources  humility  hype  hyperlocal  IBM  ideas  idea_generation  IDF  ignorance  imagery  imperfections  incubators  Indian-Americans  indoctrination  Industrial_Internet  industry_expertise  inefficiencies  inequality  inferences  inflection_points  infographics  infoliteracy  informational_advantages  information_asymmetry  information_flows  information_overload  information_sources  infrastructure  ingenuity  inspiration  insta-bae  Instagram  institutional_investors  insurance  intangibles  Intel  intellectual_diversity  intelligence_analysts  interestingness  internal_systems  interpersonal_interactions  interviews  investing  investment_management  investment_strategies  investors  IPOs  ISIS  Israel  Israeli  iTunes  Jack_Welch  Japan  Jet  Jevons_paradox  Jim_Collins  Jim_Manzi  job_destruction  job_displacement  Joel_Spolsky  JPMorgan_Chase  judgment  juices  knowledge_economy  Konrad_Yakabuski  Kroger  Køge  large_companies  large_markets  last_mile  Laurence_Fink  LBMA  leadership  lead_generation  lead_time  legacy_artists  liberal_arts  licensing  licensing_rights  limitations  linear_regression  location_based_services  logic_&_reasoning  logistics  long-term  Louis_Dreyfus  low-income  loyalty_management  Luddites  Lyft  machine_learning  management_consulting  Manhattan  mapping  Marcus  Margaret_Wente  marketing  Martin_Chavez  massive_data_sets  mass_media  match-making  mathematics  Max_Levchin  McKinsey  meat_space  medical  mens'_health  mental_health  Mesh  messiness  metadata  microtargeting  middle_class  millennials  mindsets  MIT  mobile  mobile_applications  mobile_phones  modelling  momentum  Moneyball  money_management  MTV  multiple_targets  municipalities  music  musicians  music_catalogues  music_industry  music_labels  music_publishing  mybestlife  mydata  myths  naivete  Nate_Silver  Nebula  nerds  Netflix  networks  network_effects  New_York_City  nimbleness  noise  non-obvious  non-routine  non-traditional  Novartis  novelty  no_regrets  numeracy  Occupy_Wall_Street  offense  offensive_tactics  oligopolies  Om_Malik  on-demand  online_advertising  online_dating  op-ed  open_data  operational_tempo  order_management_system  original_programming  Outsourcing  overconfidence  overlooked  Pandora  paradoxes  patents  patterns  pattern_recognition  pay_attention  peer-to-peer  Pelmorex  perishables  personalization  personal_data  persuasion  pharmaceutical_industry  PhDs  physical_assets  piling_on  piracy  Pittsburgh  platforms  playlists  Point72  political_literacy  political_risk  positioning  postal_codes  poverty  Predata  predictions  predictive_analytics  predictive_modeling  Predix  preparation  pretense_of_knowledge  pricing  print_journalism  privacy  private_labels  problem_solving  productivity  productivity_payoffs  product_category  product_development  programmatic  property_rights  proprietary  ProQuest  prospectuses  protocols  public_goods  public_policy  public_sector  quantitative  quants  quantum_computing  questions  Quickplay  racial_disparities  radical_ideas  Ram_Charan  Rana_Foroohar  rarity  Rdio  real-time  recommendations  recruiting  regulators  relationships  resource_allocation  resource_management  retailers  RetailGenius  Rhapsody  Richard_Nixon  ride_sharing  risk-assessment  risk-management  risks  rogue_actors  ROI  royalties  ruination  SAC_Capital  safety  safe_harbour  same-day  San_Francisco  satellites  scaling  scenario-planning  securitization  security_&_intelligence  self-mythologize  sense-making  Sense_Networks  sensors  serious_incidents  services  sexism  sharing_economy  shopping_habits  signals  Silicon_Valley  skepticism  slight_edge  Slow_Movement  SMAC_stack  smart_people  Snapchat  social_media  software  software_development  soft_skills  songs  songwriters  Songza  sophisticated  Sotheby’s  speed  sports  sportscasting  Spotify  spreadsheets  spycraft  staffing  start_ups  steam_engine  STEM  Stephen_Baker  Steven_Cohen  Steve_Jobs  Steve_Ladurantaye  Steve_Lohr  stockmarkets  stock_picking  storytelling  streaming  stress-tests  students  subcontracting  subscriptions  success_rates  supermarkets  surveillance  SXSW  systematic_approaches  systemic_risks  talent  talent_management  talent_scouting  talent_spotting  Target  teams  technological_change  television  Tellagence  Terra_Firma  Tesla  testing  test_beds  theme_parks  the_right_people  the_single_most_important  thinking  thinking_tragically  think_tanks  think_threes  third-party  threats  Tidal  Tim_Harford  Tinder  tipping_points  tips  Tom_Friedman  tools  topology  tracking  traders  trading  traffic_analysis  training  training_beds  transaction_costs  transformational  transgendered  transparency  travel  trustworthiness  Twitter  U.S.  Uber  uncertainty  unconventional  under-representation  underutilization  underwriting  unevenly_distributed  unfair_advantages  unintended_consequences  Unit_9900  unpredictability  unstructured_data  urban  urban_planning  USMC  vacations  valuations  value_investing/investors  value_judgements  value_migration  Vas_Narasimhan  vc  vegetables  Venn_diagrams  venture_capital  Viasense  virtual_agents  visualization  visual_cues  vitamins  Vivienne_Ming  voting  vulnerabilities  Wal-Mart  Wall_Street  warehouses  war_for_talent  wealth_management  weather  Weatherbill  websites  white-collar  white_spaces  wireless  women  workplaces  WorldQuant  YouTube  zeitgeist 

Copy this bookmark:



description:


tags: