nhaliday + automation + network-structure   5

The Hanson-Yudkowsky AI-Foom Debate - Machine Intelligence Research Institute
How Deviant Recent AI Progress Lumpiness?: http://www.overcomingbias.com/2018/03/how-deviant-recent-ai-progress-lumpiness.html
I seem to disagree with most people working on artificial intelligence (AI) risk. While with them I expect rapid change once AI is powerful enough to replace most all human workers, I expect this change to be spread across the world, not concentrated in one main localized AI system. The efforts of AI risk folks to design AI systems whose values won’t drift might stop global AI value drift if there is just one main AI system. But doing so in a world of many AI systems at similar abilities levels requires strong global governance of AI systems, which is a tall order anytime soon. Their continued focus on preventing single system drift suggests that they expect a single main AI system.

The main reason that I understand to expect relatively local AI progress is if AI progress is unusually lumpy, i.e., arriving in unusually fewer larger packages rather than in the usual many smaller packages. If one AI team finds a big lump, it might jump way ahead of the other teams.

However, we have a vast literature on the lumpiness of research and innovation more generally, which clearly says that usually most of the value in innovation is found in many small innovations. We have also so far seen this in computer science (CS) and AI. Even if there have been historical examples where much value was found in particular big innovations, such as nuclear weapons or the origin of humans.

Apparently many people associated with AI risk, including the star machine learning (ML) researchers that they often idolize, find it intuitively plausible that AI and ML progress is exceptionally lumpy. Such researchers often say, “My project is ‘huge’, and will soon do it all!” A decade ago my ex-co-blogger Eliezer Yudkowsky and I argued here on this blog about our differing estimates of AI progress lumpiness. He recently offered Alpha Go Zero as evidence of AI lumpiness:

...

In this post, let me give another example (beyond two big lumps in a row) of what could change my mind. I offer a clear observable indicator, for which data should have available now: deviant citation lumpiness in recent ML research. One standard measure of research impact is citations; bigger lumpier developments gain more citations that smaller ones. And it turns out that the lumpiness of citations is remarkably constant across research fields! See this March 3 paper in Science:

I Still Don’t Get Foom: http://www.overcomingbias.com/2014/07/30855.html
All of which makes it look like I’m the one with the problem; everyone else gets it. Even so, I’m gonna try to explain my problem again, in the hope that someone can explain where I’m going wrong. Here goes.

“Intelligence” just means an ability to do mental/calculation tasks, averaged over many tasks. I’ve always found it plausible that machines will continue to do more kinds of mental tasks better, and eventually be better at pretty much all of them. But what I’ve found it hard to accept is a “local explosion.” This is where a single machine, built by a single project using only a tiny fraction of world resources, goes in a short time (e.g., weeks) from being so weak that it is usually beat by a single human with the usual tools, to so powerful that it easily takes over the entire world. Yes, smarter machines may greatly increase overall economic growth rates, and yes such growth may be uneven. But this degree of unevenness seems implausibly extreme. Let me explain.

If we count by economic value, humans now do most of the mental tasks worth doing. Evolution has given us a brain chock-full of useful well-honed modules. And the fact that most mental tasks require the use of many modules is enough to explain why some of us are smarter than others. (There’d be a common “g” factor in task performance even with independent module variation.) Our modules aren’t that different from those of other primates, but because ours are different enough to allow lots of cultural transmission of innovation, we’ve out-competed other primates handily.

We’ve had computers for over seventy years, and have slowly build up libraries of software modules for them. Like brains, computers do mental tasks by combining modules. An important mental task is software innovation: improving these modules, adding new ones, and finding new ways to combine them. Ideas for new modules are sometimes inspired by the modules we see in our brains. When an innovation team finds an improvement, they usually sell access to it, which gives them resources for new projects, and lets others take advantage of their innovation.

...

In Bostrom’s graph above the line for an initially small project and system has a much higher slope, which means that it becomes in a short time vastly better at software innovation. Better than the entire rest of the world put together. And my key question is: how could it plausibly do that? Since the rest of the world is already trying the best it can to usefully innovate, and to abstract to promote such innovation, what exactly gives one small project such a huge advantage to let it innovate so much faster?

...

In fact, most software innovation seems to be driven by hardware advances, instead of innovator creativity. Apparently, good ideas are available but must usually wait until hardware is cheap enough to support them.

Yes, sometimes architectural choices have wider impacts. But I was an artificial intelligence researcher for nine years, ending twenty years ago, and I never saw an architecture choice make a huge difference, relative to other reasonable architecture choices. For most big systems, overall architecture matters a lot less than getting lots of detail right. Researchers have long wandered the space of architectures, mostly rediscovering variations on what others found before.

Some hope that a small project could be much better at innovation because it specializes in that topic, and much better understands new theoretical insights into the basic nature of innovation or intelligence. But I don’t think those are actually topics where one can usefully specialize much, or where we’ll find much useful new theory. To be much better at learning, the project would instead have to be much better at hundreds of specific kinds of learning. Which is very hard to do in a small project.

What does Bostrom say? Alas, not much. He distinguishes several advantages of digital over human minds, but all software shares those advantages. Bostrom also distinguishes five paths: better software, brain emulation (i.e., ems), biological enhancement of humans, brain-computer interfaces, and better human organizations. He doesn’t think interfaces would work, and sees organizations and better biology as only playing supporting roles.

...

Similarly, while you might imagine someday standing in awe in front of a super intelligence that embodies all the power of a new age, superintelligence just isn’t the sort of thing that one project could invent. As “intelligence” is just the name we give to being better at many mental tasks by using many good mental modules, there’s no one place to improve it. So I can’t see a plausible way one project could increase its intelligence vastly faster than could the rest of the world.

Takeoff speeds: https://sideways-view.com/2018/02/24/takeoff-speeds/
Futurists have argued for years about whether the development of AGI will look more like a breakthrough within a small group (“fast takeoff”), or a continuous acceleration distributed across the broader economy or a large firm (“slow takeoff”).

I currently think a slow takeoff is significantly more likely. This post explains some of my reasoning and why I think it matters. Mostly the post lists arguments I often hear for a fast takeoff and explains why I don’t find them compelling.

(Note: this is not a post about whether an intelligence explosion will occur. That seems very likely to me. Quantitatively I expect it to go along these lines. So e.g. while I disagree with many of the claims and assumptions in Intelligence Explosion Microeconomics, I don’t disagree with the central thesis or with most of the arguments.)
ratty  lesswrong  subculture  miri-cfar  ai  risk  ai-control  futurism  books  debate  hanson  big-yud  prediction  contrarianism  singularity  local-global  speed  speedometer  time  frontier  distribution  smoothness  shift  pdf  economics  track-record  abstraction  analogy  links  wiki  list  evolution  mutation  selection  optimization  search  iteration-recursion  intelligence  metameta  chart  analysis  number  ems  coordination  cooperate-defect  death  values  formal-values  flux-stasis  philosophy  farmers-and-foragers  malthus  scale  studying  innovation  insight  conceptual-vocab  growth-econ  egalitarianism-hierarchy  inequality  authoritarianism  wealth  near-far  rationality  epistemic  biases  cycles  competition  arms  zero-positive-sum  deterrence  war  peace-violence  winner-take-all  technology  moloch  multi  plots  research  science  publishing  humanity  labor  marginal  urban-rural  structure  composition-decomposition  complex-systems  gregory-clark  decentralized  heavy-industry  magnitude  multiplicative  endogenous-exogenous  models  uncertainty  decision-theory  time-prefer 
april 2018 by nhaliday
Overcoming Bias : A Tangled Task Future
So we may often retain systems that inherit the structure of the human brain, and the structures of the social teams and organizations by which humans have worked together. All of which is another way to say: descendants of humans may have a long future as workers. We may have another future besides being retirees or iron-fisted peons ruling over gods. Even in a competitive future with no friendly singleton to ensure preferential treatment, something recognizably like us may continue. And even win.
ratty  hanson  speculation  automation  labor  economics  ems  futurism  prediction  complex-systems  network-structure  intricacy  thinking  engineering  management  law  compensation  psychology  cog-psych  ideas  structure  gray-econ  competition  moloch  coordination  cooperate-defect  risk  ai  ai-control  singularity  number  humanity  complement-substitute  cybernetics  detail-architecture  legacy  threat-modeling  degrees-of-freedom  composition-decomposition  order-disorder  analogy  parsimony  institutions  software  coupling-cohesion 
june 2017 by nhaliday
China invents the digital totalitarian state | The Economist
PROGRAMMING CHINA: The Communist Party’s autonomic approach to managing state security: https://www.merics.org/sites/default/files/2017-12/171212_China_Monitor_44_Programming_China_EN__0.pdf
- The Chinese Communist Party (CCP) has developed a form of authoritarianism that cannot be measured through traditional political scales like reform versus retrenchment. This version of authoritarianism involves both “hard” and “soft” authoritarian methods that constantly act together.
...
- To describe the social management process, this paper introduces a new analytical framework called China’s “Autonomic Nervous System” (ANS). This approach explains China’s social management process through a complex systems engineering framework. This framework mirrors the CCP’s Leninist way of thinking.
- The framework describes four key parts of social management, visualized through ANS’s “self-configuring,” “self-healing,” “self-optimizing” and “self-protecting” objectives.

China's Social Credit System: An Evolving Practice of Control: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3175792

https://news.ycombinator.com/item?id=12771302
https://twitter.com/Aelkus/status/873584698655735808
http://infoproc.blogspot.com/2017/06/face-recognition-applied-at-scale-in.html
The Chinese government is not the only entity that has access to millions of faces + identifying information. So do Google, Facebook, Instagram, and anyone who has scraped information from similar social networks (e.g., US security services, hackers, etc.).

In light of such ML capabilities it seems clear that anti-ship ballistic missiles can easily target a carrier during the final maneuver phase of descent, using optical or infrared sensors (let alone radar).

https://www.wsj.com/articles/the-all-seeing-surveillance-state-feared-in-the-west-is-a-reality-in-china-1498493020
https://twitter.com/0xa59a2d/status/880098750009659392
https://archive.is/zHmmE
China goes all-in on technology the US is afraid to do right.
US won't learn its lesson in time for CRISPR or AI.

https://www.acast.com/theeconomistasks/theeconomistasks-howdoyouwintheairace-
Artificial intelligence is developing fast in China. But is it likely to enable the suppression of freedoms? One of China's most successful investors, Neil Shen, has a short answer to that question. Also, Chinese AI companies now have the potential to overtake their Western rivals -- we explain why. Anne McElvoy hosts with The Economist's AI expert, Tom Standage

the dude just stonewalls when asked at 7:50, completely zipped lips

http://www.indiatimes.com/technology/science-and-future/this-scary-chinese-surveillance-video-is-serious-cause-for-concern-but-just-not-why-you-think-330530.html
What you’re looking at above is the work of SenseTime, a Chinese computer vision startup. The software in question, called SenseVideo, is a visual scenario analytics system. Basically, it can analyse video footage to pinpoint whether moving objects are humans, cars, or other entities. It’s even sophisticated enough to detect gender, clothing, and the type of vehicle it’s looking at, all in real time.

https://streamable.com/iyi3z

Even China’s Backwater Cities Are Going Smart: http://www.sixthtone.com/news/1001452/even-chinas-backwater-cities-are-going-smart

https://twitter.com/ctbeiser/status/913054318869217282
https://archive.is/IiZiP
remember that tweet with the ML readout of Chinese surveilance cameras? Get ready for the future (via @triviumchina)

XI praised the organization and promised to help it beef up its operations (China
Daily):
- "China will 'help ... 100 developing countries build or upgrade communication systems and crime labs in the next five years'"
- "The Chinese government will establish an international law enforcement institute under the Ministry of Public Security which will train 20,000 police for developing nations in the coming five years"

The Chinese connection to the Zimbabwe 'coup': http://www.cnn.com/2017/11/17/africa/china-zimbabwe-mugabe-diplomacy/index.html

China to create national name-and-shame system for ‘deadbeat borrowers’: http://www.scmp.com/news/china/economy/article/2114768/china-create-national-name-and-shame-system-deadbeat-borrowers
Anyone who fails to repay a bank loan will be blacklisted and have their personal details made public

China Snares Innocent and Guilty Alike to Build World’s Biggest DNA Database: https://www.wsj.com/articles/china-snares-innocent-and-guilty-alike-to-build-worlds-biggest-dna-database-1514310353
Police gather blood and saliva samples from many who aren’t criminals, including those who forget ID cards, write critically of the state or are just in the wrong place

Many of the ways Chinese police are collecting samples are impermissible in the U.S. In China, DNA saliva swabs or blood samples are routinely gathered from people detained for violations such as forgetting to carry identity cards or writing blogs critical of the state, according to documents from a national police DNA conference in September and official forensic journals.

Others aren’t suspected of any crime. Police target certain groups considered a higher risk to social stability. These include migrant workers and, in one city, coal miners and home renters, the documents show.

...

In parts of the country, law enforcement has stored DNA profiles with a subject’s other biometric information, including fingerprints, portraits and voice prints, the heads of the DNA program wrote in the Chinese journal Forensic Science and Technology last year. One provincial police force has floated plans to link the data to a person’s information such as online shopping records and entertainment habits, according to a paper presented at the national police DNA conference. Such high-tech files would create more sophisticated versions of paper dossiers that police have long relied on to keep tabs on citizens.

Marrying DNA profiles with real-time surveillance tools, such as monitoring online activity and cameras hooked to facial-recognition software, would help China’s ruling Communist Party develop an all-encompassing “digital totalitarian state,” says Xiao Qiang, adjunct professor at the University of California at Berkeley’s School of Information.

...

A teenage boy studying in one of the county’s high schools recalled that a policeman came into his class after lunch one day this spring and passed out the collection boxes. Male students were told to clean their mouths, spit into the boxes and place them into envelopes on which they had written their names.

...

Chinese police sometimes try to draw connections between ethnic background or place of origin and propensity for crime. Police officers in northwestern China’s Ningxia region studied data on local prisoners and noticed that a large number came from three towns. They decided to collect genetic material from boys and men from every clan to bolster the local DNA database, police said at the law-enforcement DNA conference in September.

https://twitter.com/nils_gilman/status/945820396615483392
China is certainly in the lead in the arena of digital-biometric monitoring. Particularly “interesting” is the proposal to merge DNA info with online behavioral profiling.

https://twitter.com/mr_scientism/status/949730145195233280
https://archive.is/OCsxs

https://www.techinasia.com/china-citizen-scores-credit-system-orwellian
https://www.theglobeandmail.com/amp/news/world/chinese-blacklist-an-early-glimpse-of-sweeping-new-social-credit-control/article37493300/

https://twitter.com/mr_scientism/status/952263056662384640
https://archive.is/tGErH
This is the thing I find the most disenchanting about the current political spectrum. It's all reheated ideas that are a century old, at least. Everyone wants to run our iPhone society with power structures dating to the abacus.
--
Thank God for the forward-thinking Chinese Communist Party and its high-tech social credit system!

https://en.wikipedia.org/wiki/Social_Credit_System

INSIDE CHINA'S VAST NEW EXPERIMENT IN SOCIAL RANKING: https://www.wired.com/story/age-of-social-credit/
http://www.wired.co.uk/article/chinese-government-social-credit-score-privacy-invasion

http://foreignpolicy.com/2017/05/24/chinese-citizens-want-the-government-to-rank-them/
The government thinks "social credit" will fix the country's lack of trust — and the public agrees.

To be Chinese today is to live in a society of distrust, where every opportunity is a potential con and every act of generosity a risk of exploitation. When old people fall on the street, it’s common that no one offers to help them up, afraid that they might be accused of pushing them in the first place and sued. The problem has grown steadily since the start of the country’s economic boom in the 1980s. But only recently has the deficit of social trust started to threaten not just individual lives, but the country’s economy and system of politics as a whole. The less people trust each other, the more the social pact that the government has with its citizens — of social stability and harmony in exchange for a lack of political rights — disintegrates.

All of which explains why Chinese state media has recently started to acknowledge the phenomenon — and why the government has started searching for solutions. But rather than promoting the organic return of traditional morality to reduce the gulf of distrust, the Chinese government has preferred to invest its energy in technological fixes. It’s now rolling out systems of data-driven “social credit” that will purportedly address the problem by tracking “good” and “bad” behavior, with rewards and punishments meted out accordingly. In the West, plans of this sort have tended to spark fears about the reach of the surveillance state. Yet in China, it’s being welcomed by a public fed up of not knowing who to trust.

It’s unsurprising that a system that promises to place a check on unfiltered power has proven popular — although it’s… [more]
news  org:rec  org:biz  china  asia  institutions  government  anglosphere  privacy  civil-liberty  individualism-collectivism  org:anglo  technocracy  authoritarianism  managerial-state  intel  sinosphere  order-disorder  madisonian  orient  n-factor  internet  domestication  multi  commentary  hn  society  huge-data-the-biggest  unaffiliated  twitter  social  trust  hsu  scitariat  anonymity  computer-vision  gnon  🐸  leviathan  arms  oceans  sky  open-closed  alien-character  dirty-hands  backup  podcast  audio  interview  ai  antidemos  video  org:foreign  ratty  postrat  expansionism  developing-world  debt  corruption  anomie  organizing  dark-arts  alt-inst  org:lite  africa  orwellian  innovation  biotech  enhancement  GWAS  genetics  genomics  trends  education  crime  criminal-justice  criminology  journos-pundits  chart  consumerism  entertainment  within-group  urban-rural  geography  org:mag  modernity  flux-stasis  hmm  comparison  speedometer  reddit  discussion  ssc  mobile  futurism  absolute-relative  apple  scale  cohesion  cooperate-defect  coordination  egalit 
january 2017 by nhaliday

bundles : dismalityeconpatterns

related tags

2016-election  absolute-relative  abstraction  acmtariat  africa  ai  ai-control  algorithms  alien-character  alignment  alt-inst  altruism  analogy  analysis  analytical-holistic  anglosphere  anomie  anonymity  antidemos  aphorism  apple  arms  article  asia  audio  authoritarianism  automation  axelrod  backup  barons  behavioral-gen  biases  big-yud  biodet  biotech  books  bostrom  business  censorship  chart  china  civic  civil-liberty  class  class-warfare  clever-rats  cog-psych  cohesion  coming-apart  commentary  comparison  compensation  competition  complement-substitute  complex-systems  composition-decomposition  computer-vision  conceptual-vocab  conquest-empire  consumerism  contrarianism  convexity-curvature  cooperate-defect  coordination  corporation  corruption  coupling-cohesion  crime  criminal-justice  criminology  crooked  crux  current-events  cybernetics  cycles  dark-arts  death  debate  debt  decentralized  decision-theory  deep-materialism  defense  degrees-of-freedom  detail-architecture  deterrence  developing-world  dirty-hands  discussion  distribution  domestication  economics  eden  eden-heaven  education  EEA  egalitarianism-hierarchy  EGT  elite  ems  endogenous-exogenous  engineering  enhancement  entertainment  entrepreneurialism  environmental-effects  epistemic  error  ethical-algorithms  evolution  evopsych  existence  exit-voice  expansionism  experiment  facebook  farmers-and-foragers  flexibility  flux-stasis  flynn  formal-values  frontier  futurism  generalization  genetics  genomics  geography  gnon  google  government  gray-econ  gregory-clark  growth-econ  guilt-shame  GWAS  hanson  hardware  hari-seldon  heavy-industry  history  hmm  hn  hsu  huge-data-the-biggest  humanity  ideas  identity-politics  idk  iidness  impact  individualism-collectivism  inequality  info-dynamics  infrastructure  innovation  insight  institutions  intel  intelligence  interests  internet  interview  intricacy  intuition  iq  iteration-recursion  journos-pundits  labor  large-factor  law  left-wing  legacy  lesswrong  leviathan  links  list  local-global  long-short-run  machine-learning  madisonian  magnitude  malaise  malthus  management  managerial-state  marginal  market-power  markets  math.DS  medieval  meta:science  metameta  military  miri-cfar  mobile  model-organism  models  modernity  moloch  moments  multi  multiplicative  mutation  n-factor  nature  near-far  network-structure  neuro  neuro-nitgrit  new-religion  news  nitty-gritty  noblesse-oblige  nonlinearity  number  oceans  open-closed  optimization  order-disorder  org:anglo  org:biz  org:edu  org:foreign  org:lite  org:mag  org:rec  organizing  orient  orwellian  parenting  parsimony  pdf  peace-violence  philosophy  plots  podcast  polanyi-marx  policy  politics  postmortem  postrat  prediction  privacy  pro-rata  propaganda  proposal  protocol-metadata  psychology  psychometrics  publishing  rationality  ratty  realness  reddit  regulation  reputation  research  revolution  rhetoric  risk  s-factor  scale  science  scitariat  search  selection  sex  shift  singularity  sinosphere  sky  slides  smoothness  social  society  software  speculation  speed  speedometer  spreading  ssc  startups  status  straussian  structure  study  studying  subculture  survey  sv  tech  technocracy  technology  the-watchers  thinking  threat-modeling  time  time-preference  track-record  tradeoffs  trends  trump  trust  twitter  unaffiliated  uncertainty  universalism-particularism  urban-rural  usa  utopia-dystopia  values  video  war  wealth  westminster  wiki  winner-take-all  within-group  wonkish  zeitgeist  zero-positive-sum  🐸 

Copy this bookmark:



description:


tags: