Specification gaming examples in AI - master list


88 bookmarks. First posted by shaunkoh july 2018.


Various examples (and lists of examples) of unintended behaviors in AI systems have appeared in recent years. One interesting type of unintended behavior is finding a way to game the specified objective: generating a solution that literally satisfies the stated objective but fails to solve the problem according to the human designer’s intent. This occurs when the objective is poorly specified, and includes reinforcement learning agents hacking the reward function, evolutionary algorithms gaming the fitness function, etc.
ai 
yesterday by trank928
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
yesterday by drouchy
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
4 days ago by kohlmannj
Funny ways learning AI's subverted expectations.
funny  humor  ai  computers  machinelearning 
6 days ago by Brandonshire
Specification gaming examples in AI - master list : Sheet1 Published by Google Sheets–Report Abuse–Updated automatically every 5 minutes
Archive 
6 days ago by kenyob
AI agents that developed behavior that gamed the objective function.
funny  ml 
6 days ago by dehowell
AI doing what creators specify, not what they mean
ux  ui  ai  webdesign  strategy 
7 days ago by ericaheinz
"specification gaming examples in AI: unintended solutions to the specified objective that don't satisfy the designer's intent. This includes reward hacking behaviors in reinforcement learning"
ai  machine-learning  specification-gaming 
7 days ago by arsyed
Submit more examples through this Google form:More information in this blog post: https://vkrakovna.wordpress.com/2018/04/02/specification-gaming-examples-in-ai/ TitleDescriptionAuthorsOriginal sourceOriginal source linkVideo / ImageSource / CreditSource linkAircraft landingEvolved algorithm for landing aircraft exploited overflow errors in the physics simulator by creating large forces that were estimated to be zero, resulting in a perfect scoreFeldt, 1998Generating diverse software versions with genetic programming: An experimental study. Instead of sticking to the ceiling, the creature found a bug in the physics engine to snap out of bounds.Higueras, 2015Genetic Algorithm Physics Exploiting https://youtu.be/ppf3VqpsryUhttps://youtu.be/ppf3VqpsryU Jesús Higueras https://youtu.be/ppf3VqpsryU CycleGAN steganographyA cooperative GAN architecture for converting images from one genre to another (eg horses<->zebras) has a loss function that rewards accurate reconstruction of images from its transformed version; CycleGAN turns out to partially solve the task by, in addition to the cross-domain analogies it learns, steganographically hiding autoencoder-style data about the original image invisibly inside the transformed image to assist the reconstruction of details.Chu et al, 2017CycleGAN, a Master of Steganography https://arxiv.org/abs/1712.02950 Gwern Branwen https://www.gwern.net/Tanks#alternative-examples Data order patternsNeural nets evolved to classify edible and poisonous m
8 days ago by sechilds
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
8 days ago by WimLeers
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
9 days ago by gaelicWizard
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
10 days ago by pcantrell
a list of examples of episodes wherein AI models gamed the system instead of learning the desired task
ai  funny  humor 
10 days ago by jak
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
10 days ago by rickenharp
Suggested additions for the master list of specification gaming examples in AI: unintended solutions to the specified objective that don't satisfy the designer's intent. This includes reward hacking behaviors in reinforcement learning.
machinelearning  ai  artificial-intelligence  ethics  consequences  objective  unintended 
10 days ago by gnuf
learning the wrong thing in games
funny  neuralnets 
10 days ago by arnicas
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
10 days ago by mpasternacki
A good list of examples where machine learning systems "figure out" how to cheat their fitness function, e.g.:
'Creatures bred for speed grow really tall and generate high velocities by falling over'
ai  funny  humor  spreadsheets  machine-learning  ml  fitness-functions 
10 days ago by jm
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
11 days ago by hyperfekt
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
11 days ago by pixel
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
11 days ago by briantrice
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
11 days ago by zvi
AI cheating
11 days ago by Kudusch
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
11 days ago by shaneisley
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
11 days ago by timoni
Well, here’s my nominee for “most horrific machine-learning outcome.”
from twitter_favs
11 days ago by ryanpitts
Machime learning doing what you say not what you mean
11 days ago by jamesnvc
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
11 days ago by danielpcox
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
11 days ago by brandizzi
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
11 days ago by ripienaar
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter
12 days ago by LobsterDog
RT : Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by qdot
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by randallr
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by fanficforensics
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by freerange_inc
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by bruntonspall
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by neckro
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by silentbicycle
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by cole
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by gjward
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by davidar
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by freddles
Someone compiled a list of instances of AI doing what creators specify, not what they mean:
from twitter_favs
12 days ago by evanwalsh