jm + automation   33

Automated unemployment insurance fraud detection system had a staggering 93% error rate in production
Expect to see a lot more cases of automated discrimination like this in the future. There is no way an auto-adjudication system would be allowed to have this staggering level of brokenness if it was dealing with the well-off:

State officials have said that between Oct. 1, 2013, when the MiDAS [automated unemployment insurance fraud detection] system came on line, and Aug. 7, 2015, when the state halted the auto-adjudication of fraud determinations and began to require some human review of MiDAS findings, the system had a 93% error rate and made false fraud findings affecting more than 20,000 unemployment insurance claims. Those falsely accused of fraud were subjected to quadruple penalties and aggressive collection techniques, including wage garnishment and seizure of income tax refunds. Some were forced into bankruptcy.

The agency is now reviewing about 28,000 additional fraud determinations that were made during the relevant period, but which involved some human review. An unknown number of those fraud findings were also false.
fraud  broken  fail  michigan  detroit  social-welfare  us-politics  computer-says-no  automation  discrimination  fraud-detection 
29 days ago by jm
Chatbot that overturned 160,000 parking fines now helping refugees claim asylum | Technology | The Guardian
The original DoNotPay, created by Stanford student Joshua Browder, describes itself as “the world’s first robot lawyer”, giving free legal aid to users through a simple-to-use chat interface. The chatbot, using Facebook Messenger, can now help refugees fill in an immigration application in the US and Canada. For those in the UK, it helps them apply for asylum support.
government  technology  automation  bots  asylum  forms  facebook 
6 weeks ago by jm
DST breaks everything
LOL as DST bug uncovers spurious automated noise complaints:
In January last year the airport unearthed a scheme whereby campaigners were using automated software to generate complaints against the airport. Officials caught out the set-up when the two anti-Heathrow enthusiasts forgot to take into account the hour going back in October, and began complaining about flights that had not yet taken off or arrived.
bugs  dst  daylight-savings-time  funny  heathrow  complaints  automation  noise 
november 2016 by jm
Google Intrusion Detection Problems
'We have lost access to multiple critical data stores because Google has an automated threat detection system that is incapable of handling false positives.'
google  security  cloud  false-positives  intrusion-detection  automation  fail 
august 2016 by jm
IMDB on automation, pt 2
Quotable: "how long can work on making a routine task more efficient before you're spending more time than you save?"
quotes  time  automation  hacks  life  imdb  productivity  efficiency 
july 2016 by jm
IMDB on automation
quotable: "I spend a lot of time on this task. I should write a program automating it!"
ifttt  quotes  automation  coding  hacks  reality 
july 2016 by jm
The tyranny of the algorithm yet again...
Paypal will no longer handle payments if the user's address includes the word "Isis":
That these place names exist won't be a surprise to anyone familiar with English limnology - the study of rivers and inland waters. As Wikipedia helpfully tells us, "The Isis is the name given to the part of the River Thames above Iffley Lock which flows through the university city of Oxford". In at least one local primary school I'm familiar with, the classes are called Windrush, Cherwell, Isis and Thames.

[...] Now PayPal has decided that they are not prepared to facilitate payments for goods to be delivered to an address which includes the word "Isis". An Isis street resident ran into some unexpected difficulties when attempting to purchase a small quantity of haberdashery on the internet with the aid of a PayPal account. The transaction would not process. In puzzlement she eventually got irritated enough to brave the 24/7 customer support telephone tag labyrinth. The short version of the response from the eventual real person she managed to get through to was that PayPal have blacklisted addresses which include the name "Isis". They will not process payments for goods to be delivered to an Isis related address, whatever state of privileged respectability the residents of such properties may have earned or inherited in their lifetimes to this point.


One has to wonder if this also brings the risk of adding the user to a secret list, somewhere. Trial by algorithm.
isis  algorithms  automation  fail  law-enforcement  paypal  uk  rivers 
june 2016 by jm
Revealed: How copyright law is being misused to remove material from the internet
Automated DMCA takedowns used to fraudulently censor online content.
In fact, no copyright infringement had occurred at all. Instead, something weirder had happened. At some point after Narey posted her comments on Mumsnet, someone had copied the entire text of one of her posts and pasted it, verbatim, to a spammy blog titled “Home Improvement Tips and Tricks”. The post, headlined “Buildteam interior designers” was backdated to September 14 2015, three months before Narey had written it, and was signed by a “Douglas Bush” of South Bend, Indiana. The website was registered to someone quite different, though: Muhammed Ashraf, from Faisalabad, Pakistan.

Quite why Douglas Bush or Muhammed Ashraf would be reviewing a builder based in Clapham is not explained in “his” post. BuildTeam says it has no idea why Narey’s review was reposted, but that it had nothing to do with it. “At no material times have we any knowledge of why this false DCMA take down was filed, nor have we contracted any reputation management firms, or any individual or a group to take such action on our behalf. Finally, and in conjunction to the above, we have never spoken with a ‘Douglas Bush,’ or a ‘Muhammed Ashraf.’”
fraud  censorship  mumsnet  dmca  takedowns  google  automation  copyright 
may 2016 by jm
Jenkins 2.0
built-in support for CI/CD deployment pipelines, driven from a checked-in DSL file. great stuff, very glad to see them going this direction. (via Eric)
via:eric  jenkins  ci  cd  deployment  pipelines  testing  automation  build 
march 2016 by jm
Diffy: Testing services without writing tests
Play requests against 2 versions of a service. A fair bit more complex than simply replaying logged requests, which took 10 lines of a shell script last time I did it
http  testing  thrift  automation  twitter  diffy  diff  soa  tests 
september 2015 by jm
One Direction Offers Remix Competition, Then Sony/Soundcloud Punish The Entrants As Copyright Infringers | Techdirt
TorrentFreak has the story of a UK-producer and songwriter named Lee Adams who took part in an official remix competition of boy band One Direction's music, put on by the band and its label, Sony Music. The stems for remixing were released on Soundcloud. The rules of the contest required entrants to upload their remixes on Soundcloud... and that's exactly what Adams did. And yet those works still got taken down via copyright claims from Sony Music as infringing.
sony  soundcloud  anti-piracy  automation  piracy  stems  remixing  one-direction  lee-adams 
july 2015 by jm
President's message gets lost in (automated) translation
In a series of bizarre translations, YouTube’s automated translation service took artistic licence with the [President's] words of warmth.

When the head of state sent St Patrick’s Day greetings to viewers, the video sharing site said US comedian Tina Fey was being “particular with me head”. As President Higgins spoke of his admiration for Irish emigrants starting new communities abroad, YouTube said the President referenced blackjack and how he “just couldn’t put the new iPhone” down. And, in perhaps the most unusual moment, as he talked of people whose hearts have sympathy, the President “explained” he was once on a show “that will bar a gift card”.


(via Daragh O'Brien)
lol  president  ireland  michael-d-higgins  automation  translation  machine-learning  via:daraghobrien  funny  blackjack  iphone  tina-fey  st-patrick  fail 
march 2015 by jm
Automating Tinder with Eigenfaces
While my friends were getting sucked into "swiping" all day on their phones with Tinder, I eventually got fed up and designed a piece of software that automates everything on Tinder.


This is awesome. (via waxy)
via:waxy  tinder  eigenfaces  machine-learning  k-nearest-neighbour  algorithms  automation  ai 
february 2015 by jm
Amazon sellers hit by nightmare before Christmas as glitch cuts prices to 1p | Technology | The Guardian
From 7-8pm on Friday, [RepricerExpress] software, used by third-party sellers to ensure their products are the cheapest on the market, went a bit haywire and reduced prices to as little as 1p.
1p  amazon  resellers  repricer-express  fail  price-cutting  automation  risks  undercutting 
december 2014 by jm
Should Airplanes Be Flying Themselves?
Excellent Vanity Fair article on the AF447 disaster, covering pilots' team-leadership skills, Clipper Skippers, Alternate Law, and autopilot design: 'There is an old truth in aviation that the reasons you get into trouble become the reasons you don’t get out of it.'

Also interesting:

'The best pilots discard the [autopilot] automation naturally when it becomes unhelpful, and again there appear to be some cultural traits involved. Simulator studies have shown that Irish pilots, for instance, will gleefully throw away their crutches, while Asian pilots will hang on tightly. It’s obvious that the Irish are right, but in the real world Sarter’s advice is hard to sell. The automation is simply too compelling. The operational benefits outweigh the costs. The trend is toward more of it, not less. And after throwing away their crutches, many pilots today would lack the wherewithal to walk.'

(via Gavin Sheridan)
airlines  automation  flight  flying  accidents  post-mortems  af447  air-france  autopilot  alerts  pilots  team-leaders  clipper-skippers  alternate-law 
november 2014 by jm
Pillar
Manages migrations for your Cassandra data stores. Pillar grew from a desire to automatically manage Cassandra schema as code. Managing schema as code enables automated build and deployment, a foundational practice for an organization striving to achieve Continuous Delivery.

Pillar is to Cassandra what Rails ActiveRecord migrations or Play Evolutions are to relational databases with one key difference: Pillar is completely independent from any application development framework.
migrations  database  ops  pillar  cassandra  activerecord  scala  continuous-delivery  automation  build 
june 2014 by jm
fcron
Fcron is a scheduler. It aims at replacing Vixie Cron, so it implements most of its functionalities. But contrary to Vixie Cron, fcron does not need your system to be up 7 days a week, 24 hours a day : it also works well with systems which are running only occasionnally (contrary to anacrontab). In other words, fcron does both the job of Vixie Cron and anacron, but does even more and better :)) ...


Thanks Craig!
via:chughes  cron  fcron  unix  linux  ops  scheduler  automation  scripts 
april 2014 by jm
Huginn
a system for building agents that perform automated tasks for you online. They can read the web, watch for events, and take actions on your behalf. Huginn's Agents create and consume events, propagating them along a directed event flow graph. Think of it as Yahoo! Pipes plus IFTTT on your own server. You always know who has your data. You do.


MIT-licensed open source, built on Rails.
ifttt  automation  huginn  ruby  rails  open-source  agents 
april 2014 by jm
Google: Our Robot Cars Are Better Drivers Than Puny Humans | MIT Technology Review
One of those analyses showed that when a human was behind the wheel, Google’s cars accelerated and braked significantly more sharply than they did when piloting themselves. Another showed that the cars’ software was much better at maintaining a safe distance from the vehicle ahead than the human drivers were. “We’re spending less time in near-collision states,” said Urmson. “Our car is driving more smoothly and more safely than our trained professional drivers.”
google  cars  driving  safety  roads  humans  robots  automation 
october 2013 by jm
Juniper Adds Puppet support
This is super-cool.

'Network engineering no longer should be mundane tasks like conf, set interfaces fe-0/0/0 unit o family inet address 10.1.1.1/24. How does mindless CLI work translate to efficiently spent time ? What if you need to change 300 devices? What if you are writing it by hand? An error-prone waste of time. Juniper today announced Puppet support for their 12.2R3,5 JUNOS code. This is compatible with EX4200, EX4550, and QFX3500 switches. These are top end switches, but this start is directly aimed at their DC and enterprise devices. Initially, the manifest interactions offered are interface, layer 2 interface, vlan, port aggregation groups, and device names.'

Based on what I saw in the Network Automation team in Amazon, this is an amazing leap forward; it'd instantly render obsolete a bunch of horrific SSH-CLI automation cruft.
ssh  cli  automation  networking  networks  puppet  ops  juniper  cisco 
august 2013 by jm
When 'Smart Homes' Get Hacked: I Haunted A Complete Stranger's House Via The Internet - Forbes
Hardware designers do their usual trick -- omit the whole security part:
[Trustwave's Crowley] found security flaws that would allow a digital intruder to take control of a number of sensitive devices beyond the Insteon systems, from the Belkin WeMo Switch to the Satis Smart Toilet. Yes, they found that a toilet was hackable. You only have to have the Android app for the $5,000 toilet on your phone and be close enough to the toilet to communicate with it. “It connects through Bluetooth, with no username or password using the pin ‘0000’,” said Crowley. “So anyone who has the application on their phone and was connected to the network could control anyone else’s toilet. You could turn the bidet on while someone’s in there.”
home  automation  insteon  security  hardware  fail  attacks  bluetooth  han  trustwave  belkin  satis 
july 2013 by jm
Testing Your Automation [slides]
Test-driven infrastructure, using Chef -- slides from Big Ruby 2013. Tools used: foodcritic (lol), Chefspec, minitest-chef-handler, fauxhai, cucumber chef. This is really good to see -- TDD applied to ops. Video at: http://confreaks.com/videos/2309-bigruby2013-testing-your-automation-ttd-for-chef-cookbooks
devops  ops  chef  automation  testing  tdd  infrastructure  provisioning  deployment 
april 2013 by jm
KDE's brush with git repository corruption: post-mortem
a barely-averted disaster... phew.

while we planned for the case of the server losing a disk or entirely biting the dust, or the total loss of the VM’s filesystem, we didn’t plan for the case of filesystem corruption, and the way the corruption affected our mirroring system triggered some very unforeseen and pathological conditions. [...] the corruption was perfectly mirrored... or rather, due to its nature, imperfectly mirrored. And all data on the anongit [mirrors] was lost.

One risk demonstrated: by trusting in mirroring, rather than a schedule of snapshot backups covering a wide time range, they nearly had a major outage. Silent data corruption, and code bugs, happen -- backups protect against this, but RAID, replication, and mirrors do not.

Another risk: they didn't have a rate limit on project-deletion, which resulted in the "anongit" mirrors deleting their (safe) data copies in response to the upstream corruption. Rate limiting to sanity-check automated changes is vital. What they should have had in place was described by the fix: 'If a new projects file is generated and is more than 1% different than the previous file, the previous file is kept intact (at 1500 repositories, that means 15 repositories would have to be created or deleted in the span of three minutes, which is extremely unlikely).'
rate-limiting  case-studies  post-mortems  kde  git  data-corruption  risks  mirroring  replication  raid  bugs  backups  snapshots  sanity-checks  automation  ops 
march 2013 by jm
OmniTI's Experiences Adopting Chef
A good, in-depth writeup of OmniTI's best practices with respect to build-out of multiple customer deployments, using multi-tenant Chef from a version-controlled repo. Good suggestions, and I am really looking forward to this bit:

'Chef tries to turn your system configuration into code. That means you now inherit all the woes of software engineering: making changes in a coordinated manner and ensuring that changes integrate well are now an even greater concern. In part three of this series, we’ll look at applying software quality assurance and release management practices to Chef cookbooks and roles.'
chef  deployment  ops  omniti  systems  vagrant  automation 
january 2013 by jm
Shell Scripts Are Like Gremlins
Shell Scripts are like Gremlins. You start out with one adorably cute shell script. You commented it and it does one thing really well. It’s easy to read, everyone can use it. It’s awesome! Then you accidentally spill some water on it, or feed it late one night and omgwtf is happening!?


+1. I have to wean myself off the habit of automating with shell scripts where a clean, well-unit-tested piece of code would work better.
shell-scripts  scripting  coding  automation  sysadmin  devops  chef  deployment 
december 2012 by jm
What can data scientists learn from DevOps?
Interesting.

'Rather than continuing to pretend analysis is a one-time, ad hoc action, automate it. [...] you need to maintain the automation machinery, but a cost-benefit analysis will show that the effort rapidly pays off — particularly for complex actions such as analysis that are nontrivial to get right.' (via @fintanr)
via:fintanr  data-science  data  automation  devops  analytics  analysis 
november 2012 by jm
High-frequency trading: The fast and the furious | The Economist

"The NYMEX panel found that Infinium had finished writing the algorithm only the day before it introduced it to the market, and had tested it for only a couple of hours in a simulated trading environment to see how it would perform. The firm's normal testing processes take six to eight weeks. When the algorithm started its frenetic buying spree, the measures designed to shut it down automatically did not work. One was supposed to turn the system off if a maximum order size was breached, but because the machine was placing lots of small orders rather than a single big one the shutdown was not triggered. The other measure was meant to prevent Infinium from selling or buying more than a certain number of contracts, but because of an error in the way the rogue algorithm had been written, this, too, failed to spot a problem."
hft  automation  trading  markets  stocks  nymex  bugs  software 
august 2012 by jm
Air France 447 Flight-Data Recorder Transcript - What Really Happened Aboard Air France 447 - Popular Mechanics
The (comp.)risks of overautomation strike again. "When trouble suddenly springs up and the computer decides that it can no longer cope—on a dark night, perhaps, in turbulence, far from land -- the humans might find themselves with a very incomplete notion of what's going on. They'll wonder: What instruments are reliable, and which can't be trusted?"
aviation  crash  flight  flying  autopilot  stalls  warnings  alarms  ui  af447  risks  automation 
december 2011 by jm
Gerrit, Git and Jenkins
This is the future of code review. Commit directly from your git checkout to the Gerrit code-review system; change is immediately web-visible and enters the review workflow; at the same time, Jenkins checks out the proposed change and runs the test suite; once it's approved, it automatically gets checked in. Brilliant!
git  coding  code-review  workflows  jenkins  gerrit  c-i  testing  automation  from delicious
february 2011 by jm
Boxee TV Show Download Automation
organise downloaded TV shows into the directory format Boxee (and by extension, XBMC) wants; some votes for Sickbeard here
tv  torrents  downloading  boxee  xbmc  automation  from delicious
january 2011 by jm
Lone Sale of $4.1 Billion in Contracts Led to ‘Flash Crash’ in May
'as the computers of the high-frequency traders traded contracts back and forth, a “hot potato” effect was created, the report said, as contracts changed hands 27,000 times in 14 seconds, but with eventually only 200 actually being bought or sold.' upshot: horrifically complex distributed feedback loops now directly impact our economies -- great :(
distributed-systems  distcomp  flash-crash  stock-market  trading  automation  via:nelson  sec  nyse  high-frequency-trading  from delicious
october 2010 by jm
FlexGet
torrent automation from RSS feeds; will work nicely with Transmission
bittorrent  automation  boxee  linux  python  rss  torrents  tv  flexget  from delicious
july 2010 by jm

related tags

1p  accidents  activerecord  af447  agents  ai  air-france  airlines  alarms  alerts  algorithms  alternate-law  amazon  analysis  analytics  anti-piracy  asylum  attacks  automation  autopilot  aviation  backups  belkin  bittorrent  blackjack  bluetooth  bots  boxee  broken  bugs  build  c-i  cars  case-studies  cassandra  cd  censorship  chef  ci  cisco  cli  clipper-skippers  cloud  code-review  coding  complaints  computer-says-no  continuous-delivery  copyright  crash  cron  data  data-corruption  data-science  database  daylight-savings-time  deployment  detroit  devops  diff  diffy  discrimination  distcomp  distributed-systems  dmca  downloading  driving  dst  efficiency  eigenfaces  fabric  facebook  fail  false-positives  fcron  flash-crash  flexget  flight  flying  forms  fraud  fraud-detection  funny  gerrit  git  google  government  hacks  han  hardware  heathrow  hft  high-frequency-trading  home  http  huginn  humans  ifttt  imdb  infrastructure  insteon  intrusion-detection  iphone  ireland  isis  jenkins  juniper  k-nearest-neighbour  kde  law-enforcement  lee-adams  life  linux  lol  machine-learning  markets  michael-d-higgins  michigan  migrations  mirroring  mumsnet  networking  networks  noise  nymex  nyse  omniti  one-direction  open-source  ops  paypal  pillar  pilots  pipelines  piracy  post-mortems  president  price-cutting  productivity  provisioning  pty  puppet  python  quotes  raid  rails  rate-limiting  reality  remixing  replication  repricer-express  resellers  risks  rivers  roads  robots  rss  ruby  safety  sanity-checks  satis  scala  scheduler  scripting  scripts  sec  security  shell-scripts  snapshots  soa  social-welfare  software  sony  soundcloud  ssh  st-patrick  stalls  stems  stock-market  stocks  sysadmin  systems  takedowns  tdd  team-leaders  technology  testing  tests  thrift  time  tina-fey  tinder  torrents  trading  translation  trustwave  tv  twitter  ui  uk  undercutting  unix  us-politics  vagrant  via:chughes  via:daraghobrien  via:eric  via:fintanr  via:nelson  via:waxy  warnings  workflows  xbmc 

Copy this bookmark:



description:


tags: