jm + twitter   147

Welcome To The Dystopia: People Are Arguing Whether This Trump Press Conference Video Is Doctored Or Not
To sum it up: A historically unreliable narrator who works for a conspiracy website tweets out a video in order to show alleged bad behavior on the part of a journalist. The clip goes viral. The White House picks up and disseminates that video and uses it as proof to ban the journalist from reporting at the White House. Outraged journalists decry the White House's use of a video taken from a historically unreliable narrator. Then, users attempt to debunk the video as "actual fake news." Others, unclear if the video is fake, urge caution, suggesting the media may be jumping the gun. An argument breaks out over the intricate technical details of doctoring a clip.

The entire ordeal is a near perfect example of a scenario disinformation experts have predicted and warned of, where the very threat of video manipulation can lead to a blurring of reality. "These technological underpinnings [of AI and photoshop, and editing programs lead] to the increasing erosion of trust,” computational propaganda researcher Renee DiResta told BuzzFeed News in early 2018. “It makes it possible to cast aspersions on whether videos — or advocacy for that matter — are real.”
twitter  disinfo  disinformation  buzzfeed  video  reality  fake-news  photoshop 
4 days ago by jm
cool twitter image compression hack
Dаvіd Вucһаnаn on Twitter:
Assuming this all works out, the image in this tweet is also a valid ZIP archive, containing a multipart RAR archive, containing the complete works of Shakespeare. This technique also survives twitter's thumbnailer :P


(via JK)
via:johnke  cool  compression  image  jpeg  twitter 
12 days ago by jm
Epic twitter thread from @colmmacc explaining why client certs and mutual-auth TLS are TERRIBAD
Ok. tweet thread time! Too long ago I promised to write a screed explaining how much I hated mutual-auth TLS and why. I got distracted, and I wasn't happy with the writing, so here it is in tweet thread form instead! But basically: Client certs and Mutual-Auth TLS is TERRIBAD.
When I say TERRIBAD, I mean that unless you've got the resources of a big security dept and folks who comb threat models for a living, using clients certs and mutual auth probably materially lessens your security. That's NUTS!


(source: https://twitter.com/colmmacc/status/1057017343438540801 )
terribad  rants  twitter  threads  tls  ssl  authentication  mtls  security 
13 days ago by jm
Saudi Arabian spy worked as a Twitter SRE
and Twitter didn't find out until the CIA told them. The spy who had root
root  spies  twitter  saudi-arabia  privacy  espionage  cia 
21 days ago by jm
Excellent Twitter thread from colmmacc on how s2n avoids protocol-state errors
using a linearized set of state transitions, and Cryptol and SAW to perform verification of the TLS state machine
cryptol  saw  formal-verification  twitter  threads  colmmacc  security  s2n 
24 days ago by jm
Muting some magic keywords fixes the Twitter timeline
Apparently, turning off some of the shittier recent features:
Muting suggest_recycled_tweet_inline and suggest_activity_tweet actually has fixed my timeline. It's all chronological and there are barely any "x and y liked" tweets.
twitter  feature-creep  muting  hacks 
7 weeks ago by jm
'The Internet of Garbage' by Sarah Jeong
Sarah Jeong's 2015 book is now free:

'I think The Internet of Garbage still provides a useful framework to begin to
talk about our new dystopia, and it continues to be surprisingly relevant in many
ways. But I wrote the book with a tone of optimism I did not feel even at the time,
hoping that by reaching the well-meaning policy teams across Silicon Valley, I
might be able to spark change for the better.
Not only did that change never quite solidify, but the coordinated,
orchestrated harassment campaigns of Gamergate that I very briefly touch on in
Chapter Two have since overtaken our national political and cultural
conversations. These twisted knots of lies, deflection, and rage are not just some
weird and terrible online garbage. They shadow executive orders, court rulings,
even the newly appointed judiciary. They will haunt us for years to come. We are
all victims of fraud in the marketplace of ideas.
I hope that in the very near future, I will be putting out a second edition of
The Internet of Garbage. In that future edition, I hope to grapple with advertising
incentives, engagement traps, international propaganda wars, the American crisis
in free speech coinciding with the rise of platform power, and search engine
optimization as the new paradigm of speech.
In the meantime, I am putting out The Internet of Garbage 1.5 as an interim
edition. I wish it were more helpful in our present reality. But as imperfect a tool
as it is, I figure we all need as much help as we can get. '
dystopia  fake-news  internet  spam  harrassment  abuse  twitter  gamergate  politics  books  free  to-read 
9 weeks ago by jm
Mastodon and the challenges of abuse in a federated system
Similar to this thread by CJ Silverio, I’m not thinking about this in terms of whether Wil Wheaton or his detractors were right or wrong. Rather, I’m thinking about how this incident demonstrates that a large-scale harassment attack by motivated actors is not only possible in the fediverse, but is arguably easier than in a centralized system like Twitter or Facebook, where automated tools can help moderators to catch dogpiling as it happens.

As someone who both administrates and moderates Mastodon instances, and who believes in Mastodon’s mission to make social media a more pleasant and human-centric place, this post is my attempt to define the attack vector and propose strategies to prevent it in the future.
mastodon  abuse  twitter  wilw  harassment  moderation 
10 weeks ago by jm
Cory Doctorow: Zuck’s Empire of Oily Rags
the sophisticated targeting systems available through Facebook, Google, Twitter, and other Big Tech ad platforms made it easy to find the racist, xenophobic, fearful, angry people who wanted to believe that foreigners were destroying their country while being bankrolled by George Soros.

Remember that elections are generally knife-edge affairs, even for politicians who’ve held their seats for decades with slim margins: 60% of the vote is an excellent win. Remember, too, that the winner in most races is “none of the above,” with huge numbers of voters sitting out the election. If even a small number of these non-voters can be motivated to show up at the polls, safe seats can be made contestable. In a tight race, having a cheap way to reach all the latent Klansmen in a district and quietly inform them that Donald J. Trump is their man is a game-changer.

Cambridge Analytica are like stage mentalists: they’re doing something labor-intensive and pretending that it’s something supernatural. A stage mentalist will train for years to learn to quickly memorize a deck of cards and then claim that they can name your card thanks to their psychic powers. You never see the unglamorous, unimpressive memorization practice. Cambridge Analytica uses Facebook to find racist jerks and tell them to vote for Trump and then they claim that they’ve discovered a mystical way to get otherwise sensible people to vote for maniacs.
facebook  politics  surveillance  cory-doctorow  google  twitter  advertising  elections  cambridge-analytica  racism  nazis 
july 2018 by jm
Twitter thread regarding GDPR-compliance for append-only logs/event sourcing systems
Martin Kleppmann: "What’s current best practice for GDPR compliance (in particular, right to deletion) in systems with append-only logs/event sourcing/blockchains, which are supposed to keep history forever?"

Ben Kehoe: "Crypto delete. The immutable store keeps an encrypted copy, and the key is stored elsewhere. Forget me = throw away the key".

That seems to be the most practical suggestion in general in this thread.
twitter  threads  gdpr  compliance  law  eu  append-only  logs  blockchain  event-sourcing  architecture  storage  kafka  kinesis 
june 2018 by jm
How Ireland Beat Dark Ads – Foreign Policy
In practice, while these recognizable attempts to disrupt the democratic debate with microtargeted ads, bot activity, and misinformation were active, they appear to have been relatively ineffective and may even have turned voters away from those employing them.

Given the battleground online discourse has become in democracies across the world, this small country’s resistance to it may offer some cause for hope. The resilience offered by the small size and close-knit nature of the Irish electorate may be difficult to reproduce in larger democracies. But the active measures taken by media, volunteer groups, and campaigners against potentially corrosive techniques can be a powerful inspiration.


+1 -- it's heartening that we were able to defeat these 21st century dirty tricks after the damage they did with Trump and Brexit.
brexit  elections  trump  fake-news  propaganda  bots  dark-ads  facebook  social-media  repealthe8th  referenda  abortion  ireland  repeal-shield  twitter 
june 2018 by jm
"Tweeps! What’s the craziest infra incident you worked on at Twitter"
great thread of Twitter outages and production incidents. I would love to hear more details about these, I love hearing about other people's outages ;) Even reading "over a month of cleanup and some permanent data loss" has me sweating....
infrastructure  engineering  twitter  ops  outages  production 
april 2018 by jm
How to report graphic abortion imagery to the gardai under Irish law
I tried to report ICBR graphic abortion imagery to the Gardai today and met a lot of resistance. The following thread gives an account of what happened and how someone can effectively report this imagery. 1/x
At 2pm on Friday the 13th of April I noticed the presence of ICBR graphic abortion imagery being displayed outside the Nassau street entrance of Trinity. I called Kevin Street Garda Station in order to make a complaint under Section 7 of the Public Order Act 1994 2/x
I was told that the Gardai had been instructed by their superiors to not intervene with such imagery and that this direction had come from the Refendum Commission itself. I then called the Refendum Commission in order to query this, as they'd never been involved previously. 3/x
A representative from the commission informed me that no such direction had been given to the Gardai as it is not in the commission's remit to influence such imagery. The representative told me that they would contact with Kevin Street Station about this miscommunication. 4/x
I then rang Kevin Street Station again to inform them of what I had been told by the Refendum Commission. I was then told that a complaint had to be made in person to either a Garda on the scene or to a local station (Trinity would be Pearse Street), which is understandable. 5/x
I informed the Gardai of a similar experience in Dundrum in which the local station had dispatched officers to move along those displaying the imagery to prevent a breach of the peace without a complaint being made in person. 6/x
I was finally told that Pearse Street Station would be contacted to have an available car dispatched to Trinity. 8/x
TLDR: If you see this imagery, report it under Section 7. If you are told that the Gardai cannot intervene, let them know that other stations have before. If they say they have been directed by the Referendum Commission, let them know there is no such directive on record. 9/x
I hope this miscommunication can be cleared up and that both @gardainfo and @RefCom_ie end up on the same page, so that Gardai can continue to do their jobs effectively and respond to public complaints of breach of the peace. 10/10


Very illuminating.
twitter  threads  abortion  propaganda  gardai  law  ireland  public-order-act 
april 2018 by jm
Timeline behind the #CopOnComrades controversy
A very exhaustive timeline of the online defence of feminism against a few left-wing men in Ireland, courtesy of Andrew Flood.
coponcomrades  left-wing  politics  ireland  twitter  facebook  irish-times 
march 2018 by jm
The 29 Stages Of A Twitterstorm In 2018
'14. Then suddenly there are Nazis everywhere.'
twitter  twitterstorms  funny  2018  nazis  alt-right  memes 
january 2018 by jm
Sarah Jeong's hilarious Twitter thread on Bitcoin
"People are sick of the Federal Reserve, sick of bailouts, sick of inflation. You know what we need? Internet money with the usability of PGP and the reliability of BART" and much, much more
bitcoin  funny  sarah-jeong  comedy  lols  pgp  twitter  threads 
january 2018 by jm
Meet the man who deactivated Trump’s Twitter account
Legend!
His last day at Twitter was mostly uneventful, he says. There were many goodbyes, and he worked up until the last hour before his computer access was to be shut off. Near the end of his shift, the fateful alert came in. This is where Trump’s behavior intersects with Duysak’s work life. Someone reported Trump’s account on Duysak’s last day; as a final, throwaway gesture, he put the wheels in motion to deactivate it. Then he closed his computer and left the building.
twitter  trump  bahtiyar-duysak  abuse  reporting  funny 
november 2017 by jm
Tech Leaders Dismayed by Weaponization of Social Media - IEEE Spectrum
“We have passed the fail-safe point,” McNamee said. “I don’t think we can get back to the Silicon Valley that I loved. At this point we just have to save America.”
propaganda  fake-news  facebook  twitter  social-media  us-politics  brexit  internet  russia  silicon-valley  usa 
november 2017 by jm
Rich "Lowtax" Kyanka on Twitter's abuse/troll problem
how did you solve this problem at Something Awful? You said you wrote a bunch of rules but internet pedants will always find ways to get around them.

The last rule says we can ban you for any reason. It's like the catch-all. We can ban you if it's too hot in the room, we can ban you if we had a bad day, we can ban you if our finger slips and hits the ban button. And that way people know that if they're doing something and it's not technically breaking any rules but they're obviously trying to push shit as far as they can, we can still ban them. But, unlike Twitter, we actually have what's called the Leper's Colony, which says what they did and has their track record. Twitter just says, “You're gone.”
twitter  communication  discussion  history  somethingawful  lowtax 
november 2017 by jm
Yonatan Zunger's twitter thread on Twitter's problem with policy issues
'I worked on policy issues at G+ and YT for years. It was *painfully* obvious that Twitter never took them seriously.'

This thread is full of good information on "free speech", nazis, Trump, Gamergate and Twitter's harrassment problem.

(Via Peter Bourgon)
via:peterbourgon  harrassment  twitter  gamergate  threads  youtube  google-plus  policy  abuse  bullying  free-speech  engagement 
october 2017 by jm
One person’s history of Twitter, from beginning to end – Mike Monteiro
Twitter, which was conceived and built by a room of privileged white boys (some of them my friends!), never considered the possibility that they were building a bomb. To this day, Jack Dorsey doesn’t realize the size of the bomb he’s sitting on. Or if he does, he believes it’s metaphorical. It’s not. He is utterly unprepared for the burden he’s found himself responsible for.
The power of Oppenheimer-wide destruction is in the hands of entitled men-children, cuddled runts, who aim not to enhance human communication, but to build themselves a digital substitute for physical contact with members of the species who were unlike them. And it should scare you.
politics  twitter  mike-monteiro  history  silicon-valley  trump 
october 2017 by jm
@tttthreads
Unroll a long twitter thread with a single tweet. I like it
unroll  threads  twitter  reading  ux  bots 
september 2017 by jm
John Gallagher on Twitter: "Lot of misinformation about this today. For clarity: I live in Britain and mince on toast is served at every meal"
Eater.com posts comically misinformed video about some kind of imaginary brit comfort food. John Gallagher's response thread is a virtuoso performance
mince-on-toast  disgusting  food  funny  wtf  twitter 
july 2017 by jm
serviette/serviette.py at master · heathervm/serviette · GitHub
Delete tweets based on search terms. Wonder why you'd want that
twitter  tweets  delete  privacy  social-media 
april 2017 by jm
Watching the hearings, I learned my "Bernie bro" harassers may have been Russian bots
However, the rest of the abuse came from accounts purporting to be supporters of Vermont Independent Senator Bernie Sanders. And these were “people” with whom I believed I shared common values and policy interests. Almost all of the accounts presented as men — mostly young and white — and used sexist and misogynistic tones and words. I was called “mom” and “grandma” as epithets by these “young men.” I was called every vile sexualized name you can imagine. For some reason that I did not understand at the time, they liked to call me a “vagina.” (I now believe non-native English — i.e. Russian — speakers wrote the algorithms controlling these bots and perhaps imagined “vagina” to be the equivalent of the c-word when hurled at a woman.) Not being conversant in the mechanisms of Russian psychological warfare techniques at the time, it never occurred to me that, like the #MAGA bots, these “Bernie Bro” accounts were actually bots too.
And the abuse from these accounts was much harder to dismiss. It went in further, emotionally speaking. The vitriol of the attacks felt like a painful betrayal. After all, “we” probably shared 99 percent of our political perspective; we just supported different candidates — which is something I said repeatedly in my attempts to appeal to reason with some of the attackers over the course of those long months. Nonetheless, even the mildest criticism of Sanders or comment of support for Clinton would bring out a swarm of these “Bernie Bro” accounts spouting off with abusive language and mockery.
bernie-bros  abuse  twitter  russia  security  bots  elections  hilary-clinton 
april 2017 by jm
UW professor: The information war is real, and we’re losing it
Starbird sighed. “I used to be a techno-utopian. Now I can’t believe that I’m sitting here talking to you about all this.”


Yep :(
journalism  media  news  fake-news  infowars  twitter  facebook 
march 2017 by jm
US grand jury decides that a GIF counts as a deadly weapon in Twitter seizure case - The Verge
While this is certainly correct to prosecute, I still think that Twitter need to rethink their UI choices that allow a random stranger to fire graphic images at their targets without any opt-in requirement.
FBI investigators seized the account late last year after tracking him through his iPhone, and allege that Rivello sent several tweets and messages about his intentions to cause Eichenwald to have a seizure — including the text “You deserve a seizure for your post.” According to NBC News, other messages specifically say that "I hope this sends him into a seizure,” while others read "Spammed this at [Eichenwald] let's see if he dies."
images  twitter  epilepsy  gifs  nam-shub  abuse  social-media  trolls 
march 2017 by jm
Colm O'Gorman, on societal responsibility for Mother & Baby Homes, Magdalene Laundries & various other church atrocities in Ireland
Excellent twitter thread on the topic. Pasted:

It is often said that everyone knew what was happening in such places, or about the rape of children by priests. That is not true.
It is true that deep veins of knowledge existed across Irish society, at all levels, but not everyone knew. Or were allowed to know.
Just like is always the case, the terrible things that were done were possible only because they were tolerated. They went unchecked.
They were tolerated by those in positions of authority who either dared not, or did not wish to, challenge the power strictures that existed
They were tolerated by those without power or position because they feared what speaking up might do to them and to their families
That was an Ireland where challenging such vile abuse by power would see you become its victim. It was brutal and vicious.
If you did not, or could not, conform to the demands of the powerful, you were in real danger. At best, ostracisation and excommunication.
But many experience far worse than that. They found themselves in the very places we now acknowledge as hell holes. Locked up in institutions
I always remember the late, great Mary Rafferty exposing the scale of such abusive institutionalisation. She pointed out that at one point
in our relatively recent history, we led the world in one regard. Per capita, we locked up more people in psychiatric institutions than
any other country on the planet. Only the Soviet Union came a distant second to us. That was how Ireland treated dissent or difference
That what was happened to many who could not conform to a brutal demand to be somehow 'acceptable' to dogma & unaccountable power
And it wasn't some ancient Ireland either. The last laundry closed in 1996. In 2002, when fighting for inquiries into child rape by priests
and it's cover up by bishops, cardinals and popes, those same princes declared themselves above the rule of the law of this Republic
insisting that the law of their church was superior to the law of this state. And their position was taken seriously by many.
It took months of dogged battle by me and others to get past that bullshit. For our political and legal system to assert itself.
The Ireland where the lives of women & children were controlled & brutalised by people who felt they had a God given right to do so is not
some other country that existed back in some other time. It is this Ireland. We have changed a lot - but it is still this Ireland.
The difference now is that we ALL know. That the truth is out, and that more is being revealed. And yes, undoubtedly there is more to come.
So it is NOT true all past members of society, or even anything close to a majority, colluded with such abuses. That is a falsehood.
It is also a falsehood to suggest that the church did what the state would not do, and provided as best it could. That is a lie.
The Catholic Church captured control of what should have been arms of the state. Health, education and social care. And it exploited them.
It used them to drive its own agendas, to enforce its own dogma. And at every turn it resisted any 'intrusion' into those realms by others.
including the state. Look at the Mother & Child Scheme for eg, or the response to the first multi-denominational schools, and much more.
Catholic orders defended themselves against accusations of appalling abuse of children in their institutions by claiming that
the state did not give them enough money to feed, clothe and properly care for the children they detained in those places. This was a lie.
in the same institutions where children went starving, clergy were well fed and housed. They went for nothing. Funded by the state and the
forced labour of the children or women they detained. The Ryan Report debunked that lie in its entirety.
Ryan found that religious orders maintained "bloated congregations" by bringing in more and more children, and therefore more and more money
And now we know. Now the threat of brutal reprisal is lifted. Now is the time for truth, to own what has been done to so many vulnerable
people in our Republic. To learn from it and ensure we identify how that same corrupting tendency manifests today. Because it does of course
It may not be quite as vicious, but it prevails.Look at how power still treats a reasonable demand for accountability: Maurice McCabe for eg
Look at how our education and health systems still allow religious dogma to exert extraordinary power over people's lives.
We are a different Ireland, but are we different enough?
mother-and-baby-homes  tuam  ireland  catholic-church  abuse  colm-o-gorman  twitter  history  priests 
march 2017 by jm
A Programmer’s Introduction to Unicode – Nathan Reed’s coding blog
Fascinating Unicode details -- a lot of which were new to me. Love the heat map of usage in Wikipedia:
One more interesting way to visualize the codespace is to look at the distribution of usage—in other words, how often each code point is actually used in real-world texts. Below is a heat map of planes 0–2 based on a large sample of text from Wikipedia and Twitter (all languages). Frequency increases from black (never seen) through red and yellow to white.

You can see that the vast majority of this text sample lies in the BMP, with only scattered usage of code points from planes 1–2. The biggest exception is emoji, which show up here as the several bright squares in the bottom row of plane 1.
unicode  coding  character-sets  wikipedia  bmp  emoji  twitter  languages  characters  heat-maps  dataviz 
march 2017 by jm
"I caused an outage" thread on twitter
Anil Dash: "What was the first time you took the website down or broke the build? I’m thinking of all the inadvertent downtime that comes with shipping."

Sample response: 'Pushed a fatal error in lib/display.php to all of FB’s production servers one Friday night in late 2005. Site loaded blank pages for 20min.'
outages  reliability  twitter  downtime  fail  ops  post-mortem 
march 2017 by jm
US immigration asking tech interview trivia questions now
what the absolute fuck. Celestine Omin on Twitter: "I was just asked to balance a Binary Search Tree by JFK's airport immigration. Welcome to America."
twitter  celestine-omin  us-politics  immigration  tests  interviews  bst  trees  data-structures  algorithms 
february 2017 by jm
Why Shopify Payments prohibit sexual content
Interesting background info from a twitter thread:

@jennschiffer Breitbart uses Shopify Payments, which is built on top of Stripe, which is sponsored by Wells Fargo merchant services AFAIK.
WF has underwriting rules that prohibit sexual content. The main reasons aren't b/c WF or Stripe are interested in policing morals.
Historically there's a higher rate of chargebacks from porn sites, which is why banks are generally anti-sexual content.
Imagine someone's partner finds a charge for pornhub on their credit cars and calls them out on it. The person will deny and file a CB.
Once porn sites started getting shut down by banks, they would change their names or submit applications claiming to be fetish sites, etc
So underwriting dept's decided the risk is too high and generally defer to no with anything sexual.
Most processors aren't inclined to challenge this position on moral grounds since there's strong precedent against it...
...and it could jeapordize their entire payments system if they get shut off.
There are exceptions of course and there are other prohibited uses that are allowed to continue.
twitter  porn  shopify  sex  chargebacks  payment 
february 2017 by jm
"Solving Imaginary Scaling Issues At Scale — Getting the wrong idea from that conference talk you attended"
Amazing virtuoso performance:

Chapter 1: Databases with cool-sounding names.
Chapter 2: using BitTorrent for everything.
Chapter 3: forget Torrents. Use the blockchain for everything.
Chapter 4: sharding the database before adding any indexes.
Chapter 5: upgrading to faster processors without checking if you're limited by disk I/O.
Chapter 6: rewriting APIs in C for speed without compressing data on the wire.
Chapter 7: putting large blobs of binary data into SQL databases for fun and profit.
Chapter 8: using protobufs to poll 300 times per second.
Chapter 9: diagnose scaling issues by grepping 10 lines of code and guessing.
Chapter 10: putting Varnish in front of everything just in case.
Chapter 11: buying boxes with gigantic amounts of RAM.
Chapter 12: realizing your HAProxy box is still a micro instance.
Chapter 13: rewriting 3 of 10 features in Go and declaring victory.
Chapter 14: split everything into 35 microservices all maintained by 1 person.
Chapter 15: 300% performance boosts by deleting data validity checks.
Chapter 16: minifying the JS of your O(n^3) to-do list.
Chapter 17: Fuck It, Let's Try Erlang.
Chapter 18: Blaming Everything On The Last Person To Quit.
Chapter 19: A Bloom Filter Will Definitely Fix This.
Chapter 20: Move all client-side processing to the server and/or vice-versa.
Chapter 21: Putting A Node.js Proxy In Front Of Our COBOL Backend Will Definitely Improve Matters.
Chapter 22: A Type-Checked Transpilation Step Will Surely Speed Things Up.
Chapter 23: Writing A New Language Almost The Same As Your Old Language But Faster (guest chapter by Facebook).
Chapter 24: Replacing an SQL DB with a NoSQL DB then implementing SQL in your ORM.
Chapter 25: Migrating From Bare Metal To The Cloud Or Vice-Versa, Whichever You're Not Currently Doing.
Chapter 26: Putting everything behind a CDN except the slow, complicated parts.
Chapter 27: Applying distributed map-reduce to less than 1 gigabyte of data.
Chapter 28: Running exactly the same software, but in Docker.
Chapter 29: Machine learning: how it will magically fix your crappy code.
Chapter 30: Blaming your package manager for slow run-time performance.
Chapter 31: Moving processing from the CPU to the GPU without changing the algorithm.
Chapter 32: Switching To Heroku Or Away From Heroku Or A Hybrid Heroku-AWS model, whichever sounds the most fun.
Chapter 33: Loading all your dependencies from somebody else's github repo.
Chapter 34: optimizing your PNGs while hosting 300MB video ads.
Chapter 35: hosting your database in memory and your images on S3.
scalability  funny  lol  twitter  oreilly 
november 2016 by jm
Anti-Brexit traitors outed on twitter
oh god this is funny. Louise Mensch and various UKIPpers fall for transparent pisstake involving "taking Article 50 out of the ring binder and shredding it. It now goes straight from 49 to 51" etc.
twitter  louise-mensch  funny  idiots  fail  brexit  ukip 
october 2016 by jm
Open Sourcing Twitter Heron
Twitter are open sourcing their Storm replacement, and moving it to an independent open source foundation
open-source  twitter  heron  storm  streaming  architecture  lambda-architecture 
may 2016 by jm
Social Network Algorithms Are Distorting Reality By Boosting Conspiracy Theories | Co.Exist | ideas + impact
In his 1962 book, The Image: A Guide to Pseudo-Events in America, former Librarian of Congress Daniel J. Boorstin describes a world where our ability to technologically shape reality is so sophisticated, it overcomes reality itself. "We risk being the first people in history," he writes, "to have been able to make their illusions so vivid, so persuasive, so ‘realistic’ that they can live in them."
algorithms  facebook  ethics  filtering  newsfeed  conspiracy-theories  twitter  viral  crazy 
may 2016 by jm
Observability at Twitter: technical overview, part II
Interesting to me mainly for this tidbit which makes my own prejudices:
“Pull” vs “push” in metrics collection: At the time of our previous blog post, all our metrics were collected by “pulling” from our collection agents. We discovered two main issues:

* There is no easy way to differentiate service failures from collection agent failures. Service response time out and missed collection request are both manifested as empty time series.
* There is a lack of service quality insulation in our collection pipeline. It is very difficult to set an optimal collection time out for various services. A long collection time from one single service can cause a delay for other services that share the same collection agent.

In light of these issues, we switched our collection model from “pull” to “push” and increased our service isolation. Our collection agent on each host only collects metrics from services running on that specific host. Additionally, each collection agent sends separate collection status tracking metrics in addition to the metrics emitted by the services.

We have seen a significant improvement in collection reliability with these changes. However, as we moved to self service push model, it becomes harder to project the request growth. In order to solve this problem, we plan to implement service quota to address unpredictable/unbounded growth.
pull  push  metrics  tcp  stacks  monitoring  agents  twitter  fault-tolerance 
march 2016 by jm
Pinboard on the Next Economy Conference (with tweets)
Maciej Ceglowski went to an O'Reilly SV-boosterish conference and produced these excellent tweets
twitter  conferences  oreilly  silicon-valley  new-economy  future  lyft  uber  unions  maciej-ceglowski 
november 2015 by jm
How both TCP and Ethernet checksums fail
At Twitter, a team had a unusual failure where corrupt data ended up in memcache. The root cause appears to have been a switch that was corrupting packets. Most packets were being dropped and the throughput was much lower than normal, but some were still making it through. The hypothesis is that occasionally the corrupt packets had valid TCP and Ethernet checksums. One "lucky" packet stored corrupt data in memcache. Even after the switch was replaced, the errors continued until the cache was cleared.


YA occurrence of this bug. When it happens, it tends to _really_ screw things up, because it's so rare -- we had monitoring for this in Amazon, and when it occurred, it overwhelmingly occurred due to host-level kernel/libc/RAM issues rather than stuff in the network. Amazon design principles were to add app-level checksumming throughout, which of course catches the lot.
networking  tcp  ip  twitter  ethernet  checksums  packets  memcached 
october 2015 by jm
Let a 1,000 flowers bloom. Then rip 999 of them out by the roots
The Twitter tech-debt story.
Somewhere along the way someone decided that it would be easier to convert the Birdcage to use Pants which had since learned how to build Scala and to deal with a maven-style layout. However at some point prior Pants been open sourced in throw it over the wall fashion and picked up by a few engineers at other companies, such as Square and Foursquare and moved forward. In the meantime, again because there weren’t enough people who’s job it was to take care of these things, Science was still on the original internally developed version and had in fact evolved independently of the open source version. However by the time we wanted to move Birdcage onto Pants, the open source version had moved ahead so that’s the one the Birdcage folks chose.


(cries)
tech-debt  management  twitter  productivity  engineering  monorepo  build-systems  war-stories  dev 
september 2015 by jm
Diffy: Testing services without writing tests
Play requests against 2 versions of a service. A fair bit more complex than simply replaying logged requests, which took 10 lines of a shell script last time I did it
http  testing  thrift  automation  twitter  diffy  diff  soa  tests 
september 2015 by jm
Improving The Weather On Twitter
lovely open-source dataviz improvement for near-term historical rainfall-radar images
dataviz  weather  rain  rainfall  radar  nws  twitter  bots  graphics  ui 
august 2015 by jm
How to Create RSS Feeds for Twitter
The latest hacky workaround to Twitter's API shortcoming
rss-feeds  feeds  twitter  favorites  api  social-media  workaround  google-script 
july 2015 by jm
That time the Internet sent a SWAT team to my mom's house - Boing Boing
The solution is for social media sites and the police to take threats or jokes about swatting, doxxing, and organized crime seriously. Tweeting about buying a gun and shooting up a school would be taken seriously, and so should the threat of raping, doxxing, swatting or killing someone. Privacy issues and online harassment are directly linked, and online harassment isn’t going anywhere. My fear is that, in reaction to online harassment, laws will be passed that will break down our civil freedoms and rights online, and that more surveillance will be sold to users under the guise of safety. More surveillance, however, would not have helped me or my mother. A platform that takes harassment and threats seriously instead of treating them like jokes would have.
twitter  gamergate  4chan  8chan  privacy  doxxing  swatting  harrassment  threats  social-media  facebook  law  feminism 
july 2015 by jm
Adrian Colyer reviews the Twitter Heron paper
ouch, really sounds like Storm didn't cut the muster. 'It’s hard to imagine something more damaging to Apache Storm than this. Having read it through, I’m left with the impression that the paper might as well have been titled “Why Storm Sucks”, which coming from Twitter themselves is quite a statement.'

If I was to summarise the lessons learned, it sounds like: backpressure is required; and multi-tenant architectures suck.

Update: response from Storm dev ptgoetz here: http://blog.acolyer.org/2015/06/15/twitter-heron-stream-processing-at-scale/#comment-1738
storm  twitter  heron  big-data  streaming  realtime  backpressure 
june 2015 by jm
Tim Hunt "jokes" about women scientists. Or not. (with image, tweets) · deborahblum · Storify
'[Tim Hunt] said that while he meant to be ironic, he did think it was hard to collaborate with women because they are too emotional - that he was trying to be honest about the problems.' So much for the "nasty twitter took my jokes seriously" claims then.
twitter  science  misogyny  women  tim-hunt  deborah-blum  journalism 
june 2015 by jm
Twitter ditches Storm
in favour of a proprietary ground-up rewrite called Heron. Reading between the lines it sounds like Storm had problems with latency, reliability, data loss, and supporting back pressure.
analytics  architecture  twitter  storm  heron  backpressure  streaming  realtime  queueing 
june 2015 by jm
The Agency - NYTimes.com
Russia's troll farms. Ladies and gentlemen -- the future
future  abuse  trolls  russia  trolling  politics  social-media  twitter  facebook 
june 2015 by jm
Rob Pike's 5 rules of optimization
these are great. I've run into rule #3 ("fancy algorithms are slow when n is small, and n is usually small") several times...
twitter  rob-pike  via:igrigorik  coding  rules  laws  optimization  performance  algorithms  data-structures  aphorisms 
april 2015 by jm
J. G. Ballard predicted social media in a 1977 essay for Vogue
'In the intro essay to High Rise it says that J G Ballard predicted social media in a 1977 essay for Vogue. Here it is'
j-g-ballard  social-media  twitter  instagram  youtube  future  society  vogue  1977  facebook  media 
april 2015 by jm
Twitter’s new anti-harassment filter
Twitter is calling it a “quality filter,” and it’s been rolling out to verified users running Twitter’s iOS app since last week. It appears to work much like a spam filter, except instead of hiding bots and copy-paste marketers, it screens “threats, offensive language, [and] duplicate content” out of your notifications feed.


via Nelson
via:nelson  harassment  spam  twitter  gamergame  abuse  ml 
april 2015 by jm
Stu Hood and Brian Degenhardt, Scala at Twitter, SF Scala @Twitter 20150217
'Stu Hood and Brian Degenhardt talk about the history of Scala at Twitter, from inception until today, covering 2.10 migration, the original Alex Payne’s presentation from way back, pants, and more. The first five years of Scala at Twitter and the years ahead!'

Very positive indeed on the monorepo concept.
monorepo  talks  scala  sfscala  stu-hood  twitter  pants  history  repos  build  projects  compilation  gradle  maven  sbt 
march 2015 by jm
Twitter's Answers architecture
Twitter's mobile-device analytics service architecture, with Kafka and Storm in full Lambda-Architecture mode
twitter  lambda-architecture  storm  kafka  architecture 
february 2015 by jm
South Korean spymaster had a team posting political comments on Twitter and rigging polls
Mad stuff. The South Korean National Intelligence Service directly interfering in a democratic election by posting fake comments and rigging online polls
web  polls  twitter  social-media  psyops  korea  south-korea  nis  sock-puppets  democracy 
february 2015 by jm
Twitter CEO: 'We suck at dealing with abuse' | The Verge
'We suck at dealing with abuse and trolls on the platform and we've sucked at it for years. It's no secret and the rest of the world talks about it every day. We lose core user after core user by not addressing simple trolling issues that they face every day.
I'm frankly ashamed of how poorly we've dealt with this issue during my tenure as CEO. It's absurd. There's no excuse for it. I take full responsibility for not being more aggressive on this front. It's nobody else's fault but mine, and it's embarrassing.
We're going to start kicking these people off right and left and making sure that when they issue their ridiculous attacks, nobody hears them.
Everybody on the leadership team knows this is vital.'


More like this!
trolls  twitter  gamergate  dickc  abuse  leaks  social-media 
february 2015 by jm
Politwoops
'All deleted tweets from politicians'. Great idea
delete  twitter  politics  politicians  ireland  social-media  news 
january 2015 by jm
Introducing practical and robust anomaly detection in a time series
Twitter open-sources an anomaly-spotting R package:
Early detection of anomalies plays a key role in ensuring high-fidelity data is available to our own product teams and those of our data partners. This package helps us monitor spikes in user engagement on the platform surrounding holidays, major sporting events or during breaking news. Beyond surges in social engagement, exogenic factors – such as bots or spammers – may cause an anomaly in number of favorites or followers. The package can be used to find such bots or spam, as well as detect anomalies in system metrics after a new software release. We’re open-sourcing AnomalyDetection because we’d like the public community to evolve the package and learn from it as we have.
statistics  twitter  r  anomaly-detection  outliers  metrics  time-series  spikes  holt-winters 
january 2015 by jm
Building a complete Tweet index
Twitter's new massive-scale twitter search backend. Sharding galore
architecture  search  twitter  sharding  earlybird 
november 2014 by jm
#Gamergate Trolls Aren't Ethics Crusaders; They're a Hate Group
#Gamergate, as they have treated myself and peers in our industry, is a hate group. This word, again, should not lend them any mystique or credence. Rather it should illuminate the fact that even the most nebulous and inconsistent ideas can proliferate wildly if strung onto the organizational framework of the hate group, which additionally gains a startling amount of power online. #Gamergate is a hate group, and they are all the more dismissible for it. And the longer we treat them otherwise, the longer I fear for our industry's growth.
harassment  gamergate  abuse  twitter  hate-groups  gaming  misogyny 
october 2014 by jm
GamerGate Death Threats - Business Insider
"It's completely insane. It's insane that you even have to say out loud that sending death threats to people who disagree with your opinion of video games is wrong. Yet here we are: Apparently, it needs to be said."
death-threats  gamergame  gaming  twitter  feminism  misogyny 
october 2014 by jm
Trouble at the Koolaid Point — Serious Pony
This is a harrowing post from Kathy Sierra, full of valid observations:
You’re probably more likely to win the lottery than to get any law enforcement agency in the United States to take action when you are harassed online, no matter how visciously and explicitly. Local agencies lack the resources, federal agencies won’t bother.


That to the power of ten in Ireland, too, I'd suspect. Fuck this. Troll culture is way out of control....
twitter  harassment  feminism  weev  abuse  trolls  4chan  kathy-sierra 
october 2014 by jm
The Womansplainer
Brilliant. Nice use of an anime avatar, to boot....

'Consulting for men who have better things to do than educate themselves about feminism. Got a question for a feminist? I would be happy to educate you! Below are my rates.'
feminism  gamergate  funny  mansplaining  misandry  misogyny  twitter  lmgtfy 
october 2014 by jm
How did Twitter become the hate speech wing of the free speech party?
Kevin Marks has a pretty good point here:
Your tweet could win the fame lottery, and everyone on the Internet who thinks you are wrong could tell you about it. Or one of the "verified" could call you out to be the tribute for your community and fight in their Hunger Games.

Say something about feminism, or race, or sea lions and you'd find yourself inundated by the same trite responses from multitudes. Complain about it, and they turn nasty, abusing you, calling in their friends to join in. Your phone becomes useless under the weight of notifications; you can't see your friends support amongst the flood.

The limited tools available - blocking, muting, going private - do not match well with these floods. Twitter's abuse reporting form takes far longer than a tweet, and is explicitly ignored if friends try to help.
harassment  twitter  4chan  abuse  feminism  hate-speech  gamergate  sea-lions  filtering  social-media  kevin-marks 
october 2014 by jm
sferik/t
"A command-line power tool for Twitter." It really is -- much better timeline searchability than the "real" Twitter UI, for example
twitter  ruby  github  cli  tools  unix  search 
october 2014 by jm
Emergent
"Snopes for Twitter". great idea
aggregator  facebook  twitter  snopes  urban-legends  news  rumours 
october 2014 by jm
How Twitter Uses Redis to Scale
'105TB RAM, 39MM QPS, 10,000+ instances.' Notes from a talk given by Yao Yu of Twitter's Cache team, where she's worked for 4 years. Lots of interesting insights into large-scale Redis caching usage -- as in, large enough to max out the cluster hosts' network bandwidth.
twitter  redis  caching  memcached  yao-yu  scaling 
september 2014 by jm
Fighting spam with BotMaker
Some vague details of the antispam system in use at Twitter.
The main challenges in supporting this type of system are evaluating rules with low enough latency that they can run on the write path for Twitter’s main features (i.e., Tweets, Retweets, favorites, follows and messages), supporting computationally intense machine learning based rules, and providing Twitter engineers with the ability to modify and create new rules instantaneously.
spam  realtime  scaling  twitter  anti-spam  botmaker  rules 
august 2014 by jm
inotify one-liner hack
install inotify-tools, then: 'while true do inotifywait -r -e modify -e create -e close . ./run.sh done' #opscookie
inotify  al-tobey  one-liners  unix  hacks  opscookie  twitter 
august 2014 by jm
« earlier      
per page:    204080120160

related tags

4chan  8chan  abortion  abuse  accounts  advertising  afp  agents  aggregation  aggregator  aggregators  agtweet  ai  air  al-tobey  alcoholics  alexis-madrigal  algorithms  alt-right  analytics  android  annoying  anomaly-detection  anonymity  anti-phishing  anti-spam  antisemitism  aphorisms  api  apparel  append-only  appliances  architecture  art  ash  australia  authentication  automation  backpressure  bahtiyar-duysak  banking  batch  bbc  belong.io  bernie-bros  big-data  bitcoin  blockchain  blocking  blocklists  bmp  boards  boards.ie  boing-boing  books  botmaker  botnet  bots  bounding-boxes  brazil  brexit  bridging  bst  build  build-systems  bullying  bureaucracy  business  buzzfeed  cache  caching  cambridge-analytica  canary-requests  canvas  cap  cassandra  catholic-church  celestine-omin  censorship  character-sets  characters  chargebacks  cheats  checkpoints  checksums  chris-andrews  cia  cli  clothing  cloudera  cms  cnet  coal  coding  colm-o-gorman  colmmacc  columnar-storage  columns  comedy  comments  communication  community  companies  compilation  compliance  compression  conferences  consistency  conspiracy-theories  consume  consumers  conversion  cool  coponcomrades  copyright  corman-herley  cory-doctorow  crazy  creepy  cryptol  css  culture  curl  cyber-bullying  cyberbullying  cyberstalking  dailywtf  damien-mulley  daniel-morel  dark-ads  data  data-structures  database  databases  datamining  datasift  dataviz  death-threats  debate  deborah-blum  debugging  delete  democracy  design  dev  development  devore  dickc  diff  diffy  discussion  disgusting  disinfo  disinformation  distcomp  distraction  distributed  distributed-systems  dns  dole  downtime  doxxing  dpi  dremel  driving  drugs  dublin  dutch  dylan-collins  dystopia  earlybird  earth  ebooks  elections  email  emergent  emi  emoji  enforcement  engagement  engineering  epilepsy  espionage  ethernet  ethics  eu  event-logging  event-processing  event-sourcing  events  eventual-consistency  facebook  fail  fake-followers  fake-news  false-positives  fame  family  fault-tolerance  favorites  feature-creep  feature-team  feed  feeds  feminism  fianna-fail  filtering  fine-gael  firehose  flickr  food  formal-verification  fossil-fuels  fraud  free  free-speech  freedom  fridges  funny  future  game-mechanics  gamergame  gamergate  games  gaming  gardai  gc  gdpr  geodata  george-soros  geotagging  gephi  getty-images  gifs  github  gizzard  google  google-news  google-plus  google-script  government  gradle  graham-linehan  graphics  graphs  grep  guardian  guidelines  gwibber  hacks  hadoop  haiti  harassment  harrassment  hashtags  hate-groups  hate-speech  hbase  heat-maps  heron  high-scalability  hilary-clinton  history  holidays  holland  holt-winters  home  horse_ebooks  hotspot  html  http  hunter-energy  ibm  idiots  ifttt  image  images  immigration  infowars  infrastructure  innodb  inotify  instagram  internet  interviewing  interviews  iot-group  ip  ireland  irish-times  irony  j-g-ballard  java  javascript  jobs  jokes  journalism  jpeg  jvm  kafka  kathy-sierra  ketamine  kevin-marks  kinesis  korea  labour  lambda-architecture  languages  latin-america  law  laws  layout  leaks  left-wing  legal  linkedin  links  live  lmgtfy  logging  logs  lol  lols  louise-mensch  low-latency  lowtax  lyft  maciej-ceglowski  management  manhattan  mansplaining  mapping  mapreduce  maps  markov-chain  markov-chains  marref  marshalling  mary-mcaleese  mastodon  maven  media  memcache  memcached  memes  memory  merging  metrics  michael-hayden  microsoft  mike-monteiro  mince-on-toast  misandry  misogyny  ml  mobiles  moderation  monitoring  monorepo  mother-and-baby-homes  mtls  mugus  music  muting  mysql  nam-shub  nathan-marz  nazis  network  networking  networks  new-economy  news  newsfeed  nigerian-scam  nis  nosql  nsa  number-crunching  nws  oauth  one-liners  online  open-source  opendns  ops  opscookie  optimization  oreilly  osama-bin-laden  outages  outliers  outrage  packets  paddy  pants  parquet  passwords  patty  payment  pdf  perf  performance  pgp  phishing  phones  photography  photos  photoshop  pig  pinboard  pintman  pintmen  piracy  pitchforks  policing  policy  politicians  politics  polls  porn  post-mortem  power  presentation  presentations  pricehound  priests  privacy  production  productivity  profiling  projects  prolife  propaganda  proxy  proxying  psyops  public  public-order-act  pubs  pull  push  quangos  queueing  r  racism  radar  rage  rain  rainfall  random  rants  reading  reality  realtime  record-shredding  red-tape  redis  referenda  reliability  repeal-shield  repealshield  repealthe8th  replication  reporting  repos  research  retweets  revenue  riak  rob-pike  robots  root  rss-feeds  ruby  rules  rumours  russia  s2n  san-francisco  sarah-jeong  saudi-arabia  saw  sbt  scala  scalability  scale  scaling  scams  science  scribe  sea-lions  sean-sherlock  search  searching  security  service-metrics  sex  sexism  sfscala  sharding  sherblock  shopify  sigma.js  silicon-valley  slides  slogans  sms  sneachta  snopes  snow  soa  social-media  socialnetworking  society  sock-puppets  somethingawful  south-america  south-korea  spam  speed  spies  spikes  ssd  ssl  ssm  st-patricks-day  stack  stacks  stalking  standards  startups  statistics  storage  stories  storm  stream-processing  streaming  stu-hood  stupid-ideas  style  styling  summingbird  supertweet  surveillance  swatting  syria  systems  talks  targeting  tcp  teams  teamwork  tech  tech-debt  tech-talks  tee-shirts  terribad  testing  tests  text  threadless  threads  threats  thrift  til  tim-hunt  time-series  tls  to-read  tom-matzzie  tom-murphy  tools  tor  tor-bridges  torrents  tracer-requests  tracing  tragedy  trains  travel  trees  trident  trolling  trolls  trump  tsar  tsd  tuam  tumblr  tuning  turkey  tweaks  tweets  tweetstorms  twitter  twitterstorms  twoheadlines  twolicy  uber  ucd  udp  ui  uk  ukip  unicode  unions  unix  unroll  upd4t3  urban-legends  us-politics  usa  ux  vaac  via:adamshostack  via:igrigorik  via:johnke  via:micktwomey  via:nelson  via:niall-harbison  via:peterbourgon  via:pixelbeat  via:tjmcintyre  video  vips  viral  visas  vogue  volcano  voldemort  war  war-stories  warning  waxy  weather  web  web-apps  weev  welfare  wikipedia  wilw  women  word-frequency  work  workaround  world  wtf  yao-yu  yesequality  youtube  zeromq  zipkin 

Copy this bookmark:



description:


tags: