jm + war-stories   4

Let a 1,000 flowers bloom. Then rip 999 of them out by the roots
The Twitter tech-debt story.
Somewhere along the way someone decided that it would be easier to convert the Birdcage to use Pants which had since learned how to build Scala and to deal with a maven-style layout. However at some point prior Pants been open sourced in throw it over the wall fashion and picked up by a few engineers at other companies, such as Square and Foursquare and moved forward. In the meantime, again because there weren’t enough people who’s job it was to take care of these things, Science was still on the original internally developed version and had in fact evolved independently of the open source version. However by the time we wanted to move Birdcage onto Pants, the open source version had moved ahead so that’s the one the Birdcage folks chose.


(cries)
tech-debt  management  twitter  productivity  engineering  monorepo  build-systems  war-stories  dev 
september 2015 by jm
Goodbye MongoDB, Hello PostgreSQL
Another core problem we’ve faced is one of the fundamental features of MongoDB (or any other schemaless storage engine): the lack of a schema. The lack of a schema may sound interesting, and in some cases it can certainly have its benefits. However, for many the usage of a schemaless storage engine leads to the problem of implicit schemas. These schemas aren’t defined by your storage engine but instead are defined based on application behaviour and expectations.


Well, don't say we didn't warn you ;)
mongodb  mysql  postgresql  databases  storage  schemas  war-stories 
march 2015 by jm
A cautionary tale about building large-scale polyglot systems
'a fucking nightmare':
Cascading requires a compilation step, yet since you're writing Ruby code, you get get none of the benefits of static type checking. It was standard to discover a type issue only after kicking off a job on, oh, 10 EC2 machines, only to have it fail because of a type mismatch. And user code embedded in strings would regularly fail to compile – which you again wouldn't discover until after your job was running. Each of these were bad individually, together, they were a fucking nightmare. The interaction between the code in strings and the type system was the worst of all possible worlds. No type checking, yet incredibly brittle, finicky and incomprehensible type errors at run time. I will never forget when one of my friends at Etsy was learning Cascading.JRuby and he couldn't get a type cast to work. I happened to know what would work: a triple cast. You had to cast the value to the type you wanted, not once, not twice, but THREE times.
etsy  scalding  cascading  adtuitive  war-stories  languages  polyglot  ruby  java  strong-typing  jruby  types  hadoop 
march 2014 by jm
_Scaling with MongoDB_, Michael Schurter 2011 [PDF]
presentation with some rather terrifying MongoDB war stories
mongodb  performance  presentation  scaling  war-stories 
june 2011 by jm

Copy this bookmark:



description:


tags: