jm + mongodb   20

How eBay’s Shopping Cart used compression techniques to solve network I/O bottlenecks
compressing data written to MongoDB using LZ4_HIGH --dropped oplog write rates from 150GB/hour to 11GB/hour. Snappy and Gzip didn't fare too well by comparison
lz4  compression  gzip  json  snappy  scaling  ebay  mongodb 
6 weeks ago by jm
Charity Majors responds to the CleverTap Mongo outage war story
This is a great blog post, spot on:
You can’t just go “dudes it’s faster” and jump off a cliff.  This shit is basic.  Test real production workloads. Have a rollback plan.  (Not for *10 days* … try a month or two.)


The only thing I'd nitpick on is that it's all very well to say "buy my book" or "come see me talk at Blahcon", but a good blog post or webpage would be thousands of times more useful.
databases  stateful-services  services  ops  mongodb  charity-majors  rollback  state  storage  testing  dba 
october 2016 by jm
Cat-Herd's Crook
Nice approach from MongoDB:
we’ve recently gained momentum on standardizing our [cross-platform test] drivers. Human-readable, machine-testable specs, coded in YAML, prove which code conforms and which does not. These YAML tests are the Cat-Herd’s Crook: a tool to guide us all in the same direction.
mongodb  testing  unit-tests  yaml  multi-platform  coding 
march 2016 by jm
"What the hell have you built"
cut out and keep PNG for many occasions! "Why is Redis talking to MongoDB?"
mongodb  redis  funny  architecture  gifs  png  reactiongifs 
january 2016 by jm
Intercom Engineering Insights - Scale and Reliability 2015
next Intercom hiring^Wevent coming up, Dec 10th in Dublin, talking about how they scale and ops their ElasticSearch and Mongo clusters
elasticsearch  mongodb  intercom  engineering  talks  dublin 
december 2015 by jm
demonstration of the importance of server-side request timeouts
from MongoDB, but similar issues often apply in many other TCP/HTTP-based systems
tcp  http  requests  timeout  mongodb  reliability  safety 
march 2015 by jm
Goodbye MongoDB, Hello PostgreSQL
Another core problem we’ve faced is one of the fundamental features of MongoDB (or any other schemaless storage engine): the lack of a schema. The lack of a schema may sound interesting, and in some cases it can certainly have its benefits. However, for many the usage of a schemaless storage engine leads to the problem of implicit schemas. These schemas aren’t defined by your storage engine but instead are defined based on application behaviour and expectations.


Well, don't say we didn't warn you ;)
mongodb  mysql  postgresql  databases  storage  schemas  war-stories 
march 2015 by jm
Good advice on running large-scale database stress tests
I've been bitten by poor key distribution in tests in the past, so this is spot on: 'I'd run it with Zipfian, Pareto, and Dirac delta distributions, and I'd choose read-modify-write transactions.'

And of course, a dataset bigger than all combined RAM.

Also: http://smalldatum.blogspot.ie/2014/04/biebermarks.html -- the "Biebermark", where just a single row out of the entire db is contended on in a read/modify/write transaction: "the inspiration for this is maintaining counts for [highly contended] popular entities like Justin Bieber and One Direction."
biebermark  benchmarks  testing  performance  stress-tests  databases  storage  mongodb  innodb  foundationdb  aphyr  measurement  distributions  keys  zipfian 
december 2014 by jm
"Ark: A Real-World Consensus Implementation" [paper]
"an implementation of a consensus algorithm similar to Paxos and Raft, designed as an improvement over the existing consensus algorithm used by MongoDB and TokuMX."

It'll be interesting to see how this gets on in review from the distributed-systems community. The phrase "similar to Paxos and Raft" is both worrying and promising ;)
paxos  raft  consensus  algorithms  distsys  distributed  leader-election  mongodb  tokumx 
july 2014 by jm
BitCoin exchange CoinBase uses MongoDB as their 'primary datastore'
'Coinbase uses MongoDB for their primary datastore for their web app, api requests, etc.'
coinbase  mongodb  reliability  hn  via:aphyr  ops  banking  bitcoin 
december 2013 by jm
incompetent error-handling code in the mongo-java-driver project
an unexplained invocation of Math.random() in the exception handling block of this MongoDB java driver class causes roflscale lols in the github commit notes. http://stackoverflow.com/a/16833798 has more explanation.
github  commits  mongodb  webscale  roflscale  random  daily-wtf  wtf 
may 2013 by jm
Call me maybe: Carly Rae Jepsen and the perils of network partitions
Kyle "aphyr" Kingsbury expands on his slides demonstrating the real-world failure scenarios that arise during some kinds of partitions (specifically, the TCP-hang, no clear routing failure, network partition scenario). Great set of blog posts clarifying CAP
distributed  network  databases  cap  nosql  redis  mongodb  postgresql  riak  crdt  aphyr 
may 2013 by jm
High Performance MongoDB Clusters with Amazon EBS Provisioned IOPS
yeah yeah, Mongo. bookmarking for the good data on EBS+PIOPS
ebs  piops  aws  performance  tips  ops  ec2  mongodb  presentations 
april 2013 by jm
Big Data Analytics at Netflix. Interview with Christos Kalantzis and Jason Brown.
Good interview with the Cassandra guys at Netflix, and some top Mongo-bashing in the comments
cassandra  netflix  user-stories  testimonials  nosql  storage  ec2  mongodb 
february 2013 by jm
Cassandra, Hive, and Hadoop: How We Picked Our Analytics Stack
reasonably good whole-stack performance testing and analysis; HBase, Riak, MongoDB, and Cassandra compared. Riak did pretty badly :(
riak  mongodb  cassandra  hbase  performance  analytics  hadoop  hive  big-data  storage  databases  nosql 
february 2013 by jm
_Scaling with MongoDB_, Michael Schurter 2011 [PDF]
presentation with some rather terrifying MongoDB war stories
mongodb  performance  presentation  scaling  war-stories 
june 2011 by jm
The MongoDB NoSQL Database Blog - MongoDB live at Craigslist
'>MongoDB is now live at Craigslist, where it is being used to archive [10TB] of [old posts]'. iiiinteresting
mongodb  nosql  craigslist  systems 
may 2011 by jm
Graylog2
'Free open source self-hosted log management and exception tracking', loggly-style.  Basically, a nifty web data-mining UI on your syslogs (via adulau)
logging  syslog  sysadmin  mongodb  opensource  via:adulau  logs  web  ui  data-mining  from delicious
january 2011 by jm
Foursquare MongoDB outage post mortem
MongoDB was set up to write to RAM if possible, omitting immediate writes to disk -- but then the db size exceeded RAM size, the disk was hit, imposing a massive slowdown and creating a huge backlog immediately, bringing the site down (via Nelson)
via:nelson  mongodb  sharding  nosql  ouch  outage  foursquare  sysadmin  ops  from delicious
october 2010 by jm
We’re Back… so long MongoDB! · Blue74
MongoDB war story -- records going missing, eek
mongodb  mysql  nosql  rant  stability  beta  from delicious
june 2010 by jm

related tags

algorithms  analytics  aphyr  architecture  aws  banking  benchmarks  beta  biebermark  big-data  bitcoin  cap  cassandra  charity-majors  coding  coinbase  commits  compression  consensus  craigslist  crdt  daily-wtf  data-mining  databases  dba  distributed  distributions  distsys  dublin  ebay  ebs  ec2  elasticsearch  engineering  foundationdb  foursquare  funny  gifs  github  gzip  hadoop  hbase  hive  hn  http  innodb  intercom  json  keys  leader-election  logging  logs  lz4  measurement  mongodb  multi-platform  mysql  netflix  network  nosql  opensource  ops  ouch  outage  paxos  performance  piops  png  postgresql  presentation  presentations  raft  random  rant  reactiongifs  redis  reliability  requests  riak  roflscale  rollback  safety  scaling  schemas  services  sharding  snappy  stability  state  stateful-services  storage  stress-tests  sysadmin  syslog  systems  talks  tcp  testimonials  testing  timeout  tips  tokumx  ui  unit-tests  user-stories  via:adulau  via:aphyr  via:nelson  war-stories  web  webscale  wtf  yaml  zipfian 

Copy this bookmark:



description:


tags: