scalability   37442

« earlier    

Locking, Little's Law, and the USL
Excellent explanatory mailing list post by Martin Thompson to the mechanical-sympathy group, discussing Little's Law vs the USL:
Little's law can be used to describe a system in steady state from a queuing perspective, i.e. arrival and leaving rates are balanced. In this case it is a crude way of modelling a system with a contention percentage of 100% under Amdahl's law, in that throughput is one over latency.

However this is an inaccurate way to model a system with locks. Amdahl's law does not account for coherence costs. For example, if you wrote a microbenchmark with a single thread to measure the lock cost then it is much lower than in a multi-threaded environment where cache coherence, other OS costs such as scheduling, and lock implementations need to be considered.

Universal Scalability Law (USL) accounts for both the contention and the coherence costs.

When modelling locks it is necessary to consider how contention and coherence costs vary given how they can be implemented. Consider in Java how we have biased locking, thin locks, fat locks, inflation, and revoking biases which can cause safe points that bring all threads in the JVM to a stop with a significant coherence component.
usl  scaling  scalability  performance  locking  locks  java  jvm  amdahls-law  littles-law  system-dynamics  modelling  systems  caching  threads  schedulers  contention 
4 days ago by jm
Optimizing web servers for high throughput and low latency | Hacker News
I spent some time looking at this recently. I'm focussed on throughput, not high TPS. The quality of blog posts is absolutely atrocious. There's a lot of blog posts that amount to "Set this setting to this value" but no real explanation why, or what the setting actually does. Some of what I've seen are actually downright dangerous.
development  devops  scalability  nginx 
4 days ago by dano
The Archives Unleashed Project
Web archive search and data analysis tools to enable scholars and librarians to access, share, and investigate recent history since the early days of the WWW.
software  preservation  archiving  analytics  scalability  digital-humanities  via:mikael 
5 days ago by mjlassila
Delivering Billions of Messages Exactly Once · Segment Blog
Segment is the analytics API you've always wanted. It's the easiest way to install all of your favorite analytics tools at once!
architecture  scalability  kafka  distributed-systems 
10 days ago by derekharmel
PostgreSQL Automatic Failover
Pacemaker is nowadays the industry reference for High Availability. In the same fashion than for Systemd, all Linux distributions moved (or are moving) to this unique Pacemaker+Corosync stack[…]. It is able to detect failure on various services and automatically decide to failover the failing resource to another node when possible.


PostgreSQL Automatic Failover is a new [Pacemaker] Resource Agent dedicated to PostgreSQL. […]

Once your PostgreSQL cluster built using internal streaming replication, PAF is able to expose to Pacemaker what is the current status of the PostgreSQL instance on each node: master, slave, stopped, catching up, etc. […]
hold  tools  2017  distributed  scalability  postgresql 
11 days ago by giorgio_v

« earlier    

related tags

2017  @article  @comparison  @howto  access  advice  agile  agilepm  algorithms  amazon  amdahls-law  analysis  analytics  apache  apple  architecture  archiving  arstechnica  article  autoscale  aws  aws_security  backups  big-data  biology  bitcoin  blockchain  book  books  bot  busting  cache  caching  capacity  casestudy  cassandra  cdn  citus  clevermarks  cloud  cluster  clustering  concurrency  container  contention  continuousdelivery  continuousintegration  coordination  data  database  databases  datacenter  datasync  db  ddos  deployment  desktop  development  devops  digital-humanities  discord  disk  distributed-systems  distributed  distributed_systems  distributedsystem  diy  dopost  dsdm  economics  elasticsearch  elixir  engineering  esb6  essays  ethereum  event_sourcing  events  faulttolerance  fintech  frameworks  fwp  gamedev  generator  genetics  git  github  golang  google  gpu  growth  ha  heroku  hetzner  hickory  high-availability  hold  hosting  http  httpd  image  infrastructure  interesting  interview  io  issues  java  jvm  kafka  kubernetes  latency  lib  library  limited  linux  littles-law  loadbalancing  locking  locks  mariadb  memory  messaging  modelling  mongodb  monorepo  ms  mysql  myths  netflix  networking  nginx  nodejs  nosql  opensource  operations  optimization  ovum  parallel  performance  php  plasma  postgres  postgresql  preemptible  presentation  preservation  problems  profiling  programming  python  qanqc  rails  rdbms  recovery  resilience  resize  resizing  ruby  scalable  scale  scaledb  scaling  scaling_out  schedulers  science  security  server  serverless  sharding  sharting  shopify  smartcontracts  software  sql  startup  startups  stateful-sets  storage  success  synthesis  sysadmin  system-dynamics  systems  talk  tappsi  tech  techarch  technology  testing  threads  thumbor  tips  tools  tuning  twitter  unity  usability  usl  varnish  vc  venturecapital  video  virtualization  weather  web  webapp  webdev  wordpress  work  workload  ycombinator  youtube 

Copy this bookmark: