distributed   39499

« earlier    

How We Test the Stateful Autoscaling of Our Stream Processing System | Lobsters
7 How We Test the Stateful Autoscaling of Our Stream Processing System distributed testing blog.wallaroolabs.
testing  distributed 
yesterday by linkt
The Red Wedding Problem: Mitigating Write Spikes at the Edge | Lobsters
10 The Red Wedding Problem: Mitigating Write Spikes at the Edge pdf distributed christophermeiklejohn.
pdf  distributed 
yesterday by linkt
Common patterns for building resiliency in distributed systems at Scale | Lobsters
Please let me know your experiences with these resiliency patterns. Also please feel to pitch in more other patterns if you have encountered any. Thanks for this.
ask  distributed 
yesterday by linkt
Choosing BOINC projects
BOINC is used by many volunteer computing projects. Some are based at universities and research labs, others are run by private groups or individuals. You can participate in any number of these projects. The following projects are known to us at BOINC, and we believe that their descriptions are accurate.
boinc  distributed  background 
4 days ago by cyberchucktx
TCP is an underspecified two-node consensus algorithm and what that means for your proxies
"When people talk about TCP, it's easy to fall into the trap of thinking of it as a connection, with a bi-directional stream of bytes. That is the abstraction that TCP provides, but it's not what TCP is. TCP is an agreement between 2 nodes to run a simple consensus algorithm. The data that is agreed on is (roughly) how much of what I have sent have you seen and how much have I seen of what you've sent. Since there are only 2 nodes, the algorithm is much simpler than what you would see in Raft or Paxos, but like a lot of consensus algorithms, it's based on nodes agreeing on what the current highest number is. [...] Besides the streams being sent, there's another important bit of information: the state of the connection itself. Annoyingly, some of this information is not transmitted over the network. The state of the connection is based largely on heuristics of the individual TCP implementations and to make matters worse, we allow programs to change this behavior depending on the application protocol."
networking  tcp  tls  distributed 
4 days ago by mechazoidal
Bringing Arbitrary Compute to Authoritative Data - ACM Queue
Mark Cavage and David Pacheco, on the architecture of Joyent's "Manta" general-purpose distributed storage system, supporting arbitrary computation on data at rest.
In particular:
- storage is "object": no partial updates, objects are immutable.
- interface is Unix-like: stdin/stdout + pipes
- "Local" computing: when given a job to run, the job is run on the node that contains the object instead of copying it across the network.
paper  distributed  piperesearch  from:acm  storage 
4 days ago by mechazoidal
There's Just No Getting around It: You're Building a Distributed System - ACM Queue
Mark Cavage, illustrating it with a simple "image resizing" service.
"In reality, building a distributed system requires a methodical approach to requirements along the boundaries of failure domains, latency, throughput, durability, consistency, and desired SLAs for the business application at all aspects of the application."

PR: note that splitting up identity management and caching makes it look like a 9P system: but he counters with noting that filesystems choose the C in CAP, and therefore do not scale past a single node very well.
distributed  from:acm  paper  reference  design  2013 
4 days ago by mechazoidal
TCP is an underspecified two-node consensus algorithm and what that means for your proxies | Lobsters
5 TCP is an underspecified two-node consensus algorithm and what that means for your proxies distributed networking morsmachine.
ifttt  tfav  twitter  getpocket  distributed  networking 
5 days ago by linkt
Consistent Hash Rings Explained Simply
Consistent hash rings are beautiful structures, yet often poorly explained. Implementations tend to focus on clever language-specific tricks, and theoretical approaches insist on befuddling it with math and tangents irrelevant. This is an attempt... | Ax's Findings | distributed systems, programming languages, and the internals of operating systems
programming  algorithms  compsci  distributed 
5 days ago by mtwilliams

« earlier    

related tags

2013  actor  actors  advice  aggregate  ai  algorithm  algorithms  analytics  ansible  apache  architecture  ask  assemblage  authentication  automation  aws  background  balancing  beakerbrowser  beyon  big-data  birds  bitcoin  blockchain  boinc  book  books  browser  c++  career  caspaxos  centralization  clickhouse  cloud  cluster  column  columndb  community  compsci  computer-science  computing  concurrency  consensus  coo  cool-tools  cool  cryptocurrency  culture  curated  dat  data  database  decentralized-protocols  decentralized  deep  deployment  design  devops  direct  distributed-computing  distributed-systems  django  economics  elixir  elk  encryption  engineering  erlang  estimation  ethereum  evolution  experience  eye  faulty  federation  filesystem  floss  follow  framework  fritter  from:acm  geometry  getpocket  github  go  golang  google  graph-database  graph  graphdb  graphql  hadoop  health  healthcare  http  ifttt  interesting  internet  interviewing  ipfs  javascript  kafka  languages  lb  learning  lib  library  list  lists  load  location  log  logging  logs  manager  market  memory  messaging  micro  microservices  mitogen  model  network  networking  node  nosql  one  open  opensource  opentracing  ops  optimization  overview  p2p  pandas  paper  parallel  paxos  pdc  pdf  performance  personal  piperesearch  platform  prediction  privacy  processing  productivity  programming  protocol  proxy  publishing  python  quasicrystal  rdfs  rdma  reference  reinforcement  replay  resiliency  resources  response  rest  routing  rsm  ruby  sagas  sampling  scalability  scheduler  scrum  secret  sensor  services  share  shared  smb-direct  smb  smbv3  social  software  source  sql  standard  standards  storage  story  streams  sync  sysadmin  system  systems  task  tcp  testing  tfav  tiling  time  timeseries  tls  to_read  tool  tools  topology  tracing  transactions  twitter  vision  web  webdev  wechat  xmpp  yandex  youtube 

Copy this bookmark: