distributed   40526

« earlier    

NetApp Trident - Storage orchestrator for containers
A fully supported open source project maintained by NetApp. It has been designed from the ground up to help you meet the sophisticated persistence demands of your containerized applications.

Through its support for popular container platforms like Kubernetes and Docker, Trident understands the natural and evolving languages of those platforms, and translates requirements expressed or implied through them into an automated and orchestrated response from the infrastructure.
filesystem  Kubernetes  virtualization  opensource  storage  distributed 
2 days ago by liqweed
Real Differences between OT and CRDT for Co-Editors
"OT (Operational Transformation) was invented for supporting real-time co-editors in the late 1980s and has evolved to become a core technique used in today's working co-editors and adopted in major industrial products. CRDT (Commutative Replicated Data Type) for co-editors was first proposed around 2006, under the name of WOOT (WithOut Operational Transformation). Follow-up CRDT variations are commonly labeled as "post-OT" techniques capable of making concurrent operations natively commutative, and have made broad claims of superiority over OT solutions, in terms of correctness, time and space complexity, simplicity, etc. Over one decade later, however, CRDT solutions are rarely found in working co-editors, while OT solutions remain the choice for building the vast majority of co-editors. Contradictions between this reality and CRDT's purported advantages have been the source of much debate and confusion in co-editing research and developer communities. What is CRDT really to co-editing? What are the real differences between OT and CRDT for co-editors? What are the key factors that may have affected the adoption of and choice between OT and CRDT for co-editors in the real world? In this paper, we report our discoveries, in relation to these questions and beyond, from a comprehensive review and comparison study on representative OT and CRDT solutions and working co-editors based on them. Moreover, this work reveals facts and presents evidences that refute CRDT claimed advantages over OT. We hope the results reported in this paper will help clear up common myths, misconceptions, and confusions surrounding alternative co-editing techniques, and accelerate progress in co-editing technology for real world applications."
crdt  ot  collaborative  distributed  datastructure 
2 days ago by aapl
Understanding Apache Pulsar
In this post we’ll go through the Apache Pulsar design so that we can better design the failure scenarios. This post is not for people who want to understand how to use Apache Pulsar but who want to understand how it works. I have struggled to write a clear overview of its architecture in a way that is simple and easy to understand. I appreciate any feedback on this write-up.

The main claims that I am interested in are:

guarantees of no message loss (if recommended configuration applied and your whole data center doesn't burn to the ground)

strong ordering guarantees

predictable read and write latency

Apache Pulsar chooses consistency over availability as does its sister projects BookKeeper and ZooKeeper. Every effort is made to give strong consistency.
pulsar  kafka  messaging  systems  distributed  consensus 
3 days ago by whitney

« earlier    

related tags

0mq  academia  actor  actors  algorithm  algorithms  alternative  anonymous  api  architecture  async  attacks  aws  batch-size  batchsize  berlin  blockchain  blog  boinc  c++  cap  carrier  cassandra  cdn  celery  climatechange  clocks  cloud-haskell  cloudflare's  cloudflare  cluster  clustering  collaboration  collaborative  computing  concurrency  conference  conferences  consensus  consul  container  control  course  crdt  crypto  culturalanthropology  data-parallel  data  database  dataparallel  datastructure  datastructures  db  ddd  debugging  decentralization  decentralized  deep-learning  devops  devtools  discovery  displacement  displacements  distributed-systems  dl  docker  dropbox  dvcs  dynamic  edge  editing  education  encryption  environment  erlang  event-driven  example  facebook  federated  file  filesharing  filesystem  floss  framework  freenet  gateway  germany  github  globalwarming  golang  google  gpu  gradient  gradients  guid  hackernews  hadoop  haskell  highered  highereducation  history  howto  http  implementation  indieweb  infrastructure  internet  internetarchive  interval  ipfs  ipv6  java  jvm  kafka  kubernetes  languages  large  ledger  library  links  logs  medium  mesh-networking  mesh  messaging  microservices  monitoring  nats  netflix  network  networking  newsql  nosql  ntp  oct18  opensource  optimization  orchestration  oss  ot  p2p  patches  paxos  performance  phishing  physics  postgresql  privacy  private  programming  project  protocol  pulsar  python  pytorch  queue  ram  reddit  reference  research  s3  scale  scanning  scheduler  security  servers  service  services  sharing  social  socialchains  software  sql  stats  storage  streaming  streams  survey  sustainability  sysadmin  system  systems  task  testing  texas  text  through  timbernerslee  time  tool  tools  tor  tracing  training  transaction  tree  tutorial  types  uuid  vector  version  video  virtual  virtualization  vpn  waste  web  webdev  wifi  youtube  zeromq 

Copy this bookmark: