euler + synchronization   32

Neil Fraser: Writing: Differential Synchronization
Keeping two or more copies of the same document synchronized with each other in real-time is a complex challenge. This paper describes the differential synchronization algorithm. Differential synchronization offers scalability, fault-tolerance, and responsive collaborative editing across an unreliable network.
algorithms  synchronization  cs 
july 2018 by euler
Deep Dive: New bookmark sync in Nightly – Firefox Nightly News
Historically, bookmark sync has been plagued by problems that were difficult to isolate and fix.

Bookmarks would be duplicated, lost, or reordered, temporarily or permanently.
Folders with different contents would smush together.
New bookmarks wouldn’t make their way to all devices, causing them to gradually fall out of sync.
Moves would be partially or completely undone.
At the root of all these issues was an approach to syncing that didn’t consider the unique challenges of bookmarks. In this post, we’ll dive into an overview of how Sync works, why bookmarks are special, and the advantages of the new design. Whether you’re a new or long-time Sync user, we invite you to flip the pref, try the new bookmarks engine out, and send us your feedback!
synchronization  bookmarks  firefox 
may 2018 by euler
Fauna Blog | ACID transactions in a globally distributed database
In this post we described how distributed ACID transactions work in FaunaDB, as well as why they are an essential building block for applications. In the end, we consider them and strong consistency to be table stakes for any modern, distributed database.

FaunaDB’s transaction support is the foundation for many of its other features, such as temporality, global distribution, low-latency consistent reads, and operational resiliency. You can learn more about the architecture of FaunaDB in our architectural white paper.

You can also sign up for an account and try FaunaDB for yourself.

Discuss this post on Hacker News.
distributed  Database  synchronization 
november 2017 by euler
OpenReplica provides availability, reliability and fault-tolerance in distributed systems. It is designed to maintain long-lived, critical state (such as configuration information) and to synchronize distributed components. It works as follows: you define a Python object that encapsulates the state you want replicated, along with methods that can update it, and can synchronize threads that access it. You give it to OpenReplica, your object gets geographically distributed automatically, and you receive a proxy through which multiple clients can access the replicated object transparently. To the rest of your application, your replicated object appears as a regular Python object when you use the provided proxy.
distributed  synchronization  paxos  python 
may 2017 by euler
How Your Data is Stored, or, The Laws of the Imaginary Greeks
If you have made it this far, you have just learned some of the most challenging topics in distributed computing. Nearly every problem in datacenter- or planet-scale computing boils down to these issues: how do you get a bunch of computers, often distant from one another, connected via unreliable links, and prone to going down at unpredictable intervals, to nonetheless agree on what information they store?
In practice, there are four methods which are commonly used:
Single data stores (the Pseudemoxian Hermit), where a single computer keeps its own copy, everyone wishing to use it must take turns, and the system is vulnerable to a single disaster; however, the system is strongly consistent, dead-simple, and all other systems are built on top of it.
Eventually consistent replication (the Fotan system), where each participant has their own (strongly-consistent) store, and everyone changes and reads their own copy, distributing and receiving updates to all of their fellows later on. This has the advantage of speed and simplicity, as well as robustness to many kinds of disaster, but lacks the strong-consistent guarantees that once you write, all future readers will know about it. This system is very useful in cases where that guarantee isn’t needed, such as distributing copies of images (or other bulky data) which will never change after it is written, and where freshness isn’t really required.
Quorum decisions (the Paxon system — and unlike the other examples, this one is actually called “Paxos” in normal CS conversations), where reads and writes involve getting a majority of the participants to agree. This provides strong consistency and robustness, but can be very slow, especially when spread out over a wide area.
Master election (the Siranon system), where an expensive, strongly-consistent store is used to decide who is in charge of any subject for a time, and then that responsible party uses their own, smaller, strongly-consistent store to maintain the laws on that subject.
distributed  synchronization  concurrency  paxos 
may 2017 by euler
haret/ at master · vmware/haret · GitHub
One of the top priorities in developing haret is ease of operation. We have experience in the past delivering software intended for lights-out operation, and we intend to bring lessons from that experience to bear in making haret a software system that can be confidently operated as "invisible infrastructure" in many cases.
haret  reference  synchronization 
may 2017 by euler
GitHub - vmware/haret: A strongly consistent distributed coordination system, built using proven protocols & implemented in Rust.

haret provides a distributed service built on proven protocols that exposes strongly consistent coordination primitives via an easy to use API, in order to satisfy the essential dependencies of many distributed applications. You can read more about why we decided to build haret here.

This README contains quickstart instructions for both developers and users. More information about using haret can be found in the rough and incomplete User Guide.
distributed  algorithms  cs  synchronization 
may 2017 by euler
Make any algorithm lock-free with this one crazy trick : Paper Trail
This gives RCU a really simple scheme to check for a grace period after a write has finished: try to run a thread on every CPU in the system. Once the thread runs, a context switch has happened which means any previous critical sections must have completed, and so any previous version of the data structure can be reclaimed. The readers don’t have to do anything to signal that they are finished with their critical section: it’s implicit in them starting to accept context switches again!
linux  synchronization  rcu 
may 2016 by euler
What is RCU, Fundamentally? []
Read-copy update (RCU) is a synchronization mechanism that was added to the Linux kernel in October of 2002. RCU achieves scalability improvements by allowing reads to occur concurrently with updates. In contrast with conventional locking primitives that ensure mutual exclusion among concurrent threads regardless of whether they be readers or updaters, or with reader-writer locks that allow concurrent reads but not in the presence of updates, RCU supports concurrency between a single updater and multiple readers. RCU ensures that reads are coherent by maintaining multiple versions of objects and ensuring that they are not freed up until all pre-existing read-side critical sections complete. RCU defines and uses efficient and scalable mechanisms for publishing and reading new versions of an object, and also for deferring the collection of old versions. These mechanisms distribute the work among read and update paths in such a way as to make read paths extremely fast. In some cases (non-preemptable kernels), RCU's read-side primitives have zero overhead.
linux  synchronization  locking  rcu 
may 2016 by euler
Dropbox: Founder Drew Houston Simplifies the Cloud | MIT Technology Review
Achieving that experience is not simple: we have a polished exterior, but there’s this jungle of different operating systems [with which we must work] and even gnarlier stuff like operating-system bugs and incompatibilities. It’s a hostile environment: we macheted our way through that jungle of problems. It was a bunch of us spending big chunks of our 20s chasing down these obscure compatibility issues.
dropbox  synchronization 
december 2015 by euler
Z-Push | Open source push technology
Z-Push is an open-source application to synchronize ActiveSync compatible devices such as mobile phones, tablets and mobile phones. With a history of over 6 years of successful synchronization with multiple backends Z-Push is the leading open source push synchronization.
opensource  mobile  email  synchronization 
may 2015 by euler
rclone - rsync for cloud storage
Rclone is a command line program to sync files and directories to and from

Google Drive
Amazon S3
Openstack Swift / Rackspace cloud files / Memset Memstore
Google Cloud Storage
The local filesystem
rsync  data  s3  synchronization 
july 2014 by euler
Viewstamped Replication: The Less-Famous Consensus Protocol - Marc's Blog
By comparison to Paxos and Raft, one distributed consensus protocol seems frequently overlooked: Oki and Liskov's Viewstamped Replication. Introduced in May 1988 in Brian Oki's PhD thesis, Viewstamped Replication predates the first publication of Paxos by about a year. If you're looking for intrigue you may be disappointed: both Lamport and Liskov claim the inventions were independent. First, from Viewstamped Replication Revisited:
cs  synchronization  replication  protocol  paxos  raft 
may 2014 by euler
Scalable Atomic Visibility with RAMP Transactions | Peter Bailis
RAMP Transactions provide a scalable alternative to locking and other coordination-intensive solutions to atomic visibility. We’ve found them very useful (e.g., our recent 25x prior-best on the TPC-C New-Order benchmark was implemented using a variant of RAMP-Fast), and there’s a principled reason for their performance: synchronization and partition independence. Please check out our SIGMOD paper for more details, and many thanks for the encouraging feedback so far. Happy scaling!
distributed  database  synchronization 
april 2014 by euler
Synchronization in a Distributed System | 8th Light
I’ll admit that the first time I heard the terms "vector clock" or "Lamport timestamp", I figured they were some insane mathematical algorithm that I’d never understand. Perhaps more importantly, I avoided learning about the subject altogether because of this fear. Fortunately, I discovered that their behavior is actually far simpler than I imagined. Let's take a peek under the hood and find out how this all works.
synchronization  distributed  clock 
october 2013 by euler
Firebase - A scalable real-time backend for your app
Data stored in Firebase is directly accessible from the client. This means you can build dynamic, data-driven websites without worrying about servers or server code.
webdev  data  synchronization  Node.js 
june 2013 by euler
Frustrated with iCloud, Apple’s developer community speaks up en masse | Ars Technica
Apple's iCloud is marketed to us end users as a convenient and centralized way to manage data on all of our Macs and iOS devices: sync contacts and bookmarks, re-download music and apps, back up iOS devices, and sync documents and data for third-party apps as MobileMe did. The last item, syncing of documents and data, is one of the least glossy features of iCloud, but it is one of the most important, and it should be among the most straightforward. Right?
apple  icloud  cloud  storage  synchronization 
march 2013 by euler
Build collaborative apps with Google Drive Realtime API - Google Developers Blog
Because the Drive Realtime API is based on operational transformation (OT), local changes are reflected instantly, even on high-latency networks. The Drive Realtime API automatically transforms changes to the data model so that every collaborator stays in sync.

If basic collections aren't enough for your application, the Drive Realtime API supports custom objects and references, including trees and arbitrary graph structures. As with other collaborative objects, the Drive Realtime API automatically synchronizes these objects with other collaborators and stores them in Drive.

Because presence is important in collaborative applications, the Drive Realtime API also keeps track of who is connected to your application and provides your app with events for when collaborators join, leave, or make changes.
API  googledrive  google  OT  synchronization  drive 
march 2013 by euler
sickill/bitpocket - GitHub
bitpocket is a small but smart script that does 2-way directory synchronization. It uses rsync to do efficient data transfer and tracks local file creation/removal to avoid known rsync problem when doing 2-way syncing with deletion.

bitpocket can use any server which you have ssh access to for its central storage. If you have gigabytes of free disk space on your hosting server you can finally make use of it.
storage  dropbox  synchronization  rsync 
december 2011 by euler
You can take it with you
The system, called Deep Shot, exploits the fact that many Web applications use a standard format, called the uniform resource identifier (URI), to describe the states they’re in. For instance, if you search for directions on Google Maps, then click the link that says “Link,” a window pops up that says “Paste link in email or IM.” The link consists of a long string of symbols, including the addresses of the starting and ending points and codes that indicate their geographical coordinates and the approximate size of the map window. That’s a URI. The data contained in a URI can vary widely, and URIs are a common feature of many Web applications — if sometimes harder to extract than they are with Google Maps.
data  synchronization  trend  uri 
june 2011 by euler
Open Source Dropbox Alternatives To Start Building A File Storage & Sharing System
As expected, yes, there is open source for everything and found very nice resources that can help us accomplish the project with almost no (or few) coding.

Considering you may use them in your own projects/platforms or have an idea of building a file storage and sharing application, here is a complete list of open source Dropbox alternatives (that are updated regularly):
dropbox  opensource  synchronization 
june 2011 by euler
Replicating PostgreSQL with Bucardo - Gabriel Weinberg's Blog
First off, why Bucardo, especially since Postges 9.0 now has built-in hot standby/streaming replication (full docs)? Steve Signer wrote up some cases where you wouldn't want to/be able to use the built-in replication. A more detailed comparison of the various options is on the PG wiki.
postgresql  synchronization  sysadmin 
may 2011 by euler
OpenMobster - Open Source Mobile Cloud Platform: Open Source Cloud Platform for Android
* Build end-to-end Android Sync Apps. All sync modes such as two-way sync, one-way-server sync, one-way-device-sync, and boot sync (an innovation over slowsync) are supported.

* Integrate Push capabilities in your Android Apps.

* Quickly develop Android Apps using the Mobile MVC Framework. This MVC Framework seamlessly integrates with the Sync and Push services.
android  cloud  synchronization 
july 2010 by euler
Google reveals plans for Chrome cloud synchronization - Ars Technica
Browser sync will boost the value of Google's browser and advance Google's strategy of converging the desktop with the cloud. It's a particularly intriguing development in light of Google's upcoming Chrome OS. Seamless synchronization of browser data between a desktop and netbook could be a very compelling feature. It's not clear yet if Google will also bring this feature to its Android mobile browser, but it seems like an obvious move.

Google intends to document the protocols and make the client-side source code available under an open license, which means that it could be possible for third-party developers to bring Google synchronization support to other browsers
synchronization  browser  trend  cloud  rank:1 
august 2009 by euler
StorageMojo » Axxana fixes the speed of light
The Phoenix Black Box is located near the storage system at the primary data center and records a synchronous data stream from the storage. At the same time, an asynchronous data replication system is moving data to a secondary data center (the remote recovery site). The Phoenix Black Box has to protect only the Gigabytes of data that would have been lost in a typical asynchronous replication scenario. Data is protected inside the Black Box during the course of the disaster and can be immediately extracted.
daa  replication  synchronization  operations 
october 2008 by euler
SugarSync Sweetens Service With an iPhone App - GigaOM
Om thinks having an iPhone app gives Sugar Sync (and Buffalo, which offers remote access to its storage drives through an iPhone app) a real competitive advantage against folks like Dropbox or FolderShare. I’m not sure I agree with him. He’s right when he says storage startups are a dime a dozen, and have to do something to stand out from the crowd, but the iPhone is still an aspirational and consumer device. Plenty of people don’t have them yet. Small business owners are the ones buying Buffalo drives, and given the troubles with MobileMe they might wait before taking on the iPhone.
iphone  synchronization 
august 2008 by euler
WebWorkerDaily » Archive SugarSync: Fresh Take on Synchronization «
There are certainly plenty of ways to synchronize files among multiple computers: we’ve looked at entrants in this field including BeInSync, Dropbox, and GoodSync. New entrant in the field SugarSync is trying to stand out by offering support across a larger-than-usual breadth of devices, as well as some innovative features.
august 2008 by euler
Dare Obasanjo aka Carnage4Life - How Facebook Keeps Memcached Consistent Across Geo-Distributed Data Centers
Fortunately, the solution is a lot easier to explain than the problem. We made a small change to MySQL that allows us to tack on extra information in the replication stream that is updating the slave database. We used this feature to append all the data objects that are changing for a given query and then the slave database "sees" these objects and is responsible for deleting the value from cache after it performs the update to the database.
database  replication  synchronization  performance 
august 2008 by euler
Prophet, your path out of the cloud: OSCON 2008 - O'Reilly Conferences, July 21 - 25, 2008, Portland, Oregon
Prophet is a new peer to peer distributed database designed to help ease the transition to post-web-2.0 applications.
With Prophet, it’s easy to build custom database applications and synchronize your data with your friends and coworkers – all without a central server.
database  synchronization  trend  cloud 
august 2008 by euler
CalDAV support comes to Google Calendar - The Unofficial Apple Weblog (TUAW)
Well, fortunately, Google has just quietly introduced CalDAV support to Google Calendar. CalDAV is the protocol that iCal uses to transmit data over the web. Although some other mail and calendar programs support CalDAV, right now Google Calendar is only compatible with iCal. Finally, iCal and Google Calendar can sync without having to use third party programs!
osx  google  ical  caldev  synchronization 
august 2008 by euler
Dare Obasanjo aka Carnage4Life - Some Thoughts on Joel Spolsky's Live Mesh Rant
Everywhere I look data synchronization is becoming more and more important and also more commonplace. I expect this trend to continue over time given the inevitable march of the Web. Being able to synchronize my data and my actions from my desktop to the
trend  synchronization 
june 2008 by euler

Copy this bookmark: