Sync Overview

data consistency and integrity is the foundation of any data management system, particularly in distributed systems such as ditto this article offers a brief overview of how interconnected peers communicate and collaborate quickly, continuously, and reliably across the system, even in the presence of network failures for comprehensive information, see the platform manual docid 5oc2w0zdefoy9vu8ma6qd docid\ frcbubtpe3ob76brpmoib delta encoded sync data in ditto is delta encoded that is, when peer to peer syncing, instead of transmitting entire datasets, only changes (delta) are retrieved, sent, or locally stored the key advantages to ditto's delta based approach include highly frequent and efficient data transmission — avoiding redundant data transmission and reducing bandwidth consumption optimizes network performance consistency without centralized coordination — distributed peers do not need to reach consensus to converge and merge a single value without conflict sync, partition, and replication data synchronization, or "sync" for short, refers broadly to the practice of ensuring that peers always have the most up to date and accurate information however, data sync goes beyond basic updates and involves sophisticated techniques like partitioning and replication partitioning is a strategy that involves dividing the data into smaller, more manageable pieces called partitions these partitions are then asynchronously copied across connected small peers that is, the copies of the data, or replicas , are distributed across separate storage locations running locally within an end user's physical environment, such as a mobile device partitioning and replication are central to ensuring rapid and dependable data access for your end users multiplexer for intelligent replication ditto incorporates a multiplexer inside the peer to peer mesh network to facilitate the data replication process developed by ditto, the multiplexer is an intelligent replication machine that seamlessly switches between active transport types as needed, without duplicating data in addition, the multiplexer breaks data packets down into small fragments and then, once received on the other side, reassembles them when the transports collaborate in parallel and the multiplexer automatically switches between them to establish the most optimal connection, these diverse transport types collectively form what is known as the rainbow connection — each color of the rainbow symbolizes a different transport type conflict free replication in ditto's decentralized architecture, data propagates asynchronously among offline peers, eliminating the need for a centralized authority like a central server to validate and merge database operations this decentralized model, however, presents the possibility of concurrency conflicts a concurrency conflict happens when multiple end users make changes to the same data items at the same time when merging these changes, any inconsistencies collide to resolve these concurrency conflicts, as well as enable optimizations like delta based replication, ditto harnesses conflict free replicated data type ( crdt ) technology crdt is an advanced class of data type designed to manage and replicate data changes in a way that allows multiple distributed peers to make updates concurrently without the need to reach a consensus to form a single meaningful value for merge for an overview of the different crdts that ditto offers, see docid\ rb3wd2jhenojpw8xuirfy communication transports ditto’s peer to peer network architecture utilizes the following diverse set of communication transport technologies and communication channels to achieve seamless, offline first connections between devices /#bluetooth low energy /#peer to peer wi fi /#local area network /#websockets by leveraging a diverse set of technologies, ditto supports a variety of use cases, devices, and environments by default, peers use all available transports available to their devices for instructions on how to enable and disable transports, see the platform manual > docid 5oc2w0zdefoy9vu8ma6qd > docid\ q3jydbwoo rksjdqkg0 bluetooth low energy ditto utilizes both traditional wi fi and bluetooth low energy (le) to maintain a continuous mesh network of transmission control protocol (tcp) connections bluetooth le technology forms low powered and high distance connections between devices, making it highly performant in offline scenarios when replicating small amounts of data in the event of a wi fi network disruption, such as a router getting disconnected, ditto does not automatically switch transports to establish an independent bluetooth le connection as a fallback instead, ditto actively works to maintain the previously established wi fi and bluetooth le connections the following video provides a brief visual overview of how ditto uses bluetooth le to form low powered high distance connections between devices the distances and bandwidth limitations and capacities illustrated in the video are approximations and vary depending on the bluetooth® firmware installed on the device for detailed information, see platform manual > docid 5oc2w0zdefoy9vu8ma6qd peer to peer wi fi available on most devices, standard peer to peer wi fi enables direct, point to point connections between peers without requiring traditional network infrastructure like a centralized server, router, or access point in addition to standard peer to peer wi fi, ditto incorporates a customized multiplexer to increase frequency, speed, and efficiency of data transmission between peers connected in the mesh network the following table provides an overview of the wi fi technologies ditto supports platform ditto supported technology apple apple wireless direct link (awdl) android wi fi aware windows wi fi aware the following video illustrates ditto’s sophisticated and decentralized approach to peer to peer wi fi connections between distributed peers connected in the mesh network the maximum distance of a connection can vary depending on the specific peer to peer transport type being used for detailed information, see platform manual > docid 5oc2w0zdefoy9vu8ma6qd local area network when devices are connected over the same wi fi access point or through other means like an ethernet cable, they can take advantage of a local area network ( lan ), if available, to communicate directly with each other without requiring internet access a lan is an interconnected network made up of devices that are physically near each other, such as a home, enterprise, or college campus for detailed information, see platform manual > docid 5oc2w0zdefoy9vu8ma6qd websockets peers use websocket connections to enable realtime, bidirectional sync between small peers and the cloud deployment, the big peer before you can establish a connection with the big peer using a websocket, you need to specify an ip/port for instructions, see platform manual > docid\ q3jydbwoo rksjdqkg0 the defining attributes of a websocket include requires an internet connection to establish peer to peer communication unlike the other peer to peer transport types, does not automatically discover other peers physically nearby websocket connections can be made over a transmission control protocol (tcp) or local area network (lan) for detailed information, see platform manual > docid 5oc2w0zdefoy9vu8ma6qd universal serial bus when active and granted permission for universal serial bus (usb) communication, peers can establish a direct, wired connection by way of a usb connection