--- title: Mesh Networking 101 slug: v4-6/basic/mesh-networking-101 docTags: createdAt: 2024-03-05T14:12:07.366Z --- The *mesh* is an underlay for data sync within Ditto, operating independently of your queries and sync subscriptions. Data updates propagate through the mesh automatically, provided your sync subscriptions match. Whether you have two devices or two thousand devices in your mesh, Ditto incorporates several algorithms to maintain an optimal mesh network at all times. This article provides an overview of the very basics of Ditto's mesh technology, covering key topics such as: - Communication transports like Bluetooth Low Energy (LE). - The multiplexer that enables Ditto's rainbow connection. - The flood-fill design pattern. - Peer-to-peer authentication and access controls and permissions. - Concurrency conflict-resolution strategies. # Introduction Upon initiating the sync process by invoking the `startSync` function in your app, peers with the same app ID immediately form a *mesh network* using a mixture of communication transports, each with its advantages and disadvantages. For example, Ditto prioritizes Wi-Fi for its high bandwidth and only falls back to Bluetooth LE when needed to take advantage of its longer-range capabilities. Unlike typical home networks — which represent a star topology where all devices connect directly to a central router, switch, or access point — a peer-to-peer mesh network offers multiple pathways for communication. Here's a quick video overview by Ditto's Network Transports Team Lead, Tom Karpiniec: ::embed[]{url="https://www.youtube.com/watch?v=SzbqsL4KCBE"} # Transports Ditto leverages an array of *communication transports*, supporting a variety of use cases and end-user devices. Communication transports are networking protocols that facilitate data transmission across different environments. For instance, using the local area network (LAN) transport for data movement in an office-like setting. Which transports Ditto supports depends on what kind of device or Small Peer SDK you are using, for example: - A mobile phone can use Bluetooth LE, LAN, Wi-Fi Direct, and WebSocket. - A web app running in a browser can use WebSocket. - A Raspberry Pi can use Bluetooth LE, LAN, and WebSocket. While many transports are set up automatically, others, such as WebSocket, if configured with the Big Peer, require that you manually configure them in your app.  :::ExpandableHeading ## Big Peer and On-Premise Servers In addition to peer-to-peer transports like Bluetooth LE, you can opt for decentralized cloud deployment, as well as provide third-party on-prem servers like a legacy business intelligence system, which you configure by simply specifying its URL. At Ditto, we call a server like this a *Big Peer*, while regular SDKs are *Small Peers*. Many apps need to sync data with the Big Peer or an on-premise server. The Ditto SDK uses peer-to-peer and Big Peer connections simultaneously, turning your mesh global. This is as easy as setting your server's URL. ::Image[]{src="https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/FftMwF24t8jwvcUmfsgMk_playground.gif" size="58" width="746" height="416" position="flex-start" showCaption="false"} Big Peers are especially important for those who want to use Ditto in a web browser since web browsers have very limited peer-to-peer capabilities — you can configure the JavaScript SDK with the Big Peer and sync. An SDK can sync in any of the following ways: - Directly with a Big Peer - Locally with Small Peers - With both at the same time, acting as a bridge for devices without internet access. ::: ::::ExpandableHeading ## Bluetooth Low Energy Ditto utilizes both traditional Wi-Fi and Bluetooth Low Energy (LE) to maintain a continuous mesh network of Transmission Control Protocol (TCP) connections. Bluetooth LE technology forms low-powered and high-distance connections between devices, making it highly performant in offline scenarios when replicating small amounts of data.  :::hint{type="info"} In the event of a Wi-Fi network disruption, such as a router getting disconnected, Ditto does not automatically switch transports to establish an independent Bluetooth LE connection as a fallback. Instead, Ditto actively works to maintain the previously established Wi-Fi and Bluetooth LE connections. ::: The following visual overview illustrates how Ditto uses Bluetooth LE to form low-powered, high-distance connections between peer devices.  :::hint{type="info"} The distances and bandwidth limitations and capacities illustrated in the video are approximations and vary depending on the Bluetooth® firmware installed on the device. ::: ::Image[]{src="https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/xX8cTqTKbloV09pe9E5VU_ble2.gif" size="66" width="1024" height="554" position="flex-start" showCaption="false"} :::: ::::ExpandableHeading ## Peer-to-Peer Wi-Fi Available to most devices, standard peer-to-peer Wi-Fi enables direct, point-to-point connections between peers without requiring traditional network infrastructure like a centralized server, router, or access point. In addition to standard peer-to-peer Wi-Fi, Ditto incorporates a customized multiplexer to increase frequency, speed, and efficiency of data transmission between peers participating in the mesh. The following table provides an overview of the Wi-Fi technologies Ditto supports: | **Platform** | **Ditto-Supported Technology** | | ------------ | --------------------------------- | | Apple | Apple Wireless Direct Link (AWDL) | | Android | Wi-Fi Aware | | Windows | Wi-Fi Aware | The following video illustrates Ditto’s sophisticated and decentralized approach to peer-to-peer Wi-Fi connections between distributed peers connected in the mesh network: :::hint{type="info"} The maximum distance of a connection can vary depending on the specific peer-to-peer transport type being used. ::: ::Image[]{src="https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/bwiEGfjmSo3iQf4ZzXqf7_p2p2.gif" size="60" width="989" height="554" position="flex-start" showCaption="false"} :::: :::ExpandableHeading ## Local Area Network When devices are connected over the same Wi-Fi access point or through other means like an Ethernet cable, they can take advantage of a local area network (*LAN*), if available, to communicate directly with each other without requiring internet access. A LAN is an interconnected network made up of devices that are physically near each other, such as a home, enterprise, or college campus. ::: ::::ExpandableHeading ## WebSocket Peers use *WebSocket* connections to enable realtime, bidirectional sync between Small Peers and the optional cloud deployment, the *Big Peer.* :::hint{type="info"} Before you can establish a connection with the Big Peer using a WebSocket, you need to specify an IP/PORT. (See [Customizing Transport Configurations](docId\:Zuw1DwPowaImbpntCB3z3)) ::: The defining attributes of a WebSocket include: - Requires an internet connection to establish peer-to-peer communication. - Unlike the other peer-to-peer transport types, does not automatically discover other peers physically nearby. - WebSocket connections can be made over a Transmission Control Protocol (TCP) or Local Area Network (LAN). :::: # Multiplexer Ditto incorporates a *multiplexer *inside the peer-to-peer mesh network to facilitate data sync. Developed by Ditto, the multiplexer is an intelligent sync machine that seamlessly switches between active transport types as needed, without duplicating data. In addition, the multiplexer breaks down data packets into small fragments and then, once received on the other side, reassembles them. :::ExpandableHeading ## Rainbow Connection When the transports collaborate in parallel and the multiplexer automatically switches between them to establish the most optimal connection, these diverse transport types collectively form what is known as the *rainbow connection *— each color of the rainbow symbolizes a different transport type: ::embed[]{url="https://www.youtube.com/watch?v=8_A1CkYfzoM&t=3s"} ::: # Flood-Fill Process When all peers connected in the mesh need to share the same view of the data, Ditto initiates the *flood-fill *process to multihop data from one connected peer to another connected peer by way of intermediate “hops” along a given path. ::::VerticalSplit{layout="middle"} :::VerticalSplitItem Flood fill, or *flooding* for short, is a common pattern in a mesh network topology for enhancing efficiency and reliability by ensuring data propagates across connected peers efficiently and reliably, regardless of their queries. When flooding the mesh, data moves from one connected peer to another through intermediate "hops" along a defined path. For example, imagine a daisy chain from point A to point B, as illustrated in the graphic on the right. ::: :::VerticalSplitItem ![](https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/vZ7smqGfYJJ4wYD79Mm-5_multihop.png) ::: :::: :::ExpandableHeading ## Benefits of Mesh-Wide Flooding The benefits of flooding the mesh include: - Guaranteeing a consistent offline-first sync of critical data, ensuring data remains consistent even when connectivity is intermittent. - Improving the overall responsiveness of your app, leading to faster interactions. - Reducing costs by optimizing data operations since, when feasible, you can read and write data to a nearby peer instead of traversing a very fragile, low‑bandwidth data source.  ## Preconditions to Multihop For the multihop communication process to succeed, every peer within the chain, or *connecting group*, must meet the following preconditions: - Share the same subscription - Accessible through the transport mechanism For example, if peers **A**, **B**, and **C** in the following graphic are subscribed to the same query, changes are synchronized across them, meaning all peers in the chain are aware of the same dataset: ![](https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/IQRXP8wQmDFDDnwDwL_wX_multihopovsubscribed.png) ## Multihopping to the Big Peer If a peer within the mesh network gains internet access at any point, all mesh‑generated data is automatically routed to the Big Peer through multihop. This multihop connection to the Big Peer enables a Small Peer to establish a secure and encrypted connection through one or more intermediate peers. ## Ditto Link When peers establish a mesh network connection, they create multiple encrypted internal tunnels, or *links*, using the Transport Layer Security (TLS) protocol. Nearby peers then use these TLS-secured links to transmit data. Similarly to the virtual private network (VPN), these links create virtual private connections utilizing the *Noise protocol* to establish secure and protected connections. The Noise protocol is a cryptographic framework for advanced security protections. For more information, see the official documentation at [noiseprotocol.org](http://noiseprotocol.org/). ::: # Authentication and Access Ditto's security relies on digital identities for peer-to-peer authentication and access control permissions. :::ExpandableHeading ## Offline Sync Prerequisite Before participating in a mesh and exchanging data with nearby peers, each instance of Ditto running in your app must connect to the internet at least once to establish the necessary infrastructure within the Big Peer, including generating the access credentials required to participate in the mesh and sync offline. ::: ## Peer Identity Each instance of Ditto running in your app is assigned a random universally unique identifier (UUID). This `string` of alphanumeric characters uniquely identifies your app across the Ditto platform and other integrated systems and environments, including: - Change Data Capture (CDC) in Apache Kafka, third-party databases, business intelligence tools like Microsoft BI, and so on. - Establish necessary infrastructure within the Big Peer, for instance, the Big Peer uses your app ID to identify your app before issuing your access credentials. ## OnlinePlayground Authentication Using Ditto's `onlinePlayground` identity, you can sync with other peers who share the same app ID and read and write to collections without the hassle of setting up your own authentication. # Conflict-Free Sync A challenge arises in offline scenarios when two or more peers make edits independently and the data values stored by each peer diverge over time. Referred to as *conflict resolution*, Ditto's process of addressing concurrency conflicts involves a combination of a metadata database and guiding principles, as follows. :::ExpandableHeading ## Metadata Database Each peer individually maintains its own *metadata database*. The metadata database serves as a storage repository of information essential in resolving conflicts that arise when merging concurrent changes. Within this metadata database is the *version vector*. The version vector manages the following essential details for a particular peer: - The current state of sync. - A record of the data previously sent and received. - The sequence of updates. - The timestamp of last write operation. ::: :::ExpandableHeading ## Guiding Principles Ditto adheres to the following strategies to ensure that all peers ultimately reach the same value for a particular data item: - Deterministic — As part of the strategy of eventual consistency, regardless of the order in which updates from different peers are received and merged, all peers ultimately converge to the same single value. - Predictable and Meaningful** **— Instead of arbitrarily resolving conflicting `registers` to a predefined value, the resulting merge accurately represents the original input and some rational interpretation of that input. ::: # Query Subscription System The multiple replicas that result from a single partition are spread across virtual and relevant physical server nodes; that is, the Big Peer cloud deployment and local Ditto stores of *subscribing* Small Peers. By default, Ditto does not automatically replicate data to peers. Instead, peers subscribe to changes and, using a query that you've defined, indicate the data they're interested in watching for changes. Once a change is observed somewhere in the mesh, only the delta matching your query replicates to other subscribing peers. In simpler terms, replication involves a subscribing peer selectively "pulling" data from other peers, rather than remote peers automatically "pushing" data to it. ::Image[]{src="https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/9LrwNdFRSBqeg5dyH8WyN_subscription-not-subscribed.png" size="68" width="1032" height="668" position="flex-start" showCaption="false"} Partitioning and replication are central to ensuring rapid and dependable data access for your end users. With Ditto's query subscription system, devices are able to sync both Ditto document and attachment objects efficiently and intelligently. For a direct explanation of the query subscription system that controls Ditto's data sync service, watch the following video on the functionality of subscriptions. In this video, the tech lead of Ditto's replication team discusses the synchronization process between two peers — one with an empty document set and the other with seven billion documents: ::embed[]{url="https://www.youtube.com/watch?v=X9yxTxkugw0"} # Configurations The Ditto SDK features a configuration structure called `TransportConfig`, which is stored within the main `Ditto` object. By default, if you don't make any changes to this configuration, Ditto will automatically enable all available peer-to-peer transports. If you create a new `TransportConfig,` it has three main sections: - `peerToPeer` - `connect` - `listen` Most apps will only need to configure the `peerToPeer` and WebSocket settings. The `listen` parameters provide support for unusual scenarios such as web browser clients on fully offline networks. Please use these only as directed by Ditto. If misconfigured, the listener features could circumvent Ditto’s usual encryption and access control. # Peer Discovery[​](https://docs.ditto.live/ios/common/how-it-works/mesh-network#discovering-peers) When Ditto starts syncing it advertises itself on all enabled peer-to-peer transports. This advertisement is transmitted in different ways depending on the transport but it always involves two small pieces of information. - App ID Hash - Announce All devices running the same app will have the same *App ID Hash* so they know they should connect. In the rare event of a hash collision, Ditto may try to connect to a different app, but verification will fail and the connection will be canceled immediately. If you are monitoring network traffic you might come across a Ditto Announce. This is a Ditto-specific advertisement format that contains basic information about the device. This provides hints to Ditto so that it can construct a good mesh quickly and efficiently. ```none Q1RACGXmbwOgjGSGP771 Q1 Major protocol version: 1 RA Platform: Android CGXmbwOg 32-bit Network ID jGSGP771 Device name “SGP771” ``` The Network ID is randomized each time Ditto starts up. It enables peers to identify each other with high probability across all transports. If Peer A connects to Peer B over Wi-Fi they know each other’s Network IDs, so Peer B can avoid creating a duplicate connection in the reverse direction.