Network Deployment Guide

this article is intended for network systems administrators and engineers deploying ditto in their specific network environment and provides essential information about different transports and configurations, networking concepts, and various wireless protocols ditto lan the ditto local area network (lan) uses both multicast domain name system (mdns) and dns service discovery (dns sd) to automatically discover and communicate with other ditto peers for more information, see the official documentation for https //datatracker ietf org/doc/html/rfc6762 and http //www dns sd org/ when you have multiple network segments and want to ensure ditto peers automatically discover each other, configure the mdns multicast frame as follows ipv4 multicast address 224 0 0 251 ipv6 multicast address ff02 fb port udp 5353 dns sd service type http alt tcp mdns service announce message example ditto\<hashed app id> announce http alt tcp local peer discovery and data sync ditto uses various transports to sync data, all of which utilize the 2 4 and 5 ghz 802 11 device antenna, commonly known as wi fi however, there are important differences between these transports to consider when configuring and debugging your network peer to peer wi fi apple wireless direct link (awdl) and wifi aware are peer to peer links which do not require any 802 11 access points, switches, or routers in these transports either data is sent on 802 11 frames directly or one or more devices act as a wireless access point importantly, no switch, router, or firewall is present on the networking path between devices instead device discovery occurs through direct observation of bluetooth, 2 4ghz, or 5ghz 802 11 network traffic awdl and wifi aware are not implemented or configured by the ditto sdk, but are services provided by the underlying operating system thus, if any awdl service is working properly (i e airplay), then ditto peer discovery and data sync via awdl should also work provided permissions are correctly configured note awdl can only discover other awdl (ios & macos) peers and wifi aware can only discover other wifi aware (android) peers without a lan transport, the only common link between these devices is bluetooth low energy (ble) which has limited bandwidth for data sync when possible, we recommend permitting lan transport for best performance access point wi fi (802 11) the lan and websocket transports also use the device’s 2 4 ghz or 5ghz antenna at the physical layer and so are also considered “wifi”, but these peer devices are part of a layer 3 ipv4 or ipv6 network which adds important packet routing and filtering capabilities wireless access points, switches, routers, firewalls, gateways, dhcp servers and other networking hardware are present on these networks and responsible for assigning ip addresses to ditto peers in more technical language, the “lan” transport is actually a stack of ip + mdns + tcp + mtls with an optional extension to web sockets when dns resolution of peers is desired there are some key points to bear in mind ditto peer discovery occurs via mdns packets sent over udp port 5353 to a multicast ip address each ditto small peer will listen for and respond to mdns queries on udp port 5353 by default these udp datagrams provide the ip address and tcp port that each small peer is listening for lan (tcp) client both the mdns query and mdns response datagrams are required to discover a peer many network switches do not forward these frames because they can lead to network congestion if misconfigured forwarding of these multicast frames must be enabled to allow automatic ditto peer discovery further ip based networks may be segmented into subnets and vlans, preventing devices from discovering each other both tcp and websocket transports implement mtls encryption within the peer to peer connection you should consult your access point and network switch documentation how to enable multicast forwarding this may also be called “avahi proxy” or “mdns proxy” mdns security if you are concerned about mdns security, forward only the ditto multicast frames they look like `ditto\<hashed app id> announce http alt tcp local` stale mdns announcements are typical, but ditto sdk does ignore stale announcements so duplicate messages to unavailable devices are avoided if mdns peer discovery is not permitted on a network, then ditto may be configured to use either static ip addresses or traditional dns discovery this will need to be provided by your dhcp and dns servers this is generally not scalable for large numbers of dynamic peer devices, but may be a suitable fallback for some applications transport configuration each ditto peer is capable of acting as both a client and server for each ditto transport this means a ditto peer can listen for tcp and websocket connections from a particular ip address on a particular port this is configured in the ditto sdk through the transportconfig struct\<link> by default, a ditto peer will attempt to connect to the ditto big peer via websockets by resolving wss\ //\<app id> cloud ditto live on tcp port 443 peers will not listen for incoming websocket connections by default, but this can be enabled you can manually add other peers acting as servers either using their tcp/ip address or a websocket url a ditto peer is unable to send an mdns update message when it is turned off or goes out of wireless network range, so stale mdns announcements are typical network ports used by ditto network firewalls need to permit outgoing web socket connections on tcp port 443 if the ditto big peer is in use outgoing https traffic to the ditto auth provider; as in, https //\<app id> cloud ditto live on tcp 443 is also required how to connect externally into a ditto mesh running on a local network you can connect to ditto peers running on a local 802 11 ipv4 network externally by permitting incoming tcp or websocket connections the target (listener) device typically needs to have a static public ip address, a dns resolvable ip address, or appropriate network address translation configured in the network gateway the default web socket port 443 is privileged on most operating systems, so selection and configuration of an unprivileged port is generally advisable note that this device will be exposed to the public internet which may not be part of your security threat model you should take appropriate security precautions for this reason the websocket transport may be preferable or better supported both tcp (lan) and websocket transports provide mtls on the inside for authentication and encryption after the transport connection is established wireless infrastructure wifi is very sensitive to interference and the number of clients while ditto does its best to be bandwidth efficient, meshes are inherently more bandwidth intensive than a typical hub and spoke server model for larger meshes (eg more than 30 devices), lan issues can start to really play a role in performance degradation through our own testing, we’ve learned a couple of useful tips avoid 2 4ghz when possible it’s a very congested band optimize your radio channel selection check that you’re not accidentally operating in an already congested space enable automatic channel selection if it makes sense for your environment design your wifi infrastructure to handle about 30 clients per radio to clarify, a typical wifi ap has 2 or 3 radios (2 4ghz, 5ghz, and recently 6ghz) if you have 100 devices, plan on setting up 4 access points in your building, and ensuring that the devices balance out connections across the aps (something that can be obnoxiously hard to orchestrate automatically…) ensure that the connections between the aps and the switches are high bandwidth and low latency, as the aps will be communicating large amounts of traffic between each other to reach other peers experiment with enabling multicast to unicast conversion for your wifi infrastructure, often called “multicast enhancement” or something similar multicast traffic over wifi is a complicated beast, and in most circumstances, it’s actually beneficial to convert multicast packets to unicast before sending them over the radio see this stackexchange explanation for more explanation well known wireless manufacturers will advertise their aps as capable of supporting “100 clients”, but what they really mean by this is an ideal case of 34 clients on 2 4ghz, 33 on 5ghz and 33 on 6ghz 100 clients all on 5ghz is a severely degraded experience debugging techniques /#recommended approach use ditto demo app presence viewer /#other tools recommended approach use ditto demo app presence viewer the easiest way to verify your wlan is working properly is to install a ditto demo app such as sky service on several test devices the skyservice app can be downloaded from the apple and google app stores these should be connected to the site’s wireless access point and have bluetooth disabled from the settings menu, you’ll find the presence viewer which will show discovered peers verify that peers connected to different wireless access points can discover each other and are visible in the presence viewer other tools wireshark you can use the network traffic capture and analysis tool to directly observer the ditto announce messages on a variety of interfaces this powerful tool is available on windows, mac, and linux generally, it is sufficient to capture filter udp port 5353 and use the built in mdns display filter you may need to make multiple captures at different points in your network here is an example of a ditto mdns query in wireshark and here is the associated ditto mdns response you must see both the request and the response if the response is missing, it is likely network hardware is blocking your mdns request