Fixed a rare issue decompressing attachments during replication

Improved the API Key validation of the HTTP API to now automatically retry a number of time in case of transient issues, before throwing an error to the caller 

Fixed an issue where HTTP API calls might incorrectly fail when passing a transaction ID as part of the call

Fixed a failure when executing remote queries that contained non-ASCII characters in the SYNC CONTEXT clause

Fixed an issue that prevented attachment replication when a large number of documents were also being simultaneously replicated

Fixed a number of issues related to data convergence during replication in rare scenarios

Improved the resource utilization of the HTTP API when executing DQL SELECT queries.

DQL INSERT statements run via the HTTP API now support inline object literals.

The /api/v4/attachments/{attachment_id} HTTP endpoint now supports HEAD requests to check for attachment's existence.

Improved the performance of subscriptions with limits.

Improved the performance of replication during garbage collection. 

Fixed a failure in replication in cases where there are frequent subscription changes and on-device evictions.

Fixed an issue which could cause a crash during document merges when using older versions of the SDK.

Improved query performance for DQL queries that access attachments, through the HTTP data API.

Improved handling of slow WebHook targets by enforcing a 30-second timeout to outgoing WebHook calls, this prevents lockups and other unexpected behavior when using WebHook connections.

Improved replication to avoid resending metadata, resulting in a reduction in overall replication traffic and processing when sync subscriptions change.

Added support for inline object and array definition to DQL through the HTTP API

Added object_length() and deserialize_json() functions to DQL through the HTTP API

Improved the error message for cases where the query result is too large

Removed the /{collection}/distinct_values HTTP endpoint

Removed support for X-DITTO-CLIENT-ID, X-DITTO-ACTOR-ID, X-HYDRA-CLIENT-ID from the HTTP API

Fixed a hang when replicating documents or attachments larger than 64KB

Fixed the default HTTP API response format to correctly be application/json if not specified in the request

Fixed a rare edge case that could slow down sync when using very small documents

Synchronized version bump release — no changes

Potential crash as a result of a DQL query containing a multi-byte character. (#12324)

A hang in the transports multiplexer when handling large messages spanning multiple chunks larger than 64KB. (#12428)

Attachments now sync independently of documents, improving rainbow connection efficiency (#11156)

Sync crashes due to missing update files and other unexpected conditions are now handled recoverably (#11952)

Resolved legacy query language casting counters and maps with tombstones to the wrong boolean value (#10887)

Resolved Bytes and Arrays not being always comparable in the legacy query language. (#10887)

Improve performance by avoiding deserializing the entire Document if only part of it is required. (#10887)

Resolved UST initial polling unbounded error retry. (#11841)

Removed is_connected_to_big_peer field from the Small Peer Info document. (#11883)

Resolve legacy query language accepting invalid field names. (#11888)

Increased the limit on the allowable depth of the Small Peer Info JSON metadata. (#11891)

Attachments are more performant when working with many attachments concurrently

Resolved HTTP API OpenAI spec that will cause duplicated Authentication header in certain generated clients. (#11936)

Spread work more evenly across Big Peer store writer threads. (#11939)

MeshChooser now supports a limited set of runtime configurable values via the mesh_chooser_* system parameters. (#11968)

A warning message that appears when peers connect is no longer logged (No handle found to notify the shutdown; multiple message handling loops??) (#11977)

When replicating in mixed meshes from V4 to V3 full state will be sent. (#12029)

Lowered the log level of some SQLite3 notices that were being unnecessarily logged as warnings. (#12032)

The Small Peer Info collector no longer gets stuck waiting to upload logs under certain conditions. (#12070)

Duration parameters can now be set via DQL, whereas before they would fail to type check. (#12070)

Ensure document cache GC always progresses regardless of configured time slice and load. (#12091)

String functions byte_length and char_length. (#12147)

Revert "Fix DocumentCache cache wrong result for Sort + Limit queries. (#9435)

Query plan for queries using the contains function with an empty array. (#12010)

Stream-splitter consumer migration applying old offset. (#11985)

Update stream-splitter to respect Ditto login retry interval and attempts. (#11785)

Stream-splitter checks for kafka.enabled before spawning user-consumable streams. (#11788)

Tweak hydra-cdc-heartbeat to avoid stuck consumer. (#11815)

Application crashes while preparing to syncing data to remote peer. (#11843)

Fix Subserver caching wrong result for queries with sort and limit. (#9435)

DQL queries can now query for the id, len and metadata fields of attachments (#11481)

Improve UST polling resilience in HTTP API and subserver (#11701)

Do not error in DQL inserts with the DO NOTHING conflict policy (#11730)

Synchronization protocol enhanced to prevent accumulation of unsubscribed documents after subscription change, and limits redundant data exchanged when re-synchronizing. (#10440)

Reduce metadata lock contention while applying transactions (Avoid metadata read lock in apply_txns_internal #11654)

Fix race between applying of transactions and reconfiguration of replicas (#11361)

Legacy query language casting counters and maps with tombstones to the wrong boolean value

Legacy query language not allowing comparison against attachment objects.

Bug where certificate request would not check identity data hash with JWT’s

Bug where the certificate request would fail with big permissions 

An issue where an internal data decoding function could panic on certain inputs (#10376)

Improve read query performance of legacy contains([..], _id) syntax (#10987)

Parser error messages emphasize the location of the error and de-emphasize a potential fix. (#11040)

Return 500 instead of 401 upon internal error while validating API key. (#11178)

A bug where Link could run into an infinite loop. (#11188)

Reduce HTTP API latency with improved lock map version tracking (#11278)

Improved read query performance for DQL queries using array_contains (#11338)

Make actix-web keep alive timeout configurable for Big Peer (#9578)

Preliminary support for runtime configuration using DQL ALTER SYSTEM and SHOW statements (Hook up DQL entrypoint to parameter store #10401)

Peers use latest supported protocol version, even if the application is using backwards-compatible CRDT formats. (Disentangle protocol version from protocol flags #10875)

Fix deadlock when replicating attachments to Big Peer (Fix AttachmentsPeer deadlock when setting up outbound backoff #11036)

Improve read query performance on queries with multiple ranges (#10605)


Fix incorrect DQL literal comparison behavior in big peer planning. This fixes a correctness issue where big peer could return no results when it should return all documents in a collection. This also makes big peer avoid a full scan in certain cases where it should return no results. (#10940) 

SDK version/language/platform and device name are now included in the small peer info document. (#10515)


Improved read query performance by pre-filtering non-matching document ids (#10620)

Improve memory usage and performance of Documents (#10736)

Performance by implementing bipartite distance-aware mesh topology (#10632)

Mesh would choose random connections (#10711)

Fix `NOT` on non-Boolean panics on big peer (#10762)

Fixed a bug with Ditto's internal crash handler (#10022)

Initial logging events now use the same format as all other logs (#10611)

Fix hashing bug in Big Peer store node controller 

Fix a bug preventing updates to clusters from rolling out properly.

Fix bug related to Kafka transaction chunking

Improved run-time validation of corrupted attachments being inserted into documents (#10552)

Deprecated experimental time-series HTTP APIs

Big Peer queries matching on nested objects or arrays return the expected documents (#10382)


Addressed a warning from SQLite that sometimes occurs on startup about a schema change (#10501)

Released: Mon Oct 30, 2023

Fixed HTTP API output format for Attachments when using SerializedAs::LatestValues

Remove doc_ids from requery_required message in CDC

Reduced metadata accumulation after eviction by clearing remote summaries

Released: Wed Oct 11th 2023, 5:43 pm

Add attachments support for HTTP API RPC v4

Released: Thu Sep 21st 2023, 3:40 pm

Optimized Big Peer internal data serialization performance.

Optimized the updates of internal system collections when doing writes with the HTTP RPC API.

Fixed a bug around where queries could return an older version of documents around a server reboot.

Released: Thu Sep 14th 2023, 3:40 pm

Made the Big Peer more resilient to Kafka broker disconnects

Improve our HTTP API implementation to reduce Timeout related errors

Update our OpenAPI specification to include the new count request.

Released: Wed Aug 16th 2023, 11:48 a.m.

Fix bug in Big Peer device certificate CA rotation process

Released: Thu Jul 27th 2023, 5:22 pm

Improved performance when multiple peers with distinct permissions work with the same cloud endpoint

Released: Tue Jul 25th 2023, 5:20 pm

Various fixes and improvements for better reliability, security, and performance

Released: Wed Jul 19th 2023, 5:27 pm

Minor bug fixes and performance improvements.

Changed: new serialization format improves efficiency of replication, particularly in the Big Peer

Released: Thu Jul 13th 2023, 8:27 am

Fix detection of broken stream when there are multiple apps in the transaction log

Released: Fri Jul 7th 2023, 5:06 am

Initial release of the lazy deserialization optimization for the Big Peer.

Tighten network policies for increased security and stability

Released: Tue Jun 27th 2023, 1:53 pm

Fix bug where Big Peer would sometimes panic if an SDK peer managed to create multiple Ditto instances with the same persistence directory. We are seeing this error more than we originally hoped. This change makes any occurrence non-fatal until we fix the root cause. There will be another followup change that will re-enable alerting for this event.

Released: Tue Aug 23rd 2022, 1:27 pm

Fix bug where HTTP API upserts would not always be reflected in subsequent find requests.

Properly consider limit+offset+order_by for active queries

Released: Thu Aug 18th 2022, 1:27 pm

Minor bug fixes and performance improvements

Released: Sun Aug 14th 2022, 1:08 am

Supports Ditto SDK through version 2.0.x. 

Released: Wed Aug 3rd 2022, 1:08 am

Released: Mon Aug 1st 2022, 1:08 am

Add collection and document id to RequeryRequired requests as part of the live query WebHook and CDC.

Release Notes