--- title: Data-Handling Essentials slug: v4-7/basic/data-handling-essentials docTags: createdAt: 2023-08-10T20:29:15.578Z --- This article provides a complete overview of the various data types you can use with your queries to retrieve, modify, and sync data in Ditto. In addition, you'll find an introduction to related concepts like conflict resolution strategies. # Introduction In Ditto, there's a clear distinction between traditional `CREATE`, `READ`, `UPDATE`, and `DELETE` (CRUD) database operations and data sync. ## CRUD For CRUD operations, you interact with data stored locally in the Ditto store by invoking a single operation on the `store` namespace. Once executed, Ditto returns a response object encapsulating the results. The following table provides a high-level overview of the different ways you can perform CRUD in Ditto: | **Operation** | **Description** | | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `CREATE` | Using the `INSERT` statement, either insert a new document or update an existing document for a given document ID. | | `READ` | Using the `SELECT` statement, retrieve documents based on the specific criteria you pass as parameters in your function. | | `UPDATE` | Using the `UPDATE` method, write changes to Ditto. | | `DELETE` | Using the `EVICT` method, remove local data from your store.
In addition, using a *soft-delete pattern*, indicate data as deleted without physically removing it from Ditto. | ## Sync For syncing, you set up remote listeners, or *sync subscriptions*, to monitor changes to the data you're interested in. When data changes, Ditto automatically syncs the delta changes to your local Ditto store. ## Observe In addition to setting up sync subscriptions to monitor mesh-wide data changes, you can establish local listeners known as *store observers* to monitor changes in the Ditto store operating locally. A store observer is a DQL query that runs continuously and, once Ditto detects relevant changes, asynchronously triggers the callback function you defined when you set up your store observer. For instance, when the end user updates their profile, display the profile changes to the end user in realtime. # Documents and Collections Ditto stores data in structured JSON-like document objects. Each document consists of human‑readable fields that identify and represent the information the document stores. The following snippet provides an example of a basic JSON-encoded document object: :::CodeblockTabs Ditto Document ```json { "_id": "abc123", "make": "Hyundai", "year": 2018, "color": "black" } ``` ::: A single document consists of one or more fields that self‑describe the data it encodes; each field is associated with a value: ::Image[]{src="https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/uIoehA_cK_sRrfBsdtrNZ_image.png" size="38" width="1082" height="754" position="flex-start" showCaption="false"} | **Item** | **Description** | | -------- | ----------------------------------------------- | | 1 | The name identifying the data. | | 2 | The value that holds the actual data to store. | A grouping of related documents is referred to as a *collection*. Think of a collection as a *table* in a relational database table — but with greater flexibility in terms of the data it can hold — and the documents in a collection like table *rows:* ![](https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/fcqm-eUpUunzxjkGDX_4y_document-demo-table.png) In the following structure, the `location` field property is logically grouped with details about the car, such as its `make`, `year`, and `color`. It contains nested fields. also referred to as *subfields*, representing both the `coordinates` and the `address` where the car is located: ::Image[]{src="https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/nE76-8SxgOLCjFhcWaVqc_embedded-object.png" size="72" width="2259" height="1265" position="flex-start" showCaption="false"} # Dedicated Query Language Ditto Query Language (DQL) is the platform's dedicated query language for defining criteria for local database operations and data sync with remote peers. For instance, you use DQL to perform various filter operations like traditional `CREATE`, `READ`, `UPDATE`, and `DELETE` (CRUD) and filter data for sync subscriptions.  DQL is a familiar SQL‑like syntax designed specifically for Ditto's edge-sync features that enable the platform's offline-first capabilities. DQL offers features like: - Reusable statements - Clear syntax - Advanced querying capabilities ## Query Statements Overview The following table provides an overview of the types of statements you'll write in DQL and enclose within your API method operations to specify data selection criteria: - `ditto.store.execute` — Interact with data stored locally within the Small Peer device running your app. (See [ditto.store.execute](docId\:FnY_XpTERHpNUssPLV32W)) - `ditto.store.registerObserver` — Set up store listeners to monitor changes to data in the local Ditto store based on specified query criteria. (See [ditto.store.registerObserver](docId\:FnY_XpTERHpNUssPLV32W)) - `ditto.sync.registerSubscription` — Set up subscriptions for syncing data from other peers based on their associated queries. (See [ditto.sync.registerSubscription](docId\:FnY_XpTERHpNUssPLV32W)) - `ditto.presence` — Enable end-user functionality like mesh network monitoring, management, and transport optimization. (See [Presence Operations](./#presence-operations)) ## Data Types and Scalar Types When writing queries with DQL, data formatting is categorized into two buckets: ::::::WorkflowBlock :::WorkflowBlockItem *Scalar type * A scalar data type can be a `string`, `boolean`, `array`, and any other basic primitive type. In addition, a scalar type can be a JSON `blob` object capable of nesting multiple key-value pairs functioning as a single unit. ::: :::::WorkflowBlockItem *Data type* An advanced type that guarantees conflict-free resolution at merge and includes `REGISTER`, `MAP`, and `ATTACHMENT`. ::::VerticalSplit{layout="right"} :::VerticalSplitItem As illustrated on the right, each data type includes two components: - The value to be stored — encoded using scalar types like `string`, `boolean`, and so on. - The field‑specific metadata that defines the enforced merge strategy in conflict resolution. The default data type is `REGISTER` ; you'll use other data types in specific scenarios where appropriate. ::: :::VerticalSplitItem ![](https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/2Q1Om-kNAfoliRa5LCZnW_dql-scalar-ov2.png) ::: :::: ::::: :::::: ### Specifying Data Types To use a DQL type other than a `REGISTER` — the default data type in Ditto — you must explicitly specify the type in your query; otherwise, Ditto defaults to the `REGISTER` type as follows. :::hint{type="info"} In DQL syntax, enclose `string` literals within single quotes (`''`); for example `'blue'` . ::: :::CodeblockTabs DQL ```mysql SELECT * FROM COLLECTION cars WHERE features.trim = 'Standard' ``` ::: Here is an example illustrating the same `SELECT` statement query explicitly expressed as a `MAP` structure. It specifies retrieval of the `MAP` structure storing the`"features"` collection with a `"trim"` field property set to a value of `"standard"`: :::CodeblockTabs DQL ```mysql SELECT * FROM COLLECTION cars (features MAP) WHERE features.trim = 'Standard' ``` ::: # Conflict Resolution As the foundation of how Ditto exposes and models data, these data types leverage conflict-free replicated data type (CRDT) technology to ensure that no data inconsistencies occur as a result of *concurrent modifications*; that is, simultaneous edits made to the same data types in multiples local Ditto stores. All data types —`REGISTER`, `MAP`, and `ATTACHMENT`— adhere to the *causal consistency* model when resolving concurrency conflicts. The causal consistency model is a guarantee that if there is an operation that must happen before another operation — for example, events A and B, where B is a result of A — all peers agree upon and observe the same sequential order of these operations; as in, A *always* executes before B. ## Merge Strategies In Ditto's implementation, conflicts are automatically resolved, merged, and synced across peers without the need for coordination or validation from a centralized authority. Within this consistency model, there are two principles for guiding conflict resolution at merge: - *Last-write-wins* merge strategy - *Add-wins* merge strategy The following table provides a quick overview of the data types you can use to write queries, along with their merge strategies, a brief description, and a common usage scenario: | **Type** | **Merge Strategy** | **Description** | **Use Case** | | ------------ | ------------------ | ------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | | `REGISTER` | "Last write wins" | Stores a single value and allows for concurrent updates. | Store JSON‑compatible scalar subtypes, including a nested blob representing two or more fields as a single object. | | `MAP` | "Add wins" | Contains a nested object consisting of any Ditto type: `REGISTER`, `MAP`, and `ATTACHMENT`. | Enable field-level concurrent hierarchy within a Ditto document. | | `ATTACHMENT` | "Last write wins" | Stores the token you use to retrieve the `ATTACHMENT`. | Reduce Small Peer resource usage by storing data that can be retrieved *lazily*; as in, you fetch the data only when needed. | ## Representing Complex Datasets When you want to embed a hierarchical structure to represent complex parent‑children data structures within a document in DQL, you have the option to nest either of the following: - JSON blob functioning as a `REGISTER` - `MAP` The decision between the two depends on your specific use case. For instance, to represent embedded values with dependencies, such as a GPS coordinate along with its corresponding address, structure your data in a JSON object, as follows: ```json "location": { "coordinates": [-122.0308, 37.3318], "address": "123 Main St, San Francisco, CA 94105" } ``` This is because, unlike a `MAP`, a JSON object functions as a single unit. Managing both the coordinate and address as a cohesive unit ensures that any changes made to one automatically update the other. :::hint{type="info"} Similar to the JSON object, the `array` type in Ditto acts as a `REGISTER` and therefore encapsulating values function as a single unit. Representing `coordinates` as an `array`, as demonstrated in the previous snippet, is the smart choice since its latitude and longitude values function as a unified entity, always changing simultaneously. ::: To represent embedded values with no dependencies; that is, you want the flexibility to update each key-value pair independently, and structure your data as separate `REGISTER` fields in a `MAP`: :::CodeblockTabs Ditto Document ```json "features": { "trim": "Standard" "speakers" : 5 } ``` ::: To represent highly complex data structures in which you need to establish additional hierarchies, embed a `MAP` within another `MAP`, as follows: :::hint{type="warning"} Syncing large documents can significantly impact network performance. The decision to use deeply embedded `MAPS` in a single document or opt for a flat model depends on requirements, relationships between data, and tolerance for certain tradeoffs. ::: :::CodeblockTabs Ditto Document ```json "owner": { "name": "John Larson", "information": { "address": "2147 Chestnut St, Oakland, CA 94607", "phone": "555-555-5555", "updated": "June 2, 2023" } } ``` ::: ### Distinctions by Type The following graphic and corresponding table aim to demonstrate the distinct capabilities and versatility of each DQL type. :::hint{type="info"} The `REGISTER` data type functions as the default in Ditto for scalar-encoded values. So any scalar-encoded values, including embedded JSON objects and `arrays`, are assumed to be type `REGISTER`. ::: ![](https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/atukhQqE57CpdrzzUJE0__image.png) | **Item** | **Type** | **Description** | | -------- | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | `REGISTER` | A value can be any JSON-encoded primitive type: `boolean`, `numeric`, `binary`, `string`, `array`, and`NULL`. | | 2 | `REGISTER` | A hierarchical data structure of multiple JSON-encoded fields nested within a larger JSON object and serves as a single value. | | 3 | `MAP` | A hierarchical data structure of two or more key-value pairs encoded using any data type — `REGISTER`, `ATTACHMENT`, or `MAP`. | | 4 | Response Object | The response object returned after creating a new `ATTACHMENT`.
You use the `ATTACHMENT` response object within your app's code to retrieve and display the file to the end user, as appropriate, and to update or delete the file. | | 5 | Attachment Token | The pointer that Ditto uses to reference the large file's storage location when fetching.
You can use an `ATTACHMENT` for any file type, including binary data of 50 megapixels or more, such as an mp4 file, or a large document object featuring complex hierarchical structures. | ## MAP Data Type Conflicts An issue unique to `MAP` data types is the possibility for two offline peers to create a new document, in which one peer represents the field as an object (`MAP`), while the other peer represents the field as an `array`. ### Divergent Types Preventing Merge The following snippets illustrate a scenario of a type-level conflict unique to `MAP` types. Peer A creates the following new document: :::CodeblockTabs JSON ```json { "name":"Bob Jones", "address": { "street":"Long Road", "house number":10298, "zip":"90210" } } ``` ::: While at the same time Peer B creates the following new document: :::CodeblockTabs JSON ```json { "name": "Bob Jones", "address":[ 10298, "Long Road", "90210" ] } ``` ::: ### Managing Conflicts: Update History Because peer A and peer B use divergent data structures, combining an `array` with an object (`MAP`) is impossible. Rather than adhering to the default "last updated type" win principle, which can trigger a *ping-pong* alteration of types between connected peers, as demonstrated in the following snippet, retain both values for the `address` field property by creating a data structure that accommodates both the object (`MAP`) and the `array`. :::hint{type="info"} A ping-pong alteration of types occurs when distinct peers repeatedly modify the data type of a specific field in response to each other's updates, leading to a forever loop of back‑and‑forth behavior. ::: ```json { "name": "Bob Jones", "address": { "objectVersion": { "street": "Long Road", "house number": 10298, "zip": "90210" }, "arrayVersion": [ 10298, "Long Road", "90210" ] } } ``` # Store Operations There are two ways of interacting with the Ditto store operating locally on end-user devices: - Perform a one-time execution of a `create`, `read`, `update`, `delete` (CRUD) operation against the `store` namespace. - Asynchronously monitor local changes by setting up observers against the `store` namespace. :::::ExpandableHeading ## ditto.store.execute Interact with data stored locally within the Small Peer device running your app. ::::VerticalSplit{layout="middle"} :::VerticalSplitItem On the right is a graphic illustrating CRUD operations on a Small Peer end‑user device. With these one-time operations, you perform a single action, such as modifying or retrieving data, on the Ditto store at a given time. Once invoked, you must wait for the operation to complete before continuing to the next action in your code. ::: :::VerticalSplitItem ::Image[]{src="https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/HafO31uVbYTqjFexlHKY0_data-flow-local-crud.png" size="66" width="833" height="1264" position="center" showCaption="false"} ::: :::: For example, the following single execution query, once called, uses the `execute` API method and a local `SELECT` query to search within the local Small Peer store for data in a dataset named `cars`: :::CodeblockTabs ```swift let result = await ditto.store.execute(query: "SELECT * FROM cars") ``` ```kotlin val result = ditto.store.execute("SELECT * FROM cars") ``` ```javascript const result = await ditto.store.execute("SELECT * FROM cars"); ``` ```java DittoQueryResult result = (DittoQueryResult) ditto.store.execute( "SELECT * FROM cars", new Continuation<>() { @NonNull @Override public CoroutineContext getContext() { return EmptyCoroutineContext.INSTANCE; } @Override public void resumeWith(@NonNull Object o) { if (o instanceof Result.Failure) { // Handle failure } } } ); ``` ```csharp var result = await ditto.Store.ExecuteAsync("SELECT * FROM cars"); ``` ```cpp auto result = ditto.get_store().execute("SELECT * FROM cars").get(); ``` ```rust let result = ditto.store().execute("SELECT * FROM cars", None); ``` ::: ### Creating To create a new document in your local Ditto store, call `INSERT`. Following is an example of how to perform an `INSERT` operation using Ditto's SDK. :::CodeblockTabs ```swift await ditto.store.execute( query: "INSERT INTO cars DOCUMENTS (:newCar)", arguments: ["newCar": ["color": "blue"]]); ``` ```kotlin ditto.store.execute( "INSERT INTO cars DOCUMENTS (:newCar)", mapOf("newCar" to mapOf("color" to "blue"))) ``` ```javascript await ditto.store.execute( "INSERT INTO cars DOCUMENTS (:newCar)", { newCar: { color: "blue" } }); ``` ```java DittoQueryResult result = (DittoQueryResult) ditto.store.execute( "INSERT INTO cars DOCUMENTS (:newCar)", Collections.singletonMap("newCar", Collections.singletonMap("color", "blue")), ); ``` ```csharp var args = new Dictionary(); args.Add("newCar", new { color = "blue" }); await ditto.Store.ExecuteAsync( "INSERT INTO cars DOCUMENTS (:newCar)", args); ``` ```cpp std::map> args; args["newCar"] = {{"color", "blue"}}; auto result = ditto.get_store().execute( "INSERT INTO cars DOCUMENTS (:newCar)", args).get(); ``` ```rust struct Args { newCar: Car, } struct Car { color: String } // ... let args = Args { newCar: Car { color: "blue".to_string() }, }; ditto.store().execute( "INSERT INTO cars DOCUMENTS (:newCar)", args); ``` Dart (beta) ```dart ``` ::: ### Reading Search for documents within your local Ditto store by calling the `execute` API method on a `SELECT` query as follows: :::CodeblockTabs ```swift let result = await ditto.store.execute(query: "SELECT * FROM cars") ``` ```kotlin val result = ditto.store.execute("SELECT * FROM cars") ``` ```javascript const result = await ditto.store.execute("SELECT * FROM cars"); ``` ```java DittoQueryResult result = (DittoQueryResult) ditto.store.execute( "SELECT * FROM cars", new Continuation<>() { @NonNull @Override public CoroutineContext getContext() { return EmptyCoroutineContext.INSTANCE; } @Override public void resumeWith(@NonNull Object o) { if (o instanceof Result.Failure) { // Handle failure } } } ); ``` ```csharp var result = await ditto.Store.ExecuteAsync("SELECT * FROM cars"); ``` ```cpp auto result = ditto.get_store().execute("SELECT * FROM cars").get(); ``` ```rust let result = ditto.store().execute("SELECT * FROM cars", None); ``` Dart (beta) ```dart ``` ::: Establish a listener, known as a *store observer*, to watch local changes so you can respond immediately; for example, to observe updates to end-user profiles. Here is how you set up a store observer in Ditto: :::CodeblockTabs ```swift let observer = ditto.store.registerObserver( query: "SELECT * FROM cars"){ result in /* handle change */ }; ``` ```kotlin val observer = ditto.store.registerObserver("SELECT * FROM cars") { result -> /* handle change */ }; ``` ```javascript const changeHandler = (result) => { // handle change } const observer = ditto.store.registerObserver( "SELECT * FROM cars", changeHandler); ``` ```java DittoStoreObserver observer = ditto.store.registerObserver( "SELECT * FROM cars", result -> { // handle change } ); ``` ```csharp // Without Arguments var result = await ditto.Store.RegisterObserver( "SELECT * FROM cars", (result) => { // handle change }); // With Arguments var result = ditto.Store.RegisterObserver( "SELECT * FROM cars", (result) => { // handle change }); ``` ```cpp auto observer = ditto.get_store().register_observer( "SELECT * FROM cars", [&](QueryResult result) { /* handle change */ }); ``` ```rust let observer = ditto.store().register_observer( "SELECT * from cars", None, move |result: QueryResult| { // handle change }) ``` Dart (beta) ```dart ``` ::: ### Updating Modify fields within one or more documents in your local Ditto store. For example, executing an `UPDATE` operation within the `cars` collection, changing the color to `'blue'` and the mileage to `3001` in documents where the `_id` field is `'123'`, as follows: :::CodeblockTabs ```swift try await ditto.store.execute(""" UPDATE cars SET color = 'blue' WHERE _id = '123' """); ``` ```kotlin ditto.store.execute(""" UPDATE cars SET color = 'blue' WHERE _id = '123' """) ``` ```javascript await ditto.store.execute(` UPDATE cars SET color = 'blue' WHERE _id = '123'`) ``` ```java DittoQueryResult result = (DittoQueryResult) ditto.store.execute( "UPDATE cars SET color = 'blue' WHERE _id = '123'", new Continuation<>() { @NonNull @Override public CoroutineContext getContext() { return EmptyCoroutineContext.INSTANCE; } @Override public void resumeWith(@NonNull Object o) { if (o instanceof Result.Failure) { // Handle failure } } } ); ``` ```csharp await ditto.Store.ExecuteAsync( "UPDATE cars SET color = 'blue' WHERE _id = '123'"); ``` ```cpp ditto.get_store().execute( "UPDATE cars SET color = 'blue' WHERE _id = '123'").get(); ``` ```rust ditto.store().execute( "UPDATE cars SET color = 'blue' WHERE _id = '123'", None); ``` Dart (beta) ```dart ``` ::: ### Deleting Call the `Evict` method to clear one or more documents from the local Ditto store. Once invoked, the documents are no longer accessible by local queries, however, they remain accessible from other peers connected in the mesh. The following snippet shows how to write a basic `EVICT` operation to purge the document with an `_id` field of `'123'` from the local Ditto store: :::CodeblockTabs ```swift try await ditto.store.execute(""" UPDATE cars SET color = 'blue' WHERE _id = '123' """); ``` ```kotlin ditto.store.execute(""" UPDATE cars SET color = 'blue' WHERE _id = '123' """) ``` ```javascript await ditto.store.execute(` UPDATE cars SET color = 'blue' WHERE _id = '123'`) ``` ```java DittoQueryResult result = (DittoQueryResult) ditto.store.execute( "UPDATE cars SET color = 'blue' WHERE _id = '123'", new Continuation<>() { @NonNull @Override public CoroutineContext getContext() { return EmptyCoroutineContext.INSTANCE; } @Override public void resumeWith(@NonNull Object o) { if (o instanceof Result.Failure) { // Handle failure } } } ); ``` ```csharp await ditto.Store.ExecuteAsync( "UPDATE cars SET color = 'blue' WHERE _id = '123'"); ``` ```cpp ditto.get_store().execute( "UPDATE cars SET color = 'blue' WHERE _id = '123'").get(); ``` ```rust ditto.store().execute( "UPDATE cars SET color = 'blue' WHERE _id = '123'", None); ``` Dart (beta) ```dart ``` ::: ::::: :::::ExpandableHeading ## ditto.store.registerObserver Set up *store observers* to monitor changes to data in the local Ditto store based on specified query criteria. ::::VerticalSplit{layout="middle"} :::VerticalSplitItem A store observer is a continuous DQL query — when a change to the store impacts the query results, it automatically triggers a callback that you can use to perform some action in your app. Store observers are useful when you want to monitor changes from your local Ditto store and react to them immediately. For instance, when observing end-user profile updates, you can asynchronously display their changes to them in realtime. ::: :::VerticalSplitItem ![](https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/pLvheYFA77aH2SBxNQhkY_image.png) ::: :::: ::::: # Sync Operations Establish sync subscriptions to monitor changes to the data you're interested in. The queries defining your data criteria run on all remote stores subscribed to the same sync subscription. When you subscribe to a collection, the subscription query is distributed to all relevant stores in the network. Each store executes the query locally and sends updates to the subscriber whenever changes match the subscription criteria. :::ExpandableHeading ## ditto.sync.registerSubscription Following is a graphic illustrating how to set up a sync subscription to monitor changes to the data you're interested in. First, you initiate a subscription request from your local Ditto store to the remote peer. Then, when the data you're subscribed to changes on the remote peer, such as updates, deletions, or insertions, the remote peer automatically syncs the delta changes to your local Ditto store over the mesh. ::Image[]{src="https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/2zd6mcPVLIK19hsHaNGRx_data-flow.png" size="76" width="1807" height="1264" position="flex-start" showCaption="false"} Combining store observers with your sync subscriptions ensures that you receive updates from remote stores, enabling you to quickly respond in your app: ::Image[]{src="https://api.archbee.com/api/optimize/qoRkNxW5fJ81r_NqVpc8C/sJkqm6qpAG9T1i5ZQ_02f_data-flow.png" size="76" width="1907" height="1264" position="center" showCaption="false"} ::: # Presence Operations As Ditto establishes connections and forms a mesh network using all available network transports on a device, each peer discovered in the mesh creates a *presence graph* through advertising and forming connections. The presence graph is a data structure representing the current state of the mesh from a specific peer's point of view. With Ditto's Presence API, you can implement the following end-user functionality: - Intercept and decide to accept or reject incoming connection requests. - Retrieve and view the presence graph providing near realtime status of connected peers and network resources. - Retrieve and view your local device's peer key or the peer key identifying a specific remote peer device. - Input and view personal information about yourself, such as name and role. - View the personal information set by remote peers. :::hint{type="info"} For more information, see [Handling Connection Requests](docId\:VhJyUkvY6-XgltDJBRnS7), [Using Mesh Presence](docId\:ZE725plpDADL4zThnx4Ct), and [Implementing Presence Viewer](docId\:mshpP_4atHAeMelOkLOYN). ::: # Patterns This topic provides an overview of lazy-load retrieval, memory management, ping-pong effect, and utilizing a `MAP` to sync concurrent changes. Understanding these patterns will help you make informed design decisions in your app. ## Lazy-Load Retrieval To improve performance, instead of storing a file that encodes large amounts of binary data within a document, consider storing a reference to it in a separate, explicitly fetched object (token) known as an `ATTACHMENT`. With the `ATTACHMENT` data type, you can implement *lazy loading*. Lazy loading is when you delay retrieval until necessary rather than aggressively fetching the data in anticipation of hypothetical future use. This "on-demand" retrieval pattern enhances performance by optimizing resource usage. For a realworld usage scenario, see either the demo chat app for iOS or Android in the getditto > [demoapp-chat](https://github.com/getditto/demoapp-chat/tree/main) GitHub repository. For instance, in the [iOS demo chat app](https://github.com/getditto/demoapp-chat/tree/main/iOS), you can see a savvy implementation of `ATTACHMENT` with a full-resolution avatar image from a collection named `User`. ## Memory Management Data storage management is essential for preventing unnecessary resource usage, which affects sync performance, battery life, and overall end-user experience. Eviction is important for use cases like cabin crew apps where data from the last flight is not needed on the next flight. ## Ping-Pong Effect The best approach to handle conflicts that result from two peers making concurrent offline edits and then later rejoining online depends on your specific requirements and use case. Following is an overview of best approaches for handling concurrency conflicts: - Resolving concurrency conflicts — If you want to give priority to the "latest" change, use a `register`. Ditto's `register` type use Last-Write-Wins semantics so the value written last always becomes the current value. - Auditing concurrency conflicts — If you want to keep track of the changes made by different peers over time, use the `map` type to model your list of operations. Each write operation is independently tracked as a field-value pair, with the field representing the unique identifier and the value storing only the specific changes made by a given peer. - Prompting end users to choose — If you want your end users to resolve concurrency conflicts instead of Ditto, use the `map` type inside of your document and prompt end users to select the value to replicate. ## Using a MAP for Concurrent Updates Imagine a scenario in which two Ditto stores, peer A and peer B, have the following document: :::CodeblockTabs JSON ```json { "_id": "abc123", "color": "red", "make": "Toyota", "mileage": 160000, "inspections": "" } ``` ::: Peer A calls the Upsert method to change the field-value `color:red` to `color:blue`: :::CodeblockTabs ```kotlin val initialDocument = mapOf( "_id" to "abc123", "color" to "blue" ) ``` ```javascript upsert({ _id: "abc123", color: "blue" }) ``` ::: At the same time, peer B calls the Update method to change the value of the `mileage` field: :::CodeblockTabs ```kotlin findById("abc123").update(doc => { doc.mileage.incrememt(200) }) ditto.store.collection("cars").update(initialDocument) ``` ```javascript findById("abc123").update(doc => { doc.mileage.increment(200) }) ``` ::: When the changes replicate across the distributed peers, both changes merge resulting in both peer A and peer B Ditto stores having a mileage `increment` of `200` and the `color` change to `blue`: :::CodeblockTabs JSON ```json { "_id": "abc123", "color": "blue", "make": "Toyota", "mileage": 160200, "smogReports": "" } ``` ::: ## Antipatterns Following is a table summarizing the risk associated with improperly implemented patterns: | **Pattern** | **Risks** | | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | Lazy-load retrieval | - Increased latency if resources are not efficiently preloaded - Potential resource contention from simultaneous requests | | Memory management | Memory leaks that lead to increased memory consumption over time | | Ping-pong effect | * Excessive network traffic or processing overhead * Reduced system responsiveness if components engage in a cycle of redundant operations | | Using a `MAP` for sync | Race conditions causing data inconsistency |