Device Storage Management

data storage management is essential for preventing unnecessary resource usage, which affects not only performance but also battery life and overall end user experience with a soft delete pattern, you flag documents as inactive by adding the isarchived field while keeping them available if needed (see docid 2ixfv6yfqikjrobx0cujd ) however, this does not remove data from disk implementing a well thought data storage strategy can ensure that your app remains efficient, resulting in faster load times, improved battery life, and a more responsive ui using eviction use the evict api method to remove data from the local disk you will need to s et up an automatic process that evicts documents on a regular basis, such as once per day evict documents that are accessed less frequently or are no longer relevant in order to free up disk space eviction can slow down the mesh remember that before invoking the evict method, you must cancel relevant subscriptions and then restart them this results in an increase in network usage, which can degrade sync performance call evict on a regular basis, but no more than once per day during periods of minimal disruption, such as after hours and at night time to live eviction strategy implement a design pattern that focuses on efficient memory management and performance optimization when handling data in fast paced industries, such as airlines, retail, quick service restaurants, and other time‑sensitive environments the implementation process of the time based replication design pattern with a ttl eviction strategy varies based on your approach big peer management (recommended) — through regular interval http api calls, the big peer cloud deployment delegates all ttl based evictions mesh wide by opting for the big peer approach, you ensure that documents sync with the big peer before eviction to prevent any potential data loss (see docid\ coszw lyb 2nolzk8hxv ) small peer local management — each small peer device manages its own ttl based evictions this is suitable if you either are not using the big peer, or if you are confident operationally that devices will never be offline longer than the eviction process will run (see docid\ coszw lyb 2nolzk8hxv ) a common implementation is the time based replication design pattern , in which you use a time‑to‑live ( ttl ) eviction strategy to set document retention logic; for example, logic specifying that only data written to the local ditto store within the past seven days will persist on a local device example scenario imagine you're developing a meal ordering app that flight attendants use to record passenger meal orders on their devices to ensure that your app remains highly performant throughout its life cycle while also guaranteeing that flight attendants using the app always work with the most recent and relevant information, you specify a ttl of 72 hours so that only documents written to the local ditto store within the past 72 hours persist on the small peer device { "createdat" "2022 09 17t20 00 46 945z", "expiresat" "2022 09 20t20 00 46 945z", "flightno" "dit101" } big peer based eviction as ditto's recommended approach, by opting to use the big peer to manage ttl based evictions not only do you ensure that locally stored documents sync with the big peer before eviction, but you also streamline the eviction process by reducing the frequency of needing to perform subscription updates on small peers to align with the latest ttl queries to implement ttl based evictions using the big peer make a post request to the /api/v4/store/write endpoint using a top level args variable, declare your ttl include the necessary authorization headers (see docid 1zqho9hlq mo6v1aqnazl ) in your request body, indicate which documents to target for eviction (see docid\ coszw lyb 2nolzk8hxv ) configure your http server to run the ttl based eviction operation at regular intervals using http api calls for example, set it to run daily, removing all documents older than 24 hours from the current day make sure the small peer devices are evicting and subscribing to the right documents (see docid\ coszw lyb 2nolzk8hxv ) specifying targets for purge pass the update command specify the target collection; for instance, the “orders” collection using a query , define time‑based eviction criteria to select documents where the createdat time is less than or equal to $ttl ensure only eligible documents, those not previously evicted, are considered by setting evictionflag in your query to true curl curl x post 'https //{app id} cloud ditto live/api/v4/store/write' \\ \ header 'authorization bearer $api key' \\ \ header 'content type application/json' \\ \ data raw '{ "commands" \[{ "method" "update", "collection" "orders", "query" "createdat <= $ttl && evictionflag == false", 	 "commands" \[{ 	 "method" "set", 	 "path" "evictionflag", 	 "value" "true" 	 }] }] }' configuring small peers once you've configured for ttl based eviction management using the big peer, configure small peers to make sure their devices are evicting and subscribing to the right documents using the find method, evict all documents that have the evictionflag set to true using the docid\ x46m5dxeleea9hqu lznl method, execute the eviction process on the documents that match your time based eviction criteria pseudocode ditto store collection("orders") find("evictionflag == true") evict() once eviction is complete, using the find method, subscribe to all documents that have the evictionflag set to false using the subscription method , execute the sync subscription on the documents that match your query criteria make sure to remove the timestamp range from your subscription query this is because, by using the big peer for management, small peers no longer need to perform regular subscription updates to align their time based eviction criteria in their queries pseudocode subscription = ditto store collection("orders") find("locationid == 1234 && evictionflag == false") subscribe() small peer based eviction in most cases, choosing to use the big peer for ttl based eviction management is ideal; however, in select scenarios where concerns like data availability, integrity, privacy, and security require local management, opting for the small peer local management approach may be the better choice before invoking the evict method to purge documents older than the ttl, cancel any active subscriptions that may be impacted by the eviction ( see docid\ coszw lyb 2nolzk8hxv ) by stopping any subscriptions affected by the evict update, you ensure that ditto does not attempt to purge documents actively syncing across the mesh once you ensure that any ongoing subscriptions affected by the eviction update are canceled, run the eviction process (see docid\ coszw lyb 2nolzk8hxv ) once the eviction process is complete, set up a new subscription with your updated time‑based eviction criteria (see docid\ coszw lyb 2nolzk8hxv ) cancel affected subscriptions to maintain data integrity, you must cancel any active subscription that the eviction update may impact before executing your eviction process otherwise, an undesirable loop may result in which ditto continuously evicts documents from the local ditto store and then automatically re‑syncs them, leading to increased memory usage pseudocode subscription cancel() invoke eviction updates using a top level args variable, declare your ttl using the find method, define your time‑based eviction criteria using the evict method, execute the eviction process on documents matching your criteria notice that the query used for eviction is the opposite of the query used for subscription, enclosed as parameters in the find method; that is, they do not select the same documents by using opposite queries, you prevent potential conflict by ensuring that the queries you use for eviction and subscription are distinct and do not overlap pseudocode ditto store collection("flights") find("createdat <= $args ttl", args \[ "ttl" gettimetolive() ]) evict() create new subscriptions to update a subscription with your updated time‑based eviction criteria, do the following you must declare your subscription object from the top most scope of your app to ensure access throughout the life cycle of your app otherwise, you cannot modify or cancel your subscription from any part of your code, resulting in difficulty and potential errors when managing the subscription's lifecycle specify the name of the collection to watch; for example, the "flights" collection using the find method, define the query that retrieves documents where the "createdat" field is greater than the ttl using the subscribe method, initiate your subscription pseudocode subscription = ditto store collection("flights") find("createdat > $args ttl", args \[ "ttl" gettimetolive()]) subscribe()