ATTACHMENT

this article provides conceptual information for the attachment data type for instructions, see crud operations > docid\ x59fgseg99p3qozbszd8v , docid\ yswoinf8jdl zdf6ovbgw , docid\ yexrv69kde e xykjr8mv , and docid 4 uqdixl6xnddh0bjzywt use case the attachment data type lets you link extensive amounts of binary data — such as images, videos, deeply embedded documents of 50 megapixels or more — as well as any other type of file that changes rarely to a document this linking allows for on demand sync, both online or offline, without any conflicts you can create and transmit attachments across peers, even with intermittent connectivity; however, unlike documents, which are always accessible and automatically sync when relevant per sync subscriptions, peers do not automatically fetch the attachment data associated with a subscription — so a ttachments must first be explicitly fetched before you can access them for a realworld usage scenario, see either the demo chat app for ios or android in the getditto > https //github com/getditto/demoapp chat/tree/main github repository structure overview when incorporating an attachment into a document, rather than inserting the entire resource heavy attachment object into the document, internally ditto inserts a pointer object, known as the attachment token (see /#attachment object and /#attachment token ) attachment object the attachment object, shown on the right, stores your attachment outside of the local device, is not directly visible in your code, and consists of metadata and a binary large object ( blob ) store the metadata component is a set of string values providing extra details about the file or complex document, such as name and size, among other identifying information the blob store is an array of bytes representing your large file or complex document encoded as an attachment data type (see /#blob store ) for example t he following snippet in json demonstrates the structure of an attachment object storing a jpeg file named "car stock photo jpg" in this structure t he blob encodes the image the metadata contains extra details about the binary contents, including its multipurpose internet mail extension ( mime ) type and name as a set of string values structure { "blob" \[ ], // array of bytes "metadata" { "mime type" "image/jpeg", "name" "car stock photo jpg" } } mime, also known as internet media type , is a standard identifier for specifying the structure and format of a document or file it enables web browsers and other software systems to interpret and handle binary data appropriately when transmitting it over the internet ( see the official https //developer mozilla org/en us/docs/web/http/basics of http/mime types ) attachment token the attachment token, shown on the right, encapsulates information ditto uses to identify and retrieve the full attachment when needed; as in you've explicitly fetched it in your app for example h ere is an attachment token structured within a document in this structure, the picture attachment token acts as a placeholder for the full attachment storing a jpeg file named car stock photo jpg document { " id" "abc123", "make" "hyundai", "year" 2018, "color" "black", "picture" { "id" "\<unique identifier>", "metadata" { "mime type" "image/jpeg", "name" "car stock photo jpg", }, "len" 123 } } schema the following structure and corresponding table provide an overview of the schema the attachment token uses to model the data it contains json { "id" "string", "len" 42, "metadata" {} } here are the attachment token properties you can query to increase the efficiency of fetching operations in your app field detail description id attachment id a cryptographic hash of the attachment's binary contents (see /#organizing by id ) len blob length the size of the blob data in terms of length ( len ) in bytes (see /#blob store ) metadata metadata additional information about the attachment, such as its name, type, and so on blob store a json blob stores binary data representing any file type for example, it can store image files like jpeg, png, and tiff; video files such as mp4; audio files, including mp3 and wav; document files like pdf; and more the metadata is stored in the document, while the blob is stored in a dedicated location independent of documents the following table provides an overview of blob storage locations type blob store location small peer if running in the browser or a server based system, in memory storage specifically, within random access memory (ram) if running on a mobile device like an iphone, filesystem storage big peer cloud object storage service amazon webservices (aws) simple storage service (s3) organizing by id ditto organizes and stores blob data using the content addressable network ( can ) paradigm can is a distributed network architecture widely implemented in decentralized systems like ditto's where scalability, efficiency, and fault tolerance are essential requirements (see the official https //en wikipedia org/wiki/content addressable network wikipedia article ) in more straightforward terms, ditto organizes and accesses attachment data by the cryptographic hash table of its contents — for instance, attachment id — rather than its physical location a hash table is a data structure that stores key value pairs by adhering to the can framework for attachment management, ditto stores and retrieves data efficiently for instance deduplication — if the same blob is created multiple times, ditto stores only one copy similarly, if an attachment is added to multiple documents, they all reference the same blob conflict free s ync — blobs are stored based on the cryptographic hash of their contents, ensuring that conflicts are effectively impossible decentralized — no central server is required to manage attachments they can be created and accessed concurrently throughout the network, even with intermittent connectivity purging blob data blob storage is managed internally using a reference counting process known as garbage collection the garbage collection process running periodically in the background of small peers frees up space by removing blobs that no longer have any references — once an attachment has no remaining references, its blob is considered safe to remove, and the garbage collection process clears it from the device syncing attachment objects the two components comprising an attachment object — metadata stored internally within a document and blob data storing the actual binary data of your attachment — sync in different ways metadata syncs just as any other document blob data syncs with an entirely different protocol than documents; one that is asynchronous and built on top of ditto's rainbow connection technology and optimizes for large binary data transfer (see docid 7w61tegftlq39qfc8dusj ) attachment blob sync being asynchronous provides three key advantages unblocking fetching operations so document continue syncing without interruption when retrieving an attachment ditto multiplexes between these independent activities to balance needs for low latency and high throughput at the edge blob sync is resilient, so if connectivity is interrupted while transferring an attachment, progress is not lost once restored or if another peer possesses the attachment, ditto resumes the transfer from the point of interruption blob sync remains performant — ditto uses data structures like bloom filters to scale efficiently and remain light on system resources bloom filters are randomized data structures designed to represent sets in a compressed form, making them highly effective in peer to peer environments where concerns such as memory usage and accuracy are prevalent (see the official https //en wikipedia org/wiki/bloom filter wikipedia article ) garbage collection when an attachment token stored on a small peer end user device is unused, meaning there are no remaining references to it, a background process that runs periodically, every 10 minutes, automatically deletes it to free up space this reference counting process, known as garbage collection, only runs on small peer devices (not on the big peer) it involves scanning for objects, including blobs, that are no longer needed or relevant and are therefore safe to delete from its device the big peer stores its data in ditto's third party cloud object storage service provider, amazon webservices (aws) simple storage service (s3) for more information, see the official https //docs aws amazon com/s3/ lazy load retrieval pattern since the blob is stored outside the local ditto store, fetch it only when needed referred to as lazy loading , this "on demand" retrieval pattern reduces resource usage on end user devices for a realworld usage scenario, see either the demo chat app for ios or android in the getditto > https //github com/getditto/demoapp chat/tree/main github repository for instance, in the https //github com/getditto/demoapp chat/tree/main/ios , you can see a savvy implementation of attachment with a full resolution avatar image from a collection named user merge strategy last write wins the register and the attachment type follow the last write wins principle to determine the value that ultimately takes precedence and merges across distributed peers with this approach, each peer seeing the change will sequence it in the same order as every other peer, resulting in the same single value for example, one flight attendant updates a customer's seat number to 6 and another to seat 9 when the two conflicting versions merge, the edit with the highest timestamp wins put another way, by enforcing the last write wins merge strategy, for events a and b, where b is a result of a, event a always occurs before b described as unsurprising and meaningful , the last write wins strategy means that, rather than arbitrarily resolving all conflicting changes to some predefined value, the value that results and propagates across peers as the single source of truth both reflects the actual end user input and the update made by the last writer creating and working with attachments there are two separate steps in the process of creating an attachment generate the attachment in the ditto store (see docid\ x59fgseg99p3qozbszd8v ) reference the attachment token in a document (see docid\ x59fgseg99p3qozbszd8v ) to access your attachment data on demand or perform other operations, such as updating or deleting the attachment, first fetch its associated attachment token (see docid\ yswoinf8jdl zdf6ovbgw )