ipfsdhtdistributed-filesystem

How does IPFS store small files (less than 1KB)


The white paper states:

Small values (equal to or less than 1KB) are stored directly on the DHT. For values larger, the DHT stores references, which are the NodeIds of peers who can serve the block.

What does mean: "stored directly"?

Does it mean that is it just stored locally or is it broadcasted to all peers?


Solution

  • The DHT is somewhat like a phone directory. If you know the content ID (the multihash), the DHT allows you to find the peer that stores the actual content.

    This "phone directory" is so large that no single peer can hold it in its entirety. That's one of the reasons why it is distributed — each peer does actually hold only a small chunk of the directory. There is also some redundancy, so that the global directory doesn't lose information when a single peer goes offline.

    When some content is added to the network, some chunk of the distributed directory is updated with its ID and the address of the peer that holds it. When you later want to find that content by its ID, you first have to find the peer that stores the right chunk of the DHT, which will tell you where the actual content can be downloaded.

    However, if the content is so small that it can fit into a single "line" in the directory (1 KB), it is stored right in the directory chunk. When you look up that content, The peer that stores ids directory entry can serve it directly, instead of pointing you to another peer.

    Therefore, only if the content is smaller than a certain threshold, it is actually "broadcasted" to some peers (but not all of them). If the content is larger than that, only the information where to find it is broadcast.