Truly Distributed Storage
[introduction_details details=”Nearly every NFT project completely ignores the problem of storage for the underlying NFT asset. Users incorrectly believe that NFTs are minted and exist in a decentralized manner on the blockchain. However, NFTs are really only non-fungible to the extent they refer to the actual token, not ownership of the rare asset itself. Most platforms simply outsource the storage of the actual rare asset to a third-party. In this case, the owner only really owns the non-fungible token to authenticate the file stored on a distributed protocol like IPFS or even simply the URL pointing to where the file is stored, say on Google or AWS.”] [introduction_details details=”Storing an NFT’s metadata and assets on a centralized server or relying on IPFS for the maintenance of files makes any creator or owner highly vulnerable to the loss of assets if the centralized entity were to shut down or if the protocol links were to go dead.”]
Pastel is the first NFT platform to have its own completely integrated, decentralized storage layer based on advanced technology such as RaptorQ fountain codes. Pastel ensures that the digital asset itself is uploaded, verified, and registered on the Pastel blockchain — rather than just the token with which it is minted. Through a series of smart tickets living on the Pastel ledger, artists can store their masterpieces in a distributed fashion across a variety of Supernodes as opposed to just ensuring the token is non-fungible.
Our goal is to ensure that even in 100 years, the world does not lose access to a single one of the NFTs entrusted to the Pastel network.
RaptorQ for Redundant Chunk Generation
[introduction_details details=”This sophisticated storage layer, leveraging the RaptorQ fountain code algorithm, begins by breaking each asset up in a series of redundant chunks. Every chunk contains certain random fragments of the combined file and is distributed redundantly across participating Supernodes running on the network.”]
[introduction_details details=”This sophisticated storage layer, leveraging the RaptorQ fountain code algorithm, begins by breaking each asset up in a series of redundant chunks. Every chunk contains certain random fragments of the combined file and is distributed redundantly across participating Supernodes running on the network.”]
Kademlia for Random Chunk Distribution
[introduction_details details=”The sets of chunks are then auto-distributed across the network to randomly selected Supernodes using the Kademlia DHT algorithm. This provides a useful “distance metric†that can be computed for any binary string of data and automatically eliminates the need for any of the unnecessary architecture – no complex or centralized system for deciding which node is responsible for which chunk, no iteration through Supernodes to find one with the relevant chunk, and no complicated logic for handling chunk re-allocation in the case of Supernodes entering and leaving the network.”]
[introduction_details details=”Each chunk is uniquely identified by a SHA3-256 hash, just as is done with each Supernode identifier (i.e. its public key). We determine the binary representation of each hexadecimal string (both the chunk and Supernode identifier) and compute the XOR distance between the strings. The smaller the distance in this computation, the ‘closer’ the chunk is to the Supernode in the network. Supernodes are responsible for storing chunks that are ‘closest’ to them in the network. As new chunks are created and as Supernodes enter and leave the network, this set of “closest†chunks to a given Supernode changes. What’s achieved is a completely distributed, deterministic way to self-organize into a particular network topology using random outputs.”]
Self-Healing
[introduction_details details=”Despite the redundancy and self-balance introduced above, in that a replacement Supernode is automatically found when an old Supernode leaves the network, it is still conceivable that a particular chunk could be lost forever. Possibly, a very large and sudden drop in the number of available Supernodes – as a result of market forces or an attack on the network – could wipe out all the Supernodes hosting that chunk before new Supernodes can take them over.”]
[introduction_details details=”However, if this event were to occur, there is a solution. Each chunk is uniquely determined by two items: the original data and a random seed for a given chunk. The random seed is generated when the chunks are first created. The set of these random seeds, together with the file hashes for each chunk, is also contained in the artwork registration ticket. If Supernodes on the network determine that a given chunk is no longer available, then the highest-ranked Supernode can retrieve enough LT chunks to reconstruct the original file and then use the random seed corresponding to the missing LT chunk to generate from scratch the identical chunk. This process can be verified easily by computing the file hash of the “new†chunk and checking that it matches the file hash listed for that chunk in the original artwork registration ticket on the blockchain.”]
Near-Duplicate Detection
[introduction_details details=”The standard way many crypto projects characterize NFTs is by taking the hash of the file as the ID or fingerprint of the underlying image. The problem with this method is that someone can then copy the NFT, say a piece of digital art, and reupload it to the system. This would cause the hash to completely change, even though the file is identical.”] [introduction_details details=”What is needed is a way of robustly characterizing the intrinsic nature or content of an NFT into a numerical fingerprint— one that doesn’t change materially when the same file is uploaded or even if an image is modified and re-uploaded in superficial ways such as through cropping, altering colors or brightness, or applying various photoshop-type filters like edge detection or warping.”]
Pastel’s approach to near-duplicate image detection leverages Tensorflow with
Keras Applications. The method finds all the ways of transforming each image into a fixed list of around 8,000 numbers. This vector of numbers is the image fingerprint, and it allows us to compare two images by comparing their fingerprints, which take up a tiny fraction of the space of the actual images. We can think of each fingerprint as representing a point in space, where space is 8,000-dimensional versus 3-dimensional. Images with similar content have fingerprints that are “close†to each other in this 8,000-dimensional space. We find suspiciously similar images by looking at both the correlations between a given image fingerprint vector and the database of all previously registered image fingerprint vectors on Pastel that align with open databases of images (e.g. Google).
In order to intelligently generate these 8,000 numbers, we leverage the combined efforts of Google, Open-AI, and numerous other researchers in the machine learning community. We take 5 trained deep convolutional neural net models that have been proven to score well in the famous Imagenet computer vision recognition and classification competition (e.g. ResNet Model). We transform all candidate images to fit the same 225 pixel by 225 pixel resolution, and each model outputs a set of probability estimates for a collection of over 10,000 categories. The category with the highest probability is the model’s ‘best guess’ as to the content of the image.
[introduction_details details=”Essentially, we take the output of the second-to-last layer of each of the 5 neural net models (each of which are based on different approaches and architectures), which are basically a bunch of numbers that summarize the internal state of the model when it is stimulated by the given image. We then take these 5 differentiated output vectors and combine them into a single string which serves as the final fingerprint. The neural networks are highly sensitive to the images they are shown and have “learned†the semantic content, and thus they are effectively able to characterize the essence of the content numerically – even if various changes are made to the original image. Even if every single pixel in the original image is different from the duplicate image, the system will detect a level of similarity that would be unlikely to occur by chance.”]
| Rareness | Description |
|---|---|
| Unknown Rarity | NFT is signed by the artist’s PastelID private key. |
| Rare to Pastel | NFT is signed and not similar to any images already registered on Pastel. |
| Highly Rare | NFT is signed, new to Pastel, and not similar to images on Google Images and other internet sources. |
Highly Rare:
The highest designation is certified New to the Internet. This encompasses whether or not the NFT that a user is trying to get registered has been “seen†on the internet before, or something extremely close to it. The Supernodes take the candidate image file and perform a reverse image search using google images and other free, public services such as TinyEye or DupliChecker. They then take the top results from each of these services and apply Pastel’s near-duplicate detection system on each of them. If none of these are close enough to the candidate image, the NFT earns this designation, which is included directly in the original art registration ticket. This would also be clearly conveyed to the user throughout the Pastel interface.
Candidate NFTs that satisfy the Rare to Pastel and Highly Rare standards are given the highest status or level of rareness. In this way, a user can go ahead and register a photo of the Mona Lisa, but since it would only be signed by their PastelID private key, and not Leonardo’s signature, the market might not place significant value on it. Market participants could also conclude that the Pastel NFT of a particular image that was the first one registered on the system is the “best†and most valuable. If a user is the first one to register the Mona Lisa on Pastel, then they—and only they—will qualify for the Rare to Pastel designation, and so this version might be particularly valuable.
[introduction_details details=”Collectors may prefer to spend more on an NFT that is rare at the level of the pixel patterns. Other users that are insensitive to the rarity of the NFT and view it as more of a digital collectable or as a way to support their favorite creators can buy the NFTs that are signed by the artist, even if they aren’t “rare†in this stricter sense.”]
NSFW Detection
All candidate NFTs are scanned at the time of registration by one or more “NSFW†such as Tensorflow with the open-source model from Yahoo Research OpenNSFW. When OpenNSFW is applied to an image file, the output is a number between zero (least pornographic/NSFW) and 1.0 (most NSFW). This score is included in the original artwork registration ticket and thus directly accessible in the Pastel blockchain.
[introduction_details details=”Supernode Operators can decide in their settings which images they are willing to register on the network, and which images they are willing to store file chunks for in the Pastel storage layer. For example, a very careful, cautious Supernode operator living in the United States might decide to never register or store file chunks for any image with an NSFW score of over 0.95. A Supernode operator less sensitive to local regulations may be willing to register or store chunks for all images, even if they have a very high NSFW score.”]
[core_bold_text description=”Since NSFW scores are built into the system, from a user interface standpoint, default wallet settings allow for selecting an appropriate level to ensure that the platform accounts for local regulations and is “child friendly,†while also preserving the broader permissionless aspect of the system.”]
Negligible Fees
In addition to financial risk and waste, high transaction fees outside the control of the project undermine one of the core reasons why NFTs are so disruptive: the power to make a rare digital asset accessible to anyone in the world on the internet without permission.
[introduction_details details=”A system where only artists who can spend $500 worth of ETH to make a single artwork NFT would end up excluding most digital artists in the world. We keep the cost to register an artwork minimally below $.05 – close to the level that offsets the real-world costs of providing such service (e.g., the cost of file storage in perpetuity) – ensuring that transacting is affordable irrespective of network exhaustion or the underlying price of PSL. “]
We ensure low fees in Pastel with a mechanism that automatically adjusts the required registration fees based on how much the mining difficulty has increased for PSL over time. For instance, if the network hashrate were to increase by a 10x multiple because of more miners competing, then the registration cost for a particular artwork measured in PSL would only cost 1/10th as much as it cost at the original, lower network hashrate. The network hashrate correlates highly with the underlying price of PSL over time (in the case of BTC, this has been estimated at a ~90% correlation), and thus provides a highly simple, effective manner of controlling the absolute cost of fees without introducing any external dependency.
[information_readmore title=”Introduction” link=”https://pastelnetwork.wpengine.com/overview/introduction/”] [information_readmore title=”Multi-Layer Structure” link=”https://pastelnetwork.wpengine.com/overview/multi-layer-structure/”] [information_readmore title=”Built for Builders” link=”https://pastelnetwork.wpengine.com/overview/built-for-builders/”] [information_readmore title=”User-Friendly Experience” link=”https://pastelnetwork.wpengine.com/overview/user-friendly-experience/”]