Storage Challenges: Proof of Storage

One of the most compelling aspects of Pastel Network’s storage layer, Cascade, is the trustless decentralization of data storage. In Pastel, data is fragmented and randomly distributed across validators (Supernodes) operating the network. Monet 1.2 introduces Supernode Storage Challenges to further promote trustless data decentralization.

In simple terms, Storage Challenges involve Supernodes “quizzing” each other at random to ensure they are storing the required files, using a novel Proof of Storage mechanism. If a Supernode fails a Proof of Storage challenge or does not respond, they will be penalized through slashing and loss of network income.

These core concepts are fundamental to the broader Pastel Network:

• Trustless — Trust no network participant other than yourself, assuming that others may be unreliable or malicious.
• Distributed — Operations like Storage Challenges are performed by multiple active, randomly selected validators.
• Verify — The outcomes of Storage Challenge operations are verified by other fair network operators, and those verifications are further checked (Bob performs a task, Alice verifies the task to the network, Eve double-checks Alice’s verification).
• Open — Supernodes broadcast their Storage Challenges to the entire network, ensuring transparency.
• Deterministic — Each aspect of our Proof of Storage is deterministically executed as explicitly prescribed by the underlying protocol using XOR rankings and entropy from mining.

Incorporating Storage Challenges enables us to create a robust mechanism for continually verifying and validating the contents of Cascade. This ensures that Supernode operators either remain compliant or face penalties for violating the protocol (through slashing and network bans), further safeguarding the decentralization of files stored on the network.

For a deeper understanding of how Storage Challenges are implemented, we encourage you to review and contribute to our public repo.

Self-Healing Data

Storage Challenges in Cascade ensure the ongoing permanence and decentralization of data across active and compliant Supernodes. But even with Proof of Storage, what if the number of active Supernodes holding the underlying, redundant pieces of a particular file changes to the point where the entire file cannot be sufficiently reconstructed? Monet 1.2 activates the innovative concept of Self-Healing to further promote ongoing data permanence.

Cascade’s disruptive “over-provisioning” capability generates more pieces and copies of data than strictly required to reconstruct the original data. These chunks are not standard disjoint segments of data like BitTorrent but are many versions of each chunk, making them fungible. These fungible chunks of data are called symbol files.

For example, Cascade can turn a 1MB file into 12MB of 50KB symbol files. And because symbol files are each fungible, Cascade only needs to collect 1MB worth of the different 50KB files to reconstruct the original file. By generating more symbol files than necessary and by ensuring that each symbol is truly being stored via Storage Challenges, Cascade guarantees any file can be reconstructed.

In addition to over-provisioning, Cascade further reduces the risk of data loss with additional redundancy by requiring each symbol file to be stored by the 5 Supernodes that are closest to it using XOR distance. XOR is a deterministic algorithm used to define the distance between the PastelID identifier string for each active Supernode and the hash of the previous block. The Supernode’s PastelID that is “closest” to this previous block hash is then considered the “winner.” In the case of these symbol files, the 5 closest Supernodes are the winners.

However, what if these 5 Supernodes storing a set of symbol files suddenly leave the network at the same time, so that there is no chance to conduct a “handoff” of responsibility to other SNs that remain in the network? This is where self-healing comes into play.

There is a specific “seed” property in Cascade that is stored in the Pastel blockchain protocol itself. The seed enables Cascade to, after the fact, use the complete copy of the file (which can be verified by comparing it to the file’s hash — also stored directly in the Pastel blockchain) to reconstruct any of the files from the list of symbol files that were originally created. This means that Cascade can never truly lose any of the symbol files “forever” so long as the original file can be reconstructed correctly.

In practice, we use this to incorporate this concept such that whenever a symbol file goes missing, it is regenerated and redistributed to the network of Supernodes in the same way as all other symbol files: the 5 closest Supernodes to each symbol file must store each of the regenerated files.

Cascade’s self-healing property minimizes the probability of a permanent, irrecoverable loss of data through intrinsic storage redundancy.

To better understand the concept of Self Healing in practice, we encourage you to review and contribute to our public repo.

Other Features in Monet 1.2

In addition to Storage Challenges and Self-Healing, we are excited to share that Monet 1.2 will also bring the following features to Mainnet:

• Activation of Supernode Health and State Challenges: Similar to Proof of Storage, Health & State Challenges will guarantee the ongoing active and enabled state of each Supernode client on the network.
• Stabilization of Sense Protocol: Improvements to Chrome Driver usage, updates to Webtools 1.3, and implementation of additional tickets.
• Release of the OpenAPI Gateway: Production-ready Gateway will be deployed via the Mainnet with support for a number of new endpoints, asynchronous execution, Ansible support for independent deployment, and a graphical user interface for interaction.

Our Monet 1.2 Testnet Release source-code can be reviewed here.

Monet 1.2 Mainnet Release

The Pastel Network team and community are engaged in comprehensive testing of every aspect of the current release. This is to ensure that all functionalities are running smoothly, without any bugs or glitches, and can withstand different environmental and network state conditions. After rigorous unit-level, integration-based, and protocol-level stress testing, we will start upgrading existing Miners, SuperNodes, and WalletNodes, ultimately leading to the release of Monet 1.2 Mainnet.

To prepare for this release, community members will receive prior notification to upgrade their existing infrastructure to the latest release, before a specific blockHeight. Further information on the Monet 1.2 Mainnet Release will be shared soon. Stay tuned for updates.

Follow us on Twitter, Discord, Telegram, & Reddit for the most up-to-date Pastel news!

About Pastel Network

Pastel Network is a fully decentralized, developer-friendly layer-1 blockchain serving as the preeminent protocol standard for non-fungible tokens (“NFTs”) and Web3 technology.

Pastel allows for the development of third-party decentralized-applications (“DApps”) to sit on top of its Network, enabling developers to enjoy the scalable registration features, storage processes, and security of the broader ecosystem. Lightweight protocols such as Sense — which was built to assess the relative rareness of a given NFT against near-duplicate metadata — and Cascade — which conducts permanent, distributed storage of underlying NFT data — can be integrated cross-chain across various layer-1 blockchains, layer-2 protocols, or other third-party apps.

Pastel is managed by world-class developers, cryptographers, and technologists, supported alongside an experienced and extensive network of marketers, influencers, and third-party agencies. Pastel is backed by key stakeholders including Innovating Capital, a prominent venture fund.

For more information on Pastel Network, visit https://pastel.network/.

Website | Telegram | Twitter | Instagram | Github | Reddit | Discord