rfc: propose PDP offer invocation #76
Conversation
|
|
||
| There is implicit trust that the piece multihash generated by the storage node addresses the same underlying data as the blob multihash. The only way to verify the two hashes are related is to perform the hash computation, which involves reading the data. | ||
|
|
||
| An external party looking to validate a piece multihash being added to PDP would need to perform this check, in addition to asserting that the corresponding blob hash was indeed requested to be stored by the service (and accepted). Reading the data and performing the two hash computations would be time consuming and costly, so ideally an alternative can be found. |
There was a problem hiding this comment.
An external party looking to validate a piece multihash being added to PDP would need to perform this check
I'm refering to the signing service here btw if it wasn't obvious.
|
|
||
| Today we refer to data via two different hashes - a hash generated by the client (what we refer to as blob hash), and a hash generated by storage nodes as a requirement to identify data when interacting with the PDP contract. The client hash is typically sha2-256 multihash and the other is a piece multihash ([fr32-sha256-trunc254-padbintree](https://github.com/filecoin-project/FIPs/blob/master/FRCs/frc-0069.md)). | ||
|
|
||
| There is implicit trust that the piece multihash generated by the storage node addresses the same underlying data as the blob multihash. The only way to verify the two hashes are related is to perform the hash computation, which involves reading the data. |
There was a problem hiding this comment.
The only way to verify the two hashes are related
To my knowledge. Something something ZK circuit?
|
|
||
| This RFC proposes using the client generated data as a means to verify the relationship between the two hashes. | ||
|
|
||
| To do this, we propose adding two additional capabilities `space/blob/pdp/offer` and `pdp/offer`. The `space/blob/pdp/offer` invocation is sent by the client to the **upload service** and signals that the blob should be added to PDP on any node that it is stored. Note: this includes all nodes replicating the data, including nodes that may replicate the data in the future. Like the `filecoin/offer` invocation, it includes the piece hash as well as the blob hash. |
There was a problem hiding this comment.
I did consider invoking directly on the storage node, but the client actually doesn't know where to send the invocation (the URL) and also we need to authorize the upload service to make these invocations in the replication flow.
|
|
||
| To do this, we propose adding two additional capabilities `space/blob/pdp/offer` and `pdp/offer`. The `space/blob/pdp/offer` invocation is sent by the client to the **upload service** and signals that the blob should be added to PDP on any node that it is stored. Note: this includes all nodes replicating the data, including nodes that may replicate the data in the future. Like the `filecoin/offer` invocation, it includes the piece hash as well as the blob hash. | ||
|
|
||
| Once authorized, the upload service will invoke `pdp/offer` on the storage node nominated to store the blob on behalf of the space. Like the `space/blob/pdp/offer` invocation, the `pdp/offer` invocation will include both hashes, providing the node with a externally signed "proof" that the two hashes are equivalent. |
There was a problem hiding this comment.
"proof"
...because it's not proof, just an attestation of the fact (albeit not an attestation in the sense of the attestation delegations we use for did:mailto!).
hannahhoward
left a comment
hannahhoward
left a comment
There was a problem hiding this comment.
To be clear, do you propose holding PDP aggregation on the storage node until they receive this invocation?
because there is something problematic about being able to space/blob/add but not pdp-offer, which is what the node needs to get paid. IOW it's not exactly like filecoin/offer in that we don't intend for the action to be configurable apart from the space/blob/add (or, maybe we do? IOW maybe I can decide to just not worry about verifications -- the problem there being that again, payment is currently tied to PDP)
I wonder: should we just bite the bullet it and make it an additional required parameter to space/blob/add?
Alternatively, if they are seperate, should PDP/offer go all the way to the storage node or should it be retrieved seperately by the signing service (for example by pulling equality claims)
This would be my preference until we have an alternative to implement. |
Yes, we already do this, except it is coupled to the EIther way the node should not be obliged to retain the data until it receives Separately, this is a good identification of an existing problem we have (an upload without acceptance) that we need to do something about anyways.
Exactly, we have other invocations that the client makes that we don't really consider optional, e.g. we need them to upload and identify the index for the data. The incentive here is that they won't be able to retrieve without it. With I really like the idea of decoupling adding data to PDP from the process of uploading a blob. In the same way as we have decoupled indexing, filecoin, upload listings etc. |
3 participants
This RFC proposes a mechanism for verifying the equality relationship between blob hash and piece hash. TLDR; trust client provided data, since they already calculate both hashes and have no real incentive to tamper with them.
The storage node would still calculate piece hash to verify the relationship itself because it doesn't want to risk not being able to prove over the data.
📖 Preview