bug (blob): potential bug in appshares parsing

[renaynay]

On mocha-4, I did a valid request for blob get-all for namespace 72656e6572656e65

ubuntu@rene-no-touchy-3:~/go/src/github.com/renaynay/celestia-node# celestia blob get-all 7717345 72656e6572656e65
{
  "result": "sequence length 1649397 is greater than the number of bytes in the sequence 1649380"
}

proof it was retrieved from network, (succeeded in GetNamespaceData) but failed in appshare parsing

 TOTAL READ BYTES FROM ND STREAM:   3587617   in (ms):   168
client-nd: request took (ms):    169    err??  <nil>
shrexgetter.GetNamespaceData.request nd took (ms):  169
shrexgetter.GetNamespaceData.request nd verify took (ms):  7
cascade: getter returned value in  (ms):   177   getter  0
blobservice.retrieve.GetNamespaceData SUCCEEDED took (ms):    177
blobservice.retrieve.appshareparsing FAILED took (s):    0.010391332    err:  sequence length 1649397 is greater than the number of bytes in the sequence 1649380
blobservice.GetAll total time took:  (s) 0.188196003

[vgonkivs]

celestia blob get-all 7717345 72656e6572656e65 finds a blob with sequence length 1649397(this info is calculated on fly using go-square lib). 1649397 then converts to 3422 shares. I can confirm that the node downloads the correct number of shares. This can also be proved here by this condition. It ensures that we download the correct number of shares. Imo, we caught another edge case in go-square.

The sequence-data is calculated by appending 482 bytes of all shares in the sequence, except the first one(458 for the first share, as it contains additional info):
3421*482+458 = 1649380

[rootulp]

Findings so far:

• Related: fix: ony the first share should parse the signer go-square#163 which is included in go-square v2.3.1 which celestia-node currently uses.
• The PayForBlob that caused this uses share version 1 (a.k.a authored blobs): https://www.mintscan.io/celestia-testnet/tx/6bc0041f1b491909f17a391a6bb7a2c871cbfd91f848de68f1ca6e8cdc66e14c?sector=message
• 1649397 and 1649380 are only off by 17 bytes which is less than the size of the signer.
• It seems plausible that the sequence length is off by ~20 bytes because of the signer.
• @vgonkivs I'm pretty sure this blob is 3423 shares because I looked at the sequence start indicator for shares in the raw data. SparseSharesNeeded doesn't account for the space occupied by the signer if its an authored blob which seems like a bug in go-square (created SparseSharesNeeded does not account for space occupied by signer go-square#170). I think the celestia-node parser here is thinking 3422 shares instead of 3423 observed in my unit test.