[query] Rewrite relational index to no longer have metadata alongside index files themselves

[chrisvittal]

Every index file for every partition currently writes metadata alongside, this creates a structure like:

9.hmt/index
|-- part-0-30bcc9ce-4797-427a-a9ba-bd0c688bddce.idx
|   |-- index
|   `-- metadata.json.gz
`-- part-1-e0447c57-8794-4e14-8e4f-c996fb431427.idx
    |-- index
    `-- metadata.json.gz

This creates a ton of files that are very slow to list and operate on, in addition to being completely unnecessary as most fields of the metadata is duplicated for every partition.

This change returns the per partition variable index metadata in the partition results, and then serializes it in one file.

Security Assessment

Delete all except the correct answer:

• This change cannot impact the Hail Batch instance as deployed by Broad Institute in GCP