nydus-backend-proxy reload blob objects map #556
Conversation
Current code is hard to implement blobs fd map reload since blobs manager has to be mutable among different futures. Signed-off-by: Changewei Ge <[email protected]>
|
@changweige , a new test job has been submitted. Please wait in patience. |
|
@changweige , The CI test is completed, please check result:
Congratulations, your test job passed! |
||||||||||||||||||||||||
Signed-off-by: Changewei Ge <[email protected]>
Collaborating system probably put files after the server starts. So the nydus-backend-proxy http server is better to re-scan files if it can't find the requested object. Signed-off-by: Changewei Ge <[email protected]>
|
@changweige , your pull request has been updated. A new test job will be submitted. Please wait in patience. |
|
@changweige , your test job has passed, and no need to test again. |
changweige
changed the title
|
I already ran nydus-test using this nydus-banckend-proxy as storage backend without any data corruption failure. |
| blobs: HashMap<String, fs::File>, | ||
| } | ||
| impl BlobBackend { | ||
| fn populate_blobs_map(&mut self) { |
There was a problem hiding this comment.
Is it necessary to load the whole blob directory if the blob is not found on fetching?
There was a problem hiding this comment.
My thought is to load all new files into fd map after re-scanning since the nydusd-backend-proxy 's exports all the files int blobsdir , which I think is its SLO
Current code is hard to implement blobs fd map reload since blobs manager has to be mutable among different futures.
At present
nydus-backend-proxyonly loads blobs info when it starts byreaddir(). So blob which is generated and stored in theblobsdircan't be found responding to the HTTP requests.This PR refactors the proxy a bit and prepares to reload blobs, it will benefit nydus-test since nydus-test is testing container images with multiple layers.
With blob fd map reload, backend-proxy will try to
readdiragain when blob is not found.