nydus-image: fixed a bug caused by empty blob and prefetched combined scenes #495
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zyfjeff
force-pushed
the
fix-prefetch-bug
branch
5 times, most recently
from
98efc1c to
572bbdf
Compare
… scenes
When building a layer independently, if the layer does not produce any blobs, but the prefetch is set,
the position of the prefetch table and the position of the inode table will overlap,
resulting in the prefetch table overwriting part of the inode table.
blob table|prefetch table|inode table
blob table + prefetch table is a whole, 4K alignment, if blob table size is equal to 0, but prefetchtable is not equal to 0,
will cause the prefetch table to exist, but will not be 4k aligned, and overlap with the inode table.
Related code
let orig_meta_addr = bootstrap_ctx.nodes[0].offset - EROFS_BLOCK_SIZE;
let meta_addr = if blob_table_size > 0 {
align_offset(
blob_table_offset + blob_table_size + prefetch_table_size as u64,
EROFS_BLOCK_SIZE as u64,
)
} else {
orig_meta_addr
};
let root_nid = calculate_nid(
bootstrap_ctx.nodes[0].offset - orig_meta_addr + meta_addr,
meta_addr,
);
Signed-off-by: zyfjeff <[email protected]>
zyfjeff
force-pushed
the
fix-prefetch-bug
branch
from
572bbdf to
56d21d2
Compare
zyfjeff
changed the title
jiangliu
approved these changes
Jun 15, 2022
imeoer
reviewed
Jun 15, 2022
src/bin/nydus-image/core/blob.rs
Outdated
| self.dump_meta_data(blob_ctx)?; | ||
| // Dump is only required if there is chunk in the blob | ||
| if has_chunk { | ||
| self.dump_meta_data(blob_ctx)?; |
There was a problem hiding this comment.
Better to check blob_ctx.compressed_blob_size or blob_ctx.chunk_count in dump_meta_data, so that we can reuse this logic for stargz build in future.
imeoer
reviewed
Jun 15, 2022
tests/builder.rs
Outdated
| ).unwrap(); | ||
| } | ||
|
|
||
| pub fn build_empty_file_with_prefetch(&mut self) { |
There was a problem hiding this comment.
Why not follow build_lower and build_upper (create new one named builder_empty, and enable prefetch for all test cases)? so that we can reuse the test func in tests/smoke.rs.
There was a problem hiding this comment.
Bugs only occur when prefetched and empty files are combined, so there are no build_lower and build_upper used here
There was a problem hiding this comment.
This case and other tests have now been combined.
zyfjeff
force-pushed
the
fix-prefetch-bug
branch
2 times, most recently
from
ee741ee to
dcfd0dd
Compare
Signed-off-by: zyfjeff <[email protected]>
zyfjeff
force-pushed
the
fix-prefetch-bug
branch
3 times, most recently
from
a37f49b to
8145b40
Compare
Signed-off-by: zyfjeff <[email protected]>
zyfjeff
force-pushed
the
fix-prefetch-bug
branch
from
8145b40 to
f5161a1
Compare
imeoer
reviewed
Jun 16, 2022
src/bin/nydus-image/core/blob.rs
Outdated
| self.dump_meta_data(blob_ctx)?; | ||
| // Dump is only required if there is chunk in the blob | ||
| if blob_ctx.compressed_blob_size > 0 { | ||
| self.dump_meta_data(blob_ctx)?; |
There was a problem hiding this comment.
Move the if blob_ctx.compressed_blob_size > 0 check into dump_meta_data should be better.
imeoer
reviewed
Jun 16, 2022
Signed-off-by: zyfjeff <[email protected]>
zyfjeff
force-pushed
the
fix-prefetch-bug
branch
from
dd701a6 to
ce79edd
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When building a layer independently, if the layer does not produce any blobs, but the prefetch is set,
the position of the prefetch table and the position of the inode table will overlap,
resulting in the prefetch table overwriting part of the inode table.
Signed-off-by: zyfjeff [email protected]