Skip to content

Export Nydus images as block devices through NBD #1111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 7 commits into from
Closed

Export Nydus images as block devices through NBD #1111

wants to merge 7 commits into from

Conversation

jiangliu
Copy link
Collaborator

Please follow the step to test block-nbd:

  • cargo build --features block-nbd
  • mkdir images
  • target/debug/nydus-image create -D images/ src/
  • modprobe nbd
  • target/debug/nydusd nbd -D images/ -B images/75611859024605a4cff41633ed6f295fe205d71f65fc9c70f849b7e976569154 /dev/nbd0 &
  • mount -t erofs /dev/nbd0 /mnt/

@jiangliu
rafs: refine v6 related code
5310318 
Refine v6 related code and add two fields to meta info.

Signed-off-by: Jiang Liu <[email protected]>
@jiangliu
rafs: load mapped-blkaddr for each data blob
77b9784 
Load the mapped_blkaddr field for each data blob, later it will
be used compose a RAFS v6 image into a block device.

Signed-off-by: Jiang Liu <[email protected]>
@jiangliu
service: add common code to compose a block device from a RAFSv6 image
d74af78 
Add common code to compose a block device from a RAFS image,
which then can used exposed through nbd/ublk/virtio-blk/vhost-user-blk
etc.

Signed-off-by: Jiang Liu <[email protected]>
@jiangliu
service: compose a block device from a RAFSv6 image
35732f9 
Compose a block device from a RAFSv6 image, so all metadata/data
content can be accessed by block address. The EROFS fs driver can be
used to directly mount the block device.

It depends on the blob_cache subsystem and can be used to implement
nbd/ublk/virtio-blk/vhost-user-blk servers.

Signed-off-by: Jiang Liu <[email protected]>
@jiangliu
service: add nbd service to export RAFSv6 images as block devices
d9589e3 
Implement NbdService which cooperates with the Linux nbd driver to
expose RAFSv6 images as block devices. To simplify the implementation,
the NbdService will directly talk with the nbd driver, instead of
following a typical nbd-server and nbd-client architecture.

Signed-off-by: Jiang Liu <[email protected]>
@jiangliu
nydusd: add subcommand nbd to export nydus images as block devices
a40bee0 
Add subcommand nbd to export nydus images as block devices through
NBD.

Signed-off-by: Jiang Liu <[email protected]>
@jiangliu jiangliu requested a review from a team as a code owner February 26, 2023 06:37
@jiangliu jiangliu requested review from bergwolf, luodw and hsiangkao and removed request for a team February 26, 2023 06:37
@anolis-bot
Copy link
Collaborator

@jiangliu , a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53042

@jiangliu jiangliu changed the title Block nbd Export Nydus images as block devices through NBD Feb 26, 2023
@anolis-bot
Copy link
Collaborator

@jiangliu , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53043

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53179

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd❌ FAIL

Sorry, your test job failed. Please get the details in the link.

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53401

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd❌ FAIL

Sorry, your test job failed. Please get the details in the link.

@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53406

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd❌ FAIL

Sorry, your test job failed. Please get the details in the link.

@imeoer
Copy link
Collaborator

imeoer commented Feb 28, 2023

Hi @jiangliu, are there docs / tests that can be given?

@jiangliu
api: add method to load BlobCacheConfigV2 from file
fe9a91c 
Add method to load BlobCacheConfigV2 from configuration file.

Signed-off-by: Jiang Liu <[email protected]>
@anolis-bot
Copy link
Collaborator

@jiangliu , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/53436

@anolis-bot
Copy link
Collaborator

@jiangliu , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd❌ FAIL

Sorry, your test job failed. Please get the details in the link.

@hsiangkao hsiangkao deleted the branch dragonflyoss:dev/v2.3 March 3, 2023 09:14
@hsiangkao hsiangkao closed this Mar 3, 2023
@hsiangkao
Copy link
Contributor

could we reopen a pr to merge this to main branch now?

@hsiangkao
Copy link
Contributor

substituted by #1139

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bergwolf bergwolf Awaiting requested review from bergwolf bergwolf was automatically assigned from dragonflyoss/nydus-reviewers

@luodw luodw Awaiting requested review from luodw luodw was automatically assigned from dragonflyoss/nydus-reviewers

@hsiangkao hsiangkao Awaiting requested review from hsiangkao hsiangkao was automatically assigned from dragonflyoss/nydus-reviewers

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jiangliu @anolis-bot @imeoer @hsiangkao