RFC-0561: List metadata reuse

[ClSlaid]

Signed-off-by: ClSlaid [email protected]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Open an RFC for adding metadata to DirEntry

related: #558

thanks to: @sandflee

[Xuanwo]

And for this proposal's naming, list_with_metadata doesn't reflect the thing we want to do. How about reuse_metadata_during_list?

[ClSlaid]

Still not ready for review

[ClSlaid]

There are two ways of implementation:

1. add a lightweight option structure storing metadata to DirEntry

struct MetaLite {
    pub content_length: u64,  // size of file
    pub last_modified: OffsetDateTime,
    pub created: OffsetDateTime,    // time created
}

pub struct DirEntry {
    acc: Arc<dyn Accessor>,
    
    mode: ObjectMode,
    path: String,
    
    // newly add metadata struct
    metadata: Option<MetaLite>,
}

impl DirEntry {
    // get size of file
    pub fn content_length(&self) -> Option<u64> {
        self.metadata.as_ref().map(|m| m.content_length)
    }
    // get the last modified time
    pub fn last_modified(&self) -> Option<OffsetDateTime> {
        self.metadata.as_ref().map(|m| m.last_modified)
    }
    // get the create time
    pub fn created(&self) -> Option<OffsetDateTime> {
        self.metadata.as_ref().map(|m| m.created)
    }
}

2. extend DirEntry with multiple optional metadata fields

pub struct DirEntry {
    acc: Arc<dyn Accessor>,

    mode: ObjectMode,
    path: String,

    // newly add metadata fields
    content_length: Option<u64>,  // size of file
    last_modified: Option<u64>,
    created: Option<u64>,    // time created
}

impl DirEntry {
    // get size of file
    pub fn content_length(&self) -> Option<u64> {
        self.content_length
    }
    // get the last modified time
    pub fn last_modified(&self) -> Option<OffsetDateTime> {
        self.last_modified
    }
    // get the create time
    pub fn created(&self) -> Option<OffsetDateTime> {
        self.created
    }
}

I prefer the first one because it provides a compacter memory layout. The first one saves 8 bytes of space.

Implementing this RFC will increase the size of DirEntry from 40 bytes to 80 bytes, 88 bytes in the second approach.

The second approach may be a little faster because no intermediate function calls are involved. But it's worth sacrificing an infrequently used feature's performance to save memory.

[Xuanwo]

Mostly LGTM!

This RFC is really great.

[Xuanwo]

Please also update https://github.com/datafuselabs/opendal/blob/main/docs/SUMMARY.md

[Xuanwo]

Great! Congrats on your first RFC!