Skip to content

action: add benchmark on schedule #1254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 5, 2023

Conversation

Desiki-high
Copy link
Member

@Desiki-high Desiki-high commented Apr 28, 2023

Add the benchmark workflow

  1. We will cache each benchmark result and compare with the last result.
  2. See docs.github, gitHub will remove any cache entries that have not been accessed in over 7 days. We schedule benchmark on every Monday and Wednesday. You can also do this manually.

Add benchmark-schedule mode on benchmark summary

You can see the result in 1, 2, 3, 4.

The pull request trigger for dev was deleted in compare.
Next work: add more images in benchmark.

Ref:#1220

@Desiki-high Desiki-high requested a review from a team as a code owner April 28, 2023 03:24
@Desiki-high Desiki-high requested review from bergwolf, imeoer and changweige and removed request for a team April 28, 2023 03:24
@Desiki-high Desiki-high linked an issue Apr 28, 2023 that may be closed by this pull request
@hsiangkao
Copy link
Contributor

switch into daily or every two days?

@codecov
Copy link

codecov bot commented Apr 28, 2023

Codecov Report

Merging #1254 (83fbf80) into master (35aa3a2) will decrease coverage by 0.03%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1254      +/-   ##
==========================================
- Coverage   45.08%   45.06%   -0.03%     
==========================================
  Files         126      126              
  Lines       37395    37395              
  Branches    37395    37395              
==========================================
- Hits        16860    16851       -9     
- Misses      19648    19656       +8     
- Partials      887      888       +1     

see 3 files with indirect coverage changes

@Desiki-high
Copy link
Member Author

Desiki-high commented Apr 28, 2023

switch into daily or every two days?

Is that too frequent? We want to compare this benchmark with the last time benchmark.

@hsiangkao
Copy link
Contributor

switch into daily or every two days?

Is that too frequent? We want to compare this benchmark with the last time benchmark.

I'm afraid it's hard to find the real commit which causes regression with a per-week benchmark test.

@Desiki-high
Copy link
Member Author

switch into daily or every two days?

Is that too frequent? We want to compare this benchmark with the last time benchmark.

I'm afraid it's hard to find the real commit which causes regression with a per-week benchmark test.

You are right. I will switch to every two days.

@Desiki-high Desiki-high force-pushed the benchmark-weekly branch 14 times, most recently from 956b085 to d98cfc7 Compare April 30, 2023 17:57
@Desiki-high Desiki-high requested a review from a team as a code owner April 30, 2023 17:57
@Desiki-high Desiki-high force-pushed the benchmark-weekly branch 16 times, most recently from 4d42dcf to 34c2d45 Compare May 1, 2023 12:43
we will run benchmark twice per-week, and compare result with the last by cache.

Signed-off-by: Yadong Ding <[email protected]>
supprot the benchmark on schedule by new mode in benchmark_summary.

Signed-off-by: Yadong Ding <[email protected]>
@Desiki-high Desiki-high linked an issue May 3, 2023 that may be closed by this pull request
@Desiki-high Desiki-high changed the title [WIP]action: add weekly benchmark [WIP]action: add benchmark on schedule May 3, 2023
@Desiki-high Desiki-high changed the title [WIP]action: add benchmark on schedule action: add benchmark on schedule May 3, 2023
@anolis-bot
Copy link
Collaborator

@Desiki-high , the title has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/70728

@anolis-bot
Copy link
Collaborator

@Desiki-high , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

Copy link
Collaborator

@imeoer imeoer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@imeoer imeoer merged commit 7a48992 into dragonflyoss:master May 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Proposal: Add the performance test as a per-week cron task.
4 participants