Implemented data import from the old DB to the cluster on upgrade #1571
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dannyzaken
force-pushed
the
danny-cnpg-upgrade
branch
from
986fbcb to
5a92a76
Compare
alphaprinz
approved these changes
Mar 25, 2025
pkg/system/db_reconciler.go
Outdated
| r.Logger.Errorf("cnpg:: "+format, args...) | ||
| } | ||
|
|
||
| // wasClusterSpecChanged checks if any of the |
| @@ -514,6 +515,11 @@ func (r *Reconciler) SetDesiredCoreApp() error { | |||
| podSpec.ServiceAccountName = "noobaa-core" | |||
| coreImageChanged := false | |||
|
|
|||
| oneReplica := int32(1) | |||
| if r.CoreApp.Spec.Replicas != nil && *r.CoreApp.Spec.Replicas == 0 { | |||
There was a problem hiding this comment.
I would add a comment here explaining why replicas=0
dannyzaken
force-pushed
the
danny-cnpg-upgrade
branch
from
5a92a76 to
304d4b9
Compare
- When upgrading from a deployment with an existing DB, provide the DB
details in the cluster CR, to perform data import.
- Upgrade flow is as follows:
- If DB sts exist:
- Stop core\endpoints pods
- Create a cnpg cluster with import details in the CR
- The cnpg operator will start the first instance and deploy an
import job to import the data. After the import is completed, the
second instance will started.
- Wait for cluster to complete import and become Ready
- Once ready scale down the noobaa-db-pg sts to 0 replicas.
- The reconciliation will wait for the cluster to be ready, and a
temp error is returned until it is.
- The old sts is not deleted, to have it around for backup. In future
versions we should cleanup all the leftovers.
Signed-off-by: Danny Zaken <[email protected]>
dannyzaken
force-pushed
the
danny-cnpg-upgrade
branch
from
304d4b9 to
25c557a
Compare
jackyalbo
reviewed
Mar 25, 2025
| @@ -103,6 +103,31 @@ func RunInstall(cmd *cobra.Command, args []string) { | |||
| util.KubeCreateSkipExisting(cnpgRes.WebhooksService) | |||
| } | |||
|
|
|||
| // RunInstall runs the CloudNativePG operator installation | |||
There was a problem hiding this comment.
Suggested change
| // RunInstall runs the CloudNativePG operator installation | |
| // RunUpgrade runs the CloudNativePG operator upgrade |
dannyzaken
commented
Mar 25, 2025
| r.CNPGCluster.Spec.Bootstrap = &cnpgv1.BootstrapConfiguration{ | ||
| InitDB: &cnpgv1.BootstrapInitDB{ | ||
| Database: "nbcore", | ||
| Owner: "noobaa", |
There was a problem hiding this comment.
Add other default properties explicitly (like collation, etc.)
dannyzaken
commented
Mar 25, 2025
| func (r *Reconciler) reconcileClusterImport() error { | ||
|
|
||
| // The bootstrap configuration should only be set for the CR creation. | ||
|
|
There was a problem hiding this comment.
regardless of this code only running in cluster creation flow. guard it to look at the noobaaDbStatus
dannyzaken
commented
Mar 25, 2025
| // microservice type - import only the nbcore database | ||
| Type: cnpgv1.MicroserviceSnapshotType, | ||
| Databases: []string{ | ||
| "nbcore", |
There was a problem hiding this comment.
remove "magic strings" and use consts\options instead
|
I added comments based on an online review with @jackyalbo and @liranmauda. Will handle them in a different PR |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Explain the changes
noobaa upgradecommand in noobaa cli, to install the necessary cnpg resources.Gaps
Testing Instructions:
noobaa upgradewith a latest build of the CLI. this should install missing cnpg resources and update the noobaa CR to perform import