Skip to content

fix Kamaji OOM#340

Merged
Marian Koreniuk (themoriarti) merged 1 commit into
mainfrom
fix-kamaji-oom
Sep 10, 2024
Merged

fix Kamaji OOM#340
Marian Koreniuk (themoriarti) merged 1 commit into
mainfrom
fix-kamaji-oom

Conversation

@kvaps

@kvaps Andrei Kvapil (kvaps) commented Sep 10, 2024

Copy link
Copy Markdown
Member

Sometimes Kamaji can be killed due to defult limits let's expand them a little

Summary by CodeRabbit

  • New Features
    • Introduced resource management configurations for the kamaji service, enhancing control over CPU and memory allocation.
    • Added specifications for resource limits and requests to improve stability and performance in a Kubernetes environment.

Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
@coderabbitai

coderabbitai Bot commented Sep 10, 2024

Copy link
Copy Markdown
Contributor

Walkthrough

The changes introduce resource management configurations for the kamaji service in the values.yaml file. A new resources section has been added, specifying limits and requests for CPU and memory. The service is now configured with a maximum of 200 milliCPU and 500 MiB of memory, while requesting a minimum of 100 milliCPU and 100 MiB of memory.

Changes

Files Change Summary
packages/system/kamaji/values.yaml Added resources configuration with CPU and memory limits and requests.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Kubernetes
    participant Kamaji

    User->>Kubernetes: Deploy Kamaji
    Kubernetes->>Kamaji: Allocate Resources
    Kamaji->>Kubernetes: Resource Limits Set
    Kamaji->>Kubernetes: Resource Requests Made
    Kubernetes->>User: Deployment Successful
Loading

Poem

🐰 In the garden where resources grow,
Kamaji hops with a steady flow.
With limits set, it leaps with grace,
In a world of clouds, it finds its place.
A dance of CPU and memory bright,
Stability shines, a joyful sight! 🌼

Tip

Announcements
  • The review status is no longer posted as a separate comment when there are no actionable or nitpick comments. In such cases, the review status is included in the walkthrough comment.
  • We have updated our review workflow to use the Anthropic's Claude family of models. Please share any feedback in the discussion post on our Discord.
  • Possibly related PRs: Walkthrough comment now includes a list of potentially related PRs to help you recall past context. Please share any feedback in the discussion post on our Discord.
  • Suggested labels: CodeRabbit can now suggest labels by learning from your past PRs in the walkthrough comment. You can also provide custom labeling instructions in the UI or configuration file.
  • Possibly related PRs, automatic label suggestions based on past PRs, learnings, and possibly related issues require data opt-in (enabled by default).

Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 1128d0c and 9808368.

Files selected for processing (1)
  • packages/system/kamaji/values.yaml (1 hunks)
Additional comments not posted (1)
packages/system/kamaji/values.yaml (1)

9-15: LGTM, but provide more context about the service's resource usage.

The added resources section looks good. It's correctly formatted and indented, and the limits and requests values are valid.

However, it's hard to determine if the values are appropriate without more context about the service's resource usage. Can you provide more information about the service's CPU and memory usage under normal and peak loads? This will help determine if the limits and requests values are sufficient to prevent the out-of-memory issues mentioned in the PR summary.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try this, but I think this limit will need to be increased to 1GB.

@themoriarti
Marian Koreniuk (themoriarti) deleted the fix-kamaji-oom branch September 10, 2024 20:44
Andrey Kolkov (androndo) added a commit that referenced this pull request Jul 22, 2026
…ends) (#3403)

## What this PR does

Bumps the etcd-operator to **v0.5.3**, which carries the S3
request-checksum fix
([cozystack/etcd-operator#342](cozystack/etcd-operator#342)).
`git compare v0.5.2...v0.5.3` is exactly #340, #341, #342.

**Why:** scheduled etcd snapshots to non-AWS S3-compatible backends
(**Ceph RGW** confirmed) fail at upload. Since early 2025
`aws-sdk-go-v2` defaults `RequestChecksumCalculation` to
`WhenSupported`, which stamps a CRC32 on every upload; over HTTPS a
multipart part rides it as `x-amz-content-sha256:
STREAMING-UNSIGNED-PAYLOAD-TRAILER`, and RGW rejects it with `400
InvalidArgument`. v0.5.3 sets `WhenRequired` on both the S3 client and
the transfer manager, so multipart uploads (>5 MiB — every real etcd
snapshot) carry no checksum trailer.

**Changes:**
- `system/etcd-operator` — `appVersion` v0.5.2 → v0.5.3 (image tag
follows AppVersion; `values.yaml` keeps `tag: ""`), bump
`ETCD_OPERATOR_REF`, update deployment unittest expectations.
- `system/etcd-operator-crds` — re-vendor CRDs from v0.5.3.
`etcdmembers` gains the additive #340 `status.version` field +
**Running** printer column and picks up the `peerAutoTLS` spec field the
vendored copy was already missing (all additive, backward-compatible;
`etcdclusters`/`etcdsnapshots` unchanged). Fixes the stale `make update`
(upstream has no `config/crd` kustomization — CRDs live in
`charts/etcd-operator/crd-bases`) and makes it fail loudly on a fetch
error (temp file under `set -e`, not a pipe).

**Verification:** `helm unittest` 18/18 and `helm template` green on
both packages; regenerated CRDs are byte-identical to upstream
`crd-bases` at v0.5.3. End-to-end on a live Ceph RGW backend
(freedom-portal-stage): the v0.5.2 (client-only) agent fails a multipart
snapshot upload with the 400; v0.5.3 uploads an 11 MiB multipart
snapshot successfully, and the operator-driven CronJob→EtcdSnapshot path
reaches `Complete`.

### Downstream repositories

Walked the trigger map in `docs/agents/contributing.md` file-by-file
against the diff (`packages/system/etcd-operator{,-crds}/**` only — a
component version bump + additive CRD re-vendor). Nothing matches: not
an `apps/`/`extra/` package add/rename/remove, no `core/platform`
values, no variant/bundle, no platform component add/remove, no `hack/`
layout or shared-tooling change, and the provider does not type the
`etcd-operator.cozystack.io` CRDs (and the change is additive/opaque
regardless).

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:

### Release note

```release-note
fix(etcd-operator): bump to v0.5.3 — etcd snapshot uploads to non-AWS S3-compatible backends (Ceph RGW, some MinIO/R2) no longer fail with `400 InvalidArgument: x-amz-content-sha256 ...`; the snapshot agent now requests a checksum only when required, on both the S3 client and the multipart transfer manager. Also brings observed EtcdMember versions (#340) and `--watch-namespace` (#341).
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added runtime etcd version visibility to `EtcdMember` (including a
“Running” status column).
* Added `peerAutoTLS` support to `EtcdMember` for operator-managed peer
TLS behavior.

* **Updates**
  * Updated the etcd-operator to version 0.5.3.
* Refreshed bundled CRDs and ensured CRDs are preserved during Helm
lifecycle operations.

* **Tests**
* Updated deployment test expectations to use the 0.5.3 operator image.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
pull Bot pushed a commit to medampudi/cozystack that referenced this pull request Jul 22, 2026
…ends)

v0.5.3 carries the S3 request-checksum fix (cozystack/etcd-operator#342): the
snapshot agent sets RequestChecksumCalculation=WhenRequired on BOTH the S3 client
and the transfer manager, so multipart snapshot uploads (>5 MiB — every real etcd
snapshot) no longer carry the CRC32 STREAMING-UNSIGNED-PAYLOAD-TRAILER that Ceph
RGW rejects with 400 InvalidArgument. Also pulls in cozystack#340 (observed EtcdMember
versions) and cozystack#341 (--watch-namespace).

- system/etcd-operator: appVersion v0.5.2 -> v0.5.3 (image tag follows AppVersion),
  bump ETCD_OPERATOR_REF, update deployment unittest expectations.
- system/etcd-operator-crds: re-vendor CRDs from v0.5.3. etcdmembers gains the
  additive cozystack#340 status.version field + "Running" printer column and picks up the
  peerAutoTLS spec field the vendored copy was already missing (all additive,
  backward-compatible; etcdclusters/etcdsnapshots unchanged). Fix the stale
  `make update`: upstream has no config/crd kustomization — CRDs live in
  charts/etcd-operator/crd-bases — so vendor those directly and stamp
  helm.sh/resource-policy: keep.

Verified end-to-end on a Ceph RGW backend: the v0.5.2 (client-only) agent fails a
multipart snapshot upload with the 400 above; v0.5.3 uploads an 11 MiB multipart
snapshot successfully.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants