Skip to content

Supporting more properties for rabbitmq #3806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 28 commits into from
Aug 20, 2025
Merged

Supporting more properties for rabbitmq #3806

merged 28 commits into from
Aug 20, 2025

Conversation

abossard
Copy link
Contributor

@abossard abossard commented Apr 29, 2025 •
edited
Loading

Description

We want to be able to better integrate Dapr with existing RabbitMQ setups. For that we added a feature to have certain RabbitMQ message properties to go into the Dapr Metadata and also do be able to set these RabbitMQ message properties from Dapr.

For Bindings, it will just work, since Bindings are anyway RabbitMQ specific. For the PubSub, if you set supported metadata, it will be put into the RabbitMQ message properties, but you need to enable a special config, so get the RabbitMQ message properties as Metadata back. This is to prevent any accidental adding of metadata, that might conflict with metadata that you're anyway using.

Supported and tested properties:
image
(RabbitMQ Message Properties)

  • type
  • message_id
  • correlation_id
  • content_type

New pubusb config value:

  • publishMessagePropertiesToMetadata, to specify if properties should be published to metadata for a pub sub retriever.

Bindings Example:
tbd

Pubsub Example:
tbd

Issue reference

Please reference the issue this PR will close: #3650

Duplicate of

#3661
I messed up the DCO and didn't figure out how to fix it.

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

@abossard abossard requested review from a team as code owners April 29, 2025 10:06
@abossard abossard force-pushed the supporting_more_properties_for_rabbitmq_on_main branch from 5642742 to 76ad9bb Compare April 29, 2025 10:06
@cicoyle cicoyle added the autoupdate automatically keeps PR up to date against master label Jun 20, 2025
@cicoyle
Copy link
Contributor

cicoyle commented Jun 20, 2025

Hey @abossard thanks for the PR here 🎉 and your patience 🙏🏻

I just approved CI to run and it looks a bit red. Mind taking a look to fix up the related issues? (& make sure to pull latest bc I merged master into your branch) :)

Thanks!

@abossard
Copy link
Contributor Author

abossard commented Jul 4, 2025 •
edited
Loading

Thanks @cicoyle I'll finally have time next week to finish it all up. Does it require another approval to run the pipeline?

@abossard
Copy link
Contributor Author

Hi @cicoyle

Can you point me to the CI run where it has the error?

@yaron2
Copy link
Member

yaron2 commented Jul 22, 2025

Conflicts

Andre Bossard and others added 12 commits July 22, 2025 21:04
Add support for messageId, correlationId, and type in RabbitMQ bindings
5eda4a5 
Signed-off-by: Andre Bossard <[email protected]>
Squashed commit of the following:
4ddf575 
commit 3549843
Author: Josh van Leeuwen <[email protected]>
Date:   Mon Mar 24 17:50:50 2025 -0300

    go.mod: CVE updates (dapr#3713)

    Signed-off-by: joshvanl <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit 9a11502
Author: Cassie Coyle <[email protected]>
Date:   Wed Apr 23 07:33:24 2025 -0500

    Add baggage header support to http binding (dapr#3723)

    Signed-off-by: Cassandra Coyle <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit 8014a1d
Author: Josh van Leeuwen <[email protected]>
Date:   Wed Mar 26 19:55:55 2025 -0300

    CVE go mod dep update (dapr#3716)

    Signed-off-by: joshvanl <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit 72ffcd1
Author: Josh van Leeuwen <[email protected]>
Date:   Mon Mar 24 17:50:50 2025 -0300

    go.mod: CVE updates (dapr#3713)

    Signed-off-by: joshvanl <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit 12be96f
Author: Yaron Schneider <[email protected]>
Date:   Wed Jan 29 13:41:28 2025 -0800

    Add deepseek support, update Go to 1.23.5 (dapr#3659)

    Signed-off-by: yaron2 <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit a00c8f3
Merge: a395f74 397766a
Author: Andre Bossard <[email protected]>
Date:   Tue Apr 29 11:41:27 2025 +0200

    Merge branch 'dapr:main' into supporting_more_properties_for_rabbitmq_on_main

commit a395f74
Author: Andre Bossard <[email protected]>
Date:   Mon Apr 28 10:33:59 2025 +0200

    Refactor tryGetProperty to use strings.EqualFold for case-insensitive key matching

    Signed-off-by: Andre Bossard <[email protected]>

commit 1d4f3a1
Author: Anton Troshin <[email protected]>
Date:   Thu Apr 24 15:19:04 2025 -0500

    Support Oracle Connect Descriptors (dapr#3800)

    Signed-off-by: Anton Troshin <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit 4630f5c
Author: Anton Troshin <[email protected]>
Date:   Wed Apr 23 07:34:59 2025 -0500

    Solace pubsub conformance test fix (dapr#3802)

    Signed-off-by: Anton Troshin <[email protected]>
    Co-authored-by: Yaron Schneider <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit ed3f7a8
Author: Cassie Coyle <[email protected]>
Date:   Wed Apr 23 07:33:24 2025 -0500

    Add baggage header support to http binding (dapr#3723)

    Signed-off-by: Cassandra Coyle <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit af31fae
Author: MikelRev <[email protected]>
Date:   Thu Apr 17 11:54:52 2025 -0400

    Updated sqlserver auth to utilize default scope. (dapr#3698)

    Co-authored-by: Yaron Schneider <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit c5a0407
Author: Andre Bossard <[email protected]>
Date:   Wed Apr 16 13:39:26 2025 +0200

    Fix metadata handling by removing unnecessary value escaping in RabbitMQ message processing

    Signed-off-by: Andre Bossard <[email protected]>

commit 5bcb186
Author: Andre Bossard <[email protected]>
Date:   Wed Apr 16 13:13:24 2025 +0200

    Refactor RabbitMQ metadata handling to use case-insensitive key retrieval

    Signed-off-by: Andre Bossard <[email protected]>

commit 4f17a7f
Author: Emmanuel Auffray <[email protected]>
Date:   Thu Apr 10 10:06:37 2025 +1200

    Adding GoogleAI models too (dapr#3689)

    Signed-off-by: Emmanuel Auffray <[email protected]>
    Co-authored-by: Josh van Leeuwen <[email protected]>
    Co-authored-by: Yaron Schneider <[email protected]>
    Co-authored-by: Cassie Coyle <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit 652c733
Author: Josh van Leeuwen <[email protected]>
Date:   Wed Mar 26 19:55:55 2025 -0300

    CVE go mod dep update (dapr#3716)

    Signed-off-by: joshvanl <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit 71e9a4b
Author: Josh van Leeuwen <[email protected]>
Date:   Mon Mar 24 17:50:50 2025 -0300

    go.mod: CVE updates (dapr#3713)

    Signed-off-by: joshvanl <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit 621f7ef
Author: Emmanuel Auffray <[email protected]>
Date:   Mon Mar 24 05:38:17 2025 +1300

    Adding Ollama as a conversation component for local dev/running of LLMs (dapr#3688)

    Signed-off-by: Emmanuel Auffray <[email protected]>
    Co-authored-by: Mike Nguyen <[email protected]>
    Co-authored-by: Yaron Schneider <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit 229524f
Author: Emmanuel Auffray <[email protected]>
Date:   Sat Mar 22 00:52:15 2025 +1300

    Fix reference links of conversation components (dapr#3690)

    Signed-off-by: Emmanuel Auffray <[email protected]>
    Signed-off-by: Andre Bossard <[email protected]>

commit 9dbe6e3
Author: Andre Bossard <[email protected]>
Date:   Wed Apr 16 12:19:53 2025 +0200

    Add metadata handling for RabbitMQ messages and corresponding test

    Signed-off-by: Andre Bossard <[email protected]>

commit 3f0ec0b
Author: Andre Bossard <[email protected]>
Date:   Thu Mar 20 03:25:29 2025 +0100

    Add RabbitMQ metadata properties test and configuration

    Signed-off-by: Andre Bossard <[email protected]>

commit b6acda8
Author: Andre Bossard <[email protected]>
Date:   Tue Mar 18 12:25:31 2025 +0100

    Add integration test for publishing metadata properties to RabbitMQ

    Signed-off-by: Andre Bossard <[email protected]>

commit f726110
Author: Andre Bossard <[email protected]>
Date:   Tue Mar 18 09:26:49 2025 +0100

    Refactor RabbitMQ publishing: consolidate metadata application into a single function

    Signed-off-by: Andre Bossard <[email protected]>

commit 4cb5600
Author: Andre Bossard <[email protected]>
Date:   Tue Mar 18 09:16:14 2025 +0100

    Refactor RabbitMQ metadata handling: consolidate metadata retrieval into common package

    Signed-off-by: Andre Bossard <[email protected]>

commit 069b12f
Author: Andre Bossard <[email protected]>
Date:   Tue Mar 18 08:12:25 2025 +0100

    Refactor metadata function names for consistency: update messageId and correlationId to messageID and correlationID

commit d508479
Merge: af3fca0 d8ac01b
Author: Andre Bossard <[email protected]>
Date:   Tue Mar 18 07:58:36 2025 +0100

    Merge branch 'dapr:main' into supporting_more_properties_for_rabbitmq_on_main

commit af3fca0
Merge: 57ae111 637d18d
Author: Andre Bossard <[email protected]>
Date:   Wed Mar 12 03:28:26 2025 -0700

    Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main

commit 57ae111
Merge: d0c75f6 2997e47
Author: Andre Bossard <[email protected]>
Date:   Tue Mar 4 08:49:52 2025 +0100

    Merge branch 'dapr:main' into supporting_more_properties_for_rabbitmq_on_main

commit d0c75f6
Author: Andre Bossard <[email protected]>
Date:   Wed Feb 5 11:43:03 2025 +0100

    Add support for messageId, correlationId, and type in RabbitMQ bindings

    Signed-off-by: Andre Bossard <[email protected]>

Signed-off-by: Andre Bossard <[email protected]>
refactor(rabbitmq): improve metadata validation in tests and enhance …
9434604 
…error handling

Signed-off-by: Andre Bossard <[email protected]>
fix(rabbitmq): standardize metadata keys to lowercase and enhance deb…
0098f30 
…ugging in tests

Signed-off-by: Andre Bossard <[email protected]>
fix(rabbitmq): update metadata keys to include 'metadata.' prefix for…
fc95737 
… AMQP properties

Signed-off-by: Andre Bossard <[email protected]>
fix(rabbitmq): prefix custom headers in metadata with 'metadata.' for…
acbac0b 
… consistency

Signed-off-by: Andre Bossard <[email protected]>
feat(rabbitmq): add PublishMessagePropertiesToMetadata flag to contro…
98a936f 
…l metadata publishing

Signed-off-by: Andre Bossard <[email protected]>
fix(rabbitmq): add missing metadata validation logging in TestRabbitM…
d07605b 
…QMetadataProperties

Signed-off-by: Andre Bossard <[email protected]>
fix(rabbitmq): remove outdated comment regarding gRPC protocol in Tes…
31f6399 
…tRabbitMQMetadataProperties

Signed-off-by: Andre Bossard <[email protected]>
feat(rabbitmq): implement TryGetProperty function with case-insensiti…
a81a73e 
…ve matching and add unit tests

Signed-off-by: Andre Bossard <[email protected]>
@famarting
fix default value in azure eventhubs binding metadata.yaml (dapr#3883)
5b9aaf5 
Signed-off-by: Fabian Martinez <[email protected]>
@famarting
add conversation components to component metadata schema and bundle (d…
e2e682d 
…apr#3882)

Signed-off-by: Fabian Martinez <[email protected]>
@abossard abossard closed this Jul 22, 2025
@abossard abossard force-pushed the supporting_more_properties_for_rabbitmq_on_main branch from 08bf2be to 0ab52f7 Compare July 22, 2025 19:12
@abossard
Copy link
Contributor Author

abossard commented Jul 22, 2025 •
edited
Loading

Yeah, sorry. It's time consuming for me to resolve conflicts. If someone is able to point me to what else I need to complete to push the PR further, I'd be up for it.

Would anyone have time in the upcoming week to e.g. show me which pipeline or action is blocking my PR? If not, I consider this change closed.

@claude
Resolve merge conflicts with upstream/main
2cfd793 
- Updated Go dependencies to latest versions from upstream
- Resolved conflicts in go.mod, go.sum, and certification test dependencies
- Accepted upstream versions for all conflicted packages including:
  - golang.org/x/crypto v0.39.0
  - golang.org/x/net v0.41.0
  - google.golang.org/grpc v1.72.0
  - google.golang.org/protobuf v1.36.6

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Signed-off-by: Andre Bossard <[email protected]>
@abossard abossard reopened this Jul 22, 2025
@abossard
Copy link
Contributor Author

abossard commented Aug 6, 2025

Hi @yaron2 can you help me make the workflow approvals so I can check if it all runs?

dapr-bot and others added 11 commits August 12, 2025 09:42
@dapr-bot
Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main
3822add
@dapr-bot
Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main
9d6617a
updated the metadata.yaml
0dff84f 
Signed-off-by: Andre Bossard <[email protected]>
@dapr-bot
Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main
cd8ca9c
@dapr-bot
Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main
d9b8493
@dapr-bot
Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main
6ce6a44
@dapr-bot
Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main
aabe284
@dapr-bot
Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main
b106e51
@dapr-bot
Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main
e0d4842
@dapr-bot
Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main
888e524
@yaron2
Merge branch 'main' into supporting_more_properties_for_rabbitmq_on_main
e3664ee
@yaron2 yaron2 merged commit 6a45e8e into dapr:main Aug 20, 2025
26 of 88 checks passed
@yaron2 yaron2 added this to the v1.16 milestone Aug 20, 2025
@sicoyle sicoyle mentioned this pull request Aug 20, 2025
Open
@sicoyle
Copy link
Contributor

sicoyle commented Aug 20, 2025

Hey @abossard 👋 Thanks so much for this contribution. Would you help update the documentation with your latest additions please? dapr/docs#4810

jjcollinge pushed a commit to jjcollinge/components-contrib that referenced this pull request Aug 21, 2025
@abossard @famarting
@claude @dapr-bot @yaron2 @jjcollinge
Supporting more properties for rabbitmq (dapr#3806)
67a9c09 
Signed-off-by: Andre Bossard <[email protected]>
Signed-off-by: Fabian Martinez <[email protected]>
Co-authored-by: Andre Bossard <[email protected]>
Co-authored-by: Fabian Martinez <[email protected]>
Co-authored-by: Claude <[email protected]>
Co-authored-by: Dapr Bot <[email protected]>
Co-authored-by: Yaron Schneider <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@abossard abossard

Labels
autoupdate automatically keeps PR up to date against master
Projects
None yet
Milestone
v1.16
Development

Successfully merging this pull request may close these issues.

RabbitMQ Messages should have some cloud event property fields populated, e.g. message id and correlation id
6 participants
@abossard @cicoyle @yaron2 @sicoyle @famarting @dapr-bot