[Bug]: Jaeger v2.6.0 fails with OpenSearch v3.0 due to missing BodyJson field in span template

[7mza]

What happened?

jaeger 2.6.0 fails to start with opensearch 3

"jaeger_storage_exporter" exporter: cannot create trace writer: failed to create template "jaeger-main-jaeger-span": missing required fields: [BodyJson]
tracer-1 exited with code 1

everything is working with opensearch <= 2.19.2

Steps to reproduce

1. deploy jaeger v2.6.0 with opensearch v3.0
2. use defaut conf
3. start env

Expected behavior

tracer started, indexes jaeger-main & jaeger-archive created in opensearch

Relevant log output

tracer-1  | 2025/05/14 04:37:41 application version: git-commit=02128431cc2566b5cc3f38b8aea88a30f182ce3d, git-version=v2.6.0, build-date=2025-05-09T19:30:14Z
tracer-1  | 2025-05-14T04:37:41.676Z    info    service@v0.125.0/service.go:199 Setting up own telemetry...
tracer-1  | 2025-05-14T04:37:41.676Z    info    builders/builders.go:26 Development component. May change in the future.        {"otelcol.component.id": "jaeger_storage_exporter", "otelcol.component.kind": "exporter", "otelcol.signal": "traces"}
tracer-1  | 2025-05-14T04:37:41.677Z    debug   builders/builders.go:24 Beta component. May change in the future.       {"otelcol.component.id": "batch", "otelcol.component.kind": "processor", "otelcol.pipeline.id": "traces", "otelcol.signal": "traces"}
tracer-1  | 2025-05-14T04:37:41.677Z    debug   builders/builders.go:24 Stable component.       {"otelcol.component.id": "otlp", "otelcol.component.kind": "receiver", "otelcol.signal": "traces"}
tracer-1  | 2025-05-14T04:37:41.677Z    debug   otlpreceiver@v0.125.0/otlp.go:58        created signal-agnostic logger  {"otelcol.component.id": "otlp", "otelcol.component.kind": "receiver"}
tracer-1  | 2025-05-14T04:37:41.677Z    debug   builders/extension.go:48        Beta component. May change in the future.       {"otelcol.component.id": "jaeger_storage", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.677Z    debug   builders/extension.go:48        Beta component. May change in the future.       {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.677Z    info    builders/extension.go:50        Development component. May change in the future.        {"otelcol.component.id": "healthcheckv2", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.679Z    info    service@v0.125.0/service.go:266 Starting jaeger...      {"Version": "v2.6.0", "NumCPU": 12}
tracer-1  | 2025-05-14T04:37:41.679Z    info    extensions/extensions.go:41     Starting extensions...
tracer-1  | 2025-05-14T04:37:41.679Z    info    extensions/extensions.go:45     Extension is starting...        {"otelcol.component.id": "jaeger_storage", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.679Z    info    jaegerstorage/extension.go:159  Initializing storage 'another_storage'  {"otelcol.component.id": "jaeger_storage", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.699Z    info    config/config.go:302    Elasticsearch detected  {"otelcol.component.id": "jaeger_storage", "otelcol.component.kind": "extension", "version": 3}
tracer-1  | 2025-05-14T04:37:41.699Z    info    jaegerstorage/extension.go:159  Initializing storage 'some_storage'     {"otelcol.component.id": "jaeger_storage", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.717Z    info    config/config.go:302    Elasticsearch detected  {"otelcol.component.id": "jaeger_storage", "otelcol.component.kind": "extension", "version": 3}
tracer-1  | 2025-05-14T04:37:41.717Z    info    extensions/extensions.go:62     Extension started.      {"otelcol.component.id": "jaeger_storage", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.717Z    info    extensions/extensions.go:45     Extension is starting...        {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.718Z    info    grpc@v1.72.0/clientconn.go:176  [core] original dial target is: "localhost:4317"        {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.718Z    info    grpc@v1.72.0/clientconn.go:459  [core] [Channel #1]Channel created      {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.718Z    info    grpc@v1.72.0/clientconn.go:207  [core] [Channel #1]parsed dial target is: resolver.Target{URL:url.URL{Scheme:"dns", Opaque:"", User:(*url.Userinfo)(nil), Host:"", Path:"/localhost:4317", RawPath:"", OmitHost:false, ForceQuery:false, RawQuery:"", Fragment:"", RawFragment:""}}   {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.718Z    info    grpc@v1.72.0/clientconn.go:208  [core] [Channel #1]Channel authority set to "localhost:4317"    {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.718Z    error   jaegerquery/server.go:178       Cannot init traces archive storage writer       {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension", "error": "failed to create template \"jaeger-archive-jaeger-span\": missing required fields: [BodyJson]"}
tracer-1  | github.com/jaegertracing/jaeger/cmd/jaeger/internal/extension/jaegerquery.(*server).initArchiveStorage
tracer-1  |     github.com/jaegertracing/jaeger/cmd/jaeger/internal/extension/jaegerquery/server.go:178
tracer-1  | github.com/jaegertracing/jaeger/cmd/jaeger/internal/extension/jaegerquery.(*server).addArchiveStorage
tracer-1  |     github.com/jaegertracing/jaeger/cmd/jaeger/internal/extension/jaegerquery/server.go:153
tracer-1  | github.com/jaegertracing/jaeger/cmd/jaeger/internal/extension/jaegerquery.(*server).Start
tracer-1  |     github.com/jaegertracing/jaeger/cmd/jaeger/internal/extension/jaegerquery/server.go:103
tracer-1  | go.opentelemetry.io/collector/service/extensions.(*Extensions).Start
tracer-1  |     go.opentelemetry.io/collector/service@v0.125.0/extensions/extensions.go:52
tracer-1  | go.opentelemetry.io/collector/service.(*Service).Start
tracer-1  |     go.opentelemetry.io/collector/service@v0.125.0/service.go:271
tracer-1  | go.opentelemetry.io/collector/otelcol.(*Collector).setupConfigurationComponents
tracer-1  |     go.opentelemetry.io/collector/otelcol@v0.125.0/collector.go:242
tracer-1  | go.opentelemetry.io/collector/otelcol.(*Collector).Run
tracer-1  |     go.opentelemetry.io/collector/otelcol@v0.125.0/collector.go:312
tracer-1  | go.opentelemetry.io/collector/otelcol.NewCommand.func1
tracer-1  |     go.opentelemetry.io/collector/otelcol@v0.125.0/command.go:39
tracer-1  | github.com/jaegertracing/jaeger/cmd/jaeger/internal.checkConfigAndRun
tracer-1  |     github.com/jaegertracing/jaeger/cmd/jaeger/internal/command.go:84
tracer-1  | github.com/jaegertracing/jaeger/cmd/jaeger/internal.Command.func1
tracer-1  |     github.com/jaegertracing/jaeger/cmd/jaeger/internal/command.go:59
tracer-1  | github.com/spf13/cobra.(*Command).execute
tracer-1  |     github.com/spf13/cobra@v1.9.1/command.go:1015
tracer-1  | github.com/spf13/cobra.(*Command).ExecuteC
tracer-1  |     github.com/spf13/cobra@v1.9.1/command.go:1148
tracer-1  | github.com/spf13/cobra.(*Command).Execute
tracer-1  |     github.com/spf13/cobra@v1.9.1/command.go:1071
tracer-1  | main.main
tracer-1  |     github.com/jaegertracing/jaeger/cmd/jaeger/main.go:29
tracer-1  | runtime.main
tracer-1  |     runtime/proc.go:283
tracer-1  | 2025-05-14T04:37:41.718Z    info    jaegerquery/server.go:186       Metric storage not configured   {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.718Z    info    grpc@v1.72.0/server.go:690      [core] [Server #2]Server created        {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.719Z    info    app/static_handler.go:92        Using UI configuration  {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension", "path": "/etc/jaeger/jaeger-ui.json"}
tracer-1  | 2025-05-14T04:37:41.719Z    info    app/server.go:256       Query server started    {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension", "http_addr": "[::]:16686", "grpc_addr": "[::]:16685"}
tracer-1  | 2025-05-14T04:37:41.719Z    info    extensions/extensions.go:62     Extension started.      {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.719Z    info    app/server.go:284       Starting HTTP server    {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension", "port": 16686, "addr": ":16686"}
tracer-1  | 2025-05-14T04:37:41.719Z    info    extensions/extensions.go:45     Extension is starting...        {"otelcol.component.id": "healthcheckv2", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.719Z    info    app/server.go:298       Starting GRPC server    {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension", "port": 16685, "addr": ":16685"}
tracer-1  | 2025-05-14T04:37:41.719Z    info    grpc@v1.72.0/server.go:886      [core] [Server #2 ListenSocket #3]ListenSocket created  {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.719Z    debug   healthcheckv2extension@v0.125.0/extension.go:98 Starting health check extension V2      {"otelcol.component.id": "healthcheckv2", "otelcol.component.kind": "extension", "config": {"Endpoint":"localhost:13133","TLSSetting":null,"CORS":null,"Auth":null,"MaxRequestBodySize":0,"IncludeMetadata":false,"ResponseHeaders":null,"CompressionAlgorithms":null,"ReadTimeout":0,"ReadHeaderTimeout":0,"WriteTimeout":0,"IdleTimeout":0,"Middlewares":null,"Path":"/","ResponseBody":null,"CheckCollectorPipeline":null,"UseV2":true,"GRPCConfig":null,"HTTPConfig":{"Endpoint":"localhost:13133","TLSSetting":null,"CORS":null,"Auth":null,"MaxRequestBodySize":0,"IncludeMetadata":false,"ResponseHeaders":null,"CompressionAlgorithms":null,"ReadTimeout":0,"ReadHeaderTimeout":0,"WriteTimeout":0,"IdleTimeout":0,"Middlewares":null,"Config":{"Enabled":false,"Path":"/config"},"Status":{"Enabled":true,"Path":"/status"}},"ComponentHealthConfig":null}}
tracer-1  | 2025-05-14T04:37:41.720Z    info    extensions/extensions.go:62     Extension started.      {"otelcol.component.id": "healthcheckv2", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.727Z    error   graph/graph.go:438      Failed to start component       {"error": "cannot create trace writer: failed to create template \"jaeger-main-jaeger-span\": missing required fields: [BodyJson]", "type": "Exporter", "id": "jaeger_storage_exporter"}
tracer-1  | 2025-05-14T04:37:41.727Z    info    service@v0.125.0/service.go:331 Starting shutdown...
tracer-1  | 2025-05-14T04:37:41.727Z    info    extensions/extensions.go:69     Stopping extensions...
tracer-1  | 2025-05-14T04:37:41.727Z    info    healthcheckv2extension@v0.125.0/extension.go:138        discarding event received after shutdown        {"otelcol.component.id": "healthcheckv2", "otelcol.component.kind": "extension", "source": {}, "event": {}}
tracer-1  | 2025-05-14T04:37:41.727Z    info    healthcheckv2extension@v0.125.0/extension.go:138        discarding event received after shutdown        {"otelcol.component.id": "healthcheckv2", "otelcol.component.kind": "extension", "source": {}, "event": {}}
tracer-1  | 2025-05-14T04:37:41.727Z    info    app/server.go:323       Closing HTTP server     {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.727Z    info    app/server.go:291       HTTP server stopped     {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension", "port": 16686, "addr": ":16686"}
tracer-1  | 2025-05-14T04:37:41.730Z    info    app/server.go:328       Stopping gRPC server    {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.730Z    info    grpc@v1.72.0/server.go:822      [core] [Server #2 ListenSocket #3]ListenSocket deleted  {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.730Z    info    app/server.go:306       GRPC server stopped     {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension", "port": 16685, "addr": ":16685"}
tracer-1  | 2025-05-14T04:37:41.730Z    info    app/server.go:333       Server stopped  {"otelcol.component.id": "jaeger_query", "otelcol.component.kind": "extension"}
tracer-1  | 2025-05-14T04:37:41.730Z    info    grpc@v1.72.0/clientconn.go:563  [core] [Channel #1]Channel Connectivity change to SHUTDOWN      {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.730Z    info    grpc@v1.72.0/resolver_wrapper.go:111    [core] [Channel #1]Closing the name resolver    {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.730Z    info    grpc@v1.72.0/balancer_wrapper.go:160    [core] [Channel #1]ccBalancerWrapper: closing   {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.731Z    info    grpc@v1.72.0/clientconn.go:1174 [core] [Channel #1]Channel deleted      {"grpc_log": true}
tracer-1  | 2025-05-14T04:37:41.731Z    info    healthcheckv2extension@v0.125.0/extension.go:138        discarding event received after shutdown        {"otelcol.component.id": "healthcheckv2", "otelcol.component.kind": "extension", "source": {}, "event": {}}
tracer-1  | 2025-05-14T04:37:41.731Z    info    healthcheckv2extension@v0.125.0/extension.go:138        discarding event received after shutdown        {"otelcol.component.id": "healthcheckv2", "otelcol.component.kind": "extension", "source": {}, "event": {}}
tracer-1  | 2025-05-14T04:37:41.731Z    info    healthcheckv2extension@v0.125.0/extension.go:138        discarding event received after shutdown        {"otelcol.component.id": "healthcheckv2", "otelcol.component.kind": "extension", "source": {}, "event": {}}
tracer-1  | 2025-05-14T04:37:41.731Z    info    service@v0.125.0/service.go:345 Shutdown complete.
tracer-1  | Error: cannot start pipelines: failed to start "jaeger_storage_exporter" exporter: cannot create trace writer: failed to create template "jaeger-main-jaeger-span": missing required fields: [BodyJson]
tracer-1  | 2025/05/14 04:37:41 cannot start pipelines: failed to start "jaeger_storage_exporter" exporter: cannot create trace writer: failed to create template "jaeger-main-jaeger-span": missing required fields: [BodyJson]
tracer-1 exited with code 1

Screenshot

No response

Additional context

No response

Jaeger backend version

No response

SDK

No response

Pipeline

No response

Stogage backend

opensearch 3.0

Operating system

No response

Deployment model

docker

Deployment configs

[yurishkuro]

v3 is not supported currently - #7133