dhruva_RP-13176_support_multiple_indices_in_AddDocumentRequest

[dhruva-r]

Jira Ticket

RP-13176

Summary of Changes

Added support for multiple indices to be linked to a document in the same AddDocumentRequest, in nrtsearch, for indexing. Changed client command AddDocuments to accept a list of indexNames as an input. This prevents the need of sending the same document to different indices with multiple commands or requests to the AddDocumentsHandler.

Tests

Manually testing to make sure a request with both indexName and indexNames is rejected.

dhruva@HVVK4XYGRK nrtsearch % ./build/install/nrtsearch/bin/nrtsearch_client addDocuments -f docs.csv -t csv -i testIdx
[INFO ] 2025-05-29 12:44:13.812 [main] NrtsearchClientBuilder$AddDocumentsClientBuilder - Read row 0
[INFO ] 2025-05-29 12:44:13.813 [main] NrtsearchClientBuilder$AddDocumentsClientBuilder - Read row 1
[INFO ] 2025-05-29 12:44:13.817 [main] NrtsearchClient - sent async addDocumentsRequest to server...
[ERROR] 2025-05-29 12:44:13.927 [grpc-default-executor-0] NrtsearchClient - INVALID_ARGUMENT: Must provide either indexName or indexNames.
io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Must provide either indexName or indexNames.
        at io.grpc.Status.asRuntimeException(Status.java:533) ~[grpc-api-1.66.0.jar:1.66.0]
        at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481) ~[grpc-stub-1.66.0.jar:1.66.0]
        at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489) ~[grpc-core-1.66.0.jar:1.66.0]
        at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453) ~[grpc-core-1.66.0.jar:1.66.0]
        at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486) ~[grpc-core-1.66.0.jar:1.66.0]
        at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:564) ~[grpc-core-1.66.0.jar:1.66.0]
        at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:72) ~[grpc-core-1.66.0.jar:1.66.0]
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:729) ~[grpc-core-1.66.0.jar:1.66.0]
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:710) ~[grpc-core-1.66.0.jar:1.66.0]
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.66.0.jar:1.66.0]
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) ~[grpc-core-1.66.0.jar:1.66.0]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
        at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
java.lang.RuntimeException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Must provide either indexName or indexNames.
        at com.yelp.nrtsearch.server.grpc.NrtsearchClient.addDocuments(NrtsearchClient.java:260)
        at com.yelp.nrtsearch.tools.cli.AddDocumentsCommand.call(AddDocumentsCommand.java:103)
        at com.yelp.nrtsearch.tools.cli.AddDocumentsCommand.call(AddDocumentsCommand.java:36)
        at picocli.CommandLine.executeUserObject(CommandLine.java:2045)
        at picocli.CommandLine.access$1500(CommandLine.java:148)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2465)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2457)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2419)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2277)
        at picocli.CommandLine$RunLast.execute(CommandLine.java:2421)
        at picocli.CommandLine.execute(CommandLine.java:2174)
        at com.yelp.nrtsearch.tools.cli.NrtsearchClientCommand.main(NrtsearchClientCommand.java:95)
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Must provide either indexName or indexNames.
        at io.grpc.Status.asRuntimeException(Status.java:533)
        at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481)
        at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
        at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
        at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
        at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:564)
        at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:72)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:729)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:710)
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)

Sanity test to ensure that a request with only the indexName parameter still gets processed. This is to ensure and prove backwards compatibility.

dhruva@HVVK4XYGRK nrtsearch % ./build/install/nrtsearch/bin/nrtsearch_client addDocuments -f docs.csv -t csv -i testIdx
[INFO ] 2025-05-29 13:00:02.844 [main] NrtsearchClientBuilder$AddDocumentsClientBuilder - Read row 0
[INFO ] 2025-05-29 13:00:02.845 [main] NrtsearchClientBuilder$AddDocumentsClientBuilder - Read row 1
[INFO ] 2025-05-29 13:00:02.850 [main] NrtsearchClient - sent async addDocumentsRequest to server...
[INFO ] 2025-05-29 13:00:02.978 [grpc-default-executor-0] NrtsearchClient - Received response for genId: genId: "2"
primaryId: "a7c8868b-1cf7-451f-b549-bf6f9c6c67ca"

[INFO ] 2025-05-29 13:00:02.978 [grpc-default-executor-0] NrtsearchClient - Received final response from server

Generic case of adding documents to multiple index names.

dhruva@HVVK4XYGRK nrtsearch % ./build/install/nrtsearch/bin/nrtsearch_client addDocuments -f docs.csv -t csv -i testIdx,testIdx2     
[INFO ] 2025-05-29 14:03:56.816 [main] NrtsearchClientBuilder$AddDocumentsClientBuilder - Read row 0
[INFO ] 2025-05-29 14:03:56.817 [main] NrtsearchClientBuilder$AddDocumentsClientBuilder - Read row 1
[INFO ] 2025-05-29 14:03:56.822 [main] NrtsearchClient - sent async addDocumentsRequest to server...
[INFO ] 2025-05-29 14:03:56.958 [grpc-default-executor-0] NrtsearchClient - Received response for genId: genId: "2"
primaryId: "6f9bf1f9-ed03-4e6b-ae23-9c40fe49897d"

[INFO ] 2025-05-29 14:03:56.958 [grpc-default-executor-0] NrtsearchClient - Received final response from server
dhruva@HVVK4XYGRK nrtsearch %