Skip to content

fix(external): prevent task ID URL path injection#3324

Merged
danielaskdd merged 3 commits into
HKUDS:mainfrom
VectorPeak:fix/external-task-id-url-encoding
Jun 24, 2026
Merged

fix(external): prevent task ID URL path injection#3324
danielaskdd merged 3 commits into
HKUDS:mainfrom
VectorPeak:fix/external-task-id-url-encoding

Conversation

@VectorPeak

@VectorPeak VectorPeak commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Description

This fixes a request path injection risk (CWE-116: Improper Encoding or Escaping of Output) in the external parser clients, where task identifiers returned by Docling/MinerU services are interpolated directly into URL path segments before polling or result-download requests.

The affected paths are built from a configured parser endpoint plus a service-returned task identifier:

url = f"{self.endpoint}{POLL_PATH.format(task_id=task_id)}"
url = f"{self.endpoint}{RESULT_PATH.format(task_id=task_id)}"

poll_url = f"{self.official_endpoint}/api/v4/extract-results/batch/{batch_id}"
poll_url = f"{self.local_endpoint}/tasks/{task_id}"
result_url = f"{self.local_endpoint}/tasks/{task_id}/result"

task_id / batch_id are opaque values returned by the external parser service. If a compromised, malicious, or incompatible parser service returns a value containing URL-reserved characters such as /, ?, #, or dot-segment-like path components, the HTTP client can interpret that value as request structure rather than a single path segment.

The concrete impact is that LightRAG may send follow-up poll/result requests to an unintended path on the configured parser service origin. This does not change the configured host by itself, but it weakens the URL path boundary between LightRAG and the external parser service.

The parser client is the correct place to enforce this boundary because it is the component that consumes service-returned task identifiers and turns them into follow-up request paths. Relying on external services to always return URL-safe segment text leaves the final request construction too permissive.

Changes Made

  • Encode external parser task identifiers before inserting them into URL path segments.
  • Apply the same path-segment boundary to Docling poll/result requests, MinerU official batch polling, and MinerU local poll/result requests.
  • Keep existing task status handling, result download behavior, endpoint configuration, and normal URL-safe task IDs unchanged.

Checklist

  • Changes tested locally
  • Code reviewed
  • Documentation updated (if necessary)
  • Unit tests added (if applicable)

Additional Notes

Evidence:

The issue can be reproduced at the URL-construction layer with a parser-service-controlled identifier. This local reproduction uses fake parser clients only; it does not contact a real parser service.

LightRAG external parser identifier path injection evidence demo
Scope: local client URL-construction reproduction; no external service is contacted.

payload task_id: ../admin?x=1#frag
payload batch_id: ../batch?x=1#frag

Docling vulnerable follow-up URLs:
GET http://parser.local/v1/status/poll/../admin?x=1#frag?wait=1
GET http://parser.local/v1/result/../admin?x=1#frag

MinerU official vulnerable follow-up URL:
GET https://mineru.net/api/v4/extract-results/batch/../batch?x=1#frag

MinerU local vulnerable follow-up URLs:
POST http://mineru.local/tasks
GET http://mineru.local/tasks/../admin?x=1#frag
GET http://mineru.local/tasks/../admin?x=1#frag/result

Expected safe path segments:
..%2Fadmin%3Fx%3D1%23frag
..%2Fbatch%3Fx%3D1%23frag

A local mock parser service also shows the same issue at the HTTP request-target level. The client constructs a vulnerable MinerU official polling URL with a crafted batch_id; the mock server records the request path it actually receives.

from http.server import BaseHTTPRequestHandler, HTTPServer
from threading import Thread
from urllib.parse import quote

import httpx

PAYLOAD = "../admin?x=1#frag"

class Recorder(BaseHTTPRequestHandler):
    seen = []

    def do_GET(self):
        Recorder.seen.append(self.path)
        self.send_response(200)
        self.end_headers()
        self.wfile.write(b"ok")

    def log_message(self, *_):
        pass

server = HTTPServer(("127.0.0.1", 0), Recorder)
Thread(target=server.serve_forever, daemon=True).start()
base = f"http://127.0.0.1:{server.server_address[1]}"

vulnerable_url = f"{base}/api/v4/extract-results/batch/{PAYLOAD}"
fixed_url = f"{base}/api/v4/extract-results/batch/{quote(PAYLOAD, safe='')}"

print("payload:", PAYLOAD)
httpx.get(vulnerable_url)
print("\n[vulnerable]")
print("client requested:", vulnerable_url)
print("server saw      :", Recorder.seen[-1])

httpx.get(fixed_url)
print("\n[fixed]")
print("client requested:", fixed_url)
print("server saw      :", Recorder.seen[-1])

server.shutdown()

Run locally:

.\.venv\Scripts\python.exe .\poc_task_id_path_injection.py

Observed output:

payload: ../admin?x=1#frag

[vulnerable]
client requested: http://127.0.0.1:55881/api/v4/extract-results/batch/../admin?x=1#frag
server saw      : /api/v4/extract-results/admin?x=1
verdict         : escaped intended /batch/{id} path segment

[fixed]
client requested: http://127.0.0.1:55881/api/v4/extract-results/batch/..%2Fadmin%3Fx%3D1%23frag
server saw      : /api/v4/extract-results/batch/..%2Fadmin%3Fx%3D1%23frag
verdict         : stayed inside intended /batch/{id} path segment

In the vulnerable case, ../ changes the request path and ?x=1 becomes a query string. In the fixed case, quote(..., safe="") keeps the crafted identifier inside the intended path segment.

The fragment is included only to show that the unencoded value is parsed as URL syntax. It is not relied on as server-side evidence because URL fragments are normally not sent in HTTP requests.

local console PoC output

Possible call chain / impact:

Docling:
lightrag/parser/external/docling/client.py
  -> DoclingRawClient.download_into(...)
  -> self._submit(...)
  -> task_id from Docling response
  -> self._poll_until_done(client, task_id)
  -> GET /v1/status/poll/{task_id}?wait=N
  -> self._download_zip_bytes(client, task_id)
  -> GET /v1/result/{task_id}

MinerU official:
lightrag/parser/external/mineru/client.py
  -> MinerURawClient.download_into(...)
  -> self._download_official(...)
  -> batch_id from MinerU official upload URL response
  -> self._poll_official_batch(client, batch_id, upload_name)
  -> GET /api/v4/extract-results/batch/{batch_id}

MinerU local:
lightrag/parser/external/mineru/client.py
  -> MinerURawClient.download_into(...)
  -> self._download_local(...)
  -> task_id from MinerU local /tasks response
  -> self._poll_local_task(client, task_id)
  -> GET /tasks/{task_id}
  -> self._download_zip(client, /tasks/{task_id}/result, raw_dir)

This PR only changes URL path-segment encoding for parser-service identifiers. It does not change parser endpoint selection, upload payloads, polling semantics, zip extraction, caching, chunking, embedding, retrieval, or normal behavior for already URL-safe task IDs.

Validation:

  • .\.venv\Scripts\python.exe -m ruff check lightrag\parser\external\docling\client.py lightrag\parser\external\mineru\client.py - passed
  • .\.venv\Scripts\python.exe -m pre_commit run --files lightrag\parser\external\docling\client.py lightrag\parser\external\mineru\client.py - passed
  • .\.venv\Scripts\python.exe -m pytest tests\parser\external\docling\test_client.py tests\parser\external\mineru\test_client.py - passed, 30 tests
  • .\.venv\Scripts\python.exe -m pytest tests\parser\external - passed, 174 tests
  • git diff --check - passed
  • GitHub CI: Linting and Formatting, Offline Tests (3.12), and Offline Tests (3.14) - passed

@VectorPeak
VectorPeak force-pushed the fix/external-task-id-url-encoding branch from c0801e5 to 6a5239b Compare June 24, 2026 11:01
@VectorPeak
VectorPeak force-pushed the fix/external-task-id-url-encoding branch from 6a5239b to 6aea62e Compare June 24, 2026 11:12
@VectorPeak
VectorPeak marked this pull request as ready for review June 24, 2026 11:20
@VectorPeak VectorPeak changed the title fix(external): encode parser task IDs in request paths fix(external): prevent task ID path injection Jun 24, 2026
@VectorPeak VectorPeak changed the title fix(external): prevent task ID path injection fix(external): prevent task ID URL path injection Jun 24, 2026
@danielaskdd

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown
Contributor

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 6aea62e501

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

- encode local poll and result task ids as a single path segment
- prevent crafted ids from escaping `/tasks/{id}` or adding queries
- add regression tests for official and local mineru flows
- update docling test fakes to match encoded task id routing
@danielaskdd
danielaskdd merged commit 24e0144 into HKUDS:main Jun 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants