fix: allow null auto_id primary column in DataFrame insert

[VectorPeak]

This PR fixes a DataFrame insert validation crash when an auto_id primary key column is present but contains only null values.

What Problem This Solves

check_insert_schema() already treats an all-null auto_id primary key column as "no primary key values were provided" and tries to remove that column before validating the remaining DataFrame fields.

The current code builds a list of DataFrame column labels, but then attempts to remove the FieldSchema object itself:

columns = list(data.columns)
columns.remove(schema.primary_field)

Since data.columns contains column names, not FieldSchema objects, this raises:

ValueError: list.remove(x): x not in list

A minimal trigger is an auto_id schema with a DataFrame that keeps the primary key column but leaves it entirely null:

pd.DataFrame({
    "id": [None, None],
    "vec": [[1.0, 2.0], [3.0, 4.0]],
})

Change

Remove the primary key column by its field name instead of by the FieldSchema object:

columns.remove(schema.primary_field.name)

This keeps the existing behavior unchanged for non-null auto_id primary key values: if users explicitly provide primary key data, check_insert_schema() still raises DataNotMatchException.

Evidence

Before this fix, check_insert_schema() crashes before it can validate the remaining DataFrame fields:

ValueError: list.remove(x): x not in list

After this fix:

• DataFrame inserts with an all-null auto_id primary key column pass schema validation.
• DataFrame inserts with non-null auto_id primary key values are still rejected with DataNotMatchException.

Possible call chain / impact

Collection.insert(pd.DataFrame)
  -> check_insert_schema(...)
    -> auto_id primary field exists in DataFrame
    -> all primary key values are null
    -> remove auto_id primary column before validating remaining fields

This PR only changes the column removal path for all-null auto_id primary key columns in DataFrame insert validation. It does not change list-based inserts, row-based inserts, upsert validation, or the existing rejection of non-null auto_id primary key values.

Testing

• uv run --python 3.11 --group dev pytest tests/unit/orm/test_schema.py::TestCheckInsertSchema -q
• uv run --python 3.11 --group dev pytest tests/unit/orm/test_prepare.py -k auto_id -q
• uv run --python 3.11 --group dev black --check pymilvus/orm/schema.py tests/unit/orm/test_schema.py
• uv run --python 3.11 --group dev ruff check pymilvus/orm/schema.py tests/unit/orm/test_schema.py
• git diff --check -- pymilvus/orm/schema.py tests/unit/orm/test_schema.py

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.02%. Comparing base (0ee0227) to head (18a0389).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3673      +/-   ##
==========================================
+ Coverage   93.93%   94.02%   +0.08%     
==========================================
  Files          72       72              
  Lines       15541    15567      +26     
==========================================
+ Hits        14599    14637      +38     
+ Misses        942      930      -12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[VectorPeak]

Additional trigger context for reviewers:

This bug is not a broad DataFrame insert failure. It requires four conditions to line up:

1. The insert path uses Collection.insert(pd.DataFrame).
2. The collection schema has an auto_id=True primary key.
3. The incoming DataFrame still includes the primary key column, for example id.
4. That primary key column is entirely null (None / NaN).

In that case, check_insert_schema() already treats the all-null auto_id primary column as "no user-provided primary key values" and tries to remove it before validating the remaining fields. The crash happens because the code removes the FieldSchema object from a list of DataFrame column-name strings:

columns = list(data.columns)
columns.remove(schema.primary_field)

A realistic way to hit this is an ETL/DataFrame workflow where the source table or CSV/Parquet schema still has an id column, but new rows leave it null because Milvus is expected to generate IDs. From the user's perspective, the primary key was intentionally left empty for auto_id, so the client-side ValueError: list.remove(x): x not in list is surprising and hides the intended validation path.

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request

[XuanYang-cn]

/lgtm

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: VectorPeak, XuanYang-cn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [XuanYang-cn]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[pymilvus-bot]

✅ Backport Created
Hi @VectorPeak, Backport PR for 2.6 has been created: #3677

(cc @XuanYang-cn)

[pymilvus-bot]

✅ Backport Created
Hi @VectorPeak, Backport PR for 3.0 has been created: #3678

(cc @XuanYang-cn)