-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Description
This PR #25445 introduced lock-free commits in iceberg HMS catalog, however if locking is used then there is no need to pass {"expected_parameter_key": "metadata_location", "expected_parameter_value", metadataLocation} to the environmentContext. This leads to unnecessary operation on the metastore side to verify that the metadata location is still the same before commiting while Hive locking is already being used to ensure the commit is atomic, example SQL query:
UPDATE "TABLE_PARAMS"
SET "PARAM_VALUE" = 'P2' -- The new path
WHERE "TBL_ID" = 123
AND "PARAM_KEY" = 'metadata_location'
AND "PARAM_VALUE" = 'P1'; -- The expected old path
I'm using HMS 3.1.2 and this was a breaking change where commits were not succeeding (probably due to a bug in HMS or underlying MySQL DB) that caused transactions to become stuck in the DB leading to timeouts and failed commits and eventually the MySQL DB used by the HMS wasn't able to process other transactions timely leading to timeout across all other HMS operations until the underlying DB was restarted to clear the stuck transactions and the Trino version was rolled back.
This change should only affect Trino deplyoments that want to use lock-free commits and have iceberg.hive-catalog.locking-enabled
set as false