MDEV-28730 Remove internal parser usage from InnoDB fts by Thirunarayanan · Pull Request #4443 · MariaDB/server

Thirunarayanan · 2025-11-17T07:42:04Z

The Jira issue number for this PR is: MDEV-28730

Description

Remove internal parser/SQL-graph usage and migrate FTS paths to QueryExecutor

Introduced QueryExecutor (row0query.{h,cc}) and FTSQueryExecutor abstractions for
clustered, secondary scans and DML.

Refactored fetch/optimize code to use QueryExecutor::read(), read_by_index()
with RecordCallback, replacing SQL graph flows

Added CommonTableReader and ConfigReader callbacks for common/CONFIG tables

Implemented fts_index_fetch_nodes(trx, index, word, user_arg, FTSRecordProcessor, compare_mode)
and rewrote fts_optimize_write_word() to delete/insert via executor with fts_aux_data_t

Removed fts_doc_fetch_by_doc_id() and FTS_FETCH_DOC_BY_ID_* macros, updating callers to
fts_query_fetch_document()

Tightened fts_select_index{,_by_range,by_hash} return type to uint8_t;
Removed fts0sql.cc and eliminated fts_table_t from fts_query_t/fts_optimize_t.*

Release Notes

Removed the sql parser usage from fulltext subsystem

How can this PR be tested?

For QA purpose, Run RQG testing involving Fulltext subsystem

Basing the PR against the correct MariaDB version

This is a new feature or a refactoring, and the PR is based against the main branch.
This is a bug fix, and the PR is based against the earliest maintained branch in which the bug can be reproduced.

PR quality check

I checked the CODING_STANDARDS.md file and my PR conforms to this where appropriate.
For any trivial modifications to the PR, I am ok with the reviewer making the changes themselves.

CLAassistant · 2025-11-17T07:42:10Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

grooverdan · 2025-11-18T04:32:39Z

In addition to the CI failures needing correcting, does this mean storage/innobase/pars/ and the fts_parse_sql at least can be removed from storage/innobase/include/fts0priv.h?

Great to see the parser going away.

dr-m

Here are some quick initial comments.

dr-m

Here are some more comments. The error propagation is better now, but I would like to see some more effort to avoid the number of dict_sys.latch acquisitions. This should be tested as well, in a custom benchmark.

Even though we are adding quite a bit of code, I was pleasantly surprised that the size of a x86-64 CMAKE_BUILD_TYPE=RelWithDebInfo executable would increase by only 20 KiB. I believe that removing the InnoDB SQL parser (once some more code has been refactored) would remove more code than that.

dr-m · 2025-12-04T14:22:13Z

+    if (UNIV_LIKELY(error == DB_SUCCESS ||
+                    error == DB_RECORD_NOT_FOUND))
+    {
+      fts_sql_commit(trx);
+      if (error == DB_RECORD_NOT_FOUND) error = DB_SUCCESS;


What is the reason for committing and re-starting the transaction after each iteration? Is it one transaction per fetched row?

Here, the second if had better be removed. A blind assignment error= DB_SUCCESS should be shorter and incur less overhead. It is basically just zeroing out a register.

It is not single word. If no starting word is specified, fetch all words.
Otherwise, fetch words starting from the given word. This supports pagination
when memory limits are exceeded. There could be more words in all auxiliary tables and doesn't make sense
to open transaction for too long time.

I would like to address this issue as a seperate one. Because this is what fulltext behaviour before this patch.
Yes, I agree that btr_cur_t iteration is enough to retrieve all the words from auxiliary table.

dr-m · 2025-12-04T14:27:38Z

+      fts_sql_rollback(trx);
+      if (error == DB_LOCK_WAIT_TIMEOUT)
+      {
+        ib::warn() << "Lock wait timeout reading FTS index. Retrying!";
+        trx->error_state = DB_SUCCESS;
+      }
+      else
+      {
+        ib::error() << "Error occurred while reading FTS index: " << error;
+        break;


Which index and table are we reading? Why are we not disclosing the name of the index or the table?

Please, let’s avoid using ib::logger::logger in any new code, and invoke sql_print_error or sql_print_warning directly.
Is this code reachable? How would a lock wait timeout be possible?

Can this ever be a locking read? When and why would it need to be one? After all, as the code stands now, we are committing the transaction (and releasing any locks) after every successful iteration. Hence, there will be no consistency guarantees on the data that we are reading.

"Auxiliary table" in the function comment is inaccurate. Can we be more specific? Is this always reading entries from a partition of an inverted index? Which functions can write these tables? (What are the potential conflicts?)

Do we even need a transaction object here, or would a loop around btr_cur_t suffice?

Replaced ib:: with sql_print_. Yet to address how lock wait can happen.

dr-m

I tried to review the MVCC logic, but I did not fully understand it.

It would be much more convenient to review this if the tables or indexes on which the new code is expected to be invoked were prominently documented in debug assertions.

dr-m · 2026-01-22T07:51:21Z

+  mem_heap_t* version_heap= nullptr;
+  mem_heap_t* offsets_heap= nullptr;
+  rec_offs* offsets= nullptr;
+  rec_offs* version_offsets= nullptr;


Why are we not allocating offsets and version_offsets from the stack, and forcing memory to be allocated from the heap every single time?

Could we cope with a single mem_heap_t object here? Or none at all? This function is supposed to be run on some tables whose schema we know in advance, right? Can this ever be run on a user-defined table? Unfortunately, in none of the callers of QueryExecutor::process_record_with_mvcc() did I find any assertion on the table names or the schema.

dr-m · 2026-01-22T08:35:43Z

+dberr_t QueryExecutor::process_record_with_mvcc(
+   dict_index_t *clust_index, const rec_t *rec,
+   RecordCallback &callback, dict_index_t *sec_index,
+   const rec_t *sec_rec) noexcept
+{
+  ut_ad(m_mtr.trx);
+  ut_ad(srv_read_only_mode || m_mtr.trx->read_view.is_open());


We are missing ut_ad(sec_index->is_normal_btree()) and possibly other assertions.

Is this known to be limited to some specific tables or indexes? For example, if the table is not a FTS_ internal table and not mysql.innodb_index_stats or mysql.innodb_table_stats, would we know that the sec_index is FTS_DOC_ID_INDEX(FTS_DOC_ID)? Documenting the intended usage with debug assertions would make it easier to review this and to suggest possible optimization.

Added the assert in lookup_clustered_record() saying secondary index is only FTS_DOC_ID_IDX

dr-m · 2026-01-22T08:46:02Z

+  if (!rec_get_deleted_flag(result_rec,
+                            clust_index->table->not_redundant()))


In row_sel_sec_rec_is_for_clust_rec() this condition would be checked for every record version that row_sel_get_clust_rec() or Row_sel_get_clust_rec_for_mysql::operator()() is processing, but here we do it only after fetching the clustered index record version. I think that this should be OK. But, I would have appreciated a source code comment that refers to the other implementation of this logic.

However, there seems to be a bigger problem that here we are checking at most one earlier version, instead of traversing all versions that are visible in the current read view. This might be OK if this function is only going to be invoked on some specific FTS_ tables. But then there should be debug assertions that would document the assumptions about the tables, as well as source code comments that explain why we only check for one older version.

Using Row_sel_get_clust_rec_for_mysql::operator() directly. I thought better to use existing function

dr-m

The QueryExecutor bears some similarity with functions that operate on row_prebuilt_t. Could we unify them better? Would it be feasible to construct row_prebuit_t objects in the FULLTEXT subsystem, so that even higher-level functions such as row_search_mvcc() can be invoked?

dr-m · 2026-02-23T06:26:44Z

-static void row_sel_reset_old_vers_heap(row_prebuilt_t *prebuilt)
+static void row_sel_reset_old_vers_heap(mem_heap_t** old_vers_heap)
 {
-  if (prebuilt->old_vers_heap)
-    mem_heap_empty(prebuilt->old_vers_heap);
+  if (*old_vers_heap)
+    mem_heap_empty(*old_vers_heap);
  else
-    prebuilt->old_vers_heap= mem_heap_create(200);
+    *old_vers_heap= mem_heap_create(200);
 }


This change is related to some memory leaks; see for example https://buildbot.mariadb.org/#/builders/1031/builds/3522

CURRENT_TEST: mariabackup.apply-log-only-incr … ==230009==ERROR: LeakSanitizer: detected memory leaks Direct leak of 64000 byte(s) in 200 object(s) allocated from: #0 0x56452b57e2e4 in malloc (/home/buildbot/bld/sql/mariadbd+0x22a92e4) (BuildId: 4c5cec9ca993f3edf8770e1f50461ef96bb740ef) #1 0x56452d7a5177 in ut_allocator<unsigned char, true>::allocate(unsigned long, unsigned char const*, unsigned int, bool, bool) /home/buildbot/src/storage/innobase/include/ut0new.h:375:11 #2 0x56452d8fe784 in mem_heap_create_block_func(mem_block_info_t*, unsigned long, unsigned long) /home/buildbot/src/storage/innobase/mem/mem0mem.cc:276:37 #3 0x56452daae96b in mem_heap_create_func(unsigned long, unsigned long) /home/buildbot/src/storage/innobase/include/mem0mem.inl:306:10 #4 0x56452daae96b in row_sel_reset_old_vers_heap(mem_block_info_t**) /home/buildbot/src/storage/innobase/row/row0sel.cc:3268:21 #5 0x56452da93291 in row_sel_build_prev_vers_for_mysql(trx_t*, dict_index_t*, unsigned char const*, unsigned short**, mem_block_info_t**, mem_block_info_t*, unsigned char**, dtuple_t**, mtr_t*) /home/buildbot/src/storage/innobase/row/row0sel.cc:3294:2 #6 0x56452da99dfc in row_search_mvcc(unsigned char*, page_cur_mode_t, row_prebuilt_t*, unsigned long, unsigned long) /home/buildbot/src/storage/innobase/row/row0sel.cc:5355:11 #7 0x56452d749096 in ha_innobase::general_fetch(unsigned char*, unsigned int, unsigned int) /home/buildbot/src/storage/innobase/handler/ha_innodb.cc:9229:24 #8 0x56452c8c146f in handler::ha_rnd_next(unsigned char*) /home/buildbot/src/sql/handler.cc:4046:5 #9 0x56452b700eb4 in rr_sequential(READ_RECORD*) /home/buildbot/src/sql/records.cc:509:35 #10 0x56452bc778f3 in READ_RECORD::read_record() /home/buildbot/src/sql/records.h:77:30 #11 0x56452bc778f3 in sub_select(JOIN*, st_join_table*, bool) /home/buildbot/src/sql/sql_select.cc:24592:18 #12 0x56452bcfa10e in do_select(JOIN*, Procedure*) /home/buildbot/src/sql/sql_select.cc:24086:14 #13 0x56452bcf7ac3 in JOIN::exec_inner() /home/buildbot/src/sql/sql_select.cc:5125:50 #14 0x56452bcf4a1b in JOIN::exec() /home/buildbot/src/sql/sql_select.cc:4913:8 #15 0x56452bc7afa5 in mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /home/buildbot/src/sql/sql_select.cc:5439:21 #16 0x56452bc79ce1 in handle_select(THD*, LEX*, select_result*, unsigned long long) /home/buildbot/src/sql/sql_select.cc:636:10 #17 0x56452bb6d334 in execute_sqlcom_select(THD*, TABLE_LIST*) /home/buildbot/src/sql/sql_parse.cc:6212:12 #18 0x56452bb50a93 in mysql_execute_command(THD*, bool) /home/buildbot/src/sql/sql_parse.cc:3987:12 #19 0x56452bb341a3 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/src/sql/sql_parse.cc:7940:18 #20 0x56452bb2b056 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /home/buildbot/src/sql/sql_parse.cc:1896:7 #21 0x56452bb36765 in do_command(THD*, bool) /home/buildbot/src/sql/sql_parse.cc:1432:17 #22 0x56452c16bfcc in do_handle_one_connection(CONNECT*, bool) /home/buildbot/src/sql/sql_connect.cc:1503:11 #23 0x56452c16b805 in handle_one_connection /home/buildbot/src/sql/sql_connect.cc:1415:5 #24 0x56452d4912e8 in pfs_spawn_thread /home/buildbot/src/storage/perfschema/pfs.cc:2198:3 #25 0x56452b57bc66 in asan_thread_start(void*) asan_interceptors.cpp.o

dr-m · 2026-02-23T06:31:10Z

-	row_sel_reset_old_vers_heap(prebuilt);
+	row_sel_reset_old_vers_heap(&old_vers_heap);

 	return row_vers_build_for_consistent_read(
 		rec, mtr, clust_index, offsets,
-		&prebuilt->trx->read_view, offset_heap,
-		prebuilt->old_vers_heap, old_vers, vrow);
+		&trx->read_view, offset_heap,
+		old_vers_heap, old_vers, vrow);


We are leaking the possible old contents of old_vers_heap here.

dr-m · 2026-02-23T06:31:39Z

 					rec_get_offsets(rec, clust_index) */
 	mem_heap_t**	offset_heap,	/*!< in/out: memory heap from which
 					the offsets are allocated */
+	mem_heap_t*	old_vers_heap,


The purpose and the lifetime of this parameter are not documented.

dr-m · 2026-02-23T06:34:50Z

 dberr_t
 Row_sel_get_clust_rec_for_mysql::operator()(
 /*============================*/
-	row_prebuilt_t*	prebuilt,/*!< in: prebuilt struct in the handle */
+	dtuple_t*	clust_ref,/*!< in: clustered index search tuple */
+	btr_pcur_t*	clust_pcur,/*!< in/out: clustered index cursor */
+	lock_mode	select_lock_type,/*!< in: lock mode for selection */
+	trx_t*		trx,	/*!< in: transaction */
+	mem_heap_t*	old_vers_heap,/*!< in/out: memory heap for old versions */
 	dict_index_t*	sec_index,/*!< in: secondary index where rec resides */
 	const rec_t*	rec,	/*!< in: record in a non-clustered index; if
 				this is a locking read, then rec is not
 				allowed to be delete-marked, and that would
 				not make sense either */
 	que_thr_t*	thr,	/*!< in: query thread */
 	const rec_t**	out_rec,/*!< out: clustered record or an old version of
 				it, NULL if the old version did not exist
 				in the read view, i.e., it was a fresh
 				inserted version */
 	rec_offs**	offsets,/*!< in: offsets returned by
 				rec_get_offsets(rec, sec_index);
 				out: offsets returned by
 				rec_get_offsets(out_rec, clust_index) */
 	mem_heap_t**	offset_heap,/*!< in/out: memory heap from which
 				the offsets are allocated */
 	dtuple_t**	vrow,	/*!< out: virtual column to fill */
 	mtr_t*		mtr)	/*!< in: mtr used to get access to the
 				non-clustered record; the same mtr is used to
 				access the clustered index */


There already are very many parameters passed to this member function call. Adding 4 more parameters will complicate the call setup; I don’t think that any ABI would support passing this number of parameters in registers. Can we move most of the parameters to data members?

Do we need both trx and thr? Is thr->graph->trx not always accessible?

Instead of changing this interface, could we initialize a row_prebuilt_t in the code path that is missing it?

dr-m · 2026-02-23T06:45:31Z

+enum class RecordCompareAction
+{
+  /** Do not process this record, continue traversal */
+  SKIP,
+  /** Process this record via process_record */
+  PROCESS,
+  /** Stop traversal immediately */
+  STOP
+};


PROCESS is the return value of the default argument to RecordCallback as well as the first value that row0query.cc will be checking the return value against. I’d suggest to make its value 0.

dr-m · 2026-02-23T06:49:45Z

+    if (action == RecordCompareAction::PROCESS)
+    {
+      dberr_t proc_err= callback->process_record(rec, clust_index, offsets);
+      if (proc_err != DB_SUCCESS)
+      {
+        err= proc_err;
+        goto err_exit;
+      }
+    }
+    if (action == RecordCompareAction::SKIP)
+    {
+       err= DB_RECORD_NOT_FOUND;
+       goto err_exit;
+    }


Why do we need a separate variable proc_err? We could just assign to err directly.

Should the if (action == RecordCompareAction::SKIP) actually be else, so that it would also cover the STOP value? If not, then it should be else if and we’d need an assertion and a comment that documents what we are supposed to do with the STOP value.

dr-m · 2026-02-23T07:02:32Z

+  /** Constructor with processor function and optional comparator
+  @param[in] processor    Function to process each record
+  @param[in] comparator   Optional function to filter records (default: accept all) */
+  RecordCallback(
+    RecordProcessor processor,
+    RecordComparator comparator= [](const dtuple_t*, const rec_t*,
+                                    const dict_index_t*)
+      { return RecordCompareAction::PROCESS; })
+    : process_record(std::move(processor)),
+      compare_record(std::move(comparator)) {}


We should not use any [in] or [out] in new code. Why do we need std::move()?

As far as I understand, only fts_load_user_stopword() is relying on this default comparator. It exercises only a small part of the new code. Could we default to nullptr and assert in most places that the comparator function pointer has been set?

dr-m · 2026-02-23T07:03:07Z

+  /** Called for each matching record */
+  RecordProcessor process_record;
+
+  /** Comparison function for custom filtering */
+  RecordComparator compare_record;


Why are these not const? They are not supposed to change after construction, are they?

mariadb-SaahilAlam · 2026-05-15T05:44:52Z

+    if (error == DB_SUCCESS)
+    {
+      /* There must be some nodes to read. */
+      ut_a(ib_vector_size(optim->words) > 0);


Hitting this Assertion
# 2026-05-14T14:04:17 [56807] | InnoDB: Failing assertion: ib_vector_size(optim->words) > 0
Bug raised:https://jira.mariadb.org/browse/MDEV-39616

mariadb-SaahilAlam · 2026-05-15T06:25:30Z

  noexcept
 {
  ut_ad(dtuple_check_typed(&tuple));
  ut_ad(page_rec_is_leaf(rec));


Found this Assertion:-
# 2026-05-14T14:34:41 [1226396] | mariadbd: /data/Server/MDEV-28730_new/storage/innobase/page/page0cur.cc:87: int cmp_dtuple_rec_bytes(const rec_t*, const dict_index_t&, const dtuple_t&, int*, ulint): Assertion page_rec_is_leaf(rec)' failed.`
Bug raised:https://jira.mariadb.org/browse/MDEV-39617

mariadb-SaahilAlam · 2026-05-15T11:23:32Z

+                                     ULINT_UNDEFINED, &m_heap);
+
+  dberr_t err= DB_SUCCESS;
+  ulint cmpl_info= UPD_NODE_NO_ORD_CHANGE | UPD_NODE_NO_SIZE_CHANGE;


Crash found during testing

Thread 3 received signal SIGSEGV, Segmentation fault. [Switching to Thread 3906580.3917832] thd_storage_lock_wait (thd=0x0, value=11) at /data/Server/MDEV-28730_new/sql/sql_class.cc:451 451 thd->utime_after_lock+= value; (rr) bt #0 thd_storage_lock_wait (thd=0x0, value=11) at /data/Server/MDEV-28730_new/sql/sql_class.cc:451 #1 0x000056d4586cb8e3 in lock_sys_t::wait_resume (this=this@entry=0x56d459ee5c80 <lock_sys>, thd=0x0, start=start@entry=..., now=...) at /data/Server/MDEV-28730_new/storage/innobase/lock/lock0lock.cc:2108 #2 0x000056d4586c9128 in lock_wait (thr=thr@entry=0x1276380563c8) at /data/Server/MDEV-28730_new/storage/innobase/lock/lock0lock.cc:2410 #3 0x000056d45879c93d in row_mysql_handle_errors (new_err=new_err@entry=0x799151a0c4a0, trx=trx@entry=0x55aa7560ff00, thr=thr@entry=0x1276380563c8, savept=savept@entry=0x0) at /data/Server/MDEV-28730_new/storage/innobase/row/row0mysql.cc:698 #4 0x000056d4587d308d in row_search_mvcc<InnoDBPolicy<false, false> > (buf=buf@entry=0x799151a0cd90 "H\203>Y\324V", mode=mode@entry=PAGE_CUR_GE, prebuilt=0x127638056468, match_mode=match_mode@entry=0, direction=direction@entry=0) at /data/Server/MDEV-28730_new/storage/innobase/row/row0sel.cc:6051 #5 0x000056d45893409c in row_search_mvcc_callback_dml (direction=0, match_mode=0, prebuilt=<optimized out>, mode=PAGE_CUR_GE, callback=0x799151a0cd90) at /data/Server/MDEV-28730_new/storage/innobase/include/row0sel.h:202 #6 QueryExecutor::select_for_update (this=this@entry=0x799151a0ce70, table=table@entry=0x12763803fbf8, search_tuple=search_tuple@entry=0x799151a0cd20, callback=callback@entry=0x799151a0cd90) at /data/Server/MDEV-28730_new/storage/innobase/row/row0query.cc:331 #7 0x000056d458906f5f in FTSQueryExecutor::read_config_with_lock (this=this@entry=0x799151a0ce70, key=key@entry=0x56d458cb4922 "synced_doc_id", callback=...) at /data/Server/MDEV-28730_new/storage/innobase/fts/fts0exec.cc:535 #8 0x000056d4589044bf in fts_config_get_value (executor=executor@entry=0x799151a0ce70, table=table@entry=0x127638103a68, name=name@entry=0x56d458cb4922 "synced_doc_id", value=value@entry=0x799151a0ce50) at /data/Server/MDEV-28730_new/storage/innobase/fts/fts0config.cc:46 #9 0x000056d4586a3eb3 in i_s_fts_config_fill (thd=0x127638000d58, tables=<optimized out>) at /data/Server/MDEV-28730_new/storage/innobase/handler/i_s.cc:3161 #10 0x000056d45813ed14 in get_schema_tables_result (join=join@entry=0x1276380178a8, executed_place=executed_place@entry=PROCESSED_BY_JOIN_EXEC) at /data/Server/MDEV-28730_new/sql/sql_show.cc:9924 #11 0x000056d4581286d1 in JOIN::exec_inner (this=this@entry=0x1276380178a8) at /data/Server/MDEV-28730_new/sql/sql_select.cc:5086 #12 0x000056d4581288f1 in JOIN::exec (this=this@entry=0x1276380178a8) at /data/Server/MDEV-28730_new/sql/sql_select.cc:4913 #13 0x000056d4581278df in mysql_select (thd=thd@entry=0x127638000d58, tables=0x127638016628, fields=..., conds=0x0, og_num=0, order=<optimized out>, group=0x0, having=0x0, proc_param=0x0, select_options=2701396736, result=0x127638017880, unit=0x127638005278, select_lex=0x127638015ea8) at /data/Server/MDEV-28730_new/sql/sql_select.cc:5439 #14 0x000056d458127ab4 in handle_select (thd=thd@entry=0x127638000d58, lex=lex@entry=0x127638005198, result=result@entry=0x127638017880, setup_tables_done_option=setup_tables_done_option@entry=0) at /data/Server/MDEV-28730_new/sql/sql_select.cc:636 #15 0x000056d45809d35e in execute_sqlcom_select (thd=thd@entry=0x127638000d58, all_tables=0x127638016628) at /data/Server/MDEV-28730_new/sql/sql_parse.cc:6214 #16 0x000056d4580a6dd3 in mysql_execute_command (thd=thd@entry=0x127638000d58, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /data/Server/MDEV-28730_new/sql/sql_parse.cc:3988 #17 0x000056d4580ac979 in mysql_parse (thd=thd@entry=0x127638000d58, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x799151a0e3c0) at /data/Server/MDEV-28730_new/sql/sql_parse.cc:7942 #18 0x000056d4580adfcd in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x127638000d58, packet=packet@entry=0x12763800b669 "\nSELECT COUNT(*) FROM INFORMATION_SCHEMA.INNODB_FT_CONFIG /* E_R Thread6 QNO 79 CON_ID 21 */ ", packet_length=packet_length@entry=93, blocking=blocking@entry=true) at /data/Server/MDEV-28730_new/sql/sql_parse.cc:1898 #19 0x000056d4580af5d2 in do_command (thd=thd@entry=0x127638000d58, blocking=blocking@entry=true) at /data/Server/MDEV-28730_new/sql/sql_parse.cc:1432 #20 0x000056d45820859d in do_handle_one_connection (connect=<optimized out>, connect@entry=0x56d45b7f35b8, put_in_cache=put_in_cache@entry=true) at /data/Server/MDEV-28730_new/sql/sql_connect.cc:1503 #21 0x000056d4582087ba in handle_one_connection (arg=0x56d45b7f35b8) at /data/Server/MDEV-28730_new/sql/sql_connect.cc:1415 #22 0x0000764474ab2a94 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447 #23 0x0000764474b3fa34 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

Bug raised:-https://jira.mariadb.org/browse/MDEV-39621

This should be fixed in the latest patch.

mariadb-SaahilAlam · 2026-05-18T08:02:15Z

 end_sync:
 	if (error == DB_SUCCESS && !sync->interrupted) {
-		error = fts_sync_commit(sync);
+		error = fts_sync_commit(&executor, sync);


hitting this assertion on origin/main-MDEV-28730 a9665958948732b39b5a71f2b66672a97883837b

# 2026-05-15T16:50:34 [1679424] | 2026-05-15 16:50:01 22 [ERROR] InnoDB: ( Lock wait ) writing word node to FTS auxiliary index table test/table300_innodb_int_autoinc # 2026-05-15T16:50:34 [1679424] | mariadbd: /data/Server/MDEV-28730_1/storage/innobase/include/trx0trx.h:1124: void trx_t::assert_freed() const: Assertion `!lock.wait_lock' failed.

RR trace is present on SDP :-
/data/results/1778871863/MDEV-28730_05

This is not reproducible on latest commit

mariadb-SaahilAlam · 2026-05-19T05:26:19Z

+{
+  dict_table_t *table= m_aux_tables[aux_index];
+  if (table == nullptr) return DB_TABLE_NOT_FOUND;
+  dberr_t err= m_executor.lock_table(table, mode);


Assertion found on origin/main-MDEV-28730 29c838c

`# 2026-05-18T12:58:16 [3307300] | mariadbd: /data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc:985: void trx_start_low(trx_t*, bool): Assertion `!read_write' failed.` # 2026-05-18T12:59:09 [3307300] | #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44 # 2026-05-18T12:59:09 [3307300] | #1 __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78 # 2026-05-18T12:59:09 [3307300] | #2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 # 2026-05-18T12:59:09 [3307300] | #3 0x00007deb72e4526e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 # 2026-05-18T12:59:09 [3307300] | #4 0x00007deb72e288ff in __GI_abort () at ./stdlib/abort.c:79 # 2026-05-18T12:59:09 [3307300] | #5 0x00007deb72e2881b in __assert_fail_base (fmt=0x7deb72fd01e8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x6464bf127506 "!read_write", file=file@entry=0x6464bf07ab10 "/data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc", line=line@entry=985, function=function@entry=0x6464bf07ad38 "void trx_start_low(trx_t*, bool)") at ./assert/assert.c:94 # 2026-05-18T12:59:09 [3307300] | #6 0x00007deb72e3b507 in __assert_fail (assertion=0x6464bf127506 "!read_write", file=0x6464bf07ab10 "/data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc", line=985, function=0x6464bf07ad38 "void trx_start_low(trx_t*, bool)") at ./assert/assert.c:103 # 2026-05-18T12:59:09 [3307300] | #7 0x00006464bec810a7 in trx_start_low (trx=0x7deb70402e80, read_write=read_write@entry=true) at /data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc:985 # 2026-05-18T12:59:09 [3307300] | #8 0x00006464bec81274 in trx_start_if_not_started_low (trx=<optimized out>, read_write=read_write@entry=true) at /data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc:2216 # 2026-05-18T12:59:09 [3307300] | #9 0x00006464bed9cb2c in QueryExecutor::lock_table (this=this@entry=0x7deb7015b190, table=0x75eb38496c58, mode=mode@entry=LOCK_IS) at /data/Server/MDEV-28730_02/storage/innobase/row/row0query.cc:154 # 2026-05-18T12:59:09 [3307300] | #10 0x00006464bed6ddd8 in FTSQueryExecutor::lock_aux_tables (this=this@entry=0x7deb7015b190, aux_index=aux_index@entry=0 '\000', mode=mode@entry=LOCK_IS) at /data/Server/MDEV-28730_02/storage/innobase/fts/fts0exec.cc:161 # 2026-05-18T12:59:09 [3307300] | #11 0x00006464bed6fca4 in FTSQueryExecutor::read_aux (this=this@entry=0x7deb7015b190, aux_index=aux_index@entry=0 '\000', start_word=start_word@entry=0x7deb7015b100, callback=...) at /data/Server/MDEV-28730_02/storage/innobase/fts/fts0exec.cc:863 # 2026-05-18T12:59:09 [3307300] | #12 0x00006464beb0919a in i_s_fts_index_table_fill_selected (executor=executor@entry=0x7deb7015b190, reader=..., selected=selected@entry=0 '\000', word=word@entry=0x7deb7015b100) at /data/Server/MDEV-28730_02/storage/innobase/handler/i_s.cc:2679 # 2026-05-18T12:59:09 [3307300] | #13 0x00006464beb0ca39 in i_s_fts_read_aux_index_words (executor=executor@entry=0x7deb7015b190, reader=..., selected=selected@entry=0 '\000', thd=thd@entry=0x75eb4400b358, tables=tables@entry=0x75eb44020c18, conv_str=conv_str@entry=0x7deb7015b3a0, heap=0x75eb4404f8a0, words=0x75eb4404f938) at /data/Server/MDEV-28730_02/storage/innobase/handler/i_s.cc:2851 # 2026-05-18T12:59:09 [3307300] | #14 0x00006464beb0e1b9 in i_s_fts_index_table_fill_one_index (index=index@entry=0x75eb3828b638, thd=thd@entry=0x75eb4400b358, conv_str=conv_str@entry=0x7deb7015b3a0, tables=tables@entry=0x75eb44020c18) at /data/Server/MDEV-28730_02/storage/innobase/handler/i_s.cc:2942 # 2026-05-18T12:59:09 [3307300] | #15 0x00006464beb0e34e in i_s_fts_index_table_fill (thd=0x75eb4400b358, tables=0x75eb44020c18) at /data/Server/MDEV-28730_02/storage/innobase/handler/i_s.cc:3000 # 2026-05-18T12:59:09 [3307300] | #16 0x00006464be5a6d14 in get_schema_tables_result (join=join@entry=0x75eb440220b0, executed_place=executed_place@entry=PROCESSED_BY_JOIN_EXEC) at /data/Server/MDEV-28730_02/sql/sql_show.cc:9924 # 2026-05-18T12:59:09 [3307300] | #17 0x00006464be5906d1 in JOIN::exec_inner (this=this@entry=0x75eb440220b0) at /data/Server/MDEV-28730_02/sql/sql_select.cc:5086 # 2026-05-18T12:59:09 [3307300] | #18 0x00006464be5908f1 in JOIN::exec (this=this@entry=0x75eb440220b0) at /data/Server/MDEV-28730_02/sql/sql_select.cc:4913 # 2026-05-18T12:59:09 [3307300] | #19 0x00006464be58f8df in mysql_select (thd=thd@entry=0x75eb4400b358, tables=0x75eb44020c18, fields=..., conds=0x0, og_num=0, order=<optimized out>, group=0x0, having=0x0, proc_param=0x0, select_options=2701396736, result=0x75eb44022088, unit=0x75eb4400f878, select_lex=0x75eb44020498) at /data/Server/MDEV-28730_02/sql/sql_select.cc:5439 # 2026-05-18T12:59:09 [3307300] | #20 0x00006464be58fab4 in handle_select (thd=thd@entry=0x75eb4400b358, lex=lex@entry=0x75eb4400f798, result=result@entry=0x75eb44022088, setup_tables_done_option=setup_tables_done_option@entry=0) at /data/Server/MDEV-28730_02/sql/sql_select.cc:636 # 2026-05-18T12:59:09 [3307300] | #21 0x00006464be50535e in execute_sqlcom_select (thd=thd@entry=0x75eb4400b358, all_tables=0x75eb44020c18) at /data/Server/MDEV-28730_02/sql/sql_parse.cc:6214 # 2026-05-18T12:59:09 [3307300] | #22 0x00006464be50edd3 in mysql_execute_command (thd=thd@entry=0x75eb4400b358, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /data/Server/MDEV-28730_02/sql/sql_parse.cc:3988 # 2026-05-18T12:59:09 [3307300] | #23 0x00006464be514979 in mysql_parse (thd=thd@entry=0x75eb4400b358, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x7deb7015c3c0) at /data/Server/MDEV-28730_02/sql/sql_parse.cc:7942 # 2026-05-18T12:59:09 [3307300] | #24 0x00006464be515fcd in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x75eb4400b358, packet=packet@entry=0x75eb44015c39 "\nSELECT COUNT(*) FROM INFORMATION_SCHEMA.INNODB_FT_INDEX_TABLE\nLIMIT 500 /* E_R Thread2 QNO 23 CON_ID 19 */ ", packet_length=packet_length@entry=108, blocking=blocking@entry=true) at /data/Server/MDEV-28730_02/sql/sql_parse.cc:1898 # 2026-05-18T12:59:09 [3307300] | #25 0x00006464be5175d2 in do_command (thd=thd@entry=0x75eb4400b358, blocking=blocking@entry=true) at /data/Server/MDEV-28730_02/sql/sql_parse.cc:1432 # 2026-05-18T12:59:09 [3307300] | #26 0x00006464be67059d in do_handle_one_connection (connect=<optimized out>, connect@entry=0x6464c2354d68, put_in_cache=put_in_cache@entry=true) at /data/Server/MDEV-28730_02/sql/sql_connect.cc:1503 # 2026-05-18T12:59:09 [3307300] | #27 0x00006464be6707ba in handle_one_connection (arg=0x6464c2354d68) at /data/Server/MDEV-28730_02/sql/sql_connect.cc:1415 # 2026-05-18T12:59:09 [3307300] | #28 0x00007deb72e9ca94 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447 # 2026-05-18T12:59:09 [3307300] | #29 0x00007deb72f29a34 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

RRtrace is present on SDP :-
/data/results/1779134248/MDEV-28730_06

this is not reproducible on latest commit

mariadb-SaahilAlam · 2026-05-19T10:07:37Z

+      if (UNIV_LIKELY(error == DB_SUCCESS ||
+                      error == DB_FTS_EXCEED_RESULT_CACHE_LIMIT))
+      {
+        fts_sql_commit(trx);


Assertion found on release build, please check if its related:-
origin/main-MDEV-28730 29c838cc95b532d26f91e327dc684c8973b218f1

# 2026-05-19T01:01:00 [2370519] | 2026-05-19 01:00:56 0x71385c1ff6c0 InnoDB: Assertion failure in file /data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc line 1427 # 2026-05-19T01:01:00 [2370519] | InnoDB: Failing assertion: UT_LIST_GET_LEN(lock.trx_locks) == 0

# 2026-05-19T01:01:40 [2370519] | #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44 # 2026-05-19T01:01:40 [2370519] | #1 __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78 # 2026-05-19T01:01:40 [2370519] | #2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 # 2026-05-19T01:01:40 [2370519] | #3 0x000071385ee4526e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 # 2026-05-19T01:01:40 [2370519] | #4 0x000071385ee288ff in __GI_abort () at ./stdlib/abort.c:79 # 2026-05-19T01:01:40 [2370519] | #5 0x00005b1ae6645769 in ut_dbg_assertion_failed (expr=expr@entry=0x5b1ae7208c70 "UT_LIST_GET_LEN(lock.trx_locks) == 0", file=file@entry=0x5b1ae7208ac0 "/data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc", line=line@entry=1427) at /data/Server/MDEV-28730_02/storage/innobase/ut/ut0dbg.cc:60 # 2026-05-19T01:01:40 [2370519] | #6 0x00005b1ae6644a9f in trx_t::commit_in_memory (mtr=0x71385c1fcbf0, this=0x71385c202180) at /data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc:1427 # 2026-05-19T01:01:40 [2370519] | #7 trx_t::commit_persist (this=this@entry=0x71385c202180) at /data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc:1622 # 2026-05-19T01:01:40 [2370519] | #8 0x00005b1ae6e5ce7f in trx_t::commit (this=0x71385c202180) at /data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc:1631 # 2026-05-19T01:01:40 [2370519] | #9 trx_commit_for_mysql (trx=trx@entry=0x71385c202180) at /data/Server/MDEV-28730_02/storage/innobase/trx/trx0trx.cc:1746 # 2026-05-19T01:01:40 [2370519] | #10 0x00005b1ae6db264d in i_s_fts_read_aux_index_words (words=0x69382c03f8d0, heap=<optimized out>, conv_str=<optimized out>, tables=<optimized out>, thd=<optimized out>, selected=0 '\000', reader=..., executor=0x71385c1fd0b0) at /data/Server/MDEV-28730_02/storage/innobase/handler/i_s.cc:2857 # 2026-05-19T01:01:40 [2370519] | #11 i_s_fts_index_table_fill_one_index (index=index@entry=0x6938201381e0, thd=thd@entry=0x69382c000fd8, conv_str=conv_str@entry=0x71385c1fd310, tables=tables@entry=0x69382c014198) at /data/Server/MDEV-28730_02/storage/innobase/handler/i_s.cc:2942 # 2026-05-19T01:01:40 [2370519] | #12 0x00005b1ae6db2d64 in i_s_fts_index_table_fill (thd=0x69382c000fd8, tables=0x69382c014198) at /data/Server/MDEV-28730_02/storage/innobase/handler/i_s.cc:3000 # 2026-05-19T01:01:40 [2370519] | #13 0x00005b1ae6910421 in get_schema_tables_result (join=join@entry=0x69382c016010, executed_place=executed_place@entry=PROCESSED_BY_JOIN_EXEC) at /data/Server/MDEV-28730_02/sql/sql_show.cc:9924 # 2026-05-19T01:01:40 [2370519] | #14 0x00005b1ae68f120d in JOIN::exec_inner (this=this@entry=0x69382c016010) at /data/Server/MDEV-28730_02/sql/sql_select.cc:5086 # 2026-05-19T01:01:40 [2370519] | #15 0x00005b1ae68f1be2 in JOIN::exec (this=this@entry=0x69382c016010) at /data/Server/MDEV-28730_02/sql/sql_select.cc:4913 # 2026-05-19T01:01:40 [2370519] | #16 0x00005b1ae68f0155 in mysql_select (thd=thd@entry=0x69382c000fd8, tables=0x69382c014198, fields=..., conds=0x69382c014f28, og_num=1, order=<optimized out>, group=0x0, having=0x0, proc_param=0x0, select_options=2701396736, result=0x69382c015fe8, unit=0x69382c005320, select_lex=0x69382c013808) at /data/Server/MDEV-28730_02/sql/sql_select.cc:5439 # 2026-05-19T01:01:40 [2370519] | #17 0x00005b1ae68f0837 in handle_select (thd=thd@entry=0x69382c000fd8, lex=lex@entry=0x69382c005240, result=result@entry=0x69382c015fe8, setup_tables_done_option=setup_tables_done_option@entry=0) at /data/Server/MDEV-28730_02/sql/sql_select.cc:636 # 2026-05-19T01:01:40 [2370519] | #18 0x00005b1ae6866f34 in execute_sqlcom_select (thd=thd@entry=0x69382c000fd8, all_tables=0x69382c014198) at /data/Server/MDEV-28730_02/sql/sql_parse.cc:6214 # 2026-05-19T01:01:40 [2370519] | #19 0x00005b1ae68749d6 in mysql_execute_command (thd=thd@entry=0x69382c000fd8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /data/Server/MDEV-28730_02/sql/sql_parse.cc:3988 # 2026-05-19T01:01:40 [2370519] | #20 0x00005b1ae6876083 in mysql_parse (thd=0x69382c000fd8, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /data/Server/MDEV-28730_02/sql/sql_parse.cc:7942 # 2026-05-19T01:01:40 [2370519] | #21 0x00005b1ae6878362 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x69382c000fd8, packet=packet@entry=0x69382c008f09 "\nSELECT word, doc_count, doc_id\nFROM INFORMATION_SCHEMA.INNODB_FT_INDEX_TABLE\nWHERE word LIKE CONCAT( SUBSTRING( 'tell', 1, 2 ), '%' )\nORDER BY word LIMIT 100 /* E_R Thread2 QNO 9 CON_ID 19 */ ", packet_length=packet_length@entry=193, blocking=blocking@entry=true) at /data/Server/MDEV-28730_02/sql/sql_parse.cc:1997 # 2026-05-19T01:01:40 [2370519] | #22 0x00005b1ae6879a96 in do_command (thd=thd@entry=0x69382c000fd8, blocking=blocking@entry=true) at /data/Server/MDEV-28730_02/sql/sql_parse.cc:1432 # 2026-05-19T01:01:40 [2370519] | #23 0x00005b1ae699f08d in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /data/Server/MDEV-28730_02/sql/sql_connect.cc:1503 # 2026-05-19T01:01:40 [2370519] | #24 0x00005b1ae699f385 in handle_one_connection (arg=0x5b1ae8ad1d48) at /data/Server/MDEV-28730_02/sql/sql_connect.cc:1415 # 2026-05-19T01:01:40 [2370519] | #25 0x000071385ee9ca94 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447 # 2026-05-19T01:01:40 [2370519] | #26 0x000071385ef29a34 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

RR trace is present on SDP:-
/data/results/1779177603/MDEV-28730_07

This is not reproducible on latest commit

mariadb-SaahilAlam · 2026-05-20T11:08:56Z

+  else m_thr->lock_state= QUE_THR_LOCK_ROW;
+  if (m_trx->lock.wait_thr)
+  {
+    dberr_t wait_err= lock_wait(m_thr);


Server crash found here on debug build:-
origin/main-MDEV-28730 ebbcbd200a4a62ad3dbfd132a83ef927a0cab648

Thread 2 received signal SIGSEGV, Segmentation fault. [Switching to Thread 3946709.3954078] thd_storage_lock_wait (thd=0x0, value=12) at /data/Server/MDEV-28730C/sql/sql_class.cc:451 451 thd->utime_after_lock+= value; (rr) set print addr off (rr) bt #0 thd_storage_lock_wait (thd=, value=12) at /data/Server/MDEV-28730C/sql/sql_class.cc:451 #1 lock_sys_t::wait_resume (this=this@entry=<lock_sys>, thd=, start=start@entry=..., now=...) at /data/Server/MDEV-28730C/storage/innobase/lock/lock0lock.cc:2108 #2 lock_wait (thr=<optimized out>) at /data/Server/MDEV-28730C/storage/innobase/lock/lock0lock.cc:2410 #3 QueryExecutor::handle_wait (this=this@entry=, err=DB_LOCK_WAIT, table_lock=table_lock@entry=false) at /data/Server/MDEV-28730C/storage/innobase/row/row0query.cc:189 #4 QueryExecutor::insert_record (this=this@entry=, table=table@entry=, tuple=tuple@entry=) at /data/Server/MDEV-28730C/storage/innobase/row/row0query.cc:155 #5 FTSQueryExecutor::insert_aux_record (this=this@entry=, aux_index=aux_index@entry=2 '\002', aux_data=aux_data@entry=) at /data/Server/MDEV-28730C/storage/innobase/fts/fts0exec.cc:235 #6 fts_write_node (executor=executor@entry=, selected=selected@entry=2 '\002', aux_data=aux_data@entry=) at /data/Server/MDEV-28730C/storage/innobase/fts/fts0fts.cc:3278 #7 fts_sync_write_words (executor=executor@entry=, index_cache=index_cache@entry=, unlock_cache=true) at /data/Server/MDEV-28730C/storage/innobase/fts/fts0fts.cc:3348 #8 fts_sync_index (executor=executor@entry=, sync=sync@entry=, index_cache=index_cache@entry=) at /data/Server/MDEV-28730C/storage/innobase/fts/fts0fts.cc:3399 #9 fts_sync (sync=, unlock_cache=<optimized out>, wait=wait@entry=true) at /data/Server/MDEV-28730C/storage/innobase/fts/fts0fts.cc:3631 #10 fts_sync_table (table=, wait=wait@entry=true) at /data/Server/MDEV-28730C/storage/innobase/fts/fts0fts.cc:3704 #11 ha_innobase::optimize (this=, thd=) at /data/Server/MDEV-28730C/storage/innobase/handler/ha_innodb.cc:15326 #12 handler::ha_optimize (this=, thd=thd@entry=, check_opt=check_opt@entry=) at /data/Server/MDEV-28730C/sql/handler.cc:5884 #13 mysql_admin_table (thd=thd@entry=, tables=tables@entry=, check_opt=check_opt@entry=, operator_name=operator_name@entry=<msg_optimize>, lock_type=lock_type@entry=TL_WRITE, org_open_for_modify=org_open_for_modify@entry=true, no_errors_from_open=false, extra_open_options=0, prepare_func=, operator_func=<optimized out>, view_operator_func=, is_cmd_replicated=true) at /data/Server/MDEV-28730C/sql/sql_admin.cc:949 #14 Sql_cmd_optimize_table::execute (this=<optimized out>, thd=) at /data/Server/MDEV-28730C/sql/sql_admin.cc:1730 #15 mysql_execute_command (thd=thd@entry=, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /data/Server/MDEV-28730C/sql/sql_parse.cc:5902 #16 mysql_parse (thd=thd@entry=, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=) at /data/Server/MDEV-28730C/sql/sql_parse.cc:7942 #17 dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=, packet=packet@entry="", packet_length=packet_length@entry=83, blocking=blocking@entry=true) at /data/Server/MDEV-28730C/sql/sql_parse.cc:1898 #18 do_command (thd=thd@entry=, blocking=blocking@entry=true) at /data/Server/MDEV-28730C/sql/sql_parse.cc:1432 #19 do_handle_one_connection (connect=<optimized out>, connect@entry=, put_in_cache=put_in_cache@entry=true) at /data/Server/MDEV-28730C/sql/sql_connect.cc:1503 #20 handle_one_connection (arg=) at /data/Server/MDEV-28730C/sql/sql_connect.cc:1415 #21 start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447 #22 clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

RR trace is present on SDP:-
/data/results/1779260294/MDEV-28730_08

This is not reproducible on latest commit

mariadb-SaahilAlam · 2026-05-21T09:55:04Z

+    if (success)
+      trx_commit_for_mysql(trx);
+    else trx->rollback();
+    trx->free();


Assertion found here:-
MDEV-28730 CS 13.0.1 ebbcbd200a4a62ad3dbfd132a83ef927a0cab648 (Debug, Clang 18.1.3-11)

SET @@GLOBAL.INNODB_flush_log_at_trx_commit=0; CREATE TABLE t1 (c1 INT KEY,c2 VARCHAR(1),c3 TEXT,FULLTEXT KEY ftidx_9244(c2),FULLTEXT KEY ftidx_8094(c3)) ENGINE=INNODB DEFAULT CHARSET=utf8mb4;

mariadbd: /test/main-MDEV-28730_dbg/storage/innobase/trx/trx0trx.cc:397: void trx_t::free(): Assertion !commit_lsn' failed.
`

iMineLink

As this PR is rather large, I'm still not done with the review but wanted to post a few pending comments anyway before finishing the review.
In general, I see that reusing row_search_mvcc() is a really clever way to prevent most of the headaches around these changes, especially when secondary indices are involved. Since it seems that testing was already done on this revision, please feel free to discard minor comments such as those about doc_count, thank you.

iMineLink

I finished my review, all I have here are minor comments. I believe the only non-minor ones were the one (posted in previous review on the same commit) about default charset initialization to latin1 and table name construction without holding the dict_sys latch: please check those at least.
It's a great work, and reusing row_search_mvcc() is I think the best approach. Thanks!

InnoDB FTS performed all reads and DML on its auxiliary, common and CONFIG tables through the InnoDB internal SQL graph parser. Replace that path with direct B-tree access via a new query-executor layer, and delete the parser-era helpers. row0query.h, row/row0query.cc - new QueryExecutor: - General MVCC-aware record traversal and basic DML on the clustered index. Sits on a btr_pcur and a transaction-owned mtr; record processing goes through a RecordCallback that bundles two std::functions: compare_record() returns SKIP / PROCESS / STOP for a record process_record() handles each PROCESS-ed (MVCC-visible) row Public API: read() scan a clustered index with a search key read_all() full clustered scan (optional start tuple) read_by_index() scan a secondary index, fetch the matching clustered record, deliver it to the callback insert_record() insert a tuple into the clustered index delete_record() delete a row identified by tuple delete_all() delete every row in the clustered index select_for_update() position+X-lock the matching clustered row update_record() update the row select_for_update() locked, falling back to optimistic/pessimistic and external storage paths as needed replace_record() upsert: select_for_update()+update_record(), else insert_record() lock_table(), handle_wait(), commit_mtr() fts0exec.h, fts/fts0exec.cc - new FTSQueryExecutor: Thin wrapper over QueryExecutor specialised for FTS tables. Opens and locks the required tables once and exposes typed helpers keyed by table family. Auxiliary INDEX_[1..6]: open_all_aux_tables() insert_aux_record(aux_index, fts_aux_data_t) delete_aux_record(aux_index, fts_aux_data_t) read_aux() range scan from a given word read_from_range() paginated read that absorbs DB_FTS_EXCEED_RESULT_CACHE_LIMIT internally and resumes from the last word seen Common deletion tables (DELETED, DELETED_CACHE, BEING_DELETED, BEING_DELETED_CACHE): open_all_deletion_tables() insert_common_record(), delete_common_record(), delete_all_common_records(), read_all_common() CONFIG table (<key, value>): open_config_table() / set_config_table() insert_config_record(), update_config_record() (upsert), delete_config_record(), read_config_with_lock() fts_aux_data_t carries the auxiliary row payload. RecordCallback specialisations live alongside the executor: CommonTableReader collects doc_ids from common tables that share the <doc_id> schema. ConfigReader extracts <key, value> and provides compare_config_key() for fast key matching. AuxRecordReader scans auxiliary indexes with an AuxCompareMode (GREATER_EQUAL / GREATER / LIKE / EQUAL) driving the comparator; tracks the last word seen so a paginated scan can resume. fts_query() walks index and common tables via QueryExecutor::read_by_index() with RecordCallback; fts_write_node() writes auxiliary rows through FTSQueryExecutor::insert_aux_record() / delete_aux_record() with fts_aux_data_t. fts_optimize_write_word() now goes through the same insert/delete path. fts_select_index{,_by_range,_by_hash} return uint8_t (was ulint) with a simpler control flow. fts_optimize_table() binds a thd to its transaction whether invoked from a user thread or the FTS optimize thread. fts_optimize_t drops its fts_index_table and fts_common_table fts_table_t fields; fts_query_t drops fts_common_table. storage/innobase/fts/fts0sql.cc is deleted along with the commented-out and unreferenced parser-era helpers it held. dict_sys.latch is now acquired once per fts_sync_table(), fts_optimize_table() and fts_query() call to open every auxiliary and common table in one pass, instead of being re-acquired per table. Every FTS trx_create() site now takes the THD from a real caller (user trx, fts_opt_thd, DDL ctx->trx, ha_thd()) instead of either leaving it NULL or pulling current_thd. fts_commit(): Each fts_commit_table() used to create transaction and commit, producing N undo segments, N read views and N group-commit batches for N FTS tables touched by the user statement. fts_commit() now creates one internal trx, runs every fts_commit_table() under it, and commits once. fts_optimize_word() could fail when source nodes have equal boundary doc_ids. Loosen the merge predicate from '>' to '>=' so legitimate overlapping ranges that occur when nodes fill to FTS_ILIST_MAX_SIZE at doc_id boundaries are handled correctly. INFORMATION_SCHEMA.INNODB_FT_INDEX_TABLE (i_s.cc) wires the shared words_heap via set_words_heap() and empties it per batch with mem_heap_empty(); the per-fetch cleanup frees only the per-node ilist (ut_alloc'd) and skips fts_word_free().

iMineLink

Thanks for addressing my previous review points. LGTM now.

Copilot

Pull request overview

This PR refactors InnoDB Fulltext Search (FTS) execution paths to remove SQL-string parsing / internal SQL-graph execution and instead drive scans and DML through new C++ executors built on row_search_mvcc() + callback interfaces.

Changes:

Introduces QueryExecutor (row0query.{h,cc}) and FTSQueryExecutor (fts0exec.{h,cc}) abstractions for MVCC-aware reads, DML, and FTS auxiliary/common/config table operations.
Refactors row_search_mvcc() to support policy-based execution (MySQL row conversion vs. InnoDB callback; internal vs. external mtr) and updates multiple FTS call sites to use the new executors.
Updates FTS doc-id/CONFIG/sync/stopword flows to pass THD* for correct lock-wait behavior and adjusts related MTR tests.

Reviewed changes

Copilot reviewed 36 out of 36 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
storage/innobase/row/row0sel.cc	Refactors clustered-record lookup and templates `row_search_mvcc()` for policy-based callback + mtr handling.
storage/innobase/row/row0query.cc	Adds `QueryExecutor` implementation for read/lock/DML helpers built on `row_search_mvcc()`.
storage/innobase/row/row0mysql.cc	Propagates FTS doc-id allocation errors and threads `THD*` into doc-id init paths.
storage/innobase/row/row0merge.cc	Threads `THD*` through FTS doc-id allocation/sync during merge; uses `FTSQueryExecutor` to update synced doc id.
storage/innobase/row/row0ins.cc	Threads `THD*` through FTS doc-id allocation/update during cascade updates.
storage/innobase/row/row0ftsort.cc	Ensures merge thread trx inherits user `THD*` for lock-wait policy consistency.
storage/innobase/page/page0cur.cc	Exposes `cmp_dtuple_rec_bytes()` (no longer `static`) for reuse.
storage/innobase/lock/lock0lock.cc	Skips per-THD wait accounting when `THD*` is null; updates dict MDL helper usage.
storage/innobase/include/ut0new.h	Registers `fts0exec` in auto-event names list.
storage/innobase/include/row0sel.h	Adds policy aliases and callback wrappers for the new `row_search_mvcc()` template use cases.
storage/innobase/include/row0query.h	Adds `RecordCallback` API and `QueryExecutor` declaration.
storage/innobase/include/row0mysql.h	Adds `row_prebuilt_t::mtr` for external mini-transaction support in DML flows.
storage/innobase/include/page0cur.h	Declares the now-exported `cmp_dtuple_rec_bytes()`.
storage/innobase/include/fts0types.inl	Tightens FTS aux-index selector return type to `uint8_t`.
storage/innobase/include/fts0types.h	Tightens `fts_node_t::doc_count` to `uint32_t` and updates selector return type.
storage/innobase/include/fts0priv.h	Removes SQL-parser helper declarations; switches FTS internals to executor-based APIs.
storage/innobase/include/fts0fts.h	Updates FTS public/internal APIs to accept `THD*` and executor-based helpers; adds new function declarations.
storage/innobase/include/fts0exec.h	Adds `FTSQueryExecutor`, callback readers, and aux/common/config table helpers.
storage/innobase/include/dict0dict.h	Generalizes MDL acquisition helper from “shared-only” to configurable MDL mode.
storage/innobase/include/btr0cur.h	Refines `nonnull` attribute indices for `btr_cur_del_mark_set_clust_rec()`.
storage/innobase/handler/i_s.cc	Reimplements INFORMATION_SCHEMA FTS table readers using `FTSQueryExecutor` + callbacks.
storage/innobase/handler/handler0alter.cc	Threads `THD*` into FTS DDL sync operations.
storage/innobase/handler/ha_innodb.cc	Refactors stopword loading/config update via executor and threads `THD*` into optimize/sync.
storage/innobase/fts/fts0que.cc	Rewrites on-disk FTS query execution paths to use executor + callbacks (no SQL graph eval).
storage/innobase/fts/fts0exec.cc	Implements `FTSQueryExecutor` and callback readers for aux/common/config table DML/reads.
storage/innobase/fts/fts0config.cc	Migrates CONFIG get/set/ulint-set to executor + callback implementation.
storage/innobase/dict/dict0dict.cc	Implements generalized `dict_acquire_mdl` template and updates call sites/instantiations.
storage/innobase/CMakeLists.txt	Adds new executor sources/headers and removes `fts0sql.cc` from build.
mysql-test/suite/innodb_fts/t/sync.test	Stabilizes MATCH query output ordering with `ORDER BY FTS_DOC_ID`.
mysql-test/suite/innodb_fts/t/index_table.test	Updates debug injection to `fts_load_stopword_fail`.
mysql-test/suite/innodb_fts/t/fts_kill_query.test	Adds suppression for expected interrupted-operation error log line.
mysql-test/suite/innodb_fts/r/sync.result	Updates expected output ordering to match new `ORDER BY` clause.
mysql-test/suite/innodb_fts/r/index_table.result	Updates expected debug injection line to match new test.
mysql-test/suite/innodb_fts/r/fts_kill_query.result	Updates expected suppression call line.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+	    || btr_pcur_get_low_match(clust_pcur)
 	    < dict_index_get_n_unique(clust_index)) {
-		btr_cur_t*	btr_cur = btr_pcur_get_btr_cur(prebuilt->pcur);
+		btr_cur_t*	btr_cur = btr_pcur_get_btr_cur(clust_pcur);


 				 - btr_cur->rtr_info->matches->block->page.frame)
 			   >> srv_page_size_shift)
-			|| rec != btr_pcur_get_rec(prebuilt->pcur))) {
+			|| rec != btr_pcur_get_rec(clust_pcur))) {


+			if (rec != btr_pcur_get_rec(clust_pcur)) {
 				clust_rec = NULL;
                                goto func_exit;
 			}


+    RecordCompareAction action= callback->compare_record(
+      prebuilt->search_tuple, rec, index, offsets);
+


+  ut_ad(m_trx);
+  ut_ad(table);
+  ut_ad(index);
+  ut_ad(callback.compare_record);


+  ut_ad(m_trx);
+  ut_ad(table);
+  ut_ad(callback.compare_record);


+bool CommonTableReader::extract_common_fields(
+  const rec_t *rec, const dict_index_t *index, doc_id_t *doc_id)
+{
+  if (!dict_table_is_comp(index->table))
+  {
+    ulint doc_id_len;
+    ulint offset= rec_get_nth_field_offs_old(rec, 0, &doc_id_len);
+    if (offset != 0 || doc_id_len == UNIV_SQL_NULL || doc_id_len != 8)
+      return false;
+  }
+
+  *doc_id= mach_read_from_8(rec);
+  return true;
+}


+  if (ilist_data && ilist_len != UNIV_SQL_NULL && ilist_len > 0)
+  {
+    node->ilist_size_alloc= node->ilist_size= ilist_len;
+    if (ilist_len)
+    {
+      node->ilist= static_cast<byte*>(ut_malloc_nokey(ilist_len));
+      memcpy(node->ilist, ilist_data, ilist_len);
+    }
+    if (ilist_len == 0) return DB_SUCCESS_LOCKED_REC;
+  }


+  dberr_t read_all(dict_table_t *table, RecordCallback &callback,
+                   const dtuple_t *start_tuple= nullptr) noexcept;


Thirunarayanan requested a review from dr-m November 17, 2025 07:42

Thirunarayanan added the MariaDB Corporation label Nov 17, 2025

Thirunarayanan force-pushed the main-MDEV-28730 branch 3 times, most recently from 04ec1e9 to ff6a64d Compare November 21, 2025 06:13

dr-m mentioned this pull request Nov 27, 2025

MDEV-28730 Remove internal parser usage from InnoDB fulltext #4036

Closed

5 tasks

dr-m reviewed Nov 27, 2025

View reviewed changes

Thirunarayanan force-pushed the main-MDEV-28730 branch from b672350 to 53f237a Compare December 4, 2025 08:40

Thirunarayanan requested a review from dr-m December 4, 2025 08:42

Thirunarayanan force-pushed the main-MDEV-28730 branch from 53f237a to edabb01 Compare December 4, 2025 11:29

dr-m reviewed Dec 4, 2025

View reviewed changes

Thirunarayanan force-pushed the main-MDEV-28730 branch 4 times, most recently from 9f34b2c to 166235e Compare January 8, 2026 14:14

Thirunarayanan requested a review from dr-m January 13, 2026 14:59

dr-m reviewed Jan 22, 2026

View reviewed changes

Comment thread storage/innobase/fts/fts0fts.cc Outdated

dr-m reviewed Jan 22, 2026

View reviewed changes

Comment thread storage/innobase/fts/fts0fts.cc Outdated

dr-m reviewed Jan 22, 2026

View reviewed changes

Thirunarayanan force-pushed the main-MDEV-28730 branch from 166235e to 8355e6a Compare February 12, 2026 13:02

Thirunarayanan requested a review from dr-m February 12, 2026 13:02

dr-m reviewed Feb 23, 2026

View reviewed changes

Thirunarayanan force-pushed the main-MDEV-28730 branch from 8355e6a to 734db1e Compare May 14, 2026 11:33

mariadb-SaahilAlam reviewed May 15, 2026

View reviewed changes

Thirunarayanan force-pushed the main-MDEV-28730 branch from 734db1e to 7b5e60b Compare May 15, 2026 09:36

mariadb-SaahilAlam reviewed May 15, 2026

View reviewed changes

Thirunarayanan force-pushed the main-MDEV-28730 branch from 7b5e60b to a966595 Compare May 15, 2026 13:01

mariadb-SaahilAlam reviewed May 18, 2026

View reviewed changes

mariadb-SaahilAlam reviewed May 19, 2026

View reviewed changes

Thirunarayanan force-pushed the main-MDEV-28730 branch 2 times, most recently from 5506e5b to cdb604e Compare May 20, 2026 05:42

mariadb-SaahilAlam reviewed May 20, 2026

View reviewed changes

mariadb-SaahilAlam reviewed May 21, 2026

View reviewed changes

Thirunarayanan force-pushed the main-MDEV-28730 branch 2 times, most recently from e5d72aa to a5af47f Compare June 12, 2026 06:04

Thirunarayanan requested a review from iMineLink June 12, 2026 08:28

iMineLink reviewed Jun 19, 2026

View reviewed changes

iMineLink reviewed Jun 22, 2026

View reviewed changes

Thirunarayanan force-pushed the main-MDEV-28730 branch from a5af47f to 45006b2 Compare July 20, 2026 07:57

iMineLink approved these changes Jul 20, 2026

View reviewed changes

iMineLink requested a review from Copilot July 21, 2026 07:36

Copilot started reviewing on behalf of iMineLink July 21, 2026 07:36 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

		if (!rec_get_deleted_flag(result_rec,
		clust_index->table->not_redundant()))

		RecordCompareAction action= callback->compare_record(
		prebuilt->search_tuple, rec, index, offsets);

		dberr_t read_all(dict_table_t *table, RecordCallback &callback,
		const dtuple_t *start_tuple= nullptr) noexcept;

Uh oh!

Uh oh!

Conversation

Thirunarayanan commented Nov 17, 2025

Description

Release Notes

How can this PR be tested?

Basing the PR against the correct MariaDB version

PR quality check

Uh oh!

CLAassistant commented Nov 17, 2025

Uh oh!

grooverdan commented Nov 18, 2025

Uh oh!

dr-m left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dr-m left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dr-m left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Thirunarayanan Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dr-m left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Thirunarayanan Feb 12, 2026 •

edited

Loading