perf(arrow-cast): gate Dictionary -> View fast path on cardinality

[Abhisheklearn12]

Which issue does this PR close?

• Closes Fast path for Dictionary -> View cast for large types & cross cast #8985

Supersedes #9768, which I closed it implemented these fast paths ungated, and benchmarking showed that made things significantly slower. Details below.

Rationale for this change

dictionary_cast can build a view array from a dictionary two ways:

• (a) unpack_dictionary builds one view per dictionary value, then gathers with take
• (b) view_from_dict_values builds one view per row directly against the values buffer

The comment above (b) says (a) "incurs unnecessary data copy of the value buffer". That
isn't the case. impl From<&GenericByteArray> for GenericByteViewArray reuses the buffer
(it only falls back to copying when the values exceed the u32 offset a view can hold),
and take_byte_view passes data_buffers().to_vec() through untouched. Neither step
copies value data, so there was no copy to remove.

What the two actually trade is a bandwidth-bound u128 gather in take against a
per-row make_view that reads each row's payload out of the values buffer. The gather
wins once rows reach roughly 0.6x the dictionary size so (b) is a large pessimisation in
the common case, and only pays off when the dictionary is substantially larger than the
array is long, where (a) spends most of its time building views no row references.

This means the existing Utf8 -> Utf8View and Binary -> BinaryView fast paths on main
are currently 5-8x slower than unpack_dictionary at typical batch shapes.

What changes are included in this PR?

• Gate the direct path on keys.len() < values.len() / 2; everything else falls through
  to unpack_dictionary. The measured crossover is near rows ≈ 0.6 * values; the gate
  switches short of it, at rows = values / 2, where the direct path is still ahead by
  15-30%. That margin covers the crossover moving with cache size or microarchitecture.
  Being wrong in that direction only forgoes a win; being wrong the other way is a
  regression on the common case.
• Extend the direct path to the remaining combinations Fast path for Dictionary -> View cast for large types & cross cast #8985 asks for:
  LargeUtf8/LargeBinary -> Utf8View/BinaryView, and the Utf8 <-> Binary cross
  casts with the offset-fit check and UTF-8 validation the issue calls out.
• Fix view_from_dict_values dropping null dictionary values: they became empty strings
  rather than nulls, where unpack_dictionary and impl From<&GenericByteArray> both
  produce nulls.
• Bounds-check the dictionary key before indexing the offsets, turning an out-of-range key
  from undefined behaviour into an error. The gate confines this loop to short row counts,
  so it is within noise when the dictionary holds no nulls. When the dictionary does hold
  nulls, the null check costs 25-40% on the direct path an unavoidable random bitmap
  lookup per row, and the price of emitting nulls rather than the empty strings the
  previous code produced.

Benchmarks

Existing fast paths on main:

cast	rows	dict values	main	this PR	change
Utf8->Utf8View	8,192	100	26.3 µs	3.4 µs	-87.2%
Utf8->Utf8View	1,000,000	1,000	3188.9 µs	554.6 µs	-82.6%
Binary->BinaryView	8,192	100	24.9 µs	3.3 µs	-86.6%
Binary->BinaryView	1,000,000	1,000	3296.0 µs	593.6 µs	-82.0%

New arms, sparse shapes (dictionary larger than the array):

cast	rows	dict values	main	this PR	change
LargeUtf8->Utf8View	1,000	100,000	271.2 µs	3.9 µs	-98.6%
LargeUtf8->BinaryView	10,000	1,000,000	15752.5 µs	78.6 µs	-99.5%
LargeBinary->BinaryView	1,000	100,000	276.1 µs	3.9 µs	-98.6%
Utf8->BinaryView	10,000	1,000,000	3005.8 µs	73.1 µs	-97.6%
Binary->Utf8View	1,000	100,000	351.5 µs	88.6 µs	-74.8%
LargeBinary->Utf8View	10,000	1,000,000	4570.7 µs	1713.9 µs	-62.5%

(Binary/LargeBinary -> Utf8View gain less because UTF-8 validation of the dictionary
values dominates.)

Dense shapes on the new arms are unchanged, -2.9% to +1.2% the gate routes them to
unpack_dictionary. Worst cell across the whole matrix is +5.0%, on a shape where both
versions take the direct path and the delta is the null and bounds checks above.

Method

Both implementations were compiled into a single binary and timed in alternating rounds so
thermal drift cancels out of the ratio. Cells whose code is identical in both versions come
out at -2.9% to +1.2%, which bounds the method's noise; a two-run criterion comparison of
those same cells reported up to +47% drift, which is why it wasn't used. Outputs are
asserted equal per cell before timing. Measured on an i7-11700F; a second independent run
reproduced every headline result within 3.2 points and produced the identical set of cells
above 50%; the largest shift on any cell was 8.3 points, on a cell that is flat in both runs.

Are these changes tested?

Yes. Every arm is exercised through both implementations (row counts either side of the
gate), with results asserted equal to the unpack_dictionary reference in each case. Null
dictionary values, null keys, and invalid UTF-8 under both safe and strict CastOptions
are covered by dedicated tests.

Are there any user-facing changes?

Casting Dictionary<_, Utf8> -> Utf8View and Dictionary<_, Binary> -> BinaryView becomes
substantially faster at typical batch shapes 5-8x on the shapes benchmarked above.

Dictionary<_, LargeUtf8/LargeBinary> -> Utf8View/BinaryView and the Utf8 <-> Binary
cross casts gain a fast path when the dictionary holds more than twice as many values as the
array has rows. Outside that they behave as before, going through unpack_dictionary.

Two behaviour changes, called out explicitly:

1. A null dictionary value now casts to null rather than an empty string:

// values ["aa", NULL, "cc"], keys [0, 1, 2]
cast(&dict, &DataType::Utf8View)   // before: ["aa", "",   "cc"]
                                   // after:  ["aa", null, "cc"]

2. An out-of-range dictionary key now returns an InvalidArgumentError instead of being
   undefined behaviour. Only reachable for a dictionary built without validation.

Happy to split either into its own PR if you'd prefer this one stay purely about the fast path.