Port PendingRow to arrow-rs

[emilk]

• Part of Migrate from re_arrow2 to arrow #3741
• Requires More conversions to and from arrow-rs re_arrow2#15

NOTE: this will regress on memory use, because we lack .shrink_to_fit, which will arrive in

• Update arrow, pyo3, and numpy #8618

[github-actions]

Web viewer built successfully. If applicable, you should also test it:

• I have tested the web viewer

Result	Commit	Link	Manifest
✅	21d76fd	https://rerun.io/viewer/pr/8617	+nightly +main

^{Note: This comment is updated whenever you push a commit.}

[emilk]

I took the freedom to simplify this function quite a bit from the original:

rerun/crates/store/re_chunk/src/util.rs

Lines 22 to 37 in 71215db

	pub fn is_list_array_semantically_empty(list_array: &ArrowListArray<i32>) -> bool {
	let is_physically_empty = || list_array.is_empty();
	let is_all_nulls = || {
	list_array
	.validity()
	.map_or(false, |bitmap| bitmap.unset_bits() == list_array.len())
	};
	let is_all_empties = || list_array.offsets().lengths().all(|len| len == 0);
	let is_a_mix_of_nulls_and_empties =
	|| list_array.iter().flatten().all(|array| array.is_empty());
	is_physically_empty() || is_all_nulls() || is_all_empties() || is_a_mix_of_nulls_and_empties()
	}

Was the old complexity necessary for some reason I don't see, or an oversight?

[teh-cmc]

All I can say is that this logic was very important for the UX of the dataframe views and APIs.

Whether the two implementations are equivalent, I'm tempted to say yes but I might be missing something too.

[emilk]

This whole file is a port of https://github.com/rerun-io/rerun/blob/main/crates/store/re_chunk

[emilk]

• Requires Update arrow, pyo3, and numpy #8618

…but maybe it's fine to regress on this temporarily

[emilk]

@rerun-bot full-check

[github-actions]

Started a full build: https://github.com/rerun-io/rerun/actions/runs/12687112287

[teh-cmc]

Gave it a very shallow look but it all seems fairly mechanical so there shouldn't too many surprises.

A full-check will definitely tell us if something went wrong in any case.

[teh-cmc]

All I can say is that this logic was very important for the UX of the dataframe views and APIs.

Whether the two implementations are equivalent, I'm tempted to say yes but I might be missing something too.

[emilk]

@rerun-bot full-check

[github-actions]

Started a full build: https://github.com/rerun-io/rerun/actions/runs/12687313927

[emilk]

Deploying the Python docs failed O.o