feat(pageserver): enable reldirv2 by default in regress tests

[skyzh]

Problem

Part of LKB-197. This patch enables reldirv2 by default.

Using index_part to persist reldirv2 status is not reliable. In the current ingestion path when we initialize the relv2 keyspace, we will immediately set index_part to persist that we have enabled v2. However, if the pageserver is shut down or the tenant gets dropped for whatever reason, the disk persistent LSN might not catch up with the LSN where we enabled the relv2. Therefore, we will end up with a state that we have "v2 enabled" in index_part, but the v2 migration keys are not in the layer files.

This patch mainly addresses this issue by modifying the data path to look at the migrated_at LSN to decide whether to scan the v2 keyspace or not. If the request is below the migrated_at LSN, only v1 is used.

How this mechanism interacts with branches are not tested yet and will be in the upcoming patches. For example, if we create a branch below the migrated_at LSN of the current branch, we should reset it to legacy mode. Currently this is solely guarded by the maybe_enable_rel_size_v2 (see "Revert the status to legacy...") and we should also do this during timeline creation.

This patch also adds partial support for v2-only mode. This is governed by rel_size_v1_access_disabled flag. The code is here solely to make the tests pass, not 100% correct (for example, when the above situation happens during the switch from v1+v2 validation mode to v2-only mode, it might have some races). We do not plan to switch to v2-only mode in the near term. This needs more tests before we enable.

There was a previous attempt #12708 to deal with this index_part <-> reality inconsistency by persisting the migration status in the keyspace. The pros of that approach is that we get very precise LSN where the migration is done and that LSN is consistent with when we do the migration. However, the downside is that it needs to query that key on every request, and it greatly slows down the tests (all pg_regress tests are timing out on ingestion). So in this patch, we take the approach to allow the inconsistency between index_part <-> real data and detect such inconsistency on the write path + in the future branch creation time.

Basically, the way to look into the patch is like:

• Think about the case where the reldir migration is done, index_part gets uploaded, but delta layers are not uploaded yet.
• (Note that we never upload data before the index_part gets updated as we set the index_part flag before commit the changes.)
• What happens when inconsistency happens during tenant re-attach.
  • and note that inconsistency always comes with ingestion LSN < migrated_at
  • so we will always trigger the relv2 migration path when we encounter the create relation WAL again (note that we use <= migrated_lsn instead of <)
  • and either do a migration again at this point or keep the legacy mode if the config flag is unset.
  • this is discovered/covered by the timeline_offloading test case but we will have a separate one in the future.

Summary of changes

• rel_size_v1_access_disabled to fully disable v1 and transition the status to Migrated.
• Fix a bug that v2-only mode does not put the placeholder for v1 reldir keys. collect_keyspace still needs that.
• Only read v2 keyspace if the request is equal to or above the migrated_at LSN.
• Patch test_image_consistent_lsn: this test was added as part of the hadron new features. It has some assumption on the layer size and the relv2 migration process breaks that assumption. So I forced v1 in that test case, and I'll do a full fix later.

[github-actions]

9152 tests run: 8490 passed, 0 failed, 662 skipped (full report)

---

Flaky tests (1)

Postgres 16

• test_pageserver_reconnect: release-arm64-with-lfc

Code coverage* (full report)

• functions: 34.7% (8831 of 25440 functions)
• lines: 45.8% (71623 of 156471 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
5713ff3 at 2025-07-30T16:22:16.761Z :recycle:}

[erikgrinaker]

I haven't paged this back in enough to give it a thorough review, but it seems okay at a high level.

[erikgrinaker]

nit: can this get a doc comment please?

[erikgrinaker]

nit: stray line?

[erikgrinaker]

nit: not an LSN