Adaptive protocol timing: derive FTP/param timeouts and windows from measured link RTT and loss

[DonLakeFlyer]

Problem

Every timeout, retry count, and window size in the MAVLink request/response protocols is a compile-time constant:

• FTPManager::_ackOrNakTimeoutMsecs = 3000, fixed burst size, fixed session-reopen count
• ParameterManager::kWaitForParamValueAckMs = 1000, kIndexBatchMaxOutstanding = 10, kMaxInitialLoadRetrySingleParam = 5, kUnresponsiveSilentCycles = 2
• RequestMetaDataTypeStateMachine "slow download" heuristic: abort if elapsed > 5 s && progress < 0.5 && projected > 40 s
• _ftpDownloadComplete "FTP too slow, fall back to stream" heuristic: 0.0001 < progress < 0.01

The only link awareness is LinkConfiguration::isHighLatency(), a boolean meaning "satellite — skip parameter load entirely."

These constants were tuned for a healthy SiK radio and happen to also work on WiFi because everything completes well inside them. They fail in the middle: a SiK link at range, a congested WiFi/ESP bridge with a few percent loss, a slow DroneCAN node behind the autopilot. That is the environment #15114 came from, and it is why #15120 needed five review rounds of hand-tuned retry/give-up flags.

MockLink has zero latency and zero loss unless a test asks for a specific failure mode, so CI never exercises the constants against a realistic link and cannot catch this class of bug.

Proposal

Measure the link; stop configuring it.

1. Per-link RTT / loss / throughput estimator on LinkInterface (or a small LinkStats object it owns).

   • RTT samples come for free from request/response pairs we already do: PARAM_REQUEST_READ→PARAM_VALUE, FTP request→ack (seq-numbered), COMMAND_LONG→COMMAND_ACK, TIMESYNC/PING.
   • Keep SRTT / RTTVAR (Jacobson–Karels) and a loss-rate EWMA. Expose rto() = SRTT + 4·RTTVAR clamped to a sane range (e.g. 250 ms … 10 s), lossRate(), bytesPerSecond().
   • Throughput estimator can build on the existing DataRateTracker.

2. Protocols consume rto() instead of constants.

   • FTP ack/nak timeout, PARAM_REQUEST_READ / PARAM_SET ack timeout, metadata download stall detection.
   • Retries use exponential backoff on rto(); fixed retry counts stop being the tuning knob.

3. Outstanding-request window driven by loss (AIMD).

   • Replace kIndexBatchMaxOutstanding and the fixed FTP burst size with a congestion window: grow on ack, halve on timeout. WiFi opens to a large window and finishes fast; SiK settles at a few outstanding and stops flooding the radio.
   • "Component unresponsive" becomes "loss estimate to that component is 100% over N RTOs," replacing kUnresponsiveSilentCycles / kUnresponsiveMinOutstanding.

4. Transport selection from measured throughput. Decide FTP param.pck vs. PARAM_REQUEST_LIST stream up front from bytesPerSecond() and the known compressed size, instead of starting FTP, watching it crawl, and falling back.

5. Testability. Protocols take the stats provider by injection. MockLink gains link profiles (e.g. WiFi: 20 ms RTT / 0 % loss; SiKAtRange: 800 ± 300 ms RTT / 20 % loss; Congested: 200 ms / 5 %) and the existing FTPManagerTest / ParameterManagerTest suites run under each profile via _data() rows. Tests that only pass at zero latency are bugs.

Acceptance

• No static constexpr int k*Ms timeouts remain in FTPManager, ParameterManager initial load, or RequestMetaDataTypeStateMachine; all derive from the link estimator.
• Existing FTPManagerTest and ParameterManagerTest pass unchanged under the zero-latency profile, and also pass under a lossy/slow profile.
• Full PX4 parameter load over a simulated 57.6 kbps / 10 % loss link completes without declaring the autopilot or a DroneCAN component unresponsive.

Dependencies

• Refactor FTPManager onto a single operation object with queued completion #15122 (FTPManager refactor) and Refactor ParameterManager initial load around per-component load state with derived progress #15123 (ParameterManager refactor) first: timing has to be injectable before it can be adaptive. Constants become constructor/provider inputs in those PRs; this issue replaces the provider with the measured one.

Context

Follow-up from #15120 / #15114 design discussion.

[DonLakeFlyer]

Measurement design notes (from follow-up discussion)

No blocking probe in initial connect. The connect sequence is already a series of request/response pairs (REQUEST_MESSAGE(AUTOPILOT_VERSION), REQUEST_MESSAGE(PROTOCOL_VERSION), COMMAND_LONG→COMMAND_ACK, component-information requests, first PARAM_REQUEST_READs). Each is an RTT sample. Start from a conservative initial RTO (~1.5–2 s, roughly today's kWaitForParamValueAckMs) and fold samples in as they arrive, TCP-style; the estimate converges during the first second or two of the connection without adding a round trip.

Loss is passive. MAVLink headers carry a per-sender sequence number and Vehicle already derives a link loss percentage from gaps in it (feeds the telemetry-loss indicator). Use that as the loss input rather than counting request timeouts.

Karn's rule. Do not take RTT samples from retransmitted requests — the reply cannot be attributed to a specific send. During a timeout storm that yields no clean samples; a single TIMESYNC with a fresh nonce is the one unambiguous probe and is the right tool there.

Explicit probe only for stale estimates. If the link has been idle (Fly view, inbound telemetry only) and the user starts a burst operation — refresh params, log download, mission upload — send one TIMESYNC first to refresh SRTT before opening a window. Cost: one RTT. PING is deprecated in the spec; TIMESYNC is answered by both PX4 and ArduPilot.

Key the estimator by (link, sysid, compid). RTT to MAV_COMP_ID_AUTOPILOT1 (answers from RAM) and to a DroneCAN peripheral behind the autopilot (CAN round trip + bridge) can differ by an order of magnitude. Use the link-level estimate as the prior for a component with no samples yet. This is also what makes "component unresponsive" a clean per-component judgement (loss → 100 % over N RTOs for that compid) instead of a fixed silent-cycle count.

Additional acceptance

• Initial connect adds no new blocking round trip; SRTT is populated from existing connect-sequence traffic.
• Retransmitted requests contribute no RTT samples.
• Per-component RTO for a bridged component is allowed to exceed the autopilot's without triggering give-up.