fix(google-health): keep the skip's place across backfill slices

A walk that reads without the edge bound passed over everything earlier
slices wrote on every slice, and once it crossed the edge the skipped
pages were counted against the walk's page ceiling. The skip now stays
counted for the whole walk, runs on the slice's clock and its own
three-minute limit, and saves the page token it stopped at (on a limit,
a rate limit or the slice budget) so the next slice re-reads that one
page and goes on. A saved place counts as progress only when it moves
forward; a token Google refuses, or one that no longer reaches the edge,
sends the walk back to the newest page as before.
This commit is contained in:
Marc-André Bombeck committed 2026-10-15 16:38:12 +02:00
1 parent 73b9870338
commit 1c05813b82
8 files changed
+649 -184

No files matched your search

+11 -6
View File
@@ -143,12 +143,17 @@ Google answers that first resumed request with `400`, the slice reads the data
type again without the bound, passes over the readings newer than its note
without writing them, and carries on writing from there. The refusal is noted
per data type, so later slices read without the bound straight away. The pages
passed over are not counted against the slice's five minutes or 500 pages:
otherwise a history deeper than one slice would spend every slice passing over
what earlier slices wrote and never write anything new. Only the job's own
time limit stops a slice while it is passing over, so on such an account each
slice takes longer the further back the backfill has got. A `400` that names
something else (the page token, the page size) is still a failure.
passed over are not counted against the slice's 500 pages, but they do run on
its five minutes, and passing over stops after three minutes at most, so a
slice never holds the account longer than a slice. Wherever the walk stops, on
a limit or on a rate limit, it keeps Google's page token for the page it read
last, and the next slice reads that one page again and goes on from there
instead of passing over everything earlier slices wrote: the pages read stay
in proportion to the history. If Google no longer takes the token, or the page
it returns no longer reaches the noted reading, the slice starts again from the
newest page and passes over as before. A `400` that names something else (the
page size, or the page token of a walk that did not start from a kept one) is
still a failure.
The hourly sync waits for the slice that is running (up to eight minutes)
instead of skipping the account, so recent data keeps arriving while an old