Marc-André Bombeck
cbc3b3118d
fix(measurements): recompute a stored fold mean only where all its samples remain
...
The fold repair and the regular folds rewrote an existing daily or hourly
stats: mean from the samples still in the table. Fold tombstones are purged
75 days after they were written, so for an older window those samples can
be a fragment: a re-upload after an anchor reset, the rows of one later
fold run. A mean over them replaced a stored mean that had been taken from
the whole day, and on old days it moved a lot.
A stored mean is now recomputed only for a day that starts at or after the
tombstone horizon (foldConstituentHorizon): every fold tombstone of such a
day was written after dayStart + the fold threshold and is still inside the
retention, so the live samples plus the fold leftovers are the whole day.
That is the last 75 days for daily means and, since dense days fold 90 days
late, the 75 days before the raw window for hourly means. A window also
needs at least one fold leftover; without one the rows in hand are not
known to be all of it.
Beyond the horizon:
- The repair leaves the day as it is and counts it per type as
windowsSkippedBeyondHorizon.
- The daily-mean consolidation and the dense hourly fold leave a day or
hour that already has a live stats: row exactly as it is, the stored
mean and its live samples alike (daysLeftAsStored, hoursLeftAsStored).
Up to v1.41 the fold overwrote such a row with the mean of the live rows
alone, and the first v1.42 cut with the mean of those plus whatever
tombstones were left; both are a mean of part of a day. Dropping the
live rows instead would lose samples that may not be in the stored
mean, so they stay as synced. They are few, the folded_window guard
refuses new ones, and the nightly pass walks over them without writing.
A day or hour without a stored mean still folds as before.
Inside the horizon, a pass that recomputes a stored mean soft-deletes the
live samples it took in as fold leftovers instead of deleting them, so a
second pass computes the same mean from the same samples. Before, a fold
that deleted them left the repair a fragment, and the repair then rewrote
the mean from the tombstones alone.
The repair, the mean consolidation and the dense fold now take a per-account
advisory lock after the restore lock in every day transaction, and the
repair reads the day's samples inside it, so the two never compute the
same window from two views of its samples or write the same row at once.
The repair also stops at the fold boundary: a day the fold has not
finished is the fold's to complete.
2026-10-09 01:03:02 +02:00
..
2026-09-25 07:39:06 +02:00
2026-05-10 10:34:32 +02:00
2026-08-06 18:21:40 +02:00
2026-09-25 07:39:06 +02:00
2026-10-06 18:39:50 +02:00
2026-05-09 20:46:20 +02:00
2026-09-25 07:39:06 +02:00
2026-09-25 08:51:00 +02:00
2026-09-26 01:02:48 +02:00
2026-10-08 07:56:59 +02:00
2026-06-19 22:20:31 +02:00
2026-05-10 11:05:40 +02:00
2026-07-29 13:42:04 +02:00
2026-09-24 07:54:59 +02:00
2026-10-03 18:18:56 +02:00
2026-09-24 09:22:50 +02:00
2026-10-06 21:02:41 +02:00
2026-09-24 09:19:43 +02:00
2026-10-06 21:02:41 +02:00
2026-10-05 22:28:10 +02:00
2026-06-28 19:26:16 +02:00
2026-08-30 14:40:43 +02:00
2026-07-04 14:46:19 +02:00
2026-05-16 19:46:43 +02:00
2026-05-14 23:49:32 +02:00
2026-08-06 14:10:28 +02:00
2026-09-10 00:37:15 +02:00
2026-07-29 14:20:56 +02:00
2026-07-16 08:48:41 +02:00
2026-10-04 22:15:48 +02:00
2026-07-21 13:08:28 +02:00
2026-09-25 02:05:19 +02:00
2026-09-26 20:11:45 +02:00
2026-09-11 12:37:54 +02:00
2026-07-19 11:41:12 +02:00
2026-10-02 23:01:36 +02:00
2026-10-06 19:44:52 +02:00
2026-09-26 15:55:57 +02:00
2026-07-18 02:09:33 +02:00
2026-09-24 12:08:39 +02:00
2026-09-26 01:49:02 +02:00
2026-10-01 11:17:44 +02:00
2026-09-25 07:39:06 +02:00
2026-10-04 08:17:34 +02:00
2026-09-25 07:39:06 +02:00
2026-09-26 19:54:40 +02:00
2026-09-25 07:39:06 +02:00
2026-10-04 10:19:32 +02:00
2026-09-25 07:39:06 +02:00
2026-10-04 22:17:27 +02:00
2026-10-04 18:23:14 +02:00
2026-09-26 19:22:03 +02:00
2026-09-27 13:13:50 +02:00
2026-09-28 00:06:47 +02:00
2026-09-27 13:21:18 +02:00
2026-10-08 22:37:57 +02:00
2026-09-26 01:02:48 +02:00
2026-09-25 07:39:06 +02:00
2026-10-04 10:19:31 +02:00
2026-09-30 22:21:46 +02:00
2026-10-03 18:19:50 +02:00
2026-09-25 21:55:14 +02:00
2026-07-20 00:40:20 +02:00
2026-06-14 14:26:58 +02:00
2026-10-05 22:28:10 +02:00
2026-06-19 22:20:31 +02:00
2026-06-27 22:40:01 +02:00
2026-06-27 20:37:35 +02:00
2026-09-27 14:08:49 +02:00
2026-05-10 11:05:40 +02:00
2026-09-24 08:03:04 +02:00
2026-09-26 10:41:02 +02:00
2026-07-21 01:34:29 +02:00
2026-07-26 18:34:42 +02:00
2026-09-11 12:55:52 +02:00
2026-09-11 12:37:54 +02:00
2026-09-24 09:19:43 +02:00
2026-07-16 23:32:28 +02:00
2026-10-05 12:53:40 +02:00
2026-05-10 16:11:12 +02:00
2026-05-11 02:38:17 +02:00
2026-10-04 20:45:21 +02:00
2026-10-06 21:49:27 +02:00
2026-10-01 13:23:17 +02:00
2026-07-24 18:28:41 +02:00
2026-09-24 12:11:21 +02:00
2026-10-06 20:47:17 +02:00
2026-08-22 16:14:54 +02:00
2026-10-05 12:40:55 +02:00
2026-10-06 19:38:57 +02:00
2026-10-08 18:42:01 +02:00
2026-09-24 09:29:00 +02:00
2026-10-08 08:41:25 +02:00
2026-07-15 22:49:07 +02:00
2026-08-13 22:07:57 +02:00
2026-09-26 15:55:57 +02:00
2026-09-03 07:42:59 +02:00
2026-09-26 19:58:50 +02:00
2026-09-25 02:05:19 +02:00
2026-09-26 16:23:14 +02:00
2026-09-24 08:11:14 +02:00
2026-10-04 08:17:34 +02:00
2026-10-01 08:50:31 +02:00
2026-09-10 12:23:59 +02:00
2026-08-08 01:02:08 +02:00
2026-09-24 12:30:39 +02:00
2026-10-04 20:00:18 +02:00
2026-09-24 12:37:51 +02:00
2026-09-09 23:03:38 +02:00
2026-09-25 13:48:31 +02:00
2026-09-03 01:47:52 +02:00
2026-07-20 00:40:20 +02:00
2026-10-04 11:33:42 +02:00
2026-10-08 22:37:57 +02:00
2026-10-04 22:14:14 +02:00
2026-08-03 09:52:02 +02:00
2026-09-11 12:22:47 +02:00
2026-09-30 23:01:57 +02:00
2026-07-17 14:29:07 +02:00
2026-10-09 01:03:02 +02:00
2026-09-27 21:20:24 +02:00
2026-10-05 12:40:55 +02:00
2026-10-01 08:56:19 +02:00
2026-10-01 08:47:18 +02:00
2026-07-20 12:24:15 +02:00
2026-10-08 13:03:37 +02:00
2026-09-26 22:11:27 +02:00
2026-10-08 13:03:37 +02:00
2026-10-08 13:03:37 +02:00
2026-10-03 21:14:47 +02:00
2026-09-26 14:42:13 +02:00
2026-10-04 15:56:56 +02:00
2026-07-08 22:05:05 +02:00
2026-09-26 00:38:26 +02:00
2026-10-04 21:59:22 +02:00
2026-09-25 07:25:29 +02:00
2026-07-21 13:08:28 +02:00
2026-10-01 21:53:37 +02:00
2026-08-01 19:26:47 +02:00
2026-08-10 05:32:58 +02:00
2026-08-10 05:32:59 +02:00
2026-09-27 13:07:10 +02:00
2026-08-08 15:06:10 +02:00
2026-10-04 10:19:31 +02:00
2026-08-08 15:54:19 +02:00
2026-09-26 01:49:02 +02:00
2026-10-04 01:58:04 +02:00
2026-09-26 01:49:02 +02:00
2026-10-05 13:07:22 +02:00
2026-10-08 10:02:39 +02:00
2026-10-08 22:37:56 +02:00
2026-10-08 10:02:39 +02:00
2026-09-03 05:30:08 +02:00
2026-09-26 19:54:38 +02:00
2026-10-04 11:33:45 +02:00
2026-06-22 09:00:39 +02:00
2026-06-22 09:00:39 +02:00
2026-10-09 01:03:02 +02:00
2026-10-09 01:03:02 +02:00
2026-10-08 10:02:39 +02:00
2026-07-19 13:40:03 +02:00
2026-07-21 00:36:40 +02:00
2026-09-27 20:42:54 +02:00
2026-07-29 14:18:20 +02:00
2026-07-29 14:22:49 +02:00
2026-10-04 11:33:45 +02:00
2026-08-07 06:11:36 +02:00
2026-10-08 12:04:36 +02:00
2026-08-10 05:32:59 +02:00
2026-09-24 10:52:04 +02:00
2026-08-12 22:52:03 +02:00
2026-09-25 07:39:06 +02:00
2026-08-07 14:16:17 +02:00
2026-10-08 09:22:49 +02:00
2026-10-08 09:37:04 +02:00
2026-10-04 10:14:22 +02:00
2026-08-06 01:29:07 +02:00
2026-05-09 18:08:11 +02:00
2026-09-24 09:03:58 +02:00
2026-09-24 09:19:43 +02:00
2026-05-10 01:30:06 +02:00
2026-05-10 11:05:40 +02:00
2026-06-28 19:23:57 +02:00
2026-09-24 09:06:22 +02:00
2026-09-24 19:40:27 +02:00
2026-07-07 15:55:50 +02:00
2026-06-19 22:20:31 +02:00
2026-06-11 02:19:28 +02:00
2026-09-30 23:11:03 +02:00
2026-07-27 14:35:34 +02:00
2026-07-28 14:00:44 +02:00
2026-07-27 14:35:33 +02:00
2026-07-26 18:02:35 +02:00
2026-10-08 09:05:54 +02:00
2026-07-29 15:59:27 +02:00
2026-10-03 18:53:28 +02:00
2026-08-31 19:38:08 +02:00
2026-10-03 15:44:51 +02:00
2026-09-26 01:02:48 +02:00
2026-09-27 13:21:18 +02:00
2026-09-24 09:39:27 +02:00
2026-09-10 04:01:14 +02:00
2026-10-03 18:21:19 +02:00
2026-09-10 01:09:50 +02:00
2026-08-06 02:36:08 +02:00
2026-10-08 22:37:57 +02:00
2026-08-07 05:24:11 +02:00
2026-08-07 05:11:02 +02:00
2026-09-09 20:06:48 +02:00
2026-09-10 04:41:38 +02:00
2026-10-08 09:28:56 +02:00
2026-10-04 22:14:14 +02:00
2026-07-29 15:59:12 +02:00
2026-10-08 14:31:13 +02:00
2026-09-26 13:29:36 +02:00
2026-09-28 00:06:47 +02:00
2026-07-25 15:03:44 +02:00
2026-10-04 10:41:08 +02:00
2026-06-19 22:20:31 +02:00
2026-10-08 07:56:59 +02:00
2026-09-24 07:56:33 +02:00
2026-05-31 14:36:17 +02:00
2026-07-25 15:03:44 +02:00
2026-08-06 20:56:16 +02:00
2026-09-25 08:04:30 +02:00
2026-10-01 08:58:42 +02:00
2026-07-26 00:45:08 +02:00
2026-06-19 22:20:31 +02:00
2026-06-10 11:18:00 +02:00
2026-10-03 23:36:08 +02:00
2026-10-03 22:52:05 +02:00
2026-08-06 02:00:35 +02:00
2026-08-06 02:00:35 +02:00
2026-08-06 02:00:35 +02:00
2026-09-25 10:31:22 +02:00
2026-08-10 07:58:48 +02:00
2026-10-04 08:17:34 +02:00
2026-08-07 06:02:31 +02:00
2026-07-29 15:59:12 +02:00
2026-06-19 22:20:31 +02:00
2026-09-10 07:40:05 +02:00
2026-07-25 18:41:40 +02:00
2026-09-26 15:21:42 +02:00
2026-08-06 18:54:20 +02:00
2026-08-08 01:03:05 +02:00
2026-09-25 07:39:06 +02:00
2026-08-08 09:58:22 +02:00
2026-08-08 09:58:50 +02:00
2026-08-08 15:30:38 +02:00
2026-10-01 10:01:06 +02:00
2026-06-05 03:45:04 +02:00
2026-06-27 19:33:10 +02:00
2026-06-19 22:20:31 +02:00
2026-08-08 15:30:00 +02:00
2026-07-28 14:01:11 +02:00
2026-08-08 03:03:38 +02:00
2026-10-05 12:40:55 +02:00
2026-08-06 13:47:42 +02:00
2026-09-11 13:43:19 +02:00
2026-07-30 23:30:28 +02:00
2026-09-25 02:05:19 +02:00
2026-09-26 20:18:55 +02:00
2026-09-26 22:08:51 +02:00
2026-09-26 22:42:40 +02:00
2026-09-10 18:58:21 +02:00
2026-09-10 05:07:15 +02:00
2026-07-25 11:36:18 +02:00
2026-09-26 15:55:57 +02:00
2026-09-25 02:05:19 +02:00
2026-07-27 23:15:38 +02:00
2026-09-27 09:59:34 +02:00
2026-08-09 06:53:57 +02:00
2026-07-26 14:37:57 +02:00
2026-10-08 09:06:09 +02:00
2026-07-22 04:14:15 +02:00
2026-10-04 11:37:57 +02:00
2026-10-04 11:22:17 +02:00
2026-10-04 10:41:08 +02:00
2026-10-04 14:55:03 +02:00
2026-10-04 11:22:17 +02:00
2026-07-19 02:31:45 +02:00
2026-09-10 04:55:19 +02:00
2026-09-09 22:55:53 +02:00
2026-09-26 19:54:38 +02:00
2026-08-06 17:52:10 +02:00
2026-08-06 19:33:15 +02:00
2026-08-06 18:21:40 +02:00
2026-09-10 15:35:52 +02:00
2026-09-06 15:36:35 +02:00
2026-09-26 20:06:36 +02:00
2026-07-29 13:42:04 +02:00
2026-09-28 00:34:11 +02:00
2026-09-25 07:39:06 +02:00
2026-07-14 11:56:29 +02:00
2026-10-04 15:29:02 +02:00
2026-09-26 16:56:02 +02:00
2026-06-26 13:24:30 +02:00
2026-09-26 20:06:36 +02:00
2026-10-04 14:14:22 +02:00
2026-06-19 22:20:31 +02:00
2026-07-23 15:16:48 +02:00
2026-09-26 22:34:21 +02:00
2026-05-21 23:13:21 +02:00
2026-10-08 07:56:59 +02:00
2026-09-02 11:51:09 +02:00
2026-10-03 18:27:28 +02:00
2026-07-07 22:00:14 +02:00
2026-09-09 20:06:48 +02:00
2026-08-14 02:33:07 +02:00
2026-08-22 13:04:33 +02:00
2026-08-07 00:01:27 +02:00
2026-10-05 22:28:10 +02:00
2026-08-07 06:11:36 +02:00
2026-08-07 06:11:36 +02:00
2026-09-27 00:52:11 +02:00
2026-10-08 22:37:57 +02:00
2026-09-24 10:32:41 +02:00
2026-09-02 13:01:23 +02:00
2026-08-05 23:58:30 +02:00
2026-08-07 14:30:10 +02:00
2026-07-26 14:04:44 +02:00
2026-06-04 10:05:47 +02:00
2026-07-29 16:08:41 +02:00
2026-10-08 22:57:47 +02:00
2026-07-15 22:49:07 +02:00
2026-09-26 20:06:36 +02:00
2026-09-26 01:02:48 +02:00
2026-09-25 14:10:56 +02:00
2026-09-10 10:49:12 +02:00
2026-10-03 21:14:08 +02:00
2026-10-08 08:41:25 +02:00
2026-09-27 13:46:38 +02:00
2026-08-02 02:00:08 +02:00
2026-10-08 09:06:09 +02:00
2026-10-08 10:32:11 +02:00
2026-10-08 08:41:25 +02:00
2026-06-19 22:20:31 +02:00
2026-08-08 21:01:25 +02:00
2026-08-08 14:36:08 +02:00
2026-09-10 11:11:38 +02:00
2026-10-08 22:37:57 +02:00
2026-08-08 23:00:39 +02:00
2026-09-24 09:44:51 +02:00
2026-09-26 20:22:12 +02:00
2026-06-05 08:10:13 +02:00
2026-07-22 04:14:15 +02:00
2026-08-08 01:05:31 +02:00
2026-05-17 15:36:38 +02:00
2026-07-18 02:30:33 +02:00
2026-05-24 22:22:30 +02:00
2026-05-14 23:49:32 +02:00
2026-05-17 15:36:38 +02:00
2026-09-27 09:59:34 +02:00
2026-08-08 09:53:09 +02:00
2026-07-22 04:14:15 +02:00
2026-09-27 15:11:43 +02:00
2026-10-03 17:53:36 +02:00
2026-10-03 23:26:48 +02:00