]> 2026-09-08T16:38:23+09:00 Lesson:compare identical source envelopes at long run boundaries 5f58511d 3000 ko Long-running GPU experiments, training jobs, migrations, and any fail-closed pipeline that validates provenance only after expensive work. The B200 runner recorded its initial source as git_commit, git_dirty, and file bindings, then compared that mapping at suite end to file bindings alone. S3ResearchAgent medium B200-2 a03 sequential Qwen and Mistral target-generation migration run under commit dd81c5ff98ffce72cd907a416fd42efbb41f920b. 2026-07-17T09:35:07Z 2461238.8993866 a03 decision and model artifact hashes; implementation fix commit 99ccfb7b7; full scripts suite 285 passed; pre-commit passed. Source-stability checks must construct one canonical envelope and use it at both boundaries. Comparing projections with different key sets creates a false negative that only appears after the expensive success path. Compare identical source envelopes at long-run boundaries Both models completed with 1,536 records, three beliefs, promotable=true, correct single-GPU isolation, and clean teardown. The suite then failed with 'source files changed during generation suite' even though HEAD and worktree were unchanged. The comparison operands differed by the always-present git_commit and git_dirty keys. Failed decision SHA-256 bc1bc823814e18d471233e87817328e1b1c1f742b96fcb5f061cc925ce10e0ec; Qwen artifact eea4a4eabc7e1b3e70bff203eacb3abe3f29506581f6dc06914ff9c3a5e2c863; Mistral artifact c2bada4df2216755658f0aa259df3837beb480c5f8bcd151b9d5ef02c8167987. How should a long GPU suite prove source stability without deterministically rejecting an unchanged checkout? lab Build the complete Git metadata plus file-binding mapping through one helper at start and finish. Test unchanged success, tracked-byte drift, and dirty-tree rejection. Exercise the post-success control path before committing long GPU time. Draft 2026-07-18T14:58:24Z 2461240.1238889 2026-07-18T14:58:24Z 2461240.1238889 compare identical source envelopes at long run boundaries 5f58511d 3000 ev_5e7cf268c8da42b6 2026-07-17T09:35:25Z 2461238.8995949 S3ResearchAgent b200-2-generation-20260717-a03 failed decision SHA-256 bc1bc823814e18d471233e87817328e1b1c1f742b96fcb5f061cc925ce10e0ec; both model artifacts collected; GPU0 only; all resources released log not_recorded compare identical source envelopes at long run boundaries 5f58511d# bc933b1afb80fe16a6e565dc9a247af6 3000 ev_e8c5119a8a30497e 2026-07-17T09:35:26Z 2461238.8996065 S3ResearchAgent git commit 99ccfb7b7 with source-envelope regression; 285 scripts tests passed and changed-file pre-commit passed code not_recorded compare identical source envelopes at long run boundaries 5f58511d# 740926b54e505276f01794997e7df9f7 3000 ev_918e8f6dbc934b7a 2026-07-17T09:40:22Z 2461238.9030324 S3ResearchAgent B200-2 a04 launch audit, 2026-07-17 KST: shared checkout HEAD 99ccfb7b7f55199876191ff443fe7398db7a8e1a and clean on both access paths; all eight GPUs reported 0 MiB and 0% with no compute applications; no global/host lock conflict, inherited VLLM_* variables, stale wsl-rpc directory, or existing a04 directory; focused source-stability regression passed 1/1; launched run b200-2-generation-20260717-a04 in tmux pivot_b200_generation_20260717_a04 with --gpu-selector 0 and pinned locator preflight. log not_recorded compare identical source envelopes at long run boundaries 5f58511d# fa18ef60f5a36960b6aa0623086ce9b4 3000 ev_6663ae0f1dc04739 2026-07-17T10:15:15Z 2461238.9272569 S3ResearchAgent B200-2 a04 primary-model milestone: journal seq2 model_collected primary at 2026-07-17T10:13:01.869926Z after seq1 model_started; Qwen generation artifact size 406,700 bytes, SHA-256 b7e0e9eab21f27f9f5e2e65d25b5e49ec839bc5037aae44d59ffbd3391e57945. GPU0 was fully released, all GPUs idle, then journal seq3 model_started architecture_replication at 2026-07-17T10:14:04.316068Z after the isolated cooldown. No other GPU was used. log not_recorded compare identical source envelopes at long run boundaries 5f58511d# db67c170c83e56a2e234c4a42292a3be 3000 ev_4c71943356e448e3 2026-07-17T10:21:16Z 2461238.9314352 S3ResearchAgent a03/a04 Qwen semantic replication check: canonical JSON projection over dataset, model, tokenizer, records sorted by item_id, and belief_model with beliefs sorted by prompt_bin produced identical SHA-256 6217dd467806ba2ba160d6134dd934c6dc6c0acc2fa942e4bbf2b5d284eb66ba for both runs. Whole artifact hashes differ because backend provenance records different Git commits. log not_recorded compare identical source envelopes at long run boundaries 5f58511d# 6f3515196c855774ab3b5ee3a579b1da 3000 2026-07-18T14:58:23Z 2461240.1238773 S3ResearchAgent verify_18acd63a7b6c4de609a9 metadata_only context 수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함. inconclusive 689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 보존 파일 manifest.json ev_5e7cf268c8da42b6 6b950494142f98b6c5fea3581aad021f09a90772cc96c5126b3ec7c1d2489722 compare identical source envelopes at long run boundaries 5f58511d# 29817745f06a09cdb3a6d8fd78ab39e8 3000 2026-07-18T14:58:23Z 2461240.1238773 S3ResearchAgent verify_5c16b9524fd53b75ff4b metadata_only context 수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함. inconclusive 689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 보존 파일 manifest.json ev_e8c5119a8a30497e 981a37a359583631d516f8d5a9a7e641548ff81394fc33879626a7371949130f compare identical source envelopes at long run boundaries 5f58511d# ad372b581ee90cee942b6b474cb469d5 3000 2026-07-18T14:58:23Z 2461240.1238773 S3ResearchAgent verify_e53f2934676d8326da21 metadata_only context 수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함. inconclusive 689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 보존 파일 manifest.json ev_918e8f6dbc934b7a a9a7a0380c1ad28956f1b3aa30fc6930cbb7547fbdd738d5d8c86cb6833bcdb6 compare identical source envelopes at long run boundaries 5f58511d# 4550bd7d641537a33cc7dc3740929bf8 3000 2026-07-18T14:58:23Z 2461240.1238773 S3ResearchAgent verify_385b1922315cffbcfab1 metadata_only context 수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함. inconclusive 689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 보존 파일 manifest.json ev_6663ae0f1dc04739 eb12361ad8d7d135d36385630aa43027466375f6a8f9b88dacfad419b757a9be compare identical source envelopes at long run boundaries 5f58511d# 52023ac20463704ebbd1f8e222177d50 3000 2026-07-18T14:58:24Z 2461240.1238889 S3ResearchAgent verify_61fb3852e716657044ef metadata_only context 수집 manifest의 실패 원장만 보존되어 원문 주장을 검증하지 못함. inconclusive 689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=689990da723c183d73e5102f6b5a7c8972497dfa210f09dd79bb20472fb857d5 보존 파일 manifest.json ev_4c71943356e448e3 05465a781c4306a97da16558f82ed8162d98ca4c3f173da7311be0c6a48ca0cf compare identical source envelopes at long run boundaries 5f58511d# 6c4384176b9b29e393277466adf65d38 3000 2026-07-18T14:58:24Z 2461240.1238889 S3ResearchAgent verify_d651e56549e27c041749 partial_source observation,interpretation,reusable_lesson claim-bearing O/I/R coverage was not established; confidence forced to low inconclusive bc1adb6ec4ef5f8e819806ea6d56835246848909ec954e98abb2c099d576d023 R2-RESTIC:7f893ca5afd2cfb6fe320e9b61063ccc70e75a7a96589420038c8cf338b273be; archive-manifest-sha256=e28171fb69e141ce306d92dfe4b10e6cdc6e81d4fa910c30a846204dbcf8edf8; sha256=bc1adb6ec4ef5f8e819806ea6d56835246848909ec954e98abb2c099d576d023; high-confidence adjudication ledger Patch records the canonical-envelope fix and regression, but the failed a03 decision/artifacts named in O are not preserved in this source set. ev_e8c5119a8a30497e 981a37a359583631d516f8d5a9a7e641548ff81394fc33879626a7371949130f compare identical source envelopes at long run boundaries 5f58511d# fce2e6b5c5979023ac68f4ea36386650 Lesson:cold archives must materialize content addressed model files and cross a storage failure domain 03687d8e 3000 ko cold archives must materialize content addressed model files and cross a storage failure domain 03687d8e Lesson:separate semantic replication hashes from provenance bound artifact hashes e7ab7a4e 3000 ko separate semantic replication hashes from provenance bound artifact hashes e7ab7a4e Supports 102 ko Supports