{"article":"Score Centering Done Right","checked_date":"2026-10-12","scope":"Displayed real-Qwen gradient measurements and synthetic arithmetic RL results only. No new training launched. Seed-42 success is not a consistent cross-seed advantage. Batch-512 follow-up has no completed endpoint in this checked snapshot.","qwen_sampler_target":{"configuration":"lag4 FP32, eight SST-2 A/B prompts, four responses per group, all 752393024 text parameters, engine batch one","target":"E[q^4] mean(A_i score_q_i); sample-standardized group advantages ddof=1, epsilon=1e-6","aggregation":"Arithmetic mean of conditional per-prompt bias\u00b2, variance or MSE divided by that prompt\u2019s inference-reference target norm\u00b2; eight fixed SST-2 prompts, not a population estimate.","values":[{"configuration":"sgd_lag4_fp32","statistical_batch_size":4,"target":"g_q","method":"SC","relative_bias2":0.049218363209716016,"relative_variance":0.18616327269499328,"relative_mse":0.23538163590470929},{"configuration":"sgd_lag4_fp32","statistical_batch_size":4,"target":"g_q","method":"TIS","relative_bias2":0.054491045439024936,"relative_variance":0.2049938552151766,"relative_mse":0.25948490065420154},{"configuration":"sgd_lag4_fp32","statistical_batch_size":4,"target":"g_q","method":"corrected_SC","relative_bias2":0.001989021248390651,"relative_variance":0.20081530008276133,"relative_mse":0.20280432133115198},{"configuration":"sgd_lag4_fp32","statistical_batch_size":4,"target":"g_q","method":"oracle_q","relative_bias2":1.0665271410986257e-32,"relative_variance":0.19634525038011358,"relative_mse":0.19634525038011358},{"configuration":"sgd_lag4_fp32","statistical_batch_size":16,"target":"g_q","method":"SC","relative_bias2":0.049218363209716016,"relative_variance":0.04654081817374832,"relative_mse":0.09575918138346434},{"configuration":"sgd_lag4_fp32","statistical_batch_size":16,"target":"g_q","method":"TIS","relative_bias2":0.054491045439024936,"relative_variance":0.05124846380379415,"relative_mse":0.1057395092428191},{"configuration":"sgd_lag4_fp32","statistical_batch_size":16,"target":"g_q","method":"corrected_SC","relative_bias2":0.001989021248390651,"relative_variance":0.05020382502069033,"relative_mse":0.05219284626908099},{"configuration":"sgd_lag4_fp32","statistical_batch_size":16,"target":"g_q","method":"oracle_q","relative_bias2":1.0665271410986257e-32,"relative_variance":0.049086312595028395,"relative_mse":0.049086312595028395},{"configuration":"sgd_lag4_fp32","statistical_batch_size":64,"target":"g_q","method":"SC","relative_bias2":0.049218363209716016,"relative_variance":0.01163520454343708,"relative_mse":0.0608535677531531},{"configuration":"sgd_lag4_fp32","statistical_batch_size":64,"target":"g_q","method":"TIS","relative_bias2":0.054491045439024936,"relative_variance":0.012812115950948538,"relative_mse":0.06730316138997347},{"configuration":"sgd_lag4_fp32","statistical_batch_size":64,"target":"g_q","method":"corrected_SC","relative_bias2":0.001989021248390651,"relative_variance":0.012550956255172583,"relative_mse":0.014539977503563235},{"configuration":"sgd_lag4_fp32","statistical_batch_size":64,"target":"g_q","method":"oracle_q","relative_bias2":1.0665271410986257e-32,"relative_variance":0.012271578148757099,"relative_mse":0.012271578148757099},{"configuration":"sgd_lag4_fp32","statistical_batch_size":256,"target":"g_q","method":"SC","relative_bias2":0.049218363209716016,"relative_variance":0.00290880113585927,"relative_mse":0.05212716434557529},{"configuration":"sgd_lag4_fp32","statistical_batch_size":256,"target":"g_q","method":"TIS","relative_bias2":0.054491045439024936,"relative_variance":0.0032030289877371344,"relative_mse":0.05769407442676207},{"configuration":"sgd_lag4_fp32","statistical_batch_size":256,"target":"g_q","method":"corrected_SC","relative_bias2":0.001989021248390651,"relative_variance":0.0031377390637931457,"relative_mse":0.005126760312183796},{"configuration":"sgd_lag4_fp32","statistical_batch_size":256,"target":"g_q","method":"oracle_q","relative_bias2":1.0665271410986257e-32,"relative_variance":0.0030678945371892747,"relative_mse":0.0030678945371892747}],"limitations":["Restricted two-action task; not full-vocabulary multi-token generation.","Per-action IS does not change the joint advantage distribution; TIS and IS are comparative estimators, not unbiased g_q estimators.","The shared-context centering term cancels within each group.","TIS cap2 never activates in these lag4 measurements."]},"precision_variance":{"definition":"Ratios of mean absolute A/B conditional group-gradient variances across eight prompts; all statistical batches rescale by the same factor.","master_checkpoint_fixed":true,"drsc_equals_sc":true,"values":[{"configuration":"fp32_control","label":"FP32","prompts":8,"mean_full_vocab_kl_q_p":0.0,"mean_full_vocab_weight_second_moment":1.0000000000000036,"median_full_vocab_weight_second_moment":1.0000000000000049,"min_full_vocab_weight_second_moment":0.999999999999996,"max_full_vocab_weight_second_moment":1.0000000000000089,"mean_full_vocab_tis_clipped_q_mass":0.0,"mean_full_vocab_tis_retained_weight_mass":1.0000000000000036,"binary_B4_SC_variance":6641.7748716499555,"binary_B4_TIS_variance":6641.7748716499555,"binary_B4_unclipped_IS_variance":6641.7748716499555,"binary_TIS_to_SC_variance":1.0,"binary_unclipped_IS_to_SC_variance":1.0},{"configuration":"cpu_autocast_fp16","label":"FP16","prompts":8,"mean_full_vocab_kl_q_p":6.511246241429905e-06,"mean_full_vocab_weight_second_moment":1.000013016872658,"median_full_vocab_weight_second_moment":1.0000135302859405,"min_full_vocab_weight_second_moment":1.0000070851744658,"max_full_vocab_weight_second_moment":1.0000187258048028,"mean_full_vocab_tis_clipped_q_mass":0.0,"mean_full_vocab_tis_retained_weight_mass":1.0000000000000036,"binary_B4_SC_variance":6650.654749296673,"binary_B4_TIS_variance":6656.012478254832,"binary_B4_unclipped_IS_variance":6656.012478254832,"binary_TIS_to_SC_variance":1.0008055942099123,"binary_unclipped_IS_to_SC_variance":1.0008055942099123},{"configuration":"cpu_autocast_bf16","label":"BF16","prompts":8,"mean_full_vocab_kl_q_p":0.0003438334684906214,"mean_full_vocab_weight_second_moment":1.0006844884052384,"median_full_vocab_weight_second_moment":1.0006352345941008,"min_full_vocab_weight_second_moment":1.0003559734146152,"max_full_vocab_weight_second_moment":1.0012128843104193,"mean_full_vocab_tis_clipped_q_mass":0.0,"mean_full_vocab_tis_retained_weight_mass":1.0000000000000036,"binary_B4_SC_variance":6709.524035135089,"binary_B4_TIS_variance":6761.691387447001,"binary_B4_unclipped_IS_variance":6761.691387447001,"binary_TIS_to_SC_variance":1.0077751196714897,"binary_unclipped_IS_to_SC_variance":1.0077751196714897},{"configuration":"weight_int8_g128_sim","label":"W8 sim.","prompts":8,"mean_full_vocab_kl_q_p":0.0013486312621220982,"mean_full_vocab_weight_second_moment":1.0026948572004621,"median_full_vocab_weight_second_moment":1.001929722891053,"min_full_vocab_weight_second_moment":1.001214927952656,"max_full_vocab_weight_second_moment":1.00562667352813,"mean_full_vocab_tis_clipped_q_mass":0.0,"mean_full_vocab_tis_retained_weight_mass":1.0000000000000036,"binary_B4_SC_variance":6697.375061974637,"binary_B4_TIS_variance":6735.64189420463,"binary_B4_unclipped_IS_variance":6735.64189420463,"binary_TIS_to_SC_variance":1.0057137060230148,"binary_unclipped_IS_to_SC_variance":1.0057137060230148},{"configuration":"weight_int4_g128_sim","label":"W4 sim.","prompts":8,"mean_full_vocab_kl_q_p":0.2452513747726765,"mean_full_vocab_weight_second_moment":1.7363497892672797,"median_full_vocab_weight_second_moment":1.632885032308151,"min_full_vocab_weight_second_moment":1.3492774553308764,"max_full_vocab_weight_second_moment":2.3117474858956104,"mean_full_vocab_tis_clipped_q_mass":0.06265478456803243,"mean_full_vocab_tis_retained_weight_mass":0.9203390913054021,"binary_B4_SC_variance":10173.667206790198,"binary_B4_TIS_variance":12537.276811063648,"binary_B4_unclipped_IS_variance":21644.62412236692,"binary_TIS_to_SC_variance":1.2323262159288941,"binary_unclipped_IS_to_SC_variance":2.127514462820317},{"configuration":"weight_int3_g128_sim","label":"W3 sim.","prompts":8,"mean_full_vocab_kl_q_p":2.5107492302402257,"mean_full_vocab_weight_second_moment":356.9976339700077,"median_full_vocab_weight_second_moment":126.75408221648208,"min_full_vocab_weight_second_moment":27.052480401640825,"max_full_vocab_weight_second_moment":2153.1924459878182,"mean_full_vocab_tis_clipped_q_mass":0.08137246977773568,"mean_full_vocab_tis_retained_weight_mass":0.4290822459347381,"binary_B4_SC_variance":7703.108088526853,"binary_B4_TIS_variance":9951.103050105467,"binary_B4_unclipped_IS_variance":54332.71899644406,"binary_TIS_to_SC_variance":1.2918296012134138,"binary_unclipped_IS_to_SC_variance":7.053350202545929}],"configuration":{"model":"/home/weicheng/.cache/huggingface/hub/models--Qwen--Qwen3.5-0.8B/snapshots/2fc06364715b967f1860aea9cf38778875588b17","torch":"2.6.0+cu124","transformers":"5.3.0","device":"cpu","threads":2,"nice":15,"seed":20261011,"engine_batch":1,"temperature":1.0,"prompts":8,"statistical_batches":[4,16,64,256],"group_size":4,"checkpoint_lag":0,"configs":["fp32_control","cpu_autocast_fp16","cpu_autocast_bf16","weight_int8_g128_sim","weight_int4_g128_sim","weight_int3_g128_sim"],"quantization":"Deterministic round-to-nearest symmetric absmax per output-row input-group of 128. qmax=2^(bits-1)-1, codes in [-qmax,qmax]. Linear weights except lm_head; norms, embeddings, convolutions and activations remain FP32. Dequantized weights execute FP32 kernels. NOT a native INT kernel, calibrated GPTQ/AWQ, or speed benchmark.","limitations":"Full-vocabulary metrics concern forward distributions and importance weights, NOT full-parameter gradient variance. Full-parameter estimator variance is evaluated separately for the constrained A/B policy, without claiming a differentiable low-precision q oracle. Same master checkpoint, no checkpoint-lag HVP.","source_prompts_sha256":"b5c1ce5b5971fc285a8f560d1ae7965a2b5cdaea6d001b78b74ad4539fed1fe9","source_code_sha256":"96d9d713f38c96c652664800179c8f33ed3abb0a9c9f5bd8f10bd6d567aeb0c7"},"limitations":["Weight rounding simulation executes in FP32; not native low-bit inference kernels.","No quantized-engine oracle bias claim."]},"seed42_learning":{"verified_unix":1791766312.8893192,"all_three_formal_runs_complete_and_verified":true,"scope":"One seed, new operand range and fixed dataset. Post-hoc subsets are descriptive. This completes only the three-method 30-step study, not the separate long-run matrices.","source_hashes_match":true,"actual_training_data_paired":true,"all_evaluations_use_same_twenty_heldout_prompts":true,"per_run_unique_training_prompts":480,"per_run_training_responses":1920,"evaluations_per_method":7,"evaluation_responses_per_checkpoint":80,"launch_manifest_sha256":"d7baf550d718ea005b6be6bcb8c4c4adb1e10bb928b8aa0fdd9f2d9fc213b399","runs":[{"run":"formal_grpo_seed42_gpu0","settings":{"method":"grpo","seed":42,"optimizer_steps":30,"eval_interval":5,"prompts_per_step":16,"responses_per_prompt":4,"global_batch":64,"micro_batch":1,"lr":1e-06,"mode":"colocated_fully_async","physical_gpu":0},"steps":30,"actual_training_samples":1920,"actual_prompt_groups":480,"distinct_training_prompts":480,"no_multi_epoch":true,"mixed_reward_groups":14,"evaluations":{"0":0.1125,"5":0.0125,"10":0.1,"15":0.1,"20":0.1,"25":0.1,"30":0.1},"mean_age_updates":0.24166666666666667,"max_age_updates":1,"elapsed_seconds":4310.957861185074,"status_sha256":"f92a9e237962dd8500136d41dc39c64189b88ed4900d0c6de0ee96bc29a0ee7d","ordered_training_prompt_sha256":"e56ce1d42ce80d7f7bd2d9dd4357235ae3fdc0c579438dd6160392690ed304e5","prompt_plan_sha256":"162d30e7ff875bc31983a63dd1d64e957e246f311db4743659b07c3735b6cd0a","identical_preselected_prompt_order_and_batches_verified":true,"independently_recomputed_evaluations":{"0":{"accuracy":0.1125,"response_counts":{"4":7,"9":3,"5":13,"3":12,"1":5,"To":1,"2":11,"7":3,"8":10,"6":7,"0":6,"<think>":1,"The":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":6.0},"no_units_carry":{"samples":44,"correct":3.0}},"source_sha256":"88ba89e9874f4f1369974efb6b509821ff7403f978a58ae6e829d6e0529a2c00"},"5":{"accuracy":0.0125,"response_counts":{"To":71,"1":2,"First":3,"Step":2,"The":1,"$$":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":1.0},"no_units_carry":{"samples":44,"correct":0.0}},"source_sha256":"86ef0935e68f0b8ca1aea168e1bc62745010a685a1b249756a4a9f8ed2a0a627"},"10":{"accuracy":0.1,"response_counts":{"1":80},"descriptive_subsets":{"units_carry":{"samples":36,"correct":8.0},"no_units_carry":{"samples":44,"correct":0.0}},"source_sha256":"575e95628b2faee925acbf19fdebba7a1991c81ae888d06498834effc82ee8e9"},"15":{"accuracy":0.1,"response_counts":{"1":80},"descriptive_subsets":{"units_carry":{"samples":36,"correct":8.0},"no_units_carry":{"samples":44,"correct":0.0}},"source_sha256":"7e383d854e8e23ba0842077e4d7d50c4588c7d3f0a791d4d8f686edcf0beab94"},"20":{"accuracy":0.1,"response_counts":{"1":80},"descriptive_subsets":{"units_carry":{"samples":36,"correct":8.0},"no_units_carry":{"samples":44,"correct":0.0}},"source_sha256":"fdcb3595f0d2b1e1e2be4b9d617d4f07f26c1940e1253abf070d06aae0ad3244"},"25":{"accuracy":0.1,"response_counts":{"1":80},"descriptive_subsets":{"units_carry":{"samples":36,"correct":8.0},"no_units_carry":{"samples":44,"correct":0.0}},"source_sha256":"73fd5695e52a6807201452eca83f916a48adf695b3f2fde238549e4336a90a0c"},"30":{"accuracy":0.1,"response_counts":{"1":80},"descriptive_subsets":{"units_carry":{"samples":36,"correct":8.0},"no_units_carry":{"samples":44,"correct":0.0}},"source_sha256":"db8525ccc1d001cacbcbae913ec35f438fc00b5f1e6246c029e929bff3a011f0"}}},{"run":"formal_ste_seed42_gpu1","settings":{"method":"ste","seed":42,"optimizer_steps":30,"eval_interval":5,"prompts_per_step":16,"responses_per_prompt":4,"global_batch":64,"micro_batch":1,"lr":1e-06,"mode":"colocated_fully_async","physical_gpu":1},"steps":30,"actual_training_samples":1920,"actual_prompt_groups":480,"distinct_training_prompts":480,"no_multi_epoch":true,"mixed_reward_groups":66,"evaluations":{"0":0.1125,"5":0.05,"10":0.0875,"15":0.075,"20":0.1,"25":0.15,"30":0.1},"mean_age_updates":0.24166666666666667,"max_age_updates":1,"elapsed_seconds":4279.37664604187,"status_sha256":"9ba90b6abb5fc91a502f670ba51159d5c5e4a7fe8eae939904f1596ace6d87f2","ordered_training_prompt_sha256":"e56ce1d42ce80d7f7bd2d9dd4357235ae3fdc0c579438dd6160392690ed304e5","prompt_plan_sha256":"162d30e7ff875bc31983a63dd1d64e957e246f311db4743659b07c3735b6cd0a","identical_preselected_prompt_order_and_batches_verified":true,"independently_recomputed_evaluations":{"0":{"accuracy":0.1125,"response_counts":{"4":7,"9":3,"5":13,"3":12,"1":5,"To":1,"2":11,"7":3,"8":10,"6":7,"0":6,"<think>":1,"The":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":6.0},"no_units_carry":{"samples":44,"correct":3.0}},"source_sha256":"88ba89e9874f4f1369974efb6b509821ff7403f978a58ae6e829d6e0529a2c00"},"5":{"accuracy":0.05,"response_counts":{"4":13,"3":13,"(":6,"To":23,"1":2,"The":3,"$":2,"First":1,"5":7,"6":2,"<think>":2,"7":1,"2":2,"$$":2,"</think>":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":1.0},"no_units_carry":{"samples":44,"correct":3.0}},"source_sha256":"84d91ac871bcc45c9fdb1467d0111e07cc750a80e8d4e23265433a3b25ac16c7"},"10":{"accuracy":0.0875,"response_counts":{"4":35,"3":35,"1":4,"5":2,"<think>":1,"2":2,"The":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":1.0},"no_units_carry":{"samples":44,"correct":6.0}},"source_sha256":"2e7be55c3e1b63402bf7a1dab1d9ae6fca806125a1546c7c5a03d3b283370672"},"15":{"accuracy":0.075,"response_counts":{"4":6,"3":10,"5":62,"1":1,"0":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":4.0},"no_units_carry":{"samples":44,"correct":2.0}},"source_sha256":"8a0e2ed3b69e55b6952b5e293c9e58efa22a73865b81e96f9f683b3aac2813e8"},"20":{"accuracy":0.1,"response_counts":{"5":74,"3":5,"":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":7.0},"no_units_carry":{"samples":44,"correct":1.0}},"source_sha256":"9477755b6f26aeeb5ab9ba5ec4da591532564389059a0cc64073baf8933b1c5b"},"25":{"accuracy":0.15,"response_counts":{"4":15,"3":53,"5":11,"<think>":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":2.0},"no_units_carry":{"samples":44,"correct":10.0}},"source_sha256":"a6bcbbd68bcbdbdabf79c1cb84e69934d439e72179ce2a8ae383581d4bbffa99"},"30":{"accuracy":0.1,"response_counts":{"3":80},"descriptive_subsets":{"units_carry":{"samples":36,"correct":0.0},"no_units_carry":{"samples":44,"correct":8.0}},"source_sha256":"5bb5c65a0243685e25f9d283e615e84d11f9e14bd733ce3031fac49a26c94e3d"}}},{"run":"formal_corrected_seed42_gpu1","settings":{"method":"corrected","seed":42,"optimizer_steps":30,"eval_interval":5,"prompts_per_step":16,"responses_per_prompt":4,"global_batch":64,"micro_batch":1,"lr":1e-06,"mode":"colocated_fully_async","physical_gpu":1},"steps":30,"actual_training_samples":1920,"actual_prompt_groups":480,"distinct_training_prompts":480,"no_multi_epoch":true,"mixed_reward_groups":122,"evaluations":{"0":0.1125,"5":0.1,"10":0.275,"15":0.275,"20":0.3875,"25":0.75,"30":1.0},"mean_age_updates":0.24166666666666667,"max_age_updates":1,"elapsed_seconds":5672.3222579956055,"status_sha256":"ad000e6385bc02c4e29fb62f33afbaadeb65eddc49e9031ad0ca37cad7fa6615","ordered_training_prompt_sha256":"e56ce1d42ce80d7f7bd2d9dd4357235ae3fdc0c579438dd6160392690ed304e5","prompt_plan_sha256":"162d30e7ff875bc31983a63dd1d64e957e246f311db4743659b07c3735b6cd0a","identical_preselected_prompt_order_and_batches_verified":true,"independently_recomputed_evaluations":{"0":{"accuracy":0.1125,"response_counts":{"4":7,"9":3,"5":13,"3":12,"1":5,"To":1,"2":11,"7":3,"8":10,"6":7,"0":6,"<think>":1,"The":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":6.0},"no_units_carry":{"samples":44,"correct":3.0}},"source_sha256":"88ba89e9874f4f1369974efb6b509821ff7403f978a58ae6e829d6e0529a2c00"},"5":{"accuracy":0.1,"response_counts":{"4":16,"3":18,"5":16,"1":15,"2":8,"6":4,"7":2,"The":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":5.0},"no_units_carry":{"samples":44,"correct":3.0}},"source_sha256":"656127e273a291d7ed2b6325fe0927cdf57129332b63e9a71b13af46f4ad7c6a"},"10":{"accuracy":0.275,"response_counts":{"3":24,"1":10,"5":15,"6":10,"2":6,"4":9,"0":5,"9":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":4.0},"no_units_carry":{"samples":44,"correct":18.0}},"source_sha256":"8891824f52a34f1f440a0ddf8f68ec2e8324cfa9d1f8d29fdc7c7f95170ce272"},"15":{"accuracy":0.275,"response_counts":{"4":10,"5":11,"1":33,"2":6,"0":3,"3":10,"6":2,"<think>":1,"7":2,"8":1,"</think>":1},"descriptive_subsets":{"units_carry":{"samples":36,"correct":13.0},"no_units_carry":{"samples":44,"correct":9.0}},"source_sha256":"c03a3f55a1ea9b965edb66e0714865515651a60fdce6a15f91eb9e18c32e3632"},"20":{"accuracy":0.3875,"response_counts":{"3":19,"8":4,"6":13,"2":12,"0":6,"1":8,"7":5,"5":4,"4":5,"9":4},"descriptive_subsets":{"units_carry":{"samples":36,"correct":10.0},"no_units_carry":{"samples":44,"correct":21.0}},"source_sha256":"2202dd7fb34a7fd0e395ba4337898196fcafd0cb5cd68fb34d509d25bdb451af"},"25":{"accuracy":0.75,"response_counts":{"5":6,"2":7,"6":9,"1":19,"0":4,"3":11,"9":5,"7":4,"4":11,"8":4},"descriptive_subsets":{"units_carry":{"samples":36,"correct":27.0},"no_units_carry":{"samples":44,"correct":33.0}},"source_sha256":"58e1e1e4c1e6b730cba3b8e84787abcba5ae25e8960509cb8a0e1f273771b6b2"},"30":{"accuracy":1.0,"response_counts":{"5":8,"2":8,"6":8,"0":8,"1":8,"9":8,"7":8,"4":8,"3":8,"8":8},"descriptive_subsets":{"units_carry":{"samples":36,"correct":36.0},"no_units_carry":{"samples":44,"correct":44.0}},"source_sha256":"9395030520bc45d2f7f83e0a9b492467991039a478c4dae1edf9823e88453569"}}}]},"timing":{"scope":"Read-only reanalysis of completed, paired seed-42, 30-update, batch-64 runs. No training run launched or changed.","methods":{"corrected":{"directory":"results/slime_exp30_batch64_paired_async/formal_corrected_seed42_gpu1","completed":true,"settings":{"method":"corrected","seed":42,"optimizer_steps":30,"eval_interval":5,"prompts_per_step":16,"responses_per_prompt":4,"global_batch":64,"micro_batch":1,"lr":1e-06,"mode":"colocated_fully_async","physical_gpu":1},"unique_update_count":30,"per_update":[{"rollout_id":0,"source_line":1795,"metrics":{"perf/update_weights_time":0.3102073669433594,"perf/data_preprocess_time":0.018757104873657227,"perf/train_wait_time":46.899585247039795,"perf/actor_train_time":135.3694794178009,"perf/train_time":135.50833702087402,"perf/actor_train_tflops":0.06120425784022414,"perf/actor_train_tok_per_s":16.42910949768746,"perf/step_time":182.40792226791382,"perf/wait_time_ratio":0.2571137517709097}},{"rollout_id":1,"source_line":1970,"metrics":{"perf/update_weights_time":0.28691864013671875,"perf/data_preprocess_time":0.017334461212158203,"perf/train_wait_time":4.056377410888672,"perf/actor_train_time":184.4942443370819,"perf/train_time":184.61970973014832,"perf/actor_train_tflops":0.04515053570514954,"perf/actor_train_tok_per_s":12.11961927611517,"perf/step_time":188.676087141037,"perf/wait_time_ratio":0.02149916013393099}},{"rollout_id":2,"source_line":2352,"metrics":{"perf/update_weights_time":0.265857458114624,"perf/data_preprocess_time":0.017935752868652344,"perf/train_wait_time":4.159173011779785,"perf/actor_train_time":185.12923097610474,"perf/train_time":185.25485610961914,"perf/actor_train_tflops":0.044753540423086394,"perf/actor_train_tok_per_s":12.013229830177705,"perf/step_time":189.41402912139893,"perf/wait_time_ratio":0.02195810432348754}},{"rollout_id":3,"source_line":2527,"metrics":{"perf/update_weights_time":0.321258544921875,"perf/data_preprocess_time":0.01920914649963379,"perf/train_wait_time":3.529902458190918,"perf/actor_train_time":183.29642844200134,"perf/train_time":183.42195010185242,"perf/actor_train_tflops":0.04536407076902241,"perf/actor_train_tok_per_s":12.176996676758758,"perf/step_time":186.95185256004333,"perf/wait_time_ratio":0.018881345169110955}},{"rollout_id":4,"source_line":2702,"metrics":{"perf/update_weights_time":0.3194267749786377,"perf/data_preprocess_time":0.019638776779174805,"perf/train_wait_time":4.355366468429565,"perf/actor_train_time":184.28750348091125,"perf/train_time":184.41306233406067,"perf/actor_train_tflops":0.04512010849619701,"perf/actor_train_tok_per_s":12.111510318610364,"perf/step_time":188.76842880249023,"perf/wait_time_ratio":0.023072536525621117}},{"rollout_id":5,"source_line":3197,"metrics":{"perf/update_weights_time":0.3151068687438965,"perf/data_preprocess_time":0.01796102523803711,"perf/train_wait_time":5.3144824504852295,"perf/actor_train_time":183.84262323379517,"perf/train_time":183.9689495563507,"perf/actor_train_tflops":0.0452292945198982,"perf/actor_train_tok_per_s":12.14081892838058,"perf/step_time":189.28343200683594,"perf/wait_time_ratio":0.028076849590794074}},{"rollout_id":6,"source_line":3581,"metrics":{"perf/update_weights_time":0.3145921230316162,"perf/data_preprocess_time":0.016116857528686523,"perf/train_wait_time":3.8026123046875,"perf/actor_train_time":183.11907505989075,"perf/train_time":183.24431705474854,"perf/actor_train_tflops":0.045163230885647165,"perf/actor_train_tok_per_s":12.123259137661814,"perf/step_time":187.04692935943604,"perf/wait_time_ratio":0.020329723228870895}},{"rollout_id":7,"source_line":3756,"metrics":{"perf/update_weights_time":0.2870500087738037,"perf/data_preprocess_time":0.016675233840942383,"perf/train_wait_time":3.535799503326416,"perf/actor_train_time":183.06758522987366,"perf/train_time":183.19324898719788,"perf/actor_train_tflops":0.0454207780209203,"perf/actor_train_tok_per_s":12.192218503332144,"perf/step_time":186.7290484905243,"perf/wait_time_ratio":0.01893545504520601}},{"rollout_id":8,"source_line":3931,"metrics":{"perf/update_weights_time":0.31887316703796387,"perf/data_preprocess_time":0.01680278778076172,"perf/train_wait_time":3.507859230041504,"perf/actor_train_time":183.14773035049438,"perf/train_time":183.27301597595215,"perf/actor_train_tflops":0.04515615176542468,"perf/actor_train_tok_per_s":12.121362332754714,"perf/step_time":186.78087520599365,"perf/wait_time_ratio":0.01878061244853263}},{"rollout_id":9,"source_line":4313,"metrics":{"perf/update_weights_time":0.3111841678619385,"perf/data_preprocess_time":0.01932549476623535,"perf/train_wait_time":3.52670955657959,"perf/actor_train_time":185.2245123386383,"perf/train_time":185.3498830795288,"perf/actor_train_tflops":0.0449725247547505,"perf/actor_train_tok_per_s":12.071836344813875,"perf/step_time":188.8765926361084,"perf/wait_time_ratio":0.01867203080783115}},{"rollout_id":10,"source_line":4602,"metrics":{"perf/update_weights_time":0.28464460372924805,"perf/data_preprocess_time":0.01812291145324707,"perf/train_wait_time":5.458685398101807,"perf/actor_train_time":186.98052835464478,"perf/train_time":187.10590982437134,"perf/actor_train_tflops":0.04431043486126821,"perf/actor_train_tok_per_s":11.894286638134606,"perf/step_time":192.56459522247314,"perf/wait_time_ratio":0.028347295055954055}},{"rollout_id":11,"source_line":4985,"metrics":{"perf/update_weights_time":0.322906494140625,"perf/data_preprocess_time":0.015005350112915039,"perf/train_wait_time":3.7182061672210693,"perf/actor_train_time":183.71820378303528,"perf/train_time":183.8437478542328,"perf/actor_train_tflops":0.04477194471464533,"perf/actor_train_tok_per_s":12.018406203271887,"perf/step_time":187.56195402145386,"perf/wait_time_ratio":0.019823882655838457}},{"rollout_id":12,"source_line":5160,"metrics":{"perf/update_weights_time":0.31694793701171875,"perf/data_preprocess_time":0.01848888397216797,"perf/train_wait_time":3.918822765350342,"perf/actor_train_time":184.6854465007782,"perf/train_time":184.8110225200653,"perf/actor_train_tflops":0.045022887883680446,"perf/actor_train_tok_per_s":12.085413562841808,"perf/step_time":188.72984528541565,"perf/wait_time_ratio":0.020764192115051632}},{"rollout_id":13,"source_line":5335,"metrics":{"perf/update_weights_time":0.31650876998901367,"perf/data_preprocess_time":0.01682114601135254,"perf/train_wait_time":4.010698318481445,"perf/actor_train_time":183.93878626823425,"perf/train_time":184.06438541412354,"perf/actor_train_tflops":0.0451244161449673,"perf/actor_train_tok_per_s":12.112725353917211,"perf/step_time":188.07508373260498,"perf/wait_time_ratio":0.021324984888395106}},{"rollout_id":14,"source_line":5718,"metrics":{"perf/update_weights_time":0.31579017639160156,"perf/data_preprocess_time":0.018648147583007812,"perf/train_wait_time":3.7070512771606445,"perf/actor_train_time":184.20265316963196,"perf/train_time":184.327867269516,"perf/actor_train_tflops":0.04505977625156366,"perf/actor_train_tok_per_s":12.095374098375434,"perf/step_time":188.03491854667664,"perf/wait_time_ratio":0.01971469610970384}},{"rollout_id":15,"source_line":6008,"metrics":{"perf/update_weights_time":0.3212137222290039,"perf/data_preprocess_time":0.019015073776245117,"perf/train_wait_time":5.405585050582886,"perf/actor_train_time":184.0994894504547,"perf/train_time":184.22597360610962,"perf/actor_train_tflops":0.04492270310953637,"perf/actor_train_tok_per_s":12.05869721109385,"perf/step_time":189.6315586566925,"perf/wait_time_ratio":0.028505724937741585}},{"rollout_id":16,"source_line":6391,"metrics":{"perf/update_weights_time":0.2957935333251953,"perf/data_preprocess_time":0.016698360443115234,"perf/train_wait_time":3.53436279296875,"perf/actor_train_time":181.75541400909424,"perf/train_time":181.88066482543945,"perf/actor_train_tflops":0.0453376615043538,"perf/actor_train_tok_per_s":12.170201432840516,"perf/step_time":185.4150276184082,"perf/wait_time_ratio":0.019061900420728653}},{"rollout_id":17,"source_line":6566,"metrics":{"perf/update_weights_time":0.3162553310394287,"perf/data_preprocess_time":0.016843795776367188,"perf/train_wait_time":3.9600396156311035,"perf/actor_train_time":182.73158073425293,"perf/train_time":182.85718393325806,"perf/actor_train_tflops":0.04566783556289645,"perf/actor_train_tok_per_s":12.25841746127966,"perf/step_time":186.81722354888916,"perf/wait_time_ratio":0.021197401076859385}},{"rollout_id":18,"source_line":6949,"metrics":{"perf/update_weights_time":0.31535983085632324,"perf/data_preprocess_time":0.02062821388244629,"perf/train_wait_time":3.907557964324951,"perf/actor_train_time":185.22711658477783,"perf/train_time":185.3524832725525,"perf/actor_train_tflops":0.04464923505252933,"perf/actor_train_tok_per_s":11.985286176950844,"perf/step_time":189.26004123687744,"perf/wait_time_ratio":0.02064650276301198}},{"rollout_id":19,"source_line":7124,"metrics":{"perf/update_weights_time":0.28644299507141113,"perf/data_preprocess_time":0.01728987693786621,"perf/train_wait_time":4.362307071685791,"perf/actor_train_time":184.37819242477417,"perf/train_time":184.50348281860352,"perf/actor_train_tflops":0.04525999344811193,"perf/actor_train_tok_per_s":12.148942185307051,"perf/step_time":188.8657898902893,"perf/wait_time_ratio":0.023097391402751244}},{"rollout_id":20,"source_line":7620,"metrics":{"perf/update_weights_time":0.3191819190979004,"perf/data_preprocess_time":0.019453763961791992,"perf/train_wait_time":5.278115749359131,"perf/actor_train_time":184.71912264823914,"perf/train_time":184.84486556053162,"perf/actor_train_tflops":0.04509556913714265,"perf/actor_train_tok_per_s":12.104864769512888,"perf/step_time":190.12298130989075,"perf/wait_time_ratio":0.027761587331497143}},{"rollout_id":21,"source_line":7795,"metrics":{"perf/update_weights_time":0.3156411647796631,"perf/data_preprocess_time":0.01641702651977539,"perf/train_wait_time":3.9089016914367676,"perf/actor_train_time":185.13548827171326,"perf/train_time":185.26108694076538,"perf/actor_train_tflops":0.044267783069509924,"perf/actor_train_tok_per_s":11.883189012206994,"perf/step_time":189.16998863220215,"perf/wait_time_ratio":0.02066343461613636}},{"rollout_id":22,"source_line":7970,"metrics":{"perf/update_weights_time":0.32396984100341797,"perf/data_preprocess_time":0.018818378448486328,"perf/train_wait_time":3.5119848251342773,"perf/actor_train_time":183.384343624115,"perf/train_time":183.51004552841187,"perf/actor_train_tflops":0.0450164105030525,"perf/actor_train_tok_per_s":12.083910524783734,"perf/step_time":187.02203035354614,"perf/wait_time_ratio":0.018778455235969937}},{"rollout_id":23,"source_line":8353,"metrics":{"perf/update_weights_time":0.3146684169769287,"perf/data_preprocess_time":0.016664505004882812,"perf/train_wait_time":4.255553722381592,"perf/actor_train_time":182.64191055297852,"perf/train_time":182.76714253425598,"perf/actor_train_tflops":0.04552663803382667,"perf/actor_train_tok_per_s":12.220634317951733,"perf/step_time":187.02269625663757,"perf/wait_time_ratio":0.022754210090854463}},{"rollout_id":24,"source_line":8528,"metrics":{"perf/update_weights_time":0.3153655529022217,"perf/data_preprocess_time":0.02028965950012207,"perf/train_wait_time":4.012247562408447,"perf/actor_train_time":184.02745532989502,"perf/train_time":184.15315294265747,"perf/actor_train_tflops":0.04526506086498497,"perf/actor_train_tok_per_s":12.150360912134857,"perf/step_time":188.16540050506592,"perf/wait_time_ratio":0.0213229826080615}},{"rollout_id":25,"source_line":9024,"metrics":{"perf/update_weights_time":0.3142838478088379,"perf/data_preprocess_time":0.016884326934814453,"perf/train_wait_time":4.6175737380981445,"perf/actor_train_time":183.796315908432,"perf/train_time":183.9221489429474,"perf/actor_train_tflops":0.0452406900021576,"perf/actor_train_tok_per_s":12.143877797376474,"perf/step_time":188.53972268104553,"perf/wait_time_ratio":0.024491251352425816}},{"rollout_id":26,"source_line":9199,"metrics":{"perf/update_weights_time":0.3051571846008301,"perf/data_preprocess_time":0.017362117767333984,"perf/train_wait_time":3.8620662689208984,"perf/actor_train_time":183.62255549430847,"perf/train_time":183.74914240837097,"perf/actor_train_tflops":0.04544624574391493,"perf/actor_train_tok_per_s":12.198937074859687,"perf/step_time":187.61120867729187,"perf/wait_time_ratio":0.020585477254528004}},{"rollout_id":27,"source_line":9374,"metrics":{"perf/update_weights_time":0.3170592784881592,"perf/data_preprocess_time":0.019604921340942383,"perf/train_wait_time":4.006107807159424,"perf/actor_train_time":184.7015516757965,"perf/train_time":184.82751655578613,"perf/actor_train_tflops":0.045018962082935314,"perf/actor_train_tok_per_s":12.084359767143654,"perf/step_time":188.83362436294556,"perf/wait_time_ratio":0.021215013060701143}},{"rollout_id":28,"source_line":9757,"metrics":{"perf/update_weights_time":0.3159670829772949,"perf/data_preprocess_time":0.019766569137573242,"perf/train_wait_time":3.967775344848633,"perf/actor_train_time":185.6933250427246,"perf/train_time":185.81869435310364,"perf/actor_train_tflops":0.04461758935103204,"perf/actor_train_tok_per_s":11.976736371586316,"perf/step_time":189.78646969795227,"perf/wait_time_ratio":0.020906523795734232}},{"rollout_id":29,"source_line":9911,"metrics":{"perf/update_weights_time":0.31656551361083984,"perf/data_preprocess_time":0.015501976013183594,"perf/train_wait_time":3.9740257263183594,"perf/actor_train_time":183.91569304466248,"perf/train_time":184.0418825149536,"perf/actor_train_tflops":0.045048852243772405,"perf/actor_train_tok_per_s":12.092497182716862,"perf/step_time":188.01590824127197,"perf/wait_time_ratio":0.021136646167294944}}],"totals_seconds":{"perf/actor_train_time":5474.333585739136,"perf/train_time":5478.115729570389,"perf/train_wait_time":166.06553649902344,"perf/step_time":5644.181266069412,"perf/update_weights_time":9.300198316574097,"perf/data_preprocess_time":0.5380816459655762},"training_minutes":91.23889309565226,"training_mean_seconds_per_update":182.47778619130452,"end_to_end_seconds":5672.3222579956055,"end_to_end_minutes":94.53870429992676,"loss_hook_only_seconds":2159.6314411871135,"loss_hook_records":1920,"first_perf_line":1795,"last_perf_line":9911,"wall_status_line":54,"relative_to_sc":{"training_ratio":1.3433897563955126,"training_increase_percent":34.338975639551265,"end_to_end_ratio":1.32550198946431,"end_to_end_increase_percent":32.55019894643101}},"grpo":{"directory":"results/slime_exp30_batch64_paired_async/formal_grpo_seed42_gpu0","completed":true,"settings":{"method":"grpo","seed":42,"optimizer_steps":30,"eval_interval":5,"prompts_per_step":16,"responses_per_prompt":4,"global_batch":64,"micro_batch":1,"lr":1e-06,"mode":"colocated_fully_async","physical_gpu":0},"unique_update_count":30,"per_update":[{"rollout_id":0,"source_line":1796,"metrics":{"perf/update_weights_time":0.29906225204467773,"perf/data_preprocess_time":0.018031835556030273,"perf/train_wait_time":46.04880213737488,"perf/actor_train_time":136.81516313552856,"perf/train_time":136.954580783844,"perf/actor_train_tflops":0.06055753128603681,"perf/actor_train_tok_per_s":16.255508154435443,"perf/step_time":183.00338292121887,"perf/wait_time_ratio":0.2516281469900391}},{"rollout_id":1,"source_line":1971,"metrics":{"perf/update_weights_time":0.2853384017944336,"perf/data_preprocess_time":0.016937732696533203,"perf/train_wait_time":3.66998028755188,"perf/actor_train_time":137.8663296699524,"perf/train_time":137.99190163612366,"perf/actor_train_tflops":0.06042094531912026,"perf/actor_train_tok_per_s":16.218608309606218,"perf/step_time":141.66188192367554,"perf/wait_time_ratio":0.025906618193376738}},{"rollout_id":2,"source_line":2353,"metrics":{"perf/update_weights_time":0.31566548347473145,"perf/data_preprocess_time":0.019034862518310547,"perf/train_wait_time":3.5649478435516357,"perf/actor_train_time":141.68819642066956,"perf/train_time":141.81359577178955,"perf/actor_train_tflops":0.05847479699286618,"perf/actor_train_tok_per_s":15.696438067409556,"perf/step_time":145.3785436153412,"perf/wait_time_ratio":0.02452182939034094}},{"rollout_id":3,"source_line":2528,"metrics":{"perf/update_weights_time":0.3274087905883789,"perf/data_preprocess_time":0.01793670654296875,"perf/train_wait_time":3.765528678894043,"perf/actor_train_time":158.3401243686676,"perf/train_time":158.4657850265503,"perf/actor_train_tflops":0.052513992803187345,"perf/actor_train_tok_per_s":14.096237507071637,"perf/step_time":162.23131370544434,"perf/wait_time_ratio":0.023210862273672604}},{"rollout_id":4,"source_line":2703,"metrics":{"perf/update_weights_time":0.32135725021362305,"perf/data_preprocess_time":0.018541574478149414,"perf/train_wait_time":4.200732946395874,"perf/actor_train_time":146.0283055305481,"perf/train_time":146.1542956829071,"perf/actor_train_tflops":0.05694150953366055,"perf/actor_train_tok_per_s":15.284707933100554,"perf/step_time":150.35502862930298,"perf/wait_time_ratio":0.027938759246640753}},{"rollout_id":5,"source_line":3199,"metrics":{"perf/update_weights_time":0.3232450485229492,"perf/data_preprocess_time":0.017786502838134766,"perf/train_wait_time":5.478676080703735,"perf/actor_train_time":136.61733770370483,"perf/train_time":136.7426142692566,"perf/actor_train_tflops":0.06086395981149696,"perf/actor_train_tok_per_s":16.337604271287685,"perf/step_time":142.22129034996033,"perf/wait_time_ratio":0.03852219359859903}},{"rollout_id":6,"source_line":3583,"metrics":{"perf/update_weights_time":0.3182816505432129,"perf/data_preprocess_time":0.01627063751220703,"perf/train_wait_time":3.879107713699341,"perf/actor_train_time":137.28509950637817,"perf/train_time":137.41091585159302,"perf/actor_train_tflops":0.0602414180142818,"perf/actor_train_tok_per_s":16.17072798127564,"perf/step_time":141.29002356529236,"perf/wait_time_ratio":0.027454930049655938}},{"rollout_id":7,"source_line":3758,"metrics":{"perf/update_weights_time":0.3248305320739746,"perf/data_preprocess_time":0.014873981475830078,"perf/train_wait_time":4.231449127197266,"perf/actor_train_time":136.35519123077393,"perf/train_time":136.48087286949158,"perf/actor_train_tflops":0.060980972389083316,"perf/actor_train_tok_per_s":16.369013748970204,"perf/step_time":140.71232199668884,"perf/wait_time_ratio":0.030071631731703196}},{"rollout_id":8,"source_line":3933,"metrics":{"perf/update_weights_time":0.3192462921142578,"perf/data_preprocess_time":0.01830315589904785,"perf/train_wait_time":4.081442356109619,"perf/actor_train_time":138.28778839111328,"perf/train_time":138.41397356987,"perf/actor_train_tflops":0.05980460605682422,"perf/actor_train_tok_per_s":16.053478227023717,"perf/step_time":142.49541592597961,"perf/wait_time_ratio":0.028642622147436358}},{"rollout_id":9,"source_line":4315,"metrics":{"perf/update_weights_time":0.3175637722015381,"perf/data_preprocess_time":0.01799154281616211,"perf/train_wait_time":4.1476380825042725,"perf/actor_train_time":136.42008304595947,"perf/train_time":136.5456576347351,"perf/actor_train_tflops":0.061061493149286876,"perf/actor_train_tok_per_s":16.39054859134413,"perf/step_time":140.69329571723938,"perf/wait_time_ratio":0.029479998043688272}},{"rollout_id":10,"source_line":4604,"metrics":{"perf/update_weights_time":0.31899118423461914,"perf/data_preprocess_time":0.017649412155151367,"perf/train_wait_time":5.0845019817352295,"perf/actor_train_time":135.3202874660492,"perf/train_time":135.44629955291748,"perf/actor_train_tflops":0.06122650695715297,"perf/actor_train_tok_per_s":16.43508184652641,"perf/step_time":140.5308015346527,"perf/wait_time_ratio":0.03618069438308491}},{"rollout_id":11,"source_line":4987,"metrics":{"perf/update_weights_time":0.3169286251068115,"perf/data_preprocess_time":0.018646717071533203,"perf/train_wait_time":4.145098686218262,"perf/actor_train_time":134.87717080116272,"perf/train_time":135.00236558914185,"perf/actor_train_tflops":0.06098453291976297,"perf/actor_train_tok_per_s":16.370450142782545,"perf/step_time":139.1474642753601,"perf/wait_time_ratio":0.02978925061843377}},{"rollout_id":12,"source_line":5162,"metrics":{"perf/update_weights_time":0.32369446754455566,"perf/data_preprocess_time":0.018352270126342773,"perf/train_wait_time":3.592799425125122,"perf/actor_train_time":135.64077234268188,"perf/train_time":135.76582884788513,"perf/actor_train_tflops":0.06130215869417833,"perf/actor_train_tok_per_s":16.455229216485815,"perf/step_time":139.35862827301025,"perf/wait_time_ratio":0.025780961463589144}},{"rollout_id":13,"source_line":5337,"metrics":{"perf/update_weights_time":0.32100510597229004,"perf/data_preprocess_time":0.019391775131225586,"perf/train_wait_time":3.78987193107605,"perf/actor_train_time":136.09078121185303,"perf/train_time":136.21618604660034,"perf/actor_train_tflops":0.06098965898246374,"perf/actor_train_tok_per_s":16.37142486919569,"perf/step_time":140.0060579776764,"perf/wait_time_ratio":0.027069342468597574}},{"rollout_id":14,"source_line":5719,"metrics":{"perf/update_weights_time":0.3163566589355469,"perf/data_preprocess_time":0.019656658172607422,"perf/train_wait_time":3.3789961338043213,"perf/actor_train_time":135.23262548446655,"perf/train_time":135.35788202285767,"perf/actor_train_tflops":0.061376685596638,"perf/actor_train_tok_per_s":16.475314237361445,"perf/step_time":138.736878156662,"perf/wait_time_ratio":0.024355428626473427}},{"rollout_id":15,"source_line":6007,"metrics":{"perf/update_weights_time":0.31895017623901367,"perf/data_preprocess_time":0.01830911636352539,"perf/train_wait_time":5.194434642791748,"perf/actor_train_time":136.00884342193604,"perf/train_time":136.13418912887573,"perf/actor_train_tflops":0.060806683588533056,"perf/actor_train_tok_per_s":16.32246804064764,"perf/step_time":141.32862377166748,"perf/wait_time_ratio":0.03675430004316712}},{"rollout_id":16,"source_line":6390,"metrics":{"perf/update_weights_time":0.3189973831176758,"perf/data_preprocess_time":0.018761157989501953,"perf/train_wait_time":3.4057774543762207,"perf/actor_train_time":135.28623604774475,"perf/train_time":135.41148972511292,"perf/actor_train_tflops":0.06091059724671354,"perf/actor_train_tok_per_s":16.35051772169453,"perf/step_time":138.81726717948914,"perf/wait_time_ratio":0.024534249402652406}},{"rollout_id":17,"source_line":6565,"metrics":{"perf/update_weights_time":0.31664443016052246,"perf/data_preprocess_time":0.016855239868164062,"perf/train_wait_time":4.178207159042358,"perf/actor_train_time":135.1503622531891,"perf/train_time":135.27556610107422,"perf/actor_train_tflops":0.061745715231503844,"perf/actor_train_tok_per_s":16.574132415595088,"perf/step_time":139.45377326011658,"perf/wait_time_ratio":0.02996123418796955}},{"rollout_id":18,"source_line":6949,"metrics":{"perf/update_weights_time":0.32299160957336426,"perf/data_preprocess_time":0.016086816787719727,"perf/train_wait_time":4.238439559936523,"perf/actor_train_time":135.36909770965576,"perf/train_time":135.49525260925293,"perf/actor_train_tflops":0.06109406952120129,"perf/actor_train_tok_per_s":16.399606982396612,"perf/step_time":139.73369216918945,"perf/wait_time_ratio":0.030332266285532795}},{"rollout_id":19,"source_line":7124,"metrics":{"perf/update_weights_time":0.317821741104126,"perf/data_preprocess_time":0.01988840103149414,"perf/train_wait_time":4.103758811950684,"perf/actor_train_time":135.34608435630798,"perf/train_time":135.47128582000732,"perf/actor_train_tflops":0.0616564256055707,"perf/actor_train_tok_per_s":16.55016479163922,"perf/step_time":139.575044631958,"perf/wait_time_ratio":0.029401809061009324}},{"rollout_id":20,"source_line":7620,"metrics":{"perf/update_weights_time":0.31937217712402344,"perf/data_preprocess_time":0.018927574157714844,"perf/train_wait_time":5.129512786865234,"perf/actor_train_time":135.51418256759644,"perf/train_time":135.63950371742249,"perf/actor_train_tflops":0.06146968389954955,"perf/actor_train_tok_per_s":16.500117977575158,"perf/step_time":140.76901650428772,"perf/wait_time_ratio":0.03643921733806383}},{"rollout_id":21,"source_line":7795,"metrics":{"perf/update_weights_time":0.3185083866119385,"perf/data_preprocess_time":0.018453121185302734,"perf/train_wait_time":3.930436611175537,"perf/actor_train_time":134.06059408187866,"perf/train_time":134.18662405014038,"perf/actor_train_tflops":0.06113308455335134,"perf/actor_train_tok_per_s":16.410489712259004,"perf/step_time":138.11706066131592,"perf/wait_time_ratio":0.028457285380649438}},{"rollout_id":22,"source_line":7970,"metrics":{"perf/update_weights_time":0.3159444332122803,"perf/data_preprocess_time":0.01591944694519043,"perf/train_wait_time":3.3824942111968994,"perf/actor_train_time":134.15156626701355,"perf/train_time":134.2766079902649,"perf/actor_train_tflops":0.06153714877979693,"perf/actor_train_tok_per_s":16.51862935084412,"perf/step_time":137.6591022014618,"perf/wait_time_ratio":0.024571526016831605}},{"rollout_id":23,"source_line":8353,"metrics":{"perf/update_weights_time":0.31900453567504883,"perf/data_preprocess_time":0.016334056854248047,"perf/train_wait_time":4.283353805541992,"perf/actor_train_time":136.6436948776245,"perf/train_time":136.76931071281433,"perf/actor_train_tflops":0.06085221977493232,"perf/actor_train_tok_per_s":16.334452914193637,"perf/step_time":141.05266451835632,"perf/wait_time_ratio":0.030367053470192085}},{"rollout_id":24,"source_line":8528,"metrics":{"perf/update_weights_time":0.3197059631347656,"perf/data_preprocess_time":0.019730091094970703,"perf/train_wait_time":3.826890230178833,"perf/actor_train_time":137.2569773197174,"perf/train_time":137.38226509094238,"perf/actor_train_tflops":0.06068918410561098,"perf/actor_train_tok_per_s":16.290610821128663,"perf/step_time":141.20915532112122,"perf/wait_time_ratio":0.027100864823361985}},{"rollout_id":25,"source_line":9024,"metrics":{"perf/update_weights_time":0.31657958030700684,"perf/data_preprocess_time":0.016104936599731445,"perf/train_wait_time":5.16124153137207,"perf/actor_train_time":135.95982456207275,"perf/train_time":136.0850110054016,"perf/actor_train_tflops":0.06115830303793703,"perf/actor_train_tok_per_s":16.4166142990349,"perf/step_time":141.24625253677368,"perf/wait_time_ratio":0.036540732505652374}},{"rollout_id":26,"source_line":9199,"metrics":{"perf/update_weights_time":0.31701231002807617,"perf/data_preprocess_time":0.017823457717895508,"perf/train_wait_time":3.488511085510254,"perf/actor_train_time":135.57229256629944,"perf/train_time":135.69759821891785,"perf/actor_train_tflops":0.06155354920356631,"perf/actor_train_tok_per_s":16.52255012877771,"perf/step_time":139.1861093044281,"perf/wait_time_ratio":0.025063643943665215}},{"rollout_id":27,"source_line":9374,"metrics":{"perf/update_weights_time":0.32304811477661133,"perf/data_preprocess_time":0.017681121826171875,"perf/train_wait_time":3.8592872619628906,"perf/actor_train_time":135.67380928993225,"perf/train_time":135.79897570610046,"perf/actor_train_tflops":0.061287231449239074,"perf/actor_train_tok_per_s":16.451222322727446,"perf/step_time":139.65826296806335,"perf/wait_time_ratio":0.027633791083635496}},{"rollout_id":28,"source_line":9757,"metrics":{"perf/update_weights_time":0.32113146781921387,"perf/data_preprocess_time":0.0167691707611084,"perf/train_wait_time":3.7337958812713623,"perf/actor_train_time":134.5896155834198,"perf/train_time":134.71475481987,"perf/actor_train_tflops":0.061558898775877474,"perf/actor_train_tok_per_s":16.524306057041567,"perf/step_time":138.44855070114136,"perf/wait_time_ratio":0.02696883327678331}},{"rollout_id":29,"source_line":9911,"metrics":{"perf/update_weights_time":0.32076168060302734,"perf/data_preprocess_time":0.016039133071899414,"perf/train_wait_time":3.878145217895508,"perf/actor_train_time":135.09877800941467,"perf/train_time":135.224018573761,"perf/actor_train_tflops":0.06132691208134116,"perf/actor_train_tok_per_s":16.462028989226056,"perf/step_time":139.1021637916565,"perf/wait_time_ratio":0.02787983387306678}}],"totals_seconds":{"perf/actor_train_time":4114.547215223312,"perf/train_time":4118.325208425522,"perf/train_wait_time":164.85385966300964,"perf/step_time":4283.1790680885315,"perf/update_weights_time":9.535449504852295,"perf/data_preprocess_time":0.533088207244873},"training_minutes":68.57578692038854,"training_mean_seconds_per_update":137.15157384077708,"end_to_end_seconds":4310.957861185074,"end_to_end_minutes":71.8492976864179,"loss_hook_only_seconds":804.8571955887601,"loss_hook_records":1920,"first_perf_line":1796,"last_perf_line":9911,"wall_status_line":54,"relative_to_sc":{"training_ratio":1.0097010886468245,"training_increase_percent":0.9701088646824507,"end_to_end_ratio":1.0073798634136153,"end_to_end_increase_percent":0.7379863413615295}},"ste":{"directory":"results/slime_exp30_batch64_paired_async/formal_ste_seed42_gpu1","completed":true,"settings":{"method":"ste","seed":42,"optimizer_steps":30,"eval_interval":5,"prompts_per_step":16,"responses_per_prompt":4,"global_batch":64,"micro_batch":1,"lr":1e-06,"mode":"colocated_fully_async","physical_gpu":1},"unique_update_count":30,"per_update":[{"rollout_id":0,"source_line":1795,"metrics":{"perf/update_weights_time":0.30886101722717285,"perf/data_preprocess_time":0.01832413673400879,"perf/train_wait_time":46.069175243377686,"perf/actor_train_time":135.58811736106873,"perf/train_time":135.72729063034058,"perf/actor_train_tflops":0.061105565024704135,"perf/actor_train_tok_per_s":16.402617303679556,"perf/step_time":181.79646587371826,"perf/wait_time_ratio":0.2534107306320181}},{"rollout_id":1,"source_line":1970,"metrics":{"perf/update_weights_time":0.2919182777404785,"perf/data_preprocess_time":0.022076129913330078,"perf/train_wait_time":3.8126888275146484,"perf/actor_train_time":137.07478165626526,"perf/train_time":137.20051264762878,"perf/actor_train_tflops":0.06076985033778649,"perf/actor_train_tok_per_s":16.312263809451775,"perf/step_time":141.01320147514343,"perf/wait_time_ratio":0.027037814811875723}},{"rollout_id":2,"source_line":2352,"metrics":{"perf/update_weights_time":0.3201305866241455,"perf/data_preprocess_time":0.0189058780670166,"perf/train_wait_time":4.3687543869018555,"perf/actor_train_time":136.62206411361694,"perf/train_time":136.7472367286682,"perf/actor_train_tflops":0.06064312214675599,"perf/actor_train_tok_per_s":16.278483379891615,"perf/step_time":141.11599111557007,"perf/wait_time_ratio":0.030958606125112833}},{"rollout_id":3,"source_line":2527,"metrics":{"perf/update_weights_time":0.31639671325683594,"perf/data_preprocess_time":0.019461870193481445,"perf/train_wait_time":3.8400919437408447,"perf/actor_train_time":151.28575897216797,"perf/train_time":151.4112048149109,"perf/actor_train_tflops":0.054962689205146686,"perf/actor_train_tok_per_s":14.753536718618841,"perf/step_time":155.25129675865173,"perf/wait_time_ratio":0.024734685145403442}},{"rollout_id":4,"source_line":2703,"metrics":{"perf/update_weights_time":0.3336014747619629,"perf/data_preprocess_time":0.017937660217285156,"perf/train_wait_time":4.262743949890137,"perf/actor_train_time":138.69656467437744,"perf/train_time":138.82386207580566,"perf/actor_train_tflops":0.059951536442691085,"perf/actor_train_tok_per_s":16.092684092357594,"perf/step_time":143.0866060256958,"perf/wait_time_ratio":0.029791355517403383}},{"rollout_id":5,"source_line":3199,"metrics":{"perf/update_weights_time":0.3193635940551758,"perf/data_preprocess_time":0.018571138381958008,"perf/train_wait_time":5.733987808227539,"perf/actor_train_time":137.5138132572174,"perf/train_time":137.63936638832092,"perf/actor_train_tflops":0.06046717747546415,"perf/actor_train_tok_per_s":16.23109669590123,"perf/step_time":143.37335419654846,"perf/wait_time_ratio":0.03999339933392992}},{"rollout_id":6,"source_line":3583,"metrics":{"perf/update_weights_time":0.31746888160705566,"perf/data_preprocess_time":0.020246028900146484,"perf/train_wait_time":3.9252727031707764,"perf/actor_train_time":135.91684222221375,"perf/train_time":136.04225182533264,"perf/actor_train_tflops":0.060847860583567476,"perf/actor_train_tok_per_s":16.333516609887596,"perf/step_time":139.96752452850342,"perf/wait_time_ratio":0.028044167505237416}},{"rollout_id":7,"source_line":3758,"metrics":{"perf/update_weights_time":0.3227372169494629,"perf/data_preprocess_time":0.0179903507232666,"perf/train_wait_time":4.514197826385498,"perf/actor_train_time":136.60378861427307,"perf/train_time":136.72899770736694,"perf/actor_train_tflops":0.06086999662235721,"perf/actor_train_tok_per_s":16.33922472166917,"perf/step_time":141.24319553375244,"perf/wait_time_ratio":0.031960462302814116}},{"rollout_id":8,"source_line":3933,"metrics":{"perf/update_weights_time":0.3164246082305908,"perf/data_preprocess_time":0.018644332885742188,"perf/train_wait_time":3.8710830211639404,"perf/actor_train_time":137.4173505306244,"perf/train_time":137.5423972606659,"perf/actor_train_tflops":0.06018342425658192,"perf/actor_train_tok_per_s":16.155165206050583,"perf/step_time":141.41348028182983,"perf/wait_time_ratio":0.02737421505678999}},{"rollout_id":9,"source_line":4315,"metrics":{"perf/update_weights_time":0.317737340927124,"perf/data_preprocess_time":0.017351627349853516,"perf/train_wait_time":4.014739990234375,"perf/actor_train_time":136.56204080581665,"perf/train_time":136.68723368644714,"perf/actor_train_tflops":0.06099801904821268,"perf/actor_train_tok_per_s":16.373510433836174,"perf/step_time":140.70197367668152,"perf/wait_time_ratio":0.02853364373878528}},{"rollout_id":10,"source_line":4604,"metrics":{"perf/update_weights_time":0.318988561630249,"perf/data_preprocess_time":0.01774883270263672,"perf/train_wait_time":4.888437747955322,"perf/actor_train_time":133.79146194458008,"perf/train_time":133.916574716568,"perf/actor_train_tflops":0.061926137898216126,"perf/actor_train_tok_per_s":16.622884358055963,"perf/step_time":138.80501246452332,"perf/wait_time_ratio":0.03521802030891889}},{"rollout_id":11,"source_line":4987,"metrics":{"perf/update_weights_time":0.31603169441223145,"perf/data_preprocess_time":0.019011497497558594,"perf/train_wait_time":3.6217806339263916,"perf/actor_train_time":133.99181199073792,"perf/train_time":134.11685228347778,"perf/actor_train_tflops":0.06138749182238521,"perf/actor_train_tok_per_s":16.478618858834647,"perf/step_time":137.73863291740417,"perf/wait_time_ratio":0.026294588215480655}},{"rollout_id":12,"source_line":5162,"metrics":{"perf/update_weights_time":0.31657838821411133,"perf/data_preprocess_time":0.020550966262817383,"perf/train_wait_time":4.460143327713013,"perf/actor_train_time":136.0788505077362,"perf/train_time":136.20487475395203,"perf/actor_train_tflops":0.06110480887020192,"perf/actor_train_tok_per_s":16.402254954917545,"perf/step_time":140.66501808166504,"perf/wait_time_ratio":0.031707551660951086}},{"rollout_id":13,"source_line":5337,"metrics":{"perf/update_weights_time":0.3179593086242676,"perf/data_preprocess_time":0.01763010025024414,"perf/train_wait_time":4.3635430335998535,"perf/actor_train_time":135.3157229423523,"perf/train_time":135.44086146354675,"perf/actor_train_tflops":0.061338994141161646,"perf/actor_train_tok_per_s":16.465196738069977,"perf/step_time":139.8044044971466,"perf/wait_time_ratio":0.031211770825781908}},{"rollout_id":14,"source_line":5719,"metrics":{"perf/update_weights_time":0.31721973419189453,"perf/data_preprocess_time":0.016080617904663086,"perf/train_wait_time":4.114533424377441,"perf/actor_train_time":134.82874631881714,"perf/train_time":134.9540741443634,"perf/actor_train_tflops":0.06156053930177059,"perf/actor_train_tok_per_s":16.524665999130878,"perf/step_time":139.06860756874084,"perf/wait_time_ratio":0.029586356664595567}},{"rollout_id":15,"source_line":6006,"metrics":{"perf/update_weights_time":0.31384849548339844,"perf/data_preprocess_time":0.01844620704650879,"perf/train_wait_time":5.589852333068848,"perf/actor_train_time":134.30819153785706,"perf/train_time":134.43336939811707,"perf/actor_train_tflops":0.06157663663328301,"perf/actor_train_tok_per_s":16.52914818210664,"perf/step_time":140.0232217311859,"perf/wait_time_ratio":0.0399208950055452}},{"rollout_id":16,"source_line":6389,"metrics":{"perf/update_weights_time":0.31946611404418945,"perf/data_preprocess_time":0.01822209358215332,"perf/train_wait_time":4.172024250030518,"perf/actor_train_time":134.96334528923035,"perf/train_time":135.08867859840393,"perf/actor_train_tflops":0.06105632176846727,"perf/actor_train_tok_per_s":16.38963523955056,"perf/step_time":139.26070284843445,"perf/wait_time_ratio":0.029958374219690498}},{"rollout_id":17,"source_line":6564,"metrics":{"perf/update_weights_time":0.3164103031158447,"perf/data_preprocess_time":0.017041444778442383,"perf/train_wait_time":4.409390687942505,"perf/actor_train_time":133.95269775390625,"perf/train_time":134.07810378074646,"perf/actor_train_tflops":0.06229778064232117,"perf/actor_train_tok_per_s":16.722320920443565,"perf/step_time":138.48749446868896,"perf/wait_time_ratio":0.03183963075409265}},{"rollout_id":18,"source_line":6947,"metrics":{"perf/update_weights_time":0.3201899528503418,"perf/data_preprocess_time":0.018271684646606445,"perf/train_wait_time":4.516881465911865,"perf/actor_train_time":134.13910174369812,"perf/train_time":134.264164686203,"perf/actor_train_tflops":0.06165427499505779,"perf/actor_train_tok_per_s":16.5499840921985,"perf/step_time":138.78104615211487,"perf/wait_time_ratio":0.03254681810771919}},{"rollout_id":19,"source_line":7122,"metrics":{"perf/update_weights_time":0.3229255676269531,"perf/data_preprocess_time":0.019150495529174805,"perf/train_wait_time":4.288851499557495,"perf/actor_train_time":134.2130892276764,"perf/train_time":134.3392252922058,"perf/actor_train_tflops":0.06217691455535893,"perf/actor_train_tok_per_s":16.689877365091483,"perf/step_time":138.6280767917633,"perf/wait_time_ratio":0.03093782730607946}},{"rollout_id":20,"source_line":7617,"metrics":{"perf/update_weights_time":0.31348085403442383,"perf/data_preprocess_time":0.014957189559936523,"perf/train_wait_time":5.218696594238281,"perf/actor_train_time":133.8834855556488,"perf/train_time":134.00913763046265,"perf/actor_train_tflops":0.06221838288542034,"perf/actor_train_tok_per_s":16.70108894103003,"perf/step_time":139.22783422470093,"perf/wait_time_ratio":0.03748314137973147}},{"rollout_id":21,"source_line":7792,"metrics":{"perf/update_weights_time":0.3149576187133789,"perf/data_preprocess_time":0.017444849014282227,"perf/train_wait_time":4.5377197265625,"perf/actor_train_time":133.7935311794281,"perf/train_time":133.9185118675232,"perf/actor_train_tflops":0.061255111222747474,"perf/actor_train_tok_per_s":16.443246400677022,"perf/step_time":138.4562315940857,"perf/wait_time_ratio":0.03277367637641478}},{"rollout_id":22,"source_line":7967,"metrics":{"perf/update_weights_time":0.3161301612854004,"perf/data_preprocess_time":0.018236398696899414,"perf/train_wait_time":3.6180388927459717,"perf/actor_train_time":135.10261797904968,"perf/train_time":135.2278962135315,"perf/actor_train_tflops":0.06110395946358454,"perf/actor_train_tok_per_s":16.402346846777125,"perf/step_time":138.84593510627747,"perf/wait_time_ratio":0.026057938894477536}},{"rollout_id":23,"source_line":8350,"metrics":{"perf/update_weights_time":0.31651949882507324,"perf/data_preprocess_time":0.020054101943969727,"perf/train_wait_time":4.304976940155029,"perf/actor_train_time":135.5441870689392,"perf/train_time":135.6696457862854,"perf/actor_train_tflops":0.06134584102321457,"perf/actor_train_tok_per_s":16.466954786226143,"perf/step_time":139.97462272644043,"perf/wait_time_ratio":0.030755410204380162}},{"rollout_id":24,"source_line":8525,"metrics":{"perf/update_weights_time":0.3217952251434326,"perf/data_preprocess_time":0.015087604522705078,"perf/train_wait_time":3.9452619552612305,"perf/actor_train_time":134.98158979415894,"perf/train_time":135.10676288604736,"perf/actor_train_tflops":0.061712222970842986,"perf/actor_train_tok_per_s":16.56522199367931,"perf/step_time":139.0520248413086,"perf/wait_time_ratio":0.0283725602684586}},{"rollout_id":25,"source_line":9021,"metrics":{"perf/update_weights_time":0.3159477710723877,"perf/data_preprocess_time":0.019270896911621094,"perf/train_wait_time":5.4388158321380615,"perf/actor_train_time":134.0355749130249,"perf/train_time":134.1608989238739,"perf/actor_train_tflops":0.06203630757690721,"perf/actor_train_tok_per_s":16.6522954928073,"perf/step_time":139.59971475601196,"perf/wait_time_ratio":0.03896007840448567}},{"rollout_id":26,"source_line":9196,"metrics":{"perf/update_weights_time":0.31706953048706055,"perf/data_preprocess_time":0.018871545791625977,"perf/train_wait_time":4.4181764125823975,"perf/actor_train_time":134.7588849067688,"perf/train_time":134.883953332901,"perf/actor_train_tflops":0.06192508781067274,"perf/actor_train_tok_per_s":16.622280612886602,"perf/step_time":139.3021297454834,"perf/wait_time_ratio":0.031716502975616914}},{"rollout_id":27,"source_line":9371,"metrics":{"perf/update_weights_time":0.31658220291137695,"perf/data_preprocess_time":0.018037796020507812,"perf/train_wait_time":3.8010082244873047,"perf/actor_train_time":135.34629201889038,"perf/train_time":135.4714343547821,"perf/actor_train_tflops":0.06143553715081799,"perf/actor_train_tok_per_s":16.49103175791826,"perf/step_time":139.2724425792694,"perf/wait_time_ratio":0.02729189029856996}},{"rollout_id":28,"source_line":9754,"metrics":{"perf/update_weights_time":0.31446242332458496,"perf/data_preprocess_time":0.02050471305847168,"perf/train_wait_time":4.0768983364105225,"perf/actor_train_time":133.98947072029114,"perf/train_time":134.11441111564636,"perf/actor_train_tflops":0.06183462385100164,"perf/actor_train_tok_per_s":16.598319166755253,"perf/step_time":138.19130945205688,"perf/wait_time_ratio":0.02950184315190191}},{"rollout_id":29,"source_line":9908,"metrics":{"perf/update_weights_time":0.3162727355957031,"perf/data_preprocess_time":0.020314693450927734,"perf/train_wait_time":4.402186632156372,"perf/actor_train_time":134.71535658836365,"perf/train_time":134.84078907966614,"perf/actor_train_tflops":0.06150145826801495,"perf/actor_train_tok_per_s":16.508882552979138,"perf/step_time":139.2429757118225,"perf/wait_time_ratio":0.031615143310834905}}],"totals_seconds":{"perf/actor_train_time":4075.015132188797,"perf/train_time":4078.7905740737915,"perf/train_wait_time":172.59995365142822,"perf/step_time":4251.39052772522,"perf/update_weights_time":9.507475852966309,"perf/data_preprocess_time":0.5544428825378418},"training_minutes":67.91691886981329,"training_mean_seconds_per_update":135.83383773962657,"end_to_end_seconds":4279.37664604187,"end_to_end_minutes":71.32294410069784,"loss_hook_only_seconds":799.2906225034967,"loss_hook_records":1920,"first_perf_line":1795,"last_perf_line":9908,"wall_status_line":54,"relative_to_sc":{"training_ratio":1.0,"training_increase_percent":0.0,"end_to_end_ratio":1.0,"end_to_end_increase_percent":0.0}}},"formulas":{"training":"sum(perf/actor_train_time for exactly one actor record for each rollout_id 0..29)","training_mean":"training / 30","end_to_end":"run_status.json elapsed_seconds","increase_percent":"100 * (method / ste - 1)","step_time":"perf/train_time + perf/train_wait_time; a framework-derived quantity, not the top-level elapsed_seconds"},"timer_definition":{"actor_train":{"file":"vendor/slime/slime/backends/megatron_utils/actor.py","line":540,"scope":"Wall-clock time of model.train(...) block, including before-step reference hook (and first-call lazy ReferenceState initialization), all 64 forward/backward microbatches, correction/reference diagnostics, optimizer and scheduler update, and training logging. Excludes rollout generation/evaluation and main engine/model initialization as directly called stages, but concurrent worker contention is part of elapsed time."},"train":{"file":"vendor/slime/slime/backends/megatron_utils/actor.py","line":445,"scope":"Outer timer additionally includes advantage/return computation, rollout-data postprocess and logging."},"optimizer_and_backward":{"file":"vendor/slime/slime/backends/megatron_utils/model.py","lines":[557,647,679,826]},"clock":{"file":"vendor/slime/slime/observability/timer.py","lines":[20,26],"scope":"time.time elapsed, not isolated CUDA-event kernel timing. No explicit device synchronize in timer itself; training metric .item() calls and reference diagnostics perform synchronization during block."},"perf_reset":{"file":"vendor/slime/slime/observability/train_metric_utils.py","lines":[365,376,395],"scope":"Reports timer values then resets each training update; perf/step_time adds train_wait to train."},"end_to_end":{"file":"experiments/slime_exp30_paired_train.py","lines":[30,45,47,50],"scope":"Starts before ray.init, includes training/rollout model init, rollout, weight transfers, evaluation, training and diagnostics; final time recorded after train(args) returns and before ray.shutdown. CLI parse and imports precede clock."},"loss_hook":{"file":"pg_correction/slime_reference.py","lines":[130,132,250,255],"scope":"diagnostics.jsonl seconds times loss hook only; misses preceding native forward and later backward/optimizer. GRPO original policy_loss_function also occurs after diagnostic timer. Cannot label this train time."}},"limitations":["All values come from one paired seed and 30 updates. This is an instrumented study, not production throughput.","Actor training times include expensive reference/gradient audits and hooks; they do not isolate incremental HVP cost.","Async rollout can overlap actor training. Do not add worker rollout elapsed to actor elapsed to estimate end-to-end runtime.","Neither a pure forward/backward/optimizer breakdown nor diagnostic-free runtime is recorded in these logs.","Core timer/model/actor source files are clean in the vendored Git checkout, but were not individually listed in the run_status source manifest. Their current hashes are captured here; experiment launcher, hook, and vendor/train.py hashes match saved run manifests."],"source_sha256":{"results/slime_exp30_batch64_paired_async/formal_corrected_seed42_gpu1/run.log":"860130c355aa4a28ed97f8da189e30581a7b69bd76afa45ff49b96133c9e7ac9","results/slime_exp30_batch64_paired_async/formal_corrected_seed42_gpu1/run_status.json":"ad000e6385bc02c4e29fb62f33afbaadeb65eddc49e9031ad0ca37cad7fa6615","results/slime_exp30_batch64_paired_async/formal_corrected_seed42_gpu1/diagnostics.jsonl":"9a3655689ee8d99196bc9d3285296f25ee906ecb23af06b1c868a8e72fc3f14b","results/slime_exp30_batch64_paired_async/formal_corrected_seed42_gpu1/async_event_loop.json":"9dcaadb0adf2df27e4ca642b889bf87d5970b74f940de0d863edefc5a7243349","results/slime_exp30_batch64_paired_async/formal_grpo_seed42_gpu0/run.log":"c831f8f6bb99d83a2e1b2f8a1f9bc13570d7ebbc01e500f38d74153375b94c94","results/slime_exp30_batch64_paired_async/formal_grpo_seed42_gpu0/run_status.json":"f92a9e237962dd8500136d41dc39c64189b88ed4900d0c6de0ee96bc29a0ee7d","results/slime_exp30_batch64_paired_async/formal_grpo_seed42_gpu0/diagnostics.jsonl":"56728d34c8adc089685d3d034cf9e9e6c6633fe3e91a2eade7ccce562077f1f5","results/slime_exp30_batch64_paired_async/formal_grpo_seed42_gpu0/async_event_loop.json":"9dcaadb0adf2df27e4ca642b889bf87d5970b74f940de0d863edefc5a7243349","results/slime_exp30_batch64_paired_async/formal_ste_seed42_gpu1/run.log":"9ac70ca3fb6e803a281e05939878bb7eaca7e49cfcedf8b5eb07e81ebf1971cd","results/slime_exp30_batch64_paired_async/formal_ste_seed42_gpu1/run_status.json":"9ba90b6abb5fc91a502f670ba51159d5c5e4a7fe8eae939904f1596ace6d87f2","results/slime_exp30_batch64_paired_async/formal_ste_seed42_gpu1/diagnostics.jsonl":"9d62f44235351f468f45d0d4a8954904d321b29b50ca31ad0c10f2ffca34f474","results/slime_exp30_batch64_paired_async/formal_ste_seed42_gpu1/async_event_loop.json":"9dcaadb0adf2df27e4ca642b889bf87d5970b74f940de0d863edefc5a7243349","vendor/slime/slime/observability/timer.py":"8154c9a5a89673bd250dbfbbcca4a661c6c0b2533291dceca82135da5e279393","vendor/slime/slime/observability/train_metric_utils.py":"39eddc0e9ebd0448906e5c7a33e4011434753005dbe17e211326ed5cb5c4c612","vendor/slime/slime/backends/megatron_utils/actor.py":"6c25c72c7e8d8b9368f59d95884a0b161571dce9d15b203748c1c28db8b22860","vendor/slime/slime/backends/megatron_utils/model.py":"2d9c16dfe1759c43955c9763e4e446aed708fe305aa01ea1b049da4e7f38a5fa","vendor/slime/train.py":"d29b5b892e17e6d27f0a643b04e6cd41ba3fe7d0397f5928d860776316e00749","experiments/slime_exp30_paired_train.py":"9ea4614b5abd594ebea1451b2558d315a52cf1547d34c08e16ed22e91ccacebf","pg_correction/slime_reference.py":"164e0c93ee2c63f9518d106479a63bafb5343376c8574f8d75974c4e85db5c31","pg_correction/async_batch64.py":"84948d97c71c134247e91ec20c684ceca2c3aff64f73aa15839f5066cf842257","independent/qwen_fullgrad_20261011/run.py":"7170a247c101c94de8ac4831885e6f2efc4ac1e4612b80620dfed697a993bdfc","independent/qwen_fullgrad_20261011/REPORT.md":"8051b931f56402654334ea91609a8fdbe17f05f195f29379011a7406f6b7f9e8"},"run_manifest_matches":{"corrected":{"vendor/slime/train.py":{"recorded_sha256":"d29b5b892e17e6d27f0a643b04e6cd41ba3fe7d0397f5928d860776316e00749","current_sha256":"d29b5b892e17e6d27f0a643b04e6cd41ba3fe7d0397f5928d860776316e00749","matches":true},"experiments/slime_exp30_paired_train.py":{"recorded_sha256":"9ea4614b5abd594ebea1451b2558d315a52cf1547d34c08e16ed22e91ccacebf","current_sha256":"9ea4614b5abd594ebea1451b2558d315a52cf1547d34c08e16ed22e91ccacebf","matches":true},"pg_correction/slime_reference.py":{"recorded_sha256":"164e0c93ee2c63f9518d106479a63bafb5343376c8574f8d75974c4e85db5c31","current_sha256":"164e0c93ee2c63f9518d106479a63bafb5343376c8574f8d75974c4e85db5c31","matches":true},"pg_correction/async_batch64.py":{"recorded_sha256":"84948d97c71c134247e91ec20c684ceca2c3aff64f73aa15839f5066cf842257","current_sha256":"84948d97c71c134247e91ec20c684ceca2c3aff64f73aa15839f5066cf842257","matches":true}},"grpo":{"vendor/slime/train.py":{"recorded_sha256":"d29b5b892e17e6d27f0a643b04e6cd41ba3fe7d0397f5928d860776316e00749","current_sha256":"d29b5b892e17e6d27f0a643b04e6cd41ba3fe7d0397f5928d860776316e00749","matches":true},"experiments/slime_exp30_paired_train.py":{"recorded_sha256":"9ea4614b5abd594ebea1451b2558d315a52cf1547d34c08e16ed22e91ccacebf","current_sha256":"9ea4614b5abd594ebea1451b2558d315a52cf1547d34c08e16ed22e91ccacebf","matches":true},"pg_correction/slime_reference.py":{"recorded_sha256":"164e0c93ee2c63f9518d106479a63bafb5343376c8574f8d75974c4e85db5c31","current_sha256":"164e0c93ee2c63f9518d106479a63bafb5343376c8574f8d75974c4e85db5c31","matches":true},"pg_correction/async_batch64.py":{"recorded_sha256":"84948d97c71c134247e91ec20c684ceca2c3aff64f73aa15839f5066cf842257","current_sha256":"84948d97c71c134247e91ec20c684ceca2c3aff64f73aa15839f5066cf842257","matches":true}},"ste":{"vendor/slime/train.py":{"recorded_sha256":"d29b5b892e17e6d27f0a643b04e6cd41ba3fe7d0397f5928d860776316e00749","current_sha256":"d29b5b892e17e6d27f0a643b04e6cd41ba3fe7d0397f5928d860776316e00749","matches":true},"experiments/slime_exp30_paired_train.py":{"recorded_sha256":"9ea4614b5abd594ebea1451b2558d315a52cf1547d34c08e16ed22e91ccacebf","current_sha256":"9ea4614b5abd594ebea1451b2558d315a52cf1547d34c08e16ed22e91ccacebf","matches":true},"pg_correction/slime_reference.py":{"recorded_sha256":"164e0c93ee2c63f9518d106479a63bafb5343376c8574f8d75974c4e85db5c31","current_sha256":"164e0c93ee2c63f9518d106479a63bafb5343376c8574f8d75974c4e85db5c31","matches":true},"pg_correction/async_batch64.py":{"recorded_sha256":"84948d97c71c134247e91ec20c684ceca2c3aff64f73aa15839f5066cf842257","current_sha256":"84948d97c71c134247e91ec20c684ceca2c3aff64f73aa15839f5066cf842257","matches":true}}},"core_timer_sources_git_status":"clean","core_timer_sources_last_commit":"5bae5bb7928d65906e24e99f5b5d3f99a4daaf93","cpu_probe":{"source":"independent/qwen_fullgrad_20261011/REPORT.md","lines":[109,111,114],"same_recurrent_seconds":2.4697,"lag4_gradient_plus_hvp_seconds":7.5405,"ratio":3.0532048426934444,"definition":"Single-prompt CPU logit-margin full-parameter gradient basis construction; optional create_graph + one HVP in independent/qwen_fullgrad_20261011/run.py:95-104. Excludes optimizer, sampling, async scheduler and end-to-end training. Different CPU probe/configuration; cannot interpret 3.05x as native training slowdown."}},"seed42_diagnostics":{"score_transport":{"observed_unix":1791766606.986549,"target":"The negative raw action score at the behavior-policy parameters, computed by differentiating negative log probability there. This is not the current-policy score.","measurement":"L2 error of reference STE or corrected STE relative to the same behavior-score oracle; residual_fraction is corrected error divided by STE error within one observation.","selection":"The first microbatch of each update only; fresh update1 omitted. Nonzero-advantage subsets contain 1, 3, and 6 observations. These are not whole-batch gradient errors or independent seeds.","interpretation":"Within-trajectory counterfactual score comparisons; even the GRPO trajectory diagnostics compare STE and corrected STE scores, not the implemented GRPO loss gradient. Do not compare median error levels across trajectories as if their policies and sampled actions were matched, or infer that local error reduction alone causes the final accuracy difference.","source_definition":"pg_correction/slime_reference.py","source_definition_sha256":"164e0c93ee2c63f9518d106479a63bafb5343376c8574f8d75974c4e85db5c31","runs":[{"trajectory_method":"grpo","diagnostics_sha256":"56728d34c8adc089685d3d034cf9e9e6c6633fe3e91a2eade7ccce562077f1f5","all_observed_stale_scores":{"observations":29,"improved":27,"worsened_steps":[3,4],"median_residual_fraction":0.8567838226842008},"nonzero_advantage_subset":{"observations":1,"improved":1,"worsened_steps":[],"median_residual_fraction":0.8532070570925899},"observations":[{"optimizer_step":2,"nonzero_advantage":false,"ste_relative_error":3.226459673698709,"corrected_relative_error":2.2773688293165155,"residual_fraction":0.7058414050177213},{"optimizer_step":3,"nonzero_advantage":false,"ste_relative_error":1.9879134415942634,"corrected_relative_error":3.744262802357322,"residual_fraction":1.8835140021763246},{"optimizer_step":4,"nonzero_advantage":false,"ste_relative_error":0.5801172421645898,"corrected_relative_error":0.6049738829798401,"residual_fraction":1.0428476159793196},{"optimizer_step":5,"nonzero_advantage":false,"ste_relative_error":0.25303169081890686,"corrected_relative_error":0.19877562255698128,"residual_fraction":0.7855759960883465},{"optimizer_step":6,"nonzero_advantage":false,"ste_relative_error":0.14780165739015272,"corrected_relative_error":0.11880004531116604,"residual_fraction":0.8037801971162544},{"optimizer_step":7,"nonzero_advantage":false,"ste_relative_error":0.10040093910352656,"corrected_relative_error":0.08451410032937685,"residual_fraction":0.8417660341028455},{"optimizer_step":8,"nonzero_advantage":true,"ste_relative_error":0.07954674134140484,"corrected_relative_error":0.06786984108120549,"residual_fraction":0.8532070570925899},{"optimizer_step":9,"nonzero_advantage":false,"ste_relative_error":0.5613944029680142,"corrected_relative_error":0.4552606593744167,"residual_fraction":0.8109462028255302},{"optimizer_step":10,"nonzero_advantage":false,"ste_relative_error":0.4659084530248741,"corrected_relative_error":0.3753712199549566,"residual_fraction":0.8056759166267287},{"optimizer_step":11,"nonzero_advantage":false,"ste_relative_error":0.2575446667376617,"corrected_relative_error":0.21069681524693046,"residual_fraction":0.8180981494039203},{"optimizer_step":12,"nonzero_advantage":false,"ste_relative_error":0.1766466455075121,"corrected_relative_error":0.14358313772007888,"residual_fraction":0.8128268572978525},{"optimizer_step":13,"nonzero_advantage":false,"ste_relative_error":0.12553575082541038,"corrected_relative_error":0.10318394810821337,"residual_fraction":0.8219487072787502},{"optimizer_step":14,"nonzero_advantage":false,"ste_relative_error":0.09801672305153801,"corrected_relative_error":0.08139542497580161,"residual_fraction":0.8304238546416535},{"optimizer_step":15,"nonzero_advantage":false,"ste_relative_error":0.0850433613323604,"corrected_relative_error":0.07348463053425396,"residual_fraction":0.8640842669313901},{"optimizer_step":16,"nonzero_advantage":false,"ste_relative_error":0.0747948624459471,"corrected_relative_error":0.06359375415901462,"residual_fraction":0.8502422770677956},{"optimizer_step":17,"nonzero_advantage":false,"ste_relative_error":0.06269122400687899,"corrected_relative_error":0.053398872568688076,"residual_fraction":0.851775881147714},{"optimizer_step":18,"nonzero_advantage":false,"ste_relative_error":0.05351684405619323,"corrected_relative_error":0.04600843441354247,"residual_fraction":0.8597000668655488},{"optimizer_step":19,"nonzero_advantage":false,"ste_relative_error":0.047958894912232555,"corrected_relative_error":0.04078892083096541,"residual_fraction":0.8504975126222445},{"optimizer_step":20,"nonzero_advantage":false,"ste_relative_error":0.042668893705073783,"corrected_relative_error":0.03669854240170679,"residual_fraction":0.8600771947678347},{"optimizer_step":21,"nonzero_advantage":false,"ste_relative_error":0.03841404817947249,"corrected_relative_error":0.032912535043983504,"residual_fraction":0.8567838226842008},{"optimizer_step":22,"nonzero_advantage":false,"ste_relative_error":0.033653366693215614,"corrected_relative_error":0.029100323840042238,"residual_fraction":0.8647076563043766},{"optimizer_step":23,"nonzero_advantage":false,"ste_relative_error":0.0300465093121301,"corrected_relative_error":0.025770104706567907,"residual_fraction":0.85767382955744},{"optimizer_step":24,"nonzero_advantage":false,"ste_relative_error":0.026892941023209938,"corrected_relative_error":0.023436373632119478,"residual_fraction":0.8714693425271981},{"optimizer_step":25,"nonzero_advantage":false,"ste_relative_error":0.024176060587323085,"corrected_relative_error":0.021040522863625136,"residual_fraction":0.8703040260685774},{"optimizer_step":26,"nonzero_advantage":false,"ste_relative_error":0.021688727882016975,"corrected_relative_error":0.01883228512405515,"residual_fraction":0.8682982804016725},{"optimizer_step":27,"nonzero_advantage":false,"ste_relative_error":0.01931856545271915,"corrected_relative_error":0.01695699325226172,"residual_fraction":0.8777563372271499},{"optimizer_step":28,"nonzero_advantage":false,"ste_relative_error":0.017196349254642557,"corrected_relative_error":0.015056780667531789,"residual_fraction":0.8755800690351099},{"optimizer_step":29,"nonzero_advantage":false,"ste_relative_error":0.015672301199605908,"corrected_relative_error":0.013676553220264344,"residual_fraction":0.8726576299215237},{"optimizer_step":30,"nonzero_advantage":false,"ste_relative_error":0.014163217009413094,"corrected_relative_error":0.012513288691383207,"residual_fraction":0.8835061047971433}]},{"trajectory_method":"ste","diagnostics_sha256":"9d62f44235351f468f45d0d4a8954904d321b29b50ca31ad0c10f2ffca34f474","all_observed_stale_scores":{"observations":29,"improved":27,"worsened_steps":[3,24],"median_residual_fraction":0.8228296981720619},"nonzero_advantage_subset":{"observations":3,"improved":3,"worsened_steps":[],"median_residual_fraction":0.8056971706120469},"observations":[{"optimizer_step":2,"nonzero_advantage":false,"ste_relative_error":3.226424427856503,"corrected_relative_error":2.2773730720705845,"residual_fraction":0.7058504307145892},{"optimizer_step":3,"nonzero_advantage":false,"ste_relative_error":1.988066561015616,"corrected_relative_error":3.744569779369447,"residual_fraction":1.883523345142182},{"optimizer_step":4,"nonzero_advantage":false,"ste_relative_error":0.6492520359274855,"corrected_relative_error":0.6418145176205889,"residual_fraction":0.9885444821189174},{"optimizer_step":5,"nonzero_advantage":false,"ste_relative_error":0.32939379097375693,"corrected_relative_error":0.26008445812675923,"residual_fraction":0.7895851872553371},{"optimizer_step":6,"nonzero_advantage":false,"ste_relative_error":0.37088656332239656,"corrected_relative_error":0.307638071919998,"residual_fraction":0.8294667489816306},{"optimizer_step":7,"nonzero_advantage":false,"ste_relative_error":0.3606206849322487,"corrected_relative_error":0.29700736934103783,"residual_fraction":0.8236004803685565},{"optimizer_step":8,"nonzero_advantage":false,"ste_relative_error":0.45749576272517367,"corrected_relative_error":0.4112577529801829,"residual_fraction":0.8989323759643939},{"optimizer_step":9,"nonzero_advantage":false,"ste_relative_error":0.49987788366144625,"corrected_relative_error":0.4532957843177918,"residual_fraction":0.9068130420124701},{"optimizer_step":10,"nonzero_advantage":false,"ste_relative_error":0.590146369521112,"corrected_relative_error":0.466886956072333,"residual_fraction":0.7911375553342798},{"optimizer_step":11,"nonzero_advantage":false,"ste_relative_error":0.37337225557437465,"corrected_relative_error":0.3313092261066556,"residual_fraction":0.8873429162458478},{"optimizer_step":12,"nonzero_advantage":false,"ste_relative_error":0.6085095292108508,"corrected_relative_error":0.5593116766757114,"residual_fraction":0.9191502348386525},{"optimizer_step":13,"nonzero_advantage":true,"ste_relative_error":0.6707920632267823,"corrected_relative_error":0.5404552674108357,"residual_fraction":0.8056971706120469},{"optimizer_step":14,"nonzero_advantage":false,"ste_relative_error":0.9766728439314557,"corrected_relative_error":0.930717408130106,"residual_fraction":0.9529469503663449},{"optimizer_step":15,"nonzero_advantage":false,"ste_relative_error":0.608257869701812,"corrected_relative_error":0.578575223334849,"residual_fraction":0.9512005551502122},{"optimizer_step":16,"nonzero_advantage":false,"ste_relative_error":0.37638719819621597,"corrected_relative_error":0.2765240642896318,"residual_fraction":0.7346797808608673},{"optimizer_step":17,"nonzero_advantage":false,"ste_relative_error":0.5875680671300115,"corrected_relative_error":0.4767366230322962,"residual_fraction":0.8113725876236368},{"optimizer_step":18,"nonzero_advantage":false,"ste_relative_error":0.4408147563085893,"corrected_relative_error":0.347382814766015,"residual_fraction":0.7880471553970215},{"optimizer_step":19,"nonzero_advantage":false,"ste_relative_error":0.27747115693176294,"corrected_relative_error":0.2283115083096153,"residual_fraction":0.8228296981720619},{"optimizer_step":20,"nonzero_advantage":false,"ste_relative_error":0.15472625286712813,"corrected_relative_error":0.1153947610435294,"residual_fraction":0.7457994936555801},{"optimizer_step":21,"nonzero_advantage":false,"ste_relative_error":0.10627535041438312,"corrected_relative_error":0.07934300872174857,"residual_fraction":0.7465796011246124},{"optimizer_step":22,"nonzero_advantage":false,"ste_relative_error":0.21458116518563167,"corrected_relative_error":0.17783912652726705,"residual_fraction":0.8287732354021868},{"optimizer_step":23,"nonzero_advantage":false,"ste_relative_error":0.19644187129508,"corrected_relative_error":0.15867532045012658,"residual_fraction":0.8077469401204013},{"optimizer_step":24,"nonzero_advantage":false,"ste_relative_error":0.4296167121390734,"corrected_relative_error":0.4365366483974073,"residual_fraction":1.0161072324767801},{"optimizer_step":25,"nonzero_advantage":true,"ste_relative_error":0.45363188616516703,"corrected_relative_error":0.33376928940611017,"residual_fraction":0.735771226814918},{"optimizer_step":26,"nonzero_advantage":true,"ste_relative_error":0.7496146737918516,"corrected_relative_error":0.6569349991402686,"residual_fraction":0.8763635799940094},{"optimizer_step":27,"nonzero_advantage":false,"ste_relative_error":0.8774436612703903,"corrected_relative_error":0.7635251495791298,"residual_fraction":0.8701699986910546},{"optimizer_step":28,"nonzero_advantage":false,"ste_relative_error":0.33404917582227284,"corrected_relative_error":0.25724024440841414,"residual_fraction":0.770066993206042},{"optimizer_step":29,"nonzero_advantage":false,"ste_relative_error":0.29127102501516317,"corrected_relative_error":0.18482702343443588,"residual_fraction":0.6345534143837824},{"optimizer_step":30,"nonzero_advantage":false,"ste_relative_error":0.36903854525183566,"corrected_relative_error":0.25127169445573605,"residual_fraction":0.6808819774754578}]},{"trajectory_method":"corrected","diagnostics_sha256":"9a3655689ee8d99196bc9d3285296f25ee906ecb23af06b1c868a8e72fc3f14b","all_observed_stale_scores":{"observations":29,"improved":25,"worsened_steps":[3,4,6,17],"median_residual_fraction":0.8269024119830913},"nonzero_advantage_subset":{"observations":6,"improved":5,"worsened_steps":[17],"median_residual_fraction":0.8553122994761134},"observations":[{"optimizer_step":2,"nonzero_advantage":false,"ste_relative_error":3.226424427856503,"corrected_relative_error":2.2773730720705845,"residual_fraction":0.7058504307145892},{"optimizer_step":3,"nonzero_advantage":false,"ste_relative_error":1.988066561015616,"corrected_relative_error":3.744569779369447,"residual_fraction":1.883523345142182},{"optimizer_step":4,"nonzero_advantage":false,"ste_relative_error":0.8218226705315501,"corrected_relative_error":1.0324419848619955,"residual_fraction":1.2562831640969674},{"optimizer_step":5,"nonzero_advantage":false,"ste_relative_error":2.250603939020309,"corrected_relative_error":1.7477634907244437,"residual_fraction":0.7765753273697938},{"optimizer_step":6,"nonzero_advantage":false,"ste_relative_error":0.8301226308609988,"corrected_relative_error":1.31150186996385,"residual_fraction":1.5798893093705533},{"optimizer_step":7,"nonzero_advantage":false,"ste_relative_error":0.7817560604795298,"corrected_relative_error":0.6348740142035446,"residual_fraction":0.8121126861672328},{"optimizer_step":8,"nonzero_advantage":true,"ste_relative_error":1.0591875513894808,"corrected_relative_error":1.0442756929255608,"residual_fraction":0.9859214183131606},{"optimizer_step":9,"nonzero_advantage":false,"ste_relative_error":0.5340816188797004,"corrected_relative_error":0.4608209046347617,"residual_fraction":0.862828617096743},{"optimizer_step":10,"nonzero_advantage":false,"ste_relative_error":1.1853376277486587,"corrected_relative_error":0.910899195422019,"residual_fraction":0.7684723526005941},{"optimizer_step":11,"nonzero_advantage":false,"ste_relative_error":0.6243653716034523,"corrected_relative_error":0.42271832317708824,"residual_fraction":0.6770367839130669},{"optimizer_step":12,"nonzero_advantage":false,"ste_relative_error":0.9512967327084878,"corrected_relative_error":0.8504462496924742,"residual_fraction":0.8939863035912288},{"optimizer_step":13,"nonzero_advantage":true,"ste_relative_error":0.434324366066292,"corrected_relative_error":0.3619112841437808,"residual_fraction":0.8332741895685895},{"optimizer_step":14,"nonzero_advantage":false,"ste_relative_error":0.6542310281421985,"corrected_relative_error":0.6117965981219562,"residual_fraction":0.9351384630277441},{"optimizer_step":15,"nonzero_advantage":false,"ste_relative_error":0.4797549261534791,"corrected_relative_error":0.4553391268969878,"residual_fraction":0.949107767475679},{"optimizer_step":16,"nonzero_advantage":false,"ste_relative_error":0.2606159220645789,"corrected_relative_error":0.19427663401582407,"residual_fraction":0.7454518990120781},{"optimizer_step":17,"nonzero_advantage":true,"ste_relative_error":0.37666566208854313,"corrected_relative_error":0.4028163106640291,"residual_fraction":1.0694266858053516},{"optimizer_step":18,"nonzero_advantage":false,"ste_relative_error":0.6494568162698209,"corrected_relative_error":0.6391755563377023,"residual_fraction":0.9841694479531844},{"optimizer_step":19,"nonzero_advantage":false,"ste_relative_error":0.9523722094660356,"corrected_relative_error":0.8693601057348053,"residual_fraction":0.9128364909159072},{"optimizer_step":20,"nonzero_advantage":false,"ste_relative_error":0.3008526566445537,"corrected_relative_error":0.21964168170128226,"residual_fraction":0.7300639593845463},{"optimizer_step":21,"nonzero_advantage":false,"ste_relative_error":0.1538040266124948,"corrected_relative_error":0.0863314011098953,"residual_fraction":0.5613078084581297},{"optimizer_step":22,"nonzero_advantage":false,"ste_relative_error":0.3067934970301042,"corrected_relative_error":0.2536882826749205,"residual_fraction":0.8269024119830913},{"optimizer_step":23,"nonzero_advantage":true,"ste_relative_error":0.23324152261868686,"corrected_relative_error":0.16770794805251896,"residual_fraction":0.7190312692594408},{"optimizer_step":24,"nonzero_advantage":true,"ste_relative_error":0.23138451712836897,"corrected_relative_error":0.20300530082760976,"residual_fraction":0.8773504093836374},{"optimizer_step":25,"nonzero_advantage":false,"ste_relative_error":0.3299734357017447,"corrected_relative_error":0.27006247912754205,"residual_fraction":0.8184370313119546},{"optimizer_step":26,"nonzero_advantage":false,"ste_relative_error":0.2618444188626063,"corrected_relative_error":0.25460674867245553,"residual_fraction":0.9723588907428712},{"optimizer_step":27,"nonzero_advantage":false,"ste_relative_error":0.3216986641420161,"corrected_relative_error":0.18753814246700834,"residual_fraction":0.5829621424359361},{"optimizer_step":28,"nonzero_advantage":true,"ste_relative_error":0.13986602461947611,"corrected_relative_error":0.10199831426336871,"residual_fraction":0.7292572627331657},{"optimizer_step":29,"nonzero_advantage":false,"ste_relative_error":0.18827816735564407,"corrected_relative_error":0.14062943621794682,"residual_fraction":0.7469237575077295},{"optimizer_step":30,"nonzero_advantage":false,"ste_relative_error":0.21203182061683767,"corrected_relative_error":0.15429422239178292,"residual_fraction":0.7276937109859926}]}]},"first_informative_stale_group":{"observed_unix":1791769153.2737067,"optimizer_update":3,"problem":"Compute (51 + 55) modulo10","label":"6","sampled_responses":["6","2","1","1"],"scope":"Descriptive first stale mixed-reward group in the completed seed42 study; methods have the same first three sampled batches. GRPO ratios reconstructed from saved rollout and current native selected log-probabilities, before this update. Gradient norms are for the whole batch. This does not establish a universal cause of collapse, or identify why uncorrectedST later collapses.","records":[{"method":"grpo","source_sha256":"f996979354ae952cc08c7ab901ec08c8ff7a83d7a3cfb627bee133028e8217eb","samples_with_nonzero_advantage":[132,133,134,135],"samples":[{"sample_index":132,"reward":1.0,"advantage":1.4999970197677612,"behavior_logprob":-3.1317784786224365,"current_native_logprob":-10.968952178955078,"ratio":0.00039478324112425544,"ppo_clips_gradient":false},{"sample_index":133,"reward":0.0,"advantage":-0.4999989867210388,"behavior_logprob":-2.1693115234375,"current_native_logprob":-7.987980365753174,"ratio":0.002971558125610152,"ppo_clips_gradient":true},{"sample_index":134,"reward":0.0,"advantage":-0.4999989867210388,"behavior_logprob":-0.8848743438720703,"current_native_logprob":-1.4909021854400635,"ratio":0.5455134369984158,"ppo_clips_gradient":true},{"sample_index":135,"reward":0.0,"advantage":-0.4999989867210388,"behavior_logprob":-0.8848740458488464,"current_native_logprob":-1.4909021854400635,"ratio":0.545513274422767,"ppo_clips_gradient":true}],"native_logged_gradient_norm":0.02814086013215962,"native_logged_clip_fraction":0.046875},{"method":"ste","source_sha256":"04a0befa32502641d7d40b9fdf7ede6d08b23ab1be8b58454a36c826e765b286","samples_with_nonzero_advantage":[132,133,134,135],"native_logged_gradient_norm":53.65604752073036,"native_logged_clip_fraction":0.0},{"method":"corrected","source_sha256":"0a3865ca6662e8775315d52692e09b39883766d842adfa4e5eab9cb272b37cf3","samples_with_nonzero_advantage":[132,133,134,135],"native_logged_gradient_norm":92.53448470367682,"native_logged_clip_fraction":0.0}],"reconstructed_grpo_loss":0.018740709288208635,"reconstructed_grpo_clipped_sample_count":3,"loss_reconstruction_matches_native_log":true,"correct_answer_probability":{"behavior":0.04364011518334241,"current":1.7228386115115746e-05,"ratio":0.00039478324112425544}}},"completed_seed_qualification":{"paired_async_seeds":[42,43,44,45],"paired_differences":[{"seed":42,"corrected_minus_ste_endpoint_pp":90.0,"corrected_minus_ste_auc_pp":29.583333333333336,"corrected_minus_grpo_endpoint_pp":90.0},{"seed":43,"corrected_minus_ste_endpoint_pp":-5.0,"corrected_minus_ste_auc_pp":-1.6666666666666643,"corrected_minus_grpo_endpoint_pp":-81.25},{"seed":44,"corrected_minus_ste_endpoint_pp":-17.5,"corrected_minus_ste_auc_pp":-9.999999999999998,"corrected_minus_grpo_endpoint_pp":2.5},{"seed":45,"corrected_minus_ste_endpoint_pp":-17.5,"corrected_minus_ste_auc_pp":-20.416666666666664,"corrected_minus_grpo_endpoint_pp":-15.0}],"freshness_controls":[{"method":"grpo","seed":42,"sync_endpoint_percent":10.0,"async_endpoint_percent":10.0,"sync_minus_async_endpoint_pp":0.0,"sync_minus_async_auc_pp":0.0},{"method":"ste","seed":42,"sync_endpoint_percent":10.0,"async_endpoint_percent":10.0,"sync_minus_async_endpoint_pp":0.0,"sync_minus_async_auc_pp":-0.8333333333333321}],"scope":"Only fully audited 30-update runs. Seeds, not repeated answers, are replicates. One fixed 20-question test. Freshness controls cover seed 42 only; no general causal claim. AUC is trapezoidal accuracy averaged over 0..30 steps. Partial or unpaired seeds excluded from paired summaries.","retained_runs_completed":15,"retained_runs_planned":15},"batch512_status":{"pid":839947,"started_unix":1791794427.4489846,"seed":42,"global_batch":512,"microbatch":1,"status":"running","updated_unix":1791797912.7091029,"pending":["corrected_sc","tis_grpo"],"completed_runs":0,"active_run":"async_sc_seed42_gpu0","active_pid":865221,"retained_runs":["async_grpo_seed46_gpu1"]},"sources":[{"path":"independent/qwen_fullgrad_20261011/artifacts/extended/exact_metrics.csv","sha256":"56cd966b210a0b69626237350d2a4bc78863e1c136c4ff2b2642075274cb50a6","bytes":270388},{"path":"independent/qwen_fullgrad_20261011/artifacts/extended/summary.csv","sha256":"e27ee008a33ecc4ba7b4f0b36abafe683606b8878d7de1699597f0e9f487ba43","bytes":24428},{"path":"independent/qwen_fullgrad_20261011/artifacts/extended/raw_metrics.json","sha256":"8816b6e3a58a04d3747dfb4c9a2daa341de87015c217b309b168b74bc493816a","bytes":41154},{"path":"independent/qwen_fullgrad_20261011/artifacts/extended/weight_audit.json","sha256":"5aa470bae30abdc52742f40f76b50620fd2d7a8c58ce44873542bf8e3ed0d46f","bytes":19942},{"path":"results/slime_exp30_batch64_paired_async/completion_review.json","sha256":"c2453b6af2647ef20679cced0ec429ab4aab4f9e98683360cbe064e98ee9bb04","bytes":16945},{"path":"results/slime_exp24/REPORT.md","sha256":"7e40e4c707eab88c20f997233ff5c032ce8507ccb908de6b5c737a20f73d0c61","bytes":11334},{"path":"independent/qwen_fullgrad_20261011/REPORT.md","sha256":"8051b931f56402654334ea91609a8fdbe17f05f195f29379011a7406f6b7f9e8","bytes":13963},{"path":"independent/qwen_fullgrad_20261011/PROTOCOL.md","sha256":"0781ba48e47cf6edde208eca754fb7f930908a71ffb4fa165ed0a012bd6c5dd4","bytes":14872},{"path":"results/qwen_diagnostic/REPORT.md","sha256":"eeb814f4e40d00c5c8a4631427b6f8606cc74531b33fe0110047bf9f6ae58463","bytes":3926},{"path":"results/slime_gradient_audit.json","sha256":"b61b66abfaec2404dae7f7dab8b5a367a47855567ed8eced368bcfaab6097463","bytes":969},{"path":"independent/qwen_precision_stress_20261011/artifacts/summary.json","sha256":"467d579c64d4544710b3cb611fecfd6cd619c46671abdedcc7a6ae29270579f9","bytes":6415},{"path":"independent/qwen_precision_stress_20261011/artifacts/forward_metrics.json","sha256":"76a9dbc0fc1c3af43b88490455bfb1f3bfa3ac67817a949ce8284d4cf6bd8bdc","bytes":95903},{"path":"independent/qwen_precision_stress_20261011/artifacts/binary_variance.json","sha256":"7a075aa59dca285911ea4755f2758f57318718970858f5faac136e5c7e914037","bytes":348894},{"path":"independent/qwen_precision_stress_20261011/artifacts/validation.json","sha256":"b045db42fc54fe2f8f1bca92b5c1ad6d200f5554ea0aa58d63bc8f00c6d4109e","bytes":1203},{"path":"independent/qwen_precision_stress_20261011/artifacts/config.json","sha256":"705d7bdbed5264a1ff53e33cf2af993b09aaa73775f81821aef54ba77c8f7735","bytes":1456},{"path":"independent/score_centering_done_right_20261011/timing_audit_agent.json","sha256":"5763fe56342080a1ca5f6ec3d578ca72b44f61bc6478473f770a5d0e41bc63c5","bytes":71780},{"path":"independent/score_centering_done_right_20261011/math_checks.json","sha256":"27d3a17e6805d34aca998c692f8a8c9e27d8d2229b006c02cf6d095bff2e3d0c"},{"path":"independent/score_centering_done_right_20261011/verify_math.py","sha256":"0e4c8a4132796f931fd17b126baca3150169ad3df89c7c965c955d6612b00d22"},{"path":"results/slime_exp30_batch64_paired_async/completion_review.json","sha256":"c2453b6af2647ef20679cced0ec429ab4aab4f9e98683360cbe064e98ee9bb04"},{"path":"results/slime_exp30_batch64_paired_async/score_transport_diagnostics.json","sha256":"133a37bb43cf9f2366ff9d8df2cceeceefa98b22a919dc71937bcb49c0f2b2af"},{"path":"results/slime_exp30_batch64_paired_async/first_informative_stale_group.json","sha256":"d8397e4eae1754d510437737a2b2bd02234b1351721f24a4c7e4b1779d8373fa"},{"path":"results/slime_exp30_batch64_paired_async/training_reward_group_diagnostics.json","sha256":"cbfd044d7685ae9de6e4aede44a272ea9087f6477023fb01c3897e01237ca368"},{"path":"independent/qwen_fullgrad_20261011/artifacts/extended/exact_metrics.csv","sha256":"56cd966b210a0b69626237350d2a4bc78863e1c136c4ff2b2642075274cb50a6"},{"path":"results/slime_exp30_multiseed/complete_run_analysis.json","sha256":"0bfa8754211d453c493b948a17ec641574c1a2899da786d58bc21e07a0e2328f","bytes":25814},{"path":"results/slime_exp30_batch512/queue_status.json","sha256":"707cf6821ede6dbc8c5225f0329e08e36d51c3d5dde05b63d8c32277fad6b32d","bytes":372}],"qwen_checkpoint_bias":{"protocol":"Real Qwen3.5-0.8B weights, all 752393024 text parameters; eight fixed SST-2 A/B prompts; G=4; CPU FP32 execution, engine batch one; own full-parameter SGD updates at learning rate 1e-5 induce lag. No artificial independent Jacobian/logit perturbations.","values":[{"configuration":"chunk_fp32","statistical_batch_size":256,"target":"g_q","method":"SC","relative_bias2":8.97982507871135e-12,"relative_variance":0.0030678934862744893,"relative_mse":0.0030678934952543145},{"configuration":"chunk_fp32","statistical_batch_size":256,"target":"g_q","method":"corrected_SC","relative_bias2":8.97982507871135e-12,"relative_variance":0.0030678934862744893,"relative_mse":0.0030678934952543145},{"configuration":"same_recurrent_fp32","statistical_batch_size":256,"target":"g_q","method":"SC","relative_bias2":0.0,"relative_variance":0.0030678945371892747,"relative_mse":0.0030678945371892747},{"configuration":"same_recurrent_fp32","statistical_batch_size":256,"target":"g_q","method":"corrected_SC","relative_bias2":0.0,"relative_variance":0.0030678945371892747,"relative_mse":0.0030678945371892747},{"configuration":"sgd_lag1_fp32","statistical_batch_size":256,"target":"g_q","method":"SC","relative_bias2":0.005855828988622502,"relative_variance":0.0030040990735063257,"relative_mse":0.008859928062128828},{"configuration":"sgd_lag1_fp32","statistical_batch_size":256,"target":"g_q","method":"corrected_SC","relative_bias2":1.6785957817498193e-05,"relative_variance":0.0030753276990124158,"relative_mse":0.003092113656829914},{"configuration":"sgd_lag4_fp32","statistical_batch_size":256,"target":"g_q","method":"SC","relative_bias2":0.049218363209716016,"relative_variance":0.00290880113585927,"relative_mse":0.05212716434557529},{"configuration":"sgd_lag4_fp32","statistical_batch_size":256,"target":"g_q","method":"corrected_SC","relative_bias2":0.001989021248390651,"relative_variance":0.0031377390637931457,"relative_mse":0.005126760312183796}],"scope":"Exact group-standardized update expectation; not raw expected-reward gradient. Real-model controlled gradient experiment, separate from Megatron/SGLang arithmetic training. Two lag checkpoints do not verify asymptotic order."},"qwen_gradient_comparison":{"source_path":"/home/weicheng/yifei/pg-correction/independent/qwen_fullgrad_20261011/artifacts/extended/exact_metrics.csv","source_sha256":"56cd966b210a0b69626237350d2a4bc78863e1c136c4ff2b2642075274cb50a6","source_bytes":270388,"gradient_scope":"all 752393024 Qwen3.5-0.8B text parameters; constrained one-token A/B policy; real checkpoint lag via SGD lr 1e-5; no synthetic Jacobian/logit perturbation","aggregation":"Mean across eight fixed prompts of each per-prompt squared metric divided by that prompt reference-target norm squared (g_q for target=g_q; g_p for target=g_p). Conditional prompt metrics, not population-of-prompts results.","target":"Exact expected group-advantage updates g_q under q^4 and g_p under p^4; G=4, reward correctness, sample sd ddof1 and epsilon1e-6.","GRPO":{"ppo_clip":0.2,"ratio":"p/q exactly once; q is PPO denominator; no extra duplicate p/q weighting","active":"(A>=0 and p/q<=1.2) or (A<0 and p/q>=0.8)"},"TIS":{"cap":2,"weight":"min(p/q,2) with detached weights times training score","all_extended_cells_cap_inactive":true,"equal_to_unclipped_IS_in_extended_table":true},"IS":"unclipped_IS uses per-token p/q, not whole-group probability ratio; group advantages mean it is not guaranteed unbiased for g_p.","limits":["Each MSE is relative to the target named in its record; target mismatch contributes to errors when a method pursues the other update direction. The target=gp export includes lag4 only.","B=4/16/64/256 means 1/4/16/64 independently sampled groups, engine batch fixed1.","Because one-token responses share the prefix and group advantages sum0, common centering term cancels.","Exact enumeration eliminates Monte Carlo estimation uncertainty; eight prompts remain a small controlled setup.","g_p rows use per-prompt g_p norm\u00b2 normalization, so normalized g_p/g_q magnitudes cannot be directly compared as absolute errors across panels.","oracle_q is a q-score oracle and is generally biased relative to g_p; do not label it an unbiased oracle in the g_p panel."],"records":[{"configuration":"chunk_fp32","method":"GRPO","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.19634532764928586,"mean_normalized_mse":0.19634532765823853},{"configuration":"chunk_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.19634532764928586,"mean_normalized_mse":0.19634532765823853},{"configuration":"chunk_fp32","method":"TIS","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.19634532764928586,"mean_normalized_mse":0.19634532765823853},{"configuration":"chunk_fp32","method":"SC","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":8.97982507871135e-12,"mean_normalized_variance":0.19634518312156732,"mean_normalized_mse":0.19634518313054714},{"configuration":"chunk_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":8.97982507871135e-12,"mean_normalized_variance":0.19634518312156732,"mean_normalized_mse":0.19634518313054714},{"configuration":"chunk_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.19634525038011358,"mean_normalized_mse":0.19634525038011358},{"configuration":"chunk_fp32","method":"GRPO","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.049086331912321465,"mean_normalized_mse":0.04908633192127414},{"configuration":"chunk_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.049086331912321465,"mean_normalized_mse":0.04908633192127414},{"configuration":"chunk_fp32","method":"TIS","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.049086331912321465,"mean_normalized_mse":0.04908633192127414},{"configuration":"chunk_fp32","method":"SC","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":8.97982507871135e-12,"mean_normalized_variance":0.04908629578039183,"mean_normalized_mse":0.04908629578937165},{"configuration":"chunk_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":8.97982507871135e-12,"mean_normalized_variance":0.04908629578039183,"mean_normalized_mse":0.04908629578937165},{"configuration":"chunk_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.049086312595028395,"mean_normalized_mse":0.049086312595028395},{"configuration":"chunk_fp32","method":"GRPO","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.012271582978080366,"mean_normalized_mse":0.012271582987033042},{"configuration":"chunk_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.012271582978080366,"mean_normalized_mse":0.012271582987033042},{"configuration":"chunk_fp32","method":"TIS","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.012271582978080366,"mean_normalized_mse":0.012271582987033042},{"configuration":"chunk_fp32","method":"SC","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":8.97982507871135e-12,"mean_normalized_variance":0.012271573945097957,"mean_normalized_mse":0.012271573954077781},{"configuration":"chunk_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":8.97982507871135e-12,"mean_normalized_variance":0.012271573945097957,"mean_normalized_mse":0.012271573954077781},{"configuration":"chunk_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.012271578148757099,"mean_normalized_mse":0.012271578148757099},{"configuration":"chunk_fp32","method":"GRPO","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.0030678957445200916,"mean_normalized_mse":0.003067895753472767},{"configuration":"chunk_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.0030678957445200916,"mean_normalized_mse":0.003067895753472767},{"configuration":"chunk_fp32","method":"TIS","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":8.95267551433896e-12,"mean_normalized_variance":0.0030678957445200916,"mean_normalized_mse":0.003067895753472767},{"configuration":"chunk_fp32","method":"SC","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":8.97982507871135e-12,"mean_normalized_variance":0.0030678934862744893,"mean_normalized_mse":0.0030678934952543145},{"configuration":"chunk_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":8.97982507871135e-12,"mean_normalized_variance":0.0030678934862744893,"mean_normalized_mse":0.0030678934952543145},{"configuration":"chunk_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.0030678945371892747,"mean_normalized_mse":0.0030678945371892747},{"configuration":"same_recurrent_fp32","method":"GRPO","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.19634525038011358,"mean_normalized_mse":0.19634525038011358},{"configuration":"same_recurrent_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.19634525038011358,"mean_normalized_mse":0.19634525038011358},{"configuration":"same_recurrent_fp32","method":"TIS","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.19634525038011358,"mean_normalized_mse":0.19634525038011358},{"configuration":"same_recurrent_fp32","method":"SC","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.19634525038011358,"mean_normalized_mse":0.19634525038011358},{"configuration":"same_recurrent_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.19634525038011358,"mean_normalized_mse":0.19634525038011358},{"configuration":"same_recurrent_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.19634525038011358,"mean_normalized_mse":0.19634525038011358},{"configuration":"same_recurrent_fp32","method":"GRPO","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.049086312595028395,"mean_normalized_mse":0.049086312595028395},{"configuration":"same_recurrent_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.049086312595028395,"mean_normalized_mse":0.049086312595028395},{"configuration":"same_recurrent_fp32","method":"TIS","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.049086312595028395,"mean_normalized_mse":0.049086312595028395},{"configuration":"same_recurrent_fp32","method":"SC","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.049086312595028395,"mean_normalized_mse":0.049086312595028395},{"configuration":"same_recurrent_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.049086312595028395,"mean_normalized_mse":0.049086312595028395},{"configuration":"same_recurrent_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.049086312595028395,"mean_normalized_mse":0.049086312595028395},{"configuration":"same_recurrent_fp32","method":"GRPO","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.012271578148757099,"mean_normalized_mse":0.012271578148757099},{"configuration":"same_recurrent_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.012271578148757099,"mean_normalized_mse":0.012271578148757099},{"configuration":"same_recurrent_fp32","method":"TIS","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.012271578148757099,"mean_normalized_mse":0.012271578148757099},{"configuration":"same_recurrent_fp32","method":"SC","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.012271578148757099,"mean_normalized_mse":0.012271578148757099},{"configuration":"same_recurrent_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.012271578148757099,"mean_normalized_mse":0.012271578148757099},{"configuration":"same_recurrent_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.012271578148757099,"mean_normalized_mse":0.012271578148757099},{"configuration":"same_recurrent_fp32","method":"GRPO","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.0030678945371892747,"mean_normalized_mse":0.0030678945371892747},{"configuration":"same_recurrent_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.0030678945371892747,"mean_normalized_mse":0.0030678945371892747},{"configuration":"same_recurrent_fp32","method":"TIS","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.0030678945371892747,"mean_normalized_mse":0.0030678945371892747},{"configuration":"same_recurrent_fp32","method":"SC","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.0030678945371892747,"mean_normalized_mse":0.0030678945371892747},{"configuration":"same_recurrent_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.0,"mean_normalized_variance":0.0030678945371892747,"mean_normalized_mse":0.0030678945371892747},{"configuration":"same_recurrent_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.0030678945371892747,"mean_normalized_mse":0.0030678945371892747},{"configuration":"sgd_lag1_fp32","method":"GRPO","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.2053259803553693,"mean_normalized_mse":0.21231777992237658},{"configuration":"sgd_lag1_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.2053259803553693,"mean_normalized_mse":0.21231777992237658},{"configuration":"sgd_lag1_fp32","method":"TIS","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.2053259803553693,"mean_normalized_mse":0.21231777992237658},{"configuration":"sgd_lag1_fp32","method":"SC","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.005855828988622502,"mean_normalized_variance":0.19226234070440484,"mean_normalized_mse":0.19811816969302737},{"configuration":"sgd_lag1_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":1.6785957817498193e-05,"mean_normalized_variance":0.1968209727367946,"mean_normalized_mse":0.1968377586946121},{"configuration":"sgd_lag1_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.19634525038011358,"mean_normalized_mse":0.19634525038011358},{"configuration":"sgd_lag1_fp32","method":"GRPO","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.05133149508884233,"mean_normalized_mse":0.0583232946558496},{"configuration":"sgd_lag1_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.05133149508884233,"mean_normalized_mse":0.0583232946558496},{"configuration":"sgd_lag1_fp32","method":"TIS","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.05133149508884233,"mean_normalized_mse":0.0583232946558496},{"configuration":"sgd_lag1_fp32","method":"SC","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.005855828988622502,"mean_normalized_variance":0.04806558517610121,"mean_normalized_mse":0.053921414164723715},{"configuration":"sgd_lag1_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":1.6785957817498193e-05,"mean_normalized_variance":0.04920524318419865,"mean_normalized_mse":0.04922202914201615},{"configuration":"sgd_lag1_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.049086312595028395,"mean_normalized_mse":0.049086312595028395},{"configuration":"sgd_lag1_fp32","method":"GRPO","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.012832873772210582,"mean_normalized_mse":0.019824673339217856},{"configuration":"sgd_lag1_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.012832873772210582,"mean_normalized_mse":0.019824673339217856},{"configuration":"sgd_lag1_fp32","method":"TIS","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.012832873772210582,"mean_normalized_mse":0.019824673339217856},{"configuration":"sgd_lag1_fp32","method":"SC","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.005855828988622502,"mean_normalized_variance":0.012016396294025303,"mean_normalized_mse":0.017872225282647806},{"configuration":"sgd_lag1_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":1.6785957817498193e-05,"mean_normalized_variance":0.012301310796049663,"mean_normalized_mse":0.012318096753867162},{"configuration":"sgd_lag1_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.012271578148757099,"mean_normalized_mse":0.012271578148757099},{"configuration":"sgd_lag1_fp32","method":"GRPO","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.0032082184430526454,"mean_normalized_mse":0.01020001801005992},{"configuration":"sgd_lag1_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.0032082184430526454,"mean_normalized_mse":0.01020001801005992},{"configuration":"sgd_lag1_fp32","method":"TIS","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.006991799567007274,"mean_normalized_variance":0.0032082184430526454,"mean_normalized_mse":0.01020001801005992},{"configuration":"sgd_lag1_fp32","method":"SC","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.005855828988622502,"mean_normalized_variance":0.0030040990735063257,"mean_normalized_mse":0.008859928062128828},{"configuration":"sgd_lag1_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":1.6785957817498193e-05,"mean_normalized_variance":0.0030753276990124158,"mean_normalized_mse":0.003092113656829914},{"configuration":"sgd_lag1_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.0030678945371892747,"mean_normalized_mse":0.0030678945371892747},{"configuration":"sgd_lag4_fp32","method":"GRPO","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.32526582078066124,"mean_normalized_variance":0.10446350630863194,"mean_normalized_mse":0.4297293270892932},{"configuration":"sgd_lag4_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.054491045439024936,"mean_normalized_variance":0.2049938552151766,"mean_normalized_mse":0.25948490065420154},{"configuration":"sgd_lag4_fp32","method":"TIS","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.054491045439024936,"mean_normalized_variance":0.2049938552151766,"mean_normalized_mse":0.25948490065420154},{"configuration":"sgd_lag4_fp32","method":"SC","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.049218363209716016,"mean_normalized_variance":0.18616327269499328,"mean_normalized_mse":0.23538163590470929},{"configuration":"sgd_lag4_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.001989021248390651,"mean_normalized_variance":0.20081530008276133,"mean_normalized_mse":0.20280432133115198},{"configuration":"sgd_lag4_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.19634525038011358,"mean_normalized_mse":0.19634525038011358},{"configuration":"sgd_lag4_fp32","method":"GRPO","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.32526582078066124,"mean_normalized_variance":0.026115876577157984,"mean_normalized_mse":0.3513816973578192},{"configuration":"sgd_lag4_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.054491045439024936,"mean_normalized_variance":0.05124846380379415,"mean_normalized_mse":0.1057395092428191},{"configuration":"sgd_lag4_fp32","method":"TIS","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.054491045439024936,"mean_normalized_variance":0.05124846380379415,"mean_normalized_mse":0.1057395092428191},{"configuration":"sgd_lag4_fp32","method":"SC","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.049218363209716016,"mean_normalized_variance":0.04654081817374832,"mean_normalized_mse":0.09575918138346434},{"configuration":"sgd_lag4_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.001989021248390651,"mean_normalized_variance":0.05020382502069033,"mean_normalized_mse":0.05219284626908098},{"configuration":"sgd_lag4_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.049086312595028395,"mean_normalized_mse":0.049086312595028395},{"configuration":"sgd_lag4_fp32","method":"GRPO","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.32526582078066124,"mean_normalized_variance":0.006528969144289496,"mean_normalized_mse":0.33179478992495076},{"configuration":"sgd_lag4_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.054491045439024936,"mean_normalized_variance":0.012812115950948538,"mean_normalized_mse":0.06730316138997347},{"configuration":"sgd_lag4_fp32","method":"TIS","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.054491045439024936,"mean_normalized_variance":0.012812115950948538,"mean_normalized_mse":0.06730316138997347},{"configuration":"sgd_lag4_fp32","method":"SC","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.049218363209716016,"mean_normalized_variance":0.01163520454343708,"mean_normalized_mse":0.0608535677531531},{"configuration":"sgd_lag4_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.001989021248390651,"mean_normalized_variance":0.012550956255172583,"mean_normalized_mse":0.014539977503563234},{"configuration":"sgd_lag4_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.012271578148757099,"mean_normalized_mse":0.012271578148757099},{"configuration":"sgd_lag4_fp32","method":"GRPO","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.32526582078066124,"mean_normalized_variance":0.001632242286072374,"mean_normalized_mse":0.32689806306673364},{"configuration":"sgd_lag4_fp32","method":"unclipped_IS","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.054491045439024936,"mean_normalized_variance":0.0032030289877371344,"mean_normalized_mse":0.05769407442676207},{"configuration":"sgd_lag4_fp32","method":"TIS","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.054491045439024936,"mean_normalized_variance":0.0032030289877371344,"mean_normalized_mse":0.05769407442676207},{"configuration":"sgd_lag4_fp32","method":"SC","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.049218363209716016,"mean_normalized_variance":0.00290880113585927,"mean_normalized_mse":0.05212716434557529},{"configuration":"sgd_lag4_fp32","method":"corrected_SC","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.001989021248390651,"mean_normalized_variance":0.0031377390637931457,"mean_normalized_mse":0.005126760312183797},{"configuration":"sgd_lag4_fp32","method":"oracle_q","target":"g_q","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":1.0665271410986257e-32,"mean_normalized_variance":0.0030678945371892747,"mean_normalized_mse":0.0030678945371892747},{"configuration":"sgd_lag4_fp32","method":"GRPO","target":"g_p","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.3023327632083811,"mean_normalized_variance":0.11294669923776046,"mean_normalized_mse":0.41527946244614156},{"configuration":"sgd_lag4_fp32","method":"unclipped_IS","target":"g_p","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":1.2697142820950433e-05,"mean_normalized_variance":0.19754899349550864,"mean_normalized_mse":0.1975616906383296},{"configuration":"sgd_lag4_fp32","method":"TIS","target":"g_p","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":1.2697142820950433e-05,"mean_normalized_variance":0.19754899349550864,"mean_normalized_mse":0.1975616906383296},{"configuration":"sgd_lag4_fp32","method":"SC","target":"g_p","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.00216841511879374,"mean_normalized_variance":0.18114476250077458,"mean_normalized_mse":0.1833131776195683},{"configuration":"sgd_lag4_fp32","method":"corrected_SC","target":"g_p","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.044272030067611615,"mean_normalized_variance":0.19729574517509746,"mean_normalized_mse":0.2415677752427091},{"configuration":"sgd_lag4_fp32","method":"oracle_q","target":"g_p","statistical_batch_size":4,"independent_groups":1,"prompts":8,"mean_normalized_bias2":0.05104447187116277,"mean_normalized_variance":0.19320211144459257,"mean_normalized_mse":0.24424658331575533},{"configuration":"sgd_lag4_fp32","method":"GRPO","target":"g_p","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.3023327632083811,"mean_normalized_variance":0.028236674809440115,"mean_normalized_mse":0.3305694380178212},{"configuration":"sgd_lag4_fp32","method":"unclipped_IS","target":"g_p","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":1.2697142820950433e-05,"mean_normalized_variance":0.04938724837387716,"mean_normalized_mse":0.049399945516698116},{"configuration":"sgd_lag4_fp32","method":"TIS","target":"g_p","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":1.2697142820950433e-05,"mean_normalized_variance":0.04938724837387716,"mean_normalized_mse":0.049399945516698116},{"configuration":"sgd_lag4_fp32","method":"SC","target":"g_p","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.00216841511879374,"mean_normalized_variance":0.045286190625193644,"mean_normalized_mse":0.04745460574398738},{"configuration":"sgd_lag4_fp32","method":"corrected_SC","target":"g_p","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.044272030067611615,"mean_normalized_variance":0.049323936293774365,"mean_normalized_mse":0.09359596636138598},{"configuration":"sgd_lag4_fp32","method":"oracle_q","target":"g_p","statistical_batch_size":16,"independent_groups":4,"prompts":8,"mean_normalized_bias2":0.05104447187116277,"mean_normalized_variance":0.04830052786114814,"mean_normalized_mse":0.09934499973231091},{"configuration":"sgd_lag4_fp32","method":"GRPO","target":"g_p","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.3023327632083811,"mean_normalized_variance":0.007059168702360029,"mean_normalized_mse":0.3093919319107411},{"configuration":"sgd_lag4_fp32","method":"unclipped_IS","target":"g_p","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":1.2697142820950433e-05,"mean_normalized_variance":0.01234681209346929,"mean_normalized_mse":0.01235950923629024},{"configuration":"sgd_lag4_fp32","method":"TIS","target":"g_p","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":1.2697142820950433e-05,"mean_normalized_variance":0.01234681209346929,"mean_normalized_mse":0.01235950923629024},{"configuration":"sgd_lag4_fp32","method":"SC","target":"g_p","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.00216841511879374,"mean_normalized_variance":0.011321547656298411,"mean_normalized_mse":0.013489962775092151},{"configuration":"sgd_lag4_fp32","method":"corrected_SC","target":"g_p","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.044272030067611615,"mean_normalized_variance":0.012330984073443591,"mean_normalized_mse":0.05660301414105521},{"configuration":"sgd_lag4_fp32","method":"oracle_q","target":"g_p","statistical_batch_size":64,"independent_groups":16,"prompts":8,"mean_normalized_bias2":0.05104447187116277,"mean_normalized_variance":0.012075131965287035,"mean_normalized_mse":0.0631196038364498},{"configuration":"sgd_lag4_fp32","method":"GRPO","target":"g_p","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.3023327632083811,"mean_normalized_variance":0.0017647921755900072,"mean_normalized_mse":0.3040975553839711},{"configuration":"sgd_lag4_fp32","method":"unclipped_IS","target":"g_p","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":1.2697142820950433e-05,"mean_normalized_variance":0.0030867030233673225,"mean_normalized_mse":0.0030994001661882728},{"configuration":"sgd_lag4_fp32","method":"TIS","target":"g_p","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":1.2697142820950433e-05,"mean_normalized_variance":0.0030867030233673225,"mean_normalized_mse":0.0030994001661882728},{"configuration":"sgd_lag4_fp32","method":"SC","target":"g_p","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.00216841511879374,"mean_normalized_variance":0.0028303869140746027,"mean_normalized_mse":0.004998802032868343},{"configuration":"sgd_lag4_fp32","method":"corrected_SC","target":"g_p","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.044272030067611615,"mean_normalized_variance":0.003082746018360898,"mean_normalized_mse":0.047354776085972515},{"configuration":"sgd_lag4_fp32","method":"oracle_q","target":"g_p","statistical_batch_size":256,"independent_groups":64,"prompts":8,"mean_normalized_bias2":0.05104447187116277,"mean_normalized_variance":0.003018782991321759,"mean_normalized_mse":0.054063254862484526}],"target_definitions":{"g_q":"E_{group~q^4} mean(A_i score_q_i)","g_p":"E_{group~p^4} mean(A_i score_p_i)"}},"paired_seed_learning":{"source_summary":"/home/weicheng/yifei/pg-correction/results/slime_exp30_multiseed/complete_run_analysis.json","source_summary_sha256":"0bfa8754211d453c493b948a17ec641574c1a2899da786d58bc21e07a0e2328f","source_summary_updated_unix":1791797112.1849222,"scope":"Four paired completed 30-update async seeds. Each absolute endpoint read from actual per-run audit.json and cross-checked to complete_run_analysis.json; not reconstructed from pairwise differences.","eval_questions":20,"responses_per_question":4,"records":[{"seed":42,"method":"DRSC","endpoint_percent":100.0,"checkpoints_percent":{"0":11.25,"5":10.0,"10":27.500000000000004,"15":27.500000000000004,"20":38.75,"25":75.0,"30":100.0},"actual_run":"results/slime_exp30_batch64_paired_async/formal_corrected_seed42_gpu1","summary_record_run":"results/slime_exp30_batch64_paired_async/formal_corrected_seed42_gpu1","audit_sha256":"29e25f63973bc133e042971958638fbbabd4e48ab8ca5fe8668ad5897d6ee809","steps":30,"prompt_order_sha256":"e56ce1d42ce80d7f7bd2d9dd4357235ae3fdc0c579438dd6160392690ed304e5"},{"seed":42,"method":"GRPO","endpoint_percent":10.0,"checkpoints_percent":{"0":11.25,"5":1.25,"10":10.0,"15":10.0,"20":10.0,"25":10.0,"30":10.0},"actual_run":"results/slime_exp30_batch64_paired_async/formal_grpo_seed42_gpu0","summary_record_run":"results/slime_exp30_batch64_paired_async/formal_grpo_seed42_gpu0","audit_sha256":"6eea77a17940a7b63bd26e6eb606eb5cf7e70cd04ba18446f11402c365230122","steps":30,"prompt_order_sha256":"e56ce1d42ce80d7f7bd2d9dd4357235ae3fdc0c579438dd6160392690ed304e5","grpo_clip_argument_lines":["eps_clip ........................................ 0.2","eps_clip_high ................................... 0.2"],"ppo_clip_low":0.2,"ppo_clip_high":0.2},{"seed":42,"method":"SC","endpoint_percent":10.0,"checkpoints_percent":{"0":11.25,"5":5.0,"10":8.75,"15":7.5,"20":10.0,"25":15.0,"30":10.0},"actual_run":"results/slime_exp30_batch64_paired_async/formal_ste_seed42_gpu1","summary_record_run":"results/slime_exp30_batch64_paired_async/formal_ste_seed42_gpu1","audit_sha256":"eb072ef7ce81817eac3e1df5637522852068a2923d236f41b1e4219be61d5b81","steps":30,"prompt_order_sha256":"e56ce1d42ce80d7f7bd2d9dd4357235ae3fdc0c579438dd6160392690ed304e5"},{"seed":43,"method":"DRSC","endpoint_percent":17.5,"checkpoints_percent":{"0":12.5,"5":35.0,"10":10.0,"15":22.5,"20":25.0,"25":26.25,"30":17.5},"actual_run":"results/slime_exp30_multiseed/async_corrected_seed43_gpu1","summary_record_run":"results/slime_exp30_multiseed/async_corrected_seed43_gpu1","audit_sha256":"c219cb01d71dc739eab0007828e8a6efe3a3ab2ea0afb7bf5bcc0dc41dd8ba91","steps":30,"prompt_order_sha256":"568e8194bb5241cede610977bb0c58c032db8eda4b03d59beab2cee8b0571333"},{"seed":43,"method":"GRPO","endpoint_percent":98.75,"checkpoints_percent":{"0":12.5,"5":28.749999999999996,"10":21.25,"15":43.75,"20":53.75,"25":77.5,"30":98.75},"actual_run":"results/slime_exp30_multiseed/async_grpo_seed43_gpu1","summary_record_run":"results/slime_exp30_multiseed/async_grpo_seed43_gpu1","audit_sha256":"8e5518d3651be92bb3aa254608ab7a0c96d85ad5fdac47f74ff5a7c08bac216f","steps":30,"prompt_order_sha256":"568e8194bb5241cede610977bb0c58c032db8eda4b03d59beab2cee8b0571333","grpo_clip_argument_lines":["eps_clip ........................................ 0.2","eps_clip_high ................................... 0.2"],"ppo_clip_low":0.2,"ppo_clip_high":0.2},{"seed":43,"method":"SC","endpoint_percent":22.5,"checkpoints_percent":{"0":12.5,"5":33.75,"10":25.0,"15":22.5,"20":22.5,"25":22.5,"30":22.5},"actual_run":"results/slime_exp30_multiseed/async_ste_seed43_gpu0","summary_record_run":"results/slime_exp30_multiseed/async_ste_seed43_gpu0","audit_sha256":"fd48c51e8ff32cde624736b0d6ce97aad62d0c6fc6d743c4b9815e54c65363d8","steps":30,"prompt_order_sha256":"568e8194bb5241cede610977bb0c58c032db8eda4b03d59beab2cee8b0571333"},{"seed":44,"method":"DRSC","endpoint_percent":6.25,"checkpoints_percent":{"0":8.75,"5":10.0,"10":10.0,"15":10.0,"20":10.0,"25":6.25,"30":6.25},"actual_run":"results/slime_exp30_multiseed/async_corrected_seed44_gpu1","summary_record_run":"results/slime_exp30_multiseed/async_corrected_seed44_gpu1","audit_sha256":"8906b64b692ef343cfd3f4f1beec9a608928db25cd09b5b12c9416f9e354db34","steps":30,"prompt_order_sha256":"f06ce80182b6f047767b69be94e58a9f07b9744f73682e18efa2978c59f5e8f3"},{"seed":44,"method":"GRPO","endpoint_percent":3.75,"checkpoints_percent":{"0":8.75,"5":10.0,"10":11.25,"15":11.25,"20":1.25,"25":1.25,"30":3.75},"actual_run":"results/slime_exp30_multiseed/async_grpo_seed44_gpu0","summary_record_run":"results/slime_exp30_multiseed/async_grpo_seed44_gpu0","audit_sha256":"338a3a0ff46b0568e406669b344f28f656fb50e77dff5b2f1ddcdd798a09c245","steps":30,"prompt_order_sha256":"f06ce80182b6f047767b69be94e58a9f07b9744f73682e18efa2978c59f5e8f3","grpo_clip_argument_lines":["eps_clip ........................................ 0.2","eps_clip_high ................................... 0.2"],"ppo_clip_low":0.2,"ppo_clip_high":0.2},{"seed":44,"method":"SC","endpoint_percent":23.75,"checkpoints_percent":{"0":8.75,"5":25.0,"10":8.75,"15":27.500000000000004,"20":22.5,"25":13.750000000000002,"30":23.75},"actual_run":"results/slime_exp30_multiseed/async_ste_seed44_gpu0","summary_record_run":"results/slime_exp30_multiseed/async_ste_seed44_gpu0","audit_sha256":"502b5d2872fef4dd863fcfd7160baed6d16ea388f8de8237943a002932b56d87","steps":30,"prompt_order_sha256":"f06ce80182b6f047767b69be94e58a9f07b9744f73682e18efa2978c59f5e8f3"},{"seed":45,"method":"DRSC","endpoint_percent":82.5,"checkpoints_percent":{"0":8.75,"5":13.750000000000002,"10":40.0,"15":30.0,"20":46.25,"25":51.24999999999999,"30":82.5},"actual_run":"results/slime_exp30_multiseed/async_corrected_seed45_gpu0","summary_record_run":"results/slime_exp30_multiseed/async_corrected_seed45_gpu0","audit_sha256":"e63dbe8e8c0e8e2eaed1559f9fa2bd1b86fdd7516e92b22b3d343112e00021b0","steps":30,"prompt_order_sha256":"982291fb89502d1893ea2ab42f2dd9d6edec973eaec49c8507f9537ab478d20f"},{"seed":45,"method":"GRPO","endpoint_percent":97.5,"checkpoints_percent":{"0":8.75,"5":23.75,"10":73.75,"15":82.5,"20":97.5,"25":100.0,"30":97.5},"actual_run":"results/slime_exp30_multiseed/async_grpo_seed45_gpu0","summary_record_run":"results/slime_exp30_multiseed/async_grpo_seed45_gpu0","audit_sha256":"b9bf33ce45e646392fd81d01426caf9b99bf8acfd1fb3be872da522abbcb2bc8","steps":30,"prompt_order_sha256":"982291fb89502d1893ea2ab42f2dd9d6edec973eaec49c8507f9537ab478d20f","grpo_clip_argument_lines":["eps_clip ........................................ 0.2","eps_clip_high ................................... 0.2"],"ppo_clip_low":0.2,"ppo_clip_high":0.2},{"seed":45,"method":"SC","endpoint_percent":100.0,"checkpoints_percent":{"0":8.75,"5":17.5,"10":45.0,"15":60.0,"20":82.5,"25":90.0,"30":100.0},"actual_run":"results/slime_exp30_multiseed/async_ste_seed45_gpu1","summary_record_run":"results/slime_exp30_multiseed/async_ste_seed45_gpu1","audit_sha256":"8fd39386660090d8be75b5d8b860501944fd8d884cbee744086f720ed899e8eb","steps":30,"prompt_order_sha256":"982291fb89502d1893ea2ab42f2dd9d6edec973eaec49c8507f9537ab478d20f"}],"source_path_correction":"Summary record for seed43 GRPO names gpu0, but actual audited run is async_grpo_seed43_gpu1; audit endpoint matches summary. Actual_run preserves verified location."},"batch512_train_eval":{"captured_utc":"2026-10-13T05:24:40.613038+00:00","source_host":"sb","source_directory":"/home/weicheng/yifei/pg-correction/results/slime_exp30_batch512","source_sha256":{"summary.json":"fa3d6ac4391119b4e419bdd247f0c72d0af013730b548c52beb861f492e0771f","training_accuracy_vs_steps_current.csv":"b81cad4ec462f25f9a4683e86d0e72e8e076028559c5f7043ed241fe61c99fdc","evaluation_checkpoints.csv":"cf36d7d3f5761cb582970fa7a384d68f74c85532e647d5dbf0ff8ce898ad648a","learning_curves.svg":"4ec72913eb65539f7fe1bfe4842d65c5223a7d87080a1805c8a05af11ad7d86a","training_accuracy_vs_steps_current.svg":"87107b86672bebbf6b8d5aa86787317e70b542c8111781ff9beba8233d9187fc","protocol.json":"596a8f341ba44c99e2f7d3e30fa096d6254ea94dbc6167f18d07506a44ce4bc2"},"protocol":{"created_unix":1791794273.0466247,"seed":42,"ordered_methods":["sc","corrected_sc","tis_grpo"],"optimizer_steps":30,"evaluation_steps":[0,5,10,15,20,25,30],"prompts_per_step":32,"responses_per_prompt":16,"global_batch":512,"microbatch":1,"learning_rate":1e-06,"entropy_coefficient":0.0,"kl_coefficient":0.0,"optimizer":"Adam","learning_rate_schedule":"constant","weight_decay":0.0,"data_policy":"9980-question pool; identical ordered960 distinct questions per method; no multi-epoch.","evaluation":"Same fixed20 held-out questions,4 independently seeded sampled answers per question.","model":"Qwen3.5-0.8B, original checkpoint, full text-parameter FP32 training","method_definitions":{"sc":"Existing behavior-score-centered straight-through loss (previously labelled ST).","corrected_sc":"Same loss plus first-order HVP transport using each sample actual behavior snapshot.","tis_grpo":"Native PPO-clipped GRPO with detached training-policy denominator and vanilla token-level TIS: min(p_anchor/q_behavior,2). Lower bound0, no rejection sampling or batch normalization."},"tis_clip_upper":2.0,"tis_clip_lower":0.0,"ppo_clip":0.2,"async_settings":{"worker":"fully_async","prefetch_group_cap":16,"server_concurrency":128,"worker_group_concurrency":8,"engine_max_running_requests":16,"rationale":"Scale the worker group concurrency from4/16 prompts to8/32 prompts to retain comparable stale-sample exposure. Actual per-sample ages are audited; no assumed age histogram."},"execution":"Finish the two retained16x4 jobs, then strictly sequential SC -> corrected SC -> TIS-GRPO.","no_short_training_tests":true,"interpretation":"One seed and one fixed small test set. Do not infer cross-seed robustness or call this a controlled batch-size-only intervention against the old study.","user_handoff_sha256":"fa08d693a2107899e91921d57ffc946f58468fabf67b4cabf0e8f8fa46ac9a5a"},"series":{"SC":{"train":[[1,12.5],[2,9.1796875],[3,12.890625],[4,19.7265625],[5,25.78125],[6,25.1953125],[7,36.328125],[8,41.9921875],[9,31.0546875],[10,12.6953125],[11,6.4453125],[12,5.6640625],[13,33.0078125],[14,39.6484375],[15,43.5546875],[16,30.46875],[17,43.75],[18,58.3984375],[19,53.515625],[20,51.5625],[21,70.3125],[22,68.75],[23,80.46875],[24,93.1640625],[25,89.84375],[26,87.890625],[27,99.0234375],[28,84.5703125],[29,92.7734375],[30,91.40625]],"eval":[[0,11.25],[5,22.5],[10,25.0],[15,55.0],[20,67.5],[25,97.5],[30,100.0]]},"DRSC":{"train":[[1,12.5],[2,9.375],[3,10.15625],[4,16.2109375],[5,19.7265625],[6,13.8671875],[7,22.0703125],[8,19.3359375],[9,28.125],[10,38.28125],[11,38.671875],[12,40.4296875],[13,57.6171875],[14,69.7265625],[15,87.890625],[16,89.0625],[17,87.3046875],[18,96.875],[19,89.2578125],[20,97.0703125],[21,97.0703125],[22,100.0],[23,100.0],[24,100.0],[25,97.0703125],[26,100.0],[27,94.140625],[28,96.875]],"eval":[[0,11.25],[5,15.0],[10,52.5],[15,95.0],[20,96.25],[25,95.0]]},"TIS-GRPO":{"train":[[1,12.5],[2,9.9609375],[3,10.546875],[4,19.53125],[5,22.265625],[6,19.921875],[7,28.90625],[8,34.1796875],[9,41.9921875],[10,56.0546875],[11,52.34375],[12,67.1875],[13,78.3203125],[14,74.4140625],[15,89.453125],[16,91.40625],[17,94.7265625],[18,97.4609375],[19,96.484375],[20,96.875],[21,88.0859375],[22,97.65625],[23,99.0234375],[24,97.8515625],[25,96.875],[26,96.09375],[27,96.875],[28,99.609375],[29,100.0],[30,100.0]],"eval":[[0,11.25],[5,21.25],[10,68.75],[15,100.0],[20,93.75],[25,95.0],[30,97.5]]}},"run_status":[{"method":"sc","status":"completed"},{"method":"corrected_sc","status":"running"},{"method":"tis_grpo","status":"completed"}],"training_metric":"Raw correct responses / 512 from the actual rollout batch consumed before each optimizer update, with some stale behavior checkpoints; not a fixed training-set evaluation.","evaluation_metric":"Exact-match sampled answers / 80, on the same fixed 20 held-out questions with four draws each.","scope":"Single seed 42. Partial DRSC curve; no inferred endpoint. Not a batch-size-only ablation of the prior 16x4 experiment."}}