Qwen Support-Contrastive Meta-ICL
The one idea you need
Think of the worked examples in a prompt as a recipe card handed to a cook. One who memorized the dish cooks it even from a scrambled card: high score, card ignored. Contrastive training rewards only the cook who actually reads the card.
The question
When you fine-tune a small model to follow worked examples in its prompt, does it actually read them, or just recognize the task and answer from memory?
What we found
Not by default. One tuned model scored 49% whether its worked examples were intact or scrambled, meaning it had memorized the task and ignored the examples entirely. A contrastive objective forced genuine reading: 50% with intact examples, collapsing to 7% when scrambled. The catch is this honesty cost a few points versus plain tuning's peak of 56%.
Why it matters
Before trusting a fine-tuned model to follow in-prompt examples, rerun it with those examples scrambled. If accuracy barely drops, it is ignoring your demonstrations and just pattern-matching the task family, and a high raw score hides exactly that.
On this page
Results at a glance 6
How to read
Five training recipes across the bottom; three bars each for full-task accuracy with real examples, scrambled examples, and no examples in the prompt. Taller is better; a big drop from real to scrambled means the recipe truly reads its examples.
Takeaway → The contrastive recipe's real-example bar near half towers over its scrambled bar at 7%, while the shuffled-labels recipe's two bars are equal, so it ignores examples.
Data table
| training arm | normal support | shuffled support | no support |
|---|---|---|---|
| base | 17.8% | 6.7% | 2.2% |
| CE ordinary | 55.6% | 22.2% | 4.4% |
| CE counterfactual | 53.3% | 33.3% | 4.4% |
| CE shuffled-labels | 48.9% | 48.9% | 2.2% |
| contrastive | 50.4% | 7.4% | 0% |
Numbers from experiments/qwen_support_contrastive_meta_icl/analysis/aggregate_summary.csv (public_prose split)
Technical framing
Public task accuracy by training arm and support condition — Contrastive training nearly matches CE accuracy with real supports but collapses to ~7% when supports are corrupted — it actually reads them.
How to read
One pair of bars per recipe, public benchmark and synthetic tasks, showing accuracy with real examples minus accuracy with scrambled ones. Taller is better here: a bigger drop means the answer truly depends on the examples.
Takeaway → Contrastive towers over every recipe on both sets (43 and 77 points); the shuffled-labels recipe sits at zero, confirming it disregards the demonstrations.
Data table
| training arm | public benchmark | synthetic counterfactual |
|---|---|---|
| base | 11.1% | 21.7% |
| CE ordinary | 33.3% | 43.3% |
| CE counterfactual | 20% | 38.3% |
| CE shuffled-labels | 0% | 8.3% |
| contrastive | 43% | 77.2% |
Numbers from experiments/qwen_support_contrastive_meta_icl/analysis/support_gaps.csv
Technical framing
Support-binding gap (normal minus shuffled support accuracy) — The contrastive arm has the largest gap on both splits: its answers depend causally on the support examples, not just the task family.
How to read
Five recipes across the bottom on made-up transformation tasks; three bars each for real, scrambled, and no examples, as share of tasks fully correct. A high real bar next to a low scrambled bar signals the recipe reads its examples.
Takeaway → Contrastive collapses from 84% to 7% once examples are scrambled, while other recipes still score 37 to 83% on scrambled examples, the sharpest example-dependence.
Data table
| Training arm | Normal support | Shuffled support | No support |
|---|---|---|---|
| base | 43.3% | 21.7% | 1.7% |
| ce_cf | 85% | 46.7% | 0% |
| ce_ordinary | 80% | 36.7% | 0% |
| ce_shuffled_labels | 91.7% | 83.3% | 1.7% |
| contrastive_cf | 83.9% | 6.7% | 0% |
Numbers from reports/qwen_support_contrastive_meta_icl_report.md (Aggregate Metrics table, synthetic_counterfactual rows)
Technical framing
Synthetic counterfactual accuracy by arm and support condition — On the synthetic split, contrastive training collapses shuffled-support accuracy to 6.7% while CE arms still score 36.7-83.3%.
How to read
Ten task families (dates, numbers, phone, email, shipping codes and more) across the bottom; two bars each for the untrained base and the contrastive-tuned model, as share of tasks fully correct. A higher contrastive bar means the tuning helped.
Takeaway → Contrastive lifts most families, several from 0% to 100%; no family regresses, and only the billing-code family stays stuck at 0% (with zip codes already tied at 100%).
Data table
| Task family | Base model | Contrastive arm |
|---|---|---|
| ShippingCode | 0% | 100% |
| Log | 0% | 100% |
| Name | 0% | 77.8% |
| 25% | 66.7% | |
| DateTime | 6.2% | 39.6% |
| Number | 25% | 47.2% |
| Phone | 25% | 41.7% |
| BillingCode | 0% | 0% |
| ZipCode | 100% | 100% |
| Address | 0% | 100% |
Numbers from reports/qwen_support_contrastive_meta_icl_report.md (Public Family Breakdown table)
Technical framing
Public task families: base vs contrastive full-task accuracy — Contrastive gains span most families (ShippingCode/Log/Name jump 78-100 points); only Address regresses and BillingCode stays at zero.
How to read
Three training runs with different random starts across the bottom; two bars each for public benchmark and synthetic tasks, both measured with real examples. Steady bar heights across runs mean the result is reliable, not luck.
Takeaway → Public accuracy holds near half across all three runs; synthetic accuracy wobbles more, dipping to 77% on the third run.
Data table
| Training seed | Public prose | Synthetic counterfactual |
|---|---|---|
| seed 20260628 | 51.1% | 88.3% |
| seed 20260629 | 48.9% | 86.7% |
| seed 20260630 | 51.1% | 76.7% |
Numbers from reports/qwen_support_contrastive_meta_icl_report.md (Seed-Level Normal-Support Metrics table)
Technical framing
Contrastive arm accuracy across three training seeds — Public accuracy is stable across seeds (48.9-51.1%); synthetic accuracy varies more, with seed 3 dropping to 76.7%.
How to read
Three training runs across the bottom; three bars each counting, out of 45 public tasks, how many contrastive tuning helped, hurt, or left tied versus the untrained base. More helped and fewer hurt is better.
Takeaway → Every run helps 17 to 18 tasks and hurts exactly 3, so the gains are broad and repeatable, not a few lucky tasks.
Data table
| Training seed | Helped vs base | Hurt vs base | Tied |
|---|---|---|---|
| seed 20260628 | 18 | 3 | 24 |
| seed 20260629 | 17 | 3 | 25 |
| seed 20260630 | 18 | 3 | 24 |
Numbers from reports/qwen_support_contrastive_meta_icl_report.md (Contrastive Public Task Flips Versus Base table)
Technical framing
Public tasks helped vs hurt by contrastive training, per seed — Every seed helps 17-18 of 45 public tasks and hurts exactly 3, so the gain is broad and consistent, not a few lucky tasks.
In the author’s words from the Overview
The support-contrastive arm improves public strict task consistency over base and produces the cleanest support dependence: Base public full-task exact: 17.8%. Support-contrastive public full-task exact: 50.4% ± 1.3% over three seeds. Support-contrastive public corrupted-support controls: shuffled 7.4%, no-support 0.0%. The margin objective is not the best raw-accuracy recipe in this run: CE-only counterfactual control: 53.3%. CE-only ordinary control: 55.6%. CE-only shuffled-label control: 48.9%, with shuffled-support performance also 48.9%. The result is therefore a clean tradeoff: contrastive training buys causal support binding, but at this margin/weight it sacrifices several points of public normal-support accuracy relative to CE-only tuning. … Read the full result →
Overview
Standalone experiment testing whether a support-contrastive LoRA objective makes Qwen3-4B bind text-transformation answers to the provided support examples, rather than only adapting to the prompt format or task family.
Large artifacts, checkpoints, and the public benchmark mirror live under:
/workspace/large_artifacts/qwen_support_contrastive_meta_icl
Planned Arms
contrastive_cf: counterfactual synthetic episodes with positive CE plus support-corruption margin losses.ce_cf: same counterfactual synthetic episodes with positive CE only.ce_ordinary: ordinary synthetic few-shot episodes with positive CE only.ce_shuffled_labels: counterfactual synthetic episodes with shuffled support labels during CE training.
All arms use a fixed synthetic/public evaluation seed. The final report is generated as Markdown and HTML with charts.
Final Read
The support-contrastive arm improves public strict task consistency over base and produces the cleanest support dependence:
- Base public full-task exact:
17.8%. - Support-contrastive public full-task exact:
50.4% ± 1.3%over three seeds. - Support-contrastive public corrupted-support controls: shuffled
7.4%, no-support0.0%.
The margin objective is not the best raw-accuracy recipe in this run:
- CE-only counterfactual control:
53.3%. - CE-only ordinary control:
55.6%. - CE-only shuffled-label control:
48.9%, with shuffled-support performance also48.9%.
The result is therefore a clean tradeoff: contrastive training buys causal support binding, but at this margin/weight it sacrifices several points of public normal-support accuracy relative to CE-only tuning.
Reports:
- Markdown:
reports/qwen_support_contrastive_meta_icl_report.md - HTML:
reports/qwen_support_contrastive_meta_icl_report.html
Report
Rendered from reports/qwen_support_contrastive_meta_icl_latest_run_report.md
Setup
- Run:
main_ce_shuffled_labels_s1 - Objective:
ce - Train mode:
shuffled_labels - Train steps:
80 - Elapsed seconds:
397.1
Metrics
| method | split | support_mode | tasks | rows | row_exact | full_task_exact |
|---|---|---|---|---|---|---|
| adapter | public_prose | none | 45 | 135 | 8.1% | 2.2% |
| adapter | public_prose | normal | 45 | 135 | 68.9% | 48.9% |
| adapter | public_prose | shuffled | 45 | 135 | 63.0% | 48.9% |
| adapter | synthetic_counterfactual | contrast | 60 | 120 | 0.0% | 0.0% |
| adapter | synthetic_counterfactual | none | 60 | 120 | 5.0% | 1.7% |
| adapter | synthetic_counterfactual | normal | 60 | 120 | 95.8% | 91.7% |
| adapter | synthetic_counterfactual | shuffled | 60 | 120 | 90.8% | 83.3% |
Charts
Artifacts
- Run directory:
/workspace/experiments/qwen_support_contrastive_meta_icl/runs/main_ce_shuffled_labels_s1 - Checkpoint:
/workspace/large_artifacts/qwen_support_contrastive_meta_icl/checkpoints/main_ce_shuffled_labels_s1/adapter
Experiment log 13
Show the running log (13 entries)
Setup
- Created fresh standalone experiment directory:
/workspace/experiments/qwen_support_contrastive_meta_icl. - Large artifacts directory:
/workspace/large_artifacts/qwen_support_contrastive_meta_icl. - Primary question: does an explicit support-contrastive loss improve public text-transformation accuracy while making performance depend more causally on intact support examples?
- Planned process: implement harness, run smoke, run pilot, run main fixed-eval matrix, aggregate with Markdown and HTML reports.
Smoke smoke_contrastive
- Contrastive training path compiled and ran for two optimizer steps.
- Multi-forward margin loss did not OOM.
- Smoke aggregation produced Markdown and HTML reports.
- Patched shuffled-support eval seeding to use a deterministic support-mode offset.
Pilot
pilot_contrastive_cf: 18 public tasks, 24 synthetic counterfactual tasks, 40 optimizer steps.- Public full-task exact: normal
55.6%, shuffled5.6%, no support0.0%. - Synthetic full-task exact: normal
79.2%, shuffled12.5%, contrast support0.0%, no support0.0%.
- Public full-task exact: normal
pilot_ce_cf: same fixed eval, CE-only counterfactual control, 40 optimizer steps.- Public full-task exact: normal
50.0%, shuffled27.8%, no support5.6%. - Synthetic full-task exact: normal
83.3%, shuffled41.7%, contrast support0.0%.
- Public full-task exact: normal
- Pilot read: the support-contrastive objective did not improve normal public accuracy over CE in the pilot, but it produced a much cleaner corrupted-support gap. Proceeding to the full fixed-eval matrix.
Main Matrix
Completed:
main_contrastive_s1,main_contrastive_s2,main_contrastive_s3main_ce_cf_s1main_ce_ordinary_s1main_ce_shuffled_labels_s1
Fixed evaluation:
- 45 public PROSE tasks, 3 held-out rows per task.
- 60 synthetic counterfactual tasks, 2 held-out rows per task.
Public full-task exact:
- Base:
17.8%. - Support-contrastive mean:
50.4%, std1.3%. - CE counterfactual:
53.3%. - CE ordinary:
55.6%. - CE shuffled-label:
48.9%.
- Base:
Public corrupted-support controls:
- Support-contrastive: normal
50.4%, shuffled7.4%, no-support0.0%. - CE counterfactual: normal
53.3%, shuffled33.3%, no-support4.4%. - CE ordinary: normal
55.6%, shuffled22.2%, no-support4.4%. - CE shuffled-label: normal
48.9%, shuffled48.9%, no-support2.2%.
- Support-contrastive: normal
Synthetic corrupted-support controls:
- Support-contrastive: normal
83.9%, shuffled6.7%, contrast-support0.0%, no-support0.0%. - CE shuffled-label: normal
91.7%, shuffled83.3%.
- Support-contrastive: normal
- Main read: the support-contrastive objective does what it was designed to do: it makes performance causally depend on intact support examples. It is not the best raw public-accuracy recipe at this margin/weight, because CE-only ordinary training reaches higher normal-support accuracy. The live next question is tuning or scheduling the margin so it preserves CE accuracy while keeping the corruption gap.
- Final Markdown report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_report.md - Final HTML report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_report.html
Run smoke_contrastive
- Time UTC:
2026-06-28T16:41:31.417074+00:00 - Config:
{"contrast_weight": 0.35, "eval_seed": 20260702, "grad_accum": 2, "log_every": 1, "lora_alpha": 32, "lora_dropout": 0.05, "lora_r": 16, "lr": 0.0002, "margin": 0.75, "max_grad_norm": 1.0, "max_length": 768, "max_new_tokens": 24, "model": "Qwen/Qwen3-4B", "no_train": false, "objective": "support_contrastive", "public_heldout_n": 3, "public_task_limit": 2, "query_n": 2, "run_name": "smoke_contrastive", "seed": 20260628, "skip_base_eval": false, "smoke": true, "support_n": 4, "synthetic_eval_pairs": 2, "train_episodes": 16, "train_mode": "counterfactual", "train_rows_per_episode": 2, "train_steps": 2, "warmup_steps": 20, "weight_decay": 0.0} - Elapsed seconds:
107.7 - Summary rows:
14 - Latest run report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_latest_run_report.md
Run pilot_contrastive_cf
- Time UTC:
2026-06-28T16:45:45.172706+00:00 - Config:
{"contrast_weight": 0.35, "eval_seed": 20260702, "grad_accum": 4, "log_every": 5, "lora_alpha": 32, "lora_dropout": 0.05, "lora_r": 16, "lr": 0.0002, "margin": 0.75, "max_grad_norm": 1.0, "max_length": 768, "max_new_tokens": 24, "model": "Qwen/Qwen3-4B", "no_train": false, "objective": "support_contrastive", "public_heldout_n": 3, "public_task_limit": 18, "query_n": 2, "run_name": "pilot_contrastive_cf", "seed": 20260628, "skip_base_eval": false, "smoke": false, "support_n": 4, "synthetic_eval_pairs": 12, "train_episodes": 300, "train_mode": "counterfactual", "train_rows_per_episode": 2, "train_steps": 40, "warmup_steps": 20, "weight_decay": 0.0} - Elapsed seconds:
564.0 - Summary rows:
14 - Latest run report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_latest_run_report.md
Run pilot_ce_cf
- Time UTC:
2026-06-28T16:55:23.733550+00:00 - Config:
{"contrast_weight": 0.35, "eval_seed": 20260702, "grad_accum": 4, "log_every": 5, "lora_alpha": 32, "lora_dropout": 0.05, "lora_r": 16, "lr": 0.0002, "margin": 0.75, "max_grad_norm": 1.0, "max_length": 768, "max_new_tokens": 24, "model": "Qwen/Qwen3-4B", "no_train": false, "objective": "ce", "public_heldout_n": 3, "public_task_limit": 18, "query_n": 2, "run_name": "pilot_ce_cf", "seed": 20260628, "skip_base_eval": true, "smoke": false, "support_n": 4, "synthetic_eval_pairs": 12, "train_episodes": 300, "train_mode": "counterfactual", "train_rows_per_episode": 2, "train_steps": 40, "warmup_steps": 20, "weight_decay": 0.0} - Elapsed seconds:
171.4 - Summary rows:
7 - Latest run report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_latest_run_report.md
Run main_contrastive_s1
- Time UTC:
2026-06-28T16:59:13.158687+00:00 - Config:
{"contrast_weight": 0.35, "eval_seed": 20260702, "grad_accum": 4, "log_every": 10, "lora_alpha": 32, "lora_dropout": 0.05, "lora_r": 16, "lr": 0.0002, "margin": 0.75, "max_grad_norm": 1.0, "max_length": 768, "max_new_tokens": 24, "model": "Qwen/Qwen3-4B", "no_train": false, "objective": "support_contrastive", "public_heldout_n": 3, "public_task_limit": 45, "query_n": 2, "run_name": "main_contrastive_s1", "seed": 20260628, "skip_base_eval": false, "smoke": false, "support_n": 4, "synthetic_eval_pairs": 30, "train_episodes": 700, "train_mode": "counterfactual", "train_rows_per_episode": 2, "train_steps": 80, "warmup_steps": 20, "weight_decay": 0.0} - Elapsed seconds:
1135.6 - Summary rows:
14 - Latest run report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_latest_run_report.md
Run main_contrastive_s2
- Time UTC:
2026-06-28T17:18:31.066226+00:00 - Config:
{"contrast_weight": 0.35, "eval_seed": 20260702, "grad_accum": 4, "log_every": 10, "lora_alpha": 32, "lora_dropout": 0.05, "lora_r": 16, "lr": 0.0002, "margin": 0.75, "max_grad_norm": 1.0, "max_length": 768, "max_new_tokens": 24, "model": "Qwen/Qwen3-4B", "no_train": false, "objective": "support_contrastive", "public_heldout_n": 3, "public_task_limit": 45, "query_n": 2, "run_name": "main_contrastive_s2", "seed": 20260629, "skip_base_eval": true, "smoke": false, "support_n": 4, "synthetic_eval_pairs": 30, "train_episodes": 700, "train_mode": "counterfactual", "train_rows_per_episode": 2, "train_steps": 80, "warmup_steps": 20, "weight_decay": 0.0} - Elapsed seconds:
754.6 - Summary rows:
7 - Latest run report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_latest_run_report.md
Run main_contrastive_s3
- Time UTC:
2026-06-28T17:31:23.436980+00:00 - Config:
{"contrast_weight": 0.35, "eval_seed": 20260702, "grad_accum": 4, "log_every": 10, "lora_alpha": 32, "lora_dropout": 0.05, "lora_r": 16, "lr": 0.0002, "margin": 0.75, "max_grad_norm": 1.0, "max_length": 768, "max_new_tokens": 24, "model": "Qwen/Qwen3-4B", "no_train": false, "objective": "support_contrastive", "public_heldout_n": 3, "public_task_limit": 45, "query_n": 2, "run_name": "main_contrastive_s3", "seed": 20260630, "skip_base_eval": true, "smoke": false, "support_n": 4, "synthetic_eval_pairs": 30, "train_episodes": 700, "train_mode": "counterfactual", "train_rows_per_episode": 2, "train_steps": 80, "warmup_steps": 20, "weight_decay": 0.0} - Elapsed seconds:
760.6 - Summary rows:
7 - Latest run report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_latest_run_report.md
Run main_ce_cf_s1
- Time UTC:
2026-06-28T17:44:32.745451+00:00 - Config:
{"contrast_weight": 0.35, "eval_seed": 20260702, "grad_accum": 4, "log_every": 10, "lora_alpha": 32, "lora_dropout": 0.05, "lora_r": 16, "lr": 0.0002, "margin": 0.75, "max_grad_norm": 1.0, "max_length": 768, "max_new_tokens": 24, "model": "Qwen/Qwen3-4B", "no_train": false, "objective": "ce", "public_heldout_n": 3, "public_task_limit": 45, "query_n": 2, "run_name": "main_ce_cf_s1", "seed": 20260628, "skip_base_eval": true, "smoke": false, "support_n": 4, "synthetic_eval_pairs": 30, "train_episodes": 700, "train_mode": "counterfactual", "train_rows_per_episode": 2, "train_steps": 80, "warmup_steps": 20, "weight_decay": 0.0} - Elapsed seconds:
410.6 - Summary rows:
7 - Latest run report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_latest_run_report.md
Run main_ce_ordinary_s1
- Time UTC:
2026-06-28T17:51:37.497945+00:00 - Config:
{"contrast_weight": 0.35, "eval_seed": 20260702, "grad_accum": 4, "log_every": 10, "lora_alpha": 32, "lora_dropout": 0.05, "lora_r": 16, "lr": 0.0002, "margin": 0.75, "max_grad_norm": 1.0, "max_length": 768, "max_new_tokens": 24, "model": "Qwen/Qwen3-4B", "no_train": false, "objective": "ce", "public_heldout_n": 3, "public_task_limit": 45, "query_n": 2, "run_name": "main_ce_ordinary_s1", "seed": 20260628, "skip_base_eval": true, "smoke": false, "support_n": 4, "synthetic_eval_pairs": 30, "train_episodes": 700, "train_mode": "ordinary", "train_rows_per_episode": 2, "train_steps": 80, "warmup_steps": 20, "weight_decay": 0.0} - Elapsed seconds:
374.7 - Summary rows:
7 - Latest run report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_latest_run_report.md
Run main_ce_shuffled_labels_s1
- Time UTC:
2026-06-28T17:58:05.949981+00:00 - Config:
{"contrast_weight": 0.35, "eval_seed": 20260702, "grad_accum": 4, "log_every": 10, "lora_alpha": 32, "lora_dropout": 0.05, "lora_r": 16, "lr": 0.0002, "margin": 0.75, "max_grad_norm": 1.0, "max_length": 768, "max_new_tokens": 24, "model": "Qwen/Qwen3-4B", "no_train": false, "objective": "ce", "public_heldout_n": 3, "public_task_limit": 45, "query_n": 2, "run_name": "main_ce_shuffled_labels_s1", "seed": 20260628, "skip_base_eval": true, "smoke": false, "support_n": 4, "synthetic_eval_pairs": 30, "train_episodes": 700, "train_mode": "shuffled_labels", "train_rows_per_episode": 2, "train_steps": 80, "warmup_steps": 20, "weight_decay": 0.0} - Elapsed seconds:
397.1 - Summary rows:
7 - Latest run report:
/workspace/experiments/qwen_support_contrastive_meta_icl/reports/qwen_support_contrastive_meta_icl_latest_run_report.md
Figures 6
Data files 24
Result tables and metrics copied from the experiment folder — preview inline or open the raw file.
analysis/aggregate_summary.csv3.3 kBruns/main_ce_cf_s1/eval_episodes.csv58 kBruns/main_ce_cf_s1/summary.csv557 Bruns/main_ce_cf_s1/task_metrics.csv27 kBruns/main_ce_ordinary_s1/eval_episodes.csv58 kBruns/main_ce_ordinary_s1/summary.csv558 Bruns/main_ce_ordinary_s1/task_metrics.csv27 kBruns/main_ce_shuffled_labels_s1/eval_episodes.csv58 kBruns/main_ce_shuffled_labels_s1/summary.csv585 Bruns/main_ce_shuffled_labels_s1/task_metrics.csv27 kBruns/main_contrastive_s1/eval_episodes.csv58 kBruns/main_contrastive_s1/summary.csv1.0 kBruns/main_contrastive_s1/task_metrics.csv53 kBruns/main_contrastive_s2/eval_episodes.csv58 kBruns/main_contrastive_s2/summary.csv508 Bruns/main_contrastive_s2/task_metrics.csv27 kBruns/main_contrastive_s3/eval_episodes.csv58 kBruns/main_contrastive_s3/summary.csv538 Bruns/main_contrastive_s3/task_metrics.csv27 kBruns/pilot_ce_cf/eval_episodes.csv22 kBruns/pilot_ce_cf/summary.csv562 Bruns/pilot_ce_cf/task_metrics.csv11 kBruns/pilot_contrastive_cf/eval_episodes.csv22 kBruns/pilot_contrastive_cf/summary.csv978 B
4 more result files not shown here — browse the full folder on GitHub.
Reproduce
This entry is source code or analysis only — there is no separate run to reproduce.