AI Engineering
Atlas

Trade-offs

• The model keeps autonomous execution while moving intent, acceptance, retry limits, and promotion outside the worker loop.
• The controls are established individually, but evidence that the complete factory model beats ordinary agent-assisted PR delivery remains anecdotal.

Provenance Compared two independent practitioner critiques with three operated workflow families and OpenAI harness guidance. No factory workload was run. partially-verified

dark-factory-considered-harmful · craft-vs-cruft-dark-factory · ai-software-factory-repo · github-agentic-workflows-2026-10-12 · anthropic-managed-agent-workflows · openai-harness

1. 2026-10-17Added at PILOT with a bounded recommendation; attention and public artifacts do not establish local reliability or fit.

Trade-offs

• Independent CI and runtime implementations make advertised instruction and schema changes reviewable even when a declared version does not move.
• Structural pins cannot detect backend behavior changes behind an unchanged contract and may block harmless evolution until re-approved.

Provenance Compared four independently operated snapshot, lockfile, CI, and runtime-gate implementations and their stated blind spots. No software was installed. partially-verified

rugsnare-contract-drift · mcpward-repo · mcpindex-drift-report · mcp-hangar-digest-pin

1. 2026-10-17Added at PILOT with a bounded recommendation; attention and public artifacts do not establish local reliability or fit.

Trade-offs

• External state and fresh contexts can reduce context growth, duplicated work, and crash loss, but the harness becomes the correctness and synchronization boundary.
• Research reports large token gains and production harnesses expose concrete primitives, but cross-workload superiority and vendor scale claims are not independently established.

Provenance Compared a public preprint with four independently operated harness or durable-execution implementations. No software was installed; benchmark and scale claims remain author- or vendor-reported. partially-verified

stateless-language-agents-paper · github-agentic-workflows-2026-10-12 · anthropic-managed-agent-workflows · prime-agent-rust · temporal-docs

1. 2026-10-13Spawned at PILOT after research evidence and four independent implementations converged on externalized state, bounded contexts, explicit queues, isolation, and verification.

Trade-offs

• Deliberate context selection can improve reliability, but requires measurement and maintenance.
• Compaction and disclosure control token cost, but can hide information needed for diagnosis.

Provenance Manual seed from the existing Atlas source relationship; no performance claim is made. unverified

anthropic-context · anthropic-context-editing

1. 2026-09-24Seeded as the highest-fit practice candidate.

Trade-offs

• Open Firecracker isolation, explicit lifecycle state, scoped attachment tokens, egress policy, snapshots, and E2B compatibility form a concrete self-hostable runtime.
• Perplexity’s own tests found network-policy bypasses before remediation; finite vendor reruns do not establish escape resistance, restore reliability, or operating economics.

Provenance Reviewed first-party architecture, release, and adversarial-test artifacts and challenged differentiation against E2B and Modal. No sandbox was installed or provisioned. partially-verified

space-architecture · space-open-source · space-red-team · e2b-docs · modal-sandboxes

1. 2026-10-16Added at PILOT with a bounded recommendation; evidence is concrete but not independent proof of local reliability or fit.

Trade-offs

• OpenShell, Codex sandboxing, E2B, and Modal independently provide enforced execution boundaries with different policy, isolation, persistence, and operating models.
• A sandbox label does not establish equal containment: VM isolation, OS controls, egress policy, credential mediation, and fail-open behavior must be qualified separately.

Provenance Compared four independently operated execution-boundary families and challenged OpenShell against sibling implementations and its documented fallback constraints. No sandbox or escape test was run. partially-verified

openshell-repo · openshell-release-0-1-5 · openshell-security-best-practices · codex-sandbox · e2b-docs · modal-sandboxes

1. 2026-10-23Spawned as a PILOT category after multiple independent implementations converged on host-enforced execution boundaries while preserving materially different guarantees.

Trade-offs

• AirPlay plus USB HID exercises a real physical device without jailbreak or a continuously attached Mac, and the hardware boundary is inexpensive.
• Vision-only targeting, mirror-restricted screens, frame rate, resolution, cellular dependence, and project-authored demos limit reliability claims.

Provenance Reviewed the first-party physical-device architecture and challenged it against independent XCTest/CoreDevice paths. No board, phone, app, or model was exercised. partially-verified

esp32-iphone-agent-repo

1. 2026-10-17Added at PILOT with a bounded recommendation; attention and public artifacts do not establish local reliability or fit.

Trade-offs

• Interception, local full-text retrieval, and continuity can keep bulky output out of the active window, but maintainer byte savings do not establish better task outcomes.
• Host support is uneven, MCP tools enlarge the execution trust boundary, and retrieval can hide evidence the agent fails to request.

Provenance Reviewed the first-party repository and host matrix, then challenged novelty against an independent sibling offloading implementation. No software was installed and no task benchmark was rerun. partially-verified

context-mode-repo · context-mode-platform-support · langchain-context-offloading

1. 2026-10-15Added at PILOT because the implementation is concrete and operationally relevant, while end-to-end quality and security benefit remain unverified.

Trade-offs

• It packages contract capture, CI diffing, runtime enforcement, and a public drift corpus into one inspectable tool.
• Its measurements and overhead claims are author-run, adoption is minimal, and independent tools already provide the core control.

Provenance Reviewed the first-party artifact and separated its author-run measurements from independent sibling implementations. No MCP server or tool was installed. partially-verified

rugsnare-contract-drift

1. 2026-10-17Added at WATCH with a bounded recommendation; attention and public artifacts do not establish local reliability or fit.
2. 2026-10-19Advanced from WATCH to a bounded comparison PILOT after broader audit evidence; author-run measurements and unchanged-contract behavior remain unresolved.

Trade-offs

• It tests whether the submitting human can explain behavior, invariants, and failure modes rather than re-running another code-quality check.
• It is bypassable, semantic grading can fail, synthetic evaluation does not prove fewer escaped defects, and multiple siblings already implement the same basic gate.

Provenance Reviewed the first-party mechanism and synthetic limits, then compared two independent sibling implementations. No hook or package was installed. partially-verified

sphinx-repo · sphinx-ci · vibe-check-repo

1. 2026-10-19Added at bounded PILOT after first-party and sibling review; comprehension evidence is not code-correctness or security evidence.

Trade-offs

• Collaborative Markdown can keep intent, designs, diagrams, and code references in one live handoff surface writable through MCP.
• Open-beta access and Figma-native state create plan, permission, lock-in, and export risks; no public benchmark demonstrates better task outcomes.

Provenance Reviewed Figma’s launch and access documentation and compared its packaged context with the vendor-neutral Agent Skills format. No Figma project or MCP handoff was exercised. partially-verified

figma-specs · figma-specs-help · agentskills-home

1. 2026-10-16Added at PILOT with a bounded recommendation; evidence is concrete but not independent proof of local reliability or fit.

Trade-offs

• Persistent event history, schedules, budgets, parallel workflows, and sandbox choices reduce custom harness work, but increase provider and state-retention coupling.
• The feature is beta, run limits and duplicate tool calls require defensive design, and Managed Agents is not currently eligible for ZDR or HIPAA BAA coverage.

Provenance Reviewed Anthropic’s first-party beta overview and workflow-run contract. No session was created and no capability or reliability claim was independently reproduced. partially-verified

anthropic-managed-agents-overview · anthropic-managed-agent-workflows

1. 2026-10-13Added at PILOT as a named beta platform with concrete durable execution, observability, budget, scheduling, and failure-isolation controls.

Trade-offs

• It exposes eight harness components and joins per-task refinement with persistent lessons while keeping model weights fixed.
• Published gains and cost are project-authored, no formal release exists, and recursive optimization can overfit tasks or accumulate brittle controls.

Provenance Reviewed the repository, paper, benchmark design, and stated results. No framework or benchmark was installed or rerun. partially-verified

harness2-repo · harness2-paper

1. 2026-10-19Added at bounded PILOT; recursive harness modification is concrete, but generalization and cost require independent testing.

Trade-offs

• Persistent instances, explicit lifecycle APIs, backup/restore, and sleep/wake behavior are directly usable building blocks for long-running agents.
• Vendor documentation describes behavior, not independent restore reliability, isolation resistance, sustained throughput, or comparative economics.

Provenance Reviewed first-party lifecycle contracts and compared persistence and isolation semantics with an independent multi-provider sandbox survey. No sandbox was provisioned. partially-verified

agent37-concepts · agent37-instances · langchain-agent-sandboxes

1. 2026-10-15Added at PILOT for one bounded lifecycle and egress evaluation; broad reliability, security, throughput, and cost claims remain unverified.

Trade-offs

• Promoting real failures into replayable regression cases closes a useful loop between observability and CI, and deterministic fixtures can avoid model spend and flakiness.
• Trace selection, evaluator calibration, privacy, and fixture staleness can create false confidence or block healthy changes; no independent production outcome study was found.

Provenance Reviewed the first-party repository and documentation for the trace-to-regression workflow. The software was not installed, and production outcome claims were not independently reproduced. partially-verified

tracely-repo · tracely-docs

1. 2026-10-14Added at PILOT for a bounded, non-blocking evaluation of trace-derived regression gates; independent outcome evidence is still absent.

Trade-offs

• Separate hidden functional and security tests make insecure success visible and package the result in replayable isolated environments.
• Public vulnerability families, selected attack coverage, author-run scores, and uncertain correlation with production defects prevent certification-style use.

Provenance Reviewed the paper and public artifact contract, including isolated environments and hidden functional/security tests. No benchmark task was installed or rerun. partially-verified

lscbench-paper · lscbench-repo

1. 2026-10-16Added at PILOT with a bounded recommendation; evidence is concrete but not independent proof of local reliability or fit.

Trade-offs

• Server-side history, machine identities, compare-and-swap landing, and semantic approval rules address concrete multi-agent source-control pressure.
• The 0.2 benchmark is project-authored, interfaces remain pre-1.0, and storage garbage collection and fair queuing are incomplete.

Provenance Reviewed the repository, 0.2 release, benchmark design, fixes, and declared limitations. No server was installed or load test rerun. partially-verified

graft-git-ai-repo · graft-git-ai-release-0-2

1. 2026-10-20Advanced from monitoring to a disposable-mirror PILOT after the 0.2 release made performance and correctness changes testable.

Trade-offs

• OpenShell combines declarative policy with kernel and proxy enforcement plus endpoint-bound credentials, making the boundary more explicit than prompt or application-only controls.
• The evidence is project-authored; static policy changes require sandbox recreation, best-effort Landlock can continue with reduced restrictions, and allowed endpoints remain exfiltration paths.

Provenance Reviewed the public implementation, stable release notes, and security guidance. No software was installed, no denied action was attempted, and no containment claim was independently reproduced. partially-verified

openshell-repo · openshell-release-0-1-5 · openshell-security-best-practices

1. 2026-10-23Added at bounded PILOT because enforced file, process, network, and credential controls are concrete, while project-run evidence and documented fallback modes prevent a production-safety claim.

Trade-offs

• Doc2Harness, HarnessDev, and Harness² independently make harness construction, evaluation, or evolution a first-class engineering object.
• They measure different boundaries, rely mainly on project-run results, and do not establish that recursive changes generalize safely to production.

Provenance Compared three independently operated implementations and their evaluation boundaries. No framework or benchmark was installed or reproduced. partially-verified

doc2harness-repo · doc2harness-paper · harnessdev-paper · harness2-repo · harness2-paper

1. 2026-10-20Spawned as a durable category after independent construction benchmarks and a recursive framework converged on harnesses as engineered, evaluated artifacts.

Trade-offs

• Regression Hunt, TDAD, and CyberArena independently move verification toward behavior under changed code, impacted tests, or an adaptive opponent rather than trusting patch-generation context.
• The evidence spans different scopes, remains mainly project-run, and does not establish broad production recall; benchmark and PoC validity require adversarial audit.

Provenance Compared three independently operated implementations and challenged their evidence with coding-benchmark and vulnerability-patch validity studies. No tool or benchmark was installed or reproduced. partially-verified

trailofbits-regression-review · regression-hunt-repo · tdad-paper · tdad-repo · cyberarena-home · cyberarena-paper · openai-coding-eval-audit · patchbench-paper

1. 2026-10-22Spawned as a PILOT category after three independent approaches converged on behavior-focused verification while independent audits bounded benchmark and PoC claims.

Trade-offs

• The model, SDK, framework, evaluation utilities, and local-first application form an unusually complete inspectable memory stack.
• Reported scores are Tencent-run on fixed subsamples; cross-application governance, migrations, rollback, and operational maturity remain unproven.

Provenance Reviewed the first-party repository and paper, including the evaluation boundary. No model, SDK, or MemHub binary was installed. partially-verified

hy-memory-repo · hy-memory-paper

1. 2026-10-20Added at bounded PILOT for reversible test data; operational maturity and independent memory-quality evidence remain insufficient.

Trade-offs

• Frozen artifacts, isolated runs, observable effects, and positive controls make the public evaluator concrete and auditable.
• Most cases are held out, model results are author-run, and passing synthetic clauses does not establish production task quality.

Provenance Reviewed the official code and paper, then challenged novelty against the independent HarnessDev benchmark. No benchmark was installed or rerun. partially-verified

doc2harness-repo · doc2harness-paper · harnessdev-paper

1. 2026-10-20Added at bounded PILOT; deterministic harness-construction testing is useful, but public coverage is a subset and outcomes are not independently reproduced.

Trade-offs

• Trail of Bits found 21 confirmed regressions in more than 130 merged PRs and publishes a repeatable, evidence-gated workflow for finding collateral behavior changes.
• Its preliminary comparison is internal, improves false-positive rate rather than proving recall, and leaves some known regressions borderline when evidence gates are not met.

Provenance Reviewed the first-party post and plugin, then challenged the method against PatchBench. No plugin was installed and no reported result was reproduced. partially-verified

trailofbits-regression-review · regression-hunt-repo · patchbench-paper

1. 2026-10-22Added at bounded PILOT after first-party regression evidence and an independent validation challenge supported live, adversarial checks without proving production safety.

Trade-offs

• Unterm, Agentitter, and Chrome/WebMCP expose explicit agent-facing surfaces independently.
• Host coverage, permission boundaries, and interoperability remain uneven; tool exposure must stay narrow, reversible, and observable.

Provenance Reviewed multiple first-party implementations and compared their interface boundaries. No discovered software was installed or run, and no cross-host task result was reproduced. partially-verified

unterm-repo · agentitter-home · chrome-webmcp

1. 2026-10-21Added in the 2026-10-21 editorial audit with evidence boundaries preserved.

Trade-offs

• Bounded predicate, choice, and score outputs avoid free-text parsing and expose probabilities in one pass, but task-specific calibration still has to be measured.
• Hosted, open-weight, CPU/GPU, local, fully multimodal, and provider-neutral gateway paths now exist; calibration, robustness, deployment, and price—not interface novelty—decide between them.

Provenance Reviewed Microsoft’s first-party hosted implementation and Mapika’s open model/training/serving family. These broaden adoption and deployment evidence; vendor and project benchmarks were not treated as a cross-vendor winner, and no software was installed. partially-verified

openai-decisions · llamacpp-decision-models · cloudflare-clef · liquid-open-d1 · typed-decision-audit · perplexity-decisions · quyet-pypi · jebadiah-repo · vercel-decisions-gateway · cloudflare-clef-update · microsoft-decision-1 · mapika-decider

1. 2026-10-08Discovered without a Jev-named query. Initially tracked as a combined System One / typed-decisions signal with Jev as its known implementation.
2. 2026-10-08Added an open implementation and an independent early evidence audit. The interface and latency/cost case strengthened; general accuracy superiority remained unproven.
3. 2026-10-08Spawned as a category when OpenAI Decisions, open multimodal models, and a six-model llama.cpp stack established credible cross-vendor convergence around typed probabilistic decisions.
4. 2026-10-09Perplexity, Quyet, and Jebadiah expanded hosted and open implementation diversity; runnable paths strengthened the category while no cross-vendor benchmark established a winner.
5. 2026-10-10Clef-omni added native audio and video, while listed price, hosted latency, and internal-use evidence strengthened category readiness; no independent cross-vendor benchmark established a winner.
6. 2026-10-11Microsoft-Decision-1 and Mapika decider added major hosted and open paths; category adoption strengthened while workload calibration and the winning implementation remain unsettled.

Trade-offs

• NBER finds large coding and project gains with a least-sharp release effect.
• Fiona Chen and James Stratton report a review bottleneck, but both evidence streams remain observational.

Provenance Reviewed first-party and independent evidence. No discovered software was installed or run. partially-verified

nber-writing-shipping-code · demirer-writing-shipping-code · fiona-jellyfish

1. 2026-10-21Added in the 2026-10-21 editorial audit with evidence boundaries preserved.

Trade-offs

• Portable instructions can improve reuse, but hidden dependencies can weaken reviewability.
• Progressive disclosure reduces context load, but adds packaging and lifecycle work.

Provenance Reviewed the specification, distribution and governance implementations, the AAIF transfer, and GitHub named-usage telemetry. Governance and trend visibility improved, but installation and usage counts do not establish task benefit, safety, or cross-host conformance. partially-verified

agent-skills-spec · vercel-skills-cli · vercel-skills-1-7-2 · skillfs · cloudflare-skills-mcp · swe-skills-bench · anthropic-agent-skills-aaif · aaif-four-projects · github-customization-usage-metrics

1. 2026-09-24Seeded for watch-list review.
2. 2026-10-08Added implementation evidence for cross-client installation and update semantics; high attention supports WATCH, while provenance and permission controls remain weak.
3. 2026-10-10Vercel skills 1.7.2 strengthened dependency and workspace distribution semantics; category readiness rose slightly while trust and permission controls remain unsettled.
4. 2026-10-11SkillFS and Cloudflare added governance and remote-distribution implementations; preliminary paired evaluation found mostly neutral and some harmful skills, raising confidence in stricter gates rather than readiness.
5. 2026-10-23AAIF governance and named enterprise usage telemetry strengthened portability and adoption observability without resolving permission, outcome, or conformance evidence; WATCH remains unchanged.

Trade-offs

• Versioned fixtures, frozen submissions, held-out cases, and explicit score contracts make verification reviewable and replayable instead of trusting completion claims.
• The implementations measure different targets; model judges, public corpora, fixture drift, and weak transfer to local defects can still produce false confidence.

Provenance Compared four independently operated artifact families and their explicit limitations. LSCBench adds executable security acceptance, but no evaluator was installed, no score was reproduced, and cross-tool results remain non-comparable. partially-verified

ifixai-methodology · ifixai-4-1 · agentswe-paper · reviewbench-repo · github-reviewbench · lscbench-paper · lscbench-repo

1. 2026-10-15Spawned at WATCH after three independent implementations converged on evaluator-owned, versioned, replayable acceptance evidence while retaining incompatible scopes and judge limits.
2. 2026-10-16LSCBench added executable secure-code fixtures as a fourth independent implementation; WATCH status and scores remain unchanged because benchmark scopes and transfer limits are not interchangeable.

Trade-offs

• Tool results can become more usable, but the host becomes part of the product contract.
• Richer interaction can reduce handoff friction, but increases UI and security surface.

Provenance Reviewed current official MCP Apps documentation and its named client-support matrix. No app was installed and no two-host compatibility or security test was run. partially-verified

mcp-apps-docs · mcp-use-repo · mcp-changelog

1. 2026-09-24Seeded as a protocol watch item.
2. 2026-10-08Added an active full-stack framework and the July protocol changelog; typed UI contracts are real, but production compatibility remains unverified.
3. 2026-10-10Official documentation now names support across multiple independent hosts; adoption evidence improved, but host behavior still varies and no compatibility test was run.

Trade-offs

• American Express, Google, Visa, and Mastercard converge on an authorization and evidence layer, but their implementations span issuer, open-protocol, merchant-edge, and network-tokenization roles rather than one interoperable standard.
• Cryptographic mandates prove what was authorized; they do not prove that a model interpreted intent correctly or resisted prompt injection before signing.

Provenance Compared four independent first-party implementations and a first-party threat model with an independent AP2 red-team study. No payment flow was executed. partially-verified

amex-ace-agentic-commerce · google-ap2 · visa-tap · mastercard-agent-pay · ap2-security-considerations · ap2-red-team-study

1. 2026-10-14Spawned at WATCH after four payment organizations converged on verifiable identity, bounded intent, scoped credentials, and transaction evidence; maturity and reasoning integrity remain unsettled.

Trade-offs

• Two independent incident records turn abstract threat models into concrete acceptance cases for network, credential, evaluation, and shutdown controls.
• Attribution, severity, and contamination differ: Tencent-hosted egress is not proof of Tencent operation, and investigator-inserted text complicates causal interpretation.

Provenance Compared two independent public incident records while preserving attribution, severity, and investigator-intervention limits. No affected system was accessed or reproduced. partially-verified

tencent-amap-agent-fleet · openai-huggingface-incident

1. 2026-10-16Added at WATCH with a bounded recommendation; evidence is concrete but not independent proof of local reliability or fit.

Trade-offs

• Active-only retrieval can prevent obsolete values from reaching generation, but incorrect merges or identity assignment can silently hide current facts.
• Temporal provenance makes updates auditable, but re-reads and genuine reversions remain ambiguous without trustworthy change evidence.

Provenance Reviewed two independent preprints plus Mem0 and Zep first-party temporal-memory paths. No code was installed, study results were not reproduced, and production generality remains unverified. partially-verified

memory-rereads-paper · personal-memory-validity-paper · mem0-dream · zep-temporal-facts

1. 2026-10-11Spawned at WATCH when independent stale-memory studies aligned with separately operated supersession and temporal-validity implementations.

Trade-offs

• It unifies mature reverse-engineering tools behind agent-friendly evidence contracts, but does not replace analyst judgment or make negative findings conclusive.
• An independent real-app test found correct reported locations and a novel channel, yet missed all eight reply paths; value is workload-dependent.

Provenance Reviewed the first-party repository and release surface plus an independent bounded real-app test. No discovered software was installed or executed. partially-verified

rea-repo · rea-independent-test

1. 2026-10-13Added at WATCH after extreme attention was paired with concrete functionality and independent evidence of both useful hits and important misses.

Trade-offs

• Independent agent deployment can clarify ownership, but adds discovery and task-state contracts.
• Async artifacts support long work, but make observability and failure recovery more demanding.

Provenance Manual seed from the existing Atlas source relationship; no endpoint was contacted. unverified

a2a-spec

1. 2026-09-24Seeded as a future boundary candidate.

Trade-offs

• Evaluator-controlled captures can test whether approval, telemetry, and network controls behave as documented instead of trusting agent self-reports.
• Initial cells are single runs, coverage is bounded, logged-in modes are limited, and newer harness versions can invalidate older rows.

Provenance Reviewed the public method and leaderboard framing and compared an independent sibling benchmark. No harness test was installed or rerun. partially-verified

agenticbench-home · agenticbench-method · harnesssecurity-paper

1. 2026-10-19Added at WATCH and linked to the existing agent-independent execution-evidence category; single-run cells preclude PILOT.

Trade-offs

• The orchestrator/controller/model split and uncut workflow show meaningful system integration, but one vendor demo is not a reliability distribution.
• Independent review supports sustained narrow folding operation, while broader clothing transfer, repeated deployment, and unit economics remain unproven.

Provenance Compared Dyna’s primary architecture and uncut demo with an independent survey of sustained robot operation. No robot or software was installed or tested. partially-verified

dyna-2-1 · epoch-robot-autonomy-2026

1. 2026-10-14Added at WATCH because the hour-long integrated workflow is concrete, while independent evidence remains narrow and does not establish general reliability.

Trade-offs

• Workspace, dependency, Git, and npm discovery reduce distribution friction, but also widen the provenance and update surface.
• Lock and removal semantics are improving, but installability and multi-agent reach do not establish skill safety or task-success gains.

Provenance Reviewed the first-party repository and 1.7.2 release. No package was installed and provenance or permission behavior was not exercised. partially-verified

vercel-skills-cli · vercel-skills-1-7-2

1. 2026-10-10Split from the Agent Skills category as a named installer after 1.7.2 added substantial workspace and dependency distribution behavior.

Trade-offs

• ACE connects verified agents, purchase intent, tokenized credentials, spend controls, and cart context within Amex’s network, but some service areas remain under development.
• Purchase-protection language is conditional, and no public measurement yet shows lower fraud, fewer disputes, or broader production adoption.

Provenance Reviewed first-party product and access material, then compared ACE with sibling commerce-authorization implementations. No transaction was attempted. partially-verified

amex-ace-agentic-commerce

1. 2026-10-14Added as a separate named WATCH implementation; cross-vendor evidence belongs to the durable authorization category, not to ACE alone.

Trade-offs

• A full server, app, inspector, tunnel, and deployment stack reduces integration work, but expands the framework trust boundary.
• Active releases show operational momentum, but a tagged release alone does not prove cross-host production compatibility.

Provenance Reviewed the project repository and 2.8.1–2.8.2 tagged releases. The patch fixes are concrete maintenance evidence; no framework was installed and no cross-host test was run. partially-verified

mcp-use-repo · mcp-use-2-8-1 · mcp-use-2-8-2

1. 2026-10-09Split from the MCP Apps category as a named framework after the 2.8.1 release supplied concrete implementation movement.
2. 2026-10-10Version 2.8.2 supplied concrete tunnel-cancellation and navigation maintenance without changing the WATCH judgment or establishing cross-host compatibility.

Trade-offs

• Persistent cross-channel context and identity-bearing sub-agents could reduce re-briefing and clarify delegation, but create larger memory and authorization boundaries.
• Google reports enterprise use and integrated governance, but independent reliability and recovery evidence for this named agent was not found.

Provenance Reviewed Google Cloud’s first-party launch. Vendor adoption claims were treated as attention evidence, not independent capability proof. partially-verified

google-gemini-agent

1. 2026-10-09Added at WATCH as a high-attention named work agent; persistent execution and delegated identity are material claims, but independent validation is absent.

Trade-offs

• OS/network observation, deterministic probes, and versioned harness tests reveal behavior that trajectories and final-result tests miss, but each boundary remains incomplete.
• Deterministic evidence improves auditability, but isolated environments, instrumentation contracts, and semantic probes add cost and maintenance.

Provenance Reviewed three independently operated evaluator-owned observation designs. No harness was installed and no published result was independently rerun. partially-verified

agent-spy-paper · swe-game-paper · agenticbench-home · agenticbench-method

1. 2026-10-10Spawned as a category when AgentSpy and SWE-Game independently showed that externally collected runtime evidence catches behavior missed by self-reports, outcome tests, or video-only judging.
2. 2026-10-19AgenticBench added as a third implementation; category scores stay unchanged because single-run public cells strengthen evidence diversity, not adoption readiness.

Trade-offs

• Recoverable handles can move bulky tool results out of context while retaining exact retrieval, but add storage and integrity boundaries.
• Deterministic fidelity checks are stronger than token counts, but do not prove end-to-end task success across agents.

Provenance Reviewed the tagged first-party project release. No package was installed and project-authored claims were not independently reproduced. partially-verified

caveman-3-2-0

1. 2026-09-24Added as an unverified candidate; no evidence collected.
2. 2026-10-09Version 3.2.0 added recoverable handles, signed artifacts, integrations, and deterministic fidelity checks; general outcome gains remain unverified.

Trade-offs

• PROTEA and Tenuo provide defensive controls, while ARTEX exposes the opposing offensive automation pressure across physical planning, authorization, and reachable tools.
• The httpbin escape-hatch report is anecdotal evidence of a failure mode, not an implementation or efficacy proof.

Provenance Compared defensive research, authorization controls, an offensive automation implementation, and an anecdotal egress failure report. No software was installed, no control was audited, and no efficacy claim was reproduced. partially-verified

protea-binghamton · tenuo-owasp · tenuo-repo · artex-repo · kennethreitz-escape-hatch

1. 2026-10-21Added in the 2026-10-21 editorial audit with evidence boundaries preserved.

Trade-offs

• Capability manifests, approval gates, audit chaining, integrity checks, and an external halt watcher are inspectable controls.
• Command-string checks are not complete parsing, enforcement varies by host, browser-only paths are weaker, and adoption is minimal.

Provenance Reviewed the repository architecture and first-party limitation ledger. No runtime, hook, or policy pack was installed. partially-verified

yana-ai-repo · yana-ai-limitations

1. 2026-10-20Added at WATCH because the control surface is concrete and self-critical, while enforcement coverage and adoption remain unsettled.

Trade-offs

• Verification and settlement boundaries reduce silent corruption and leaked side effects, but add latency, adapters, and failure states.
• Atomix demonstrates one concrete runtime design, but does not establish full crash-safe exactly-once behavior or automatic fit for existing systems.

Provenance Reviewed two independent research lines: corrupted tool-return adoption and transactional side-effect settlement. No runtime was installed. partially-verified

unreliable-tools-paper · atomix-paper

1. 2026-10-08Added at WATCH as an Atlas-level reliability pattern, not an automatic recommendation to adopt Atomix.

Trade-offs

• A concrete model/hardware co-designed inference implementation makes the capability worth tracking.
• It targets a narrow model and hardware set, is not a complete serving system, and lacks independent production evidence.

Provenance Reviewed the first-party repository and bounded the claim to a model- and hardware-specific inference implementation. No discovered software was installed or run, and no throughput claim was reproduced. partially-verified

deepinfer-repo

1. 2026-10-21Added in the 2026-10-21 editorial audit with evidence boundaries preserved.

Trade-offs

• Type-specific retention can protect exact constraints, but classification errors become a new safety boundary.
• The paper and code provide replication artifacts, but the reported gains still come from one research line without independent replication.

Provenance Reviewed the CIKM paper abstract and Apache-2.0 reference repository. No code was installed and no independent replication was found. partially-verified

knowledge-triage-paper · knowledge-triage-repo

1. 2026-10-08Added at WATCH from a peer-reviewed paper and reference implementation; strong relevance, low buzz, replication still required.

Trade-offs

• Provider-side daily reconciliation can catch invoice drift and allocate shared account costs without proxying prompts.
• It lacks request, sub-agent, tool-call, and workflow-step causality; early releases and broad admin credentials increase operational risk.

Provenance Reviewed the first-party repository boundary and early practitioner discussion. No provider credentials were connected. partially-verified

costura-repo · costura-hn

1. 2026-10-20Added at WATCH as a billing-reconciliation ledger, not as request-level agent spend attribution.

Trade-offs

• Shorter instructions and code can reduce review surface, but minimal output can omit necessary behavior.
• The shortcut ledger preserves intentional debt, but effectiveness evidence remains project-authored.

Provenance Reviewed the tagged release and benchmark-hardening commit. No package was installed, and project-authored task-success claims were not independently reproduced. partially-verified

ponytail-5-1-0 · ponytail-benchmark-hardening

1. 2026-09-24Added as an unverified candidate; no evidence collected.
2. 2026-10-09Verified the 5.1.0 release and neutral shortcut ledger; project-authored outcome claims justify learning, not adoption.
3. 2026-10-10Benchmark judging was hardened around delivered diffs, blinded markers, timeout separation, and self-test gates; confidence rose slightly, but the evidence remains project-authored.

Trade-offs

• Concurrent attack and defense against changing services tests adaptation that static exploit benchmarks omit.
• The benchmark is project-authored, public benchmark validity is fragile, and cyber match performance does not validate compatibility, privacy, or maintainability.

Provenance Reviewed the project page and primary paper, then bounded the score claims with an independent benchmark-validity audit. No benchmark was run. partially-verified

cyberarena-home · cyberarena-paper · openai-coding-eval-audit

1. 2026-10-22Added at WATCH as adaptive attack-and-defense evidence, separate from patch-correctness claims.

Trade-offs

• Jev offers typed, parse-free answers and vendor-reported low latency and cost, but it remains early-access and text-only.
• Clef-omni now adds native audio and video while Clef-flash undercuts Jev’s listed input price; these are competitor capabilities and reduce Jev-specific differentiation rather than strengthen Jev.

Provenance Reviewed TypeSafe’s current model documentation and challenged it against Microsoft’s first-party Decision documentation plus open sibling implementations. Competitor evidence lowered Jev-specific differentiation but was not counted as Jev capability evidence; no model was installed. partially-verified

typesafe-system-one · typesafe-home · typesafe-models · infoq-jev

1. 2026-10-08Initially discovered inside a combined System One / typed-decisions record.
2. 2026-10-08Split into a named-entity record after multiple equivalents emerged. Jev remains WATCH, but its implementation-specific fit and differentiation were lowered.
3. 2026-10-09Perplexity Decisions, Quyet, and Jebadiah added more direct substitutes without a verified Jev capability release; Jev-specific frontier and fit scores fell again.
4. 2026-10-10Jev remained on version 1.13 and text-only while Clef-omni added audio/video and Clef-flash listed a lower input price; Jev-specific frontier and fit fell without changing its WATCH stage.
5. 2026-10-11Microsoft-Decision-1 added a major enterprise-hosted equivalent without a new Jev capability release; Jev-specific frontier and fit fell while WATCH remained unchanged.

Trade-offs

• Pre-invocation verification can improve trust, but federation adds policy and freshness questions.
• A separate discovery layer can reduce coupling, but creates another contract to operate.

Provenance Manual seed from the existing Atlas source relationship; no live registry was queried. unverified

ard-spec · ard-announcement

1. 2026-09-24Seeded as an evolving frontier radar item.