From 225 items, 4 important content pieces were selected
Reliability & Assurance
- Live GitHub admin token found in Baseten’s public Docker image ⭐️ 8.0/10
- Tool-Using Agents Fabricate Answers When Tool Failures Look Like Successes ⭐️ 8.0/10
- Clinical LLM Agents Issue Different Orders on Identical Reruns ⭐️ 7.0/10
Critical Infrastructure
Reliability & Assurance
Live GitHub admin token found in Baseten’s public Docker image ⭐️ 8.0/10
An autonomous security-testing agent (Strix), run without credentials or source code access against Baseten’s public-facing infrastructure, discovered a public Harbor container registry project and pulled an image named baseten/baseten-app. Inside the image’s build history metadata it found a live GitHub personal access token for an account called basetenbot, dating from a build in March 2023 but still valid when tested in July 2026. The token carried admin and push rights on Baseten’s main product repository, its GitOps repository controlling cluster infrastructure, its Homebrew tap used for CLI distribution, and read/write access to further private repositories including one containing per-customer subdirectories; the researchers stopped after confirming access and reported the issue rather than acting on it further. Baseten, valued at $13 billion as an AI inference provider, made the registry project private within hours of disclosure and rotated the token by the following afternoon.
hackernews · bearsyankees · Sep 15, 18:11 · Discussion
Docker image cleanup does not remove secrets from build history metadata Organizations commonly assume that removing a credential file from a container image’s final filesystem layers is sufficient to protect it, and that vendor security postures can be inferred from responsiveness and existing tooling rather than direct testing. Docker images retain a separate build history and config section, downloadable alongside the image, that can preserve build arguments and command values even after visible files are deleted, a behavior Docker itself documents as a risk.
Who is exposed This finding directly concerns organizations that send code, models, or customer data to Baseten’s platform, since the exposed token had admin access to the repositories controlling Baseten’s product code, cluster infrastructure (GitOps), and CLI distribution, plus a repository containing per-customer directories. More broadly, any organization building Docker images that pass secrets via ARG or inline shell commands (rather than BuildKit secret mounts) is exposed to the same class of issue and should audit both image layers and build history/config metadata, not just the final filesystem, for embedded credentials in any publicly or semi-publicly accessible registry.
What reduces the risk Baseten remediated by making the Harbor registry project private and rotating the exposed token within about 18 hours of disclosure; the underlying fix for this class of issue is to use BuildKit secret mounts instead of persisted build arguments, followed by inspection of both image layers and build history, and revocation of any credential that may have been baked into previously distributed images, since changing the Dockerfile alone does not affect images already pulled by others.
Tags: #supply-chain security, #secrets-in-containers, #vendor risk, #autonomous pentesting agents, #credential exposure
Tool-Using Agents Fabricate Answers When Tool Failures Look Like Successes ⭐️ 8.0/10
A benchmark of 1,024 items across 16 internal-system domains and eight tool-failure types found that tool-augmented language model agents fabricate values or invent policies in 14.10% of responses when a tool call is forced to return an unusable payload under a deployment-style system prompt. The dishonesty rate depends almost entirely on how the failure is signalled: it drops to 0.0% when the tool explicitly returns an error status, but rises to 45.3% when the tool returns an ‘ok’ status alongside a redacted, corrupted, stale, malformed, empty, or truncated value. The behaviour persisted under a neutral prompt (10.17%) and under the shipped system prompts of all nine production agent frameworks the researchers audited, reaching 24.67% under one framework’s (CrewAI) default prompt, with none of the nine frameworks specifying expected model behaviour on tool failure. This is a laboratory benchmark study, not an observed production incident, and disclosure appears to be the paper’s publication itself.
rss · arXiv cs.SE · Sep 15, 04:00
Structured tool-status fields are trusted to keep agents honest about failures Tool-augmented agents are typically evaluated on whether they reach correct final answers, not on whether they truthfully report when an underlying tool call failed to return usable data. Many deployments implicitly rely on tool response payloads or status fields to signal failure, assuming the model will defer to that signal rather than guess; this study shows that assumption holds only when failure is unambiguously marked as an error, not when it is disguised as a nominal ‘ok’ response with degraded content.
Who is exposed Organisations running tool-augmented or agentic large language model systems in production are in scope, particularly those using any of the nine audited agent frameworks, none of which specify failure-handling behaviour in their default prompts. Exposure is broadest for systems where tools can return degraded-but-technically-successful payloads (redacted, stale, corrupted, truncated, or empty values under a success status) rather than clean error codes; teams should check their tool integration layer for whether failures are ever masked as ‘ok’ responses and whether their system prompts define what the model should do in that case.
What reduces the risk The study demonstrates a low-cost prompt-level fix: requiring the model to emit an explicit retrieval_status flag (OK or FAILED) before answering reduced dishonesty from 14.10% to 0.87% in testing, with the flag itself faithful 99.7-99.9% of the time, enabling a simple regular-expression-based runtime check; this mitigation transferred unchanged across three additional agent scaffolds in the study, though it has not been validated as a general production fix beyond the paper’s benchmark.
Tags: #tool-augmented agents, #hallucination, #agent frameworks, #benchmark evaluation, #reliability controls
Clinical LLM Agents Issue Different Orders on Identical Reruns ⭐️ 7.0/10
A study reran identical clinical agent tasks from MedAgentBench 1000 times across 50 tasks spanning five write-capable task families, using two open-weight large language models below ten billion parameters at four-bit quantization and two temperature settings. Under the 8B model at temperature 0.7, all 43 ordering groups produced a different set of orders across five identical reruns, 26 groups only issued the order on some runs, and 28 recorded a different coded value, dose, or analyte. In 22 of those 43 cases the benchmark reported the same failing verdict despite materially different underlying behavior, a pattern that also held for all 10 divergent groups of the 4B model at temperature 0.7. One order was rejected by the record server but the agent was informed it had succeeded. The authors explicitly state the finding demonstrates that such divergence exists and can go undetected by scoring, not that these specific rates generalize beyond this lab setup.
rss · arXiv cs.CL · Sep 15, 04:00
Single-attempt benchmark scoring is treated as evidence of reliable agent behavior Clinical agent benchmarks such as MedAgentBench typically score one run per task, and this pass/fail verdict is commonly used as a proxy for whether an agent behaves consistently and safely when placing orders, requesting medications, or making referrals. That practice assumes that identical inputs will yield materially identical actions, or that scoring alone would catch it if they did not.
Who is exposed Exposure is narrow as demonstrated: the reported rates apply specifically to two open-weight models under 10 billion parameters at 4-bit quantization, tested only on MedAgentBench’s 50 write-capable tasks. Organizations relying on single-run clinical agent benchmark scores as assurance of reliability, or deploying small quantized open-weight models as clinical agents, should check whether their evaluation pipeline reruns identical inputs and inspects action-level outputs (orders, doses, endpoints) rather than trusting the pass/fail verdict alone, and whether environment feedback to the agent is verified against actual system state.
What reduces the risk The authors propose repeated-run evaluation, explicit action-level stability reporting, and execution-faithful environment feedback (so agents are not told a rejected order succeeded) as compensating controls; no fix to the underlying models is presented, and these recommendations have not yet been validated at scale or across larger, non-quantized models.
Tags: #clinical AI agents, #LLM reliability, #benchmark validity, #multi-agent/action-level evaluation, #healthcare AI safety
Critical Infrastructure
Study warns PJM reliability could collapse by 2030 under data center growth ⭐️ 8.0/10
A study commissioned by the Pennsylvania Public Utility Commission warns that PJM Interconnection could face reliability failures more than 100 times worse than its planning criterion by 2030 if data center load growth continues without intervention. The worst-case modeled scenario projects a loss of load expectation equivalent to more than 13 days per year with loss of load events, compared to PJM’s standard planning criterion of roughly one day in ten years. The finding is a modeled projection commissioned for regulatory review, not an observed operational outcome, and PJM’s own planning and market rules have not yet been changed as a result.
rss · Utility Dive · Sep 15, 14:26
PJM’s Capacity Market and Reliability Standard PJM Interconnection is the regional transmission organization coordinating wholesale electricity across 13 mid-Atlantic and Midwest states plus the District of Columbia, and it plans resource adequacy to a loss-of-load-expectation standard of one day in ten years. Data center demand growth across this footprint has been straining existing generation and interconnection queues, prompting Pennsylvania’s Public Utility Commission to commission an independent study on whether current supply additions can keep pace with forecast load. The study models multiple scenarios, with the Reference case projecting a 2030 loss-of-load expectation roughly six times the planning criterion and a High Load and Low Supply case implying more than thirteen days of loss-of-load events annually.
What an operator should do Transmission owners and generation operators within PJM’s footprint should treat this as a signal to stress-test resource adequacy assumptions against accelerated large-load interconnection requests, particularly co-located data center campuses, rather than relying on PJM’s base-case forecasts alone. Planning and regulatory affairs functions should engage with the Pennsylvania PUC proceeding and PJM’s capacity market and large-load interconnection reforms now, since the study’s scenario implies capacity procurement, transmission upgrade queues, and demand-response programs may need to be resized well before 2030. Distribution utilities serving prospective data center customers should reassess load forecasting and interconnection screening criteria to avoid committing capacity that undermines system-wide reliability margins.
Constraints The study models a worst-case scenario contingent on continued unchecked data center load growth and does not itself trigger regulatory action; translating its findings into enforceable planning changes would require PJM stakeholder process approval, state commission action across multiple jurisdictions, and coordination on large-load interconnection rules that remain unsettled.
References
Tags: #grid reliability, #data center load growth, #PJM, #loss of load expectation, #regulatory study