←
Signal65483 posts
Signal65 avatar

Signal65

@Signal_65

We are here to ensure our partners become the signal of innovation in the noise of the technology markets.

◷ Joined December 2023▧ 172 Photos and videos
Signal65 avatar
Signal65@Signal_65·Nov 11

Claude Fable 5.2 passed 28 of 30 Frontier Class coding jobs on Signal65 PINNACLE. No other model has passed more than 21. It also costs $30.51 per passed job in our testing, 3.9x Claude Opus 5.5. Best model on the board, priced like it.

Signal65 avatar
Signal65@Signal_65·13h

Running open models on dedicated, private GPUs has meant paying for GPU hours PLUS the engineering time to build and tune a serving stack, and agentic workloads push both costs up. Signal65 tested @Microsoft Foundry Managed Compute on AMD Instinct MI300X GPUs to see how much of that work a managed template removes, and what it can provide in performance and agentic capability. Key findings from our testing: ➡️ Four open models went from the Foundry catalog to a live, dedicated endpoint in as little as 14 minutes, with no serving configuration ➡️ A single MI300X GPU served gpt-oss-120b to 32 concurrent AI agents, and four GPUs served DeepSeek V4.1 Flash to 128 agents with a one-million-token context ➡️ Default Foundry templates reached 93% to 100% of the per-agent speed of the same models hand-tuned on bare-metal MI300X servers ➡️ A budget of just $10,000 of MI300X time covered about 112,000 complete gpt-oss-120b agent sessions, roughly $0.09 of GPU time each Full Lab Insight Report: signal65.com/research/ai/dep… @Azure @AMD

Post attachmentPost attachmentPost attachmentPost attachment
Ryan Shrout avatar
⟳ Signal65 retweeted
Ryan Shrout@ryanshrout·Nov 16

NVIDIA is using SC26 to argue that AI agents will take on more and more scientific work, and today's announcements cover most of the stack that idea needs, from open models to a CPU-only rack for simulation codes to a new HPC GPU. Nemotron for Science pairs the open Nemotron 3.5 Super VL model with a 50,000-question chemistry dataset and post-training recipes, so research teams can build agents on their own data. Lila Sciences says its tuned version hit 61% accuracy on quantum dot synthesis versus 54% for GPT-6 Astra at 10x to 20x lower inference cost, and KTH used a Nemotron agent to fix 81% of broken GROMACS setups overnight on a single DGX Spark. An open model a lab can tune and run on its own hardware takes care of the cost and data-control worries that keep a lot of institutions off closed frontier APIs. Vera is the one I think matters most in the near term, since a huge share of climate, weather and CFD code still runs on CPUs and NVIDIA didn't have a product for it until now. The new rack packs 192 Vera CPUs (88 Olympus cores each) on Quantum-X800 InfiniBand with more than 200 TB/s of aggregate memory bandwidth, and NVIDIA claims 1.9x the performance and 2.4x the performance per watt of x86 on OpenFOAM. DKRZ already picked Vera for its HLRE-5 climate system, and Dell, HPE, Lenovo and Supermicro ship racks in early 2027. We tested Vera on agentic workloads at @Signal_65 earlier this fall, and its move into simulation gives x86 a real new competitor in HPC. NVIDIA also announced Rubin HPC, a 2028 GPU in the Vera Rubin family with more than 50,000 CUDA cores and a design that scales data movement along with compute. That speaks to HPC buyers who watched native FP64 on the standard Rubin GPU land at 33 TFLOPS, but without precision or memory specs it's too early to judge. Put it together and @nvidia is building every layer a science agent needs, which means AMD, Intel and the Arm server players chasing HPC budgets now have to compete with NVIDIA at each of them.

Ready to build your own agent to speed scientific discovery? 🧬 Introducing Nemotron for Science, helping researchers develop agents that reason through scientific problems and put specialized research tools to work. Used by @argonne, @RIKEN_en & more. #SC26 🔗 nvda.ws/4C2tIuu

Post attachment
Signal65 avatar
Signal65@Signal_65·Nov 10

AI agents spend much of their working time running tools on CPUs, so the processor decides how many agents a server can carry and how quickly each one finishes its work. That puts CPU choice back into the AI infrastructure budget, alongside the GPUs. Signal65 tested two AMD EPYC Venice processors ahead of today's launch. In our early testing, Zen 6 ran CPU-bound agent tool work 22% to 60% faster per core than the previous-generation EPYC 9755. A preview of the upcoming Signal65 PINNACLE CPU Benchmark shows the 256-core EPYC 9996 finishing each agent's tool work 22% faster than the 96-core EPYC 9686F at 1,024 concurrent agents, while the 9686F's faster cores lead when the server is lightly loaded. A full PINNACLE CPU Benchmark deep dive is coming soon. The choice between the two Venice designs comes down to how many agents you run and how quickly each one needs to move. More testing soon!

Ryan Shrout @ryanshrout Nov 10 Article First look at AMD EPYC Venice and the CPU side of agentic AI AMD launched its sixth-generation EPYC processors, code-named Venice, today, and Signal65 had a short window before launch to test two of them on an AMD reference platform. We ran the 256-core EPYC

Signal65 avatar
Signal65@Signal_65·Nov 9

Reasoning models only pay off when people get answers fast enough to stay in the flow of their work, and every extra GPU needed to hold that response time adds to the cost of serving them. Signal65 analyzed the audited MLPerf Inference v6.1 DeepSeek-R1 results to see which cloud delivers the most reasoning throughput under the tightest latency limits. The Interactive scenario holds every system to a 1.5s limit on time to first token and 15ms per output token, the closest MLPerf match for conversational and coding workloads. We compared each Azure result against systems with the same GPU generation and GPU count. Key findings from our analysis: ➡️ Azure ND GB300 v6 leads Interactive throughput by 10.3% over the next cloud submission, at 260,098 output tokens per second on 72 GPUs ➡️ Azure ND GB200 v6 scales from one NVL72 rack to four at 93.6% efficiency in Server and 96.4% in Offline ➡️ Four racks reach 2.09M output tokens per second in Offline, the only multi-rack NVL72 result from a cloud provider this round ➡️ Moving from GB200 to GB300 raises Server throughput 38.1% at the same 72-GPU scale ➡️ Azure ND GB200 v6 posts the top Server and Offline throughput of all Available 72-GPU GB200 systems, 1.7% and 2.3% ahead of the closest cloud submission Read the full Lab Insight: signal65.com/research/ai/mic… @Azure @nvidia

Post attachmentPost attachment
Signal65 avatar
Signal65@Signal_65·Nov 9

Somewhat surprisingly for a medium-sized option, a 65B open-weight model that you can run on its own hardware just posted the highest agentic job completion of any open model on Signal65 PINNACLE. Live on the Model Score and cost views at pinnacle.signal65.com Ripple-65B-Preview is a 64.8B dense model from @RicursiveAI, a US lab new to the board, scored on the same protocol as every other model. In our testing: ➡️ It finished 98.2% of the multi-step enterprise jobs, the highest rate of any open-weight row, ahead of Qwen3.8-Flash-Next (97.1%) and Beam-1.0 NVFP4 (96.1%), with the frontier tier at 96.7% ➡️ It scores 3,101 on the PINNACLE Model Score, the best result under 100B parameters, 22% fewer weighted errors than Qwen3.8-27B and within 5% of Muse Spark Open NVFP4 and GLM-5.2 at a twelfth of their size ➡️ On one 8× NVIDIA B300 node it holds 40 concurrent agents at 10 tok/s each, which prices a correct task at $0.39, 193 correct tasks per node-hour; the speed term is a single 172-attempt run and the row is a preview build ➡️ The NVFP4 build is not at parity with BF16. It scores 2,772 (97.5% jobs finished, 12.6% fabrication) at $0.34, 20% more throughput on the same node for an 11% lower score The agentic half of the result is already there. Close the retrieval gap and this model moves well up the board.

Post attachmentPost attachmentPost attachment
Signal65 avatar
Signal65@Signal_65·Nov 2

How does a chip that’s 1.2x faster on average deliver ~2x the work per rack? Because power — not processors — is now the limit. When capacity planning starts with megawatts, rack density becomes the number that decides how much agentic work your infrastructure can actually host. That’s the story behind our new Lab Insight on the @Arm AGI CPU vs. a leading x86 competitor, and it’s the conversation @RyanShrout and Russ Fellows just had with Arm’s Jeff Defilippi and Ravi Malhotra on Six Five Media: how hyperscalers really do this math, how agentic benchmarks like SWE-bench and Terminal-Bench 2 are changing what we measure on CPUs, what Arm’s software ecosystem looks like in 2026, and when infrastructure teams can get hands on the silicon. 📷 Watch the conversation: youtube.com/watch?v=acYo88OG… 📷 Read the full analysis (modeled rack results, per-workload data, methodology): signal65.com/research/ai/arm…

Ryan Shrout avatar
⟳ Signal65 retweeted
Ryan Shrout@ryanshrout·Oct 31

I continue to believe that this sell-off of $INTC is pretty absurd. If you were betting on @Intel before this week, you were betting on FOUNDRY as the growth engine, with compute as the more modest impact piece. Nothing in that model changed this week. Certainly not by 30%!

Intel, $INTC, shares are down 30% today following the company’s earnings report. @RonWestfallDX and @ryanshrout discuss the results and the market reaction. For more: schwabnetwork.com/?CID=SM:Tw…

8:16
Ryan Shrout avatar
⟳ Signal65 retweeted
Ryan Shrout@ryanshrout·Oct 29

Today we are launching the first major upgrade to @Signal_65 PINNACLE since it went live in August. The PINNACLE Coding Agent test is a separate score for whether a model can build the right software in a codebase it has never seen, graded by running what it builds, with a price on every passed job. Coding agents are among the most useful tools to reach software teams in years, and anyone who has handed one serious work has watched it hand back a program that runs, passes its own tests but can miss the point, because the agent never worked out how the system is meant to BEHAVE. That failure is what this test is built to catch. In our testing one hosted model answers 97% of its samples correctly and passes none of its 30 complete jobs. Three things I would pull out. 🟢 The best enterprise model is not necessarily the best coding model, and the two rankings part ways at the top. 🟢 Cost per passed job puts the failed tries in the bill, and the hosted frontier runs from $1.35 to $73.76 a passed open-ended job. 🟢 And the open models that pass fully scoped work for well under a dollar are the size that runs on a developer's own machine, which is where PINNACLE goes next client testing. The full article, and the results views on pinnacle.signal65.com, are here: pinnacle.signal65.com/articl…

Enterprises are handing coding agents real engineering work and choosing the model on benchmarks that basically just count passing unit tests. Those benchmarks do not see the failure every engineering team has met, code that runs and misses the point, and they say nothing about what a passed job costs. Signal65 PINNACLE now has a Coding Agent test, published today beside the PINNACLE Model Score. ➡️ A model is dropped into a codebase it has never seen, with a business brief, 74K tokens of documentation and 7,200 seconds, and asked to build the software ➡️ Grading is all code. The program it builds is run on 200 unseen samples, every answer is recomputed, and the design is checked across samples, so a generator that never varies fails the job ➡️ Two classes of work, open-ended Frontier Class jobs and fully scoped Local Class jobs, weighted equally into one PINNACLE Coding Agent Score ➡️ The best enterprise model is not necessarily the best coding model, and the rankings part ways at the top of the field ➡️ One passed open-ended job costs from $1.35 to $73.76 across the hosted frontier, failed tries included, with open-weights models priced on a rented @NVIDIA B300 node 63 configurations, 3,690 Frontier Class and 3,660 Local Class attempts, developed in close partnership with @Kamiwaza. Read the article and the results: pinnacle.signal65.com/articl…

Post attachment
Signal65 avatar
Signal65@Signal_65·Oct 29

In our testing Claude-Opus-5.5 at medium effort passes 70% of open-ended jobs, Gemini-4-Argon at high effort 67%, GPT-6-Astra 63% and GPT-6.1-Sol 57%, and with 30 attempts per hosted configuration the four intervals overlap. Below them Gemini-4-Argon at medium effort passes 43% and Claude-Sonnet-5.5 37%. The open-weights leaders are closer to that tier than their enterprise results. GLM-5.3 at low effort passes 35% of Frontier Class jobs and DeepSeek-V4.1-Flash at max effort 32%, and each of them passes every Local Class job. Below that line most of the field passes no open-ended job at all, which is the reason the test has a fully scoped half, and it is where models sized for a PC and workstation show what they can do. pinnacle.signal65.com/articl…

Post attachment
Signal65 avatar
Signal65@Signal_65·Oct 29

A model chosen on enterprise results and handed to an engineering team carries an assumption, that the capability transfers. In our testing it does not, and the cost of getting it wrong is an engineering team running on the wrong model for months. The Signal65 PINNACLE Coding Agent Score is published beside the PINNACLE Model Score, and across the 57 configurations run at the same effort setting on both tests the two rankings agree broadly, a rank correlation of 0.88, and part ways at the top, among the models an enterprise is most likely to be choosing between. Claude-Opus-5.5 at medium effort is fifth on enterprise work and first on coding. Gemini-3.8-Flash at medium effort is third on enterprise work and seventeenth on coding. GLM-5.3 at low effort is thirty-first on enterprise work and seventh on coding, passing every Local Class job. The practical reading is to pick per workload, and the coding table carries both numbers for every configuration.

Post attachment
Signal65 avatar
Signal65@Signal_65·Oct 29

Enterprises are handing coding agents real engineering work and choosing the model on benchmarks that basically just count passing unit tests. Those benchmarks do not see the failure every engineering team has met, code that runs and misses the point, and they say nothing about what a passed job costs. Signal65 PINNACLE now has a Coding Agent test, published today beside the PINNACLE Model Score. ➡️ A model is dropped into a codebase it has never seen, with a business brief, 74K tokens of documentation and 7,200 seconds, and asked to build the software ➡️ Grading is all code. The program it builds is run on 200 unseen samples, every answer is recomputed, and the design is checked across samples, so a generator that never varies fails the job ➡️ Two classes of work, open-ended Frontier Class jobs and fully scoped Local Class jobs, weighted equally into one PINNACLE Coding Agent Score ➡️ The best enterprise model is not necessarily the best coding model, and the rankings part ways at the top of the field ➡️ One passed open-ended job costs from $1.35 to $73.76 across the hosted frontier, failed tries included, with open-weights models priced on a rented @NVIDIA B300 node 63 configurations, 3,690 Frontier Class and 3,660 Local Class attempts, developed in close partnership with @Kamiwaza. Read the article and the results: pinnacle.signal65.com/articl…

Post attachment
Snowflake avatar
⟳ Signal65 retweeted
Snowflake@Snowflake·Oct 29

68 minutes vs. 316 minutes. In an independent, hands-on evaluation, @Signal_65 found Snowflake CoCo completed a seven-phase enterprise workflow in 68 minutes, compared to 316 minutes via manual workflow. That’s 4.6x faster. See the full evaluation: bit.ly/4ix1386

Post attachment