Complete Solidity security: Wake Framework (open-source) for testing & fuzzing + Wake Arena for AI audits with graph-driven reasoning. By @AckeeBlockchain
Four AI auditing tools. Fourteen protocols. The shape tells the story.
Wake Arena 3.1's blue area contains the other three on every axis.
Where all systems struggle — Burve, Notional — the gap closes.
Where the work matters — Lend, Munchables, Virtuals — the edge is visible and consistent.
What the chart doesn't show: 49 of the 63 found vulnerabilities were confirmed by more than one Wake Arena agent independently.
The shape reflects consensus, not a single model's guess.
Learn more ↓
8/
This is the design rationale of Wake: contracts in Solidity, tests and fuzzing in Python, identical EVM semantics.
Generation falls on the language where measured model capability is highest; human review concentrates on test logic rather than language mechanics.
→ getwake.io
7/
The defensible conclusion: the disparity persists by default and closes only with targeted data or tooling.
A contract's language is fixed by its deployment target. Its test suite's language is not — and it determines which region of model competence AI-written tests exercise.
6/
The evaluation ecosystem remains Python-centric.
HumanEval and MBPP are Python-only; SWE-bench Multilingual covers nine languages and Aider polyglot six — none includes Solidity.
5/
On test generation: the only published measurement of LLM-generated Solidity unit tests (EASE 2025, three Code4rena projects) reports 2–4.6% correctness in single-shot generation.
No comparable Solidity-versus-Python test-generation study exists to date.
3/
At current scale (Cheng et al., 2026): GPT-5 attains 96.3% on Python HumanEval but ~4.3% pass@1 generating the low-resource language Cangjie; 5 of 6 frontier models score under 5%.
An agentic scaffold with compiler feedback raises GPT-5 to 77.6% — the deficit is knowledge, not reasoning.
2/
The mechanism is training data: the StarCoder corpus contains 60.40 GB of Python and 0.85 GB of Solidity — a ~71× disparity (Li et al., 2023).
Cassano et al. (IEEE TSE 2023): same models, same problems, 18+ languages — 30.4 pass@1 on Python vs ~10–12 on low-resource languages (p<0.001).
1/
LLM performance is not uniform across programming languages: it tracks training-data representation, and Solidity sits at the low-resource end.
In 2026, the strongest agent evaluated detects 45.9% of real Solidity vulnerabilities; LLM-written Solidity tests are 2–4.6% correct.
The evidence:
On 94 high-severity findings from Code4rena and Sherlock protocols:
Wake Arena 3.1 caught 63 — 67%.
Nearest competitors, GPT-5.2 xhigh and Zellic v12, caught 41 — 44%.
Public, reproducible, labels taken as-is. Some models may have trained on these codebases — fair caveat.
→ github.com/Ackee-Blockchain/…
Bridges fail because testing one chain at a time misses the interaction bugs.
Cross-chain protocols introduce synchronization challenges that single-chain tests cannot replicate.
Different block times.
Event relay timing.
State consistency across networks.
Testing these in isolation creates false confidence.
The contract works on Ethereum.
Works on Polygon.
But fails when they interact.
Wake was designed from day one with multi-chain as a first-class citizen.
Not an afterthought.
Not a plugin.
Built-in.
Spin up multiple blockchain instances in one test suite.
Each maintains independent state, accounts, and configuration.
Fork mainnet on both chains simultaneously.
Test realistic cross-chain scenarios with production token contracts and actual network conditions.
The framework enforces chain-specific account usage.
Can't accidentally deploy an Ethereum contract using a Polygon account.
Can't transfer funds across chains without explicit relaying logic.
This prevents an entire class of testing mistakes while making complex multi-chain scenarios straightforward to express.
Real bridge exploits often emerge from timing assumptions, failed relay attempts, or reorg handling.
Wake lets you simulate these edge cases systematically: chain reorganizations, message ordering issues, and timeout scenarios.
One cross-chain DEX aggregator caught a critical synchronization bug during Wake testing that would have drained user funds during high network congestion.
The bug only manifested when both chains processed transactions simultaneously with specific timing.
Multi-chain protocols are complex.
Testing infrastructure shouldn't make them harder.
With built-in mainnet forking, independent time manipulation per chain, and intuitive Python APIs for cross-chain logic, Wake gives teams the testing foundation that distributed protocols require.
Cross-chain testing should be straightforward, not heroic.
Discover Wake's multi-chain capabilities ↓
Wake@WakeFramework 2 Oct 2025 Article Cross-Chain Testing with Wake: Building Robust Multi-Chain Smart Contract Tests Introduction As blockchain ecosystems become increasingly interconnected, testing cross-chain interactions has become critical for DeFi protocols, bridges, and multi-chain applications. Wake, a
Learning to write security tests?
"Awesome Wake Tests" collects real production test suites from Aave, Lido, Axelar + comprehensive reentrancy defense patterns + educational resources (workshops, Ethernaut solutions, benchmarks).
Study how protocols securing billions test their code ↓
Wake@WakeFramework Jan 13 Article Awesome Wake Tests: A Collection of Security Resources Check out "Awesome Wake Tests," a curated list of testing resources, guides, and real-world examples for Wake users. This repository collects production test suites, performance benchmarks, and
4/ Security gates in CI mean auditors review cleaner code.
• Follow @WakeFramework for more insights
• Retweet the 1st tweet to help others
Get Wake:
getwake.io
3/ Pair detectors with fuzzing for layered coverage:
Static analysis catches known bad patterns.
`wake test -P 5` explores unexpected runtime behavior.
Structure + behavior — the gap most CI pipelines leave open.
2/ What gets caught by Wake before a premium audit:
• Reentrancy patterns
• Write-after-write bugs
• ERC-4337 storage violations
• Access control issues
Same detector set from 200+ Ackee audits.
Now running on every commit.