Mongoose Docs

Every scanner tells you "this looks broken." Mongoose shows you the transaction that broke it.

Key Results

Quick Start

bashgit clone https://github.com/Azeru548/ottersec.git
cd ottersec
npm install

bashcp .env.example .env
# Edit .env and set GROQ_API_KEY from https://console.groq.com/keys

bashnpm run setup:dataset
# Clones coral-xyz/sealevel-attacks into data/sealevel-attacks

bash# Run on a single program (no verifier needed):
npm run otter -- --program path/to/src --skip-verify

# Run on all in-scope families (detector only):
npm run otter:inscope

# Full pipeline with verification (requires Solana CLI):
solana-test-validator --reset --quiet &
./scripts/build-and-deploy.sh
npm run otter -- --dataset ./data/sealevel-attacks

Prerequisites

Pipeline Architecture

Mongoose uses a three-stage pipeline where each stage is independently gradeable. If the Extractor is wrong, the Detector's input is garbage — you can catch that in isolation. If the Detector over-flags, the Verifier filters false positives before they reach the user.

[1] Extractor No LLM

Purpose: Remove noise. LLMs are unreliable at reading raw Rust macros — parse them deterministically so the Detector works with structured data, not raw text.

Implementation: Uses web-tree-sitter with the tree-sitter-rust grammar. Walks the AST for:

• #[derive(Accounts)] structs and their fields
• #[account(...)] attributes: has_one, owner, address, signer, mut
• Handler body checks: is_signer, owner comparisons, discriminants, PDA derivation
• Each account mapped to its constraints

Output: Structured JSON per instruction, including account name, signer/mut flags, owner constraints, and a human-readable constraint summary.

json{
  "instructions": [{
    "name": "withdraw",
    "accounts": [
      { "name": "user", "is_signer": false, "is_mut": false },
      { "name": "vault", "is_signer": false, "is_mut": true }
    ],
    "constraint_summary": "No signer constraints listed"
  }]
}

Failure mode: If the Extractor fails, the pipeline halts. The Detector never runs on garbage input. This is intentional — a silent bad parse is worse than an explicit error.

[2] Detector LLM Agent

Purpose: Reason about vulnerability classes using structured context, not raw code.

Provider: Grok Cloud (GROQ_API_KEY, model: openai/gpt-oss-20b by default, overridable via DETECTOR_MODEL).

System prompt design:

• Receives the 5-class Solana vulnerability taxonomy as context
• Input is structured JSON from the Extractor, not raw source code
• Conservative by design — only flags what is actually absent
• Outputs: class, instruction, account, reasoning (2-3 sentences), confidence (HIGH/MEDIUM/LOW)

False-positive memory: A registry (data/fp-memory.json) tracks known false positives from previous runs. The Detector is instructed to avoid re-flagging them.

Failure mode: The Detector may hallucinate missing checks or miss subtle ones. The Verifier catches false positives; the changelog tracks Detector precision per iteration.

[3] Verifier No LLM

Purpose: Close the loop. Turn suspicion into proof.

Architecture:

• Local solana-test-validator with deployed target program
• Ephemeral test accounts created on startup
• Raw @solana/web3.js transactions (no Anchor .rpc() convenience wrappers)
• Per-class exploit construction:

Vulnerability Classes

Sourced from Neodyme's sealevel-attacks — the reference dataset for known Solana program vulnerabilities. Classes 1-3 get full Verifier coverage because the exploit is a single malformed transaction. Classes 4-5 are flagged as "Suspected" with detailed reasoning.

Class 1 Missing Signer Check

An instruction doesn't verify the expected authority actually signed the transaction.

How it works: The Extractor identifies accounts where is_signer is false and no explicit #[account(signer)] constraint exists. The Detector flags this as a potential vulnerability. The Verifier constructs a transaction omitting the authority's signature — if it succeeds on-chain, the finding is PROVEN.

Real-world impact: An attacker can drain vaults, transfer tokens, or modify state without the owner's authorization. This is the most common and most dangerous Solana vulnerability class.

rust// Vulnerable: no signer constraint on 'authority'
pub fn withdraw(ctx: Context<Withdraw>) -> Result<()> {
    let amount = ctx.accounts.vault.amount;
    **ctx.accounts.vault.to_account_info().try_borrow_mut_lamports()? -= amount;
    **ctx.accounts.user.to_account_info().try_borrow_mut_lamports()? += amount;
    Ok(())
}

// Fix: add signer constraint
#[derive(Accounts)]
pub struct Withdraw<'info> {
    #[account(mut)]
    pub user: Signer<'info>,  // ← must sign
    // ...
}

Class 2 Missing Owner Check

A program doesn't verify an account is owned by the expected program, allowing a malicious user to pass an account owned by a different program.

How it works: The Verifier passes an account owned by the System Program (or a fake program) instead of the expected one. If the instruction accepts it, the finding is PROVEN.

rust// Vulnerable: no owner constraint
pub fn process(ctx: Context<Process>) -> Result<()> {
    let data = ctx.accounts.data.try_borrow_data()?;
    // ... reads data without verifying ownership
}

// Fix: add owner constraint
#[derive(Accounts)]
pub struct Process<'info> {
    #[account(
        has_one = authority,
        seeds = [...],
        bump
    )]
    pub data: Account<'info, MyData>,  // ← auto-verified
}

Class 3 Type Cosplay (Account Type Confusion)

Program accepts an account of the wrong type due to missing discriminator check. An account with the correct owner but wrong data layout passes validation.

How it works: The Verifier constructs an account that has the expected owner but contains data matching a different type. Without a discriminator check (typically via Anchor's Account<'info, T>), the instruction happily processes the wrong account type.

rust// Vulnerable: UncheckedAccount, no discriminator
pub fn update(ctx: Context<Update>) -> Result<()> {
    let mut user = ctx.accounts.user.try_borrow_mut_data()?;
    user[0] = 1;  // overwrites arbitrary data
}

// Fix: use typed account with discriminator
#[derive(Accounts)]
pub struct Update<'info> {
    pub user: Account<'info, UserProfile>,  // ← Anchor checks discriminator
}

Class 4 Missing Relationship Constraint

The has_one or account-data matching constraint is missing — two accounts that should be linked aren't verified as connected.

Verifier coverage: SUSPECTED only. Requires complex multi-instruction state setup that's unreliable to automate within the current time budget. The Detector flags the missing constraint with detailed reasoning.

Class 5 Insecure PDA Seeds

Seeds allow derivation of colliding accounts. A seed derivation can look completely correct by inspection and only reveals its flaw when you actually try to derive a colliding account.

Why this is hard: This is the class where static detection caps out. The flaw only reveals under derivation — purely textual analysis can't catch it. Mongoose flags it as SUSPECTED with reasoning about the seed structure.

CLI Reference

Flags & Options

Environment Variables

CI Pipeline (GitHub Actions)

The CI pipeline runs on ubuntu-latest and builds programs, starts the validator, deploys fixtures, runs exploits, and uploads artifacts.

Workflow Steps

Programs Covered

Reproduction Guide

Full reproduction of the evaluation results on your local machine.

1. Setup

bashgit clone https://github.com/Azeru548/ottersec.git
cd ottersec
npm install
cp .env.example .env   # Set GROQ_API_KEY

2. Setup Dataset

bashnpm run setup:dataset
# → data/sealevel-attacks (coral-xyz/sealevel-attacks)

3. Run Mongoose

bash# Full pipeline on test set:
npm run otter -- --dataset ./data/sealevel-attacks --output otter_results.json

# Baseline only:
npm run baseline -- --dataset ./data/sealevel-attacks --output baseline_results.json

# Generate comparison:
npm run evaluate -- --baseline baseline_results.json --otter otter_results.json

4. Run with Verifier

bash# Requires Solana CLI + Anchor installed:
solana-test-validator --reset --quiet &
./scripts/build-and-deploy.sh
OTTER_SIGNALS_ONLY=1 npm run verify:ci

Expected Output

Evaluation Metrics

Detailed Results

Per-program results across all 18 in-scope test cases (6 families × 3 variants each):

Baseline Comparison

The baseline uses a single generic prompt with no Solana-specific context, no structured extraction, and no verification:

prompt"Review the following Rust code for security vulnerabilities.
List any issues you find."

The baseline receives raw .rs files. It does not know the 5-class taxonomy. It does not attempt exploits. This establishes a fair floor: Mongoose must beat a generic LLM prompt using the same test set.

Ground Rules

Pinned Versions

FAQ

Why can't I just use a generic LLM to find vulnerabilities?

Generic LLMs have no Solana-specific account-model knowledge. They hallucinate missing checks on secure programs and miss real vulnerabilities hidden in Anchor macros. Mongoose uses structured extraction to give the LLM grounded context, then verifies its output with real exploit transactions.

Why are Classes 4-5 only "Suspected"?

Classes 4 (missing relationship) and 5 (insecure PDA seeds) require complex multi-instruction state setup to exploit. Automating this reliably within a time budget isn't feasible yet. Mongoose is honest about what it can prove and what it can only flag.

Does Mongoose work on mainnet?

No. All exploit attempts run against a local solana-test-validator. No transactions ever touch devnet, testnet, or mainnet. This is a hard ground rule.

What if the Extractor fails to parse my program?

The pipeline halts. The Detector never runs on garbage input. This is intentional — a silent bad parse is worse than an explicit error. The Extractor uses tree-sitter-rust which handles most Anchor patterns, but non-standard code may require manual inspection.

How do I add support for new vulnerability classes?

Extend the 5-class taxonomy in the Detector's system prompt and add corresponding exploit construction logic in the Verifier. The modular pipeline makes this straightforward — each stage is independent.

Can I use a different LLM provider?

Yes. Set DETECTOR_MODEL to any model available on Groq Cloud. The Detector uses the OpenAI-compatible API, so any model that supports structured JSON output should work.