collina.tech

AI Testing

Fri, 03 Jul 2026 00:00:00 +0000

$ ./ai-testing --scope models,rag,agents,tools

Machine-learning systems fail in ways classic software does not. The vulnerability isn’t a missing bounds check — it’s that a model reads attacker-controlled text as instructions, that a retrieval pipeline cannot forget a poisoned document, and that an agent with real credentials can be talked into using them against its owner. This track is where I keep field notes on breaking — and therefore securing — AI systems.

Why AI testing is its own discipline

The trust boundary moved into natural language. There is no reliable syntax that separates “data” from “commands” in a prompt. Every channel that puts text into the context window — user input, a retrieved document, a tool’s output, another agent’s message — is an injection surface.
Findings are probabilistic. The same payload can succeed 7 times out of 10. A pentest report has to speak in Attack Success Rate, pinned to model, version, and temperature — not a single screenshot.
The blast radius is the tooling, not the chat box. A model that “says something bad” is a safety issue. A model wired to email, a database, or a shell that can be made to act is a security issue. That’s where the money is.

The mental model: the lethal trifecta

The clearest framing of agentic risk, from Simon Willison (June 2025): an agent is exploitable for data theft when it holds all three of these at once —

Pentesting LLM Applications: A Field Methodology

Fri, 03 Jul 2026 00:00:00 +0000

A repeatable, architecture-led workflow for testing LLM apps and agents — scoping a non-deterministic target, mapping the five attack surfaces, running OWASP LLM Top-10 test cases, and reporting stochastic findings.

Prompt Injection & the Lethal Trifecta

Thu, 02 Jul 2026 00:00:00 +0000

Why prompt injection has no clean fix, how indirect injection turns retrieved content into code, and how the 2025 zero-click incidents (EchoLeak, ShadowLeak, ForcedLeak) are all the same three ingredients.

whoami

Thu, 02 Jul 2026 00:00:00 +0000

$ whoami
collina
$ id
uid=1337(collina) groups=pentest,osint,ctf

I break web apps and APIs for a living, and chase loose threads through open sources for fun. This is where I keep field notes — the writeups, the tooling, the tradecraft that didn’t fit in a report.

What you’ll find here

Pentest — web/API exploitation walkthroughs, methodology, and the occasional CTF.
Investigation — OSINT pivots, entity mapping, and how to turn a single artifact into a full picture.
Tooling — small scripts and setups that pull their weight.

Everything here is my own work and opinion. Findings from real engagements are sanitized — no client data, no live targets, no crossing the line.

The AI Testing Toolkit & Frameworks

Wed, 01 Jul 2026 00:00:00 +0000

The frameworks that give an AI pentest its vocabulary, the scanners that give it coverage, and a safe practice-lab recipe for rehearsing every attack offline.

Hunting IDOR / BOLA in the Wild

Sun, 28 Jun 2026 00:00:00 +0000

A repeatable workflow for finding broken object-level authorization in modern APIs — from mapping object references to proving impact.

OSINT: Pivoting From a Single Artifact

Sat, 20 Jun 2026 00:00:00 +0000

How to turn one email, username, or image into a mapped network of entities — a disciplined pivot chain that avoids rabbit holes.

A Recon Workflow That Actually Scales

Fri, 12 Jun 2026 00:00:00 +0000

Turning a wildcard scope into a prioritized attack surface without melting your VPS — passive first, resolve, probe, then triage.