Glossary

What it is

Security defects produced by AI code generators (like Lovable, Replit, Manus, Cursor, GitHub Copilot, or Claude Code) that ship in production applications without the developer realising they exist.

Why it matters

AI tools have rewired how code gets written. Most production codebases today contain a meaningful percentage of AI-generated code — and the bugs that come with it. Traditional SAST tools were built to catch the kinds of bugs humans write. The bugs AI writes are often semantic in nature (broken auth flows, BOLA, race conditions, logic errors) and fall outside the rule databases pattern-matching scanners rely on. Our Vibe Coded & Vulnerable research found 561 vulnerabilities across 24 AI-built apps — including 300 critical or high severity findings.

Example

An AI tool generates a CRUD endpoint with an auth check on top, but the database query inside doesn't filter by the authenticated user's ID. Syntactically clean. Semantically broken. Pattern-matching SAST won't catch it.

How Kolega handles it

Kolega's engine reads data flow across your codebase rather than pattern matching. It catches the semantic flaws that AI-generated code tends to ship, regardless of which builder produced the code.

What it is

Software that doesn't just find a vulnerability — it generates a code patch that fixes it, typically as a pull request that an engineer can review and merge.

Why it matters

Detection without remediation is half a product. Most security tools surface findings and stop there. Engineers then have to context-switch out of feature work, understand the bug, write the fix, write a regression test, open the PR. Most don't. Findings sit in a queue, slowly rotting. Autofix collapses the loop: the fix lands as a PR before the engineer has to decide whether to prioritise it.

Example

Kolega finds a BOLA in a getOrder endpoint, generates a one-line fix that adds the missing user_id filter, writes a regression test proving the fix prevents the BOLA, and opens a PR for review. Five minutes from detection to merge-ready PR.

How Kolega handles it

Every finding above a noise threshold gets an autofix PR generated automatically. PRs include regression tests by default so reviewers can verify the fix without rebuilding the whole exploit.

What it is

A vulnerability where an application correctly checks whether a user is authenticated, but fails to check whether that user is authorised to access the specific object they're requesting. The auth gate passes; the authorisation logic doesn't.

Why it matters

BOLA is one of the most common and most damaging vulnerabilities in modern web applications. OWASP lists it as the #1 API security risk. The classic example: a user logs in successfully, then changes the ID in the URL from /users/123 to /users/124 and sees someone else's data. The application accepted them as authenticated. It just didn't check whether they should be allowed to see record 124.

Example

@app.get("/orders/{order_id}")
def get_order(order_id: int, user: User = Depends(get_current_user)):
    # user is authenticated, but we never check if THIS user owns THIS order
    return db.query(Order).filter(Order.id == order_id).first()

The fix is one line — filtering by Order.user_id == user.id — but spotting that the line is missing is hard. The code looks complete. The auth check is there. The query runs. Traditional SAST tools rarely catch this because there's no pattern that distinguishes a correct query from a vulnerable one.

How Kolega handles it

Semantic data flow analysis tracks where the authenticated user object goes and what it's used to filter. If a query returns an object that isn't filtered by the authenticated user, Kolega flags it — even when the auth check higher up in the function passes.

What it is

The growing gap between what your engineering team can ship and what your security and governance processes can keep up with. As tools, AI assistants, and automation accelerate the writing of code, the controls designed to govern it (reviews, scans, approvals, audits) fall further behind.

Why it matters

Control Drift is the underlying reason most modern security incidents happen. It's not that teams don't care about security — it's that the velocity of shipping has outrun the velocity of reviewing. A team that could ship 5 PRs per week in 2022 might ship 30 today with AI assistance. The security review process is still designed for the 5-PR-per-week world. Things slip through, not because nobody's looking, but because nobody can look fast enough.

Example

Your team uses Cursor, Claude Code, and Copilot to write 60% of your codebase. Your scanner runs on main once a week. Your security review is gated on a once-a-quarter audit. The gap between when AI writes the bug and when anyone looks at it can be months.

How Kolega handles it

Kolega scans on every PR, not on a schedule. The control sits where the code is written, not where it's audited. Closing the gap is the entire point of the product.

What it is

A scanning architecture where the code being analysed is cloned into a fresh, isolated container that exists only for the duration of the scan. When the scan finishes, the container and everything in it is destroyed.

Why it matters

Traditional security tools store your source code in their infrastructure indefinitely. That's a perpetual breach risk: a compromise at the vendor exposes every customer's code. Ephemeral scanning eliminates the at-rest copy entirely. Your code lives in our infrastructure only as long as the scan takes, and it leaves nothing behind.

Example

Kolega clones your repo, runs the semantic analysis inside an isolated container, extracts the findings (file path, line number, severity, fix suggestion), and destroys the container. The source code itself never enters our database.

How Kolega handles it

This is the default architecture for every scan. No customer code is stored at rest. Enterprise customers can run self-hosted runners so code never leaves their infrastructure at all. See the Trust Center for the full lifecycle.

What it is

The scoring metric used by the RealVuln benchmark to measure how well a security tool detects real vulnerabilities. F3 is a weighted F-score that emphasises recall — finding the bugs that exist — over precision, because missing a real vulnerability typically costs more than reviewing one extra alert.

Why it matters

Vendors often quote precision metrics (“only 5% false positives!”) that look great in marketing but hide poor recall (“we only find 15% of the actual bugs”). F3 forces the comparison to surface what most teams actually care about: are you catching the vulnerabilities or aren't you?

Example

A tool that flags 100 issues and gets 95 of them right (95% precision) but misses 80% of the real vulnerabilities in the codebase scores poorly on F3. A tool that flags 110 issues with 90 correct but catches almost every real vulnerability scores much higher.

How Kolega handles it

Kolega scores 92.4% on RealVuln's F3 metric — the highest of 21 systems tested, including every frontier model. We publish the methodology and every result.

What it is

The hidden cost a team pays when a security tool generates more false positives than true positives. Every false alert costs engineering time to triage, dismiss, or investigate. Multiply that across a year and the cost is measurable, even if it never shows up on a balance sheet.

Why it matters

Most security programs fail not because no tool was bought, but because the tool that was bought drowned the team in noise until they stopped looking at it. A scanner that's accurate is cheap. A scanner that's loud is expensive in time, attention, and trust — even if the licence cost looks the same.

Example

A team of 10 engineers spends 30 minutes each per week triaging false positives. That's 5 engineer-hours per week, ~260 hours per year, or roughly $30k–$50k in fully-loaded engineering cost just to ignore findings that shouldn't have existed. Try the Noise Tax Calculator to estimate your own.

How Kolega handles it

Because semantic analysis generates fewer false positives at the engine level than pattern matching does, Kolega's noise floor is materially lower than rule-based scanners. The fix happens upstream of triage.

What it is

A class of vulnerability where the security of the system depends on the order in which two or more operations execute — but the order isn't guaranteed. A check-then-use sequence (verify the user has permission, then perform the operation) can be exploited if an attacker can slip an operation in between the check and the use.

Why it matters

Race conditions are nearly invisible to pattern-matching SAST because the bug isn't in the shape of the code — it's in the timing of when the code runs relative to other concurrent code. They produce some of the highest-impact bugs in real systems: TOCTOU (time-of-check to time-of-use) vulnerabilities, double-spend bugs in fintech, balance manipulation in payment flows.

Example

A withdrawal endpoint checks the user's balance is sufficient, then deducts the amount. If two requests fire simultaneously, both check the balance (sees $100), both pass the check, both deduct $100 — and the user has now withdrawn $200 from a $100 balance. The code looks fine. The bug is in the concurrency.

How Kolega handles it

Semantic analysis tracks the relationship between checks and the operations they're meant to guard. When the check and the use can be interleaved by concurrent execution, Kolega flags it.

What it is

An open-source benchmark for security scanners, built and maintained by Kolega. RealVuln tests 676 real vulnerabilities pulled from 26 production repositories with verified CVEs and known fix commits. Every tool is scored using the same methodology against the same code.

Why it matters

Most security vendor benchmarks are run by the vendor on synthetic test cases they designed. The numbers can't be reproduced and the methodology can't be inspected. RealVuln is open: the code is on GitHub, the methodology is documented, any vendor can run it on their own tool and publish results.

Example

On the current leaderboard, Kolega scores 92.4% on F3. CodeQL (the engine behind GitHub Advanced Security) scores 27.5%. Semgrep scores 17.5%. Snyk scores 16.7%. All on the same 676 vulnerabilities, scored the same way.

How Kolega handles it

We built it because we wanted to make detection depth comparable and verifiable. The benchmark is at realvuln.com and the source is at github.com/kolega-ai/Real-Vuln-Benchmark.

What it is

A category of security tools that analyse source code without running it, looking for vulnerabilities. SAST tools sit in CI or run on PR open and scan the static text of your code for security issues.

Why it matters

SAST is the workhorse of application security — most teams' first and only security tooling. The category includes everything from pattern-matching rule-based scanners (Semgrep, CodeQL, SonarQube) to semantic analysis engines (Kolega). All call themselves SAST. They work very differently underneath.

Example

When you commit a change with a SQL injection, a SAST tool that runs in CI catches the vulnerability before it ships. When the bug is more subtle — say a BOLA where the auth check passes but the query doesn't filter by user — most SAST tools miss it because they're matching patterns rather than reasoning about meaning.

How Kolega handles it

Kolega is a SAST tool, but with a semantic analysis engine rather than a pattern-matching one. The “static” part is the same; the “analysis” part is what differs.

What it is

A type of static analysis that reads the meaning of code — what it does, where data comes from, what touches it, and what the security implications are — rather than matching code against a database of known patterns.

Why it matters

Most SAST tools work by pattern matching. They have a database of rules that say “code shaped like X is a vulnerability.” This works well for known patterns (SQL injection, deprecated APIs, common XSS) and fails on semantic flaws (BOLA, race conditions, logic errors, missing authorisation checks). Semantic analysis catches the bugs pattern matching can't see, because it reasons about the code the way a senior engineer would.

Example

Two functions both query the database. Pattern matching sees both as “database query” and applies the same rules. Semantic analysis sees that one is filtering by the authenticated user and one isn't — and flags the second as a BOLA. Same syntax, different meaning, different security implications.

How Kolega handles it

Semantic analysis is the foundation of Kolega's engine. We score 92.4% on the RealVuln benchmark, compared to 16.7% for Snyk and 17.5% for Semgrep, because we read code semantically rather than matching it against rules.

What it is

The category of security challenges introduced by “vibe coding” — the practice of building applications by describing what you want in natural language and letting an AI tool generate the code. Tools in this category include Lovable, Replit, Manus, Bolt, v0, Cursor, and Windsurf.

Why it matters

Vibe coding has changed who builds software. Apps that would have required a team of engineers in 2022 can now be built in an afternoon by a single person using AI. That's genuinely useful — but it also means the security expertise that used to be baked into the engineering team isn't necessarily present anymore. The AI doesn't know what threat model your app sits in. It doesn't know which CORS settings are safe for your use case. It ships defaults that work for the demo, not for production. Our Vibe Coded & Vulnerable research found that 23 out of 24 AI-built apps shipped with vulnerabilities — most of them critical or high severity.

Example

A founder builds a SaaS app on Lovable in a weekend. The app ships with permissive CORS so the preview rendered, hardcoded API keys in the frontend bundle so the demo didn't need an env var setup, and database row-level security disabled because it broke the seeded data. All of these defaults shipped to production because nobody reviewed them — there was no engineering team to review them.

How Kolega handles it

Kolega is built specifically for codebases that contain large amounts of AI-generated code. The engine catches the semantic flaws that AI tools tend to produce, regardless of which builder generated the code.