# Code Checker (LLM-as-Judge)

- **Severity:** Low
- **Signature ID:** `netzilo-llmjudge-code-checker-001`
- **OWASP:** LLM09 (Misinformation)

## Summary

This entry describes a quality-checking template that asks an AI model to judge whether generated code correctly solves a stated problem. It is not designed to catch attacks — it flags code that looks wrong or incomplete, for review purposes only.

## How the attack works

A downstream process (like an evaluation pipeline) submits code output to a scanner. The scanner is given a rubric asking it to judge correctness and problem-solving quality, and it returns a verdict based on that rubric. The verdict is informational only — it never blocks anything and its maximum severity is 'report'. If the AI scanner is unavailable or has no API keys configured, the check silently passes everything (fails open).

## Why it matters

There is no security impact from this rule itself since it does not block or restrict any action; it only produces a quality report on code correctness that a human or process can choose to read.

## What you can do

- Do not rely on this check for security review — it evaluates code quality, not malicious intent.
- Treat 'no verdict' or 'fail open' results as no signal at all, since missing AI credentials silently allow everything through.
- If using this pattern to gate code merges, add a separate, non-AI check for cases where the scanner is unavailable.
- Review flagged items manually since natural discussion of security topics or test data can trigger similar wording to real defects.

## Known benign look-alikes

- Legitimate content discussing the flagged topic (security research, documentation, test data)
- AI scanner unavailable or no AI keys configured (fails open -> allow)

## References

- https://docs.smith.langchain.com/evaluation

---
Source: https://www.netzilo.com/threats/llm-as-judge-code-checker