Avoiding Security Vulnerabilities from AI-Generated Code

1. Why AI-generated code carries its own risks

Large language models like Claude generate code based on patterns seen across vast amounts of training data. A significant portion of that data comes from public repositories, tutorials, and Stack Overflow answers, many of which were simplified for teaching purposes or are simply outdated. A tutorial snippet that shows a database query without a prepared statement because the lesson is about loops is not automatically flagged as a bad example by the model, unless the context explicitly demands otherwise.

The result is not malicious behavior on the part of the AI, but a statistical likelihood: if the training corpus contains many examples of a certain pattern without a safeguard, the model is likely to reproduce that pattern unless the prompt contains a counter-instruction. Developers who adopt AI-generated code without reflection implicitly hand off security decisions to a system that bears no responsibility for the consequences and has no awareness of the specific threat context of their own application. The following sections walk through concrete vulnerability patterns and how to counter them with prompting, review, and tooling.

2. Missing input validation and sanitization

The most common vulnerability type in quickly generated code involves the handling of user input. An AI assistant asked to write a function that processes form data often delivers a functional solution that takes values directly from $_POST or a request object and passes them along without validation. The model optimizes primarily for the requested functionality, not for implicit security requirements that were never mentioned in the prompt. In a Magento environment, this concretely means missing escaping calls in phtml templates, unchecked parameters in controller actions, or string concatenation in SQL statements that should actually go through the query builder interface.

What makes this particularly tricky is that generated code often looks syntactically flawless and appears to work at first glance, because test data during development is rarely malicious. The gap only becomes visible when an attacker deliberately injects special characters, overlong strings, or nested payloads. That is why every function processing external input should be explicitly checked for validation and escaping logic, regardless of whether the code came from a human or an AI. Asking Claude specifically about escaping strategies, for example escapeHtml(), escapeUrl(), or escapeJs() in a Hyvä context, generally produces correct code, because the model then explicitly reacts to that stated requirement.

<?php
declare(strict_types=1);

namespace Mironsoft\Contact\Controller\Index;

use Magento\Framework\App\Action\HttpPostActionInterface;
use Magento\Framework\App\RequestInterface;
use Magento\Framework\Controller\Result\JsonFactory;
use Magento\Framework\Controller\ResultInterface;
use Magento\Framework\Escaper;

/**
 * Handles contact form submissions with explicit input validation.
 */
final class Submit implements HttpPostActionInterface
{
    /**
     * @param RequestInterface $request HTTP request object
     * @param JsonFactory $resultJsonFactory Factory for JSON responses
     * @param Escaper $escaper Magento escaper for output encoding
     */
    public function __construct(
        private readonly RequestInterface $request,
        private readonly JsonFactory $resultJsonFactory,
        private readonly Escaper $escaper
    ) {
    }

    /**
     * Executes the controller action.
     *
     * @return ResultInterface
     */
    public function execute(): ResultInterface
    {
        $result = $this->resultJsonFactory->create();

        // WRONG pattern an unguided AI assistant frequently produces:
        // $email = $this->request->getParam('email');
        // $message = $this->request->getParam('message');
        // $this->mailSend($email, $message); // no validation at all

        // RIGHT: explicit validation before any further processing
        $email = trim((string) $this->request->getParam('email', ''));
        $message = trim((string) $this->request->getParam('message', ''));

        if (!filter_var($email, FILTER_VALIDATE_EMAIL)) {
            return $result->setData(['success' => false, 'error' => 'Invalid email address']);
        }

        if ($message === '' || mb_strlen($message) > 5000) {
            return $result->setData(['success' => false, 'error' => 'Invalid message length']);
        }

        // Output encoding applied explicitly before echoing back
        $safeMessage = $this->escaper->escapeHtml($message);

        return $result->setData(['success' => true, 'preview' => $safeMessage]);
    }
}

3. Spotting outdated cryptography suggestions

Training data for language models spans code from many years and projects of varying quality. For cryptographic operations, this creates a real risk: a model may suggest md5() or sha1() for password hashing, because these functions appear in countless older tutorials and legacy projects, even though both algorithms have long been considered unsuitable for password hashing. The same applies to hand-rolled encryption routines, static initialization vectors, or hardcoded keys in example code that was only ever meant to illustrate a concept.

The solution is not to distrust AI-generated cryptography code across the board, but to explicitly ask for current, vetted standard functions whenever a security-critical operation is involved. PHP already ships with password_hash() and password_verify(), secure adaptive hashing functions that handle salt generation and cost factors automatically. For symmetric encryption, sodium_crypto_secretbox() from the libsodium extension is the current standard, not the deprecated mcrypt extension, which was removed as of PHP 7.2 but still shows up in older training examples. A prompt like "use password_hash with PASSWORD_ARGON2ID, no custom hash implementation" eliminates this risk almost entirely.

<?php
declare(strict_types=1);

namespace Mironsoft\Customer\Model;

/**
 * Demonstrates the difference between an outdated crypto suggestion
 * and the currently recommended approach for password hashing.
 */
final class PasswordHasher
{
    // WRONG: an AI assistant without explicit security context may suggest this,
    // because md5() appears frequently in older training examples
    public function hashInsecure(string $password): string
    {
        return md5($password); // broken: fast, unsalted, reversible via rainbow tables
    }

    /**
     * Hashes a password using PHP's built-in adaptive hashing function.
     *
     * @param string $password Plaintext password to hash
     * @return string Argon2id hash including salt and cost parameters
     */
    public function hashSecure(string $password): string
    {
        // RIGHT: explicit prompt "use password_hash with PASSWORD_ARGON2ID"
        // produces this instead of a hand-rolled or outdated algorithm
        return password_hash($password, PASSWORD_ARGON2ID, [
            'memory_cost' => 65536,
            'time_cost' => 4,
            'threads' => 2,
        ]);
    }

    /**
     * Verifies a plaintext password against a stored Argon2id hash.
     *
     * @param string $password Plaintext password to verify
     * @param string $hash Stored hash to verify against
     * @return bool True if the password matches the hash
     */
    public function verify(string $password, string $hash): bool
    {
        return password_verify($password, $hash);
    }
}

4. Insecure defaults from training data

Another recurring pattern is overly permissive default settings. Asked to create a CORS configuration, a file upload function, or an API route, an AI assistant's answer, without explicit constraints, often trends toward the most open variant, because that version appears most frequently in example code and raises the fewest follow-up questions. Access-Control-Allow-Origin: *, upload directories without a file-type whitelist, or admin endpoints without an explicit role check are typical symptoms. The model often delivers the code that "works" fastest, not the code that is fit for production.

This pattern is especially relevant for Magento extensions, because ACL configuration and permission checks in acl.xml and isAllowed() calls are easily forgotten unless explicitly requested in the prompt. A backend controller missing an ADMIN_RESOURCE constant or with incorrect ACL resource inheritance is harder for a human reviewer to spot than an obvious SQL injection vector, because the code appears complete at first glance. The countermeasure is to make certain phrases a fixed part of your own prompting vocabulary: "with least-privilege access", "only for logged-in admin users with resource X", or "whitelist instead of blacklist" as a standard formulation for any request that touches access rights. Without that addition, an AI assistant typically delivers Access-Control-Allow-Origin: * instead of an origin whitelist, an upload directory without a file-type check instead of a MIME whitelist check, and an admin endpoint without an ADMIN_RESOURCE constant instead of an enforced isAllowed() check, three patterns that a single prompt addition like "apply least-privilege defaults" reliably prevents.

5. Security-conscious prompting in practice

The most effective lever against insecure AI-generated suggestions is not fixing things after the fact, but the prompting itself. A prompt that explicitly names security requirements demonstrably leads Claude to produce different output than a purely functional prompt. Instead of "write a function that uploads a file", asking for "write a function that uploads a file, with a MIME-type whitelist, size limit, random filename, and storage outside the webroot" systematically yields more secure code, because the model treats the stated requirements as hard constraints, not optional add-ons.

For recurring tasks, a project-wide security context, for example in a CLAUDE.md file, pays off. It permanently establishes baseline requirements such as prepared statements, mandatory escaping in templates, ACL checks in controllers, and rejection of outdated crypto functions. This context is automatically loaded on every request and does not need to be repeated in each individual prompt. It also helps to explicitly ask Claude to run a brief self-check of security-relevant aspects after generating the code, for example with a closing instruction like "review the generated code for injection risks, missing validation, and insecure defaults before responding".

{
  "weak_prompt": "Write a PHP function that queries products by category id from the request.",
  "strong_prompt": "Write a PHP function that queries products by category id from the request. Requirements: validate the category id as a positive integer before use, use Magento's repository/collection API with parameterized filters (never string-concatenated SQL), throw a NoSuchEntityException on invalid input, and add strict_types=1 with full PHPDoc.",
  "why_it_matters": "The weak prompt leaves input handling, SQL construction, and error behavior fully up to the model's training-data priors. The strong prompt turns each risk into an explicit constraint the model must satisfy.",
  "project_level_context_file": "CLAUDE.md",
  "recommended_standing_instructions": [
    "Always use prepared statements or repository APIs, never raw string concatenation for SQL",
    "Always escape output in phtml templates via $escaper",
    "Always require ACL resource checks in admin controllers",
    "Never suggest md5, sha1 or mcrypt for security-relevant hashing or encryption",
    "Ask before applying permissive defaults (CORS, file upload types, public routes)"
  ]
}

6. Magento- and Hyvä-specific pitfalls

Magento ships with its own security layers that a generically trained language model does not automatically know about or apply correctly if the context is missing. The query builder API via Magento\Framework\DB\Select or repository interfaces with SearchCriteriaBuilder replace raw SQL, but a generically trained model will not automatically prefer them when asked for "a database query" without Magento being mentioned explicitly. Equally important: CSRF protection via form keys in classic forms and correct use of isAjax() checks on AJAX endpoints are often skipped when the prompt only describes the functional requirement.

In a Hyvä context, another layer comes into play: Alpine.js components using x-html instead of x-text open up XSS holes when dynamic user input is written into the DOM unchecked, and Hyvä's CSP mechanism requires inline scripts to be explicitly registered via $hyvaCsp->registerInlineScript(). An AI assistant asked for an interactive component without this background frequently generates code that is functional but either CSP-incompatible or XSS-prone. Documenting these Magento- and Hyvä-specific requirements in the project context substantially reduces the risk without needing a reminder in every single prompt.

<?php
declare(strict_types=1);

namespace Mironsoft\Catalog\Model;

use Magento\Catalog\Api\ProductRepositoryInterface;
use Magento\Framework\Api\SearchCriteriaBuilderFactory;
use Magento\Framework\Exception\NoSuchEntityException;

/**
 * Loads products by category using the repository API instead of raw SQL.
 */
final class CategoryProductLoader
{
    /**
     * @param ProductRepositoryInterface $productRepository Repository for product entities
     * @param SearchCriteriaBuilderFactory $searchCriteriaBuilderFactory Factory for search criteria
     */
    public function __construct(
        private readonly ProductRepositoryInterface $productRepository,
        private readonly SearchCriteriaBuilderFactory $searchCriteriaBuilderFactory
    ) {
    }

    /**
     * Returns products filtered by a validated category id.
     *
     * @param int $categoryId Positive category entity id
     * @return \Magento\Catalog\Api\Data\ProductInterface[]
     * @throws NoSuchEntityException
     */
    public function getByCategoryId(int $categoryId): array
    {
        if ($categoryId <= 0) {
            throw new NoSuchEntityException(__('Invalid category id'));
        }

        $searchCriteriaBuilder = $this->searchCriteriaBuilderFactory->create();
        $searchCriteria = $searchCriteriaBuilder
            ->addFilter('category_id', $categoryId, 'eq')
            ->create();

        return $this->productRepository->getList($searchCriteria)->getItems();
    }
}

7. Treating AI code in review like any other contribution

The fundamental rule for handling AI-generated code is that it deserves exactly the same level of scrutiny as code written by a junior developer, regardless of how convincingly it is formatted and commented. A common mistake in teams newly adopting AI assistants is a kind of authority bias: cleanly formatted, well-documented code appears more trustworthy, regardless of its actual correctness. Claude typically produces readable, consistently formatted code, which increases the risk that reviewers apply a less thorough review than they would for visibly sloppy code.

A resilient review process treats the origin of the code as irrelevant to the depth of review, while still making it transparent in the pull request, for example with a note like "generated with Claude Code, manually reviewed". That creates traceability without blanket distrust or blanket privilege for AI code. Three areas deserve particular attention: any place that accepts external input, any place that accesses the filesystem, database, or external services, and any place involving permissions or authentication. These three categories cover the vast majority of security-relevant defects, regardless of whether the code came from a human or an AI.

8. Static analysis and automated security checks

Human review alone does not scale reliably, especially once AI assistants significantly increase the volume of code produced per developer. Static analysis tools such as PHPStan with security-relevant rules, PHP_CodeSniffer with the Magento coding standard, and specialized security scanners like psalm-plugin-security-analysis or progpilot catch many of the patterns described above automatically and consistently, without requiring every reviewer to manually check every detail. These tools are not a substitute for human judgment, but they are a reliable safety net that applies the same rules regardless of a reviewer's mood or experience level on any given day.

In the CI pipeline, this safety net can be configured so that a pull request with AI-generated code containing a known insecure function like md5() for password hashing, or an unparameterized SQL query, is automatically blocked before a human even needs to review it. Composer audit (composer audit) additionally checks whether third-party packages suggested by Claude contain known vulnerabilities, an aspect that is easily overlooked when adopting AI recommendations for new dependencies, since the model rarely has up-to-date information about a package's current CVEs in its training corpus.

#!/usr/bin/env bash
# ci-security-gate.sh - runs before merging any PR, including AI-generated ones
set -euo pipefail

echo "[1/4] PHPStan static analysis (level 5)"
bin/analyse app/code/Mironsoft --level=5

echo "[2/4] Magento coding standard + security sniffs"
bin/phpcs --standard=Magento2 app/code/Mironsoft

echo "[3/4] Dependency vulnerability audit"
bin/composer audit

echo "[4/4] Grep for known insecure patterns as a fast pre-filter"
if grep -rEn "md5\(|sha1\(|mcrypt_|eval\(" app/code/Mironsoft --include="*.php"; then
  echo "[BLOCKED] Insecure pattern found - manual review required" >&2
  exit 1
fi

echo "[OK] Security gate passed"

9. Insecure versus secure prompts compared

The following overview uses concrete tasks to show how an unspecific prompt leads to risk-laden code, and how an explicit security requirement in the prompt systematically changes the outcome.

The pattern is consistent: as soon as a security aspect is explicitly named in the prompt, Claude treats it as a hard requirement and delivers correspondingly hardened code. Left unmentioned, the model decides based on statistical likelihood drawn from training data, which can produce unacceptable results in security-critical contexts. This insight translates directly into team standards, for example as a mandatory prompt checklist for security-relevant requests.

10. Summary

Security vulnerabilities from AI-generated code rarely stem from obviously broken code, but from silently adopted patterns in training data: missing input validation, outdated crypto functions like md5() for passwords, and overly permissive defaults for CORS, file uploads, or access rights. These patterns can be substantially reduced with targeted, security-conscious prompting, for example through explicit requirements like "use password_hash with ARGON2ID" or "whitelist instead of blacklist", ideally stored permanently in a project-wide CLAUDE.md.

The decisive principle stays the same regardless of the tool in use: AI-generated code deserves exactly the same review depth as any other contribution to the project, neither more trust because of clean formatting nor blanket distrust because of its origin. Static analysis, security scanners, and CI gates reliably automate part of that check, but they do not replace human judgment on input validation, access control, and data access, the three areas with the highest defect density.

11. FAQ: Avoiding Security Vulnerabilities from AI-Generated Code