Automating Import/Export Profiles in Magento 2 with Cron

1. Why manual triggering falls short for recurring imports

Magento's standard import/export area is deliberately designed as a manually triggered toolbox: an admin user picks an entity, uploads a file, checks the validation result, and only starts the actual import after deliberate confirmation. For one-off migrations that flow is exactly right, but for a daily supplier feed with several thousand rows it quickly becomes a bottleneck, because automation is blocked by the lack of a way to trigger the same flow regularly without a human in the loop.

This article deliberately doesn't cover performance optimization of large imports themselves, that's already its own topic, but focuses purely on how an existing or new import profile runs reliably on a schedule, documents its result, and notifies the right people on failure instead of failing silently.

2. Creating a custom scheduled operation entity

Instead of hard-wiring import configuration and file path into code, a custom entity declared via db_schema.xml pays off, storing per automated profile the entity type, source file path or SFTP directory, schedule expression, and the status of the last run. This entity is managed through a dedicated repository following the usual service contract conventions and maintained in the admin area through a dedicated UI component grid, so new automated profiles can be created without a code deployment.

The schedule itself isn't stored as a rigid cron expression in module code, but as a configurable field per profile, letting different profiles run at different frequencies without needing a separate crontab.xml entry for every new frequency. The actual cron job instead runs at a fixed, short interval and checks on every run which profiles are due according to their stored schedule.

<!-- app/code/Mironsoft/ScheduledImportExport/etc/db_schema.xml -->
<table name="mironsoft_scheduled_import_profile" resource="default" engine="innodb">
    <column xsi:type="int" name="entity_id" identity="true" nullable="false"/>
    <column xsi:type="varchar" name="entity_type" length="64" nullable="false"/>
    <column xsi:type="varchar" name="source_path" length="255" nullable="false"/>
    <column xsi:type="varchar" name="cron_expression" length="32" nullable="false"/>
    <column xsi:type="varchar" name="last_status" length="32" nullable="true"/>
    <column xsi:type="timestamp" name="last_run_at" nullable="true"/>
    <constraint xsi:type="primary" referenceId="PRIMARY">
        <column name="entity_id"/>
    </constraint>
</table>

3. Registering the periodic trigger as a cron job

The actual cron job is registered like any other via crontab.xml and cron_groups.xml and typically runs every five or ten minutes, noticeably more often than any individual profile actually imports. Inside the job class, every run first checks which stored profiles are currently due according to their cron_expression field, usually through a small wrapper library that evaluates a cron expression against the current time.

A locking mechanism is critical for reliable operation, preventing two overlapping cron runs from processing the same profile in parallel, for example when an import runs unusually long and the next regular cron tick already starts. A simple but robust approach uses a cache entry with a short TTL as a lock, set when a profile import starts and released on completion or in a finally block.

<?php

declare(strict_types=1);

namespace Mironsoft\ScheduledImportExport\Cron;

use Mironsoft\ScheduledImportExport\Model\ResourceModel\Profile\CollectionFactory;
use Mironsoft\ScheduledImportExport\Model\ProfileImportRunner;
use Mironsoft\ScheduledImportExport\Model\ProfileLockManager;
use Mironsoft\ScheduledImportExport\Model\CronExpressionEvaluator;

/**
 * Periodically checks due import profiles and triggers
 * their processing under lock protection.
 */
class RunDueImportProfiles
{
    /**
     * @param CollectionFactory $profileCollectionFactory Provides all stored profiles
     * @param CronExpressionEvaluator $cronEvaluator Checks due status against the cron expression
     * @param ProfileLockManager $lockManager Prevents parallel processing of the same profile
     * @param ProfileImportRunner $importRunner Runs the actual import
     */
    public function __construct(
        private readonly CollectionFactory $profileCollectionFactory,
        private readonly CronExpressionEvaluator $cronEvaluator,
        private readonly ProfileLockManager $lockManager,
        private readonly ProfileImportRunner $importRunner,
    ) {
    }

    /**
     * Runs the periodic check across all profiles.
     *
     * @return void
     */
    public function execute(): void
    {
        $collection = $this->profileCollectionFactory->create();

        foreach ($collection as $profile) {
            if (!$this->cronEvaluator->isDue($profile->getCronExpression())) {
                continue;
            }
            if (!$this->lockManager->acquire((int) $profile->getId())) {
                continue;
            }

            try {
                $this->importRunner->run($profile);
            } finally {
                $this->lockManager->release((int) $profile->getId());
            }
        }
    }
}

4. Calling Magento's import model programmatically instead of via the admin controller

The actual import runs through the same core logic used by the admin controller, namely Magento\ImportExport\Model\Import. Called programmatically, the detour through an HTTP request and session disappears, and the flow splits into two separate steps: validateSource checks the source file against the chosen entity and behavior schema without writing any data yet, and importSource only performs the actual import after successful validation.

This split is deliberate and should be preserved in the automated flow too: a profile whose source file is obviously broken, for example due to a changed column layout from the supplier, should never be partially imported. The automated runner therefore aborts completely on a hard validation error instead of only importing the valid rows and silently discarding the rest.

<?php

declare(strict_types=1);

namespace Mironsoft\ScheduledImportExport\Model;

use Magento\ImportExport\Model\Import;
use Magento\ImportExport\Model\Import\Source\Csv;

/**
 * Runs a single automated import cycle through
 * Magento's import model, called programmatically.
 */
class ProfileImportRunner
{
    /**
     * @param Import $import Magento core import model
     * @param ImportResultNotifier $resultNotifier Evaluates the result and notifies on failure
     */
    public function __construct(
        private readonly Import $import,
        private readonly ImportResultNotifier $resultNotifier,
    ) {
    }

    /**
     * Runs validation and import for a single profile.
     *
     * @param \Mironsoft\ScheduledImportExport\Model\Profile $profile Profile to process
     * @return void
     */
    public function run(Profile $profile): void
    {
        $this->import->setData([
            'entity' => $profile->getEntityType(),
            'behavior' => Import::BEHAVIOR_APPEND,
        ]);

        $source = new Csv($profile->getSourcePath(), $this->import->getWorkingDir());

        $validationResult = $this->import->validateSource($source);
        if (!$validationResult) {
            $this->resultNotifier->notifyFailure($profile, $this->import->getErrorAggregator());
            return;
        }

        $this->import->importSource();
        $this->resultNotifier->notifyResult($profile, $this->import->getErrorAggregator());
    }
}

5. Evaluating results through the error aggregator

Both validateSource and importSource collect issues in the error aggregator instead of throwing a single exception, which makes sense for bulk data since one broken record shouldn't automatically block the entire import. The aggregator distinguishes errors by severity, critical errors that stop the import entirely, and row-level errors that flag individual bad records without blocking the rest of the import.

For automated evaluation, a simple hasFatalExceptions or getErrorsCount check on the aggregator isn't enough to tell the recipient of a failure notification what actually went wrong. It's better to read the individual error messages via getAllErrors with row number and error text and include a compact, human-readable summary in the notification instead of sending a generic failed message.

6. Notification for failed automated imports

An automated import that fails silently is more dangerous than one that was never automated in the first place, because trust in current data stays deceptively intact. Notification happens through Magento\Framework\Mail\Template\TransportBuilder with a dedicated email template containing profile name, timestamp, count of processed and failed rows, and the most important error messages from the error aggregator.

For teams with existing Slack or Teams integration, a simple webhook call alongside the email is worth adding, since email notifications are easy to miss day to day while a chat message tends to get noticed much faster. Both channels should be allowed to fail independently, an unreachable webhook should never prevent the email notification from being sent.

7. Logging run history for traceability

Every automated run should be logged in a dedicated history table regardless of outcome, with start and end time, result status, and a reference to the full error list. Without this history it's nearly impossible to reconstruct after the fact whether a given automated import actually ran on a given day or why it failed, which matters especially for intermittent network issues against a supplier's FTP server.

A dedicated admin grid on top of this history table, built following the same UI component pattern as every other grid in the project, makes the history traceable for admin users without database access and lets them filter specifically for failed runs of a given profile, with no additional reporting tool needed.

8. Distinguishing this from performance optimization of large catalogs

This article deliberately doesn't cover how to make a single import with several million rows fast, batch processing, indexing strategy during the import, or memory usage of large CSV files are their own topic with their own solution patterns. The automation layer described here works independently of the size of the individual import and can be combined with a performance-optimized import implementation without the two layers affecting each other.

In practice this means: teams that already have a performant import pipeline for large catalogs only need to add the trigger timing, error evaluation, and notification for automation, without touching the actual import logic. The two topic areas complement each other but are technically clearly separated.

9. Handling credentials and source files securely

Automated profiles frequently reach out to external sources, an SFTP directory from a supplier or an API endpoint with its own access token, and these credentials should never end up as a plain-text column on the scheduled operation entity. Magento's own mechanism for encrypted configuration values, accessible via EncryptorInterface, also works for values managed through a custom entity rather than system.xml, as long as encryption and decryption are consistently encapsulated in one place in the code.

A clear separation between the directory where supplier files get dropped and the rest of the shop's file system matters just as much. An import profile that can accidentally access an arbitrary, user-supplied path opens a path traversal attack surface, which is why the source path should be validated server-side against a fixed base directory instead of passing user input straight through to the file system.

10. Summary

11. FAQ: Automated Import/Export Profiles in Magento 2: The Essentials