Background Work as Architecture: Laravel Jobs and Eventual Consistency

A queue does more than speed up an HTTP response. It introduces an asynchronous boundary: the request can say that an invoice is being prepared while the PDF, the accounting export, and the email do not yet exist. A worker can die after calling a provider but before recording the response. A deploy can replace the code while a reserved payload still refers to the old class. Those are ordinary operating states, so a job needs an explicit business contract.

That contract answers five questions: which durable input identifies the work; what success means; which effects can safely repeat; how long retry is useful; and how an operator recovers a permanent failure. We will use invoice creation to make each decision concrete.

At least once is the useful default

Queue drivers aim for at-least-once delivery. If a worker loses a reserved job, the driver may make it available again. There is no switch for exactly-once execution because a network failure can always leave an unknown outcome. Build for repetition with a stable business key, a unique database constraint, and an idempotency key for any external API.

Pass scalar identifiers through the queue, then load current state in handle(). Do not serialize a request, the authenticated user, or a large Eloquent graph. The request is gone when the worker runs, relations can be stale, and hidden serialization makes the job's database dependency unclear.

<?php

declare(strict_types=1);

namespace App\Jobs;

use App\Models\Order;
use App\Services\InvoiceGenerator;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

final class GenerateInvoice implements ShouldQueue
{
    use Dispatchable;
    use InteractsWithQueue;
    use Queueable;
    use SerializesModels;

    public int $timeout = 120;
    public int $tries = 3;

    /** @var array<int, int> */
    public array $backoff = [10, 60, 300];

    public function __construct(public readonly int $orderId)
    {
    }

    public function handle(InvoiceGenerator $invoiceGenerator): void
    {
        $order = Order::query()->findOrFail($this->orderId);

        if (! $order->is_paid || $order->invoice_path !== null) {
            return;
        }

        $invoiceGenerator->generateFor($order);
    }
}

The early return is a useful first line, not the final guarantee. Two workers can both observe invoice_path as null. Put the actual invariant where the write occurs: create an invoice row with a unique order_id, allocate a unique invoice number, or make an atomic conditional update. A job reduces duplicate opportunities; the database decides whether two records can exist.

Commit the cause before dispatching its consequence

A worker can otherwise win the race against the transaction that created its input. It looks up an uncommitted order, throws ModelNotFoundException, and wastes a retry. Dispatch after commit when the job depends on a row written in the current transaction.

<?php

declare(strict_types=1);

namespace App\Actions;

use App\Jobs\GenerateInvoice;
use App\Models\Order;
use Illuminate\Support\Facades\DB;

final class MarkOrderPaid
{
    public function execute(Order $order): void
    {
        DB::transaction(function () use ($order): void {
            $order->update(['paid_at' => now()]);

            GenerateInvoice::dispatch($order->getKey())->afterCommit();
        });
    }
}

afterCommit() orders dispatch after a successful commit; it does not replace the transaction, nor does it freeze business state. The job must still handle a later cancellation. This is eventual consistency: the system advances through visible states, rather than pretending every subsystem changed atomically.

Idempotency belongs beyond the database

Your database can reject a duplicate key. An accounting API is outside that transaction. Give it an idempotency key derived from the business operation (not the attempt number), so attempt one and attempt three mean the same thing.

<?php

declare(strict_types=1);

namespace App\Services;

use App\Models\Order;
use Illuminate\Http\Client\Factory as HttpFactory;

final class AccountingExporter
{
    public function __construct(private readonly HttpFactory $http)
    {
    }

    public function export(Order $order): void
    {
        $this->http->acceptJson()
            ->timeout(15)
            ->connectTimeout(5)
            ->withHeaders([
                'Idempotency-Key' => "order:{$order->getKey()}:accounting-export",
            ])
            ->post(config('services.accounting.url').'/invoices', [
                'order_id' => $order->getKey(),
                'total_cents' => $order->total_cents,
            ])
            ->throw();
    }
}

When a provider cannot deduplicate, persist an outbox record with a unique business key before dispatching and mark it sent only after a confirmed reply. If the reply is lost after a successful request, the unknown outcome has a durable record for reconciliation instead of becoming an invisible duplicate.

Retry for the business deadline

Retries must not hammer an outage and must stop when continuing is no longer useful. Keep the queue connection's retry_after comfortably above a job's timeout; otherwise a second worker can receive the same job while the first still runs. For a payment reconciliation that may reasonably wait until close of business, use a deadline instead of an arbitrary number of attempts.

<?php

declare(strict_types=1);

namespace App\Jobs;

use DateTimeInterface;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

final class ReconcilePayment implements ShouldQueue
{
    use Dispatchable;
    use Queueable;

    public int $tries = 0;

    /** @var array<int, int> */
    public array $backoff = [60, 300, 900, 1800];

    public function __construct(public readonly string $providerPaymentId)
    {
    }

    public function retryUntil(): DateTimeInterface
    {
        return now()->endOfDay();
    }
}

Validation errors, deleted orders, and an API's “invalid account” response are not transient. Retrying them only delays a decision. Record an actionable state in failed() so the operator sees the business object, not just a failed queue payload.

public function failed(\Throwable $exception): void
{
    Order::query()->whereKey($this->orderId)->update([
        'invoice_status' => 'failed',
    ]);

    report($exception);
}

Unique dispatches are not a data invariant

ShouldBeUnique prevents duplicate dispatch for one uniqueId() while its lock exists. It suits one CRM sync per customer. ShouldBeUniqueUntilProcessing releases the lock when handling begins, which is useful only when a new sync may wait behind the current one.

final class SyncCustomerFromCrm implements ShouldBeUnique, ShouldQueue
{
    use Dispatchable;
    use Queueable;

    public int $uniqueFor = 900;

    public function __construct(public readonly int $customerId)
    {
    }

    public function uniqueId(): string
    {
        return (string) $this->customerId;
    }
}

It cannot protect the same operation invoked by a command, webhook, and old job. For a short critical section take an explicit Cache::lock(), but retain the database invariant because a lock may expire during slow work. Locks serialize work, so they are a poor default for broad imports or an unhealthy upstream; use them for scarce, per-entity coordination.

Cache::lock("invoice:{$order->getKey()}", 180)->block(5, function () use ($order): void {
    app(InvoiceGenerator::class)->generateFor($order);
});

Batch independent work and chain dependencies

CSV chunks can run independently, so model them with a batch and expose batch progress. Invoice generation, accounting export, and notification have a strict order, so model them as a chain.

Bus::batch($chunkPaths->map(fn (string $path) => new ImportCustomersChunk($path))->all())
    ->name('September customer import')
    ->then(fn () => FinalizeCustomerImport::dispatch($importId))
    ->catch(fn ($batch, \Throwable $exception) => MarkImportFailed::dispatch($importId))
    ->dispatch();

Bus::chain([
    new GenerateInvoice($orderId),
    new ExportInvoiceToAccounting($orderId),
    new SendInvoiceReadyNotification($orderId),
])->dispatch();

Neither is a distributed transaction. A chain can export successfully and stop before notification. Persist observable milestones such as exported_at and notification_sent_at, then let a repair command safely resume the missing step instead of rerunning everything blindly.

Workers are deployed application code

Workers are long-running processes and will not load new PHP automatically. After compatible code and schema are available, use php artisan queue:restart so standard workers exit after their current job; terminate Horizon during a Horizon deployment so its manager replaces workers. Keep reserved old payloads compatible with the release: adding an optional field is safer than removing a job class or renaming a serialized property.

Horizon is excellent for separating high, default, and low queues, observing wait time, failures, memory, and process balance. It cannot determine whether an external call succeeded immediately before a crash.

Test observable safety

Test that the action dispatches the job, that completed work is a no-op, and that an external request receives the stable key. Bus::fake() is appropriate at the dispatch boundary; a direct job test with a mocked generator verifies the idempotent branch. Add an integration test with a real queue driver when serialization itself matters.

it('does not generate an invoice twice', function (): void {
    $order = Order::factory()->create([
        'paid_at' => now(),
        'invoice_path' => 'invoices/123.pdf',
    ]);
    $generator = Mockery::mock(\App\Services\InvoiceGenerator::class);
    $generator->shouldNotReceive('generateFor');

    (new GenerateInvoice($order->getKey()))->handle($generator);
});

Do not queue a small, user-critical write simply because queues look architectural: checkout should reserve stock synchronously. Do not queue work that must be visible in the same HTTP response. For short, non-critical post-response work on one server, defer() may be clearer. A queue is the right boundary when work is slow, independently retryable, and its delay is safe and visible to the user.