How to pause and control queues in Laravel step by step

Informatec Digital » Resources » How to pause and control queues in Laravel step by step

Working with queues in Laravel is almost mandatory in any halfway serious project: sending emails, processing CSV files, generating PDFs, resizing images, calling external APIs… If all of that is done in the main HTTP request, the website becomes slow, 500 errors appear, and the user ends up desperately staring at a blank screen.

The good news is that Laravel offers a powerful and flexible queuing infrastructure that allows you to delegate heavy lifting to background processes, manage them by priority, group them into batches, monitor them, retry them when they fail, and even temporarily pause workers for maintenance. Let's take a close look (and leave nothing out) at how to get the most out of it, including how to pause, resume, and control its behavior in production.

Basic concepts: connections, queues, and drivers

In Laravel, before pausing anything, you need to understand the difference between queue connections and queues themselves. In config/queue.php you have an array connections where you define each backend: database, redis, beanstalkd, sqs, sync, null, failover, rabbitmq (if you add it), etc.

Each connection can have several logical queues (names like default, emails, high, low, importsThe attribute queue Each connection indicates which is the default queue where jobs will go if no other is specified. onQueue()This way you can do things like:

• A quick queue for emails or short transactions.
• Another low priority for long tasks such as processing videos or large reports.
• Queues segmented by task type: imports, notificaciones, reportes, etc.

When you launch the worker with php artisan queue:workBy default, it listens to the queue specified as default on the selected connection. If you want to prioritize, you can pass multiple comma-separated queues:

php artisan queue:work redis --queue=high,default,low

This way, the worker processes high first , then default , and only then low , which is crucial when the system is loaded and you don't want a 30-minute report to block high-priority emails.

Queue drivers and prerequisites

Laravel offers a unified API for several queue drivers , but each has its own specific features and dependencies. Choosing the right driver is crucial before discussing pausing workers.

Driver database

The driver database It stores the jobs in an SQL table (usually jobsIt's easy to set up and perfect for small projects or development environments where you don't want to mess around with Redis or SQS.

Basic steps:

• Generate the migration: php artisan queue:table (or in newer versions, it comes as create_jobs_table default).
• Run migrations: php artisan migrate.
• Setup QUEUE_CONNECTION=database at the .env and confirm in config/queue.php that 'default' => env('QUEUE_CONNECTION','database').

It is convenient and robust enough, but with many queues and a lot of traffic it can become a bottleneck , since everything goes through the same relational database.

Driver Redis

The Redis driver is the standard workhorse in production. It's fast, supports advanced features (locking, rate limiting, atomic locks, Horizon, etc.), and scales very well.

Important points:

• You need to configure the Redis connection in config/database.php.
• The options serializer y compression from Redis They are not compatible with the queue driver.
• If you use Redis ClusterThe names of the queues must have a hash tag, for example {default}so that all the keys in that queue fall into the same slot.
• The option block_for It allows the worker to block while waiting for jobs for a few seconds, instead of continuously polling: saves CPU and reduces queries to the Redis server.

For example:

'redis' => ,

If you put block_for = 0The worker remains blocked indefinitely until a job becomes available, something to keep in mind if you want it to respond quickly to signals like SIGTERM or pause commands.

Other drivers: SQS, Beanstalkd, null, failover…

In addition to databases and redis, Laravel supports other systems :

• Amazon SQSAWS managed queues, with support for FIFOMessage groups, deduplication, and advanced visibility settings. Requires the SDK. aws/aws-sdk-php.
• BeanstalkdVery quick and easy, using the package pda/pheanstalk.
• sync: performs the tasks inmediatamente in the same process. Useful in development or testing, but in production it's practically like not using queues at all.
• null: Discards jobs as soon as they are "dispatched". Used to completely disable the queuing system.
• fail overThis allows you to configure multiple connections, and if the primary one fails, Laravel attempts to push the job to the next one in the list. Perfect for environments where the High availability is critical.

At the level of pausing and resuming workers, the logic is practically the same regardless of the driver, because it is managed from the Artisan commands and the Worker class , not from the queue backend.

Job creation and model serialization

Un Jobs In Laravel, it's simply a class that encapsulates a task to be executed in a queue. It's typically generated with Artisan and saved in app/Jobs.

To create one:

php artisan make:job ProcessPodcast

The generated class implements ShouldQueue and usually uses traits Dispatchable, InteractsWithQueue, Queueable y SerializesModelsThe heart is in the method handle()which is what the worker does when the job arrives.

A particularly powerful feature is the automatic serialization of Eloquent models . If you accept a model in the Job constructor, Laravel doesn't add the entire object to the queue, only its identifier. When the job is processed, it reconstructs the model from the database with its relationships loaded.

Typical example (in pseudocode, to avoid repeating it verbatim):

• Builder receives Podcast $podcast.
• The trait SerializesModels It is responsible for saving the ID.
• En handle(AudioProcessor $processor) You do the hard work (processing audio, for example).

If you don't want the relationships in a model to be serialized (so the payload isn't huge), you can use $model->withoutRelations() or in PHP 8 the attribute # on the property promoted by the builder.

Dependency injection in the handle method

Another advantage is that you can inject services directly into the handle method (as if it were a controller), because Laravel also uses the service container in Jobs. For example, you can request a AudioProcessorHTTP clients, repositories, etc., and the framework takes care of resolving them.

If you need ultra-fine control of what's called the handle, you can use Container::bindMethod() in a service provider and define the injection logic yourself, but in most cases there is no need to complicate things so much.

Job Middleware

Just as routes have middleware, Jobs can also have specific middleware that run before and after the job. This greatly reduces repetitive code in the methods handle() and it is especially useful for:

• Rate limiting with Redis.
• Avoid overlaps of works that touch on the same resource.
• Regular exceptions for unstable APIs.
• Skipping jobs under certain conditions (Skip middleware).

Middleware is usually placed in app/Jobs/Middleware and are returned from the method middleware() From Job. Conceptual example:

public function middleware(): array { return ;

Special types of jobs: unique, encrypted, and locked

Laravel allows you to mark certain jobs as only to ensure that no two identical instances are ever queued at the same time. This is achieved by implementing the interface ShouldBeUnique o ShouldBeUniqueUntilProcessingand supports defining a key uniqueId() and a time uniqueFor.

This is ideal for things like:

• Recalculate a product's search index: only once per product until it finish.
• Synchronizations with external services that do not allow concurrent requests for the same resource.

Underneath, Laravel uses a atomic lock of cache (Redis, memcached, database, etc.). You can also choose the cache driver for that lock with uniqueVia().

In addition, there is the interface ShouldBeEncrypted, with which you indicate that the complete Job is Enter your code before entering the queueVery useful when passing sensitive data and you don't want it to travel in plain text through Redis, SQS or the database.

Dispatch, delay, and execute jobs in different modes

Once your job is defined, you launch it with Job::dispatch() from controllers, event listeners, Artisan commands, or even closures. The parameters you pass to dispatch They go directly to the builder of Job.

Interesting variations:

• dispatchIf() y dispatchUnless() to condition the shipment in a single line.
• delay() so that the job cannot be processed until a specific date/time.
• dispatchAfterResponse() o afterResponse() en dispatch() Closures, which defer execution until the HTTP response has been sent, but within the same PHP process.
• dispatchSync() to run the Job synchronously (without going through the queue) but reusing the same class.
• Special connections such as deferred o background to execute after the response, in the same process or in a new PHP process.

Finally, you can specify connection and queue with chained methods onConnection() y onQueue(), either when dispatching or within the Job builder.

Fine control: retries, backoff, timeouts, and error handling

A big part of working with queues in Laravel is deciding how many times to retry a job , how long to wait between attempts, and when to consider it a failure.

Available options:

• --tries en queue:work to set the maximum number of attempts for all jobs of that worker (unless the Job defines its own) $tries o tries()).
• $tries or a method tries() in the Job class for per-Job control.
• retryUntil() to say: “try again as you wish until this time, after that, failure.” If it coincides with tries, send retryUntil.
• $maxExceptions to cut off when the problem is due to repeated unhandled exceptions.
• $timeout o --timeout to indicate how many seconds a Job can run before the worker kills the child process.
• $failOnTimeout to mark the job as failed in case of timeout.
• $backoff the method backoff() to define how long to wait between retries, even with exponential values.

If a job throws an uncaught exception, Laravel will... releases back into the tail until the configured attempts are exhausted. You can control this manually with release($segundos) o fail($exception) when you want to deliberately mark a job as failed.

Failed jobs, failed_jobs table and pruning

When a Job exhausts his attempts, he comes to be considered failed work and Laravel inserts it into the table failed_jobs (or in DynamoDB if you configure it that way). That table stores UUID, connection, queue, job payload, and the exception that was thrown.

Useful commands:

• php artisan queue:failed to list the failed ones.
• php artisan queue:retry <id|all> to try them again.
• php artisan queue:forget <id> to delete one.
• php artisan queue:flush to delete all (with option for hours in newer versions).
• php artisan queue:prune-failed --hours=48 to automatically prune old records.

In Jobs you can define a method failed(Throwable $exception) For extra cleanup: notify the user, revert partial actions, write specialized logs, etc. Note that a new instance of the Job is created to call failed()Therefore, do not rely on property changes made in handle().

Job batching: batches of jobs and combining them with strings

When you need to launch hundreds or thousands of related jobs (for example, importing a giant CSV in chunks, processing files in batches, cleaning data, etc.), it's advisable to use Laravel's batch functionality .

Basic flow:

• create the table job_batches with php artisan queue:batches-table && php artisan migrate.
• Mark the Jobs that will be part of batches with the trait Batchable.
• Use Bus::batch() to send a set of Jobs, with callbacks then, catch y finally.
• Access the batch from each Job with $this->batch() and check if it's cancelled.
• Possibility to add more Jobs to an ongoing batch with $this->batch()->add().

Batches integrate beautifully with chains : you can have chains within a batch or batches as elements of a larger chain. This is especially useful when you want multiple chains to run "in parallel" and trigger something when they've all completed.

Worker execution, priorities, and deployment

All the magic of queues happens because you have one or more workers running the command queue:work (o queue:listen(although the latter is less efficient). These processes are long-running daemons that listen to their respective queues.

Key options of queue:work:

• Connection: php artisan queue:work redis, queue:work database, etc.
• Drives: --queue=high,default,low to prioritize.
• -eleven: processes a single job and exits.
• –max-jobs: processes X jobs and terminates (to periodically restart workers and free up memory).
• –max-time: processes for N seconds and then exits.
• –sleep: seconds of waiting when no jobs are available.
• –stop-when-empty: empties the queue and closes cleanly (ideal in Docker environments or one-off jobs).
• –timeout: seconds of timeout per job, as we mentioned before.
• –Force: to process jobs even if the app is in maintenance mode.

Because these processes are long-running, in production a process monitor like Supervisor (on Linux) is almost always used to start X workers, restart them if they crash, and manage logs. A typical Supervisor configuration block for Laravel defines:

• command ’s relationship with the php artisan queue:work ... .
• numprocs to indicate how many workers to launch in parallel.
• Log paths, user, automatic restarts, etc.

Pausing and resuming queues in Laravel

Let's get to what's often important in maintenance: how to pause queue workers without abruptly stopping them and without leaving jobs unfinished. Laravel offers several mechanisms, some newer, some classic, to control this.

Pause: queue:pause and interrupt signals

In recent versions, Laravel adds specific commands to pause and resume the execution of workers:

• php artisan queue:pause: tells the system to stop taking new jobs, but to let the ones already in progress finish.
• php artisan queue:continue: resumes processing new jobs from where it left off.

These commands work by sending signals to the workers through the caching mechanism that Laravel uses internally. That's why it's important to have a properly configured cache driver (memcached, redis, database, file, etc.) even if your actual cache queue is something else.

Important: Pausing a queue or worker does not cancel or delete jobs already queued ; it simply prevents more jobs from being removed from the queue while the pause lasts.

Polling of disruptions and impact on performance

Every time a worker finishes a job, Laravel checks the cache for restart, pause, or continue signals . This has a negligible cost, but it exists. If you're extremely focused on performance and know you won't need to pause or restart workers from commands, you can disable this polling by calling:

Queue::withoutInterruptionPolling();

Or even adjust static properties of Illuminate\Queue\Worker to disable specific touches ($restartable, $pausableHowever, if you do this and then try to use queue:pause o queue:restartthe workers They won't find outIt's a design decision that should be carefully considered.

Pause manually with Supervisor or Docker

In addition to Laravel's own commands , in production environments, processes are often "paused" indirectly:

• Stopping the program in Supervisor (supervisorctl stop laravel-worker:*), which kills or stops the workers.
• Using –stop-when-empty in deployments to let the workers finish what's in the queue and then shut down.
• In Docker, letting the container process with --stop-when-empty and then it turns off.

The difference with queue:pause By completely stopping the processes, you will stop processing jobs until the workers restart. internal pause You can keep the processes in memory, waiting for the command continueThis is useful for quick roll-back deployments or short maintenance periods.

Database transactions and after_commit

A classic mistake: dispatching jobs within database transactions . If you're not careful, the worker might execute the job before the commit is made, and the model or data you expect might not yet be in the database.

To avoid this dance, you have two options:

• Activate 'after_commit' => true in the queue connection configuration. Thus, any Job dispatched within a transaction is postponed internally until it is confirmed.
• Use per-job methods afterCommit() o beforeCommit() when dispatching, to force the behavior only in specific Jobs.

If the transaction is reversed, the jobs dispatched during it are discarded without ever entering the queue, which is precisely what is usually desired to maintain consistency.

Monitoring, queue cleaning, and testing

Once you have queues up and running in earnest, it's not enough to just throw out Jobs and forget about it; you should monitor and test how everything behaves.

Useful tools and commands:

• queue:clear to empty a specific queue (SQS, Redis, database).
• queue:monitor to monitor the number of jobs in certain queues and trigger the event QueueBusy if they exceed a threshold.
• Job Events , the Queue::before, Queue::after, Queue::looping y Queue::failing to hook custom logic (logs, metrics, rollbacks, etc.).
• Fakes in testing using Queue::fake() y Bus::fake() to verify that Jobs, strings or batches have been dispatched without actually executing them.

In tests you can state things like “this Job has been queued X times” , “a chain with these Jobs has been dispatched in this order” or “no particular Job has been queued”, which provides a lot of security in complex flows.

With this entire ecosystem—drivers like Redis or SQS, well-designed jobs, middleware for fine-tuning, batches, chains, retry configuration, Supervisor-managed workers, and commands like queue:pause y queue:continue— Laravel lets you master background processing down to the millimeter, from typical email sending to distributed queue architectures with failover, ensuring your application continues to respond quickly at the HTTP layer while heavy work is being done underneath without blocking the user.