job-queue

0.0.4 • Public • Published

Job QueueBuild Status

A queueing system to schedule jobs and have multiple rate-limited consumers.

API

Setting up a Job Queue

jobQueue = new JobQueue consumerslimitperiod
Parameter Type Description
consumers Function A function that will accept a job as the only parameter.
limit Integer The maximum number of jobs the consumer should process in period milliseconds.
period Integer The number of milliseconds limit applies to.

Adding Consumers

jobQueue.addConsumers consumerslimitperiod

Function signature for addConsumers is same as the above constructor.

Enqueuing a Job

jobQueue.enqueue job
Parameter Type Description
job Anything The job can either be any type (Object, Function, Number, ...). Its type depends on what the consumer takes as its argument.

Getting Count of Pending Jobs

pendingJobs = jobQueue.pendingJobs

Example Usage

JobQueue = require "job-queue"
 
makeConsumer = (consumerId) ->
    (job) ->
        console.log "Consumer #{consumerId} processing job #{job.id}"
        job.process consumerId
 
# Create a process queue with 5 consumers where each supports up to 5 requests per second 
processQueue = new JobQueue [1..5].map(makeConsumer)51000
 
# Add another 5 consumers to the process queue where each supports up to 80 jobs per minute 
processQueue.addConsumers [6..10].map(makeConsumer)8060 * 1000
 
# Adding 10k jobs to the process queue 
for jobId in [1..10000then do (jobId) ->
    processQueue.enqueue
        id: jobId
        process: (consumerId) ->
            # The rate limited section (e.g. some API call) 
            console.log "Job #{jobId} processed by consumer #{consumerId}"

Package Sidebar

Install

npm i job-queue

Weekly Downloads

4

Version

0.0.4

License

BSD

Last publish

Collaborators

  • gotemb