|
# Async-Limiter |
|
|
|
A module for limiting concurrent asynchronous actions in flight. Forked from [queue](https://github.com/jessetane/queue). |
|
|
|
[](http://www.npmjs.org/async-limiter) |
|
[](https://travis-ci.org/STRML/async-limiter) |
|
[](https://coveralls.io/r/STRML/async-limiter) |
|
|
|
This module exports a class `Limiter` that implements some of the `Array` API. |
|
Pass async functions (ones that accept a callback or return a promise) to an instance's additive array methods. |
|
|
|
## Motivation |
|
|
|
Certain functions, like `zlib`, have [undesirable behavior](https://github.com/nodejs/node/issues/8871#issuecomment-250915913) when |
|
run at infinite concurrency. |
|
|
|
In this case, it is actually faster, and takes far less memory, to limit concurrency. |
|
|
|
This module should do the absolute minimum work necessary to queue up functions. PRs are welcome that would |
|
make this module faster or lighter, but new functionality is not desired. |
|
|
|
Style should confirm to nodejs/node style. |
|
|
|
## Example |
|
|
|
``` javascript |
|
var Limiter = require('async-limiter') |
|
|
|
var t = new Limiter({concurrency: 2}); |
|
var results = [] |
|
|
|
// add jobs using the familiar Array API |
|
t.push(function (cb) { |
|
results.push('two') |
|
cb() |
|
}) |
|
|
|
t.push( |
|
function (cb) { |
|
results.push('four') |
|
cb() |
|
}, |
|
function (cb) { |
|
results.push('five') |
|
cb() |
|
} |
|
) |
|
|
|
t.unshift(function (cb) { |
|
results.push('one') |
|
cb() |
|
}) |
|
|
|
t.splice(2, 0, function (cb) { |
|
results.push('three') |
|
cb() |
|
}) |
|
|
|
// Jobs run automatically. If you want a callback when all are done, |
|
// call 'onDone()'. |
|
t.onDone(function () { |
|
console.log('all done:', results) |
|
}) |
|
``` |
|
|
|
## Zlib Example |
|
|
|
```js |
|
const zlib = require('zlib'); |
|
const Limiter = require('async-limiter'); |
|
|
|
const message = {some: "data"}; |
|
const payload = new Buffer(JSON.stringify(message)); |
|
|
|
// Try with different concurrency values to see how this actually |
|
// slows significantly with higher concurrency! |
|
// |
|
// 5: 1398.607ms |
|
// 10: 1375.668ms |
|
// Infinity: 4423.300ms |
|
// |
|
const t = new Limiter({concurrency: 5}); |
|
function deflate(payload, cb) { |
|
t.push(function(done) { |
|
zlib.deflate(payload, function(err, buffer) { |
|
done(); |
|
cb(err, buffer); |
|
}); |
|
}); |
|
} |
|
|
|
console.time('deflate'); |
|
for(let i = 0; i < 30000; ++i) { |
|
deflate(payload, function (err, buffer) {}); |
|
} |
|
q.onDone(function() { |
|
console.timeEnd('deflate'); |
|
}); |
|
``` |
|
|
|
## Install |
|
|
|
`npm install async-limiter` |
|
|
|
## Test |
|
|
|
`npm test` |
|
|
|
## API |
|
|
|
### `var t = new Limiter([opts])` |
|
Constructor. `opts` may contain inital values for: |
|
* `q.concurrency` |
|
|
|
## Instance methods |
|
|
|
### `q.onDone(fn)` |
|
`fn` will be called once and only once, when the queue is empty. |
|
|
|
## Instance methods mixed in from `Array` |
|
Mozilla has docs on how these methods work [here](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array). |
|
### `q.push(element1, ..., elementN)` |
|
### `q.unshift(element1, ..., elementN)` |
|
### `q.splice(index , howMany[, element1[, ...[, elementN]]])` |
|
|
|
## Properties |
|
### `q.concurrency` |
|
Max number of jobs the queue should process concurrently, defaults to `Infinity`. |
|
|
|
### `q.length` |
|
Jobs pending + jobs to process (readonly). |
|
|
|
|