binary-stream-multiplexer
binary-stream-multiplexer
allows you to record multiple binary streams simultaneously, and to play them back later. It multiplexes up to 254 streams to a single, compact binary stream (or file). Later, you can replay a recorded multiplexed file, which will be de-multiplexed back into N streams.
Because of Node.js
's excellent streaming interface, virtually all streams in Node look exactly the same (HTTP streams, file streams, serial port streams, ... etc, all have the same interface). If you have an application where you are relying on live streaming data, it can be useful to have the ability to play back that data in real-time (that is, with simulated delays that mimic the original real-world delays of the data). The more streams you have, the more useful this is.
A trivial but real-world example: let's say you have three servers that expose a streaming binary interface. The binary interfaces emit events in a binary protocol at about 15 times per second, or about every 66 milliseconds. You have a dashboard that consumes these streams in real times and shows summary statistics. About every 10 minutes, the dashboard crashes - what to do? Now you can collect a single log of all the streaming input to your application, and replay it as many times as you need to isolate and fix the bug - and you also have a test fixture to run through tests repeatedly!
See test/index.js
for examples of how to hook up basic streams and demultiplex them.
Install
Install as a dependency to a project:
npm install --save binary-stream-multiplexer
Install CLI utilities:
npm install --global binary-stream-multiplexer
CLI Usage
You cannot record streams using the CLI; you must use the API to record streams. With a recorded file, you can:
See all options:
$ binary-stream-multiplexer --helper
Display the header of a multiplexed file:
$ binary-stream-multiplexer header /path/to/multiplexed/file
Extract one of the encoded streams by ID:
$ binary-stream-multiplexer extract /path/to/multiplexed/file 3 > binary_for_stream_3.out
API Usage
The following functions are all available when you require('binary-stream-multiplexer')
:
recordStreams(streams, filename)
Pass an array of binary streams and a filename. It will start multiplexing the streams'
output to the filename
.
This function returns a stream
that you can .end()
to end multiplexing.
multiplexStreams(streams)
Similar to recordStreams
but doesn't log out to a file - it only returns the output stream.
demultiplexFile(filename, useRealOffsets, callback)
Demultiplexes the binary data at filename
.
useRealOffsets
: a boolean value indicating whether or not the original timing of
the data should be replicated. If so, timeouts will be introduced to the output streams
to mimic the timing of the original streams.
callback(err, streams)
: called with an error or with a list of output streams. The streams
are in the same order that they were when you recorded the data with recordStreams
.
demultiplexBuffer(buf, useRealOffsets, callback)
Pass in a Buffer of multiplexed data. All other parameters
are the same as demultiplexFilename
.
demultiplexStream(stream, useRealOffsets, callback)
Pass in a Stream of multiplexed data. All other parameters
are the same as demultiplexFilename
. This is the primitive function used internally by demultiplexFilename
(which
uses fs.createReadStream()
) and demultiplexBuffer
(which creates a stream and writes the entire buffer to
it immediately). It can also be used externally.
Binary protocol (V1)
TODO
Caveats
The binary protocol (v1) limits you to a maximum of 254 streams.
Binary output size will be artificially inflated if you have many large gaps in data that are more than 60 seconds long.
License
MIT license. See LICENSE
file.