Compactr
Schema based serialization made easy
What is this and why does it matter?
Protocol Buffers are awesome. Having schemas to deflate and inflate data while maintaining some kind of validation is a great concept. Compactr's goal is to build on that to better suit the Javascript ecosystem.
Install
npm install compactr
Implementation
const Compactr = require('compactr'); // Defining a schemaconst userSchema = Compactr.schema({ id: { type: 'number' }, name: { type: 'string' }}); // EncodinguserSchema.write({ id: 123, name: 'John' }); // Get the header bytesconst header = userSchema.headerBuffer(); // Get the content bytes const partial = userSchema.contentBuffer(); // Get the full payload (header + content bytes)const buffer = userSchema.buffer(); // Decoding a full payloadconst content = userSchema.read(buffer); // Decoding a partial payload (content)const content = userSchema.readContent(partial);
Size comparison
JSON: {"id":123,"name":"John"}
: 24 bytes
Compactr (full): <Buffer 02 00 01 01 04 7b 4a 6f 68 6e>
: 10 bytes
Compactr (partial): <Buffer 7b 4a 6f 68 6e>
: 5 bytes
Protocol details
Data types
Type | Count bytes | Byte size |
---|---|---|
boolean | 0 | 1 |
number | 0 | 8 |
int8 | 0 | 1 |
int16 | 0 | 2 |
int32 | 0 | 4 |
double | 0 | 8 |
string | 1 | 2/char |
char8 | 1 | 1/char |
char16 | 1 | 2/char |
char32 | 1 | 4/char |
array | 1 | (x)/entry |
object | 1 | (x) |
unsigned | 0 | 8 |
unsigned8 | 0 | 1 |
unsigned16 | 0 | 2 |
unsigned32 | 0 | 4 |
- Count bytes range can be specified per-item in the schema*
See the full Compactr protocol
Benchmarks
[Array] JSON x 737 ops/sec ±1.04% (91 runs sampled)
[Array] Compactr x 536 ops/sec ±1.25% (82 runs sampled)
[Array] size: { json: 0, compactr: 10 }
[Boolean] JSON x 996 ops/sec ±0.84% (93 runs sampled)
[Boolean] Compactr x 1,439 ops/sec ±1.24% (83 runs sampled)
[Boolean] Protobuf x 2,585 ops/sec ±1.32% (91 runs sampled)
[Boolean] size: { json: 23, compactr: 5, protobuf: 5 }
[Float] JSON x 713 ops/sec ±1.29% (88 runs sampled)
[Float] Compactr x 1,056 ops/sec ±1.38% (86 runs sampled)
[Float] Protobuf x 2,432 ops/sec ±1.45% (88 runs sampled)
[Float] size: { json: 41, compactr: 12, protobuf: 12 }
[Integer] JSON x 954 ops/sec ±0.77% (93 runs sampled)
[Integer] Compactr x 1,520 ops/sec ±1.21% (89 runs sampled)
[Integer] Protobuf x 2,676 ops/sec ±1.22% (90 runs sampled)
[Integer] size: { json: 24, compactr: 8, protobuf: 7 }
[Object] JSON x 512 ops/sec ±0.62% (90 runs sampled)
[Object] Compactr x 412 ops/sec ±1.46% (81 runs sampled)
[Object] Protobuf x 898 ops/sec ±0.92% (93 runs sampled)
[Object] size: { json: 46, compactr: 13, protobuf: 25 }
[String] JSON x 432 ops/sec ±0.79% (88 runs sampled)
[String] Compactr x 428 ops/sec ±0.87% (87 runs sampled)
[String] Protobuf x 738 ops/sec ±0.71% (90 runs sampled)
[String] size: { json: 57, compactr: 14, protobuf: 28 }
Want to help ?
You are awesome! Open an issue on this project, identifying the feature that you want to tackle and we'll take the discussion there!
License
Apache 2.0 (c) 2019 Frederic Charette