Feat: Pulse with quantization

README.md

@@ -67,3 +67,201 @@ In this case, there is no problem with when each PR is merged. It's recommendabl
 ## Comms
 
 TODO
+
+---
+
+# Bitwise Serialization Plugin (`protoc-gen-bitwise`)
+
+A custom protoc plugin that generates C# partial classes with typed float
+accessors for quantized `uint32` fields in high-frequency MMO networking
+messages (position deltas, player input, etc.).  It runs alongside
+`--csharp_out` in the same protoc invocation; the two output files coexist
+via C# `partial class`.
+
+## How it works
+
+Protobuf encodes `uint32` values as varints, which are already compact for
+small values: a value up to 2¹⁴−1 costs 2 bytes, up to 2²¹−1 costs 3 bytes.
+Rather than a separate binary packing layer, the plugin leverages this:
+
+1. Declare quantized fields as `uint32` in the `.proto` schema and annotate
+   them with `[(decentraland.common.quantized)]` to specify the float range
+   and bit resolution.
+2. `--csharp_out` generates the standard protobuf class with the raw `uint32`
+   property (e.g. `PositionX`).
+3. `--bitwise_out` (this plugin) generates a `partial class` extension with a
+   cached float accessor (e.g. `PositionXQuantized`) that encodes/decodes
+   transparently via `Quantize.Encode` / `Quantize.Decode`.
+
+The wire representation is a standard protobuf message — any protobuf-capable
+client can read it without knowledge of the plugin.
+
+## Prerequisites
+
+| Requirement | Version |
+|---|---|
+| Python | 3.10+ |
+| `protobuf` Python package | 4.x or 3.20+ |
+| `protoc` | 3.19+ |
+
+```bash
+pip install protobuf
+```
+
+## Step 1 — Annotate your `.proto` file
+
+Declare quantized fields as `uint32` and import `options.proto`:
+
+```protobuf
+syntax = "proto3";
+
+import "decentraland/common/options.proto";
+
+package decentraland.kernel.comms.v3;
+
+message PositionDelta {
+  // Float range [-100, 100] quantized to 16 bits ≈ 0.003-unit precision.
+  // Stored as uint32 on the wire; protobuf encodes it as a 3-byte varint.
+  uint32 dx        = 1 [(decentraland.common.quantized)  = { min: -100.0, max: 100.0, bits: 16 }];
+  uint32 dy        = 2 [(decentraland.common.quantized)  = { min: -100.0, max: 100.0, bits: 16 }];
+  uint32 dz        = 3 [(decentraland.common.quantized)  = { min: -100.0, max: 100.0, bits: 16 }];
+
+  // Unannotated uint32: protobuf varint encodes small values compactly by default.
+  uint32 entity_id = 4 [(decentraland.common.bit_packed) = { bits: 20 }];
+}
+```
+
+### Annotation reference
+
+| Annotation | Target type | Parameters | Effect |
+|---|---|---|---|
+| `[(decentraland.common.quantized)]` | `uint32` | `min`, `max`, `bits` | Plugin emits a cached `float {Name}Quantized` accessor |
+| `[(decentraland.common.bit_packed)]` | `uint32` | `bits` | Documents the value range; protobuf handles varint compaction automatically |
+
+### Wire cost at worst-case (all bits set)
+
+| Quantization bits | Max value | Varint bytes | Tag (field ≤ 15) | Total per field |
+|---|---|---|---|---|
+| 8 | 255 | 2 | 1 | 3 B |
+| 12 | 4 095 | 2 | 1 | 3 B |
+| 14 | 16 383 | 2 | 1 | 3 B |
+| 16 | 65 535 | 3 | 1 | 4 B |
+| 20 | 1 048 575 | 3 | 1 | 4 B |
+
+Proto3 omits fields equal to their default value (0), so average cost is lower.
+
+## Step 2 — Run protoc
+
+```bash
+protoc \
+  --proto_path=proto \
+  --proto_path=/path/to/google/protobuf/include \
+  --csharp_out=generated/cs \
+  --plugin=protoc-gen-bitwise=protoc-gen-bitwise/plugin.py \
+  --bitwise_out=generated/cs \
+  proto/decentraland/kernel/comms/v3/comms.proto
+```
+
+The plugin emits one `*.Bitwise.cs` file (PascalCase, flat in the output
+directory) for each `.proto` file that contains at least one `[(quantized)]`
+field.
+
+## Step 3 — Copy the runtime
+
+Copy `Quantize.cs` into your project:
+
+```
+Assets/
+└── Scripts/
+    └── Networking/
+        └── Bitwise/
+            └── Quantize.cs   ← protoc-gen-bitwise/runtime/cs/Quantize.cs
+```
+
+`Quantize.cs` lives in the `Decentraland.Networking.Bitwise` namespace and
+provides two static methods used by the generated accessors:
+
+```csharp
+public static class Quantize
+{
+    public static uint  Encode(float value, float min, float max, int bits);
+    public static float Decode(uint encoded, float min, float max, int bits);
+}
+```
+
+## Step 4 — Use the generated code
+
+The plugin emits a `partial class` that adds float accessors on top of the
+standard protobuf-generated `uint32` properties:
+
+```csharp
+using Decentraland.Kernel.Comms.V3;
+
+// --- Build and send ---
+var delta = new PositionDelta();
+delta.DxQuantized = 3.14f;   // encodes to uint32, stored in delta.Dx
+delta.DyQuantized = 0f;
+delta.DzQuantized = -7.5f;
+delta.EntityId    = 42u;
+
+byte[] bytes = delta.ToByteArray();   // standard protobuf serialization
+SendOnChannel1(bytes);
+
+// --- Receive and read ---
+var received = PositionDelta.Parser.ParseFrom(receivedBytes);
+float x = received.DxQuantized;   // decoded on first access, cached thereafter
+float y = received.DyQuantized;
+float z = received.DzQuantized;
+
+// If raw uint32 fields are mutated directly after construction, invalidate the cache:
+received.ResetDecodedCache();
+```
+
+## Generated file example
+
+For the `PositionDelta` message above the plugin emits `PositionDelta.Bitwise.cs`:
+
+```csharp
+// <auto-generated>
+//   Generated by protoc-gen-bitwise. DO NOT EDIT.
+//   Source: decentraland/kernel/comms/v3/comms.proto
+// </auto-generated>
+
+using Decentraland.Networking.Bitwise;
+
+namespace Decentraland.Kernel.Comms.V3
+{
+    public partial class PositionDelta
+    {
+        private float? _dx;
+        public float DxQuantized
+        {
+            get => _dx ??= Quantize.Decode(Dx, -100.0f, 100.0f, 16);
+            set { _dx = value; Dx = Quantize.Encode(value, -100.0f, 100.0f, 16); }
+        }
+
+        private float? _dy;
+        public float DyQuantized
+        {
+            get => _dy ??= Quantize.Decode(Dy, -100.0f, 100.0f, 16);
+            set { _dy = value; Dy = Quantize.Encode(value, -100.0f, 100.0f, 16); }
+        }
+
+        private float? _dz;
+        public float DzQuantized
+        {
+            get => _dz ??= Quantize.Decode(Dz, -100.0f, 100.0f, 16);
+            set { _dz = value; Dz = Quantize.Encode(value, -100.0f, 100.0f, 16); }
+        }
+
+        /// <summary>Clears all cached decoded values. Call after mutating raw uint32 fields directly.</summary>
+        public void ResetDecodedCache()
+        {
+            _dx = null;
+            _dy = null;
+            _dz = null;
+        }
+    }
+
+} // namespace Decentraland.Kernel.Comms.V3
+```

package.json

@@ -25,9 +25,10 @@
     "protobufjs": "7.2.4"
   },
   "files": [
-    "proto",
+    "proto/decentraland",
     "out-ts",
     "out-js",
-    "public"
+    "public",
+    "protoc-gen-bitwise"
   ]
 }

proto/decentraland/common/options.proto

@@ -0,0 +1,30 @@
+syntax = "proto3";
+
+package decentraland.common;
+
+import "google/protobuf/descriptor.proto";
+
+// Options for quantizing a float value stored as a uint32 field.
+// The float is clamped to [min, max] and uniformly quantized to N bits;
+// protobuf encodes the resulting uint32 as a varint on the wire.
+// The generator emits a float {Name}Quantized accessor in the partial class.
+message QuantizedFloatOptions {
+  float  min  = 1;
+  float  max  = 2;
+  uint32 bits = 3;
+}
+
+// Options for bit-packing an integer field into fewer than 32 bits.
+message BitPackedOptions {
+  uint32 bits = 1;
+}
+
+extend google.protobuf.FieldOptions {
+  // Apply to uint32 fields to enable quantized float encoding.
+  // Example: uint32 dx = 1 [(decentraland.common.quantized) = { min: -100.0, max: 100.0, bits: 16 }];
+  QuantizedFloatOptions quantized  = 50001;
+
+  // Apply to uint32 fields to pack into fewer bits.
+  // Example: uint32 entity_id = 4 [(decentraland.common.bit_packed) = { bits: 20 }];
+  BitPackedOptions      bit_packed = 50002;
+}

proto/decentraland/common/quantization_example.proto

@@ -0,0 +1,132 @@
+syntax = "proto3";
+
+package decentraland.common;
+
+import "decentraland/common/options.proto";
+
+// High-frequency player state messages sent on Channel 1 (unreliable sequenced).
+//
+// Every message below uses the protoc-gen-bitwise annotations to minimise
+// wire size.  Wire costs are listed per-message so the trade-offs are clear.
+//
+// Annotation cheat-sheet:
+//   [(decentraland.common.quantized)  = { min: F, max: F, bits: N }]
+//       → stores a float as a uint32 quantized to N bits over [min, max].
+//       → protobuf encodes the uint32 as a varint: values ≤ 2^14-1 cost 2 bytes,
+//         values ≤ 2^21-1 cost 3 bytes; the generated partial class adds a
+//         float {Name}Quantized accessor that encodes/decodes transparently.
+//   [(decentraland.common.bit_packed) = { bits: N }]
+//       → documents that this uint32 uses at most N bits; protobuf encodes it
+//         as a varint (same savings, no generated accessor needed).
+//   (no annotation)
+//       → standard protobuf encoding (bool/double/etc. at natural width).
+//
+// Varint byte costs at worst-case (all bits set):
+//   ≤ 7 bits  → 1 byte   (max 127)
+//   ≤ 14 bits → 2 bytes  (max 16 383)
+//   ≤ 21 bits → 3 bytes  (max 2 097 151)
+// Proto3 omits fields equal to their default value (0), so average cost is lower.
+
+// ---------------------------------------------------------------------------
+// PositionDelta — Δ position relative to last acknowledged full snapshot.
+//
+// Sent every client tick (~10 Hz) on Channel 1.
+//
+//  Field       | Type   | Range          | Q bits | Wire worst-case
+//  ------------|--------|----------------|--------|-----------------------
+//  dx          | uint32 | [-100, 100]    |   16   | tag 1B + varint 3B = 4B
+//  dy          | uint32 | [-100, 100]    |   16   | tag 1B + varint 3B = 4B
+//  dz          | uint32 | [-100, 100]    |   16   | tag 1B + varint 3B = 4B
+//  entity_id   | uint32 | [0, 1 048 575] |   20   | tag 1B + varint 3B = 4B
+//  sequence    | uint32 | [0,     4 095] |   12   | tag 1B + varint 2B = 3B
+//  ---------------------------------------------------------------------------
+//  Worst-case: 19 B  (vs. 20 B raw: 3×float + 2×uint32)
+//  Step: dx/dy/dz ≈ 0.003 units
+// ---------------------------------------------------------------------------
+message PositionDelta {
+  uint32 dx        = 1 [(decentraland.common.quantized)  = { min: -100.0, max: 100.0, bits: 16 }];
+  uint32 dy        = 2 [(decentraland.common.quantized)  = { min: -100.0, max: 100.0, bits: 16 }];
+  uint32 dz        = 3 [(decentraland.common.quantized)  = { min: -100.0, max: 100.0, bits: 16 }];
+  uint32 entity_id = 4 [(decentraland.common.bit_packed) = { bits: 20 }];
+  uint32 sequence  = 5 [(decentraland.common.bit_packed) = { bits: 12 }];
+}
+
+// ---------------------------------------------------------------------------
+// PlayerInput — client input snapshot for server-side reconciliation.
+//
+// Sent every client frame (~30 Hz) on Channel 1.
+//
+//  Field       | Type   | Range          | Q bits | Wire worst-case
+//  ------------|--------|----------------|--------|-----------------------
+//  move_x      | uint32 | [-1, 1]        |    8   | tag 1B + varint 2B = 3B
+//  move_z      | uint32 | [-1, 1]        |    8   | tag 1B + varint 2B = 3B
+//  yaw         | uint32 | [-180, 180]    |   12   | tag 1B + varint 2B = 3B
+//  buttons     | uint32 | bitmask        |    8   | tag 1B + varint 2B = 3B
+//  sequence    | uint32 | [0, 4 095]     |   12   | tag 1B + varint 2B = 3B
+//  ---------------------------------------------------------------------------
+//  Worst-case: 15 B  (vs. 20 B raw: 3×float + 2×uint32)
+//  Step: move_x/move_z ≈ 0.008; yaw ≈ 0.088°
+// ---------------------------------------------------------------------------
+message PlayerInput {
+  // Normalised joystick axes in [-1, 1].
+  uint32 move_x   = 1 [(decentraland.common.quantized)  = { min: -1.0,   max: 1.0,   bits:  8 }];
+  uint32 move_z   = 2 [(decentraland.common.quantized)  = { min: -1.0,   max: 1.0,   bits:  8 }];
+
+  // Horizontal look direction in degrees.
+  uint32 yaw      = 3 [(decentraland.common.quantized)  = { min: -180.0, max: 180.0, bits: 12 }];
+
+  // Bitmask of active buttons (see ButtonFlags below).
+  uint32 buttons  = 4 [(decentraland.common.bit_packed) = { bits:  8 }];
+
+  // Rolling input sequence number used by the server for reconciliation.
+  uint32 sequence = 5 [(decentraland.common.bit_packed) = { bits: 12 }];
+}
+
+// Bitmask values for PlayerInput.buttons.
+enum ButtonFlags {
+  BF_NONE      = 0;
+  BF_JUMP      = 1;   // bit 0
+  BF_SPRINT    = 2;   // bit 1
+  BF_INTERACT  = 4;   // bit 2
+  BF_EMOTE     = 8;   // bit 3
+  BF_CROUCH    = 16;  // bit 4
+  // bits 5-7 reserved
+}
+
+// ---------------------------------------------------------------------------
+// AvatarStateSnapshot — full authoritative state, sent on Channel 0 (reliable)
+// or on resync requests.  Demonstrates wider ranges and mixed encodings.
+//
+//  Field            | Type   | Range            | Q bits | Wire worst-case
+//  -----------------|--------|------------------|--------|-----------------------
+//  x                | uint32 | [-4096, 4096]    |   16   | tag 1B + varint 3B = 4B
+//  y                | uint32 | [-256,   256]    |   14   | tag 1B + varint 2B = 3B
+//  z                | uint32 | [-4096, 4096]    |   16   | tag 1B + varint 3B = 4B
+//  pitch            | uint32 | [-90,    90]     |   10   | tag 1B + varint 2B = 3B
+//  yaw              | uint32 | [-180,  180]     |   12   | tag 1B + varint 2B = 3B
+//  entity_id        | uint32 | [0, 1 048 575]   |   20   | tag 1B + varint 3B = 4B
+//  animation_state  | uint32 | [0,       63]    |    6   | tag 1B + varint 1B = 2B
+//  is_grounded      | bool   | —                |   —    | tag 1B + varint 1B = 2B
+//  timestamp        | double | —                |   —    | tag 1B + fixed64  8B = 9B
+//  ---------------------------------------------------------------------------
+//  Worst-case: 34 B  (vs. 45 B raw: 5×float + 2×uint32 + bool + double)
+//  Step: x/z ≈ 0.125 units; y ≈ 0.031 units; pitch ≈ 0.176°; yaw ≈ 0.088°
+// ---------------------------------------------------------------------------
+message AvatarStateSnapshot {
+  // World-space position.
+  uint32 x               = 1 [(decentraland.common.quantized)  = { min: -4096.0, max: 4096.0, bits: 16 }];
+  uint32 y               = 2 [(decentraland.common.quantized)  = { min:  -256.0, max:  256.0, bits: 14 }];
+  uint32 z               = 3 [(decentraland.common.quantized)  = { min: -4096.0, max: 4096.0, bits: 16 }];
+
+  // View angles.
+  uint32 pitch           = 4 [(decentraland.common.quantized)  = { min:  -90.0,  max:   90.0, bits: 10 }];
+  uint32 yaw             = 5 [(decentraland.common.quantized)  = { min: -180.0,  max:  180.0, bits: 12 }];
+
+  // Identity and animation state.
+  uint32 entity_id       = 6 [(decentraland.common.bit_packed) = { bits: 20 }];
+  uint32 animation_state = 7 [(decentraland.common.bit_packed) = { bits:  6 }];
+
+  // Un-annotated fields — encoded at their natural width.
+  bool   is_grounded     = 8;
+  double timestamp       = 9;   // server epoch milliseconds
+}