feat(java): long array serializer support varint encoding

[Pigsy-Monk]

What does this PR do?

This PR adds variable-length encoding serializers for long[] arrays in Java, which provides more space-efficient serialization for arrays containing many small values.

Changes:

• Enhance LongArraySerializer with variable-length encoding support: Added supportVarLenEncoding parameter to LongArraySerializer constructor, allowing it to optionally use variable-length encoding when enabled.

• Add comprehensive test cases:

  • testVariableLengthLongArray(): Tests serialization/deserialization of long arrays with various value ranges (empty, small, mixed, large, negative values)
  • testVariableLengthEncodingEfficiencyForSmallValues(): Demonstrates that variable-length encoding produces significantly smaller serialized data (50%+ reduction) for arrays containing many small values

Test Details:

• testVariableLengthLongArray:

  • Tests empty arrays
  • Tests arrays with small values (0-255)
  • Tests arrays with mixed small and large values (including Long.MAX_VALUE and Long.MIN_VALUE)
  • Tests arrays with negative values
  • Tests large arrays (1000 elements) with many small values

• testVariableLengthEncodingEfficiencyForSmallValues:

  • Compares serialization size between fixed-length encoding (8 bytes per long element) and variable-length encoding (1-2 bytes per small element)
  • Tests with arrays containing values 0-127 (optimal for variable-length encoding)
  • Tests with arrays containing values 0-1023 (still benefits from variable-length encoding)
  • Verifies at least 50% size reduction for small values
  • Outputs detailed efficiency metrics (bytes, bytes per element, percentage reduction)

Performance Benefits:

For arrays containing many small values:

• Fixed-length encoding: 8 bytes per long element + overhead
• Variable-length encoding: 1-2 bytes per small long element + overhead
• Space savings: Up to 75%+ reduction for arrays with values in the 0-127 range

Related Issues:

• Addresses the need for more efficient serialization of primitive arrays with small values
• Enables space-optimized serialization for use cases like sparse arrays, indices, counters, etc.

Does this PR introduce any user-facing change?

No, this PR only adds new serializer classes and test cases. The default serializers remain unchanged. Users can opt-in to variable-length encoding by using the enhanced LongArraySerializer with supportVarLenEncoding=true.

[chaokunyang]

@Pigsy-Monk We have org.apache.fory.serializer.CompressedArraySerializers, could you use this serializer instead?

[Pigsy-Monk]

Hi @chaokunyang , I have researched the CompressedLongArraySerializer algorithm. My understanding is that it assumes all elements in the array are compressible, and that the achievable compression ratio is limited by the largest element in the array. In my use case, the majority of the long values are small, with only a small fraction being large.
In addition, CompressedLongArraySerializer is recommended for compressing large arrays (size > 512), whereas in my use case there are many small arrays, often well below that size.

[chaokunyang]

@Pigsy-Monk Could you fix code styel errors?

[Pigsy-Monk]

Sure I am trying to figure out where it went wrong.

[chaokunyang]

ForyConfig has compressIntArray and compressLongArray options, how about use that directly, and for LongArraySerializer, we compress based on LongEncoding

[Pigsy-Monk]

Sounds better. I will submit a commit.

[chaokunyang]

could you switch on LongEncoding to invoke different functions to use different comrpession algorithms?

[Pigsy-Monk]

Yes I am working on it.

[chaokunyang]

We can have two functions to move the switch on LongEncoding outside the loop

[Pigsy-Monk]

private void writeInt64s(MemoryBuffer buffer, long[] value, LongEncoding longEncoding) {
  int length = value.length;
  buffer.writeVarUint32Small7(length);
  
  if(longEncoding == LongEncoding.SLI){
    for (int i = 0; i < length; i++) {
      buffer.writeSliInt64(value[i]);
    }
    return;
  }      
  for (int i = 0; i < length; i++) {
    buffer.writeVarInt64(value[i]);
  }
}

[Pigsy-Monk]

I am not sure if I get you right. Do you mean something like this?

[chaokunyang]

LGTM, would you like to open another PR to add varint compression to int[] array?

[Pigsy-Monk]

Yes that's what i plan to do after merge this one.

[chaokunyang]

@Pigsy-Monk Please fix checkstyle error

[Pigsy-Monk]

Sure, I am trying to figure out where it went wrong.

[Pigsy-Monk]

It would been easier if the error message indicates the line number.

[Pigsy-Monk]

@chaokunyang Could you please tell me which plugin you use to check the code style?

[chaokunyang]

@chaokunyang Could you please tell me which plugin you use to check the code style?

You can just run ci/format.sh --java

And see maven pom.xml for detailed plugin we used. We use spotless For code format

[Pigsy-Monk]

Thanks. I run ci/format.sh --java. Hope it work this time.

[Pigsy-Monk]

Hope it works this time.