v0.1.1

BC-breaking changes

• Replace run_pipeline_in_subprocess impl by @moto-meta in #780

  Previously run_pipeline_in_subprocess returned an iterator and was not reusable.
  Now it returns an iterable, which can be iterated multiple times.
  Following this change, the object returned from run_pipeline_in_subprocess does not provide iterator-specific method such as next

  Example

  # Construct a builder
  builder = (
      spdl.pipeline.PipelineBuilder()
      .add_source(...)
      .pipe(...)
      ...
      .add_sink(...)
  )
  
  # Move it to the subprocess, build the Pipeline
  iterable = run_pipeline_in_subprocess(builder, ...)
  
  # Iterate - epoch 0
  for item in iterable:
      ...
  
  # Iterate - epoch 1
  for item in iterable:
      ...

What's Changed

• Support multiple initializers in iterate_in_subprocess and run_pipeline_insubprocess by @mthrok in #783
  Now you can pass multiple initializer functions to iterate_in_subprocess and run_pipeline_insubprocess which are called in the subprocess before the first iteration starts.
• Allow Zero Weights in MergeIterator by @vbourgin in #784
  Previously, the MergeIterator did not allow 0 value as weights. Now this is relaxed and you can pass 0.
  The iterators with 0 weight are not iterated.
• Shuffle Before Iterating by Default for embed_shuffle by @vbourgin in #785
  Previously, embed_shuffle defaulted to shuffle after each iteration, but this was counter-intuitive.
  Now the default behavior is to shuffle before each iteration.
• Adhere to PEP 561 (Fixes a mypy bug) by @alxmrs in #790
  Now SPDL package contain py.typed file so that type-checkers can analyze the code.
• Ensure core bindings is loaded when loading CUDA binding by @mthrok in #792
  When loading CUDA extension in spdl.io module, it ensures that the CPU extension is loaded.
• Support batch loading images with NVJPEG by @mthrok in #794
  The spdl.io.decode_image_nvjpeg function supports loading multiple images. (Note that the function is still experimental.)
• Enable compilation warnings by @mthrok in #806, #821
  The C++ code of spdl.io is hardened by turning few selected compiler warnings into error.
• Fix nanobind option for archive module by @mthrok in #814
  The extension module for archive (zip) parsing was not compiled with free-threading support.
• Mypy type-checking feedback for all "pyre safe" sources. by @alxmrs in #801
  Now CI mypy type checking. The mypy compatibility is not enforced at the moment as pyre has been (and still is) used. We plan to gradually make the codebase compatible with mypy.

New Contributors

• @alxmrs made their first contribution in #790

Full Changelog: v0.1.0...v0.1.1