feat(retriever): Add bm25 strategy and fix related init/retrieve bugs #17
+183
−90
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
examples/run_lightmem_bm25.py:
bm25strategy end-to-end.src/lightmem/configs/base.py:
"bm25"to theLiteraltypes forindex_strategyandretrieve_strategyto allow the new config.src/lightmem/configs/memory_manager/base.py:
ValidationErrorby changingconfigs: Optional[dict]toOptional[Any]to match the validator's logic.src/lightmem/configs/retriever/bm25.py:
ValidationErrorby adding the requiredcollection_nameandpathfields toBM25Config.src/lightmem/memory/lightmem.py:
retrievemethod, which was hardcoded for "embedding", with a new version that correctly checks theretrieve_strategyand calls thecontext_retrieverfor "bm25".offline_updateto correctly callindex()whenindex_strategy="bm25".AttributeErrors in__init__related tocompressor,model_name, andtext_embedderto allow non-embedding strategies to initialize correctly.