Skip to content

Conversation

@adrianlizarraga
Copy link
Contributor

@adrianlizarraga adrianlizarraga commented Nov 19, 2025

Description

  • Updates the ep_weight_sharing_ctx_gen tool to support specifying a plugin EP configuration (via JSON).
  • Mark the ep_weight_sharing_ctx_gen tool as deprecated and add notification to README that recommends the use the public Python ORT APIs instead.
  • Added an example Python script in the README.
  • Added a Python unit test that tests compiling models with weight sharing using an example plugin EP.

Tool usage

Create a JSON file that contains information about the plugin EP to load/use (e.g., example_plugin_ep_config.json):

{
    "ep_library_registration_name": "example_plugin_ep",
    "ep_library_path": "example_plugin_ep.dll",
    "selected_ep_name": "example_plugin_ep",
    "default_ep_options": { "option_key": "option_value" }
}

Call the ep_weight_sharing_ctx_gen tool with the -p command-line option to specify the location of the above configuration file:

$ ep_weight_sharing_ctx_gen.exe -p example_plugin_ep_config.json model_1.onnx,model_2.onnx

Motivation and Context

Close the functionality gap between traditional provider-bridge EPs and plugin EPs. This PR allows using plugin EPs with the tool that compiles models with weight sharing.

# ONNXRuntime EP Context Model Generation with Weight Sharing

> [!NOTE]
> This tool is deprecated. Please use the public ONNX Runtime Python APIs to compile models with resource sharing. Refer to the example Python script at the end of this document.
Copy link
Contributor Author

@adrianlizarraga adrianlizarraga Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking we remove this tool after ORT 1.24. At that point we can consider replacing it with a Python version if we deem it necessary (although a simple example script in the docs is probably enough).

Ort::Env env(logging_level, "ep_weight_sharing");

ORT_TRY {
std::optional<PluginEpLibraryRegistrationHandle> plugin_ep_library_registration_handle = std::nullopt;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: optional<unique_ptr> feels unnecessary. can we just use an empty unique_ptr to represent the empty state?

Test compiling two example models using weight sharing (via example plugin EP)
"""
if sys.platform != "win32":
self.skipTest("Skipping test because device discovery is only supported on Windows")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there should be some non-Windows support for device discovery now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants