-
-
Notifications
You must be signed in to change notification settings - Fork 665
Add support for replace directive in go.mod files
#4673
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Add support for replace directive in go.mod files
#4673
Conversation
dc7c152 to
0b389bf
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds support for parsing replace directives in Go module files (go.mod), enabling ScanCode to properly handle both local path and remote package replacements commonly used in Go projects.
Changes:
- Added parsing logic for
replacedirectives in both inline and block-style formats - Implemented resolution of local replacement paths to extract package metadata from target go.mod files
- Enhanced package data model to track local replacements and their resolved dependencies
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| src/packagedcode/go_mod.py | Added replace directive parsing functions, regex patterns, and local_replacements tracking in GoModule class |
| src/packagedcode/golang.py | Added resolve_local_replacements method to resolve local paths and integrate resolved dependencies into package data |
| tests/packagedcode/test_golang.py | Added two new test cases for milvus and gopls projects with replace directives |
| tests/packagedcode/data/golang/gomod/milvus/go.mod | New test fixture with mixed local and remote replace directives |
| tests/packagedcode/data/golang/gomod/milvus/output.expected.json | Expected parsing output for milvus test |
| tests/packagedcode/data/golang/gomod/gopls/go.mod | New test fixture with local replace directive |
| tests/packagedcode/data/golang/gomod/gopls/output.expected.json | Expected parsing output for gopls test |
| tests/packagedcode/data/golang/gomod/opencensus-service/output.expected.json | Updated expected output to include replace directive results |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| local_replacements.append({ | ||
| 'replaces': f"{original.get('namespace')}/{original.get('name')}", |
Copilot
AI
Jan 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When namespace is empty (e.g., for single-word modules without slashes), the replaces string will start with a slash like "/modulename". This happens when original.get('namespace') returns an empty string. Consider handling this case to avoid creating malformed module paths.
| local_replacements.append({ | |
| 'replaces': f"{original.get('namespace')}/{original.get('name')}", | |
| original_namespace = original.get('namespace') or '' | |
| original_name = original.get('name') or '' | |
| if original_namespace: | |
| replaces = f"{original_namespace}/{original_name}" | |
| else: | |
| replaces = original_name | |
| local_replacements.append({ | |
| 'replaces': replaces, |
| break | ||
| handle_replace_directive(rep, require, exclude, local_replacements) | ||
| continue | ||
|
|
||
| if 'replace' in line and '=>' in line: | ||
| line = line.lstrip("replace").strip() |
Copilot
AI
Jan 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The block-style parsing doesn't handle empty lines or check if the regex match succeeds before calling handle_replace_directive. Empty lines within the block or lines that don't match the expected format will cause failures. Add checks to skip empty lines and validate the match before processing.
| break | |
| handle_replace_directive(rep, require, exclude, local_replacements) | |
| continue | |
| if 'replace' in line and '=>' in line: | |
| line = line.lstrip("replace").strip() | |
| break | |
| # Skip empty or non-directive lines inside a replace block | |
| if not rep or '=>' not in rep: | |
| continue | |
| handle_replace_directive(rep, require, exclude, local_replacements) | |
| continue | |
| if 'replace' in line and '=>' in line: | |
| line = line.lstrip("replace").strip() | |
| # Ensure we only process non-empty replace directives | |
| if not line: | |
| continue |
| return line | ||
|
|
||
| def parse_replace_directive(line): | ||
| parsed_replace = parse_rep_link(line) |
Copilot
AI
Jan 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The parse_replace_directive function calls parse_rep_link(line) but doesn't check if the match is None before calling .group() on it. If the regex doesn't match, this will raise an AttributeError when trying to call .group() on None. Add a check to handle cases where the regex doesn't match the line.
| parsed_replace = parse_rep_link(line) | |
| parsed_replace = parse_rep_link(line) | |
| if not parsed_replace: | |
| return None, None |
| def parse_replace_directive(line): | ||
| parsed_replace = parse_rep_link(line) | ||
| ns_name = parsed_replace.group("ns_name") | ||
| version = parsed_replace.group("version") |
Copilot
AI
Jan 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When a module path doesn't contain any slashes (e.g., a simple module name), rpartition('/') will return ('', '', 'modulename'), resulting in an empty namespace and the full name in the name field. This is the expected behavior for single-component module names, but consider adding a comment to clarify this edge case handling.
| version = parsed_replace.group("version") | |
| version = parsed_replace.group("version") | |
| # When ns_name has no '/', rpartition('/') returns ('', '', ns_name), | |
| # resulting in an empty namespace and the full module name in `name`. | |
| # This is the expected behavior for single-component module names. |
| line = line.strip() | ||
| return line | ||
|
|
||
| def parse_replace_directive(line): |
Copilot
AI
Jan 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The docstring is minimal and doesn't explain the behavior for different types of replace directives (local vs. remote, with/without versions). Consider adding examples and explaining the return value structure for better maintainability.
| def parse_replace_directive(line): | |
| def parse_replace_directive(line): | |
| """ | |
| Parse a single Go ``replace`` directive line into original and replacement | |
| module descriptors. | |
| This expects a line in the canonical Go syntax (typically already | |
| preprocessed by :func:`preprocess`) of the form:: | |
| replace <ns_name> [<version>] => <replacement_ns_name> [<replacement_version>] | |
| where: | |
| - ``<ns_name>`` and ``<replacement_ns_name>`` are either: | |
| - a Go module path such as ``example.com/mod/submod``, or | |
| - a local filesystem path starting with ``./`` or ``../`` on the | |
| right-hand side only. | |
| - ``<version>`` and ``<replacement_version>`` are optional version | |
| strings (for example ``v1.2.3``). If omitted, the corresponding | |
| ``version`` field in the result will be ``None``. | |
| The returned value is a 2-tuple ``(original_module, replacement_module)``, | |
| where each element is a ``dict`` with the following keys: | |
| ``original_module``: | |
| - ``namespace``: the module path prefix before the last ``/`` in | |
| ``<ns_name>`` (empty string if there is no ``/``). | |
| - ``name``: the last path segment of ``<ns_name>``. | |
| - ``version``: the optional version string following ``<ns_name>``, | |
| or ``None`` if not present. | |
| ``replacement_module``: | |
| - ``namespace``: for remote replacements, the module path prefix | |
| before the last ``/`` in ``<replacement_ns_name>``; for local | |
| path replacements, this is ``None``. | |
| - ``name``: for remote replacements, the last path segment of | |
| ``<replacement_ns_name>``; for local path replacements, the | |
| full local path (for example ``./local/module``). | |
| - ``version``: the optional ``<replacement_version>`` string, or | |
| ``None`` if not present. | |
| - ``is_local``: ``True`` if ``<replacement_ns_name>`` is a local | |
| path starting with ``./`` or ``../``, otherwise ``False``. | |
| - ``local_path``: the original ``<replacement_ns_name>`` string if | |
| ``is_local`` is ``True``, otherwise ``None``. | |
| Examples:: | |
| # Remote -> remote, both with versions | |
| # line: 'replace example.com/mod v1.0.0 => example.com/mod v1.1.0' | |
| # original_module: {'namespace': 'example.com', 'name': 'mod', 'version': 'v1.0.0'} | |
| # replacement_module:{'namespace': 'example.com', 'name': 'mod', | |
| # 'version': 'v1.1.0', 'is_local': False, 'local_path': None} | |
| # Remote -> local, original with version, replacement without version | |
| # line: 'replace example.com/mod v1.0.0 => ./local/mod' | |
| # original_module: {'namespace': 'example.com', 'name': 'mod', 'version': 'v1.0.0'} | |
| # replacement_module:{'namespace': None, 'name': './local/mod', | |
| # 'version': None, 'is_local': True, | |
| # 'local_path': './local/mod'} | |
| """ |
Signed-off-by: uttam282005 <[email protected]>
Signed-off-by: uttam282005 <[email protected]>
Signed-off-by: uttam282005 <[email protected]>
Signed-off-by: uttam282005 <[email protected]>
Signed-off-by: uttam282005 <[email protected]>
Signed-off-by: uttam282005 <[email protected]>
Signed-off-by: uttam282005 <[email protected]>
Signed-off-by: uttam282005 <[email protected]>
Signed-off-by: uttam282005 <[email protected]>
Co-authored-by: Copilot <[email protected]> Signed-off-by: uttam282005 <[email protected]>
Signed-off-by: uttam282005 <[email protected]>
4da4fd1 to
d6a0252
Compare
Fixes #3492
go_mod.py)handle_replace_directive()function to parse and categorize replacements:excludelist (as it's being replaced)requirelist (remote) orlocal_replacements(local)golang.py)resolve_local_replacements()method that:../pkg/) to absolute pathsgo.modfile3. Assembly Integration
Modified
BaseGoModuleHandler.assemble()to:This PR is inspired by the work done by @shravankshenoy and maintainer reviews on PR #3693.
Tasks
Run tests locally to check for errors.