Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .rubocop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,11 @@ Metrics/BlockNesting:
Max: 2

Metrics/ClassLength:
Max: 320
Max: 330

Metrics/ModuleLength:
Description: Avoid modules longer than 100 lines of code.
Max: 420
Max: 440
Exclude:
- "lib/simplecov.rb"

Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ Unreleased
* `SimpleCov::Result.new` is roughly 7× faster for already-string-keyed input (the `SimpleCov.collate` hot path). The previous implementation deep-cloned each file's coverage data with `JSON.parse(JSON.dump(coverage))` per source file — a useful normalization for live `Coverage.result` symbol keys, but pure overhead for resultsets loaded from disk that already have string keys. `Result` now stringifies the outer hash keys with `transform_keys` only when needed; the inner branch/method-key shape is already handled by `SourceFile#restore_ruby_data_structure`. See #916.

## Bugfixes
* Added `:eval_generated` tokens to `SimpleCov.ignore_branches` and the new `SimpleCov.ignore_methods` so projects using macros like Rails' `delegate` (or any pattern that calls `module_eval(body, __FILE__, __LINE__)`) can drop the synthetic branch and method entries those macros inject. Ruby's `Coverage` attributes eval'd code to the caller's `__FILE__` / `__LINE__`, so a `delegate :foo, to: :bar` line surfaces as if it had a `def foo` and an `if` branch right there. Detection uses Prism to walk the static source and treats any Coverage entry whose start_line lacks a real `def` keyword (for methods) or branch construct (for branches) as eval-generated. Opt in with `ignore_methods :eval_generated` and / or `ignore_branches :eval_generated`. Prism ships with Ruby 3.3+; on older Rubies `gem install prism` enables the filter, otherwise the setting is a no-op. See #1046.
* Files added via `cover` / `track_files` that were never `require`'d during the run now contribute branch and method entries to the report, not just lines. Previously `SimulateCoverage` left those fields as empty hashes (because parsing source ourselves felt risky), which made unloaded files invisible to the branch and method denominators while their lines DID count — so a `cover "{app,lib}/**/*.rb"` glob over files without specs silently inflated branch% relative to line% (the OP's reproduction was via SonarQube, which surfaces the asymmetry more visibly than the SimpleCov HTML report). Branches and methods are now enumerated statically via `SimpleCov::StaticCoverageExtractor`, which uses Prism to walk the AST and emits Coverage-shaped tuples without loading the file. The shape matches what Ruby's own `Coverage` library reports for the same source: `:if` / `:case` / `:while` / `:until` constructs plus their `:then` / `:else` / `:when` / `:in` / `:body` arms, with the synthetic `:else` for case-without-explicit-else that the `ignore_branches :implicit_else` setting (see Enhancements) targets. Prism is bundled with Ruby 3.3+; on older Rubies `gem install prism` enables the fix, otherwise SimulateCoverage falls back to the previous "empty hashes" behavior. See #1059.
* HTML report: two groups whose names share an alphanumeric suffix but differ only in a leading non-letter (e.g. `">100LOC"` / `"<10LOC"`, or any pair using different special characters) no longer render into the same DOM container. The JS that built HTML ids from group names stripped every non-letter prefix and then every remaining non-alphanumeric char, so both names sanitized to `"LOC"` and the second group silently replaced the first in the rendered tabs. The new encoding (`"g-" + each-non-id-char-as-hex`) preserves uniqueness across all input shapes. See #1038.
* `SimpleCov::Result` now warns when it drops source files because their absolute paths aren't on the local filesystem, instead of silently producing an empty `0 / 0 (100.00%)` report. The most common trigger is `SimpleCov.collate` invoked from a machine or working directory different from where the individual resultsets were generated — when *every* entry is missing the warning explicitly names that case and points at the issue; when only some are missing the warning is quieter and lists up to five paths with a `(+N more)` suffix. See #980.
Expand Down
33 changes: 29 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -397,10 +397,35 @@ SimpleCov.start do
end
```

`ignore_branches` is variadic — only `:implicit_else` is currently supported, but the
shape leaves room for future synthetic branch types. Calling it before (or without)
`enable_coverage :branch` is harmless: the setting is stored and applies once branch
coverage is enabled. Explicit `else` arms still count.
`ignore_branches` is variadic. `:implicit_else` and `:eval_generated` (described below)
are the currently supported tokens. Calling it before (or without) `enable_coverage :branch`
is harmless: the setting is stored and applies once branch coverage is enabled.
Explicit `else` arms still count.

### Ignoring eval-generated branches and methods

Rails' `delegate` (and other macros that call `module_eval(body, __FILE__, __LINE__)`)
make Ruby's `Coverage` library attribute the eval'd code to the macro's source line.
The result is a `delegate :foo, to: :bar` line that surfaces in the report as if it
had its own `def foo` and an `if` branch — both reported as missed when the delegated
method isn't called from the suite. Drop those synthetic entries:

```ruby
SimpleCov.start do
enable_coverage :branch
enable_coverage :method
ignore_branches :eval_generated
ignore_methods :eval_generated
end
```

`ignore_methods` is variadic; `:eval_generated` is the only currently supported token.
Both filters detect eval-generated entries by walking the static source with
[Prism](https://github.com/ruby/prism) and dropping any Coverage entry whose start
line lacks a real `def` keyword (for methods) or branch construct (for branches).
Prism is bundled with Ruby 3.3+; on older Rubies `gem install prism` enables the
filter, otherwise it's a silent no-op. Real `def`s and branches that share a line
with an eval-generated entry are kept (line-presence is the matcher).

**Is branch coverage strictly better?** No. Branch coverage really only concerns itself with
conditionals - meaning coverage of sequential code is of no interest to it. A file without
Expand Down
79 changes: 65 additions & 14 deletions lib/simplecov/configuration.rb
Original file line number Diff line number Diff line change
Expand Up @@ -807,29 +807,40 @@ def disable_coverage(criterion)
end

# Branch coverage entries that should not count toward the report when
# they appear in the raw `Coverage.result`. The only currently supported
# token is `:implicit_else` — synthetic `else` arms that Ruby's Coverage
# library reports for constructs without a literal `else` keyword (e.g.
# `case/in` without `else`, `case/when` without `else`, `||=`, `&&=`,
# and `if` without `else`). They show up as "missed" branches and
# depress the branch-coverage percentage even though the source has no
# corresponding code to exercise. See #1033.
#
# Variadic — pass one or more tokens. Multiple calls union:
# they appear in the raw `Coverage.result`. Supported tokens:
#
# * `:implicit_else` (see #1033) drops synthetic `else` arms that
# Ruby's Coverage library reports for constructs without a literal
# `else` keyword (`case/in` without `else`, `case/when` without
# `else`, `||=`, `&&=`, `if` without `else`). They show up as
# "missed" branches and depress the branch-coverage percentage
# even though the source has no corresponding code to exercise.
# * `:eval_generated` (see #1046) drops branches whose source range
# does not correspond to a real conditional in the file. Ruby's
# Coverage attributes `module_eval(body, __FILE__, __LINE__)` to
# the calling file/line, so macros like Rails' `delegate` inject
# "missed" entries into otherwise clean source files when
# `enable_coverage :eval` is on. Detection uses Prism to walk the
# real source and treats any Coverage entry whose start_line does
# not coincide with a real branch construct (`if`, `unless`,
# ternary, `case/when`, `case/in`, `while`, `until`) as
# eval-generated.
#
# Variadic. Pass one or more tokens. Multiple calls union:
#
# SimpleCov.start do
# enable_coverage :branch
# ignore_branches :implicit_else
# ignore_branches :implicit_else, :eval_generated
# end
#
# The setting is recorded regardless of whether branch coverage is
# enabled at call time, so call order doesn't matter:
# enabled at call time, so call order doesn't matter.
# `ignore_branches :implicit_else` before `enable_coverage :branch`
# (or vice versa) both apply the filter. If branch coverage is never
# enabled, the stored setting has nothing to filter and produces no
# observable change in the report. Unknown tokens raise
# `SimpleCov::ConfigurationError` immediately to catch typos.
IGNORABLE_BRANCH_TYPES = %i[implicit_else].freeze
IGNORABLE_BRANCH_TYPES = %i[implicit_else eval_generated].freeze

def ignore_branches(*types)
types.each { |type| raise_if_branch_type_unsupported(type) }
Expand All @@ -845,6 +856,38 @@ def ignored_branch?(type)
ignored_branches.include?(type)
end

# Method coverage entries that should not count toward the report
# when they appear in the raw `Coverage.result`. The only currently
# supported token is `:eval_generated` (see #1046), which drops
# method entries whose source position does not correspond to a
# real `def` keyword in the file. Macros that synthesize methods
# via `module_eval` / `class_eval` (Rails' `delegate`, ActiveRecord
# associations, `attr_accessor`-style helpers) inject "missed"
# method entries when `enable_coverage :eval` is on. Detection uses
# Prism to walk the real source and treat any Coverage method
# entry whose start_line does not match a real `def` as
# eval-generated.
#
# Variadic. Same lifecycle as `ignore_branches`: setting is recorded
# regardless of whether method coverage is enabled, applies once
# method coverage is enabled, no observable effect if it never is.
# Unknown tokens raise `SimpleCov::ConfigurationError`.
IGNORABLE_METHOD_TYPES = %i[eval_generated].freeze

def ignore_methods(*types)
types.each { |type| raise_if_method_type_unsupported(type) }
ignored_methods.concat(types).uniq!
ignored_methods
end

def ignored_methods
@ignored_methods ||= []
end

def ignored_method?(type)
ignored_methods.include?(type)
end

def primary_coverage(criterion = nil)
if criterion.nil?
@primary_coverage ||= default_primary_coverage
Expand Down Expand Up @@ -966,8 +1009,16 @@ def raise_if_branch_type_unsupported(type)
return if IGNORABLE_BRANCH_TYPES.member?(type)

raise SimpleCov::ConfigurationError,
"Unsupported branch type #{type.inspect} for `ignore_branches`; " \
"supported values are #{IGNORABLE_BRANCH_TYPES.inspect}"
"Unsupported branch type #{type.inspect} for `ignore_branches`. " \
"Supported values are #{IGNORABLE_BRANCH_TYPES.inspect}"
end

def raise_if_method_type_unsupported(type)
return if IGNORABLE_METHOD_TYPES.member?(type)

raise SimpleCov::ConfigurationError,
"Unsupported method type #{type.inspect} for `ignore_methods`. " \
"Supported values are #{IGNORABLE_METHOD_TYPES.inspect}"
end

def minimum_possible_coverage_exceeded(coverage_option)
Expand Down
59 changes: 58 additions & 1 deletion lib/simplecov/source_file.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
require "ripper"
require "set"
require_relative "directive"
require_relative "static_coverage_extractor"

module SimpleCov
#
Expand Down Expand Up @@ -335,12 +336,36 @@ def build_branches_report
def build_branches
coverage_branch_data = coverage_data["branches"] || {}
branches = coverage_branch_data.flat_map do |condition, coverage_branches|
next [] if eval_generated_condition_to_ignore?(condition)

build_branches_from(condition, coverage_branches)
end

process_skipped_branches(branches)
end

# Detect a Coverage-reported branch condition that originates from
# `eval`/`module_eval`/`class_eval`/`instance_eval` rather than from
# the file's literal source. Coverage attributes such branches to the
# caller's `__FILE__`/`__LINE__`, so a Rails `delegate :foo, to: :bar`
# call surfaces inside the source file as if there were branches at
# the `delegate` line. Prism never sees those branches in the static
# source, so a condition whose start_line isn't in the real-source
# branch set must be eval-generated. Only consulted when the user has
# opted in via `SimpleCov.ignore_branches :eval_generated`. See #1046.
def eval_generated_condition_to_ignore?(condition)
return false unless SimpleCov.ignored_branch?(:eval_generated)

positions = real_source_positions
# simplecov:disable branch — nil branch fires only when Prism is unavailable
return false unless positions

# simplecov:enable branch

_type, _id, start_line, * = restore_ruby_data_structure(condition)
!positions[:branches].include?(start_line)
end

def process_skipped_branches(branches)
chunks = no_cov_chunks + directive_chunks.fetch(:branch)
return branches if chunks.empty?
Expand Down Expand Up @@ -511,14 +536,46 @@ def branch_coverage_statistics
end

def build_methods
methods = coverage_data.fetch("methods", {}).map do |info, hit_count|
methods = coverage_data.fetch("methods", {}).filter_map do |info, hit_count|
info = restore_ruby_data_structure(info)
next if eval_generated_method_to_ignore?(info)

SourceFile::Method.new(self, info, hit_count)
end

process_skipped_methods(methods)
end

# See `eval_generated_condition_to_ignore?` for the rationale. Coverage
# reports an eval'd `def` at the eval caller's line and name, so a
# method whose `(name, start_line)` is absent from the real-source
# `def` set is eval-generated. Only consulted when the user has opted
# in via `SimpleCov.ignore_methods :eval_generated`. See #1046.
def eval_generated_method_to_ignore?(info)
return false unless SimpleCov.ignored_method?(:eval_generated)

positions = real_source_positions
# simplecov:disable branch — nil branch fires only when Prism is unavailable
return false unless positions

# simplecov:enable branch

_class_name, name, start_line, * = info
!positions[:methods].include?([name, start_line])
end

# Memoize the Prism-derived set of real source positions (branches at
# which lines, methods at which (name, line) pairs). Returns nil when
# Prism is unavailable on this Ruby (older than 3.3 without the gem)
# or when parsing fails. A nil return makes both eval_generated
# filters short-circuit to "keep everything" — no false drops when
# we can't see the static source clearly.
def real_source_positions
return @real_source_positions if defined?(@real_source_positions)

@real_source_positions = StaticCoverageExtractor.real_source_positions(src.join)
end

def process_skipped_methods(methods)
method_chunks = directive_chunks.fetch(:method)
return methods if method_chunks.empty?
Expand Down
31 changes: 31 additions & 0 deletions lib/simplecov/static_coverage_extractor.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# frozen_string_literal: true

require "set"

begin
require "prism"
rescue LoadError
Expand Down Expand Up @@ -70,6 +72,35 @@ def call(source)
# simplecov:enable line
end

# Summarize a source file's REAL branch and method positions, for the
# `:eval_generated` filter (SimpleCov.ignore_branches /
# SimpleCov.ignore_methods, #1046). Returns a hash:
#
# {
# branches: Set[start_line, ...], # e.g., [3, 12, 20]
# methods: Set[[name, start_line], ...] # e.g., [[:foo, 7], [:bar, 13]]
# }
#
# Branch matching is start_line-only because Coverage's condition type
# vocabulary (`:if`, `:unless`, `:case`, `:while`, `:until`) does not
# always match Prism's emitted type (the existing visitor reports
# `:if` for `unless` and ternary). Coincidental line-sharing between
# a real branch and an eval-generated one will keep both, which is
# an acceptable false-negative for an opt-in filter. Method matching
# uses (name, start_line) since a method name is unique at any line.
#
# Returns nil when Prism is unavailable or parsing fails, signaling
# callers to keep every Coverage entry (no false drops).
def real_source_positions(source)
extracted = call(source)
return nil unless extracted

{
branches: extracted["branches"].keys.to_set { |tuple| tuple[2] },
methods: extracted["methods"].keys.to_set { |tuple| [tuple[1], tuple[2]] }
}
end

# simplecov:disable branch
# The `else` arm (Prism missing) is unreachable on engines where the
# dogfood report runs; the Visitor class only matters when Prism is
Expand Down
42 changes: 41 additions & 1 deletion spec/configuration_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -842,7 +842,7 @@

it "names the supported tokens in the error message" do
expect { config.ignore_branches :nope }
.to raise_error(SimpleCov::ConfigurationError, /supported values are \[:implicit_else\]/)
.to raise_error(SimpleCov::ConfigurationError, /Supported values are \[:implicit_else, :eval_generated\]/)
end

it "stores the setting even when branch coverage is not enabled" do
Expand All @@ -861,6 +861,46 @@
expect(config.coverage_criteria).to include :branch
expect(config.ignored_branch?(:implicit_else)).to be true
end

it "accepts :eval_generated alongside :implicit_else" do
config.ignore_branches :implicit_else, :eval_generated

expect(config.ignored_branch?(:implicit_else)).to be true
expect(config.ignored_branch?(:eval_generated)).to be true
end
end

describe "#ignore_methods" do
it "starts empty" do
expect(config.ignored_methods).to eq []
end

it "records the requested token" do
config.ignore_methods :eval_generated

expect(config.ignored_methods).to eq [:eval_generated]
expect(config.ignored_method?(:eval_generated)).to be true
end

it "deduplicates across calls" do
config.ignore_methods :eval_generated
config.ignore_methods :eval_generated

expect(config.ignored_methods).to eq [:eval_generated]
end

it "raises on an unknown token" do
expect { config.ignore_methods :nope }
.to raise_error(SimpleCov::ConfigurationError,
/Unsupported method type :nope.*Supported values are \[:eval_generated\]/m)
end

it "stores the setting even when method coverage is not enabled" do
expect(config.coverage_criteria).to contain_exactly :line
config.ignore_methods :eval_generated

expect(config.ignored_method?(:eval_generated)).to be true
end
end

describe "#disable_coverage" do
Expand Down
7 changes: 7 additions & 0 deletions spec/fixtures/eval_generated.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
class EvalHost
def_delegators :receiver, :hello
def initialize(receiver)
receiver ? @receiver = receiver : nil
end
attr_reader :receiver
end
Loading
Loading