Output for contain_exactly/match_array is worse than stock RSpec

[myronmarston]

Given superdiff_contains.rb:

require "bundler/inline"

gemfile do
  source "https://rubygems.org"
  gem "rspec", "3.13.0"
  gem "super_diff", "0.12.1"
end

require "rspec/autorun"
require "super_diff/rspec" if ENV["SUPERDIFF"]

RSpec.describe "A contain_exactly/match_array expectation" do
  it "fails with an extra element" do
    expect(ENV.keys).to contain_exactly("EXTRA", *ENV.keys)
  end

  it "fails with a missing element" do
    expect(ENV.keys).to match_array(ENV.to_h.except("HOME").keys)
  end
end

When I run it without superdiff, I get this output:

$ ruby superdiff_contains_exactly.rb
FF

Failures:

  1) A contain_exactly/match_array expectation fails with an extra element
     Failure/Error: expect(ENV.keys).to contain_exactly("EXTRA", *ENV.keys)

       expected collection contained:  [redacted so as not to leak the ENV vars on my machine]
       actual collection contained:    [redacted so as not to leak the ENV vars on my machine]
       the missing elements were:      ["EXTRA"]
     # superdiff_contains_exactly.rb:14:in `block (2 levels) in <main>'

  2) A contain_exactly/match_array expectation fails with a missing element
     Failure/Error: expect(ENV.keys).to match_array(ENV.to_h.except("HOME").keys)

       expected collection contained:  [redacted so as not to leak the ENV vars on my machine]
       actual collection contained:    [redacted so as not to leak the ENV vars on my machine]
       the extra elements were:        ["HOME"]
     # superdiff_contains_exactly.rb:18:in `block (2 levels) in <main>'

Finished in 0.01349 seconds (files took 0.05948 seconds to load)
2 examples, 2 failures

Failed examples:

rspec superdiff_contains_exactly.rb:13 # A contain_exactly/match_array expectation fails with an extra element
rspec superdiff_contains_exactly.rb:17 # A contain_exactly/match_array expectation fails with a missing element

When I run it with superdiff, I get this output:

 ~/Development/sq-elasticgraph-ruby/tmp/ [myron/make-superdiff-opt-in] SUPERDIFF=1 ruby superdiff_contains_exactly.rb
FF

Failures:

  1) A contain_exactly/match_array expectation fails with an extra element
     Failure/Error: expect(ENV.keys).to contain_exactly("EXTRA", *ENV.keys)

                 Expected [redacted so as not to leak the ENV vars on my machine]
       to contain exactly [redacted so as not to leak the ENV vars on my machine]
     # /Users/myron/.rvm/gems/ruby-3.2.3/gems/super_diff-0.12.1/lib/super_diff/rspec/monkey_patches.rb:43:in `handle_failure'
     # superdiff_contains_exactly.rb:14:in `block (2 levels) in <main>'

  2) A contain_exactly/match_array expectation fails with a missing element
     Failure/Error: expect(ENV.keys).to match_array(ENV.to_h.except("HOME").keys)

                  Expected [redacted so as not to leak the ENV vars on my machine]
       to match array with [redacted so as not to leak the ENV vars on my machine]
     # /Users/myron/.rvm/gems/ruby-3.2.3/gems/super_diff-0.12.1/lib/super_diff/rspec/monkey_patches.rb:43:in `handle_failure'
     # superdiff_contains_exactly.rb:18:in `block (2 levels) in <main>'

Finished in 0.03072 seconds (files took 0.0479 seconds to load)
2 examples, 2 failures

Failed examples:

rspec superdiff_contains_exactly.rb:13 # A contain_exactly/match_array expectation fails with an extra element
rspec superdiff_contains_exactly.rb:17 # A contain_exactly/match_array expectation fails with a missing element

(...but with the redacted bits not redacted...).

The stock RSpec output makes it very easy to see what the extra elements or missing elements are. With superdiff it's super hard to tell.

Could superdiff retain the the missing elements were: and the extra elements were: parts of the standard RSpec output?

[jas14]

Hey @myronmarston , thanks for the detailed report + repro case.

I believe this is due to a change to the RSpec (private) API in 3.13.0; see #250. To confirm, I just tried this out myself with RSpec 3.12.0 and I got the diff output below. The good news is that a fix (#258) is already up; I just need to get it back up to date and review it, at which point I'll cut another version.

  1) A contain_exactly/match_array expectation fails with an extra element
     Failure/Error: expect(['a', 'b']).to contain_exactly('a', 'b', 'EXTRA')

       Expected ["a", "b"] to contain exactly "a", "b" and "EXTRA".
       Diff:

         [
           "a",
           "b",
       -   "EXTRA"
         ]
     # 253.rb:14:in `block (2 levels) in <main>'

  2) A contain_exactly/match_array expectation fails with a missing element
     Failure/Error: expect(['a', 'b', 'MISSING']).to contain_exactly('a', 'b')

       Expected ["a", "b", "MISSING"] to contain exactly "a" and "b".
       Diff:

         [
           "a",
           "b",
       +   "MISSING"
         ]
     # 253.rb:18:in `block (2 levels) in <main>'