sometimes method fails to be mocked

[akostadinov]

There is a really weird issue trying to stub a particular method. It can be reproduced by running ruby test.rb

And test.rb is:

# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  gem "mocha", "2.1.0"
  gem "activesupport", "6.0.6.1"
end

require "active_support"
require "minitest/autorun"
require "mocha/minitest"

def timeout
end

class BugTest < Minitest::Test
  def test_stuff
    config = ActiveSupport::OrderedOptions.new.tap do |config|
      config.merge!(
        timeout: 4,
      )
    end

    assert_equal 4, config.timeout

    config.stubs(:timeout)

    assert_nil config.timeout
  end
end

I would expect this test to pass. But in fact it fails with Expected 4 to be nil.

[floehopper]

@akostadinov

Thanks for reporting this - it does seem very odd. I suspect the problem is related to ActiveSupport::OrderedOptions using #method_missing to implement the #timeout method in your example.

I haven't had a lot of time, but I have confirmed that whether or not the top-level #timeout method is defined, Mocha is correctly prepending a module to the instance of ActiveSupport::OrderedOptions and defining a #timeout method on that module. However, for some reason when the top-level #timeout method is defined it looks like the ActiveSupport::OrderedOptions#method_missing implementation is intercepting the invocation before or instead of the Mocha-defined stub method. At the moment I have no idea why that's happening. I'll try to do some more digging when I have time.

Out of interest, in your real scenario can you stub the method that returns the ActiveSupport::OrderedOptions instance and substitute in a modified version of the options instance?

[floehopper]

@akostadinov

@tomstuart has pointed out to me that methods defined on the top-level main object become private instance methods of Object:

def foobar = 42 

Object.private_instance_methods.grep(/foo/) # => [:foobar]

Object.new.send(:foobar) # => 42

This is almost certainly the root cause of the problem, but I'll need to spend some more time digging into it...

[akostadinov]

wrt original use case, it was easy enough to just change the config instead of stubbing. Stubbing it is a little bit more lazy so probably that's why original author of the test used this approach. So no issue with that.

So you are right, I noticed the #timeout method was private. I didn't check OrderedOptions though. A simpler reproducer without active_support is here:

# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  gem "minitest"
  gem "mocha", "2.1.0"
end

require "minitest/autorun"
require "mocha/minitest"

def timeout
end

class UsingMethodMissing
  def method_missing(name, *args)
    name == :timeout ? name : super
  end
end

class BugTest < Minitest::Test
  def test_stuff
    config = UsingMethodMissing.new

    assert_equal :timeout, config.timeout

    config.stubs(:timeout)

    assert_nil config.timeout
  end
end

[akostadinov]

Probably you already know that, but mocha 1.x series didn't exhibit this problem. I've hit it on upgrade.

[floehopper]

Probably you already know that, but mocha 1.x series didn't exhibit this problem. I've hit it on upgrade.

That's useful to know - thanks.

[floehopper]

I've got a bit further with this...

Firstly, it's important to realise that before Mocha gets involved, the config object in your last example effectively has two implementations of #timeout: one implemented via #method_missing on UsingMethodMissing and the other private one implemented on Object. At this point, if you call config.timeout you'll get the return value from #method_missing, i.e. :timeout in your last example. But if you call config.send(:timeout) you'll get the return value from the private method on Object, i.e. nil in your last example. This already seems pretty confusing to me, but it doesn't take away from the fact that Mocha is doing something surprising/incorrect...

It seems as if Mocha is successfully stubbing the method, but because it sees the private method on Object, it gives the stub the same visibility. Thus the object effectively ends up with three #timeout methods: the two mentioned previously, but also the stub method defined on a prepended module. Importantly the latter also has private visibility, which means that calling config.timeout still returns the result of the #method_missing implementation; whereas calling config.send(:timeout) returns the stubbed value, nil. Note that this is not the result of the #timeout implementation on Object which confusingly would also be nil in your last example. To confirm this I think you can change assert_nil config.timeout in your test to assert_nil config.send(:timeout) and it will pass.

The bug in Mocha is that it should be calling #respond_to? and (assuming #respond_to_missing? is correctly defined for UsingMethodMissing) thus determining that there is effectively a public implementation of #timeout. That would then cause the stub to be given public visibility and the stub would then take precedence when you call config.timeout. I'm pretty sure there used to be some code that did this, but it looks like it got lost during some refactoring sometime around the v2 release. The fix isn't going to be completely straightforward, so you'll have to bear with me until I have a decent chunk of time between client projects so that I can work on this.

[floehopper]

I've created a less pathalogical failing acceptance test to demonstrate the problem in https://github.com/freerange/mocha/compare/fix-stubbing-method-implemented-by-method-missing. It's worth noting that a private method in a superclass is enough to cause the problem - no need for the method on the top-level main object.

[akostadinov]

I wonder what the right resolution would be. Current behavior seems that private methods are stubbed as private and public ones are stubbed as public. But in this case mocha doesn't seem to have a good way to know what the intention is.

In my opinion it makes sense in such situation (private method available and at the same time a custom #method_missing, then mocha should raise an error and request usage of a keyword parameter (visibility: :public or visibility: :private). So that the intention is always clear... unless everybody wants to go back to 1.x behavior.