banister / method_source Goto Github PK

Hi @banister

When I talked with the author of bacon by email, he told me that Bacon development had halted.
https://github.com/chneukirchen/bacon/issues

I want to replace bacon that is used for method_source to RSpec 3 or Minitest.

How do you think?

I have a Rails scope:

  scope :positive_amount,                    -> { where("amount > 0") }

But when I do:

Transaction.method(:positive_amount).source.display

I get:

            singleton_class.send(:define_method, name) do |*args|
              scope = all
              scope = scope._exec_scope(*args, &body)
              scope = scope.extending(extension) if extension
              scope
            end

Not the -> { where("amount > 0") } that I'd like.

Any ideas?

This is a test case to demonstrate what I mentioned in the title:

#!/usr/bin/env ruby
# test.rb

class TestClass
  source = <<-EOF
  def test_m
    "test_m here"
  end
  EOF

  eval(source)
end

[1] pry(main)> require './test.rb'                                                                                                                                                                                                                                      
=> true
[2] pry(main)> TestClass.new.test_m                                                                                                                                                                                                                                     
=> "test_m here"
[3] pry(main)> m  = TestClass.new.method :test_m                                                                                                                                                                                                                        
=> #<Method: TestClass#test_m>

[4] pry(main)> m.source                                                                                                                                                                                                                                                 
MethodSource::SourceNotFoundError: Could not load source for : No such file or directory @ rb_sysopen - (eval)
from /home/yanying/.rvm/gems/ruby-2.1.5/gems/method_source-0.8.2/lib/method_source.rb:55:in `rescue in lines_for'

[5] pry(main)> m.source_location                                                                                                                                                                                                                                        
=> ["(eval)", 1]
[6] pry(main)>      

> MethodSource.complete_expression?('$\\')
 => false 

$\ is a global variable however, and thus it shouldn't be determined as incomplete

Extracted source continues to the next expression on the same line

def foo; end; def bar
  "baz"
end

method(:foo).source.display

output:

def foo; end; def bar
  "baz"
end

I expected:

def foo; end; def bar

Is this bug or feature?

We should revert #51 once JRuby releases the fix for
jruby/jruby#5262 that is currently unreleased by on their master.

The code there is temporary, to let users enjoy JRuby 9.2.0.0 with Pry. Once JRuby 9.2.0.1 or whatnot is released, the patch won't be needed anymore, so it's safe to delete it.

Hi, it seems method_source v0.9.1 is available.
https://rubygems.org/gems/method_source/versions/0.9.1

However, there is no tags for v0.9.1 on GitHub. Could you push it?

see here: https://gist.github.com/748033

> MethodSource.complete_expression?('%\a\\')
 => false

%\a\ is a valid string however and thus it should be determined as complete.

At the moment we have to specify the gem for this (by way of 'pry') by pointing to master in our project's Gemfile:

gem 'method_source', git: 'https://github.com/banister/method_source', branch: 'master'

I know it has been a while, but could you please make a new release to Rubygems?

The "method_source" gem seems not to have a license at all. Unless a license that specifies otherwise is included, nobody else can use, copy, distribute, or modify that library without being at risk of take-downs, shake-downs, or litigation.

I know, that this gem has a license on github, however it's missing one at rubygems and in a gemspec.

Running the "rails generate" command when pry-rails is installed leads to the error "undefined method `create_command' for #Pry::CommandSet:0x00000000057ccb08". The "rails generate" command works if pry-rails is not installed.

The pry-rails gem was last updated on December 30, 2018. The pry gem (dependency of pry-rails) was last updated on November 12, 2018. The coderay gem (dependency of pry) was last updated on September 3, 2017.

The Gemfile.lock file shows that version 1.0.0 (latest one) of the method_source gem is in use. This version of the gem was published today, March 19, 2020.

e.g

if meth.owner == Symbol then :test.method(meth).source_location

Related to #22

For blocks, source_location reports the line where the block opens as its line number. If the block is part of a multiline expression, this can be partway through the expression.

Maybe expression_at needs to start looking backwards (at line numbers prior to the provided one) if the first line has a syntax error?

class A
  attr_reader :blk

  def initialize(_stuff, &blk)
    @blk = blk
  end
end

a = A.new("a" => 1, "b" => 2) do
  "Block!"
end

b = A.new("a" => 1,
          "b" => 2) do # line 14
  "Block!"
end

puts a.blk.source #=> works
puts b.blk.source_location #=> a.rb 14
puts b.blk.source #=> MethodSource::SourceNotFoundError

MethodSource::SourceNotFoundError: Could not parse source for #<Proc:0x00007f85ebcb34a0 ~/a.rb:14>: (eval):2: syntax error, unexpected =>, expecting end-of-input
          "b" => 2) do
              ^~

from ~/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/method_source-1.0.0/lib/method_source.rb:29:in `rescue in source_helper'
Caused by SyntaxError: (eval):2: syntax error, unexpected =>, expecting end-of-input
          "b" => 2) do
              ^~

from ~/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:71:in `eval'

The gem metadata on rubygems.org doesn't set the source code URI, so there's no easy way to jump from the gem to its source repo. I had to search on GitHub and then compare the list of release tags to the set of versions on rubygems.org to verify I found the correct repo.

If someone with permissions could add the value to rubygems.org, that would be greatly appreciated.

> MethodSource.complete_expression?('x = :"\xC3"')
 => EncodingError: invalid encoding symbol

Although this code string has invalid utf-8 chars, it is complete, and thus true should be returned.

I had low expectations of method_source helping me, so I might be way out of bounds here.

I'm writing a little gem called arg-that which provides a method that takes a block. To give better test messages, I was hoping to use method_source (or similar) to print the content of the arg_that blocks.

Here's a little example:

    expect(
      :a => 1,
      :b => 99
    ).to eqish(
      :a => 1,
      :b => arg_that {|arg| arg > 98 && arg < 100 }
    )

In the above, calling #inspect on the block passed to arg_that will raise the following:

Failure/Error: Unable to find matching line from backtrace
     MethodSource::SourceNotFoundError:
       Could not parse source for #<Proc:0x007ffc8bc73898@/Volumes/Macintosh HD/Users/justin/code/vagrants/oss/projects/ruby/arg_that/spec/arg_that_spec.rb:42>: (eval):2: syntax error, unexpected tASSOC, expecting $end
                 :b => arg_that {|arg| arg > 98 && arg < 100 }
                      ^
     # ./lib/arg_that/that_arg.rb:13:in `inspect'
     # ./lib/arg_that/eqish.rb:19:in `block (2 levels) in <top (required)>'

Because

I wonder, does method_source need to filter SyntaxError messages or could it just keep adding inputs until no more SyntaxError (or EOF)?

Ref: ruby/prism#2734

truffleruby-head works fine but I was not able to add it to CI by myself

Issue

Ran into this issue while writing parameterized tests that include an optional block argument. Simplest implementation of the problem:

require "method_source"

a = [
  {block: ->(e) { e << %i[a c] }},
]

puts a.first[:block].source

Output:

/Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source.rb:29:in `rescue in source_helper': Could not parse source for #<Proc:0x0000000100999ef8 ./source.rb:4 (lambda)>: (eval):2: syntax error, unexpected ',', expecting end-of-input (MethodSource::SourceNotFoundError)
...block: ->(e) { e << %i[a c] }},
...                              ^
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source.rb:23:in `source_helper'
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source.rb:110:in `source'
	from ./source.rb:7:in `<main>'
/Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:71:in `eval': (eval):2: syntax error, unexpected ',', expecting end-of-input (SyntaxError)
...block: ->(e) { e << %i[a c] }},
...                              ^
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:71:in `block in complete_expression?'
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:70:in `catch'
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:70:in `complete_expression?'
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:97:in `block in extract_first_expression'
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:95:in `each'
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:95:in `extract_first_expression'
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:30:in `expression_at'
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source.rb:27:in `source_helper'
	from /Users/me/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/method_source-1.0.0/lib/method_source.rb:110:in `source'
	from ./source.rb:7:in `<main>'

Possible fix

I recognize this is a major change for the gem, and may not be possible due to the variety of ruby implementations this supports, but the parser gem does an excellent job turning any source file into an AST. The AST can easily be searched for proc/lambda/def calls, and the line numbers and source are easily accessible from any matching node.

This is the workaround I'm using for now to get Proc#source working for my case. I can adapt this and create a pull request for method_source if the maintainers feel that changing the parser is a good idea.

require "dry/core/cache"
require "parser/current"

module ProcSource
  extend Dry::Core::Cache
  extend AST::Sexp

  class SourceNotFound < StandardError; end

  # parse a ruby source file and return the AST; result is cached
  def self.parse(path)
    fetch_or_store(path) do
      source_buffer = Parser::Source::Buffer.new(path).read
      parser = Parser::CurrentRuby.new
      parser.diagnostics.all_errors_are_fatal = true
      parser.diagnostics.ignore_warnings      = true
      parser.parse(source_buffer)
    end
  end

  PROC_NODES = [
    s(:send, nil, :lambda),
    s(:send, nil, :proc),
    s(:send, s(:const, nil, :Proc), :new),
  ]

  module Helpers
    def source
      file, line = source_location
      root = ProcSource.parse(file)

      queue = [root]
      until queue.empty?
        node = queue.shift
        next unless node.is_a?(Parser::AST::Node)
        queue.unshift(*node.children)

        next unless node.type == :block
        next unless node.loc.line == line

        # verify the first child is a send node
        ch = node.children.first
        next unless ch.is_a?(Parser::AST::Node)
        next unless ch.type == :send

        # verify we're calling lambda, proc, or Proc.new
        next unless ProcSource::PROC_NODES.include?(ch)

        return node.loc.expression.source
      end

      raise SourceNotFound, "unable to find source for %p" % self
    end
  end

  Proc.prepend(Helpers)
end

Hit a case where I need the Proc source without the enclosure. For example:

p = Proc.new do
  puts 'proc'
end

p.source currently gives me all three lines, but I really want:

puts 'proc'

There doesn't appear to be a way to do this. Possible to add?

To reproduce:

require 'rubygems'
require 'method_source'

block     = proc { puts "Hello, world!" }
original = block.source

File.open('/tmp/foo.txt', 'a') { |f| f.write(original) }

buf     = File.read '/tmp/foo.txt'
block = eval(buf, binding, __FILE__, __LINE__)

block.source

If you change the mode from a to w, it works fine.
It looks like method_source loops past the end of the file (valid_expression? never returns true).

Proc#to_s contains all info necessary for a Proc#source_location implementation on ruby 1.8

pry(#<Bacon::Context:0x1015b4738>)> proc {}
=> #<Proc:0x0000000000000000@(pry):2>

Note the portion after the @

[40] pry(main)> ActiveRecord::Base.method(:before_save).source
EOFError: end of file reached
from /Users/reg/.rvm/gems/ruby-1.9.3-p125@pry_sbox/gems/method_source-0.7.1/lib/method_source.rb:44:in `readline'

/Users/reg/.rvm/gems/ruby-1.9.3-p125@pry_sbox/gems/activemodel-3.2.3/lib/active_model/callbacks.rb: 109

    def _define_before_model_callback(klass, callback) #:nodoc:
      klass.class_eval <<-CALLBACK, __FILE__, __LINE__ + 1
        def self.before_#{callback}(*args, &block)
          set_callback(:#{callback}, :before, *args, &block)
        end
      CALLBACK
    end

As you can see below, the # symbol will be interpreted as a comment so valid_expression? would never return true and readline just keeps going

       def self.before_#{callback}(*args, &block)
          set_callback(:#{callback}, :before, *args, &block)
        end

Problem would have been solved if syntax checking is done at the start of klass.class_eval line instead of def self.before_ as suggested by @banister.

I keep encountering this on multiple VMs on an aws a1.xlarge running the openSUSE Leap 15.1 ARM marketplace ami image.

(I am running into this running GitLab)

/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:71: [BUG] Segmentation fault at 0x00000000000000dc
ruby 2.6.6p146 (2020-03-31 revision 67876) [aarch64-linux]

-- Control frame information -----------------------------------------------
c:0063 p:---- s:0355 e:000354 CFUNC  :eval
c:0062 p:0021 s:0350 e:000349 BLOCK  /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:71 [FINISH]
c:0061 p:---- s:0347 e:000346 CFUNC  :catch
c:0060 p:0014 s:0342 e:000341 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:70
c:0059 p:0030 s:0336 e:000335 BLOCK  /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:97 [FINISH]
c:0058 p:---- s:0332 e:000331 CFUNC  :each
c:0057 p:0039 s:0328 e:000327 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:95
c:0056 p:0077 s:0320 e:000319 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source/code_helpers.rb:30
c:0055 p:0060 s:0310 e:000309 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source.rb:27
c:0054 p:0031 s:0301 e:000300 METHOD /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/method_source-1.0.0/lib/method_source.rb:110
c:0053 p:0051 s:0297 e:000296 BLOCK  /opt/gitlab/embedded/service/gitlab-rails/config/initializers/zz_metrics.rb:181

Called from the GitLab code here: https://gitlab.com/gitlab-org/gitlab/-/blob/718eaa448d40a69781415335ca4275578d5585e2/config/initializers/zz_metrics.rb#L181

GitLab bug tracking the issue is here: https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5645

Would you push tag v0.8.2?

As far as I can tell there is no license information for this code. Could you please add a license?

Given a line like this:

Then { expect(:foo).to eqish arg_that {|arg| arg.kind_of?(String) } }

Asking the block passed to arg_that its source will return a string of the entire line, not only what was passed to the block.

I imagine this is just a limitation of what #source_location tells you (e.g. no column information). Is there any other way to get at this? Would an AST approach perhaps work?

In Ruby 1.9.3, there is no '.' in file search path, therefore the test suite needs to be executed by:

$ bacon -I. test/test.rb

But I consider the '-I.' superfluous and the correct approach should be to use the File.expand_path to obtain full path to require paths, e.g. the following command will do the change:

$ sed -i 's|direc = File.dirname(__FILE__)|direc = File.expand_path(File.dirname(__FILE__))|' test/test.rb

Thank you for consideration.

strip it