tenderlove / recma Goto Github PK

View Code? Open in Web Editor NEW

194.0 194.0 58.0 471 KB

Pure ruby javascript parser and interpreter.

Ruby 88.76% Yacc 11.24%

recma's People

Contributors

Stargazers

Watchers

recma's Issues

A fork of RKelly

Just writing to let you know that I've created a fork of RKelly: https://github.com/nene/rkelly-remix

I needed a proper support for line/char positions of each node to be able to use it as a replacement for JS parser in JSDuck. As I couldn't get these changes into the main RKelly development line fast enough, I went on the route of forking and releasing my own gem.

I would be most happy to get all the fixes I've made into the main RKelly repository, but until then I'll need to maintain my own.

PS. Thanks for RKelly, it has saved the JSDuck project.

"a = function b() {}" parsed as "a = b() {}"

This:

irb(main):004:0> parser.parse("a = function b() {}").to_ecma()
=> "a = b() {\n\n};"

Parsing it again gives a syntax error.

Typo in visitor.rb

DotAccessor is mistakenly written DotAcessor in ALL_NODES declaration in visitor.rb, causing some concrete visitors to skip DotAccessorNodes.

¡ Keep up the great work !

invalid multibyte escape: /(\A[\s\xA0]|[\s\xA0]\Z)/

0$ gem install rkelly
Successfully installed rkelly-1.0.7
Parsing documentation for rkelly-1.0.7
Done installing documentation for rkelly after 1 seconds
1 gem installed
0$ irb
irb(main):001:0> require 'rkelly'
SyntaxError: /home/user/.gem/ruby/2.0.0/gems/rkelly-1.0.7/lib/rkelly/visitors/evaluation_visitor.rb:318: invalid multibyte escape: /(\A[\s\xA0]*|[\s\xA0]*\Z)/
  from /usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
  from /usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
  from /home/user/.gem/ruby/2.0.0/gems/rkelly-1.0.7/lib/rkelly/visitors.rb:3:in `block in '
  from /home/user/.gem/ruby/2.0.0/gems/rkelly-1.0.7/lib/rkelly/visitors.rb:2:in `each'
  from /home/user/.gem/ruby/2.0.0/gems/rkelly-1.0.7/lib/rkelly/visitors.rb:2:in `'
  from /usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
  from /usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:45:in `require'
  from /home/user/.gem/ruby/2.0.0/gems/rkelly-1.0.7/lib/rkelly.rb:3:in `'
  from /usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:110:in `require'
  from /usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:110:in `rescue in require'
  from /usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:35:in `require'
  from (irb):1
  from /usr/bin/irb:12:in `'
0$ ruby -v
ruby 2.0.0p195 (2013-05-14 revision 40734) [x86_64-linux]
0$ echo $LANG
ru_RU.utf8

Is it possible to convert some Javascript code to Ruby using rkelly (or rkelly-remix)?

This guide showed the basics of how it could be done but is very out-of-date...

http://tenderlovemaking.com/2007/04/15/converting-javascript-to-ruby-with-rkelly.html

Parsing a large file a gets a nil back

I tried to parse Angular.JS uncompressed (500K). The parser.parse method return nil.

I am using JRuby 1.7.3

This returns different results on JRuby vs MRI

RKelly::Parser.new.parse "try { 2 + 2 } catch(ex) { }"

I suspect a bug in the pure ruby racc runtime

for in statemet is some weird...^^;

RKelly::Parser.new.parse("for (var i in a) alert(1);", file_name).to_ecma
->
for( i in a) alert(1);

Unable to parse sea.js

Executing this

require 'open-uri'
RKelly::Parser.new.parse(open('http://seajs.com/build/sea.js') {|f|f.read})

Would give this error

RuntimeError: something bad happened, please report a bug with sample JavaScript
    from [redacted]/gems/rkelly-1.0.3/lib/rkelly/parser.rb:44:in `yyabort'
    from parser.y:708:in `_reduce_264'
    from (eval):4:in `_reduce_264'
    from [redacted]/gems/rkelly-1.0.3/lib/rkelly/tokenizer.rb:64:in `_racc_do_parse_c'
    from [redacted]/.rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/racc/parser.rb:99:in `catch'
    from [redacted]/.rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/racc/parser.rb:99:in `_racc_do_parse_c'
    from [redacted]/.rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/racc/parser.rb:99:in `__send__'
    from [redacted]/.rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/racc/parser.rb:99:in `do_parse'
    from [redacted]/gems/rkelly-1.0.3/lib/rkelly/parser.rb:39:in `parse'
    from (irb):1

Error parsing a function literal

I get erros by parsing this code:

[code]
function() {
return 4
}
[/code]

RuntimeError: Parsing Errors: [""("", "error value: SINGLE_CHAR: (", "error stack: [#<RKelly::Token:0x5f92452e @name=:RAW_IDENT, @line=1, @transformer=#Proc:0x5d65c981@/Users/federico/.rvm/gems/jruby-1.7.3/gems/rkelly-1.0.7/lib/rkelly/tokenizer.rb:83, @value="function">]"].

Can't Require for Ruby 2.2.1

In pry I try require 'rkelly' and get the following:

SyntaxError: /Users/chase/.rvm/gems/ruby-2.2.1/gems/rkelly-1.0.7/lib/rkelly/visitors/evaluation_visitor.rb:318: invalid multibyte escape: /(\A[\s\xA0]*|[\s\xA0]*\Z)/
from /Users/chase/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'

support for "web ecmascript"

Hi,

I'm wondering if rkelly could optionally support stuff like <!-- for single line comments.

Regards,
Henrik

EDIT: There is a list referencing those differences here, the comment peculiarity is documented in the HTML comments section.

Problematic linking of comments to AST nodes

The algorithm for linking comments to AST nodes is very simplistic and the resulting outcome is therefore not very useful.

For example:

// comment A
function foo(a, b) { // comment B
    var x = 1;
}

Here both comments get linked to the FunctionDeclNode. Which looks fine. But when we just add one extra newline:

// comment A
function foo(a, b)
{ // comment B
    var x = 1;
}

The comment B gets linked to VarStatementNode instead. Not too bad. But when we eliminate the var statement:

// comment A
function foo(a, b) { 
    // comment B
}

Now the comment B vanishes altogether. Like it hadn't been in the source code at all. However when we add another function after foo:

// comment A
function foo(a, b) { 
    // comment B
}

function bar() {
}

The comment B is now linked to function bar instead, although it clearly sits inside function foo.

There are more weird cases I could bring out...

There are many culprits collaborating in this problem:

Only some AST nodes have the line @line field assigned (seemingly arbitrarily).
The number in @line field refers sometimes to the line where the node begins. Sometimes to the line where it ends. (Seemingly arbitrarily again.)
Just one line number is not enough information about Node's position - one really needs to know both where a node begins and where it ends.
With just line numbers data everything falls apart when all the comments and code are on one long line - one really needs to know the character positions too.
The comments linking algorithm is simply too naive.

rkelly (and rkelly-remix) unable to parse non-ASCII identifiers

The following is valid JavaScript syntax:

ą.ś = {ą: 'ś'}

rkelly-remix fails badly when trying to parse it:

require 'rkelly'
puts RKelly::Parser.new.parse("ą.ś = {ą: 'ś'}").to_ecma

C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rkelly-remix-0.0.6/lib/rkelly/tokenizer.rb:201:in `raw_tokens': undefined method `name' for nil:NilClass (NoMethodError)
        from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rkelly-remix-0.0.6/lib/rkelly/parser.rb:40:in `parse'
        from rkelly.rb:2:in `<main>'

According to ECMAScript Language Specification - ECMA-262 Edition 5.1, identifiers can contain any Unicode letters and digits (and some other things too). According to rkelly and rkelly-remix, they may only contain [_\$0-9A-Za-z].

@nene, It shouldn't be very hard to extend the tokenizer to support this? I could probably do the boring patch creating. :)

(This was encountered in the wild with JSDuck trying to parse this file: https://gerrit.wikimedia.org/r/#/c/147828/1/modules/ve/ui/inspectors/ve.ui.SpecialCharacterInspector.js – this diff removes the quotes around accented characters to satisfy jscs's disallowQuotedKeysInObjects rule. Original error log can be seen here: https://integration.wikimedia.org/ci/job/VisualEditor-jsduck/1705/console.)

(Let me also note that the error messages returned when encountering unknown syntax are unpleasant, e.g. the above undefined method name for nil:NilClass (NoMethodError) or something bad happened, please report a bug with sample JavaScript (RuntimeError) for var a = 100abc;.)

> rake
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake/alt_system.rb:32: Use RbConfig instead of obsolete and deprecated Config.
(in /Users/rpai/workspace/rkelly)
racc -o lib/rkelly/generated_parser.rb lib/parser.y
1 shift/reduce conflicts
/Users/rpai/.rbenv/versions/1.9.3-p448/bin/ruby -w -Ilib:ext:bin:test -e 'require "rubygems"; require "test/unit"; require "test/execution_contexts/test_10_1_3-1.rb"; require "test/expressions/test_11_3_1.rb"; require "test/expressions/test_11_3_2.rb"; require "test/expressions/test_11_4_2.rb"; require "test/expressions/test_11_4_3.rb"; require "test/expressions/test_11_4_4.rb"; require "test/expressions/test_11_4_5.rb"; require "test/expressions/test_11_4_6.rb"; require "test/expressions/test_11_4_8.rb"; require "test/expressions/test_11_4_9.rb"; require "test/expressions/test_11_5_1.rb"; require "test/expressions/test_11_5_2.rb"; require "test/expressions/test_11_5_3.rb"; require "test/expressions/test_11_6_1-1.rb"; require "test/expressions/test_11_9_1.rb"; require "test/function/test_15_3_1_1-1.rb"; require "test/global_object/test_15_1_1_1.rb"; require "test/global_object/test_15_1_1_2.rb"; require "test/global_object/test_15_1_1_3.rb"; require "test/object/test_15_2_1_1.rb"; require "test/object/test_15_2_1_2.rb"; require "test/object/test_15_2_2_1.rb"; require "test/statements/test_12_5-1.rb"; require "test/test_add_node.rb"; require "test/test_arguments_node.rb"; require "test/test_array_node.rb"; require "test/test_assign_expr_node.rb"; require "test/test_automatic_semicolon_insertion.rb"; require "test/test_bit_and_node.rb"; require "test/test_bit_or_node.rb"; require "test/test_bit_x_or_node.rb"; require "test/test_bitwise_not_node.rb"; require "test/test_block_node.rb"; require "test/test_bracket_accessor_node.rb"; require "test/test_break_node.rb"; require "test/test_case_block_node.rb"; require "test/test_case_clause_node.rb"; require "test/test_comma_node.rb"; require "test/test_comments.rb"; require "test/test_conditional_node.rb"; require "test/test_const_statement_node.rb"; require "test/test_continue_node.rb"; require "test/test_delete_node.rb"; require "test/test_divide_node.rb"; require "test/test_do_while_node.rb"; require "test/test_dot_accessor_node.rb"; require "test/test_ecma_visitor.rb"; require "test/test_element_node.rb"; require "test/test_empty_statement_node.rb"; require "test/test_equal_node.rb"; require "test/test_evaluation_visitor.rb"; require "test/test_expression_statement_node.rb"; require "test/test_false_node.rb"; require "test/test_for_in_node.rb"; require "test/test_for_node.rb"; require "test/test_function_body_node.rb"; require "test/test_function_call_node.rb"; require "test/test_function_decl_node.rb"; require "test/test_function_expr_node.rb"; require "test/test_function_visitor.rb"; require "test/test_getter_property_node.rb"; require "test/test_global_object.rb"; require "test/test_greater_node.rb"; require "test/test_greater_or_equal_node.rb"; require "test/test_if_node.rb"; require "test/test_in_node.rb"; require "test/test_instance_of_node.rb"; require "test/test_label_node.rb"; require "test/test_left_shift_node.rb"; require "test/test_less_node.rb"; require "test/test_less_or_equal_node.rb"; require "test/test_line_number.rb"; require "test/test_logical_and_node.rb"; require "test/test_logical_not_node.rb"; require "test/test_logical_or_node.rb"; require "test/test_modulus_node.rb"; require "test/test_multiply_node.rb"; require "test/test_new_expr_node.rb"; require "test/test_not_equal_node.rb"; require "test/test_not_strict_equal_node.rb"; require "test/test_null_node.rb"; require "test/test_number_node.rb"; require "test/test_object_literal_node.rb"; require "test/test_op_and_equal_node.rb"; require "test/test_op_divide_equal_node.rb"; require "test/test_op_equal_node.rb"; require "test/test_op_l_shift_equal_node.rb"; require "test/test_op_minus_equal_node.rb"; require "test/test_op_mod_equal_node.rb"; require "test/test_op_multiply_equal_node.rb"; require "test/test_op_or_equal_node.rb"; require "test/test_op_plus_equal_node.rb"; require "test/test_op_r_shift_equal_node.rb"; require "test/test_op_u_r_shift_equal_node.rb"; require "test/test_op_x_or_equal_node.rb"; require "test/test_parameter_node.rb"; require "test/test_parser.rb"; require "test/test_pointcut_visitor.rb"; require "test/test_postfix_node.rb"; require "test/test_prefix_node.rb"; require "test/test_property_node.rb"; require "test/test_regexp_node.rb"; require "test/test_resolve_node.rb"; require "test/test_return_node.rb"; require "test/test_right_shift_node.rb"; require "test/test_rkelly.rb"; require "test/test_runtime.rb"; require "test/test_scope_chain.rb"; require "test/test_setter_property_node.rb"; require "test/test_source_elements.rb"; require "test/test_strict_equal_node.rb"; require "test/test_string_node.rb"; require "test/test_subtract_node.rb"; require "test/test_switch_node.rb"; require "test/test_this_node.rb"; require "test/test_throw_node.rb"; require "test/test_tokenizer.rb"; require "test/test_true_node.rb"; require "test/test_try_node.rb"; require "test/test_type_of_node.rb"; require "test/test_unary_minus_node.rb"; require "test/test_unary_plus_node.rb"; require "test/test_unsigned_right_shift_node.rb"; require "test/test_var_decl_node.rb"; require "test/test_var_statement_node.rb"; require "test/test_void_node.rb"; require "test/test_while_node.rb"; require "test/test_with_node.rb"' 
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- test/execution_contexts/test_10_1_3-1.rb (LoadError)
    from /Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from -e:1:in `<main>'
rake aborted!
Command failed with status (1): [/Users/rpai/.rbenv/versions/1.9.3-p448/bin...]

(See full trace by running task with --trace)

Rake 10.1.0

rake --trace
warning: couldn't activate the clean plugin, skipping
warning: couldn't activate the debug plugin, skipping
warning: couldn't activate the deps plugin, skipping
warning: couldn't activate the package plugin, skipping
warning: couldn't activate the publish plugin, skipping
warning: couldn't activate the rcov plugin, skipping
warning: couldn't activate the signing plugin, skipping
warning: couldn't activate the test plugin, skipping
warning: couldn't activate the gemspec plugin, skipping
warning: couldn't activate the git plugin, skipping
rake aborted!
Don't know how to build task 'check_manifest'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/task_manager.rb:49:in `[]'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/task.rb:348:in `[]'
/Users/rpai/workspace/rkelly/Rakefile:29:in `<top (required)>'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/rake_module.rb:25:in `load'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/rake_module.rb:25:in `load_rakefile'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/application.rb:637:in `raw_load_rakefile'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/application.rb:94:in `block in load_rakefile'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/application.rb:165:in `standard_exception_handling'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/application.rb:93:in `load_rakefile'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/application.rb:77:in `block in run'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/application.rb:165:in `standard_exception_handling'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/lib/rake/application.rb:75:in `run'
/Users/rpai/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rake-10.1.0/bin/rake:33:in `<top (required)>'
/Users/rpai/.rbenv/versions/1.9.3-p448/bin/rake:23:in `load'
/Users/rpai/.rbenv/versions/1.9.3-p448/bin/rake:23:in `<main>'

tenderlove / recma Goto Github PK

recma's People

Contributors

Stargazers

Watchers

Forkers

recma's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs