chriswailes / rltk Goto Github PK
View Code? Open in Web Editor NEWThe Ruby Language Toolkit
Home Page: http://chriswailes.github.io/RLTK/
License: University of Illinois/NCSA Open Source License
The Ruby Language Toolkit
Home Page: http://chriswailes.github.io/RLTK/
License: University of Illinois/NCSA Open Source License
Hi,
I am currently working on a natural language parser and I wonder whether it is possible to reject a clause during the parsing.
What I mean is that I defined a grammar but, as in the end all tokens are words, I need to be able to reject some constructions if the word is, say, a verb and the parser is currently in a clause needing a noun. Without that, I don't know how to specify the differences between each class of words.
Cheers,
Emmanuel
I'm curious about using RLTK for some experiments regarding converting an existing DSL into something which can be executed natively, as well as some general experimenting with compiler-writing, because compilers are awesome.
However, it appears that the last commit is close to six months old, and there are no apparent branches that have more recent work. Is RLTK abandoned/moved, or is it just reached a point of stability.
Thanks, RLTK looks like a great project, and I'm really hoping to use it, but I don't want to hang on something that isn't being maintained.
If you could please check a gemspec into the repository then I will be able to use RLTK head from Bundler.
LLVM 3.2 was released a couple of days ago. RLTK's LLVM bindings should be updated.
Do you have any examples of child elements of an ASTNode that is an array?
the first problem I ran into was the value being initialized to nil
which caused the initialization to error out, I started initializing the array children with []
and this gets past initialization, but the AST cannot be traversed (since during traversal it encounters an Array as the value and calls .each(:pre)
which violates the arity of a normal Array)
Was hoping that I am just not understanding something here? (also happy to submit a patch with default initialization and an Array subclass with an appropriate arity on each
to handle traversal)
Here's a snippet:
require "rltk"
require "rltk/cg"
require "rltk/cg/module"
require "rltk/cg/contractor"
@module = RLTK::CG::Module.new("Hello")
f = @module.functions.add("hello", RLTK::CG::VoidType, [RLTK::CG::NativeIntType])
f.params[0].type
# ArgumentError: wrong number of arguments (0 for 1..2)
#0: /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/type.rb:158:in `initialize'
#1: /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/type.rb:50:in `new'
#2: /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/type.rb:50:in `from_ptr'
#3: /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/value.rb:125:in `type'
The exact version of LLVM from brew I'm using is this:
llvm34: stable 3.4.2 (bottled), HEAD
Let's say that I have the following rule in the lexer:
rule(/a/)
and the following string (with 3 leading spaces):
aaab
Lexing it, what I get is a RLTK::LexingError whose line_offset
is supposed to be 6
, but I actually get a way lower number (I guess in this case it would be 3
)
Let me know if you want me to write a test that fails for this bug, in case my description is not clear enough :^)
cc: @libo
I've been working on a new programming language, Rip, for a while now. I am able to construct the AST. I didn't know about RLTK before I got started, so I'm just trying to use RLTK to generated LLVM IR. I've compiled LLVM 3.5 from source based on the official instructions. The binaries and libraries are in a non-standard location (/Users/thomas/Code/llvm/build/Debug+Asserts/{bin,lib}
) though.
Running RLTK::CG::Module.new('foo_module')
results in the following error. What is the preferred way to install LLVM so I can use RLTK?
/usr/local/opt/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/ffi-1.9.3/lib/ffi/library.rb:133:in `block in ffi_lib': Could not open library 'LLVM-3.0': dlopen(LLVM-3.0, 5): image not found. (LoadError)
Could not open library 'libLLVM-3.0.dylib': dlopen(libLLVM-3.0.dylib, 5): image not found
from /usr/local/opt/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/ffi-1.9.3/lib/ffi/library.rb:100:in `map'
from /usr/local/opt/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/ffi-1.9.3/lib/ffi/library.rb:100:in `ffi_lib'
from /usr/local/opt/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rltk-2.2.1/lib/rltk/cg/bindings.rb:46:in `<module:Bindings>'
from /usr/local/opt/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rltk-2.2.1/lib/rltk/cg/bindings.rb:44:in `<module:CG>'
from /usr/local/opt/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rltk-2.2.1/lib/rltk/cg/bindings.rb:23:in `<top (required)>'
from /usr/local/opt/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rltk-2.2.1/lib/rltk/cg/module.rb:11:in `require'
from /usr/local/opt/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rltk-2.2.1/lib/rltk/cg/module.rb:11:in `<top (required)>'
from /Users/thomas/Code/rip-rip/lib/rip/nodes/module.rb:1:in `require'
from /Users/thomas/Code/rip-rip/lib/rip/nodes/module.rb:1:in `<top (required)>'
from /Users/thomas/Code/rip-rip/lib/rip/nodes.rb:28:in `require'
from /Users/thomas/Code/rip-rip/lib/rip/nodes.rb:28:in `<top (required)>'
from /Users/thomas/Code/rip-rip/lib/rip.rb:47:in `require'
from /Users/thomas/Code/rip-rip/lib/rip.rb:47:in `<top (required)>'
from ./bin/rip:3:in `require_relative'
from ./bin/rip:3:in `<main>'
This very simple snippet doesn't seem to work for me and I don't know why:
mod = RLTK::CG::Module.new("Hello")
mod.globals.add(RLTK::CG::Int32Type, "hello")
Stacktrace:
ArgumentError: :pointer argument is not a valid pointer
from /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/module.rb:391:in `add_global'
Running against LLVM-3.4 from Homebrew.
To parse list of items is so usual that I'd like some syntax sugar about it.
Kazoo example:
production(:arg_defs) do
clause('') { || [] }
clause('arg_def_list') { |args| args }
end
production(:arg_def_list) do
clause('IDENT') { |i| [i] }
clause('IDENT COMMA arg_def_list') { |i, _, defs| [i] + defs }
end
I know this is the LALR(1) way, but something like the following is shorter and easier to read:
production(:arg_defs, separator: 'COMMA', required: false) do
clause { |items| defs << items }
end
This way it would be optional, of course. Don't be afraid, LALR(1) purists!
Thank you again for RLTK!
Change the required version of RLTK to LLVM 3.6.
Problem: Windows paths contain a colon
Solution: rltk-3.0.1\lib\rltk\parser.rb:564
Current:
def_file = caller()[2].split(':')[0] if opts[:use]
Should be (choose one):
def_file = caller()[2].split(':')[0..-3].join(':') if opts[:use]
def_file = caller()[2][/^(.*):[^:]+:[^:]+$/, 1] if opts[:use]
Hey there.
This is awesome. I'm wondering if there's a way to get access to the groups captured by a particular rule's regex, as in String#scan
. For example:
class MyLexer < RLTK::Lexer
rule(/\.(\w+)\[/) { [:THING, $1] }
end
Currently it looks like the block is being passed only the whole match. I'm happy to patch it to be more like the core ruby matching methods, but I wanted to check first :)
Affected Versions
Affects version 3.0.0
and version 3.0.1
. Does not affect version 2.2.1
or prior.
Behavior
In the more recent versions of the application when attempting to use a trivial parser we get a NoMethodError.
ast = TaskGrammar::TaskParser::parse(TaskGrammar::TaskLexer::lex(" "))
#=> NoMethodError (undefined method `subclass_of?' for TaskGrammar::TaskCompletion:Class)
When running this with 2.2.1
of the gem, this completes successfully.
ast = TaskGrammar::TaskParser::parse(TaskGrammar::TaskLexer::lex(" "))
=> #<TaskGrammar::TaskAst:0x000055d585cb2e00 @notes={}, @parent=nil, @c=#<TaskGrammar::TaskCompletion:0x000055d585cb34e0 @notes={}, @parent=#<TaskGrammar::TaskAst:0x000055d585cb2e00 ...>, @prefix="a", @infix="b">>
Investigation
It appears that the ast.rb file moved the subclass_of?
definition to the rltk/util/monkeys
file in commit 4565d5c
(4565d5c).
In version 2.2.1 the util folder exists and contains the monkeys.rb
file that defines subclass_of?
(https://github.com/chriswailes/RLTK/tree/v2.2.1/lib/rltk). In version 3.0.0
the utils folder has been deleted (https://github.com/chriswailes/RLTK/tree/v3.0.0/lib/rltk), but no replacement method of subclass_of?
appears to be defined.
LLVM 3.6 is removing support for the legacy JIT. The bindings need to be updated to use only the MCJIT.
We have a JRuby-on-Rails application that interprets TWS definitions using RLTK that then inserts them into database tables. The application previously was using this version and has now been updated to 2.2.1.
We initially saw an error saying NameError exception occurred: undefined local variable or method 'scanner' for #<RLTK::Lexer::Environment:0x2838abe5>
, which seemed to be related to these lines in lib/tws_parser/parser.rb:
# Punctuation
rule(/#/) {
# A hash indicates a comment-line iff it is the first character on the line
scanner.pos -= 1
token = scanner.bol? ? nil : :HASH
if token
scanner.pos += 1
else
scanner.scan_until(/\n/)
end
token
}
After replacing these lines we now receive a new error:
18-Aug-2020 21:02:17.190 INFO [http-nio-7100-exec-10] org.apache.catalina.core.ApplicationContext.log Parsing schedules...
18-Aug-2020 21:02:17.198 INFO [http-nio-7100-exec-10] org.apache.catalina.core.ApplicationContext.log RLTK::NotInLanguage exception occurred: String not in language.
18-Aug-2020 21:02:17.200 INFO [http-nio-7100-exec-10] org.apache.catalina.core.ApplicationContext.log lib/tws_parser/builder.rb:72:in `process'
app/controllers/tivoli_uploads_controller.rb:37:in `create'
app/controllers/tivoli_uploads_controller.rb:19:in `create'
I believe the issue is in the same file as before, here is the contents of lib/tws_parser/parser.rb, lines 103 to 129:
# A hash and then a space indicates a comment-line
rule(/^#/) { push_state :comment }
rule(/\n/, :comment) { pop_state }
rule(/./, :comment)
# Punctuation
rule(/,/) { :COMMA }
rule(/:/) { :COLON }
rule(/\./) { :DOT }
rule(/@/) { :AT_SYM }
rule(/\+/) { :PLUS }
rule(/\-/) { :MINUS }
# Text
rule(/"(\\"|[^"])*"/) do |t|
str = t[1..-2].gsub('\"', '"')
str.gsub!(/%([^%]+)%/) do |m|
(@mappings && @mappings[$1]) || m
end
[:STRING, str]
end
rule(/\([^\)]*\)/) { |t| [:QUALIFIER, t] }
rule(/[A-Za-z%][A-Za-z0-9_\-\\%]*/) do |t|
t =~ /^%(.+)%$/
mapped = $1 && @mappings && @mappings[$1]
[:NAME, mapped || t]
end
I guess the issue is related to the block that assigns [:STRING, str]
but I don't know what I am supposed to change it to - can you please help?
When attempting to run the test suite against Rubinius I get the following backtrace:
An exception occurred running /Users/jnh/.rvm/rubies/rbx-head-rbx19/bin/testrb
undefined method `id' on nil:NilClass. (NoMethodError)
Backtrace:
Kernel(NilClass)#id (method_missing) at kernel/delta/kernel.rb:81
{ } in RLTK::Parser.grammar_prime at lib/rltk/parser.rb:689
Array#each at kernel/bootstrap/array.rb:68
{ } in RLTK::Parser.grammar_prime at lib/rltk/parser.rb:686
Array#each at kernel/bootstrap/array.rb:68
{ } in RLTK::Parser.grammar_prime at lib/rltk/parser.rb:681
{ } in RLTK::Parser::State#each at lib/rltk/parser.rb:1505
Array#each at kernel/bootstrap/array.rb:68
RLTK::Parser::State#each at lib/rltk/parser.rb:1505
{ } in RLTK::Parser.grammar_prime at lib/rltk/parser.rb:676
Array#each at kernel/bootstrap/array.rb:68
RLTK::Parser.grammar_prime at lib/rltk/parser.rb:675
{ } in RLTK::Parser.prune at lib/rltk/parser.rb:1076
Array#each at kernel/bootstrap/array.rb:68
{ } in RLTK::Parser.prune at lib/rltk/parser.rb:1074
Array#each at kernel/bootstrap/array.rb:68
{ } in RLTK::Parser.prune at lib/rltk/parser.rb:1068
Array#each at kernel/bootstrap/array.rb:68
RLTK::Parser.prune at lib/rltk/parser.rb:1058
RLTK::Parser.finalize at lib/rltk/parser.rb:615
ParserTester::EmptyListParser0.__class_init__ (EmptyListParser0) at test/tc_parser.rb:111
ParserTester.__class_init__ (ParserTester) at test/tc_parser.rb:108
Object#__script__ at test/tc_parser.rb:26
Rubinius::CodeLoader.require at kernel/common/codeloader.rb:212
Kernel(Object)#require at kernel/common/kernel.rb:649
Object#__script__ at test/ts_rltk.rb:34
Rubinius::CodeLoader.require at kernel/common/codeloader.rb:212
Kernel(Test::Unit::AutoRunner::Runner)#require at kernel/common/kernel.rb:649
{ } in Test::Unit::RequireFiles(Test::Unit::AutoRunner::Runner)#non_options at /Users/jnh/.rvm/rubies/rbx-head-rbx19/lib/19/test/unit.rb:221
Array#each at kernel/bootstrap/array.rb:68
Test::Unit::RequireFiles(Test::Unit::AutoRunner::Runner)#non_options at /Users/jnh/.rvm/rubies/rbx-head-rbx19/lib/19/test/unit.rb:215
Test::Unit::Options(Test::Unit::AutoRunner::Runner)#process_args at /Users/jnh/.rvm/rubies/rbx-head-rbx19/lib/19/test/unit.rb:52
Test::Unit::AutoRunner#process_args at /Users/jnh/.rvm/rubies/rbx-head-rbx19/lib/19/test/unit.rb:625
Object#__script__ at /Users/jnh/.rvm/rubies/rbx-head-rbx19/bin/testrb:5
Rubinius::CodeLoader#load_script at kernel/delta/codeloader.rb:68
Rubinius::CodeLoader.load_script at kernel/delta/codeloader.rb:118
Rubinius::Loader#script at kernel/loader.rb:615
Rubinius::Loader#main at kernel/loader.rb:816
Coverage report generated for Unit Tests to /Users/jnh/Dev/forks/RLTK/coverage. 699 / 763 LOC (91.61%) covered.
rake aborted!
Command failed with status (1): [/Users/jnh/.rvm/rubies/rbx-head-rbx19/bin/...]
Tasks: TOP => test
(See full trace by running task with --trace)
The backtrace points at lib/rltk/parser.rb:689
as being the culprit:
def grammar_prime
if not @grammar_prime
@grammar_prime = CFG.new
@states.each do |state|
state.each do |item|
lhs = "#{state.id}_#{item.next_symbol}".to_sym
next unless CFG::is_nonterminal?(item.next_symbol) and not @grammar_prime.productions.keys.include?(lhs)
@grammar.productions[item.next_symbol].each do |production|
rhs = ""
cstate = state
production.rhs.each do |symbol|
rhs += "#{cstate.id}_#{symbol} "
cstate = @states[cstate.on?(symbol).first.id] # <-- here
end
@grammar_prime.production(lhs, rhs)
end
end
end
end
@grammar_prime
end
The parser code is too complex for me to figure out at first blush, so I thought I'd raise an issue in the hopes that you had any ideas/pointers on where I should look?
Hi!
First of all thanks for this tool!
I have noticed that the following line in the ASTNode
require 'filigree/match'
https://github.com/chriswailes/RLTK/blob/master/lib/rltk/ast.rb
Actually will include this in the main Object
def match(*objects, &block)
me = Filigree::MatchEnvironment.new
me.instance_exec &block
me.find_match(objects)
end
https://github.com/chriswailes/filigree/blob/master/lib/filigree/match.rb#L157
It looks like RLTK doesn't use #match
but only the module Filigree::Destructurable
as described in this commit that introduced the inclusion: 3cf628c
I was wondering if we could avoid leaking the match
method in a global scope.
I have noticed since including rltk/ast started to conflict with rspec
Thanks!
I am trying to use the error production feature for handling typos in a really simple context-free grammar. Could you please give a more advanced example?
The code generation library, used to interact with LLVM, will be moved to its own project. This will allow developers to upgrade RLTK and the LLVM bindings independently, easing the upgrade path for both RLTK and LLVM installations.
If this will cause any major issues for anyone please let me know.
A new testcase has been added that checks the interplay between EBNF elemination and the production value selectors. It currently doesn't work and should be fixed.
Happy birthday to v3.0.1 - its been 5 years ๐ฐ
On the serious side: Can we get a release?
Today I fought a bit with the BadToken error and was confused as the code in the repo didn't match the error received. #50 improved this error a bit. Although not a lot has happened, there are a few things in master that might be worth releasing.
Cheers
With the new dependency on filigree
, gem
will now choke trying to install on Ruby versions lower than 2.4.0:
steven@macbook:~/workspace/apps/ios $ gem install rltk
Fetching: ffi-1.9.25.gem (100%)
Building native extensions. This could take a while...
Successfully installed ffi-1.9.25
Fetching: filigree-0.4.1.gem (100%)
ERROR: Error installing rltk:
filigree requires Ruby version >= 2.4.0.```
It seems like it doesn't matter what I do, the result's always the same. Every time I try to call run_function
, it blows up.
require "rltk"
require "rltk/cg"
require "rltk/cg/module"
require "rltk/cg/contractor"
require "rltk/cg/execution_engine"
# ENV["DYLD_FALLBACK_LIBRARY_PATH"] = "/usr/local/lib/llvm-3.4/lib/"
RLTK::CG::LLVM.init(:X86)
@module = RLTK::CG::Module.new("Hello")
@engine = RLTK::CG::JITCompiler.new(@module)
f = @module.functions.add("hello", RLTK::CG::VoidType, [])
f.blocks.append('entry') do
ret_void
end
f.verify!
@engine.run_function(f, [])
/Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/execution_engine.rb:104: [BUG] Segmentation fault at 0x00000000000000
ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-darwin14.0]
-- Crash Report log information --------------------------------------------
See Crash Report log file under the one of following:
* ~/Library/Logs/CrashReporter
* /Library/Logs/CrashReporter
* ~/Library/Logs/DiagnosticReports
* /Library/Logs/DiagnosticReports
for more details.
-- Control frame information -----------------------------------------------
c:0004 p:---- s:0020 e:000019 CFUNC :run_function
c:0003 p:0075 s:0013 e:000011 METHOD /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/execution_engine.rb:104
c:0002 p:0156 s:0005 E:000b60 EVAL binding_test.rb:26 [FINISH]
c:0001 p:0000 s:0002 E:000868 TOP [FINISH]
binding_test.rb:26:in `<main>'
/Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/execution_engine.rb:104:in `run_function'
/Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/execution_engine.rb:104:in `run_function'
-- C level backtrace information -------------------------------------------
0 ruby 0x00000001036a6e50 rb_vm_bugreport + 144
1 ruby 0x00000001035589b1 report_bug + 305
2 ruby 0x0000000103558874 rb_bug + 180
3 ruby 0x00000001036291bc sigsegv + 156
4 libsystem_platform.dylib 0x00007fff8f1f3f1a _sigtramp + 26
5 libLLVM-3.4.dylib 0x000000010429534a LLVMRunFunction + 96
6 ??? 0x00007fc92b226560 0x0 + 140501988828512
-- Other runtime information -----------------------------------------------
* Loaded script: binding_test.rb
* Loaded features:
0 enumerator.so
1 enc/encdb.so
2 enc/trans/transdb.so
3 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/2.1.0/x86_64-darwin14.0/rbconfig.rb
4 thread.rb
5 thread.so
6 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/compatibility.rb
7 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/defaults.rb
8 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/deprecate.rb
9 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/errors.rb
10 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/version.rb
11 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/requirement.rb
12 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/platform.rb
13 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/basic_specification.rb
14 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/stub_specification.rb
15 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/util/stringio.rb
16 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/specification.rb
17 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/exceptions.rb
18 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_gem.rb
19 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/2.1.0/monitor.rb
20 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb
21 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems.rb
22 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/path_support.rb
23 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/site_ruby/2.1.0/rubygems/dependency.rb
24 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk.rb
25 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg.rb
26 /Users/gunca/.rvm/gems/ruby-2.1.5/extensions/x86_64-darwin-14/2.1.0-static/ffi-1.9.8/ffi_c.bundle
27 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/platform.rb
28 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/types.rb
29 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/library.rb
30 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/errno.rb
31 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/pointer.rb
32 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/memorypointer.rb
33 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/struct_layout_builder.rb
34 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/struct.rb
35 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/union.rb
36 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/managedstruct.rb
37 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/callback.rb
38 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/io.rb
39 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/autopointer.rb
40 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/variadic.rb
41 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/enum.rb
42 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi/ffi.rb
43 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/ffi-1.9.8/lib/ffi.rb
44 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/filigree-0.3.2/lib/filigree/boolean.rb
45 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/version.rb
46 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/generated_bindings.rb
47 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/bindings.rb
48 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/context.rb
49 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/module.rb
50 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/filigree-0.3.2/lib/filigree/class_methods_module.rb
51 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/2.1.0/ostruct.rb
52 /Users/gunca/.rvm/rubies/ruby-2.1.5/lib/ruby/2.1.0/singleton.rb
53 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/filigree-0.3.2/lib/filigree/abstract_class.rb
54 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/filigree-0.3.2/lib/filigree/types.rb
55 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/filigree-0.3.2/lib/filigree/class.rb
56 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/filigree-0.3.2/lib/filigree/match.rb
57 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/filigree-0.3.2/lib/filigree/visitor.rb
58 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/type.rb
59 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/value.rb
60 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/instruction.rb
61 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/builder.rb
62 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/contractor.rb
63 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/pass_manager.rb
64 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/triple.rb
65 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/target.rb
66 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/execution_engine.rb
67 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/llvm.rb
68 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/basic_block.rb
69 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/function.rb
70 /Users/gunca/.rvm/gems/ruby-2.1.5/gems/rltk-3.0.0/lib/rltk/cg/generic_value.rb
[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
Don't forget to include the above Crash Report log file.
For details: http://www.ruby-lang.org/bugreport.html
Abort trap: 6
I have the following associativity map:
left :MOD
left :GT, :GTE, :LT, :LTE
left :SUB
right :ADD
left :DIV
right :MUL
right :EXP
right :DOLLAR
right :VAR
I want DOLLAR and VAR to have higher precedence than anything else. However, since RLTK sets operator precedence for left-assoc and right-assoc operators separately, if I add another left-associative operators up top (e.g. left :QUESTION), DOLLAR, which right now has precedence 4, will suddenly become lower-precedence than DIV, which currently also has precedence 4.
This seems wrong to me - I don't see why precedence should be tied to left- right- or non-associativity.
There is also another issue that setting precedence based on the token type does not always work out, since the same symbol may have multiple meanings - for example I have COLON showing up in a ternary expression production clause and also in a dictionary item label clause - it doesn't make sense to set one precedence for COLON since these are two very different operations.
There seems to be a problem with regard to nondeterminism in the productions results:
parsing AB
under S-> (A B? B?)*
results in duplicated output. Never mind the global variable, its just to test that the output is indeed generated twice, and not just duplicated.
https://gist.github.com/ranmrdrakono/dfbbe4dbfae599f5b8c7
The output should be (in my opinion) [:body, [[:a, 1]]]
but acctually is [:body, [[:a, 1], [:a, 2]]]
RLTK targets an older version of the LLVM (3.4). Is there any future plans of upgrading to support newer versions of the LLVM?
That gem version combination raises:
#<ArgumentError: wrong number of arguments (given 4, expected 2)>
.../gems/ruby-2.5.1/gems/filigree-0.4.1/lib/filigree/types.rb:30:in `check_type'
.../gems/ruby-2.5.1/gems/rltk-3.0.1/lib/rltk/ast.rb:169:in `block in define_accessor'
.../gems/ruby-2.5.1/gems/rltk-3.0.1/lib/rltk/ast.rb:458:in `block in initialize'
.../gems/ruby-2.5.1/gems/rltk-3.0.1/lib/rltk/ast.rb:457:in `each'
.../gems/ruby-2.5.1/gems/rltk-3.0.1/lib/rltk/ast.rb:457:in `initialize'
.../gems/ruby-2.5.1/gems/filigree-0.4.1/lib/filigree/abstract_class.rb:75:in `new'
.../gems/ruby-2.5.1/gems/filigree-0.4.1/lib/filigree/abstract_class.rb:75:in `new'
while instantiating an RLTK::ASTNode
derived class like this:
class Participant < RLTK::ASTNode
value :id, String
value :text, String
end
due to updated method signature of check_type
from filigree gem.
check_type(obj, type, blame: nil, nillable: false, strict: false)
It is called from RLTK like so:
self.instance_variable_set(ivar_name, check_type(value, type, nil, true))
I believe the documentation for RLTK::Parser.empty_list '(as well as nonempty_list) is either a little unspecific or the functions do not behave as expected. If one uses an expressions such as "COMMA NL?" for the seperator only the first list value will be returned. This behaviour is fixed when the expression is outsourced into an own rule.
See this example for clarification:
In the example, both MUL and DIV are using right associativity, which should be left. It is misleading to some extent.
To illustrate right associativity, better to use exponentiation in the example.
I'm working on a ruby runtime for the Handlebars.js templating language. I ran into an issue where the parser erroneously raises a string not in language exception when tokens contain underscores.
Changing the following rule:
rule /{{/, :default do
push_state :expression
[ :EXPRESSION_START ]
end
to
rule /{{/, :default do
push_state :expression
[ :EXPRESSIONSTART ]
end
Allows the parser to succeed.
I have prepared examples of working and non-working lexers, parsers and specs for your enjoyment at jamesotron/FlavourSaver on the broken_parser
and working_parser
branches. In both branches if you run the specs you will find that the lexer output passes but the parser will change behaviour.
Thanks for RLTK, it's an awesome library!
Hi, the generation of the yardoc produce this error:
[debug]: Parsing ["{lib,app}/**/*.rb", "ext/**/*.{c,cc,cxx,cpp,rb}"] with `ruby` parser
[debug]: Parsing lib/rltk.rb
[debug]: Parsing lib/rltk/cg.rb
[debug]: Parsing lib/rltk/ast.rb
[debug]: Parsing lib/rltk/cfg.rb
[debug]: Parsing lib/rltk/lexer.rb
[debug]: Parsing lib/rltk/token.rb
[debug]: Parsing lib/rltk/parser.rb
Traceback (most recent call last):
25: from /home/didacus/.gem/ruby/2.7.0/bin/yardoc:23:in `<main>'
24: from /home/didacus/.gem/ruby/2.7.0/bin/yardoc:23:in `load'
23: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/bin/yardoc:13:in `<top (required)>'
22: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/cli/command.rb:14:in `run'
21: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/cli/yardoc.rb:258:in `run'
20: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/registry.rb:210:in `lock_for_writing'
19: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/registry_store.rb:202:in `lock_for_writing'
18: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/serializers/yardoc_serializer.rb:56:in `lock_for_writing'
17: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/cli/yardoc.rb:259:in `block in run'
16: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard.rb:20:in `parse'
15: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/source_parser.rb:113:in `parse'
14: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/logging.rb:182:in `enter_level'
13: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/source_parser.rb:114:in `block in parse'
12: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/source_parser.rb:371:in `parse_in_order'
11: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/source_parser.rb:45:in `parse'
10: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/logging.rb:82:in `capture'
9: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/source_parser.rb:46:in `block in parse'
8: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/source_parser.rb:442:in `parse'
7: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/ruby/ruby_parser.rb:17:in `parse'
6: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/ruby/ruby_parser.rb:56:in `parse'
5: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/ruby/ruby_parser.rb:56:in `parse'
4: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/ruby/ruby_parser.rb:165:in `on_def'
3: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/ruby/ruby_parser.rb:238:in `visit_event'
2: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/ruby/ast_node.rb:64:in `source_range'
1: from /home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/ruby/ast_node.rb:348:in `reset_line_info'
/home/didacus/.gem/ruby/2.7.0/gems/yard-0.9.24/lib/yard/parser/ruby/ast_node.rb:348:in `first': cannot get the first element of beginless range (RangeError)
Is it a problem of rltk or yard ?
By the way, this is an amazing project. It's fun creating programming languages with this library. Thank you!
Traversing the AST fails if some children have Array types:
I've been following the tutorial to build Kazoo, but have come across 2 problems:
S:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/rltk-3.0.1/lib/rltk/parser.rb:568:in `mtime': No such file or directory @ rb_file_s_mtime - J (Errno::ENOENT)
from S:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/rltk-3.0.1/lib/rltk/parser.rb:568:in `finalize'
from J:/Programs/ruby/beetbasicrb/parser.rb:58:in `<class:Parser>'
from J:/Programs/ruby/beetbasicrb/parser.rb:4:in `<module:BeetBasic>'
from J:/Programs/ruby/beetbasicrb/parser.rb:3:in `<top (required)>'
from <internal:S:/Ruby31-x64/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
from <internal:S:/Ruby31-x64/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
from main.rb:4:in `<main>'
My parser.rb
calls
finalize({:use => 'bbparser.tbl'})
and my directory structure looks like:
When I delete the file, it generates as planned.
S:/Ruby31-x64/lib/ruby/gems/3.1.0/gems/rltk-3.0.1/lib/rltk/ast.rb:120:in `child': undefined method `subclass_of?' for BeetBasic::Expression:Class (NoMethodError)
if not t.subclass_of?(ASTNode)
^^^^^^^^^^^^^
Did you mean? subclasses
from J:/Programs/ruby/beetbasicrb/ast.rb:17:in `<class:Binary>'
from J:/Programs/ruby/beetbasicrb/ast.rb:16:in `<module:BeetBasic>'
from J:/Programs/ruby/beetbasicrb/ast.rb:3:in `<top (required)>'
from <internal:S:/Ruby31-x64/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
from <internal:S:/Ruby31-x64/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
from main.rb:5:in `<main>'
I guess they must have removed this method? I'm defining Expression and Binary like this:
class Expression < RLTK::ASTNode; end
#...
class Binary < Expression
child :left, Expression #* child is a method that takes a symbol and a type. The lack of parenthesis is kiling me.
child :right, Expression
end
I don't think I can fix this one on my end.
I'm new to Ruby (although not new to programming) so if I made a rookie mistake let me know. I'm using the Gem version of RLTK and the latest release of Ruby, if that wasn't clear.
If you need a closer look, this project is a public repo.
As Ruby 1.9 is starting to find its way into mainline distributions of Linux it doesn't seem to unreasonable to drop support for Ruby 1.8 syntax and code. Unless anyone who is currently using RLTK has any major objections I will be making some changes to the codebase that will prevent RLTK from running with Ruby versions < 1.9.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.