refreezed / dumbluaparser Goto Github PK

View Code? Open in Web Editor NEW

29.0 6.0 3.0 646 KB

Lua parsing library capable of optimizing and minifying code.

Home Page: http://refreezed.com/luaparser/

License: MIT License

Lua 100.00%

lua parser pure-lua library lua51 lua52 lua53 lua-module abstract-syntax-tree tokenizer

dumbluaparser's People

Contributors

Stargazers

Watchers

Forkers

lukadev-0 foul11 dseeni

dumbluaparser's Issues

Error processing escape character in strings

Very simple code for demo:

local parser = require('dumbParser')

local ast = parser.parse('print("\\x00")') -- passed string: print("\x00")
--local ast = parser.parseExpression('print("\\x00")') -- same thing
--local ast = parser.parseFile('test.lua') -- same thing

parser.printTree(ast) -- toLua gives the same effect
                      -- lua51:        ... ARG1 literal (string="x00") / Wrong.
                      -- lua53/luajit: ... ARG1 literal (string="{NUL}") / OK.


local ast = parser.parse('print("\\\\x00")') -- passed string: print("\\x00") / OK
--local ast = parser.parse('print("\\0000")') -- passed string: print("\0000") / OK
--local ast = parser.parse('print("\\1000")') -- passed string: print("\1000") / OK

parser.printTree(ast) -- toLua gives the same effect
                      -- lua51:        ... ARG1 literal (string="\x00") / OK.
                      -- lua53/luajit: ... ARG1 literal (string="\x00") / OK.

This affects the recording of escape characters.

In Lua5.1 they turn into plain text.
In Lua5.3 or Luajit there is no such problem.

Creating a rock

Hey ReFreezed,

Pretty happy to stumble upon your project, I hope you keep maintaining it :)

Will be interesting adding your package into luarock, so we can easily consume it.

I can open a PR including the spec, but some steps shall be done by you like creating an account in luarocks server and also tagging the release.

Here is how: https://github.com/luarocks/luarocks/wiki/Creating-a-rock

Let me know what you think.

Suggestion for optimize function

replaces calls to global functions with calls to local functions and generates a single line of local variable assignments at the top

so something like :

local function doSomething(s)
      s = string.rep(s, 100)
      print(s)
end

turns into

local string_rep, pring = string.rep, print
local function doSomething(s)
      s = string_rep(s, 100)
      print(s)
end

Incorrect parsing of binary operator chain, when metatables are involved

Currently, it is assumed that the following operators may be chained while omitting parentheses: ["..", "and", "or", "+", "*", "&", "|"].
However, this becomes an issue when a metatable that has special operator functions is operated on. For example, consider this piece of code:
local ret = 4 + (setmetatable({}, { __add = function(_, val) return val * 2; end }) + 64) return ret

Here, the metatable will be operated on with the + operator and the number 64 FIRST, and then 4 will be added to the result, however these semantics are thrown out after parsing the script, leaving us with this result:
local ret = 4 + setmetatable({}, {__add=function(_, val) return val * 2; end}) + 64; return ret;
Now, because the parentheses were omitted, the behavior has changed, and no longer will the metatable function be called with 64 first.

Proposed solution: Consider adding a check to make sure that each member of the binary operator chain is a number, before assuming parentheses are safe to remove.

Cannot parse "^" expression, when a negative number is on the right side

Currently, due to this line:

DumbLuaParser/dumbParser.lua

Line 2051 in a14fbbe

and OPERATOR_PRECEDENCE.unary > lastPrecedence

the following code fails to parse correctly:
local _ = 6 ^ -4

Proposed solution: Change the line to include a check for "^" (not sure if this breaks anything)
and ((OPERATOR_PRECEDENCE.unary > lastPrecedence) or lastPrecedence == 11)

Minification problems

Hello, I am using your parser with the following tool for the Love2D engine: https://github.com/2dengine/love.maker
and we are getting incorrect output after code minification (difficult to say, but it may be related to parsing literals).

I ran another of my tools called SUPERSTRICT https://github.com/2dengine/sstrict.lua over the source code of the parser and we found some issues:

/minify.lua:436: unused variable 'x'
/minify.lua:435: unused variable '_ENV'
/minify.lua:485: undefined variable 'bit32'
/minify.lua:1080: variable name 'i1' redefinition
/minify.lua:1219: variable name 'i1' redefinition
/minify.lua:1219: variable name 'i2' redefinition
/minify.lua:1256: variable name 'i1' redefinition
/minify.lua:1256: variable name 'i2' redefinition
/minify.lua:1277: empty code block
/minify.lua:1344: variable name 'i1' redefinition
/minify.lua:1350: variable name 'b1' redefinition
/minify.lua:1350: variable name 'b2' redefinition
/minify.lua:1363: variable name 'i2' redefinition
/minify.lua:1162: unused variable 'BYTES_NUM_OR_DOT'
/minify.lua:1653: variable name 'tokNext' redefinition
/minify.lua:1653: variable name 'err' redefinition
/minify.lua:1709: variable name 'tokNext' redefinition
/minify.lua:1709: variable name 'err' redefinition
/minify.lua:2184: variable name 'tokNext' redefinition
/minify.lua:2184: variable name 'err' redefinition
/minify.lua:2213: variable name 'tokNext' redefinition
/minify.lua:2213: variable name 'err' redefinition
/minify.lua:2236: variable name 'tokNext' redefinition
/minify.lua:2236: variable name 'err' redefinition
/minify.lua:2250: variable name 'expr' redefinition
/minify.lua:2250: variable name 'tokNext' redefinition
/minify.lua:2250: variable name 'err' redefinition
/minify.lua:2260: variable name 'block' redefinition
/minify.lua:2260: variable name 'tokNext' redefinition
/minify.lua:2260: variable name 'err' redefinition
/minify.lua:2269: variable name 'block' redefinition
/minify.lua:2269: variable name 'tokNext' redefinition
/minify.lua:2269: variable name 'err' redefinition
/minify.lua:2310: variable name 'ok' redefinition
/minify.lua:2310: variable name 'tokNext' redefinition
/minify.lua:2310: variable name 'err' redefinition
/minify.lua:2316: empty code block
/minify.lua:2328: variable name 'tokNext' redefinition
/minify.lua:2328: variable name 'err' redefinition
/minify.lua:2355: variable name 'tokNext' redefinition
/minify.lua:2355: variable name 'err' redefinition
/minify.lua:2374: variable name 'tokNext' redefinition
/minify.lua:2374: variable name 'err' redefinition
/minify.lua:2387: variable name 'tokNext' redefinition
/minify.lua:2387: variable name 'err' redefinition
/minify.lua:2414: variable name 'tokNext' redefinition
/minify.lua:2414: variable name 'err' redefinition
/minify.lua:2438: variable name 'ok' redefinition
/minify.lua:2438: variable name 'tokNext' redefinition
/minify.lua:2438: variable name 'err' redefinition
/minify.lua:2522: variable name 'tokNext' redefinition
/minify.lua:2522: variable name 'err' redefinition
/minify.lua:2537: variable name 'ok' redefinition
/minify.lua:2537: variable name 'tokNext' redefinition
/minify.lua:2537: variable name 'err' redefinition
/minify.lua:2758: empty code block
/minify.lua:3178: variable name 'i' redefinition
/minify.lua:3290: empty code block
/minify.lua:3406: empty code block
/minify.lua:3622: variable name 'node' redefinition
/minify.lua:3941: empty code block
/minify.lua:4106: variable name 'node' redefinition
/minify.lua:4311: empty code block
/minify.lua:4623: variable name 'slot' redefinition
/minify.lua:4634: variable name 'slot' redefinition
/minify.lua:4666: variable name 'statement' redefinition
/minify.lua:4801: variable name 'optimize' redefinition
/minify.lua:5066: variable name 'ok' redefinition
/minify.lua:5092: variable name 'pretty' redefinition
/minify.lua:5114: variable name 'ok' redefinition
/minify.lua:5168: variable name 'ok' redefinition
/minify.lua:5424: variable name 'ok' redefinition
/minify.lua:5541: variable name 'ok' redefinition
/minify.lua:5600: variable name 'ok' redefinition
/minify.lua:5621: variable name 'ok' redefinition
/minify.lua:5646: variable name 'ok' redefinition
/minify.lua:5656: variable name 'ok' redefinition
/minify.lua:5669: variable name 'ok' redefinition
/minify.lua:5700: variable name 'ok' redefinition
/minify.lua:5723: variable name 'ok' redefinition
/minify.lua:5749: variable name 'ok' redefinition
/minify.lua:5759: variable name 'ok' redefinition
/minify.lua:6094: empty code block
/minify.lua:6104: empty code block
/minify.lua:6297: empty code block
/minify.lua:6453: variable name 'v' redefinition
/minify.lua:6453: unused variable 'v'

The "redefinition" warnings are not important, although there are some questionable parts like:
HANDLE_ENV = not pcall(function() local x = _G end)
where "x" is an unused variable and "local x = _G end" does nothing.

Any tips on how we can pinpoint what is causing the incorrect output of the minification?
Thank you!

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

Jobs

Jooble