stephens2424 / php Goto Github PK
View Code? Open in Web Editor NEWParser for PHP written in Go
License: BSD 3-Clause "New" or "Revised" License
Parser for PHP written in Go
License: BSD 3-Clause "New" or "Revised" License
Hello
How run this library?
Its get file and parse file by file, and don't parse a project?
Thank you
Not sure if this intentional / a non-issue for your tooling, but I was hoping to iterate over each token to find out the class this way. Now I'm required to do additional ObjectOperator/ScopeResolutionOperator checks with the previous token.
If the file uses only CR instead of LF/CRLF as line endings, the parser will treat everything after a line comment as part of that line comment.
Example (the \r
character represents carriage-return, there's no \n
anywhere in the file):
<?php\r
// comment\r
$a = 1;
Will yield // comment\r$a = 1;
as a line comment.
To fix that I changed
func lexLineComment(l *lexer) stateFn {
lineLength := strings.Index(l.input[l.pos:], "\n") + 1
to
func lexLineComment(l *lexer) stateFn {
lineLength := strings.IndexAny(l.input[l.pos:], "\r\n") + 1
and it seems to work fine.
<?php
echo 2 * (2 - 3);
(ast.EchoStmt)Echo
(ast.BinaryExpr)-
(*ast.Literal)Literal-float: 2
(*ast.Literal)Literal-float: 3
I am attempting to help flesh out the fmt passes. I do not see how you can differentiate between ++$a
and $a++
. Both appear to have the same values:
node.Children() # => ['$a']
node.Position() # => 0
node.Operator # => '++'
node.Type # => 'integer|float'
node.Operand1 # => '$a'
node.Operand2 # => nil
node.Operand3 # => nil
Hi,
I think I have found a problem with the parsing of simple function calls. Given the following code:
<?php
sort($myArray);
sort() will be parsed to ast.ExpressionStmt. If you use a the function call in an assignment like this:
<?php
$ok = sort($myArray);
sort() will be parsed as ast.FunctionCallExpression. Maybe I missed some point but it seems a bit confusing.
If this is a real bug I could fix it by myself... I would appreciate some hints :)
To do an intermediate transpile, we will need to translate common PHP functions and classes to Go equivalents. This may happen at two levels. The first is to provide functional equivalent implementations of the same functions. These functions would be available to transpiled programs as a package. The second is to provide compiler passes that translate these functions to their idiomatic Go equivalents.
This issue will be closed when the first of these is completed.
test file:
<?php
if (true)
if (true)
echo 1;
else
echo 2;
if (true) echo 1;
?>
got:
Found echo:"echo", expected [;]
I got golint on the top level package, but all the directories...
Hi,
I'm using your tool to parse PHP code with the secret hope of building some kind of static analysis tool with it; I wondered if it would be feasible and/or desirable to track line numbers in Node structures? I don't mind lending a hand for that, but if it's definitely unwanted I'd rather not waste time :)
Working with the php parser in cannot find the namespace node.
How I can access to the namespace node?
Best reagards
Parser errors always display line=0 instead of the real line number.
I tried to rebuild the package. It requires some work to built it again.
I suppose you know about https://go.dev/doc/go-get-install-deprecation
We are currently parsing
echo (int)"0" || false;
As if it was
echo (int)("0" || false);
<?php
$var1 = "test";
$var2 = "{$var1}Append"; //<-- not a literal string
Hey Stephen! Thanks for this project. I'm finally getting around to playing with it!
It appears to be an issue when an 'OR' is reached.
The simplest test case I can give you is this:
<?php defined('BASEPATH') OR exit('No direct script access allowed');
This gives the following error:
(ast.ExitStmt)Literal-string: 'No direct script access allowed'
Compiled 1 files. 0 files with errors - 100.000000% success
In all the tokens from the nodes are missing the Start and End structs.
Is a pending job or I can work on that?
There is a corner case causing an infinite loop that occasionally occurs on the phpconsole program.
test file:
<?php
class A {
function f($x) {
if ($x)
return 1;
else
return 2;
}
}
?>
got:
unexpected class member PHP End:"?>"
The parser will hog the CPU when working on data like
<?php
$a = ‘test‘; // Note that ‘ is a 3-byte UTF-8 character %E2%80%98, not a single quote
I get those files from users that copy-paste PHP code from online articles that apply formatting on PHP code. The reason it hangs is that *lexer.acceptRun
doesn't accept multibyte characters since they are not in the valid
set. A quick fix I did was add an additional check for utf8.RuneLen
to accept multibyte strings, but I'm not sure it's the best workaround.
Several places in the code look for ast.Foo but also *ast.Foo.
Example in ast/printer/printer.go:
case ast.AssignmentExpression:
p.PrintAssignmentExpression(&n)
case *ast.AssignmentExpression:
p.PrintAssignmentExpression(n)
At the end of the day we are not sure about the kind of object inside in AST.
Was looking at this project to look into making a fast PHP autocomplete daemon. Out of curiosity, any reason why the lexer is running in a Goroutine? Wouldn't it be the job of the Parser / user-code to run the lexer in a go-routine (if at all?)
File:
https://github.com/Codeception/Codeception/blob/2.2/tests/data/Invalid.php
File Contents:
<?php
$I do nothing here
My Golang loop:
for {
token := lexer.Next()
if token.Typ == PHPToken.EOF {
break
}
// ... more
}
I got a test php file:
<?php
/* Foo */
// Bar
$baz = "hey";
and a small go tool to parse this:
package main
import (
"fmt"
"github.com/stephens2424/php"
"github.com/stephens2424/php/passes/printing"
"io/ioutil"
"log"
)
func main() {
content, err := ioutil.ReadFile("./test.php")
if err != nil {
log.Fatal(err)
}
fmt.Println(string(content))
p := php.NewParser()
a, err := p.Parse("test.php", string(content))
if err != nil {
log.Fatal(err)
}
w := printing.NewWalker()
for _, node := range a.Nodes {
w.Walk(node)
}
}
The output looks like:
go run main.go
<?php
/* Foo */
// Bar
$baz = "hey";
(ast.ExpressionStmt)=
(ast.AssignmentExpression)=
(*ast.Variable)$baz
(*ast.Identifier)baz
(*ast.Literal)Literal-string: "hey"
As you see the single line + multi line comments are nor parsed in the AST.
Do i something wrong or are they not reflected as node?
To do an intermediate transpile, we will need to translate common PHP functions and classes to Go equivalents. This may happen at two levels. The first is to provide functional equivalent implementations of the same functions. These functions would be available to transpiled programs as a package. The second is to provide compiler passes that translate these functions to their idiomatic Go equivalents.
This issue will be closed when the both of these are completed.
Hey @stephens2424, I stumbled onto your blog post and was intrigued. I wanted to test and play. What I'd like to do is print the golang equivalent of a php file. I edited the bulk_test.go file as a broken attempt.
...
p := NewParser(string(src))
astNodes, errs := p.Parse() // Returns []ast.Node
var buf bytes.Buffer
// printer.Fprint(os.Stdout, nil, astNodes)
printer.Fprint(&buf, nil, astNodes)
s := buf.String()
fmt.Println(s, astNodes)
//The Go parser and printer package are
I'm trying to figure out what I need to do with astNodes and how to create a token.FileSet.
Thanks in advance.
Add a testdata file with a simple foreach.php
file, for example:
<?php foreach ( $arr as $key => $val ){ echo 1; }
Now tune it to have actual logic inside:
<?php
foreach ( $arr as $key => $val ){
if ($x === $y = $z) {
echo "true";
} else {
echo "false";
}
}
And this breaks with:
--- FAIL: TestFiles (0.01s)
bulk_test.go:28: foreach.php :10: Expected expression. Found EOF
:10: Found EOF, statement or expression
:10: Found EOF, statement or expression
Do you have plans to integrate your parser with Srclib? https://srclib.org/
This would allow for intelligent PHP code completion in a number of editors. I'm working on a vim plugin for srclib right now, so it could be very cool!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.