bblfsh / bash-driver Goto Github PK
View Code? Open in Web Editor NEWLicense: GNU General Public License v3.0
License: GNU General Public License v3.0
As discovered in bblfsh/javascript-driver#81 (comment) and noted at bblfsh/javascript-driver#81 (comment) - bash driver does not handle string literals in semantic mode the same way as other drivers do.
On
bc = "b\nc"
//uast:String
Path query results in
"@type":"uast:String","Format":"","Value":"b\\nc"
where the expected result for semantic mode would be b\ncq
.
The type Parameter_expansion_operator_':-'
is missing the bash:
prefix, unlike Parameter_expansion_operator_'!'
or Parameter_expansion_operator_'-'
,in Semantic mode.
You can see this by parsing export foo=${foo:-bar}
.
When parsing export foo=(bar)
, the array_assignment_list
node has token value bar)
instead of either bar
or bar)
. This is also true if the list has multiple element, e.g. export foo=(bar baz)
yields an array_assignment_list
node with token value bar baz)
.
The underlying combined_word
node(s) have the correct value(s) though, although the parentheses are not contained - I was expecting bash:LP
or bash_LT
nodes but maybe this is expected behavior ?
Running the following command (where ./utils/install_server.sh
is from a fresh clone of Redis):
/target # bblfsh-cli -a localhost:9432 -l bash -m native --v2 ./utils/install_server.sh
Produces a UAST but also generates the following warning:
WARN: com.ansorgit.plugins.bash.lang.lexer._BashLexer
java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
at com.ansorgit.plugins.bash.lang.util.HeredocSharedImpl.trimNewline(HeredocSharedImpl.java:56)
at com.ansorgit.plugins.bash.lang.util.HeredocSharedImpl.cleanMarker(HeredocSharedImpl.java:29)
at com.ansorgit.plugins.bash.lang.lexer.HeredocLexingState$HeredocMarkerInfo.<init>(HeredocLexingState.java:73)
at com.ansorgit.plugins.bash.lang.lexer.HeredocLexingState.pushMarker(HeredocLexingState.java:54)
at com.ansorgit.plugins.bash.lang.lexer._BashLexerBase.advance(_BashLexerBase.java:1642)
at com.intellij.lexer.FlexAdapter.locateToken(FlexAdapter.java:110)
at com.intellij.lexer.FlexAdapter.getTokenType(FlexAdapter.java:69)
at com.intellij.lexer.MergingLexerAdapterBase.locateToken(MergingLexerAdapterBase.java:75)
at com.intellij.lexer.MergingLexerAdapterBase.getTokenType(MergingLexerAdapterBase.java:48)
at com.intellij.lang.impl.PsiBuilderImpl.a(PsiBuilderImpl.java:270)
at com.intellij.lang.impl.PsiBuilderImpl.<init>(PsiBuilderImpl.java:191)
at com.intellij.lang.impl.PsiBuilderImpl.<init>(PsiBuilderImpl.java:158)
at com.intellij.lang.impl.PsiBuilderFactoryImpl.createBuilder(PsiBuilderFactoryImpl.java:78)
at bblfsh.bash.Parser.builder(Parser.java:56)
at bblfsh.bash.Parser.parse(Parser.java:30)
at bblfsh.bash.Driver.processRequest(Driver.java:62)
at bblfsh.bash.Driver.processOne(Driver.java:35)
at bblfsh.bash.Driver.run(Driver.java:20)
at bblfsh.bash.Main.main(Main.java:14)
Can this warning be fixed/silenced? Does the lack of this ClassDef adversely affect the generated UAST?
We use enry for language detection both on bblfshd side as well as on our applications. Enry, as well as github/linguist, do not detect Bash itself, but just Shell (this includes the whole family of POSIX-like shell languages).
This generates the following problem:
Since last night I am trying to generate tests for the bash-driver unsuccessfully.
First, my source files where .sh
, the bblfsh-sdk test
was running without errors, but no output generated.
I changed my files to .bash
and I see the followings.
TestBashDriver/semantic/bench_ethopian_multiplication (2.01s)
require.go:794:
Error Trace: fixtures.go:241
Error: Received unexpected error:
driver failure: read |0: i/o timeout
github.com/bblfsh/bash-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/native.(*Driver).Parse
/go/src/github.com/bblfsh/bash-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/native/native.go:213
github.com/bblfsh/bash-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/fixtures.(*Suite).testFixturesUAST.func1
/go/src/github.com/bblfsh/bash-driver/vendor/gopkg.in/bblfsh/sdk.v2/driver/fixtures/fixtures.go:232
testing.tRunner
/usr/local/go/src/testing/testing.go:777
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:2361
Test: TestBashDriver/semantic/bench_ethopian_multiplication
FAIL
If you think there might be something wrong with my files, I can make a PR with them.
From RosettaCode, the source code of the files I used is inUnix shell
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.