Another subset of #43! Covers:

• Abs(x as Float) as Float
• Cdbl(x as Integer) as Float
• Cint(x as Float) as Integer
• Csng(x as Integer) as Float
• Fix(x as Float) as Integer
• Int(x as Float) as Integer
• Sgn(x as Float) as Integer
• Sgn(x as Integer) as Integer

The Reference BrightScript Implementation (RBI) allows the next keyword to be used as a synonym for end for as best I can tell. brs doesn't support that right now, but it definitely should

Broken snippet:

for each num in [1,2,3]
    print num
next

Originally reported by @TwitchBronBron. Thanks friend!

The Reference BrightScript Implementation allows as object to be used as a type suffix for function arguments and function returns, but it currently causes a parse failure here:

Failing file:

sub Main()
    test(invalid)
end sub

sub test(foo as object)
    print "if you can read this, the bug has been fixed"
end sub

Main()

Failed output:

$ brs ./object-mwe.brs
[Line 5] Function parameter [object Object] is of invalid type 'object'
[Line 7] Found unexpected token 'end sub'
Error occurred

See #30 (comment) for details, but exitfor needs to be a valid variable identifier and not an alias for exit for. This'll be interesting to fix.

It's currently possible to crash the entire interpreter by calling a function with the wrong argument types, e.g.:

• attempting to lowercase a non-string: LCase(false)
• attempting to uppercase a non-string: UCase(invalid)

Rather than crashing with an esoteric JavaScript stack trace, we should provide a helpful error message that tells users which arguments are incorrectly typed and in which function.

From the Roku Developers slack workspace:

Georgejecook [1 minute ago]
perhaps you (or some other generous soul) can add it to the vscode ide?
I can see real value in that.
see https://github.com/TwitchBronBron/vscode-brightscript-language/

I'm not super sure what all this would involve (or even if this would happen in a separate repo?), but it sounds useful!

Right now when the interpreter fails, it's not clear where it failed and how it got to that point. The brightscript stack trace format exposes the current call stack as well as the local scope. We should do the same.

Example crash from "hello, world" application (ignore thread-related output):

BrightScript Micro Debugger.
Enter any BrightScript statement, debug commands, or HELP.

Suspending threads...
Thread selected:  1*   pkg:/components/helloworld.xml(40)      y()

Current Function:
038:    sub foo()
039:       x = 3
040:*      y()
041:    end sub
Function Call Operator ( ) attempted on non-function. (runtime error &he0) in pkg:/components/helloworld.xml(40)
040:      y()
Backtrace:
#1  Function foo() As Void
   file/line: pkg:/components/helloworld.xml(40)
#0  Function init() As Dynamic
   file/line: pkg:/components/helloworld.xml(30)
Local Variables:
global           Interface:ifGlobal
m                roAssociativeArray refcnt=3 count:3
x                Integer val:3 (&h3)
y                <uninitialized>
Threads:
ID    Location                                Source Code
 0    pkg:/source/Main.brs(17)                screen.show()
 1*   pkg:/components/helloworld.xml(40)      y()
  *selected

The index.d.ts file included in the npm module [email protected] includes some references to Users/sean.barag

The interpreter will need to be able to process multiple files within the same scope. For example, a function defined in file A should be callable in file B after file A has been executed.

$ brs test1.brs test2.brs ...

Errors during runtime should refer to the file where that function/expression were defined.

It's not used super often from what I can tell with a quick Github search, but its availability allows for some neat use-cases. It's a nice to have, but should enable some deep unit testing of brightscript if used appropriately!

Docs for the RBI version: https://sdkdocs.roku.com/display/sdkdoc/Runtime+Functions#RuntimeFunctions-Run(filenameasString[,Args%E2%80%A6])Asdynamic

In the Reference BrightScript Interpreter (RBI), the following is not valid:

(sub()
  print "this is immediately invoked!"
end sub)()

But it is valid in brs. As it turns out, RBI doesn't support parenthetical "groupings" to be part of expression statements. A simpler case demonstrates this well:

(2 + 3)

That's valid in JavaScript, as it's modeled something like:

ExpressionStatement (
    Group (
        Add (
            lhs: Literal(2)
            rhs: Literal(3)
        )
    )
)

It seems like function calls are the only things that are allowed to appear in expression statements, as the following is also invalid in RBI, but valid in brs and node

a = 5
a

The REPL works on only single-line input. This is practical for most expressions, but prevents many multi-line, valid code blocks to be entered. For example, multi-line bodies do not work.

brs> x = {
[Line 1] (At end of file) Expected identiier or string as associative array key, but received ''

Ideally a user could enter code line by line similar to other REPLs like python:

>>> x = {
...     "a" : 3
... }
>>>

I imagine this is accomplished by generating AST line by line, but only actually executing the code (or failing) if the AST input is at the top of a tree (for example, has a closing bracket or "end sub").

Unlike JavaScript (which allows return without a value that implicitly returns undefined), BrightScript in the reference implementation requires a value to follow a return statement within a function that doesn't return void. BRS should do the same!

RBI error

Return must return a value. (compile error &ha9) in pkg:/source/mwe.brs(79)

MWE

function bar()
    return ' currently succeeds; this should fail
end function

sub main()
    bar()
end sub

StrToI is a global "utility" function related to string parsing we should implement to match the spec (https://sdkdocs.roku.com/display/sdkdoc/Global+Utility+Functions#GlobalUtilityFunctions-StrToI(strasString)asDynamic).

• StrToI(str as String) as Dynamic

Stack trace:

$ brs
module.js:549
    throw err;
    ^

Error: Cannot find module 'node-int64'
    at Function.Module._resolveFilename (module.js:547:15)
    at Function.Module._load (module.js:474:25)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> ($HOME/.local/lib/node_modules/brs/lib/lexer.js:3:15)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)

Repro steps: yarn global add brs && brs

Versions:

• node: 8.11.2
• yarn: 1.6.0
• Mac OS X: 10.12.6

Implement the global utilities related to file reading and writing (https://sdkdocs.roku.com/display/sdkdoc/Global+Utility+Functions).

• ReadAsciiFile(filepath as String) as String
• WriteAsciiFile(filepath as String, text as String) as Boolean

A subset of #43! Covers these functions:

• Atn(x as Float) as Float
• Cos(x as Float) as Float
• Sin(x as Float) as Float
• Tan(x as Float) as Float

It's become obvious that keywords and reserved words receive separate treatment within the Reference BrightScript Implementation (RBI), e.g.:

1. The as keyword is not a reserved word, so is a valid variable name.
2. The Type reserved word is not a keyword - just a reserved identifier.

Since these properties seem to be able to vary independently, it makes sense to handle them separately within the lexer and parser created here.

• Reserved words should be lexed as regular identifiers. Handling of reserved word usage can be handled in one of two ways:
  • (Preferred, and probably simpler) Identifiers (see Token.ts) gain a new boolean member isReserved; the interpreter simply checks for truthiness of that value.
  • (Slower and likely involves lots of repeated logic) The interpreter checks all identifiers against the reserved word list at each use.
  • Example: type("foo") gets lexed into something like [ Identifier("type"), LefParen, Literal("foo"), RightParen ]
• Keywords are lexed into Tokens with the kind property equal to their own Lexemes.
  • Example: foo and bar gets lexed into something like [ Identifier("foo"), And, Identifier("bar") ]

We've already got four of these, but it'd be great if we had a few more! Specifically, there's a bunch of global string functions in BrightScript:

• Instr(start as Integer, text as String, substring as String) as Integer
• Left(s as String, n as Integer) as String
• Len(s as String) as Integer
• Mid(s as String, p as Integer, [n as Integer]) as String
• Right(s as String, n as Integer) as String
• Str(value as Float) as String
• StrI(value as Integer) as String
• StrI(value as Integer, radix as Integer) as String
• String(n as Integer, str as String ) as String
• StringI(n as Integer, ch as Integer) as String
• Val(s as String) as Float
• Val(str as String, radix as Integer) as Integer
• Substitute(str as String, arg0 as String, arg1 = "" as String, arg2 = "" as String, arg3 = "" as String) as String

Implement the global utilities related to file operations besides read/write (https://sdkdocs.roku.com/display/sdkdoc/Global+Utility+Functions).

• ListDir(path as String) as Object
• CopyFile(source as String, destination as String) as Boolean
• MoveFile(source as String, destination as String) as Boolean
• DeleteFile(file as String) as Boolean
• DeleteDirectory(dir as String) as Boolean
• CreateDirectory(dir as String) as Boolean
• FormatDrive(drive as String , fs_type as String) as Boolean

FormatDrive() should be stubbed out as we shouldn't ever implement it.

This line seems to cause a parse failure! Looks like we're missing support for : to separate things where \n is expected.

fn = Function(x): return x: End FUnction

The BrightScript language supports 17 global math functions:

• Abs(x as Float) as Float
• Atn(x as Float) as Float
• Cdbl(x as Integer) as Float
• Cint(x as Float) as Integer
• Cos(x as Float) as Float
• Csng(x as Integer) as Float
• Exp(x as Float) as Float
• Fix(x as Float) as Integer
• Int(x as Float) as Integer
• Log(x as Float) as Float
• Rnd(range as Integer) as Integer
• Rnd(0) as Float
• Sgn(x as Float) as Integer
• Sgn(x as Integer) as Integer
• Sin(x as Float) as Float
• Sqr(x as Float) as Float
• Tan(x as Float) as Float

We should get those implemented! Luckily they seem to all be wrappers around JavaScript's Math native functions, so these should be relatively easy to implement as native functions. Also luckily: they all operate on either Float or Int32, so we shouldn't have to worry about dealing with 64-bit integer math!

Brightscript implements a couple functions for working with JSON (https://sdkdocs.roku.com/display/sdkdoc/Global+Utility+Functions#GlobalUtilityFunctions-ParseJson(jsonStringasString)asObject). We should implement both of them as they heavily used in a typical codebase.

• ParseJson(jsonString as String) as Object
• FormatJson(json as Object, flags = 0 as Integer) as String

RBI doesn't allow subroutines (functions declared with sub) to return values, but this project does!

RBI's error

Return can not have a return-value if inside a Sub or Function with Void return type. (compile error &haa) in pkg:/source/mwe.brs(73)

MWE

sub foo()
    return "I break in RBI!"
end sub

sub main()
    foo()
end sub

The brightscript roDateTime object and the methods associated with it are used often in many codebases, so we should implement it.

https://sdkdocs.roku.com/display/sdkdoc/ifDateTime

Not having the column numbers and file locations for tokens was fine when things were only located in single (small) files, but the inputs to brs can now be much more substantial! To make the parser (or interpreter)'s output more meaningful to users with multiple long files, we need to report with more granularity where errors occurred.

I kind of wish these fit into a better bucket, but they don't. See the BrightScript documentation for details on what each one does, but this should cover:

• Instr(start as Integer, text as String, substring as String) as Integer
• Len(s as String) as Integer
• String(n as Integer, str as String ) as String
• StringI(n as Integer, ch as Integer) as String
• Substitute(str as String, arg0 as String, arg1 = "" as String, arg2 = "" as String, arg3 = "" as String) as String

A user of brs won't care where in the language the error came from - what they actually want to know is that the file they tried to execute doesn't exist.

Let's give them a good error message - let's look to node, python, cat, ls, git, and a few other tools for precedent. This should be pretty simple with fs.exists either way though.

Repro Steps

$ brs does-not-exist
/home/sean/src/brs/lib/lexer/index.js:47
    return current >= source.length;
                             ^

TypeError: Cannot read property 'length' of undefined
    at isAtEnd (/home/sean/src/brs/lib/lexer/index.js:47:30)
    at Object.scan (/home/sean/src/brs/lib/lexer/index.js:31:13)
    at run (/home/sean/src/brs/lib/index.js:39:26)
    at ReadFileContext.fs.readFile [as callback] (/home/sean/src/brs/lib/index.js:13:9)
    at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:420:13)

Per BrightScript's documentation (emphasis mine):

A function is anonymous if it does not have a name. Note that Anonymous Functions do not currently create closures.

Consider the following BrightScript file:

function Main()
    a = "before"

    showA = sub()
        print a
    end sub

    showA()

    a = "after"
    showA()
end function

Main()

The reference implementation prints

<UNINITIALIZED>
<UNINITIALIZED>

which indicates that the value of a wasn't captured by the creation of the anonymous function showA. The current brs implementation does capture the value of a:

before
after

While closures are certainly useful, their presence in brs will lead to this project supporting an incompatible superset of the reference implementation. Programs that execute as-expected in brs will behave unexpectedly in the reference implementation due to its different scoping rules, and vice-versa.

See the BrightScript documentation for details on what each one does, but this should cover:

• GetInterface(object as Object, ifname as String) as Interface
• FindMemberFunction(object as Object, funName as String) As Interface

This is currently blocked until objects are implemented, but when implemented objects one should keep this function in mind as it will need access to which interface a function came from.

See the BrightScript documentation for details on what each one does, but this should cover:

• Sleep(milliseconds as Integer) as Void
• Wait(timeout as Integer, port as Object) as Object

This will require the roMessagePort object to be implemented first.

Blank lines in a source file shouldn't crash the CLI (they don't crash the REPL, surprisingly). Instead they should do nothing, like they do in BrightScript.

Repro Steps

user@hostname$ cat has-blank-line.brs
a = 1
b = 2

print a + b

user@hostname$ brs has-blank-line.brs
Error: [line 4] Found unexpected token '
'
    at runtime (/home/sean/src/brs/lib/Error.js:11:17)
    at Object.make (/home/sean/src/brs/lib/Error.js:6:12)
    at Object.make (/home/sean/src/brs/lib/parser/ParseError.js:10:14)
    at primary (/home/sean/src/brs/lib/parser/index.js:141:30)
    at unary (/home/sean/src/brs/lib/parser/index.js:123:12)
    at exponential (/home/sean/src/brs/lib/parser/index.js:109:16)
    at multiplicative (/home/sean/src/brs/lib/parser/index.js:100:16)
    at additive (/home/sean/src/brs/lib/parser/index.js:91:16)
    at relational (/home/sean/src/brs/lib/parser/index.js:81:16)
    at boolean (/home/sean/src/brs/lib/parser/index.js:72:16)

Yet another subset of #43. Covers:

• Exp(x as Float) as Float
• Log(x as Float) as Float
• Sqr(x as Float) as Float

And these two non-multiplicative functions because I don't know where else to put them 🤷‍♂️

• Rnd(range as Integer) as Integer
• Rnd(0) as Float

BrightScript's print statement is surprisingly powerful. It supports tabulated output (via print "foo", "bar", "baz"), where a comma prints the next expression at the next 16-character "tab stop". It also supports printing multiple expressions with no space in between with a semiclon (e.g. print "hello"; "world" ' prints 'helloworld'). It's all rather complicated.

https://sdkdocs.roku.com/display/sdkdoc/Program+Statements#ProgramStatements-PRINTitemlist has the official docs. I'm not sure if fixing this issue requires pos(x) and tab(expression) to also be implemented, since they're "global functions" but are probably only valid within an print statement.

I should have set one of these up a long time ago to keep myself honest, but it's even more important if others are contributing! The goal here is explicitly not to enforce a consistent style (besides a few basics), but rather to prevent errors in the future. Some rules to consider:

• Use spaces for indentation
• Require semicolons (only because JavaScript is really weird with that, and being explicit helps a lot)
• Use === for comparisons (with an exception for foo == null as shorthand for foo === null || foo === undefined)
• Prevent eval
• Maybe something for unused locals if the compiler doesn't already handle that?
• 🤷‍♀️
• 🤷‍♂️

Turns out orbs already exists - I could have sworn it didn't! It's got extremely low download rates, so I might be able to ask for a name transfer, but I should prepare for changing the name.

Suggestions are welcome!

Right now if the brightscript runtime executes a file, it will execute every expression it finds outside of a function. This doesn't match the brightscript implementation where expressions must be inside functions.

...
print "running something"

sub Main()
    print "in showChannelSGScreen"
    ...

which raises a syntax error the following error:

Syntax Error. (compile error &h02) in pkg:/source/Main.brs(10)

Should we match the device implementation and look for a main function, or should we enable a flag to toggle it so we can still run expressions in the REPL?

While running latest brs with manifest support I've found some properties can't be parsed correctly:

• Relative paths to images like mm_icon_focus_hd=pkg:/images/myIcon.jpg
• Hex color codes splash_color=#000000 (interpreter thinks is conditional compilation)
• Special resolution values uri_resolution_autosub=$$RES$$,720p,720p,1080p

My suggestion is to have a list of ignored properties to fix this.

In RBI, end function and End Function are supposed to be equivalent. For some reason, using End Function in brs causes a parse failure.

ifArrayJoin includes the join() function on roArray, which is an important and often used function in many codebases.

https://sdkdocs.roku.com/display/sdkdoc/ifArrayJoin

I'm using the Lexer and Parser classes to try and validate a brightscript file. How can I get a list of parse errors?

I'm doing the following in TypeScript:

import { Lexer, Parse } from 'brs';
let tokens = Lexer.scan(text);
let statements = Parser.parse(tokens);

Problem is, it just seems to eat any errors it encounters. For example, based on this line in Parser.ts, there should be a parse error thrown whenever you declare a sub with an as clause.

Here's a very simple brightscript file that should have at least the above mentioned parse error

sub SayHello() as string 
    print "hello world"
end sub

I don't get any exceptions thrown...the statements array contains a single entry, representing the print line. If I remove the as string, then the statements array has one entry at the top level represneting the SayHello function (as expected).

What I am hoping to get is a list of parse errors, and their line numbers (which in this case would be 1 parse error, something to the effect of

{
    error: `"'Sub' functions are always void returns, and can't have 'as' clauses""`, 
    line: 1,
    column: 14,
    file: "theFile.brs"
}

Is this functionality available? If not, do you think it's do-able?

Implement the global utility MatchFiles to fetch files by pattern (https://sdkdocs.roku.com/display/sdkdoc/Global+Utility+Functions#GlobalUtilityFunctions-MatchFiles(pathasString,pattern_inasString)asObject).

The only way to avoid invalid dereferences is to ensure something isn't invalid, which means comparing against it. Since everything is comparable to invalid, the current logic in https://github.com/sjbarag/brs/blob/128d5217e3d5c8592c04c7ff0d01d0c7165e14fd/src/brsTypes/index.ts#L79-L82. It'll probably have to be not a typeguard, as everything can be compared to invalid. @strattonbrazil you were right!

The architecture detailed in Crafting Interpreters uses a Resolver layer to handle static variable binding. That allows a level of lexical analysis (e.g. overwriting global functions) that's currently only possible in the Interpreter layer of this project.

Consider adding a Resolver or something so that deeper analysis can be performed. The good news is it shouldn't affect runtime performance significantly since it doesn't have a lot of work to do!

BrightScript values including uninitialized variables are all valid parameters to the global Type function that returns a string describing the variable's type. Note that there's an optional second parameter, but it's not super clear how that affects the output. It seems like type(foo, 3) returns a BrightScript Component name, and type(foo, 2) doesn't? I'm not entirely sure yet, but this will likely require a change to the BrsValue interface to ensure all values are capable of handling this on their own.

Reference Implementation docs

depends on #79

See the BrightScript documentation for details on what each one does, but this should cover:

• Left(s as String, n as Integer) as String
• Mid(s as String, p as Integer, [n as Integer]) as String
• Right(s as String, n as Integer) as String

The RBI docs mention that then is optional! The current implementation requires then after an if statement.

The official BrightScript docs specify that the following is valid syntax:

add = function(a, b)
    return a + b
end function

print add(1, 2)

Let's make that work.

Attempting to assign an anonymous function to a property of an associative array seems to cause a parse failure! Originally reported by @TwitchBronBron – thanks dude!

Here's another fun one (throws "Found unexpected token 'function'")

o = CreateObject("roAssociativeArray")
o.getSomething = function()
    return "something"
end function

but this works fine (define function and assign it on separate lines)

    o = CreateObject("roAssociativeArray")
    action = function()
        return "something"
    end function
    o.action = action

See the BrightScript documentation for details on what each one does, but this should cover:

• Str(value as Float) as String
• StrI(value as Integer) as String
• StrI(value as Integer, radix as Integer) as String
• Val(s as String) as Float
• Val(str as String, radix as Integer) as Integer