drksephy / pascal-compiler Goto Github PK

View Code? Open in Web Editor NEW

21.0 21.0 7.0 940 KB

Repository for a pascal compiler built using Python

License: MIT License

Pascal 2.28% Python 97.72%

pascal-compiler's Introduction

pascal-compiler

A Pascal compiler built using Python.

Restrictions

As a means to learn, no libraries will be used for heavy lifting of the implementation of the scanner, parser or abstract syntax tree. The only libraries being used within this project are:

Pretty Tables : Library for printing ascii tables, useful for formatting debug statements and generated tokens to console.

Progress

The following is a list of features that have been implemented.

###Compiler Research Links

######General

######Compiler Theory

######Parsing Methods and Grammar theory

######Bytecode

Python Disassembly

Compiling Pascal on Mac OS X

If you do not want to use an IDE to write and compile Pascal code, it is highly recommended to simply compile pascal programs through the command line. To do so, follow these steps.

Download 'fpc' (Free Pascal Compiler) here. If running on an intel-based Mac, download and installfpc-2.6.0.intel-macosx.dmg, and if you are on a PowerPC Macbook, download and install fpc-2.6.0.powerpc-macosx.dmg.

This should be all you need. Given the following sample program, hello.pas:

program HelloWorld;
uses crt; 

(* Here the main program block starts *)
begin
    writeln('Hello World!');
end.

You can compile hello.pas using fpc hello.pas on the command line, producing hello.o and hello. You can now execute the executable using ./hello, printing 'Hello World!'.

LICENSE

The MIT License (MIT)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

pascal-compiler's People

Contributors

Stargazers

Watchers

Forkers

paulodiniz wangchengyu coderzzg skeptycal yitaodong sahwar programmermohr hanyedeleng

pascal-compiler's Issues

Store integers

Real integers need to be stored in the table of tokens, as TK_REAL_INT.

Write methods for visualizing Parse Tree

Before implementing more Grammars, it would be useful to be able to print out the parse tree using no outside libraries.

Create byte array

Using the output of the parser, a byte array needs to be built containing the addresses and values of the nodes.

Investigate how to jump to future address

When implementing if/else statements, we need to do a forward jump to an address which we do not know yet. To do this, we need to look at "patch-ups".

Generate tokens in scanner

scanner.py is able to scan a <target program> character by character. Next it needs to produce valid tokens.

Write tests for scanner

Once the scanner nears completion, tests should be written to verify functionality, and to ensure the scanner throws errors on strings it cannot tokenize.

Create function for expected token

A function should be created which takes in a specific token as input. This is useful for debugging and throwing errors if the wrong token is consumed.

Investigate parenthesized expression

According to the grammar,

2 < 1 and 1 < 3

Works fine due to precedence. However, the following should work as expected:

( 2 < 1 ) and (1 < 3 )

Currently, errors are thrown when processing these tokens.

Create lexer

A lexer needs to be written in order to scan the target program and create tokens.

Invalid tokens

Scanner should fail for invalid tokens such as /*abc, 123abc, ****, ", (, '.

Investigate generating Bytecode Python

In order to execute our input program, we have to compile it into the appropriate form. Currently stack machine ASM is being generated from the parsing. The other options are:

* Generate x86 Assembly and output it to a file to be compiled by TASM/MASM. 
* Generate bytecode to be executed by the simulator.

Of these methods, we will focus on building a bytecode simulator. Research needs to be done to figure out how to generate bytecode using Python and to execute it.

Create parser

After creating the tokens using a lexer, we need to build a parser to consume the tokens and create an Abstract Syntax Tree.

Build simulator

The simulator main loop needs to be implemented. Pseudo-code of the simulator is shown below:

void run( ){
    ip = 0;
    while(1){
        switch(code[ip]){
            .
            .
            .
            // Populate all cases of instructions
        }
 }

Parser must handle for statements

Similar to if and while statements, the parser must be able to build parse trees of for statements.

Update parser to do type checking

In order to properly check types of operands, the parser grammar needs to be reimplemented and combined into a while loop as opposed to using Tail Recursion.

Create lexer driver

A "driver" needs to be implemented to run the lexer to scan Pascal code.

Implement for statements

For statements need to be implemented for the Compiler

Implement if-then-else statements

If-then-else statements need to be implemented.

Create system table

Scanner needs to be able to read system functions, such as writeln.

Begin writing parser class

A parser needs to be written in order to produce an AST for a given program.

Build symbol tables

Whenever assignments occur within the parsing segment, a symbol table containing all variables, types and number of bytes needed to store it must be built.

Implement switch statements

Similar to if/else statements, switch statements need to be implements.

Test more complicated programs

More complicated programs such as factorial or fizzbuzz should be implemented to further test the compiler.

Create parser driver

A driver needs to be written to consume all the tokens output by parsing the target program.

Create library for building the AST

After the parser is built, we need to build a library that creates the AST for us and determines how the program will be run.

Create function for grabbing one token at a time

Instead of using a for loop to iterate over all tokens, a function should be written which grabs one token at a time.

Implement decorated grammar for basic arithmetic

Currently, the basic arithmetic parser is almost done. Now we need to implement the decorated grammar so that we can generate the stack machine code.

Store strings in table

Any proper strings need their values retained and need to be saved as TK_STRING.

Store floats

Floats need to be stored within the token table as well as real ints.

Implement repeat statement

Repeat statements need to be implemented, as they are the easiest of all loops/iterations to implement.

Generate metadata for better print statements

By storing curr_col, curr_row, curr_token, and tokens, we can build better debug statements and outputs.

Build Symbol Table

Symbol table must be built which contains all variables and assignments, and possibly scope as well.

Tokenize more symbols

Scanner currently does not work for ', ", (, ).

Build parser for arithmetic

Now that the grammar has been obtained for handling arithmetic, the parser must be built up. The goal is to be able to parse and evaluate expressions such as 2 * 4 / 5.

Re-implement repeat/while statements

The current structure of repeat/while statement implementations is unstable and does not extend to having multiple statements within.

Extend grammar to have assignment

The current grammar needs to handle assignment of variables in order to move forward with the simulator.

Detect carriage return in scanner

Given the following code:

var  
age : integer;

The scanner cannot detect carriage return, and will only work if there are manual spaces after var.

Implement while statements

While statements for the compiler need to be implemented.

Create parser error handler

A function should be built which can be called with the expected token as input. This function will throw the proper errors to the user. For instance, if we are expecting an operator token but instead encounter an assignment token, the assignment token would be thrown as the error.

Different implementations for lexer

Instead of using regular expressions to match keywords, it might be worth considering looking into using a state machine to do this. Regex take much longer in terms of running time to search a string and capture the input, so it might be worthwhile in the long run to swap in a new method for generating tokens.

Scan comments

When scanning comments, we need to ignore all of the comment within them until we break out of the comment state.

Create method for code generation

A method which will handle pushing elements onto the stack and carrying out operations needs to be implemented. For starters, we would like to implement stack machine assembly. Given the following program:

a * b + c * d

The stack machine code generated is:

push a
push b
mul
push c
push d
mul
add

Grammar for handling arithmetic

In order to build expressions within the parser for handling arithmetic, an unambiguous grammar must be constructed.

Implement functions for stack

All assignments and integer values need to be pushed and popped off the stack. Methods are needed for this.

Implement Writeln( )

Out of all the functions/procedures/system calls discussed in class, implementing Writeln( ) is important for being able to print to the console.

Implement op codes

In order to perform operations on the byte arrays generated by the parser, op codes need to be implemented to simulate hardware. Some instructions of importance are:

pushi
push
add
sub
`mult
pop

Parser must handle while statements

Similar to if statements, the parser must be able to properly build parse trees for while statements.

Implement arrays

The last feature planned before the end of the semester is to implement array operations, such as being able to print out the values of an array.

Tokenize EOF character

The currently method of reading lines inside of scanner.py does not show an end of file character. Inside of emulator.py, the EOF token is manually appended to the end of the list of tokens after scanning is complete, however a better method to handle this should be done.

Parser must handle if statements

The parser must be able to handle if statements, which includes having a proper grammar as well as handling "jumps". Jumps lead to "holes" which need to be patched using the instruction pointer.