thenerd247 / javacsvreader

Java CSV reader lib for FIRST Robotics Competition robots. Using Java FRC standard API.

JAVA CSV READER Library

FOR: Java based FRC robots.
LICENSE: BSD 2-Clause.

DESCRIPTION

JavaCSVReader is a very light-weight library built for implementing a custom and standard CSV file parsing/operations. This library is also the base of the FRC-CSV API that is used for configuring FRC robots.

JAVACSV FILE FORMAT

The JavaCSV file is a custom CSV file that is used to contain multiple spreadsheets of data. Spreadsheets are grouped together in what is called a data-header (or more simply a header). Each header begins with two lines of header-metadata and ends with the last element of the data for that header. Data within a header is formatted into a table.

###DELIMITERS

In order to allow for the parser to segregate multiple headers of data and their properties each line must contain special delimiter characters to denote what data the line of text contains.

3 delimiters are used:

• Header-Delimiter: a special character to denote meta-data for a header
• Line-Delimiter: a special character to denote a row of data for a header
• Data-Delimiter: a special character to use to seperate data and meta-data

Because each user can have any kind of data stored - and thus any kind of characters - in the JavaCSV file, the delimiters can be set to any character that is unique and not found in the data or meta-data.

FILE META-DATA

On the first line of the JavaCSV file there MUST be a file meta-data sequence. This line sets the delimiters that are used in the file. The file meta-data line must go as follows:

Header-DelimiterLine-DelimiterData-Delimiter

Where is a unique character from the 3 delimiters.

Below is an example of what the file meta-data line is to look like.

|#|;|,|

In this case the "|" character is the and the 3 delimiters are in their proper order.

HEADERS

Headers are formatted to allow for custom formatting of tables. Each header must contain two lines of meta-data that are each denotenoted by the header-delimiter character at the beginning of each line. The first line of the header-meta-data contains the header's title (this is the equivalent of the spreadsheet's name). The second line of the meta-data contains the title's of the header's columns. Each column title is seperated by the data-delimiter character.

Below is an example of how a properly formatted header's meta-data should look.

#Header Title
#ColumnTitle-1,ColumnTitle-2,ColumnTitle-1

In this case the header-delimiter character is the "#". Notice how each header-delimiter character is the first character in the lines. This is a MUST. Also, notice each column title is seperated by a comma. Naturally this is obvious, however the comma could be replaced by another character (as long as it is the data-delimiter character which is a MUST).

Data

The data for a header begins directly under the header-metadata lines. Each new row of data is denoted by the line-delimiter character as the very first character in the row of data. The data in a row is seperated by a data-delimiter character.

Below is an example of how two rows of data should appear in a JavaCSV file.

;data1,data2,data3
;dataA1,,dataA3
;dataB1,dataB2,

NOTE: Data in a row does not have to fill all columns of data. As seen above column 2 was skipped in row 2, and column 3 was skipped in row 3.

Comments

Some times it's good to leave comments in the file to tell people details about the data. Comments are simple: insert a line of text whos first character does not begin with either the line-delimiter or header-delimiter character, and the comment is not found in a row of data or meta-data.
For example:

This is a comment
that spans multiple lines 
of text. As long as the first 
character in each line does not contain the header-delmiter character (#) 
or the line-delimiter character (;) the parser will ignore these lines.
#Header-Title
#Column1,Column2 
This is another comment that spans a single line
;firstData,secondData

For most programers the norm of creating comments is using the "//" or "/* */" characters for single line or multi-line comments. This also works with the JavaCSV file as long as the comments are not on the same line of text.

Below is an example of how not to use comments

#DONE PLACE COMMENTS HERE Header_Title DONT PLACE COMMENTS HERE
#Column1,Colum2 THIS COMMENT WILL BECOME PART OF Column2's title
;Data,data THIS COMMENT WILL BECOME PART OF THE LAST DATA IN THE ROW

EXAMPLE JAVACSV FILE

Below is an example JavaCSV file. It contains 2 headers of data (equivalent to 2 spreadsheets of data).

|#|;|,|

#First Header Title
#Column1,Column2
;data1,data2

#Second Header Title 
#Column1,Column2,Column3
;data,data,moredata
;evenmore data

FRC-CSV API

DESCRIPTION

The frcCSVAPI package is an api that implements robot configuration and data storage. It uses the JavaCVSReader library as an interface for low level data operations such as reading and writing data and managing headers.

REQUIREMENTS

Java FRC API 2010 or greater

USING THE FRC-CSV API

Implementing the frcCSVAPI in your FRC robot project is very easy. To allow a robot project to use the API follow the simple steps below.

1. Download and "install" frcCSVAPI package and its dependencies.

   1. To download the api go to follow [this][1] link and download the zip file available. Extract the files to your desktop or an arbitrary location.

   2. Copy the directories (or folders) located in the src directory that you extracted from the zip file from above to the location of your FRC project source code. For example: Let's say my source files for the FRC project are located in C:\Users\Developer\Desktop\frcProject\src\edu\wpi\first\wpilibj
      (That is in this directory you would find your RobotMain.java). I would copy the frcCSVAPI and JavaCSVReader directories to this directory.

   NOTE step 2 must be performed for every frc project you wish to use. If you don't want to copy the files every time you create a new project, modify your class path variable (this is different for each IDE you use) to point to the location of the src folder from the zip you downloaded in step 1.

2. Import the package into your RobotMain class and implement usage.

3. In your RobotMain class copy the below code and add it to the constructor of the class.

   //Uncomment the below line to add custom DeviceParser classes //DeviceParserManager.addParser(...)

   //read configuration data and create devices RobotConfig robotConfig = new RobotConfig();

NOTE You must call the above lines before you call methods from devices created by the RobotConfig.

USAGE FOR PROGRAMMERS

Below is some basic usage of the frcCSVAPI. For more details and documentation please see the javadoc included in the download.

ADDING PARSERS

Because each robot device has a different constructor set up, unique parser classes are needed for each device created. By default the frcCSVAPI contains parsers for the below devices. The constructors for the devices are also shown. NOTE a new parser must be created if you wish to use a different constructor for a given device.

• Accelerometer(int slot, int channel)
• Compressor(int pressureSwitch_Slot, int pressureSwitch_Channel, int relaySlot, int RelayChannel)
• Encoder(int a_slot, int b_slot, int a_channel, int b_channel)
• Gyro(int slot, int channel)
• Jaguar(int slot, int channel)
• Relay(int slot, int channel)
• Solenoid(int slot, int channel)
• Victor(int slot, int channel)

If the device or constructor for your device is not found worry not! You can make your own. Simply implement the DeviceParser interface in your custom device class, and set the name variable to "[insert name of device here]Parser". Then add the parser to the DeviceParserManager by inserting the above code BEFORE you call the RobotConfig constructor.

DeviceParserManager.addParser(new [insert name of device here]Parser());

This will add the parser to the list of parsers and allow you to add devices to the JavaCSV file that use this parser (see FRC JavaCSV file format).

GETTING A DEVICE

Once the constructor of the RobotConfig class is called all the devices found are created. To use a device created by the RobotConfig class follow the below template.

<Device Class> varName = (<Device Class>)RobotConfig.getDevice("<name of device>")

For example: let's say I want a Jaguar that is named "FrontLeftMotor". Below is the code I would use to access this device:

Jaguar flMotor = (Jaguar)RobotConfig.getDevice("FrontLeftMotor");

FRC JAVACSV FILE FORMAT (the frcCSV file)

In order for the frcCSVAPI to work a certain file needs to be downloaded onto the cRIO. This file MUST be in JavaCSV format and must be ASCII encoded. Further, the file must be specifically formatted in order for the parsers to work correctly in the frcCSVAPI. (For this documentation we will refer to this file as the frcCSV file).

First the frcCSV file must contain ALL of its device config data in a single header that has the header title "Devices". Column titles are not parsed as so can be named anything. However, the order of the data is very important.

• The first column of the header MUST be the name of the device.

• The second column MUST be the name of the parser to use for that row of data. For example: use JaguarParser if that row contains data for a Jaguar device.

• The rest of that column data in a row contains the argument values you wish to use for that device's constructor. The argument values are the values you would normally use for that device's constructor. The order, type, and amount of values are determined by the type of parser used. In the example below the JaguarParser parser is used. This is a default parser and uses this Jaguar constructor: Jaguar(int slot, int channel). So then the third column for that row would be the slot value, and the fourth column for that row would be the channel value.

Below is an example of how the device header should be formatted:

#Devices
#Name,ParserType,Arg1,Arg2,Arg3

//Jaguar devices:           slot,channel
;FrontLeftMotor,JaguarParser,1,2

//Sensors : aSlot,bSlot,aChnl,bChnl,reverse?
;Encoder,EncoderParser,1,1,1,2,true

USAGE FOR NON-PROGRAMMERS

This api has a very easy to use interface. Simply edit the frcCSV file and upload it to the cRIO and the reboot the robot! This makes it easy to change ports and other data for robot devices on the fly and makes it very usefull during competitions.

A GUI interface (ergo you don't have to edit a file anymore) can also be used to change config data.

javacsvreader's Issues

Documentation for Javadoc

If someone would like to manage the javadoc clean up behind the latest stable releases then that would be awesome. Since we are using the git-flow style of work management this should be easy. Pull requests to master on the origin repo will be reviewed. Once all the code is settled and works then we'll make the call to whom ever is handling javadoc and we'll wait on them to make the last commit to the code - which would contain the properly formatted javadoc. Once the javadoc is included then we'll accept the pull request.

Any other ideas on how we can manage the javadoc for this?

DataHeader Write/Read-ability

Now that DataHeaders can be created and read from, lets clean up the format that we read data (no more accessing data through hard coding) and let's allow users to write and save data back into a CSV file.

FRC Object Parsing

OK, so now we are on to phase 2 of the project!

Now, we are on to creating, the API for FRC Robots!

Some problems I would like to discuss are the formatting of how we are going to store data for FRC devices in the JavaCSV file.
My idea is to put all devices under one header, and create a list of parsing objects that determine how to read the rows of data and create devices from them. In each row the first two items will determine how to parse the data and store it. For example we have a row as follows:
;FrontLeftMotor,Jaguar,1,1,2

A controller will look at the second item in the list, and select a parser based upon it, to create an frc object. Example code would be :

CSVDataHeader devs = getDevices();
Vector row = devs.getRow(0);
DeviceParser parser = parserList.getParser((String)row.elementAt(1));
RobotDevice newDevice = parser.createDevice(row);
Jaguar jag = (Jaguar)newDevice.getDevice();

Just a thought. Any other ideas?

Versions

Any ideas on how we should do the version-ing of our code?

Data Comprehension

CSV is becoming to simple: Possibly going to create a custom "config language" for data comprehension.
Considered options of understanding data: multiple headers in one file to break down the data.

License For this LIB

I realized the other day that this code has yet to be brought under license. As seen the master branch contains no deployable code yet I think we should let this slide for now. However the FIRST merge to the master branch must contain an open source license.

PS: I suggest GNU Public License.

FRC-Based GUI

I figured that the GUI support for the CSVReader lib should be restricted to FRC support only. Editing non-FRC based files should be via a text editor. This keeps things simple and easy to use. FRC-based GUI will allow users to:

-change the ports of installed devices on the robot.
-add properties for robot sensors (such as distance per count for rotary encoders)
-.....possibly adding new devices to the robot ---> this will need an extension the library of the JavaCSVReader.

The interface will include a list of the different devices, and other details that will be changeable. Appropriate interfaces will be created for these

GUI Structure for Users

OK, now that data can be written to a CSV file and read and manipulated, it's time to work on an end user interface for managing the data in a CSV file WITHOUT having to know the Java Primitive data types or the file formatting.

Somethings to note: -Simplicity: we want the users to not get scared when it comes to managing data
-Speed: a relatively lightweight processing power consumption for this utility
-Sharp Looking: lets make our GUI look and feel like a pro.

CSV Data Header Title

I noticed some problems involving an end programmer interface for accessing the data. First is that the programmer needs to know how the data is placed in the CSV file. Also, the names of two headers could be the same for instance you could store students in one header with the data formatted as: (String)Name,(int)Age and a header for teachers in the same way. With the current system there is no way to tell the difference between the two.

So then here are the suggested steps we take in making this change:

-CLEANUP: field names throughout all the files. The meaning of "header" seems to have changed its meaning. It should mean: a group of related data. --->see README for changes
-CLEANUP: the format of the CSVDataHeader class --> make things nice and clean: data in one field, data types in another, names of columns in another.

-ADD: field in CSVDataHeader to contain the Title for a header.

Data Container

The way to go from CSV to String[][] is done. Now to go from String[][] to some kind of sorted data structure that will hold all of the data. Hashtable sounds like a good plan.

NOTICE: Stable Code for Master

Stable code for master is being pulled together from the various branches on the origin (theNerd247/JavaCSVReader) repo. Please wait just a bit longer.

Clean up Hierarchy

CSVFile.java needs to be refactored into a separate util class for working with strings.

Universal CSV Compatibility

Moved the compatibility problem from isssue #8 to its own issue.

To recap: need methods in CSVFile that make it able to read/write to a standard CSV file - this should create only one header in the file as seen 1 CSV file only holds 1 sheet full of data.