s92025592025 / checkstyle-idea Goto Github PK

Users should be able to type in the first couple letter of the rule to search

For big long preview, the current implementation doesn't allow scrolling

(1/4 pts) You can follow the instructions to reproduce the results in the paper.
The instructions in week10/report.pdf say to use Python 3 but I found that generate_figures.sh actually uses Python 2.7. Attempting to run the original with python 2.7 and changing the command to use python 3 yielded the same error: i.e. that jpg files are not supported.

Traceback (most recent call last):
  File "./reports/week10/evaluation/generate_figures.py", line 209, in <module>
    main()
  File "./reports/week10/evaluation/generate_figures.py", line 146, in main
    plot_time_to_complete(time_to_complete_dict(ttc), path.join(directory, 'figures'))
  File "./reports/week10/evaluation/generate_figures.py", line 71, in plot_time_to_complete
    plt.savefig(path.join(out_dir, 'average_completion_time.jpg'))
  File "/home/eric/.local/lib/python2.7/site-packages/matplotlib/pyplot.py", line 695, in savefig
    res = fig.savefig(*args, **kwargs)
  File "/home/eric/.local/lib/python2.7/site-packages/matplotlib/figure.py", line 2062, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "/home/eric/.local/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 2173, in print_figure
    canvas = self._get_output_canvas(format)
  File "/home/eric/.local/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 2105, in _get_output_canvas
    .format(fmt, ", ".join(sorted(self.get_supported_filetypes()))))
ValueError: Format 'jpg' is not supported (supported formats: eps, pdf, pgf, png, ps, raw, rgba, svg, svgz)

After changing all occurrences of jpg to png, the figures were reproduced on both python 2.7 and python 3, except for average_completion_time which appears to be different from the one present in the report:

(2/2 pts) These instructions are fully automated.
Besides the jpg error, the script does produce the results automatically. No further input is needed.

(4/4 pts) The code and experimental results are well documented.
All of the code appears to be well documented with ample javadoc style comments for methods and classes as well as comments for fields and within methods, when necessary. The experiments are well documented and it is clear who the participants are and what questions/tasks were used. The analysis of the results is also good: there are many graphs to clearly show the results, and the discussion focuses on both the supporting and opposing evidence and draws conclusions from the results appropriately, as well as mentioning future improvements based on the results.

(1/1 pt) The report or the README contains instructions to reproduce the initial results.
-The instructions are located in the Notes/InitialResultInstruction.md file of the init-result branch. The report points to this branch/file for instructions to reproduce results.
(3/3 pts) The initial results are reproducible.
-I was able to reproduce the demos of the XML read/write tool and the GUI prototype
(2/2 pts) The report has a discussion of the results.
-The report does have a discussion of the results to far
(0.5/1 pt) The report expands on the methodology and is complete.
-The project has changed, so the methodology is different from previous reports. The report is mostly complete, but does not contain any evaluation information/results.
(1/1 pt) The schedule is updated and there is a reasonable discussion on why it changed.
-The schedule has changed from the previous report to reflect the new project focus. The feedback section does discuss the change in the project.
(1/2 pts) The user manual is updated.
-The manual appears to be the original Checkstyle-IDEA manual, but with a small section added for generating configuration files. This is difficult to gauge as the project has changed, so the old manual is no longer applicable, and because the team is building upon another existing tool with its own manual.

In your instructions in the MD file for reproducing the results, there are hotkeys given for Windows. It might help to instead explain how to navigate the menus (for example, I am using a Mac so on step 3 I instead found the settings by going to “IntelliJ IDEA”->”Preferences”)

In your instructions for reproducing the GUI in your report, on step 3, instead of ‘./gradlew.sh runIde’ I had to run ‘./gradlew runIde’ for it to work.

In the report, under the “Implementation Plan” section, the description of how users interact with your GUI is a bit confusing. You might want to find a way to break that up somehow into smaller/easier to follow steps by concisely listing what the various buttons do.

(0/1 pt) The report or the README contains instructions to reproduce the initial results.
Neither the latest version of the report (week 7 as of 2/28/19) or the README contain information about results or how to produce them
(0/3 pts) The initial results are reproducible.
There was no information provided on how to reproduce any results
(0/2 pts) The report has a discussion of the results.
Because this project focus has shifted, there were no results yet to discuss in the latest report
(0/1 pt) The report expands on the methodology and is complete.
This is difficult to gauge. As the project has changed, this report is a re-write, so it does not expand on any previous methodologies.
(1/1 pt) The schedule is updated and there is a reasonable discussion on why it changed.
The schedule has changed from the previous report to reflect the new project focus. The feedback section does discuss the change in the project.
(1/2 pts) The user manual is updated.
The manual appears to be the original Checkstyle-IDEA manual, but with a section added for generating configuration files. Again, this is difficult to gauge as the project has changed, so the old manual is no longer applicable.

In the report, under the “Implementation Plan” section, the description of how users interact with your GUI is a bit confusing. You might want to find a way to break that up somehow into smaller/easier to follow steps by concisely listing what the various buttons do. Also, it’s a bit unclear what freedom users have when defining their own custom rules.

Should have a feature to show XML String for the controller to show them as preview

I can't get onboard with https://github.com/s92025592025/checkstyle-idea/blob/init-result/Notes/InitialResultInstruction.md

(1/1pt) overall organization is sensible
-The report was organized well.
(1/1pt) introduction motivates the problem and describes the approach
-The Background/Problem/Solution sections combine well to describe the issue and the general approach.
(1/1pt) the technical details of the approach are well-described
-The architecture diagram and its description are very helpful for understanding the technical details.
(1/1pt) the report contains a clear description of a reproducable methodology
-I think the overall methodology would be reproducable given the explanation of the solution and architecture
(0.5/1pt) results are presented clearly and described in text
-I think the Initial Results section is mostly good, but could use a little more evaluation of the results so far (given that your project recently changed, are you satisfied with your initial results so far? Why/why not? etc.)
(1/1pt) the report describes related work and contains academic references
-The related work seems like reasonable examples. The bibliography includes a paper by Christakis and Bird but I couldn’t find where it was referenced in the document (I might have missed it, otherwise referencing its bibliography entry in the report might be a good idea to help support the corresponding idea/claim in the report)

Your explanations of the flow of user actions and the GUI mockups to go with them improved a lot since the last report, and it was easy to follow along with the steps. I like the minimalist layout of the GUI, it seems like it would be very easy to use. I think your idea of evaluating partly by having the Checkstyle-IDEA team check/review your work is a clever idea.

I like the lessons learned section. It’s obvious your group had multiple difficulties, but you were able to reflect on those difficulties and articulate what you’ve learned throughout this process.

In your Initial Results section, one of the steps notes to run “.\gradlew runIde”, where it should be “./gradlew runIde” (the slash was flipped). Also the step that says “Open an IntelliJ project in the IDE” could use a little extra explanation. I had no projects to import so I had to create a new one, but I had to wade through a couple of errors for a minute before I learned what settings to use in the window for creating a new project.

As for your GUI, on the screen where the user configures a rule’s parameters, it might help to change the “OK” button to instead say “Add to Active Rules” or something along those lines to be more explicit.

In your Implementation Plan section, you mention you evaluate/claim the success of your XML reader/writer by comparing its conversions between actual/easier to understand formats, I think this would be better to have in your Evaluation section instead.

Also with regards to user testing to determine the success of your project, why are you only going to use TAs to test it? Is there a specific reason or purpose for just using TAs? Being more explicit about why them specifically, or just generalize with something like “users familiar with check-style already, such as the TAs”.

The manual doesn’t really seem to have changed from last week, but then again you are building off Checkstyle’s manual so this is hard to evaluate. Their “Use” section is kind of short, maybe you can expand on this to help people who are new to IntelliJ?

Editor should validate user input before generating the configuration file