marketsquare / robotframework-robocop

Tool for static code analysis of Robot Framework language

---

Robocop

• Introduction
• Documentation
• Requirements
• Installation
• Usage
• Example Output
• Values
• Fixing issues
• FAQ

Watch our talks from RoboCon 2021 and RoboCon 2022 about Robocop & Robotidy and learn more about these cool tools! 🤖

---

Introduction

Robocop is a tool that performs static code analysis of Robot Framework code.

It uses official Robot Framework parsing API to parse files and runs number of checks, looking for potential errors or violations to code quality standards (commonly referred as linting issues).

Hosted on GitHub. 🎖️

Documentation

Full documentation is available here. 📖

Most common questions with answers can be found at the bottom ⬇ of this README file.

Requirements

Python 3.8+ 🐍 and Robot Framework 3.2.2+ 🤖.

Installation

You can install the latest version of Robocop simply by running:

pip install -U robotframework-robocop

Usage

Robocop runs by default from the current directory and it discovers supported files recursively. You can simply run:

robocop

All command line options can be displayed in help message by executing:

robocop -h

Example Output

Executing command:

robocop --report rules_by_error_type test.robot

Will result in following output:

\Users\OCP\test.robot:7:1 [W] 0509 Section '*** Variables ***' is empty (empty-section)
\Users\OCP\test.robot:22:1 [E] 0801 Multiple test cases with name "Simple Test" (first occurrence in line 17) (duplicated-test-case)
\Users\OCP\test.robot:42:1 [E] 0810 Both Task(s) and Test Case(s) section headers defined in file (both-tests-and-tasks)
\Users\OCP\test.robot:48:1 [W] 0302 Keyword 'my keyword' does not follow case convention (wrong-case-in-keyword-name)
\Users\OCP\test.robot:51:13 [I] 0606 Tag 'mytag' is already set by Test Tags in suite settings (tag-already-set-in-test-tags)

Found 5 issues: 2 ERRORs, 2 WARNINGs, 1 INFO.

Values

Original RoboCop - a fictional cybernetic police officer 👮‍♂️ - was following 3 prime directives which also drive the progress of Robocop linter:

First Directive: Serve the public trust 👨‍👩‍👧‍👦

Which lies behind the creation of the project - to serve developers and testers as a tool to build applications they can trust.

Second Directive: Protect the innocent 👶

The innocent testers and developers have no intention to produce ugly code but sometimes, you know, it just happens, so Robocop is there to protect them.

Third Directive: Uphold the law 🏛️

Following the coding guidelines established in the project are something very important to keep the code clean, readable and understandable by others and Robocop can help to uphold the law.

Fixing issues

Many issues in your code reported by Robocop can be fixed using auto-formatting tool, Robotidy. Check out the Robotidy documentation.

FAQ

Can I integrate Robocop with my code editor (IDE)?

Yes, Robocop integrates nicely with popular IDEs like PyCharm or VSCode thanks to Robot Framework Language Server. Read simple manual (README) in that project to figure out how to install & use it.

You can also use Robocop in PyCharm easily as an external tool. To configure it, go to: File → Settings → Tools → External Tools and click + icon. Then put values based on official instructions or this screenshot:

If you're using Python virtual environment in your project, make sure to provide correct path to robocop.exe located in venv\Scripts\robocop.exe. Now, you can run Robocop by right-clicking on a file or directory and choosing External tools → Robocop.

We suggest also to add a keyboard shortcut (e.g. Ctrl + , (comma)) to quickly run Robocop on selected files. You can map the shortcut in Settings → Keymap.

Can I load configuration from file?

Yes, there are multiple ways to configure Robocop:

Argument file

You can add command line options to an argument file, preferably one option with value for a line. Such file can be used as an input for Robocop with --argumentfile / -A option, e.g. robocop -A robocop.cfg. You can mix arguments from a file with ones provided in run command.

Example argument file:

--exclude *doc*
--exclude 0510
--threshold W
--configure inconsistent-assignment:assignment_sign_type:equal_sign
--configure line-too-long:line_length:140
--reports all
--output robocop.log

---

.robocop file

It is a default file that is loaded only when no command line options are provided for Robocop. When running plain robocop command, it looks for .robocop file from place where it was run until it finds .git file. Options can be provided like in the example above.

---

pyproject.toml file

If there is no .robocop file and toml module is installed, Robocop will try to load configuration from pyproject.toml file (if it exists). Options have the same names as command line arguments and need to be placed under [tool.robocop] section.

Example configuration file:

[tool.robocop]
paths = [
    "tests\\atest\\rules\\bad-indent",
    "tests\\atest\\rules\\duplicated-library"
]
include = ['W0504', '*doc*']
exclude = ["0203"]
reports = [
    "rules_by_id",
    "scan_timer"
]
ignore = ["ignore_me.robot"]
ext-rules = ["path_to_external\\dir"]
filetypes = [".txt", ".tsv"]
threshold = "E"
format = "{source}:{line}:{col} [{severity}] {rule_id} {desc} (name)"
output = "robocop.log"
configure = [
    "line-too-long:line_length:150",
    "0201:severity:E"
]
no_recursive = true

I use different coding standards. Can I configure rules so that they fit my needs?

Yes, some rules are configurable. You can list them by running robocop --list-configurables or just robocop -lc.

Configuring is done by using -c / --configure command line option followed by pattern <rule>:<param_name>:<value> where:

• <rule> can either be rule name or its id
• <param_name> is a public name of the parameter
• <value> is a desired value of the parameter

For example:

--configure line-too-long:line_length:140

is equivalent to

-c 0508:line_length:140

---

Each rule's severity can also be overwritten. Possible values are e/error, w/warning or i/info and are case-insensitive. Example:

-c too-long-test-case:severity:e

---

If there are special cases in your code that violate the rules, you can also exclude them in the source code.

Example:

Keyword with lowercased name  # robocop: disable

More about it in our documentation.

Can I define custom rules?

Yes, you can define and include custom rules using -rules / --ext-rules command line option by providing a path to a file containing your rule(s). The option accepts comma-separated list of paths to files or directories, e.g.

robocop -rules my/own/rule.py --ext-rules rules.py,external_rules.py

If you feel that your rule is very helpful and should be included in Robocop permanently, you can always share your solution by submitting a pull request. You can also share your idea by creating an issue.

More about external rules with code examples in our documentation.

Can I use Robocop in continuous integration (CI) tools?

Yes, it is easy to integrate Robocop with CI and other tools. For more details read our documentation.

Can I configure return status / code?

Yes, by default Robocop returns code 0 if number of found issues does not exceed quality gates.

Quality gates are the number specified for each severity (error, warning, info) that cannot be exceeded. Every violation of quality gates increases the return code by 1 up to maximum of 255. Default values for quality gates are:

quality_gate = {
          'E': 0,
          'W': 0,
          'I': -1
      }

which shows the accepted number of issues by severity. In that case each error and warning increases the return code. Rules with INFO severity do not affect the return code.

To configure quality gates, you simply use -c / --configure command line option with following pattern --configure return_status:quality_gates:<severity>=limit. You can change all limits at once. Example:

--configure return_status:quality_gates:E=0:W=100:I=-1

which means that no errors are accepted, up to 100 warnings are tolerated and issues with INFO severity do not affect the return code.

What is the difference between Robocop and rflint?

Robocop is better in every case because it:

• has maaaaany rules that check the quality of your Robot Framework code
• is integrated with popular IDE tools
• is highly configurable
• has very good defaults that work out of the box
• can be configured in source code
• uses the latest Robot Framework Parsing API
• is actively developed & fixed
• is easy to integrate with external tools
• can redirect output to a file
• displays nice reports
• is easy to extend it with new rules
• is cool 🤓

Still not convinced? Watch our talk about Robocop & Robotidy and see for yourself! 🧐

---

Excuse me, I have to go. Somewhere there is a crime happening. - Robocop

robotframework-robocop's Issues

Design Robocop logo and publish it

Run tests in parallel

Remove astroid dependency (required for importing checkers)

Parse pylama's output and fail step in Unit tests only on errors

Now pylama returns status other than 0 if any issue is found. Warnings and refactors should not fail unit tests but because pylama does not have an option to control return code, we need to send it to null and then parse the output independently.

Add raw file checker type and visitor type (default, all current rules are visitor type)

Rule: too-many-arguments

Too many arguments used in keyword call or definition. For example:

My Keyword  ${var1}  1  smth  ${var2}  10 ...

Resolve variables in import paths

For example:

Resource  ${path}/file.robot

It requires loading variables and remembering current scope.

Add type of message (node based or line based - ie from parsing raw file, not using robot api)

Improve methods of disabling rule from comment (# robocop=disable:rulename,rulename2)

parse robot parsing erorrs as rule violations

space in tags, reserved word (OR, AND -> just info with potential issue)

Allow to ignore file

Add a way to ignore particular files ie given directory. Support glob patterns:

tests/
test.robot
resource.robot
test2.robot
othersuite.robot

If robocop parse whole directory with --ignore test*.robot it should only run tests on resource.robot and othersuite.robot

Method for configuring rules from cli

Add verbose output

Add CLI option to turn on verbose output and also a method that write lines to output if verbose is enabled.

Create acceptance tests and add automation via GitHub Actions

Create acceptance tests that checks robocop functionality and add automated mechanism to run acceptance tests for each created Pull Request

Flag for listing all enabled rules (or all rules with enabled/disabled info)

Create external documentation on all features, user guide

Load external rules

Add proper readme

Allow to disable rule by default (and run only if it was enabled with --include)

Extend help message and command line options handling

Write more extensive help for different command line options and organize them into groups.

Severity value should be removed if used in include/exclude/configure arguments

We're referring to rules either by message name or id. Id is usually in format of "0101" etc. A lot of people will try to use it together with severity value, for example given this log:

C:\repo\path\file.robot:196:0 [W] 0507 Line is too long (145/120)

if we want to ignore it we need to add this:
robocop --exclude 0507 paths

But some people will try to use this:
robocop --exclude W0507 paths

We need to make sure this situation is handled

Allow to override message severity

Add reports (allow rules to define reports for storing some data, statistic that could be displayed together with lint results)

line too long rule

todo comments as warning

Too short,long keyword (counting keyword calls, not lines of code)

too-short-keyword (lines of code)
too-long-keyword (lines of code)
too-many-keywords (keyword calls)

Unify checkers scan entrypoint (for code clarity)

All types of checkers should use the same method name for scan entrypoint. Right now we have "visit" for VisitorChecker and "parse_file" or RawFileChecker (and we're checking for type in robocop run.py)

invalid sections (in general and per file type, resource, suite, __init__)

Parse list of paths instead of first positional path given

Normalize robot paths - replace ${/} (done), CURDIR, EXECDIR etc in resource paths

Parse only allowed extensions (*.robot for now)

Rule: trailing blankline

Robocop should not parse directory recursively by default, only with option enabled from cli

Support glob patterns in rule include/exclude arguments

auto generate help info on settings like configurable rules fields, available messages

Allow to disable rules by severity (all info, all warnings etc)

Register possible messages while registering rules in common store

Allow to use custom message formatting

Save output to file

Rule: in line comments

Test
  Keyword1
  # Keyword2
  Keyword3

If someone is commenting out the whole lines, it should report warning

Add run statistic

Add methods for including / excluding rules

We need a way for including / excluding linter rules.
It should support:
recognizing rule by rule name or id (missing-doc-keyword or 0201 for example)
if nothing is included all rules should be used
if something is included only rules included should be run
excluding work on top of previous two features. For example:

--include 0201 --exclude 0201
nothing will run

--exclude 0201
will run all rules except 0201

--include 0201
will only run 0201 rule

Rules could be given as separate commands (--include rule1 --include rule2) or as comma separated list (--include rule1,rule2)

file-too-long rule

Load configuration from file and/or command line

Add more rules

Rule: missing documentation in testcase, suite

Its partly handled already but it will not detect missing documentation if whole section (*** settings ***) is missing.

Handle invalid messages (duplicates, not existing severity)

Depends on #6

Create unit tests and add automation via GitHub Actions

Create unit tests that checks robocop methods and add automated mechanism to run unit tests for each created Pull Request

don't use keywords after [Return] statement