svenskaspel / har2locust

Convert HTTP Archive (.har file) to a Locust script

har2locust

Creating a locust file from scratch is sometimes hard. Don't you wish you could just generate it automatically from a browser session?

Well, now you can! har2locust converts your browser recordings (HAR files) into locust files.

Here's an example of a generated file.

At its core har2locust uses a jinja2 template to define the output. You can easily change that to customize your output, or you can go even further and use the plugin system to make any kind of changes to the processing/output.

Installation

pip install har2locust

Usage

1. Navigate the web with your browser while recording your activity. Then export the recording into a HAR file. Here is an example using Chrome Devs Tools

2. Run har2locust myrecording.har > locustfile.py

> har2locust --help
usage: har2locust [-h] [-t TEMPLATE] [--plugins PLUGINS] [--disable-plugins DISABLE_PLUGINS]
                  [--resource-types RESOURCE_TYPES] [--version] [--loglevel LOGLEVEL]
                  input

positional arguments:
  input                 har input file

options:
  -h, --help            show this help message and exit
  -t TEMPLATE, --template TEMPLATE
                        jinja2 template used to generate locustfile. Defaults to locust.jinja2. Will check current
                        directory/relative paths first and har2locust built-ins second
  --plugins PLUGINS     Comma separated list of extra python files to source OR packages to import, containing
                        decorated methods for processing the har file.
  --disable-plugins DISABLE_PLUGINS
                        Temporarily disable default plugins. Specified by comma separated list of default plugin
                        python files to source.
  --resource-types RESOURCE_TYPES
                        Commas separated list of resource types to be included in the locustfile. Supported type are
                        `xhr`, `script`, `stylesheet`, `image`, `font`, `document`, `other`. Defaults to
                        xhr,document,other.
  --version, -V         show program's version number and exit
  --loglevel LOGLEVEL, -L LOGLEVEL

Simplest usage:
har2locust myrecording.har > locustfile.py

Load extra plugins by import path and/or filename:
har2locust --plugins har2locust.extra_plugins.plugin_example,myplugin.py myrecording.har

Disable one of the default plugins:
har2locust --disable-plugins=rest.py myrecording.har

Parameters can also be set using environment variables or config files (har2locust.conf or ~/.har2locust.conf) For details about how to set parameters see https://goo.gl/R74nmi

3. Run locust!

Advanced usage

• Filter your recording using the files .urlignore and .headerignore (read from your current directory). Populate them with regexes to filter any unwanted requests or headers from your recordings. Some headers are always ignored (cookie, content-length and chrome's "fake" headers) Here are some examples: .urlignore, .headerignore

• Use the plugin system for more advanced processing.

Notes

har2locust builds upon har2py, modified to generate a locustfile instead of a basic Python file and extended to support plugins.

har2locust is very new, and before 1.0 there may be changes to interfaces without notice. If you encounter an issue, PRs are very welcome.

Also, dont expect that the generated file will always be very useful out of the box. You'll want to add response validations to ensure the quality of your test, and perhaps parametrize dynamic data like usernames.

har2locust's Issues

Invlid generated class name

HAR file names with invalid characters for an identifier (parentheses, brackets, etc.) generate invalid class names (and code).

Currently the only characters being replaced are "-" and ".":

name = har_path.stem.replace("-", "_").replace(".", "_")

Template with multiple tagged tasks in separate functions

A use case I'm looking into would be to split the generated locust file into multiple tasks according to the path detected in the url.

For example I'm browsing on a web application that has multiple modules, for example /todo, /calendar, /news, and I would love to have har2locust generate separate tasks for each "prefix" so that I can launch locust to load tests only one module at a time...

# snip 
    @tag('news')
    @task
    def news(self):
        with self.client.request(
            "GET",
            "/news/latest",
# snip

ERROR:root:invalid syntax when parsing rendered template

har2locust --loglevel DEBUG zlxy.har > ab.py

ERROR:root:invalid syntax when parsing rendered template
Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Users\aihole\AppData\Local\Programs\Python\Python311\Scripts\har2locust.exe_main.py", line 7, in
File "c:\users\aihole\appdata\local\programs\python\python311\Lib\site-packages\har2locust_main.py", line 31, in main
py = render(args.template, template_values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\users\aihole\appdata\local\programs\python\python311\Lib\site-packages\har2locust_main_.py", line 103, in render
tree = ast.parse(py, type_comments=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\users\aihole\appdata\local\programs\python\python311\Lib\ast.py", line 50, in parse
return compile(source, filename, mode, flags,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 142
'sec-ch-ua-platform': '"Windows"',}, data=,catch_response=True) as resp:
^
SyntaxError: invalid syntax

C:\Users\aihole\Desktop>har2locust -V
har2locust 0.9.3
C:\Users\aihole\Desktop>har2locust --loglevel DEBUG zlxy.har > ab.py
zlxy.zip

Have an option to generate with HttpUser not FastHttpUser

I'm having some segfaults with FastHttpUser, and to fast track to a working version I'm switching the generated file to HttpUser for it not to segfault.

I then need to replace all self.rest( with self.client.request( for the locustfile to work.

Would it be possible to have an option for this ? I know this can probably be done with a template, but this might interest some other users.

(oh and by the way, I ♥ har2locust, it's really awesome for my use case! )

How to elegantly handle the jwt token inside the header？

  with self.rest(
            "GET",
            "/api/xxxxx?systemId=7228a882ce2b4805aef2ba007f452ab2&time=1703906292335",
            headers={
                "Accept": "application/json, text/javascript, */*; q=0.01",
                "Authorization": "Bearer xxxxxxxxxx",
                "Referer": "xxxxx",
                "Sec-Fetch-Dest": "empty",
                "Sec-Fetch-Mode": "cors",
            },

Since every request contains "Authorization": "Bearer xxxxxxxxxxxx", is there a plugin that provides placeholders instead of me replacing them in my production code?

Error parsing har file

I am trying to generate a locust file from a har fetched from peterjacksons.com site. When running har2locust www.peterjacksons.com > locustfile.py I get the following error:

har2locust www.peterjacksons.com.har
ERROR:root:invalid decimal literal when parsing rendered template (set log level DEBUG to see the whole output)
Traceback (most recent call last):
  File "/opt/homebrew/bin/har2locust", line 8, in <module>
    sys.exit(__main__())
             ^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/har2locust/__main__.py", line 28, in __main__
    py = render(args.template, template_values)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/har2locust/__main__.py", line 98, in render
    tree = ast.parse(py, type_comments=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.8/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<unknown>", line 80

and the resulting file is empty.
I attached the har file for reference.
www.peterjacksons.com.har.tgz

F841 Local variable `resp` is assigned to but never used

C:\Users\aihole\Desktop>har2locust index.har > locustfile.py
-:25:84: F841 Local variable resp is assigned to but never used

C:\Users\aihole\Desktop>har2locust -V
har2locust 0.9.2

Why is there such an error?
Operation environment: win10 cmd
har2locust version: 0.9.2

Specifying `--resource-types` gives `IndexError: list index out of range`

har2locust --resource-types document,xhr performance/an_undergraduate_student_joins.har > performance/an_undergraduate_student_joins.py

yields

Traceback (most recent call last):
  File "/Users/russ/.pyenv/versions/3.12.0/bin/har2locust", line 8, in <module>
    sys.exit(__main__())
             ^^^^^^^^^^
  File "/Users/russ/.pyenv/versions/3.12.0/lib/python3.12/site-packages/har2locust/__main__.py", line 29, in __main__
    template_values = process(har, args)
                      ^^^^^^^^^^^^^^^^^^
  File "/Users/russ/.pyenv/versions/3.12.0/lib/python3.12/site-packages/har2locust/__main__.py", line 77, in process
    values |= p(entries, args) or {}
              ^^^^^^^^^^^^^^^^
  File "/Users/russ/.pyenv/versions/3.12.0/lib/python3.12/site-packages/har2locust/default_plugins/default_headers.py", line 39, in default_headers
    urlparts = urlsplit(entries[0]["request"]["url"])
                        ~~~~~~~^^^
IndexError: list index out of range

Removing the --resource-types param generates the script. Also, adding the default har2locust --resource-types xhr,document,other performance/an_undergraduate_student_joins.har > performance/an_undergraduate_student_joins.py generates the script with no errors.