tristanlatr / wpwatcher Goto Github PK

Shows like it has an alert, but it doesn't display the alert details :/

The log file actually shows :

INFO - Email sent: WPWatcher WARNING report - http://URL.ca - 2020-10-13T18-02-26 to

Currently, reports are loaded as a whole at the beginning of the script and wrote to file when a scan end using the json.dump() method. See WPWatcher. update_and_write_wp_reports() method.

Could use https://pypi.org/project/tinydb/
It will increase the performance of the script.

But do not log in terminal with FIXED keyword
Expected behaviour is not to display twice the issue: only at the end of the email with [false positive] prefix

hi tristanlatr,
please make deploy to heroku button for don't use my personal system hardware & vps req.
its so important for me
waiting for your response
finest regards

On version 2.0.5 only
Need to fix WPWatcherNotification.build_message

An bug slipped through the cracks and errors don't get reported in Json database (Version 2.0)

The code that choose what to store in database and what will be included in emails will be changed in order to always save everything in database and exclude infos or warnings from messages afterwards

Could run Wordpress installs locally

Ressources for vulnerable docker wordpress images

• https://github.com/vianasw/dvwps
• https://github.com/vulhub/vulhub/tree/master/wordpress/pwnscriptum
• https://github.com/wpscanteam/VulnerableWordpress

Hello,

I have a problem where WPWatcher refuses to do all of the work when configured as a cron job. I set it to run with verbose mode and to output logging to a file in my home folder, and all I can se it doing is sort of faking a run, almost as if it was dry-running.

0 * * * * /home/user/.local/bin/wpwatcher -v --conf /home/user/wpwatcher.conf >> /home/user/wpwatcher.log

Approximate config:

DEBUG - WPWatcher configuration:
wp_sites                        =       [{"url": "..."}, {"url": "..."}, {"url": "..."}, {"url": "..."}]
send_email_report               =       True
send_errors                     =       True
email_to                        =       ["[email protected]"]
send_infos                      =       True
quiet                           =       True
verbose                         =       True
attach_wpscan_output            =       True
fail_fast                       =       False
api_limit_wait                  =       False
daemon                          =       False
daemon_loop_sleep               =       0:00:00
resend_emails_after             =       0:00:00
wp_reports                      =       /home/user/.wpwatcher/wp_reports.json
asynch_workers                  =       4
log_file                        =       /home/user/.wpwatcher/wpwatcher.log
follow_redirect                 =       True
wpscan_output_folder            =       /home/user/.wpwatcher/wpscan-results/
wpscan_args                     =       ["--format", "json", "--random-user-agent", "--update"]
scan_timeout                    =       0:25:00
false_positive_strings          =       []
send_warnings                   =       True
email_errors_to                 =       ["[email protected]"]
wpscan_path                     =       wpscan
smtp_server                     =       smtp.example.com:587
smtp_auth                       =       True
smtp_user                       =       [email protected]
smtp_pass                       =       ***
smtp_ssl                        =       True
from_email                      =       [email protected]
use_monospace_font              =       True
syslog_server                   =
syslog_port                     =       514
syslog_stream                   =       SOCK_STREAM
syslog_kwargs                   =       {"enterprise_id": 42, "msg_as_utf8": true, "utc_timestamp": true}

It outputs the correct config, it then continues to output:

INFO - Starting scans on 22 configured sites
INFO - Scanning site https://www.my-first-site-to-scan.com/
INFO - Scanning site https://www.the-next-site-to-scan.com/
DEBUG - Running WPScan command: wpscan --version --format json --no-banner
.....

It all looks like it is running correctly, but looking at HTOP I see that WPWatcher is running from start to stop in less than half a second, and I receive no emails. If I run the exact same command as the same user but not in cron it runs as it should, it takes some time and I receive emails for all scanned sites etc.

I'm running a fresh install of Ubuntu 20.04.1 LTS with Python 3.8.5, ruby 2.7.0p0, WPWatcher 2.4.8 and WPScan 3.8.15.

Any idea what I'm missing here? Is this a bug?

Description

The current documentation for WPWatcher, both on ReadTheDocs and within the codebase, could benefit from improvements to help users better understand the application and make it more accessible to contributors.

Suggested improvements include

1. Expand ReadTheDocs documentation: Enhance the existing Quickstart Guide to include sections on Troubleshooting and Common Issues, Tips for Effective Scanning, Contribution and Support, and License Information. This would provide a more comprehensive guide for users and encourage community involvement.
2. In-code documentation: Add more detailed docstrings and comments throughout the codebase to clarify the purpose and functionality of classes, methods, and functions. This would help contributors better understand the code, making it easier for them to contribute and maintain the project.
3. Best practices and troubleshooting: Provide a section in the ReadTheDocs documentation dedicated to best practices when using WPWatcher, as well as common troubleshooting tips to help users resolve issues they may encounter.

By implementing these improvements, we can make WPWatcher more accessible and user-friendly, ultimately benefiting both end-users and contributors to the project.

Description

Currently, WPWatcher relies on the Ruby-based WPScan tool to perform WordPress vulnerability scans. This adds a dependency on Ruby and the WPScan tool, which can complicate the installation process and might affect performance.

An alternative approach would be to use the WPScan API directly within the Python application. This could potentially improve performance, simplify dependencies, and allow for more control over the scanning process. However, this change would require a significant refactor of the existing codebase to integrate the WPScan API directly.

Benefits

• Improved performance: By using the API directly, we can potentially reduce overhead and improve the performance of WPWatcher.
• Simplified dependencies: Removing the dependency on the Ruby-based WPScan tool could make it easier to install and use WPWatcher.
• Increased control: By interacting with the WPScan API directly, we can have more control over the scanning process and possibly offer more features or customisations.

Potential Drawbacks

• Significant refactoring: This change would require a substantial rewrite of the existing codebase to integrate the WPScan API directly.

This issue is to discuss the feasibility of this change and gather input from contributors on whether it would be a valuable improvement to WPWatcher.

ERROR - Unable to send syslog messages for site URL
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/wpwatcher/scan.py", line 507, in scan_site
    self.syslog.emit_messages(wp_report)
  File "/usr/local/lib/python3.6/site-packages/wpwatcher/syslogout.py", line 48, in emit_messages
    for m in self.get_messages(wp_report):
  File "/usr/local/lib/python3.6/site-packages/wpwatcher/syslogout.py", line 68, in get_messages
    c.set_prefix("deviceVersion", VERSION)
NameError: name 'VERSION' is not defined

This bug was introduced when improving the setup.py

Description

When running WPWatcher with multiple instances or scheduled scans, there is a risk of data corruption and failures due to concurrent access to the JSON report database file. This issue impacts users who have multiple scans running at the same time or who use a combination of cron jobs and the Linux service.

Proposed Solution

To address these concurrency issues, I propose migrating from JSON file storage to SQLite for storing scan reports. SQLite provides built-in support for concurrent access and ensures data consistency even when multiple processes access the database simultaneously. By using SQLite, WPWatcher can run multiple instances without the risk of data corruption due to concurrent access.

• Update the DataBase class to use SQLite instead of a JSON file.
• Create a table named reports in the SQLite database to store report data.
• Modify the write and find methods to work with the SQLite database.
• Ensure the updated implementation maintains backward compatibility with existing configurations and usage.
• Test the updated implementation with multiple instances of WPWatcher running concurrently to confirm that the concurrency issues are resolved.

Benefit

By implementing SQLite for report storage, WPWatcher will be able to handle multiple instances and concurrent scans without the risk of data corruption or failures. This change will greatly enhance the reliability and robustness of the tool for users who need to perform frequent or overlapping scans.

WPWatcher should be able to run docker images of WPScan seamlessly.

Currently wpscan_path only accept single path values. Meaning that il will fail if wpscan_path=docker run -it --rm wpscanteam/wpscan.

WPWatcher should also have it's own docker image.

Description

The config.py module is responsible for handling WPWatcher's configuration. To improve maintainability and readability, we should address the issues reported by Code Climate in the config.py module.

Code Climate Recommendations

1. Cyclomatic complexity: Break down complex functions into smaller, more focused functions. This will make the code more readable and maintainable.
2. Cognitive complexity: Simplify the logic in some of the functions by using helper functions or more straightforward control structures.
3. Maintainability: Refactor the code to improve its maintainability and readability.

Specific Recommendations

• Refactor the _ini2conf function to reduce its complexity by breaking it into smaller, more focused functions.
• Simplify the _load_files function by using more straightforward control structures.
• Refactor the _find_files function to reduce its complexity by breaking it into smaller, more focused functions.

By addressing these issues, we can improve the overall quality and maintainability of the WPWatcher codebase.

Fixed items should be after warnings.
Issue details is not really relevant.
Only 2 lines per fixed items like :

Issue regarding component "Potential Vulnerability: Chained Quiz < 1.0 - Multiple XSS" has been fixed since last report.
Last report sent the 2020-07-06T12-54-48.

Even if all functionalities are working, WPWatcher surely have bugs and should be properly tested.

Wiki pages:

• Basic configuration overview and exemples
• False positives use cases
• Email reports statuses and config
• Linux service
• Docker install
• All options and config table
• Json database summary
• Library usage

It's a little bit complicated in there and it would require structural changes to make it more clear.

Description

To increase the flexibility and adaptability of WPWatcher for various user preferences and requirements, it would be beneficial to extend the application with more notification methods beyond the existing email notifications. The proposed enhancements include the integration of Slack and custom webhooks as notification channels for vulnerability alerts.

Proposed Methods

1. Slack: Enable users to configure WPWatcher to send alerts to a Slack channel using either incoming webhooks or the Slack API.
2. Webhooks: Support user-defined webhooks for custom integrations with other services, applications, or incident management platforms.

By implementing a variety of notification methods, WPWatcher can better cater to user preferences, streamline vulnerability alert reception, and facilitate seamless integration with other tools and services.

Even if wpscan output can be attached to emails, we should be able to save the raw output locally on demand.

Should use strip() on URL strings in WPWatcher. format_site()

We should be able to customize email report messages.

A field in the config: email_message read and added at the beginning of every reports.

Current behaviour

When using default format (cli), vulnerabilities or warnings regarding one component (i.e. Wordpress version) are displayed all together in one big message. So one message per component.

Ignore specific Warnings regarding a component and still display other is impossible, it'll ignore the entire message because false positive strings are tested against the entire message string, that contain ALL vulnerabilities titles.

Future behaviour

Split messages like when using --format json in WPScan arguments

Workaround : Use --format json in WPScan arguments

WPWatcher will currently crash.
Would be best to report an error, then continue with the scanning, and exit with a non-zero code.

Description

The cli.py module currently contains multiple responsibilities, such as configuration handling, argument parsing, and command execution. To improve maintainability and readability, we should divide the cli.py module into smaller, more focused modules, as well as address the issues reported by Code Climate.

Proposed Split

• args.py: Manages command-line argument parsing and validation.
• cli.py: Executes WPWatcher commands using the parsed arguments and config file.

Code Climate Recommendations

In addition to splitting the cli.py module, we should also address the following issues reported by Code Climate:

1. Cyclomatic complexity: Break down complex functions into smaller, more focused functions. This will make the code more readable and maintainable.
2. Cognitive complexity: Simplify the logic in some of the functions by using helper functions or more straightforward control structures.

By addressing these issues and splitting the cli.py module, we can improve the overall quality and maintainability of the WPWatcher codebase.

The scan output is parsed as the exit code was 0 and miraculously all issues are flagged as fixed and INFO email is sent...
Need some investigation on this one, error handling is somewhat failing.

Use https://rfc5424-logging-handler.readthedocs.io/en/latest/basics.html#sending-rfc5424-specific-fields

Add the following config options:

• syslog_server: String
• syslog_port: Int
  And other

For each scanned, send syslog TCP packet with the scan results as the structured data

Error sample :

ERROR - WPScan command 'wpscan --format cli --no-banner --random-user-agent --disable-tls-checks --enumerate t,p,tt,cb,dbe,u,m --url ' failed with exit code 3 . WPScan output: [+] URL: *** [] [+] Effective URL: ***/ [+] Started: Thu Apr 16 00:06:37 2020 [!] No WPVulnDB API Token given, as a result vulnerability data has not been output. [!] You can get a free API token with 50 daily requests by registering at https://wpvulndb.com/users/sign_up [+] Finished: Thu Apr 16 00:06:37 2020 [+] Requests Done: 4 [+] Cached Requests: 0 [+] Data Sent: 1.354 KB [+] Data Received: 75.075 KB [+] Memory used: 37.961 MB [+] Elapsed time: 00:00:00 Scan Aborted: No such file or directory @ rb_sysopen - /tmp/wpscan/cache/b20515b38b68645f5c87c5ac518148bb/1655927643 Trace: [...]

Currently the program tries to delete /tmp/wpscan and fails silently if errors.
Initially, it delete /tmp/wpscan to save space, the cache can rapidly increase size to a few Gigas.

Could do a clear_wpscan_cache=Yes/No in the config and --clear argument

If the alert is still triggered after the second email alert, add a notice like:

Warning: Unfixed issue

Description

To facilitate a more structured and efficient issue reporting process, it is recommended to add issue templates to the WPWatcher repository. These templates will help guide users in submitting issues and ensure that all necessary information is provided, ultimately making it easier for maintainers to address and resolve issues.

Proposed Issue Templates

1. Bug Report: A template that prompts users to provide details on the unexpected behaviour or errors they encountered, steps to reproduce the issue, and any relevant logs or error messages.
2. Feature Request: A template that guides users in describing a new feature or enhancement they would like to see added to WPWatcher, including the benefits, use cases, and any potential implementation considerations.
3. Documentation Improvement: A template that enables users to report issues or suggest improvements related to the project documentation, such as unclear explanations, missing sections, or outdated information.

Benefits:

By adding issue templates to the WPWatcher repository, the project can benefit from a more organised and efficient issue reporting process. These templates will help users provide comprehensive and relevant information, making it easier for maintainers to understand, triage, and address reported issues.

The timeout wrapper is dump and raise exception while waiting 24h...

Workaround : set scan_timeout=25h

WPWatcher should be able to run scans without API token first and then use token on site that have outdated WordPress version or plugin only to save API calls .

Would need to do some research to see if that's a safe way to scan sites anyway.

Would not work if API Token is written to wpscan config file or environnement variable

Traceback (most recent call last):
  File "/home/ec2-user/.local/bin/wpwatcher", line 8, in <module>
    sys.exit(main())
  File "/home/ec2-user/.local/lib/python3.9/site-packages/wpwatcher/cli.py", line 72, in main
    exit_code, reports = wpwatcher.run_scans()
  File "/home/ec2-user/.local/lib/python3.9/site-packages/wpwatcher/core.py", line 230, in run_scans
    self._run_scans(self.wp_sites)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/wpwatcher/core.py", line 207, in _run_scans
    self.new_reports.append(f.result())
  File "/usr/lib64/python3.9/concurrent/futures/_base.py", line 446, in result
    return self.__get_result()
  File "/usr/lib64/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/usr/lib64/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/ec2-user/.local/lib/python3.9/site-packages/wpwatcher/core.py", line 179, in _scan_site
    self.wp_reports.write([wp_report])
  File "/home/ec2-user/.local/lib/python3.9/site-packages/wpwatcher/db.py", line 111, in write
    raise RuntimeError("The file lock must be acquired before writing data. ")
RuntimeError: The file lock must be acquired before writing data. 

I don't understand what is tripping it off or how to suppress this error?

Description:

The email.py file has several issues reported by Code Climate. These issues include high cyclomatic complexity, cognitive complexity, and maintainability. In order to improve the code quality, we should address the following:

1. High cyclomatic complexity: Break down complex functions into smaller, more focused functions. This will make the code more readable and maintainable.
2. High cognitive complexity: Simplify the logic in some of the functions by using helper functions or more straightforward control structures.
3. Maintainability: Refactor the code to improve its maintainability and readability.

Some areas to focus on for improvement include:

• send_email: This function has a high cyclomatic complexity and cognitive complexity. Consider breaking down the function into smaller, more focused functions.
• smtp_send: This function has a high cyclomatic complexity. Splitting the function into smaller functions that handle specific tasks, such as establishing a connection and sending the email, can help reduce complexity.

These improvements should help address the issues reported by Code Climate and improve the overall quality of the email.py file.

When I try and use wp_sites in the wpwatcher.conf file and set wp_args for a site, then running wpwatcher fails with the following:

 INFO - WPWatcher -  Automating WPscan to scan and report vulnerable Wordpress sites
Traceback (most recent call last):
  File "/usr/local/bin/wpwatcher", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/dist-packages/wpwatcher/cli.py", line 408, in main
    WPWatcherCLI()
  File "/usr/local/lib/python3.6/dist-packages/wpwatcher/cli.py", line 53, in __init__
    configuration = self.build_config_cli(args)
  File "/usr/local/lib/python3.6/dist-packages/wpwatcher/cli.py", line 343, in build_config_cli
    configuration, files = WPWatcherConfig(files=conf_files).build_config()
  File "/usr/local/lib/python3.6/dist-packages/wpwatcher/config.py", line 55, in __init__
    self.parser.read_file(fp)
  File "/usr/lib/python3.6/configparser.py", line 718, in read_file
    self._read(f, source)
  File "/usr/lib/python3.6/configparser.py", line 1092, in _read
    fpname, lineno)
configparser.DuplicateOptionError: While reading from '/root/.wpwatcher/wpwatcher.old' [line 19]: option '{"url"' in section 'wpwatcher' already exists

Here's my conf with the actual URLs omitted:

[wpwatcher]
wp_sites=[
{"url":"url1.com","wpscan_args":["--ignore-main-redirect"]},
{"url":"url2.com"} ]

If I remove the wpscan_args protion, and only set the url, then it runs fine. However, I need to set different wpscan_args for each site I'll be scanning.

Any assistance with this would be appreciated.

Description

To provide users with an alternative, easily shareable format for vulnerability reports, it would be advantageous to add an HTML page export feature to WPWatcher. This feature would allow users to generate an interactive, visually appealing report that can be viewed in any web browser, shared with team members, or archived for future reference.