fabriziosalmi / websites-monitor

Websites monitoring via GitHub Actions (expiration, security, performances, privacy, SEO)

Websites Monitor

Project Description

This project aims to continuously monitor various aspects of specified websites. It runs a variety of checks, ranging from performance to security considerations. The GitHub Action is scheduled to run once per day, updating this README with the latest results.

How to Use

1. Fork this repository.
2. Add the websites you want to monitor in the websites.txt file, one per line.
3. Enable GitHub Actions if not already enabled.
4. The README will be automatically updated with the latest check results once a day.

Monitoring Checks

Check Type	audiolibri.org	get.domainsblacklists.com	example.com
Domain breach	🔘	🔘	🔘
Domain Expiration	🟢 (253 days left)	🟢 (341 days left)	🟢 (355 days left)
SSL Expiration	🟢 (80 days left)	🟢 (54 days left)	🟢 (191 days left)
DNS Blacklists (Spamhaus + Spamcop)	🟢	🟢	🟢
DNS Blacklist (DomainsBlacklists)	🟢	🟢	🟢
HSTS	🟢	🟢	🔴
XSS Protection	🟢	🟢	🔴
Redirect chains	🟢	🟠	🟢
Pagespeed Performances	98	99	100
Website Load Time	🟢	🟢	🟢
Rate limited	🔴	🔴	🔴
CDN	🔴	🔴	🔴
Brotli	🟢	🔴	🔴
Deprecated libs	🟢	🟢	🟢
Client rendering	🔴	🔴	🟢
Mixed content	🔴	🟢	🟢
Content-Type headers	🟢	🟢	🟢
Internationalization	⚪	⚪	⚪
FLOC	🔴	🔴	🔴
AMP	🔴	🔴	🔴
Robots.txt	🔴	🔴	🔴
Sitemap	🔴	🔴	🔴
Favicon	🟢	🟢	🔴
Alt Tags	🟢	🟢	🔴
Open Graph Protocol	🟢	🔴	🟢
Semantic Markup	🔴	🔴	🔴
Ad and tracking	🟢	🟢	🟢
Privacy-Protected Whois	🔴	🔴	🔴
Privacy Exposure	🔴	🔴	🟢

---

Last Updated: 2024-08-22 04:03:54

websites-monitor's Issues

Move websites list to external file

with open("websites.txt", "r") as f:
websites = f.readlines()

Report export

Report export in PDF, XLS, CSV, JSON formats.

Check server software

import requests

def check_server_software(website):
    """
    Detect the web server software of the website.
    
    Args:
    - website (str): URL of the website to be checked.
    
    Returns:
    - str: "🟢" if server software is suppressed or not easily identifiable,
           "🔴" if server software is openly exposed,
           "⚪" for any errors.
    """

    try:
        response = requests.get(website)
        server_header = response.headers.get("Server")

        if server_header:
            print(f"Server software identified: {server_header}")
            return "🔴"
        else:
            return "🟢"

    except Exception as e:
        print(f"Error occurred: {e}")
        return "⚪"

print(check_server_software("https://example.com"))

Domain breach check

• https://breachdirectory.com/#api-pricing 15€/m

Split login

..into several files (main, checks)

v2

main.py

from datetime import datetime

# Import checks
from checks import (
    check_domain_breach, check_domain_expiration, check_ssl_cert, check_dns_blacklist,
    check_domainsblacklists_blacklist, check_hsts, check_xss_protection, check_redirect_chains,
    check_pagespeed_performances, check_website_load_time, check_rate_limiting, check_cdn,
    check_brotli_compression, check_deprecated_libraries, check_clientside_rendering,
    check_mixed_content, check_content_type_headers, check_internationalization, check_floc,
    check_amp_compatibility, check_robot_txt, check_sitemap, check_favicon, check_alt_tags,
    check_open_graph_protocol, check_semantic_markup, check_ad_and_tracking, check_privacy_protected_whois,
    check_privacy_exposure
)

# Initialize an error log
error_log = []


def log_error(message):
    """A simple function to log errors for later use."""
    error_log.append(message)
    print(message)  # This will also print the error in the console


# Read websites from external file
with open('websites.txt', 'r') as f:
    websites = [line.strip() for line in f.readlines()]

# Initialize Markdown report
report_md = "# Websites Monitor\n"

# Read the project description and usage instructions
with open('project_description.md', 'r') as f:
    report_md += f"{f.read()}\n"

with open('usage_instructions.md', 'r') as f:
    report_md += f"{f.read()}\n"

# Initialize the table
report_md += "\n## Monitoring Checks\n"
report_md += "| Check Type | " + " | ".join(websites) + " |\n"
report_md += "|------------|" + "---|" * len(websites) + "\n"

# List of check functions and their human-readable names
check_functions = [
    ("Domain breach", check_domain_breach),
    # ... [other checks]
    ("Privacy Exposure", check_privacy_exposure),
]

# Populate the table with check results
for check_name, check_func in check_functions:
    report_md += f"| {check_name} | "
    for website in websites:
        try:
            result = check_func(website)
            report_md += f"{result} | "
        except Exception as e:
            err_msg = f"Error occurred with {check_name} for {website}: {e}"
            log_error(err_msg)
            report_md += "⚪ | "  # Add a default grey indicator for errors
    report_md += "\n"

# Timestamp
current_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
report_md += f"\n---\nLast Updated: {current_time}\n"

# Write the Markdown report to a file
with open("README.md", "w") as f:
    f.write(report_md)

# Exit with a non-zero code if errors were encountered
if error_log:
    exit(1)