greenbone / ospd

OSPd is a framework for vulnerability scanners which share the same communication protocol: OSP (Open Scanner Protocol)

ospd

⚠️ ospd got merged into ospd-openvas. Therefore this repository is unmaintained and will not get any further changes!

ospd is a base class for scanner wrappers which share the same communication protocol: OSP (Open Scanner Protocol). OSP creates a unified interface for different security scanners and makes their control flow and scan results consistently available under the central Greenbone Vulnerability Manager service.

OSP is similar in many ways to GMP (Greenbone Management Protocol): XML-based, stateless and non-permanent connection.

The design supports wrapping arbitrary scanners with same protocol OSP, sharing the core daemon options while adding scanner specific parameters and options.

Table of Contents

• Table of Contents
• Releases
• Installation
  • Requirements
  • Install using pip
• How to write your own OSP Scanner Wrapper
• Support
• Maintainer
• Contributing
• License

Releases

 All release files are signed with the Greenbone Community Feed integrity key. This gpg key can be downloaded at https://www.greenbone.net/GBCommunitySigningKey.asc and the fingerprint is 8AE4 BE42 9B60 A59B 311C 2E73 9823 FAA6 0ED1 E580.

Installation

Requirements

ospd requires Python >= 3.7 along with the following libraries:

• paramiko
• lxml
• defusedxml
• deprecated
• psutil

Install using pip

You can install ospd from the Python Package Index using pip:

python3 -m pip install ospd

Alternatively download or clone this repository and install the latest development version:

python3 -m pip install .

How to write your own OSP Scanner Wrapper

As a core you need to derive from the class OSPDaemon from ospd.py. See the documentation there for the single steps to establish the full wrapper.

See the file doc/INSTALL-ospd-scanner.md about how to register a OSP scanner at the Greenbone Vulnerability Manager which will automatically establish a full GUI integration for the Greenbone Security Assistant (GSA).

For an example implementation see ospd-example-scanner.

Support

For any question on the usage of OSPD please use the Greenbone Community Portal. If you found a problem with the software, please create an issue on GitHub.

Maintainer

This project is maintained by Greenbone Networks GmbH.

Contributing

Your contributions are highly appreciated. Please create a pull request on GitHub. For bigger changes, please discuss it first in the issues.

For development you should use poetry to keep you python packages separated in different environments. First install poetry via pip

python3 -m pip install --user poetry

Afterwards run

poetry install

in the checkout directory of ospd (the directory containing the pyproject.toml file) to install all dependencies including the packages only required for development.

The ospd repository uses autohooks to apply linting and auto formatting via git hooks. Please ensure the git hooks are active.

poetry install
poetry run autohooks activate --force

License

Copyright (C) 2009-2020 Greenbone Networks GmbH

Licensed under the GNU Affero General Public License v3.0 or later.

ospd's Issues

[2.0.0] xml errors in the log

I often see this error message in the log:

journal: OSPD - openvas: ERROR: (ospd.ospd) While handling client command:#012Traceback (most recent call last):#12 File "/usr/lib/python3.6/site-packages/ospd/ospd.py", line 762, in handle_client_stream#012 response = self.handle_command(data)#12 File "/usr/lib/python3.6/site-packages/ospd/ospd.py", line 1624, in handle_command#012 return self.handle_get_vts_command(tree)#12 File "/usr/lib/python3.6/site-packages/ospd/ospd.py", line 1011, in handle_get_vts_command#012 vts_xml = self.get_vts_xml(vt_id, filtered_vts)#12 File "/usr/lib/python3.6/site-packages/ospd/ospd.py", line 1545, in get_vts_xml#012 vts_xml.append(self.get_vt_xml(vt_id))#12 File "/usr/lib/python3.6/site-packages/ospd/ospd.py", line 1487, in get_vt_xml#012 vt_id, vt.get('affected')#12 File "/usr/lib/python3.6/site-packages/ospd_openvas/daemon.py", line 684, in get_affected_vt_as_xml_str#012 _affected.text = affected#012 File "src/lxml/etree.pyx", line 1018, in lxml.etree._Element.text.set#12 File "src/lxml/apihelpers.pxi", line 710, in lxml.etree._setNodeText#012 File "src/lxml/apihelpers.pxi", line 698, in lxml.etree._createTextNode#012 File "src/lxml/apihelpers.pxi", line 1493, in lxml.etree._utf8#012ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

[2.0.0] Add option to set the group of the socket

By default the socket will created as user:group of the calling process.
But ospd-openvas must run as root to let run openvas with root rights.
So far so nice, but gvmd does not need to run with root rights. But now gvmd can't access the socket of ospd-openvas. And change the mode to 666 is not an good idea.
Better will be, to change group of the socket to one group in which gvmd will be an member of.
But for this an option is needed, to set the group of the socket.

GetMemoryUsageTestCase fails on macos

On macos 10.15, ospd 21.4.3, python 3.8, psutil 5.8.0:

______________ GetMemoryUsageTestCase.test_with_main_process_only ______________

self = <tests.command.test_commands.GetMemoryUsageTestCase testMethod=test_with_main_process_only>

    def test_with_main_process_only(self):
        cmd = GetMemoryUsage(None)
    
        request = et.fromstring('<get_memory_usage />')
    
>       response = et.fromstring(cmd.handle_xml(request))

tests/command/test_commands.py:458: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
ospd/command/command.py:672: in handle_xml
    self._add_memory_info(process_element, current_process.pid, unit)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'ospd.command.command.GetMemoryUsage'>
process_element = <Element 'process' at 0x1098dab80>, pid = 49273, unit = None

    @classmethod
    def _add_memory_info(
        cls, process_element: Element, pid: int, unit: str = None
    ):
        try:
            ps_process = psutil.Process(pid)
        except psutil.NoSuchProcess:
            return
    
        memory = ps_process.memory_info()
    
        rss_element = Element('rss')
        rss_element.text = cls._get_memory(memory.rss, unit)
    
        process_element.append(rss_element)
    
        vms_element = Element('vms')
        vms_element.text = cls._get_memory(memory.vms, unit)
    
        process_element.append(vms_element)
    
        shared_element = Element('shared')
>       shared_element.text = cls._get_memory(memory.shared, unit)
E       AttributeError: 'pmem' object has no attribute 'shared'

ospd/command/command.py:659: AttributeError

The other GetMemoryUsageTestCase tests fail similarly. The psutil docs suggest that shared is only available on linux.

OSP documentation: type patterns not in HTML

For PR#19 I discovered that the type patterns in OSP.xml are not visible
in the generated html file. The problem likely is in doc/HTMl.xsl.

ospd 21.4.3 has requirement psutil<6.0.0,>=5.7.2, but you'll have psutil 5.5.1 which is incompatible.

Please when you say you build will work on buster then I expect your setup works without running own PIP installs.

https://greenbone.github.io/docs/gvm-21.04/index.html is outdated then and on some points wrong.

To fix this:

echo 'deb http://deb.debian.org/debian buster-backports main' >/etc/apt/sources.list.d/backports.list
apt-get update
apt-get install python3-psutil/buster-backports

[2.0.1] Source file signature missing

The signature of the source tar ball is missing.

SSL Handshake error doing mutual authentication

Hey OSPD team,

some time ago, i openend a ticket on ospd-openvas to track a connection error between GVMD and the remote listening (IP) ospd-openvas instance.

I think the root cause is within OSPD.
When connecting from GVMD I am getting the following error:

Exception happened during processing of request from ('xxx.xxx.xxx.xx', 55500)
Traceback (most recent call last):
  File "/usr/lib/python3.7/socketserver.py", line 650, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.7/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.7/socketserver.py", line 720, in __init__
    self.handle()
  File "/opt/openvas/lib/python3.7/site-packages/ospd-2.0.0-py3.7.egg/ospd/server.py", line 127, in handle
    self.server.handle_request(self.request, self.client_address)
  File "/opt/openvas/lib/python3.7/site-packages/ospd-2.0.0-py3.7.egg/ospd/server.py", line 167, in handle_request
    self.server.handle_request(request, client_address)
  File "/opt/openvas/lib/python3.7/site-packages/ospd-2.0.0-py3.7.egg/ospd/server.py", line 297, in handle_request
    req_socket = self.tls_context.wrap_socket(request, server_side=True)
  File "/usr/lib/python3.7/ssl.py", line 423, in wrap_socket
    session=session
  File "/usr/lib/python3.7/ssl.py", line 870, in _create
    self.do_handshake()
  File "/usr/lib/python3.7/ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
OSError: [Errno 0] Error

Fore a detailed analysis of the issue, please read the issue within ospd-openvas

All details and versions can be found in the above ticket

older ospd version

If I use older version ospd like v1.0.0, does python3 environment is also required? (‘coz our team may use openvas 8, Thanks)

Could not find suitable distribution for Requirement.parse

I have this error when I run
python3 setup.py install --prefix=/usr/local/

running install
running bdist_egg
running egg_info
writing ospd_openvas.egg-info/PKG-INFO
writing dependency_links to ospd_openvas.egg-info/dependency_links.txt
writing entry points to ospd_openvas.egg-info/entry_points.txt
writing requirements to ospd_openvas.egg-info/requires.txt
writing top-level names to ospd_openvas.egg-info/top_level.txt
reading manifest file 'ospd_openvas.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.csv' under directory 'tests'
writing manifest file 'ospd_openvas.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/init.py -> build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/version.py -> build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/daemon.py -> build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/db.py -> build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/dryrun.py -> build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/errors.py -> build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/lock.py -> build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/nvticache.py -> build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/openvas.py -> build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/preferencehandler.py -> build/bdist.linux-x86_64/egg/ospd_openvas
copying build/lib/ospd_openvas/vthelper.py -> build/bdist.linux-x86_64/egg/ospd_openvas
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/init.py to init.cpython-39.pyc
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/version.py to version.cpython-39.pyc
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/daemon.py to daemon.cpython-39.pyc
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/db.py to db.cpython-39.pyc
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/dryrun.py to dryrun.cpython-39.pyc
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/errors.py to errors.cpython-39.pyc
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/lock.py to lock.cpython-39.pyc
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/nvticache.py to nvticache.cpython-39.pyc
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/openvas.py to openvas.cpython-39.pyc
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/preferencehandler.py to preferencehandler.cpython-39.pyc
byte-compiling build/bdist.linux-x86_64/egg/ospd_openvas/vthelper.py to vthelper.cpython-39.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying ospd_openvas.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ospd_openvas.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ospd_openvas.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ospd_openvas.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ospd_openvas.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ospd_openvas.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/ospd_openvas-21.10.0.dev1-py3.9.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing ospd_openvas-21.10.0.dev1-py3.9.egg
Removing /usr/local/lib/python3.9/dist-packages/ospd_openvas-21.10.0.dev1-py3.9.egg
Copying ospd_openvas-21.10.0.dev1-py3.9.egg to /usr/local/lib/python3.9/dist-packages
ospd-openvas 21.10.0.dev1 is already the active version in easy-install.pth
Installing ospd-openvas script to /usr/local/bin

Installed /usr/local/lib/python3.9/dist-packages/ospd_openvas-21.10.0.dev1-py3.9.egg
Processing dependencies for ospd-openvas==21.10.0.dev1
Searching for ospd>=21.10.0.dev1
Reading https://pypi.org/simple/ospd/
No local packages or working download links found for ospd>=21.10.0.dev1
error: Could not find suitable distribution for Requirement.parse('ospd>=21.10.0.dev1')

OSP documentation: Allow examples for elements.

This would improve the documentation of elements and it would be possible
to drop the (sometimes long) example elements in the command documentation examples.
It would mean to extend HTML.xsl. Note: This could benefit the documentation of GMP as
well where a similar HTML.xsl is used.

[21.4.0] Source code signature file missing

The file is missing for the released files.

broken link

this para in the main readme: There are some online resources about this topic: https://docs.greenbone.net/GSM-Manual/gos-3.1/en/osp.html#how-to-write-your-own-osp-wrapper

returns a 404

Write your own OSPD wrapper link in README is broken

I wanted to learn how to create an OSPD wrapper myself, but the link on how to create one in the README is broken. Can you please fix the link? Thanks!

[20.8.1] tests are installed

After call setup.py install, the following files will installed:
/usr/lib/python3.6/site-packages/tests/command/init.py
/usr/lib/python3.6/site-packages/tests/command/pycache/init.cpython-36.opt-1.pyc
/usr/lib/python3.6/site-packages/tests/command/pycache/init.cpython-36.pyc
/usr/lib/python3.6/site-packages/tests/command/pycache/test_command.cpython-36.opt-1.pyc
/usr/lib/python3.6/site-packages/tests/command/pycache/test_command.cpython-36.pyc
/usr/lib/python3.6/site-packages/tests/command/pycache/test_commands.cpython-36.opt-1.pyc
/usr/lib/python3.6/site-packages/tests/command/pycache/test_commands.cpython-36.pyc
/usr/lib/python3.6/site-packages/tests/command/pycache/test_registry.cpython-36.opt-1.pyc
/usr/lib/python3.6/site-packages/tests/command/pycache/test_registry.cpython-36.pyc
/usr/lib/python3.6/site-packages/tests/command/test_command.py
/usr/lib/python3.6/site-packages/tests/command/test_commands.py
/usr/lib/python3.6/site-packages/tests/command/test_registry.py

I think, the test files should not be installed on the target.