GithubHelp home page GithubHelp logo

juju4 / ansible-misp Goto Github PK

View Code? Open in Web Editor NEW
51.0 5.0 20.0 1.51 MB

ansible role to setup MISP, Malware Information Sharing Platform & Threat Sharing

License: BSD 2-Clause "Simplified" License

Shell 13.05% Ruby 62.47% PHP 0.52% Jinja 23.96%

ansible-misp's Introduction

Actions Status - Master Actions Status - Devel

MISP ansible role

Ansible role to setup MISP, Malware Information Sharing Platform & Threat Sharing

Alternatives

Requirements & Dependencies

Ansible

It was tested on the following versions:

  • 2.0
  • 2.2
  • 2.3
  • 2.4
  • 2.5

Operating systems

Tested on Ubuntu 20.04, 22.04 and centos 8-Stream

Example Playbook

Just include this role in your list. For example

- hosts: all
  roles:
    - juju4.MISP

default admin credentials ([email protected] / admin)

Variables

Nothing specific for now.

Continuous integration

This role has a travis basic test (for github), more advanced with kitchen and also a Vagrantfile (test/vagrant). Default kitchen config (.kitchen.yml) is lxd-based, while (.kitchen.vagrant.yml) is vagrant/virtualbox based.

Once you ensured all necessary roles are present, You can test with:

$ gem install kitchen-ansible kitchen-lxd_cli kitchen-sync kitchen-vagrant
$ cd /path/to/roles/juju4.MISP
$ kitchen verify
$ kitchen login
$ KITCHEN_YAML=".kitchen.vagrant.yml" kitchen verify

or

$ cd /path/to/roles/juju4.MISP/test/vagrant
$ vagrant up
$ vagrant ssh

Role has also a packer config which allows to create image for virtualbox, vmware, eventually digitalocean, lxc and others. When building it, it's advise to do it outside of roles directory as all the directory is upload to the box during building and it's currently not possible to exclude packer directory from it (hashicorp/packer#1811)

$ cd /path/to/packer-build
$ cp -Rd /path/to/juju4.MISP/packer .
## update packer-*.json with your current absolute ansible role path for the main role
## you can add additional role dependencies inside setup-roles.sh
$ cd packer
$ packer build packer-*.json
$ packer build -only=virtualbox-iso packer-*.json
## if you want to enable extra log
$ PACKER_LOG_PATH="packerlog.txt" PACKER_LOG=1 packer build packer-*.json
## for digitalocean build, you need to export TOKEN in environment.
##  update json config on your setup and region.
$ export DO_TOKEN=xxx
$ packer build -only=digitalocean packer-*.json
## for Azure
$ . ~/.azure/credentials
$ packer build azure-packer-centos7.json
$ packer build -var-file=variables.json azure-packer-centos7.json

Troubleshooting & Known issues

Troubleshooting

$ tail /var/log/apache2/misp.*
$ tail /var/www/MISP/app/tmp/logs/*.log
$ cd /var/www/MISP/app/Console && ./cake CakeResque.CakeResque tail

Known bugs

  • in /var/www/MISP/app/tmp/logs/error.log Error: [MissingTableException] Table logs for model Log was not found in datasource default. check misp database exists in mysql and is filled

  • MISP curl_tests.sh is made to run once unlike kitchen verify. If repeated, this test will fail.

  • if using privileged docker and a host with mysql, you might have the following issue

mysqld[29176]: /usr/sbin/mysqld: error while loading shared libraries: libaio.so.1: cannot stat shared object: Permission denied

moby/moby#7512

  • docker and redis can have issue too and it might be necessary to edit systemd config on xenial see task 'docker redis workaround ???'

  • CI failing sometimes on Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction. Seems related to Issue 5004 - Open

  • Ubuntu 22.04 seems unsupported as php8.1 and app/composer.json requires php >=7.2.0,<8.0.0

  • Error: Database connection \"Mysql\" is missing, or could not be created. can be cause if multiple php versions are present and wrong version is called from cli.

  • PHP Fatal error: Uncaught TypeError: Return value of Symfony\\Component\\Process\\Process::close() must be of the type int, null returned in phar:///usr/local/bin/composer/vendor/symfony/process/Process.php:1466 (rhel/rockylinux8 and 9): root cause not identified, possibly container/docker related as only failing in molecule/docker and not bare github-hosted images.

FAQ

TODO

License

BSD 2-clause

ansible-misp's People

Contributors

dependabot[bot] avatar egypcio avatar juju4 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

ansible-misp's Issues

Getting an SELinux labelling issue on /var/www/MISP/app/tmp/cache/persistent/

Using: CentOS Linux release 7.5.1804 (Core)

I'm getting the following after running a playbook to apply the role:

==> /var/log/httpd/misp.local_error.log <==
[Sat Jun 30 00:20:55.445260 2018] [php7:warn] [pid 96330] [client 192.168.235.1:60756] PHP Warning:  _cake_core_ cache was unable to write 'cake_dev_en-au' to File cache in /var/www/MISP/app/Lib/cakephp/lib/Cake/Cache/Cache.php on line 327
[Sat Jun 30 00:20:55.445317 2018] [php7:warn] [pid 96330] [client 192.168.235.1:60756] PHP Warning:  /var/www/MISP/app/tmp/cache/persistent/ is not writable in /var/www/MISP/app/Lib/cakephp/lib/Cake/Cache/Engine/FileEngine.php on line 389

[Sat Jun 30 00:20:55.445360 2018] [php7:error] [pid 96330] [client 192.168.235.1:60756] PHP Fatal error:  Uncaught CacheException: Cache engine "_cake_core_" is not properly configured. Ensure required extensions are installed, and credentials/permissions are correct in /var/www/MISP/app/Lib/cakephp/lib/Cake/Cache/Cache.php:186\nStack trace:\n#0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Cache/Cache.php(151): Cache::_buildEngine('_cake_core_')\n#1 /var/www/MISP/app/Config/core.php(270): Cache::config('_cake_core_', Array)\n#2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Core/Configure.php(72): include('/var/www/MISP/a...')\n#3 /var/www/MISP/app/Lib/cakephp/lib/Cake/bootstrap.php(439): Configure::bootstrap(true)\n#4 /var/www/MISP/app/webroot/index.php(81): include('/var/www/MISP/a...')\n#5 {main}\n  thrown in /var/www/MISP/app/Lib/cakephp/lib/Cake/Cache/Cache.php on line 186

Running ausearch -m avc -ts recent show's it's an SELinux labelling issue and setenforce 0 works around the problem:

time->Sat Jun 30 00:17:11 2018
type=PROCTITLE msg=audit(1530281831.940:679): proctitle=2F7573722F7362696E2F6874747064002D44464F524547524F554E44
type=SYSCALL msg=audit(1530281831.940:679): arch=c000003e syscall=87 success=yes exit=0 a0=7fef46a818b8 a1=1 a2=7fef46a818bf a3=7fef5576f870 items=0 ppid=96327 pid=96332 auid=4294967295 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=4294967295 comm="httpd" exe="/usr/sbin/httpd" subj=system_u:system_r:httpd_t:s0 key=(null)
type=AVC msg=audit(1530281831.940:679): avc:  denied  { unlink } for  pid=96332 comm="httpd" name="myapp_cake_core_cake_console_eng" dev="dm-0" ino=51354795 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:httpd_sys_content_t:s0 tclass=file

The SELinux label for this directory, and possibly sub-directories, should be set to something that t_httpd can write to as part of the Ansible role.

Just adding a ticket now before I forget. I may find time to come back and fix it up with a PR but probably not any time soon.

Ansible Galaxy role/collection resolves to version 1.10

Hi.. when installing from ansible galaxy, the latest version of ansible-MISP is 1.1.0, which fails for me on Ubuntu 20.04 and Centos 7. When installing the role directly from GitHub it works correctly.

I have everything working fine, but others may be running into issues.

pip install latest lief version is broken on ubuntu 18.04

Well, lief released new version 0.10.0 and 0.10.1 and current pip releases for python 2.7 are broken on ubuntu 18.04. See travis job log here https://travis-ci.org/juju4/ansible-MISP/jobs/617449500

Collecting lief==0.10.1
  Using cached https://files.pythonhosted.org/packages/ee/b1/57241e2f5f7aac93d8d8d3ad46bf3f104a4f4ef171ca2eef38803f3868aa/lief-0.10.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-qm4nyK/lief/setup.py", line 306
        def format_version(version: str, fmt: str = fmt_dev, is_dev: bool = False):
                                  ^
    SyntaxError: invalid syntax
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-qm4nyK/lief/

Using version 0.9.0 fixes the problem. MISP team still has lief 0.9.0 in their installation documentation, so i dont think its a problem to "downgrade" here

Import Remi rpm repository keys error

Dear juju4,

I use Centos7 and I always get this error. I use two centos 7 instances on vmware. I updated all of the instances with this ansible module and added some usefull stuff. The preview yml file that I pasted here can be wierd so sorry about that.

  • name: install required programs
    hosts: [ansible_clients]
    remote_user: root
    vars:
    reboot_connect_timeout: 5
    reboot_post_reboot_delay: 15
    reboot_timeout: 600
    become: true

    tasks:

Upgrade RHEL family OS packages

  • name: Upgrade RHEL Family OS packages
    yum:
    name: '*'
    state: latest
    when: ansible_os_family == "RedHat"

Reboot after upgrade

  • name: Reboot host
    reboot:
    connect_timeout: "{{ reboot_connect_timeout }}"
    post_reboot_delay: "{{ reboot_post_reboot_delay }}"
    reboot_timeout: "{{ reboot_timeout }}"
    #install epel release
  • name: install epel
    yum:
    name: epel-release
    state: latest

Install everything else

  • name: install git
    yum:
    name: git
    state: latest

  • name: centos | installing open-vm-tools
    yum: name=open-vm-tools state=present
    when: ansible_os_family == "RedHat" and ansible_virtualization_type == "VMware"

  • name: centos | starting and enabling open-vm-tools
    service: name=vmtoolsd.service state=restarted enabled=yes
    when: ansible_os_family == "RedHat" and ansible_virtualization_type == "VMware"

  • name: install wget
    yum:
    name: wget
    state: latest

  • name: install python3
    yum:
    name: python3
    state: latest

  • name: install nano
    yum:
    name: nano
    state: latest

Install MISP

  • name: misp install
    hosts: [ansible_clients]
    roles:
    • juju4.misp

The point is that I always get thios error :

image
I hope you can help me.

Some modules don'tr appear to be installed properly

Adminsitration -> Server Settings -> Diagnostic :
Advanced attachment handler :
PyMisp : Not installed or version outdated.
[root@machine PyMISP]# git status
HEAD detached at 1dc2f66

Some of the libraries related to STIX are not installed. Make sure that all libraries listed below are correctly installed.

How do I figure which ones ?

Cortex module system…System not enabled

httpd listening on IPv6 only

Dear juju4,

Currently with the Template file "templates/apache2-misp.conf.j2" and the config file "defaults/main.yaml", the listening port doesn't have a host/IP specified.

This can cause some issues, wherein the httpd server might listen only on IPv6, thus not responding to requests made on IPv4.

A solution would be to add a variable for the "misp_listening_ip" and put in front of the listening port in the template.
templates/apache2-misp.conf.j2 -> Line 6

Listen {{ misp_listening_ip }}:{{ misp_base_port }}

The issue was observed on a fresh CentOS7 install using your Ansible Playbook.

Best regards,
Krypterya

Works with VMware vSphere, between Ubuntu 20.04 servers?

Hi, I have tried many times to install from an Ubuntu server to an other Ubuntu server, but the playbook always failed or I could not reach the starting screen from the browser. I have checkd many things, but I dont know what more I can do. So if anyone know this role works with VMware vSphere client, with ubuntu server 20.04, or centos 8?

Clean install has small nits / issues

After installing MISP using the role I went to fix issues in the UI and got the following marked as red :
Advanced attachment handler
The advanced attachment tools are used by the add attachment functionality to extract additional data about the uploaded sample.
PyMISP:… Not installed or version outdated.

[[email protected] PyMISP]$ git status
HEAD detached at 1dc2f66
nothing to commit, working directory clean
[[email protected] PyMISP]$

Current libraries status…Some of the libraries related to STIX are not installed. Make sure that all libraries listed below are correctly installed.
STIX library version…
CYBOX library version…
MIXBOX library version…
MAEC library version…
PYMISP library version…

Any idea why?

Installing role from ansible-galaxy fails

$ ansible-galaxy install juju4.MISP
- downloading role 'MISP', owned by juju4
- downloading role from https://github.com/juju4/ansible-MISP/archive/v0.8.tar.gz
- extracting juju4.MISP to /home/yaleman/.ansible/roles/juju4.MISP
- juju4.MISP (v0.8) was installed successfully
- adding dependency: juju4.redhat-epel
- downloading role 'redhat-epel', owned by juju4
 [WARNING]: - juju4.redhat-epel was NOT installed successfully: - sorry, juju4.redhat-epel was not found on https://galaxy.ansible.com.

Ansible Installation on Ubuntu 22.04 error installing pip requirements

Prerequisites

  • Ensure no duplicate issue
  • Using an up-to-date latest release or tag
  • Tested an up-to-date latest HEAD
  • Collected play logs on verbose mode aka ansible-playbook -vvv playbook.yml. Redact any sensitive information.
  • Ensuring using latest stable underlying software (ansible, operating systems...)

Your Environment

  • Version used: Python 3.10.12
  • Server type and version: Virtual Machine on VMWare ESXi
  • Operating System and version: Ubuntu 22.04 server

Expected behavior

Ansible Skript will install misp by executing the provided role.

Actual behavior

I included the Role in an extra Playbook:

---
- name: install ansible-misp role
  hosts: all
  gather_facts: yes
  remote_user: "{{ remoteuser }}"
  become: True
  become_user: root
  roles:
    - ansible-MISP

remoteuser is a working ssh user with sudo privileges.

If i run the role i get the output shown in 240328_misp_ansible_ubuntu2204_error.

If i run the role with -vvv i get the output shown in 240328_misp_ansible_ubuntu2204_error-vvv.

If i try to run the Command directly i get the errors shown in 240328_misp_ansible_ubuntu2204_error_manueller_fehler.

Steps to reproduce

  • Set up a new Ubuntu 22.04 server
  • run the ansible role

Possible Solution (Not obligatory)

No response

More context

I am not able to get misp on Ubuntu 22.04 server running. There seems to be an issue installing the pip requirements. I tried installing them by hand but kept running in errors.

Relevant log output

See attached Files

Extra attachments

240328_misp_ansible_ubuntu2204_error_manueller_fehler.txt
240328_misp_ansible_ubuntu2204_error-vvv.txt
240328_misp_ansible_ubuntu2204_error.txt

Code of Conduct

  • I agree to follow this project's Code of Conduct

Rocky Linux support

Hi I'd like this role to support rocky linux.
It's a drop in replacement for CentOS afaik so I was hoping it wouldn't be much work.

Thanks in advance 😄

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.