abuseio / abuseio

AbuseIO is a toolkit to receive, process, correlate and notify about abuse reports received by network operators, typically hosting and access providers.

AbuseIO - Abusemanagement tools

AbuseIO is a toolkit to receive, process, correlate and notify end users about abuse reports received by network operators, typically hosting and access providers. The purpose is to consolidate efforts by various companies and individuals to automate and improve the abuse handling process.

Official Documentation

Documentation for AbuseIO can be found in its own repository.

An online version of the documentation can be found on https://docs.abuse.io/

Contributing

Thank you for considering contributing to AbuseIO! The contribution guide can be found in the AbuseIO website.

Security Vulnerabilities

If you discover a security vulnerability within AbuseIO, please send an e-mail to the AbuseIO CERT at [email protected] (GPG available on Key servers). All security vulnerabilities will be promptly addressed.

License

AbuseIO is open-sourced software licensed under the GNUv2 license

abuseio's Issues

Add a RBL scanner

implementing a rblscanner that can can ASN, ipblock, or file based IP set

Override return-path header in outgoing notifications

If the return path is not explicitly specified it's set to the user running the notifier script, causing mail bounces to be sent to that address (which might not be what you want).

So:

• Make it a configurable option in settings.conf (NOTIFICATIONS_BOUNCE ?)
• If the option is not set, fall back to the NOTIFICATIONS_FROM address

PHP's mail() method does not support it directly, but the return path can be set using the mail() additional_parameters option set to "-f $return_address"

Add a EML uploader

Add a EML uploader for MDA processing

create a JSON RPC interface

and include issue #16

add language selection

Add a button to select available languages to change the language within the view.

optionally provide a default language setting to a specific customer

Add more feed parsers / collectors

Datafeeds including collection URL(s):

Arbor SSH Attackers feed

http://atlas-public.ec2.arbor.net/public/ssh_attackers

Autoshun bots, worms and spam engines

http://www.autoshun.org/files/shunlist.csv

CleanMX

http://support.clean-mx.de/clean-mx/xmlphishing?response=alive&format=csv&domain= http://support.clean-mx.de/clean-mx/xmlviruses?response=alive&format=csv&domain=

Brute Force Blocker project

http://danger.rulez.sk/projects/bruteforceblocker/blist.php

DragonBot

http://dragonresearchgroup.org/insight/sshpwauth.txt https://dragonresearchgroup.org/insight/vncprobe.txt

Malc0de Database Feed

http://malc0de.com/rss

abuse.ch

https://zeustracker.abuse.ch/rss.php https://palevotracker.abuse.ch/?rssfeed https://feodotracker.abuse.ch/feodotracker.rss

Open blacklist

https://www.openbl.org/lists/date_all.txt

phishtank:

http://data.phishtank.com/data/%s/online-valid.xml.bz2

ProjectHoneyPotBot

    "http://www.projecthoneypot.org/list_of_ips.php?by=1&rss=1",
    "http://www.projecthoneypot.org/list_of_ips.php?by=4&rss=1",
    "http://www.projecthoneypot.org/list_of_ips.php?by=7&rss=1",
    "http://www.projecthoneypot.org/list_of_ips.php?by=10&rss=1",
    "http://www.projecthoneypot.org/list_of_ips.php?by=13&rss=1",
    "http://www.projecthoneypot.org/list_of_ips.php?by=16&rss=1",
    "http://www.projecthoneypot.org/list_of_ips.php?by=19&rss=1

descriptions = {'H': 'harvester', 'S': 'spam server', 'D': 'dictionary attacker', 'W': 'bad web host', 'C': 'comment spammer', 'R': 'rule breaker'} http://www.projecthoneypot.org/ip_

extra: https://zeltser.com/malicious-ip-blocklists/

add setting in feedConfigurations which disables notifications for a feed

Some feeds you want only to log and dont want that ticket thats generated being notified to the contact (e.g. for internal use only). It should be hidden from the customer optionally so

hidden_from_contact = true
disable_notifications = true

in parser/config/Parser.php or something

Option to create support bundle

Somethings people ask for support, but created local modification or settings that might interfere with operations. When a support bundle is triggered then:

• Validate all files by a MD file provided with each installation
• Bundle all modifications by create diffs in a patch style to path a new installation with the same code.
• Encrypt the data with a GPG key from AbuseIO packaged into the release
• Manually set the e-mail address where to send the data to
• A local validation / dont submit option where a compressed file with results is saved. Also a simple report that shows if the software is correctly installed / not modified.

Bug in ticket list: Ticket type 'Escalation' is shown as 'types.type..name'

Reproduction

• Create a new event and fill in the required fields
• Select event type Escalation
• Submit
• Check the value for your entry that's in the Type column, it will be 'types.type..name.
• You will also notice that the type_id field is empty in the database entry for your column. This is because the ENUM expects ESCALATION, not ESCALATED.

I've submitted a PR that fixes this

Complete the ASH info texts

The ASH information texts should be expanded to a point where a end-user was send to page where everything is listed what he needs to resolve it. There is no limit on the document size.

500 Internal server error at /admin/tickets when searching

When typing something in the search field the following url is being requested which yields into an http 500 error.

Server:nginx/1.4.6 (Ubuntu)
X-Powered-By:PHP/5.5.9-1ubuntu4.14

/admin/tickets/search/query/?draw=2&columns%5B0%5D%5Bdata%5D=&columns%5B0%5D%5Bname%5D=&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=false&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D=id&columns%5B1%5D%5Bname%5D=id&columns%5B1%5D%5Bsearchable%5D=true&columns%5B1%5D%5Borderable%5D=true&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B1%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B2%5D%5Bdata%5D=ip&columns%5B2%5D%5Bname%5D=ip&columns%5B2%5D%5Bsearchable%5D=true&columns%5B2%5D%5Borderable%5D=true&columns%5B2%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B2%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B3%5D%5Bdata%5D=domain&columns%5B3%5D%5Bname%5D=domain&columns%5B3%5D%5Bsearchable%5D=true&columns%5B3%5D%5Borderable%5D=true&columns%5B3%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B3%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B4%5D%5Bdata%5D=type_id&columns%5B4%5D%5Bname%5D=type_id&columns%5B4%5D%5Bsearchable%5D=true&columns%5B4%5D%5Borderable%5D=true&columns%5B4%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B4%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B5%5D%5Bdata%5D=class_id&columns%5B5%5D%5Bname%5D=class_id&columns%5B5%5D%5Bsearchable%5D=true&columns%5B5%5D%5Borderable%5D=true&columns%5B5%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B5%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B6%5D%5Bdata%5D=event_count&columns%5B6%5D%5Bname%5D=event_count&columns%5B6%5D%5Bsearchable%5D=false&columns%5B6%5D%5Borderable%5D=true&columns%5B6%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B6%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B7%5D%5Bdata%5D=notes_count&columns%5B7%5D%5Bname%5D=notes_count&columns%5B7%5D%5Bsearchable%5D=false&columns%5B7%5D%5Borderable%5D=true&columns%5B7%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B7%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B8%5D%5Bdata%5D=status_id&columns%5B8%5D%5Bname%5D=status_id&columns%5B8%5D%5Bsearchable%5D=true&columns%5B8%5D%5Borderable%5D=true&columns%5B8%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B8%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B9%5D%5Bdata%5D=actions&columns%5B9%5D%5Bname%5D=&columns%5B9%5D%5Bsearchable%5D=false&columns%5B9%5D%5Borderable%5D=false&columns%5B9%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B9%5D%5Bsearch%5D%5Bregex%5D=false&order%5B0%5D%5Bcolumn%5D=0&order%5B0%5D%5Bdir%5D=asc&start=0&length=10&search%5Bvalue%5D=O&search%5Bregex%5D=false&_=1452093251886

Query strings:

draw:2
columns[0][data]:
columns[0][name]:
columns[0][searchable]:true
columns[0][orderable]:false
columns[0][search][value]:
columns[0][search][regex]:false
columns[1][data]:id
columns[1][name]:id
columns[1][searchable]:true
columns[1][orderable]:true
columns[1][search][value]:
columns[1][search][regex]:false
columns[2][data]:ip
columns[2][name]:ip
columns[2][searchable]:true
columns[2][orderable]:true
columns[2][search][value]:
columns[2][search][regex]:false
columns[3][data]:domain
columns[3][name]:domain
columns[3][searchable]:true
columns[3][orderable]:true
columns[3][search][value]:
columns[3][search][regex]:false
columns[4][data]:type_id
columns[4][name]:type_id
columns[4][searchable]:true
columns[4][orderable]:true
columns[4][search][value]:
columns[4][search][regex]:false
columns[5][data]:class_id
columns[5][name]:class_id
columns[5][searchable]:true
columns[5][orderable]:true
columns[5][search][value]:
columns[5][search][regex]:false
columns[6][data]:event_count
columns[6][name]:event_count
columns[6][searchable]:false
columns[6][orderable]:true
columns[6][search][value]:
columns[6][search][regex]:false
columns[7][data]:notes_count
columns[7][name]:notes_count
columns[7][searchable]:false
columns[7][orderable]:true
columns[7][search][value]:
columns[7][search][regex]:false
columns[8][data]:status_id
columns[8][name]:status_id
columns[8][searchable]:true
columns[8][orderable]:true
columns[8][search][value]:
columns[8][search][regex]:false
columns[9][data]:actions
columns[9][name]:
columns[9][searchable]:false
columns[9][orderable]:false
columns[9][search][value]:
columns[9][search][regex]:false
order[0][column]:0
order[0][dir]:asc
start:0
length:10
search[value]:O
search[regex]:false

Evidence from failed jobs are getting pruned while they should not be

Because of an logic/handler error only running jobs are checked before a pruning starts. If a job was failed then the EML evidence would be deleted from the the disk while the jobs is still depending on it. Because the EML is deleted there is no way to recover or retry the failed job.

All installations from AbuseIO 4 up to RC3 are affected by this bug and its highly recommended to upgrade to RC4 or newer.

Details on this fix can be found here:
0dc500a

commit #0dc500a22c03eb4ab9084267465c191980261d1c

This bug was reported by @suchpool

Update report view with customer resolved/ignored fields

there is no resovled/ignored in the view yet, but abusedesks need to know such status. I suggest using some colours and a legend to show different statusses.

Project Honeypot parser does not match reports

I don't know if i misconfigured something at project honeypot or if they changed their format. The mails i receive look like this:

Below is your Project Honey Pot Monitor Report for 09/02/2015 - 10/03/2015. To view your report in more detail, including more information on any of the listed IPs, please visit your online report at:

http://www.projecthoneypot.org/monitor_report.php

The report summarizes the IPs you currently are monitoring. If you have changed your Monitor Settings during the report period then the report will reflect the settings at the end of the period.
Need to monitor more IPs? You can now sign up for the Advanced Monitor Service. For more details, click here:

https://www.projecthoneypot.org/monitor_upgrade_1.php

This email was generated and sent to your account by a Project Honey Pot account held by [email protected].

---------------------------------------


Report Date Range: 09/02/2015 - 10/03/2015
IPs Monitored: 4,608
Malicious IPs : 5
- Harvesting IPs: 3
- Email Spamming IPs: 0
- Dictionary Attacks IPs: 0
- Comment Spamming IPs: 2
Other Potentially Suspicious IPs : 0

IPs Engaged In Harvesting On Your Network:

1.2.3.4


IPs Engaged In Email Spamming On Your Network:

none


IPs Engaged In Dictionary Attacks On Your Network:

none


IPs Engaged In Comment Spamming On Your Network:

1.2.3.4


Other Potentially Suspicious IPs On Your Network:

none

---------------------------------------

Please see your Monitor Report for more details on the activity of the reported IPs. You have elected to receive this message Weekly. To change the frequency with which you receive this message, or stop receiving it entirely, please visit:

http://www.projecthoneypot.org/monitor_settings.php

If you would rather not receive this report unless there is activity on your network, please visit:

http://www.projecthoneypot.org/monitor_settings.php

Use MVC framework and/or DAL (database abstraction layer)

Rewriting all the code to MVC. The main developer group decided on Laravel for now, with CakePHP as a secondary choice.

• work includes PDO driver
• work includes deadlock prevention (due to indexing)

Join our development team, Any help is greatly appriciated!

Add tooling to manually scan local network

It would be great to add the tooling to people can do (re)scans themselves. This would allow quicker uptime reports or check if a customer actually solved the problem. If the tools already exist we include them, so don't rebuild existing open source tools but rather include them!

add NIPAP support module in AbuseIO4

Based on the example from Jurrian van Iersel

requires : "zendframework/zend-xmlrpc": "2.5.*"

getHttpClient(); $httpClient->setAuth($username, $password); $result = $RpcClient->call( 'search_prefix', array( array( 'auth' => array( 'authoritative_source' => 'nipap' ), 'query' => array( 'operator' => 'equals', 'val1' => 'prefix', 'val2' => '1.0.0.1/32' ), 'search_options' => array( 'include_all_parents' => true, ), ) ) ); /* * Walk results in reverse to get the most specific match to use. (if host has no contact, then move up * every prefix until something is found. NIPAP does not inherit AVPS objects which we should ask them to do */ if (is_array($result['result'])) { $resultRev = array_reverse($result['result']); $foundCustomerID = false; foreach ($resultRev as $key => $resultRevSet) { if (!empty($resultRevSet['avps']['AbuseIO_Name']) && !empty($resultRevSet['avps']['AbuseIO_Contact']) && !empty($resultRevSet['avps']['AbuseIO_AutoNotify']) ) { print_r($resultRevSet['avps']); } if (!empty($resultRevSet['customer_id']) !empty($foundCustomerID) && ) { $foundCustomerID = $resultRevSet['customer_id']; } } } /* * If no matches try to poll a customer portal API that holds the Name, Contact and AutoNotify data * which we collect by using the $result['result'][0] of the most specific match */ if (!empty($foundCustomerID)) { // TODO } ``` }

Option to send statistics

Several research institutions are interested in how abuse is handled and mostly their uptime from the ISP point of view. Such parties only get feed information and have no idea if an ISP actually received or acted on a abuse incident.

Any kind of statistics should be fully anonymised with a locally generated key with a one-way encryption. The new ID's should be the same for the same IP's so aggregation can still be done. TU/d should provide us with information on data collection options.

This option should be disabled by default and need to be very carefully enabled, e.g. a key must manually be generated, enable the option and manually set upload locations.

call to $this->failed() does not work in the command structure

In a failed attempt to parse events, the call to $this->failed() does not work in the command structure:

[2015-07-24 22:10:05] local.ERROR: exception 'ErrorException' with message 'call_user_func() expects parameter 1 to be a valid callback, cannot access protected method AbuseIO\Commands\EmailProcess::failed()' in /opt/abuseio/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php:77
Stack trace:
#0 [internal function]: Illuminate\Foundation\Bootstrap\HandleExceptions->handleError(2, 'call_user_func(...', '/opt/abuseio/ve...', 77, Array)
#1 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(77): call_user_func(Array, Object(AbuseIO\Commands\EmailProcess))
#2 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Queue/Jobs/Job.php(206): Illuminate\Queue\CallQueuedHandler->failed(Array)
#3 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(243): Illuminate\Queue\Jobs\Job->failed()
#4 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(201): Illuminate\Queue\Worker->logFailedJob('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob))
#5 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(159): Illuminate\Queue\Worker->process('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), '1', '0')
#6 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(109): Illuminate\Queue\Worker->pop(NULL, 'emails', '0', '3', '1')
#7 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(71): Illuminate\Queue\Console\WorkCommand->runWorker(NULL, 'emails', '0', '128', false)
#8 [internal function]: Illuminate\Queue\Console\WorkCommand->fire()
#9 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Container/Container.php(502): call_user_func_array(Array, Array)
#10 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Console/Command.php(149): Illuminate\Container\Container->call(Array)
#11 /opt/abuseio/vendor/symfony/console/Command/Command.php(259): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Console/Command.php(135): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /opt/abuseio/vendor/symfony/console/Application.php(878): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /opt/abuseio/vendor/symfony/console/Application.php(195): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /opt/abuseio/vendor/symfony/console/Application.php(126): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(100): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /opt/abuseio/artisan(36): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 {main}

Add a downstream to endusers

Endusers (ISP customers) that are large enough might be interested in having their own AbuseIO. Using RPC and a CustomerCode based token we could allow certain customer to receive a downstream of reports and upstream of notes/status to the ISP.

This way the enduser will not have to parse the data on their own.

/admin/users/create doesn't save user

When creating a user via the /admin/users/create page;

• It's not being saved to the DB
• Doesn't show an error when both password fields are empty
• If not already present; add a field to add roles to the user (as you cannot log in without assigned roles)

Branding templates

I'd like to have a branding possibility in the product. A brand contains some basic information like company name, contact info, logo and ash template.

The relations would be:
Netblock->hasOne('AbuseIO\Models\Brand')
Domain->hasOne('AbuseIO\Models\Brand')

Brand->hasMany('AbuseIO\Models\Netblock')
Brand->hasMany('AbuseIO\Models\Domain')

Add option to ignore certain classes

SHADOWSERVER_IGNORE should be replaced and implemented in a more generic way, to allow users to ignore other classes as well.

Back button in admin/ticket/# view resolves to itself

Reproduce with:

• Open ticket: /admin/tickets/5
• Change status to "Closed"
• Press "Back" and find yourself being directed to "/admin/tickets/5" instead of "/admin/tickets"

Make 'Analytics and statistics' sortable by count

Please make the 'Analytics and statistics' page sortable by the number of occurences (the 'count' column), so you can easily see the most 'popular' classifications.

Unable to filter closed tickets

So I tried to search for "Open" which yields 0 results, upon asking @kruisdraad he said:
By default the field will search all columns for the value but the "Open" status equals to "1". Searching for "1" however will list all rows containing a "1" in one of the fields.

Being able to filter on a specific column value ("Status: open") would be very convenient but for now the solution should make it possible to work trough only open tickets.

Create migration script from 3.1 to 4.0

We need a script so 3.x users can migrate their data into 4.0. As the format is heavily changed the process should be providing two options:

• Quick migration

This basically only grabs the ticket data and add that data (duplicate) for the number of times the event was seen so they reach the same counts, start and end dates. As AbuseIO3 only used the parsed evidence from the first evidence report it does not provide extra info, but does migrate all existing (viewable) data into the system.

• Full migration

This processes the entry evidence again that was logged per ticket. Only looking at the specific IP in that evidence and then adding each event into the database. This will take a long time, however this would add a lot more details into the parsed evidence and search collections.

In both cases the migration should:

• Lock down the system
• Check the destination system is blank
• Make sure all ticket numbers are kept the same
• Finish with a validation of old vs new data

Prevent sending "Alert: There are x jobs that are stuck" every minute

When jobs are stuck the "php /opt/abuseio/artisan schedule:run" cronjob will sent out an alert every minute which seems a bit excessive.

Perhaps you could put some threshold in place to prevent flooding mailboxes?

Implement bounce to mailbox for unparseable abuse reports

Currently unparseable abuse reports are forwarded as an attachment. This makes following up from a ticket system quite cumbersome.

It should be possible to bounce those reports instead, so the originator is unmodified and the report be easily replied to. Also no attachments need to be downloaded and opened to see what the report is about.

The forwarding behavior should be configurable ("bounce" or "forward as attachment").

Bouncing might cause trouble with things like SPF, so this should be investigated and explained in the documentation. SPF checks should perhaps be bypassed for all mail sent from an abuseio instance, for example.

Improve logging

Instead of using FUNCTION in every logger call, determine and log the context directly in logger() method.

Find all logger calls and improve text or remove if too redundant.

Also split out debug, info, warn, crit from console messages

create an (csv) export option from search results

create an (csv) export option from search results so abusedeskers can use the data in other ways like manual reporting to customers

Generate information for a Transparancy Report

Creating a tool that would generate (additional) information for a Transparancy Report. Some ISP's do this yearly ... others might do it if they wouldn't need to spend days collecting all the information needed to such a report.

It should report stuff like:

Amount of cases per classification
Types of cases (if typing is implemented, as INFO types arent really abuse)
Amount of cases that are resolved (or not)
Average time (perhaps per classification) for customers to resolve the case
etc

Add demo setting

In demo mode (demo.abuse.io, but also demonstrations) it would be handy to anonymize some of the data shown like the IP into 213.213.x.x and hide customer information after the first letter.

Translate the mail template example into english

Translate the mail template example into english. Its currently in dutch, but AbuseIO's native language is english. We dutch variant could be left as mail.template.example.NL

cidr notation conversion

When entering IP ranges in netblocks, it would be nice to have cidr notated ranges automatically converted to start / last ip's and filled into the respective fields.

Create fieldMaps for infoblobs

Create fieldMaps to create more user friendly infoblobs.

IPv6 events cannot link to contact data

At some points long2ip is used for database storage, however this only works for IPv4 addresses. The event is parsed correctly, but a contact will never be resolved.

In the v4.0 branch this was solved by switching to varbinary(16) fields.

test irc notifications

Allow parser mapping to be a configurable item

switch the parser from static to configurable settings for ISP's that require a little tweaking. For example SpamExperts parser the from and body are whitelabeled and unique.

Add a set of CLI tools that provide same functions as the webgui

Not everyone likes to used graphical interface and it would be handy to have CLI tools to create scripted interactions with AbuseIO

Date/Time repeatedly strtotime()'d in view

The culprit is this code:

    if (strtotime($input['timestamp']) !== false) {
        $input['timestamp'] = strtotime($input['timestamp']);
    }

So the solution that involves the least amount of changing things is to solve the above which I've done, and then in the view we convert the timestamp field to a string with date('Y-m-d H:i:s'). This is the path of least friction that I can see. But also there's a little problem there. Laravel Form Builder has a set order of preference when considering what value to put in a field even when we specify what we want. It will look to old input first, and then consider the one we've provided. We want to override what's in the old input. A hack would be to create a helper function that simply overrides the old input, but I don't think we want hacks in the codebase?

So for now I'm engaging the maintainers of the Form Builder package to see if they can consider and accept my PR which enables us (as an option) to tell the Form Builder to use the value provided if we want that, and not to look for old input.

Whatever the end result will be, soon as we have this fixed I'll drop in the datepicker

Password reset yields: InvalidArgumentException in FileViewFinder.php line 137: View [emails.password] not found.

Credits @michelzimmerman:

When trying to request a password link at /password/email by filling in the e-mail address and submitting the form the following error shows.

Jan 15 11:08:38 abuse01 abuseio[29401]: production.ERROR: exception 'InvalidArgumentException' with message 'View [emails.password] not found.' in /opt/abuseio/vendor/laravel/framework/src/Illuminate/View/FileViewFinder.php:137 Stack trace: #0 /opt/abuseio/vendor/laravel/framework/src/Illuminate/View/FileViewFinder.php(79): Illuminate\View\FileViewFinder->findInPaths('emails.password', Array) #1 /opt/abuseio/vendor/laravel/framework/src/Illuminate/View/Factory.php(151): Illuminate\View\FileViewFinder->find('emails.password') #2 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Mail/Mailer.php(464): Illuminate\View\Factory->make('emails.password', Array) #3 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Mail/Mailer.php(331): Illuminate\Mail\Mailer->getView('emails.password', Array) #4 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Mail/Mailer.php(171): Illuminate\Mail\Mailer->addContent(Object(Illuminate\Mail\Message), 'emails.password', NULL, NULL, Array) #5 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Auth/Passwords/PasswordBroker.php(119): Illuminate\Mail\Mailer->send('emails.password', Array, Object(Closure)) #6 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Auth/Passwords/PasswordBroker.php(93): Illuminate\Auth\Passwords\PasswordBroker->emailResetLink(Object(AbuseIO\Models\User), '965c2ffc9d26fed...', Object(Closure)) #7 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(219): Illuminate\Auth\Passwords\PasswordBroker->sendResetLink(Array, Object(Closure)) #8 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Foundation/Auth/ResetsPasswords.php(37): Illuminate\Support\Facades\Facade::__callStatic('sendResetLink', Array) #9 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Foundation/Auth/ResetsPasswords.php(37): Illuminate\Support\Facades\Password::sendResetLink(Array, Object(Closure)) #10 [internal function]: AbuseIO\Http\Controllers\Auth\PasswordController->postEmail(Object(Illuminate\Http\Request)) #11 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(256): call_user_func_array(Array, Array) #12 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(164): Illuminate\Routing\Controller->callAction('postEmail', Array) #13 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(112): Illuminate\Routing\ControllerDispatcher->call(Object(AbuseIO\Http\Controllers\Auth\PasswordController), Object(Illuminate\Routing\Route), 'postEmail') #14 [internal function]: Illuminate\Routing\ControllerDispatcher->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request)) #15 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(139): call_user_func(Object(Closure), Object(Illuminate\Http\Request)) #16 /opt/abuseio/app/Http/Middleware/RedirectIfAuthenticated.php(45): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request)) #17 [internal function]: AbuseIO\Http\Middleware\RedirectIfAuthenticated->handle(Object(Illuminate\Http\Request), Object(Closure)) #18 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(124): call_user_func_array(Array, Array) #19 [internal function]: Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request)) #20 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): call_user_func(Object(Closure), Object(Illuminate\Http\Request)) #21 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(114): Illuminate\Pipeline\Pipeline->then(Object(Closure)) #22 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(69): Illuminate\Routing\ControllerDispatcher->callWithinStack(Object(AbuseIO\Http\Controllers\Auth\PasswordController), Object(Illuminate\Routing\Route), Object(Illuminate\Http\Request), 'postEmail') #23 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/Route.php(203): Illuminate\Routing\ControllerDispatcher->dispatch(Object(Illuminate\Routing\Route), Object(Illuminate\Http\Request), 'AbuseIO\Http\Co...', 'postEmail') #24 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/Route.php(134): Illuminate\Routing\Route->runWithCustomDispatcher(Object(Illuminate\Http\Request)) #25 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/Router.php(708): Illuminate\Routing\Route->run(Object(Illuminate\Http\Request)) #26 [internal function]: Illuminate\Routing\Router->Illuminate\Routing\{closure}(Object(Illuminate\Http\Request)) #27 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(139): call_user_func(Object(Closure), Object(Illuminate\Http\Request)) #28 [internal function]: Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request)) #29 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): call_user_func(Object(Closure), Object(Illuminate\Http\Request)) #30 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/Router.php(710): Illuminate\Pipeline\Pipeline->then(Object(Closure)) #31 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/Router.php(675): Illuminate\Routing\Router->runRouteWithinStack(Object(Illuminate\Routing\Route), Object(Illuminate\Http\Request)) #32 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Routing/Router.php(635): Illuminate\Routing\Router->dispatchToRoute(Object(Illuminate\Http\Request)) #33 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(236): Illuminate\Routing\Router->dispatch(Object(Illuminate\Http\Request)) #34 [internal function]: Illuminate\Foundation\Http\Kernel->Illuminate\Foundation\Http\{closure}(Object(Illuminate\Http\Request)) #35 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(139): call_user_func(Object(Closure), Object(Illuminate\Http\Request)) #36 /opt/abuseio/app/Http/Middleware/Locale.php(51): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request)) #37 [internal function]: AbuseIO\Http\Middleware\Locale->handle(Object(Illuminate\Http\Request), Object(Closure)) #38 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(124): call_user_func_array(Array, Array) #39 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(50): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request)) #40 [internal function]: Illuminate\Foundation\Http\Middleware\VerifyCsrfToken->handle(Object(Illuminate\Http\Request), Object(Closure)) #41 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(124): call_user_func_array(Array, Array) #42 /opt/abuseio/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php(49): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request)) #43 [internal function]: Illuminate\View\Middleware\ShareErrorsFromSession->handle(Object(Illuminate\Http\Request), Object(Closure)) #44 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(124): call_user_func_array(Array, Array) #45 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(62): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request)) #46 [internal function]: Illuminate\Session\Middleware\StartSession->handle(Object(Illuminate\Http\Request), Object(Closure)) #47 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(124): call_user_func_array(Array, Array) #48 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request)) #49 [internal function]: Illuminate\Cookie\Middleware\AddQueuedCookiesToResponse->handle(Object(Illuminate\Http\Request), Object(Closure)) #50 /opt/abuseio/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(124): call_user_func_arra

Column selections

The option to chose what columns you want to see or hide. This can be applied to all table views.

Bug in New Event form: Information field JSON keeps nesting itself

Reproduction

• Create a new event, don't put any values (or put any values in the Information field)
• Submit
• Check the value for now in the _Information_field. Make any edits (or not) and submit again
• The JSON keeps getting nested

I've submitted a PR that fixes this. But still, the textarea placeholder hints that YAML is what should be in here. Do we make for YAML then to be what shows here and is expected? There are parsers that can make this not too hard.

Send the original subject on "Exception notification" mails

I think that would be good, cause hard to seek mails back with a same subject. Maybe it shoud me a bool option in the config file...

Show 'missing role' error on /auth/login

If the user has no role(s) he will not be able to use his login. This isn't clear for the user as no error is being shown.

Default IPA support for nipap and phpmyipam

Build in two packages to show how contact data can be collected by using nipap and phpmyipam as these are the commonly used IPA systems. A third would be an RIPE API example (as people used different fields) which is usefull.

Empty contacts shows DataTables error in 4.0.1

Fresh install of AbuseIO 4.0.1 on Ubuntu 16.04 (PHP7) without seeding the DB returns an AJAX error on the Contacts page:

DataTables warning: table id=contacts-table - Ajax error. For more information about this error, please see http://datatables.net/tn/7

Add a Feedback loop parsers

Many receivers, including AOL, Yahoo and Hotmail, offer feedback loops to notify senders when complaints are generated by their users via the "This is Spam/Junk" button.

AbuseIO should be able to parse such notifications

Disregard reports for IP addresses not in Customer blocks

Hi,

Sometimes you subscribe for feeds that are not so configurable that will contain addresses not in your customer ranges. Would it make sense to add an option to disregard these reports ? Of course this can be coded in each parser..