GithubHelp home page GithubHelp logo

ntopng-udm's Introduction

ntopng for UDM/UDM pro

Distributed under MIT license

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Project Notes

Author: Carlos Talbot (@tusc69 on ubnt forums)

Installing

This is a prebuilt image of ntopng to run directly on an UDM or UDM PRO. The Docker image has been configured to perserve data between upgrades. ntopng uses an in memory database known as redis so writes to disk are minimal. In addition, the timeseries database (RRD) does not require much disk space. https://www.ntop.org/ntopng/ntopng-disk-requirements-for-timeseries-and-flows/. You can optionally configure the timeseries database to point to an external Influx database.

In order to install this image you will need to log into the UDM via ssh and type the following command:

podman pull tusc/ntopng-udm:latest

This will download the latest image to the UDM.

Next, we'll need to create two directories and download config files that will be saved between upgrades. This is a one time operation.

mkdir -p /mnt/data/ntopng/redis
mkdir -p /mnt/data/ntopng/lib
touch /mnt/data/ntopng/GeoIP.conf
curl -Lo /mnt/data/ntopng/ntopng.conf https://github.com/tusc/ntopng-udm/blob/master/ntopng/ntopng.conf?raw=true
curl -Lo /mnt/data/ntopng/redis.conf https://github.com/tusc/ntopng-udm/blob/master/ntopng/redis.conf?raw=true

Next, we want to create a container with ntopng running on https port 3001 using this image with the above config files.

podman run -d --net=host --restart always \
   --name ntopng \
   -v /mnt/data/ntopng/GeoIP.conf:/etc/GeoIP.conf \
   -v /mnt/data/ntopng/ntopng.conf:/etc/ntopng/ntopng.conf \
   -v /mnt/data/ntopng/redis.conf:/etc/redis/redis.conf \
   -v /mnt/data/ntopng/lib:/var/lib/ntopng \
   docker.io/tusc/ntopng-udm:latest

NOTE: If you prefer to use the external drive on the UMD pro to store the persistent data you can use the following to start up ntopng. Make sure to replace all references above from /mnt/data to /mnt/data_ext:

podman run -d --net=host --restart always \
   --name ntopng \
   -v /mnt/data_ext/ntopng/GeoIP.conf:/etc/GeoIP.conf \
   -v /mnt/data_ext/ntopng/ntopng.conf:/etc/ntopng/ntopng.conf \
   -v /mnt/data_ext/ntopng/redis.conf:/etc/redis/redis.conf \
   -v /mnt/data_ext/ntopng/lib:/var/lib/ntopng \
   docker.io/tusc/ntopng-udm:latest

Open a web browser page to your UDM's ip address with port 3001 at the end using https. For example: https://192.168.1.1:3001

If you have to reboot the UDM you'll have to restart the container. You can do so by typing the following:

podman start ntopng

Fortunately you can also take advantage of boostchicken's great tool to automatically start a Docker container after a reboot: https://github.com/boostchicken/udm-utilities/tree/master/on-boot-script

If you're interested in compiling your own version I have a Dockerfile available here that compiles ntopng from source: https://github.com/tusc/ntopng-udm/blob/master/source/Dockerfile

GeoIP integration

If you want to see country flags next to hosts you'll need to setup a free account with maxmind.com. Follow the instructions from the link below and save the downloaded GeoIP.conf file on the UDM in the path /mnt/data/ntopng/GeoIP.conf. You can use scp or winscp to transfer the file over. https://github.com/ntop/ntopng/blob/dev/doc/README.geolocation.md#using-geolocation-in-ntopng. Step 0 (geoipudate) has been done for you as it's included in this image.

When prompted on the version of geoipupdate select the option for older than 3.1.1.

Once you are done you can start the container. Anytime the docker container is started it will run a geoipupdate to download the latest GeoIP data.

Customize settings

The default instance will listen on the LAN interface (br0). You can edit the file /mnt/data/ntopng/ntopng.conf on the UDM to change the settings. The default is -i=br0 (LAN), n=1 ( Decode DNS responses and resolve all numeric IPs ) and -W3001 (enable HTTPS port)

NOTE If you comment out the -i interface and let ntopng startup listening to all interfaces you will have to wait up to 30 seconds for all interfaces to register. This will also consume additional CPU and memory resources so be careful with this option.

You can also customize the settings for the redis database if you want to eliminates database saves to storage. That file is located at /mnt/data/ntopng/redis.conf

Disable Redis

If you want to disable Redis and use an external server just set the env var "DISABLE_REDIS"

docker run -e DISABLE_REDIS=true tusc/ntopng-udm

Building

Build on your UDM or build on another device using buildx and targeting arm64

docker buildx build --platform linux/arm64 -t ntopng-udm:latest --load .

Upgrades

Whenever there is a new version of ntopng you can easily perform an upgrade by doing the following commands:

podman pull tusc/ntopng-udm:latest
podman stop ntopng
podman rm ntopng
podman run -d --net=host --restart always \
   --name ntopng \
   -v /mnt/data/ntopng/GeoIP.conf:/etc/GeoIP.conf \
   -v /mnt/data/ntopng/ntopng.conf:/etc/ntopng/ntopng.conf \
   -v /mnt/data/ntopng/redis.conf:/etc/redis/redis.conf \
   -v /mnt/data/ntopng/lib:/var/lib/ntopng \
   docker.io/tusc/ntopng-udm:latest

Uninstalling

To remove the docker instance and image you'll need to type the following at the UDM ssh prompt:

podman stop ntopng
podman rm ntopng
podman rmi docker.io/tusc/ntopng-udm  (or "docker rmi ntopng-image" if you installed the first release)

Console Lockout

If for whatever reason you find yourself locked out of the ntopng login prompt you can follow the steps on this page for resetting the password: https://www.ntop.org/guides/ntopng/faq.html#cannot-login-into-the-gui

You have to connect to the containter in order to run the redis commands as referenced in the FAQ. Do so by typing the following below. You can type "exit" to get out of the container when you're done.

podman exec -it ntopng bash

ntopng-udm's People

Contributors

boostchicken avatar roberttheprofessional avatar tusc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ntopng-udm's Issues

ntopng.conf -W vs -w

It seems that the -w is case sensitive. Your default one has -W and my UDM Pro wasn't happy

No podman on my UDM

Hi

I have this error on my UDM SE

root@UDM-SE:/# podman pull tusc/ntopng-udm:latest
-bash: podman: command not found

no WAN interface

I know ntopng can monitor wan traffic but if I edit the config to include the wan its never an option when I go back in. I've also just had it listen on all interfaces and WAN was still not an option. Am I missing something?

just wanted to say thanks

@tusc I wanted to say thanks for the NTP docker build you did, but i think they locked that thread over at the unifi site.
couldn't figure out a way to msg you there. so just wanted to drop a note to say thanks for that, it was exactly what i was looking for!

Unable to change password

When I log into ntopng for the first time with admin/admin credentials I'm prompted to change the password. After entering a new password I click the "Change Password" button, but nothing happens, so now I'm stuck.

I have ntopng running with podman on my UDM-P as per the instructions in the readme, I've done no tweaking of it at all.

smtp endpoint

I have setup a gmail smtp endpoint on pfsense ntopng no problem.

But with this docker setup I get this error:

Email send error: SSL peer certificate or SSH remote key was not OK.

any ideas how to debug?

HTTPS warning on connect

Why is the podman container running with https by default? I don't have a certificate for 192.168.1.1 and so my browser gives me a warning that the site is using https but the certificate is invalid. Does ntopng really need to run with https, wouldn't http suffice as it's an internal service?

If https makes sense, then could you please update the readme as to how to make it work nicely out of the box?

pull fails

# docker login docker.io
Authenticating with existing credentials...
Existing credentials are valid. Already logged in to docker.io
# docker pull tusc/ntopng-udm:latest
Trying to pull docker.io/tusc/ntopng-udm:latest...
  denied: requested access to the resource is denied
Trying to pull quay.io/tusc/ntopng-udm:latest...
  unauthorized: access to the requested resource is not authorized
Trying to pull registry.fedoraproject.org/tusc/ntopng-udm:latest...
  manifest unknown: manifest unknown
Error: error pulling image "tusc/ntopng-udm:latest": unable to pull tusc/ntopng-udm:latest: 3 errors occurred:
        * Error initializing source docker://tusc/ntopng-udm:latest: Error reading manifest latest in docker.io/tusc/ntopng-udm: errors:
denied: requested access to the resource is denied
unauthorized: authentication required

        * Error initializing source docker://quay.io/tusc/ntopng-udm:latest: Error reading manifest latest in quay.io/tusc/ntopng-udm: unauthorized: access to the requested resource is not authorized
        * Error initializing source docker://registry.fedoraproject.org/tusc/ntopng-udm:latest: Error reading manifest latest in registry.fedoraproject.org/tusc/ntopng-udm: manifest unknown: manifest unknown

The podman run command given fails to mount the /mnt/data/ntopng/redis dir into the container...

The README.md @0b67c64 offers this command to run the container:

podman run -d --net=host --restart always \
   --name ntopng \
   -v /mnt/data/ntopng/GeoIP.conf:/etc/GeoIP.conf \
   -v /mnt/data/ntopng/ntopng.conf:/etc/ntopng/ntopng.conf \
   -v /mnt/data/ntopng/redis.conf:/etc/redis/redis.conf \
   -v /mnt/data/ntopng/lib:/var/lib/ntopng \
   docker.io/tusc/ntopng-udm:latest

Shouldn't there also be a -v /mnt/data/ntopng/redis:/var/lib/redis argument?

No settings are saved

After following the instructions and mapping volumes the container runs fine and I can log in and use ntopng. However, it saves almost no settings at all. Even the password resets so that after the container is restarted it is always reset to admin:admin.

Mapping the redis dir as in issue #9 does not seem to help. Still nothing is saved and every time the container restarts it is starting fresh.

Volumes and files are mapping, because the settings in mapped conf files are read (e.g., the GeoIP.conf and ntopng.conf). However, nothing changed in the web UI is ever saved.

Add support for TLS certificates

As documented, ntopng runs on port 3001 but with an old certificate. Safari doesn't really like that.

It's possible to provide a certificate to ntopng (howto) so that browsers are happy.

In the case of ntopng-udm, we need to put a private key and certificate chain into /usr/share/ntopng/httpdocs/ssl/ntopng-cert.pem. If using @kchristensen's https://github.com/kchristensen/udm-le to generate certs with LetsEncrypt, this information is available locally. Thus you can:

cd /mnt/data/ntopng
cat ../udm-le/lego/certificates/*.key ../udm-le/lego/certificates/*.crt > ntopng-cert.pem

And then create the container with:

podman run -d --net=host --restart always \
   --name ntopng \
   -v /mnt/data/ntopng/GeoIP.conf:/etc/GeoIP.conf \
   -v /mnt/data/ntopng/ntopng.conf:/etc/ntopng/ntopng.conf \
   -v /mnt/data/ntopng/redis.conf:/etc/redis/redis.conf \
   -v /mnt/data/ntopng/lib:/var/lib/ntopng \
   -v /mnt/data/ntopng/ntopng-cert.pem:/usr/share/ntopng/httpdocs/ssl/ntopng-cert.pem \
   docker.io/tusc/ntopng-udm:latest

Bonus points, it is necessary to update udm-le.sh to update ntopng-cert.pem. Ideally we'd build some sort of registry of things that need to get updated so that we can de-couple udm-le.sh from knowing specifically about ntopng but for now a quick hack this diff should work.

diff --git a/udm-le.sh b/udm-le.sh
index c4ca95d..85debe8 100755
--- a/udm-le.sh
+++ b/udm-le.sh
@@ -82,6 +82,7 @@ renew)
        ${PODMAN_CMD} ${LEGO_ARGS} renew --days 60 && deploy_cert
        if [ "${NEW_CERT}" = "yes" ]; then
                add_captive && unifi-os restart
+                cat ${UDM_LE_PATH}/lego/certificates/${CERT_NAME}.key ${UDM_LE_PATH}/lego/certificates/${CERT_NAME}.crt > /mnt/data/ntopng/ntopng-cert.pem && podman restart ntopng
        fi
        ;;
 bootrenew)

Thoughts? I can throw together a PR to update the docs if you want.

Unable to run Ping/SpeedTest Active Monitoring

When setting up a ping active monitor (plugins/active_monitoring_stats.lua), the ping operation fails. The following is in the log trace:

Ping.cpp:89] ERROR: Ping IPv6 socket creation error: Success
[Ping.cpp:83] ERROR: Ping IPv4 socket creation error: Operation not permitted
[Ping.cpp:67] ERROR: Unable to enable capabilities
[LuaEngineNtop.cpp.inc:1508] ERROR: Unable to create ping socket: are you root?

There doesn't seem to be any log trace for speedtest, it just publishes the alert: Speedtest execution failed

I set up this today, following the instructions in the README.

Update to new version

Hi,

first of all, many thanks for your work, ntopng works like a charm on UDM pro!

How is it possible to update ntopng to a new version? I see in the Dockerfile you just copy the "200711" packages to /tmp and install them. Is it possible to simply replace the current "ntopng_4.1.200711-10754_arm64.deb" with a current one, e.g. "ntopng_4.2.201120-12281_amd64.deb" (from https://packages.ntop.org/apt-stable/buster/x64/) ?

Additionally: what's the "ntopng-data_4.1.200711_all.deb" package? Is this mandatory?

best
Daniel

use /mnt/data_ext for more disk space when using a hard drive installed in UDM Pro

I've installed a 256G SSD in my UDM Pro and I've deployed this project on that drive.

Here's the command I use to start the container:

podman run -d --net=host --restart always \
   --name ntopng \
   -v /mnt/data_ext/ntopng/GeoIP.conf:/etc/GeoIP.conf \
   -v /mnt/data_ext/ntopng/ntopng.conf:/etc/ntopng/ntopng.conf \
   -v /mnt/data_ext/ntopng/redis.conf:/etc/redis/redis.conf \
   -v /mnt/data_ext/ntopng/lib:/var/lib/ntopng \
   docker.io/tusc/ntopng-udm:latest

License Ntopng

Hi,
I'm really excited about the package you provided for my UDM pro. I am wondering how I can add a license of the PRO version of ntopng to the package. I tried by saving a ntopng.license file in different locations (/etc/ and /mnt/data/ntopng/), this should trigger the daemon to start in the PRO version but this doesn't work. I am using the instructions as I found here: https://www.ntop.org/guides/ntopng/adding_a_license/index.html
Is there a way to add my license key to the docker container (maybe with the -v option)?
Thanks

UniFi OS 3

Hi, is this working of UniFi OS 3? Anyone have it running?

GeoIP configured but no flags

Hi,

I have just installed ntopng-udm for the first time, configured my GeoIP account which is also successfully recognised.
I have also stopped and started the podman instance.

But I see nowhere flags showing up.
Where should I expect the flags or why does it not work, please?
thanks
Mike

help please am getting the following errors etc

podman pull tusc/ntopng-udm:latest

Trying to pull docker.io/tusc/ntopng-udm:latest...
Getting image source signatures
Copying blob 919fa42d3a05 skipped: already exists
Copying blob 29ade854e0dc skipped: already exists
Copying blob a0a5d74ac1bc skipped: already exists
Copying blob f52b7a278135 skipped: already exists
Copying blob c6bc2d0421f1 [--------------------------------------] 0.0b / 0.0b
Copying config da456e3aff done
Writing manifest to image destination
Storing signatures
da456e3aff0b2d6ba8094228dd712857d0ae0dca03b660bf87a7482d7e857906

mkdir -p /mnt/data/ntopng/redis

mkdir -p /mnt/data/ntopng/lib

touch /mnt/data/ntopng/GeoIP.conf

curl -Lo /mnt/data/ntopng/ntopng.conf https://github.com/tusc/ntopng-udm/blob/master/ntopng/ntopng.conf?raw

=true
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 130 100 130 0 0 742 0 --:--:-- --:--:-- --:--:-- 742
100 141 100 141 0 0 536 0 --:--:-- --:--:-- --:--:-- 536
100 3112 100 3112 0 0 7040 0 --:--:-- --:--:-- --:--:-- 7040

curl -Lo /mnt/data/ntopng/redis.conf https://github.com/tusc/ntopng-udm/blob/master/ntopng/redis.conf?raw=t

rue
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 129 100 129 0 0 712 0 --:--:-- --:--:-- --:--:-- 716
100 140 100 140 0 0 507 0 --:--:-- --:--:-- --:--:-- 507
100 46755 100 46755 0 0 98k 0 --:--:-- --:--:-- --:--:-- 98k

podman run -d --net=host --restart always \

--name ntopng
-v /mnt/data/ntopng/GeoIP.conf:/etc/GeoIP.conf
-v /mnt/data/ntopng/ntopng.conf:/etc/ntopng/ntopng.conf
-v /mnt/data/ntopng/redis.conf:/etc/redis/redis.conf
-v /mnt/data/ntopng/lib:/var/lib/ntopng
docker.io/tusc/ntopng-udm:latest
WARN[0000] Failed to add conmon to cgroupfs sandbox cgroup: open /sys/fs/cgroup/libpod_parent/conmon/cpuset.cpus: open /sys/fs/cgroup/libpod_parent/conmon/cpuset.cpus.effective: no such file or directory
Error: OCI runtime error: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: process_linux.go:508: setting cgroup config for procHooks process caused: bpf_prog_query(BPF_CGROUP_DEVICE) failed: function not implemented

thank you so much

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.