Upgrade Greenbone to debian-master (Report outdated / end-of-life Scan Engine) about gvm-docker

[Dexus]

Thank you @markdesilva and @willemdh I'll do my best, it's also in my own interest.

[Dexus]

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

It use POSTGRESQL 13.

What are meaning the tags:

docker pull deineagenturug/gvm:latest               # is an not initialized basic image, that first start its initialization on the first start (short: no db & no PDF report support)
docker pull deineagenturug/gvm:latest-full          # is an not initialized basic image, that first start its initialization on the first start (short: no db, PDF report is support)
docker pull deineagenturug/gvm:latest-data          # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db, but no PDF report support)
docker pull deineagenturug/gvm:latest-data-full     # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db & PDF report is support)

It can be used as direct replacement for the here provided image, with the same arguments on docker run.

⚠️ I have not tested the migration from the alpine image, but this should work out of the box now. Maybe you can give a notice it it works as I expected.

[Dexus]

Why do you switch from https://hub.docker.com/r/securecompliance/gvm to https://hub.docker.com/r/deineagenturug/gvm?

Because I feel like I'm the only one working on it. So I created my own fork so that I am able to push new versions and build all Docker images directly and correctly without having to wait for a merge. Unfortunately, that's what we've seen here in the past and again currently, no one cares. So someone has to do it who also has all the rights. And since I don't work for SCS LLC, I have no choice but to use my own fork and hardware so I can do my work and support best possible.

[markdesilva]

Thank you @Dexus for all the hard work on this and the new DeineAgenturUG images, I for one am grateful all the time and effort put in!

Cheers!

[willemdh]

Thank a lot @Dexus for everything you did. Working for free on projects such as these is not always easy. If you are the only one working on this project then we should all start following https://hub.docker.com/r/deineagenturug/gvm imho and hope for you to find the time to keep maintaining this useful project.

[devMarc5598]

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

It use POSTGRESQL 13.

What are meaning the tags:

docker pull deineagenturug/gvm:latest               # is an not initialized basic image, that first start its initialization on the first start (short: no db & no PDF report support)
docker pull deineagenturug/gvm:latest-full          # is an not initialized basic image, that first start its initialization on the first start (short: no db, PDF report is support)
docker pull deineagenturug/gvm:latest-data          # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db, but no PDF report support)
docker pull deineagenturug/gvm:latest-data-full     # is an initialized basic image, that can run directly without extra VOLUMES added (short: included db & PDF report is support)

It can be used as direct replacement for the here provided image, with the same arguments on docker run.

warning I have not tested the migration from the alpine image, but this should work out of the box now. Maybe you can give a notice it it works as I expected.

Why do you switch from https://hub.docker.com/r/securecompliance/gvm to https://hub.docker.com/r/deineagenturug/gvm?

[Kelvinloucosta]

I'm using the "docker pull deineagenturug/gvm:latest-data-full" but when I go to check the openvas scan added, the service appears unavailable, does anyone have a link to openvas correct or have you faced this problem?

[FiveOFive]

I think I have had the same or a similar issue. The gvmd logs shows "osp_scanner_feed_version: failed to get scanner_feed_version. OSPd OpenVAS is still starting"

@Dexus - Any thoughts? Thanks for your efforts keeping this project going!

[Dexus]

I think I have had the same or a similar issue. The gvmd logs shows "osp_scanner_feed_version: failed to get scanner_feed_version. OSPd OpenVAS is still starting"

I will investigate, since It build the latest-data-full without problems I think not there is a problem.

But will check this.

I'm using the "docker pull deineagenturug/gvm:latest-data-full" but when I go to check the openvas scan added, the service appears unavailable, does anyone have a link to openvas correct or have you faced this problem?

How did you run the Image? What is your run command? While it is building the image successful I think there is a unknown problem which I can't currently reproduce.

[xenago]

Thanks for your efforts, @Dexus! Unfortunately I'm unable to start up deineagenturug/gvm:latest-full and deineagenturug/gvm:latest-data-full (versions published today) on top of my existing install. I will continue to use securecompliance/gvm:debian-master-data-full in the meantime.

Hopefully this info helps...

With latest-data-full, it throws repo signature errors:
E: The repository 'http://deb.debian.org/debian bullseye-backports InRelease' is not signed.
W: GPG error: http://apt.postgresql.org/pub/repos/apt bullseye-pgdg InRelease: At least one invalid signature was encountered.

With latest-full, it throws postgres startup errors:
2022-03-21 15:08:32,173 INFO spawned: 'postgresql' with pid 653
2022-03-21 15:08:32,188 INFO exited: postgresql (exit status 1; not expected)
....
2022-03-21 15:08:52,251 INFO gave up: postgresql entered FATAL state, too many start retries too quickly

Docker-compose:

services:
    gvm:
#        image: securecompliance/gvm:debian-master-data-full
        image: deineagenturug/gvm:latest-data-full
    restart: unless-stopped
    volumes:
      - /gvm_data/database:/opt/database
      - /gvm_data/plugins:/var/lib/openvas/plugins
      - /gvm_data/gvm:/var/lib/gvm
      - /gvm_data/ssh:/etc/ssh
    ports:
      - "443:9392"
      - "5432:5432"
      - "2222:22"
    environment:
      - DB_PASSWORD="passwordhere"
      - USERNAME="admin"
      - PASSWORD="passwordhere"
      - TZ="America/Toronto"
      - SSHD=true
      - HTTPS=false
      - AUTO_SYNC=true
      - RELAYHOST=172.17.0.1
      - SMTPPORT=25

[Dexus]

Thanks for your efforts, @Dexus! Unfortunately I'm unable to start up deineagenturug/gvm:latest-full and deineagenturug/gvm:latest-data-full (versions published today) on top of my existing install. I will continue to use securecompliance/gvm:debian-master-data-full in the meantime.

Hopefully this info helps...

With latest-full, it throws postgres startup errors:
2022-03-21 15:08:32,173 INFO spawned: 'postgresql' with pid 653
2022-03-21 15:08:32,188 INFO exited: postgresql (exit status 1; not expected)
....
2022-03-21 15:08:52,251 INFO gave up: postgresql entered FATAL state, too many start retries too quickly

hi @xenago i think you have to migrate your DB something like: https://www.kostolansky.sk/posts/upgrading-to-postgresql-12/

currently we have no automatic ways for this - what makes me a bit unhappy but without full time work on it, it will take a while :(

If you use latest-data-full or latest-data together with volumes, you can simply run latest-full or latest as image tag. Because you don't will benefit from the pre-initalized images.

With latest-data-full, it throws repo signature errors:
E: The repository 'http://deb.debian.org/debian bullseye-backports InRelease' is not signed.
W: GPG error: http://apt.postgresql.org/pub/repos/apt bullseye-pgdg InRelease: At least one invalid signature was encountered.

When you get this message? On my local system I never get this message.

[xenago]

Ok thanks for the info regarding the DB upgrade, I appreciate it!

When you get this message?

I see the repo errors in the docker logs when running the docker-compose as-included in my previous comment. I'm not sure if it has anything to do with the environment variables or volumes I used or if it has something to do with the image itself.

[Dexus]

Ok thanks for the info regarding the DB upgrade, I appreciate it!

When you get this message?

I see the repo errors in the docker logs when running the docker-compose as-included in my previous comment. I'm not sure if it has anything to do with the environment variables or volumes I used or if it has something to do with the image itself.

are you behind an proxy? I'm not able here to reproduce this, maybe a newer image will solve this issue.

[Dexus]

For further support, use https://github.com/DeineAgenturUG/greenbone-gvm-openvas-for-docker

[devMarc5598]

@Dexus
Using the image deineagenturug/gvm:latest-full worked for me. Now I have the version 21.4.3.
1)
But I cant do Scans without an error...
It shows "Task interrupted unexpectedly "and "Scan process Failure" after I run a task.
2)
I cant start the container with a preinitialized volume...
So I don't have some of the old data in the current version.

[Dexus]

@devMarc5598
to point 1: your feed sync is done before you start the scan?
to point 2: do you have a DB problem? did you use before the debian image with pg11/12 or the alpine image? if the debian with pg11/12 then you need to upgrade the DB. But I'm no DB Admin so I can only ref to https://www.kostolansky.sk/posts/upgrading-to-postgresql-12/

Edit:

I will try, to provide a Docker Image to Upgrade the DB. But this may take some days.

[devMarc5598]

@Dexus
no, I created a simple task and started it. -> it stoped after a view seconds and shows this errors in the report:

Before: deineagenturug/gvm:latest-data I was using the Alpine Version securecompliance/gvm:21.4.3-v1 from 7 months ago

[Dexus]

@Dexus no, I created a simple task and started it. -> it stoped after a view seconds and shows this errors in the report:

Before: deineagenturug/gvm:latest-data I was using the Alpine Version securecompliance/gvm:21.4.3-v1 from 7 months ago

what is the status of <url>/feedstatus are there any sync in progress?

[Dexus]

plase exec docker exec -ti <container_id/name> cat /var/log/gvm/* and docker exec -ti <container_id/name> cat /var/log/supoervisor/* this will showing some problems maybe.

[devMarc5598]

My Main-Problem is:
I want to update from my current greenbone alpine version to the newest version because I get this issue
-> Report outdated / end-of-life Scan Engine / Environment (local)

But I dont want to use the debian version because of the db downgrade and some other reasons.

So the first small part of the solution is: Using a Image of https://github.com/DeineAgenturUG/
But now I need to get this work with my old alpine data.

[Dexus]

My Main-Problem is: I want to update from my current greenbone alpine version to the newest version because I get this issue -> Report outdated / end-of-life Scan Engine / Environment (local)

But I dont want to use the debian version because of the db downgrade and some other reasons.

So the first small part of the solution is: Using a Image of https://github.com/DeineAgenturUG/ But now I need to get this work with my old alpine data.

This should not be a problem since the alpine version was with PG 13.3 so it should work out of the box with the current debian based image with PG 13.6.

[Dexus]

My Main-Problem is: I want to update from my current greenbone alpine version to the newest version because I get this issue -> Report outdated / end-of-life Scan Engine / Environment (local)

But I dont want to use the debian version because of the db downgrade and some other reasons.

So the first small part of the solution is: Using a Image of https://github.com/DeineAgenturUG/ But now I need to get this work with my old alpine data.

@devMarc5598 I have found 2 bugs, that may have impact of the things you mentioned. I have have a run open to publish a fix for it - but it only releases an amd64 image for now.

[Dexus]

But I cant do Scans without an error...
It shows "Task interrupted unexpectedly "and "Scan process Failure" after I run a task.

@devMarc5598 i mean this point of failure

[Kelvinloucosta]

I have a doubt, I just uploaded two dockers, one with openvas and the other with gvm, I now want to install an openvas on the clients so that I can manage everything in a single panel that would be GSA, is this possible? Point to the public address of the GVM?

[Dexus]

@Kelvinloucosta

I have a doubt, I just uploaded two dockers, one with openvas and the other with gvm, I now want to install an openvas on the clients so that I can manage everything in a single panel that would be GSA, is this possible? Point to the public address of the GVM?

you can use the gvm image as "MASTER" and the openvas-scanner image as "SENSOR" in your DMZ. Use the Images from https://hub.docker.com/u/deineagenturug

Once you registered the "SENSOR" in "MASTER" you are able to select the Scanner for your new Scans.

[Kelvinloucosta]

thanks @Dexus , but I would like to install openvas-scanner on another network by public ip. it's possible?

Can anyone help me explain this design to me, sorry about that but I would like to know, why use redis? what is the function of postgresql? what does ospd do?

[Dexus]

thanks @Dexus , but I would like to install openvas-scanner on another network by public ip. it's possible?

Yes this would work, but I would recommend a connection via VPN. if you make the SSH port from MASTER listen public, you can connect the openvas-scanner to it.

Can anyone help me explain this design to me, sorry about that but I would like to know, why use redis? what is the function of postgresql? what does ospd do?

If you need more details for this, please look around at Greenbone

[Kelvinloucosta]

thanks again @Dexus

[Dexus]

@Kelvinloucosta you're welcome. But you should know, that the performance via WAN is not the same as via LAN. So you should be know that this may have a quit big different.

[Kelvinloucosta]

yeah you right

[gdanldo]

I have the same problem.
I have found this link https://community.greenbone.net/t/report-outdated-end-of-life-scan-engine-environment-local/5430/2

"You can edit the following script to turn off the message:

/var/lib/openvas/plugins/gcf/gb_outdated_scan_engine.nasl

Insert the versions for your scanner and libs, mine is 5.1.3 and 9.0.3:

#expected_gce_ver = “6.0.7”;
expected_gce_ver = “5.1.3”;
#expected_libs_ver1 = “10.0.2”;
expected_libs_ver1 = “9.0.3”;
#expected_libs_ver2 = “11.0.1”;
expected_libs_ver2 = “9.0.3”;

Now it won’t report the scanner.

An alternative is to set an override for the script."

This workarround work but the file change after stop and restart container.
Override doesn't work for me.
I don't have find how to update scan engine :(

[Dexus]

@gdanldo do read the full thread
https://community.greenbone.net/t/report-outdated-end-of-life-scan-engine-environment-local/5430/21

[4920441]

Hi,

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

Sorry, nice try, but there are too many problems with this container - the most recognizeable one is, that all reports are empty - only logging information is shown in the reports. There are many small problems with their container additionaly. For me, it is not useable as a drop in replacement at all :-(

[Dexus]

Hi,

You can now use the https://hub.docker.com/r/deineagenturug/gvm images, which are using the latest gvmd and components releases.

Sorry, nice try, but there are too many problems with this container - the most recognizeable one is, that all reports are empty - only logging information is shown in the reports. There are many small problems with their container additionaly. For me, it is not useable as a drop in replacement at all :-(

It's up to you to report problems. But I will not work on this repo anymore, to fix issues. You should know: I was the maintainer for the last big improvements here, but also the only one.

[4920441]

It's up to you to report problems. But I will not work on this repo anymore, to fix issues. You should know: I was the maintainer for the last big improvements here, but also the only one.

I didn't want to blame you or anything, but I wanted to point out it sounds like a drop in replacement, which it isn't - so I used up many hours to make it work, only to get more flaws whcih are preventing me from using it...

[Dexus]

It's up to you to report problems. But I will not work on this repo anymore, to fix issues. You should know: I was the maintainer for the last big improvements here, but also the only one.

I didn't want to blame you or anything, but I wanted to point out it sounds like a drop in replacement, which it isn't - so I used up many hours to make it work, only to get more flaws whcih are preventing me from using it...

It has nothing changed from the usage of the image from here to my image. And it should - yes mistakes included sometimes - work out of the box to switch.

Btw. your mentioned PDF report problem I have created at Greenbone an Issue AP-1987 for it - tacked DeineAgenturUG/greenbone-gvm-openvas-for-docker#9

[landonstewart]

Can @Secure-Compliance-Solutions simply update their docker images?

[Kelvinloucosta]

Hi @Dexus which gvm image and openvas are you using?
Is it running normally without any problems?

[Dexus]

Can @Secure-Compliance-Solutions simply update their docker images?

Sure, they can...

Hi @Dexus which gvm image and openvas are you using? Is it running normally without any problems?

https://github.com/DeineAgenturUG/greenbone-gvm-openvas-for-docker look there.

I use https://hub.docker.com/r/deineagenturug/openvas-scanner and https://hub.docker.com/r/deineagenturug/gvm

[Kelvinloucosta]

Which redis image are you guys using? Official?

[Dexus]

We have a "development" DB Upgrade tool for the migration from here to my deineagenturug/gvm.

Details you can read here: DeineAgenturUG/greenbone-gvm-openvas-for-docker#16 (comment)

Any feedback in the issue on my repo is welcome.

[weskerjiang]

Hello，i use docker pull deineagenturug/gvm:latest-data-full to pull the image and i use docker run -d -p 9390:9392 -e PASSWORD="xxx" --name=gvm deineagenturug/gvm:latest-data-full,it runs but the browser can't visit https://127.0.0.1:9390