Checking Links in published documentation about foreman-documentation

[adamlazik1]

Nevermind all that!
After six hours of yesterday's work, I learned that linkchecker does work if you specify it in the make like this: make linkchecker BUILD=satellite. HOWEVER, it is strange that it does not seem to detect all the links. make linkechecker BUILD=satellite on 3.1 branch in doc-Upgrading_and_Updating checks 29 URLs, while a grep search for different URLs returns 38, so my work may not be completely in vain 😅. Attaching output.

[ekohl]

This is configured via the ignore option in linkchecker.ini:

foreman-documentation/guides/common/linkchecker.ini

Lines 7 to 23 in eb0b825

	ignore=example.com
	cdn.redhat.com
	file:///modules
	file:///js/versions.js
	file:///js/nav.js
	tools.ietf.org
	host/unattended/provision
	projects.theforeman.org
	access.redhat.com/solutions
	forge.puppetlabs.com
	keycloak.com
	rhsso.com
	sources/
	atixservice.zendesk.com
	# This isn't published downstream yet
	access.redhat.com/documentation/en-us/red_hat_satellite/6.10/html-single/managing_configurations_using_puppet_integration/index
	access.redhat.com/documentation/en-us/red_hat_satellite/6.11

As you can see, it ignores 6.11. In feb42e7 I turned it on for 3.1.

#1750 is an effort to properly resolve it for master. You're (very) welcome to take over the effort.

[adamlazik1]

Still, not all the links are checked for the 3.1 branch as can be seen from the output above. Nevertheless I can take over #1750 if you wish, but the work will probably be postponed because I will be less active in the coming days.

[ekohl]

I'd recommend running linkchecker with --verbose and compare URLs.

[adamlazik1]

I tested this on 3.1 for the Upgrading guide. Here are the commands that I used and the output:

$ make BUILD=satellite
$ linkchecker --verbose ../build/Upgrading_and_Updating/index-satellite.html | grep 'Real URL' | tr -s ' ' | cut -d' ' -f3 | sort > linkchecker-checked-urls.txt
$ grep -oE 'href="http[^"]+' ../build/Upgrading_and_Updating/index-satellite.html | sed -e 's/href="//' | sort | uniq > grepped-urls.txt
$ diff linkchecker-checked-urls.txt grepped-urls.txt

1,3d0
< file:///home/alazik/repos/foreman-documentation/guides/build/Upgrading_and_Updating/index-satellite.html
< file:///js/nav.js
< file:///js/versions.js
11c8,9
< https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html/release_notes/index
---
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html/release_notes/index#
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html/release_notes/index#ref_known-issues_assembly_introducing-red-hat-satellite
12a11,14
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/administering_red_hat_satellite/index#Performing_an_Incremental_Backup_admin
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/administering_red_hat_satellite/index#Renaming_Server_admin
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/administering_red_hat_satellite/index#Restoring_from_a_Full_Backup_admin
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/administering_red_hat_satellite/index#Restoring_from_Incremental_Backups_admin
13a16,17
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_capsule_server/index#
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_capsule_server/index#deploying-a-custom-ssl-certificate-to-capsule-server_capsule
14a19,20
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_satellite_server_in_a_connected_network_environment/index#
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_satellite_server_in_a_connected_network_environment/index#configuring-external-services
15a22,25
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_satellite_server_in_a_disconnected_network_environment/index#
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_satellite_server_in_a_disconnected_network_environment/index#configuring-the-base-operating-system-with-offline-repositories-in-rhel-7_satellite
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_satellite_server_in_a_disconnected_network_environment/index#configuring-the-base-operating-system-with-offline-repositories-in-rhel-8_satellite
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_satellite_server_in_a_disconnected_network_environment/index#downloading-the-binary-dvd-images_satellite
16a27
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/installing_satellite_server_in_a_disconnected_network_environment/index#resolving-package-dependency-errors_satellite
17a29
> https://access.redhat.com/documentation/en-us/red_hat_satellite/6.11/html-single/managing_hosts/index#Configuring_and_Setting_Up_Remote_Jobs_managing-hosts

According to this, there are quite a few URLs that didn't get checked by the linkchecker.
Attaching the two output files.

grepped-urls.txt
linkchecker-checked-urls.txt

[ekohl]

If I strip off the #ref (which the server doesn't care about) it looks like there is no difference:

$ diff -Nut <(linkchecker --verbose ../build/Upgrading_and_Updating/index-satellite.html | grep 'Real URL' | tr -s ' ' | cut -d' ' -f3 | sed 's/#.*//' | sort) <(grep -oE 'href="http[^"]+' ../build/Upgrading_and_Updating/index-satellite.html | sed -e 's/href="// ; s/#.*//' | sort -u)
INFO linkcheck.cmdline 2022-12-01 14:02:49,057 MainThread Checking intern URLs only; use --check-extern to check extern URLs.
--- /dev/fd/63	2022-12-01 14:02:48.886036197 +0100
+++ /dev/fd/62	2022-12-01 14:02:48.887036210 +0100
@@ -1,6 +1,3 @@
-file:///home/ekohl/dev/foreman-documentation/3.1/guides/build/Upgrading_and_Updating/index-satellite.html
-file:///js/nav.js
-file:///js/versions.js
 https://access.redhat.com/articles/3664871
 https://access.redhat.com/articles/4977891
 https://access.redhat.com/articles/6393361

Can we close this?

[adamlazik1]

Sure. I realized later that link-check does not solve that problem either.