Upgrade will install unnecessary packages about pkgin

[jperkin]

So arguably this is an incorrect pkg_summary. There shouldn't be multiple packages with the same PKGPATH listed in the same pkg_summary, and historically trying to support this has just resulted in problems. pbulk doesn't generate these as it uses the list of built packages as input instead of just e.g. running pkg_info -X *.tgz inside ${PACKAGES}, so having leftover packages in there doesn't affect the generation.

I don't yet know if I want to help support this, as ultimately even if it was handled correctly you'll still run into issues, and it would require quite a large reworking of the code, but I'm happy to leave this open for a while if we want to discuss some use-case I'm not aware of.

To answer a potential query around this, the only way to correctly support multiple packages with the same name is to ensure they have different PKGPATHs, so if there are older versions that you specifically want instead of the latest then they need to be built from a custom copy in sysutils/htop310 or whatever.

[suominen]

The pkgin manual page says that pkgin uses pkg_summary(5) files.

The pkg_summary(5) manual page says that the pkg_summary file is generated using the pkg_info(1) -X option.

Nothing in the file format description even hints at multiple version not being supported.

So documentation does not support the argument that the pkg_summary file would be incorrect if it has multiple versions listed.

However, in practice I probably wouldn't mind if the old versions are not in the summary. I would mind having to always rebuild all packages, though.

If there was a tool to build the pkg_summary for pkgin using pkgin_summary_builder *.tgz where the glob might expand to multiple versions of the same package but were then filtered by the tool, that would be helpful for my use case.

I would rather not remove the old packages immediately in case I ran into a reason to roll back.

Being able to roll back without rebuilding pkg_summary would be more flexible, though.

[suominen]

In case it is helpful: I'm rebuilding packages for the repository with pkg_chk -r && pkg_chk -aks and then rebuilding pkg_summary with pkg_info -X *.tgz in ${PACKAGES}.

[jperkin]

There is pkgtools/pkg_summary-utils, perhaps that includes something that already handles this?

I agree that it'd be better to have a tool that generates a "correct" pkg_summary from an arbitrary PACKAGES directory, and maybe we should look at improving the documentation to be more explicit about some of the edge cases that should be avoided if creating pkg_summary manually. Most users use pbulk and do not have to think about this stuff which is probably why it hasn't been documented yet.

[suominen]

Or just document the fact that only pbulk is supported -- then others wouldn't waste time trying to use pkgin with another setup.

Currently the pkgsrc guide doesn't really make it clear that pkgin is only for the official pkgsrc repositories.

It does introduce pbulk under "Creating binary packages for everything in pkgsrc," but I specifically do not need to build binary packages for everything in pkgsrc, only for select few pieces of software.

[jperkin]

It's not that only pbulk is supported, it's that using other tools is under-documented. For example I've had a couple of occasions where people have reported bugs in pkgin because the pkg_summary they generated manually was done based on their installed packages. Obviously that makes absolutely no sense, but there's just no documentation that explains exactly how to produce a correct repository using manual tools so users are left to run random commands and hope it works.

This would be better coming from people who are actually in the not-using-pbulk camp, it's been well over 10 years since I used pkg_chk so I have no idea what the current best practises are.

[suominen]

The following makefile rule works, although is suboptimal as it doesn't detect errors in the command pipeline.

equinoxe:~> cat /p/sandbox/root/home/sandbox-exit.makefile
# Create pkg_summary.xz

pkg_summary.xz: *.tgz
        @echo Updating '`'"${.TARGET}'"
        @ls -f1 "${.CURDIR}" \
        | sed -n '/\.tgz$$/ {s,-[0-9].*,,; s,^.*/\([^/]*\)$$,\1,; p;}' \
        | sort -u \
        | while read pkg; do pkg_admin -d "${.CURDIR}" lsbest "$${pkg}"; done \
        | xargs pkg_info -X \
        | xz -c > ".${.TARGET}.new"
        @mv ".${.TARGET}.new" "${.TARGET}"