fortran-lang / fpm-registry

Centralized registry of fpm packages

fpm-registry

Centralized registry of fpm packages for the fortran-lang site.

How to submit a new package

Your new registry submittal should first meet the general package criteria required of any package listed at the packages listing.

Please submit a pull request against this repository, adding the new package into the file registry.toml in alphabetical order. It is recommended that you explicitly list each version using the tag keyword, otherwise the tip of the default branch will be used. An example:

[datetime]
"1.7.0" = {git="https://github.com/wavebitscientific/datetime-fortran", tag="v1.7.0"}
"latest" = {git="https://github.com/wavebitscientific/datetime-fortran"}

Test your edits

Before submitting the pull request, you can validate the registry.toml file locally using the provided Python script. First set up a new virtual environment and install dependencies:

python3 -m venv venv
source venv/bin/activate
pip install -U pip
pip install -r requirements.txt

To validate registry.toml, run:

python load_registry.py

Registry

When a pull request is merged it will then appear at the fpm registry.

How to request help

If you have a great Fortran package that you are interested in registering as an fpm package but terms like fpm, PR, toml, and python are putting you off, then please open an issue at the fpm Registry site listing your Repository and let us know how we can help. Get as far as you can and identify where you got stuck. We are always happy to help.

The Fortran Discourse forum is another valuable avenue for help as well as for general discussion and announcements related to the Fortran Programming Language.

fpm-registry's Issues

Duplicated entries in fpm registry

Introduced with #30, both M_CLI2 and M_CLI now have two entries in the registry each. See https://fortran-lang.org/packages/search/?query=m_cli

Workflow is committing artifacts to default branch

Our current workflow for building the package index is committing it to the default branch. Instead a separate branch or an upload of the package index seems preferable to me. What do you think?

Maintain ODEPACK under fortran-lang

Similar to FFTPACK (#40) and QUADPACK (#43), we should also maintain ODEPACK under fortran-lang. This has been first mentioned in scipy/scipy#13229, where they have to maintain ODEPACK specific patches inside SciPy.

Instead, we should maintain a new de-facto "upstream" home for ODEPACK, maintaining the code to ensure it compiles with recent Fortran compilers on all architectures. Then SciPy can just copy the sources into their tree. (Down the road, as fpm becomes more widely used, we can discuss other ways of SciPy depending on these packages.)

Broken link to registry in About section

The link to https://fpm.fortran-lang.org/en/registry/ in the About section is broken.

How to handle package name conflicts

Right now if somebody submits a PR with a duplicate name of a new package, such as:

--- a/registry.toml
+++ b/registry.toml
@@ -2,6 +2,9 @@
 "1.7.0" = {git="https://github.com/wavebitscientific/datetime-fortran", tag="v1.7.0"}
 "latest" = {git="https://github.com/wavebitscientific/datetime-fortran"}
 
+[datetime]
+"1.0.0" = {git="https://github.com/other/my_package", tag="v1.0.0"}
+
 [M_calculator]
 "latest" = {git="https://github.com/urbanjost/M_calculator"}

Then our CI will give an error:

$ python load_registry.py 
Traceback (most recent call last):
  File "load_registry.py", line 3, in <module>
    d = toml.load("registry.toml")
  File "/home/ondrej/miniconda3/envs/reg/lib/python3.8/site-packages/toml/decoder.py", line 134, in load
    return loads(ffile.read(), _dict, decoder)
  File "/home/ondrej/miniconda3/envs/reg/lib/python3.8/site-packages/toml/decoder.py", line 477, in loads
    raise TomlDecodeError("What? " + group +
toml.decoder.TomlDecodeError: What? datetime already exists?{'datetime': {'1.7.0': {'git': 'https://github.com/wavebitscientific/datetime-fortran', 'tag': 'v1.7.0'}, 'latest': {'git': 'https://github.com/wavebitscientific/datetime-fortran'}}} (line 5 column 1 char 171)

Which is great, so there will be no duplicates.

However, we should discuss how to handle package name conflicts. Two main options:

1. All package names will be made unique by prefixing them with their GitHub/GitLab organization name, so M_calculator becomes urbanjost/M_calculator. I believe that is what Go does.

2. Package names are not prefixed, and we need to provide some guidelines. We can expect a similar situation as in Rust, when people will be submitting lots of new packages with the most obvious names, such as "spline", "harmonics", "fft", "lapack" and they might not necessarily end up the most maintained. And changing a name of a package means breaking people's builds.

The advantages of 1:

• We can always switch to 2. later if needed
• The chance of collision of the GitHub / GitLab / ... prefix is relatively low (if it happens, we'll handle it on a case by case basis)
• We can list the main package, say fortran-lang/stdlib as well as a fork, say certik/stdlib, if the fork has some versions or branches that the main repository does not

Disadvantages of 1:

• I remember the package by its name, such as "datetime", and I don't want to remember what the prefix is. Although in practice I assume one would do fpm search datetime to obtain the actual version to use (which I won't remember anyway). And fpm search datetime can print out the exact line to put into fpm.toml, such as wavebitscientific/datetime = "1.7.0". So this might not be any more difficult than just datetime = "1.7.0".

Advantages of 2:

• Shorter names
• Possible to change where the package lives, for example if wavebitscientific/datetime is moved to fortran-lang/datetime, then nothing changes in the fpm.toml. With the option 1., one would really need to stick to wavebitscientific/datetime as a name, or provide some kind of a redirect.

Disadvantages of 2:

• First serve first come basis, in a few years all the lucrative names will be already taken
• Possible conflicts

Adding Cantera to fpm

Disclaimer:it's been about ten years since I was routinely working in Fortran, but I'm getting back into working with it via Cantera (https://github.com/Cantera/cantera).

I'm curious about getting cantera into fpm - it's primarily written in C++, but has Fortran bindings/interface (https://github.com/Cantera/cantera/tree/main/samples/f90)

So - is this even feasible/reasonable or is cantera outside the scope of fpm since it's not primarily Fortran source code?

Thanks

Implement full metadata extraction to JSON

I've made a start on this as part of listing fpm packages on fortran-lang.org, so creating an issue to assign.

As discussed in fortran-lang/fpm#34, this is just a script that reads the registry.toml and generates an index.json file containing relevant data from the fpm.toml in each package. I'll then use this JSON file to list the fpm packages on the website.

add `forlab` package

Keurfonluu originally developed the two packages of Forlab and FFTPack -- a interface package, which I think are good. Now I submit the Forlab package separately to the fpm-registry. I hope I will try to maintain this package later.
Thank keurfonluu!

---

Maybe it's not perfect right now, but it will get better.

Add QUADPACK to `fpm` registry

I actually would be willing to do all of the work here but am not sure where to start.

AFAICT the original source has been converted to Fortran 90 by John Burkardt (single and double precision). The big concerns then are (A) LICENSE, this is the biggest (B) build tooling e.g. Make or CMake, not sure what fpm prefers (C) documentation.

Improve error reporting in CI

It's currently difficult to diagnose CI failures when submitting new packages because of how github actions outputs stdout and stderr. In short, the relevant part of the error message is not output near to which package is being processed.

I'm open to possible solutions. Perhaps:

• redirect stderr to stdout in CI workflow
• print error messasge to stdout in build_index.py

Index build is failing on default branch

The index build failed on the default branch, probably due to a failed API request on a Gitlab repository (jsonff).

See https://github.com/fortran-lang/fpm-registry/runs/2006247844 for c650134

Move minpack under fortran-lang

A candidate repository to move could be this one:

https://github.com/certik/minpack

Required keywords for registry

I wonder why we require license, maintainer, author and copyright to be present in the package manifest of a submitted package. Shouldn't license and maintainer be sufficient for a package to be valid?

Then the question is how we specify the maintainer properly, for all packages registered it is an email address so far, while fpm new with both bootstrap fpm and Fortran fpm will currently create a name and an email address.

Also, for all packages so far, we have single author projects, which will change once we start porting larger projects to work with fpm, those usually have more than one maintainer and/or author. fpm itself is probably the simplest example for this purpose, there is just a placeholder for the maintainer available right now.
I would suggest to allow or better require the maintainer entry to be an array of strings.
Similar arguments hold for the author field, most packages will have more than one author.

Additionally, I want to add that putting up an email address might not be the easiest way to reach out to the maintainer. Depending on the project emailing the maintainer might work, but usually creating an issue is the preferred way to report bugs or request features, while questions are usually handled by various other platforms, like gitter, zulip or a forum. In short, this information is usually present on the homepage of the project or the projects README.

The copyright fields is especially tricky, since it is not always that easy to pin down. I have projects where I cannot put a copyright statement in one line and even if I could put several lines using a TOML multiline string, those wouldn't apply for the complete project.
Another thing I was wondering, in case a package is dedicated to the public domain (Unlicense, CC0, ...), I wouldn't include a copyright statement at all, due to it being in public domain.

Finally, there seems to be no check if the license fields is actually a valid license or matches the license contained in the package itself. An easy way would be to follow Cargo or conda-forge and require at least a valid SPDX identifier for the license.

Store size of package in the index

Should the fpm.toml file contain information about the size of the package?

The subroutines new_package and check, in the module fpm_manifest_package, contain 18 keywords with information about a package, name, author, version, etc. However, no keyword provides information about the size of the package. 

An idea,

1. Build the package
2. Update the file fpm.toml with information about the size of the package

The command execute_command_line can get this information from shell.

Documentation is needed

The documentation of the fpm-registry is currently not in a good shape. Good documentation is crucial to make a project like a package registry appealing for developers to actually submit their packages.

Currently, upon visiting the registry there are no guidelines on the scope and expectations of the registry, the packaging guidelines linked are for the package index at fortran-lang.org and not fpm specific. Especially the required keys for submitting to the registry are not documented.

Also, the package validation is tricky, the validation should happen with the load_registry.py script if I read the README correctly, but the actual validation of the fpm-enabled projects will only happen with build_index.py --check-new.

Cannot set branch in registry

Currently we cannot set the branch we want to fetch in the registry file.

The workaround is to exploit that branches and tags are handled sufficiently similar that we can just pretend that our branch is a tag in things will work. This workaround will be okay with fpm since we are currently not making a distinction between branches and tags in the fpm.

On the other hand, this behaviour can also be exploited to add commits from forks using the tag key in the registry, like I have done in the past here: 4e4da02

Implement manual registry

The details are discussed at:

fortran-lang/fpm#34

We should have a TOML file with a package name and a version / url combo for each version, something like:

[package1]
"0.1.5" = "https://somewhere/package1-0.1.5.tar.gz"
"0.1.6" = "https://somewhere/package1-0.1.6.tar.gz"

[stdlib]
"1.0.0" = {git="htttps://github.com/fortran-lang/stdlib", tag="v1.0.0"}
"latest" = {git="htttps://github.com/fortran-lang/stdlib", branch="master"}  # This would checkout the latest master

We can discuss if we should allow the "latest" --- perhaps we should simply only allow specific versions, that seems to be the case with other package managers.

Renaming default branch

See fortran-lang/fpm#421 for details.

Please collect references in @fortran-lang projects to files inside this repository such that we can fix those links after renaming.

Registry format / layout

I just checked out this registry for fpm and noticed some rough edges. It is somewhat related to #1, but maybe worth a separate issue.

Specifying a project with multiple releases quickly gets redundant, imagine a package which releases on a regular basis, keeping the registry data up-to-date will quickly become a copy-and-paste approach

[project-name]
"1" = {git="https://github.com/project-name/project-name", tag="v1"}
"2" = {git="https://github.com/project-name/project-name", tag="v2"}
"2.1" = {git="https://github.com/user-name/project-name", tag="v2-server.1"}
"3" = {git="https://github.com/project-name/project-name", tag="v3"}
"3.1" = {git="https://github.com/project-name/project-name", tag="v3.1"}
"4" = {git="https://github.com/project-name/project-name", tag="v4"}
"latest" = {git="https://github.com/project-name/project-name", branch="main"}

A slight improvement in my opinion would be to allow specifying the upstream URL in the project section and allow to overwrite it per version basis.

[project-name]
git = "https://github.com/project-name/project-name"
"1" = {tag="v1"}
"2" = {tag="v2"}
"2.1" = {git="https://github.com/user-name/project-name", tag="v2-server.1"}
"3" = {tag="v3"}
"3.1" = {tag="v3.1"}
"4" = {tag="v4"}
latest.branch = "main"

Failure in master?

I got an email with this link with an error at our CI:

https://github.com/fortran-lang/fpm-registry/runs/1183248590

But I am not quite sure what the cause of it is.

@LKedward, any idea?

Add `fortran-lang/stdlib` package (branch: stdlib-fpm)

Originally, I wanted to submit fftpack (has been passed, #53 ) and stdlib-fpm together, but found that it seems that fpm-registry does not support branch analysis like this:

[stdlib]
"latest" = { git = "https://github.com/fortran-lang/stdlib", branch = "stdlib-fpm" }

Is this an issue?

Index builder does not recognize valid manifest entries

The index build currently warns for unknown entries in manifests like, categories, keywords, homepage and install. Those entries are valid as defined in the manifest reference in the fpm repository but not correctly recognized by the index builder.

link to formatted registry list

A few suggestions for minor changes to the "fpm packages" page and this registry README.md ...

It would be nice if this repository had a link to the fpm packagespage.

Note that page seems to be inconsistently sorted. It would be nice if the page were sortable or selectable by categories as more packages are registered but for now it starts off looking sorted in a case-insensitive manner, but then jumps back. So if the user is supposed to enter their PR in a (case-insensitive?) order in this TOML file it seems like the packages page should reflect that.