raku / rea

Raku Programming Language Ecosystem Archive

Raku Programming Language Ecosystem Archive

The Raku Programming Language Ecosystem Archive (REA for short) attempts to keep a copy of any distribution that has ever been released into the Raku ecosystem.

Synopsis

Because you may (and should) specify your module dependencies with version (and possibly with authority and api version), you are guaranteed that an application will continue to run, even though its dependencies may have gotten updated (with a higher version number, or with a different authority).

This however opens up a possible problem when trying to install an application from scratch, and a particular version of a dependency is no longer available in the "live" ecosystem. In such a case, one should be able to download and install the dependency from this repository, or from any server front-end to this repository.

Note on CPAN ecosystem

Because the CPAN ecosystem only checked on author and module name and version, and not on the :auth of the module, it is happened that authors uploaded modules with an :auth different from the expected :auth<cpan:AUTHOR> identity. This has been corrected in this archive.

Updating

This repository is expected to be updated using the Ecosystem::Archive::Update module.

Directory structure

META.json
archive
  |- ...
  |- M
     |- ...
     |- Module::Name
         |- ...
         |- Module::Name:ver<0.0.1>:auth<foo:BAR>:api<1>.tar.gz
         |- Module::Name:ver<0.0.2>:auth<foo:BAR>:api<1>.tar.gz
         |- Module::Name:ver<0.0.3>:auth<zef:bar>:api<1>.tar.gz
         |- ...
     |- ...
  |- ...
meta
  |- ...
  |- M
     |- ...
     |- Module::Name
         |- Module::Name:ver<0.0.1>:auth:<github:baz>:api<1>.json
         |- Module::Name:ver<0.0.2>:auth:<github:baz>:api<1>.json
         |- ...
     |- ...
  |- ...

rea's Issues

Datetime::Math:ver<0.6.2> typo in source-url

wget source-url gives 404 error because it contains Datetime instead of DateTime.

REA impacting zef install on windows

See ugexe/zef#452

The longer paths needed by REA cause the windows installs to fail.

Not sure we can do anything other than document the issue in zef or REA.

dist meta key does not contain the api value when api is set to 0

for example the below meta file "dist" is App::CPAN:ver<0.0.1>:auth<cpan:TYIL> while I think it should be App::CPAN:ver<0.0.1>:auth<cpan:TYIL>:api<0>

https://github.com/Raku/REA/blob/main/meta/A/App::CPAN/App::CPAN:ver%3C0.0.1%3E:auth%3Ccpan:TYIL%3E.json

Can't install File::Directory::Tree on Windows

This is another variant of #3. This time it's the * in the filename (caused by the ver<*> of that module). * is a forbidden character on Windows. The filename ends up being File%3A%3ADirectory%3A%3ATree%3Aver%3C*%3E%3Aauth%3Cgithub%3Alabster%3E.tar.gz. I guess obvious fix is to also % encode *s.

Test::Output archive is not updated

It looks like Test::Output:ver<1.001002>:auth<github:raku-community-modules> archive doesn't match it's META6.json, the META6 has resources but the archive corresponding to that META6 file doesn't have resources directory:

> tar tf Test-Output-ver<1.001002>-auth<github-raku-community-modules>.tar.gz
Test-Output-master/
Test-Output-master/.gitignore
Test-Output-master/Changes
Test-Output-master/LICENSE
Test-Output-master/META6.json
Test-Output-master/README.md
Test-Output-master/_chromatin/
Test-Output-master/_chromatin/sub-signature.png
Test-Output-master/_chromatin/sub-usage-example.png
Test-Output-master/examples/
Test-Output-master/examples/output-test.t
Test-Output-master/lib/
Test-Output-master/lib/Test/
Test-Output-master/lib/Test/Output.pm6
Test-Output-master/logotype/
Test-Output-master/logotype/logo_32x32.png
Test-Output-master/t/
Test-Output-master/t/00-use.t
Test-Output-master/t/01-capture.t
Test-Output-master/t/02-Test.pm.t
Test-Output-master/xt/
Test-Output-master/xt/meta.t

This causes error when trying to install that module from the archive (using its corresponding META6 file).

(The project on github has resources direcctory)

source-url for some dists gives 404 Not Found

For some dists the source-url is incorrect, it has .meta extension instead of .tar.gz

For example source-url in the below META files:
https://github.com/Raku/REA/blob/main/meta/T/TAP/TAP:ver%3C0.3.5%3E:auth%3Ccpan:LEONT%3E.json
https://github.com/Raku/REA/blob/main/meta/G/Gnome::N/Gnome::N:ver%3C0.19.5%3E:auth%3Ccpan:MARTIMM%3E.json

Failed to find META6.json file for Gnome::Gtk3

Upgrading Gnome::Gtk3 failed with this error message:

===> Extraction: Failed to find a META6.json file for https://raw.githubusercontent.com/raku/REA/main/archive/G/Gnome::Gtk3/Gnome::Gtk3:ver%3C0.47.2%3E:auth%3Ccpan:MARTIMM%3E.tar.gz -- failure is likely
[https://raw.githubusercontent.com/raku/REA/main/archive/G/Gnome::Gtk3/Gnome::Gtk3:ver%3C0.47.2%3E:auth%3Ccpan:MARTIMM%3E.tar.gz] Extracting with plugin Zef::Service::Shell::tar+{<anon|1>} aborted.
Type check failed for return value; expected IO::Path but got Any (Any)

It was upgraded without errors without REA.

Versions are missing from the ecosystem

Either the indexer lags behind or missing module versions. The case I stumbled upon is about Digest. p6c currently has v0.5.0 of the module, but REA only knows about v0.4.0. From module's git log I assume that 0.5.0 was published on Feb 6, i.e. a month ago. Yet, REA repository was updated 12 days ago (https://github.com/Raku/REA/tree/main/meta/D/Digest) and the latest version is not there.

Remove version of a distribution from REA

I need to remove Pakku:ver<pupa> from the index, because when initially created this version I was trying out different approaches, I did not think much about proper version thought I can easily remove or change later. and because that naively chosen version pupa shows higher than the version scheme I'm using now, it's causing some issues
(e.g. when I do want to upgrade).

I can't think of any harm will be done when removing that version, since the api changed long time ago and not working now. And by removal I mean removal from the META.json index so it can't be installed. the archive and meta should still be there.

Another option I can think of is to create may be "deprecated": true key in META file, to indicate that a version is deprecated and should not be installed. but in the mean time can it be removed from the index until installation tooling support this new META file field.

p6doc should be removed

The installs corresponding to Raku/doc (in P/p6doc) should probably be removed.

Raku/doc repository hasn't been installable for some time, and if there is a version in here that is installable, the docs in it are very out of date (as is the probably broken build code); even when it was installed, it was using a version of the tools to read the docs that doesn't work that way anymore and is now in its own repo.