test resources is not found by brew about brew

[MikeMcQuaid]

regardless (and not a blocker for fixing this 🐛) we should probably have something like this in future instead:

resource "homebrew-test-audio" => :test do
  url "https://raw.githubusercontent.com/openai/whisper/7858aa9c08d98f75575035ecd6481f462d66ca27/tests/jfk.flac"
  sha256 "63a4b1e4c1dc655ac70961ffbf518acd249df237e5a0152faae9a4a836949715"
end

[fxcoudert]

Maybe we should revert temporarily. This is causing several big PR testing to fail on homebrew-core.

[Bo98]

Test resources working is purely accidental tbh so am not surprised it breaks.

That's why the signature is "wrong" - it was never supposed to be used this way, someone figured out it worked and it slowly became widespread from there.

Long-term ambition is full support via :test tag or similar but this hasn't been picked up yet.

[MikeMcQuaid]

@fxcoudert Can you (or someone) try to git bisect to figure out when this happened?

I'm not sure if the issue is that the test resource cannot be found or that brew fetch is trying to fetch something found in a test do block.

[fxcoudert]

I'm traveling so I can't debug properly for now, or bisect. But this happens only in the test step, all those steps before succeeded:

==> brew fetch --retry openai-whisper --build-bottle --force
==> brew install --verbose --build-bottle openai-whisper
==> brew style --formula openai-whisper
==> brew audit --formula openai-whisper --online --git --skip-style
==> brew bottle --verbose --json openai-whisper --only-json-tab
==> brew bottle --merge --write --no-commit --no-all-checks ./openai-whisper--20231117_3.arm64_sonoma.bottle.json
==> brew uninstall --force openai-whisper
==> brew uninstall --force asciidoc asciidoctor autoconf automake berkeley-db@5 bison boost cargo-c cmake curl docbook docbook-xsl doxygen flatbuffers gdbm gnu-getopt gobject-introspection googletest libgit2 libssh2 libtool lit llvm@17 lua m4 meson ncurses ninja openldap perl pkg-config protobuf@21 python-setuptools [email protected] rtmpdump ruby rust scons source-highlight sphinx-doc swig util-macros vim xcb-proto xmlto xtrans
==> brew install --only-dependencies ./openai-whisper--20231117_3.arm64_sonoma.bottle.tar.gz
==> brew install ./openai-whisper--20231117_3.arm64_sonoma.bottle.tar.gz
==> brew linkage --test openai-whisper
==> brew linkage --cached --test --strict openai-whisper
==> brew linkage --cached openai-whisper
==> brew install --only-dependencies --include-test openai-whisper

and only this one fails:

==> brew test --verbose openai-whisper

https://github.com/Homebrew/homebrew-core/actions/runs/8899611195/job/24497962540?pr=170392

I've been able to test locally that it happens with any formula. It's also not a download failure, it doesn't even initiate the download.

In case it helps, the full backtrace (where I added the same in my local gmp formula for easier testing) is:

An exception occurred within a child process:
  ResourceMissingError: gmp does not define resource "homebrew-test-audio"
/opt/homebrew/Library/Homebrew/software_spec.rb:136:in `block in resource'
/opt/homebrew/Library/Homebrew/software_spec.rb:136:in `fetch'
/opt/homebrew/Library/Homebrew/software_spec.rb:136:in `resource'
/opt/homebrew/Library/Homebrew/formula.rb:557:in `resource'
/opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula/g/gmp.rb:64:in `block in <class:Gmp>'
/opt/homebrew/Library/Homebrew/formula.rb:2793:in `block (3 levels) in run_test'
/opt/homebrew/Library/Homebrew/extend/kernel.rb:520:in `with_env'
/opt/homebrew/Library/Homebrew/formula.rb:2792:in `block (2 levels) in run_test'
/opt/homebrew/Library/Homebrew/formula.rb:1198:in `with_logging'
/opt/homebrew/Library/Homebrew/formula.rb:2791:in `block in run_test'
/opt/homebrew/Library/Homebrew/mktemp.rb:75:in `block in run'
/opt/homebrew/Library/Homebrew/mktemp.rb:75:in `chdir'
/opt/homebrew/Library/Homebrew/mktemp.rb:75:in `run'
/opt/homebrew/Library/Homebrew/formula.rb:3102:in `mktemp'
/opt/homebrew/Library/Homebrew/formula.rb:2785:in `run_test'
/opt/homebrew/Library/Homebrew/test.rb:46:in `block in <main>'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/timeout.rb:107:in `block in timeout'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/timeout.rb:36:in `block in catch'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/timeout.rb:36:in `catch'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/timeout.rb:36:in `catch'
/opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/timeout.rb:123:in `timeout'
/opt/homebrew/Library/Homebrew/test.rb:50:in `<main>'

[fxcoudert]

Done bisecting:

caf87c0336dea14bafe35441ea90dfd32be27d85 is the first bad commit
commit caf87c0336dea14bafe35441ea90dfd32be27d85
Author: Markus Reiter <[email protected]>
Date:   Fri Apr 26 20:55:51 2024 +0200

    Warn about undocumented non-private APIs.

@reitermarkus

[reitermarkus]

The problem is Formula.resource works on class level, and I thought that Formula#resource at the instance level is only used for accessing the class level resource definitions, hence only the name argument is needed.

In openai-whisper we are using Formula#resource in test to define a new resource with a block, so the signature is wrong.

[MikeMcQuaid]

resource "homebrew-test-audio" => :test do

this is what is needed