Core plugins tests are failing in Node 18 about iframely

[iparamonau]

Good point about IPv6, we'll need to do some experiments. For now, let's leave the issue open - we'll swap the test URL there, and then resolve it ourselves.

Thanks for reporting!

[iparamonau]

Is it just for that test BBC URL or for others too? Also, what do you get if you cURL that URL with our user-agent. Also, what UA string do you? The default one?

[j0k3r]

Ok maybe I wasn't clear enough.
All tests are green on Node 16.
If I switch to Node 18, 403 appears, no matter the website.
Everything is at default, the repo is clean and up to date.

$ nvm use 16
Now using node v16.18.1 (npm v9.6.0)

$ yarn test-core-plugins
yarn run v1.22.19
$ mocha --exit test/core-plugins.js
Using cache engine: node-cache
No local domains file detected...
-- [23-03-17 08:40:18]:8999 Loading domains list from https://iframely.com/qa/domains.json
Invalid mixin "ld-newsarticle-logo" in plugin "sverigesradio.se"
Invalid mixin "ld-author" in plugin "sverigesradio.se"
Invalid mixin "ld-date" in plugin "sverigesradio.se"
Iframely plugins loaded:
   - custom domains: 140
   - generic & meta: 95


  Core plugins
    meta
-- [23-03-17 08:40:18]:8999 Domains list activated. 1915 domains, including disabled ones
-- [23-03-17 08:40:18]:8999    -- plugin redirect (by "htmlparser") https://www.bbc.com/news/science-environment-23767323
-- [23-03-17 08:40:18]:8999    -- plugin redirect (by "htmlparser") https://www.bbc.com/news/av/science-environment-23767323
      ✔ not error
      ✔ has meta
    oembed
-- [23-03-17 08:40:19]:8999    -- plugin redirect (by "htmlparser") https://www.ted.com/talks/kent_larson_brilliant_designs_to_fit_more_people_in_every_city
      ✔ not error
      ✔ has valid oEmbed object
    image size
      ✔ has correct size and type (40ms)


  5 passing (381ms)

✨  Done in 1.44s.

$ nvm use 18
Now using node v18.15.0 (npm v9.5.0)

$ yarn test-core-plugins
yarn run v1.22.19
$ mocha --exit test/core-plugins.js
Using cache engine: node-cache
No local domains file detected...
-- [23-03-17 08:40:27]:9197 Loading domains list from https://iframely.com/qa/domains.json
Invalid mixin "ld-newsarticle-logo" in plugin "sverigesradio.se"
Invalid mixin "ld-author" in plugin "sverigesradio.se"
Invalid mixin "ld-date" in plugin "sverigesradio.se"
Iframely plugins loaded:
   - custom domains: 140
   - generic & meta: 95


  Core plugins
    meta
-- [23-03-17 08:40:28]:9197    -- plugin redirect (by "htmlparser") https://www.bbc.com/news/science-environment-23767323
-- [23-03-17 08:40:28]:9197 Domains list activated. 1915 domains, including disabled ones
-- [23-03-17 08:40:28]:9197    -- plugin response: {"plugin":"htmlparser","response":403,"uri":"https://www.bbc.com/news/science-environment-23767323"}
      1) not error
      2) has meta
    oembed
-- [23-03-17 08:40:28]:9197    -- plugin redirect (by "htmlparser") https://www.ted.com/talks/kent_larson_brilliant_designs_to_fit_more_people_in_every_city
      ✔ not error
      ✔ has valid oEmbed object
    image size
      ✔ has correct size and type (48ms)


  3 passing (464ms)
  2 failing

  1) Core plugins
       meta
         not error:
     AssertionError: expected 403 to equal null
      at Context.<anonymous> (file:///Users/jeremy/Sites/github/iframely/test/core-plugins.js:64:20)
      at process.processImmediate (node:internal/timers:476:21)

  2) Core plugins
       meta
         has meta:
     AssertionError: Target cannot be null or undefined.
      at Context.<anonymous> (file:///Users/jeremy/Sites/github/iframely/test/core-plugins.js:71:20)
      at process.processImmediate (node:internal/timers:476:21)



error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[iparamonau]

Thank you, I can confirm the 403 on that test URL from BBC using Node 18.

However,

403 appears, no matter the website.

Every other URL seems to be working fine and as expected. Can you please confirm that you get 403 across the board, for all the URLs you try?

[j0k3r]

Well, BBC doesn't work for sure but can't find an other one that is failing and can't recall which one was failing from that morning.

So I guess I can close the issue as it seems to be only related to BBC?
Can this be related to the fact that since Node 17 IPv6 is preferred over IPv4?