GithubHelp home page GithubHelp logo

kikobeats / xml-urls Goto Github PK

View Code? Open in Web Editor NEW
6.0 3.0 0.0 209 KB

Get all urls from a Feed/Atom/RSS/Sitemap xml markup.

Home Page: https://nicedoc.io/kikobeats/xml-urls

License: MIT License

JavaScript 100.00%
xml-url xml xml-markup

xml-urls's Introduction

xml-urls

Last version Coverage Status NPM Status

Get all URLs detected inside a Feed/Atom/RSS/Sitemap xml markup.

Install

$ npm install xml-urls --save

Usage

const xmlUrls = require('xml-urls')

;(async () => {
  const url = process.argv[2]
  if (!url) throw new TypeError('Need to provide an url as first argument.')
  const urls = await xmlUrls(url)

  urls.forEach(url => console.log(url))

  // => [
  //  'http://www.sitemappro.com/',
  //  'http://www.sitemappro.com/download.html',
  //  'http://www.sitemappro.com/register.html',
  //  'http://www.sitemappro.com/examples.html',
  //  'http://www.sitemappro.com/company.html',
  //  'http://www.sitemappro.com/contact.html',
  //  ...
  // ]
})()

See more at examples.

API

xmlUrls(urls, [options])

url

Required
Type: string

options

Type: object

Use it for providing html-get#options.

whitelist

Type: array
Default: []

A list of links to be excluded from the final output. It supports regex patterns.

See [matcher](https://github.com/sindresorhus/matcher#matcher-= for know more.

Related

  • html-urls – Get all urls from a HTML markup.
  • css-urls – Get all URLs referenced from stylesheet files.

License

xml-urls © Kiko Beats, released under the MIT License.
Authored and maintained by Kiko Beats with help from contributors.

kikobeats.com · GitHub @Kiko Beats · Twitter @Kikobeats

xml-urls's People

Contributors

dependabot-preview[bot] avatar dependabot[bot] avatar greenkeeper[bot] avatar kikobeats avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

xml-urls's Issues

An in-range update of @metascraper/helpers is breaking the build 🚨

The dependency @metascraper/helpers was updated from 5.4.2 to 5.4.3.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@metascraper/helpers is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.1.0 to 2.1.1.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Release Notes for v2.1.1

Bug Fixes

Commits

The new version differs by 3 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.0.4 to 2.0.5.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Release Notes for v2.0.5

Bug Fixes

  • package: update browserless to version 5.0.0 (f946861)
Commits

The new version differs by 3 commits.

  • d691ddb chore(release): 2.0.5
  • 817aa8f Merge pull request #26 from Kikobeats/greenkeeper/browserless-5.0.0
  • f946861 fix(package): update browserless to version 5.0.0

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

Version 1.0.9 of html-get was just published.

Branch Build failing 🚨
Dependency html-get
Current Version 1.0.8
Type dependency

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build could not complete due to an error (Details).

Commits

The new version differs by 3 commits.

  • 564bd9c Release 1.0.9
  • f6a9462 Merge pull request #6 from Kikobeats/greenkeeper/got-9.0.0
  • 33e5d52 fix(package): update got to version 9.0.0

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

Version 1.0.10 of html-get was just published.

Branch Build failing 🚨
Dependency html-get
Current Version 1.0.9
Type dependency

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build could not complete due to an error (Details).

Commits

The new version differs by 5 commits.

  • 7737b78 Release 1.0.10
  • e520dbc Merge pull request #8 from Kikobeats/greenkeeper/browserless-4.1.0
  • bb9b68c fix(package): update browserless to version 4.1.0
  • a84f537 Merge pull request #7 from Kikobeats/greenkeeper/got-9.1.0
  • 065bd3d fix(package): update got to version 9.1.0

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.0.17 to 2.0.18.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Commits

The new version differs by 2 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of @metascraper/helpers is breaking the build 🚨

The dependency @metascraper/helpers was updated from 5.4.0 to 5.4.1.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@metascraper/helpers is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.2.0 to 2.2.1.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Commits

The new version differs by 2 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.2.3 to 2.2.4.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Commits

The new version differs by 2 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.1.1 to 2.1.2.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Release Notes for v2.1.2

Bug Fixes

Commits

The new version differs by 2 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.2.5 to 2.2.6.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Commits

The new version differs by 2 commits.

  • b72e0bd chore(release): 2.2.6
  • 4a2d392 build: add more html compatible mime types

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.2.4 to 2.2.5.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Commits

The new version differs by 2 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.0.7 to 2.0.8.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Release Notes for v2.0.8

Bug Fixes

  • add missing dev dependency (26b51ca)
Commits

The new version differs by 4 commits.

  • 5a020e9 chore(release): 2.0.8
  • 26b51ca fix: add missing dev dependency
  • d192957 build: update dependencies
  • ce0baa0 build: auto detect browserless

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Improve format detection

Looks like not all the sitemap follows the same format.

Need to investigate a bit to understand how different can the mark up be.

An in-range update of lodash is breaking the build 🚨

The dependency lodash was updated from 4.17.10 to 4.17.11.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

lodash is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.0.8 to 2.0.9.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Release Notes for v2.0.9

Bug Fixes

  • package: update got to version 9.6.0 (bcb3b9e)
Commits

The new version differs by 3 commits.

  • 5c4b054 chore(release): 2.0.9
  • bd71217 Merge pull request #30 from Kikobeats/greenkeeper/got-9.6.0
  • bcb3b9e fix(package): update got to version 9.6.0

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of @metascraper/helpers is breaking the build 🚨

The dependency @metascraper/helpers was updated from 5.4.6 to 5.4.7.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@metascraper/helpers is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of aigle is breaking the build 🚨

The dependency aigle was updated from 1.13.0 to 1.13.1.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

aigle is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of @metascraper/helpers is breaking the build 🚨

The dependency @metascraper/helpers was updated from 5.4.5 to 5.4.6.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@metascraper/helpers is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.2.6 to 2.2.7.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Commits

The new version differs by 2 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of @metascraper/helpers is breaking the build 🚨

The dependency @metascraper/helpers was updated from 5.4.3 to 5.4.5.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@metascraper/helpers is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.0.10 to 2.0.11.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Release Notes for v2.0.11

Bug Fixes

  • package: update p-cancelable to version 1.1.0 (2631d34)
  • package: update time-span to version 3.0.0 (b53deef)
Commits

The new version differs by 5 commits.

  • 268fc52 chore(release): 2.0.11
  • 8fd14e7 Merge pull request #32 from Kikobeats/greenkeeper/time-span-3.0.0
  • aecad4d Merge pull request #33 from Kikobeats/greenkeeper/p-cancelable-1.1.0
  • 2631d34 fix(package): update p-cancelable to version 1.1.0
  • b53deef fix(package): update time-span to version 3.0.0

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.0.16 to 2.0.17.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Release Notes for v2.0.17

Bug Fixes

Commits

The new version differs by 14 commits.

  • 905bd27 chore(release): 2.0.17
  • 32c45bb build: add og:type as metadata (#48)
  • 70ca8f1 build: update dependencies
  • fc18507 fix: command
  • 27d52c6 build: use html detection based on headers and content
  • c5e099e build: use html detection based on file type
  • 664651c refactor: better html generation
  • 57bd613 build: add og:type as metadata
  • 961517d Update @metascraper/helpers to the latest version 🚀 (#47)
  • e3eef70 fix(package): update @metascraper/helpers to version 5.2.0
  • 72e403a Merge pull request #43 from Kikobeats/greenkeeper/time-span-3.1.0
  • 1001609 fix(package): update time-span to version 3.1.0
  • 567f707 Merge pull request #42 from Kikobeats/greenkeeper/@metascraper/helpers-5.1.0
  • 7bff17c fix(package): update @metascraper/helpers to version 5.1.0

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of got is breaking the build 🚨

Version 8.3.2 of got was just published.

Branch Build failing 🚨
Dependency got
Current Version 8.3.1
Type dependency

This version is covered by your current version range and after updating it in your project the build failed.

got is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push The Travis CI build failed Details

Release Notes v8.3.2

Fix Got throwing an error in some cases when trying to pipe one got.stream into another one. 7ac705f

Commits

The new version differs by 2 commits.

  • ad7b361 8.3.2
  • 7ac705f fix Buffer.byteLength(req._header) throwing error (#490)

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of @metascraper/helpers is breaking the build 🚨

The dependency @metascraper/helpers was updated from 5.3.0 to 5.3.4.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@metascraper/helpers is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.2.1 to 2.2.2.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Release Notes for v2.2.2

Bug Fixes

  • setup default content type (00615c4)
Commits

The new version differs by 4 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of html-get is breaking the build 🚨

The dependency html-get was updated from 2.1.2 to 2.1.3.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

html-get is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Commits

The new version differs by 2 commits.

  • 92e40d7 chore(release): 2.1.3
  • 4c468d7 build: determinate host from reachable url

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.