madchops1 / sitemap-generator Goto Github PK

I have a website created in Angular 4. When crawling, it recognises url's but when it finishes download is not starting even tho' in the console log I can see this:

download 9scripts.5c94af11.js:1

Screen flashes for a split second but nothing happens.
Browser is Google Chrome Version 59.0.3071.115 (Official Build) (64-bit)

After successfully crawling the page, when I hit Download, the button simply disappears. No download. Happens in Safari and Firefox on OSX

Trying to generate a sitemap for my site, BotMap.io goes to a bunch of urls before switching to facebook, urls all starting with http://www.facebook.com/share.php?u=...

This is surprising, shouldn't you filter out urls unrelated to the domain of the site being crawled ?
I tried to disallow www.facebook.com, but didn't work.

If the crawler is Stuck on a URL, it would be really slick to allow moving forward on to the next URL in the list. This could be related to current #4 issue

• website URL 1
• website URL 2
• website URL 3
  [ X ] website URL 4 (STUCK)
• website URL 5 (next on list)

To avoid the above error in nodejs i am proposing if it is possible to use this
<url> <loc><![CDATA[http://www.example.com/?id=10&param2=112]]></loc> </url>
or (Note & has been replaced with &amp)
<url> <loc>http://www.example.com/?id=10&amp;param2=112</loc> </url>

Your super crawler is down, any chances bringing it up again?

403 Forbidden
Code: AllAccessDisabled
Message: All access to this object has been disabled
RequestId: A43FE6ED196E4528
HostId: T4OTrDWVfVWIl/SP8coAp/9o6ObPXFSS8YSiKNYBvbh0Zwhf9wElVbm5H9bHQmix8dDjNsovuzQ=

When I try to crawl our site, in Chrome, I only receive the error that the scripts are not installed. But in Safari and Firefox it works (on OSX :) )