Comments (7)
Could you paste the command line results of one of the errors?
In general, domain-scan
scanners should fail gracefully, in that they print out an error or note in the saved data that it's an error, but it should never crash the scanner itself.
from domain-scan.
Note that some of these (possibly all) are because they use meta-redirects (here's a partial list of domains that do so).
from domain-scan.
#114 is taking a crack at this.
from domain-scan.
Alas, it's still happening, here's what is in the terminal after I let afadavantage.gov run for 9 hours and finally had to skip it with control + C
.
[afadvantage.gov][a11y]
the_domain_is_cached: False
the_cache_is_not_forced: True
Not cached.
[afadvantage.gov][a11y]
^CTraceback (most recent call last):
File "./scan", line 178, in <module>
run(options)
File "./scan", line 84, in run
scan_domains(scans, domains)
File "./scan", line 142, in scan_domains
executor.map(process_scan, tasks)
File "/usr/lib/python3.4/concurrent/futures/_base.py", line 574, in __exit__
self.shutdown(wait=True)
File "/usr/lib/python3.4/concurrent/futures/thread.py", line 131, in shutdown
t.join()
File "/usr/lib/python3.4/threading.py", line 1060, in join
self._wait_for_tstate_lock()
File "/usr/lib/python3.4/threading.py", line 1076, in _wait_for_tstate_lock
elif lock.acquire(block, timeout):
KeyboardInterrupt
Writing to cache: afadvantage.gov
Writing data for afadvantage.gov
Here's the same but for banknet.gov:
[bankhelp.gov][a11y]
the_domain_is_cached: False
the_cache_is_not_forced: True
Not cached.
[bankhelp.gov][a11y]
Writing to cache: bankhelp.gov
Writing data for bankhelp.gov
[banknet.gov][a11y]
the_domain_is_cached: False
the_cache_is_not_forced: True
Not cached.
[banknet.gov][a11y]
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib/python3.4/concurrent/futures/thread.py", line 38, in _python_exit
t.join()
File "/usr/lib/python3.4/threading.py", line 1060, in join
self._wait_for_tstate_lock()
File "/usr/lib/python3.4/threading.py", line 1076, in _wait_for_tstate_lock
elif lock.acquire(block, timeout):
KeyboardInterrupt
from domain-scan.
A number of these are no longer factors b/c the DAP exclusion list removes them at an earlier stage. These still remain though:
afadvantage.gov
banknet.gov
biomassboard.gov
dea.gov
export.gov
flightschoolcandidates.gov
grantsolutions.gov
gsaadvantage.gov
gsaauctions.gov
hrsa.gov
hydrogen.gov
idmanagement.gov
nationalbank.gov
pay.gov
realestatesales.gov
safetyact.gov
sciencebase.gov
selectusa.gov
stopfakes.gov
usdebitcard.gov
from domain-scan.
@gbinal @micahsaul - I'm seeing some hangs during a11y
scans too, though it's not necessarily the same as this list. Do you still see issues with these domains?
from domain-scan.
Augh! No, I hadn't been, I'll take a look this week.
from domain-scan.
Related Issues (20)
- Make Lambda packaging aware of third party dependencies, incorporate them in repackaging
- Move from an options dict to a "config" (or similar) dict
- Scanner dependency problem HOT 3
- public-suffic-list.txt deployed to the wrong local path HOT 2
- Packaged version of sslyze in lambda/envs/domain-scan.zip is out of date HOT 4
- Path.resolve() throws exception when results file doesn't exist HOT 4
- lambda: public-suffix-list.txt not found HOT 1
- Update to SSLyze 2.0.0, and detect TLS 1.3 0-RTT
- Confirm best way to search for USWDS elements
- Add cookie scanner
- Adopt some of the clean-up from dhs-ncats/gatherer HOT 1
- Use Node 8 runtime for Lambda
- PSL cache location not obeyed
- Unnecessary prepending of "www." on domains of form "www#." HOT 3
- Summarize USWDS scan in plain language
- Scanning hosts on a network that is not connected to the Internet HOT 1
- While trying to Scan a list of domains from a CSV, the 'HSTS Preloaded' column in pshtt.csv is giving incorrect results
- Upgrade sslyze to 3.x
- System UI HOT 1
- Can't get the code to run on the M1 chip
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from domain-scan.