Comments (23)
The point I’m making is that it is poor url handling is reality (in both url parsing classes, servers and more) that we must and should recognize, regardless of standards.
caddyserver/caddy#1582 - worth reading in the context of what is expected and accepted behavior by a web server and what is not.
My point is - ASVS should not go and tolerate going against specifications.
from asvs.
Verify that when dynamically building URLs, untrusted data is encoded according to its context (e.g., URL encoding or base64url encoding for query or path parameters). Ensure that only safe URL protocols are permitted (e.g., disallow javascript: or data:).
I removed safely as it may be a little redundant but can you PR this for 5.3?
Should be tagged as [ADDED, SPLIT FROM 5.3.1]
from asvs.
Related: #1590 (comment)
Also related:
V5.5.5 [ADDED, MERGED FROM 13.1.1] Verify that different parsers used in the application for the same data type (e.g. JSON parsers, XML parsers, URL parsers), perform parsing in a consistent way and use the same character encoding mechanism to avoid issues such as JSON Interoperability vulnerabilities or different URI or file parsing behavior being exploited in Remote File Inclusion (RFI) or Server-side Request Forgery (SSRF) attacks.
from asvs.
from asvs.
from asvs.
I believe both. However, I believe it’s not our job to set requirements for web servers in my opinion. I state with respect. I do believe we need to be aware of the differences between web servers.
vs
One of my concerns around URL safety is that bad URL handling to lead to a variety of different major security issues. This includes XSS, SSRF and open redirects.
I think those two statements are in conflict - if a web server handles URL incorrectly, it is exactly our goal to provide requirements for that and point to the specs, of how it must be done. It is not expected, that program on one web server is safe and on the other one is not just because they handle URL differently.
The provided topics are quite vague to comment on. For example, how "URL output encoding", "complete URL output encoding" and "safe complete URL output encoding" are different from the requirement we already have? Please provide example attacks or vulnerabilities that should be addressed.
from asvs.
from asvs.
If it behaves like you have it implemented, it's done incorrectly. Recommended reading:
https://datatracker.ietf.org/doc/html/rfc3986#section-2.4
Under normal circumstances, the only time when octets within a URI are percent-encoded is during the process of producing the URI from its component parts.
Once produced, a URI is always in its percent-encoded form.
When a URI is dereferenced, the components and subcomponents significant to the scheme-specific dereferencing process (if any) must be parsed and separated before the percent-encoded octets within those components can be safely decoded, as otherwise the data may be mistaken for component delimiters.
Implementations must not percent-encode or decode the same string more than once, as decoding an already decoded string might lead to misinterpreting a percent data octet as the beginning of a percent-encoding, or vice versa in the case of percent-encoding an already percent-encoded string.
In total, you listed 3 problems: please list examples for all of them, one by one, keeping the entire coverage of the risen problem.
from asvs.
Like I demonstrated on my own server, if you url encode untrusted data and place it on a path, path traversal is possible on some (popular) HTTP servers. So URL encoding untrusted data on a path is not universally safe. This is why the JWT standard opted for base64url encoding and not url encoding.
Correct URL encoding in this situation is for functionality to work correctly.
If you have a path traversal vulnerability on the server side, you can not fix it with URL encoding for the client, it is not a trusted service layer in this context. You need to fix it on the server side - with validation and sanitization like every other user input.
So, your point of view (to say that path traversal on the server side is the result of encoding URL incorrectly for the client side) is also conceptually just wrong.
from asvs.
from asvs.
The point I’m making is that it is poor url handling is reality (in both url parsing classes, servers and more) that we must and should recognize, regardless of standards. Many of our other requirements would not be needed if dev’s and components just followed standards, but they do not.
Many of our other requirements would not be needed if dev’s and components just followed standards, but they do not.
This is the first statement I agree with - I recommend you start from your own server :) We are not going to tolerate vulnerabilities just because they are widespread.
As with 3 attempts here and 10s of comments before, I was not able to get answers or understand, what is the problem with current requirements, or what the proposals are to improve them, then I'll give it over to @tghosth .
My summary:
Output encoding
I stand for the statement, that the output encoding is covered with 5.3.1
- safe complete URL output encoding (XSS)
- safe inserting of untrusted data into query string parameters and paths (url encoding and possibly base64encoding)
Safe inserting means that we need to transfer the data correctly to the next component. If the problem is in the next component, it can not be solved with output encoding.
More detailed comment for that is written here: #1589 (comment)
To use base64uri or whatever other "URL-safe" encoding, it is a question of application logic to transfer its data between its components. If URL encoding is done correctly, we can not require the use of base64uri instead.
Validation
JavaScript URL’s alone cause over a third of real world XSS attacks and is on the rise.
- url validation (SSRF and open redirects)
We have requirement 5.1.4 for validating the data from the user, we can mention URL to give it more spotlight (issue #1552)
# | Description | L1 | L2 | L3 | CWE |
---|---|---|---|---|---|
5.1.4 | [GRAMMAR] Verify that structured data is strongly typed and validated against a defined schema including allowed characters, length and pattern (e.g. credit card numbers, e-mail addresses, telephone numbers, or validating that two related fields are reasonable, such as checking that suburb and zipcode match). | ✓ | ✓ | ✓ | 20 |
If validation is done and was not enough, the value must be sanitized.
# | Description | L1 | L2 | L3 | CWE |
---|---|---|---|---|---|
5.2.2 | Verify that unstructured data is sanitized to enforce safety measures such as allowed characters and length. | ✓ | ✓ | ✓ | 138 |
For open redirect and SSRF we have:
# | Description | L1 | L2 | L3 | CWE |
---|---|---|---|---|---|
5.1.5 | [MODIFIED, SPLIT TO 50.7.1] Verify that the application will only automatically redirect the user to a different URL directly from an application URL where the destination appears on an allow list. | ✓ | ✓ | ✓ | 601 |
5.2.6 | Verify that the application protects against SSRF attacks, by validating or sanitizing untrusted data or HTTP file metadata, such as filenames and URL input fields, and uses allow lists of protocols, domains, paths and ports. | ✓ | ✓ | ✓ | 918 |
from asvs.
from asvs.
Thanks both for the comments and direction.
However, I believe it’s not our job to set requirements for web servers in my opinion. I state with respect. I do believe we need to be aware of the differences between web servers.
I think this is a subtle but important point. We are not writing requirements for how to write a web server (e.g. nginx or Apache or IIS or whatever), we are providing requirements for applications which run on these web servers.
As such, we need to advise on generating standards compliant URLs and how to manually process URLs in a safe way but I don't think we can necessarily account for every edge case or every instance where the underlying web server does not work in a standards compliant way.
In terms of the areas to cover:
1) safe complete URL output encoding (XSS)
@jmanico I would appreciate a more specific example of how this specific point should be covered. The way I understand it is that if a URL needs to be encoded for a web page based on the context where it is being written (HTML element, HTML attribute, Javascript, etc) in the same way that any content would need to be. I am not sure what is needed here in addition to what is in 5.3.1.
2) safe inserting of untrusted data into query string parameters and paths (url encoding and possibly base64encoding)
I agree with @jmanico that this has some specific points which would benefit from their own requirement. Do you think you could provide a suggested draft text for a requirement here @jmanico?
3) url validation (SSRF and open redirects)
In this case I think I agree with @elarlang that we already have a number of requirements that handle these aspects, @jmanico I am not sure we need anything additional here unless you wanted to suggest specific changes to the requirements Elar noted above.
from asvs.
from asvs.
Output encoding complete URL’s in a src or href context is not a complete defense since JavaScript: URL’s still execute XSS even when encoded.
Specifically, you mean that the JavaScript URL will execute when it is clicked on, i.e. it does not execute when the page is rendered but rather when the link is clicked (which is the theoretical purpose of the JavaScript URL anyway). Did I understand correctly?
First, build dynamic URL’s safely....
Is there any OWASP cheatsheet for this as it feels like the rules here are going to be a little complicated?
Also, validate host names and validate schemes/protocols using allow-list validation.
I am going to look to see how we currently talk about this related to SSRF and other stuff and whether we need to add more content to the existing requirements. (I will get back to you :) )
from asvs.
Notes from call:
- Validating a complete URL to safely place in a webpage
Handled in #1589
- Output encoding a complete URL when placing into a HTML document
Handled in #1589
- Validation a complete URL for server side use (redirect/SSRF)
@tghosth to check what requirements we already have for SSRF/redirects, i.e. where we have control over the interpreter.
- Assembling a dynamic URL safely
4a) Placing untrusted data in a querystring
Need a new requirement(s) @jmanico
4b) Placing untrusted data in the path of a URL (REST Style)
Need a new requirement(s) @jmanico
4c) Any other URL contexts?
Need a new requirement(s) @jmanico
from asvs.
This is not just a caddy issue. Other servers have the same struggle. I understand your point, but I want to acknowledge the reality that URL encoding is not a universal defense. base64encoding is.
I plan to build a requirement that suggests URL encoding and also suggests base64url encoding when URL encoding is not sufficient to address both of our concerns. Again, this is why the JWT standard uses base64url encoding. I'm not the only one who acknowledges this reality.
from asvs.
URL handling
So, if I point to RFC - you say (#1589 (comment)) your understanding of the URL handling is still right. When I point to the Caddy issue, which describes the same behavior as a bug/security issue, you keep saying, that your own server, which handles the URL the same-incorrect way, is still right?
Other servers have the same struggle.
Please provide names and facts. We should only talk about facts and examples, not "but The Others". Also the reason and explanations, why they do so. Or... is it someones configuration issue.
URL building
I plan to build a requirement that suggests URL encoding and also suggests base64url encoding
I'm waiting for your proposals, but note, that a suggestion or recommendation is not a security requirement.
This is what I have pointed out before - if you want to suggest, that it is easier for the interpreter to handle the URL, you can send the data in base64uri encoding. It is not a security hole, when someone does not use base64uri encoding instead of just using correctly URL encoding.
from asvs.
from asvs.
Hi @jmanico , I believe you have suggested the following requirements:
- If building dynamic urls, be sure to URL encode when placing
untrusted data in a querystring parameter and validate the complete URLand
- If building dynamic urls, be sure to URL encode or base64url encode
when placing untrusted data in a path parameter and validate the
complete URL
Do you think these could be merged?
Do these cover all cases related to URLs? Do we need to consider javascript: urls or something as well?
from asvs.
I can live with these being merged. Perhaps:
Verify that when building dynamic URLs, untrusted data is URL encoded or base64url encoded, and ensure the complete URL is validated to confirm it is legal and the protocol is safe.
from asvs.
Verify that when dynamically building URLs, untrusted data is safely encoded based on context (e.g. URL encoded for query parameters, URL or ideally base64url encoded for path parameters) and only safe URL protocols are allowed (e.g. not javascript).
@jmanico what do you think about this one?
from asvs.
I like it am am happy with it. Can I suggest a little cleanup just to make it a little more general?
Verify that when dynamically building URLs, untrusted data is safely encoded according to its context (e.g., URL encoding or base64url encoding for query or path parameters). Ensure that only safe URL protocols are permitted (e.g., disallow javascript: or data:).
from asvs.
Related Issues (20)
- discussion OAuth/OIDC: accepted flows and grants HOT 7
- 4.3.1 and 4.3.3 HOT 7
- Password Storage Algorithms 2.4.1 revisited HOT 3
- Make 2.1.14 easier and more simplified HOT 7
- Implement Requirement for Anomalous Behavior Detection HOT 4
- cloud config scanning HOT 1
- Link checker is temperamental and apparently deprecated HOT 2
- 5.5.4 should be removed as a duplicate of 5.2. HOT 1
- 5.3.4 should also mention stored procedures. HOT 2
- Do we need CSS to be mentioned in both 5.3.1 and 5.2.8. HOT 2
- Add HTML comments to 5.3.1
- Dynamically generated HTML HOT 11
- Proposal/discussion: OIDC nonce claim requirement HOT 1
- Proposal/discussion: OIDC requirement to ensure issuer URL == issuer claim HOT 2
- Proposal/discussion: OIDC requirement about ID token only being used to prove that the user has been authenticated HOT 7
- 5.6.2 Adding Server-side validation proposal HOT 11
- 7.1 Proposal for Masked Data Logging Need HOT 4
- 8.2.2 advice on sessionStorage breaks multi-tab web application use unlike cookies HOT 13
- Set Account Lockout ASVS Levels 1-3 Aligned to NIST, PCI-DSS, CIS et al
- CWE indicated in requirement 50.2.1 (previously v4.0.3-14.4.3) seems incorrect HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from asvs.