facebook / sapp Goto Github PK
View Code? Open in Web Editor NEWPost Processor for Facebook Static Analysis Tools.
License: MIT License
sapp's Issues
KeyError: 'caller_port'
When using:
sapp --tool=mariana-trench analyze .
I got error:
/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/sqlalchemy/orm/relationships.py:1994: SAWarning: Setting backref / back_populates on relationship Run.issue_instances to refer to viewonly relationship IssueInstance.run should include sync_backref=False set on the Run.issue_instances relationship. (this warning may be suppressed after 10 occurrences)
util.warn_limited(
/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/sqlalchemy/orm/relationships.py:1994: SAWarning: Setting backref / back_populates on relationship IssueInstance.run to refer to viewonly relationship Run.issue_instances should include sync_backref=False set on the IssueInstance.run relationship. (this warning may be suppressed after 10 occurrences)
util.warn_limited(
2022-06-01 20:47:21,491 [INFO] Parsing analysis output...
Traceback (most recent call last):
File "/Users/shijian03/.venvs/mariana-trench/bin/sapp", line 8, in
sys.exit(cli())
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/click/decorators.py", line 84, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/sapp/cli_lib.py", line 208, in analyze
pipeline.run(analysis_output, summary_blob)
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/sapp/pipeline/init.py", line 363, in run
next_input, summary = step.run(next_input, summary)
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 236, in run
self.analysis_output_to_dict_entries(
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 188, in analysis_output_to_dict_entries
for typ, key, e in self._analysis_output_to_parsed_tuples(inputfile):
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 135, in _analysis_output_to_parsed_tuples
for e in entries:
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser.py", line 351, in parse
yield from self.parse_handle(handle)
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser.py", line 367, in parse_handle
for postcondition in self._parse_postconditions(model):
File "/Users/shijian03/.venvs/mariana-trench/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser.py", line 528, in _parse_postconditions
port=Port.from_json(generation["caller_port"], "source"),
KeyError: 'caller_port'
Looking forward to hearing from you.
Thanks!
Cannot read properties of undefined (reading 'slice')
Error while reading issues with the following description
User input flows into WebView load: Values from user-controlled source may eventually flow into a Webview load potentially causing XSS
sapp --database-name=sapp.db server --source-directory=src/main/java
# Python 3.8.10
# Ubuntu 20.01
# Installed Using: pip install mariana-trench
function adjustRange(range: Range, lines: $ReadOnlyArray<string>): Range {
// TODO(T78595608): workaround for inaccurate Pysa locations with leading and
// trailing whitespaces.
// Assuming all ranges are single line.
const source = lines[range.from.line].slice(range.from.ch, range.to.ch); // <<<<< ERROR
const leadingWhitespace = source.search(/\S/);
const trailingWhitespace = source.length - source.trimEnd().length;
return {
from: {
line: range.from.line,
ch: range.from.ch + leadingWhitespace,
},
to: {
line: range.to.line,
ch: range.to.ch - trailingWhitespace,
},
};
}react-dom.production.min.js:209 TypeError: Cannot read properties of undefined (reading 'slice')
at Source.js:40
at Source.js:80
at Array.map (<anonymous>)
at te (Source.js:63)
at ne (Source.js:183)
at $i (react-dom.production.min.js:153)
at Ia (react-dom.production.min.js:175)
at mc (react-dom.production.min.js:263)
at ls (react-dom.production.min.js:246)
at cs (react-dom.production.min.js:246)
KeyError: 'port'
When I install fb-sapp from Pypi and analyze the output it returns the following error.
sapp -v "DEBUG" --tool=mariana-trench analyze .
/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/relationships.py:1994: SAWarning: Setting backref / back_populates on relationship Run.issue_instances to refer to viewonly relationship IssueInstance.run should include sync_backref=False set on the Run.issue_instances relationship. (this warning may be suppressed after 10 occurrences)
util.warn_limited(
/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/relationships.py:1994: SAWarning: Setting backref / back_populates on relationship IssueInstance.run to refer to viewonly relationship Run.issue_instances should include sync_backref=False set on the IssueInstance.run relationship. (this warning may be suppressed after 10 occurrences)
util.warn_limited(
2022-08-23 11:00:16,264 [DEBUG] Context: Context(database=<sapp.db.DB object at 0x1121fa1c0>, parser_class=<class 'sapp.pipeline.mariana_trench_parser_v2.Parser'>, repository='/Volumes/Samsung_T5/2. Lab Project/SAST-Mobile/Android-InsecureBankv2', ipython_extensions=[], tool='mariana-trench')
2022-08-23 11:00:16,269 [INFO] Parsing analysis output...
Traceback (most recent call last):
File "/usr/local/bin/sapp", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 84, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sapp/cli_lib.py", line 206, in analyze
pipeline.run(analysis_output, summary_blob)
File "/usr/local/lib/python3.9/site-packages/sapp/pipeline/__init__.py", line 365, in run
next_input, summary = step.run(next_input, summary)
File "/usr/local/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 235, in run
self.analysis_output_to_dict_entries(
File "/usr/local/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 187, in analysis_output_to_dict_entries
for typ, key, e in self._analysis_output_to_parsed_tuples(inputfile):
File "/usr/local/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 136, in _analysis_output_to_parsed_tuples
for e in entries:
File "/usr/local/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 468, in parse
yield from self.parse_handle(handle)
File "/usr/local/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 482, in parse_handle
for precondition in self._parse_preconditions(model):
File "/usr/local/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser_v2.py", line 764, in _parse_condition
port=Port.from_json(leaf_model["port"], leaf_kind),
KeyError: 'port'
The temporary solution is to use the previous version pip install fb-sapp==0.5.2
Instead of using the latest version pip install fb-sapp==0.5.3 released on Aug 19, 2022
KeyError: 'port' when following mariana-trench guide
sapp/sapp/pipeline/mariana_trench_parser.py
Line 850 in e6f541a
I am trying to make mariana-trench work and have followed their guide on there repo (https://github.com/facebook/mariana-trench). Everything seems to run successfully but when I want to use sapp to analyse, I get "KeyError: 'port'". Here is the full error:
/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/sqlalchemy/orm/relationships.py:1994: SAWarning: Setting backref / back_populates on relationship Run.issue_instances to refer to viewonly relationship IssueInstance.run should include sync_backref=False set on the Run.issue_instances relationship. (this warning may be suppressed after 10 occurrences) util.warn_limited( /home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/sqlalchemy/orm/relationships.py:1994: SAWarning: Setting backref / back_populates on relationship IssueInstance.run to refer to viewonly relationship Run.issue_instances should include sync_backref=False set on the IssueInstance.run relationship. (this warning may be suppressed after 10 occurrences) util.warn_limited( 2023-02-06 00:03:09,854 [INFO] Parsing analysis output... Traceback (most recent call last): File "/home/hukad/.venvs/mariana-trench/bin/sapp", line 33, in <module> sys.exit(load_entry_point('fb-sapp==0.5.4', 'console_scripts', 'sapp')()) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/click/decorators.py", line 84, in new_func return ctx.invoke(f, obj, *args, **kwargs) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/sapp/cli_lib.py", line 206, in analyze pipeline.run(analysis_output, summary_blob) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/sapp/pipeline/__init__.py", line 365, in run next_input, summary = step.run(next_input, summary) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/sapp/pipeline/base_parser.py", line 231, in run self.analysis_output_to_dict_entries( File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/sapp/pipeline/base_parser.py", line 183, in analysis_output_to_dict_entries for typ, key, e in self._analysis_output_to_parsed_tuples(inputfile): File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/sapp/pipeline/base_parser.py", line 132, in _analysis_output_to_parsed_tuples for e in entries: File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/sapp/pipeline/mariana_trench_parser.py", line 541, in parse yield from self.parse_handle(handle) File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/sapp/pipeline/mariana_trench_parser.py", line 555, in parse_handle for precondition in self._parse_preconditions(model): File "/home/hukad/.venvs/mariana-trench/lib/python3.10/site-packages/sapp/pipeline/mariana_trench_parser.py", line 850, in _parse_condition port=Port.from_json(leaf_model["port"], leaf_kind), KeyError: 'port'
This happens both when installing fb-sapp from pip or directly from github.
AttributeError: module 'zstd' has no attribute 'ZstdCompressor'
zstd library changed and there is no ZstdCompressor in it.
json diagnostics
Missing Trace Frame
Hi all
I got Missing Trace Frame in traces UI page very often. I want to know where can I add log to find out what happen in source code?
or the codes is related to the trace frame str to show ?
thanks a lot!
Unable to update a filter field of a saved filter
Thanks for your continued interest in helping us out with Pysa and SAPP, @m0mosenpai and @gracewgao!
Scenario
Currently, we don't support updating existing SAPP filters. If a user wants to change a field for an existing filter, but keep the name of the filter the same, they would need to delete the filter before re-saving the filter with the same name
For example, suppose we have an existing filter Test Filter 1 that filters only for Code==5008 and we want to overwrite Test Filter 1 to only filter for Code==5003. We expect to perform this action by loading the saved filter Test Filter 1, changing the value of code from 5008 to 5003, and finally saving the change to Test Filter 1. However to perform this action now, we will need to delete Test Filter 1, set the code to filter for 5003, and save a new filter with the same name Test Filter 1
Steps to Reproduce
- Set up your dev virtual environment
git clone [email protected]:facebook/sapp.git && cd sapp
python3 -m venv ~/.venvs/sapp
source ~/.venvs/sapp/bin/activate
pip3 install -r requirements.txt
cd sapp/ui/frontend && npm install
npm run-script build
- Run Pysa on a repo and store the results to pass into SAPP to ingest
pyre analyze --no-verify --save-results-to .
- Use SAPP to ingest the results from the Pysa run
python3 -m sapp.cli analyze taint-output.json
- Run SAPP server and go to http://localhost:5000
python3 -m sapp.cli server
- Create a filter with for any field (eg. filter for code
5008) and save the filter asTest Filter 1
- Fill some different filter fields from the ones you have set in step 5 and save the filter with the same name as in step 5 (e.g.
Test Filter 1) - Notice that the filter fails to overwrite the previously saved filter and you get the following traceback in SAPP server logs
2021-06-01 19:03:22,221 [ERROR] Traceback (most recent call last):
...
graphql.error.located_error.GraphQLLocatedError: (sqlite3.IntegrityError) UNIQUE constraint failed: filters.name
[SQL: INSERT INTO filters (name, description, json) VALUES (?, ?, ?)]
[parameters: ('test', None, '{"features":[{"mode":"all of","features":[]}],"codes":[5001]}')]
(Background on this error at: http://sqlalche.me/e/14/gkpj)
Related files
The following list of files that might be good starting points:
Resources
AttributeError: 'LocalStack' object has no attribute '__ident_func__'
Bug
Bug description
Following the instructions on the Mariana Trench home page to run but encountered issue.
Reproduction steps
Follow the post processing steps on https://github.com/facebook/mariana-trench
Expected behavior
Give a clear and concise description of what you expected to happen.
Logs
File "/usr/local/bin/sapp", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 84, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sapp/cli_lib.py", line 236, in server
start_server(ctx.database, debug, static_resources, source_directory, editor_schema)
File "/usr/local/lib/python3.9/site-packages/sapp/ui/server.py", line 75, in start_server
scopefunc=_app_ctx_stack.__ident_func__,
AttributeError: 'LocalStack' object has no attribute '__ident_func__'
Additional context
Add any other context about the problem here.
Web UI lags
HI all,
I encountered unbearable UI stutters.
There are over 100 times runs and tens of thousands of isuues in my DB. Sapp UI lags because of query DB
More accurate ，following code takes a long time:
Ask for help
Please tell me how to improve and reduce time costs
Filter has bugs
I added some simple rules to Filter. Please note that the filter with code = 3 in the following picture
First I select it to do filter.It works fine and gives me the issues with code = 3
After several random clicking different filter rules via web UI. I reselected the code=3 filler .It is frustrated to me that the filter result is empty . The reason is the filter query has a unnecessary feature(statuses:do_not_care)。It is weird for same filter has different results
Can the sapp support to bind 0.0.0.0 other than localhost ?
After run sapp --database-name=sapp.db server, we can only visit the issues by http://localhost:5000. But does the sapp can support to bind 0.0.0.0, so that all the people in a team can visit the issues by http://x.x.x.x:500/ ? or is there any roadmap to support this feature ？
sapp.pipeline.ParseError: Unexpected trace fragment
While trying to analyze Pysa output with command sapp --database-name sapp.db analyze taint-output.json, I get the following output.
2023-01-03 14:07:22,447 [INFO] Parsing analysis output...
Traceback (most recent call last):
File "/usr/local/bin/sapp", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/click/decorators.py", line 84, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/sapp/cli_lib.py", line 207, in analyze
pipeline.run(analysis_output, summary_blob)
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/__init__.py", line 441, in run
next_input, summary = step.run(next_input, summary)
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/base_parser.py", line 254, in run
summary.get("old_linemap_file"),
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/base_parser.py", line 203, in analysis_output_to_dict_entries
for typ, key, e in self._analysis_output_to_parsed_tuples(inputfile):
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/base_parser.py", line 141, in _analysis_output_to_parsed_tuples
for e in entries:
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/pysa_taint_parser.py", line 99, in parse
for entry in self.parse_handle(handle):
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/pysa_taint_parser.py", line 106, in parse_handle
yield from self._parse_by_type(entry)
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/pysa_taint_parser.py", line 172, in _parse_by_type
yield from self._parse_issue(entry["data"])
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/base_parser.py", line 87, in wrapper
yield from func(self, json, *args)
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/pysa_taint_parser.py", line 239, in _parse_issue
) = self._parse_issue_traces(json["traces"], "backward", "sink")
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/pysa_taint_parser.py", line 298, in _parse_issue_traces
return self._parse_issue_trace_fragments(leaf_port, trace["roots"])
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/pysa_taint_parser.py", line 311, in _parse_issue_trace_fragments
for fragment in self._parse_trace_fragment(leaf_port, trace):
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/pysa_taint_parser.py", line 351, in _parse_trace_fragment
yield from self._parse_trace_fragment_v3(leaf_port, trace)
File "/usr/local/lib/python3.7/dist-packages/sapp/pipeline/pysa_taint_parser.py", line 468, in _parse_trace_fragment_v3
raise ParseError("Unexpected trace fragment.", received=trace)
sapp.pipeline.ParseError: Unexpected trace fragment.
Received: `{'kinds': [{'leaves': [{'name': 'print'}], 'kind': 'Print'}], 'origin': {'filename': 'imprecision1.py', 'line': 5, 'start': 6, 'end': 22}}
Program being analyzed:
n: int = input("Input a number: ")
i: int = 'I am a number'
res = n+i
print(res.bit_length())
And the taint-output.json I want to analyze:
{"file_version":3,"config":{"repo":"/Users/tiraboschi/pysa_project"}}
{"kind":"issue","data":{"callable":"imprecision1.$toplevel","callable_line":1,"code":5002,"line":5,"start":6,"end":22,"filename":"imprecision1.py","message":"Data from [UserControlled] source(s) may reach [Print] sink(s)","traces":[{"name":"forward","roots":[{"kinds":[{"leaves":[{"name":"input"}],"kind":"UserControlled"}],"local_features":[{"always-via":"obscure:model"},{"always-type":"integer"},{"always-type":"scalar"},{"always-via":"tito"}],"tito_positions":[{"line":4,"start":6,"end":7},{"line":5,"start":6,"end":9}],"origin":{"filename":"imprecision1.py","line":2,"start":9,"end":34}}]},{"name":"backward","roots":[{"kinds":[{"leaves":[{"name":"print"}],"kind":"Print"}],"origin":{"filename":"imprecision1.py","line":5,"start":6,"end":22}}]}],"features":[{"always-via":"obscure:model"},{"always-type":"integer"},{"always-type":"scalar"},{"always-via":"tito"}],"sink_handle":{"kind":"Call","callee":"print","index":0,"parameter":"formal(*rest0)"},"master_handle":"imprecision1.$toplevel:5002:0:Call|print|0|formal(*rest0):f92c08fecfc0d6d0890769682e75edf0"}}
{"kind":"issue","data":{"callable":"imprecision2.convert","callable_line":7,"code":5002,"line":10,"start":10,"end":14,"filename":"imprecision2.py","message":"Data from [UserControlled] source(s) may reach [Print] sink(s)","traces":[{"name":"forward","roots":[{"kinds":[{"leaves":[{"name":"input"}],"kind":"UserControlled"}],"local_features":[{"always-via":"tito"},{"always-via":"obscure:unknown-callee"}],"tito_positions":[{"line":9,"start":20,"end":21}],"origin":{"filename":"imprecision2.py","line":8,"start":8,"end":33}}]},{"name":"backward","roots":[{"kinds":[{"leaves":[{"name":"print"}],"kind":"Print"}],"origin":{"filename":"imprecision2.py","line":10,"start":10,"end":14}}]}],"features":[{"always-via":"tito"},{"always-via":"obscure:unknown-callee"}],"sink_handle":{"kind":"Call","callee":"print","index":0,"parameter":"formal(*rest0)"},"master_handle":"imprecision2.convert:5002:0:Call|print|0|formal(*rest0):b730aacd59b2cd2e9376c74ea615a9bb"}}
{"kind":"model","data":{"callable":"imprecision2.get_zero","tito":[{"port":"formal(n)[__mul__]","taint":[{"kinds":[{"return_paths":{"":0},"kind":"LocalReturn"}],"local_features":[{"always-via":"obscure:unknown-callee"}],"tito":null}]}]}}
{"kind":"issue","data":{"callable":"imprecision3.convert","callable_line":7,"code":5002,"line":13,"start":14,"end":18,"filename":"imprecision3.py","message":"Data from [UserControlled] source(s) may reach [Print] sink(s)","traces":[{"name":"forward","roots":[{"kinds":[{"leaves":[{"name":"input"}],"kind":"UserControlled"}],"local_features":[{"always-via":"tito"},{"always-via":"obscure:unknown-callee"}],"tito_positions":[{"line":9,"start":20,"end":21}],"origin":{"filename":"imprecision3.py","line":8,"start":8,"end":33}}]},{"name":"backward","roots":[{"kinds":[{"leaves":[{"name":"print"}],"kind":"Print"}],"origin":{"filename":"imprecision3.py","line":13,"start":14,"end":18}}]}],"features":[{"always-via":"tito"},{"always-via":"obscure:unknown-callee"}],"sink_handle":{"kind":"Call","callee":"print","index":0,"parameter":"formal(*rest0)"},"master_handle":"imprecision3.convert:5002:0:Call|print|0|formal(*rest0):05da74f5000e93661778704068697bda"}}
{"kind":"model","data":{"callable":"imprecision3.get_zero","tito":[{"port":"formal(n)[__mul__]","taint":[{"kinds":[{"return_paths":{"":0},"kind":"LocalReturn"}],"local_features":[{"always-via":"obscure:unknown-callee"}],"tito":null}]}]}}
{"kind":"issue","data":{"callable":"imprecision4.convert","callable_line":7,"code":5002,"line":13,"start":14,"end":18,"filename":"imprecision4.py","message":"Data from [UserControlled] source(s) may reach [Print] sink(s)","traces":[{"name":"forward","roots":[{"kinds":[{"leaves":[{"name":"input"}],"kind":"UserControlled"}],"local_features":[{"always-via":"tito"},{"always-via":"obscure:unknown-callee"}],"tito_positions":[{"line":9,"start":20,"end":21}],"origin":{"filename":"imprecision4.py","line":8,"start":8,"end":33}}]},{"name":"backward","roots":[{"kinds":[{"leaves":[{"name":"print"}],"kind":"Print"}],"origin":{"filename":"imprecision4.py","line":13,"start":14,"end":18}}]}],"features":[{"always-via":"tito"},{"always-via":"obscure:unknown-callee"}],"sink_handle":{"kind":"Call","callee":"print","index":0,"parameter":"formal(*rest0)"},"master_handle":"imprecision4.convert:5002:0:Call|print|0|formal(*rest0):5d60f5a756115ff632602c48995522a3"}}
{"kind":"model","data":{"callable":"imprecision4.get_zero","tito":[{"port":"formal(n)[__mul__]","taint":[{"kinds":[{"return_paths":{"":0},"kind":"LocalReturn"}],"local_features":[{"always-via":"obscure:unknown-callee"}],"tito":null}]}]}}
{"kind":"model","data":{"callable":"input","sources":[{"port":"result","taint":[{"kinds":[{"kind":"UserControlled"}],"declaration":null}]}],"modes":["Obscure"]}}
{"kind":"model","data":{"callable":"os.system","sinks":[{"port":"formal(command)","taint":[{"kinds":[{"kind":"RemoteCodeExecution"}],"declaration":null}]}],"modes":["Obscure"]}}
{"kind":"model","data":{"callable":"print","sinks":[{"port":"formal(*rest0)","taint":[{"kinds":[{"kind":"Print"}],"declaration":null}]}],"modes":["Obscure"]}}
{"kind":"issue","data":{"callable":"source.convert","callable_line":7,"code":5001,"line":9,"start":22,"end":32,"filename":"source.py","message":"Data from [UserControlled] source(s) may reach [RemoteCodeExecution] sink(s)","traces":[{"name":"forward","roots":[{"kinds":[{"leaves":[{"name":"input"}],"kind":"UserControlled"}],"origin":{"filename":"source.py","line":8,"start":17,"end":38}}]},{"name":"backward","roots":[{"kinds":[{"features":[{"always-via":"format-string"},{"always-via":"tito"}],"leaves":[{"name":"os.system"}],"length":1,"kind":"RemoteCodeExecution"}],"call":{"position":{"filename":"source.py","line":9,"start":22,"end":32},"resolves_to":["source.get_image"],"port":"formal(url)"}}]}],"features":[{"always-via":"format-string"},{"always-via":"tito"}],"sink_handle":{"kind":"Call","callee":"source.get_image","index":0,"parameter":"formal(url)"},"master_handle":"source.convert:5001:0:Call|source.get_image|0|formal(url):71e3ce573ab7da30d5a55d88882319a9"}}
{"kind":"model","data":{"callable":"source.get_image","sinks":[{"port":"formal(url)","taint":[{"kinds":[{"leaves":[{"name":"os.system"}],"kind":"RemoteCodeExecution"}],"local_features":[{"always-via":"format-string"},{"always-via":"tito"}],"tito_positions":[{"line":4,"start":40,"end":43}],"origin":{"filename":"source.py","line":5,"start":21,"end":28}}]}],"tito":[{"port":"formal(url)","taint":[{"kinds":[{"return_paths":{"":0},"length":1,"kind":"LocalReturn"}],"local_features":[{"always-via":"obscure:model"},{"always-type":"integer"},{"always-via":"format-string"},{"always-type":"scalar"},{"always-via":"tito"}],"tito_positions":[{"line":4,"start":40,"end":43},{"line":5,"start":21,"end":28}],"tito":null}]}]}}
{"kind":"model","data":{"callable":"pstats.FunctionProfile.__init__","tito":[{"port":"formal(tottime)","taint":[{"kinds":[{"return_paths":{"[tottime]":999999},"kind":"LocalReturn"}],"tito":null}]},{"port":"formal(percall_tottime)","taint":[{"kinds":[{"return_paths":{"[percall_tottime]":999999},"kind":"LocalReturn"}],"tito":null}]},{"port":"formal(percall_cumtime)","taint":[{"kinds":[{"return_paths":{"[percall_cumtime]":999999},"kind":"LocalReturn"}],"tito":null}]},{"port":"formal(ncalls)","taint":[{"kinds":[{"return_paths":{"[ncalls]":999999},"kind":"LocalReturn"}],"tito":null}]},{"port":"formal(line_number)","taint":[{"kinds":[{"return_paths":{"[line_number]":999999},"kind":"LocalReturn"}],"tito":null}]},{"port":"formal(file_name)","taint":[{"kinds":[{"return_paths":{"[file_name]":999999},"kind":"LocalReturn"}],"tito":null}]},{"port":"formal(cumtime)","taint":[{"kinds":[{"return_paths":{"[cumtime]":999999},"kind":"LocalReturn"}],"tito":null}]}]}}
{"kind":"model","data":{"callable":"pstats.StatsProfile.__init__","tito":[{"port":"formal(total_tt)","taint":[{"kinds":[{"return_paths":{"[total_tt]":999999},"kind":"LocalReturn"}],"tito":null}]},{"port":"formal(func_profiles)","taint":[{"kinds":[{"return_paths":{"[func_profiles]":999999},"kind":"LocalReturn"}],"tito":null}]}]}}
I don't understand why I am getting this error: is it my fault or some incompatibility?
what can i do to apply a list of filters and get a merge results in web interface??
hi all ,I noticed that a list of filters can be applied to a single list of issues and show me a merged results by Command-Line Interface.
But It seems that web interface don't has this feature.How can i apply several filters and get a merged reaults in web interface??
Failed to parse the result of Mariana-trench
What's the Problem
I found Mariana-trench support "field" type sources/sinks : https://mariana-tren.ch/docs/models/#field-models
I tried to use this feature. and I catch an error when I use sapp to show the result. as follow:
Traceback (most recent call last):
File "/Users/listennter/.venvs/dev-mariana/bin/sapp", line 8, in <module>
sys.exit(cli())
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/click/decorators.py", line 84, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/sapp/cli_lib.py", line 207, in analyze
pipeline.run(analysis_output, summary_blob)
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/sapp/pipeline/__init__.py", line 441, in run
next_input, summary = step.run(next_input, summary)
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 251, in run
self.analysis_output_to_dict_entries(
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 203, in analysis_output_to_dict_entries
for typ, key, e in self._analysis_output_to_parsed_tuples(inputfile):
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/sapp/pipeline/base_parser.py", line 141, in _analysis_output_to_parsed_tuples
for e in entries:
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser.py", line 338, in parse
yield from self.parse_handle(handle)
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser.py", line 348, in parse_handle
for precondition in self._parse_precondition(model):
File "/Users/listennter/.venvs/dev-mariana/lib/python3.9/site-packages/sapp/pipeline/mariana_trench_parser.py", line 448, in _parse_precondition
caller = Method.from_json(model["method"])
KeyError: 'method'
What Cause this Problem
When mariana find an issue which source/sink is an "Field". It will contain an json like this in its output
{
"field": "Lcom/unicom/xiaowo/account/shield/e/g;.d:Ljava/lang/String;",
"position": {},
"sinks": [
{
"callee_port": "Leaf",
"field_callee": "Lcom/unicom/xiaowo/account/shield/e/g;.d:Ljava/lang/String;",
"field_origins": [
"Lcom/unicom/xiaowo/account/shield/e/g;.d:Ljava/lang/String;"
],
"kind": "N1rv0us-sink"
}
],
"sources": [
{
"callee_port": "Leaf",
"field_callee": "Lcom/unicom/xiaowo/account/shield/e/g;.d:Ljava/lang/String;",
"field_origins": [
"Lcom/unicom/xiaowo/account/shield/e/g;.d:Ljava/lang/String;"
],
"kind": "SSID"
}
],
"line": 9805
}but when sapp pasing precondition/postcondition, It will assume that all results are method,thus ignoring the parsing of field.
some code in sapp/pipeline/mariana_trench_parser.py
https://github.com/facebook/sapp/blob/main/sapp/pipeline/mariana_trench_parser.py
def _parse_precondition(self, model: Dict[str, Any]) -> Iterable[Precondition]:
caller = Method.from_json(model["method"])
caller_position = Position.from_json(model["position"], caller)
...
def _parse_postconditions(self, model: Dict[str, Any]) -> Iterable[Postcondition]:
caller = Method.from_json(model["method"])
caller_position = Position.from_json(model["position"], caller)
... How to Fix
Hope to support the parsing of Field results soon.
Error installing `fb-sapp` due to dependency issue
Hi Folks,
While trying to install fb-sapp along with pyre-check, I am receiving the following error:
Because no versions of pyre-check match >0.9.18,<0.10.0
and pyre-check (0.9.18) depends on pyre-extensions (>=0.0.29), pyre-check (>=0.9.18,<0.10.0) requires pyre-extensions (>=0.0.29).
And because fb-sapp (0.5.4) depends on pyre-extensions (0.0.27)
and no versions of fb-sapp match >0.5.4,<0.6.0, pyre-check (>=0.9.18,<0.10.0) is incompatible with fb-sapp (>=0.5.4,<0.6.0).
So, because test-project depends on both fb-sapp (^0.5.4) and pyre-check (^0.9.18), version solving failed.
The pyre-check version I am trying to install is the latest as of writing this issue, which is 0.9.18 and I have tried poetry and pip both. This installation works correctly with pyre-check version 0.9.15.
I believe the pyre-extension package is outdated in fb-sapp (which is 0.0.27) and needs to be in-line with the latest pyre-check version (which is 0.0.30)
Feature: --database-name should take a URL
I'm looking to tinker with using SAPP with multiple users, so I'd like to be able to run with some client/server style database (I'm thinking postgres or possibly python-dqlite. ). However the DB class does not handle URLs directly. It is typically in SQLAlchemy to use a URL to define all the connection parameters of a database. The DBType enum is somewhat superfluous, since the driver can be inferred from a proper URL. It would be pretty straightforward to refactor it to pass in a URL and just pass that directly to sqlalchemy.engine.url.make_url, and fall back if it's a file path.
If you wanted to keep the same interface, you could have a helper function something like
class DBType(Enum):
XDB = "xdb" # not yet implemented
INFER = "infer"
SQLITE = "sqlite"
MEMORY = "memory"
def _make_url(name_or_url: Optional[Union[str, sqlalchemy.engine.url.URL]] = None,
dbtype: Union[DBType, str] = DBType.INFER,
default_db_file: str = 'sapp.db') -> sqlalchemy.engine.url.URL:
if dbtype is DBType.MEMORY or name_or_url == ':memory:':
return sqlalchemy.engine.url.URL('sqlite', database=":memory:")
if dbtype is DBType.SQLITE:
return sqlalchemy.engine.url.URL('sqlite', database=name_or_url or default_db_file)
if dbtype is DBType.INFER:
return sqlalchemy.engine.url.make_url(name_or_url)
raise errors.AIException(f'unsupported database type: {dbtype}')
This would keep the existing CLI behavior exactly as is, while allowing folks to pass in different URLs. Obviously plugging into postgres would require a bit more tooling (seems there is some graphene integration which relies on some particular functions/stored procedures - it looks tractable) but I think this would be a start in the right direction. It's a useful feature in its own right - URLs are a pretty standard way to deal with database connections. Caveat: I'm not sure what XDB is. Also is there any reason why DBType is a sqlalchemy.Enum as opposed to enum.Enum?
I could PR this if you want. Just sketched this idea out and tests pass.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.