intelowlproject / pyintelowl Goto Github PK

I have an Intel Owl server set up using Traefik for Let's Encrypt, HTTPS is exposed to the internet. I can use the web app just fine but when I attempt to use pyintelowl CLI I get an SSL certificate error. I've tried creating a PEM with the full cert chain and a PEM with just the first certificate in the chain by piping to openssl ( | openssl x509 -outform PEM ) instead of using sed like in the screenshot, and no luck using either PEM file in my config.

Error:

[14:38:31] CRITICAL HTTPSConnectionPool(host='<redacted>', port=443): Max retries         jobs.py:55
                    exceeded with url: /api/jobs (Caused by SSLError(SSLCertVerificationError(1, '[SSL:
                    CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get issuer certificate
                    (_ssl.c:1123)')))

Let me know if I can provide any other info that'd be helpful.

Like in IntelOwl. See this PR: intelowlproject/IntelOwl#305

As per my GSoC Project, I would now be moving forward to adding PyIntelOwl support for my playbooks feature once it has been merged.

Part of intelowlproject/IntelOwl#638

Use TLP instead of force_privacy, disable_external_analyzers and private_job booleans

Should also add a IntelOwl.delete_job_by_id(job_id: int) function in pyintelowl and create a corresponding command.

Originally posted by @eshaan7 in intelowlproject/IntelOwl#324 (comment)

Cmd syntax: pyintelowl jobs rm {id}

Not sure what I'm doing wrong here. Installing on WSL 1 in a Debian instance.

I blew out my old client and cloned this from scratch.

Ran "pip3 install -r requirements.txt" and everything appears to have pulled in successfully.

I've installed from source with "pip3 install -e pyintelow/" with seemingly no errors.

However, when I try to invoke pyintelowl (even just with the -h parameter to get help doc) I get this:

[08:38:14] neil@CPI-0745:~/git$ pyintelowl --version Traceback (most recent call last): File "/usr/local/bin/pyintelowl", line 11, in <module> load_entry_point('pyintelowl', 'console_scripts', 'pyintelowl')() File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 489, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2792, in load_entry_point raise ImportError("Entry point %r not found" % ((group, name),)) ImportError: Entry point ('console_scripts', 'pyintelowl') not found

Something about entry points.

Is it possible to extract screenshots taken by rendertron?

Just tried to launch an analysis on a filename observable as : 'apk.xrat'

But since the method send_observable_analysis_request() don't allow us to classify the observable

And the classification method detect it as an domain.

Is there a way to launch an observable on generic ? If yes could you tell me how ?
Thanks

I tried to run the following command

-k -i -aa observable -v 8.8.8.8

But while the -aa file version succedes, this one gives

['400 Client Error: Bad Request for url: http:///api/ask_analysis_availability?md5=40ff44d9e619b17524bf3763204f9cbb&running_only=True'] md5:40ff44d9e619b17524bf3763204f9cbb

Is it an error or is it supposed to work only for the file type?

-j, --json as a boolean flag, if set then the response will be displayed as raw JSON.

Related to intelowlproject/IntelOwl#534.

This generic method to perform API requests to IntelOwl does provide logs that are made for the CLI only.

PyIntelOwl should be developed to be used the same way both as a library and as a CLI.

Logs like this one should not be used.

https://github.com/Eshaan7/click-creds

Currently, there isn't a way to create/edit tags via command line. It would be nice to have those features.

PS: Working on this.

It looks like when using pyintelowl on Windows terminals (Terminal/PowerShell/CMD) unicode formatting does not get displayed properly.

These screenshots are from the get-analyzer-config command.

Hi guys,

When investigating the intelligence, our steps are often the iteration of extracting info and then analyzing the info.

For instance, when digging into a binary, we might want to know if there are base64 encoded strings inside. So our first step would be extract encoded strings that are most likely base64 encoded. Then we can start to analyze the decoded strings.

Or it could be the scenario that we'd like to find (extract) URLs inside the binary and then analyze them.

I think IntelOwl is doing very well of what mentioned above. And I think it would be so nice if we can explore the threat intelligence with enhenced pyintelowl.

To do so, we need to design more Intelowl APIs making the data extraction and piping it to the analyzers more easily.

Here's the concept.

$> open /home/test/binaryfile
file successfully opened
session id: 1345873

$> file.Type()
binaryfile: Dalvik dex file version 035

$> URLs = file.findURLs()
["https://a.b.c", "https://a.a.c"]

$> InspectURL(URLs[1])
Then show analysis results from URL related analyzers. 

To make the above concept works, we need to

1. Design APIs that can extract basic information from the tartget. (Binary is just one of a kind)
2. Design APIs that make the analyzers easy to be used. Easy to pipe in the data and easy to read and reuse the results.

So what do you guys think? :D
Please correct me if I'm wrong with something.

New member function def kill_running_job(job_id: int) -> bool in pyintelowl.

Originally posted by @eshaan7 in intelowlproject/IntelOwl#225 (comment)

Is there an option to run all enabled analyzers for an observable (where applicable)? If not, could this be considered?

If not immediately available, is there an option to perform a GET to the API to retrieve the list of currently enabled analyzers and their supported data types so that we could feed this list to a request for analysis?

referred to #145

Hi,

I installed pyintelowl to use it as a CLI but, upon running, it is trying to load CLI module and it will not run. Is there any possible explanation for that? Did I miss any steps before running it?

Issue:

Currently, pyintelowl is very limited in the functionality but holds a lot of potential.

Idea:

Could rewrite pyintelowl with https://github.com/pallets/click, to make it user friendly and add support for more of IntelOwl's API endpoints (or commands in this context).

Proposal:

Command map:

• pyintelowl jobs --status "running" - to list all running jobs, --status reported_without_fails and so on.

• pyintelowl jobs poll <job_id> --interval 10 --max-tries 5 - to poll for a job with interval being 10 seconds and maximum requests being 5

• pyintelowl config --set api-key=<token> - to persist the token (user won't have to specify in the command line each time)

• pyintelowl config --set intelowl-url=localhost:80 - same as above

• pyintelowl config --get - prints current config

• pyintelowl tags -a - to list all tags

• pyintelowl tags <tag_id>- to view a tag by ID

• pyintelowl jobs -a or pyintelowl jobs --all - to list of all jobs (ideally, only top 5 by

• pyintelowl jobs <job_id> - to view a job by ID

• pyintelowl analyse - to send new analysis request

  • pyintelowl analyse observable google.com -al zoomeye -al shodan
  • pyintelowl analyse file malware.exe -al zoomeye -al shodan

• pyintelowl get_analyzer_config or pyintelowl gac for short (should be displayed as a table, can follow same one on web
  .. and more! (ideas welcome)

• We should leverage the Rich text library for pretty formatting for a super-charged UX (#31).

• We should use third party click extensions such as click-spinner, click-didyoumean, click-completion too.

intelowlproject/IntelOwl#397

Corresponding methods would have to be written for the new endpoints made along with cli commands for executing them.

• Endpoint - get-connector-config (PR- #112)
• Plugin actions- kill/ retry / health_check (#111)
• Support for showing connector_reports in job view command. (#115)
• Add column "Connectors called" to table in the jobs ls command. (#116)

We can add a template for pull requests so that new contributors can follow a format/keep track of a checklist.

For ref, see pull_request_template.md in intelowlproject/IntelOwl

There should be functionality where User provides a formated file(csv, json etc... ) which contains information regarding what analysis should be performed.

After all, Command line should bring some more automation than Web Interface, Isn't it?

there is a formatting issue to be investigated

https://stackoverflow.com/questions/50504500/deprecationwarning-invalid-escape-sequence-what-to-use-instead-of-d
seems to suggest an alternative.

• Analyzers: (#110)

These 2 lists should instead be extracted out as requirements.txt and requirements.dev.txt files. And then these files can be read using pathlib (see here: https://github.com/Eshaan7/click-creds/blob/main/setup.py)

Cmd syntax: pyintelowl tags rm {id}
endpoint: DELETE /api/tags/{id}

Other than continuous-integration, It is really necessary to prepare unit-tests as in the case of IntelOwl.

edited by eshaan7:

• test cases for CLI commands
• test cases for IntelOwl class' member methods.
• tox setup and it's integration with GH workflow action file. (See #106)

@eshaan7:

https://github.com/willmcgugan/rich
Using this effectively in pyintelowl could make the user experience 100x better

So let's try this :P

issue on IntelOwl: intelowlproject/IntelOwl#397

Corresponding methods would have to be written for the new endpoints made along with cli commands for executing them.

Hello, would it be possible to split Python SDK and CLI app projects so that there is a lower probability that SDK clashes with some other projects because of CLI only library? If not, would it be possible to not put such strict requirements on the rich library?