t0pcyber / hawk Goto Github PK
View Code? Open in Web Editor NEWPowershell Based tool for gathering information related to O365 intrusions and potential Breaches
Home Page: https://cloudforensicator.com/
License: MIT License
Powershell Based tool for gathering information related to O365 intrusions and potential Breaches
Home Page: https://cloudforensicator.com/
License: MIT License
Describe the bug
When running the Search-HawkTenantEXOAuditLog cmdlet I see the following line in the output:
The forwarding_recipients.csv is created but it does not contain any content.
Simple_forwarding _changes.csv is created successfully and contains content.
Steps to reproduce the behavior:
Run the Search-HawkTenantEXOAuditLog cmdlet.
Additional context
Have tried running this multiple times with the same result.
Is it possible to parse the JSON data in the Exchange_UAL_Audit_.csv log file into seperate columns for ease of readability?
Thank you!!
Script calls Get-SweepRules cmdlet, but this does not exist. It should be Get-SweepRule
Start-HawkUserInvestigation -UserPrincipalName [email protected] -Verbose
Get-SweepRules : The term 'Get-SweepRules' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At C:\Program Files\WindowsPowerShell\Modules\HAWK\1.8.7\User\Get-HawkUserInboxRule.ps1:95 char:23 + $SweepRules = Get-SweepRules -Mailbox $User + ~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (Get-SweepRules:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException
I tried to install and configure the Hack framework and after downloading and running the Powershell installation script, I got the following error messages, do you know what could be wrong?
PS C:\Windows\system32> Start-HawkUserInvestigation
Start-HawkUserInvestigation : The 'Start-HawkUserInvestigation' command was found in the module 'Hawk', but the module
could not be loaded. For more information, run 'Import-Module Hawk'.
At line:1 char:1
+ CategoryInfo : ObjectNotFound: (Start-HawkUserInvestigation:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CouldNotAutoloadMatchingModule
PS C:\Windows\system32> import-module hawk
import-module : The required module 'PSFramework' is not loaded. Load the module or remove the module from
'RequiredModules' in the file 'C:\Program Files\WindowsPowerShell\Modules\hawk\2.0.0\hawk.psd1'.
At line:1 char:1
+ CategoryInfo : ResourceUnavailable: (C:\Program File...2.0.0\hawk.psd1:String) [Import-Module], Missing
MemberException
+ FullyQualifiedErrorId : Modules_InvalidManifest,Microsoft.PowerShell.Commands.ImportModuleCommand
At line 71 of the Start-HawkUserInvestigation, another "param" statement is made. This will result in an error. It looks like this is an accidental copy from the param state made a couple of lines above, I suggest removal.
param : The term 'param' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At C:\Program Files\WindowsPowerShell\Modules\HAWK\1.10.1\User\Start-HawkUserInvestigation.ps1:71 char:5 + param + ~~~~~ + CategoryInfo : ObjectNotFound: (param:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException
Is the 50,000 record limit still in play? I'm getting issues...
Snippet:
[27/08/2019 11:03:07] - Search-UnifiedAuditLog -UserIDs xxxxx -RecordType ExchangeItem -StartDate '05/29/2019 00:00:00' -EndDate '08/28/2019 00:00:00' -SessionCommand ReturnLargeSet -resultsize 1000 -sessionid 110307
[27/08/2019 11:03:24] - Retrieved:1000 Total: 254452
…
[27/08/2019 11:25:55] - Retrieved:48000 Total: 254452
[27/08/2019 11:26:05] - Retrieved:49000 Total: 254452
[27/08/2019 11:26:17] - Retrieved:50000 Total: 254452
[27/08/2019 11:26:21] - Retrieved:50000 Total: 254452
[27/08/2019 11:26:23] - Retrieved:50000 Total: 254452
[27/08/2019 11:26:24] - Retrieved:50000 Total: 254452
[27/08/2019 11:26:26] - Retrieved:50000 Total: 254452
When it hits 50000, there are stdout errors on the screen. Sorry - forgot to capture them.
Thanks
Describe the bug
I'm getting multiple instances of this error when running a user analysis.
[6/9/2022 3:57:17 PM] - Converting AuditData
[6/9/2022 3:57:17 PM] - [ERROR] - 0 Entries failed JSON Conversion
[6/9/2022 3:57:17 PM] - No Data Found
Get-IPGeolocation : Cannot bind argument to parameter 'IPAddress' because it is null.
At line:109 char:62
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Additional context
When I ran this yesterday I was seeing a lot of output messages that just said failed to look up geo information, but that was before I acquired an API key so was to be expected. I've since added an API key to the Hawk.json file in my AppData folder.
When performing Start-HawkUserInvestigation
the console outputs in a first line "Initilizing Application Insights" - this should be corrected to "Initializing Application Insights".
The current methodology creates a file similar to
yyyymmddhh_mm\emailaddress\User_Changes.csv
Each user has their own file in their own directory but they are all called user_changes.csv
Attempting to double click to open in Excel gives an error that you can't open more than one file with the same name.
Suggestion would be to encode the upn in the filename as well as the foldername.
Should this be a pull request with the changes made? Should it be a feature request?
Review all the code to remove MSOnline required cmdlets and update to V2 Azure AD cmdlets.
Update Folder Name Creation to use AAD Module
After detecting a compromised account, in my experience it's useful to check the sent items for emails that the user has sent out to see how bad the scope of compromise is (assuming the attack spammed out everyone in the contact list).
Would adding a feature to the script that runs a message trace to a csv for the user for the past X days (based on the original tenant day score) or the past 2 days be useful or would this be considered bloat for the project?
Update Hawk with Solarwinds IOCs.
Script to alter:
HAWK\<version>\internal\functions\Initialize-HawkGlobalObject.ps1
Line to alter:
81
The change:
Possible to add the following to get the default domain of the tenant to be used in the file name?
$TenantName = (Get-MsolDomain | Where-Object {$_.isDefault}).Name
$FileName = $TenantName.Substring(0, $TenantName.IndexOf('.'))
After change:
New directory name would look like
Hawk_contoso_20220227_1513
This would be super cool. This project is fantastic. Keep up the great work!
After upgrading to todays version (1.13.2 from 1.10.1) , running start-hawktenantinvestigation runs in a seemingly infinite loop of "Checking for latest version online"
Looking to see why now but wanted to post the issue in case anyone else had seen it
Search-UnifiedAuditLog will only return 50k items. If the search gets back >50k items we have two issues:
Currently Dates are handled as a string conversion into the Hawk Global Object. This works great as long as you are using the US date format. From non-us date formats this is causing issues.
Need to move this to holding an actual Date object instead of a string version.
All of the searches are going to need to move to proper processing of dates (some of them don't take date objects and require strings)
It would be nice to install the Hawk Module if it could detect if you are already running Module: AzureAD or Module: AzureADPreview.
VERBOSE: Completed downloading 'https://www.powershellgallery.com/api/v2/package/HAWK/1.15.0'.
VERBOSE: Completed downloading 'HAWK'.
VERBOSE: Hash for package 'HAWK' does not match hash provided from the server.
VERBOSE: InstallPackageLocal' - name='HAWK', version='1.15.0',destination='C:\Users\Jordan\AppData\Local\Temp\2046577437'
VERBOSE: Validating the 'AzureAD' module contents under 'C:\Users\Jordan\AppData\Local\Temp\2046577437\AzureAD.2.0.2.16' path.
VERBOSE: Test-ModuleManifest successfully validated the module manifest file 'C:\Users\Jordan\AppData\Local\Temp\2046577437\AzureAD.2.0.2.16'.
VERBOSE: Validating the authenticode signature and publisher of the catalog file or module manifest file of the module 'AzureAD'.
VERBOSE: Catalog file 'AzureAD.cat' is not found in the contents of the module 'AzureAD' being installed.
VERBOSE: Valid authenticode signature found in the file 'AzureAD.psd1' for the module 'AzureAD'.
VERBOSE: Checking for possible command collisions for the module 'AzureAD' commands.
PackageManagement\Install-Package : The following commands are already available on this system:'Add-AzureADApplicationOwner
Shorten the list of all CmdLets
,Set-AzureADUserPassword,Set-AzureADUserThumbnailPhoto,Update-AzureADSignedInUserPassword'. This module 'AzureAD' may override the existing commands. If you still want to install this module 'AzureAD', use
-AllowClobber parameter.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\2.2.3\PSModule.psm1:9685 char:34
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The Search-UnifiedAuditLog command that gets generated seems to generate with dates and times in the local format. As a UK native our format of dd/mm/yyyy does not get accepted by that cmdlet, causing an error:
Cannot process argument transformation on parameter 'EndDate'. Cannot convert value "19/09/2019" to type "Microsoft.Exchange.ExchangeSystem.ExDateTime". Error: "String was not recognized as a valid DateTime."
I ran the Start-HawkUserInvestigation command with the Verbose switch, and this returned some Verbose output stating it was doing a IP lookup for IP "null", which will lead to a fail because resolving a null IP won't work of course :)
[7/29/2019 12:34:59 PM] - Recording HawkAppData to file C:\Users\User\AppData\Local\Hawk\Hawk.json VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:00 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:00 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:01 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:01 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:01 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:01 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:01 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:02 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:02 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:02 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:02 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:03 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:03 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:03 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:03 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:04 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:04 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:04 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:04 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:05 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:05 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:05 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:05 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:06 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:06 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:06 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:06 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:07 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:07 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:07 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:07 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:08 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:08 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:08 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:08 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:09 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:09 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:09 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:09 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:10 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:10 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:10 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:10 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:10 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:11 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:11 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:11 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:11 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:11 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:12 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:12 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:12 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:12 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:13 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:13 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:13 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:13 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:13 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:14 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:14 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:14 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:14 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:15 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:15 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:15 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:15 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:15 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:16 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:16 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:16 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:16 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:16 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:17 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:17 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:17 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:17 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:18 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:18 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:18 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:18 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:18 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:19 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:19 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:19 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:19 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:20 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:20 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:20 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:20 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:20 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:21 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:21 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:21 PM] - Failed to retreive location for IP <null> VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 [7/29/2019 12:35:21 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:22 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:22 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:22 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:22 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:23 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:23 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:23 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:23 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:23 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:24 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:24 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:24 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:24 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:25 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:25 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/<null>?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:25 PM] - Failed to retreive location for IP <null>
VERBOSE: GET http://api.ipstack.com/84.26.178.59?access_key=xxxx with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
[7/29/2019 12:35:25 PM] - Building MSFTIPList
[7/29/2019 12:35:25 PM] - Loading Networking functions from C:\Program Files\WindowsPowerShell\Modules\HAWK\1.10.1\System.Net.IPNetwork.dll
VERBOSE: GET https://endpoints.office.com/endpoints/Worldwide?ClientRequestId=xxx with 0-byte payload
VERBOSE: received 64723-byte response of content type application/json; charset=utf-8
[7/29/2019 12:35:27 PM] - Found 196 unique MSFT IPv6 address ranges
[7/29/2019 12:35:27 PM] - Found 81 unique MSFT IPv4 address ranges
[7/29/2019 12:35:27 PM] - Creating global variable $MSFTIPList
When running the Search-HawkTenantActivityByIP command to investigate an IP address, the command fails with the below error. It always reports 0 users were accessed but generates a report "Unique_Users_Attempted.csv" with successful UserLoggedIn events
Select-UniqueObject : Cannot bind argument to parameter 'ObjectArray' because it is null.
At C:\Program Files\WindowsPowerShell\Modules\HAWK\1.10.1\Tenant\Search-HawkTenantActivityByIP.ps1:49 char:76
~~~~~~~~~~~~~~~~
Need to be able to specify date ranges for instances where we know activity occurred during a window vs. now back.
To make sure that this work probably need to call it out as a separate cmdlet.
Describe the bug
Non-ASCII characters can't output in CSV.
HAWK outputs CSV files with only ASCII characters.
All non-ASCII characters such as Chinese, Japanese, Russian are converted to symbols of ?
.
The reason for this is using Export-Csv cmdlets without specifying encoding in the Out-MultipleFileType.ps1. The cmdlet uses the ASCII encoding.
According to the Microsoft Docs and my research, each cmdlet uses the following encodings by default.
Cmdlet | Default encoding |
---|---|
Export-Csv | ASCII |
Export-Csv (with -Append ) |
UTF-8 without BOM (It matches the existing encoding when the target file contains a BOM. In the absence of a BOM, it uses UTF-8 encoding.) |
Export-CliXml | UTF-16LE with BOM |
Out-File | UTF-16LE with BOM |
Currently, the HAWK's output encoding of xml
is UTF-16LE(BOM), and txt
is UTF-16LE(BOM), and csv
is ASCII.
So, HAWK's output of xml
and txt
files can contain non-ASCII characters, but csv
files can't.
To fix it
It could be resolve by using Export-Csv cmdlet with encoding option -Encoding <encoding name>
.
I recommend using UTF-16LE(BOM) with specifying -Encoding Unicode
.
(However, UTF-8(BOM) may be better to work with Excel.)
The pros and cons of encodings are as below.
Encoding | Pros | Cons |
---|---|---|
UTF-16LE(BOM) | Same encoding with XML and TXT files in HAWK outputs. Most tools correctly work. | Excel can't double click open and show UTF-16 CSV file properly. Excel shows it as not delimited. It reproduces on the current version, too. However, the From Text/CSV Import button can correctly import such CSV files. So it doesn't seem to be a big deal. |
UTF-8(BOM) | Excel correctly works. | Some tools do not consider the existence of BOM of UTF-8, so make trouble |
UTF-16LE(no-BOM) | - | (Can't specified as encoding option in PowerShell v5) |
UTF-8(no-BOM) | - | (Can't specified as encoding option in PowerShell v5) |
To Reproduce
Steps to reproduce the behavior:
テストabc1
.Get-HawkUserInvestigation <your mail address>
Exchange_Mailbox_Audit_<username>.csv
.The CSV file will be ASCII encoding and contain some ?
symbols in the log record of deleting operation.
All non-ASCII characters are replaced with ?
symbols.
"PSComputerName","RunspaceId","PSShowComputerName","Operation","OperationResult","LogonType","ExternalAccess","DestFolderId","DestFolderPathName","FolderId","FolderPathName","FolderName","MemberRights","MemberSid","MemberUpn","ClientInfoString","ClientIPAddress","ClientIP","ClientMachineName","ClientProcessName","ClientVersion","InternalLogonType","MailboxOwnerUPN","MailboxOwnerSid","DestMailboxOwnerUPN","DestMailboxOwnerSid","DestMailboxGuid","CrossMailboxOperation","LogonUserDisplayName","LogonUserSid","SourceItems","SourceFolders","SourceItemIdsList","SourceItemSubjectsList","SourceItemAttachmentsList","SourceItemFolderPathNamesList","SourceFolderPathNamesList","SourceItemInternetMessageIdsList","ItemId","ItemSubject","ItemAttachments","ItemInternetMessageId","DirtyProperties","OriginatingServer","SessionId","OperationProperties","AuditOperationsCountInAggregatedRecord","AggregatedRecordFoldersData","AppId","ClientAppId","ItemIsRecord","ItemComplianceLabel","MailboxGuid","MailboxResolvedOwnerName","LastAccessed","Identity","IsValid","ObjectState"
"outlook.office365.com","1111111-dummy","FALSE","MoveToDeletedItems","Succeeded","Owner","FALSE","LgAAAAAAAAAAAAAAAAAADUMY","\????????","LgAAAAAAAAAAAAAAAAA","\?????","","","","","Client=OWA;Action=ViaProxy","2001:db8::","2001:db8::","","","","Owner","[email protected]","S-1-1111111111DUMY","","","","FALSE","user1","S-1-1111111111DUMY","RgAAAAAAADUMY","","RgAAAAAAADUMY","???abc1","","?????","","<[email protected]>","","","","","","OS1P123456 (10.00.000.000)","c1111-1111","","","","00000002-0000-0000-000-000000000000","","","","aa111-0000","user1","2021/9/28 18:08","AAAAA=","TRUE","New"
Expected (better) behavior
HAWK can output CSV files containing non-ASCII characters, such as テストabc1
.
"PSComputerName","RunspaceId","PSShowComputerName","Operation","OperationResult","LogonType","ExternalAccess","DestFolderId","DestFolderPathName","FolderId","FolderPathName","FolderName","MemberRights","MemberSid","MemberUpn","ClientInfoString","ClientIPAddress","ClientIP","ClientMachineName","ClientProcessName","ClientVersion","InternalLogonType","MailboxOwnerUPN","MailboxOwnerSid","DestMailboxOwnerUPN","DestMailboxOwnerSid","DestMailboxGuid","CrossMailboxOperation","LogonUserDisplayName","LogonUserSid","SourceItems","SourceFolders","SourceItemIdsList","SourceItemSubjectsList","SourceItemAttachmentsList","SourceItemFolderPathNamesList","SourceFolderPathNamesList","SourceItemInternetMessageIdsList","ItemId","ItemSubject","ItemAttachments","ItemInternetMessageId","DirtyProperties","OriginatingServer","SessionId","OperationProperties","AuditOperationsCountInAggregatedRecord","AggregatedRecordFoldersData","AppId","ClientAppId","ItemIsRecord","ItemComplianceLabel","MailboxGuid","MailboxResolvedOwnerName","LastAccessed","Identity","IsValid","ObjectState"
"outlook.office365.com","1111111-dummy","FALSE","MoveToDeletedItems","Succeeded","Owner","FALSE","LgAAAAAAAAAAAAAAAAAADUMY","\削除済みアイテム","LgAAAAAAAAAAAAAAAAA","\受信トレイ","","","","","Client=OWA;Action=ViaProxy","2001:db8::","2001:db8::","","","","Owner","[email protected]","S-1-1111111111DUMY","","","","FALSE","user1","S-1-1111111111DUMY","RgAAAAAAADUMY","","RgAAAAAAADUMY","テストabc1","","受信トレイ","","<[email protected]>","","","","","","OS1P123456 (10.00.000.000)","c1111-1111","","","","00000002-0000-0000-000-000000000000","","","","aa111-0000","user1","2021/9/28 18:08","AAAAA=","TRUE","New"
Screenshots
N/A
File (please complete the following information):
Additional context
N/A
In the past couple of runs I've got both the Sweep Rules and the IP rules issue that were fixed in previous versions.
Running an investigation tells me that I'm on the current version (but doesn't document what that version is on the line but later on it tells me the version is 1.8.7 and hawk.psd1 tells me we are on 1.10.1
Would it be good to a) Document the version at the top of the runtime b)Have the dates of the changes in hawk.psd1 c) have the module self update if not actually up to date?
`Checking for latest version online
Latest Version Installed
Skipping Upgrade
Setting Up initial Hawk environment variable
DISCLAIMER:
THE SAMPLE SCRIPTS ARE NOT SUPPORTED UNDER ANY MICROSOFT STANDARD SUPPORT
Disclaimer
Do you agree with the above disclaimer?
[Y] Yes [N] No [?] Help (default is "Y"): Y
Please provide an output directory: c:\temp\hawk\ Creating subfolder with name c:\temp\hawk\20190712_1200
How far back in the past should we search? (1-90 Default 90): 14
Advanced Azure AD License NOT Found
Setting up Global Hawk environment variable
[7/12/2019 12:00:27 PM] - Global Variable Configured
[7/12/2019 12:00:28 PM] - Version 1.8.7`
Get-HawkUserPWNCheck.ps1:57 char:94
~
You must provide a value expression following the '+' operator.
Fix coming shortly.
Break out Initialize-HawkModule into individual ps1 files. The internal functions aren't made public and keep failing validation.
Hi,
Just started using HAWK to pull out all of the audit information on my 365 tenant but im coming across streams of errors relating to the date/time - i am running this within the UK, im not sure if this has any impact on your powershell code?
It then gets stuck in an infinite loop when searching a date range and carrying out various other tasks. It fails to pull anything from the audit files because of these errors.
Any ideas?
Thanks.
The module will throw errors during the IP lookup from ipstack:
Start-HawkUserInvestigation -UserPrincipalName [email protected] -Verbose
[4-7-2019 08:59:35] - Creating global variable $MSFTIPList Cannot index into a null array. At C:\Program Files\WindowsPowerShell\Modules\HAWK\1.8.7\Hawk.psm1:936 char:13 + $test = [System.Net.IPNetwork]::Contains($MSFTIPList.ipv4 ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : NullArray VERBOSE: GET http://api.ipstack.com/XXXX?access_key=XXXXX with 0-byte payload VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8 Cannot index into a null array. At C:\Program Files\WindowsPowerShell\Modules\HAWK\1.8.7\Hawk.psm1:936 char:13 + $test = [System.Net.IPNetwork]::Contains($MSFTIPList.ipv4 ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : NullArray
Describe the bug
Get mailbox audit log returns records past the end of the specified date range. This is because there is no check to see if $RangeEnd is greater than $EndDate within the Get-MailboxAuditLogsFiveDaysAtATime do...while loop.
File (please complete the following information):
Get-HawkUserMailboxAuditing.ps1
I've come into scenarios where the ip's are not looked up and no converted_authentication_log file is generated. Is there a reason/criteria for when this occurs? In a compromised account today, a search over the past 10 days did run through and create the logs however a search for the past 120 days did not.
On a related note it would be good to be able to search for logs between X and Y days such as 80-90 days ago to get more details if you knew from other experience that an account was compromised around a certain date in the past.
Describe the bug
When reading the comment based help it seems that Get-HawkUserHiddenRule should take a credential object when using the -EWSCredential parameter:
Get-HawkUserHiddenRule -UserPrincipalName [email protected] -EWSCredential (get-credential)
However, -EWSCredential is currently a switch statement.
To Reproduce
Pass a credential object to the parameter -EWSCredential
Expected behavior
The parameter -EWSCredential should accept a credential object
Install-Module -Name HAWK
Produces error on multiple computers and nothing gets installed:
PackageManagement\Install-Package : The following commands are already available on this system:'Add-AzureADApplicationOwner,Add-AzureADDeviceRegisteredOwner,Add-AzureADDeviceRegisteredUser,Add-AzureADDirec
toryRoleMember,Add-AzureADGroupMember,Add-AzureADGroupOwner,Add-AzureADMSLifecyclePolicyGroup,Add-AzureADServicePrincipalOwner,Confirm-AzureADDomain,Connect-AzureAD,Disconnect-AzureAD,Enable-AzureADDirector
yRole,Get-AzureADApplication,Get-AzureADApplicationExtensionProperty,Get-AzureADApplicationKeyCredential,Get-AzureADApplicationLogo,Get-AzureADApplicationOwner,Get-AzureADApplicationPasswordCredential,Get-A
zureADApplicationProxyApplication,Get-AzureADApplicationProxyApplicationConnectorGroup,Get-AzureADApplicationProxyConnector,Get-AzureADApplicationProxyConnectorGroup,Get-AzureADApplicationProxyConnectorGrou
pMembers,Get-AzureADApplicationProxyConnectorMemberOf,Get-AzureADApplicationServiceEndpoint,Get-AzureADContact,Get-AzureADContactDirectReport,Get-AzureADContactManager,Get-AzureADContactMembership,Get-Azure
ADContactThumbnailPhoto,Get-AzureADContract,Get-AzureADCurrentSessionInfo,Get-AzureADDeletedApplication,Get-AzureADDevice,Get-AzureADDeviceConfiguration,Get-AzureADDeviceRegisteredOwner,Get-AzureADDeviceReg
isteredUser,Get-AzureADDirectoryRole,Get-AzureADDirectoryRoleMember,Get-AzureADDirectoryRoleTemplate,Get-AzureADDomain,Get-AzureADDomainNameReference,Get-AzureADDomainServiceConfigurationRecord,Get-AzureADD
omainVerificationDnsRecord,Get-AzureADExtensionProperty,Get-AzureADGroup,Get-AzureADGroupAppRoleAssignment,Get-AzureADGroupMember,Get-AzureADGroupOwner,Get-AzureADMSDeletedDirectoryObject,Get-AzureADMSDelet
edGroup,Get-AzureADMSGroup,Get-AzureADMSGroupLifecyclePolicy,Get-AzureADMSIdentityProvider,Get-AzureADMSLifecyclePolicyGroup,Get-AzureADOAuth2PermissionGrant,Get-AzureADObjectByObjectId,Get-AzureADServiceAp
pRoleAssignedTo,Get-AzureADServiceAppRoleAssignment,Get-AzureADServicePrincipal,Get-AzureADServicePrincipalCreatedObject,Get-AzureADServicePrincipalKeyCredential,Get-AzureADServicePrincipalMembership,Get-Az
ureADServicePrincipalOAuth2PermissionGrant,Get-AzureADServicePrincipalOwnedObject,Get-AzureADServicePrincipalOwner,Get-AzureADServicePrincipalPasswordCredential,Get-AzureADSubscribedSku,Get-AzureADTenantDet
ail,Get-AzureADTrustedCertificateAuthority,Get-AzureADUser,Get-AzureADUserAppRoleAssignment,Get-AzureADUserCreatedObject,Get-AzureADUserDirectReport,Get-AzureADUserExtension,Get-AzureADUserLicenseDetail,Get
-AzureADUserManager,Get-AzureADUserMembership,Get-AzureADUserOAuth2PermissionGrant,Get-AzureADUserOwnedDevice,Get-AzureADUserOwnedObject,Get-AzureADUserRegisteredDevice,Get-AzureADUserThumbnailPhoto,New-Azu
reADApplication,New-AzureADApplicationExtensionProperty,New-AzureADApplicationKeyCredential,New-AzureADApplicationPasswordCredential,New-AzureADApplicationProxyApplication,New-AzureADApplicationProxyConnect
orGroup,New-AzureADDevice,New-AzureADDomain,New-AzureADGroup,New-AzureADGroupAppRoleAssignment,New-AzureADMSGroup,New-AzureADMSGroupLifecyclePolicy,New-AzureADMSIdentityProvider,New-AzureADMSInvitation,New-
AzureADServiceAppRoleAssignment,New-AzureADServicePrincipal,New-AzureADServicePrincipalKeyCredential,New-AzureADServicePrincipalPasswordCredential,New-AzureADTrustedCertificateAuthority,New-AzureADUser,New-
AzureADUserAppRoleAssignment,Remove-AzureADApplication,Remove-AzureADApplicationExtensionProperty,Remove-AzureADApplicationKeyCredential,Remove-AzureADApplicationOwner,Remove-AzureADApplicationPasswordCrede
ntial,Remove-AzureADApplicationProxyApplication,Remove-AzureADApplicationProxyApplicationConnectorGroup,Remove-AzureADApplicationProxyConnectorGroup,Remove-AzureADContact,Remove-AzureADContactManager,Remove
-AzureADDeletedApplication,Remove-AzureADDevice,Remove-AzureADDeviceRegisteredOwner,Remove-AzureADDeviceRegisteredUser,Remove-AzureADDirectoryRoleMember,Remove-AzureADDomain,Remove-AzureADGroup,Remove-Azure
ADGroupAppRoleAssignment,Remove-AzureADGroupMember,Remove-AzureADGroupOwner,Remove-AzureADMSDeletedDirectoryObject,Remove-AzureADMSGroup,Remove-AzureADMSGroupLifecyclePolicy,Remove-AzureADMSIdentityProvider
,Remove-AzureADMSLifecyclePolicyGroup,Remove-AzureADOAuth2PermissionGrant,Remove-AzureADServiceAppRoleAssignment,Remove-AzureADServicePrincipal,Remove-AzureADServicePrincipalKeyCredential,Remove-AzureADServ
icePrincipalOwner,Remove-AzureADServicePrincipalPasswordCredential,Remove-AzureADTrustedCertificateAuthority,Remove-AzureADUser,Remove-AzureADUserAppRoleAssignment,Remove-AzureADUserExtension,Remove-AzureAD
UserManager,Reset-AzureADMSLifeCycleGroup,Restore-AzureADDeletedApplication,Restore-AzureADMSDeletedDirectoryObject,Revoke-AzureADSignedInUserAllRefreshToken,Revoke-AzureADUserAllRefreshToken,Select-AzureAD
GroupIdsContactIsMemberOf,Select-AzureADGroupIdsGroupIsMemberOf,Select-AzureADGroupIdsServicePrincipalIsMemberOf,Select-AzureADGroupIdsUserIsMemberOf,Set-AzureADApplication,Set-AzureADApplicationLogo,Set-Az
ureADApplicationProxyApplication,Set-AzureADApplicationProxyApplicationConnectorGroup,Set-AzureADApplicationProxyApplicationCustomDomainCertificate,Set-AzureADApplicationProxyApplicationSingleSignOn,Set-Azu
reADApplicationProxyConnector,Set-AzureADApplicationProxyConnectorGroup,Set-AzureADDevice,Set-AzureADDomain,Set-AzureADGroup,Set-AzureADMSGroup,Set-AzureADMSGroupLifecyclePolicy,Set-AzureADMSIdentityProvide
r,Set-AzureADServicePrincipal,Set-AzureADTenantDetail,Set-AzureADTrustedCertificateAuthority,Set-AzureADUser,Set-AzureADUserExtension,Set-AzureADUserLicense,Set-AzureADUserManager,Set-AzureADUserPassword,Se
t-AzureADUserThumbnailPhoto,Update-AzureADSignedInUserPassword'. This module 'AzureAD' may override the existing commands. If you still want to install this module 'AzureAD', use -AllowClobber parameter.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.0.0.1\PSModule.psm1:1809 char:21
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#What is your question?
Hi, I followed all the instructions but getting the below error when I run the Hawk, any suggestions?
Start-HawkTenantInvestigation : The 'Start-HawkTenantInvestigation' command was found in the module 'Hawk', but the
module could not be loaded. For more information, run 'Import-Module Hawk'.
At line:1 char:1
+ CategoryInfo : ObjectNotFound: (Start-HawkTenantInvestigation:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CouldNotAutoloadMatchingModule
Describe the bug
When doing a user investigation, getting the error:
Loading Networking functions from C:\Program Files\WindowsPowerShell\Modules\HAWK\3.0
.0\System.Net.IPNetwork.dll
Exception calling "LoadFile" with "1" argument(s): "The system cannot find the file specified. (Exception
from HRESULT: 0x80070002)"
At line:38 char:9
[Reflection.Assembly]::LoadFile($dll)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The file mentioned is actually at: C:\Program Files\WindowsPowerShell\Modules\HAWK\3.0
.0\Bin\
Copying the file from \Bin to \ removes the error.
To Reproduce
Run Start-HawkUserInvestigation -UserPrincipalName
Additional context
I've uninstalled and reinstalled the module, but the issue persists
Need to update all older Exchange Online cmdlets if required.
Describe the bug
Get-MailboxAuditLogsFiveDaysAtATime terminates prematurely if $StartDate and $EndDate are in different calendar years. This is a result of $RangeEnd being declared as a string rather than a datetime, which forces an implicit type conversion of $EndDate to string in the comparison clause of the while loop, which in turn forces a lexical comparison of $RangeStart and $EndDate, which is incorrect. In the US, for example, "12/31/2020" > "01/05/2021", which will cause an immediate termination of the while loop after the first execution of Search-MailboxAuditLog.
File (please complete the following information):
Get-HawkUserMailboxAuditing.ps1
While searching the mailbox audit logs, multiple errors will be returned, which will eventually end up in what looks like a loop:
Start-HawkUserInvestigation -UserPrincipalName [email protected] -Verbose
[4-7-2019 09:00:12] - Searching Exchange Mailbox Audit Logs (this can take some time)
Get-Date : Cannot bind parameter 'Date'. Cannot convert value "05/27/2019" to type "System.DateTime". Error: "String was not recognized as a valid DateTime."
At C:\Program Files\WindowsPowerShell\Modules\HAWK\1.8.7\User\Get-HawkUserMailboxAuditing.ps1:66 char:53
~~~~~~~~~~~~~
[4-7-2019 09:00:12] - Searching Range 05/27/2019 To
Get-Date : Cannot bind parameter 'Date' to the target. Exception setting "Date": "Cannot convert null to type "System.DateTime"."
At C:\Program Files\WindowsPowerShell\Modules\HAWK\1.8.7\User\Get-HawkUserMailboxAuditing.ps1:66 char:53
~~~~~~~~~~~~~
[4-7-2019 09:00:44] - Searching Range To
Get-Date : Cannot bind parameter 'Date' to the target. Exception setting "Date": "Cannot convert null to type "System.DateTime"."
At C:\Program Files\WindowsPowerShell\Modules\HAWK\1.8.7\User\Get-HawkUserMailboxAuditing.ps1:66 char:53
~~~~~~~~~~~~~
The last few investigations are not outputting any authentication logs and based on the PowerShell output it doesnt appear that it is trying. Has this feature been depreciated?
I recently got a new computer, ran Hawk for the first time for a long time and was not prompted to enter the geoip access key. As a result, all the geoip lookups were empty and the converted_authentication_logs had blank values in the CountryName and City columns.
Verbose mode shows that the access key was never used in the querystring.
output below
[7/14/2021 8:51:22 AM] - Failed to retreive location for IP 136.53.77.94
VERBOSE: GET http://api.ipstack.com/136.53.77.94?access_key= with 0-byte payload
VERBOSE: received -1-byte response of content type application/json; Charset=UTF-8
Found the source code and ran the line manually and verified that the ip lookup still works.
`$accesskey="80b6f3"
$ipaddress="128.146.161.32"
$resource = "http://api.ipstack.com/" + $ipaddress + "?access_key=" + $Accesskey
$geoip = Invoke-RestMethod -Method Get -URI $resource
$geoip
ip : 128.146.161.32
type : ipv4
continent_code : NA
continent_name : North America
country_code : US
country_name : United States
region_code : OH
region_name : Ohio
city : Dublin
zip : 43065
latitude : 40.15196990966797
longitude : -83.09722900390625
location : @{geoname_id=5152333; capital=Washington D.C.; languages=System.Object[];
country_flag=http://assets.ipstack.com/flags/us.svg; country_flag_emoji=🇺🇸;
country_flag_emoji_unicode=U+1F1FA U+1F1F8; calling_code=1; is_eu=False}`
Attempting to read or add the access_key with read-hawkappdata or add-hawkappdata returns the name not found
read-hawkappdata
read-hawkappdata : The term 'read-hawkappdata' is not recognized as the name of a cmdlet, function, script file, or
operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
again.
Add-HawkAppData -name access_key -value "80f3"
Add-HawkAppData : The term 'Add-HawkAppData' is not recognized as the name of a cmdlet, function, script file, or
operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
again.
How do we add the access key back into the system again?
[16/10/2019 9:15:04 PM] - Starting Tenant Sweep
Out-Log : The term 'Out-Log' is not recognized as the name of a cmdlet, function, script file, or operable program.
Out-Log "Running Get-HawkTenantConfiguration" -action"
Should it be
Out-LogFile "Running Get-HawkTenantConfiguration" -action"
Describe the bug
On my recently deployed (1 month) windows 10 boxes, I get the following error when Connecting to EXO using CloudConnect Module:
AADSTS50011: The reply URL specified in the request does not match the reply URLs configured for the application
To Reproduce
Steps to reproduce the behavior:
AADSTS50011: The reply URL specified in the request does not match the reply URLs configured for the application: 'a0c73c16-a7e3-4564-9a95-2bdf47383716'.
Expected behavior
Expected the Hawk tenant investigation to execute as normal
Additional context
Tested on two fully patched windows 10 boxes, tested on multiple 365 tenants.
Is it technically feasible for HAWK to pull RiskDetections such as those found in https://portal.azure.com/#blade/Microsoft_AAD_IAM/IdentityProtectionMenuBlade/RiskDetections, possibly through the Graph request below? In environments that do not have UAL turned on, this is often the only historical logging of unauthorized sign-ins so it is of great value.
Currently the IP address lookup code test all IP addresses and then determines if they belong to MSFT according to the MSFT JSON data from https://endpoints.office.com/endpoints/Worldwide?ClientRequestId=
This is resulting in queries being made to the GEO Location endpoint that are not needed. If it is an MSFT Address we don't need to look up the location.
Should look to optimize this further.
Thank you for the warning in the disclaimer that data is being collected. Is it documented anywhere what data you are collected? If not, would you consider doing that?
The module calls some functions from the CloudConnect module, which is not installed by default and will throw an error. I suggest just prompting the user to connect to EXO (just like the MSOL prompt at the start of the script).
Start-HawkUserInvestigation -UserPrincipalName [email protected] -Verbose
The report would be much more friendly if these two columns are next to each other. I always move the column manually as soon as I open the report.
Hi,
We are seeing a potential issue with the User log extractions on the latest version of Hawk - it seems that it is only pulling the first few available days of logs from the Exchange logs for the user selected and not the full 90 days (for example at the time of writing, I am only getting 9/10/11/12/13 of October and nothing else - I'm sure there's further activity after this!) - the output logging shows:
[07/01/2020 15:53:31] - Searching Exchange Mailbox Audit Logs (this can take some time)
[07/01/2020 15:53:31] - Searching Range 10/09/2019 00:00:00 To 10/14/2019 00:00:00
[07/01/2020 15:53:47] - Found 397 Exchange Mailbox audit records.
[07/01/2020 15:53:47] - Writing Data to .\20200107_1547...
Anyone else confirm please?
EDIT
It seems like something on the O365 back-end that has caused this issue - rolling back to an older version of Hawk has also produced the same problem now.
Also, the script seems to hang on "Looking Up Ip Address Locations" and then spits out a number of subsequent errors:
[07/01/2020 15:48:42] - Loading Networking functions from C:\Program Files\WindowsPowerShell\Modules\HAWK\1.15.0\System.Net.IPNetwork.dll Invoke-WebRequest : The underlying connection was closed: An unexpected error occurred on a receive. At C:\Program Files\WindowsPowerShell\Modules\HAWK\1.15.0\Hawk.psm1:889 char:22 + ... MSFTJSON = (Invoke-WebRequest -uri ("https://endpoints.office.com/end ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc eption + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand [07/01/2020 15:52:52] - Found 0 unique MSFT IPv6 address ranges [07/01/2020 15:52:52] - Found 0 unique MSFT IPv4 address ranges [07/01/2020 15:52:52] - Creating global variable $MSFTIPList Cannot index into a null array. At C:\Program Files\WindowsPowerShell\Modules\HAWK\1.15.0\Hawk.psm1:969 char:13 + $test = [System.Net.IPNetwork]::Contains($MSFTIPList.ipv4 ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : NullArray
*** Edit 2 ***
I've run the PowerShell command for Mailbox audit logging independently of Hawk and it works.
Thanks
Update all old Exchange Online cmdlets to modern V2 cmdlets. Ensure that old cmdlets are there for default use but use the primary V2 cmdlets by default.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.