Error handling about pswinreporting

[PrzemyslawKlys]

Hrmms, you are correct that ErrorAction Stop is required on most of the stuff (something I've learned recrently).

Now the question is do we want the script to stop if any error occurs. For example Get-EventLogSize is not critical for the script to continue. If I were running a script for 3 days (like last time I did to rescan 500gb logs) and it would throw an error at the end I would be heavily disappointed.

As for Test-Key I believe it was throwing me errors when I used Set-StrictMode -Version 2.0 that's why I added try/catch and it worked. After I stopped using Set-StrictMode it works like you say, terminating silently.

[PrzemyslawKlys]

For the Get-ADDomain and similar I actually wanted or maybe even did something (can't remember from top of my head) a function that would check for availability of it. I think it's in PSSharedGoods as I use it for PSWinDocumentation and PSAutomator. Or maybe I'm just remembering something I wanted to do. Anyways... if you have time and want to make it general function that can be used across all those prrojects be my guest :-) I actually need something for all other type of services...

[snd3r]

If I were running a script for 3 days (like last time I did to rescan 500gb logs) and it would throw an error at the end I would be heavily disappointed.

I know this feel.

A few years ago (on my prev work), I was very pleased that powershell missed errors and continued to work. Urgent installation of patches on all working computers, distribution of the configuration file to access to the central database, search for an event in one of the logs or any other disaster on 15k computers should not stop because one computer is unavailable.

But in the case of a domain change report, ignoring errors is an illusion of reliability, imho.

If the script is launched from the task scheduler and works to the end ignoring errors, how do you know that errors occurred?
If one domain controller was unavailable (some problem with network connection), and the script ignored log from it, could this behavior be considered correct?
The script generated and sent a letter without informing about the inaccessibility of some of the events, who will know about it?

In this case, I think the "Fail Fast" approach is very important.
In some other script, perhaps not.

Set-StrictMode -Version latest is very important too.

[PrzemyslawKlys]

Well then to be honest there's more to work this out if put your perspective like that:

• improve verification of all fields from configuration (right now it's half done)
• verify it for actual values (as in check whether it's from ranges it's supposed to be (expected true/false, int, int array and so on)
• do critical checks much sooner
• catch all errors into variable and attach them to email (so if things were skipped it shouldn't exit the script but should send them to email, teams, or whatever else is defined as notifications)
• add control on how user wants to be notified of those errors
• detect if it's run as task scheduler or by hand (to maybe give user a choice whether errors should be reported or not?)
• add check for version into email/notifications so that user is notified when there's a new version (usually new versions play with config file so if user does 'update-module' by accident) he may not know things went side ways in his Task Scheduler
• add more logic to handling errors and getting events from domain controllers. 1 DC being down shouldn't terminate whole script. It should just let in email that things aren't working for everything
• addressing more logic into PSEventViewer for reporting errors - actually PSEventViewer is the one that should get modifications because whole error logic and checking for events is there. It terminates silently if there are no events, but it actually most likely ignores some stuff. So we would need (and it's planned) modify PSEventViewer so that output is a bit more verbose and passed correctly to PSWinreporting
• PSEventViewer needs a rewrite kinda to support more direct questions for events.
• Set-ScriptMode according to many shouldn't be used in production, hence it's removed
• It's important to monitor events daily but as well to run the report weekly. that's why i added rescan of events for Forwarders, DCs and files at same time so that one can go back and verify what was missed. That's why I have SQL export since my Client asked so that all events are exported there and every now and then I rerun the tool and tool exports everything again just to see what was missed.

From other stuff that needs fixing (as I have you here):

• Group Policy report is actually a report of changes across multiple fields - this one report should be split into few other reports because those events report group policy changes but also user changes but on granullar level - I just didn't had time to see what is what
• I want to add scanning for any events you want (kind of custom scanning). This means I would like it to be able to point it into workstation, server, fileserver and tell it to get this type of events with this fields and so on.
• To achieve that most likely further rewrite of PSWinReporting would be needed so that functions like Get-ComputerStatus could actually be defined in $Configuration variable and you would just need a dynamically build report. It's not impossible, and I have some concept in my head for this. Most likely we would need to define something along the lines of:

            ComputerCreatedChanged = @{
                Enabled          = $true
                EnabledSqlGlobal = $false
                Events           = 4741, 4742 # created, changed
                LogName          = 'Security'
                IgnoreWords      = ''
               Properties = 'Action', 'Group Name' ... 
            }

and then all those public functions would have to be merged into one function that dynamically would build the report.

I guess one would need to create each of those points as a issue and go one by one fixing those. I'm sure some other stuff can be fixed, optimized. For example there's this PSHTML project that could be used to build dynamic reports for this. Sort of summary for when you just need a peak ;-)

You up for any of those ? :-D I appreciate any help I can get. I've got 7+ projects that are currently in 'active' development and only pair of hands ;p

[PrzemyslawKlys]

There's also a problem with PSEventViewer and Get-EventLogSizes that if DC or any machine that is being scanned is down it will wait for 40+ seconds on that one. I wrote this function: Test-ComputerAvailability that I wanted to start using as a step to verify the machine is up because 40 seconds per server really kills me.

[PrzemyslawKlys]

This also needs fixing. As in IsIncluded was supposed to let me know that DC was actually scanned. And it's just not there. IsPDC is already fixed by you. Not sure why I've never noticed it ;)

[snd3r]

Let's do it!

I think we need to enable the wiki and create a page with all these points.

And what about PR?
I'm ready to send new with Test-Configuration changes.

[PrzemyslawKlys]

Sure. Send it over. The one you sent looks good. I can accept this one and you can push more changes.

[snd3r]

I think this is a suitable issue.
On the way to the moment all the errors are catched and logged, I am really tired of copying Write-Color to the catch block from any other place. This is an amazing cmdlet, but I think we should simplify its use.
An example of how I decided to abstract it:
https://github.com/snd3r/PSSharedGoods/blob/master/Public/Logging/Get-Logger.ps1

In the wild it will be used like this:

    $Params = @{
        LogPath = Join-Path $LoggerParameters.LogsDir "$([datetime]::Now.ToString('yyyy.MM.dd_hh.mm'))_ADReporting.log"
        ShowTime = $LoggerParameters.ShowTime
        TimeFormat = $LoggerParameters.TimeFormat
    }
    $Logger = Get-Logger @Params

https://github.com/snd3r/PSWinReporting/blob/07944cb3a41ab905256c9e284129652991a56116/Public/Start-ADReporting.ps1#L26

    if (Test-Key $ReportOptions "ReportOptions" "KeepReports" ) {
        if (-not (Test-Path $ReportOptions.KeepReportsPath -PathType Container)) {
            $Success = $false
            $Logger.AddErrorRecord('Path in configuration of ReportOptions.KeepReportsPath doesn''t exist.')
        }
    }

https://github.com/snd3r/PSWinReporting/blob/07944cb3a41ab905256c9e284129652991a56116/Private/Test-Configuration.ps1#L65

Since the first implementation is intentionally simplified, the console output has lost several colors:

Bonus we get the highlight in text editors:

What do you think about this?

[PrzemyslawKlys]

Do I read this right? You want to stop using PSWritecolor and start using Get-Logger for everything? What is the benefit in the long run?

Don't get me wrong. I don't take things personally. If removing PSWriteColor and putting Get-Logger in it's place is going to improve things or simplify things so be it. I just want to understand. We can even improve PSWriteColor to expand it to the needs of this and other projects. Remember I control it, so not a big deal ;-)

[snd3r]

What is the benefit in the long run?

The main benefit is ease of use and quick changes.

What do I mean when I talk about ease of use?
I don’t want to remember the format of the output line we use, I don’t always want to select “error” color from time to time in manual mode, I don’t want to remember what symbol we encode error or warning.
I just want to write "AddErrorRecord", and everything else will work automatically.

What do I mean when I talk about quick changes?
If we want to change all "[-]" icons to some word, we must do it manually in all the code.
if we use the abstraction offered by me, we will need to change only one line of the method and the output will change everywhere.

Write-Color is a really cool cmdlet, it is worth expanding and making it more flexible, but all its power is revealed just wrapped in an additional layer of abstraction.
Easy to use, flexible to configure. I think so.

[PrzemyslawKlys]

Now I get it. I didn't notice you're using Write-Color beneath. I did the same for PSAutomator:

https://github.com/EvotecIT/PSAutomator

I went for a similar generalized approach. Take a look. And yes it's very ok to go ahead with this.

[PrzemyslawKlys]

@snd3r I am not sure how you test things but you should really get better testing env.

Also having strict checking means you can't have "problems" or non-existing variables in your config. So you either check for the existence of every variable in config or you remove strict checking.

According to best practices, it shouldn't be on production code.

[PrzemyslawKlys]

Closed by accident. After some changes it seems to work now. You really should build a bit broken Test Env. I have 1 forest with 2 domains and 4 domain controllers. I have one domain, 1 controller down. I do this to make sure I cover different scenarios. With your approach to not hide errors things would not work at all.

[snd3r]

My bad.
I really need a more complete environment for testing.

[PrzemyslawKlys]

Another drawback of using ErrorAction Stop globally

Without it:

With it:

It's like impossible to find out where it terminated.

[snd3r]

In the first approximation, your statement seems to be true, but if you look closely - no.
The first exception info, although it tells us where the error occurred, does not give us an understanding of why it occurred.

I think we should change the logic a bit, remove the extra validation inside the script (Test-KeyVerifyBoth() and Test-KeyVerify() funcs) and improve the input validation. You suggested this idea earlier, I suggest to return to it.

Main change:

PSWinReporting/Private/Test-Configuration.ps1

Line 183 in a540269

$Success = (Test-Key $ReportTimes "ReportTimes" "PastHour" -DisplayProgress -ValueType 'Boolean') -and $Success

PR: #33

[PrzemyslawKlys]

1. In the first error, I can see a place where the error happens so I can easily check and eventually fix. In a second I've no clue where to even start looking.

2. Verification is a good idea - but it needs to work in different scenarios. You seem to only test the Test main command for generating reports. There are 3 other commands that need this type of verification. Also one of the commands shouldn't display anything.

3. You would also need to update your logger to be a bit more flexible. It shouldn't crash or ask for input if it doesn't exist. It should take defaults (I've modified it but I'm pretty sure this could be done better). It should allow silent output if needed. For example, I've added a manual way to check for events and basic parameters are defined inside a module. And I think you should do the 'join-path' inside a logger function and just take params. Nicer input, less code (since you can just do splatting). Maybe allow a user to define how the file is named as well (probably some conversion would be needed).

4. Test-KeyVerifyBoth (because Test-KeyVerify is not used) - was added because it's not only about verification of input but also because I had problems where the output of if ($Key) would return $true while I expected to check if it was set to true or false directly for actual value from the user, and it was just telling me that key does exist.

[PrzemyslawKlys]

Your change will check for the existence of the key and will execute the script every time for each of those values.

That's why I created Test-KeyVerifyBoth. Because it checks for actual value of $ReportTimes.CurrentHour.Enabled = $true/$false

In other words, it also checks if $ReportTimes.CurrentHour has actual value of $true/$false and is boolean. In your case, it will always return $true at least from my testing.

The way you test it now, it will just check if $ReportTimes.CurrentHour exists. And it does, always. That's why my newly introduced change.

[PrzemyslawKlys]

Also why you remove -Merge ?

Add-ToArrayAdvanced -List $Events -Element $FoundEvents -SkipNull -Merge

It actually merges two arrays. if you don't do that it may fail.

[snd3r]

Also why you remove -Merge ?

My mistake. Accidentally broke upstream sync and kept an outdated copy.

[snd3r]

In other words, it also checks if $ReportTimes.CurrentHour has actual value of $true/$false and is boolean. In your case, it will always return $true at least from my testing.

I can`t reproduce this behavior:

[PrzemyslawKlys]

Test this on PastMonth, OnDay, PastQuarter.

[snd3r]

Test this on PastMonth, OnDay, PastQuarter.

To start checking you must add an argument with the expected type -ValueType 'Boolean':

Test-Key $ReportTimes.OnDay "ReportTimes.OnDay" "Enabled" -DisplayProgress -ValueType 'Boolean'

[PrzemyslawKlys]

Checking yes, but I mean even if the values are correct $ReportTimes.OnDay will return $True. Same as $ReportTimes.PastMonth will return $true and everything under this will execute. It's not about checking config. It's about execution of get-choosendates.

[snd3r]

In these cases, I check the property "Enabled":

    # Report Per Week
    if ($ReportTimes.OnDay.Enabled) {
        foreach ($Day in $ReportTimes.OnDay.Days) {
            $DatesReportOnDay = Find-DatesPastWeek $Day
            if ($DatesReportOnDay -ne $null) {
                $Dates += $DatesReportOnDay
            }
        }
    }
    # Report Per Month
    if ($ReportTimes.PastMonth.Enabled) {
        # Find-DatesMonthPast runs only on 1st of the month unless -Force is used
        $DatesMonthPrevious = Find-DatesMonthPast -Force $ReportTimes.PastMonth.Force
        if ($DatesMonthPrevious -ne $null) {
            $Dates += $DatesMonthPrevious
        }
    }

[PrzemyslawKlys]

Ok. It's a way to do this. But then I need logger and other stuff to address my comments from above? Or you see it differently?

[snd3r]

You would also need to update your logger to be a bit more flexible.

I am happy to help, but I do not understand what is missing in the current implementation. Maybe I need a case.

It shouldn't crash or ask for input if it doesn't exist. It should take defaults (I've modified it but I'm pretty sure this could be done better). It should allow silent output if needed.

I cannot reproduce the problem on the version of the function with your corrections. I do not think that I can do something much better than it is now, except that a small fix of copy/past mistakes: EvotecIT/PSSharedGoods#5

I think you should do the 'join-path' inside a logger function and just take params. Nicer input, less code (since you can just do splatting). Maybe allow a user to define how the file is named as well (probably some conversion would be needed).

I understand what you mean, but I do not understand why make these changes.
I do not see the complexity of this piece of code:

$Logger = Get-Logger -ShowTime -LogPath $(Join-Path 'C:\temp' 'Current.log')

[PrzemyslawKlys]

What I did is define $Script:Variable and I am checking for user input. If there is no user input I use that instead. That's what I did. My problem if you start any function outside of main starting functions (the ones from Private folder) it will crash (unless you already run Public one), as in error out as not defined (which is true). While it's not necessarily needed for users I think it would be nice to have a fail-safe in all places so that one can run a command without getting error messages if you just want to test some function outside of your standard scenarios? Hope it's clear now?

What I would like is:

• No logging option, currently I have this defined in PSWinReporting and it should be in PSsharedGoods (if the path to logs is not given do not write)

PSWinReporting/Public/Start-ADReporting.ps1

Lines 16 to 37 in 6645d3b

	[bool] $WarningNoLogger = $false
	#$ErrorActionPreference = 'Stop'
	#Set-StrictMode -Version Latest
	<#
	Set logger
	#>
	if (-not $LoggerParameters) {
	$LoggerParameters = $Script:LoggerParameters
	$WarningNoLogger = $true
	}
	$Params = @{
	LogPath = if ([string]::IsNullOrWhiteSpace($LoggerParameters.LogsDir)) { '' } else { Join-Path $LoggerParameters.LogsDir "$([datetime]::Now.ToString('yyyy.MM.dd_hh.mm'))_ADReporting.log" }
	ShowTime = $LoggerParameters.ShowTime
	TimeFormat = $LoggerParameters.TimeFormat
	}
	$Logger = Get-Logger @Params
	if ($WarningNoLogger) {
	$Logger.AddWarningRecord("New version of PSWinReporting requires Logger Parameter. Please read documentation. No logs will be written to disk.")
	}

If you move Join-Path to PSSharedGoods (and the whole check for whitespace on line 29) you can actually shorten this part to:

  [bool] $WarningNoLogger = $false
    if (-not $LoggerParameters) {
        $LoggerParameters = $Script:LoggerParameters
        $WarningNoLogger = $true
    } 
  $Logger = Get-Logger @LoggerParameters

    if ($WarningNoLogger) {
        $Logger.AddWarningRecord("New version of PSWinReporting requires Logger Parameter. Please read documentation. No logs will be written to disk.")
    }

I was also thinking the user should be able to define the file naming convention? They may want to move logs/trim in a different way?

$([datetime]::Now.ToString('yyyy.MM.dd_hh.mm'))_ADReporting.log" }

[snd3r]

Got it.
EvotecIT/PSSharedGoods#6

[PrzemyslawKlys]

Looks good.

[PrzemyslawKlys]

@snd3r I know now why I've implemented my helper function. Because I changed things a bit:

PSWinReporting/Private/Parameters/Script.ReportTimes.ps1

Lines 1 to 65 in 2213b99

	$Script:ReportTimes = @{
	# Report Per Hour
	PastHour = @{
	Enabled = $false # if it's 23:22 it will report 22:00 till 23:00
	}
	CurrentHour = @{
	Enabled = $false # if it's 23:22 it will report 23:00 till 00:00
	}
	# Report Per Day
	PastDay = @{
	Enabled = $false # if it's 1.04.2018 it will report 31.03.2018 00:00:00 till 01.04.2018 00:00:00
	}
	CurrentDay = @{
	Enabled = $false # if it's 1.04.2018 05:22 it will report 1.04.2018 00:00:00 till 01.04.2018 00:00:00
	}
	# Report Per Week
	OnDay = @{
	Enabled = $false
	Days = 'Monday'#, 'Tuesday'
	}
	# Report Per Month
	PastMonth = @{
	Enabled = $false # checks for 1st day of the month - won't run on any other day unless used force
	Force = $true # if true - runs always ...
	}
	CurrentMonth = @{
	Enabled = $false
	}
	# Report Per Quarter
	PastQuarter = @{
	Enabled = $false # checks for 1st day fo the quarter - won't run on any other day
	Force = $true
	}
	CurrentQuarter = @{
	Enabled = $false
	}
	# Report Custom
	CurrentDayMinusDayX = @{
	Enabled = $false
	Days = 7 # goes back X days and shows just 1 day
	}
	CurrentDayMinuxDaysX = @{
	Enabled = $false
	Days = 3 # goes back X days and shows X number of days till Today
	}
	CustomDate = @{
	Enabled = $false
	DateFrom = get-date -Year 2018 -Month 03 -Day 19
	DateTo = get-date -Year 2018 -Month 03 -Day 23
	}
	Last3days = @{
	Enabled = $false
	}
	Last7days = @{
	Enabled = $false
	}
	Last14days = @{
	Enabled = $false
	}
	Everything = @{
	Enabled = $false
	}
	}

And with your removal of my support function:

[Info] Getting events for dates 12/09/2018 19:00:00 to 12/09/2018 20:00:00
[Info] Events found 0
[Info] Getting events for dates 12/09/2018 20:00:00 to 12/09/2018 21:00:00
[Info] Events found 0
[Info] Getting events for dates 12/08/2018 00:00:00 to 12/09/2018 00:00:00
[Info] Events found 0
[Info] Getting events for dates 12/09/2018 00:00:00 to 12/09/2018 23:59:59
[Info] Events found 0
[Info] Getting events for dates 12/01/2018 00:00:00 to 12/31/2018 23:59:59
[Info] Events found 14
[Info] Getting events for dates 10/01/2018 00:00:00 to 12/31/2018 23:59:59
[Info] Events found 14
[Info] Getting events for dates 01/01/1600 20:25:01 to 01/01/2300 20:25:01
[Info] Events found 14
[Info] Running Group Policy Changes Report
[Info] Ending Group Policy Changes Report

And that way it supported both scenarios without me having to play with it. I'm currently rebuilding PSWinReporting to support a dynamic building of reports, but I would like to have the old and new way of working.

Do you think I can bring back my function or you have other views on this?

[snd3r]

The expansion of the number of supported cases leads to an increase in the amount of code and the time of its support.
I think it is worth choosing the second path whereby specifying the "Enabled" property will be required.