cdhunt / pselect

PowerShell DSL for aggregating data

PSelect

A SQL-ish DSL in PowerShell to assist in aggregating collections of data.

Example - FromPipeline

Get-Process | PSelect {
    Field ProcessName
    Field WorkingSet -as Count -Count
    Field WorkingSet -Sum
    
    GroupBy ProcessName

    SortData
}

ProcessName                           Count WorkingSet
-----------                           ----- ----------
ApplicationFrameHost                      1   27492352
AppVShNotify                              1    7405568
armsvc                                    1    5464064
audiodg                                   1   18952192
BleServicesCtrl                           1    9011200
bmservice                                 1   46051328
chrome                                   25 1875648512
Code                                      7  357859328
CodeHelper                                1   13967360
conhost                                   7   38752256
csrss                                     2   16859136
dasHost                                   1   10018816
DataExchangeHost                          1   12623872
dllhost                                   3   26742784
...

Example - FromCsv

PSelect {
    Field category
    Field raisedAmt -as AvgRaisedAmt -Average -Unit Currency
    Field raisedAmt -as TotalRaisedAmt -Sum -Unit Currency
    Field raisedAmt -as MaxRaisedAmt -Maximum -Format "{0:N}"
    Field raisedAmt -as MinRaisedAmt -Min -Format "{0,13:C}"
    Field raisedAmt -as SDRaisedAmd -StdDev -Format "{0:f}"
    Field raisedAmt -as Rounds -Count
    
    GroupBy category
    
    FromCsv TechCrunchcontinentalUSA.csv

    SortData

} | Format-Table -AutoSize

category   AvgRaisedAmt   TotalRaisedAmt     MaxRaisedAmt   MinRaisedAmt  SDRaisedAmd Rounds
--------   ------------   --------------     ------------   ------------  ----------- ------
           $15,554,166.67 $373,300,000.00    150,000,000.00   $900,000.00 29888793.60     24
biotech    $19,312,500.00 $77,250,000.00     37,000,000.00    $250,000.00 13920954.30      4
cleantech  $18,492,857.14 $258,900,000.00    57,000,000.00  $1,000,000.00 14340525.21     14
consulting $6,427,000.00  $32,135,000.00     13,000,000.00     $10,000.00 5576451.92       5
hardware   $21,141,025.64 $824,500,000.00    130,000,000.00   $100,000.00 25244059.65     39
mobile     $6,729,583.33  $323,020,000.00    25,000,000.00     $20,000.00 5752943.80      48
other      $7,490,625.00  $119,850,000.00    29,000,000.00    $300,000.00 6911092.05      16
software   $9,979,823.53  $1,017,942,000.00  60,000,000.00     $10,000.00 10046611.33    102
web        $9,739,300.29  $11,765,074,750.00 300,000,000.00     $6,000.00 19034496.45   1208

pselect's Issues

Example FromPipeline - error on PowerShell 6 (Core)

Field : Cannot validate argument on parameter 'Unit'. The argument "Field" does not belong to the set "Currency;Percentage;Seconds;Minutes;Hours;Days;Weeks;Months;Years" specified by the ValidateSet attribute. Supply an argument that is in the set and then try the command again.
At line:1 char:51
+ Get-Process | PSelect {     Field ProcessName     Field WorkingSet -a ...
+                                                   ~~~~~
+ CategoryInfo          : InvalidData: (:) [Field], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationError,Field

GroupBy is required.
At F:\cygwin\home\thorsten\.config\powershell\Modules\PSelect\0.2.2\PSelect.psm1:45 char:13
+             Throw "GroupBy is required."
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : OperationStopped: (GroupBy is required.:String) [], RuntimeException
+ FullyQualifiedErrorId : GroupBy is required.

The same code works fine on Windows PowerShell 5.1

Clarification: difference and similarity to Group-Object

Can you give some information regarding the difference and similarity between PSelect and PowerShell's Group-Object?

Problem transforming Group-Object to PSelect

I've got some example code. The code reads a file with about 300,000 words - one per line - and groups the words by length. The grouped output is sorted by count:

$file    = 'F:\cygwin\home\thorsten\python\Wordlist.txt'
$content = Get-Content -Path $file -ReadCount 0

$content | Group-Object -Property Length | Sort-Object -Property Count

This is the output:

Count Name                      Group
----- ----                      -----
    5 24                        {formaldehydesulphoxylate, pathologicopsychological, scientificophilosophical, tetraiodophenolphthalein...}
   17 23                        {anthropomorphologically, blepharosphincterectomy, epididymodeferentectomy, formaldehydesulphoxylic...}
   31 1                         {*, 0, 1, 2...}
[...]
43555 8                         {!@#$%^&*, 0racl38i, 0racl39i, 0racle10...}
46919 10                        {abalienate, abaptiston, abasedness, abbagliamo...}
48228 9                         {0racle10i, 123456789, 199220706, <invalid>...}

The PSelect code I came up with looks like this

$content | PSelect {
               Field -Name Length -As Count -Count
               Field -Name Length

               GroupBy -Name Length

               SortData
           } 

The output looks like this:

Count Length
----- ------
   31 1
46919 10
26902 11
[...]
34384 7
43555 8
48228 9

Now my questions and observations:

1. the Group-Object code takes about 10 seconds to run. PSelect takes 4:17min to run. That's about 25 times longer.
2. Is it possible to sort the output on the Count column and not the Length column?
3. The Length column is obviously sorted alphabetically and not numerically. This looks like a bug.
4. Is it possible to add the "raw data" (the Group column from Group-Object)?