gcollapse: Support for rawsum / selective weights in general about stata-gtools

[mcaceresb]

Nice; unfortunately for me the way I wrote gcollapse's internals requires a different set of functions for each of these things. Mainly copy/paste with small tweaks, but still.

[mcaceresb]

Mmm... Yes, I will implement rawsum. At this point, it bothers me that I have to keep noting "rawsum" is not implemented. It will only take a few minutes, but testing might take a bit longer.

I agree it's a good idea to add "rawstat" so you can pass a list of targets, for which weights will be ignored. I won't make it too smart, so it will be slightly slower than regular gcollapse, but it should certainly be much faster than doing it two gcollapses.

It won't take very long to implement either, but I'm rather busy this week. Perhaps after the weekend or during the next. Cheers!

[Danferno]

Great!

[sergiocorreia]

I think having a general rawstat is a great idea, but I'm wondering about a corner case:

sysuse auto
collapse (sum) turn (rawsum) rawturn=turn [fw=foreign]
list

You'll notice that sum and rawsum give the same result, because rawsum also ignores cases where foreign==0 (i.e. zero weights).

It's something you won't expect unless you know the details about the implementation (that on an early step we exclude obs. where the weights are zero), so it's a bit troublesome (but also a problem of collapse in general)

[mcaceresb]

I see what you mean, but the only plausible workaround would involve very annoying additions to the internals. Perhaps a warning noting that observations with weights that are 0 or missing are excluded even for raw stats? "Obs with 0 or missing weights dropped for all stats (including raw)"? Something like that.

I mean, this is also a bit of a conceptual issue, right? Are 0 and missing weights "0" or are they indicating the observation should be dropped? If the latter, then the raw stat is the raw sum among observations with valid weights/that are not dropped/etc. If the former, then certainly it would be confusing.

To be honest I tend to think about it in terms of the former, but given how annoying it would be to bypass this restriction, I'll conveniently re-think my interpretation of weights.

[sergiocorreia]

Agree with you. Did a quick commit to ftools that just does it the easy way. I think just adding a warning on the help file will suffice.