Alternative participation approach: Assign randomly rather than rank about c-tam

[MaxGhenis]

@Amy-Xu

[Amy-Xu]

This is a valid concern. Dan has also brought up this issue before. Records with positive predictors are a lot more likely to be imputed, while not all of the predictors are available in the CPS dataset. With regard to income, lower income people are much more likely to be imputed than higher income people and on top of that, maybe they are imputed at a higher rate than they 'should be'. Dan has asked Robert Moffitt whether the participation should be imputed by income range according to the estimated probability. His view is, in short, depending on what you want to do.

So I have two questions. First, do you have a 'correct' correlation in mind for income and participation? Or the goal is simply to reserve the fuzziness? Second, is the correlation particularly important your project?

Anyways, I do think it's worthwhile to add on more options for imputing procedure. Your solution is sensible and easy to implement. The caveat, from my perspective, is how to factor in the records weight. CPS weights are population/family/household counts. If we simply implement the weighted sum in the argument p, the weight of each record could confound the actual probability.

[MaxGhenis]

First, do you have a 'correct' correlation in mind for income and participation? Or the goal is simply to reserve the fuzziness? Second, is the correlation particularly important your project?

A simple example could be modeling the after-tax income distributional impact of repealing SNAP. The current data overstates SNAP participation among low-income HHs, and understates it among higher-income HHs. So SNAP repeal would look more regressive than it actually is. Other reforms like shifting the SNAP budget to UBI would also look overly regressive.

The caveat, from my perspective, is how to factor in the records weight.

Yes that's trickier, and I think would require a looping budget logic rather than numpy.random.choice. I asked for ideas in this SO question. I think it requires adding households one at a time according to the probabilities, until you can't add any more.

[MaxGhenis]

Here's the code from my answer to the SO question:

def weighted_budgeted_random_sample(df, budget):
    """ Produce a weighted budgeted random sample.

    Args:
        df: DataFrame with columns for `prob` and `weight`.
        budget: Total weight budget.

    Returns:
        List of index values of df that constitute the sample.

    """
    ids = []
    total = 0
    while total < budget:
        remaining = budget - total
        df = df[df.weight <= remaining]
        # Stop if there are no records with small enough weight.
        if df.shape[0] == 0:
            break
        # Select one record.
        selection = random.choice(df.index, p=(df.prob / df.prob.sum()))
        total += df.loc[selection].weight
        df.drop(selection, inplace=True)
        ids.append(selection)
    return ids

So it'd be called as weighted_budgeted_random_sample(df, budget=total_snap_participation) where df had prob as predicted by the regression model, and weight=s006.