Need reference for PCMAX (default setting for PC in CL apparently) about causal-learn

[jdramsey]

Sorry, I should say, if you can find a published reference for it, use that. Or at least describe the procedure in the documentation. I think.

[kunwuz]

Sure, thanks so much for your suggestions. I've added your paper as the reference for maxP and definiteMaxP (uc_rule). Please let me know if anything is wrong.

BTW, I'm not totally sure that our default setting is PCMAX, since the default value for 'uc_rule' is 0, as follows:

def pc( data: ndarray, alpha=0.05, indep_test=fisherz, stable: bool = True, uc_rule: int = 0, uc_priority: int = 2, mvpc: bool = False, correction_name: str = 'MV_Crtn_Fisher_Z', background_knowledge: BackgroundKnowledge | None = None, verbose: bool = False, show_progress: bool = True, node_names: List[str] | None = None, ):

where

uc_rule: how unshielded colliders are oriented. Default: 0.
0: run uc_sepset.
1: run maxP. Orient an unshielded triple X-Y-Z as a collider with an additional CI test.
2: run definiteMaxP. Orient only the definite colliders in the skeleton and keep track of all the definite non-colliders as well.

It's very likely that I misunderstood something, as I'm not familiar with PCMAX. It might be the case that we are using PCMAX somewhere else?

[jdramsey]

Here is the performance profile I get--before I grabbed the latest repository CL I was getting near identical performance between default PC_CL ("causality Lab") and Tetrad PCMAX_CC ("causal-cmd" = Tetrad); now it goes back and forth a little (though that could just be statistical variation do due a difference of the order in which operations are done). I include some other CC algorithms as well in the comparison. Maybe I don't understand what "UC" is--let me look it up.

`Algorithms:

1. PC_CL
2. PC_CC
3. PCMAX_CC
4. FGES_CC
5. GRaSP_CC

Weighting of Statistics:

Write weighting here

AVERAGE STATISTICS:

All edges

Alg   AP      AR     AHP   AHPCE     AHR   AHRCE   McAdj   McArr   F1Adj   F1Arr     SHD    E
1	0.97	0.69	0.81	0.84	0.51	0.76	0.78	0.62	0.80	0.63	16.10	
2	0.96	0.69	0.44	0.46	0.31	0.45	0.78	0.31	0.80	0.36	25.10	
3	0.96	0.69	0.84	0.88	0.56	0.82	0.78	0.66	0.80	0.67	15.70	
4	0.84	0.84	0.69	0.81	0.68	0.81	0.79	0.65	0.83	0.68	18.80	
4	0.99	0.87	0.99	0.99	0.84	0.96	0.91	0.90	0.93	0.91	5.40`

[jdramsey]

Sorry, this is for a problem with 20 nodes, 40 edges, 1000 samples, averaged over 10 runs.

p = 20
a = 4
d = a / (p - 1)
n = 1000
runnum = 10

[jdramsey]

Sorry, linear, Gaussian, using Fisher Z for PC_CL and PCMAX_CC--but perhaps the tests are behaving differently in the two platforms--I'll look into it...

[jdramsey]

Anyway, I am very curious about this. If you are getting PCMAX performance from the PC sepsets step alone (without maxp) that is something I don't know how to do. Wayne Lam did a study of this for his MA thesis in Python and didn't see how to do it either, with several variants of the procedure. So I need to figure out exactly what the CL code is doing. I'll forward future comments offline and close this.

[kunwuz]

Hi Joe, thanks for this great discovery! I'm also very curious about that. I've asked several contributors of PC-CL and we are not aware of the reason yet, as we thought only by selecting 'maxP' or 'definiteMaxP' as the 'uc_rule' (how to orient unshielded colliders) can we perform strategies based on the maximization of the P-value.

We are looking into the detailed code to see the reason and will update you ASAP.

[jdramsey]

Yeah I get the same performance whether I set ucrule to 0 or 1. Setting it to 2 gives a better result, close to GRASP_CC but with lower recalls.

Alg AP AR AHP AHPCE AHR AHRCE McAdj McArr F1Adj F1Arr SHD E 1 1.00 0.71 0.95 0.95 0.52 0.73 0.81 0.68 0.83 0.67 12.40