adematti / cosmopipe Goto Github PK

View Code? Open in Web Editor NEW

4.0 4.0 3.0 13.6 MB

License: Other

Python 49.19% Dockerfile 0.16% Jupyter Notebook 50.64%

cosmopipe's People

Contributors

Stargazers

Watchers

Forkers

lhior gqcristhian echaussidon

cosmopipe's Issues

figure out how to iterate gaussian_covariance

Gaussian covariance assumes a model, which might not be very correct. After fitting you have a model that hopefully describes the observed power well, so you'd like to be able to automatically recalculate the covariance and redo the fit. Just need to figure out how that should look as a script.

print dof with chi^2

power spectrum estimation uses too much memory?

384^3 box peak memory usage >23Gb (one double array this size is 0.45 Gb)
Doesn't seem likely necessary...

wrong alpha calculation?

Shouldn't alpha weighting include FKP weight? I.e., we're trying to make weighted randoms represent mean of weighted field as well as possible and, to the extent that it matters, FKP weight is part of that...

cosmopipe/cosmopipe/estimators/utils.py

Line 61 in a8d2e3c

alpha = data.sum('weight_comp')/randoms.sum('weight_comp')

demo_mock_covariance.yaml singular matrix

pypescript demo_mock_covariance.yaml
eventually hits a “singular matrix” for me, as follows:

version: 0.0.2 date: 2021-07-11

[000000.00] [0/1] 07-11 20:18 BasePipeline INFO Init module BasePipeline [main].
[000000.03] [0/1] 07-11 20:18 BaseModule INFO Importing module cosmopipe.parameters.parameterisation [params].
[000000.95] [0/1] 07-11 20:18 BaseModule INFO Found description file /Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/parameters/parameterisation.yaml.
[000000.95] [0/1] 07-11 20:18 BaseModule INFO No ['setup', 'execute', 'cleanup'] functions found in module [params], trying to load class Parameterisation.
[000000.95] [0/1] 07-11 20:18 BaseModule INFO Init module Parameterisation [params].
[000000.95] [0/1] 07-11 20:18 BaseModule INFO Importing module cosmopipe.theory.primordial.fiducial [fiducial].
[000001.40] [0/1] 07-11 20:18 BaseModule INFO Found description file /Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/theory/primordial/fiducial.yaml.
[000001.40] [0/1] 07-11 20:18 BaseModule INFO No ['setup', 'execute', 'cleanup'] functions found in module [fiducial], trying to load class Fiducial.
[000001.40] [0/1] 07-11 20:18 BaseModule INFO Init module Fiducial [fiducial].
[000001.40] [0/1] 07-11 20:18 BaseModule INFO Importing module pypescript [mpi].
[000001.40] [0/1] 07-11 20:18 BaseModule INFO No description file provided at /Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/pypescript/pypescript.yaml.
[000001.41] [0/1] 07-11 20:18 MPIPipeline INFO Init module MPIPipeline [mpi].
[000001.41] [0/1] 07-11 20:18 BaseModule INFO Importing module cosmopipe.estimators.power_spectrum.nbodykit_survey [survey_estimator].
[000003.08] [0/1] 07-11 20:18 BaseModule INFO Found description file /Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/estimators/power_spectrum/nbodykit_survey.yaml.
[000003.08] [0/1] 07-11 20:18 BaseModule INFO No ['setup', 'execute', 'cleanup'] functions found in module [survey_estimator], trying to load class SurveyPowerSpectrum.
[000003.08] [0/1] 07-11 20:18 SurveyPowerSpectrum INFO Init module SurveyPowerSpectrum [survey_estimator].
[000003.08] [0/1] 07-11 20:18 BaseModule INFO Importing module cosmopipe.data_vector.mock_covariance_matrix [cov].
[000003.11] [0/1] 07-11 20:18 BaseModule INFO Found description file /Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/data_vector/mock_covariance_matrix.yaml.
[000003.11] [0/1] 07-11 20:18 BaseModule INFO No ['setup', 'execute', 'cleanup'] functions found in module [cov], trying to load class MockCovarianceMatrix.
[000003.12] [0/1] 07-11 20:18 BaseModule INFO Init module MockCovarianceMatrix [cov].
[000003.13] [0/1] 07-11 20:18 BaseTaskManager INFO Entering BaseTaskManager.
[000003.13] [0/1] 07-11 20:18 Catalog INFO Loading _catalog/lognormal_data_1.fits.
[000003.38] [0/1] 07-11 20:18 Catalog INFO Loading _catalog/lognormal_randoms.fits.
[000005.11] [0/1] 07-11 20:18 FKPCatalog INFO cartesian coordinate range: [0.00125445 0.00054827 0.0002208 ] : [1387.20757471 1387.33654312 1386.97592007]
[000005.11] [0/1] 07-11 20:18 FKPCatalog INFO BoxSize = 4000.0
[000005.11] [0/1] 07-11 20:18 FKPCatalog INFO BoxCenter = [693.60441458 693.6685457 693.48807044]
[000005.14] [0/1] 07-11 20:18 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'first'
[000005.14] [0/1] 07-11 20:18 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'second'
[000011.72] [0/1] 07-11 20:19 CatalogMesh INFO Chunk 0 ~ 4194304 / 787849
[000012.41] [0/1] 07-11 20:19 CatalogMesh INFO painted 787849 out of 787849 objects to mesh
[000012.73] [0/1] 07-11 20:19 CatalogMesh INFO painted 787849 out of 787849 objects to mesh
[000012.73] [0/1] 07-11 20:19 CatalogMesh INFO mean particles per cell is 0.375676
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000012.73] [0/1] 07-11 20:19 CatalogMesh INFO sum is 787849
[000012.74] [0/1] 07-11 20:19 FKPCatalogMesh INFO data painted.
[000012.76] [0/1] 07-11 20:19 CatalogMesh INFO Chunk 0 ~ 4194304 / 1576395
[000014.35] [0/1] 07-11 20:19 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000014.65] [0/1] 07-11 20:19 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000014.65] [0/1] 07-11 20:19 CatalogMesh INFO mean particles per cell is 0.751684
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000014.65] [0/1] 07-11 20:19 CatalogMesh INFO sum is 1.5764e+06
[000014.66] [0/1] 07-11 20:19 FKPCatalogMesh INFO randoms painted.
[000014.68] [0/1] 07-11 20:19 FKPCatalogMesh INFO volume per cell is 30517.6
[000014.68] [0/1] 07-11 20:19 FKPCatalogMesh INFO field: (FKPCatalog(species=['data', 'randoms']) as CatalogMesh) painting done
[000014.68] [0/1] 07-11 20:19 ConvolvedFFTPower INFO tsc painting of 'first' done
[000014.80] [0/1] 07-11 20:19 ConvolvedFFTPower INFO ell = 0 done; 1 r2c completed
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:531: ComplexWarning: Casting complex values to real discards the imaginary part
xgrid = [xx.astype('f8') + offset[ii] for ii, xx in enumerate(density2.slabs.optx)]
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:536: ComplexWarning: Casting complex values to real discards the imaginary part
kgrid = [kk.astype('f8') for kk in cfield.slabs.optx]
[000023.35] [0/1] 07-11 20:19 ConvolvedFFTPower INFO normalized power spectrum with `randoms.norm = 236.354700`
[000023.94] [0/1] 07-11 20:19 ConvolvedFFTPower INFO ell = 2 done; 5 r2c completed
[000025.43] [0/1] 07-11 20:19 ConvolvedFFTPower INFO ell = 4 done; 9 r2c completed
[000025.84] [0/1] 07-11 20:19 ConvolvedFFTPower INFO higher order multipoles computed in elapsed time 00:00:02.49
[000035.32] [0/1] 07-11 20:19 DataVector INFO Saving to _data/lognormal_power_spectrum_1.txt.
[000035.36] [0/1] 07-11 20:19 Catalog INFO Loading _catalog/lognormal_data_2.fits.
[000035.81] [0/1] 07-11 20:19 Catalog INFO Loading _catalog/lognormal_randoms.fits.
[000037.52] [0/1] 07-11 20:19 FKPCatalog INFO cartesian coordinate range: [0.00125445 0.00054827 0.0002208 ] : [1387.20757471 1387.33654312 1386.97592007]
[000037.52] [0/1] 07-11 20:19 FKPCatalog INFO BoxSize = 4000.0
[000037.52] [0/1] 07-11 20:19 FKPCatalog INFO BoxCenter = [693.60441458 693.6685457 693.48807044]
[000037.53] [0/1] 07-11 20:19 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'first'
[000037.53] [0/1] 07-11 20:19 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'second'
[000042.10] [0/1] 07-11 20:19 CatalogMesh INFO Chunk 0 ~ 4194304 / 789379
[000042.78] [0/1] 07-11 20:19 CatalogMesh INFO painted 789379 out of 789379 objects to mesh
[000043.08] [0/1] 07-11 20:19 CatalogMesh INFO painted 789379 out of 789379 objects to mesh
[000043.08] [0/1] 07-11 20:19 CatalogMesh INFO mean particles per cell is 0.376405
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000043.08] [0/1] 07-11 20:19 CatalogMesh INFO sum is 789379
[000043.09] [0/1] 07-11 20:19 FKPCatalogMesh INFO data painted.
[000043.11] [0/1] 07-11 20:19 CatalogMesh INFO Chunk 0 ~ 4194304 / 1576395
[000044.63] [0/1] 07-11 20:19 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000044.92] [0/1] 07-11 20:19 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000044.92] [0/1] 07-11 20:19 CatalogMesh INFO mean particles per cell is 0.751684
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000044.92] [0/1] 07-11 20:19 CatalogMesh INFO sum is 1.5764e+06
[000044.94] [0/1] 07-11 20:19 FKPCatalogMesh INFO randoms painted.
[000044.96] [0/1] 07-11 20:19 FKPCatalogMesh INFO volume per cell is 30517.6
[000044.96] [0/1] 07-11 20:19 FKPCatalogMesh INFO field: (FKPCatalog(species=['data', 'randoms']) as CatalogMesh) painting done
[000044.96] [0/1] 07-11 20:19 ConvolvedFFTPower INFO tsc painting of 'first' done
[000045.09] [0/1] 07-11 20:19 ConvolvedFFTPower INFO ell = 0 done; 1 r2c completed
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:531: ComplexWarning: Casting complex values to real discards the imaginary part
xgrid = [xx.astype('f8') + offset[ii] for ii, xx in enumerate(density2.slabs.optx)]
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:536: ComplexWarning: Casting complex values to real discards the imaginary part
kgrid = [kk.astype('f8') for kk in cfield.slabs.optx]
[000053.68] [0/1] 07-11 20:19 ConvolvedFFTPower INFO normalized power spectrum with `randoms.norm = 236.813700`
[000054.28] [0/1] 07-11 20:19 ConvolvedFFTPower INFO ell = 2 done; 5 r2c completed
[000055.73] [0/1] 07-11 20:19 ConvolvedFFTPower INFO ell = 4 done; 9 r2c completed
[000056.13] [0/1] 07-11 20:19 ConvolvedFFTPower INFO higher order multipoles computed in elapsed time 00:00:02.45
[000065.44] [0/1] 07-11 20:19 DataVector INFO Saving to _data/lognormal_power_spectrum_2.txt.
[000065.48] [0/1] 07-11 20:19 Catalog INFO Loading _catalog/lognormal_data_3.fits.
[000065.72] [0/1] 07-11 20:19 Catalog INFO Loading _catalog/lognormal_randoms.fits.
[000067.27] [0/1] 07-11 20:20 FKPCatalog INFO cartesian coordinate range: [0.00125445 0.00054827 0.0002208 ] : [1387.20757471 1387.33654312 1386.97592007]
[000067.28] [0/1] 07-11 20:20 FKPCatalog INFO BoxSize = 4000.0
[000067.28] [0/1] 07-11 20:20 FKPCatalog INFO BoxCenter = [693.60441458 693.6685457 693.48807044]
[000067.29] [0/1] 07-11 20:20 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'first'
[000067.29] [0/1] 07-11 20:20 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'second'
[000071.59] [0/1] 07-11 20:20 CatalogMesh INFO Chunk 0 ~ 4194304 / 788609
[000072.24] [0/1] 07-11 20:20 CatalogMesh INFO painted 788609 out of 788609 objects to mesh
[000072.54] [0/1] 07-11 20:20 CatalogMesh INFO painted 788609 out of 788609 objects to mesh
[000072.54] [0/1] 07-11 20:20 CatalogMesh INFO mean particles per cell is 0.376038
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000072.54] [0/1] 07-11 20:20 CatalogMesh INFO sum is 788609
[000072.56] [0/1] 07-11 20:20 FKPCatalogMesh INFO data painted.
[000072.58] [0/1] 07-11 20:20 CatalogMesh INFO Chunk 0 ~ 4194304 / 1576395
[000074.10] [0/1] 07-11 20:20 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000074.39] [0/1] 07-11 20:20 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000074.39] [0/1] 07-11 20:20 CatalogMesh INFO mean particles per cell is 0.751684
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000074.39] [0/1] 07-11 20:20 CatalogMesh INFO sum is 1.5764e+06
[000074.41] [0/1] 07-11 20:20 FKPCatalogMesh INFO randoms painted.
[000074.43] [0/1] 07-11 20:20 FKPCatalogMesh INFO volume per cell is 30517.6
[000074.43] [0/1] 07-11 20:20 FKPCatalogMesh INFO field: (FKPCatalog(species=['data', 'randoms']) as CatalogMesh) painting done
[000074.43] [0/1] 07-11 20:20 ConvolvedFFTPower INFO tsc painting of 'first' done
[000074.55] [0/1] 07-11 20:20 ConvolvedFFTPower INFO ell = 0 done; 1 r2c completed
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:531: ComplexWarning: Casting complex values to real discards the imaginary part
xgrid = [xx.astype('f8') + offset[ii] for ii, xx in enumerate(density2.slabs.optx)]
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:536: ComplexWarning: Casting complex values to real discards the imaginary part
kgrid = [kk.astype('f8') for kk in cfield.slabs.optx]
[000083.29] [0/1] 07-11 20:20 ConvolvedFFTPower INFO normalized power spectrum with `randoms.norm = 236.582700`
[000083.93] [0/1] 07-11 20:20 ConvolvedFFTPower INFO ell = 2 done; 5 r2c completed
[000085.47] [0/1] 07-11 20:20 ConvolvedFFTPower INFO ell = 4 done; 9 r2c completed
[000085.89] [0/1] 07-11 20:20 ConvolvedFFTPower INFO higher order multipoles computed in elapsed time 00:00:02.59
[000095.16] [0/1] 07-11 20:20 DataVector INFO Saving to _data/lognormal_power_spectrum_3.txt.
[000095.20] [0/1] 07-11 20:20 Catalog INFO Loading _catalog/lognormal_data_4.fits.
[000095.60] [0/1] 07-11 20:20 Catalog INFO Loading _catalog/lognormal_randoms.fits.
[000097.99] [0/1] 07-11 20:20 FKPCatalog INFO cartesian coordinate range: [0.00125445 0.00054827 0.0002208 ] : [1387.20757471 1387.33654312 1386.97592007]
[000097.99] [0/1] 07-11 20:20 FKPCatalog INFO BoxSize = 4000.0
[000097.99] [0/1] 07-11 20:20 FKPCatalog INFO BoxCenter = [693.60441458 693.6685457 693.48807044]
[000098.02] [0/1] 07-11 20:20 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'first'
[000098.02] [0/1] 07-11 20:20 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'second'
[000104.45] [0/1] 07-11 20:20 CatalogMesh INFO Chunk 0 ~ 4194304 / 786535
[000105.22] [0/1] 07-11 20:20 CatalogMesh INFO painted 786535 out of 786535 objects to mesh
[000105.56] [0/1] 07-11 20:20 CatalogMesh INFO painted 786535 out of 786535 objects to mesh
[000105.56] [0/1] 07-11 20:20 CatalogMesh INFO mean particles per cell is 0.375049
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000105.56] [0/1] 07-11 20:20 CatalogMesh INFO sum is 786535
[000105.57] [0/1] 07-11 20:20 FKPCatalogMesh INFO data painted.
[000105.59] [0/1] 07-11 20:20 CatalogMesh INFO Chunk 0 ~ 4194304 / 1576395
[000107.20] [0/1] 07-11 20:20 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000107.50] [0/1] 07-11 20:20 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000107.50] [0/1] 07-11 20:20 CatalogMesh INFO mean particles per cell is 0.751684
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000107.50] [0/1] 07-11 20:20 CatalogMesh INFO sum is 1.5764e+06
[000107.52] [0/1] 07-11 20:20 FKPCatalogMesh INFO randoms painted.
[000107.53] [0/1] 07-11 20:20 FKPCatalogMesh INFO volume per cell is 30517.6
[000107.54] [0/1] 07-11 20:20 FKPCatalogMesh INFO field: (FKPCatalog(species=['data', 'randoms']) as CatalogMesh) painting done
[000107.54] [0/1] 07-11 20:20 ConvolvedFFTPower INFO tsc painting of 'first' done
[000107.66] [0/1] 07-11 20:20 ConvolvedFFTPower INFO ell = 0 done; 1 r2c completed
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:531: ComplexWarning: Casting complex values to real discards the imaginary part
xgrid = [xx.astype('f8') + offset[ii] for ii, xx in enumerate(density2.slabs.optx)]
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:536: ComplexWarning: Casting complex values to real discards the imaginary part
kgrid = [kk.astype('f8') for kk in cfield.slabs.optx]
[000116.19] [0/1] 07-11 20:20 ConvolvedFFTPower INFO normalized power spectrum with `randoms.norm = 235.960500`
[000116.78] [0/1] 07-11 20:20 ConvolvedFFTPower INFO ell = 2 done; 5 r2c completed
[000118.20] [0/1] 07-11 20:20 ConvolvedFFTPower INFO ell = 4 done; 9 r2c completed
[000118.61] [0/1] 07-11 20:20 ConvolvedFFTPower INFO higher order multipoles computed in elapsed time 00:00:02.42
[000128.04] [0/1] 07-11 20:21 DataVector INFO Saving to _data/lognormal_power_spectrum_4.txt.
[000128.09] [0/1] 07-11 20:21 Catalog INFO Loading _catalog/lognormal_data_5.fits.
[000128.36] [0/1] 07-11 20:21 Catalog INFO Loading _catalog/lognormal_randoms.fits.
[000130.01] [0/1] 07-11 20:21 FKPCatalog INFO cartesian coordinate range: [0.00125445 0.00054827 0.0002208 ] : [1387.20757471 1387.33654312 1386.97592007]
[000130.01] [0/1] 07-11 20:21 FKPCatalog INFO BoxSize = 4000.0
[000130.01] [0/1] 07-11 20:21 FKPCatalog INFO BoxCenter = [693.60441458 693.6685457 693.48807044]
[000130.02] [0/1] 07-11 20:21 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'first'
[000130.02] [0/1] 07-11 20:21 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'second'
[000134.42] [0/1] 07-11 20:21 CatalogMesh INFO Chunk 0 ~ 4194304 / 789783
[000135.09] [0/1] 07-11 20:21 CatalogMesh INFO painted 789783 out of 789783 objects to mesh
[000135.40] [0/1] 07-11 20:21 CatalogMesh INFO painted 789783 out of 789783 objects to mesh
[000135.40] [0/1] 07-11 20:21 CatalogMesh INFO mean particles per cell is 0.376598
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000135.40] [0/1] 07-11 20:21 CatalogMesh INFO sum is 789783
[000135.41] [0/1] 07-11 20:21 FKPCatalogMesh INFO data painted.
[000135.43] [0/1] 07-11 20:21 CatalogMesh INFO Chunk 0 ~ 4194304 / 1576395
[000136.89] [0/1] 07-11 20:21 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000137.18] [0/1] 07-11 20:21 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000137.18] [0/1] 07-11 20:21 CatalogMesh INFO mean particles per cell is 0.751684
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000137.18] [0/1] 07-11 20:21 CatalogMesh INFO sum is 1.5764e+06
[000137.20] [0/1] 07-11 20:21 FKPCatalogMesh INFO randoms painted.
[000137.21] [0/1] 07-11 20:21 FKPCatalogMesh INFO volume per cell is 30517.6
[000137.22] [0/1] 07-11 20:21 FKPCatalogMesh INFO field: (FKPCatalog(species=['data', 'randoms']) as CatalogMesh) painting done
[000137.22] [0/1] 07-11 20:21 ConvolvedFFTPower INFO tsc painting of 'first' done
[000137.33] [0/1] 07-11 20:21 ConvolvedFFTPower INFO ell = 0 done; 1 r2c completed
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:531: ComplexWarning: Casting complex values to real discards the imaginary part
xgrid = [xx.astype('f8') + offset[ii] for ii, xx in enumerate(density2.slabs.optx)]
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:536: ComplexWarning: Casting complex values to real discards the imaginary part
kgrid = [kk.astype('f8') for kk in cfield.slabs.optx]
[000145.91] [0/1] 07-11 20:21 ConvolvedFFTPower INFO normalized power spectrum with `randoms.norm = 236.934900`
[000146.53] [0/1] 07-11 20:21 ConvolvedFFTPower INFO ell = 2 done; 5 r2c completed
[000148.01] [0/1] 07-11 20:21 ConvolvedFFTPower INFO ell = 4 done; 9 r2c completed
[000148.42] [0/1] 07-11 20:21 ConvolvedFFTPower INFO higher order multipoles computed in elapsed time 00:00:02.51
[000158.03] [0/1] 07-11 20:21 DataVector INFO Saving to _data/lognormal_power_spectrum_5.txt.
[000158.07] [0/1] 07-11 20:21 Catalog INFO Loading _catalog/lognormal_data_6.fits.
[000158.34] [0/1] 07-11 20:21 Catalog INFO Loading _catalog/lognormal_randoms.fits.
[000160.06] [0/1] 07-11 20:21 FKPCatalog INFO cartesian coordinate range: [0.00125445 0.00054827 0.0002208 ] : [1387.20757471 1387.33654312 1386.97592007]
[000160.06] [0/1] 07-11 20:21 FKPCatalog INFO BoxSize = 4000.0
[000160.06] [0/1] 07-11 20:21 FKPCatalog INFO BoxCenter = [693.60441458 693.6685457 693.48807044]
[000160.08] [0/1] 07-11 20:21 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'first'
[000160.08] [0/1] 07-11 20:21 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'second'
[000164.63] [0/1] 07-11 20:21 CatalogMesh INFO Chunk 0 ~ 4194304 / 789629
[000165.33] [0/1] 07-11 20:21 CatalogMesh INFO painted 789629 out of 789629 objects to mesh
[000165.65] [0/1] 07-11 20:21 CatalogMesh INFO painted 789629 out of 789629 objects to mesh
[000165.65] [0/1] 07-11 20:21 CatalogMesh INFO mean particles per cell is 0.376524
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000165.65] [0/1] 07-11 20:21 CatalogMesh INFO sum is 789629
[000165.67] [0/1] 07-11 20:21 FKPCatalogMesh INFO data painted.
[000165.69] [0/1] 07-11 20:21 CatalogMesh INFO Chunk 0 ~ 4194304 / 1576395
[000167.23] [0/1] 07-11 20:21 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000167.53] [0/1] 07-11 20:21 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000167.53] [0/1] 07-11 20:21 CatalogMesh INFO mean particles per cell is 0.751684
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000167.53] [0/1] 07-11 20:21 CatalogMesh INFO sum is 1.5764e+06
[000167.54] [0/1] 07-11 20:21 FKPCatalogMesh INFO randoms painted.
[000167.56] [0/1] 07-11 20:21 FKPCatalogMesh INFO volume per cell is 30517.6
[000167.57] [0/1] 07-11 20:21 FKPCatalogMesh INFO field: (FKPCatalog(species=['data', 'randoms']) as CatalogMesh) painting done
[000167.57] [0/1] 07-11 20:21 ConvolvedFFTPower INFO tsc painting of 'first' done
[000167.69] [0/1] 07-11 20:21 ConvolvedFFTPower INFO ell = 0 done; 1 r2c completed
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:531: ComplexWarning: Casting complex values to real discards the imaginary part
xgrid = [xx.astype('f8') + offset[ii] for ii, xx in enumerate(density2.slabs.optx)]
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:536: ComplexWarning: Casting complex values to real discards the imaginary part
kgrid = [kk.astype('f8') for kk in cfield.slabs.optx]
[000176.29] [0/1] 07-11 20:21 ConvolvedFFTPower INFO normalized power spectrum with `randoms.norm = 236.888700`
[000176.89] [0/1] 07-11 20:21 ConvolvedFFTPower INFO ell = 2 done; 5 r2c completed
[000178.31] [0/1] 07-11 20:21 ConvolvedFFTPower INFO ell = 4 done; 9 r2c completed
[000178.71] [0/1] 07-11 20:21 ConvolvedFFTPower INFO higher order multipoles computed in elapsed time 00:00:02.42
[000187.87] [0/1] 07-11 20:22 DataVector INFO Saving to _data/lognormal_power_spectrum_6.txt.
[000187.91] [0/1] 07-11 20:22 Catalog INFO Loading _catalog/lognormal_data_7.fits.
[000188.17] [0/1] 07-11 20:22 Catalog INFO Loading _catalog/lognormal_randoms.fits.
[000189.78] [0/1] 07-11 20:22 FKPCatalog INFO cartesian coordinate range: [0.00125445 0.00054827 0.0002208 ] : [1387.20757471 1387.33654312 1386.97592007]
[000189.78] [0/1] 07-11 20:22 FKPCatalog INFO BoxSize = 4000.0
[000189.78] [0/1] 07-11 20:22 FKPCatalog INFO BoxCenter = [693.60441458 693.6685457 693.48807044]
[000189.79] [0/1] 07-11 20:22 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'first'
[000189.79] [0/1] 07-11 20:22 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'second'
[000194.22] [0/1] 07-11 20:22 CatalogMesh INFO Chunk 0 ~ 4194304 / 788980
[000194.90] [0/1] 07-11 20:22 CatalogMesh INFO painted 788980 out of 788980 objects to mesh
[000195.20] [0/1] 07-11 20:22 CatalogMesh INFO painted 788980 out of 788980 objects to mesh
[000195.20] [0/1] 07-11 20:22 CatalogMesh INFO mean particles per cell is 0.376215
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000195.20] [0/1] 07-11 20:22 CatalogMesh INFO sum is 788980
[000195.22] [0/1] 07-11 20:22 FKPCatalogMesh INFO data painted.
[000195.24] [0/1] 07-11 20:22 CatalogMesh INFO Chunk 0 ~ 4194304 / 1576395
[000196.72] [0/1] 07-11 20:22 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000197.01] [0/1] 07-11 20:22 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000197.01] [0/1] 07-11 20:22 CatalogMesh INFO mean particles per cell is 0.751684
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000197.01] [0/1] 07-11 20:22 CatalogMesh INFO sum is 1.5764e+06
[000197.03] [0/1] 07-11 20:22 FKPCatalogMesh INFO randoms painted.
[000197.04] [0/1] 07-11 20:22 FKPCatalogMesh INFO volume per cell is 30517.6
[000197.05] [0/1] 07-11 20:22 FKPCatalogMesh INFO field: (FKPCatalog(species=['data', 'randoms']) as CatalogMesh) painting done
[000197.05] [0/1] 07-11 20:22 ConvolvedFFTPower INFO tsc painting of 'first' done
[000197.16] [0/1] 07-11 20:22 ConvolvedFFTPower INFO ell = 0 done; 1 r2c completed
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:531: ComplexWarning: Casting complex values to real discards the imaginary part
xgrid = [xx.astype('f8') + offset[ii] for ii, xx in enumerate(density2.slabs.optx)]
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:536: ComplexWarning: Casting complex values to real discards the imaginary part
kgrid = [kk.astype('f8') for kk in cfield.slabs.optx]
[000205.80] [0/1] 07-11 20:22 ConvolvedFFTPower INFO normalized power spectrum with `randoms.norm = 236.694000`
[000206.42] [0/1] 07-11 20:22 ConvolvedFFTPower INFO ell = 2 done; 5 r2c completed
[000207.90] [0/1] 07-11 20:22 ConvolvedFFTPower INFO ell = 4 done; 9 r2c completed
[000208.31] [0/1] 07-11 20:22 ConvolvedFFTPower INFO higher order multipoles computed in elapsed time 00:00:02.51
[000217.89] [0/1] 07-11 20:22 DataVector INFO Saving to _data/lognormal_power_spectrum_7.txt.
[000217.93] [0/1] 07-11 20:22 Catalog INFO Loading _catalog/lognormal_data_8.fits.
[000218.20] [0/1] 07-11 20:22 Catalog INFO Loading _catalog/lognormal_randoms.fits.
[000219.80] [0/1] 07-11 20:22 FKPCatalog INFO cartesian coordinate range: [0.00125445 0.00054827 0.0002208 ] : [1387.20757471 1387.33654312 1386.97592007]
[000219.80] [0/1] 07-11 20:22 FKPCatalog INFO BoxSize = 4000.0
[000219.80] [0/1] 07-11 20:22 FKPCatalog INFO BoxCenter = [693.60441458 693.6685457 693.48807044]
[000219.82] [0/1] 07-11 20:22 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'first'
[000219.82] [0/1] 07-11 20:22 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'second'
[000224.28] [0/1] 07-11 20:22 CatalogMesh INFO Chunk 0 ~ 4194304 / 787748
[000224.96] [0/1] 07-11 20:22 CatalogMesh INFO painted 787748 out of 787748 objects to mesh
[000225.27] [0/1] 07-11 20:22 CatalogMesh INFO painted 787748 out of 787748 objects to mesh
[000225.27] [0/1] 07-11 20:22 CatalogMesh INFO mean particles per cell is 0.375628
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000225.27] [0/1] 07-11 20:22 CatalogMesh INFO sum is 787748
[000225.29] [0/1] 07-11 20:22 FKPCatalogMesh INFO data painted.
[000225.31] [0/1] 07-11 20:22 CatalogMesh INFO Chunk 0 ~ 4194304 / 1576395
[000226.84] [0/1] 07-11 20:22 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000227.14] [0/1] 07-11 20:22 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000227.14] [0/1] 07-11 20:22 CatalogMesh INFO mean particles per cell is 0.751684
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000227.14] [0/1] 07-11 20:22 CatalogMesh INFO sum is 1.5764e+06
[000227.16] [0/1] 07-11 20:22 FKPCatalogMesh INFO randoms painted.
[000227.18] [0/1] 07-11 20:22 FKPCatalogMesh INFO volume per cell is 30517.6
[000227.18] [0/1] 07-11 20:22 FKPCatalogMesh INFO field: (FKPCatalog(species=['data', 'randoms']) as CatalogMesh) painting done
[000227.18] [0/1] 07-11 20:22 ConvolvedFFTPower INFO tsc painting of 'first' done
[000227.30] [0/1] 07-11 20:22 ConvolvedFFTPower INFO ell = 0 done; 1 r2c completed
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:531: ComplexWarning: Casting complex values to real discards the imaginary part
xgrid = [xx.astype('f8') + offset[ii] for ii, xx in enumerate(density2.slabs.optx)]
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:536: ComplexWarning: Casting complex values to real discards the imaginary part
kgrid = [kk.astype('f8') for kk in cfield.slabs.optx]
[000236.07] [0/1] 07-11 20:22 ConvolvedFFTPower INFO normalized power spectrum with `randoms.norm = 236.324400`
[000236.69] [0/1] 07-11 20:22 ConvolvedFFTPower INFO ell = 2 done; 5 r2c completed
[000238.15] [0/1] 07-11 20:22 ConvolvedFFTPower INFO ell = 4 done; 9 r2c completed
[000238.56] [0/1] 07-11 20:22 ConvolvedFFTPower INFO higher order multipoles computed in elapsed time 00:00:02.49
[000247.83] [0/1] 07-11 20:23 DataVector INFO Saving to _data/lognormal_power_spectrum_8.txt.
[000247.87] [0/1] 07-11 20:23 Catalog INFO Loading _catalog/lognormal_data_9.fits.
[000248.15] [0/1] 07-11 20:23 Catalog INFO Loading _catalog/lognormal_randoms.fits.
[000249.79] [0/1] 07-11 20:23 FKPCatalog INFO cartesian coordinate range: [0.00125445 0.00054827 0.0002208 ] : [1387.20757471 1387.33654312 1386.97592007]
[000249.79] [0/1] 07-11 20:23 FKPCatalog INFO BoxSize = 4000.0
[000249.79] [0/1] 07-11 20:23 FKPCatalog INFO BoxCenter = [693.60441458 693.6685457 693.48807044]
[000249.81] [0/1] 07-11 20:23 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'first'
[000249.81] [0/1] 07-11 20:23 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'second'
[000254.37] [0/1] 07-11 20:23 CatalogMesh INFO Chunk 0 ~ 4194304 / 787876
[000255.05] [0/1] 07-11 20:23 CatalogMesh INFO painted 787876 out of 787876 objects to mesh
[000255.36] [0/1] 07-11 20:23 CatalogMesh INFO painted 787876 out of 787876 objects to mesh
[000255.36] [0/1] 07-11 20:23 CatalogMesh INFO mean particles per cell is 0.375689
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000255.36] [0/1] 07-11 20:23 CatalogMesh INFO sum is 787876
[000255.37] [0/1] 07-11 20:23 FKPCatalogMesh INFO data painted.
[000255.39] [0/1] 07-11 20:23 CatalogMesh INFO Chunk 0 ~ 4194304 / 1576395
[000256.87] [0/1] 07-11 20:23 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000257.16] [0/1] 07-11 20:23 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000257.16] [0/1] 07-11 20:23 CatalogMesh INFO mean particles per cell is 0.751684
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000257.16] [0/1] 07-11 20:23 CatalogMesh INFO sum is 1.5764e+06
[000257.18] [0/1] 07-11 20:23 FKPCatalogMesh INFO randoms painted.
[000257.20] [0/1] 07-11 20:23 FKPCatalogMesh INFO volume per cell is 30517.6
[000257.20] [0/1] 07-11 20:23 FKPCatalogMesh INFO field: (FKPCatalog(species=['data', 'randoms']) as CatalogMesh) painting done
[000257.20] [0/1] 07-11 20:23 ConvolvedFFTPower INFO tsc painting of 'first' done
[000257.32] [0/1] 07-11 20:23 ConvolvedFFTPower INFO ell = 0 done; 1 r2c completed
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:531: ComplexWarning: Casting complex values to real discards the imaginary part
xgrid = [xx.astype('f8') + offset[ii] for ii, xx in enumerate(density2.slabs.optx)]
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:536: ComplexWarning: Casting complex values to real discards the imaginary part
kgrid = [kk.astype('f8') for kk in cfield.slabs.optx]
[000265.88] [0/1] 07-11 20:23 ConvolvedFFTPower INFO normalized power spectrum with `randoms.norm = 236.362800`
[000266.49] [0/1] 07-11 20:23 ConvolvedFFTPower INFO ell = 2 done; 5 r2c completed
[000267.97] [0/1] 07-11 20:23 ConvolvedFFTPower INFO ell = 4 done; 9 r2c completed
[000268.39] [0/1] 07-11 20:23 ConvolvedFFTPower INFO higher order multipoles computed in elapsed time 00:00:02.51
[000278.16] [0/1] 07-11 20:23 DataVector INFO Saving to _data/lognormal_power_spectrum_9.txt.
[000278.20] [0/1] 07-11 20:23 Catalog INFO Loading _catalog/lognormal_data_10.fits.
[000278.47] [0/1] 07-11 20:23 Catalog INFO Loading _catalog/lognormal_randoms.fits.
[000280.10] [0/1] 07-11 20:23 FKPCatalog INFO cartesian coordinate range: [0.00125445 0.00054827 0.0002208 ] : [1387.20757471 1387.33654312 1386.97592007]
[000280.10] [0/1] 07-11 20:23 FKPCatalog INFO BoxSize = 4000.0
[000280.10] [0/1] 07-11 20:23 FKPCatalog INFO BoxCenter = [693.60441458 693.6685457 693.48807044]
[000280.11] [0/1] 07-11 20:23 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'first'
[000280.11] [0/1] 07-11 20:23 ConvolvedFFTPower INFO using compensation function CompensateTSC for source 'second'
[000284.67] [0/1] 07-11 20:23 CatalogMesh INFO Chunk 0 ~ 4194304 / 788353
[000285.37] [0/1] 07-11 20:23 CatalogMesh INFO painted 788353 out of 788353 objects to mesh
[000285.69] [0/1] 07-11 20:23 CatalogMesh INFO painted 788353 out of 788353 objects to mesh
[000285.69] [0/1] 07-11 20:23 CatalogMesh INFO mean particles per cell is 0.375916
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000285.69] [0/1] 07-11 20:23 CatalogMesh INFO sum is 788353
[000285.70] [0/1] 07-11 20:23 FKPCatalogMesh INFO data painted.
[000285.72] [0/1] 07-11 20:23 CatalogMesh INFO Chunk 0 ~ 4194304 / 1576395
[000287.30] [0/1] 07-11 20:23 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000287.61] [0/1] 07-11 20:23 CatalogMesh INFO painted 1576395 out of 1576395 objects to mesh
[000287.61] [0/1] 07-11 20:23 CatalogMesh INFO mean particles per cell is 0.751684
/Users/pmcdonal/opt/anaconda3/lib/python3.7/logging/init.py:369: ComplexWarning: Casting complex values to real discards the imaginary part
msg = msg % self.args
[000287.61] [0/1] 07-11 20:23 CatalogMesh INFO sum is 1.5764e+06
[000287.63] [0/1] 07-11 20:23 FKPCatalogMesh INFO randoms painted.
[000287.65] [0/1] 07-11 20:23 FKPCatalogMesh INFO volume per cell is 30517.6
[000287.65] [0/1] 07-11 20:23 FKPCatalogMesh INFO field: (FKPCatalog(species=['data', 'randoms']) as CatalogMesh) painting done
[000287.65] [0/1] 07-11 20:23 ConvolvedFFTPower INFO tsc painting of 'first' done
[000287.77] [0/1] 07-11 20:23 ConvolvedFFTPower INFO ell = 0 done; 1 r2c completed
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:531: ComplexWarning: Casting complex values to real discards the imaginary part
xgrid = [xx.astype('f8') + offset[ii] for ii, xx in enumerate(density2.slabs.optx)]
/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/nbodykit/algorithms/convpower/fkp.py:536: ComplexWarning: Casting complex values to real discards the imaginary part
kgrid = [kk.astype('f8') for kk in cfield.slabs.optx]
[000296.88] [0/1] 07-11 20:23 ConvolvedFFTPower INFO normalized power spectrum with `randoms.norm = 236.505900`
[000297.49] [0/1] 07-11 20:23 ConvolvedFFTPower INFO ell = 2 done; 5 r2c completed
[000298.98] [0/1] 07-11 20:23 ConvolvedFFTPower INFO ell = 4 done; 9 r2c completed
[000299.40] [0/1] 07-11 20:23 ConvolvedFFTPower INFO higher order multipoles computed in elapsed time 00:00:02.52
[000309.34] [0/1] 07-11 20:24 DataVector INFO Saving to _data/lognormal_power_spectrum_10.txt.
[000309.41] [0/1] 07-11 20:24 MockCovarianceMatrix INFO Saving to _data/covariance.txt.
[000309.47] [0/1] 07-11 20:24 Exception CRITICAL

Traceback (most recent call last):
File "/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/pypescript/module.py", line 181, in wrapper
fun(*args,**kwargs)
File "/Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/data_vector/mock_covariance_matrix.py", line 25, in setup
self.data_block[section_names.covariance,'invcov'] = cov.get_invcov()
File "/Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/lib/data_vector/covariance_matrix.py", line 183, in get_invcov
return utils.blockinv(cov,inv=inv)
File "/Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/lib/utils.py", line 212, in blockinv
invD = blockinv([b[1:] for b in blocks[1:]],inv=inv)
File "/Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/lib/utils.py", line 212, in blockinv
invD = blockinv([b[1:] for b in blocks[1:]],inv=inv)
File "/Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/lib/utils.py", line 212, in blockinv
invD = blockinv([b[1:] for b in blocks[1:]],inv=inv)
[Previous line repeated 2 more times]
File "/Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/lib/utils.py", line 209, in blockinv
return _inv(A)
File "/Users/pmcdonal/Programs/Arnaud/cosmopipe/cosmopipe/lib/utils.py", line 199, in _inv
toret = inv(mat)
File "<array_function internals>", line 6, in inv
File "/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 545, in inv
ainv = _umath_linalg.inv(a, signature=signature, extobj=extobj)
File "/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 88, in _raise_linalgerror_singular
raise LinAlgError("Singular matrix")
numpy.linalg.LinAlgError: Singular matrix

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/pypescript/pipeline.py", line 207, in wrapper
func(*args,**kwargs)
File "/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/pypescript/pipeline.py", line 185, in execute
todo()
File "/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/pypescript/pipeline.py", line 30, in call
func()
File "/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/pypescript/module.py", line 183, in wrapper
raise RuntimeError('Exception in function {} of {} [{}].'.format(name,self.class.name,self.name)) from exc
RuntimeError: Exception in function setup of MockCovarianceMatrix [cov].

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/pmcdonal/opt/anaconda3/bin/pypescript", line 8, in
sys.exit(main())
File "/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/pypescript/main.py", line 36, in main
return pypescript_main(config_block=opt.config_block_fn,pipe_graph_fn=opt.pipe_graph_fn,data_block=opt.data_block_fn,save_data_block=opt.save_data_block_fn)
File "/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/pypescript/main.py", line 24, in main
pipeline.execute()
File "/Users/pmcdonal/opt/anaconda3/lib/python3.7/site-packages/pypescript/pipeline.py", line 209, in wrapper
raise RuntimeError('Exception in function {} of {} [{}].'.format(name,self.class.name,self.name)) from exc
RuntimeError: Exception in function execute of BasePipeline [main].

[000309.47] [0/1] 07-11 20:24 Exception CRITICAL An error occured.

demo comments

Just a sentence or two at the top of the demo yamls indicating what they’re supposed to be doing would be very helpful.

unify power and window common parts

It would be good to unify, e.g., the catalog reading, cutting, etc. for window and power. E.g., here, after setting up zmin and zmax options for power, I am needing to duplicate them for window, even though the catalogs should be the same.

cosmopipe/cosmopipe/estimators/window_function/fft_window_function.py

Line 29 in 0f15bd7

 self.catalog_options = {'z':'Z','ra':'RA','dec':'DEC','position':None,'weight_comp':None,'nbar':{},'weight_fkp':None,'P0_fkp':0.} 

Catalogs probably should be read upstream and just passed down to power and window, and power should pass along things that are supposed to be the same like normalization.

(option to) plot estimated power spectrum points with error bars

Instead of current connected line, with error band on theory when present

What is going on here in nbodykit_survey.py?

I edited it to put in zeff, so... not comfortable not understanding what I did...
See comment "what is going on here with the multiplication when there is more than one catalog"

cosmopipe/cosmopipe/estimators/power_spectrum/nbodykit_survey.py

Lines 72 to 86 in 6564c97

 wdata2 = 1. 

 wran = 1. 

 zeffdata = 1. 

 zeffran = 1. 

 for data,randoms in zip(input_data,input_randoms): 

 data = data.mpi_to_state('scattered') 

 if randoms is not None: randoms = randoms.mpi_to_state('scattered') 

 data,randoms = utils.prepare_survey_catalogs(data,randoms,cosmo=cosmo,**self.catalog_options) 

 fkp = FKPCatalog(data.to_nbodykit(),randoms.to_nbodykit() if randoms is not None else None,BoxPad=self.BoxPad,nbar='nbar') 

 list_mesh.append(fkp.to_mesh(position='position',fkp_weight='weight_fkp',comp_weight='weight_comp',nbar='nbar',**self.mesh_options)) 

 #what is going on here with the multiplication when there is more than one catalog?!? 

 wdata2 *= mpi.sum_array(data['weight_comp']*data['weight_fkp'],mpicomm=data.mpicomm) 

 wran *= mpi.sum_array(randoms['weight_comp']*randoms['weight_fkp'],mpicomm=data.mpicomm) 

 zeffdata *= mpi.sum_array(data['z']*data['weight_comp']*data['weight_fkp'],mpicomm=data.mpicomm) 

 zeffran *= mpi.sum_array(randoms['z']*randoms['weight_comp']*randoms['weight_fkp'],mpicomm=data.mpicomm)

probably harmless weird code

This redefinition of origin_catalogs doesn't make any sense... does it?

cosmopipe/cosmopipe/estimators/utils.py

Lines 29 to 31 in 2542523

 origin_catalogs = {'data':data,'randoms':randoms} 

 catalogs = {} 

 origin_catalogs = {'data':data}

strange fit result plots

Fitting e.g., BOSS, parameter results look reasonable, and chi^2 good (albeit for Gaussian approximate errors), but theory curve far from data in plot.

compare random-data power to random-random power

How do we know randoms are "correct"? (non-trivial, but it seems we mostly take it on faith at power measurement level) Well, one good test seems to be comparing data-random power to (shot-noise-subtracted) random-random power (or similar for correlation). These are supposed to be identical. Of course this is in the "simple selection function" model. With fiber collision-type things one would need to think more... but thinking is good...

import Florian BOSS power, window, etc and do fit

compare to cosmopipe estimate

save best fit model to reload

Having done a fit to some data set, it would be good to print the results in a form, and have code to read it, such that it is easy to effectively use those parameter results as the default (starting) model in some later run.

on the fly Gaussian mocks

Key to most basic testing of power, window, and covariance estimates is being able to efficiently generate Gaussian fields with known power spectrum to feed into power estimation. chi^2 of power measured from these and predicted using window, weighted by covariance, should be appropriately distributed. The distribution of parameter fit results can also be studied, since chi^2 is a blunt instrument. I'd say little point in doing other code tests (e.g., with fancy mocks) before passing this one.

figure out how to calculate/print a window matrix

I.e., a matrix to multiply theory power vector by to produce power summary statistics. If the code is producing this... just need to find it.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

Jobs

Jooble

	wdata2 = 1.
	wran = 1.
	zeffdata = 1.
	zeffran = 1.
	for data,randoms in zip(input_data,input_randoms):
	data = data.mpi_to_state('scattered')
	if randoms is not None: randoms = randoms.mpi_to_state('scattered')
	data,randoms = utils.prepare_survey_catalogs(data,randoms,cosmo=cosmo,**self.catalog_options)
	fkp = FKPCatalog(data.to_nbodykit(),randoms.to_nbodykit() if randoms is not None else None,BoxPad=self.BoxPad,nbar='nbar')
	list_mesh.append(fkp.to_mesh(position='position',fkp_weight='weight_fkp',comp_weight='weight_comp',nbar='nbar',**self.mesh_options))
	#what is going on here with the multiplication when there is more than one catalog?!?
	wdata2 = mpi.sum_array(data['weight_comp']data['weight_fkp'],mpicomm=data.mpicomm)
	wran = mpi.sum_array(randoms['weight_comp']randoms['weight_fkp'],mpicomm=data.mpicomm)
	zeffdata = mpi.sum_array(data['z']data['weight_comp']*data['weight_fkp'],mpicomm=data.mpicomm)
	zeffran = mpi.sum_array(randoms['z']randoms['weight_comp']*randoms['weight_fkp'],mpicomm=data.mpicomm)

	origin_catalogs = {'data':data,'randoms':randoms}
	catalogs = {}
	origin_catalogs = {'data':data}