sarodyatawatta / smart-calibration

Deep reinforcement learning for smart calibration of radio telescopes. Automatic hyper-parameter tuning.

Smart Calibration

Using reinforcement learning for hyperparameter tuning in calibration of radio telescopes, and in other data processing pipelines (like elastic net regression). Code to accompany the paper Deep reinforcement learning for smart calibration of radio telescopes (preprint). Enhancements based on spatial regularization (preprint) are included.

RL agent code is based on this code. Prioritized experience replay memory code is based on this code.

Implemented in PyTorch, using openai.gym. Algorithms tested are: DDPG, TD3 and SAC. The figure below shows the performance of the three.

Elastic net regression

Run main_{ddpg|td3|sac}.py to use DDPG or TD3 or SAC. Prioritized experience replay can be enabled by using the flag prioritized=True. See also this for distributed learning.

Files included are:

autograd_tools.py: utilities to calculate Jacobian, inverse Hessian-vector product etc.

enetenv.py: openai.gym environment

enet_td3.py : TD3 agent

enet_ddpg.py: DDPG agent

enet_sac.py: SAC agent

enet_eval.py: evaluation and compare with sklearn GridSearchCV

lbfgsnew.py: LBFGS optimizer

main_ddpg.py: run this for DDPG

main_td3.py: run this for TD3

main_sac.py: run this for SAC

Calibration

Additional packages: python-casacore, astropy, openmpi

Simulation and Calibration software: SAGECal

Imaging software: Excon

Influence maps

Influence maps give a visual representation of the influence function of radio interferometric calibration. Here is a sample:

We use influence maps as part of the state representation. An analogy to this is visualizing an untrained and a trained CNN model. The figures below show the influence function for CIFAR10 classifier using AlexNet (untrained), AlexNet (trained) and ResNet18 (trained). With training, the figures become less random, but still has non-zero features implying overfitting in the training.

Note the difference between the trained AlexNet (65% accuracy) and ResNet18 (80% accuracy), the latter has much less bias.

Run main_{ddpg|td3|sac}.py to use DDPG or TD3 or SAC. You can copy some example data from here.

The figure below shows small areas in the sky before (left) and after (right) calibration.

Files included are:

calibration_tools.py: utility routines

simulate.py: generated sky models, systematic errors for data simulation

calibenv.py: openai.gym environment

analysis.py: calculate influence function/map

calib_td3.py: TD3 agent

calib_ddpg.py: DDPG agent

calib_sac.py: SAC agent

lbfgsnew.py: LBFGS optimizer

main_ddpg.py: run this for DDPG

main_td3.py: run this for TD3

main_sac.py: run this for SAC

docal.sh: shell wrapper to run calibration

doinfluence.sh: shell wrapper to run influence mapping

dosimul.sh : shell wrapper to run simulations

inspect_replaybuffer.py: inspect the replay buffer contents

The following scripts are for handling radio astronomical (MS) data

addcol.py: add new column to write data

changefreq.py: change observing frequency

readcorr.py: read data and output as text

writecorr.py: write text input and write to MS

addnoise.py : add AWGN to data

calmean.sh: calculate mean image

doall.sh: wrapper to do simulation, calibration, and influence map generation