kyrylo-gr / labmate Goto Github PK

... That should return the length of the loop current level.

At the moment, the progress bar in the following has no final length

for voltage in tqdm(loop(voltages)): pass

we need to switch from the current system where the parameters such as yoko voltage, anapico frequencies and powers are totally undefined to a system where at the beginning of each acquisition, the parameters are set according to variables in the config file.

It seems to me like the cleanest way to do that would be to implement the init_acquisition_cell.py call in labmate.

Something like this should do the trick

I know we have argued about that quite a lot already, but at the moment we are clearly in a suboptimal situation:

The analysis gets saved with the figure only the first time save_fig is called... After that, one needs to explicitly change the code to save_fig_and_analysis_cell to have the analysis saved with the figure...

I am fine with having an explicit User Interface, but here it's all but explicit since save_fig has a different behavior the first time it is called and latter...

Other problem: a typical use-case is to have two figures, each generated by a different analysis:

• for instance, there is the "default" analysis, that just plots the data
• and then the "fit" analysis that also shows the fit
  (this is what I just encounter with fit of the single-tone vs flux map)

I should be able to use

• aqm.save_analysis_and_fig() # for the simple default analysis (that should save the latest version of the analysis in "analysis_cell")
• aqm.save_analysis_and_fig(name="fit_single_tone") # for the fit (that should save the latest version of the fit analysis in "analysis_cell_fit_single_tone", and the figure in ...single_tone_fit_single_tone.pdf)

then, save_fig should do what it says: only save the figure...

then, I agree that due to laziness and historical reasons, people will have a tendency to use save_fig, which is very bad because the analysis will never be saved, so what I suggest is to rename it to save_fig_only (and then, we can discuss if save_analysis_and_fig should not simply be renamed to save_fig, maybe with a parameter save_analysis defaulting to True)

and check anapico driver

The idea is that it would save also the latest snapshot of those extra_cells within (..)_ACQUISTION_CELL.py and (..)_ANALYSIS_CELL.py.

Typical usecase is when there is for instance QUA code for state preparation in a separate cell, or when fitting procedures are all defined in one given cell...

It happened to us more than once to loose 5 minutes before realizing what we were looking at was not the figure corresponding to the last run, but some old sequence

... and replace it by proper management of aqm.current_acquisition is None

• access cfg thought aqm.data.cfg and not aqm.cfg (or both)
• Maybe add parsing of simple evaluation like:

x = 1.2
y = x

or even

def abc():
    return 3
x = abc()

Doesn't properly strip the .h5 from the directory name (see exception below)

ValueError: Cannot load data from C:\Users\CryoPc\My Drive\mecaflux\measurements\Cooldown_2023_01_06_CEAv1_die8\\ge_flux_sweep.h5\2023_01_10__17_27_50__ge_flux_sweep.h5

Add something like every time filename of h5 file is logged out

html = f'(<a href="file://{file}">open</a>'
display.HTML(str(html))

1st ex when opening doc is from the pre-labmate era

path:

labmate/docs/acquisition_notebook.md

do it as a optional parameter

With pickle savefig works only on the same platform and it's not that easy to reopen the fig.

So it should be rewritten to become universal and easy to use

I believe what we would like is something like the following:

%%acquisition_cell some_experiment

data = AQM.create_data_file()

data.loop_voltage = AcqusitionLoop()

voltages = np.arange(0, 1, 100)
for i in data.loop_voltage(voltages):
    res = get_some_datasets_from_somewhere()
    data.loop_voltage.add_kwds(**res) ## The data are saved immediately in the corresponding row of the array, ideally in an efficient way

data.add_kwds(voltages=voltages, yoko_voltage=yoko.get_voltage()) # it would also be natural to be able to do data.key = value

Other use case: retrieve data being accumulated in a later cell:

%%acquisition_cell some_experiment

job.execute(some_program)

%%analysis_cell

if not AQM.is_loaded_data() and job is not None;
    res = job.fetch_data()
    data = AQM.create_data_file() # overwrite entirely the data file
    data.add_kwds(**res)

I quite often encounter an annoying scenario:

1. You modify the configuration.py
2. you re-run the same acquisition_cell...
3. There's an autoreload problem, or the sequence won't compile or anything that prevents the new acquisition from being launched...
4. Unfortunately, you don't notice this because you don't look at the exception traceback, or because you don't notice the print of the failed autoreload
5. You run the analysis cell and end-up re-analyzing the old data, unfortunately, in the mean-time the metadata corresponding to your new configuration have been saved in the temp file
6. You scratch your head wondering why the curves didn't change the way you were expecting (why they didn't change at all actually).
7. Eventually, you scroll up and notice that in fact you are still analyzing a run that was launched with the old parameters...
   --->Unfortunately, the saved data and figures don't have the right metadata, which is very annoying.

I guess the solution to this problem is not unique:
a. updating temp only once the acquisition_cell execution is over (and no exception occured) would at least prevent the wrong metadata from being saved with the new data, but anyways, it is not great because some parameters might have changed (like the yoko voltage for instance) while others would not (like the if_frequencies for instance) ---> This is not ideal.
b. In fact we should maybe go further and really prevent from analyzing anything when the acquisition_cell didn't go to the last statement '''job = execute(program)''', this could be done very simply by setting '''job = None''' in the init_acquisition_cell.py. As I understood, anyways, you are working at pickling the job, so we should probably think of a solution where the pickle is set to None at the beginning of acquisition_cells ?
c. Even all this is not ideal in the case where autoreload has silently failed (for instance, imagine a syntax error in configuration.py), then you would create a job with the wrong parameter and analyze it... For this, I am not sure how to do... Maybe there's a way to force an exception when autoreload fails...

We only get the following (the line that ends up being executed in ipython by spyder)

runcell(0, 'C:/Users/CryoPc/My Drive/mecaflux/measurements/Cooldown_2022_05_30_g_fluxonium_v1_is_back/untitled1.py')

... I know that it makes some sense, but it would be nice if there was a simple workaround...

I think I fixed this behavior by changing lin 143 of acquisition_manager_notebook.py from:
self.analysis_cell_str = None ==> self.analysis_cell_str = cell

Just wanted to be sure this is the right way of fixing it...

+create title from scalar values

aqm.analysis_cell()
if True:
    x, y = acquire_data() # noqa
    aqm.save_acquisition(x=x, y=y, t1=test.a(), t2=test.b);

def abc(param_inside_abc):
    return param_inside_abc

This code will lint: param_inside_abc and test, while it should not be a case

There is possible weird behavior if data and file provided.

Now it does not load any data from file, but it's never tested.

I made a beautiful figure but can't save it due to the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-611-06c9707b9724> in <module>
     32 print(f"Mean e pop = {aqm.data.left.mean():.4f}")
     33 fig.tight_layout()
---> 34 aqm.save_fig(fig)
     35 plt.show()

~\Documents\GitHub\acquisition_utils\quanalys\acquisition_notebook\acquisition_analysis_manager.py in save_fig(self, fig, name, cell, **kwds)
    150     ) -> AcquisitionAnalysisManager:
    151 
--> 152         self.save_fig_only(fig=fig, name=name, **kwds)
    153         self.save_analysis_cell(name=name, cell=cell)
    154         return self

~\Documents\GitHub\acquisition_utils\quanalys\acquisition_notebook\acquisition_analysis_manager.py in save_fig_only(self, fig, name, **kwds)
    119             raise ValueError('No data set')
    120 
--> 121         self._analysis_data.save_fig(fig=fig, name=name, **kwds)
    122         return self
    123 

~\Documents\GitHub\acquisition_utils\quanalys\acquisition_utils\analysis_data.py in save_fig(self, fig, name, **kwargs)
    108                 import pickle
    109                 import codecs
--> 110                 pickled = codecs.encode(pickle.dumps(fig), "base64").decode()
    111                 self[f"figures/{fig_name}"] = pickled
    112                 self.save([f"figures/{fig_name}"])

AttributeError: Can't pickle local object '_make_inset_locator.<locals>.inset_locator'

The minimum example to reproduce the problem should be:

fig, axes = plt.subplots(2, 1, sharex=True, figsize=(8, 8))

cax = axes[0].inset_axes([1.04, 0.2, 0.05, 0.6])
c = axes[0].pcolorfast(aqm.data.qm_voltages, aqm.data.freqs*1e-6, aqm.data.left.T)
fig.colorbar(c, ax=axes[0], cax=cax).set_label("population in e")

I am not sure how to handle the problem, but worst case scenario, I would have the savefig simply write a warning that it cannot save the figure in case for some reason the figure is not pickle-able...

... eventhough, the length of the arrays inside should not change anymore at that point...

Could be that it exists and I missed it, but if not, we really need something like:

aqm.get_single_kwd_from_config("configuration.py", "if_spectro_eg") --> returns the value and not a dict (maybe the parameter configuration.py could default somehow to "look in any possible configuration file"...

In which case, one thing that could be even simpler would be:
aqm.config_data.if_spectro_eg

I would find it legitimate to be able to write:

for data in loop[:10]:

at the moment, I get

`---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in
20
21 z_fitted = np.zeros(z_corr.shape, dtype=complex)
---> 22 for index, data in enumerate(loop_voltage[:10]):
23 if index>10:
24 break

~\Documents\GitHub\acquisition_utils\quanalys\acquisition_utils\analysis_data.py in getitem(self, _AnalysisData__key)
51
52 def getitem(self, __key: str) -> Optional[Any]:
---> 53 return self._data.getitem(__key)
54
55 @editing

TypeError: unhashable type: 'slice'`

This function should make a string key1=value1, key2=value2...

where the key,value pairs are taken, by order of priority from aqm.data, config1, config2...

I was thinking also that this function could automatically display all "scalar" values in aqm.data, for instance if display_all_scalar_data=True is provided

i.e. now it's done by:
aqm.aq['something'] = ... -> aqm['something'] = ...

probably just setitem, not getitem to prevent people from using this data instead of analysis_data

This way an exception would be raised if trying to analyse a run that has nothing to do with the analysis

Then, a relatively straightforward improvement would be to store the various analysis_cell that apply to each acquisition_cell for latter use (there would be also a parameter default=True/False, and name, to make sure several analysis can be registered for each type of acquisition)

At the moment it is extremely difficult to understand the relationships between classes in the code because the name "Manager" is employed randomly in class names.

I am expecting that SomestuffManager means "a factory for Somestuff". i.e: you expect this thing to have methods that can create new instances of Somestuff

So if we keep AcquisitionManagerNotebook, then, what is now called AnalysisManager should be named Acquisition (or AcquisitionDataset maybe)

My understanding is that aqm.am is somehow "the current acquisition_data". Could the name be more explicit (something like aqm.current_acquisition_data or aqm.cad) ?

Now save_on_edit saves everything each time something is changed. It should be save only the values that were changed