qcodes / qcodes_loop

Loop and matching dataset that used to be part of qcodes

QCoDeS

QCoDeS is a Python-based data acquisition framework developed by the Copenhagen / Delft / Sydney / Microsoft quantum computing consortium. While it has been developed to serve the needs of nanoelectronic device experiments, it is not inherently limited to such experiments, and can be used anywhere a system with many degrees of freedom is controllable by computer. To learn more about QCoDeS, browse our homepage .

To get a feeling of QCoDeS read 15 minutes to QCoDeS, and/or browse the Jupyter notebooks in docs/examples .

QCoDeS is compatible with Python 3.10+. It is primarily intended for use from Jupyter notebooks, but can be used from traditional terminal-based shells and in stand-alone scripts as well. The features in qcodes.utils.magic are exclusively for Jupyter notebooks.

Default branch is now main

The default branch in QCoDeS has been renamed to main. If you are working with a local clone of QCoDeS you should update it as follows:

• Run git fetch origin and git checkout main
• Run git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/main to update your HEAD reference.

Install

In general, refer to here for installation.

Docs

Read it here . Documentation is updated and deployed on every successful build in main.

We use sphinx for documentations, makefiles are provided both for Windows, and *nix, so that you can build the documentation locally.

Make sure that you have the extra dependencies required to install the docs

pip install -r docs_requirements.txt

Go to the directory docs and

make html

This generate a webpage, index.html, in docs/_build/html with the rendered html.

QCoDeS Loop

The modules qcodes.data, qcodes.plots, qcodes.actions, qcodes.loops, qcodes.measure, qcodes.extensions.slack and qcodes.utils.magic that were part of QCoDeS until version 0.37.0. have been moved into an independent package called qcodes_loop. Please see it's repository and documentation for more information.

For the time being it is possible to automatically install the qcodes_loop package when installing qcodes by executing pip install qcodes[loop].

Code of Conduct

QCoDeS strictly adheres to the Microsoft Open Source Code of Conduct

Contributing

The QCoDeS instrument drivers developed by the members of the QCoDeS community but not supported by the QCoDeS developers are contained in

https://github.com/QCoDeS/Qcodes_contrib_drivers

See Contributing for general information about bug/issue reports, contributing code, style, and testing.

License

See License.

qcodes_loop's Issues

qcodes dataset cannot be stored easily

The qcodes dataset can only be stored to disk using a qcodes formatter. To save results totgether with other data it would be good to be able to serialize a dataset to a string or something like pickle, json or xml. So someting like:

s=dataset.tostring()
dataset2 = qcodes.load_dataset_string(s)

or

mystring=pickle.dumps(dataset)
dataset2=pickle.loads(mystring)

The method used for converting the dataset to a string could be taken from the formatter.

plot simple data array

Right now I can measure a simple array and plot using either MatPlot or QtPlot. But when I start processing this data, there is no good option to plot the results with the same axis and labels.

An example:

import matplotlib.pyplot as plt
import copy
import numpy as np
import qcodes as qc

#%% Make toy model
from toymodel import AModel, MockGates, MockSource, MockMeter, AverageGetter, AverageAndRaw
model = AModel()
gates = MockGates('gates', model=model)
source = MockSource('source', model=model)
meter = MockMeter('meter', model=model)
station = qc.Station(gates, source, meter)

station.set_measurement(meter.amplitude)

c0, c1, c2, vsd = gates.chan0, gates.chan1, gates.chan2, source.amplitude


#%% 1D loop to make data

data = qc.Loop(c0[-20:20:0.2], 0.01).run(location='data/testsweep', overwrite=True)
data.complete()

Plotting the data is fine!

plotQ = qc.QtPlot(data.amplitude)

Now suppose we want to calculate the derivative and show results. We can convert to Numpy and show results, but then we have no axis labels...

X=np.array(data.amplitude)
Y=np.convolve(X, [1.,0,-1.])
plotQ = qc.QtPlot(X) # labels

Trying to make a copy of the DataArray and setting values fails...

dd=copy.deepcopy(data.amplitude) # all kinds of errors possible!
dd.ndarray = Y # fails, dd is a Numpy array

plotQ = qc.QtPlot(dd)

Finally: apply the derivative to the DataArray itself. No labels again...

Y=np.convolve(data.amplitude, [1.,0,-1.])
plotQ = qc.QtPlot(X) # no labels

Options I see:

• Make a constructor for a DataArray accepting another DataArray and a plain numpy array
• Adopt the BasePlot so we can do a .add([DataSet], [numpy array])
• Add an add_array option to the DataSet with a Numpy array as argument (not a DataArray)
• Make a separate showImage(data_set, image) function that duplicates part of the functionality of BasePlot

Any suggestions on which way to go?

Failure in slack test

The following test has started failing test_slack_inst_should_not_fail_upl_latest_wo_plot
not clear why but perhaps a change in slacksdk

HDF5 backend does not roundtrip list of strings in snapshot correctly

The hdf5backend does not correctly round trip a list of strings as they end up as an array of bytes on load

See #48 for changes required to disable this

Encoding of io//location is broken on windows.

Steps to reproduce

1. name something with a greek letter
2. try to save to disk
3. depression

qcodes\data\gnuplot_format.py in write(self, data_set, io_manager, location, force_write, write_metadata)
   276             with io_manager.open(fn, open_mode) as f:
   277                 if overwrite:
--> 278                     f.write(self._make_header(group))
   279
   280                 for i in range(save_range[0], save_range[1] + 1):

lib\encodings\cp1252.py in encode(self, input, final)
    17 class IncrementalEncoder(codecs.IncrementalEncoder):
    18     def encode(self, input, final=False):
---> 19         return codecs.charmap_encode(input,self.errors,encoding_table)[0]
    20
    21 class IncrementalDecoder(codecs.IncrementalDecoder):

UnicodeEncodeError: 'charmap' codec can't encode character '\u03bc' in position 110: character maps to <undefined>

System

operating system
Windows of course

qcodes branch
Master, any commit will do.

manual creation of dataset

The following code created a dataset, but only the z array is shown.

There are two issues here:

• The x and y arrays do not show. This is something related to the .action_indices
• According to the documentation https://github.com/qdev-dk/Qcodes/blob/master/qcodes/data/data_set.py#L76 the argument arrays should be a dict, but that generates an error

import qcodes
from qcodes.tests.data_mocks import *

def DataSet2D(location=None):
    # DataSet with one 2D array with 4 x 6 points
    yy, xx = numpy.meshgrid(range(4), range(6))
    zz = xx**2+yy**2
    # outer setpoint should be 1D
    xx = xx[:, 0]
    x = DataArray(name='x', array_id='x', label='X', preset_data=xx, is_setpoint=True)
    y = DataArray(name='y', array_id='y',  label='Y', preset_data=yy, set_arrays=(x,),
                  is_setpoint=True)
    z = DataArray(name='z',  array_id='z', label='Z', preset_data=zz, set_arrays=(x, y))
    #return new_data(arrays={'x': x, 'y': y, 'z': z}, location=location) # this would fail
    return new_data(arrays=[x,y,z], location=location)

d=DataSet2D()
print(d)

@alexcjohnson @giulioungaretti

DataArray subset

At the moment there is no functionality to make a subset of a DataSet or DataArray, e.g. if data have a DataSet with a 2D data array amplitude and setpoint arrays x, y one should be able to do

subdataset = dataset.subset('amplitude', 0:10, 0:30) 

assert(subdataset.x.size==10)
assert(subdataset.y.shape==(10, 30) )

Some questions

• Which notation do we want to use?
• Do we copy the metadata, or just create an empty metadata?

Reduce save interval

Currently data is saved every 5 seconds. This may be bad if measurements are interrupted as the data from the last 5 seconds is missing. We should investigate getting rid of this

• Benchmark if there are any real performance implications. I suspect not as the data will be buffered by the os anyway
• Reduce the save interval -> possibly to zero
• Get rid of the save interval completely

QtPlot xlabel does not show up for SI units with prefix 'u', 'm', 'n' (e.g. 'uV')

Steps to reproduce

1.liveplot using QtPlot, in measurement sweeping parameter with unit with prefix 'u', 'm', 'n' (e.g. using 'uV')

Expected behaviour

Should see the xlabel of the plot

Actual behaviour

the x label disappears in a few 100ms. The labels is visualized correctly if I use 'V', but disappears using e.g. 'uV'
note: this only happens with latest release of pyqtgraph (v0.11.1) with prev version (v0.11.0) fixes the problem.

System

Win 10

master branch, qcodes v0.23.0

Errors during live plotting

While doing a long measurement, I often get errors from the live-plotting:

error reading file 2016-05-01\22-56-15\data.dat
Traceback (most recent call last):
  File "E:\Git\Qcodes\qcodes\data\format.py", line 94, in read
    self.read_one_file(data_set, f, ids_read)
  File "E:\Git\Qcodes\qcodes\data\format.py", line 346, in read_one_file
    myindices, indices)
ValueError: ('inconsistent setpoint values', 0.0135, 0.0, 'z_fldC', (63,), [63, 79])

error reading file 2016-05-01\22-56-15\data.dat
Traceback (most recent call last):
  File "E:\Git\Qcodes\qcodes\data\format.py", line 94, in read
    self.read_one_file(data_set, f, ids_read)
  File "E:\Git\Qcodes\qcodes\data\format.py", line 330, in read_one_file
    values = tuple(map(float, line.split()))
ValueError: could not convert string to float: '1.8953e-'

error reading file 2016-05-01\22-56-15\data.dat
Traceback (most recent call last):
  File "E:\Git\Qcodes\qcodes\data\format.py", line 94, in read
    self.read_one_file(data_set, f, ids_read)
  File "E:\Git\Qcodes\qcodes\data\format.py", line 330, in read_one_file
    values = tuple(map(float, line.split()))
ValueError: could not convert string to float: '-'

error reading file 2016-05-01\22-56-15\data.dat
Traceback (most recent call last):
  File "E:\Git\Qcodes\qcodes\data\format.py", line 94, in read
    self.read_one_file(data_set, f, ids_read)
  File "E:\Git\Qcodes\qcodes\data\format.py", line 262, in read_one_file
    ids = self._read_comment_line(f).split()
  File "E:\Git\Qcodes\qcodes\data\format.py", line 360, in _read_comment_line
    raise ValueError('expected a comment line, found:\n' + s)
ValueError: expected a comment line, found:

Is it really reading the file? I thought it would sync the data from the measurement server?
If I look at the file itself, I cannot find a 1.8953e- that is broken, only a few occurrences but they look fine.
So is one process reading the file while another one is writing to it?

Negative Delay

This issue might be related to microsoft/Qcodes#44 and microsoft/Qcodes#47
upon writing a numerical simulation as an instrument several problems occurred (described in separate issues)

I want to sweep over some numerical parameter such the energy of the system and measure some transmission property. the loop i wrote is:

loop = qcodes.Loop(system.parameters["energy"][0:1:0.1], delay=0)
data = loop.run(background=False)

where system is the variable containing the numerical simulation instrument
(the background=false is part of a separate issue)
upon running this code i get the same warning as @AdriaanRol got before:
WARNING:root:negative delay -0.000041 sec

In the case of such loops (containing numerical work rather than experiments) I feel that there should be some easy/trivial way to remove this sort of delays from the loop without causing this warning.
I feel that this behavior should be default for the syntaxes:

qcodes.Loop(system.parameters["energy"][0:1:0.1], delay=0)
qcodes.Loop(system.parameters["energy"][0:1:0.1], delay=None)
qcodes.Loop(system.parameters["energy"][0:1:0.1])

I understand that this warning is very important in some loops, but that is only the case if delay>0.
For loops not containing a delay, the actual time taken will always be larger than 0 and the warning is meaningless.

drop nans for datasets

I just made a new issue to rephrase microsoft/Qcodes#208 to a feature request

When loading an experiment file, that was not written to the end, i.e. by aborting a measurement, the missing values from the original intended sizes are just filled with nan's.

It would be nice to have a dropna function like in pandas that removes all dataset rows at the end of the dataset which are filled with nans, maybe also all colums that are nans only:

DataSet.dropna(axis=0, how='all') #removes all columns with nans-only or setpoints-only
DataSet.dropna(axis=1, how='all') #removes all rows that have nans-only

including the inplace feature that updates the dataset file, and the snapshot arrays.

Even though this discusses the DataSet it should also be possible for a single DataArray :)

memory consumption when loading data

I have a dataset on disk it consumes 250mb, when I load this after the measurement has finished, the python process consumes ~1GB of memory, does that somehow make sense? Shoudnt it be much smaller in memory?
How can I profile this?

cc: @alexcjohnson @giulioungaretti

QtPlot with sweeps

Hi!
Thanks for the good work on the environment/library.

I was trying to enable dynamic 2d-map plotting with QtPlot. What I need is a way to add traces one by one, i.e. sweep by sweep (which appears to be a hassle in every aspects with QCoDeS). What I tried so far is

1. extract x_data, y_data, z_data as qc.DataArrays of sizes (n,) (1,), and (n,), respectively
2. after the first trace, create plot2d = qc.QtPlot(x_data, y_data, z_data)
3. add subsequent traces with the add method, as plot2d.add(x_data_new, y_data_new, z_data_new) (here, x_data_new, y_data_new, z_data_new are qc.DataArrays of sizes (n,) (1,), and (n,) as before)
4. Then call plot2d.autorange() to keep to the original figure size.

What I want: one 2d-map plot with sweeps added one by one, position based on their y-value. One x-axis, one y-axis, one z-axis and associated colorbar get their ranges updated. (What else I want: exactly one DataSet created over the course of all the sweeps.)

The above otherwise works, but has the unexpected feature that each add-method also adds a separate colorbar, which is inconvenient if there are going to be 500 traces. I found no ways to control the colorbars (it's not done via update_colorbar = True in autorange). I failed to interface with the likes of plot.update() to get what I want, probably because I fail to employ loops to get what I want*.

*Also, any suggestions on how to natively use DataSets which allow adding data sweep by sweep would be greatly appreciated. The reason I need this is that the sweep ranges and sweep directions might change at every sweep. I also might address each trace separately later.

I'm working on an Mac, and installed qcodes a week or two ago on the machine.

Best wishes, and thanks for your time!

Loop forever or for n steps, or for given time.

During measurements I often encounter situations where I want to run a loop over and over again, i.e. measuring IV-traces during cooldown.

Is there an easy solution for that?

What I can do right now is something like this:

step_param = ManualParameter('step', initial_value=0)
swp = SweepFixedValues(step_param,start=0,stop=100,step=1)
data = qc.Loop(swp).loop(k1v.sweep(-0.006, 0.006, num=51), 0.01).each(curr, vdrp).run()

It would be nice if we had loops that don't do anything than repeating the inner loop, or loops that would repeat the inner loop for a given amout of time.

qc.Loop_forever().each(BreakIf(sunshine==True), qc.Loop(...)
qc.Loop_forever(steps=100).loop(...)
qc.Loop_forever(time=3600).loop(...)

@alexcjohnson or anyone else, any ideas on how to do this?