btel / spikesort Goto Github PK

extracelluar recordings are stored in 2d array, where first dimension are the samples and second channels (contacts). However from point of view of optimisation it is better to store the fastest changing index in the second dimension (C ordering). rewrite IO, tests and extract to use this ordering

Personally, I find Tk to be very ugly (no judgment on the choice to use it though, it can be MUCH easier to work with Tk than other GUI toolkits in Python), and it doesn't jive with Gnome or OS X very well.

class dictproperty(object):
    """implements collection properties with dictionary-like access.
    Adapted from `Active State Recipe 440514:
    <http://code.activestate.com/recipes/440514-dictproperty-properties-for-dictionary-attributes/>`_
    """

is imho rather a copy of the code from there so please be nice and at least list original author/license (not sure yet how PSF would play with your BSD-2, probably making the whole project under PSF...)

insted of reading whole spike waveforms into memory. read them by chunks and copy to pytables/memmap array.

running browse_data.py example brings up the spike browser window, but with no buttons. clicking in the area where buttons should appear works!

this method is currently in SpikeDetector component. Applying this function results in updating all the observers, including ClusterAnalyzer, which vanishes all previous clustering results in the current session.

When running the tutorial I get the following exception:

Traceback (most recent call last):
  File "examples/sorting/cluster_beans.py", line 58, in <module>
    browser.show()
  File "/usr/local/lib/python2.7/dist-packages/SpikeSort-0.1-py2.7.egg/spike_beans/components.py", line 475, in show
    self._draw()
  File "/usr/local/lib/python2.7/dist-packages/SpikeSort-0.1-py2.7.egg/spike_beans/components.py", line 466, in _draw
    self._set_data()
  File "/usr/local/lib/python2.7/dist-packages/SpikeSort-0.1-py2.7.egg/spike_beans/components.py", line 488, in _set_data
    labels = self.label_src.labels
  File "/usr/local/lib/python2.7/dist-packages/SpikeSort-0.1-py2.7.egg/spike_beans/base.py", line 111, in __get__
    self.result = self.Request(obj)
  File "/usr/local/lib/python2.7/dist-packages/SpikeSort-0.1-py2.7.egg/spike_beans/base.py", line 130, in Request
    % (obj, self.feature)
AssertionError: The value <spike_beans.components.ClusterAnalyzer object at 0x39f2550> of 'LabelSource' does not match the specified criteria

Using SpikeBrowser() works fine. Not sure exactly what's going on here. Looks like the labels attribute of instances of the SpikeBrowser class doesn't exist, but the RequiredFeature class makes it a necessity to pass the (currently failing) assertion.

PyTables is very specialised dependency and it may be hard to install on some systems. It may increase the "entry level" for perspective user. Replacing PyTables with memmapped numpy should not affect performance too much, so pytables can be dropped in future versions.

User-defined classes should (ultimately) inherit from Python's root object object.

Make code conform to PEP8 style for easier reading.

if type(thresh) is str or type(thresh) is unicode:
    # ...

should be changed to

if isinstance(thresh, basestring):
    # ...

Also,

type(someInstance) is TypeObject

should be avoided.

I personally find it easier to grok code faster if the classes are separated into their own individual files (with related support functions if necessary). What I do in the case that they aren't in their own files is run the code and grep until I find what I'm looking for. So really, this issue could be called "Reduce the baseline amount of grep necessary to understand the code".

add functions for basic analysis of spike trains:

• PSTH
• ISIH
• auto- and cross-correlations
• raster plots

The names of the features which required by the components are currently hard coded, like so:

class SpikeDetector(base.Component):
    """Detect Spikes with alignment"""
    waveform_src = base.RequiredFeature("SignalSource",
                                        base.HasAttributes("signal"))

which imposes some limitations on how those components may be interconnected. For example, if one wants to use different filter for spike extraction, it will not be possible without inheriting the current SpikeExtractor and modifying the code.

One possible way to overcome this limitation without change in the API:

class SpikeDetector(base.Component):
    """Detect Spikes with alignment"""
    def __init__(self, ... , waveform_src = "SignalSource"):
        self.waveform_src = base.RequiredFeature(waveform_src,
                                             base.HasAttributes("signal"))

use numpy docs convention for docstrings: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt#docstring-standard

Dashboard can sometimes throw an exception when:

1. call dash.show(some_cell)
2. change the threshold
3. update SpikeDetector

components.py, line 665:
UnboundLocalError: local variable 'spt' referenced before assignment

add LICENSE file to the root directory

It would be nice to have electrode labels on the spike browsing plot, since it is fairly common for electrodes to not be linearly ordered on their respective shanks, e.g., the first shank (going from left to right) from top to bottom is numbered 1, 3, 2, 6. It would be nice to see these numbers in the spike browsing plot.

Just like "+" and "-" allow scaling of the Y-axis, it would be great if e.g., "Shift-+" and "Shift--" scaled the X-axis. That way one wouldn't have to restart the GUI every time one wanted to see a different time scale.

cdist does the same thing as _metric_euclidean in cluster.py and has a more efficient C implementation. It also has different distance metrics should the need for that ever arise in the future.

A quick benchmark using IPython's timeit functionality reveals the following:

def euc(a, b):
    m, n = a.shape
    p, k = b.shape
    if n != k:
        raise TypeError('a and b must have the same number of columns')
    delta = np.zeros((m, p))
    for d in xrange(n):
        delta += np.subtract.outer(a[:, d], b[:, d]) ** 2
    return np.sqrt(delta)

from scipy.spatial.distance import cdist
a = randn(100, 100)
b = randn(100, 100)
timeit euc(a, b) # 100 loops, best of 3: 7.72 ms per loop
timeit cdist(a, b) # 1000 loops, best of 3: 1.45 ms per loop

Some of the code in, for example, src/spike_sort/core/evaluate.py uses peak_to_peak = avg_spike.max()-avg_spike.min(). NumPy has such functionality built in as the ptp function.

Support to new dataformats can be added via the neo library.

on clean debian system unit test complain about missing modules:

patterns

Remove the import.

fix spike_sort.core.extract.remove_doubles()

It's getting no so easy to get matplotlib 1.0.1 running with new libraries. In particular, it's already impossible to compile it against libpng>=1.1 and new tk module in python 2.7.2.

I've generated couple of patches based on the new mpl releases, so we can probably mention them in the documentation. Faced this problem in rolling-release distro (Arch).

Patches are here: https://gist.github.com/2294220

Some weird plotting artefats can be observed when using spikesort with matplotlib 1.1.0. Among those: absence of text in the Legend plot, dark (almost black) colored feature plots

A little background just in case. With the advent of OOP came one of its most useful behaviors: polymorphism. This allows for repetitive code to be abstracted into a base class from which related classes inherit and can (or must) override one or more methods. In this case, many of the filtering classes perform the same action of checking the coefficient cache and then updating it if necessary. This allows one to create a method, call it m, that calls the method that must be overloaded, call it om. Since m is inherited by all subclasses, a given subclass, call it Sub, can call m and m will know to call the om method of Sub. I've written this code, but I need to write a test (and pass it!) before pushing it.

cluster_manual takes forever to run and does not show any windows

highlighted spikes should come up in the foreground

mask = features.get('is_valid')
if mask is not None:

    valid_data = data[mask,:]    # <-- here it fails

    cl = cluster_func(valid_data, *args, **kwargs)
    labels = np.zeros(data.shape[0], dtype='int')-1

'mask' array is full-lengt and 'data' is truncated by ClusterAnalyzer to hold features only for selected cell

is_masked is now counterintuitive - the name suggests that it is True for masked (invalid) spike, but in fact it is currently False for invalid and True for valid spikes.

Documentation says it is the correct (intuitive) way, which is obviously wrong!

It would be extremely useful to be able to see the detected spikes highlighted on the "features" plot simultaneously as one is scrolling through them in the spike browser.

extract_spikes and align_spikes are brutally slow right now. I think it would be useful to attempt to speed up the loops in these functions using Cython. Of course, if there's something I'm missing about how to make this faster using, e.g., PyTables (somewhat) fast I/O then please let me know.

Around line 305 in src/spike_sort/core/extract.py the following code spits out an exception:

spWave[:, i, :] = sp_data[contacts, sp + win[0]:sp + win[1]].T

The reason is because spWave[:, i, :] has dimension (n, 1) and sp_data[contacts, sp + win[0]:sp + win[1]].T has dimension (n,). My version of Numpy doesn't allow such arrays to be assigned to one another.

I'll submit a fix for this: change sp_data[...].T to np.atleast_2d(sp_data[...]).T. This adds a dimension to any 0 or 1D arrays to make them into a 2D array and does nothing for arrays with ndim >= 2.

This is a VERY rough idea, but I've run in to an issue where I need to threshold each channel separately (we have many recordings that are not performed with tetrodes). It might be nice to have a parameter that allows one to run an arbitrary Python callable that conforms to the interface of detect_spikes.

I often find myself spending quite some time scrolling around in the SpikeBrowser to locate a cell which has few spikes. It would be nice to have a kind of selective browsing. One possible way is to have an additional attribute, something like:

>>> browser.browse_lablel
'all'
>>> browser.browse_label=1

Pressing the next button will then scroll around the cells with label=1 only.

Spinx 1.1 complains about not being able to import *_src attributes of SpikeBeans components. These attributes are instatiated on runtime and they give an exception when the source was not defined.

Solution:

make __src private by prefixing them with underscore, that is: ___src

plotting large number of spikes >10000 by PlotSpikes component takes a couple of seconds and significantly affects spike sorting experience

as we agreed:

• push to dev until the next release is ready (creating branches for features and large fixes if needed)
• merge dev to master
• tag the release

• read documentation and make it up-to-date
• check spelling mistakes
• describe the datastructures

highlighted spikes should come up in the foreground

The API changes are:

1. There's no f_filter argument in GenericSource anymore
2. All filtering stuff is moved from core.extract.py to core.filters.py