I think we ought to keep in line with the PEP8 style guidelines.

Commits 4149aff, 1a4444b, ff5a1c8 and b709dca fix most of these issues.

I had to make some breaking changes in order to do this, mostly with the names of some functions and some optional arguments. Before, we were using camelCase, now everything (I think) uses underscore_case.

speaking of which, we should have a changelog

Let's use the __all__ attribute to define what functions each module exports (see http://stackoverflow.com/questions/44834/can-someone-explain-all-in-python for more info)

Then we don't have to put underscores before all of our numpy/scipy calls 😄

spiketools.estfr does not divide by the binsize, meaning the estimated firing rate is not actually in Hz. This should be included.

in estnln, (line 157) is computing the conditional probability of the linear prediction given a spike. It is not taking into consideration the firing rate. If two (or more) spikes fall in the same bin they will be contributing as a single spike to the conditional distribution.

as a toy example consider an on/off cell that fires one spike when the stim is on and two (whithin 1ms) when the stim is off. The NL should look u shaped with the slope in the positive side being twice that in the negative side. Current code will make both slopes identical.

Returning tax for every getsta is very redundant. I added a flag to decide what gets returned. By default returns both sta and tax as before.

absolute or relative imports ❓

May want to add try - except block around call to np.linalg.svd in lowranksta, which may raise a LinAlgError if the SVD never actually converges. This should be handled gracefully.

photodiode.py, stimulusTools.py, and subspaceTools.py are left over from the initial merge of Niru's code. Most of the functions within are probably specific to his stuff, but just go through and find a new home for anything that seems generally useful, and then git rm the files.

(In particular, subspaceAngles seems moderately useful in general)

stimulustools.getcov uses an iterative procedure to estimate the covariance matrix of the given stimulus. Currently, this prints out the progress of this computation. We should have a flag, defaulting to False, that allows the caller to set this behavior.

• Spatial RF should use a symmetric color range and have white as the color for the median value (0).
• clean up plotcells function, remove blue background and add option for border around cells
• remove colorbar from playsta

A list of files with duplicate code / functionality:

• raster and sta visualization code in viz.py and visualizations.py
• estfr in spktools.py and computePSTH in psth.py

I couldn't see others but surely they exist?

Remove loadspikes function (all data storage should be specific to an individual user). Move psth and raster to viz module.

new usage will be:

stim_us, time_us = upsamplestim(stim, upfact, time=None)
stim_ds, time_ds = downsamplestim(stim, downfact, time=None)

if time is None, a value of None for time_us / time_ds will be returned

we need some

(uploading these here to add to README.md)

I think docstrings should somehow be clear about how data is being returned (as lists, dicts, numpy arrays, etc.)

For example, readbin returns data - how is it shaped?? - unclear from the docstring

I think this should not be a hidden function, since sometimes you might want to get out the center of the receptive field, or the area, or angle.

Another way to do this would be to have additional wrapper functions getcenter and getangle or something ... thoughts?

Some functions return things as (output, time), and others as (time, output). This should be standardized to the extent possible and reasonable.

Typing out all of the underscores in imports (e.g. import numpy as _np) is getting to be an inconvenience. This post suggests that underscores (by convention) represent undocumented parts of the API, which doesn't really fit with how we are using them (for importing external libraries). Also, I haven't seen other projects use the import numpy as _np convention, they just use np, for example.

Therefore: I vote we remove underscores. thoughts?

Certain pyret functions do not support the use of HDF5 datasets instead of numpy arrays. For example, fitlertools.getsta takes in a stimulus that is a numpy array, but it performs an operation (reshape) that is valid on numpy arrays but not Datasets that are from the h5py module.

We should come up with a list of dependencies that pyret requires, that don't come with a base python installation.

So far I'm assuming it needs:

• matplotlib
• numpy
• scipy

I'd like to also add the following:

• seaborn (for visualization, colormaps, etc.)

In filtertools, we have three functions (getsta, getstc, getste) that are 90% the exact same. I'm thinking that we can reduce these into a single function that selects out the spike-triggered stimuli and into an iterable sequence and then collapsing the sequence down as necessary for the STA and STC, or returning the whole sequence for the STE.

sample data file can't be found.

Load experiment data

data = np.load('../sample_data/sample_expt.npz')
data.keys()

FileNotFoundError: [Errno 2] No such file or directory: '../sample_data/sample_expt.npz'

The filtertools.getste function includes the following

for idx, val in enumerate(nzhist):
    ste[idx, :, :] = hist[val] * cstim[:, val - filterlength : val]

The stimulus frames preceding a spike are multiplied by the actual number of spikes. This is useful because it allows us to simply average ste over the right axis to compute the STA. However, we never actually do this in the code.

Furthermore, this multiplication makes the resulting array not really the STE, which should probably just include each stimulus slice exactly once.

What should we do about this?

Create a Cell class, which holds basic functionality and data specific to each cell recorded in a retinal experiment.

Data:
ste, sta, spike times

Functions:
getste, getsta, plot (plots spatiotemporal RFs if they exist), raster, psth

Write a file describing the standard coding conventions for anyone wishing to contribute. How are docstrings formatted? In what order are arguments passed and returned?

TODOs for viz tools in visualizations.py:

• a plot_cells function which takes a list of cells and created a single spatial plot of 1 s.d. gaussian ellipses fit to RFs for those cells
• a plot_psth function which allowed for plotting the PSTH from different conditions / cells for comparison
• make visualizations compatible with different RF types (spatiotemporal, temporal, bars, etc.)
• add option to plot PSTH on top of spike rasters?
• give options for each function to customize labels / titles / legends

add STC function to filtertools / cell class

The new version of the pyret package all seems reasonable, but nothing has actually been tested yet. Make sure that it all actually works, and fix what doesn't.

I think it would be interesting to have a .pyret file with some setup variables that will change between users but are probably constant throughout what each one of us does.

I'm thinking paths, name conventions, etc.

I'm not sure exactly where this is happening, but it occurs when I call filtertools.lowranksta so it must be happening in lowranksta or in a function that lowranksta calls, (filterpeak and np.linalg.svd).

Here is the bug: after calling lowranksta with some STA as input, the function modifies that input and corrupts it. This only happens after the first time the function is called, which makes it a weird bug. If I call visualizations.plotsta (which uses filtertools.lowranksta) with the same input multiple times, I get different results:

First call:
[1]: viz.plotsta(sta)

Second call:
[2]: viz.plotsta(sta)

Fourth call:
[4]: viz.plotsta(sta)

You can see these corrupted values showing up as single pixels in the spatial plots.

I flipped them around, but assumed they're 3-dimensional. Need to use np.roll or np.take and only specify the last dimension.

The STE/STA arrays are computed such that the time of the spike is at the back of the array, but it seems more natural to have them at the start. Just flip the lil' buggers.

Currently, getsta in filtertools.py has the norm flag set to True by default. I think I would prefer if this was False by default, since I think the default should be normalize by the number of spikes. Thoughts?

Add a rolling_window function that takes an array and returns a view on the same array (no copying of data) such that the new view is made up of shifted copies of the original array.

This is useful for turning convolution with a filter into a dot product with the filter, without requiring any more space on disk.

psth and raster are still from Ben's old codebase, adapt them to the new style.

The functions decompse and lowranksta work just fine. But they often return spatial and temporal kernels with opposite sign. It should be pretty easy to ensure that the point of maximal deviation from the mean has the same sign as that corresponding point in the STA. The kernels are no longer true singular vectors, but I think that it's important. It could be an optional argument or something.

Currently the function binspikes only accepts a binWidth parameter. This is useful if you want to bin spikes into fixed width time bins. I think it would also be useful to have it accept a time vector and use that for binning spikes.

This could be a separate function, but that seems unnecessary..

Add a function to spiketools.py that takes a firing rate as an input and returns spikes drawn from a time-varying Poisson process with the given rate. Allow for the option of sampling multiple trials of this process.

nonlinearities.estnln now computes nonlinearities as the ratio of the spike-triggered and unconditional histograms of the stimulus projected onto a linear filter. It would be useful also have a way of computing the nonlinearity by comparing the stimulus projection and the actual firing rate. This is probably more standard in the physiology literature, while the ratio-of-histograms method is more accepted in the theory literature.

The two functions in binary.py, readbin and readbinhdr, are specific to the binary file types used in the Baccus lab. It seems like to make pyret truly lab independent, that we should keep these functions in a separate repository or location.

Thoughts? It isn't hard for those of us that need it to keep binary.py somewhere else on our python path.

1. spktools --> spiketools
2. lntools --> filtertools
3. viz --> visualizations
4. rbin --> binary

There are several bugs in the viz module, something about colorbars being NoneType. Not sure how that happened.

I'm going to reorder the assumed dimensions of the stimulus in the getste/getsta functions. I initially wanted time to be the first dimension, because this allows the fastste extension to do things more efficiently. But I've removed that extension (I found a bug), and so I'm going to revert to the the "standard" way of storing a stimulus array. They will now be of shape (space1, space2, time)

Currently, a data matrix is defined that is size (nsamples, nchannels). This is inefficient if the user requests a small number of channels via the chanlist argument.

The line should read:

data = _np.empty((hdr['nsamples'], len(chanlist)))

• estfr needs to use filtfilt instead of lfilter
• both binspikes and estfr try and access a variable called cell that does not exist
• returned bins should be centered, not left bins

I want to add a module, nonlinearities.py, that does the following:

1. Returns parameters for different parametric nonlinearities (e.g. sigmoids, exponential, soft thresholding functions) fit to noisy data, using lsqcurvefit and similar routines.
2. Fits nonlinearities listed above given two histograms (fit to the ratio of the two histograms)
3. Fit 1d nonlinearities given just a stimulus projected onto a linear filter and corresponding firing rates.

[I'll do this in a dev branch and keep it separate for now]

Lots of code overlap, pull Niru's better versions from spikeTriggeredAnalysis.py into filtertools.py. Make sure it stays kosher with the Cell class.

Write a template analysis script showing how the various modules might be used.