damcb / tyssue Goto Github PK

from tyssue.collisions.solvers import CollidingBoxes
from tyssue.collisions import self_intersections
position_buffer = hdf5.load_datasets('../data/hf5/ellipsoid_sheet_init.hf5',
                                     data_names=['vert'])['vert'][list('xyz')]

specs = {'T1': {'critical_length': 0.3},
 'abc': [84.99999999999999, 84.99999999999999, 149.99999999999997],
 'delamination': {'basal_contract_rate': 1.001,
  'contract_neighbors': True,
  'contract_span': 3,
  'contraction_column': 'surface_tension',
  'critical_area': 5,
  'critical_area_neighbors': 12,
  #'geom': invagination.ellipsoid.EllipsoidGeometry,
  'nb_iteration': 0,
  'nb_iteration_max': 30,
  'radial_tension': 30},
 'geometry': 'cylindrical',
 'height_axis': 'z',
 'lumen_vol': 4450958.086576193,
 'nrj_norm_factor': 784,
 'threshold_length': 0.001,
 'vitelline_space': 0.2}

falty = hdf5.load_datasets('res', 'falty.hf5')
intersecting_edges = self_intersections(falty)
boxes = CollidingBoxes(sheet, position_buffer, intersecting_edges)

collisionSolverFailed.zip

Instead of generalizing available sheet geometries, as discussed in #29 , it might be more interesting to allow compound geometries, as for example a cylinder caped by two hemispheres.

This would allow us to construct complicated shapes while maintaining a relatively simple underlying structure.

First of all pandas is pure python, so I imagine it's just something like boost::python::object matter to retrieve column names, and them resorting to numpy for managing the data itself

We have two solutions here:

• https://github.com/ndarray/Boost.NumPy
  and:
• https://github.com/mdboom/numpy-boost

Here is the output of nose coverage as of sept. 1st 2016, ordered by priority:

Core, config and IO

• tyssue/core/objects.py 297 98 67%
• tyssue/core/sheet.py 39 27 31%
• tyssue/core/generation.py 250 34 86%
• tyssue/config/json_parser.py 23 1 96%
• tyssue/io/hdf5.py 22 10 55%

Geometry & topology

• tyssue/geometry/base_geometry.py 15 0 100%
• tyssue/geometry/planar_geometry.py 28 5 82%
• tyssue/geometry/sheet_geometry.py 63 3 95%
• tyssue/dynamics/base_gradients.py 8 1 88%
• tyssue/topology/sheet_topology.py 142 44 69%

Dynamics

• tyssue/dynamics/effectors.py 8 0 100%
• tyssue/dynamics/planar_gradients.py 23 20 13%
• tyssue/dynamics/planar_vertex_model.py 79 52 34%
• tyssue/dynamics/sheet_gradients.py 29 6 79%
• tyssue/dynamics/sheet_isotropic_model.py 72 34 53%
• tyssue/dynamics/sheet_vertex_model.py 68 7 90%

Miscelanous (low priority)

• tyssue/utils/utils.py 53 28 47%
• tyssue/core/multisheet.py 53 6 89%
• tyssue/geometry/multisheetgeometry.py 19 0 100%
• tyssue/solvers/sheet_vertex_solver.py 38 4 89%
• tyssue/core/monolayer.py 0%

There is some problem running Cell_Division.ipynb.

The specific problem occurs for Division in 3D Epithelium, when the cell
division is undertaken as per In [24]:

http://tyssue.readthedocs.io/en/latest/notebooks/Cell_Division.html#Division-in-a-3D-single-layer-epithelium

This gives error with the following traceback:

TypeError Traceback (most recent call last)
in ()
1 mother = 8
2 daughter = cell_division(monolayer, mother,
----> 3 orientation='vertical')

~/anaconda3/envs/fenty/lib/python3.6/site-packages/tyssue-0.2rc1-py3.6.egg/tyssue/topology/monolayer_topology.py in cell_division(monolayer, mother, orientation, psi)
63
64 daughter = bulk_division(monolayer, mother,
---> 65 MonoLayerGeometry, vertices)
66
67 # Correct segment assignations for the septum

~/anaconda3/envs/fenty/lib/python3.6/site-packages/tyssue-0.2rc1-py3.6.egg/tyssue/utils/decorators.py in with_bckup(*args, **kwargs)
7 def with_bckup(*args, **kwargs):
8 eptm = args[0]
----> 9 eptm.backup()
10 try:
11 return func(*args, **kwargs)

~/anaconda3/envs/fenty/lib/python3.6/site-packages/tyssue-0.2rc1-py3.6.egg/tyssue/core/objects.py in backup(self)
184 """
185 log.info('Backing up')
--> 186 self._backups.append(self.copy(deep_copy=True))
187
188 def restore(self):

~/anaconda3/envs/fenty/lib/python3.6/site-packages/tyssue-0.2rc1-py3.6.egg/tyssue/core/objects.py in copy(self, deep_copy)
175 identifier = self.identifier+'_copy'
176 new = type(self)(identifier, datasets,
--> 177 specs=self.specs, coords=None)
178 return new
179

TypeError: init() got an unexpected keyword argument 'coords'

Got this error after commit @ 49939cb :

make test
cd tyssue/ && py.test && cd ..
Fatal Python error: Py_Initialize: can't initialize sys standard streams
AttributeError: module 'io' has no attribute 'OpenWrapper'

It appears to be due to the I/O module being named tyssue.io

http://grokbase.com/t/python/docs/144bbs5j58/issue21202-naming-a-file%60-io-py%60-causes-cryptic-error-message

Move test up to tyssue to solve that... see #48, but then coveralls fails

this is actually data_dicts_sheet

remove third argument in return triangular_mesh.

Not sure if this is practical...

I think this is what's done in isotropic_relax, should be generalized, and made available front-end: height is scaled by a certain amount and so is the area. Might be nice to work in non-dimentionalized space all the time.

Simply:

find $\sigma_o$ such that $E(\sigma_o) = \min E(\sigma)$ where $\sigma$ is an isotropic scale factor.

I am unable to use the area_grad function from tyssue.dynamics.sheet_gradients with an AnnularSheet created with tyssue.generation.generate_ring.
The error is: 'AnnularSheet' object has no attribute 'ncoords'
Attached notebook demonstrates the error.
AnnularSheet_ncoords_missing.ipynb.tar.gz

This code should go in utils with tests and all - saving for later

def face_centered_patch(sheet, face, neighbour_order):

    faces = pd.Series(face).append(
        sheet.get_neighborhood(face, order=neighbour_order)['face'])
    
    edges = sheet.edge_df[sheet.edge_df['face'].isin(faces)]

    vertices = sheet.vert_df.loc[set(edges['srce'])]
    pos = vertices[sheet.coords].values - vertices[sheet.coords].mean(axis=0).values[None, :]
    u, v, rotation = np.linalg.svd(pos, full_matrices=False)
    rot_pos = pd.DataFrame(np.dot(pos, rotation.T), 
                           index=vertices.index,
                           columns=sheet.coords)

    patch_dset = {'vert': rot_pos,
                  'face': sheet.face_df.loc[faces].copy(),
                  'edge': edges.copy()}

    patch = Sheet('patch', patch_dset, sheet.specs)
    patch.reset_index()
    return patch

This version of draw face doesn't color face according to a condition like area. It seems to color face randomly.

def draw_face(sheet, coords, ax, **draw_spec_kw):
    """Draws epithelial sheet polygonal faces in matplotlib
    Keyword values can be specified at the element
    level as columns of the sheet.face_df
    """

    draw_spec = sheet_spec()['face']
    draw_spec.update(**draw_spec_kw)
    collection_specs = parse_face_specs(draw_spec, sheet)

    if 'visible' in sheet.face_df.columns:
        edges = sheet.edge_df[sheet.upcast_face(sheet.face_df['visible'])].index
        sheet = get_sub_eptm(sheet, edges)
        color = collection_specs['facecolors']
        if isinstance(color, np.ndarray):
            faces = sheet.face_df['face_o'].values.astype(np.uint32)
            collection_specs['facecolors'] = color.take(faces, axis=0)

    polys = sheet.face_polygons(coords)
    polys = polys.reindex(sheet.face_df.sort_values('z').index)
    p = PolyCollection(polys, closed=True, **collection_specs)
    ax.add_collection(p)
    return ax

Color face according to a condition works find with the oldest version of draw_face :

def draw_face(sheet, coords, ax, **draw_spec_kw):
    """Draws epithelial sheet polygonal faces in matplotlib
    Keyword values can be specified at the element
    level as columns of the sheet.face_df
    """
    draw_spec = sheet_spec()['face']
    draw_spec.update(**draw_spec_kw)
    polys = sheet.face_polygons(coords)
    patches = []
    for idx, poly in polys.items():
        patch = Polygon(poly,
                        fill=True,
                        closed=True)
        patches.append(patch)
    collection_specs = parse_face_specs(draw_spec, sheet)
    p = PatchCollection(patches, False,
                        **collection_specs)
    ax.add_collection(p)
    return ax

List of formats to be implemented

• JSON for the configuration
• HDF5 - adapt hdfgraph
• Wavefront .OBJ (blender I/O)
• CSV - on the fly through pandas

Re-targeted to 0.2

• OFF format (LCC/Polyhedron_3)
• ais format (LCC)

less likely to be implemented

• NetCDF (?)
• STL (optional, but would be fun)

• readthedocs hook http://tyssue.readthedocs.io
• index.rst
• Geometries examples doc/notebooks/Geometries_showcase.ipynb
• Make coverage work, document API

Where necessary, i.e for topology dependant indexing elements

https://pypi.python.org/pypi/cached-property

then complete reset_topo by flushing (del the property in self.__dict__ like indicated in the examples)

Situation at the 0.1 version

The elastic area term of the potential is taken as $\propto$ the face area. It is both more judicious and stable to put the per cell total area.

mono_apical_cut = mono.copy(deep_copy=True)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-45-d382c3a9d92c> in <module>()
----> 1 mono_apical_cut = mono.copy(deep_copy=True)

~/miniconda3/envs/tyssue/lib/python3.6/site-packages/tyssue-0.2rc1-py3.6.egg/tyssue/core/objects.py in copy(self, deep_copy)
    175         identifier = self.identifier+'_copy'
    176         new = type(self)(identifier, datasets,
--> 177                          specs=self.specs, coords=None)
    178         return new
    179 

TypeError: __init__() got an unexpected keyword argument 'coords'

Should be a quick fix, no time right now

Distinct get edge opposite for 2D and 3D sheet.

When the monolayer is created, there is no value put in the num_sides column in face_df.
Fix it temporarily by adding this line mono.update_num_sides() when created the monolayer in the notebook.

specs = config.geometry.bulk_spec()
sheet = Sheet.planar_sheet_3d('flat', 6, 8, 1, 1)
sheet.sanitize()
datasets = extrude(sheet.datasets, method='translation')
mono = Monolayer('mono', datasets, specs)
geom.center(mono)
geom.update_all(mono)
dyn_specs = config.dynamics.quasistatic_bulk_spec()
mono.update_specs(dyn_specs, reset=True)
mono.update_num_sides()
geom.update_all(mono)
res = Solver.find_energy_min(mono, geom, model)

For now we need root privileges just to install the doc in /usr/local, that should change

Hi @glyg

I'm getting a different error with the fix, could you look into this:

C:\Users\siang\Anaconda2\envs\py3k\lib\site-packages\tyssue-0.2.0-py3.5.egg\tyssue\core\objects.py in get_extra_indices(self)
493
494 if not 'opposite' in self.edge_df.columns:
--> 495 self.get_opposite()
496
497 self.dble_edges = self.edge_df[self.edge_df['opposite'] >= 0].index

AttributeError: 'Sheet' object has no attribute 'get_opposite'

Living on the bleeding edge, what can I say...

@hadim any idea how to do that? I tried

find_package(CGAL 4.6 REQUIRED)

But Travis was not ok with that (and it had no apparent effect on my box)

I'm not satisfied with the current handling of this variable:

    @staticmethod
    def update_height(sheet):
        """
        Update the height of the sheet vertices, based on the geometry
        specified in the sheet settings:

        `sheet.settings['geometry']` can be set to

          - `cylindrical`: the vertex height is
             measured with respect to the distance to the the axis
             specified in sheet.settings['height_axis'] (e.g `z`)
          - `flat`: the vertex height is
             measured with respect to the position on the axis
             specified in sheet.settings['height_axis']
          - 'spherical': the vertex height is measured with respect to its
             distance to the coordinate system centers

        In all the cases, this distance is shifted by an amount of
        `sheet.vert_df['basal_shift']`
        """
        w = sheet.settings['height_axis']
        u, v = (c for c in sheet.coords if c != w)
        if sheet.settings['geometry'] == 'cylindrical':
            sheet.vert_df['rho'] = np.hypot(sheet.vert_df[v],
                                            sheet.vert_df[u])
            sheet.vert_df['height'] = (sheet.vert_df['rho'] -
                                       sheet.vert_df['basal_shift'])

        elif sheet.settings['geometry'] == 'flat':
            sheet.vert_df['rho'] = sheet.vert_df[w]
            sheet.vert_df['height'] = (sheet.vert_df[w] -
                                       sheet.vert_df['basal_shift'])

        elif sheet.settings['geometry'] == 'spherical':
            sheet.vert_df['rho'] = np.linalg.norm(sheet.vert_df[sheet.coords],
                                                  axis=1)
            sheet.vert_df['height'] = (sheet.vert_df['rho'] -
                                       sheet.vert_df['basal_shift'])

It seems to me a more generic version could be useful by expanding the (height, basal_shift, height_axis) to a pair of 3D vectors, shifting all the vertices.

This might mean subclassing the geometries

see Okuda et al. 2014

import quantities as pq

ModuleNotFoundError: No module named 'quantities'

Can you help me?

Geometry from sets of points:

• The from_voronoi2d function is 2D in 2D only, so we need to investigate
  how to retrieve a Sheet dataset from a set of 2D data points (here the face centers) in 3D
• retrieve junctions graph from a clean segmented image (watershed) in 2D
  • generate mockups
  • see whether some style of 2D in 3D generalisation is accessible

Feature extraction from an image (SIFT/SURF like):

From Steven Leach on linkedin

If you want quicker response I suggest WillowGarage ORB 30 FPS

• https://www.willowgarage.com/sites/default/files/orb_final.pdf
• http://stackoverflow.com/questions/7232651/how-does-opencv-orb-feature-detector-work
• http://opencv-users.1802565.n2.nabble.com/how-to-use-the-ORB-descriptor-td6606424.html

This is a work in progress.

The current state of affairs can be seen in the following notebook:

http://nbviewer.jupyter.org/github/glyg/notebooks/blob/master/tyssue/Rheology/Equations.ipynb

Two questions:

• Why use only half the chord length, $a_{ij}$ in the equations? Although it simplifies some equations, there's a factor of two in most of them, and it is not very instinctive. Plus the library already computes the edge length...
• How to set correct initial conditions? I suspect my actual poor choice is responsible for not finding a solution to the differential equations. If the initial conditions violate the equations, the differential can't be solved, right?

Well, we can discuss that here...
cc @cypriengay (I don't have François github handle yet ;)

Main hurdle (in my view):

Having a single data representation (in the sense of the underlying data structure, I/O and view) for multiple physical models of the epithelium.

Components

Core (frontend) data:

Four pandas DataFrame objects:

• cell_df
• face_df
• vert_df
• edge_df

Those dataframes are contained in a datasets dictionary, with cell, face, vert and edge as respective keys.

Specification

The various options for a simulation are stored in a nested dictionary called specs, with cell, face, vert and edge and settings as keys

Core structure (or things that shouldn't move):

• Epithelium object (the whole mesh):
  • datasets -- A dictionary of the above mentioned dataframes.
  • specs -- A nested dictionary containing the specification and configuration options.

Note that specific parts of the epithelium, for example the various sheets (apical, basal or sagital) could be isolated or manipulated by the use of boolean marks, reminiscent of graph_tool's edge filters.

Events or Behaviors?

see comment bellow

Behaviors / signals

• Behavior base class
  • triggered by user, the simulation scenario or in response to a signal
  • produces Changes in Epithelium (such as a change in topology / state space / state vector/ attributes)
  • in quasi-static models, triggers a new optimization

Two classes of Changes can take place:

• AttributeChange : a change in the cells or junction attributes, for example:
  • CellGrowth / CellShrink for cells
  • TensionIncrease for junctions
• TopologicalChange: Changes the model topology, thus the space state.
  • CellDivision
  • T1Transition
  • CellEllimination: a cell is removed from the tissue, or a specific tissue sheet
• Signal
  • is sent by a Cell to it's neighbors (notion of propagation)
  • can trigger a behavior (such as a TensionIncrease event in neighbouring cells as happens in apoptotic cells).

Models

Models are implemented functionnaly, there are no class associated

Graphical representation

Here we'll use a functional approach. We plan on using both vispy and matplotlib

• EpitheliumView interfaces (or adapter?)
  • adapts an Epithelium object to be passed to viewing functions

Speaking of interfaces

Good modular design will require some care in the interplay between the various elements both in C++ and python. On that subject, @hadim pointed out this discussion on vispy's ML

Also of interest, this vispy issue

What's in Python, what's in C++ ?

The frontier might be moving, but basically:

The solver will likely be split up between python and C++, for example the computing of the gradient
could be done in C++, but the optimizer called by python, to profit from the ease of access to various algorithms in scipy.optimize

Where we are so far

This issue to start a discussion about ontologies, and ways to integrate our modelling efforts with the community:

http://bioinformatics.oxfordjournals.org/content/early/2014/05/18/bioinformatics.btu210.long

IDK how to do this, there are some discussion in the above paper. Comments / class attributes / other?

• [X ] Setup a proper compilation / distribution strategy
• Start implementing base classes
• Start writing tests
• implement CI

I am using the most recent version of tyssue (0.2.rc1.dev0+a69d5c1). However,
I am have some problems running the most basic code (Basic_Usage.ipynb).
Specifically, the problem occurs during the Energy Minimisation step:

res = Solver.find_energy_min(sheet, PlanarGeometry, PlanarModel)

The result of

print(res['success'])

is 'False'

Even more specifically, I can see that, at the end of the minimisation step,
the vertices at the boundaries do not move at all. This is quite unlike the
picture which I see in the Documentation:

http://tyssue.readthedocs.io/en/latest/notebooks/Basic_Usage.html#Easy-creation-of-a-2D-epithelial-sheet

after Energy Minimisation (In [11]:).

Am I doing something stupid? Could you please help?

BTW, over the last couple of months I have experimented quite a bit with
tyssue, and really like what you have done. I have also been successfully able
to run number of example Notebooks, and indeed understand the general
framework. That said, I have been having specific troubles in executing some
of the notebooks. So I am finally collecting my questions and planning to ask
them to you one-by-one -- the question above is being the first.

Thank you very much in advance for your help.
best,
mandar

falty_mono.zip

The view for now is that geometries should be described by simple primitives.

Simple primitives

Sheets (Polygon)

• cylindrical
• planar
• spherical
• elipsoidal -> won't implement for now, see discussion in #36
• implicit -> won't implement at all, see discussion in #36

Epithelia (LinearCellComplex)

Hey, those are self sustained, so no need for an explicit geometry here

While working on an optimization procedure I stumbled upon something weird.
My framework is as follow :

1. I initiate an AnnularSheet.
2. I create a copy of this AnnularSheet.
3. I call Solver.find_energy_min on the copy.
   After this step the lumen volume of the original AnnularSheet, stored in the settings dictionnary, is modified.
   The issue is demonstrated in the attached notebook.
   issue_lumen_vol.ipynb.tar.gz

the set_specs method doesn't do what it's supposed to do

see Okuda et al. 201

Implementation goes on here:

https://github.com/DamCB/tyssue-notebooks/blob/master/core/ReversibleNetworkReconnection.ipynb

I think the gnrl idea of specs is good, but not sure it is correctly updated

relevant:
http://stackoverflow.com/questions/3232943/update-value-of-a-nested-dictionary-of-varying-depth#3233356

Should be simple to fix
Code's here:
https://github.com/DamCB/tyssue/blob/master/src/tyssue/draw/plt_draw.py#L247

This is needed to be able to refer to a specific cell later on.

This is bad design:

nondim_specs = load_default('dynamics', 'sheet')

Because there's no way to discover the available default settings

Memo for obsolete / deprecated code to be marked as such or to be rewritten.

• core/objects.py : Epithelium.from_points (rewrite), Epithelium.set_specs (deprecated)
• config/json_parser.py : load_default (deprecated)

• Core data structures
• geometry and model classes
• topology
• timetables and events

They should only contain classmethods, but I think it would be easier to mix models (e.g. for a compound tissue)

This error appear randomly.

It append at this line :

> ~/anaconda3/lib/python3.6/site-packages/tyssue-0.2.2-py3.6.egg/tyssue/core/sheet.py in get_opposite(edge_df)
>     366     flipped.names = ['srce', 'trgt']
>     367 
> --> 368     opposite = st_indexed.reindex(flipped)['edge'].values
>     369     opposite[np.isnan(opposite)] = -1

KeyError: 'opposite' when calling get_extra_indices()

@glyg "You need to call sheet.get_opposite() before calling get_extra_indices() . Could you open an Issue reporting the bug? I'll fix it later today."

As this needs to be fast and called frequently, and it's not trivial, it's safe to assume it's better to use something already coded. CGAL has what we need:

https://doc.cgal.org/4.2/CGAL.CGAL.Intersecting-Sequences-of-dD-Iso-oriented-Boxes/html/index.html#chapterBoxIntersection

And an example with Polyhedron_3 objects (which are close to our Sheet geometry):

https://github.com/CGAL/cgal/blob/master/Polyhedron/examples/Polyhedron/polyhedron_self_intersection.cpp

So:

• 1. Create a CGAL::Polyhedron_3 object from the sheet face, srce, trgt coordinates.

• 2. Apply the intersection detection on this object and get the intersecting triangles back

• 3. Correct for the collision.

Note

The CGAL code is GPL licenced, so the collision detection module should be licenced accordingly

Add a "geometry": "ellipsoid" key to the settings and implement the corresponding height computation