triangulator memory corruption about manifold

[pca006132]

It seems that inactivePairs_ somehow contains an invalid pointer. Perhaps related to splice iterators: https://stackoverflow.com/questions/143156/splice-on-stdlist-and-iterator-invalidation/143165#143165

Edit: we got a cycle in inactivePairs_.

index d0affe3..c266358 100644
--- a/src/polygon/src/polygon.cpp
+++ b/src/polygon/src/polygon.cpp
@@ -88,10 +88,16 @@ class Monotones {
 
     if (precision_ < 0) precision_ = bound * kTolerance;
 
-    if (SweepForward()) return;
+    if (SweepForward()) {
+      std::cout << "sweepforward return" << std::endl;
+      return;
+    }
     Check();
 
-    if (SweepBack()) return;
+    if (SweepBack()) {
+      std::cout << "sweepback return" << std::endl;
+      return;
+    }
     Check();
   }
 
@@ -141,6 +147,27 @@ class Monotones {
   // A variety of sanity checks on the data structure. Expensive checks are only
   // performed if params.intermediateChecks = true.
   void Check() {
+    std::vector<PairItr> pairs;
+    for (PairItr pair = activePairs_.begin(); pair != activePairs_.end(); pair++) {
+      if (std::find(pairs.begin(), pairs.end(), pair) != pairs.end()) {
+        std::cerr << "cycle" << std::endl;
+        abort();
+      }
+      pairs.push_back(pair);
+    }
+    unsigned int num = pairs.size();
+    for (PairItr pair = inactivePairs_.begin(); pair != inactivePairs_.end(); pair++) {
+      auto iter = std::find(pairs.begin(), pairs.end(), pair);
+      if (iter != pairs.end()) {
+        if (std::distance(pairs.begin(), iter) >= num)
+          std::cerr << "inactivePairs_ cycle" << std::endl;
+        else
+          std::cerr << "duplicate" << std::endl;
+        abort();
+      }
+      pairs.push_back(pair);
+    }
+
     if (!params.intermediateChecks) return;
     std::vector<Halfedge> edges;
     for (VertItr vert = monotones_.begin(); vert != monotones_.end(); vert++) {
@@ -914,7 +941,9 @@ std::vector<glm::ivec3> Triangulate(const Polygons &polys, float precision) {
   std::vector<glm::ivec3> triangles;
   try {
     Monotones monotones(polys, precision);
+    monotones.Check();
     monotones.Triangulate(triangles);
+    monotones.Check();
     if (params.intermediateChecks) {
       CheckTopology(triangles, polys);
       CheckGeometry(triangles, polys, precision);

It seems that the cycle is formed after failed sweep back (return non-zero in sweep back).

[elalish]

Thanks for the analysis! I should be able to use that to spit out a test case polygon. Good sleuthing!

[elalish]

I got address sanitizer working, but your patch isn't printing anything for me. Is there anything else required to repro? Also, if you replace abort() with ALWAYS_ASSERT, then the triangulator will catch the exception and print the problem polygon, which is what we need to make a minimal test case.

[pca006132]

Did you change the max iteration to 10? The current one in master will not trigger the error.

[elalish]

Indeed I did; it fails, but on the munmap_chunk(): invalid pointer or double free or corruption (out) before any of your print statements hit.

[pca006132]

Weird, or maybe the cycle is formed due to memory corruption and compiling this with different options somehow changed the behavior? No idea.

[elalish]

Okay, I managed to dump out the polygon and create a simple repro test.