Compiler error (illegal read) on tuple unpacking in a `for` loop about nim

[juancarlospaco]

!nim c

iterator it(): bool = discard
for (_) in it(): discard

[metagn]

As I've mentioned in the issue, it doesn't matter if the returned type is a tuple or not

My bad, didn't see.

There is a separate bug simplified from that example on 2.0 independent from tuple unpacking (probably same as #21476, same error, same workaround):

proc main =
  let x = false
  iterator it(): bool =
    discard x

  for a in it():
    discard
main()

which is obtained by removing the parentheses from that example, so the problem does still seem to be with the parentheses.

[github-actions]

🐧 Linux bisect by @juancarlospaco (collaborator)

devel 👎 FAIL

Output

Error: Command failed: nim c --run  -d:nimDebug -d:nimDebugDlOpen -d:ssl -d:nimDisableCertificateValidation --forceBuild:on --colors:off --verbosity:0 --hints:off --lineTrace:off --nimcache:/home/runner/work/Nim/Nim --out:/home/runner/work/Nim/Nim/temp /home/runner/work/Nim/Nim/temp.nim
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

IR

Compiled filesize 0 bytes (0 bytes)

Stats

• Started 2024-01-07T23:04:15
• Finished 2024-01-07T23:04:15
• Duration

AST

nnkStmtList.newTree(
  nnkIteratorDef.newTree(
    newIdentNode("it"),
    newEmptyNode(),
    newEmptyNode(),
    nnkFormalParams.newTree(
      newIdentNode("bool")
    ),
    newEmptyNode(),
    newEmptyNode(),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  ),
  nnkForStmt.newTree(
    nnkVarTuple.newTree(
      newIdentNode("_"),
      newEmptyNode()
    ),
    nnkCall.newTree(
      newIdentNode("it")
    ),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  )
)

stable 👎 FAIL

Output

Error: Command failed: nim c --run  -d:nimDebug -d:nimDebugDlOpen -d:ssl -d:nimDisableCertificateValidation --forceBuild:on --colors:off --verbosity:0 --hints:off --lineTrace:off --nimcache:/home/runner/work/Nim/Nim --out:/home/runner/work/Nim/Nim/temp /home/runner/work/Nim/Nim/temp.nim
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

IR

Compiled filesize 0 bytes (0 bytes)

Stats

• Started 2024-01-07T23:04:16
• Finished 2024-01-07T23:04:17
• Duration

AST

nnkStmtList.newTree(
  nnkIteratorDef.newTree(
    newIdentNode("it"),
    newEmptyNode(),
    newEmptyNode(),
    nnkFormalParams.newTree(
      newIdentNode("bool")
    ),
    newEmptyNode(),
    newEmptyNode(),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  ),
  nnkForStmt.newTree(
    nnkVarTuple.newTree(
      newIdentNode("_"),
      newEmptyNode()
    ),
    nnkCall.newTree(
      newIdentNode("it")
    ),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  )
)

2.0.0 👎 FAIL

Output

Error: Command failed: nim c --run  -d:nimDebug -d:nimDebugDlOpen -d:ssl -d:nimDisableCertificateValidation --forceBuild:on --colors:off --verbosity:0 --hints:off --lineTrace:off --nimcache:/home/runner/work/Nim/Nim --out:/home/runner/work/Nim/Nim/temp /home/runner/work/Nim/Nim/temp.nim
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

IR

Compiled filesize 0 bytes (0 bytes)

Stats

• Started 2024-01-07T23:04:20
• Finished 2024-01-07T23:04:20
• Duration

AST

nnkStmtList.newTree(
  nnkIteratorDef.newTree(
    newIdentNode("it"),
    newEmptyNode(),
    newEmptyNode(),
    nnkFormalParams.newTree(
      newIdentNode("bool")
    ),
    newEmptyNode(),
    newEmptyNode(),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  ),
  nnkForStmt.newTree(
    nnkVarTuple.newTree(
      newIdentNode("_"),
      newEmptyNode()
    ),
    nnkCall.newTree(
      newIdentNode("it")
    ),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  )
)

1.6.14 👎 FAIL

Output

Error: Command failed: nim c --run  -d:nimDebug -d:nimDebugDlOpen -d:ssl -d:nimDisableCertificateValidation --forceBuild:on --colors:off --verbosity:0 --hints:off --lineTrace:off --nimcache:/home/runner/work/Nim/Nim --out:/home/runner/work/Nim/Nim/temp /home/runner/work/Nim/Nim/temp.nim
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

IR

Compiled filesize 0 bytes (0 bytes)

Stats

• Started 2024-01-07T23:04:23
• Finished 2024-01-07T23:04:23
• Duration

AST

nnkStmtList.newTree(
  nnkIteratorDef.newTree(
    newIdentNode("it"),
    newEmptyNode(),
    newEmptyNode(),
    nnkFormalParams.newTree(
      newIdentNode("bool")
    ),
    newEmptyNode(),
    newEmptyNode(),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  ),
  nnkForStmt.newTree(
    nnkVarTuple.newTree(
      newIdentNode("_"),
      newEmptyNode()
    ),
    nnkCall.newTree(
      newIdentNode("it")
    ),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  )
)

1.4.8 👎 FAIL

Output

Error: Command failed: nim c --run  -d:nimDebug -d:nimDebugDlOpen -d:ssl -d:nimDisableCertificateValidation --forceBuild:on --colors:off --verbosity:0 --hints:off --lineTrace:off --nimcache:/home/runner/work/Nim/Nim --out:/home/runner/work/Nim/Nim/temp /home/runner/work/Nim/Nim/temp.nim
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

IR

Compiled filesize 0 bytes (0 bytes)

Stats

• Started 2024-01-07T23:04:25
• Finished 2024-01-07T23:04:25
• Duration

AST

nnkStmtList.newTree(
  nnkIteratorDef.newTree(
    newIdentNode("it"),
    newEmptyNode(),
    newEmptyNode(),
    nnkFormalParams.newTree(
      newIdentNode("bool")
    ),
    newEmptyNode(),
    newEmptyNode(),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  ),
  nnkForStmt.newTree(
    nnkVarTuple.newTree(
      newIdentNode("_"),
      newEmptyNode()
    ),
    nnkCall.newTree(
      newIdentNode("it")
    ),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  )
)

1.2.18 👎 FAIL

Output

Error: Command failed: nim c --run  -d:nimDebug -d:nimDebugDlOpen -d:ssl -d:nimDisableCertificateValidation --forceBuild:on --colors:off --verbosity:0 --hints:off --lineTrace:off --nimcache:/home/runner/work/Nim/Nim --out:/home/runner/work/Nim/Nim/temp /home/runner/work/Nim/Nim/temp.nim
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

IR

Compiled filesize 0 bytes (0 bytes)

Stats

• Started 2024-01-07T23:04:27
• Finished 2024-01-07T23:04:27
• Duration

AST

nnkStmtList.newTree(
  nnkIteratorDef.newTree(
    newIdentNode("it"),
    newEmptyNode(),
    newEmptyNode(),
    nnkFormalParams.newTree(
      newIdentNode("bool")
    ),
    newEmptyNode(),
    newEmptyNode(),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  ),
  nnkForStmt.newTree(
    nnkVarTuple.newTree(
      newIdentNode("_"),
      newEmptyNode()
    ),
    nnkCall.newTree(
      newIdentNode("it")
    ),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  )
)

1.0.10 👎 FAIL

Output

Error: Command failed: nim c --run  -d:nimDebug -d:nimDebugDlOpen -d:ssl -d:nimDisableCertificateValidation --forceBuild:on --colors:off --verbosity:0 --hints:off --lineTrace:off --nimcache:/home/runner/work/Nim/Nim --out:/home/runner/work/Nim/Nim/temp /home/runner/work/Nim/Nim/temp.nim
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

IR

Compiled filesize 0 bytes (0 bytes)

Stats

• Started 2024-01-07T23:04:30
• Finished 2024-01-07T23:04:30
• Duration

AST

nnkStmtList.newTree(
  nnkIteratorDef.newTree(
    newIdentNode("it"),
    newEmptyNode(),
    newEmptyNode(),
    nnkFormalParams.newTree(
      newIdentNode("bool")
    ),
    newEmptyNode(),
    newEmptyNode(),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  ),
  nnkForStmt.newTree(
    nnkVarTuple.newTree(
      newIdentNode("_"),
      newEmptyNode()
    ),
    nnkCall.newTree(
      newIdentNode("it")
    ),
    nnkStmtList.newTree(
      nnkDiscardStmt.newTree(
        newEmptyNode()
      )
    )
  )
)

Stats

• GCC 11.4.0
• Clang 14.0.0
• NodeJS 19.0
• Created 2024-01-07T23:03:48Z
• Comments 1
• Commands nim c --run -d:nimDebug -d:nimDebugDlOpen -d:ssl -d:nimDisableCertificateValidation --forceBuild:on --colors:off --verbosity:0 --hints:off --lineTrace:off --nimcache:/home/runner/work/Nim/Nim --out:/home/runner/work/Nim/Nim/temp /home/runner/work/Nim/Nim/temp.nim

🤖 Bug found in 18 minutes bisecting 7 commits at 0 commits per second

[metagn]

Putting parentheses makes it unpack tuples in for loops (old but broken and undocumented behavior, there are other issues open because of this: #19805, #15924). Since bool is not a tuple type, the compiler crashes (I surmise). So we need to check that the type is a tuple. (As written below, this is not the cause, but the issue does still seem to be with the parentheses) The for loop tuple unpacking code probably needs cleaning up, it wasn't touched during the recent tuple unpacking refactors

[ZoomRmc]

Putting parentheses makes it unpack tuples in for loops (old but broken and undocumented behavior, there are other issues open because of this: #19805, #15924). Since bool is not a tuple type, the compiler crashes (I surmise). So we need to check that the type is a tuple. The for loop tuple unpacking code probably needs cleaning up, it wasn't touched during the recent tuple unpacking refactors

As I've mentioned in the issue, it doesn't matter if the returned type is a tuple or not, it still crashes. I just minimized the code as much as I possibly could.

proc main() =
  let x = false
  iterator it(): (bool, bool) =
    if x: discard
    discard

  for (_, _) in it():
    discard

main()

On the second thought, to crash with the actual tuple we need to access a captured local variable. So it's might be a separate bug :)

[ringabout]

I think the first example is invalid code, just as

let (a) = false

gives

Error: tuple expected for tuple unpacking, but got 'int'

[metagn]

There is still a bug like #21476 for parentheses but doesn't work with the workaround for #21476. Maybe it is still the same issue though.

proc main() =
  let x = false
  iterator it(): (bool, bool) {.closure.} = # normally {.closure.} here makes #21476 work
    discard x

  for (_, _) in it():
    discard

main()

Output is segfault.

[ringabout]

Putting parentheses makes it unpack tuples in for loops (old but broken and undocumented behavior, there are other issues open because of this:

It's quite weird that the tuple unpacking works for nested arrays in the for loop. I didn't know this feature/bug before. It is quite inconsistent with var tuples unpacking.

[metagn]

I think the purpose is to prevent an implicit pairs iterator while allowing items but we need to make sure this is the only difference and document it

[ringabout]

I checked #19805. It seems to me that it happened to compile because there is no restriction on the tuple types for tuple unpacking. Though I will wait for #23185 to see whether it breaks something.

errWrongNumberOfVariables is cute though since it doesn't work for array types (iterAfterVarLent.len is always 2), it's reasonable to suppose there would have been a check for tuple types.

if n[0].len-1 != iterAfterVarLent.len:
  return localErrorNode(c, n, n[0].info, errWrongNumberOfVariables)

I think it's most likely unintentional. The cases in the #19805 failed twice to get itself compiled.

[ringabout]

I think the example two is the same case as #15924 because there is no var tuple unpacking for closures

[ringabout]

There is still a bug like #21476 for parentheses but doesn't work with the workaround for #21476. Maybe it is still the same issue though.

proc main() =
  let x = false
  iterator it(): (bool, bool) {.closure.} = # normally {.closure.} here makes #21476 work
    discard x

  for (_, _) in it():
    discard

main()

Output is segfault.

fixed by #23205

[ZoomRmc]

fixed by #23205

proc main() =
  let x = false
  iterator it(): (bool, bool) =
    if x: discard
    discard
  for (_, _) in it(): discard

main()

> Error: internal error: proc has no result symbol

[metagn]

That should be the same issue as #21476, as long as it doesn't matter if we use (_, _) or just _. Still worth mentioning in #21476 if it is the same issue though since the original example in #21476 is a bit misleading.