list index out of range error about twecoll

[lucahammer]

This happens when there is an empty line in the .dat file. I am not sure yet when they happen but a quick fix is to remove the empty line and start the fetch command again.

[jasonkgreen]

Thanks. This seems to have solved it.

[jdevoo]

Thank you @lucahammer ! I wonder how these empty lines come about... Any chance you can share the handle to reproduce this @jasonkgreen ?

[lucahammer]

For the first time I was able to reproduce the error. It's probably something in the location field. Example of an account where the error happens: https://twitter.com/Atomroflbomber

[lucahammer]

I am testing location = data['location'].strip() in line 399 as a fix. Didn't have the problem again, but will test it some more before submitting a PR. I wonder if there are cases where it doesn't work because the problematic characters are surrounded by the location text. Then it would be better to replace \n and \t.

[lucahammer]

As expected it wasn't enough. This account triggered another out of range error: https://twitter.com/luXus_Suizid.

I added the following as a workaround.
location = location.replace('\n', ' ') location = location.replace('\t', ' ')

[jdevoo]

cool - we can probably chain them into a single line... wonder if this needs to be applied to other attributes as well

[jdevoo]

made the change but left out the tab case.