GithubHelp home page GithubHelp logo

contributor-tools's People

Contributors

jamesparrott avatar llllvvuu avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar

Forkers

jamesparrott

contributor-tools's Issues

Support unicode ๐Ÿ‘ on platforms with a non-unicode compliant default locale (e.g. Windows)

These look like great useful scripts - thanks @llllvvuu . Unfortunately as soon as it tried to write "๐Ÿ‘" to a file on Windows, pull_github_issues.py worked as well as I'd expect raw output from GPT 4 to work.

Luckily the fix for this one is simple (PR is incoming, explicit is better than implicit, add encoding = 'utf8' to all the calls to open).

INFO:root:Processing llllvvuu/contributor-tools
...
INFO:root:Processing rclone/rclone
Traceback (most recent call last):
  File "...\pull_github_issues.py", line 146, in <module>
    main()
  File "...\contributor-tools\github_issues\pull_github_issues.py", line 142, in main
    writer.writerows(all_issues)
  File "...\AppData\Local\Programs\Python\Python311\Lib\csv.py", line 157, in writerows
    return self.writer.writerows(map(self._dict_to_list, rowdicts))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\AppData\Local\Programs\Python\Python311\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f44d' in position 3785: character maps to <undefined>

Minimal example to reproduce the same error:

import string
import csv
import tempfile


thumbs_up = '\U0001f44d'


def test_chars():
    yield from string.printable
    yield thumbs_up


def write_chars_to_tmp_file(mode = 'w', chars_gen = test_chars, **kwargs):  
    with tempfile.TemporaryFile(mode = mode, **kwargs) as f:
        w = csv.writer(f)
        for c in chars_gen():
            w.writerow(c)


write_chars_to_tmp_file('w', encoding = 'utf8')

print(f'{thumbs_up=}')

write_chars_to_tmp_file('w')
py write_unicode_to_csv.py
thumbs_up='๐Ÿ‘'
Traceback (most recent call last):
  File "...\Coding\Python\write_unicode_to_csv.py", line 21, in <module>
    write_chars_to_tmp_file('w')
  File "...\Coding\Python\write_unicode_to_csv.py", line 15, in write_chars_to_tmp_file
    w.writerow(c)
  File "...\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 483, in func_wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "...\AppData\Local\Programs\Python\Python311\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f44d' in position 0: character maps to <undefined>

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.