Comments (8)
Ah. Got it. :)
So we'd definitely accept patches to improve Windows support but I'll let you know up front that the testing burden might be a little high, since none of us have easy access to Windows machines at the moment nor any expertise on developing/debugging Python on Windows issues.
I guess at a minimum we'll want to see some unit tests added if they don't already exist that test that 3 digit years are zero padded without %04D
on Windows and that they continue to be padded on Mac/Linux.
In terms of the general design I don't think duck typing is necessarily the wrong way to go here. First try with the Mac/Linux format string, fall back to the windows format string on an exception, otherwise raise.
Sound good?
from singer-python.
Pushed 5.3.3 to PyPI. Thanks! :)
from singer-python.
I agree that Python 3.7.1's datetime
module exhibits this behavior. What's the issue for singer-python
?
from singer-python.
@timvisher Isn't this a OS specific problem, not Python version? %04Y
is not supported on Windows.
Here is Python 3.5.2 on Windows,
$ "C:\Users\MLM\Downloads\python-3.5.2-embed-amd64\python.exe"
Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:18:55) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime
>>> datetime(90, 1, 1).strftime("%04Y")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: Invalid format string
Can we use something like this? Probably the wrong way to patch this 🤷♂️
Windows by default zero-pads year values with %Y
DATETIME_FMT_SAFE = "%Y-%m-%dT%H:%M:%S.%fZ"
def strftime(dtime, format_str=DATETIME_FMT):
if dtime.utcoffset() != datetime.timedelta(0):
raise Exception("datetime must be pegged at UTC tzoneinfo")
dt_str = None
try:
dt_str = dtime.strftime(format_str)
if dt_str.startswith('4Y'):
dt_str = dtime.strftime(DATETIME_FMT_SAFE)
except ValueError:
dt_str = dtime.strftime(DATETIME_FMT_SAFE)
return dt_str
from singer-python.
Sorry, I just actually wasn't sure what problem you were seeing.
So is the issue that you're identifying that utils.strftime
behaves badly on Windows?
from singer-python.
@timvisher Correct, I can't parse anything because %04Y
usage built into format string within this library just throws ValueError: Invalid format string
on Windows
from singer-python.
@timvisher I created #87
We already have tests for zero-padding a small year. The tests failed before and now pass on Windows.
from singer-python.
This is still an issue on my Windows machine. I fixed by updating DATETIME_FMT to be the same as DATETIME_FMT_MAC
DATETIME_FMT = "%Y-%m-%dT%H:%M:%S.%fZ"
from singer-python.
Related Issues (20)
- Incompatible taps & targets HOT 4
- Support for inline configuration strings HOT 6
- `log_debug` cannot work...for very long HOT 3
- Bump pytz version to >= 2018.9 HOT 1
- Ratelimit helper does not support multi-threading or async
- Schema fails to be turned into dictionary if it contains "anyOf"
- 5.9.0: Transformer.filter_data_by_metadata() doesn't filter unselected nodes where selected unspecified HOT 1
- Feature Request: Add support for --stream_name argument HOT 1
- Messages are not valid JSON
- Transformer dumps JSON incompatible string
- Async support
- JSONSchema Draft 7 array Tuple Validation unsupported? Schema.from_dict() raise exception HOT 1
- Add UTF-8 validity checking to schema
- Support Draft 7 validation keys in object schemas
- --catalog vs -p (properties) parameters HOT 4
- Loosen version spec in setup.py HOT 4
- `utils.strptime` and `utils.strftime` are asymmetrical HOT 7
- use singer taps and targets programatically HOT 5
- Is requirement for backoff==1.3.2 necessary HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from singer-python.