algorithmiaio / algorithmia-python Goto Github PK

Having a non-empty dir at this URL: data://asli/xgboost_iris_demo:

• First trying algo rmdir xgboost_iris_demo and I get no response, so as a user I'm not sure if the CLI deleted this dir or not. - So I go and check and see that it's still there. For this case, I would expect to get a message like "dir not found, you should specify it like this etc"
• Then I re-try with algo rmdir asli/xgboost_iris_demo and the CLI warns me. About this step, do we want to expose all our error trace to the end user? Shouldn't we just return a message to the user?

The example of the api docs for python looks like:
t800File = client.file("data://.my/robots/T-800.png").getFile()

but no matter how i try to save the response I keep getting errors.

if i try to read the response in order to write to a file, i get this error:

    with open("test.png", "w") as f:
        f.write(response.read())

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

if i try to save the response directly, i get this:

    with open("test.png", "w") as f:
        f.write(response)

TypeError: write() argument must be str, not _io.TextIOWrapper

Any help is appreciated.

Hi Algorithmia,

It would appear most exceptions within this python client are the general Exception class:

$ grep -r "Exception" *
Algorithmia/algo_response.py:            raise AlgoException(responseJson['error'])
Algorithmia/algo_response.py:class AlgoException(Exception):
Binary file Algorithmia/acl.pyc matches
Algorithmia/util.py:            raise Exception('Invalid path')
Algorithmia/datadirectory.py:            raise Exception("Directory creation failed: " + str(response.content))
Algorithmia/datadirectory.py:            raise Exception(result['error']['message'])
Algorithmia/datadirectory.py:            raise Exception('Unable to get permissions:' + str(response.content))
Algorithmia/datadirectory.py:            raise Exception('Unable to update permissions: ' + response.json()['error']['message'])
Algorithmia/datadirectory.py:                raise Exception("Directory iteration failed: " + str(response.content))
Binary file Algorithmia/datadirectory.pyc matches
Binary file Algorithmia/datafile.pyc matches
Binary file Algorithmia/algorithm.pyc matches
Algorithmia/algorithm.py:            raise Exception('Invalid algorithm URI: ' + algoRef)
Algorithmia/algorithm.py:                raise Exception(response.text)
Algorithmia/algorithm.py:                raise Exception(response.text)
Algorithmia/algorithm.py:                raise Exception(responseJson['error']['message'])
Algorithmia/acl.py:            raise Exception('Response does not contain read ACL')
Algorithmia/acl.py:                raise Exception('Invalid acl string %s' % (acl_list[0]))
Binary file Algorithmia/util.pyc matches
Algorithmia/datafile.py:            raise Exception('file does not exist - {}'.format(self.path))
Algorithmia/datafile.py:            raise Exception('file does not exist - {}'.format(self.path))
Algorithmia/datafile.py:            raise Exception('file does not exist - {}'.format(self.path))
Algorithmia/datafile.py:            raise Exception('file does not exist - {}'.format(self.path))
Algorithmia/datafile.py:                raise Exception(result['error']['message'])
Algorithmia/datafile.py:            raise Exception("Must put strings or binary data. Use putJson instead")
Algorithmia/datafile.py:            raise Exception(result['error']['message'])
Algorithmia/datafile.py:                raise Exception(result['error']['message'])
Algorithmia/datafile.py:            raise Exception(result['error']['message'])
Binary file Algorithmia/algo_response.pyc matches
README.md:API errors and Algorithm exceptions will result in calls to `pipe` throwing an `AlgoException`:
README.md:# Algorithmia.algo_response.AlgoException: algorithm algo://util/whoopsWrongAlgo not found
Test/datafile_test.py:        except Exception as e:
Test/datafile_test.py:        except Exception as e:
Test/datadirectorytest.py:        self.assertRaises(Exception, dd.delete)
Test/utiltest.py:        self.assertRaises(Exception, getParentAndBase, '/')
Test/utiltest.py:        self.assertRaises(Exception, getParentAndBase, '')
Test/utiltest.py:        self.assertRaises(Exception, getParentAndBase, 'a/')

It would be useful to have more specific Exceptions for try: except handling. For example, for the exception here, perhaps an IOError would be more suitable?

Thanks

How does one connect to algorithmia when behind a proxy? Found no documentation online

Hi, i am trying to build and run locally my algorithm on my machine (mac os 10.13.6). I have python 3.6.6 installed as main python (pyenv). When i execute command:
algo runlocal -D test.json
The message 'Error running 'algo serve' appears after a while..

Out of curiosity i run: algo serve and after the installation of packages finishes, there is a build error:
INFO ALGOERR - class RegexFlag(enum.IntFlag):
INFO ALGOERR - AttributeError: module 'enum' has no attribute 'IntFlag'

After searching a little bit into the wild internet, i found the following thread enum34 isn't compatible with python 3.6.

Algorithmia requires python version > 3.4 and also requires presence of enum34.

Is it something caused by my python environment or is it something that may have to be addressed on your side?

I've added a github actions for this repo in evaluation, all seems well however this path here seems to fail consistently with the following error:

self = <Test.algo_test.AlgoTest testMethod=test_json_unicode>

    def test_json_unicode(self):
        telephone = [u"\u260E"]
    
        #Unicode input to pipe()
        result1 = self.client.algo('util/Echo').pipe(telephone)
>       self.assertEquals('json', result1.metadata.content_type)
E       AssertionError: 'json' != 'text'
E       - json
E       + text

Test/algo_test.py:37: AssertionError

This occurs in both python 2.7, and python 3.6

Metadata such as duration might be useful for consumers

The dependency on requests (and any others) should be set as "install_requires" within the package so that the user doesn't need to complete further set up steps after installing algorithmia.

The Python client looks for an environment variable ALGORITHMIA_API_KEY to override the default API address: https://github.com/algorithmiaio/algorithmia-python/blob/master/Algorithmia/client.py#L19

All other client implementations which check env look for ALGORITHMIA_API

Suggest adding ALGORITHMIA_API but retaining ALGORITHMIA_API_KEY as well for backward-compatibility

Hi,

I'm having trouble to make this code work:

client = Algorithmia.client(ALGO_TOKEN)

algo_text = client.algo('util/Html2Text/0.1.4')
algo_summary = client.algo('nlp/Summarizer/0.1.3')

link_text = algo_text.pipe(title).result
summary = algo_summary.pipe(link_text).result

The first result come back without problem, but I cannot give it as an input to the next API. What am I doing wrong? I tried to encode/decode to/from unicode. I couldn't figure it out.

Log:

2016-10-21 11:14:58,167 - ERROR: Unexpected error: Traceback (most recent call last):
File "/Users/iseckert/DevProjects/bot/linkcrawler-python-bot/bot/slack_bot.py", line 53, in start
event_handler.handle(event)
File "/Users/iseckert/DevProjects/bot/linkcrawler-python-bot/bot/event_handler.py", line 18, in handle
self._handle_by_type(event['type'], event)
File "/Users/iseckert/DevProjects/bot/linkcrawler-python-bot/bot/event_handler.py", line 27, in _handle_by_type
self._handle_message(event)
File "/Users/iseckert/DevProjects/bot/linkcrawler-python-bot/bot/event_handler.py", line 85, in _handle_message
summary = algo_summary.pipe(link_text).result
File "/usr/local/lib/python2.7/site-packages/Algorithmia/algorithm.py", line 39, in pipe
return AlgoResponse.create_algo_response(self.client.postJsonHelper(self.url, input1, *_self.query_parameters))
File "/usr/local/lib/python2.7/site-packages/Algorithmia/client.py", line 53, in postJsonHelper
response = requests.post(self.apiAddress + url, data=input_json, headers=headers, params=query_parameters)
File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 107, in post
return request('post', url, data=data, json=json, *_kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 53, in request
return session.request(method=method, url=url, *_kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, *_send_kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, *_kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 376, in send
timeout=timeout
File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
body=body, headers=headers)
File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 353, in _make_request
conn.request(method, url, *_httplib_request_kw)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1057, in request
self._send_request(method, url, body, headers)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1097, in _send_request
self.endheaders(body)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1053, in endheaders
self._send_output(message_body)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 901, in _send_output
self.send(message_body)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 873, in send
self.sock.sendall(data)
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 743, in sendall
v = self.send(data[count:])
File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 709, in send
v = self._sslobj.write(data)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf3' in position 14: ordinal not in range(128)

Hi there :)
Would it be possible to have an upgrade of the python flavors? Currently the Algorithmia client only works with python2 , or 3.3 max. But python is already up to 3.7, and even support for 3.4 will be discontinued soon.
When do you think it might be possible ?

Thank you in advance.

the new languages endpoint is super useful for helping algo devs understand how to programmatically access algorithms; however the current version can sometimes swallow errors by assuming that the output is always properly formatted.

This form of exception can occur either with a 500 or 400 response:

$ algo languages
Name                      Description                        
Traceback (most recent call last):
  File "/usr/local/anaconda3/bin/algo", line 8, in <module>
    sys.exit(main())
  File "/usr/local/anaconda3/lib/python3.8/site-packages/Algorithmia/__main__.py", line 207, in main
    print("{:<25} {:<35}".format(lang['name'],lang['display_name']))
TypeError: string indices must be integers

To rectify this, we should check to ensure that the lang response is properly formatted and contains all relevant fields first, otherwise return the response unaltered.

Hi,

I've been troubleshooting a bug where this client will hang forever waiting for a response in some cases.

I noticed that calls using the requests library are not using the timeout= option which is recommended:
https://2.python-requests.org/en/master/user/quickstart/#timeouts

You can tell Requests to stop waiting for a response after a given number of seconds with the timeout parameter. Nearly all production code should use this parameter in nearly all requests. Failure to do so can cause your program to hang indefinitely

I've put together a PR #40 for changing this to 5 seconds (if this seems low, the notes from requests make this a little clearer: it's the time between bytes rather than the entire request) to establish a sensible default for the library to be making calls under.

More info from the docs:

timeout is not a time limit on the entire response download; rather, an exception is raised if the server has not issued a response for timeout seconds (more precisely, if no bytes have been received on the underlying socket for timeout seconds). If no timeout is specified explicitly, requests do not time out.

Currently only accepts a path and errors on a file
https://github.com/algorithmiaio/algorithmia-python/blob/master/Algorithmia/datafile.py#L95-L102

I came across a few unexpected behaviors, when installing the Python Client on deepPurple, for my own user. I'll list them altogether in this issue.

After the client is installed through the wheel, as per the instructions:

Trying to authorize, I would expect that the API adress is provided by default, as api.algorithmia.com and I would be able to pass this step without entering anything. Pressing enter gives the following error:

asli@deeppurple:~$ algo auth
Configuring authentication for profile: default
enter API address [api.algorithmia.com]:
Traceback (most recent call last):
  File "/home/asli/.local/bin/algo", line 10, in <module>
    sys.exit(main())
  File "/home/asli/.local/lib/python2.7/site-packages/Algorithmia/__main__.py", line 113, in main
    APIaddress = input("enter API address [api.algorithmia.com]:")
  File "<string>", line 0
    
    ^
SyntaxError: unexpected EOF while parsing

Now entering api.algorithmia.com as the API address, but got the following error.
Apparently it’s asking me to use string quotes, but I don’t think this is the expectation. We’re more used to entering string inputs without the quotes.

asli@deeppurple:~$ algo auth
Configuring authentication for profile: default
enter API address [api.algorithmia.com]:api.algorithmia.com
Traceback (most recent call last):
  File "/home/asli/.local/bin/algo", line 10, in <module>
    sys.exit(main())
  File "/home/asli/.local/lib/python2.7/site-packages/Algorithmia/__main__.py", line 113, in main
    APIaddress = input("enter API address [api.algorithmia.com]:")
  File "<string>", line 1, in <module>
NameError: name 'api' is not defined

The same goes for the API key, again entering it without the quotes gives the same error.

asli@deeppurple:~$ algo auth
Configuring authentication for profile: default
enter API address [api.algorithmia.com]:"api.algorithmia.com"
enter API key:sim0/CA0mCa6Xz3FAkyoHb45G5I1
Traceback (most recent call last):
  File "/home/asli/.local/bin/algo", line 10, in <module>
    sys.exit(main())
  File "/home/asli/.local/lib/python2.7/site-packages/Algorithmia/__main__.py", line 114, in main
    APIkey = input("enter API key:")
  File "<string>", line 1, in <module>
NameError: name 'sim0' is not defined

Now using the quotes for both fields and the client is good to go.

asli@deeppurple:~$ algo auth
Configuring authentication for profile: default
enter API address [api.algorithmia.com]:"api.algorithmia.com"
enter API key:"sim0/CA0mCa6Xz3FAkyoHb45G5I1"
usage: algo [-h] [--profile PROFILE]
            {auth,clone,run,ls,rm,mkdir,rmdir,cp,cat,help} ..

Hi, I have a problem of accessing file in my Hosted Data. Here is the code snippet

import Algorithmia

client = Algorithmia.client('sim0b1n3***pxFqn1')
cred_json = client.file("data://.my/models/cred.json").getJson()

This works fine when running on my local machine, I use conda env, python3.5.4 as #14. However, when I run this in Algorithmia IDE, I can't access the same file
Algorithmia.errors.DataApiError: unable to get file .my/models/cred.json - None

My Algorithm setup is python3, closed source, allow internet access. I use the default api key. The read access of .my/models/ is public

There are no exceptions thrown if the user is doing Algorithmia.file().get on someone else's collection, or a non-existent collection. Then, let's say I tried to retrive a Json from a file in the data api and get the field "acces_key". The error I get when I run this code is: "Error: 'access_key'". This does not help me debug what went wrong at all.

Allows for consistent experience with local testing + checked in testing since algorithms within the system can't have api keys set.

foo = client.dir("dropbox+XXXXX://XXXXX")
for entry in foo.list():
print(entry.path)
print(entry.url)

Error message below:
DataApiError: Directory iteration failed: b'{"error":{"message":"path not found"}}'

dropbox already linked to Algorithmia