Using the Metaseq API about metaseq

[stephenroller]

That's what I do, but you'll need to open the port or forward the port somehow.

[Mrs-Hudson]

Thanks @stephenroller . I tried querying the server this way:

`import requests
import json
from flask import jsonify

url = 'http://127.0.0.1:6010/completions'
headers = {'Content-Type': 'application/json'}

filters = {'prompt':'LinkedIn is a great company2'}
#print(jsonify(filters))
#params = dict(json=json.dumps(filters))
print(filters)
print(json.dumps(filters))
response = requests.post(url, json=filters,headers=headers)
#print(response.text)
assert response.status_code == 200
print(response.json())
`

But I see the following error in the server logs:
In completion b'{"prompt": "LinkedIn is a great company2"}' {'environ': {'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.input': <_io.BufferedReader name=47>, 'wsgi.errors': <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>, 'wsgi.multithread': True, 'wsgi.multiprocess': False, 'wsgi.run_once': False, 'werkzeug.server.shutdown': <function WSGIRequestHandler.make_environ.<locals>.shutdown_server at 0x7f9ef59cf820>, 'SERVER_SOFTWARE': 'Werkzeug/1.0.1', 'REQUEST_METHOD': 'POST', 'SCRIPT_NAME': '', 'PATH_INFO': '/completions', 'QUERY_STRING': '', 'REQUEST_URI': '/completions', 'RAW_URI': '/completions', 'REMOTE_ADDR': '127.0.0.1', 'REMOTE_PORT': 36506, 'SERVER_NAME': '0.0.0.0', 'SERVER_PORT': '6010', 'SERVER_PROTOCOL': 'HTTP/1.1', 'HTTP_HOST': '127.0.0.1:6010', 'HTTP_USER_AGENT': 'python-requests/2.27.1', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate', 'HTTP_ACCEPT': '*/*', 'HTTP_CONNECTION': 'keep-alive', 'CONTENT_TYPE': 'application/json', 'CONTENT_LENGTH': '42', 'werkzeug.request': <Request 'http://127.0.0.1:6010/completions' [POST]>}, 'shallow': False, 'url_rule': <Rule '/completions' (OPTIONS, POST) -> completions>, 'view_args': {}, 'stream': <werkzeug.wsgi.LimitedStream object at 0x7f9f0e918a30>, 'form': ImmutableMultiDict([]), 'files': ImmutableMultiDict([]), '_cached_data': b'{"prompt": "LinkedIn is a great company2"}', 'data': b'{"prompt": "LinkedIn is a great company2"}', 'url': 'http://127.0.0.1:6010/completions'} In completion b'{"prompt": "LinkedIn is a great company2"}' {'environ': {'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.input': <_io.BufferedReader name=111>, 'wsgi.errors': <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>, 'wsgi.multithread': True, 'wsgi.multiprocess': False, 'wsgi.run_once': False, 'werkzeug.server.shutdown': <function WSGIRequestHandler.make_environ.<locals>.shutdown_server at 0x7f9ef59cfb80>, 'SERVER_SOFTWARE': 'Werkzeug/1.0.1', 'REQUEST_METHOD': 'POST', 'SCRIPT_NAME': '', 'PATH_INFO': '/completions', 'QUERY_STRING': '', 'REQUEST_URI': '/completions', 'RAW_URI': '/completions', 'REMOTE_ADDR': '127.0.0.1', 'REMOTE_PORT': 37038, 'SERVER_NAME': '0.0.0.0', 'SERVER_PORT': '6010', 'SERVER_PROTOCOL': 'HTTP/1.1', 'HTTP_HOST': '127.0.0.1:6010', 'HTTP_USER_AGENT': 'python-requests/2.27.1', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate', 'HTTP_ACCEPT': '*/*', 'HTTP_CONNECTION': 'keep-alive', 'CONTENT_TYPE': 'application/json', 'CONTENT_LENGTH': '42', 'werkzeug.request': <Request 'http://127.0.0.1:6010/completions' [POST]>}, 'shallow': False, 'url_rule': <Rule '/completions' (OPTIONS, POST) -> completions>, 'view_args': {}, 'stream': <werkzeug.wsgi.LimitedStream object at 0x7f9ef59e48b0>, 'form': ImmutableMultiDict([]), 'files': ImmutableMultiDict([]), '_cached_data': b'{"prompt": "LinkedIn is a great company2"}', 'data': b'{"prompt": "LinkedIn is a great company2"}', 'url': 'http://127.0.0.1:6010/completions'} 2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad request version ('\x00') 2022-05-08 22:38:17 | INFO | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] "âÞô+±½ÞKxÅ4ðYæâË?È#ÈÃIr¤ÌÌ] 'ÄcÁ{w¿�-{V�ýhdñ� ½�Ïzÿ� " HTTPStatus.BAD_REQUEST - 2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad request version ('\x00') 2022-05-08 22:38:17 | INFO | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] "âÞ­[f¥ô=:© ë"�²GÈ$¿¥_ç8ê7·­·QÃ[�� �ëúWJ²ÅIf9úÝ9Þ}«ý2*ÊQ]SÚÑ��ªÿ� " HTTPStatus.BAD_REQUEST - 2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad request version ("\x00\x00|À,À0\x00£\x00\x9f̨̩̪À¯À\xadÀ£À\x9fÀ]ÀaÀWÀSÀ+À/\x00¢\x00\x9eÀ®À¬À¢À\x9eÀ\\ÀÀVÀRÀ$À(\x00k\x00jÀ#À'\x00g\x00@À")
-ZcWJþ¿Rßõg�t#øPxw¡�Ðñ°7h­| wer|À,À0£�̨̩̪À¯À­À£À�À]ÀaÀWÀSÀ+À/¢�À®À¬À¢À�À\ÀÀVÀRÀ$À(kjÀ#À'g@À" HTTPStatus.BAD_REQUEST -
2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad request version ("Wág¾\x9dèL¦\x9d\x18\x96\x00\x00¦À,À0\x00£\x00\x9f̨̩̪À¯À\xadÀ£À\x9fÀ]ÀaÀWÀSÀ+À/\x00¢\x00\x9eÀ®À¬À¢À\x9eÀ\ÀÀVÀRÀ$À(\x00k\x00jÀsÀw\x00Ä\x00ÃÀ#À'\x00g\x00@ÀrÀv\x00¾\x00½À") =�A�¾x�ùìÝNÏÎEßy Wág¾�èL¦��¦À,À0£�̨̩̪À¯À­À£À�À]ÀaÀWÀSÀ+À/¢�À®À¬À¢À�À\ÀÀVÀRÀ$À(kjÀsÀwÄÃÀ#À'g@ÀrÀv¾½À" HTTPStatus.BAD_REQUEST -
2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad HTTP/0.9 request type ('\x16\x03\x01\x00\xad\x01\x00\x00©\x03\x02')
2022-05-08 22:38:17 | INFO | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] "­©á×(�Ék4üâ�A~0&54:ã¥jnðW�ÎFE�ûf>ÀÀ" HTTPStatus.BAD_REQUEST -
2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad request version ('î\x8ed¿Ra\x05ÏØFþ®1\x9cùDÝ=¨ø\x00½É.\x8b´ëh\x00\x00¢À\x14À')
«H î�d¿RaÏØFþ®1�ùDÝ=¨ø½É.�´ëh¢ÀÀ" HTTPStatus.BAD_REQUEST -May/2022 22:38:17] "
2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad HTTP/0.9 request type ('\x16\x03\x01\x00\xad\x01\x00\x00©\x03\x01\x87BÐ\x1bé»ìü\x96\x7fx\x04\x1b\x91\x8f\x8aT\x86Êå\x80õc')
2022-05-08 22:38:17 | INFO | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] "­©�BÐ�»ìü��x���T�Êå�õc
d{Û0¢ïàë>ÀÀ" HTTPStatus.BAD_REQUEST -
2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad HTTP/0.9 request type ('\x16\x03\x01\x01\x11\x01\x00\x01')
�¸ÜÜ�¶ÿVùN$³QÚ¦Ë�î_��=�bò�Ñ5¦¢ÀÀ" HTTPStatus.BAD_REQUEST -May/2022 22:38:17] "
2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad HTTP/0.9 request type ('\x16\x03\x00\x00j\x01\x00\x00f\x03\x00µ<\x82.\x15U¶\x87ù\x9cÀ')
2022-05-08 22:38:17 | INFO | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] "jfµ<�.U¶�ù�ÀM(q�YûÈïÑ­¿Z�ðõg¥>ÀÀ" HTTPStatus.BAD_REQUEST - 2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad request version ('¾\x92¹\x00\x00¢À\x14À') 2022-05-08 22:38:17 | INFO | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] "ÎÊÆöÿ�@np�^��²Õc�F¸�Îc?OÖ¾�¹¢ÀÀ" HTTPStatus.BAD_REQUEST - 2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad HTTP/0.9 request type ('\x16\x03\x01\x01\x19\x01\x00\x01\x15\x03\x03O\x9b4Å°føuBÔ\x00\x9f-\x7f&i\x98,') 2022-05-08 22:38:17 | INFO | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] "O�4Å°føuBÔ�-�&i�,�QXü�ÑLÝ�Îè�t��À0À,À(À$ÀÀ" HTTPStatus.BAD_REQUEST - 2022-05-08 22:38:17 | ERROR | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] code 400, message Bad HTTP/0.9 request type ('\x16\x03\x01\x01\x87\x01\x00\x01\x83\x03\x03\x1bæ\x8f?>äÜ\x0ehWtåëf\x9eµ@') 2022-05-08 22:38:17 | INFO | werkzeug | 127.0.0.1 - - [08/May/2022 22:38:17] "����?>äÜhWtåëf�µ@H¯M_þ;¾�J÷ÿOôÀ0À,À(À$ÀÀ" HTTPStatus.BAD_REQUEST -

[Mrs-Hudson]

If I replace http with https in the API URL I see the below:
return request('post', url, data=data, json=json, **kwargs) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/requests/sessions.py", line 529, in request resp = self.send(prep, **send_kwargs) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/requests/sessions.py", line 645, in send r = adapter.send(request, **kwargs) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/requests/adapters.py", line 517, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='127.0.0.1', port=6010): Max retries exceeded with url: /completions (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1123)')))

[Mrs-Hudson]

This issue was solved by adding verfiy=False to the post request

[stephenroller]

Here's an example of hitting it:

curl -k http://localhost:6010/completions \
  -H "Content-Type: application/json" \
  -d '{
  "prompt": "A chat between a teacher and student who wants to learn about tacos.\n\nTeacher: Hi there. What would you like to learn about today?\nStudent:",
  "temperature": 1.0,
  "max_tokens": 32, "min_tokens": 4,
  "top_p": 0.9, "n": 1,
  "echo": false, "stop": "\n"
}'| jq .

[nickums]

I cannot find metaseq-api-local anywhere in OPT