server.py stuck on thread about unity3d-python-communication

[offchan42]

My repo teaches you how to send data to Unity. But it doesn't care about how you generate that data. You seem to have the problem generating that data which means that your issue is quite unrelated to this repo. But I will help for the sake of it.

It seems that you haven't identified clearly the problem that you are having. The first step is to identify your specific problem. Find out which line of code produce the issue. Ask yourself what you expect to happen. And what actually happened.
Use print() function to assist in your debugging.

My guess is that you are trying to read an image from Unity, run CNN in python, and then send the prediction back to Unity. So I would suggest you to inspect each step of this whole process. Check whether you got the image, first of all. Print the image's shape. After that, run the model, and see whether you get the prediction. Then check whether Unity got the prediction from ZeroMQ. Definitely one of these steps will be broken.

If any step of the process went wrong then at least you have narrowed down the potential source of problem. If you are not narrowing down the problem then that's not helping you fix the bug.

[fm64hylian]

thank you for your quick response, I am actually not sending anything back to unity (except for a message), the script generates the binary mask and places it on a folder, but I just restarted the PC and it went through the script, however at the end it threw this error:

Traceback (most recent call last):
  File "server.py", line 47, in <module>
    socket.send(response.encode())
  File "PATH\PythonStuff\envs\imageseg\lib\site-packages\zmq\sugar\socket.py", line 400, in send
    return super(Socket, self).send(data, flags=flags, copy=copy, track=track)
  File "zmq/backend/cython/socket.pyx", line 728, in zmq.backend.cython.socket.Socket.send
  File "zmq/backend/cython/socket.pyx", line 775, in zmq.backend.cython.socket.Socket.send
  File "zmq/backend/cython/socket.pyx", line 247, in zmq.backend.cython.socket._send_copy
  File "zmq/backend/cython/socket.pyx", line 242, in zmq.backend.cython.socket._send_copy
  File "zmq/backend/cython/checkrc.pxd", line 26, in zmq.backend.cython.checkrc._check_rc
    raise ZMQError(errno)
zmq.error.ZMQError: Operation cannot be accomplished in current state

the final message from python wasn't received on unity.

[offchan42]

OK. Now that's a legit question that's based purely on the design of ZeroMQ request-reply pattern.
In ZeroMQ's request-reply pattern, you are expected to send a request and then wait for a reply. You must exactly do one request and then wait for exactly one reply. If you only send requests or only wait for replies, or send one request and then not wait for a reply, things will break. The state machine of ZeroMQ expects strongly that you follow its rule.

This means that in Unity, you must send a request and must also wait for a reply. In python, you must receive a request and send the reply. Both sides must do these things in a loop. If either server or client violate the rule, things break.

Let me know if that solves your problem.

[fm64hylian]

You are right, I am doing 2 socket.send from server.py but only 1 from unity, I will test it later to see how it goes.

thank you

[fm64hylian]

Ok, it went through smoothly, I removed the first socket.send and got rid of the thread on server.py. I am just starting to read the documentation on zeromq, and I know you mentioned that the server and client need to be reset or terminated to not crash unity. I check on client that you could do either client.Disconnect() or client.Dispose(), which one would you recommend to use after I got the response I wanted from the server? also should I disconnect manually form server.py as well to not crash unity?

Thank you

[offchan42]

This code receives messages 10 times on Unity, and it works without crashing.
Just break after you got your message so that NetMQConfig.Cleanup() can be called. And that should be enough to end your client.

[fm64hylian]

yeah I'm calling both on unity and stopping the thread to see if the ports and everything was closed smoothly (I'm also doing socket.close() and context.term() on server.py just in case) I was wondering if there is a way to activate an environment from script so I don't have to call everything myself in the console, since I pretend to make an android build, I know its kind of unrelated to the repository, but do you know by chance how could we call server.py from unity? I'm using packages such as tensorflow and pycocotools, so I need python3 and can't use ironpython or python for unity since they do not support python 3 yet.

Any help would be appreciated, if not I'll just close the issue since we went out of scope here

[offchan42]

Try running the python command and environment activations using this method at the start of the unity scene.
https://stackoverflow.com/questions/1469764/run-command-prompt-commands
I've never tried it but it should work because it's similar to how you run commands in cmd.

And if you want to run multiple commands, try something like this: https://stackoverflow.com/questions/437419/execute-multiple-command-lines-with-the-same-process-using-net
You can do

conda activate YOUR_ENVIRONMENT
python server.py

When you quit the game, make sure to kill the Process that you have created.

Another thing is about packaging your python stuff, you need to learn how to freeze all your python packages into a folder so that it can be distributed with the Unity Build. I've never tried that much. But I'm sure you could do it.
Here's the starting point:
https://docs.python-guide.org/shipping/packaging/

[fm64hylian]

That's perfect, I wasn't sure even I if did all that if it would work on an android build, I also tried to set an environment without anaconda with plain python since anaconda is much heavier. Thanks for the information about how to create python packages as well