GithubHelp home page GithubHelp logo

caproto-sandbox's People

Contributors

tacaswell avatar vstadnytskyi avatar vstadnytskyi-fda avatar

Watchers

avatar avatar

Forkers

tacaswell vstadnytskyi-fda abdelrahmannasser20

caproto-sandbox's Issues

Strings passing between Caproto and pyepics GUI

Code:
server_string_pv.py has 3 pvs. Str_in is a write PV that can be changed by clients. The str_out PV is all caps of the str_in. The N_chr PV shows how many characters in the string.
server_string_pv_gui.py pyepics gui

str_in = pvproperty(value='', dtype = str, max_length = 1000)
str_out = pvproperty(value='', dtype = str, read_only = True)
N_chr = pvproperty(value=3, read_only = True)

Problems:
caproto server truncates all input PV written into str_in if the input string exceeds the max_length. The str_out which is read-only doesn't suffer the same. I guess because if originates from the server itself.

pyepics problems:
The PVTextCtrl field accepts the value in the field after dirty_timeout, return key or if focus is moved away. If I selected the GUI window the last known field will be selected. Now, if I move away without making any changes the server will get PV associated with that field.

image

caproto pvproperty string PV issue

I have a caproto server with a string PV. The max length of the PV is 1. I get an error on the server side If I use pyepics to write into this PV. Interestingly there are no problems with caproto.threading.client.write() function and I can write one character without a problem.
it works fine, if the default value is longer than 1 or i set max_length large than 1. So there is something special about how caproto processes the string sent from pyepics wx.PVTextCtrl field.

Codes:
caproto-sandbox/server_string_pv.py
caproto-sandbox/server_string_pv_client.py
caproto-sandbox/server_string_pv_gui.py

If I use pyepics GUI
works:

VALVE = pvproperty(value='nnnnnn', dtype=str)
VALVE = pvproperty(value='', dtype=str, max_length = 2)
VALVE = pvproperty(value='nn', dtype=str)

doesn't work:

VALVE = pvproperty(value='', dtype=str)
VALVE = pvproperty(value='', dtype=str, max_length = 1)

If I use caproto threading client it works always.

In [14]: client.VALVE.write('b')
Out[14]: WriteNotifyResponse(data_type=<ChannelType.CHAR: 4>, data_count=1, status=CAStatusCode(name='ECA_NORMAL', code=0, code_with_severity=1, severity=<
CASeverity.SUCCESS: 1>, success=1, defunct=False, description='Normal successful completion'), ioid=0)
In [15]: client.VALVE.write('o')
Out[15]: WriteNotifyResponse(data_type=<ChannelType.CHAR: 4>, data_count=1, status=CAStatusCode(name='ECA_NORMAL', code=0, code_with_severity=1, severity=<
CASeverity.SUCCESS: 1>, success=1, defunct=False, description='Normal successful completion'), ioid=1)

Error:

ERROR:caproto.circ:Invalid write request by user-13 (user 13.niddk.nih.gov): WriteRequest(data=array([0], dtype=uint8), data_type=<ChannelType.CHAR: 4>, d
ata_count=1, sid=0, ioid=3, metadata=None)
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/server/common.py", line 471, in handle_write
    user_address=self.circuit.address)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_data.py", line 434, in auth_write
    flags=flags))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_data.py", line 971, in write_from_dbr
    await super().write_from_dbr(*args, flags=flags, **kwargs)

  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_data.py", line 462, in write_from_dbr
    direction=ConversionDirection.FROM_WIRE)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_backend.py", line 346, in convert_values
    convert_from=from_dtype)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_numpy_backend.py", line 64, in python_to_epics
    return np.asarray(values).astype(type_map[dtype])
ValueError: invalid literal for int() with base 10: ''

The example worker_thread_pc raises exception

The example worker_thread_pc raises error. I literally copied it from the caproto examples. Version in my repository (unchanged)

>>> caproto.__version__
'0.4.0'
$ python3 caproto_sandbox/worker_thread_pc.py 
[I 16:22:34.648          server:  161] Asyncio server starting up...
[I 16:22:34.648          server:  174] Listening on 0.0.0.0:5064
[I 16:22:34.649          server:  260] Server startup complete.
* request method called at server startup
[E 16:22:34.649          server:  278] Server error. Will shut down
    Traceback (most recent call last):
      File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/asyncio/server.py", line 265, in run
        await asyncio.gather(*tasks)
      File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/server/server.py", line 137, in _server_startup
        return await self.startup(self, async_lib)
      File "caproto_sandbox/worker_thread_pc.py", line 29, in request
        self.Event = async_lib.Event
    AttributeError: 'AsyncioAsyncLayer' object has no attribute 'Event'
[I 16:22:34.650          server:  281] Server exiting....
Traceback (most recent call last):
  File "caproto_sandbox/worker_thread_pc.py", line 53, in <module>
    run(ioc.pvdb, **run_options)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/server/__init__.py", line 11, in run
    return run(pvdb, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/asyncio/server.py", line 312, in run
    loop.run_until_complete(task)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete
    return future.result()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/asyncio/server.py", line 300, in start_server
    ret = await ctx.run(log_pv_names=log_pv_names)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/asyncio/server.py", line 265, in run
    await asyncio.gather(*tasks)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/server/server.py", line 137, in _server_startup
    return await self.startup(self, async_lib)
  File "caproto_sandbox/worker_thread_pc.py", line 29, in request
    self.Event = async_lib.Event
AttributeError: 'AsyncioAsyncLayer' object has no attribute 'Event'
> pm()                                                                                                                              
caproto_sandbox/worker_thread_pc.py(29)request()
-> self.Event = async_lib.Event
(Pdb) async_lib
<caproto.asyncio.server.AsyncioAsyncLayer object at 0x10f1f4b10>

I can import caproto and create caproto.asyncio.server.AsyncioAsyncLayer

In [4]: import caproto                                                                                                                    
In [5]: p = caproto.asyncio.server.AsyncioAsyncLayer                                                                                      
In [6]: p.__dict__                                                                                                                        
Out[6]: 
mappingproxy({'__module__': 'caproto.asyncio.server',
              'name': 'asyncio',
              'ThreadsafeQueue': None,
              'library': <module 'asyncio' from '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/__init__.py'>,
              '__init__': <function caproto.asyncio.server.AsyncioAsyncLayer.__init__(self, loop=None)>,
              '__doc__': None})

caproto server issue

I have two test server:

  1. one PV camera server
  2. several PVs io_device server
    I have made even simpler server with multiple PVs and still have the same problem

Both servers generate an image 3960,3960 with float64 ranging between 0 and 255.

I am trying to retrieve the image and print: actual array, array.shape,array.max(),array.mean()

Here is the client

And here are relevant pieces of the code. The full code check in links above.

image = epics.PV(pvname = default_prefix+'image', connection_timeout = 20)
def pyepics_for_loop():
    for i in range(4):
        img = image.get(timeout = 20)
        print(img,img.shape,img.max(),img.mean(),t1.get(),t2.get())
        sleep(1)

ctx = Context()
ca_img = ctx.get_pvs(default_prefix+'image')
def caproto_for_loop():
    for i in range(4):
        img = ca_img.read().data
        print(img,img.shape,img.max(),img.mean(),ca_t2.read().data[0])
        sleep(1)

It works well with the camera server

In [1]: run caproto_sandbox/io_device_client_simple.py

In [2]: default_prefix
Out[2]: 'camera:'

In [3]: caproto_for_loop()

[179. 144. 167. ... 183. 177. 237.] (15681600,) 255.0 127.49332746658504
[211. 218. 249. ...  86. 102. 245.] (15681600,) 255.0 127.48166220283645
[ 77. 131.  72. ...   5. 189.  63.] (15681600,) 255.0 127.48323914651566
[135. 116. 222. ...  22. 126. 156.] (15681600,) 255.0 127.51444514590348

In [4]: pyepics_for_loop()
[ 16. 131. 169. ...  50. 169. 128.] (15681600,) 255.0 127.48961655698398
[  7. 139. 180. ... 253. 210.  83.] (15681600,) 255.0 127.4769012090603
[109. 156. 108. ... 127. 194.  67.] (15681600,) 255.0 127.49470283644526
[115. 164. 119. ... 227. 127. 214.] (15681600,) 255.0 127.4787924063871

The pyepics client fails to get data from the caproto "io_device" server. The max value in the array appears to be close to the timestamp.

In [2]: run caproto_sandbox/io_device_client_simple.py

In [3]: default_prefix

Out[3]: 'io_device:'

In [4]: caproto_for_loop()
[ 75. 201. 122. ...  84. 163. 116.] (15681600,) 255.0 127.49252474237322
[191.  80. 253. ... 200. 251. 105.] (15681600,) 255.0 127.51174880114274
[ 25. 108. 146. ... 235. 125. 251.] (15681600,) 255.0 127.51386178706254
[ 60.  93. 240. ... 221.  98. 159.] (15681600,) 255.0 127.51358974849505

In [5]: pyepics_for_loop()
CAC: Undecipherable TCP message ( bad response type 16412 ) from myserver.gov:5064
CA.Client.Exception...............................................
    Warning: "Virtual circuit disconnect"
    Context: "myserver.gov:5064"
    Source File: ../cac.cpp line 1237
    Current Time: Tue Oct 08 2019 17:29:11.768520171
..................................................................
[115. 112. 216. ... 182. 169. 119.] (15681600,) 1570570166.1864076 327.79232126943043
[ 47. 198.  19. ... 124. 181. 190.] (15681600,) 255.0 127.54112769105194
CAC: Undecipherable TCP message ( bad response type 16468 ) from myserver.gov:5064
CA.Client.Exception...............................................
    Warning: "Virtual circuit disconnect"
    Context: "myserver.gov:5064"
    Source File: ../cac.cpp line 1237
    Current Time: Tue Oct 08 2019 17:29:20.073010817
..................................................................
[234. 188.  57. ... 177.  56.  68.] (15681600,) 1570570175.0612175 327.81672102381924
[157. 103.   4. ...  40.   2. 224.] (15681600,) 255.0 127.49858777165595

threading.client consumes a lot of CPU while not active

I have noticed that my CPU utilization goes up(+25% one core) substantially when I create a very simple threading.client before even connection to any PVs. consumes a lot of CPU while not active.

image

My test code:
https://github.com/vstadnytskyi/caproto-sandbox/blob/master/caproto_sandbox/threading_client_bug.py

#!/usr/bin/env python3
import psutil
from time import sleep
sleep(2)
print(psutil.cpu_percent(percpu = True),'start')

from caproto.threading.client import Context
sleep(2)
print(psutil.cpu_percent(percpu = True), 'import Context')

ctx = Context()
sleep(2)
print(psutil.cpu_percent(percpu = True),'2 seconds after ctx = Context()')
sleep(2)
print(psutil.cpu_percent(percpu = True),'4 seconds after ctx = Context()')
sleep(2)
print(psutil.cpu_percent(percpu = True),'6 seconds after ctx = Context()')

Output when I run the code. See how

$ python3 caproto_sandbox/threading_client_test.py
[13.6, 4.0, 13.0, 2.5] start
[25.5, 5.7, 20.9, 4.3] import Context
[37.1, 11.4, 56.2, 10.9] 2 seconds after ctx = Context()
[43.5, 20.0, 51.0, 15.4] 4 seconds after ctx = Context()
[51.5, 25.9, 60.5, 20.4] 6 seconds after ctx = Context()

Some unknown errors

I would randomly get this error:

Ignoring late response with ioid=1 regarding PV named BEAMLINE:m39.VAL because it arrived 11.781 seconds after the deadline specified by the timeout.

or this

Server failed to process command: EventAddRequest(data_type=<ChannelType.TIME_ENUM: 17>, data_count=0, sid=33, subscriptionid=67, low=0.0, high=
0.0, to=0.0, mask=5)
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_data.py", line 367, in subscribe
    metadata, values = self._content[data_type]
KeyError: <ChannelType.TIME_ENUM: 17>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_backend.py", line 114, in enum_to_int
    return enum_strings.index(v)
ValueError: tuple.index(x): x not in tuple

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_backend.py", line 325, in convert_values
    enum_strings=enum_strings)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_backend.py", line 123, in _preprocess_enum_values
    return [enum_to_int(v) for v in values]
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_backend.py", line 123, in <listcomp>
    return [enum_to_int(v) for v in values]
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_backend.py", line 116, in enum_to_int
    raise CaprotoConversionError(f'Invalid enum string: {v!r}')
caproto._utils.CaprotoConversionError: Invalid enum string: ''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/server/common.py", line 182, in _command_queue_ite
ration
    response = await self._process_command(command)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/server/common.py", line 533, in _process_command
    sub)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_data.py", line 371, in subscribe
    metadata, values = await self._read(data_type)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_data.py", line 410, in _read
    direction=ConversionDirection.TO_WIRE)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/_backend.py", line 327, in convert_values
    raise CaprotoConversionError() from ex
caproto._utils.CaprotoConversionError

Retrieve timestamp from Context() in caproto.threading.client 

>>> ctx = Context()
>>> img,t1,t2 = ctx.get_pvs(default_prefix+'image',default_prefix+'t1',default_prefix+'t2')
>>> t1
<PV name='io_device:t1' priority=0 address=('255.255.5.59', 5064), circuit_state=States.CONNECTED, channel_state=States.CONNECTED>

threading.client behaves different from sync.client 

In [11]: ctx.get_pvs?
timeout : number or None, optional
    Number of seconds before a CaprotoTimeoutError is raised. This
    default can be overridden for any specific operation. By default,
    fall back to the default timeout set by the Context. If None, never
    timeout.

In [8]: RBV, = ctx.get_pvs('NIH:TEMP.RBV', timeout = 20)

Exception in thread search:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/threading/client.py", line 1113, in _process_search_results_l
oop
    cm = self.get_circuit_manager(address, pv.priority)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/threading/client.py", line 1161, in get_circuit_manager

    cm = VirtualCircuitManager(self, circuit, self.selector)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/caproto/threading/client.py", line 1313, in __init__
    raise CaprotoTimeoutError(f"Circuit with server at {host}:{port} "
caproto._utils.CaprotoTimeoutError: Circuit with server at 128.231.5.169:5068 did not connect within 2.0-second timeout.

user:T user-13$ caproto-get NIH:TEMP.RBV 
NIH:TEMP.RBV                              [23.403]
user:T user-13$ caproto-get NIH:TEMP.RBV 
NIH:TEMP.RBV                              [23.403]

In [14]: from caproto.sync.client import read

In [15]: read('NIH:TEMP.RBV')
Out[15]: ReadNotifyResponse(data=array([23.40296364]), data_type=<ChannelType.DOUBLE: 6>, data_count=1, status=CAStatusCode(name='ECA_NORMAL', code=0, code
_with_severity=1, severity=<CASeverity.SUCCESS: 1>, success=1, defunct=False, description='Normal successful completion'), ioid=0, metadata=None)

Report issue TODO

the string PV from

epics.wx.PVTextCtrl(self.panel, pv= prefix+'VALVE')

doesn't get passed propertly to caproto server with

VALVE = pvproperty(value='n', max_length = 1, dtype=str)

caproto vs pyepics client speed

I have been working on writing server and clients using caproto and pyepics libraries. I have a simple server written with caproto that has only one PV, '.image’. I can get the image with pyepics or caproto clients. It seems that pyepics client is 2 times slower than caproto. Does anyone know why caproto is faster than pyepics? Is it because the server is written in caproto?

The server code: https://github.com/vstadnytskyi/caproto-sandbox/blob/master/caproto_sandbox/io_camera_server.py
The client code: https://github.com/vstadnytskyi/caproto-sandbox/blob/master/caproto_sandbox/io_camera_client.py

Server is running on the same machine. The image size is 1,3960,3960

Results:
One request:
pyepics client: 2.068 seconds
caproto client: 0.856 seconds
Average of 10:
pyepics client: 1.203 seconds
caproto client: 0.628 seconds

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.