Need Barometer from Acurite -get from a file/socket? about weewx-sdr

[matthewwall]

create a weewx service that gets the pressure (from a sensor or from a file or from a web page or whatever) and adds that value to the loop or record (depending on the service binding).

https://github.com/weewx/weewx/wiki/add-sensor

[n7qnm]

Hi, Matt – thanks for the quick response. Hope you’ll forgive the “newbie” questions. Got the example modified and working – I’m a bit unclear about the timing and implications around converting this to a socket reader. What I want to do is read a continuous stream of APRS data, and pluck off one station’s pressure. The data I want shows up about once every 10 minutes, so the service code would just be “spinning” waiting for that to happen. Is that going to cause issues with the rest of weewx? And, when I find the data and post it, should I just keep reading, or stop and restart. Thanks! Clay From: Matthew Wall [mailto:[email protected]] Sent: Sunday, February 5, 2017 5:44 AM To: matthewwall/weewx-sdr <[email protected]> Cc: n7qnm <[email protected]>; Author <[email protected]> Subject: Re: [matthewwall/weewx-sdr] Need Barometer from Acurite -get from a file/socket? (#11) create a weewx service that gets the pressure (from a sensor or from a file or from a web page or whatever) and adds that value to the loop or record (depending on the service binding). https://github.com/weewx/weewx/wiki/add-sensor — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AYYIqejkUqkHgMHQTjuW5yFiuDDha8Snks5rZdIfgaJpZM4L3a2N> . <https://github.com/notifications/beacon/AYYIqTT_CVWbgwlIT5bk-PHPAudvIwGNks5rZdIfgaJpZM4L3a2N.gif>

[matthewwall]

The data I want shows up about once every 10 minutes, so the service code would just be “spinning” waiting for that to happen. Is that going to cause issues with the rest of weewx? And, when I find the data and post it, should I just keep reading, or stop and restart.

a data collection service can be bound to each NEW_LOOP_PACKET event or to each NEW_ARCHIVE_RECORD event. in your case, go with the latter. so on each archive record your service would be invoked. no 'spinning' involved. just make sure that whatever you do to get the barometric data does not block for a very long time.

[n7qnm]

OK – it’ll basically block for 10 minutes or more. Here’s the code I’m currently using and it seems to be working (sloppy, I know – just testing right now) import socket import sys import syslog import weewx from weewx.wxengine import StdService class APRSBaroService(StdService): def __init__(self, engine, config_dict): super(APRSBaroService, self).__init__(engine, config_dict) d = config_dict.get('APRSBaroService', {}) self.bind(weewx.NEW_ARCHIVE_RECORD, self.read_socket) def read_socket(self, event): sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server = ('Linux2', 2023) sock.connect(server) while 1 : data = sock.recv(250) if data.find("N7QNM-10") != -1 : ind = data.find("b") if ind == -1 : break value = float(data[ind+1:ind+6]) syslog.syslog(syslog.LOG_DEBUG, "APRSBaro: found value of %s" % value) event.record['barometer'] = value * .002953 break sock.close From: Matthew Wall [mailto:[email protected]] Sent: Sunday, February 5, 2017 6:53 PM To: matthewwall/weewx-sdr <[email protected]> Cc: n7qnm <[email protected]>; Author <[email protected]> Subject: Re: [matthewwall/weewx-sdr] Need Barometer from Acurite -get from a file/socket? (#11) The data I want shows up about once every 10 minutes, so the service code would just be “spinning” waiting for that to happen. Is that going to cause issues with the rest of weewx? And, when I find the data and post it, should I just keep reading, or stop and restart. a data collection service can be bound to each NEW_LOOP_PACKET event or to each NEW_ARCHIVE_RECORD event. in your case, go with the latter. so on each archive record your service would be invoked. no 'spinning' involved. just make sure that whatever you do to get the barometric data does not block for a very long time. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AYYIqftoFVgAzgtQkFgqCNLGowaWF4ZBks5rZosigaJpZM4L3a2N> . <https://github.com/notifications/beacon/AYYIqazgMawablKxG3tY5TSkJ0KS-L3Uks5rZosigaJpZM4L3a2N.gif>

[matthewwall]

OK – it’ll basically block for 10 minutes or more.

you don't want to block in a service - it can screw up the rest of the weewx data processing.

put the socket read in a thread. the thread just opens a socket and waits for data. when it times out, it checks to see if the thread should still run, then tries to connect again. when it receives data, it puts data into a common variable.

then the NEW_ARCHIVE_RECORD callback (running in the main thread) just reads whatever is in the variable. (if you want to distinguish the age of the data, then use a queue instead of a dict)

[n7qnm]

OK – that’s what I was kinda thinking/afraid of. I’m a bit new to Python (and weewx), and am going to have to figure that out. Meantime, I think what I’ll do is just write a daemon that reads the socket forever and writes a file, and then read the file to get the data into weewx. Not elegant; but it will work :). Thanks! Clay From: Matthew Wall [mailto:[email protected]] Sent: Sunday, February 5, 2017 10:09 PM To: matthewwall/weewx-sdr <[email protected]> Cc: n7qnm <[email protected]>; Author <[email protected]> Subject: Re: [matthewwall/weewx-sdr] Need Barometer from Acurite -get from a file/socket? (#11) OK – it’ll basically block for 10 minutes or more. you don't want to block in a service - it can screw up the rest of the weewx data processing. put the socket read in a thread. the thread just opens a socket and waits for data. when it times out, it checks to see if the thread should still run, then tries to connect again. when it receives data, it puts data into a common variable. then the NEW_ARCHIVE_RECORD callback (running in the main thread) just reads whatever is in the variable. (if you want to distinguish the age of the data, then use a queue instead of a dict) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AYYIqX9d93j26a_TYJK1FiW7WQnXYnafks5rZrjrgaJpZM4L3a2N> . <https://github.com/notifications/beacon/AYYIqT_YI7RpdlzoDGG1cU9_U_BkYAUyks5rZrjrgaJpZM4L3a2N.gif>

[matthewwall]

threading is really easy in python. take a look at this driver for an example:

https://github.com/matthewwall/weewx-ws3000

[n7qnm]

OK – so, if I’m reading this right, I’d put the thread call into the Service? And then thread would just print the value when it captured it? Thanks! From: Matthew Wall [mailto:[email protected]] Sent: Sunday, February 5, 2017 10:24 PM To: matthewwall/weewx-sdr <[email protected]> Cc: n7qnm <[email protected]>; Author <[email protected]> Subject: Re: [matthewwall/weewx-sdr] Need Barometer from Acurite -get from a file/socket? (#11) threading is really easy in python. take a look at this driver for an example: https://github.com/matthewwall/weewx-ws3000 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AYYIqdOrgwPrSNJBYVlifFleA667e2x7ks5rZryDgaJpZM4L3a2N> . <https://github.com/notifications/beacon/AYYIqeXzsaKcXPmpWO9yVCWCx8Z-BdKWks5rZryDgaJpZM4L3a2N.gif>

[matthewwall]

something like this (the socket logic could be streamlined, but this should convey the idea):

import socket
import time
import weewx
from weewx.engine import StdService

class PressureService(StdService):
    def __init__(self, engine, config_dict):
        super(PressureService, self).__init__(engine, config_dict)
        d = config_dict.get('PressureService', {})
        self.server_name = d.get('server', 'localhost')
        # the thread will set this each time it reads a new value.
        # the service will read this each archive interval.
        self.pressure = None
        # start the thread that captures the pressure value
        self._thread = PressureThread(self, self.server_name)
        self._thread.start()
        # bind this service to happen with each new archive record
        self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record)

    def shutDown(self):
        # when the service shuts down, tell the thread to die and wait for it
        if self._thread:
            self._thread.running = False
            self._thread.join()
            self._thread = None

    def new_archive_record(self, event):
        # add the pressure value to the record.  this assumes that
        # the pressure is in units appropriate to the record's unit system.
        event.record['barometer'] = self.pressure

class PressureThread(threading.Thread):
    def __init__(self, service, server_name):
        threading.Thread.__init__(self)
        self.service = service
        self.server_name = server_name
        self.running = False

    def run(self):
        self.running = True
        while self.running:
            try:
                sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                server = (self.server_name, 2023)
                sock.connect(server)
                while self.running:
                    data = sock.recv(1024)
                    if data and data.find("N7QNM-10") != -1:
                        idx = data.find("b")
                        if idx >= 0:
                            p = float(data[idx+1:idx+6]) * 0.002953
                            service.pressure = p
            except socket.error:
                pass
            time.sleep(1)

[n7qnm]

Wow! Didn’t expect a rewrite. Thank you VERY much! This looks good, and I definitely learned something. Clay From: Matthew Wall [mailto:[email protected]] Sent: Monday, February 6, 2017 11:29 AM To: matthewwall/weewx-sdr <[email protected]> Cc: n7qnm <[email protected]>; Author <[email protected]> Subject: Re: [matthewwall/weewx-sdr] Need Barometer from Acurite -get from a file/socket? (#11) something like this (the socket logic could be streamlined, but this should convey the idea): import socket import time import weewx from weewx.engine import StdService class PressureService(StdService): def __init__(self, engine, config_dict): super(PressureService, self).__init__(engine, config_dict) d = config_dict.get('PressureService', {}) self.server_name = d.get('server', 'localhost') # the thread will set this each time it reads a new value. # the service will read this each archive interval. self.pressure = None # start the thread that captures the pressure value self._thread = PressureThread(self, self.server_name) self._thread.start() # bind this service to happen with each new archive record self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record) def shutDown(self): # when the service shuts down, tell the thread to die and wait for it if self._thread: self._thread.running = False self._thread.join() self._thread = None def new_archive_record(self, event): # add the pressure value to the record. this assumes that # the pressure is in units appropriate to the record's unit system. event.record['barometer'] = self.pressure class PressureThread(threading.Thread): def __init__(self, service, server_name): threading.Thread.__init__(self) self.service = service self.server_name = server_name self.running = False def run(self): self.running = True while self.running: try: sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server = (self.server_name, 2023) sock.connect(server) while self.running: data = sock.recv(1024) if data and data.find("N7QNM-10") != -1: idx = data.find("b") if idx >= 0: p = float(data[idx+1:idx+6]) * 0.002953 service.pressure = p except socket.error: pass time.sleep(1) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AYYIqSca_PNOFzz6qaimDzX_H-WybcXqks5rZ3R4gaJpZM4L3a2N> . <https://github.com/notifications/beacon/AYYIqWuqDJYBSJPyCetmdL0zG_pl5Aerks5rZ3R4gaJpZM4L3a2N.gif>

[n7qnm]

Hi – one more quick question, if I may. I assume that this (self.server_name = d.get('server', 'localhost') is a call to the dict “get” method that will return the server name if it’s there, or default to localhost. So, somewhere in weewx.config, I’d put something like: [PressureService] Server = Linux2 Is that correct? So, given that, I could put the callsign (N7QNM-10) and the port (2023) in parameters as well, correct? Is there anything critical in the placement of that stanza? Thanks again! Clay From: Matthew Wall [mailto:[email protected]] Sent: Monday, February 6, 2017 11:29 AM To: matthewwall/weewx-sdr <[email protected]> Cc: n7qnm <[email protected]>; Author <[email protected]> Subject: Re: [matthewwall/weewx-sdr] Need Barometer from Acurite -get from a file/socket? (#11) something like this (the socket logic could be streamlined, but this should convey the idea): import socket import time import weewx from weewx.engine import StdService class PressureService(StdService): def __init__(self, engine, config_dict): super(PressureService, self).__init__(engine, config_dict) d = config_dict.get('PressureService', {}) self.server_name = d.get('server', 'localhost') # the thread will set this each time it reads a new value. # the service will read this each archive interval. self.pressure = None # start the thread that captures the pressure value self._thread = PressureThread(self, self.server_name) self._thread.start() # bind this service to happen with each new archive record self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record) def shutDown(self): # when the service shuts down, tell the thread to die and wait for it if self._thread: self._thread.running = False self._thread.join() self._thread = None def new_archive_record(self, event): # add the pressure value to the record. this assumes that # the pressure is in units appropriate to the record's unit system. event.record['barometer'] = self.pressure class PressureThread(threading.Thread): def __init__(self, service, server_name): threading.Thread.__init__(self) self.service = service self.server_name = server_name self.running = False def run(self): self.running = True while self.running: try: sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server = (self.server_name, 2023) sock.connect(server) while self.running: data = sock.recv(1024) if data and data.find("N7QNM-10") != -1: idx = data.find("b") if idx >= 0: p = float(data[idx+1:idx+6]) * 0.002953 service.pressure = p except socket.error: pass time.sleep(1) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AYYIqSca_PNOFzz6qaimDzX_H-WybcXqks5rZ3R4gaJpZM4L3a2N> . <https://github.com/notifications/beacon/AYYIqWuqDJYBSJPyCetmdL0zG_pl5Aerks5rZ3R4gaJpZM4L3a2N.gif>

[matthewwall]

Is that correct? So, given that, I could put the callsign (N7QNM-10) and the port (2023) in parameters as well, correct?

right-o!

    self.server_name = d.get('server', 'localhost')
    self.port = int(d.get('port', 2023))
    self.callsign = d.get('callsign', 'N7QNM-10')

d is a python dict, extracted from config_dict, which is a dict created from weewx.conf.

the keys of d are case-sensitive.

[n7qnm]

Hi, Matt – well, I think I’m ALMOST there – I added some debug statements to the code and I’m getting pressure readings; but, when I get to the event_record, self.pressure is always “None”. Looks like it’s not getting passed back from the thread. Any suggestions? Thanks! Clay import socket import threading import time import syslog import weewx from weewx.engine import StdService class PressureService(StdService): def __init__(self, engine, config_dict): super(PressureService, self).__init__(engine, config_dict) d = config_dict.get('PressureService', {}) self.server_name = d.get('server', 'Linux2') # the thread will set this each time it reads a new value. # the service will read this each archive interval. self.pressure = None # start the thread that captures the pressure value self._thread = PressureThread(self, self.server_name) self._thread.start() # bind this service to happen with each new archive record self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record) def shutDown(self): # when the service shuts down, tell the thread to die and wait for it if self._thread: self._thread.running = False self._thread.join() self._thread = None def new_archive_record(self, event): # add the pressure value to the record. this assumes that # the pressure is in units appropriate to the record's unit system. event.record['barometer'] = self.pressure syslog.syslog(syslog.LOG_DEBUG, "APRSBaro: event pressure %s" % self.pressure) class PressureThread(threading.Thread): def __init__(self, service, server): threading.Thread.__init__(self) self.service = service self.server_name = server self.running = False def run(self): self.running = True while self.running: try: sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server = (self.server_name, 2023) sock.connect(server) while self.running: data = sock.recv(1024) if data and data.find("N7QNM-10") != -1: idx = data.find("b") if idx >= 0: p = float(data[idx+1:idx+6]) * 0.002953 syslog.syslog(syslog.LOG_DEBUG, "APRSBaro: pressure %s" % p) service.pressure = p except socket.error: pass time.sleep(1) From: Matthew Wall [mailto:[email protected]] Sent: Monday, February 6, 2017 11:29 AM To: matthewwall/weewx-sdr <[email protected]> Cc: n7qnm <[email protected]>; Author <[email protected]> Subject: Re: [matthewwall/weewx-sdr] Need Barometer from Acurite -get from a file/socket? (#11) something like this (the socket logic could be streamlined, but this should convey the idea): import socket import time import weewx from weewx.engine import StdService class PressureService(StdService): def __init__(self, engine, config_dict): super(PressureService, self).__init__(engine, config_dict) d = config_dict.get('PressureService', {}) self.server_name = d.get('server', 'localhost') # the thread will set this each time it reads a new value. # the service will read this each archive interval. self.pressure = None # start the thread that captures the pressure value self._thread = PressureThread(self, self.server_name) self._thread.start() # bind this service to happen with each new archive record self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_record) def shutDown(self): # when the service shuts down, tell the thread to die and wait for it if self._thread: self._thread.running = False self._thread.join() self._thread = None def new_archive_record(self, event): # add the pressure value to the record. this assumes that # the pressure is in units appropriate to the record's unit system. event.record['barometer'] = self.pressure class PressureThread(threading.Thread): def __init__(self, service, server_name): threading.Thread.__init__(self) self.service = service self.server_name = server_name self.running = False def run(self): self.running = True while self.running: try: sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server = (self.server_name, 2023) sock.connect(server) while self.running: data = sock.recv(1024) if data and data.find("N7QNM-10") != -1: idx = data.find("b") if idx >= 0: p = float(data[idx+1:idx+6]) * 0.002953 service.pressure = p except socket.error: pass time.sleep(1) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AYYIqSca_PNOFzz6qaimDzX_H-WybcXqks5rZ3R4gaJpZM4L3a2N> . <https://github.com/notifications/beacon/AYYIqWuqDJYBSJPyCetmdL0zG_pl5Aerks5rZ3R4gaJpZM4L3a2N.gif>

[matthewwall]

what does the log say?

that code won't run - you'll get an exception in the pressure thread as soon as you hit the service.pressure = p line. try self.service.pressure = p instead.

please continue this discussion by posting at the weewx dev forum:

http://groups.google.com/group/weewx-development

[n7qnm]

Thanks! Actually, that change did the trick – and I’m getting a better understanding of scoping in Python :). Thanks again for all of the assistance, I’ll move back to the forums now. Clay From: Matthew Wall [mailto:[email protected]] Sent: Wednesday, February 8, 2017 4:05 AM To: matthewwall/weewx-sdr <[email protected]> Cc: n7qnm <[email protected]>; Author <[email protected]> Subject: Re: [matthewwall/weewx-sdr] Need Barometer from Acurite -get from a file/socket? (#11) what does the log say? that code won't run - you'll get an exception in the pressure thread as soon as you hit the service.pressure = p line. try self.service.pressure = p instead. please continue this discussion by posting at the weewx dev forum: http://groups.google.com/group/weewx-development — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AYYIqYLj9QQA01fppUL48NszUdy8-vd7ks5raa9igaJpZM4L3a2N> . <https://github.com/notifications/beacon/AYYIqSsm7-AorMlKGxETMmKqCgeZlAZcks5raa9igaJpZM4L3a2N.gif>