rpj / rpi Goto Github PK
View Code? Open in Web Editor NEWRPJiOS: RPJ's RPi OS, a sensor data platform for the Raspberry Pi built with python2.7 and redis.
Home Page: https://rpjios.com
License: MIT License
RPJiOS: RPJ's RPi OS, a sensor data platform for the Raspberry Pi built with python2.7 and redis.
Home Page: https://rpjios.com
License: MIT License
Per this calculator from the EPA, concentrate-to-AQI JS source here.
REDIS_URL
handling needs to be fixed, is really the point.
_runloop
is really _tick
: fix this, and improve the implementation in general (to allow both!)
It's going to have to happen someday...
On a reboot, a device may have a localtime value that is stale from the last boot, and before the device updates time (from NTP) it can start publishing sensor data with stale timestamps. This isn't a huge deal, but it does cause dirty data.
A couple folks on Hackster.io have gotten in touch with trouble building the SPS30 submodule via setup.sh
, and I've been able to reproduce it on stock current Raspbian (2018-11-13):
pi@raspberrypi:~/rpi $ ./setup.sh
*** RPi platform detected: building sensor drivers and including hardware interface modules.
* Looking for virtualenv: NOT FOUND
* Trying to install virtualenv...
* Looking for redis-server: NOT FOUND
* Trying to install redis-server...
* Looking for zip: NOT FOUND
* Trying to install zip...
* Looking for python-dev: NOT FOUND
* Trying to install python-dev...
* Looking for libjpeg9-dev: NOT FOUND
* Trying to install libjpeg9-dev...
* Looking for python-smbus: NOT FOUND
* Trying to install python-smbus...
* Fetching embedded-sps submodule: The authenticity of host 'github.com (192.30.253.113)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
./setup.sh: line 45: pushd: embedded-sps: No such file or directory
done
* Building embedded-sps submodule: ./setup.sh: line 51: pushd: release/sps30-i2c: No such file or directory
./setup.sh: line 52: pushd: hw_i2c: No such file or directory
mv: cannot stat 'sensirion_hw_i2c_implementation.c': No such file or directory
./setup.sh: line 55: popd: directory stack empty
failure!
./setup.sh: line 64: popd: directory stack empty
./setup.sh: line 65: popd: directory stack empty
* Initializing virtualenv: ^Cfailed! Cannot continue.
Could even implicitly enable via a URL scheme for -o
such as thingspeak://WRITE_API_KEY@CHANNEL_ID
. Also, downsample
really needs to be re-factored. Should probably file a separate bug for that...: #10
Need to make this configurable and be sure it's setup to happen every so often (~once/week)!
Will require a re-thinking of sensor runlooping (and therefore is related to #3), as ideally any NMEA stream-producing sensor should act in a push-only manner rather than the current sensor-src
-tick-based-pull mechanism (confusingly named _runloop
, hence #3).
Redis 5 streams would come in really handy here, but an implementation based on lists and using BLPOP
to block consumers until a new datum is available could work well too.
Doit. Doit.
Ugh.
Redis 5.0 landed the feature that this code base half-implements! The examples in the docs are even sensor-based! Re-implement using streams, obviously. (Will need relatively wide adoption of v5, though)
So the discover information auto-cleans itself up after a bit of time (could be awhile too, 15 minutes or even an hour).
The thingspeak
tool seems to hang very occasionally after running fine for long periods (~days), and seems to do so somewhere in the PSubscriber code for listening to redis. That's about all I know now.
Hello,
I am a new to these types of systems and following the guild I managed to get my system up and running and data feeding into Thingspeak, Channel ID: 802605
Issue is the code is failing to a bad internet connection I have, as I run your system through a VPN.
Thought I set up the device in tehran for a laugh as I am on Holiday here.
Are you able to just keep requesting the URL in a constant loop until it connects again?
More code here:
Success: entry #792 added
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/pi/rpi/env/local/lib/python2.7/site-packages/rpjios/SubscriberBase.py", line 54, in _sub_thread
map(lambda sf: sf(m), self._subs)
File "/home/pi/rpi/env/local/lib/python2.7/site-packages/rpjios/SubscriberBase.py", line 54, in
map(lambda sf: sf(m), self._subs)
File "./bin/thingspeak", line 39, in msg_rx
_resp = urllib2.urlopen(_url)
File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
raise URLError(err)
URLError: <urlopen error [Errno -3] Temporary failure in name resolution>
^CTraceback (most recent call last):
File "./bin/thingspeak", line 57, in
time.sleep(0.5)
KeyboardInterrupt
Plus great job on the code even for a total nobe I got it working.
Cheers.
Mehdi
On charlie, the forwarding invocations of this seem to be leaking quite badly over long periods (these have been running for about 9 days):
ryan 3038 0.1 17.1 2233072 2090796 pts/1 S+ Jan06 20:54 python ./bin/downsample -i redis://u:XXX@localhost -o redis://h:[email protected]:15979 -p * -r 10 -t list:suffix=.list,limit=5000 -m flatten:mapping=mapping.json,include-raw=1
ryan 4796 0.1 16.5 2166996 2024552 pts/3 S+ Jan06 20:10 python ./bin/downsample -i redis://u:XXX@localhost -o redis://h:[email protected]:15979 -p * -r 10 -m flatten:mapping=mapping.json,include-raw=1 -t key
The cause isn't immediately obvious, though I'm suspecting the internal windowed cache isn't windowing or something.
The downsample
tool has grown far too ungainly as it has gained features (and still needs to gain another feature with the integration of the thingspeak
forwarder, #8). It needs to re-factored, maybe re-thought. There's also that send-outside-the-send-thread problem with lists. That's bad, mmkay.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.