Comments (44)
I can second this problem. Everything sounds identical. Any updates/information would be appreciated.
from weewx-belchertown.
from weewx-belchertown.
Just thinking.... do I have to open both ports (MQTT and Websockets) on the router?
from weewx-belchertown.
Well this guide from HiveMQ suggests that you could be right ;
https://www.hivemq.com/blog/mqtt-security-fundamentals-securing-mqtt-systems
......
Firewall
Every connection to an MQTT broker should pass at least one firewall that implements sophisticated rules for accessing downstream components. If your are able to block attackers at the firewall level, they won’t be able to access other systems in your infrastructure. Unfortunately, there is no universally applicable solution. Firewall rules need to be configured individually for each use case. Many commercial and open source firewall solutions are currently available.
A solid rule of thumb is: only forward expected traffic to downstream systems. Block any traffic that you don’t expect in your downstream applications. For an MQTT broker the following traffic is probably worth blocking:
UDP: MQTT uses TCP, you can block all UDP datagram packets.
ICMP: While it may not be the smartest idea to block all ICMP traffic, ICMP traffic, ping and traceroute ICMP packets could be worth investigating as candidates to block.
It’s also a good idea to block traffic to any ports that are not needed for your MQTT system.
Don’t block the following standard MQTT ports with your firewall:
1883: This is the default MQTT port. 1883 is defined at IANA as MQTT over TCP.
8883: This is the default MQTT port for MQTT over TLS. It’s registered at IANA for Secure MQTT.
If you are in full control of your MQTT system and know the IPs of your MQTT clients, only allow traffic for the needed IP ranges. This precaution locks out any clients that are not in the defined IP ranges.
from weewx-belchertown.
I think I have solved my problem. I don't know if this is related, as I didn't realize before that you were going through a router.
I've got weewx and mosquitto running on the same server all local. My problem was that in the weewx.conf file, I had the server listed as "localhost" which seems to be an ipv6 address. This works fine for port 1883 as mosquitto supports ipv4 and ipv6 for publishing, but for websockets only ipv4 is supported (or that's what it seems like due to to this post. So I just changed the server to the local FQDN using ipv4 and it's working perfectly.
Sorry if this isn't related to your problem, I just figured I would post this here so it might help someone in the future.
from weewx-belchertown.
Thanks for all your suggestions. I replaced localhost with the IP address in the weewx.conf and it did NOT solve my issue. Since the station is remote, the changes in the router (forward port 1883) have to wait until I have access to the router (hopefully on the week-end). I will keep you posted.
from weewx-belchertown.
Right, well all I opened in my home firewall was TCP 9001 and it's working fine off my home mqtt broker @ www.41south.net.nz.
from weewx-belchertown.
Oh no, you are killing me... that was my best hope to solve the issue :(
from weewx-belchertown.
If you want to test your skin config you could subscribe to my simulated weewx data. Just point your skin.conf mqtt at 41south.net.nz:9001 (ssl = 0) - the topic is weather/41south/loop (or #)
from weewx-belchertown.
Thanks, using an online websocket client I can connect to your broker and susbscribe to your data. However, using the same websocket client, I can connect to my system but I can't subscribe. It 'feels' like the connection is not fully established. I changed my mosquitto conf file to log everything and I see the following:
1539820643: New client connected from 199.19.95.109 as clientId-53vcqTiEW6 (c1, k60).
1539820643: Sending CONNACK to clientId-53vcqTiEW6 (0, 0)
1539820694: Socket error on client clientId-53vcqTiEW6, disconnecting.
At least I have now an error message to investigate :)
from weewx-belchertown.
How's it going? Any progress?
from weewx-belchertown.
@mash55 I'm going to close this. Let me know if you need more help though
from weewx-belchertown.
No progress... thanks for asking. I continue to get SOCKET ERROR message when trying to access data via websockets. Tried different OS version (Jessie and Wheezy) without any luck. Not sure whether I should try and compile mosquito with websocket support from scratch on a raspberry pi. Anyhow, it has to wait until I am back up in the mountain. Any other suggestions would be appreciated.
from weewx-belchertown.
Were you able to reach the MQTT data using an MQTT client?
from weewx-belchertown.
Nope :(
Same errorin log file
from weewx-belchertown.
from weewx-belchertown.
I would agree. From the same host that has MQTT installed on it, try to connect locally. Firewall shouldn't be in play at that point.
I also wrote a blog post that may be helpful if you haven't seen it yet.
from weewx-belchertown.
Well, I have both ports open on the router/firewall. A local MQTT client supporting Websockets might be an option. Will go up to the cabin on the weekend and will see to find a client which supports websockets on windows or android so I can test within the local area network. Any suggestions for such a client?
Thanks...
from weewx-belchertown.
Edit: Updating some details for clarity
At this point I think it might be important to just have MQTT working. Once you can verify that weewx is sending data to MQTT and your mosquitto_sub
client can see that data, then I think you can look into websockets. See if local websockets work, then try externally.
Remember that websockets is a layer on-top of the MQTT broker. So it's important to have MQTT itself working.
-
I'd try
mosquitto_sub
on the Linux host first (see my blog post for details - linked above) -
Then I'd try your phone to connect to the broker, subscribe to the topic and check for data. Your weewx topic should be the
loop
topic. For examplemashNET/loop
. You can subscribe tomashNET/#
for testing but only use that in testing.
- You mention Android, so on Android I use MQTT Client. This does MQTT and websockets.
-
So in the app add a new broker and do not select websockets. Fill in the host, port and client ID then save. Connect to the broker and subscribe to
mashNET/#
and see if you see anything. If so great, move on. If not, stop and re-check your broker settings, your firewall, and make sure mosquitto is running on the server. -
Add a 2nd broker in the app and this time tick the websocket checkbox, and make sure you define your websockets port. Fill out the host and client ID and subscribe to the topic and you should see everything. If you do, it's a matter of configuring the skin settings and restarting weewx.
Hope this helps! Do read my blog post, I know it has helped others in past who were stuck.
from weewx-belchertown.
MQTT is running just fine. weewx is publishing and I can subscribe just fine (LAN and WAN). Problem is related to websockets. I read your blog (almost memorized it by now ;) )
Thanks for the android client... will test during the week-end.
from weewx-belchertown.
OK cool.
Yeah I saw your initial post - but when I get stuck I always start back at the basics. So I wanted to lay out what I'd try first.
So if your apps are connecting fine, then it's something with the skin settings or firewall ports on the websocket server or router or even the port forwarding from router to the websocket server. So many elements to double check.
from weewx-belchertown.
from weewx-belchertown.
Thanks guys, will report back after the week-end.
from weewx-belchertown.
Her the latest update:
Installed MQTTBox on Windows 10 machine.
MQTTBox and mosquitto broker on same subnet.
Connected (to IP address) and subscribed via mqtt/tcp protocol
Worked
Connected (to IP adddress) and subscribed via ws protocol
Client shows a connection but collapses after a few seconds
Entry in mosquitto log-file:
1542480562: New client connected from 10.66.78.136 as mash_test_windows1542479922590 (c1, k10).
1542480562: Sending CONNACK to mash_test_windows1542479922590 (0, 0)
1542480577: Client mash_test_windows1542479922590 has exceeded timeout, disconnecting.
1542480590: Socket error on client mash_test_windows1542479922590, disconnecting.
I think its safe to say that my issue is NOT related to the router/firewall but to the mosquitto version/setup/config.
Not sure how to continue from here.
from weewx-belchertown.
from weewx-belchertown.
Still here :)
mosquitto conf file:
# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example
pid_file /var/run/mosquitto.pid
persistence true
persistence_location /var/lib/mosquitto/
log_dest file /var/log/mosquitto/mosquitto.log
include_dir /etc/mosquitto/conf.d
Thats the original file. I am overwriting with my own config file (/etc/mosquitto/conf.d/mystart.conf):
persistence false
log_type websockets error
# log_type all
allow_anonymous true
listener 1883
protocol mqtt
listener 9001
protocol websockets
hostname: cabin.schnizer.com
ports: 1883 (mqqt) and 9001 (ws) respectively.
Appreciate your help...
from weewx-belchertown.
from weewx-belchertown.
mashNET/loop
from weewx-belchertown.
from weewx-belchertown.
you could connect to 1883, right? Just websockets is a nogo?
Thanks for trying...
from weewx-belchertown.
from weewx-belchertown.
Dropout might have been due to heavy snowfall (wireless link up here)...
from weewx-belchertown.
from weewx-belchertown.
from weewx-belchertown.
I also was able to connect to your MQTT broker without any issue and subsribed to mashNET/loop
too.
I added your MQTT details to my lab server and was getting the following errors. It connects to your websocket, but doesn't seem to subscribe correctly or it's constantly being disconnected and reconnecting or something. I'm a little confused by it. It's retrying many times per millisecond - so something's gotta be off somewhere.
Edit: re-read you said you installed it from another repo. I wonder if that repo's install has something wrong with the websockets portion?
I haven't installed the broker on a Pi yet, but I have a couple laying around. I might have to try it out tomorrow.
from weewx-belchertown.
I'm looking at the copy/paste you did of your /etc/mosquitto/conf.d/mystart.conf
and I don't know if it's GitHub or not, but it looks like you have extra spaces at the end of each config line. Can you confirm if they are there on the system? If so, remove them, then restart and see if that helps.
The MQTT broker is super picky about its config (like white spaces).
It might be a long-shot, but worth investigating.
from weewx-belchertown.
The broker drops the connection (via ws) even without subscribing.... it takes a few seconds (~25) until the error and retry.
I checked the start-up file for white spaces but they were introduced during copy/paste.
The reason for using a different repo is the fact that the default repo version of mosquitto is not compiled with WS support. Installed it with 'apt-get' from 'repo.mosquitto.org/debian/mosquitto-jessie.list' on a 'Raspberry Pi 3 Model B Rev 1.2' using 'Raspbian GNU/Linux 8 (jessie)'. Also tried the broker on a 'Raspbian GNU/Linux 7 (wheezy)' version with the same result.
Let me know what you find out on raspberry platform if you have the time.
Many thanks
from weewx-belchertown.
I did some testing today with good results.
- tl;dr Raspberry Pi Stretch and doing
sudo apt-get install mosquitto mosquitto-clients
works with websockets and the Belchertown skin just fine. Read on for more details.
My Raspberry Pi I'm using has Stretch installed
pi@garagepi:~ $ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
pi@garagepi:~ $
I then ran sudo apt update
and sudo apt-get install mosquitto mosquitto-clients
from the standard apt repository.
pi@garagepi:~ $ sudo apt-get install mosquitto mosquitto-clients
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
lxkeymap python-cairo python-gobject python-gobject-2 python-gtk2 python-xklavier
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
libc-ares2 libev4 libmosquitto1 libwebsockets8
Suggested packages:
apparmor
The following NEW packages will be installed:
libc-ares2 libev4 libmosquitto1 libwebsockets8 mosquitto mosquitto-clients
0 upgraded, 6 newly installed, 0 to remove and 107 not upgraded.
Need to get 418 kB of archives.
After this operation, 897 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
I can see from the initial command that it installed libwebsockets8
so I just continued on.
Once it was installed, I added a quick config with sudo nano /etc/mosquitto/conf.d/myconfig.conf
and added a very basic, non-secured config:
persistence false
allow_anonymous true
listener 1883
protocol mqtt
# websockets
listener 9001
protocol websockets
Then I tried to run this config with
pi@garagepi:~ $ sudo mosquitto -c /etc/mosquitto/mosquitto.conf
pi@garagepi:~ $
But it failed. It didn't do anything. So I wanted to see if the port was open or not and did sudo netstat -tulpn
and it returned this:
pi@garagepi:~ $ sudo netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 516/mysqld
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 406/sshd
tcp 0 0 0.0.0.0:1883 0.0.0.0:* LISTEN 4715/mosquitto
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 382/blink1-tiny-ser
tcp6 0 0 :::22 :::* LISTEN 406/sshd
tcp6 0 0 :::1883 :::* LISTEN 4715/mosquitto
udp 0 0 0.0.0.0:42445 0.0.0.0:* 284/avahi-daemon: r
udp 0 0 0.0.0.0:68 0.0.0.0:* 334/dhcpcd
udp 0 0 0.0.0.0:5353 0.0.0.0:* 284/avahi-daemon: r
udp6 0 0 :::546 :::* 334/dhcpcd
udp6 0 0 :::41606 :::* 284/avahi-daemon: r
udp6 0 0 :::5353 :::* 284/avahi-daemon: r
So if my config didn't work, but port 1883
is open then something else is running it. So I did a ps aux
and found that it was running under a new user mosquitto
that I didn't know about.
pi@garagepi:~ $ ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.6 27112 6116 ? Ss Oct28 0:27 /sbin/init splash
<TRUNCATED OUTPUT>
mosquit+ 4715 0.0 0.4 6332 3992 ? S 17:05 0:00 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
So I force killed that process with sudo kill -9 4715
and re-ran sudo mosquitto -c /etc/mosquitto/mosquitto.conf
. This time it didn't exit so quickly, it sat there (looking like it's done nothing). Checking the ports again I can see it's opened 1883
and 9001
:
pi@garagepi:~ $ sudo netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:9001 0.0.0.0:* LISTEN 5267/mosquitto
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 516/mysqld
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 406/sshd
tcp 0 0 0.0.0.0:1883 0.0.0.0:* LISTEN 5267/mosquitto
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 382/blink1-tiny-ser
tcp6 0 0 :::22 :::* LISTEN 406/sshd
tcp6 0 0 :::1883 :::* LISTEN 5267/mosquitto
udp 0 0 0.0.0.0:42445 0.0.0.0:* 284/avahi-daemon: r
udp 0 0 0.0.0.0:68 0.0.0.0:* 334/dhcpcd
udp 0 0 0.0.0.0:5353 0.0.0.0:* 284/avahi-daemon: r
udp6 0 0 :::546 :::* 334/dhcpcd
udp6 0 0 :::41606 :::* 284/avahi-daemon: r
udp6 0 0 :::5353 :::* 284/avahi-daemon: r
I opened 2 more SSH connections to the Pi.
On new connection 1 I ran mosquitto_sub -h localhost -t weather/#
to subscribe to the topic.
On new connection 2 I ran mosquitto_pub -h localhost -t "weather/test" -m "hello"
to publish to that topic.
So far so good - connection 1 can see the published update.
pi@garagepi:~ $ mosquitto_sub -h localhost -t weather/#
hello
Then on another server I have, I ran this websocket test utility that I found on GitHub (just download as zip, add to your webserver then browse to it with http://where-you-installed-it). They also have it hosted on the internet which may be easier - http://hobbyquaker.github.io/mqtt-admin but just know that GitHub will promote this to HTTPS and your broker is not using Secure Websockets it won't connect and will give you a white page. This is why I opted to host it myself on a non-https server. If you get stuck at a white screen you need to clear your cookies.
Anyways, once past that hurdle, I added my Pi and port:
Which connected me no problem. I switched to the subscribe tab and subscribed to weather/#
Then back into SSH connection 2 I re-published a couple of messages:
(MQTT saw them too, so it all appears to be working so far)
pi@garagepi:~ $ mosquitto_sub -h localhost -t weather/#
hello
hello
hello websockets?
hello websockets!
Finally I got around to weewx. I updated my weewx.conf with
[[Belchertown]]
skin = Belchertown
HTML_ROOT = /var/www/html/btownwx
[[[Extras]]]
mqtt_enabled = 1
mqtt_host = 192.168.0.34
mqtt_port = 9001
mqtt_ssl = 0
mqtt_topic = weather/weewx/loop
Restarted weewx and forced a skin update: root@weewxlab:~# service weewx restart; wee_reports; tail -f /var/log/syslog -n 50
.
I don't have any real data going to this new Pi broker, so I submitted some fake data:
pi@garagepi:~ $ mosquitto_pub -h localhost -t "weather/weewx/loop" -m '{"UV": "0.68", "barometer_inHg": "30.4306066431", "outHumidity": "45.8", "maxSolarRad": "399.067737313", "dewpoint_F": "17.6078693897", "cloudbase_foot": "4723.66604779", "heatindex_F": "36.5", "altimeter_inHg": "30.3995019668", "dayRain_in": "0.0", "outTemp_F": "36.5", "appTemp_F": "31.2879264251", "windchill_F": "36.5", "hourRain_in": "0.0", "humidex_F": "36.5", "rainRate_inch_per_hour": "0.0", "pressure_inHg": "29.9401590264", "usUnits": "1.0", "txBatteryStatus": "0.0", "windSpeed_mph": "1.0", "rain24_in": "0.0", "dateTime": "1542561977", "radiation_Wpm2": "124.813456"}'
and the page updated
So, I think if you can upgrade or install Raspbian Stretch, and just do a normal install of mosquitto you'll be in good shape? I'm not sure how you can uninstall the old mosquitto install - so maybe a fresh install of Raspbian if possible?
Let me know your thoughts.
from weewx-belchertown.
Thank you soooo much for all the work and infos. I was thinking about STRRECH before but as far as I know there is no supported uprade path and installing from scratch with all the other services on this box is a lot of work. I guess I have to look into ugrading again and make a decision how to get to STRETCH.
from weewx-belchertown.
from weewx-belchertown.
http://schnizer.net/ => :)
Decided to install from scratch on a virgin STRETCH release et voila...
Only issue right now is the 'disconnect_live_visitor' parameter. Would like to disconnect users after 5 minutes but no matter what I am using it disconnects after the default 30 min - puzzling
Thanks again for all your help (incl. Colin)... best from Colorado
from weewx-belchertown.
Looks great! Working for me here!
Ah, you may have found a bug introduced between versions! I changed the config name for that and will update in 0.8 from disconnect_live_visitor
, to disconnect_live_website_visitor
. If you want to get a head-start on that setting, try using disconnect_live_website_visitor
in your weewx.conf overrides.
Don't forget to restart weewx after you've updated it.
from weewx-belchertown.
from weewx-belchertown.
disconnect_live_website_visitor
worked perfectly - thanks
from weewx-belchertown.
Related Issues (20)
- Missing Charts HOT 11
- Upgrading to 1.3 from 1.2 HOT 2
- "drizzle" icon is missing during install and will show as broken in forecasts HOT 2
- Can't find locale HOT 1
- If Aeris throws an exception Belchertown doesn't update HOT 7
- Forecast finds the database locked, exception issue HOT 1
- Change in default charts on home page HOT 2
- Wind Rose does not seem accurate on the longer time lines HOT 1
- Kiosk.html not updating HOT 12
- Pi page showing "undefined" and unformatted values after access/refresh HOT 2
- Belchertown taking forever on some weewx v5 systems HOT 22
- 29th February HOT 1
- Graphs appear with delay on refresh
- In js/belchertown.js.tmpl function get_outTemp_color returns incorrect color when outTemp <= 0 and unit is "degree_C"
- Data is being rounded to 0
- View from Reports / NOAA
- display miles instead of kilometer HOT 2
- I am getting errors after upgrading to weewx v5.0.1 and belchertown 1.3.1 HOT 1
- Missing forecast icons from pages (except home page) HOT 9
- Current Conditions Stuck/Stale
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from weewx-belchertown.