markqvist / lxmf Goto Github PK

Thank you for your project it is very interesting !
Beginner here so there is certainly an issue with my configuration.

I am trying to setup a propagation node on a linux computer, using a LORA module, when I run LXMD I have a suspicious warning and then error messages every 5 seconds. Also, I am not able to confirm that propagation/message retention works (I can see the propagation node from another node, I have a few messages "On Propagation Net", sent from another node, through sideband, but when I open nomadenet from the propagation node, these messages never change status and never appear in my inbox)).

Suspicious message is :
[2023-01-27 14:13:00] [Warning] Clint authentication was enabled, but no identity hashes could be loaded from /home/hugo/.lxmd/allowed. Nobody will be able to sync messages from this propagation node.

This is suspicious because LXMD configuration contains auth_required = no.

Edit : after reading the code, it might be a false warning, from lxmd.py:325, the number of allowed entities is checked wether the auth_required is True or False.

Errors are :
[2023-01-27 14:13:10] [Error] An error occurred while running periodic jobs. The contained exception was: unsupported operand type(s) for +: 'float' and 'NoneType'

Edit : this error is due to the fact that the following line was commented : announce_interval = 360 under [lxmf] in the LXMD configuration file.

My configuration is the following :
LXMD

# This is an example LXM Daemon config file.
# You should probably edit it to suit your
# intended usage.

[propagation]

# Whether to enable propagation node
enable_node = true

# Automatic announce interval in minutes.
# 6 hours by default.
announce_interval = 360

# Whether to announce when the node starts.
announce_at_start = yes

# Wheter to automatically peer with other
# propagation nodes on the network.
autopeer = yes

# The maximum peering depth (in hops) for
# automatically peered nodes.
autopeer_maxdepth = 4


# By default, any destination is allowed to
# connect and download messages, but you can
# optionally restrict this. If you enable
# authentication, you must provide a list of
# allowed identity hashes in the a file named
# "allowed" in the lxmd config directory.
auth_required = no


[lxmf]

# The LXM Daemon will create an LXMF destination
# that it can receive messages on. This option sets
# the announced display name for this destination.
display_name = My Node

# It is possible to announce the internal LXMF
# destination when the LXM Daemon starts up.
announce_at_start = yes

[logging]
loglevel = 4

Reticulum

# This is the default Reticulum config file.
# You should probably edit it to include any additional,
# interfaces and settings you might need.

# Only the most basic options are included in this default
# configuration. To see a more verbose, and much longer,
# configuration example, you can run the command:
# rnsd --exampleconfig


[reticulum]

  # If you enable Transport, your system will route traffic
  # for other peers, pass announces and serve path requests.
  # This should only be done for systems that are suited to
  # act as transport nodes, ie. if they are stationary and
  # always-on. This directive is optional and can be removed
  # for brevity.

  enable_transport = True


  # By default, the first program to launch the Reticulum
  # Network Stack will create a shared instance, that other
  # programs can communicate with. Only the shared instance
  # opens all the configured interfaces directly, and other
  # local programs communicate with the shared instance over
  # a local socket. This is completely transparent to the
  # user, and should generally be turned on. This directive
  # is optional and can be removed for brevity.

  share_instance = Yes


  # If you want to run multiple *different* shared instances
  # on the same system, you will need to specify different
  # shared instance ports for each. The defaults are given
  # below, and again, these options can be left out if you
  # don't need them.

  shared_instance_port = 37428
  instance_control_port = 37429


  # You can configure Reticulum to panic and forcibly close
  # if an unrecoverable interface error occurs, such as the
  # hardware device for an interface disappearing. This is
  # an optional directive, and can be left out for brevity.
  # This behaviour is disabled by default.

  panic_on_interface_error = No


[logging]
  # Valid log levels are 0 through 7:
  #   0: Log only critical information
  #   1: Log errors and lower log levels
  #   2: Log warnings and lower log levels
  #   3: Log notices and lower log levels
  #   4: Log info and lower (this is the default)
  #   5: Verbose logging
  #   6: Debug logging
  #   7: Extreme logging

  loglevel = 4


# The interfaces section defines the physical and virtual
# interfaces Reticulum will use to communicate on. This
# section will contain examples for a variety of interface
# types. You can modify these or use them as a basis for
# your own config, or simply remove the unused ones.

[interfaces]

  # This interface enables communication with other
  # link-local Reticulum nodes over UDP. It does not
  # need any functional IP infrastructure like routers
  # or DHCP servers, but will require that at least link-
  # local IPv6 is enabled in your operating system, which
  # should be enabled by default in almost any OS. See
  # the Reticulum Manual for more configuration options.

  [[Default Interface]]
    type = AutoInterface
    enabled = Yes
    enable_node = Yes

  [[RNode LoRa Interface]]
    type = RNodeInterface
    enabled = Yes
    enable_node = Yes
    node_name = sdr
    announce_at_start = Yes
    announce_interval = 360
    port = /dev/ttyACM0
    frequency = 433920000
    bandwidth = 125000
    txpower = 7
    spreadingfactor = 8
    codingrate = 5

Hi Mark,

and another issue.
This time it is a show stopper.

I tried to send a message using LXMF to a Group Destination.
LXMF terminates at line 240:

Traceback (most recent call last):
  File "D:/JetBrains/PyCharmProjects/nexus/bsbdock.nexus_context/nexus_server/test2.py", line 62, in <module>
    lxrouter.handle_outbound(lxmessage)
  File "D:\JetBrains\PyCharmProjects\nexus\venv\lib\site-packages\LXMF\LXMF.py", line 1102, in handle_outbound
    lxmessage.pack()
  File "D:\JetBrains\PyCharmProjects\nexus\venv\lib\site-packages\LXMF\LXMF.py", line 240, in pack
    if content_size > single_packet_content_limit:
UnboundLocalError: local variable 'single_packet_content_limit' referenced before assignment

The resaon is obvious and simply because with OPPORTUNISTIC this code does not initialize the single_packet_content_limit at all. Neither within the snippet below nor earlier in the code.

            if self.desired_method == LXMessage.OPPORTUNISTIC:
                if self.__destination.type == RNS.Destination.SINGLE:
                    single_packet_content_limit = LXMessage.ENCRYPTED_PACKET_MAX_CONTENT
                elif self.__destination.type == RNS.Destination.PLAIN:
                    single_packet_content_limit = LXMessage.PLAIN_PACKET_MAX_CONTENT

                if content_size > single_packet_content_limit:
                    raise TypeError("LXMessage desired opportunistic delivery method, but content exceeds single-packet size.")
                else:
                    self.method = LXMessage.OPPORTUNISTIC
                    self.representation = LXMessage.PACKET
                    self.__delivery_destination = self.__destination

Using DIRECT instead does work with that part, but no path is found at all during outbound handling until max delivery attemtps are used up.

I tried to announce the GROUP destination(s) to publish destination hashes so a path could be registered.
However LXMF told me that GROUP targets can not be announced.

So that essentially means, even with the above bug fixed, group targets may not work with LXMessages at all.
At least actually. Right?

Here is my little test program to indicate what I tried.
Maybe there is still something fundamental wrong with my understandig.
Any suggestions are appreciated.

import os
import RNS
import LXMF
import time

APP_NAME = "myapp"
ASPECT = "cockpit"

# LXMF storage
LXMF_STORAGE_PATH = os.path.expanduser("~") + "/.myapp/lxmf"

RNS.Reticulum()

if not os.path.isdir(LXMF_STORAGE_PATH):
    # Create storage path
    os.makedirs(LXMF_STORAGE_PATH)
    # Log that storage directory was created
    RNS.log("Storage path was created")
# Log storage path
RNS.log("Storage path is " + LXMF_STORAGE_PATH)

key = b'\xab/\x91\x02t\xae\xe5\xa1-\xf3\xc9\xf8\xcd\x06\x0b\\\xfa\xcddU\t\xc3O\xa5\x93gH\xe61\xad\xa4 '

destination_to1 = RNS.Destination(
    None, RNS.Destination.IN, RNS.Destination.GROUP, APP_NAME, ASPECT
)
destination_to1.load_private_key(key)

destination_to2 = RNS.Destination(
    None, RNS.Destination.IN, RNS.Destination.GROUP, APP_NAME, ASPECT
)
destination_to2.load_private_key(key)

destination_from = RNS.Destination(
    RNS.Identity(), RNS.Destination.OUT, RNS.Destination.SINGLE, APP_NAME, ASPECT
)

#destination_to1.announce()
#destination_to2.announce()
destination_from.announce()

RNS.log("Destination To1 is " + str(destination_to1))
RNS.log("Destination To1 Hash is " + RNS.prettyhexrep(destination_to1.hash))
RNS.log("Destination To2 is " + str(destination_to2))
RNS.log("Destination To2 Hash is " + RNS.prettyhexrep(destination_to2.hash))
RNS.log("Destination From is " + str(destination_from))
RNS.log("Destination From Hash is " + RNS.prettyhexrep(destination_from.hash))

message_text = "Test Content"
message_title = "Test Title"

lxmessage = LXMF.LXMessage(
    destination=destination_to1,
    source=destination_from,
    content=message_text,
    title=message_title,
    desired_method=LXMF.LXMessage.DIRECT
    # desired_method=LXMF.LXMessage.OPPORTUNISTIC
)

lxrouter = LXMF.LXMRouter(storagepath=LXMF_STORAGE_PATH)
lxrouter.handle_outbound(lxmessage)

while True:
    time.sleep(1)

Best
Stephan

It would be good if the transfer of messages would not be randomly mixed like now but would be "first in first out" or "last in first out" based on the timestamp.

Then additionally as a parameter like the limit which the client sends to the propagation node. The limit makes no sense if the messages are transmitted in a random way.

This would also improve the message processing in the client apps.

Hi Mark,
on my investigation regarding how I can get a notifcation from LXMF that a message has arrived at a destination, I took a deep look into the LXMF.py code.
I pulled the file into my PyCharm IDE and got quite a lot Warnings and code hints of all kind.
Some of them may be quite critical...

First: Line 233

            # If no desired delivery method has been defined,
            # one will be chosen according to these rules:
            if self.desired_method == None:
                self.desired_method == LXMessage.DIRECT
            # TODO: Expand rules to something more intelligent

Guess here is the reason why leaving desired_method unset while calling LXMessage() does not work.
Instead of:
self.desired_method == LXMessage.DIRECT
I think it should state:
self.desired_method = LXMessage.DIRECT

Second: Line 314

    def determine_transport_encryption(self):
        if self.method == LXMessage.OPPORTUNISTIC:
            if self.__destination.type == RNS.Destination.SINGLE:
                self.transport_encrypted = True
                self.transport_encryption = LXMessage.ENCRYPTION_DESCRIPTION_EC
            elif destination_type == RNS.Destination.GROUP:
                self.transport_encrypted = True
                self.transport_encryption = LXMessage.ENCRYPTION_DESCRIPTION_AES
            else:
                self.transport_encrypted = False
                self.transport_encryption = LXMessage.ENCRYPTION_DESCRIPTION_UNENCRYPTED

Here the line:
elif destination_type == RNS.Destination.GROUP:
May probably should state ...
elif self.__destination_type == RNS.Destination.GROUP:
... since it is used later this way.
But in this case the object has no such member declared to judge on that assumption.
In general there are lots of members that are not declared explicitely, making reading the code even more cumbersome.

Third: Line 1269

        except Exception as e:
            RNS.log("Could not enable propagation node. The contained exception was: " + str(e), RNS.LOG_ERROR)
            raise e
            RNS.panic()

Here RNS.panic() is just dead code, that can not be reached,
And to use raise e is discouraged, because simply raise ususally puts the line into the log that contains the error and not the line of the raise statement as with raise e.

So probably the above snipped should be:

        except Exception as e:
            RNS.log("Could not enable propagation node. The contained exception was: " + str(e), RNS.LOG_ERROR)
            raise

Since I have no clue if the RNS.panic() was placed there with purpose, I cant really correct that one properly.

Forth: Line 1637

        for peer in culled_peers:
            RNS.log("Removing peer " + RNS.prettyhexrep(peer) + " due to excessive unreachability", RNS.LOG_WARNING)
            try:
                # TODO: Check this
                if peer_id in self.peers:
                    self.peers.pop(peer_id)
            except Exception as e:
                RNS.log(
                    "Error while removing peer " + RNS.prettyhexrep(peer_id) + ". The contained exception was: " + str(
                        e), RNS.LOG_ERROR)

Since PyCharm told me that peer_id within
if peer_id in self.peers:
migth be accessd prior its initialized, the snippet might be changed into:

        for peer_id in culled_peers:
            RNS.log("Removing peer " + RNS.prettyhexrep(peer_id) + " due to excessive unreachability", RNS.LOG_WARNING)
            try:
                # TODO: Check this
                if peer_id in self.peers:
                    self.peers.pop(peer_id)
            except Exception as e:
                RNS.log(
                    "Error while removing peer " + RNS.prettyhexrep(peer_id) + ". The contained exception was: " + str(
                        e), RNS.LOG_ERROR)

Same as before, I'm not sure if thats what you intendet, especially because there is this ToDo for checking this there.
Thats where I stopped analysing all the hints and warnigs indicated by PyCharm.

And there is still lots more of other stuff, that may need some attention too:

• Unresoved attribute references
• Variable references before assignments
• Deprecated setDaemon() usage
• And dozens of other warnings and PEP 8: E711 code warnings

Is that file the most recent one?
If so why not putting some love into cleaning up that awsome piece of network code in a way, that it does not have that many warnigs and improvement hints any longer?

Cheers
Stephan

Hi Mark,

I tried to send a message using LXMF.
I wrote the following python script using two of my many test peers in out test net:

# Sowassi Sixtus [DA:02]  <0d89bdafd4d3a4703158>
# Harlekin Simplex [DA:01] <c0562a3f00fbc11efcc0>
import LXMF
lxm_router = LXMF.LXMRouter()
message = LXMF.LXMessage("0d89bdafd4d3a4703158", "c0562a3f00fbc11efcc0", "Hello World!")
lxm_router.handle_outbound(message)

Tried to run it with:

python lxmf-test.py

and got the following error message:

Traceback (most recent call last):
  File "/home/pi/LXMF/lxmf-test.py", line 6, in <module>
    lxm_router = LXMF.LXMRouter()
  File "/home/pi/.local/lib/python3.9/site-packages/LXMF/LXMF.py", line 841, in __init__
    raise ValueError("LXMF cannot be initialised without a storage path")
ValueError: LXMF cannot be initialised without a storage path

Nomandnet is running fine on that node.
So reticulum schould be not the problem.
Since I found no docs regarding LXMF so far, I tried to find a solution by peeking into the code.
It seems that LXMF shall create or use some folders here. However I did not found out what I am missing here.

pls adv
Cheers
Stephan

I am trying to install and test out rns and LXMF on a Cortex A9 device. During the installation I get dependency conflict. As mentioned below. Is there any way i can solve this issue.

ERROR: Cannot install lxmf because these package versions have conflicting dependencies.

The conflict is caused by:
rns 0.2.2 depends on cryptography>=3.4.7
rns 0.2.1 depends on cryptography>=3.4.7

PS:
Cryptography version currently installed : cryptography-3.2.1-cp35-cp35m-linux_armv7l.whl

similar to the reticulum examples https://github.com/markqvist/Reticulum/tree/master/Examples

It would be good to see some basic examples for how to

• create a new user
• log a user back in
• create a new conversation (different conversation types like group chat vs public channel?)

When attempting to set a message's source or destination, for example (L is an LXMessage):

  L.set_destination(lxmf_destination)
  L.set_source(self.server_lxmf_delivery)

then attempting to use handle_outbound(L) results in the following:

  [Error] Error occurred while handling request. The contained exception was: 'LXMessage' object has no attribute 'destination'

Setting them improperly, eg:

  L.destination = lxmf_destination
  L.source = self.server_lxmf_delivery

avoids this error (but fails elsewhere with a nonetype not having hash). It seems likely this is related to the destination vs __destination discussion I saw elsewhere.

Is this an issue with a rarely used function or am I missing a step? It won't send or pack and the way the exceptions are wrapped means I'll have to walk through the code to find the specific error line or further troubleshoot.