theaeryan / simple-nlm

A simple Pytorch implementation of Neural Logic Machines (arxiv.org/abs/1904.11694).

Simple NLM

A simple Pytorch implementation of Neural Logic Machines (arxiv.org/abs/1904.11694).

New in version 1.1

• Fixed bug related to the Python typing module. Simple-NLM now also works for Python<3.9.
• Improved efficiency of both padded and unpadded NLMs, by pruning inner predicates which are not used for computing the NLM output.

How to use

Simply download from PyPI by doing pip install neural-logic-machine. An alternative option is to clone the repository.

Assuming it has been installed from PyPI, using it is as simply as importing the neural-logic-machine package in Python:

from neural_logic_machine import NLM
import torch

# We create an NLM with two layers. The first layer receives different predicates
# and outputs 8 predicates for arities 0-3. The second layer receives these predicates and outputs 2 nullary predicates and one unary predicate.
# We do not need to specify the input size of the NLM (this is automatically inferred from the first forward pass).
nlm = NLM(hidden_features=[[8,8,8,8]], out_features=[2,1,0,0])

# Create a random input for the NLM
# The batch will contain two samples, the first one with 3 objects and the second one with 5 objects
# Note that we are not masking out those tensor positions corresponding to invalid objects (e.g., object 4 for the first batch element,
# despite it only having 3 objects). The reason is that there's no need to, as the NLM does that internally when computing the reduce operations
num_objs = [3,5]
input_NLM = [torch.randn((2,) + (5,)*r + (1,)) for r in range(3)] + [None] # The last position is None, as there are no ternary predicates
# Shapes of tensors in input_NLM: [(2,1), (2,5,1), (2,5,5,1), None]

# Forward pass
output = nlm(input_NLM, num_objs)
print(output) # Note in the result how invalid positions (those correspondings to objects 3 and 4 for the first batch element) are masked to -inf

>>> [tensor([[-0.0104,  0.4631],
        [-0.4774,  0.4594]], grad_fn=<AddmmBackward0>), tensor([[[-0.0508],
         [-0.0756],
         [-0.0551],
         [   -inf],
         [   -inf]],

        [[ 0.0538],
         [ 0.1004],
         [ 0.1422],
         [-0.0173],
         [ 0.0232]]], grad_fn=<MaskedFillBackward0>), None, None]

For training, we recommend using Pytorch Lightning.

Dependencies

Python 3 and Pytorch.

Authors

• Carlos Núñez Molina
• Masataro Asai

simple-nlm's Issues

Question about the output of NLM

Hello, and thank you very much for sharing this user-friendly PyTorch implementation of the neural logic machine!

I was experimenting with the code example provided in the README and noticed that the model can output negative values. I would greatly appreciate it if you could provide some insight into why this occurs, as I was under the impression that the output values should represent the likelihood of concluded predicates and therefore range from 0 to 1.