GithubHelp home page GithubHelp logo

gbossert / pylstar Goto Github PK

View Code? Open in Web Editor NEW
51.0 6.0 14.0 129 KB

An implementation of the LSTAR Grammatical Inference Algorithm

License: GNU General Public License v3.0

Python 100.00%
state-machine protocol grammatical-inference-algorithm automaton

pylstar's Introduction

pylstar : An implementation of the LSTAR Grammatical Inference Algorithm

Continuous integration

Code coverage

About pylstar

pylstar is a free and open source Python implementation of the LSTAR Grammatical inference algorithm. It can be use to automaticaly infer the state machine that best describe the internal of a deterministic black box. To achieve this, pylstar observes the behavior of the target when stimulated with sequence of messages.

It has succesfully been used to infer various protocols such as Botnet protocols, Smart Cards protocols, Cryptographic protocols and Web Servers.

Sample usage

One that wants to use pylstar must write a class that communicates with the targeted black box (i.e. it exposes the Minimaly Adequate Teacher of the targeted reactive System). This can be done by subclassing pylstar.ActiveKnowledgeBase.ActiveKnowledgeBase. If the targeted process is a network server, one can solely subclass pylstar.NetworkActiveKnowledgeBase.

For example, the following class can be use to start and stop a fake coffee machine (coffeemachine.py) through it API (localhost:3000). This class inherits from pylstar.NetworkActiveKnowledgeBase which exposes methods that can send (and read) network messages to (and by) the coffee machine API.

import time
import subprocess

from pylstar.NetworkActiveKnowledgeBase import NetworkActiveKnowledgeBase

class CoffeeMachineKnowledgeBase(NetworkActiveKnowledgeBase):

    def __init__(self):
        super(CoffeeMachineKnowledgeBase, self).__init__("localhost", 3000)
        self.__sp = None

    def start(self):
    """This methods starts the coffee machine (to be triggered before the learning process)."""
        self.__sp = subprocess.Popen("/usr/bin/python coffeemachine.py", shell=True)
        # lets wait 5 seconds for the coffee machine to start
        time.sleep(5)

    def stop(self):
    """This method stops the coffee machine (to be triggered after the learning process)."""
        if self.__sp is not None:
            self.__sp.kill()

Given this wrapper, the following snippet can be used to trigger the automatic inference of the coffee machine. This code declares the messages accepted by the API, an instance of our wrapper and returns a pylstar.automata.Automata.Automata (and prints its DOT code) that best describes the behavior of the coffee machine.

from pylstar.LSTAR import LSTAR
from CoffeeMachineKnowledgeBase import CoffeeMachineKnowledgeBase

# list of messages accepted by the coffee machine
input_vocabulary = [
    "REFILL_WATER",
    "REFILL_COFFEE",
    "PRESS_BUTTON_A",
    "PRESS_BUTTON_B",
    "PRESS_BUTTON_C"    
]
# instanciates our CoffeeMachine MAT
coffeeBase = CoffeeMachineKnowledgeBase()
try:
    # starts the coffee machine
    coffeeBase.start()
    # learns its grammar
    lstar = LSTAR(input_vocabulary, coffeeBase, max_states = 10)
    # stores the coffee machine state machine
    coffee_state_machine = lstar.learn()

    # displays the DOT code of the state machine
    print(coffee_state_machine.build_dot_code())
finally:
   coffeeBase.stop()

The execution of this sample returns the state machine illustrated below:

State Machine of the CoffeeMachine Implementation

A runnable example of the coffee machine inference is available in test/src/test_pylstar/coffee_machine_example.

Installation

Pylstar is a typical python library. It relies on a setup.py file to describe its installation process.The following command can be use to install pylstar on your system:

$ python setup.py install 

Main Features

Playing with Automata

The implementation of automata in pylstar follows the definition of Mealy Machines. An automaton is made of a unique initial state, states and transitions.

States

A state (pylstar.automata.state.State) is defined by its name (str) and some transitions (list<pylstar.automata.transition.Transition>). Per default, a state has no transition.

from pylstar.automata.State import State

q0 = State(name="Example state")
q1 = State("Another state")

N.B: Two states are said equivalent if their name equals.

Transitions

A transition (pylstar.automata.transition.Transition) denotes a directed edge between two states. An edge is attached to a source state and is defined by a triplet:

  • a name (str),
  • an input letter (pylstar.Letter.Letter),
  • an output letter (pylstar.Letter.Letter),
  • a destination state (pylstar.automata.State.State).

The following snippet defines a transition (t0) that can be use to reach "destination state" (q1) from "origin state" (q0) if input letter "a" (la) is received. Executing this transition triggers the emission of letter "0" (l0).

from pylstar.letter import Letter
from pylstar.automata.State import State
from pylstar.automata.Transition import Transition

la = Letter("a")
l0 = Letter("0")
q0 = State("origin state")
q1 = State("destination state")
t0 = Transition("Example Transition", q1, la, l0) 
q0.transitions.append(t0)

Automaton

An automaton (pylstar.automata.Automata.Automata) is defined by its initial state (pylstar.automata.State.State) and an optional name (str). For example, the following snippet illustrates the creation of an automaton:

from pylstar.automata.Automata import Automata
from pylstar.automata.State import State

q0 = State(name="Initial State")
simple_automata = Automata(initial_state = q0, name = "Simple Automata")

An automaton exposes the following methods:

  • build_dot_code() - Returns the DOT code (str) that represents the automaton.
  • get_states() - Returns all the states (list<pylstar.automata.State.State>) that can be reached from the initial state of the automaton.
  • play_word(`pylstar.Word.Word` w, `pylstar.automata.State.State` s = None) - Visits the automaton according to the specified sequence of input messages w starting from state s (if None, it starts from the initial state). It returns a tupple made of the produced messages and the states reached while visiting the automaton ( (pylstar.Word.Word, list<pylstar.automata.State.State>)).

Tests

This project uses DocTests for testing and documentation purposes. To trigger the tests, please use the following command:

$ python setup.py test

References

The LSTAR algorithm was introduced by Dana Angluin in the article

@article{Angluin:1987,
 author = {Angluin, Dana},
 title = {Learning Regular Sets from Queries and Counterexamples},
 journal = {Inf. Comput.},
 issue_date = {November 1, 1987},
 publisher = {Academic Press, Inc.},
} 

This implementation also relies on the description of LSTAR provided by Colin de la Higuera in the book

@book{ColindelaHiguera,
  author = {de la Higuera, Colin},
  title = {Grammatical Inference: Learning Automata and Grammars},
  year = {2010},
  isbn = {0521763169, 9780521763165},
  publisher = {Cambridge University Press},
  address = {New York, NY, USA},
 }

Bugs and enhancements

I'm almost certain this code contains bugs. Please, report any bug found by opening a ticket and/or by submiting a pull requests.Obvisouly, the projet is opened to any minor and major enhancements.

Author

License

This software is licensed under the GPLv3 License. See the COPYING.txt file in the top distribution directory for the full license text.

pylstar's People

Contributors

gbossert avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

pylstar's Issues

CoffeeMachineInferer.py example is not running

Hello,
I'm running the "next" branch,
but I get error:
Exception: Query 'OutputQuery(I = [Letter('REFILL_WATER')], O = None)' could not be queried

tnx
coffee1
coffee2

Also when running the test as you described in the readme I get error
python setup.py test

@gbossert

CoffeeMachineInferer.py example is not running

Hello,
I'm running the "next" branch,
but I get error:

Server is started and listenning
Connected with 127.0.0.1:52198
5025: [ERROR] NetworkActiveKnowledgeBase:_submit_letter: a bytes-like object is required, not 'str'
5026: [ERROR] NetworkActiveKnowledgeBase:_submit_letter: a bytes-like object is required, not 'str'
5027: [ERROR] NetworkActiveKnowledgeBase:_submit_letter: a bytes-like object is required, not 'str'
5028: [ERROR] NetworkActiveKnowledgeBase:_submit_letter: a bytes-like object is required, not 'str'
5028: [ERROR] NetworkActiveKnowledgeBase:_submit_letter: a bytes-like object is required, not 'str'
5029: [ERROR] NetworkActiveKnowledgeBase:_submit_letter: a bytes-like object is required, not 'str'
5029: [ERROR] NetworkActiveKnowledgeBase:_submit_letter: a bytes-like object is required, not 'str'
5029: [ERROR] ObservationTable:__execute_query: Input and output words do not have the same size
Traceback (most recent call last):
  File "/workspaces/pylstar/src/pylstar/KnowledgeBase.py", line 113, in _resolve_word
    return self.knowledge_tree.get_output_word(word)
  File "/workspaces/pylstar/src/pylstar/KnowledgeTree.py", line 181, in get_output_word
    raise Exception("No path found")
Exception: No path found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspaces/pylstar/src/pylstar/ObservationTable.py", line 866, in __execute_query
    self.knowledge_base.resolve_query(query)
  File "/workspaces/pylstar/src/pylstar/KnowledgeBase.py", line 103, in resolve_query
    query.output_word = self._resolve_word(query.input_word)
  File "/workspaces/pylstar/src/pylstar/KnowledgeBase.py", line 123, in _resolve_word
    self.knowledge_tree.add_word(input_word = word, output_word = output)
  File "/workspaces/pylstar/src/pylstar/KnowledgeTree.py", line 220, in add_word
    raise Exception("Input and output words do not have the same size")
Exception: Input and output words do not have the same size
Stoping coffeemachine
Coffee machine process is forced to stop
Traceback (most recent call last):
  File "/workspaces/pylstar/test/src/test_pylstar/coffee_machine_example/CoffeeMachineInferer.py", line 78, in <module>
    main()
  File "/workspaces/pylstar/test/src/test_pylstar/coffee_machine_example/CoffeeMachineInferer.py", line 62, in main
    coffee_state_machine = lstar.learn()
  File "/workspaces/pylstar/src/pylstar/LSTAR.py", line 274, in learn
    self.__initialize()
  File "/workspaces/pylstar/src/pylstar/LSTAR.py", line 368, in __initialize
    self.observation_table.initialize()
  File "/workspaces/pylstar/src/pylstar/ObservationTable.py", line 87, in initialize
    self.__add_word_in_S(Word([EmptyLetter()]))
  File "/workspaces/pylstar/src/pylstar/ObservationTable.py", line 803, in __add_word_in_S
    self.__add_word_in_SA(new_word)
  File "/workspaces/pylstar/src/pylstar/ObservationTable.py", line 843, in __add_word_in_SA
    raise Exception("Query '{}' could not be queries".format(output_query))
Exception: Query 'OutputQuery(I = [Letter('REFILL_WATER'), Letter('REFILL_WATER')], O = None)' could not be queries
Connected with 127.0.0.1:52202
Connected with 127.0.0.1:52210
Connected with 127.0.0.1:52224
Connected with 127.0.0.1:52228
Connected with 127.0.0.1:52236

How to resolve the above error @gbossert ?

NetworkActiveKnowledgeBase not found

Hello, I am learning about protocol reverse engineering, thanks for your Opening source it's great! I have been learning your project for a long time. And there is one problem, when i tried to use pylstar followed the 'readme', it showed me that 'No module named pylstar.NetworkActiveKnowledgeBase', actually , i haven't found it in the source code, did you lose it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.