robotframework / rammbock

Rammbock - generic network protocol tester

Robot Framework

Introduction

Robot Framework ® is a generic open source automation framework for acceptance testing, acceptance test driven development (ATDD), and robotic process automation (RPA). It has simple plain text syntax and it can be extended easily with generic and custom libraries.

Robot Framework is operating system and application independent. It is implemented using Python which is also the primary language to extend it. The framework has a rich ecosystem around it consisting of various generic libraries and tools that are developed as separate projects. For more information about Robot Framework and the ecosystem, see http://robotframework.org.

Robot Framework project is hosted on GitHub where you can find source code, an issue tracker, and some further documentation. Downloads are hosted on PyPI.

Robot Framework development is sponsored by non-profit Robot Framework Foundation. If you are using the framework and benefiting from it, consider joining the foundation to help maintaining the framework and developing it further.

Installation

If you already have Python with pip installed, you can simply run:

pip install robotframework

For more detailed installation instructions, including installing Python, see INSTALL.rst.

Robot Framework requires Python 3.8 or newer and runs also on PyPy. The latest version that supports Python 3.6 and 3.7 is Robot Framework 6.1.1. If you need to use Python 2, Jython or IronPython, you can use Robot Framework 4.1.3.

Example

Below is a simple example test case for testing login to some system. You can find more examples with links to related demo projects from http://robotframework.org.

*** Settings ***
Documentation     A test suite with a single test for valid login.
...
...               This test has a workflow that is created using keywords in
...               the imported resource file.
Resource          login.resource

*** Test Cases ***
Valid Login
    Open Browser To Login Page
    Input Username    demo
    Input Password    mode
    Submit Credentials
    Welcome Page Should Be Open
    [Teardown]    Close Browser

Usage

Tests (or tasks) are executed from the command line using the robot command or by executing the robot module directly like python -m robot .

The basic usage is giving a path to a test (or task) file or directory as an argument with possible command line options before the path:

robot tests.robot
robot --variable BROWSER:Firefox --outputdir results path/to/tests/

Additionally, there is the rebot tool for combining results and otherwise post-processing outputs:

rebot --name Example output1.xml output2.xml

Run robot --help and rebot --help for more information about the command line usage. For a complete reference manual see Robot Framework User Guide.

Documentation

• Robot Framework User Guide
• Standard libraries
• API documentation
• General documentation

Support and Contact

• Slack
• Forum
• robotframework-users mailing list

Contributing

Interested to contribute to Robot Framework? Great! In that case it is a good start by looking at the CONTRIBUTING.rst. If you do not already have an issue you would like to work on, you can check issues with good new issue and help wanted labels.

Remember also that there are many other tools and libraries in the wider Robot Framework ecosystem that you can contribute to!

License and Trademark

Robot Framework is open source software provided under the Apache License 2.0. Robot Framework documentation and other similar content use the Creative Commons Attribution 3.0 Unported license. Most libraries and tools in the ecosystem are also open source, but they may use different licenses.

Robot Framework trademark is owned by Robot Framework Foundation.

rammbock's Issues

message_stream.get function will drop into endless loop

We found robot can not stop sometimes. After research found the core reason is that message_stream.get will drop into endless loop when no expected message found and socket never empty (receive data persistently).

Add option to receive latest from buffer instead of first

Now receiving will get the first matching message from buffer. There should be an option to get the latest.

Rammbock messages seem list like to Robot

Robot thinks that Rammbock message objects are lists, because iter works on them. This can cause for example Log variables to fail.

Messages should probably raise TypeError from __iter__.

new_protocol has no way to set endianess

In the function new_protocol

    def new_protocol(self, protocol_name):
        """Start defining a new protocol template.

        All messages sent and received from a connection that uses a protocol
        have to conform to this protocol template.
        """
        if self._protocol_in_progress:
            raise Exception('Can not start a new protocol definition in middle of old.')
        if protocol_name in self._protocols:
            raise Exception('Protocol %s already defined' % protocol_name)
        self._init_new_message_stack(Protocol(protocol_name, library=self))
        self._protocol_in_progress = True

The line self._init_new_message_stack(Protocol(protocol_name, library=self)) gives no option to set the Protocol constructor option for little_endian. Neither does the new_protocol method.

So, there does not seem to be a way to use the robot api, and have this little_endian feature. It would be very useful if the endian setting could be set from the robot api.

At least, that's how it looks to me atm.

Thanks for reviewing this issue.

Cheers,
Matt

way to extend rammbock with primitives and data types

probably with different encodings/decodings

XMPP

Hi everybody!

Does it make sense to use Rammbock for creating protocol and message templates for XMPP? An alternative would be to use a python library like http://xmpppy.sourceforge.net/ and build a Robot Framework library upon this.

Which of these two ways would you recommend?

Regards

Support of KW polimorfism based on number of arguments

It would be great supports KW polimorfism based on number os arguments.

Protocol definition with conditional segments

New Feature request - adding conditional segments to protocol template.
With such functionality there will be possibility to test protocols with dynamic structure (multiple segment types with variable length and fields).

Implementation may be as follows:

New Protocol    ConditionalProtocol
    u8      control
    u16     session         0x0102
    u8      extensions      0x00
    Conditional Segment   control == (0|1|2|3)  data_segment
        u8  data  0x00
        u8  lenght  
        pdu  lenght
    End Conditional Segment
    Conditional Segment  control == (4|5|6|7)  report_segment
        u8  report
    End Conditional Segment  
End Protocol

(...)
New Message  Data  ConditionalProtocol  header:control:0x03

(...)
Server Receives Message  header:control:0x05

Example protocol (http://tools.ietf.org/html/rfc5326) with varied segments decoded by Wireshark. Please notice that length of pdu is located outside of static header.

1. Data

Licklider Transmission Protocol
    LTP Header
        LTP Version: 0
        LTP Type: 3 (Red data, Checkpoint, EORP, EOB)
        Session ID
            Session originator: 1
            Session number: 3
        Header Extension Count: 0
        Trailer Extension Count: 0
    Data Segment
        Client service ID: 1
        Offset: 0
        Length: 60
        Checkpoint serial number: 202
        Report serial number: 0
        Data[1]

1. Report

Licklider Transmission Protocol
    LTP Header
        LTP Version: 0
        LTP Type: 8 (Report segment)
        Session ID
            Session originator: 1
            Session number: 3
        Header Extension Count: 0
        Trailer Extension Count: 0
    Report Segment
        Report serial number: 10656
        Checkpoint serial number: 202
        Upper bound: 60
        Lower bound: 0
        Reception claim count: 1
        Reception claims
            Offset[0] : 0
            Length[0] : 60

better sequence diagram functionality

Right now we only support seqdiag format with 15 last messages. In future we would like to somehow present also sequences of hundreds of messages.

Support to use structural field for dynamic length

Currently dynamic length is supported when field is not inside a structure. This is to enhance the support for dynamic length so that length field can be given inside struct

Saving second message sequence image in same test overwrites the previous

The Embed sequence diagram keyword simply uses the <test name>.png as name for the saved message sequence diagram. This will cause subsequent diagrams in the same test to overwrite the previous. There should be somekind of automatic indexing for the images and also possibly a possibility to give the sequence filename.

Conditional keyword missing - incosistency in atests

Executing acceptance test

pybot atest/structured_messages/conditional.txt 

RF is returning errors becouse of missing Conditional keyword.
No keyword with name 'Conditional' found

KEYWORD: Conditional message Expand All
Start / End / Elapsed: 20140227 15:56:59.212 / 20140227 15:56:59.214 / 00:00:00.002
KEYWORD: Rammbock.New Message ConditionalExample, Example, header:messageType:0xb0b0 Expand All
KEYWORD: Rammbock.U 8 condition, 0 Expand All
KEYWORD: Conditional condition == 1, conditional Expand All
Start / End / Elapsed: 20140227 15:56:59.214 / 20140227 15:56:59.214 / 00:00:00.000
15:56:59.214 FAIL No keyword with name 'Conditional' found.

when running full atest regression

./execute_regression_tests.sh

Result is shown as PASSed

Simple conditional message | PASS |

Tools versions:
Robot Framework 2.8.4 (Python 2.7.5+ on linux2)
robotframework-rammbock Version: 0.3.0

Long timeout in receiving can prevent the background handling from other message streams

Receiving with timeout goes to a synchronized block. If the timeout is longer than the poll interval for background handlers, this can cause the other message streams to not get the lock during this time and they can not handle their background messages.

The receiving of protocol headers in MessageStream should be done so that other message streams either can operate at the same time, or can get the lock periodically.

Filling array elements with [*] in send

Filling dynamic array elements in structs could be more handy while sending. It would be useful to be able to fill all elements easily using the syntax below. This test is also in atest/structured_messages/dynamic_messages

Filling array of free length dynamic message in send
[Tags]
Dynamic message
Client sends message numberOfPairs:4
... pairList[].first:0xaa
... pairlist[].second:0xbb
${msg} = Server receives message
Should be equal as integers ${msg.pairlist[3].second.int} 4

IPv6 support

We should support ipv6

Create actual Release 0.4.0 ?

as we now have version 0.4.0 in the change log commit, create an actual Release 0.4.0 ?

label sequence field

similar to string, but dots are converted to byte with value of length of the following word.
example:

label sequence .foo.bar.zig
would be encoded to \x03foo\x03bar\x03zig
and vice versa

Convert Rammbock runner scripts from shell to python

atest and utest runner scripts should be converted from shell scripts to python scripts. It shall be done so that changing python version in run is trivial.

Add support for bitfields

A lot of protocols use only a few bits as functional fields e.g. CAN, I2C,... or use lengths that don't match u8 u16,... .
Allowing bit fields makes testing network protocols a lot more general.
The knowledge of these fields should only be located at the conversion (reading/writing of the data).
Internally the fields can be expanded to the nearest known type if necessary.

Support for SSL

There should be support for SSL including client side certificates and various SSL/TLS versions.

I would like to create server something like this:

Start SSL Server 127.0.0.1 443 name=MyServer
... protocol=Example keystore=ServerKeys.pem
... sslversion=TLS1.2 ciphers=something

I would like to use client something like this:

Start SSL Client name=MyClient protocol=Example ca_keys=ServerKeys.pem
Connect 127.0.0.1 443 client_certificate=ClientKeys.pem

there is a new branch where work has been started.

Support copying structured elements from received messages to other messages

Example test
Populate dynamic message 2 0x01 0x02 0x11 0x22
${list msg} = Get message
Value pair ${list msg.pairList[0]}
Message with struct
${pair msg} = Get message
Should be equal ${pair msg.pair.first.hex} 0x01

Populate dynamic message
[Arguments] ${len} ${0 first} ${0 second} ${1 first} ${1 second}
Dynamic Message
value numberOfPairs ${len}
value pairList[0].first ${0 first}
value pairList[0].second ${0 second}
value pairList[1].first ${1 first}
value pairList[1].second ${1 second}

test case example for diameter protocol

Is it possible to embed one protocol into another?

So we have a TCP-based protocol (lets call it protocol A for now), that basically adds some additional fields to address specific components within our product.
These components may speak different protocols, so the payload of that first protocol can be a number of different protocols (protocol B, C, D, ...).

Is it possible to tell Rammbock that protocol B is prefixed with the header of protocol A?

pretty printing of fields

Support regular expression matching for values of type Chars

Currently, we are unable to match the values of type Chars by using a regular expression.
Also, it would be nice to filter the incoming messages using header_filter=REGEXP:.

Got unexpected token at line 2 column 28

0.3.0 version fails to generate the Sequence diagram. the reason is IP:Port is dumped in .seqdiag file, which the seqdiag tool fails to interpret!.

Ex:
diagram {
DATA -> 192.168.1.1:91181 [label = "Message:XXX"];
}

Support bag case size 0

It would be useful sometimes to explicitly define a case of size 0. Meaning that we know that a certain element might appear, but we want to fail at validation when that happens.

Possibility to close clients and servers

Currently it is not possible to close individual clients, or servers. There should be something like Close Server and Close Client keywords. As there is also no keyword for changing the current client or server, those should be created as well.

Supporting a template without any message and header fields

Hi,

I want to define a protocol which uses multicast udp connection and its packet is a simple string without any packet header field or message field. Also, the encode message method checks for ':' and '=' characters whereas I do not need such checks.

I just need rammbock to send string to a UDP multicast group. Can anyone please help me understand how to enable this feature?

Thanks in advance.

pdu_length has invalid data

Removed.

sctp support for clients and servers

atest: Background server related testes are executed using shell which is preventing graceful kill of process.

atest: Background server related testes are executed using shell which is preventing graceful kill of process.

conditional field or struct

The syntax could be for example:
u8 previous_field 1
new struct type name condition=previous_field==1
u8 struct_field 2
end struct

this functionality is needed for some gtpv2 information elements and probably for other protocols as well.

Error when using _Freelength pdu

When I use the pdu length as a free length while defining the protocol.

...
...
self.pdu('*')
self.end_protocol()

I get a python error saying

"""
pydev debugger: starting
Traceback (most recent call last):
File "...\eclipse\plugins\org.python.pydev_2.7.3.2013031601\pysrc\pydevd.py", line 1397, in
debugger.run(setup['file'], None, None)
File "...\eclipse\plugins\org.python.pydev_2.7.3.2013031601\pysrc\pydevd.py", line 1090, in run
pydev_imports.execfile(file, globals, locals) #execute the script
File "...py", line 55, in
msg = myObject.server_receives_without_validation()
File "C:\Python27\lib\site-packages\Rammbock\core.py", line 478, in server_receives_without_validation
with self._receive(self._servers, *parameters) as (msg, _):
File "C:\Python27\lib\contextlib.py", line 17, in enter
return self.gen.next()
File "C:\Python27\lib\site-packages\Rammbock\core.py", line 503, in _receive
msg = node.get_message(self._get_message_template(), **configs)
File "C:\Python27\lib\site-packages\Rammbock\networking.py", line 73, in get_message
return self._get_from_stream(message_template, self._message_stream, timeout=timeout, header_filter=header_filter)
File "C:\Python27\lib\site-packages\Rammbock\networking.py", line 76, in _get_from_stream
return stream.get(message_template, timeout=timeout, header_filter=header_filter)
File "C:\Python27\lib\site-packages\Rammbock\templates\message_stream.py", line 34, in get
header, pdu_bytes = self._protocol.read(self._stream, timeout=timeout)
File "C:\Python27\lib\site-packages\Rammbock\templates\containers.py", line 165, in read
length_param = header[self.pdu_length.field].int
AttributeError: '_FreeLength' object has no attribute 'field'

"""

Document limitations

Document that HTTP/HTTPS might be easier to handle with other libraries. Also that ASN.1 PER encoding is a bit too difficult with the current syntax.

Nicer error message for empty named cache

If named cache is empty and Rammbock tries to get the default entry, we get a cryptic KeyError. For example if a TCP server has not accepted any connections, but tries to receive data.

Check that dynamic array length does not exceed overall message length

Rammbock blindly believes the length of a dynamic array. It does not even check if it fits to overall message length. This lead into a hard-to debug problem where the memory of Python virtual machine was exhausted when the production code left the dynamic message length field uninitialized.

There should be at least a check for dynamic array size being smaller than overall message length. Probably also a global limit for message length which one could optionally set when importing Rammbock.

Support string values for header_filter

Currently we are unable to filter the incoming messages by using a header field in the protocol declared as "chars".
i.e. header_filter=<string_header_field>

templating support for asn.1 protocols

new feature: support network node that need to deal with heart beat or keep alive on the same socket

Currently the network node are bound with a certain protocol, this prohibits the usage of
rammbock from end/terminal that equipped with alive detection mechanism.
Since alive detection packet normally are not compatible with interaction protocol on the same socket,
we need to do some ugly hacking in _receive_msg_ip_port function in _NetworkNode class in order to involving rammbock in that kinds of scenario.
I come from Nokia, and our team hopes that our test cases can keep synchronized with robot and rammbock update.

Static pdu length does not work with TCP nodes

If the pdu length is fixed, receiving messages fails on TCP.

`Client/server receives message` keyword is considering latest parameter as true for 'False' as input #61

With "latest" parameter of client/server receives message keyword we can select the latest or old messages from message stack. But it is not working properly.

Expected behavior:
latest=true: Client/server should pick latest message from the stack and validate.
latest=False: Client/server should pick old message from the stack and validate.

Actual behavior:
latest=true: Client/server is retrieving the latest message from stack and validating.
latest=False: Client/server is retrieving the latest message from stack and validating.

I can add Generic Diameter Server which I use

Hi Jussi,

How are you doing?
The generic diameter server which I demoed while you were here is now complete and has been tested & being used by a good number of testers. If you feel it could be useful for more people I can add it here, after you have a look at it.

Regards,
Waman

Server Receives Message with Header Value gives false positive

I was running the acceptance test and came across this when I ran the following Test case after modifiying the value for header:reserved:0xff to header:reserved:0xaa

• Atest
  • Default Values
    • Default values for header fields

Here's the Test Case with the modified Keyword

"""
Client sends    Header value ExampleMessage
${msg}= Server receives    Header value 0xAA ExampleMessage
Should be equal ${msg._header.reserved.hex} 0xff
"""

Here's the Modified Keyword
Header value 0xAA ExampleMessage

"""
Value ExampleMessage**      
Value    header:reserved    0xaa
"""   

With the Keyword Modified above, the test case should have failed.
It seems like specifying the header value for reserved is making no effect.

My Apologies if I am not understanding this correctly.

Please correct me if I am wrong.

Thanks,
Harsh

Multicast support in Rammbock using IGMP

Is there any way I can send IGMP messages so that I can Join a multicast group and send and receive UDP messages from that group?

I know the IGMP protocol is not currently support by rammbock directly, but some insight on how it can be done will be really appreciated.

If required, I can provide more details.

Thanks.

Accept connection should support timeouts

If Accept connection is called when no connection is coming, the keyword hangs. There should be support for providing an optional timeout for accepting the connection.

How can I use bin container's field as pdu length during define protocol

I want to use like this:

New protocol    SL
u16    password    0x0102
u8    functionCode
new binary container    payloadLen
bin    4    direction
bin    12    length
end binary container
u8    payloadBeginTag    0x02
pdu    payloadLen.length + 3
End protocol

but error 'Length field payloadLen.length unknown', How can I use bin container's field as pdu length during define protocol?

AttributeError occurs when receive message with fix pdu length

I just modify atest key word " Define example protocol", and run case "Client sends server receives", got error "AttributeError: '_StaticLength' object has no attribute 'field'"

Define example protocol
[Arguments] ${name}=Example
New protocol ${name}
u8 version 0x01
u8 reserved 0x00
u16 messageType # Empty on purpose, each message template defines the type for it
Comment u16 length # Empty on purpose, the length is pdu length
u16 flags 0x0000
pdu 4
End protocol

Cannot use BinaryContainer field as header_filter in server_receives_message

I have a message that doesn't have message fields to it.
The message is a type of (ACK)acknowledgement to another message and it contains only header part of the protocol.
The protocol is our company's custom protocol and contains binary fields in the header
and the Binary Containers name is 'Flags'

How can I verify that the received message is my ACK message?

While doing the

'''
${msg} =     Server Receives Message      header_filter = Flags  
'''

I get the following error

File "C:\Python27\lib\contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "C:\Python27\lib\site-packages\Rammbock\core.py", line 503, in _receive
    msg = node.get_message(self._get_message_template(), **configs)
  File "C:\Python27\lib\site-packages\Rammbock\networking.py", line 73, in     get_message
    return self._get_from_stream(message_template, self._message_stream,     timeout=timeout, header_filter=header_filter)
  File "C:\Python27\lib\site-packages\Rammbock\networking.py", line 76, in     _get_from_stream
    return stream.get(message_template, timeout=timeout, header_filter=header_filter)
  File "C:\Python27\lib\site-packages\Rammbock\templates\message_stream.py", line     35, in get
    if self._matches(header, header_fields, header_filter):
  File "C:\Python27\lib\site-packages\Rammbock\templates\message_stream.py", line 60, in _matches
    if header[header_filter].bytes != to_bin(fields[header_filter]):
  File "C:\Python27\lib\site-packages\Rammbock\message.py", line 38, in __getattr__
    return self[name]
  File "C:\Python27\lib\site-packages\Rammbock\message.py", line 35, in __getitem__
return self._fields[str(name)]
KeyError: 'bytes'

Can the Individual fields of the Binary container be used as header filters?

Support For Logical Conditions in Conditional Element

Currently the Conditional Elements support one condition, we would like to enhance it to support multiple conditions combined using logical operators '||', '&&'.