Comments (7)
Also can you verify that self.iterators
is supposed to be self.groups
? I had to make this patch to make a simple test work with generated code:
diff --git a/generator/sbe_message.tmpl b/generator/sbe_message.tmpl
index 12aa5f6..dd16a01 100644
--- a/generator/sbe_message.tmpl
+++ b/generator/sbe_message.tmpl
@@ -10,7 +10,7 @@ ${docstring}
'''
%endif
-from sbedecoder.message import SBEMessage, SBERepeatingGroupIterator
+from sbedecoder.message import SBEMessage
from sbedecoder.message import TypeMessageField, EnumMessageField, CompositeMessageField, SetMessageField
## Build a class definition for each message that is defined
@@ -141,12 +141,12 @@ class ${message_def.name}(${message_def.base_classes}):
])
%endfor
- self.iterators = [
+ self.groups = [
%for group in message_def.iterators:
self.${group.name},
%endfor
]
%else:
- self.iterators = []
+ self.groups = []
%endif
</%def>
from sbedecoder.
@tamboril Sorry about this. I haven't been using the generator and honestly just didn't verify that it still worked after making the group changes. I actually find the dynamically generated classes pretty easy to work with. Sort of ironic as I was the one who requested the class generator. Anyway, I'll take a look at it right now.
from sbedecoder.
Thanks, Bud. I almost have it fixed up, but I'll wait for your patch.
The class generator is a great idea, especially with the template idea. It's important to me, because I don't want to require other developers to have to have all the non-standard python libraries in order to build my project, and the CME templates rarely change.
from sbedecoder.
@tamboril I just merged in a fix for the generator. It doesn't handle the case where there are nested groups yet. That's going to take a bit more work. The current MDP 3.0 templates don't have any nested groups so if that's what you are decoding, you should be fine. I'll leave this open to remind me to fix that.
from sbedecoder.
Bud, Thanks. There is some other problem with the latest. The error (code is attached):
pydecoder.zip
Traceback (most recent call last):
File "/home/jharris/tmp/pydecoder/pydecoder.py", line 43, in <module>
for msg in parser.parse(buf2, offset):
File "/home/jharris/repos/sbedecoder/sbedecoder/parser.py", line 8, in parse
message = self.factory.build(message_buffer, msg_offset)
File "/home/jharris/repos/sbedecoder/sbedecoder/message.py", line 309, in build
message.wrap(msg_buffer, offset)
File "/home/jharris/repos/sbedecoder/sbedecoder/message.py", line 294, in wrap
group_offset += group_iterator.wrap(msg_buffer, msg_offset, group_offset)
File "/home/jharris/repos/sbedecoder/sbedecoder/message.py", line 237, in wrap
block_length = self.block_length_field.value
File "/home/jharris/repos/sbedecoder/sbedecoder/message.py", line 52, in value
_raw_value = self.raw_value
File "/home/jharris/repos/sbedecoder/sbedecoder/message.py", line 72, in raw_value
self.msg_offset + self.relative_offset + self.field_offset)[0]
struct.error: unpack_from requires a buffer of at least 2 bytes
from sbedecoder.
@tamboril yeah, so not sure where you got that message from but remember how you said that the CME doesn't change the templates that often? Well it turns out they do change them. The current version is 8 and the message you are trying to decode is a version 5 message. Unfortunately we don't support versioning (issue #6). It is definitely something I need to figure out and get working...
Not sure why you couldn't get the tests to run. I was able to clone a new instance of the repo and run them with no issues:
bud@ws-bud ~/tmp $ [master|6] git clone https://github.com/tfgm/sbedecoder.git
Cloning into 'sbedecoder'...
remote: Counting objects: 138, done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 138 (delta 0), reused 4 (delta 0), pack-reused 128
Receiving objects: 100% (138/138), 49.74 KiB | 0 bytes/s, done.
Resolving deltas: 100% (57/57), done.
Checking connectivity... done.
bud@ws-bud ~/tmp $ [master|6] cd sbedecoder/
bud@ws-bud ~/tmp/sbedecoder $ [master] virtualenv venv
New python executable in /home/bud/tmp/sbedecoder/venv/bin/python
Installing setuptools, pip, wheel...done.
bud@ws-bud ~/tmp/sbedecoder $ [master] source venv/bin/activate
(venv) bud@ws-bud ~/tmp/sbedecoder $ [master] python setup.py install
[snip]
Finished processing dependencies for sbedecoder==0.1
(venv) bud@ws-bud ~/tmp/sbedecoder $ [master] nosetests -v tests/test_sbe_parser.py
test_sbe_parser.TestSBEParserLibrary.test_incremental_refresh_multiple_messages ... ok
test_sbe_parser.TestSBEParserLibrary.test_incremental_refresh_trade_summary ... ok
test_sbe_parser.TestSBEParserLibrary.test_incremental_refresh_verify_group_attributes ... ok
test_sbe_parser.TestSBEParserLibrary.test_incremental_refresh_verify_groups ... ok
test_sbe_parser.TestSBEParserLibrary.test_security_status ... ok
test_sbe_parser.TestSBEParserLibrary.test_security_status_reset_statistics ... ok
----------------------------------------------------------------------
Ran 6 tests in 0.966s
OK
(venv) bud@ws-bud ~/tmp/sbedecoder $ [master] cd generator/
(venv) bud@ws-bud ~/tmp/sbedecoder/generator $ [master] ./sbe_class_generator.py --schema /mnt/cme-data/2017/08/18/Templates/MDP3/templates_FixBinary.xml
(venv) bud@ws-bud ~/tmp/sbedecoder/generator $ [master] cp generated.py ../sbedecoder
(venv) bud@ws-bud ~/tmp/sbedecoder/generator $ [master] cd ..
(venv) bud@ws-bud ~/tmp/sbedecoder $ [master] nosetests -v tests/test_sbe_parser.py
test_sbe_parser.TestSBEParserLibrary.test_incremental_refresh_multiple_messages ... ok
test_sbe_parser.TestSBEParserLibrary.test_incremental_refresh_trade_summary ... ok
test_sbe_parser.TestSBEParserLibrary.test_incremental_refresh_verify_group_attributes ... ok
test_sbe_parser.TestSBEParserLibrary.test_incremental_refresh_verify_groups ... ok
test_sbe_parser.TestSBEParserLibrary.test_security_status ... ok
test_sbe_parser.TestSBEParserLibrary.test_security_status_reset_statistics ... ok
----------------------------------------------------------------------
Ran 6 tests in 0.572s
OK
from sbedecoder.
OK. You're right. That was an old message that my current C++ printers, which I'm trying to replace, handled by virtue of the RealLogic decoders handling the older version.
I put a big new message in there, and it doesn't crash.
I didn't have a problem with the unit tests.
Thanks for the quick patch.
from sbedecoder.
Related Issues (15)
- sample data HOT 1
- Question about CME Market Depth data HOT 3
- is_string_type fails with null strings HOT 3
- orderbook isn't included in egg anymore HOT 1
- Generator Does Not Support SubGroups HOT 2
- Make sbedecoder publicly available on pypi.org HOT 4
- Add support for Python 3 HOT 1
- Support for Market Depth HOT 3
- Fixed header size HOT 8
- Support for ref in composite elements
- Decoder doesn't support message versioning HOT 1
- Verify compatibility with the FIX Trading Community specification HOT 6
- errors: invalid tcpdump header HOT 1
- compile errors with version 3.5.1 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sbedecoder.