Sax Parsing Segfault by Ruby GC about ox

[ohler55]

If you can help me that would be great. Can you try running with the class_cache turned off? What other options are you using?

[griffinmyers]

Sure, would be happy to try that out. Unfortunately I can't find any documentation on how to turn class_cache off. Additionally, I'm not running with any options. Just passing in an IO object and a vanilla Sax Doc.

[ohler55]

Wrong gem. Ox does not have the class_cache option, only Oj. Vanilla Sax is enough for me to get started though.

[ohler55]

I checked in a fix. Can you give it a whirl on your system. Of by one on buffer size.

[griffinmyers]

Trying it out now. I'll give it a day or so and report back! Thanks for the prompt response, it is seriously awesome to see a gem maintained so well.

[ohler55]

Thanks.

[griffinmyers]

I've gotten two crashes with the same .crash report. They seem to be happening less frequently, but happen nonetheless. I used revision: bb30dbe8db25b9bf34d393a0296f4e10f3c2cb87.

[ohler55]

Can you give me a file to test with?

[griffinmyers]

The Sax document file I presume?

[ohler55]

The XML file that triggers the crash.

[griffinmyers]

There is no XML file that triggers the crash. The crash occurs during ruby garbage collection, often times well after XML parsing has completed.

[ohler55]

Can you provide a copy of the XML that is parsed before the crash? I'd like to try and recreate the problem.

[griffinmyers]

This is within a Rails app; there may be 100 XML documents parsed before it faults. That said I'll see if I can get it to occur by hitting only one document so you can test against it.

[ohler55]

Thanks.

[griffinmyers]

I've been trying to isolate this with a single XML document but not finding much luck. You know, I'm wondering: If I raise an exception from within a SAX call back like #start_element or #end_element, and that's not caught until we are well into the Rails stack, would that cause the memory allocation issue?

[griffinmyers]

In fact, I just ran a little test, and indeed that seems to be the cause of this. When I don't raise from within the callbacks, I don't get any segfaults. I'm not having this problem with nokogiri, is this something that can be fixed?

[ohler55]

Thank you for digging into the problem. That gives me something to work with. I'll see what I can do over the weekend if not sooner.

[ohler55]

I'm not having much success reproducing the problem. Do you have a test you can share?

[ohler55]

Were you able to find a file to share for testing?

[griffinmyers]

Hey, sorry sorry, I've gotten tied up with some other work at the moment, should hopefully have a solid test case for you soon. Apologies for the delay

[ohler55]

No worries. Just checking up on outstanding issues.

[griffinmyers]

Sorry for the delay, I got a test case for you finally. You can clone it here.

[ohler55]

Thats great. Thanks. I'll give it a try tonight.

[ohler55]

What platform are you running on? OS X? Linux?

[ohler55]

I added a print statement to the Ox initializer function and it was never called. Are you sure the test you have is a valid test?

[griffinmyers]

Yep, OSX 10.6. I just tried on a different machine running 10.7 and got the segfault. Are you visiting http://localhost:3000/parse/ once the rails server is running? I see it raising from within end_element

[ohler55]

I missed the /parse. I get an error in soap.rb in end_element. Is that expected?

[ohler55]

Eventually a crash. Cool, I can reproduce it here.

[ohler55]

Got it fixed. I want to add some stack use optimizations before releasing though. A couple of days at most.

[ohler55]

You made it very easy to find. Thank you for the help.

[griffinmyers]

Cool, glad I could help. I see the commits, but could you share a bit about what was going on that was causing it not to be exception safe?

[ohler55]

I had not wrapped the parse call with a rb_protect(). With that in place I can get the exception back, cleanup, and then raise the exception. An oversight on my part.

[ohler55]

Good to close?

[griffinmyers]

I think so. A couple days now with no hint of a problem. Thanks again, awesome to see this well maintained.

[ohler55]

I do my best. Thanks for the feedback and the help identifying the problem.