kwarc / rust-libxml

Rust wrapper for libxml2

Rust wrapper for libxml2.

The main goal of this project is to benefit from libxml2's maturity and stability while the native Rust XML crates mature to be near-drop-in replacements.

As of the 0.2.0 release of the crate, there are some modest safety guarantees:

• Mutability, as well as ownership - we use Rc<RefCell<T>> wrappers to ensure runtime safety of libxml2 operations already in the Rust layer.
• Memory safety guarantees - in particular Node and Document objects have automatic bookkeeping and deallocation on drop, for leak-free wrapper use.
• No thread safety - libxml2's global memory management is a challenge to adapt in a thread-safe way with minimal intervention

Coverage: Only covers a subset of libxml2 at the moment, contributions are welcome. We try to increase support with each release.

Welcome! With these caveats, the contributors to the project are migrating production work towards Rust and find a continuing reliance on libxml2 a helpful relief for initial ports. As such, contributions to this crate are welcome, if your workflow is not yet fully supported.

Installation prerequisites

Before performing the usual cargo build/install steps, you need to have the relevant components for using the original libxml2 code. These may become gradually outdated with time - please do let us know by opening a new issue/PR whenever that's the case.

Linux/Debian

On linux systems you'd need the development headers of libxml2 (e.g. libxml2-dev in Debian), as well as pkg-config.

MacOS

Community contributed:

$ brew install libxml2 # e.g. version 2.9.12 
$ ln -s /usr/local/Cellar/libxml2/2.9.12/lib/libxml2.2.dylib /usr/local/lib/libxml-2.0.dylib
$ export LIBXML2=/usr/local/Cellar/libxml2/2.9.12/lib/pkgconfig/libxml-2.0.pc

FreeBSD

Community contributed

$ pkg install libxml2 pkgconf

Windows

Community contributed:

• manually install builds tools c++ and english language by visiting BuildTools
• launch cmd prompt with admin privileges and execute these commands sequentially:

C:\> git clone https://github.com/microsoft/vcpkg
C:\> .\vcpkg\bootstrap-vcpkg.bat
C:\> setx /M PATH "%PATH%;c:\vcpkg" && setx VCPKGRS_DYNAMIC "1" /M
C:\> refreshenv
C:\> vcpkg install libxml2:x64-windows
C:\> vcpkg integrate install

rust-libxml's Issues

Schema validation errors

Before I dig too deeply in to this issue, am I right in thinking that schema validation errors are not currently captured?

When I input schema error in to ./tests/schema_tests.rs the error is printed to the terminal by (I assume) the libxml2 library and is not part of the messages returned in the Err from SchemaValidationContext::from_parser(...).

XML with BOM not parsed

I found trouble with some files, inside containing BOM.
It's just not parsed as a XML with parse_file method.

missing ability to get all attributes including namespace

Consider this:

<foo xmlns:ns="http://example.com" bar="BAR" ns:bar="NS BAR" />

How can I find out that attributes bar and ns:bar both exist?

The Node API offers get_attributes which returns a HashMap with all attributes, but this loses namespace information and I only get bar once.

I can access individual attributes by namespace using get_attribute_ns, but I have no way to find out which attributes exist in the first place. I would expect something like get_attributes_ns which returns a vec of Node (of type AttributeNode, like what is returned by get_attribute_node) in order to find this out. Or alternatively a get_attribute_names_ns which gives a vec of attribute name, attribute namespace tuples.

Am I missing something or this is something the API doesn't implement yet?

Some struct bindings are missing XML Schema

I am using your crate for parsing XML schema (XDS), and ran into the need to traverse the raw parsed schema tree. There are a few cases where libxml2 does explicit casts between pointer types (says it's a xmlSchemaType * when it's actually a xmlSchemaParticle *).

This is fine; the generated bindings properly reflect the type of the struct itself and can't do anything about their casting it. However, I think this has caused some structs to not be included in the bindings. The ones that I encountered and had to generate bindings for manually were:

• _xmlSchemaTreeItem
• xmlSchemaTreeItemPtr
• _xmlSchemaParticle

_xmlSchemaTreeItem is a bit of a weird case since it's "The abstract base type for tree-like structured schema components". You may be against including that in the bindings which I'd understand, but I do think adding at least adding _xmlSchemaParticle to the bindings would be valuable since users can obtain pointers to them via the existing API (even though the libxml2 types may lie about what kind of pointer they are).

I don't know if this was something that was manually removed from the generated bindings or if the bindgen tool simply missed them for some reason, but either way if this is something you'd like to add I could go about making a PR to implement that if it's easier for you.

Encoding issue with libxml 2.11.1, 2.11.2, 2.11.3 (OK with libxml 2.11.0)

Hi,

I've a strange encoding issue started with libxml 2.11.1+, (released a week ago https://gitlab.gnome.org/GNOME/libxml2/-/tags) with libxml rust crate 0.3.2.

My sample:

• I've the following html document <data>café</data>
• I evaluate the following xpath expression normalize-space(//data).

Sample code:

use std::ffi::CStr;
use std::os::raw;
use libxml::parser::{Parser, ParserOptions};
use libxml::xpath::Context;

fn main() {
    let parser = Parser::default_html();
    let options = ParserOptions { encoding: Some("utf-8"), ..Default::default()};
    let data = "<data>café</data>";
    let doc = parser.parse_string_with_options(data, options).unwrap();

    let context = Context::new(&doc).unwrap();
    let result = context.evaluate("normalize-space(//data)").unwrap();

    assert_eq!(unsafe { *result.ptr }.type_, libxml::bindings::xmlXPathObjectType_XPATH_STRING);
    let value = unsafe { *result.ptr }.stringval;
    let value = value as *const raw::c_char;
    let value = unsafe { CStr::from_ptr(value) };
    let value = value.to_string_lossy();
    println!("{value}")
}

With libxml 2.11.0, the value printed is café, with libxml 2.11.1 the value printed is café:

• With libxml 2.11.0:

$ export LIBXML2=/Users/jc/Documents/Dev/libxml/libxml2-2.11.0/lib/libxml2.2.dylib
$ cargo clean && cargo run
$ café

• With libxml 2.11.3:

$ export LIBXML2=/Users/jc/Documents/Dev/libxml/libxml2-2.11.3/lib/libxml2.2.dylib
$ cargo clean && cargo run
$ café

I've the impression that the encoding value of ParserOptions is not evaluated correctly through the crate (note: to reproduce the bug, you've to use Parser::default_html() and not Parser::default())

To confirm this, I've tested the "equivalent" code in plain C with libxml 2.11.3:

#include <string.h>
#include <libxml/HTMLparser.h>
#include <libxml/xpath.h>

int main() {
    xmlDocPtr doc = NULL;
    xmlXPathContextPtr context = NULL;
    xmlXPathObjectPtr result = NULL;

    // <data>café</data> in utf-8:
    char data[] = (char[]) {0x3c, 0x64, 0x61, 0x74, 0x61, 0x3e, 0x63, 0x61, 0x66, 0xc3, 0xa9, 0x3c, 0x2f, 0x64, 0x61,
                            0x74, 0x61, 0x3e};
    doc = htmlReadMemory(data, strlen(data), NULL, "utf-8",
                         HTML_PARSE_RECOVER | HTML_PARSE_NOERROR | HTML_PARSE_NOWARNING);

    // Creating result request
    context = xmlXPathNewContext(doc);
    result = xmlXPathEvalExpression((const unsigned char *) "normalize-space(//data)", context);
    if (result->type == XPATH_STRING) {
        printf("%s\n", result->stringval);
    }

    xmlXPathFreeObject(result);
    xmlXPathFreeContext(context);
    xmlFreeDoc(doc);
    return 0;
}

• With libxml 2.11.0:

$ gcc -L/Users/jc/Documents/Dev/libxml/libxml2-2.11.0/lib -l xml2 test.c
$ ./a.out
$ café

• With libxml 2.11.3:

$ gcc -L/Users/jc/Documents/Dev/libxml/libxml2-2.11.3/lib -l xml2 test.c
$ ./a.out
$ café

My suspision is in

rust-libxml/src/parser.rs

Line 292 in a10a5a6

pub fn parse_string_with_options<Bytes: AsRef<[u8]>>(

When I debug the following code:

   // Process encoding.
    let encoding_cstring: Option<CString> =
      parser_options.encoding.map(|v| CString::new(v).unwrap());
    let encoding_ptr = match encoding_cstring {
      Some(v) => v.as_ptr(),
      None => DEFAULT_ENCODING,
    };

    // Process url.
    let url_ptr = DEFAULT_URL;

If parser encoding is initialized with Some("utf-8"), encoding_ptr is not valid just before // Process url (it points to a null char).
So the call to the binding htmlReadMemory is made with no encoding... The unsafe part of the code is my Rust limit of understanding so I'm unable to see if there is something bad here. I hope my issue is clear, and, I should have started by this, thank you for your work on this crate !

Regards,

Jc

Allow user-specified ParserOptions

My use-case is parsing an HTML fragment, manipulating it and then serializing it again as a fragment. Currently there is no way around serializing it as a full HTML document and having to remove the "chrome" again afterwards:

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html><body>
</body></html>

As far as I understand HTML_PARSE_NOIMPLIED = 8192, // Do not add implied html/body... elements would solve this for me. But the crate doesn't offer the option to pass options for parsing. There is however a comment in the code suggesting that the only reason that this didn't happen is time/manpower.
https://github.com/KWARC/rust-libxml/blob/master/src/parser.rs#L201

If it is okay with you @dginev I would like to implement this in a similar fashion as #59.

edit: there is also the workaround of creating a new empty document and moving all the relevant nodes from the old to the new doc. But it's not super obvious and a bit cumbersome.

Build failing on MacOS X

When I try using the libxml crate on a Mac OS X laptop, I get the following build error:

thread 'main' panicked at 'Could not find libxml2.', /Users/balls/.cargo/registry/src/github.com-1ecc6299db9ec823/libxml-0.3.0/build.rs:25:5

This appears to be similar to other issues (#81, #72), but they were on Windows so I don't think the answer/resolution will be the same.

Is the crate looking for an existing installation of libxml2? If so then how can I point the build to it?

Publish on crates.io

I am currently looking into publishing the libxml wrapper to crates.io.

One big missing part are publicly hosted API docs, which I am working on jotting down with the help of adding #![deny(missing_docs)] directive to lib.rs, which turns all missing documentation into compile-time errors.

A question to solve with @jfschaefer is whether he wants to keep the wrapper hosted on his own repo, which is fine, but implies longer term commitment to maintaining the wrapper.

Creating new documents from retrieved nodes

Hello,

I am trying to create a new document from the results of get_nodes_as_vec and creating a new struct that contains a document but I get a memory error since documet.set_root_element uses a reference which is gone by the time the get_nodes_as_vec` loop is over (that's what I think is happening at least).

This is the code that generates the error:

extern crate libxml;

const XML: &str = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
    <notes>
    <note>
      <to>Tove</to>
      <from>Jani</from>
      <heading>Reminder</heading>
      <body>Don't forget me this weekend!</body>
    </note>
    </notes>";

#[derive(Clone)]
struct Node {
  xml_doc: libxml::tree::Document
}

impl Node {
  pub fn new(data: &str) -> Node {
    let parser = libxml::parser::Parser::default();
    let document = parser.parse_string(data.as_bytes()).unwrap();
    Node { xml_doc: document.clone() }
  }

  pub fn find_nodes(&self, query: &str) -> Vec<Node>  {
    let ctx = libxml::xpath::Context::new(&self.xml_doc).unwrap();
    let result = ctx.evaluate(query).unwrap();
    let mut v: Vec<Node> = Vec::new();

    for node in result.get_nodes_as_vec() {
      let mut doc = libxml::tree::Document::new().unwrap();
      doc.set_root_element(&node);
      v.push(Node{ xml_doc: doc.clone() });
    }
    v
  }
}

fn main() {
  let xml_node = Node::new(XML);
  let nodes = xml_node.find_nodes("//note");
}

This is the error I get when running it:

free(): invalid pointer
error: process didn't exit successfully: `/home/lrossetti/Work/sumatraproject/rumatra/target/debug/deps/rumatra-f41b8ad71503704e` (signal: 6, SIGABRT: process abort signal)

Is there another way to create a new document that contains a node? Should I directly use the libxml bindings?

Error handling

Hi!

First of all, thanks for your work on this project. I've recently started using this crate for XML validation and when running tests with syntactically incorrect XML files (which might be the result from bad merges) I noticed that internal errors from libxml2 result in a panic.

I'm happy to contribute on this matter myself, I was just wondering which route you might want to take here. I've had a look at all the TODOs regarding error handling - turning the null-pointer checks in
https://github.com/KWARC/rust-libxml/blob/da4369a035557667650a221bc89d5e59d0efb247/src/schemas/parser.rs#L24C1-L26C6
from panics to results would necessitate either:

• making breaking API changes due to the changed return-type of the function
• to deprecate these functions and introduce new versions that support error handling

All the other occurrences of panics due to missing error-handling already return a Result<Self, Vec<StructuredError>>, so implementing these should only be a matter of creating new StructuredErrors, i.e. StructuredError::null_ptr() and StructuredError::internal().

Any and all feedback is appreciated!

All `StructuredError` returned by `SchemaValidationContext::validate_*` are identical

Description

When multiple validation issues are present in a document, all of the StructuredError objects returned by validate_document or validate_file are the exact same object in memory.

The unit test for schema validation seems to suggest that this is expected behavior, but other FFI wrappers around libxml (such as: https://github.com/marudor/libxmljs2/wiki#validating-against-xsd-schema) do not exhibit this behavior. I was hoping to learn more about why this is happening and if there is any way to get each unique schema validation issue that's present.

Reproduction

Change the following code in tests/schema_tests.rs, https://github.com/KWARC/rust-libxml/blob/master/tests/schema_tests.rs#L93-L98 to the following:

for err in &errors {
  println!("{:#?}", err);
  assert_eq!(
    "Element 'bad': This element is not expected. Expected is ( to ).\n",
    err.message()
  );
}

Then run the tests using cargo test -- --nocapture. In the test output, you should see something like:

     Running tests/schema_tests.rs (target/debug/deps/schema_tests-b2c35c81d74ebf18)

running 2 tests
StructuredError(
    0x0000000120e05748,
)
StructuredError(
    0x0000000120e05748,
)
StructuredError(
    0x0000000120e05748,
)
StructuredError(
    0x0000000120e05748,
)
StructuredError(
    0x0000000120e05748,
)
test schema_from_string ... ok
test schema_from_string_generates_errors ... ok

Expected vs Actual Behavior

A unique StructuredError object for each unique validation issue present in the document. Instead, a Vec containing the same repeated StructuredError is returned.

System Information

• cargo version: cargo 1.70.0 (ec8a8a0ca 2023-04-25)
• rustup show:
  • Default host: aarch64-apple-darwin
  • Platform: stable-aarch64-apple-darwin (default)
  • Compiler Version: rustc 1.70.0 (90c541806 2023-05-31)

Looking for wellformed check

Hello,
I am the creator or node-libxml, I would like to based my lib on your's instead of the C implementation.
I am facing difficulty to try to perform wellformed check

main.rs

use libxml::{tree::Document, parser::XmlParseError};



fn main() {
    let parser = libxml::parser::Parser::default(); 
    let xml_file = parser.parse_file("tests/data/test-not-wellformed.xml");
    // let root_name = xml_file.unwrap().get_root_element().unwrap().get_name();
    dbg!(is_wellformed(xml_file.unwrap().get_root_element()));
}

fn is_wellformed(doc: Result<Document, XmlParseError>)-> bool{
    match doc {
        Err(_error) => {
            false
        },
        Ok(_doc) => {
            true
        },
    }
}

tests/data/test-not-wellformed.xml

<!DOCTYPE article PUBLIC "my doctype of doom" "mydoctype.dtd">
<xpath>
    <to>
        <my>
            <infos>trezaq</infos>
    </to>
</xpath>
``

return me true, should return me false because it's not wellformed.

Furthermore I would need DTD & XSD validation and path parsing but I suppose I will need other libraries or contribute to your ;) 

STATUS_ACCESS_VIOLATION with Evaluate function while parsing xpath [Urgent]

I'm trying to implement the code to parse the Xpath, but the issue I'm facing is when I'm returning the variable of type evaluate(&self, xpath: &str) -> Result<Object, ()> from the funtion, my application is crashing.

fn parse_xpath(test_val: &str) -> Object{

   let parser = Parser::default();
    let doc = parser.parse_file("file1.xml").unwrap();
    let xpath_c = Context::new(&doc).unwrap();
    let  xpath_res = xpath_c.evaluate(test_val).unwrap();

    xpath_res
}

fn main() {

    let xpath_res = parse_xpath("//Root/Os/Win");

    let node_cnt = xpath_res.get_number_of_nodes();
    println!("{} nodes are present", node_cnt);
    
    for node in &xpath_res.get_nodes_as_vec(){
            println!("Found: {}", node.get_name());                                  //program crashing after reaching here.
            println!("Found: {:?}", node.get_attributes());
            println!("Found: {:?}", node.get_child_elements());
    }

}

I want to know the way I'm returning the Object of type evaluate correctly or what is the correct way to do that?

Program output:

2 nodes are present
Found: Plugin
error: process didn't exit successfully: target\debug\SampleProgram.exe (exit code: 0xc0000005, STATUS_ACCESS_VIOLATION)

Link issue with Homebrew on Linux

Hi,

First of all, thank you for your crate, it's really invaluable.

I'm trying to build a Homebrew formula for Hurl (in Homebrew vocabulary, a formula is a ruby script to compile and install binaries). Hurl uses libxml crate for xpath evaluation. With a standard setup (lixmb2 installed with apt-get for instance), everything is ok and we can build and run our cli without any problem.

On macOS, I don't have any problem and my formula is building.

On Linux (Homebrew requires formulae to be available on Linux), I've some problem building my binary.

In this setup, libxml2 is provided by Homebrew, and installed in /home/linuxbrew/.linuxbrew/Cellar/libxml2/2.9.12

The directory contains:

$ ls /home/linuxbrew/.linuxbrew/Cellar/libxml2/2.9.12
AUTHORS  ChangeLog  Copyright  INSTALL_RECEIPT.json  NEWS  README  TODO  bin  include  lib  share

In lib directory:

$ find lib
lib/libxml2.so
lib/libxml2.a
lib/libxml2.so.2
...
lib/pkgconfig/libxml-2.0.pc
...

(note the name of pkgconfig file: libxml-2.0.pc)

When I'm building the Brew formula, I call cargo install --path packages/hurl --locked in the Homebrew install script.

If I try to build as it, I've a compilation error (rightfully raised by libxml/build.rs):

thread 'main' panicked at 'Could not find libxml2.', /root/.cache/Homebrew/cargo_cache/registry/src/github.com-1ecc6299db9ec823/libxml-0.3.0/build.rs:25:5

Then, I set the LIBXML2 env variable to the Brew libxml2 pkgconfig file:

export LIBXML2=/home/linuxbrew/.linuxbrew/opt/libxml2/lib/pkgconfig/libxml-2.0.pc

Now, I pass the compilation phase, but I've an error at link:

error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "/
  ...
  "-Wl,-Bdynamic" "-lxml-2.0" "-lssl" 
  ...
    = note: /home/linuxbrew/.linuxbrew/bin/ld: cannot find -lxml-2.0
          collect2: error: ld returned 1 exit status

The link flag -lxml-2.0 seems suspicious, and, if my understanding is good, this flag is build from the LIBXML2 environment
variable in libxml/build.rs:

rust-libxml/build.rs

Lines 2 to 10 in 40020e1

	if let Ok(ref s) = std::env::var("LIBXML2") {
	// println!("{:?}", std::env::vars());
	// panic!("set libxml2.");
	let p = std::path::Path::new(s);
	let fname = std::path::Path::new(p.file_name().expect("no file name in LIBXML2 env"));
	assert!(p.is_file());
	println!("cargo:rustc-link-lib={}", fname.file_stem().unwrap().to_string_lossy().strip_prefix("lib").unwrap());
	println!("cargo:rustc-link-search={}", p.parent().expect("no library path in LIBXML2 env").to_string_lossy());
	} else {

Now, if a rename the pkgconfig to be /home/linuxbrew/.linuxbrew/opt/libxml2/lib/pkgconfig/libxml2.pc (instead of libxml-2.0.pc), the link phase is good and I can build my binary....

I don't know if this is an issue of the libxml2 Brew Formula, and certainly this setup is rather unusual (build a rust projet on linux with c dependencies provided by Homebrew). Currently, I can't rename the libxml-2-0.pc as it's a readonly dependency provided by Homebrew. Do you have any idea on how I can "give" libxml crate the right build paths?

I've the impression (maybe naive), that we could explicitely pass the value of cargo:rustc-link-lib and cargo:rustc-link-search instead of deriving those values from LIBXML2...

Thank you,

Jc

Nokogiri like api

I've been playing around with a Nokogiri like api, would we like something like this?

#![allow(non_snake_case)]
use tree::Document;
/// XML parsing module
pub mod xml;

use std::io::{Read, BufReader};
use std::fs::File;
use std::path::Path;

pub trait XmlInput {
    const IS_PATH: bool = false;
    fn is_path(&self) -> bool { Self::IS_PATH }
    fn data(&self) -> String;
}

impl XmlInput for str {
    fn data(&self) -> String {
        String::from(self)
    }
}

impl XmlInput for String {
    fn data(&self) -> String {
        self.to_owned()
    }
}

impl XmlInput for Path {
    const IS_PATH: bool = true;
    fn data(&self) -> String {
        String::from(self.to_str().expect("Could not get path"))
    }
}

impl XmlInput for File {
    const IS_PATH: bool = false;
    fn data(&self) -> String {
        let mut tmp = String::new();
        {
            let mut a = BufReader::new(self);
            a.read_to_string(&mut tmp).expect("Could not read_to_string");
        }
        tmp
    }
}


/// Convenience
pub fn XML<R: Read>(mut r: R) -> Result<Document, &'static str> {
    let mut xml_str = String::new();
    r.read_to_string(&mut xml_str).expect("Could not read_to_string");
    println!("ergo::XML()");
    xml::parse_string(&xml_str)
}

pub fn XML2<R: XmlInput + ?Sized>(r:&R) -> Result<Document, &'static str> {
    println!("ergo::XML2()");
    match r.is_path() {
        true => xml::parse_file(&r.data()),
        false => xml::parse_string(&r.data())
    }
}


#[cfg(test)]
mod tests {
    use std::fs::File;
    use super::*;
    #[test]
    fn ergo_test(){
        assert!(XML("<root></root>".as_bytes()).is_ok());
        assert!(XML(File::open("tests/resources/file01.xml").unwrap()).is_ok());
        assert!(XML2("<root></root>").is_ok());
        assert!(XML2(&String::from("<root></root>")).is_ok());
        assert!(XML2(&File::open("tests/resources/file01.xml").unwrap()).is_ok());
        assert!(XML2(Path::new("tests/resources/file01.xml")).is_ok());
    }
}

Expose xmlTextWriterSetIndent

Can you expose the ability to indent an XML Document? Aka something like adding indent: int to SaveOptions for use in to_string_with_options.

I believe the appropriate libxml API is xmlTextWriterSetIndent. I'm trying to replicate something akin to xmllint --format.

windows cargo build fails on libxml

Hi,

I try to build my application on windows, and during the compilation the libxml compilation fails:

error: failed to run custom build command for `libxml v0.2.15`

Caused by:
  process didn't exit successfully: `C:\Applications\target\debug\build\libxml-19fee8ddbe173fc8\build-script-build` (exit code: 101)
  --- stderr
  thread 'main' panicked at 'Could not find libxml2.', C:\Users\frxn7205\.cargo\registry\src\github.com-1ecc6299db9ec823\libxml-0.2.15\build.rs:16:3
  stack backtrace:
     0:     0x7ff6a2cf016e - std::backtrace_rs::backtrace::dbghelp::trace
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:108
     1:     0x7ff6a2cf016e - std::backtrace_rs::backtrace::trace_unsynchronized
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
     2:     0x7ff6a2cf016e - std::sys_common::backtrace::_print_fmt
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\sys_common\backtrace.rs:67
     3:     0x7ff6a2cf016e - std::sys_common::backtrace::_print::{{impl}}::fmt
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\sys_common\backtrace.rs:46
     4:     0x7ff6a2cffc1b - core::fmt::write
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\core\src\fmt\mod.rs:1078
     5:     0x7ff6a2cecdd8 - std::io::Write::write_fmt<std::sys::windows::stdio::Stderr>
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\io\mod.rs:1518
     6:     0x7ff6a2cf2cfd - std::sys_common::backtrace::_print
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\sys_common\backtrace.rs:49
     7:     0x7ff6a2cf2cfd - std::sys_common::backtrace::print
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\sys_common\backtrace.rs:36
     8:     0x7ff6a2cf2cfd - std::panicking::default_hook::{{closure}}
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\panicking.rs:208
     9:     0x7ff6a2cf28d8 - std::panicking::default_hook
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\panicking.rs:227
    10:     0x7ff6a2cf366f - std::panicking::rust_panic_with_hook
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\panicking.rs:593
    11:     0x7ff6a2c91fa1 - std::panicking::begin_panic::{{closure}}<str*>
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\library\std\src\panicking.rs:522
    12:     0x7ff6a2c9566f - std::sys_common::backtrace::__rust_end_short_backtrace<closure-0,!>
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\library\std\src\sys_common\backtrace.rs:141
    13:     0x7ff6a2c91ed7 - std::panicking::begin_panic<str*>
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\library\std\src\panicking.rs:521
    14:     0x7ff6a2c9102f - build_script_build::main
                                 at C:\Users\frxn7205\.cargo\registry\src\github.com-1ecc6299db9ec823\libxml-0.2.15\build.rs:16
    15:     0x7ff6a2c910fb - core::ops::function::FnOnce::call_once<fn(),tuple<>>
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\library\core\src\ops\function.rs:227
    16:     0x7ff6a2c9109b - std::sys_common::backtrace::__rust_begin_short_backtrace<fn(),tuple<>>
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\library\std\src\sys_common\backtrace.rs:125
    17:     0x7ff6a2c91351 - std::rt::lang_start::{{closure}}<tuple<>>
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\library\std\src\rt.rs:66
    18:     0x7ff6a2cf3873 - core::ops::function::impls::{{impl}}::call_once
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\library\core\src\ops\function.rs:280
    19:     0x7ff6a2cf3873 - std::panicking::try::do_call
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\panicking.rs:381
    20:     0x7ff6a2cf3873 - std::panicking::try
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\panicking.rs:345
    21:     0x7ff6a2cf3873 - std::panic::catch_unwind
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\panic.rs:396
    22:     0x7ff6a2cf3873 - std::rt::lang_start_internal
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\/library\std\src\rt.rs:51
    23:     0x7ff6a2c91323 - std::rt::lang_start<tuple<>>
                                 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca\library\std\src\rt.rs:65
    24:     0x7ff6a2c91060 - main
    25:     0x7ff6a2d04010 - invoke_main
                                 at d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
    26:     0x7ff6a2d04010 - __scrt_common_main_seh
                                 at d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    27:     0x7ffcbab27974 - BaseThreadInitThunk
    28:     0x7ffcbb79a0b1 - RtlUserThreadStart
warning: build failed, waiting for other jobs to finish...
error: build failed

however I installed gcc and the libxml2 on my windows with win-builds and vcpkg:

$ vpkg list
libxml2:x86-windows                                2.9.10#2         Libxml2 is the XML C parser and toolkit develope...

$ where gcc.exe
C:\Applications\winbuilds\bin\gcc.exe

Can you help me with this? Is it even possible to compile under windows with the use of libxml ?

thks :)

Removal of Error trait for XmlParseError

Happened in e10459c. Any particular reason for this?

NodeReader trait

I continue using mostly RoNode in my main code, since most of my traversals are read-only and the benefit is significant. I just ended up adding a handful of identical methods to both Node and RoNode and the redundancy is quite obvious+painful.

I think all of the read-only methods can be extracted out as a trait, and then have very simple default implementations for both structs:

impl NodeReader for Node {}
impl NodeReader for RoNode {}

and cut the code footprint in half. Not urgent, but definitely needed if the two structs are to coexist in peace going forward.

Issues with unliking nodes

With the release of rust 1.39 I hopped on the async train for the first time. However I am having some difficulties with libxml related code. I'm not sure if it's a problem with my code or libxml itself. I'll try to describe my problems here. Hopefully someone can help.

I'm running my unit tests with #[tokio::test] which states Uses current_thread runtime.. So there should be no multi threading shenanigans.

The first test straight up crashes with this message:

malloc_consolidate(): invalid chunk size

I nailed it down to the node.unlink() line in this function:

fn strip_id_or_class(context: &Context, id_or_class: &String) -> Result<(), ScraperError> {
   let xpath = &format!("//*[contains(@class, '{}') or contains(@id, '{}')]", id_or_class, id_or_class);
   evaluate_xpath!(context, xpath, node_vec);
   for mut node in node_vec {
      node.unlink();
   }
   Ok(())
}

A very similar function does not trigger this error. Maybe it's a very specific case that is triggered here.

The second problem I'm having is because async functions don't support recursing right now. So I refactored my code into a loop that looks like this:

let mut xpath_ctx = xpath_ctx;

loop {
    if let Some(url) = self.check_for_next_page(&xpath_ctx, config) {
        let mut html = ArticleScraper::download(&url, &self.client).await?;
        parse_html!(html, config, new_xpath_ctx);
        ArticleScraper::strip_junk(&new_xpath_ctx, config, &url);
        ArticleScraper::extract_body(&new_xpath_ctx, root, config)?;
        xpath_ctx = new_xpath_ctx;
    } else {
        break;
    }
}

However in the second iteration of the loop the check_for_next_page function gets stuck evaluating an xpath expression (for which I wrote a macro)

let res = $context.evaluate($xpath).map_err(|()| {
    error!("Evaluation of xpath {} yielded no results", $xpath);
    ScraperErrorKind::Xml
})?;

let $node_vec = res.get_nodes_as_vec();

The complete code is over here. So if anyone has any ideas I'd be happy to hear them.

If the code base of my crate is too big I can try to whip up some minimal samples.
Thank you!

Node Iterators

Hello,

thanks for very good library. Is there any chance, that you can implement iterators for Node and RoNode?

No error on malformed html?

Is it right that the library doesn't give an error on malformed html? I have a website I was trying to scrape, but I wasn't able to find my elements with xpath. I managed to succeed on the same website using pythons beautifulsoup in combination with lxml.

Crossing the boundary with context data for error handling callbacks in libxml2

Hi,

I'm currently wrapping XSD functionality and banging my head against the table while attempting to capture structured errors from the parser and later validation contexts.

Does any one of you have experience in passing a pointer to a Vec across the FFI together with the callback, so the callback can then on call by libxml2 stuff the error in there, allowing for the parser or validation context wrappers on the other side to return them in a structured fashion?

Callback context passed through the FFI;

impl SchemaParserContext
{
    fn from_raw(parser: *mut bindings::_xmlSchemaParserCtxt) -> Self
    {
        let errors = Rc::new(RefCell::new(Vec::new()));

        unsafe {
            bindings::xmlSchemaSetParserStructuredErrors(
                parser,
                Some(common::structured_error_handler),
                Box::into_raw(Box::new(errors.clone())) as *mut _
            );
        }

        Self {inner: parser, errlog: errors}
    }
}

Callback passed through the FFI;

/// Provides a callback to the C side of things to accumulate xmlErrors to be
/// handled back on the Rust side.
pub fn structured_error_handler(ctx: *mut c_void, error: bindings::xmlErrorPtr)
{
    let errlog = unsafe {
        Box::from_raw(ctx as *mut Rc<RefCell<Vec<XmlStructuredError>>>)
    };

    let error = XmlStructuredError::from_raw(error);

    errlog.borrow_mut()
        .push(error);
}

The issue with this is that the Rc increments only once while settings the callback with libxml2. But on each error, it decrements in the callback. So it essentially frees the Vec away. I'm at loss on what else to use.

Any ideas?

Document::get_root_element

So I've started working on pointer book keeping. The method below returns a Node with the doc_ptr as node_ptr if there is no root element. Is there a reason for this? I think this will add complexity as I would have to add it to a list of known nodes and do a lot of checks if the node is infact a node or a document. Would this be something we can change? I know we would like to keep api compatbility.

rust-libxml/src/tree.rs

Line 100 in 75f0655

pub fn get_root_element(&self) -> Node {

Consider using bindgen

It might be useful to use bindgen to generate an initial and comprehensive set of raw Rust bindings (in place of the incomplete hand-written bindings) and then refactor the lifecycle management wrappers to go around those.

Add bindings for XSLT and XSD

Unfortunately no single crate out there supports either one, however libxml supports both. I can't find docs on them yet. Maybe take inspiration from lxml?

• https://git.launchpad.net/lxml/tree/src/lxml/xslt.pxi
• https://git.launchpad.net/lxml/tree/src/lxml/xmlschema.pxi

dtd suport

I just bumped into this crate. Thank you for this!

I can see that the latest work you've been doing is around xml schemas. With this in mind, are you planning to add bindings for DTD validation?

Thread safety?

I'm really curious if we can make the wrapper thread-safe though, since if that were possible there may be some interesting parallel execution ideas. Quite fond of rayon's patterns for example, and am curious if we can do some libxml2 magic to make possible e.g.

nodes.par_iter_mut().for_each(|node| node.set_attribute("..."));

When it comes to hardcore tree manipulations, there are definite scenarios where one ends up CPU-bound. For example big rule-based tree rewrites, that are -- very importantly -- not in XSLT, but instead follow some more involved computational machinery - for example embedding a list of leaf nodes into a tree based on some grammar derivation.

Enable crate on 32-bit architectures

full bindgen trace

failures:
---- bindings::bindgen_test_layout_UConverterFromUnicodeArgs stdout ----
thread 'bindings::bindgen_test_layout_UConverterFromUnicodeArgs' panicked at 'assertion failed: `(left == right)`
  left: `28`,
 right: `56`: Size of: UConverterFromUnicodeArgs', src/bindings.rs:10144:3
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---- bindings::bindgen_test_layout__G_fpos_t stdout ----
thread 'bindings::bindgen_test_layout__G_fpos_t' panicked at 'assertion failed: `(left == right)`
  left: `12`,
 right: `16`: Size of: _G_fpos_t', src/bindings.rs:744:3
---- bindings::bindgen_test_layout__IO_marker stdout ----
thread 'bindings::bindgen_test_layout__IO_marker' panicked at 'assertion failed: `(left == right)`
  left: `12`,
 right: `24`: Size of: _IO_marker', src/bindings.rs:829:3
---- bindings::bindgen_test_layout_UConverterToUnicodeArgs stdout ----
thread 'bindings::bindgen_test_layout_UConverterToUnicodeArgs' panicked at 'assertion failed: `(left == right)`
  left: `28`,
 right: `56`: Size of: UConverterToUnicodeArgs', src/bindings.rs:10253:3
---- bindings::bindgen_test_layout__G_fpos64_t stdout ----
thread 'bindings::bindgen_test_layout__G_fpos64_t' panicked at 'assertion failed: `(left == right)`
  left: `12`,
 right: `16`: Size of: _G_fpos64_t', src/bindings.rs:783:3
---- bindings::bindgen_test_layout__IO_FILE stdout ----
thread 'bindings::bindgen_test_layout__IO_FILE' panicked at 'assertion failed: `(left == right)`
  left: `124`,
 right: `216`: Size of: _IO_FILE', src/bindings.rs:910:3
---- bindings::bindgen_test_layout___pthread_cond_s__bindgen_ty_1 stdout ----
thread 'bindings::bindgen_test_layout___pthread_cond_s__bindgen_ty_1' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of __pthread_cond_s__bindgen_ty_1', src/bindings.rs:12079:3
---- bindings::bindgen_test_layout___pthread_cond_s stdout ----
thread 'bindings::bindgen_test_layout___pthread_cond_s' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of __pthread_cond_s', src/bindings.rs:12211:3
---- bindings::bindgen_test_layout___pthread_cond_s__bindgen_ty_2 stdout ----
thread 'bindings::bindgen_test_layout___pthread_cond_s__bindgen_ty_2' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of __pthread_cond_s__bindgen_ty_2', src/bindings.rs:12174:3
---- bindings::bindgen_test_layout___pthread_internal_list stdout ----
thread 'bindings::bindgen_test_layout___pthread_internal_list' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: __pthread_internal_list', src/bindings.rs:11866:3
---- bindings::bindgen_test_layout___pthread_mutex_s stdout ----
thread 'bindings::bindgen_test_layout___pthread_mutex_s' panicked at 'assertion failed: `(left == right)`
  left: `32`,
 right: `40`: Size of: __pthread_mutex_s', src/bindings.rs:11912:3
---- bindings::bindgen_test_layout___sigset_t stdout ----
thread 'bindings::bindgen_test_layout___sigset_t' panicked at 'assertion failed: `(left == right)`
  left: `64`,
 right: `128`: Size of: __sigset_t', src/bindings.rs:11537:3
---- bindings::bindgen_test_layout___pthread_rwlock_arch_t stdout ----
thread 'bindings::bindgen_test_layout___pthread_rwlock_arch_t' panicked at 'assertion failed: `(left == right)`
  left: `48`,
 right: `56`: Size of: __pthread_rwlock_arch_t', src/bindings.rs:11721:3
---- bindings::bindgen_test_layout___va_list_tag stdout ----
thread 'bindings::bindgen_test_layout___va_list_tag' panicked at 'assertion failed: `(left == right)`
  left: `16`,
 right: `24`: Size of: __va_list_tag', src/bindings.rs:24263:3
---- bindings::bindgen_test_layout__htmlElemDesc stdout ----
thread 'bindings::bindgen_test_layout__htmlElemDesc' panicked at 'assertion failed: `(left == right)`
  left: `36`,
 right: `64`: Size of: _htmlElemDesc', src/bindings.rs:15756:3
---- bindings::bindgen_test_layout__htmlEntityDesc stdout ----
thread 'bindings::bindgen_test_layout__htmlEntityDesc' panicked at 'assertion failed: `(left == right)`
  left: `12`,
 right: `24`: Size of: _htmlEntityDesc', src/bindings.rs:15918:3
---- bindings::bindgen_test_layout__uconv_t stdout ----
thread 'bindings::bindgen_test_layout__uconv_t' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _uconv_t', src/bindings.rs:10987:3
---- bindings::bindgen_test_layout__xlinkHandler stdout ----
thread 'bindings::bindgen_test_layout__xlinkHandler' panicked at 'assertion failed: `(left == right)`
  left: `12`,
 right: `24`: Size of: _xlinkHandler', src/bindings.rs:13339:3
---- bindings::bindgen_test_layout__xmlAttr stdout ----
thread 'bindings::bindgen_test_layout__xmlAttr' panicked at 'assertion failed: `(left == right)`
  left: `48`,
 right: `96`: Size of: _xmlAttr', src/bindings.rs:3275:3
---- bindings::bindgen_test_layout__xmlAttribute stdout ----
thread 'bindings::bindgen_test_layout__xmlAttribute' panicked at 'assertion failed: `(left == right)`
  left: `64`,
 right: `120`: Size of: _xmlAttribute', src/bindings.rs:2209:3
---- bindings::bindgen_test_layout__xmlBuffer stdout ----
thread 'bindings::bindgen_test_layout__xmlBuffer' panicked at 'assertion failed: `(left == right)`
  left: `20`,
 right: `32`: Size of: _xmlBuffer', src/bindings.rs:1952:3
---- bindings::bindgen_test_layout__xmlChRangeGroup stdout ----
thread 'bindings::bindgen_test_layout__xmlChRangeGroup' panicked at 'assertion failed: `(left == right)`
  left: `16`,
 right: `24`: Size of: _xmlChRangeGroup', src/bindings.rs:16304:3
---- bindings::bindgen_test_layout__xmlCharEncodingHandler stdout ----
thread 'bindings::bindgen_test_layout__xmlCharEncodingHandler' panicked at 'assertion failed: `(left == right)`
  left: `28`,
 right: `56`: Size of: _xmlCharEncodingHandler', src/bindings.rs:11034:3
---- bindings::bindgen_test_layout__xmlDOMWrapCtxt stdout ----
thread 'bindings::bindgen_test_layout__xmlDOMWrapCtxt' panicked at 'assertion failed: `(left == right)`
  left: `16`,
 right: `32`: Size of: _xmlDOMWrapCtxt', src/bindings.rs:4100:3
---- bindings::bindgen_test_layout__xmlDoc stdout ----
thread 'bindings::bindgen_test_layout__xmlDoc' panicked at 'assertion failed: `(left == right)`
  left: `96`,
 right: `176`: Size of: _xmlDoc', src/bindings.rs:3817:3
---- bindings::bindgen_test_layout__xmlDtd stdout ----
thread 'bindings::bindgen_test_layout__xmlDtd' panicked at 'assertion failed: `(left == right)`
  left: `64`,
 right: `128`: Size of: _xmlDtd', src/bindings.rs:3081:3
---- bindings::bindgen_test_layout__xmlElement stdout ----
thread 'bindings::bindgen_test_layout__xmlElement' panicked at 'assertion failed: `(left == right)`
  left: `56`,
 right: `112`: Size of: _xmlElement', src/bindings.rs:2809:3
---- bindings::bindgen_test_layout__xmlEntity stdout ----
thread 'bindings::bindgen_test_layout__xmlEntity' panicked at 'assertion failed: `(left == right)`
  left: `76`,
 right: `136`: Size of: _xmlEntity', src/bindings.rs:7018:3
---- bindings::bindgen_test_layout__xmlEnumeration stdout ----
thread 'bindings::bindgen_test_layout__xmlEnumeration' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _xmlEnumeration', src/bindings.rs:2151:3
---- bindings::bindgen_test_layout__xmlError stdout ----
thread 'bindings::bindgen_test_layout__xmlError' panicked at 'assertion failed: `(left == right)`
  left: `52`,
 right: `88`: Size of: _xmlError', src/bindings.rs:5134:3
---- bindings::bindgen_test_layout__xmlGlobalState stdout ----
thread 'bindings::bindgen_test_layout__xmlGlobalState' panicked at 'assertion failed: `(left == right)`
  left: `516`,
 right: `968`: Size of: _xmlGlobalState', src/bindings.rs:14023:3
---- bindings::bindgen_test_layout__xmlID stdout ----
thread 'bindings::bindgen_test_layout__xmlID' panicked at 'assertion failed: `(left == right)`
  left: `24`,
 right: `48`: Size of: _xmlID', src/bindings.rs:3423:3
---- bindings::bindgen_test_layout__xmlNode stdout ----
thread 'bindings::bindgen_test_layout__xmlNode' panicked at 'assertion failed: `(left == right)`
  left: `60`,
 right: `120`: Size of: _xmlNode', src/bindings.rs:3598:3
---- bindings::bindgen_test_layout__xmlLocationSet stdout ----
thread 'bindings::bindgen_test_layout__xmlLocationSet' panicked at 'assertion failed: `(left == right)`
  left: `12`,
 right: `16`: Size of: _xmlLocationSet', src/bindings.rs:24133:3
---- bindings::bindgen_test_layout__xmlNodeSet stdout ----
thread 'bindings::bindgen_test_layout__xmlNodeSet' panicked at 'assertion failed: `(left == right)`
  left: `12`,
 right: `16`: Size of: _xmlNodeSet', src/bindings.rs:17021:3
---- bindings::bindgen_test_layout__xmlNotation stdout ----
thread 'bindings::bindgen_test_layout__xmlNotation' panicked at 'assertion failed: `(left == right)`
  left: `12`,
 right: `24`: Size of: _xmlNotation', src/bindings.rs:2075:3
---- bindings::bindgen_test_layout__xmlElementContent stdout ----
thread 'bindings::bindgen_test_layout__xmlElementContent' panicked at 'assertion failed: `(left == right)`
  left: `28`,
 right: `48`: Size of: _xmlElementContent', src/bindings.rs:2415:3
---- bindings::bindgen_test_layout__xmlParserCtxt stdout ----
thread 'bindings::bindgen_test_layout__xmlParserCtxt' panicked at 'assertion failed: `(left == right)`
  left: `472`,
 right: `752`: Size of: _xmlParserCtxt', src/bindings.rs:7749:3
---- bindings::bindgen_test_layout__xmlOutputBuffer stdout ----
thread 'bindings::bindgen_test_layout__xmlOutputBuffer' panicked at 'assertion failed: `(left == right)`
  left: `32`,
 right: `56`: Size of: _xmlOutputBuffer', src/bindings.rs:14895:3
---- bindings::bindgen_test_layout__xmlParserInput stdout ----
thread 'bindings::bindgen_test_layout__xmlParserInput' panicked at 'assertion failed: `(left == right)`
  left: `60`,
 right: `104`: Size of: _xmlParserInput', src/bindings.rs:7319:3
---- bindings::bindgen_test_layout__xmlNs stdout ----
thread 'bindings::bindgen_test_layout__xmlNs' panicked at 'assertion failed: `(left == right)`
  left: `24`,
 right: `48`: Size of: _xmlNs', src/bindings.rs:2982:3
---- bindings::bindgen_test_layout__xmlParserInputBuffer stdout ----
thread 'bindings::bindgen_test_layout__xmlParserInputBuffer' panicked at 'assertion failed: `(left == right)`
  left: `36`,
 right: `64`: Size of: _xmlParserInputBuffer', src/bindings.rs:14780:3
---- bindings::bindgen_test_layout__xmlParserNodeInfoSeq stdout ----
thread 'bindings::bindgen_test_layout__xmlParserNodeInfoSeq' panicked at 'assertion failed: `(left == right)`
  left: `12`,
 right: `24`: Size of: _xmlParserNodeInfoSeq', src/bindings.rs:7570:3
---- bindings::bindgen_test_layout__xmlParserNodeInfo stdout ----
thread 'bindings::bindgen_test_layout__xmlParserNodeInfo' panicked at 'assertion failed: `(left == right)`
  left: `20`,
 right: `40`: Size of: _xmlParserNodeInfo', src/bindings.rs:7498:3
---- bindings::bindgen_test_layout__xmlRef stdout ----
thread 'bindings::bindgen_test_layout__xmlRef' panicked at 'assertion failed: `(left == right)`
  left: `20`,
 right: `40`: Size of: _xmlRef', src/bindings.rs:3510:3
---- bindings::bindgen_test_layout__xmlSAXLocator stdout ----
thread 'bindings::bindgen_test_layout__xmlSAXLocator' panicked at 'assertion failed: `(left == right)`
  left: `16`,
 right: `32`: Size of: _xmlSAXLocator', src/bindings.rs:8669:3
---- bindings::bindgen_test_layout__xmlSAXHandlerV1 stdout ----
thread 'bindings::bindgen_test_layout__xmlSAXHandlerV1' panicked at 'assertion failed: `(left == right)`
  left: `112`,
 right: `224`: Size of: _xmlSAXHandlerV1', src/bindings.rs:9512:3
---- bindings::bindgen_test_layout__xmlSchemaAttributeGroup stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaAttributeGroup' panicked at 'assertion failed: `(left == right)`
  left: `60`,
 right: `120`: Size of: _xmlSchemaAttributeGroup', src/bindings.rs:19863:3
---- bindings::bindgen_test_layout__xmlSchemaAttributeLink stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaAttributeLink' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _xmlSchemaAttributeLink', src/bindings.rs:19620:3
---- bindings::bindgen_test_layout__xmlSchema stdout ----
thread 'bindings::bindgen_test_layout__xmlSchema' panicked at 'assertion failed: `(left == right)`
  left: `84`,
 right: `160`: Size of: _xmlSchema', src/bindings.rs:21003:3
---- bindings::bindgen_test_layout__xmlSchemaElement stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaElement' panicked at 'assertion failed: `(left == right)`
  left: `104`,
 right: `200`: Size of: _xmlSchemaElement', src/bindings.rs:20499:3
---- bindings::bindgen_test_layout__xmlSAXHandler stdout ----
thread 'bindings::bindgen_test_layout__xmlSAXHandler' panicked at 'assertion failed: `(left == right)`
  left: `128`,
 right: `256`: Size of: _xmlSAXHandler', src/bindings.rs:9143:3
---- bindings::bindgen_test_layout__xmlSchemaFacet stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaFacet' panicked at 'assertion failed: `(left == right)`
  left: `40`,
 right: `72`: Size of: _xmlSchemaFacet', src/bindings.rs:20787:3
---- bindings::bindgen_test_layout__xmlSchemaFacetLink stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaFacetLink' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _xmlSchemaFacetLink', src/bindings.rs:20084:3
---- bindings::bindgen_test_layout__xmlSchemaAnnot stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaAnnot' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _xmlSchemaAnnot', src/bindings.rs:19346:3
---- bindings::bindgen_test_layout__xmlSchemaType stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaType' panicked at 'assertion failed: `(left == right)`
  left: `120`,
 right: `224`: Size of: _xmlSchemaType', src/bindings.rs:20154:3
---- bindings::bindgen_test_layout__xmlSchemaNotation stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaNotation' panicked at 'assertion failed: `(left == right)`
  left: `20`,
 right: `40`: Size of: _xmlSchemaNotation', src/bindings.rs:20912:3
---- bindings::bindgen_test_layout__xmlSchemaAttribute stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaAttribute' panicked at 'assertion failed: `(left == right)`
  left: `76`,
 right: `152`: Size of: _xmlSchemaAttribute', src/bindings.rs:19406:3
---- bindings::bindgen_test_layout__xmlSchemaWildcard stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaWildcard' panicked at 'assertion failed: `(left == right)`
  left: `44`,
 right: `72`: Size of: _xmlSchemaWildcard', src/bindings.rs:19715:3
---- bindings::bindgen_test_layout__xmlSchemaWildcardNs stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaWildcardNs' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _xmlSchemaWildcardNs', src/bindings.rs:19663:3
---- bindings::bindgen_test_layout__xmlShellCtxt stdout ----
thread 'bindings::bindgen_test_layout__xmlShellCtxt' panicked at 'assertion failed: `(left == right)`
  left: `28`,
 right: `56`: Size of: _xmlShellCtxt', src/bindings.rs:18498:3
---- bindings::bindgen_test_layout__xmlURI stdout ----
thread 'bindings::bindgen_test_layout__xmlURI' panicked at 'assertion failed: `(left == right)`
  left: `44`,
 right: `88`: Size of: _xmlURI', src/bindings.rs:21381:3
---- bindings::bindgen_test_layout__xmlValidCtxt stdout ----
thread 'bindings::bindgen_test_layout__xmlValidCtxt' panicked at 'assertion failed: `(left == right)`
  left: `64`,
 right: `112`: Size of: _xmlValidCtxt', src/bindings.rs:6449:3
---- bindings::bindgen_test_layout__xmlXPathAxis stdout ----
thread 'bindings::bindgen_test_layout__xmlXPathAxis' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _xmlXPathAxis', src/bindings.rs:17358:3
---- bindings::bindgen_test_layout__xmlXPathContext stdout ----
thread 'bindings::bindgen_test_layout__xmlXPathContext' panicked at 'assertion failed: `(left == right)`
  left: `204`,
 right: `352`: Size of: _xmlXPathContext', src/bindings.rs:17490:3
---- bindings::bindgen_test_layout__xmlXPathObject stdout ----
thread 'bindings::bindgen_test_layout__xmlXPathObject' panicked at 'assertion failed: `(left == right)`
  left: `40`,
 right: `72`: Size of: _xmlXPathObject', src/bindings.rs:17090:3
---- bindings::bindgen_test_layout__xmlXPathParserContext stdout ----
thread 'bindings::bindgen_test_layout__xmlXPathParserContext' panicked at 'assertion failed: `(left == right)`
  left: `48`,
 right: `88`: Size of: _xmlXPathParserContext', src/bindings.rs:17924:3
---- bindings::bindgen_test_layout__xmlXPathType stdout ----
thread 'bindings::bindgen_test_layout__xmlXPathType' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _xmlXPathType', src/bindings.rs:17215:3
---- bindings::bindgen_test_layout__xmlXPathVariable stdout ----
thread 'bindings::bindgen_test_layout__xmlXPathVariable' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _xmlXPathVariable', src/bindings.rs:17256:3
---- bindings::bindgen_test_layout_drand48_data stdout ----
thread 'bindings::bindgen_test_layout_drand48_data' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of drand48_data', src/bindings.rs:12849:3
---- bindings::bindgen_test_layout_fd_set stdout ----
thread 'bindings::bindgen_test_layout_fd_set' panicked at 'assertion failed: `(left == right)`
  left: `64`,
 right: `128`: Size of: fd_set', src/bindings.rs:11646:3
---- bindings::bindgen_test_layout_ldiv_t stdout ----
thread 'bindings::bindgen_test_layout_ldiv_t' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: ldiv_t', src/bindings.rs:11348:3
---- bindings::bindgen_test_layout_lldiv_t stdout ----
thread 'bindings::bindgen_test_layout_lldiv_t' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of lldiv_t', src/bindings.rs:11392:3
---- bindings::bindgen_test_layout_pthread_attr_t stdout ----
thread 'bindings::bindgen_test_layout_pthread_attr_t' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of pthread_attr_t', src/bindings.rs:12364:3
---- bindings::bindgen_test_layout_pthread_barrier_t stdout ----
thread 'bindings::bindgen_test_layout_pthread_barrier_t' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of pthread_barrier_t', src/bindings.rs:12598:3
---- bindings::bindgen_test_layout_pthread_cond_t stdout ----
thread 'bindings::bindgen_test_layout_pthread_cond_t' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of pthread_cond_t', src/bindings.rs:12456:3
---- bindings::bindgen_test_layout_pthread_mutex_t stdout ----
thread 'bindings::bindgen_test_layout_pthread_mutex_t' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of pthread_mutex_t', src/bindings.rs:12405:3
---- bindings::bindgen_test_layout_pthread_rwlock_t stdout ----
thread 'bindings::bindgen_test_layout_pthread_rwlock_t' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of pthread_rwlock_t', src/bindings.rs:12507:3
---- bindings::bindgen_test_layout__xmlXPathFunct stdout ----
thread 'bindings::bindgen_test_layout__xmlXPathFunct' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _xmlXPathFunct', src/bindings.rs:17305:3
---- bindings::bindgen_test_layout_pthread_rwlockattr_t stdout ----
thread 'bindings::bindgen_test_layout_pthread_rwlockattr_t' panicked at 'assertion failed: `(left == right)`
  left: `4`,
 right: `8`: Alignment of pthread_rwlockattr_t', src/bindings.rs:12557:3
---- bindings::bindgen_test_layout_timespec stdout ----
thread 'bindings::bindgen_test_layout_timespec' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: timespec', src/bindings.rs:11606:3
---- bindings::bindgen_test_layout_random_data stdout ----
thread 'bindings::bindgen_test_layout_random_data' panicked at 'assertion failed: `(left == right)`
  left: `28`,
 right: `48`: Size of: random_data', src/bindings.rs:12693:3
---- bindings::bindgen_test_layout__xmlSchemaTypeLink stdout ----
thread 'bindings::bindgen_test_layout__xmlSchemaTypeLink' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: _xmlSchemaTypeLink', src/bindings.rs:20041:3
---- bindings::bindgen_test_layout_timeval stdout ----
thread 'bindings::bindgen_test_layout_timeval' panicked at 'assertion failed: `(left == right)`
  left: `8`,
 right: `16`: Size of: timeval', src/bindings.rs:11567:3
failures:
    bindings::bindgen_test_layout_UConverterFromUnicodeArgs
    bindings::bindgen_test_layout_UConverterToUnicodeArgs
    bindings::bindgen_test_layout__G_fpos64_t
    bindings::bindgen_test_layout__G_fpos_t
    bindings::bindgen_test_layout__IO_FILE
    bindings::bindgen_test_layout__IO_marker
    bindings::bindgen_test_layout___pthread_cond_s
    bindings::bindgen_test_layout___pthread_cond_s__bindgen_ty_1
    bindings::bindgen_test_layout___pthread_cond_s__bindgen_ty_2
    bindings::bindgen_test_layout___pthread_internal_list
    bindings::bindgen_test_layout___pthread_mutex_s
    bindings::bindgen_test_layout___pthread_rwlock_arch_t
    bindings::bindgen_test_layout___sigset_t
    bindings::bindgen_test_layout___va_list_tag
    bindings::bindgen_test_layout__htmlElemDesc
    bindings::bindgen_test_layout__htmlEntityDesc
    bindings::bindgen_test_layout__uconv_t
    bindings::bindgen_test_layout__xlinkHandler
    bindings::bindgen_test_layout__xmlAttr
    bindings::bindgen_test_layout__xmlAttribute
    bindings::bindgen_test_layout__xmlBuffer
    bindings::bindgen_test_layout__xmlChRangeGroup
    bindings::bindgen_test_layout__xmlCharEncodingHandler
    bindings::bindgen_test_layout__xmlDOMWrapCtxt
    bindings::bindgen_test_layout__xmlDoc
    bindings::bindgen_test_layout__xmlDtd
    bindings::bindgen_test_layout__xmlElement
    bindings::bindgen_test_layout__xmlElementContent
    bindings::bindgen_test_layout__xmlEntity
    bindings::bindgen_test_layout__xmlEnumeration
    bindings::bindgen_test_layout__xmlError
    bindings::bindgen_test_layout__xmlGlobalState
    bindings::bindgen_test_layout__xmlID
    bindings::bindgen_test_layout__xmlLocationSet
    bindings::bindgen_test_layout__xmlNode
    bindings::bindgen_test_layout__xmlNodeSet
    bindings::bindgen_test_layout__xmlNotation
    bindings::bindgen_test_layout__xmlNs
    bindings::bindgen_test_layout__xmlOutputBuffer
    bindings::bindgen_test_layout__xmlParserCtxt
    bindings::bindgen_test_layout__xmlParserInput
    bindings::bindgen_test_layout__xmlParserInputBuffer
    bindings::bindgen_test_layout__xmlParserNodeInfo
    bindings::bindgen_test_layout__xmlParserNodeInfoSeq
    bindings::bindgen_test_layout__xmlRef
    bindings::bindgen_test_layout__xmlSAXHandler
    bindings::bindgen_test_layout__xmlSAXHandlerV1
    bindings::bindgen_test_layout__xmlSAXLocator
    bindings::bindgen_test_layout__xmlSchema
    bindings::bindgen_test_layout__xmlSchemaAnnot
    bindings::bindgen_test_layout__xmlSchemaAttribute
    bindings::bindgen_test_layout__xmlSchemaAttributeGroup
    bindings::bindgen_test_layout__xmlSchemaAttributeLink
    bindings::bindgen_test_layout__xmlSchemaElement
    bindings::bindgen_test_layout__xmlSchemaFacet
    bindings::bindgen_test_layout__xmlSchemaFacetLink
    bindings::bindgen_test_layout__xmlSchemaNotation
    bindings::bindgen_test_layout__xmlSchemaType
    bindings::bindgen_test_layout__xmlSchemaTypeLink
    bindings::bindgen_test_layout__xmlSchemaWildcard
    bindings::bindgen_test_layout__xmlSchemaWildcardNs
    bindings::bindgen_test_layout__xmlShellCtxt
    bindings::bindgen_test_layout__xmlURI
    bindings::bindgen_test_layout__xmlValidCtxt
    bindings::bindgen_test_layout__xmlXPathAxis
    bindings::bindgen_test_layout__xmlXPathContext
    bindings::bindgen_test_layout__xmlXPathFunct
    bindings::bindgen_test_layout__xmlXPathObject
    bindings::bindgen_test_layout__xmlXPathParserContext
    bindings::bindgen_test_layout__xmlXPathType
    bindings::bindgen_test_layout__xmlXPathVariable
    bindings::bindgen_test_layout_drand48_data
    bindings::bindgen_test_layout_fd_set
    bindings::bindgen_test_layout_ldiv_t
    bindings::bindgen_test_layout_lldiv_t
    bindings::bindgen_test_layout_pthread_attr_t
    bindings::bindgen_test_layout_pthread_barrier_t
    bindings::bindgen_test_layout_pthread_cond_t
    bindings::bindgen_test_layout_pthread_mutex_t
    bindings::bindgen_test_layout_pthread_rwlock_t
    bindings::bindgen_test_layout_pthread_rwlockattr_t
    bindings::bindgen_test_layout_random_data
    bindings::bindgen_test_layout_timespec
    bindings::bindgen_test_layout_timeval
test result: FAILED. 12 passed; 84 failed; 0 ignored; 0 measured; 0 filtered out

Noticed that when runnning cargo test on i686 and armv7hl.

Basic thread safety

Follow-up to #20 which didn't get too far. It would be great if we could have Sync nodes, so that we could e.g. do a rayon parallel iteration over a node's children, speeding up dramatically various read-only traversals.

What I don't currently understand is whether we must pay the price of an Arc<Mutex<...>> wrapper, or we could satisfy the guarantees in a lighter fashion. It would be great if I could find a different Rust wrapper that is already taking care of concurrency and learn from their implementation of Send + Sync guarantees for the wrapped pointers...

My current goal is to ensure thread-safety withing a single document, which should be substantially simpler than ensuring multi-document thread-safety. And just getting the read-only guarantees would still be a decent win, since that's most of what one needs in a huge class of applications (e.g. reading in config files, parsing statistics...)

creating a text node after add_next_sibling and then also trying to add_next_sibling() it causes mutably shared node Err

This is on 0.3. So the scenario is appending a newly created text node, after you call add_next_sibling, and then call add_next_sibling with that causes a mutable reference Err. Replacing the text nodes with "real" nodes works correctly. Could be some underlying libxml muckery when merging adjacent text nodes? I somewhat remember a long while ago running into a bug in the perl libxml wrapper in that ballpark.

You could ask "why would you insert another text node rather than append_data?" which is a valid question, but this should still work, or at least give a more concise error as to what's up

the fact the first 2 add_next_sibling work fine, it is only after creating another text node after the add_next_sibling call where this falls apart.

thanks!

use libxml::parser::Parser; 
use libxml::tree::*;


fn xml_nodes_works()
{
    println!("xml_nodes_works");

    let mut d = Document::new().unwrap();

    let mut div = Node::new("DIV", None, &d).unwrap();
    d.set_root_element(&div);

    let mut n = Node::new("SPAN", None, &d).unwrap();
    let mut nt = Node::new("SPAN", None, &d).unwrap();
    let mut nt2 = Node::new("SPAN", None, &d).unwrap();

    // add text
    div.add_child(&mut n).unwrap();

    // lets skip the fact it is questionable why I'd be adding multiple text nodes

    // but is it because we mutate it twice.
    n.add_next_sibling(&mut nt).unwrap(); // ok
    n.add_next_sibling(&mut nt2).unwrap(); // ok

    println!("{}", d.node_to_string(&div)); // "n nt nt2 "

    let mut nt4 = Node::new("SPAN", None, &d).unwrap();
    n.add_next_sibling(&mut nt4).unwrap(); // works.

    println!("{}", d.node_to_string(&div)); // "n nt nt2 "
}

fn xml_text_broken()
{
    println!("xml_text_broken");

    let mut d = Document::new().unwrap();

    let mut div = Node::new("DIV", None, &d).unwrap();
    d.set_root_element(&div);

    let mut n = Node::new_text("n ", &d).unwrap();
    let mut nt = Node::new_text("nt ", &d).unwrap();
    let mut nt2 = Node::new_text("nt2 ", &d).unwrap();

    // add text
    div.add_child(&mut n).unwrap();

    // lets skip the fact it is questionable why I'd be adding multiple text nodes like this..

    n.add_next_sibling(&mut nt).unwrap(); // ok
    n.add_next_sibling(&mut nt2).unwrap(); // ok

    println!("{}", d.node_to_string(&div)); // "n nt nt2 "

    let mut nt4 = Node::new_text("nt4 ", &d).unwrap();
    n.add_next_sibling(&mut nt4).unwrap(); // panic! with reference error.

    println!("{}", d.node_to_string(&n));
}

fn main() {

    xml_nodes_works();
    xml_text_broken();
}

output:

<DIV><SPAN/><SPAN/><SPAN/></DIV>
<DIV><SPAN/><SPAN/><SPAN/><SPAN/></DIV>
xml_text_broken
<DIV>n nt nt2 </DIV>
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: "Can not mutably reference a shared Node \"text\"! Rc: weak count: 0; strong count: 3"', src/main.rs:61:34

Support Rust stable

Would be nice to use this library from Rust stable. The StaticMutex is a problem. Do you think something like this could work instead? https://github.com/ramn/rust-libxml/tree/replace_static_mutex

I can make a pull request if you think the approach is ok.

Implement correct deep copy of nodes/documents

copy should be done via xmlCopyNode/xmlCopyDocument

Ecosystem alignment and benchmarks

I have been doing some basic reading through the rust XML ecosystem and am only now realizing a crate exists that is centered around the read-only approach of the RoNode PR ( #53 ), called roxmltree. Its engine is xmlparser instead of libxml2, and it may be interesting to compare performance on their benchmarks.

Fails to build on windows

I am using libxml as a dependency for one of my projects but it fails to build on windows. Any pointers to resolve this issue?

error: failed to run custom build command for `libxml v0.2.14`

Caused by:
  process didn't exit successfully: `C:\Users\xyz\work\rust\bombardier\target\release\build\libxml-77d8a84e5c34f954\build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Could not find libxml2 using pkg-config', C:\Users\xyz\.cargo\registry\src\github.com-1ecc6299db9ec823\libxml-0.2.14\build.rs:5:5

Document to string with xmlSaveOptions

I would very much welcome some way to serialize a Document that is able to take xmlSaveOptions into account.
I hacked together a branch of rust-libxml locally that does exactly what I need for my application:

  pub fn to_string_custom(&self) -> String {
    unsafe {
      // allocate a buffer to dump into
      let buf = xmlBufferCreate();
      let c_utf8 = CString::new("UTF-8").unwrap();
      let options = xmlSaveOption_XML_SAVE_NO_EMPTY + xmlSaveOption_XML_SAVE_AS_HTML;

      let save_ctx = xmlSaveToBuffer(buf, c_utf8.as_ptr(), options as i32);
      let _size = xmlSaveDoc(save_ctx, self.doc_ptr());

      let result = xmlBufferContent(buf);
      let c_string = CStr::from_ptr(result as *const c_char);
      let node_string = c_string.to_string_lossy().into_owned();
      xmlBufferFree(buf);

      node_string
    }
  }

It would be cool to somehow generalize this and upstream it. Since there is already to_string() that uses xmlDocDumpFormatMemoryEnc I'm not sure how something that uses the xmlSaveCtxt would fit into the picture.

Is the functionality something that could find its way into the code base? And if so, how would you implement it?
I'd like to submit a PR, but think I need some guidance on how this should be implemented.

Move bindgen bindings into separate libxml-sys crate

So downstream crates that enhance the rust-libxml wrapper link against the same bindings instead of parallel ones, and eventually enable libxml2 vendoring.

Would benefit siblings like xmlsec greatly, since it allows it to link against the C library through the same bindings.

What are your thoughts about this?

XmlSec sister project

Hi,

just wanted to drop a message informing of the newly birthed xmlsec wrapper, which is bases itself partly on this one and expands it by XML signing, signode templating and verification functionalities.

Have fun!

Xpath does not query paths, a little bug in the node.get_contents()

I noticed I'm not able to query paths using xpaths from my html file, while running tests as well as from my personal module.

Node get_content() also returns all child text instead of text of the inner node.

Can you please look into it?

"pointer being freed was not allocated"

I have a test case in the drop_test branch of my github fork. The test commit is here. It's ugly. I tried to make it a more suitable testcase that you could use (smaller and use more obfuscated xml), but everything I did caused the problem to go away.

When I run cargo test on Mac OS X I get this:

tree_tests-249744ee950f6197(93787,0x700006f99000) malloc: *** error for object 0x7f8b87001236: pointer being freed was not allocated
tree_tests-249744ee950f6197(93787,0x700006f99000) malloc: *** set a breakpoint in malloc_error_break to debug

When I run on Debian, I get this:

free(): invalid pointer

I tried the breakpoint suggested in the mac error message and caught it. The backtrace is this:

* thread #2, name = 'flex_drop', stop reason = breakpoint 1.1
  * frame #0: 0x00007fff6315a0de libsystem_malloc.dylib`malloc_error_break
    frame #1: 0x00007fff6314d676 libsystem_malloc.dylib`malloc_vreport + 437
    frame #2: 0x00007fff6314d4a3 libsystem_malloc.dylib`malloc_report + 152
    frame #3: 0x00007fff62ca3792 libxml2.2.dylib`xmlFreeProp + 150
    frame #4: 0x00007fff62ce79a9 libxml2.2.dylib`xmlFreeNode + 307
    frame #5: 0x0000000100073221 tree_tests-249744ee950f6197`_$LT$libxml..tree..node.._Node$u20$as$u20$core..ops..drop..Drop$GT$::drop::hfac9065cf8822916(self=0x00000001003ef258) at node.rs:72:10
    frame #6: 0x00000001000719f5 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::h7f7f8242f4051cfa((null)=0x00000001003ef258) at ptr.rs:193
    frame #7: 0x00000001000719a5 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::h7d3944f20601a66a((null)=0x00000001003ef258) at ptr.rs:193
    frame #8: 0x0000000100071959 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::h7b25fc37c0635283((null)=0x00000001003ef250) at ptr.rs:193
    frame #9: 0x0000000100071489 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::h3273663067bf5143((null)=0x00000001003ef240) at ptr.rs:193
    frame #10: 0x000000010006abde tree_tests-249744ee950f6197`_$LT$alloc..rc..Rc$LT$T$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::h0d1452789d9da09e [inlined] core::ptr::drop_in_place::he713afdf06d1726a(to_drop=0x00000001003ef240) at ptr.rs:183:4
    frame #11: 0x000000010006abd5 tree_tests-249744ee950f6197`_$LT$alloc..rc..Rc$LT$T$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::h0d1452789d9da09e(self=0x0000000100829fb8) at rc.rs:861
    frame #12: 0x0000000100071ec5 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::hd11988989e4c8320((null)=0x0000000100829fb8) at ptr.rs:193
    frame #13: 0x00000001000712a5 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::h059cd52019b17386((null)=0x0000000100829fb8) at ptr.rs:193
    frame #14: 0x0000000100071c19 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::hb40400a6a2299f6d((null)=0x0000000100829fb0) at ptr.rs:193
    frame #15: 0x000000010006b824 tree_tests-249744ee950f6197`std::collections::hash::table::RawTable$LT$K$C$V$GT$::rev_drop_buckets::h641b04379c4b80b3 [inlined] core::ptr::drop_in_place::hb75486d4a1a356cc(to_drop=0x0000000100829fb0) at ptr.rs:183:4
    frame #16: 0x000000010006b81b tree_tests-249744ee950f6197`std::collections::hash::table::RawTable$LT$K$C$V$GT$::rev_drop_buckets::h641b04379c4b80b3(self=0x0000000100304e60) at table.rs:836
    frame #17: 0x0000000100070fd2 tree_tests-249744ee950f6197`_$LT$std..collections..hash..table..RawTable$LT$K$C$V$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::hdb3d5376ec1745e5(self=0x0000000100304e60) at table.rs:1119:16
    frame #18: 0x00000001000713a5 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::h1be0a1e9f032c3f5((null)=0x0000000100304e60) at ptr.rs:193
    frame #19: 0x00000001000712c9 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::h0a9bd580f437cb9b((null)=0x0000000100304e50) at ptr.rs:193
    frame #20: 0x00000001000720fa tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::hedb4b90123f3f789((null)=0x0000000100304e48) at ptr.rs:193
    frame #21: 0x0000000100072185 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::hee6d103f969acea1((null)=0x0000000100304e48) at ptr.rs:193
    frame #22: 0x0000000100071d79 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::hbf83bdd2e3a2d069((null)=0x0000000100304e40) at ptr.rs:193
    frame #23: 0x0000000100071b79 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::ha3bb801b7aa28ee0((null)=0x0000000100304e30) at ptr.rs:193
    frame #24: 0x000000010006ad5e tree_tests-249744ee950f6197`_$LT$alloc..rc..Rc$LT$T$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::hf02e0c3ff1eb8844 [inlined] core::ptr::drop_in_place::h405055e68682654a(to_drop=0x0000000100304e30) at ptr.rs:183:4
    frame #25: 0x000000010006ad55 tree_tests-249744ee950f6197`_$LT$alloc..rc..Rc$LT$T$GT$$u20$as$u20$core..ops..drop..Drop$GT$::drop::hf02e0c3ff1eb8844(self=0x0000700006973618) at rc.rs:861
    frame #26: 0x000000010000ffc5 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::hf075550537525141((null)=0x0000700006973618) at ptr.rs:193
    frame #27: 0x000000010000ffa5 tree_tests-249744ee950f6197`core::ptr::real_drop_in_place::he27effd3de3d6a8c((null)=0x0000700006973618) at ptr.rs:193
    frame #28: 0x00000001000017b4 tree_tests-249744ee950f6197`tree_tests::flex_drop_fn_2::h0c7bad2f17434443(glyph_string=(data_ptr = "8assertion failed: !node_mock.is_text_node()assertion failed: node_mock.to_hashable() > 0assertion failed: null_node.is_null()assertion failed: second_null_node.is_null()examplekeyexamplevalueassertion failed: hello_element.set_attribute(key, value).is_ok()assertion failed: hello_element.get_content().is_empty()hello assertion failed: hello_element.append_text(\"hello \").is_ok()world!assertion failed: hello_element.append_text(\"world!\").is_ok()hello world!http://example.com/ns/mockassertion failed: mock_ns_result.is_ok()http://example.com/ns/secondassertion failed: second_ns_result.is_ok()assertion failed: root_node.get_namespace().is_none()assertion failed: root_node.set_namespace(&mock_ns_result.unwrap()).is_ok()assertion failed: active_ns_opt.is_some()mock1http://example.com/ns/mock1mock2http://example.com/ns/mock2\bassertion failed: root_node_opt.is_some()abcdeassertion failed: root_node.add_child(&mut a).is_ok()assertion failed: root_node.add_child(&mut b).is_ok()assertion failed: root_node.add_child(&mut "..., length = 1)) at tree_tests.rs:1201
    frame #29: 0x00000001000074b8 tree_tests-249744ee950f6197`tree_tests::flex_drop::hf8f3e9ebcdaacf37 at tree_tests.rs:1206:8
    frame #30: 0x000000010000e301 tree_tests-249744ee950f6197`tree_tests::flex_drop::_$u7b$$u7b$closure$u7d$$u7d$::h331952987d4e89ce((null)=0x0000700006973870) at tree_tests.rs:1204
    frame #31: 0x000000010000f6f1 tree_tests-249744ee950f6197`core::ops::function::FnOnce::call_once::h1d4e49fd071a3f4c((null)=closure @ 0x0000700006973870, (null)=<unavailable>) at function.rs:231:4
    frame #32: 0x000000010001a1c2 tree_tests-249744ee950f6197`call_box<(),closure> [inlined] {{closure}} at lib.rs:1513:29 [opt]
    frame #33: 0x000000010001a1bd tree_tests-249744ee950f6197`call_box<(),closure> [inlined] call_once<closure,()> at function.rs:231 [opt]
    frame #34: 0x000000010001a1bd tree_tests-249744ee950f6197`call_box<(),closure> at boxed.rs:749 [opt]
    frame #35: 0x000000010008a70f tree_tests-249744ee950f6197`__rust_maybe_catch_panic at lib.rs:87:7 [opt]
    frame #36: 0x00000001000356c7 tree_tests-249744ee950f6197`{{closure}} [inlined] try<(),std::panic::AssertUnwindSafe<alloc::boxed::Box<FnBox<()>>>> at panicking.rs:272:12 [opt]
    frame #37: 0x0000000100035682 tree_tests-249744ee950f6197`{{closure}} [inlined] catch_unwind<std::panic::AssertUnwindSafe<alloc::boxed::Box<FnBox<()>>>,()> at panic.rs:388 [opt]
    frame #38: 0x0000000100035682 tree_tests-249744ee950f6197`{{closure}} at lib.rs:1468 [opt]
    frame #39: 0x0000000100011925 tree_tests-249744ee950f6197`__rust_begin_short_backtrace<closure,()> at backtrace.rs:136:4 [opt]
    frame #40: 0x0000000100015ec5 tree_tests-249744ee950f6197`do_call<std::panic::AssertUnwindSafe<closure>,()> [inlined] {{closure}}<closure,()> at mod.rs:469:16 [opt]
    frame #41: 0x0000000100015eb2 tree_tests-249744ee950f6197`do_call<std::panic::AssertUnwindSafe<closure>,()> [inlined] call_once<(),closure> at panic.rs:309 [opt]
    frame #42: 0x0000000100015eb2 tree_tests-249744ee950f6197`do_call<std::panic::AssertUnwindSafe<closure>,()> at panicking.rs:293 [opt]
    frame #43: 0x000000010008a70f tree_tests-249744ee950f6197`__rust_maybe_catch_panic at lib.rs:87:7 [opt]
    frame #44: 0x000000010001a2e5 tree_tests-249744ee950f6197`call_box<(),closure> [inlined] try<(),std::panic::AssertUnwindSafe<closure>> at panicking.rs:272:12 [opt]
    frame #45: 0x000000010001a2ac tree_tests-249744ee950f6197`call_box<(),closure> [inlined] catch_unwind<std::panic::AssertUnwindSafe<closure>,()> at panic.rs:388 [opt]
    frame #46: 0x000000010001a2ac tree_tests-249744ee950f6197`call_box<(),closure> [inlined] {{closure}}<closure,()> at mod.rs:468 [opt]
    frame #47: 0x000000010001a26e tree_tests-249744ee950f6197`call_box<(),closure> at boxed.rs:749 [opt]
    frame #48: 0x000000010008a04c tree_tests-249744ee950f6197`thread_start [inlined] call_once<(),()> at boxed.rs:759:8 [opt]
    frame #49: 0x000000010008a049 tree_tests-249744ee950f6197`thread_start [inlined] start_thread at thread.rs:14 [opt]
    frame #50: 0x0000000100089fce tree_tests-249744ee950f6197`thread_start at thread.rs:80 [opt]
    frame #51: 0x00007fff63189305 libsystem_pthread.dylib`_pthread_body + 126
    frame #52: 0x00007fff6318c26f libsystem_pthread.dylib`_pthread_start + 70
    frame #53: 0x00007fff63188415 libsystem_pthread.dylib`thread_start + 13

This makes me think it's the drop code for _Node in node.rs. I tried looking at it but as I'm not really familiar with the code, there was nothing super obvious to me.

Powerpc/ARM doesn't compile

From what I gather this is probably due to the definitions here here. Short version is the type needs to be defined depending on the target or this won't work for powerpc/ARM platforms. I'm not sure if it's possible to setup a gist for targets like this or I'd create one.

error[E0308]: mismatched types
  --> /home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libxml-0.2.13/src/schemas/parser.rs:51:60
   |
51 |     let parser = unsafe { bindings::xmlSchemaNewParserCtxt(path_ptr) };
   |                                                            ^^^^^^^^ expected `u8`, found `i8`
   |
   = note: expected raw pointer `*const u8`
              found raw pointer `*const i8`

error[E0308]: mismatched types
  --> /home/vagrant/.cargo/registry/src/github.com-1ecc6299db9ec823/libxml-0.2.13/src/schemas/validation.rs:61:66
   |
61 |     let rc = unsafe { bindings::xmlSchemaValidateFile(self.ctxt, path_ptr, 0) };
   |                                                                  ^^^^^^^^ expected `u8`, found `i8`
   |
   = note: expected raw pointer `*const u8`
              found raw pointer `*const i8`

error: aborting due to 2 previous errors

Not able to link statically

When passing -C target-feature=+crt-static to the Rust compiler for the target x86_64-unknown-linux-gnu, the following error appears at linking:

= note: /usr/bin/ld: /home/temrix/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-89bc084783fdc439.rlib(std-89bc084783fdc439.std.5f6d52e5-cgu.0.rcgu.o): in function `std::sys::unix::os::home_dir::fallback':
          /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/sys/unix/os.rs:638: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
          /usr/bin/ld: /home/temrix/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-89bc084783fdc439.rlib(std-89bc084783fdc439.std.5f6d52e5-cgu.0.rcgu.o): in function `<std::sys_common::net::LookupHost as core::convert::TryFrom<(&str,u16)>>::try_from::{{closure}}':
          /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/sys_common/net.rs:207: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
          /usr/bin/ld: /home/temrix/Dokumente/coding/my-project/target/x86_64-unknown-linux-gnu/debug/deps/libreqwest-e9d0c520f0b95258.rlib(reqwest-e9d0c520f0b95258.reqwest.32e49cb3-cgu.13.rcgu.o): undefined reference to symbol '__tls_get_addr@@GLIBC_2.3'
          /usr/bin/ld: /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2: error adding symbols: DSO missing from command line
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

It cannot link libxml. If I try without it, everything goes well and I can execute the binary.

I also tried to build it under Alpine, which resulted in a segfault when executing.

Is there anything I can do to compile with libxml statically?

Logic for drop

So I think I rather like to know how we should solve this. Now there's no drop logic for nodes at all.

Lets start with my use case, I traverse a document, find node that I want to remove and call unlink() on them. As it is today those unlinked nodes gives a memory leak.

What I would propose is to add a pub fn remove(self){} to node, that would both unlink and free.

Segmentation fault with xml serialisation

I'm tracking a bug in Hurl, our cli tool that is using libxml crate.
I think I've a very small and reproductible segmentation fault that is occurring in a multithreaded context (for instance in Rust unit test):

use libxml::tree::{Document, Node};

fn main() {
}


fn to_xml() -> String {
    let doc = Document::new().unwrap();
    let mut books = Node::new("books", None, &doc).unwrap();
    books.add_text_child(None, "book", "Dune").unwrap();
    books.add_text_child(None, "book", "Les Misérables").unwrap();
    doc.node_to_string(&books)
}

#[cfg(test)]
mod tests {
    use crate::to_xml;

    #[test]
    fn it_works_0() {
        assert_eq!(to_xml(), "<books><book>Dune</book><book>Les Misérables</book></books>");
    }

    #[test]
    fn it_works_1() {
        assert_eq!(to_xml(), "<books><book>Dune</book><book>Les Misérables</book></books>");
    }

    #[test]
    fn it_works_2() {
        assert_eq!(to_xml(), "<books><book>Dune</book><book>Les Misérables</book></books>");
    }

    #[test]
    fn it_works_3() {
        assert_eq!(to_xml(), "<books><book>Dune</book><book>Les Misérables</book></books>");
    }

}

To reproduce:

1. execute test with cargo test
2. execute the binary constructed by cargo test in a loop
3. wait

$ cargo test
    Finished test [unoptimized + debuginfo] target(s) in 0.11s
     Running unittests (target/debug/deps/testxml-18c058ecf11c8588)

running 4 tests
test tests::it_works_0 ... ok
test tests::it_works_1 ... ok
test tests::it_works_3 ... ok
test tests::it_works_2 ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
$ ./run.sh
running 4 tests
test tests::it_works_0 ... ok
test tests::it_works_2 ... ok
test tests::it_works_1 ... ok
test tests::it_works_3 ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
...
running 4 tests
./run.sh: line 7: 59200 Segmentation fault: 11  target/debug/deps/testxml-18c058ecf11c8588

with run.sh:

#!/bin/bash
set -eu

for ((i=0; i<1000; i++))
do
target/debug/deps/testxml-18c058ecf11c8588
done

My environnement:

Segmentation fault on:

• macOS 10.14.6
• Rust 1.58.1 and Rust 1.56

I'm trying to setup an environment (like a Github action) where I can easily reproduce it.

I'm really not sure if this is a bug of the crate but the code seems so innocuous that I don' t know where to look.

By the way, thank you very much for your invaluable work, we're otherwise very happy with with crate,

Jc

XPath issues when doc goes out of scope

On my way debugging and separating the issues listed in #60 I came across another thing that isn't related to async or threading.

An xpath context is non-functional after the document from which it was created goes out of scope and is dropped.
There is no warning or error. It just doesn't return any results any more.

Here is a small example. The parse_html function causes the xpath evaluation to return an empty vector, but no unwrap fails. Doing the same thing in the unit test itself (the commented out code) works as expected:

#[cfg(test)]
mod tests {
    use std::fs::File;
    use std::io::Read;
    use libxml::parser::Parser;
    use libxml::xpath::Context;
    use libxml::tree::Node;

    #[test]
    fn test1() {
        let mut file = File::open("html.txt").unwrap();
        let mut html = String::new();
        file.read_to_string(&mut html).unwrap();

        // let parser = Parser::default_html();
        // let doc = parser.parse_string(html).unwrap();
        // let ctx = Context::new(&doc).unwrap();
        
        let ctx = parse_html(&html);
        
        let res = evaluate_xpath(&ctx, "//article/header");
        assert_eq!(res.len(), 1);
    }

    fn parse_html(html: &str) -> Context {
        let parser = Parser::default_html();
        let doc = parser.parse_string(html).unwrap();
        Context::new(&doc).unwrap()
    }

    fn evaluate_xpath(ctx: &Context, xpath: &str) -> Vec<Node> {
        let res = ctx.evaluate(xpath).unwrap();
        res.get_nodes_as_vec()
    }
}

I see that the XPath context does contain a weak reference to the document to prevent exactly this issue. But from my understanding the context needs a strong reference to extend the lifetime of the document.

Weak documentation:

Since a Weak reference does not count towards ownership, it will not prevent the inner value from being dropped, and Weak itself makes no guarantees about the value still being present and may return None when upgraded.

Not sure if I understand everything correctly, but the Document implementing drop directly and freeing the doc pointer seems wrong to me. Shouldn't the Document just drop its reference to the _Document and only when all references to _Document are dropped _Document itself calls xmlFreeDoc
-> oops, misread the code. It works exactly as I just explained. Guess it's a bit confusing that Document is declared in L82 and right after that the drop trait is implemented for _Document which is declared before Document.

Cargo test locking up

I have come across an issue with cargo locking up, I expect due to some race condition? The following code exhibits the behavior on my local development machine and on our CI servers:

use libxml::parser::Parser;
use libxml::schemas::{SchemaParserContext, SchemaValidationContext};

fn main() {
    println!("Hello, world!");
}

pub fn do_xml_things(schema: &str, xml: &str) {
    let mut schema_parser = SchemaParserContext::from_buffer(schema);
    let mut xsd = SchemaValidationContext::from_parser(&mut schema_parser).unwrap();

    let doc = Parser::default().parse_string(xml).unwrap();
    xsd.validate_document(&doc).unwrap();
}

#[cfg(test)]
mod test {

    use super::*;

    const SCHEMA: &str = r#"<?xml version="1.0"?>
    <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
    
    <xs:element name="note">
      <xs:complexType>
        <xs:sequence>
          <xs:element name="to" type="xs:string"/>
          <xs:element name="from" type="xs:string"/>
          <xs:element name="heading" type="xs:string"/>
          <xs:element name="body" type="xs:string"/>
        </xs:sequence>
      </xs:complexType>
    </xs:element>
    
    </xs:schema>"#;

    #[test]
    fn test_a() {
        do_xml_things(
            SCHEMA,
            "<note><to>Anyone</to><from>Me</from><heading /><body /></note>",
        );
    }

    #[test]
    fn test_b() {
        do_xml_things(
            SCHEMA,
            "<note><to>Anyone</to><from>Me</from><heading /><body /></note>",
        );
    }

    #[test]
    fn test_c() {
        do_xml_things(
            SCHEMA,
            "<note><to>Anyone</to><from>Me</from><heading /><body /></note>",
        );
    }

    #[test]
    fn test_d() {
        do_xml_things(
            SCHEMA,
            "<note><to>Anyone</to><from>Me</from><heading /><body /></note>",
        );
    }

    #[test]
    fn test_e() {
        do_xml_things(
            SCHEMA,
            "<note><to>Anyone</to><from>Me</from><heading /><body /></note>",
        );
    }
}

Running cargo test sometimes (1 in 3 times for me) results in all the tests locking up. Running RUST_TEST_THREADS=1 cargo test removes the issue altogether.

I will implement setting the tests as a workaround, but I am not sure if this is an underlying libxml2 issue, or something specific to the crate.

Thoughts?

Implement Send for Document

Document is declared as follows:

pub(crate) struct _Document {
  /// pointer to a libxml document
  pub(crate) doc_ptr: xmlDocPtr,
  /// hashed pointer-to-Node bookkeeping table
  nodes: HashMap<xmlNodePtr, Node>,
}

Now, looking into libxml sources, neither struct _xmlDoc nor struct _xmlNode contain any thread local data. Therefore, it should be possible for the Document to have Send trait.

Help Wanted | Vaidate XPath Syntax Independent of Context

Hello,

I would like to use this library to validate that some xpaths are syntactically correct. I don't really need the context, I just need to be able to catch malformed xpaths, like "//div[text(}="Hello"]". Is this doable in libxml?

Thanks,

GPLv2 license — intentionally?

Maybe, as it is library, it will be better to use LGPL license, so non-GPL projects could use it? Now, even GPLv3 project can't use it, if it is "GPLv2 only" https://www.gnu.org/licenses/gpl-faq.en.html#AllCompatibility.

Thank you

Hey, thank you for the nice crate! So far it's the only solution I found that works well with all types of crazy HTMLs and supports XPath as expected.

I got couple times frustrated with the situation about XML parsers in Rust ecosystem, but just tried out you bindings and they work just like magic! 👍 ❤️