Problems appear to occur on big endian architectures about xcffib

[tych0]

On Tue, Jun 21, 2016 at 04:20:49PM -0700, Iain Learmonth wrote:

https://buildd.debian.org/status/fetch.php?pkg=xcffib&arch=mips&ver=0.4.2-1&stamp=1464555138

Interesting. It's pretty obvious why the union packing test fails, and
I'll push a fix for that today, but I'm not sure why the WM_PROTOCOLS
test fails. I'll see if I can dig up a big endian machine to test
with.

Thanks for the report!

---

You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
#77

[irl]

Any progress on this?

[tych0]

Whoops, sorry about that. I just pushed the fix for the one I understand. I haven't found a big endian machine yet, but I also only asked one person. Thanks for the ping, I'll ask around.

[tych0]

Hi, so in looking at this again, I suspect our hardcoding of alignments is probably wrong. I wonder if this patch:

diff --git a/module/__init__.py b/module/__init__.py
index 37f707f..341b06c 100644
--- a/module/__init__.py
+++ b/module/__init__.py
@@ -452,7 +452,7 @@ class List(Protobj):

     def to_atoms(self):
         """ A helper for converting a List of chars to an array of atoms """
-        return struct.unpack("=" + "I" * (len(self) // 4), b''.join(self))
+        return struct.unpack("I" * (len(self) // 4), b''.join(self))

     def buf(self):
         return b''.join(self.list)

would fix it. If it does, we should probably not hardcode alignments anyway. Unfortunately, I don't have access to a big endian machine, but if anyone does, testing that patch would be much appreciated :)

[irl]

Will hopefully manage to test this soon. Thanks for looking into this.

[irl]

I figured, as this is already broken in unstable and not migrated to testing, there isn't much to lose from just adding the patch and uploading to Debian. The buildds can do the test builds on all the architectures and report back. (:

https://buildd.debian.org/status/package.php?p=xcffib&suite=sid

The version with the patches is 0.4.2-2. If they all build nicely, I'll close this issue.

[irl]

@tych0 Almost!

https://buildd.debian.org/status/fetch.php?pkg=xcffib&arch=powerpc&ver=0.4.2-2&stamp=1474285421

I: pybuild base:184: cd /«PKGBUILDDIR»/.pybuild/pythonX.Y_2.7/build; python2.7 -m nose test
..F.....................S.........
======================================================================
FAIL: test.test_connection.TestConnection.test_ChangeProperty_WM_PROTOCOLS
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/«PKGBUILDDIR»/.pybuild/pythonX.Y_2.7/build/test/test_connection.py", line 229, in test_ChangeProperty_WM_PROTOCOLS
    assert reply.value.to_atoms() == (wm_delete_window,)
AssertionError

----------------------------------------------------------------------
Ran 34 tests in 8.281s

FAILED (SKIP=1, failures=1)

One failure left.

[irl]

Ping?

[tych0]

Thanks for the ping; I still don't have access to powerpc hardware, but I pinged someone else who might have it. In the meantime, if you know of a way I could get access to a powerpc vm for a couple of hours to debug this, that would be great :)

[irl]

Ok, I've asked the Debian PowerPC team to see if they have a box you can use.

[tych0]

Awesome, thanks! The guy I pinged will probably be around Monday, so hopefully I'll hear from him then.

[irl]

The PowerPC team suggested you could test using QEMU. I can't find a recent qcow image though, so it's not an ideal solution. Let me know if you haven't heard anything after a while and I'll look at building a QEMU box and giving you remote access.

[tych0]

So I finally got access to a machine, and I have no idea what's going on. Consider,

root@yak:~/xcffib# git diff
diff --git a/test/test_connection.py b/test/test_connection.py
index 8815cb9..130ea94 100644
--- a/test/test_connection.py
+++ b/test/test_connection.py
@@ -232,6 +232,8 @@ class TestConnection(XcffibTest):

         reply = self.xproto.GetProperty(False, wid, wm_protocols, xcffib.xproto.Atom.ATOM, 0, 1).reply()

+       print "value: %s, wm_delete_window: %s" % (reply.value.to_atoms(), (wm_delete_window,))
+
         assert reply.value.to_atoms() == (wm_delete_window,)

         wm_take_focus = self.intern("WM_TAKE_FOCUS")
root@yak:~/xcffib# make && nosetests -v -d test.test_connection:TestConnection.test_ChangeProperty_WM_PROTOCOLS
cabal --config-file=/dev/null build
Building xcffib-0.4.2...
Preprocessing library xcffib-0.4.2...
In-place registering xcffib-0.4.2...
Preprocessing executable 'xcffibgen' for xcffib-0.4.2...
Preprocessing test suite 'PyHelpersTests' for xcffib-0.4.2...
Preprocessing test suite 'GeneratorTests.hs' for xcffib-0.4.2...
./dist/build/xcffibgen/xcffibgen --input /usr/share/xcb --output ./xcffib
xf86vidmode
xc_misc
xproto
glx
dpms
render
dri2
shm
ge
xtest
shape
xvmc
damage
xselinux
xinerama
dri3
xfixes
present
xf86dri
bigreq
sync
randr
xv
xinput
record
res
composite
xevie
screensaver
cp ./module/*py ./xcffib/
sed -i "s/__xcb_proto_version__ = .*/__xcb_proto_version__ = \"1.11\"/" xcffib/__init__.py
test.test_connection.TestConnection.test_ChangeProperty_WM_PROTOCOLS ... FAIL

======================================================================
FAIL: test.test_connection.TestConnection.test_ChangeProperty_WM_PROTOCOLS
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/xcffib/test/test_connection.py", line 237, in test_ChangeProperty_WM_PROTOCOLS
    assert reply.value.to_atoms() == (wm_delete_window,)
AssertionError: 
-------------------- >> begin captured stdout << ---------------------
value: (4009754624,), wm_delete_window: (239,)

--------------------- >> end captured stdout << ----------------------
>>  assert <xcffib.xproto.GetPropertyReply object at 0x3ffb16beb38>.value.to_atoms() == (239,)


----------------------------------------------------------------------
Ran 1 test in 0.210s

FAILED (failures=1)

Ok, looks like an endian-ness bug. So if I make to_atoms() always return little endian (e.g. because maybe the X protocol always uses little-endian or something),

root@yak:~/xcffib# git diff
diff --git a/module/__init__.py b/module/__init__.py
index 37f707f..3bfd751 100644
--- a/module/__init__.py
+++ b/module/__init__.py
@@ -452,7 +452,7 @@ class List(Protobj):

     def to_atoms(self):
         """ A helper for converting a List of chars to an array of atoms """
-        return struct.unpack("=" + "I" * (len(self) // 4), b''.join(self))
+        return struct.unpack("<" + "I" * (len(self) // 4), b''.join(self))

     def buf(self):
         return b''.join(self.list)
diff --git a/test/test_connection.py b/test/test_connection.py
index 8815cb9..71a6ad0 100644
--- a/test/test_connection.py
+++ b/test/test_connection.py
@@ -232,6 +232,8 @@ class TestConnection(XcffibTest):

         reply = self.xproto.GetProperty(False, wid, wm_protocols, xcffib.xproto.Atom.ATOM, 0, 1).reply()

+       print "value: %s, wm_delete_window: %s" % (reply.value.to_atoms(), (wm_delete_window,))
+
         assert reply.value.to_atoms() == (wm_delete_window,)

         wm_take_focus = self.intern("WM_TAKE_FOCUS")
@@ -242,6 +244,7 @@ class TestConnection(XcffibTest):

         reply = self.xproto.GetProperty(False, wid, wm_protocols, xcffib.xproto.Atom.ATOM, 0, 1).reply()

+       print "value: %s, wm_take_focus : %s" % (reply.value.to_atoms(), (wm_take_focus,))
         assert reply.value.to_atoms() == (wm_take_focus,)

     def test_GetAtomName(self):
root@yak:~/xcffib# make && nosetests -v -d test.test_connection:TestConnection.test_ChangeProperty_WM_PROTOCOLS
cabal --config-file=/dev/null build
Building xcffib-0.4.2...
Preprocessing library xcffib-0.4.2...
In-place registering xcffib-0.4.2...
Preprocessing executable 'xcffibgen' for xcffib-0.4.2...
Preprocessing test suite 'PyHelpersTests' for xcffib-0.4.2...
Preprocessing test suite 'GeneratorTests.hs' for xcffib-0.4.2...
./dist/build/xcffibgen/xcffibgen --input /usr/share/xcb --output ./xcffib
xf86vidmode
xc_misc
xproto
glx
dpms
render
dri2
shm
ge
xtest
shape
xvmc
damage
xselinux
xinerama
dri3
xfixes
present
xf86dri
bigreq
sync
randr
xv
xinput
record
res
composite
xevie
screensaver
cp ./module/*py ./xcffib/
sed -i "s/__xcb_proto_version__ = .*/__xcb_proto_version__ = \"1.11\"/" xcffib/__init__.py
test.test_connection.TestConnection.test_ChangeProperty_WM_PROTOCOLS ... FAIL

======================================================================
FAIL: test.test_connection.TestConnection.test_ChangeProperty_WM_PROTOCOLS
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/root/xcffib/test/test_connection.py", line 248, in test_ChangeProperty_WM_PROTOCOLS
    assert reply.value.to_atoms() == (wm_take_focus,)
AssertionError: 
-------------------- >> begin captured stdout << ---------------------
value: (239,), wm_delete_window: (239,)
value: (4026531840,), wm_take_focus : (240,)

--------------------- >> end captured stdout << ----------------------
>>  assert <xcffib.xproto.GetPropertyReply object at 0x3ffbc2bde48>.value.to_atoms() == (240,)


----------------------------------------------------------------------
Ran 1 test in 0.210s

FAILED (failures=1)

wm_take_focus fails, because it is big endian. If I switch it and comment out the first assert, then it works:

root@yak:~/xcffib# git diff
diff --git a/module/__init__.py b/module/__init__.py
index 37f707f..ffc4bcf 100644
--- a/module/__init__.py
+++ b/module/__init__.py
@@ -452,7 +452,7 @@ class List(Protobj):

     def to_atoms(self):
         """ A helper for converting a List of chars to an array of atoms """
-        return struct.unpack("=" + "I" * (len(self) // 4), b''.join(self))
+        return struct.unpack(">" + "I" * (len(self) // 4), b''.join(self))

     def buf(self):
         return b''.join(self.list)
diff --git a/test/test_connection.py b/test/test_connection.py
index 8815cb9..a54497e 100644
--- a/test/test_connection.py
+++ b/test/test_connection.py
@@ -232,7 +232,9 @@ class TestConnection(XcffibTest):

         reply = self.xproto.GetProperty(False, wid, wm_protocols, xcffib.xproto.Atom.ATOM, 0, 1).reply()

-        assert reply.value.to_atoms() == (wm_delete_window,)
+       print "value: %s, wm_delete_window: %s" % (reply.value.to_atoms(), (wm_delete_window,))
+
+        # assert reply.value.to_atoms() == (wm_delete_window,)

         wm_take_focus = self.intern("WM_TAKE_FOCUS")

@@ -242,6 +244,7 @@ class TestConnection(XcffibTest):

         reply = self.xproto.GetProperty(False, wid, wm_protocols, xcffib.xproto.Atom.ATOM, 0, 1).reply()

+       print "value: %s, wm_take_focus : %s" % (reply.value.to_atoms(), (wm_take_focus,))
         assert reply.value.to_atoms() == (wm_take_focus,)

     def test_GetAtomName(self):
root@yak:~/xcffib# make && nosetests -v -d test.test_connection:TestConnection.test_ChangeProperty_WM_PROTOCOLS
cabal --config-file=/dev/null build
Building xcffib-0.4.2...
Preprocessing library xcffib-0.4.2...
In-place registering xcffib-0.4.2...
Preprocessing executable 'xcffibgen' for xcffib-0.4.2...
Preprocessing test suite 'PyHelpersTests' for xcffib-0.4.2...
Preprocessing test suite 'GeneratorTests.hs' for xcffib-0.4.2...
./dist/build/xcffibgen/xcffibgen --input /usr/share/xcb --output ./xcffib
xf86vidmode
xc_misc
xproto
glx
dpms
render
dri2
shm
ge
xtest
shape
xvmc
damage
xselinux
xinerama
dri3
xfixes
present
xf86dri
bigreq
sync
randr
xv
xinput
record
res
composite
xevie
screensaver
cp ./module/*py ./xcffib/
sed -i "s/__xcb_proto_version__ = .*/__xcb_proto_version__ = \"1.11\"/" xcffib/__init__.py
test.test_connection.TestConnection.test_ChangeProperty_WM_PROTOCOLS ... ok

----------------------------------------------------------------------
Ran 1 test in 0.204s

OK

So. I am tempted to just skip this test on big endian arches for now, since it seems like X is confused in this case. I'll keep digging, but if I can't figure out anything else, that may be what I do.

[tych0]

Oh, never mind. I think I figured it out. I'll push a fix shortly.

[tych0]

Ok, I think that should fix it everywhere. Let me know if you run into more troubles. Thanks for filing the bug, and sorry it took so long to fix!