althonos / iocursor

A zero-copy file-like wrapper for Python byte buffers, inspired by Rust's std::io::Cursor.

io‸cursor

A zero-copy file-like wrapper for Python byte buffers, inspired by Rust's std::io::Cursor.

🗺️ Overview

iocursor.Cursor lets you wrap an allocated buffer (i.e. a Python object implementing the buffer protocol), and interfacing with it through the API of a file-like object. It shares some common points with io.BytesIO but with the following main differences:

• zero-copy VS copy: Cursor will not copy the data you give it at initialisation, while BytesIO will. This makes Cursor more efficient when you are using it for read-only operations.
• static VS growable: Cursor will only use the buffer you give it at static memory, while BytesIO will use its dedicated, growable buffer.

🔧 Installing

Install directly from PyPI, using pip:

$ pip install iocursor

Pre-built wheels are available on Linux and OSX for all supported Python3 versions. Otherwise, building from source only requires a working C compiler.

🧶 Thread-safety

iocursor.Cursor instances are not thread-safe. Using several Cursor instances with the same backend memory only for reading should be fine. Use a lock when interfacing otherwise.

💡 Examples

• Use iocursor.Cursor when you have bytes you need to pass to an interface that only accepts file-like objects. For instance, pass a PNG image decoded from base64 to PIL, without copy:

  import base64
  from iocursor import Cursor
  from PIL import Image
  
  imgdata = base64.b64decode("iVBORw0KGgoAAAANSUhEU...")
  img = Image.open(Cursor(imgdata))

• Use iocursor.Cursor when you want to use the file-like API to write to a buffer of known size. For instance, retrieve a file using the pysmb API, which only accepts file-like objects:

  from SMB.SMBConnection import SMBConnectSMBConnection
  
  smb = SMBConnection('guest', '', 'client', 'server')
  smb.connect("192.168.0.1")
  
  info = smb.getAttributes("Music", "The Clash/Rock the Casbah.mp3")
  cursor = Cursor(bytearray(shared_file.file_size))
  smb.retrieveFile("Music", "The Clash/Rock the Casbah.mp3", cursor)
  
  buffer = cursor.getvalue()

• Use iocursor.Cursor when you want to do direct I/O on a type implementing the buffer protocol. For instance, initialize a numpy array by writing bytes to it:

  import numpy
  
  array = numpy.empty(4, dtype="int16")
  cursor = Cursor(array)
  cursor.write(b"\x01\x00\x02\x00\x03\x00\x04\x00")
  print(array)  # array([1, 2, 3, 4], dtype=int16)

💭 Feedback

⚠️ Issue Tracker

Found a bug ? Have an enhancement request ? Head over to the GitHub issue tracker if you need to report or ask something. If you are filing in on a bug, please include as much information as you can about the issue, and try to recreate the same bug in a simple, easily reproducible situation.

🏗️ Contributing

Contributions are more than welcome! See CONTRIBUTING.md for more details.

⚖️ License

This library is provided under the MIT License.

iocursor's Issues

pytohon 3.11 on Windows 11 cant compile

Windows 11
python 3.11 x86
MS Visual Studio 2022

      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win32-cpython-311
      creating build\lib.win32-cpython-311\iocursor
      copying iocursor\__init__.py -> build\lib.win32-cpython-311\iocursor
      running egg_info
      writing iocursor.egg-info\PKG-INFO
      writing dependency_links to iocursor.egg-info\dependency_links.txt
      writing top-level names to iocursor.egg-info\top_level.txt
      reading manifest file 'iocursor.egg-info\SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'COPYING'
      writing manifest file 'iocursor.egg-info\SOURCES.txt'
      copying iocursor\cursor.c -> build\lib.win32-cpython-311\iocursor
      copying iocursor\cursor.h -> build\lib.win32-cpython-311\iocursor
      copying iocursor\cursor.pyi -> build\lib.win32-cpython-311\iocursor
      copying iocursor\py.typed -> build\lib.win32-cpython-311\iocursor
      running build_ext
      building 'iocursor.cursor' extension
      creating build\temp.win32-cpython-311
      creating build\temp.win32-cpython-311\Release
      creating build\temp.win32-cpython-311\Release\iocursor
      "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\bin\HostX86\x86\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -DCPYTHON=1 -ID:\Dev\PycharmProjects\gameextractor\venv\include -IC:\Pythons\Pytho
n311-32\include -IC:\Pythons\Python311-32\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\AT
LMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\
\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /Tciocursor\cursor.c /Fobuild\temp.win32-cpython-311\Release\iocursor\cursor.obj
      cursor.c
      iocursor\cursor.c(382): error C2036: 'void *': unknown size
      iocursor\cursor.c(455): error C2036: 'void *': unknown size
      iocursor\cursor.c(981): warning C4129: '%': unrecognized character escape sequence
      iocursor\cursor.c(983): warning C4129: '%': unrecognized character escape sequence
      error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.33.31629\\bin\\HostX86\\x86\\cl.exe' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for iocursor
Failed to build iocursor
ERROR: Could not build wheels for iocursor, which is required to install pyproject.toml-based projects

Tried every c++ version in Visual Studio, same resaults.