zerointensity / pointers.py

Bringing the hell of pointers to Python.

pointers.py

Bringing the hell of pointers to Python

Why would you ever need this

• Documentation
• Source
• PyPI

Examples

from pointers import _

text: str = "hello world"
ptr = _&text  # creates a new pointer object
ptr <<= "world hello"
print(text)  # world hello

from pointers import c_malloc, c_free, strcpy, printf

ptr = c_malloc(3)
strcpy(ptr, "hi")
printf("%s\n", ptr)  # hi
c_free(ptr)

from pointers import malloc, free

my_str = malloc(103)
my_str <<= "hi"
second_str = my_str[51]
second_str <<= "bye"
print(*my_str, *second_str)  # hi bye
free(my_str)

Features

• Fully type safe
• Pythonic pointer API
• Bindings for the entire C standard library and CPython ABI
• Segfaults

Why does this exist?

The main purpose of pointers.py is to simply break the rules of Python, but has some other use cases:

• Can help C/C++ developers get adjusted to Python
• Provides a nice learning environment for programmers learning how pointers work
• Makes it very easy to manipulate memory in Python
• Why not?

Installation

Linux/macOS

python3 -m pip install -U pointers.py

Windows

py -3 -m pip install -U pointers.py

pointers.py's Issues

Use a unary operator for dereferencing

Trying to use * for dereferencing doesn't work very well, because it isn't a unary operator. This means it only works in some contexts. Also it is "stealing" the iterability of the referenced object.

For example

xs = [1, 2, 3]
p_xs = to_ptr(xs)
print(**p_xs, sep="\n")  # fails, splat + deref is treated as kwarg splat

A better deref operator would be unary +:

xs = [1, 2, 3]
p_xs = to_ptr(xs)
print(*+p_xs, sep="\n")

+ looks a bit like *, it just has one fewer spoke.

Segfault when using fopen on /dev/null in write mode

Modernize the repository

right now its using outdated build systems and every is generally pretty messy. hopefully this isnt too much of a headache to change

Broken calloc implementation

it seems my initial guess of why calloc caused so many segfaults was wrong.

example:

from pointers import calloc, free

mem = calloc(200, 28)
mem += 200
mem <<= 1
mem -= 200
print(~mem)  # works just fine!

free(mem)

mem_2 = calloc(4, 28)
mem_2 += 1
mem_2 <<= 1
mem_2 -= 1
print(*mem_2)  # segfault

it seems that i need to skip 24 digits when using pointer arithmetic, not 1.

mem = calloc(200, 28)
mem += 24
mem <<= 1
mem -= 24
print(~mem)  # works correctly, no segfault

another example:

mem = calloc(200, 28)
mem += 23
mem <<= 1
mem -= 23
print(~mem)  # OverflowError

Provide bindings for most of the C standard lib

working with ctypes isn't exactly nice, so this library could provide a nice alternative. this will take a while.

Optimize struct implementation

Struct currently repeats dataclass logic every time it instantiates. instead, it should only do it once and store it in a class variable.

validation is also only executed when the class is instantiated, instead of when its created. most of this fix will just be moving things over to __init_subclass__ instead of __init__

Uses Cython implementation details

The way this library gets the memory addresses (the id() builtin) isn't valid in all versions of python. It's a Cython implementation detail that it actually points towards the memory address - the only guarantee across implementations is that the value is unique.

For reference: https://docs.python.org/3/library/functions.html#id

Pointer arithmetic not working correctly

pointer arithmetic edits the instance, instead of returning a new one

from pointers import calloc

memory = calloc(4, 28)
memory + 1 # this reassigns it as well 
memory += 1 # equivalent to above

ctypes, cffi provides just the same, but much more?

I'm just wondering whether there is anything here which is not in ctypes or cffi.

The documentation does not mention this at all. Maybe this can be added.

Why

Would you do this

3.0.0

its time this library gets another rewrite.

this should include the following:

• all existing features
• patches for all ffi problems
• fixed ci
• better unit testing

Error in subprocess while pip install

While installing the package in mac M2, I found an error. The steps to recreate are the following:
python:

python3 -m venv myvenv
source myvenv/bin/activate
python3 -m pip install -U pointers.py

The output of the instalation is the following:

python3 -m pip install -U pointers.py
Collecting pointers.py
Using cached pointers.py-2.0.0.tar.gz (21 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [57 lines of output]
/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py:76: _MissingDynamic: license defined outside of pyproject.toml is ignored.
!!

          ********************************************************************************
          The following seems to be defined outside of `pyproject.toml`:
  
          `license = 'MIT'`
  
          According to the spec (see the link below), however, setuptools CANNOT
          consider this value unless `license` is listed as `dynamic`.
  
          https://packaging.python.org/en/latest/specifications/pyproject-toml/#declaring-project-metadata-the-project-table
  
          To prevent this problem, you can list `license` under `dynamic` or alternatively
          remove the `[project]` table from your file and rely entirely on other means of
          configuration.
          ********************************************************************************
  
  !!
    _handle_missing_dynamic(dist, project_table)
  /private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py:83: SetuptoolsWarning: `install_requires` overwritten in `pyproject.toml` (dependencies)
    corresp(dist, value, root_dir)
  Traceback (most recent call last):
    File "/Users/anibalfermandois/Desktop/Tech_proyects/python/learning_venv/myvenv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/Users/anibalfermandois/Desktop/Tech_proyects/python/learning_venv/myvenv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/anibalfermandois/Desktop/Tech_proyects/python/learning_venv/myvenv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
      self.run_setup()
    File "/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 7, in <module>
    File "/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 104, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 159, in setup
      dist.parse_config_files()
    File "/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 631, in parse_config_files
      pyprojecttoml.apply_configuration(self, filename, ignore_option_errors)
    File "/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/config/pyprojecttoml.py", line 69, in apply_configuration
      return _apply(dist, config, filepath)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py", line 57, in apply
      _apply_project_table(dist, config, root_dir)
    File "/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py", line 83, in _apply_project_table
      corresp(dist, value, root_dir)
    File "/private/var/folders/st/h3c3v7wj6wsbzw4c3gd11s380000gn/T/pip-build-env-c341qnvc/overlay/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py", line 184, in _license
      _set_config(dist, "license", val["text"])
                                   ~~~^^^^^^^^
  KeyError: 'text'
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Fix random segfaults when moving

pretty sure these are caused by moving items of the different size, will add a validator for this later

don't do this 😳️

don't do this

import ctypes
ctypes.string_at(0)

i warned u

it just causes a segfault dw, you can also do it by typing ctypes.cast(1, ctypes.py_object)

Add support for functions with C bindings

might be able to let ctypes do the heavy lifting here with CFUNCTYPE

Feature Request: malloc

Why not?

Tricky question

Will this work on PyPy? ;)

Enhancement

Make this faster bozo 🤬🤬🤬