serpentai / d3dshot

Extremely fast and robust screen capture on Windows with the Desktop Duplication API

Serpent.AI - Game Agent Framework (Python)

Update: Revival (May 2020)

Development work has resumed on the framework with the aim of bringing it into 2020: Python 3.8+, Less Dependencies, Ease of Use (Installer, GUI) and much more! Still open-source with a permissive license and looking into a Steam distribution for non-technical users. 🐍

Warning: End of life (November 2018)

Serpent.AI is a simple yet powerful, novel framework to assist developers in the creation of game agents. Turn ANY video game you own into a sandbox environment ripe for experimentation, all with familiar Python code. The framework's raison d'être is first and foremost to provide a valuable tool for Machine Learning & AI research. It also turns out to be ridiculously fun to use as a hobbyist (and dangerously addictive; a fair warning)!

The framework features a large assortment of supporting modules that provide solutions to commonly encountered scenarios when using video games as environments as well as CLI tools to accelerate development. It provides some useful conventions but is absolutely NOT opiniated about what you put in your agents: Want to use the latest, cutting-edge deep reinforcement learning algorithm? ALLOWED. Want to use computer vision techniques, image processing and trigonometry? ALLOWED. Want to randomly press the Left or Right buttons? sigh ALLOWED. To top it all off, Serpent.AI was designed to be entirely plugin-based (for both game support and game agents) so your experiments are actually portable and distributable to your peers and random strangers on the Internet.

Serpent.AI supports Linux, Windows & macOS.

The next version of the framework will officially stop supporting macOS. Apple's aversion to Nvidia in their products means no recent macOS machine can run CUDA, an essential piece of technology for Serpent.AI's real-time training. Other decisions like preventing 32-bit applications from running in Catalina and deprecating OpenGL do not help make a case to support the OS.

Experiment: Game agent learning to defeat Monstro (The Binding of Isaac: Afterbirth+)

Background

The project was born out of admiration for / frustration with OpenAI Universe. The idea is perfect, let's be honest, but some implementation details leave a lot to be desired. From these, the core tennets of the framework were established:

1. Thou shall run natively. Thou shalt not use Docker containers or VNC servers.
2. Thou shall allow a user to bring their own games. Thou shalt not wait for licensing deals and special game APIs.
3. Thou shall encourage diverse and creative approaches. Thou shalt not only enable AI flavors of the month.

Want to know more about how Serpent.AI came to be? Read The Story Behind Serpent.AI on the blog!

Documentation

Guides, tutorials and videos are being produced and added to the GitHub Wiki. It currently is the official source of documentation.

Experiment: Game agent learning to match tiles (You Must Build a Boat)

Business Contact: [email protected]

d3dshot's Issues

Feature to wait for any change

I needed to detect if a specific region has changed. So I've built a loop waiting for any changes in a specific region, but it used too much CPU. I've found out my program was running very fast, around 180 fps.

If there was a method to wait for any potention change in the region, it would help to reduce CPU usage drastically. I think it's possible to do that cheaply using IDXGIOutputDuplication::GetFrameDirtyRects(). (Or simply waiting for AcquireNextFrame to return would also work, although it would not benefit from specifying a small region.)

API proposal
Signature: D3DShot.wait_for_change(self, timeout=None)
Returns: True if a potantial change was detected, False if timeout was reached
Precondition: A high-speed screen capture was started. (i.e. capture() was called)
Description: Blocks until a potential change in the region is detected. Blocks at most timeout seconds. (None means forever.)
Note: The system may collapse several dirty rects into one giant rect, so spurious awakenings might happen. It means that if you need to be certain that the region has changed at all, you need to check it yourself.

Capture window

Hi there.

First of all thanks for library.
This is more of a feature request - I didn't find it - but it would be nice to be able capture just the window of any opened/selected program.

Investigate capturing in a separate process and exposing the frame buffer with Python 3.8's shared memory capabilities

Probably better to escape the GIL at the library level rather than placing the burden on the user.

This wasn't initially considered because the performance loss of IPC with serialization/deserialization steps would have been too important.

With Python 3.8's shared memory capabilities, it might be possible to copy the frame to the main thread with very little overhead and therefore be able to process it without the overhead having to run on the same CPU core.

Slow performance

I just tried out D3DShot on my PC and the performance is pretty slow. My native display resolution is 3840x2160, and the graphics card is Nvidia GTX 1080, the average fps I'm getting is about 20. I tried benchmark with both the included benchmark method, and by counting the total frames added. Both yield similarly slow performance. Below is one of the many benchmark runs:

Capture Output: NumpyCaptureOutput
Display: <Display name=Generic PnP Monitor adapter=NVIDIA GeForce GTX 1080 resolution=3840x2160 rotation=0 scale_factor=2.5 primary=True>

Capturing as many frames as possible in the next 60 seconds... Go!
Done! Results: 20.4833 FPS

I looked at other issues with D3DShots and tried out some ways to improve this, for example, I tried running a Windowed or Fullscreen game when benchmarking, I also tried just staying in normal desktop environment, both yield similar results.

My graphics card driver is up-to-date (GeForce 460.89, 12/15/2020), my Windows 10 version is 2004, build 21277.1000.

To clarify, everything else graphics related is normal, games would run at normal FPS (60+), and nothing lags or is slow.

Fill in the AMD Catalyst Control Center section in Installation Note: Laptops section of the wiki

I don't have an hybrid GPU laptop with an AMD dedicated GPU so I can't properly document the steps to force Python to use the onboard GPU.

If anyone can document the steps here (including screenshots, if possible), I will append the information to the wiki article.

Thanks!

get_latest_frame() returns None

I think d.get_latest_frame() doesn't waits for the frame buffer to populate after d.capture() is called.

While manually waiting for that resolves the issue:

Just curious if the checks have intentionally been ignored considering performance....

Screenshot from wrong display is sometimes returned

When capturing screenshots from multiple displays sometimes screenshot for wrong display is returned. The solution is to self.previous_screenshot in d3dshot.py be a dict with key of display or to move previous_screenshot to Display class.

Getting garbled image on screenshot

Hi,

I´ve been trying to use D3DShot but it´s been returning a garbled image, like this:

I´m thinking this has something to do with drivers, any thoughts?

It randomly raises an indexerror.

The printed 0 value works but when the value changes from 0 to 51, it raises an annoying indexerror.

Error: Access violation

Hi,

First off, thanks for making this brilliant library =)
I was unable to get it to work properly on my machine however, I'm getting a strange memory error when I run this simple code:

import d3dshot

d = d3dshot.create()
d.screenshot()

Traceback (most recent call last):
  File "C:/Users/coolq/OneDrive/Projects/Python/main.py", line 4, in <module>
    d = d3dshot.create()
  File "C:\Users\coolq\OneDrive\Projects\Python\venv\lib\site-packages\d3dshot\__init__.py", line 70, in create
    pytorch_gpu_is_available=pytorch_gpu_is_available
  File "C:\Users\coolq\OneDrive\Projects\Python\venv\lib\site-packages\d3dshot\d3dshot.py", line 23, in __init__
    self.detect_displays()
  File "C:\Users\coolq\OneDrive\Projects\Python\venv\lib\site-packages\d3dshot\d3dshot.py", line 192, in detect_displays
    self.displays = Display.discover_displays()
  File "C:\Users\coolq\OneDrive\Projects\Python\venv\lib\site-packages\d3dshot\display.py", line 85, in discover_displays
    dxgi_factory = d3dshot.dll.dxgi.initialize_dxgi_factory()
  File "C:\Users\coolq\OneDrive\Projects\Python\venv\lib\site-packages\d3dshot\dll\dxgi.py", line 189, in initialize_dxgi_factory
    create_factory_func(IDXGIFactory1._iid_, ctypes.byref(handle))
OSError: exception: access violation writing 0x4DBAF26F

I'm running Windows 10 Pro 64-bit, I have two screens driven by a GTX 1060.

I'm running this with Python 3.6.

Any ideas? 😮

The specified device interface or feature level is not supported on this system

Hi
I encounter a problem in my W10 64bit laptop P51s, it's two video cards, nvidia and integrated intel.
I've setup nvdia as primary use and Dxdiag shows that Directx 12 is running normally.
Here is my code:

import time
import numpy as np
import d3dshot
p1=time.time()
d = d3dshot.create(capture_output="numpy")
img2=d.screenshot()
p2=time.time()
print ("it takes "+str(p2-p1)+ " seconds to capture in D3D.")

Traceback (most recent call last):
File "C:/Users/jianghu/PycharmProjects/diaexplo/mess_d3d.py", line 14, in
d = d3dshot.create()
File "C:\Python38\lib\site-packages\d3dshot_init_.py", line 64, in create
d3dshot = D3DShot(
File "C:\Python38\lib\site-packages\d3dshot\d3dshot.py", line 24, in init
self.detect_displays()
File "C:\Python38\lib\site-packages\d3dshot\d3dshot.py", line 194, in detect_displays
self.displays = Display.discover_displays()
File "C:\Python38\lib\site-packages\d3dshot\display.py", line 109, in discover_displays
display = cls(
File "C:\Python38\lib\site-packages\d3dshot\display.py", line 40, in init
self.dxgi_output_duplication = self._initialize_dxgi_output_duplication()
File "C:\Python38\lib\site-packages\d3dshot\display.py", line 66, in _initialize_dxgi_output_duplication
return d3dshot.dll.dxgi.initialize_dxgi_output_duplication(self.dxgi_output, self.d3d_device)
File "C:\Python38\lib\site-packages\d3dshot\dll\dxgi.py", line 257, in initialize_dxgi_output_duplication
dxgi_output.DuplicateOutput(d3d_device, ctypes.byref(dxgi_output_duplication))
_ctypes.COMError: (-2005270524, 'The specified device interface or feature level is not supported on this system.', (None, None, None, 0, None))

Process finished with exit code 1

Could you suggest what I should do to fix the issue?

pip install d3dshot isn't working

I follow https://pypi.org/project/d3dshot/

My python version is 3.10

I install Pillow package using pip install Pillow
then I run pip install d3dshot on Windows PowerShell and get this error :

PS C:\Users\Yanothai Chaitawat> pip install d3dshot
Collecting d3dshot
  Using cached D3DShot-0.1.5-py3-none-any.whl (24 kB)
Collecting comtypes<1.2.0,>=1.1.7
  Using cached comtypes-1.1.11-py2.py3-none-any.whl (167 kB)
Collecting pillow<7.2.0,>=7.1.2
  Using cached Pillow-7.1.2.tar.gz (38.9 MB)
  Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for pillow, since package 'wheel' is not installed.
Installing collected packages: pillow, comtypes, d3dshot
  Attempting uninstall: pillow
    Found existing installation: Pillow 9.1.0
    Uninstalling Pillow-9.1.0:
      Successfully uninstalled Pillow-9.1.0
    Running setup.py install for pillow ... error
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Yanothai Chaitawat\\AppData\\Local\\Temp\\pip-install-8ciz7jnr\\pillow_a2725d3477104ef19c3fdc565e33e203\\setup.py'"'"'; __file__='"'"'C:\\Users\\Yanothai Chaitawat\\AppData\\Local\\Temp\\pip-install-8ciz7jnr\\pillow_a2725d3477104ef19c3fdc565e33e203\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\Yanothai Chaitawat\AppData\Local\Temp\pip-record-_hls1_ye\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\Include\pillow'
         cwd: C:\Users\Yanothai Chaitawat\AppData\Local\Temp\pip-install-8ciz7jnr\pillow_a2725d3477104ef19c3fdc565e33e203\
    Complete output (175 lines):
    C:\Users\Yanothai Chaitawat\AppData\Local\Temp\pip-install-8ciz7jnr\pillow_a2725d3477104ef19c3fdc565e33e203\setup.py:42: RuntimeWarning: Pillow 7.1.2 does not support Python 3.10 and does not provide prebuilt Windows binaries. We do not recommend building from source on Windows.
      warnings.warn(
    running install
    C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.10
    creating build\lib.win-amd64-3.10\PIL
    copying src\PIL\BdfFontFile.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\BlpImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\BmpImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\BufrStubImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ContainerIO.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\CurImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\DcxImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\DdsImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\EpsImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ExifTags.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\features.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\FitsStubImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\FliImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\FontFile.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\FpxImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\FtexImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\GbrImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\GdImageFile.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\GifImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\GimpGradientFile.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\GimpPaletteFile.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\GribStubImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\Hdf5StubImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\IcnsImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\IcoImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\Image.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageChops.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageCms.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageColor.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageDraw.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageDraw2.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageEnhance.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageFile.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageFilter.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageFont.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageGrab.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageMath.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageMode.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageMorph.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageOps.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImagePalette.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImagePath.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageQt.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageSequence.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageShow.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageStat.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageTk.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageTransform.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImageWin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\ImtImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\IptcImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\Jpeg2KImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\JpegImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\JpegPresets.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\McIdasImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\MicImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\MpegImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\MpoImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\MspImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PaletteFile.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PalmImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PcdImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PcfFontFile.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PcxImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PdfImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PdfParser.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PixarImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PngImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PpmImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PsdImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PSDraw.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\PyAccess.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\SgiImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\SpiderImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\SunImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\TarIO.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\TgaImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\TiffImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\TiffTags.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\WalImageFile.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\WebPImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\WmfImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\XbmImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\XpmImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\XVThumbImagePlugin.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\_binary.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\_tkinter_finder.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\_util.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\_version.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\__init__.py -> build\lib.win-amd64-3.10\PIL
    copying src\PIL\__main__.py -> build\lib.win-amd64-3.10\PIL
    running egg_info
    warning: no files found matching '*.c'
    warning: no files found matching '*.h'
    warning: no files found matching '*.sh'
    warning: no previously-included files found matching '.appveyor.yml'
    warning: no previously-included files found matching '.coveragerc'
    warning: no previously-included files found matching '.editorconfig'
    warning: no previously-included files found matching '.readthedocs.yml'
    warning: no previously-included files found matching 'azure-pipelines.yml'
    warning: no previously-included files found matching 'codecov.yml'
    warning: no previously-included files matching '.git*' found anywhere in distribution
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no previously-included files matching '*.so' found anywhere in distribution
    no previously-included directories found matching '.azure-pipelines'
    no previously-included directories found matching '.ci'
    writing manifest file 'src\Pillow.egg-info\SOURCES.txt'
    running build_ext


    The headers or library files could not be found for zlib,
    a required dependency when compiling Pillow from source.

    Please see the install instructions at:
       https://pillow.readthedocs.io/en/latest/installation.html

    Traceback (most recent call last):
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Temp\pip-install-8ciz7jnr\pillow_a2725d3477104ef19c3fdc565e33e203\setup.py", line 860, in <module>
        setup(
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\__init__.py", line 155, in setup
        return distutils.core.setup(**attrs)
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\core.py", line 148, in setup
        return run_commands(dist)
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\core.py", line 163, in run_commands
        dist.run_commands()
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\dist.py", line 967, in run_commands
        self.run_command(cmd)
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
        cmd_obj.run()
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\command\install.py", line 68, in run
        return orig.install.run(self)
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\command\install.py", line 662, in run
        self.run_command('build')
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
        cmd_obj.run()
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\command\build.py", line 135, in run
        self.run_command(cmd_name)
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
        cmd_obj.run()
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\_distutils\command\build_ext.py", line 339, in run
        self.build_extensions()
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Temp\pip-install-8ciz7jnr\pillow_a2725d3477104ef19c3fdc565e33e203\setup.py", line 694, in build_extensions
        raise RequiredDependencyException(f)
    __main__.RequiredDependencyException: zlib

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\Yanothai Chaitawat\AppData\Local\Temp\pip-install-8ciz7jnr\pillow_a2725d3477104ef19c3fdc565e33e203\setup.py", line 914, in <module>
        raise RequiredDependencyException(msg)
    __main__.RequiredDependencyException:

    The headers or library files could not be found for zlib,
    a required dependency when compiling Pillow from source.

    Please see the install instructions at:
       https://pillow.readthedocs.io/en/latest/installation.html


    ----------------------------------------
  Rolling back uninstall of Pillow
  Moving to c:\users\yanothai chaitawat\appdata\local\programs\python\python310\lib\site-packages\pil\
   from C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\Lib\site-packages\~il
  Moving to c:\users\yanothai chaitawat\appdata\local\programs\python\python310\lib\site-packages\pillow-9.1.0.dist-info\
   from C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\Lib\site-packages\~illow-9.1.0.dist-info
ERROR: Command errored out with exit status 1: 'C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Yanothai Chaitawat\\AppData\\Local\\Temp\\pip-install-8ciz7jnr\\pillow_a2725d3477104ef19c3fdc565e33e203\\setup.py'"'"'; __file__='"'"'C:\\Users\\Yanothai Chaitawat\\AppData\\Local\\Temp\\pip-install-8ciz7jnr\\pillow_a2725d3477104ef19c3fdc565e33e203\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\Yanothai Chaitawat\AppData\Local\Temp\pip-record-_hls1_ye\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\Include\pillow' Check the logs for full command output.
WARNING: You are using pip version 21.3.1; however, version 22.0.4 is available.
You should consider upgrading via the 'C:\Users\Yanothai Chaitawat\AppData\Local\Programs\Python\Python310\python.exe -m pip install --upgrade pip' command.

And during d3dshot even tried to uninstall the Pillow package.
I don't see why this module should do that.

Do you have plan to add audio capture?

Thank you very much for the great work on this python lib. Could I suggest that it'll be even greater if you can add audio capture too? This can make it richer in feature and more useful.

Thanks,
Quan

1,000,000+ fps in screen capture

How can I start capturing on the same thread as my code?
Because getting the last frame is not an option, since I cannot find out when it changed.

example:

import d3dshot
from time import sleep, time
d = d3dshot.create(capture_output="numpy", frame_buffer_size=90)
d.capture(region=(653, 354, 713, 404))
sleep(1)
fps = 0
display_time = 1
start_time = 0
while True:
    test = d.get_latest_frame()
    fps+=1
    TIME = time() - start_time
    if (TIME) >= display_time :
        print("FPS: ", int(fps / (TIME)))
        start_time = time()
        fps = 0

Output:

FPS:  1562927
FPS:  1544829
FPS:  1702034

wtf? 1000000 fps?

D3Dshot on linux platform

As I understand this library is only for Windows,

Will this be made for linux any time soon?

Receiving following error:

import d3dshot

File "/usr/local/lib/python3.7/dist-packages/d3dshot/init.py", line 4, in
from d3dshot.d3dshot import D3DShot
File "/usr/local/lib/python3.7/dist-packages/d3dshot/d3dshot.py", line 8, in
from d3dshot.display import Display
File "/usr/local/lib/python3.7/dist-packages/d3dshot/display.py", line 1, in
import d3dshot.dll.dxgi
File "/usr/local/lib/python3.7/dist-packages/d3dshot/dll/dxgi.py", line 2, in
import ctypes.wintypes as wintypes
File "/usr/lib/python3.7/ctypes/wintypes.py", line 20, in
class VARIANT_BOOL(ctypes._SimpleCData):
ValueError: type 'v' not supported

[WinError 126] The specified module could not be found (shcore)

Windows 7
Python 3.7 64bit
pip install d3dshot

Collecting d3dshot
  Using cached https://files.pythonhosted.org/packages/e8/8d/64838f60f2a3b080024b9774673148020ec2893398c0d1ca41f12d6439f6/D3DShot-0.1.3-py3-none-any.whl
Requirement already satisfied: Pillow>=5.0.0 in ...\lib\site-packages (from d3dshot) (5.2.0)
Requirement already satisfied: comtypes>=1.1.5 in ...\lib\site-packages (from d3dshot) (1.1.7)
Installing collected packages: d3dshot
Successfully installed d3dshot-0.1.3

When running samples from Readme.md:

  File ".test.py", line 3, in <module>
    d = d3dshot.create()
  File "...\lib\site-packages\d3dshot\__init__.py", line 70, in create
    pytorch_gpu_is_available=pytorch_gpu_is_available
  File "...\lib\site-packages\d3dshot\d3dshot.py", line 24, in __init__
    self.detect_displays()
  File "...\lib\site-packages\d3dshot\d3dshot.py", line 194, in detect_displays
    self.displays = Display.discover_displays()
  File "...\lib\site-packages\d3dshot\display.py", line 107, in discover_displays
    scale_factor = d3dshot.dll.shcore.get_scale_factor_for_monitor(hmonitor)
  File "...\lib\site-packages\d3dshot\dll\shcore.py", line 7, in get_scale_factor_for_monitor
    ctypes.windll.shcore.GetScaleFactorForMonitor(hmonitor, ctypes.byref(scale_factor))
  File "...\lib\ctypes\__init__.py", line 426, in __getattr__
    dll = self._dlltype(name)
  File "...\lib\ctypes\__init__.py", line 356, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found

The self._name is "shcore" when the exception is raised

Support OpenCV Mat as a capture format

Capturing window rather than desktop?

I realise I could write my own wrapper that uses win32 to identify a windows rect/monitor then call d3dshots api with the right settings, however two cases aren't handled:

1. if the window moves (I think I'd write a wrapper to check window moving and then reinit d3dshots with right arguments)
2. if there is another window covering the target window.

Windows bitblt handles this fine,I was wondering if d3d can handle this (and it's just a matter of implementation) or if d3d can only do full screen capture followed by cropping?

How does OBS studio handle window capture? Bitblt or d3d?

Importing OpenCV alongside D3DShot results in COMError at create()

Hello,

I have run into a peculiar issue using this library (thanks for creating this btw!). When I try to import OpenCV (cv2 package) I run into this error at d3dshot.create():

(-2005270524, 'The specified device interface or feature level is not supported on this system.', (None, None, None, 0, None))

Here is my current code:

import d3dshot
import time
import cv2
d = d3dshot.create() #Error happens here

Removing the import cv2 line lets the code run without any errors. I have tried running the code as Administrator with same results.

This issue isn't urgent at all (I just wanted OpenCV for rendering, so I found a workaround) and seems to be a niche case, but I still wanted to leave this issue here in case others had similar issues.

Question

Very nice!

Tho I'd like to ask something directly, since you're more familiar with the code.
If I wanted to not receive duplicate frames when there's no change, how should I go about altering the functions so it'd return just the new data and a value of number of how many frames have been the same so far ? Thanks in advance :)

Do we own the memory pointed to by the returned screenshots? Also: Warning about using the screenshots with OpenCV!

Hey, great library, huge thanks for all of your great work!

So, I'm curious... When we grab images, it's internally done by asking Windows via COM to make a screenshot, and then return the pointer to us...

What does D3DShot do with that pointer? Does it copy the data to our own internal memory within Python, or do we continue to use the Windows pointer to memory owned by Windows?

The reason why I ask is that I've seen some really strange behavior. Perhaps a bug in d3dshot?

import cv2
import d3dshot
import numpy as np

d = d3dshot.create(capture_output="numpy")
img = d.screenshot()
print(img.flags.writeable)
img[:,:,1] = 255 # Write maximum green into every pixel (makes img hue green)
#img = img.copy() # You must uncomment this, otherwise cv2 can't draw on it.
cv2.rectangle(img, (0,0), (100,100), (0,0,255), 6)

cv2.imshow("img", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

As the comment/code explains:

• The Numpy array returned by d.screenshot() is marked as writeable.
• Writing directly into the array with img[:,:,1] = 255 works perfectly (fills the green channel with 255 in every pixel).
• Passing the img Numpy array to cv2.rectangle does NOT draw on the image.
• The only way to draw on the image is to FIRST copy it to a brand new Numpy object and THEN pass the NEW Numpy object to OpenCV.

This is very strange. It seems kinda like a bug. I really can't understand why the Numpy object is marked as writeable and we're able to write to it manually, but OpenCV cannot write to it unless we first copy it. So clearly SOMETHING differs between these objects, but I have no idea what...

---

Actually, I might have just solved it while writing this ticket, thanks to knowing quite a lot about OpenCV internals... Okay, if you look at print(img.flags), the raw screenshot says "CONTIGUOUS = FALSE" which means that the underlying RAM is not laid out in the same way as what the Numpy array is claiming. (You can read more about RAM layout and Contiguous in my article here: https://answers.opencv.org/question/219040/fastest-way-to-convert-bgr-rgb-aka-do-not-use-numpy-magic-tricks/)

So, the following code shows us what is going on:

img = d.screenshot()
print(img.flags) # Contiguous: False, Writeable: True
print(img.strides) # (1920, 1, 2073600)
img = img.copy() # Makes a new, contiguous object
print(img.strides) # (5760, 3, 1)

Okay, the next part of understanding this problem is to know about what OpenCV does internally when it receives a Python object. That's described in the link I gave above. Basically, if you give OpenCV a non-contiguous Numpy object, it can't read the RAM as-is. Instead, it creates a new Numpy object with contiguous RAM storage and then reads THAT object's RAM data, and passes it into the internal OpenCV function (ie draw rectangle).

So, OpenCV is taking the screenshot Numpy object (which is incorrectly laid out in RAM), and then it is creating a new (temporary) Numpy object with proper layout, and then it passes that object to the internal C++ based drawing function.

In other words, we're drawing on a temporary object.

Here's the proof:

import cv2
import d3dshot
import numpy as np
import time

d = d3dshot.create(capture_output="numpy")
img = d.screenshot()
#img = img.copy() # Uncomment this to check speed difference!
start = time.perf_counter()
cv2.rectangle(img, (0,0), (100,100), (0,0,255), 6)
#cv2.dnn.blobFromImage(img)
elapsed = (time.perf_counter() - start) * 1000
print("elapsed:", elapsed, "ms")

cv2.imshow("img", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

Results: If we're passing the original image given by d3dshot into cv2.rectangle, the costly conversion happens in OpenCV's "Python-to-OpenCV" wrapper... and I am getting 148.2 ms runtime for a call to cv2.rectangle. If we instead first do img = img.copy() to make a new, contiguous Numpy array, the cv2.rectangle completes in just 0.09 ms.

I also checked with blobFromImage and the time was 37 ms if img.copy() was first used, and 46 ms otherwise (meaning with the bad RAM data layout). So, around +9 ms to call the blob converter when given badly laid out RAM. Note that the "9ms" is no coincidence. That's the average amount of time it takes to run img = img.copy(), which confirms that blobFromImage has to create a fixed (contiguous) temporary, internal copy of the Numpy array when you call that function.

So yes, it is now confirmed:

D3DShot Numpy Arrays are HARMFUL if used with OpenCV. Every time you call ANY OpenCV function and give it the original d3dshot Numpy array, you are causing a MASSIVE slowdown since
the data has to be internally converted by OpenCV to a temporary Numpy array with corrected memory layout.

What's the solution then? Well, there are a few solutions. All involve various ways to pre-build a contiguous ndarray so that OpenCV doesn't have to do any conversion when you give it the ndarray later.

• img = img.copy(), which takes 9.3 milliseconds on my i7-8750H Laptop at 1920x1080.
• img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) to actually use the OpenCV color converter. This takes 11.3 milliseconds consistently on my computer (of that, 9.3 milliseconds is the time it takes Numpy to copy the non-contiguous data to a contiguous one first, which happens internally in OpenCV when given non-contiguous ndarrays). And this method is obviously only useful if you WANT to change the channel order.
• Perhaps it can be fixed in D3DShot itself, by making it lay out the data contiguously in RAM? I doubt it, but it's worth asking. If we're doing a memory copy of the temporary data that Windows gave us over COM, then perhaps we can do that copy in a way that lays it out contiguously in RAM. That way, the Numpy arrays would be valid for OpenCV API usage, without needing conversion overhead.

Help me. Why is the video in reverse sequence at the end

I apologize if I took up your time, I found the problem and solved it.

Is this maintained? feature query

Hi, just wondering if this repo is still being maintained and if there is a way to capture last frame with a region.
screenshot works for my project but seems to be delayed slightly.

monitor resolution not detected properly

I'm using a 2560x1440p monitor with no scaling set. I double checked in nvidia control panel that I'm running native 2560x1440p resolution. When I call d.devices, I get this:

<Display name=Generic PnP Monitor adapter=NVIDIA GeForce RTX 2070 SUPER resolution=2048x1152 rotation=0 scale_factor=1.25 primary=True>

According to this, my resolution is only 2048x1152 with a scale factor of 1.25. I believe this is causing an issue for me, because I am not able to capture the entire output of my monitor, even when I set region=(0, 0, 2559, 1439).

Any ideas?

Screenshot's somewhat taken, but it's only distorted colors

Multiple instances of d3dshot.create = Impossible

import d3dshot
a = d3dshot.create()
b = d3dshot.create()

Traceback (most recent call last):
  File "C:\Users\Secret\Downloads\demo.py", line 3, in <module>
    b = d3dshot.create()
  File "C:\Users\Secret\AppData\Local\Programs\Python\Python37\lib\site-packages\d3dshot\__init__.py", line 70, in create
    pytorch_gpu_is_available=pytorch_gpu_is_available
  File "C:\Users\Secret\AppData\Local\Programs\Python\Python37\lib\site-packages\d3dshot\d3dshot.py", line 24, in __init__
    self.detect_displays()
  File "C:\Users\Secret\AppData\Local\Programs\Python\Python37\lib\site-packages\d3dshot\d3dshot.py", line 194, in detect_displays
    self.displays = Display.discover_displays()
  File "C:\Users\Secret\AppData\Local\Programs\Python\Python37\lib\site-packages\d3dshot\display.py", line 119, in discover_displays
    dxgi_adapter=dxgi_adapter
  File "C:\Users\Secret\AppData\Local\Programs\Python\Python37\lib\site-packages\d3dshot\display.py", line 40, in __init__
    self.dxgi_output_duplication = self._initialize_dxgi_output_duplication()
  File "C:\Users\Secret\AppData\Local\Programs\Python\Python37\lib\site-packages\d3dshot\display.py", line 66, in _initialize_dxgi_output_duplication
    return d3dshot.dll.dxgi.initialize_dxgi_output_duplication(self.dxgi_output, self.d3d_device)
  File "C:\Users\Secret\AppData\Local\Programs\Python\Python37\lib\site-packages\d3dshot\dll\dxgi.py", line 257, in initialize_dxgi_output_duplication
    dxgi_output.DuplicateOutput(d3d_device, ctypes.byref(dxgi_output_duplication))
_ctypes.COMError: (-2147024809, 'The parameter is incorrect.', (None, None, None, 0, None))

Would this be solvable by making d3dshot share some memory between all instances of the class? For example, the d3dshot module's namespace can be used for storing a single variable that is used by all files that import d3dshot. That way, you could store a single COM connection used by all instances of the class, etc... It may be possible to make multiple instances play nicely together that way...

Personally, I made a wrapper module around d3dshot, which just basically does something like this (mine is class-based, this is just an example):

import d3dshot

ScreenGrabber = d3dshot.create()

Then I can from blah import ScreenGrabber from any other file and can use ScreenGrabber.screenshot() etc, all without getting COM errors, since they all share the exact same connection.

The important thing is to store d3dshot.create as a static variable in the module itself (outside any classes). And then ensuring that everything else only interacts with that single connection.

The display will flash quickly for a while after calling d3dshot.create(), and also when exiting the program.

I don't know when this phenomenon started. It didn't appear when I first used the d3dshot package. It was very strange, and it did not improve after restarting the computer.

bug_video

System version: Windows10 1909
Python version: 3.7.6
d3dshot version: 0.1.5

The keyed mutex was abandoned

When any DX11 game in the fullscreen mode is focused - "The keyed mutex was abandoned." is thrown from the AcquireNextFrame. Library keeps producing last screenshot forever and unfortunately it does not recover from this state until the capturing process is restarted from scratch.

Also if capturing is started after you focus fullscreen game, it works fine. But the moment you alt-tab back to desktop, it fails.

capture() fails with WinError[995]

I am trying to get high speed capture using capture->delay->stop. When calling creat(), I always got this exception:

Unhandled exception in event loop:
  File "C:\Python38\lib\asyncio\proactor_events.py", line 768, in _loop_self_reading
    f.result()  # may raise
  File "C:\Python38\lib\asyncio\windows_events.py", line 808, in _poll
    value = callback(transferred, key, ov)
  File "C:\Python38\lib\asyncio\windows_events.py", line 457, in finish_recv
    raise ConnectionResetError(*exc.args)

Exception [WinError 995] The I/O operation has been aborted because of either a thread exit or an application request
Press ENTER to continue...

This issue seems only to impact the ipython environment.

COMError at create()

import time
import d3dshot

screen = d3dshot.create(capture_output="numpy")

  File "C:\Users\mehmet\AppData\Roaming\Python\Python37\site-packages\d3dshot\display.py", line 69, in _initialize_dxgi_output_duplication
    self.dxgi_output, self.d3d_device
  File "C:\Users\mehmet\AppData\Roaming\Python\Python37\site-packages\d3dshot\dll\dxgi.py", line 284, in initialize_dxgi_output_duplication
    dxgi_output.DuplicateOutput(d3d_device, ctypes.byref(dxgi_output_duplication))
_ctypes.COMError: (-2005270524, 'Belirtilen aygıt arabirimi veya özellik düzeyi bu sistemde desteklenmiyor.', (None, None, None, 0, None))

Thats Turkish 'Belirtilen aygıt arabirimi veya özellik düzeyi bu sistemde desteklenmiyor." but in English "The specified device interface or feature level is not supported on this system."

RTX2060, Laptop, Windows 10 version 2004

I unistalled nvidia driver 452.06 and its work but with drivers it dosent work!

import time
import d3dshot
import cv2

screen = d3dshot.create(capture_output="numpy")
print(screen.display)

No way to delete D3DShot instance.

Can we add possibility to force delete instance? Like in : https://stackoverflow.com/questions/43619748/destroying-a-singleton-object-in-python . I've tested it locally and it's work for me.

I need to delete / re-create instance to recover D3DShot from some situation. For ex. D3DShot will stuck in case of D3DCtx changed. Issuse: #30

AttributeError: 'NoneType' object has no attribute 'capture'

Hello!

I'm trying to use the basic examples, but when trying to create a screenshot with:

import d3dshot
d = d3dshot.create()
d.screenshot()

I'm getting

File "", line 1, in
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\site-packages\d3dshot\d3dshot.py", line 89, in screenshot
frame = self.display.capture(self.capture_output.process, region=region)
AttributeError: 'NoneType' object has no attribute 'capture'

What am I doing wrong?

Inverted Colors

I can't tell if this is open cv or this library, but one of the channels is messed up

vs what it's supposed to look like:

I didn't see this issue with other libraries, but I might still be doing something wrong

Code doesn't work

windows 10
screen 1366x768
Notebook (with nvidia 940m)
python 3.8

I tried to run the code (Any) and it doesn't work for me.

Traceback (most recent call last):
  File "E:\NEWEXEC2\Hello.py", line 3, in <module>
    d = d3dshot.create(capture_output="numpy")
  File "C:\Users\andre\AppData\Local\Programs\Python\Python38\lib\site-packages\d3dshot\__init__.py", line 68, in create
    d3dshot = D3DShot(
  File "C:\Users\andre\AppData\Local\Programs\Python\Python38\lib\site-packages\d3dshot\d3dshot.py", line 17, in __call__
    cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
  File "C:\Users\andre\AppData\Local\Programs\Python\Python38\lib\site-packages\d3dshot\d3dshot.py", line 37, in __init__
    self.detect_displays()
  File "C:\Users\andre\AppData\Local\Programs\Python\Python38\lib\site-packages\d3dshot\d3dshot.py", line 215, in detect_displays
    self.displays = Display.discover_displays()
  File "C:\Users\andre\AppData\Local\Programs\Python\Python38\lib\site-packages\d3dshot\display.py", line 119, in discover_displays
    display = cls(
  File "C:\Users\andre\AppData\Local\Programs\Python\Python38\lib\site-packages\d3dshot\display.py", line 39, in __init__
    self.dxgi_output_duplication = self._initialize_dxgi_output_duplication()
  File "C:\Users\andre\AppData\Local\Programs\Python\Python38\lib\site-packages\d3dshot\display.py", line 68, in _initialize_dxgi_output_duplication
    return d3dshot.dll.dxgi.initialize_dxgi_output_duplication(
  File "C:\Users\andre\AppData\Local\Programs\Python\Python38\lib\site-packages\d3dshot\dll\dxgi.py", line 284, in initialize_dxgi_output_duplication
    dxgi_output.DuplicateOutput(d3d_device, ctypes.byref(dxgi_output_duplication))
_ctypes.COMError: (-2005270524, 'Указанный интерфейс устройства или уровень компонента не поддерживается в данной системе.', (None, None, None, 0, None))

when I use cv2.imshow to see the screen shoot it appears scrambled

I'm trying to print the screenshot taken with D3DShot

d = d3dshot.create(capture_output="numpy")
while True:
frame = d.screenshot(region=(140, 400, 540, 550))
cv2.imshow("screenshot", frame)

frame comes scrambled.... can anyone explain?

thank you !

Fill in the Nvidia Control Panel section in Installation Note: Laptops section of the wiki

I don't have an hybrid GPU laptop with an Nvidia dedicated GPU so I can't properly document the steps to force Python to use the onboard GPU.

If anyone can document the steps here (including screenshots, if possible), I will append the information to the wiki article.

Thanks!

RuntimeError: deque mutated during iteration

import d3dshot
import time

d = d3dshot.create(capture_output="numpy")
d.capture(region=(100,100,300,300), target_fps=30)
time.sleep(20)
d.stop()
d.frame_buffer_to_disk()

dumping frame buffer to disk always yields following error:
File "C:\ProgramData\Anaconda3\envs\testing_env\lib\site-packages\d3dshot\d3dshot.py", line 119, in frame_buffer_to_disk for i, frame in enumerate(self.frame_buffer): RuntimeError: deque mutated during iteration

Bump pillow version for Python 3.9 support on Windows

Currently, d3dshot won't install with Python 3.9 on Windows.
You can fix this, by bumping the pillow version requirement to 8.2.0

Sceenshot active monitor

I know we can screenshot specific monitor by
d.display = d.displays[1]
I'm looking for a way to automatically screenshot the active monitor

Benchmark results

Im getting the dame results un benchmarks when i run It with numpy and when i run It with default PIL. Both configurations achieve 78 FPS. Is this working as expected?
Thanks in advanced.

Create an anaconda package

Since this is a Windows specific solution, it makes sense to create a anaconda package rather that just pip.

Does not work with games

I am trying to screen record my gameplay and it does not work with games

Support for resizing (downscaling) the output

In a lot of use cases it's often better to have a resized output that's smaller than the native resolution, of course we can do this manually after the output is received, but it would be much faster to be able to down sample on GPU and then output. I'm not at all familiar with DirectX, but from what I gathered so far, we can either do this by creating and outputing mipmaps of the original frame, re-render the original frame to a smaller target-sized texture, or use some sort of compute shader to do the scaling manually. I'm not sure which way would be the fastest, from what I've learned so far, I'm guessing mipmaps would be the fastest if the scale factor is a power of two.

I'm not sure about how to approach the other methods, but mipmaps should be relatively easy to implement. However, I couldn't find a way to generate the appropriate methods & classes used by DirectX's mipmaps stuff, I tried using comtypes' GetModule('d3d11.dll'), but it doesn't seem to work and crashes with "Error loading type library/DLL".

Changing the resolution results in "frozen" screenshots - D3D doesn't update to current screen

When I change the resolution of my laptop after running my Python code with D3DShot, screenshot_to_disk_every() stops returning the current screen and instead returns the screen before resolution change. I don't know if this is an issue with Window's Desktop Duplication or this library.

My code:

import d3dshot
import time
import matplotlib.pyplot as plt
d = d3dshot.create()
d.screenshot_to_disk_every(10)

This issue occurs even with using other screenshot functions.

Profile capture process and identify bottlenecks

Document. Understand where time is spent and what optimizations are possible (if any)

Probably a good wiki entry.

Any chance of 32 bit support?

I realise that the bit-ness of python needs to match the bit-ness of the underlying Windows install. What if we're using 32-bit python in 32-bit windows?

Can you help me understand region?

@nbrochu I'm trying to wrap my head around region and clearly failing. mss has something like:

{"top": 332, "left": 752, "width": 416, "height": 416}

I have a 416 by 416 block offset top by 332 and left by 752. This gives me a 416 by 416 block dead center of the screen.

I cannot get the same thing using this project. Could you give me a pointer, im sure I'm just being dumb

Investigate WindowsGraphicsCapture API

As discovered by using OBS Studio, there is a new Window API for screen capture that has great extra features like capturing windows even if they are occluded or moved.

Investigate WindowsGraphicsCapture to determine if:

• It can be hooked into by Python
• It has good performance
• It is compatible with the existing D3DShot API

COMError with pytorch

hi,thanks for your great lib. when i use it in a conda environment installed pytorch_gpu,it doensn't work, error occurs
Traceback (most recent call last): File "C:/CenterNet/src/d3d_demo.py", line 10, in <module> d = d3dshot.create(capture_output="numpy") #capture_output="numpy" File "C:\Users\yao\Anaconda3\envs\CenterNet\lib\site-packages\d3dshot\__init__.py", line 70, in create pytorch_gpu_is_available=pytorch_gpu_is_available File "C:\Users\yao\Anaconda3\envs\CenterNet\lib\site-packages\d3dshot\d3dshot.py", line 24, in __init__ self.detect_displays() File "C:\Users\yao\Anaconda3\envs\CenterNet\lib\site-packages\d3dshot\d3dshot.py", line 194, in detect_displays self.displays = Display.discover_displays() File "C:\Users\yao\Anaconda3\envs\CenterNet\lib\site-packages\d3dshot\display.py", line 119, in discover_displays dxgi_adapter=dxgi_adapter File "C:\Users\yao\Anaconda3\envs\CenterNet\lib\site-packages\d3dshot\display.py", line 40, in __init__ self.dxgi_output_duplication = self._initialize_dxgi_output_duplication() File "C:\Users\yao\Anaconda3\envs\CenterNet\lib\site-packages\d3dshot\display.py", line 66, in _initialize_dxgi_output_duplication return d3dshot.dll.dxgi.initialize_dxgi_output_duplication(self.dxgi_output, self.d3d_device) File "C:\Users\yao\Anaconda3\envs\CenterNet\lib\site-packages\d3dshot\dll\dxgi.py", line 257, in initialize_dxgi_output_duplication dxgi_output.DuplicateOutput(d3d_device, ctypes.byref(dxgi_output_duplication)) _ctypes.COMError: (-2005270524, 'The specified device interface or feature level is not supported on this system。', (None, None, None, 0, None))
when I uninstalled pytorch, it can capture frames well,(but first run frame is full of noise).
I wanna use d3dshot and pytorch to make a aimbot ,may help me solve this problems,thank u very much!

Errors in Virtual Machines

I am using the d3dshot module in a virtual machine.

The d3dshot module works normally the first time after booting the virtual machine, but it does not work after that.
Restarting after logging out of the session does not work.

I don't know what the problem is.

However, the following message occurs:
I've been trying for days to find the cause,
but I can't figure it out.

I need help~~~

========= Error Message ==================
Traceback (most recent call last):

File "C:\Program Files (x86)\BATEM\Common\Resource\embedded\python36.zip\runpy.py", line 193, in _run_module_as_main

File "C:\Program Files (x86)\BATEM\Common\Resource\embedded\python36.zip\runpy.py", line 85, in _run_code

File "C:\Program Files (x86)\BATEM\Common\Resource\engine\rpa\library\screen_capture.py", line 5, in

__INSTANCE__ = d3dshot.create()

File "C:\Program Files (x86)\BATEM\Common\Resource\embedded\lib\site-packages\d3dshot_init_.py", line 74, in create

pytorch_gpu_is_available=pytorch_gpu_is_available,

File "C:\Program Files (x86)\BATEM\Common\Resource\embedded\lib\site-packages\d3dshot\d3dshot.py", line 17, in call

cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)

File "C:\Program Files (x86)\BATEM\Common\Resource\embedded\lib\site-packages\d3dshot\d3dshot.py", line 37, in init

self.detect_displays()

File "C:\Program Files (x86)\BATEM\Common\Resource\embedded\lib\site-packages\d3dshot\d3dshot.py", line 216, in detect_displays

self.displays = Display.discover_displays()

File "C:\Program Files (x86)\BATEM\Common\Resource\embedded\lib\site-packages\d3dshot\display.py", line 129, in discover_displays

dxgi_adapter=dxgi_adapter,

File "C:\Program Files (x86)\BATEM\Common\Resource\embedded\lib\site-packages\d3dshot\display.py", line 39, in init

self.dxgi_output_duplication = self._initialize_dxgi_output_duplication()

File "C:\Program Files (x86)\BATEM\Common\Resource\embedded\lib\site-packages\d3dshot\display.py", line 69, in _initialize_dxgi_output_duplication

self.dxgi_output, self.d3d_device

File "C:\Program Files (x86)\BATEM\Common\Resource\embedded\lib\site-packages\d3dshot\dll\dxgi.py", line 284, in initialize_dxgi_output_duplication

dxgi_output.DuplicateOutput(d3d_device, ctypes.byref(dxgi_output_duplication))

ctypes.COMError: (-2147418113, 'Error', (None, None, None, 0, None))