imageio / imageio-ffmpeg

FFMPEG wrapper for Python

IMAGEIO

Website: https://imageio.readthedocs.io/

Imageio is a Python library that provides an easy interface to read and write a wide range of image data, including animated images, video, volumetric data, and scientific formats. It is cross-platform, runs on Python 3.8+, and is easy to install.

Professional support is available via Tidelift.

Example

Here's a minimal example of how to use imageio. See the docs for more examples.

import imageio.v3 as iio
im = iio.imread('imageio:chelsea.png')  # read a standard image
im.shape  # im is a NumPy array of shape (300, 451, 3)
iio.imwrite('chelsea.jpg', im)  # convert to jpg

API in a nutshell

As a user, you just have to remember a handful of functions:

• imread() - for reading
• imwrite() - for writing
• imiter() - for iterating image series (animations/videos/OME-TIFF/...)
• improps() - for standardized metadata
• immeta() - for format-specific metadata
• imopen() - for advanced usage

See the API docs for more information.

Features

• Simple interface via a concise set of functions
• Easy to install using Conda or pip
• Few dependencies (only NumPy and Pillow)
• Pure Python, runs on Python 3.8+, and PyPy
• Cross platform, runs on Windows, Linux, macOS
• More than 295 supported formats
• Read/Write support for various resources (files, URLs, bytes, FileLike objects, ...)
• Code quality is maintained via continuous integration and continuous deployment

Dependencies

Minimal requirements:

• Python 3.8+
• NumPy
• Pillow >= 8.3.2

Optional Python packages:

• imageio-ffmpeg (for working with video files)
• pyav (for working with video files)
• tifffile (for working with TIFF files)
• itk or SimpleITK (for ITK plugin)
• astropy (for FITS plugin)
• imageio-flif (for working with FLIF image files)

Citing imageio

If you use imageio for scientific work, we would appreciate a citation. We have a DOI!

Security contact information

To report a security vulnerability, please use the Tidelift security contact. Tidelift will coordinate the fix and disclosure.

ImageIO for enterprise

Available as part of the Tidelift Subscription.

The maintainers of imageio and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source dependencies you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact dependencies you use. Learn more.

Details

The core of ImageIO is a set of user-facing APIs combined with a plugin manager. API calls choose sensible defaults and then call the plugin manager, which deduces the correct plugin/backend to use for the given resource and file format. The plugin manager then adds sensible backend-specific defaults and then calls one of ImageIOs many backends to perform the actual loading. This allows ImageIO to take care of most of the gory details of loading images for you, while still allowing you to customize the behavior when and where you need to. You can find a more detailed explanation of this process in our documentation.

Contributing

We welcome contributions of any kind. Here are some suggestions on how you are able to contribute

• add missing formats to the format list
• suggest/implement support for new backends
• report/fix any bugs you encounter while using ImageIO

To assist you in getting started with contributing code, take a look at the development section of the docs. You will find instructions on setting up the dev environment as well as examples on how to contribute code.

imageio-ffmpeg's Issues

_io.read_frames causes "ResourceWarning: unclosed file"

Description

read_frames.close() leads to a ResourceWarning: unclosed file, e.g. when running unittest tests.

This implies that the standard streams are not being closed properly, similar to imageio/imageio#697:

stderr is opened here but not explicitly closed (here?).

How to reproduce

Add the following test to test_io.py and run (tested on Windows 10, Python 3.8):

def test_read_frames_resource_warning():
    with pytest.warns(None) as warnings:
        gen = imageio_ffmpeg.read_frames(test_file1)
        next(gen)
        gen.close()
    # there should not be any warnings, but show warning messages if there are
    assert not [w.message for w in warnings]

Custom number of bytes per pixel in read_frames

Y′UV format can have a different number of bytes per pixel:
https://en.wikipedia.org/wiki/YUV#Converting_between_Y%E2%80%B2UV_and_RGB

And some variants such as:

YUV420p   6 bytes per 4 pixels

can't be set as bpp for read_frames. As a result, there is broken framesize and we have RuntimeError with End of file reached before full frame could be read

ffmpeg executable permission issue in linux, Runtime error: No ffmpeg could be found

On inspecting ffmpeg executable installed after installing imageio-ffmpeg package is not readable and executable which is causing following runtime error:
RuntimeError: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.

Enclosing Dockerfile for replicating issue in linux:

FROM continuumio/miniconda3 AS build

# Install dependencies and tools
RUN DEBIAN_FRONTEND=noninteractive apt-get update -yqq && \
    apt-get upgrade -yqq && \
    apt-get install -yqq --no-install-recommends \
#    wget \
#    curl \
    libssl-dev \
    build-essential \
    apt-utils \
    zip \
    unzip \
    gcc \
    git \
    locales \
    nano \
    libgl1-mesa-glx \
    libsm6 libxext6 \
    --no-install-recommends \
    && rm -rf /var/lib/apt/lists/* \
    && rm -rf /src/*.deb \
    && apt-get clean

RUN pip install imageio_ffmpeg
ENV SHELL /bin/bash

Now calling def get_ffmpeg_exe() method after importing imageio_ffmpeg causes this runtime issue.
Additionally I can see ffmpeg binary ffmpeg-linux64-v4.2.2 in site-packages /opt/conda/lib/python3.8/site-packages/imageio_ffmpeg-0.4.3-py3.8-linux-x86_64.egg/imageio_ffmpeg/binaries.
Manually setting permission using chmod +x ffmpeg-linux64-v4.2.2 fixes issue.
This should be automatically handled by code while installing.

yoru cdn with s3

SIGSEGV when using threads

This script will segfault pretty reliably when using imageio. It's hard to say if this is an imageio error, or even an imageio-ffmpeg error (maybe this python build is broken or something), but doesn't happen when using the image loading backend and may be related to #17.

https://github.com/christopherhesse/imageio-sigsegv

Migrate to pathlib?

Since imageio-ffmpeg requires Python>=3.4, I would suggest migrating to pathlib.
I could create a PR for that.

On Ubuntu, get_ffmpeg_exe causes AttributeError: 'PosixPath' object has no attribute 'startswith'

Hey, there appears to be a bug on Ubuntu (and potentially other linux systems), where importing get_ffmpeg_exe results in AttributeError: 'PosixPath' object has no attribute 'startswith'. Additional info: the code was being executed on remote VMs like runpod, but running Ubuntu.

I am reporting this one the behalf of some other peeps who ran into this issue. Here is the full stack trace to help narrow down the problem. My intuition says something just needs to be cast to string, because on linux/Ubuntu, something returns a PosixPath.

2023-10-18T14:14:15.336368834Z   File "/usr/lib/python3.10/pkgutil.py", line 417, in get_importer
2023-10-18T14:14:15.336371751Z     importer = sys.path_importer_cache[path_item]
2023-10-18T14:14:15.336374287Z KeyError: PosixPath('/workspace/ComfyUI')
2023-10-18T14:14:15.336376547Z
2023-10-18T14:14:15.336378896Z During handling of the above exception, another exception occurred:
2023-10-18T14:14:15.336381189Z
2023-10-18T14:14:15.336383674Z Traceback (most recent call last):
2023-10-18T14:14:15.336385957Z   File "/workspace/ComfyUI/nodes.py", line 1735, in load_custom_node
2023-10-18T14:14:15.336388253Z     module_spec.loader.exec_module(module)
2023-10-18T14:14:15.336390524Z   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
2023-10-18T14:14:15.336392763Z   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2023-10-18T14:14:15.336400783Z   File "/workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite/__init__.py", line 1, in <module>
2023-10-18T14:14:15.336403599Z     from .videohelpersuite.nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
2023-10-18T14:14:15.336406051Z   File "/workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite/videohelpersuite/nodes.py", line 26, in <module>
2023-10-18T14:14:15.336408501Z     from imageio_ffmpeg import get_ffmpeg_exe
2023-10-18T14:14:15.336410748Z   File "/venv/lib/python3.10/site-packages/imageio_ffmpeg/__init__.py", line 7, in <module>
2023-10-18T14:14:15.336412943Z     from ._io import count_frames_and_secs, read_frames, write_frames
2023-10-18T14:14:15.336415104Z   File "/venv/lib/python3.10/site-packages/imageio_ffmpeg/_io.py", line 8, in <module>
2023-10-18T14:14:15.336417279Z     from ._parsing import LogCatcher, cvsecs, parse_ffmpeg_header
2023-10-18T14:14:15.336419453Z   File "/venv/lib/python3.10/site-packages/imageio_ffmpeg/_parsing.py", line 5, in <module>
2023-10-18T14:14:15.336421683Z     from ._utils import logger
2023-10-18T14:14:15.336424533Z   File "/venv/lib/python3.10/site-packages/imageio_ffmpeg/_utils.py", line 7, in <module>
2023-10-18T14:14:15.336426910Z     from pkg_resources import resource_filename
2023-10-18T14:14:15.336429098Z   File "/venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3328, in <module>
2023-10-18T14:14:15.336431308Z     def _initialize_master_working_set():
2023-10-18T14:14:15.336433538Z   File "/venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3302, in _call_aside
2023-10-18T14:14:15.336435741Z     f(*args, **kwargs)
2023-10-18T14:14:15.336438355Z   File "/venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3340, in _initialize_master_working_set
2023-10-18T14:14:15.336440525Z     working_set = WorkingSet._build_master()
2023-10-18T14:14:15.336442738Z   File "/venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 622, in _build_master
2023-10-18T14:14:15.336444996Z     ws = cls()
2023-10-18T14:14:15.336448457Z   File "/venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 615, in __init__
2023-10-18T14:14:15.336450715Z     self.add_entry(entry)
2023-10-18T14:14:15.336452955Z   File "/venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 671, in add_entry
2023-10-18T14:14:15.336455188Z     for dist in find_distributions(entry, True):
2023-10-18T14:14:15.336457402Z   File "/venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2075, in find_distributions
2023-10-18T14:14:15.336459588Z     importer = get_importer(path_item)
2023-10-18T14:14:15.336461772Z   File "/usr/lib/python3.10/pkgutil.py", line 421, in get_importer
2023-10-18T14:14:15.336464000Z     importer = path_hook(path_item)
2023-10-18T14:14:15.336466217Z   File "<frozen importlib._bootstrap_external>", line 1632, in path_hook_for_FileFinder
2023-10-18T14:14:15.336468383Z   File "<frozen importlib._bootstrap_external>", line 1504, in __init__
2023-10-18T14:14:15.336470552Z   File "<frozen importlib._bootstrap_external>", line 182, in _path_isabs
2023-10-18T14:14:15.336472757Z AttributeError: 'PosixPath' object has no attribute 'startswith'
2023-10-18T14:14:15.336474952Z
2023-10-18T14:14:15.336477128Z Cannot import /workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite module for custom nodes: 'PosixPath' object has no attribute 'startswith'```

0.4.0 regression on Linux: count_frames() freezes

When updating to imageio-ffmpeg 0.4.0, my code which uses the imageio.plugins.ffmpeg.FfmpegFormat.Reader.count_frames() function freezes with 0% CPU utilization. This happened on two different Linux systems and multiple video files from different sources today, so I investigated a bit and found that reverting the change in this line a19c761#diff-f32de983d7bf3c1014870e92de0d1882R41 back to the previous version

out = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=ISWIN)

fixes the freezing on my system.
I don't know what exactly is happening there, I only noticed that when sending a KeyboardInterrupt, the traceback always shows that it was currently executing the line

out = subprocess.check_output(cmd, stderr=subprocess.STDOUT, **_popen_kwargs())

in imageio_ffmpeg/_io.py.

A minimal code example to reproduce this, given that a.mp4 is a regular video file:

import imageio
r = imageio.get_reader('a.mp4')
r.count_frames()

Pyinstaller import

Hello,
I've built an application using pyinstaller that relies on imageio-ffmpeg for movie creation.

When freezing the application in a non-windowed manner, I was having no issue with writing movies. However, when building a -windowed application on macOS, I was getting an import error telling me that the writer couldn't establish a link to the file path.

I found that this issue could be simply solved by adding the following argument to a build command:

--hidden-import=imagio-ffmpeg

I wasn't sure where else to report this, but I figured I would share it here for any help.

I'm using pyinstaller==4.3 and imagio-ffmpeg==0.4.3

pix_fmt may be badly reported for `yuv420p(tv, progressive)`

If I run the test suite, I believe that the following can be reported from ffmpeg.

'  Stream #0:0[0x1](und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, progressive), 64x64, 1 kb/s, 10 fps, 10 tbr, 10240 tbn (default)'

The string parser, will split the string by commas returning the pixel format:

yuv420p(tv

instead of

yuv420p(tv, progressive)

I think this answer from SO give a potential solution:
https://stackoverflow.com/a/26634150

quality=None is not respected in ffmpeg imageio

imageio-ffmpeg/imageio_ffmpeg/_io.py

Line 344 in 8de24f7

quality = quality or 5

Documentation says that setting to None will enable us to specify the output parameters, but it seems that it still specifies them for us.

https://imageio.readthedocs.io/en/stable/_autosummary/imageio.plugins.ffmpeg.html#module-imageio.plugins.ffmpeg

What if a user wants to bring her/his own ffmpeg?

• For pip one can use --no-binary to force installing the source-only version.
• For conda, I propose that we make a conda recipe that does not include the ffmpeg exe, but depends on the ffmpeg conda package. Then one can use conda install --no-deps imageio-ffmpeg to install just the wrapper.

Thoughts?

Encoders are not always fully compatible with all image shapes

So we ran into this, and I figured I would share back some findings since I did contribute the code in question:

In recent version of imageio-ffmpeg, an attempt is made to find the "best" encoder.

imageio-ffmpeg/imageio_ffmpeg/_io.py

Line 16 in 7bfe71d

h264_encoder_preference["libx264"] = 100

However, the choice of encoder also needs to be conditioned, among other things, on the encoder capabilities.

Different things, like the pixel type, but most critically, the image shape, can affect the best choice for the given encoder.
See for example the bug report: https://trac.ffmpeg.org/ticket/9251

I've added a shape parameter to the test to ensure that the choice of encoder matches the shape of the video.

You probably also want to test with the compatibility of the filters. Though for me, this is outside the scope of my immediate needs.

Color range should be specified in parameters

I've really been trying to better understand how ffmpeg is really treating our data.

I found that there is a difference between "tv" color range, and "pc" color range. Where the "tv" color range is compressed, and the "pc" color range is "full".

The code in FFMPEG:

static int convert_range(enum AVColorRange color_range)
{
    switch (color_range) {
    case AVCOL_RANGE_UNSPECIFIED:
    case AVCOL_RANGE_MPEG:
        return ZIMG_RANGE_LIMITED;
    case AVCOL_RANGE_JPEG:
        return ZIMG_RANGE_FULL;
    }
    return ZIMG_RANGE_LIMITED;
}

Thus if you leave the color range unspecified, it will be compressed, very likely not what you want in modern image processing.

I've been adding code to specify the color range as 2 in hopes to avoid color compression. I'm really not sure if it does the right thing since it seems that the color range of yuv isn't specified in the log of the implicit conversion that ffmpeg adds if your input and output colors do not match.

Some related discussion here:
https://www.facebook.com/permalink.php?story_fbid=2413101932257643&id=100006735798590

Race condition: We had to kill ffmpeg to stop it.

I'm trying to encode several mp4s at a time, and all of the larger ones give me the message We had to kill ffmpeg to stop it. Most of the files for which this message is printed are corrupted. Looking at the code, there appears to be a race condition in there where it only gives ffmpeg 2.5s to clean up. Is this needed? Or is there a way to disable that?

imageio_ffmpeg We had to kill ffmpeg to stop it, video reader not write!

Hi,
I'm looking for ffmpeg timeout param for video reader.
I need to read a video with resolution 512X512 in batch of 16 frames and do some operations.
How I can increase the timeout for leave the video reader open?
Thanks

BrokenPipeError: [Errno 32] Broken pipe

hi, I had the following error:
[could not acquire lock for <_io.BufferedReader name=10>..]
as suggested in previous issue
--> updated imageio with the updated release imageio-ffmpeg 0.4.0
rerun my code and getting the following error:

[BrokenPipeError: [Errno 32] Broken pipe
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/xxx/opt/anaconda3/envs/cvA2Zcourse/lib/python3.7/site-packages/imageio_ffmpeg/_io.py", line 483, in write_frames
    raise IOError(msg)
OSError: [Errno 32] Broken pipe
FFMPEG COMMAND:
/Users/xxx/opt/anaconda3/envs/cvA2Zcourse/bin/ffmpeg -y -f rawvideo -vcodec rawvideo -s 1920x1080 -pix_fmt rgb24 -r 30.00 -i - -an -vcodec libx264 -pix_fmt yuv420p -crf 25 -vf scale=1920:1088 -v warning /Users/xxx/PycharmProjects/cvA2Zcourse/output.mp4
FFMPEG STDERR OUTPUT:
]

the code is simple read-write example:
(Doing some Object Detection on a video)

reader = imageio.get_reader('funny_dog.mp4')
fps = reader.get_meta_data()['fps']
writer = imageio.get_writer('output.mp4', fps = fps)
for i, frame in enumerate(reader):
    frame = detect(frame, net.eval(), transform)
    writer.append_data(frame)
    print(i)
writer.close()

run on mac

@almarklein

Originally posted by @shimrite in imageio/imageio#482 (comment)

FFMPEG reader fails after KeyboardInterrupt

After a manual KeyboardInterrupt ([Ctrl]+[c]), reading a frame with (index > 0) causes an FFMPEG error.

import imageio
reader = imageio.get_reader("imageio:cockatoo.mp4",format="ffmpeg")

#reader.get_data(2) #Initializing the reader by reading some frame here will cause the error to go away

try:
    input("Do a manual KeyboardInterrupt [Ctrl]+[c] (raising an error with code won't trigger the error)")
except:
    print(reader.get_data(1)) #Reading some frame, n>0 fails.
    print(reader.get_data(0)) #Reading frame 0 works fine.

System:
Imageio 2.4.1
Python 3.6.5
FFMPEG 3.2.4
Linux 64bit

count_frames doesn't exist in 2.3.0 version

I found the following code is not applicable.

reader = imageio.get_reader(x_train[0], 'ffmpeg')
reader.count_frames()

RuntimeError: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.

Just regular call to imageio, and it works during the same runtime. But suddenly, this stops working and gives the following error:

Traceback (most recent call last):
  File "/Users/ge/mit/dmc_gen/dmc_gen_analysis/__init__.py", line 200, in thunk
  File "/home/gridsan/geyang/jaynes-mount/dmc_gen/2021-03-20/113123.645983/dmc_gen/dmc_gen/train.py", line 100, in train
    evaluate(env, agent, Args.eval_episodes, save_video=f"videos/{step:08d}_train.mp4")
  File "/home/gridsan/geyang/jaynes-mount/dmc_gen/2021-03-20/113123.645983/dmc_gen/dmc_gen/train.py", line 31, in evaluate
    logger.save_video(frames, key=save_video)
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/ml_logger/ml_logger.py", line 1147, in save_video
    imageio.mimsave(ntp.name, img_as_ubyte(frame_stack), format=format, fps=fps, **imageio_kwargs)
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio/core/functions.py", line 418, in mimwrite
    writer.append_data(im)
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio/core/format.py", line 502, in append_data
    return self._append_data(im, total_meta)
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio/plugins/ffmpeg.py", line 561, in _append_data
    self._initialize()
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio/plugins/ffmpeg.py", line 619, in _initialize
    self._write_gen.send(None)
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio_ffmpeg/_io.py", line 377, in write_frames
    cmd = [_get_exe(), "-y", "-f", "rawvideo", "-vcodec", "rawvideo", "-s", sizestr]
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio_ffmpeg/_io.py", line 19, in _get_exe
    exe = get_ffmpeg_exe()
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio_ffmpeg/_utils.py", line 50, in get_ffmpeg_exe
    "No ffmpeg exe could be found. Install ffmpeg on your system, "
RuntimeError: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.

Actual Behavior

This is not the first time it is called in this RunTime. For background, this is running on an HPC system. The first time this happened I also set the "IMAGEIO_FFMPEG_EXE" variable. Now I always set that variable.

The real red flag is that this function call is okay in these runs before this error is suddenly gets raised.

Steps to Reproduce the Problem

This is hardware-specific. And it takes a while to reproduce inside a ML training loop. These are the constrains I have no control over.

Specifications

• Python Version: 3.6.9
• ImageIO Version:

  imageio                 2.9.0
  imageio-ffmpeg          0.4.3
  

• Platform Name: Linux 4.14.224-llgrid-10ms
• Platform Version: Linux 4.14.224-llgrid-10ms

tarball downloaded via pip install

I am not sure yet who to blame, but the wheel is not always downloaded via pip install imageio-ffmpeg.

This combination for example downloaded the tar ball instead.

• pip==18.1 (previous major release)
• wheel==0.32.0 (latest)
• imageio-ffmpeg==0.2.0

Demo in docker for a 100% clean environment:

$ docker run --rm --tty python:3.7 bash -c "\
   pip install pip==18.1 wheel==0.33.0 \
&& pip install imageio-ffmpeg==0.2.0 \
&& find /usr/local/lib/python3.7/site-packages/imageio_ffmpeg/binaries/"
# <snip>
# /usr/local/lib/python3.7/site-packages/imageio_ffmpeg/binaries/
# /usr/local/lib/python3.7/site-packages/imageio_ffmpeg/binaries/README.md

---

Related issue: Zulko/moviepy#906

No such filter: 'yadif_cuda'

When I put this on my command line:

ffmpeg -i in.mkv -vf "hwupload_cuda,yadif_cuda=0:-1:0" -c:v h264_nvenc out.mp4

it works, but when I do this:

params = "-vf \"hwupload_cuda,yadif_cuda=0:-1:0\" -c:v h264_nvenc".split(" ")
stream = ffmpeg.read_frames(a, output_params=params)
meta = stream.__next__()
writer = ffmpeg.write_frames(out_file, meta["size"], output_params=params)

it tells me:

No such filter: 'yadif_cuda'
No such filter: 'hwupload_cuda'

any chance I might be doing something stupid?

Find ffmpeg in frozen applications

When this lib is included in a frozen application, the ffmpeg executable must be shipped along too. We should:

• Perhaps provide a simple function that can be used in the freeze script to copy the ffmpeg exe.
• Make get_ffmpeg_exe() look in the frozen application directory.

While we don't have this, please copy the ffmpeg executable yourself, and use the IMAGEIO_FFMPEG_EXE environment variable to point to it.

encode with audio using file path

Hi I made a simple change to the write_frames method to allow for an audio file to be provided. Is it within scope / would you like a pull request?

https://github.com/pierogis/imageio-ffmpeg/blob/audio/imageio_ffmpeg/_io.py#L377-L386

Can I use ffmpeg gpu build from source?

Will it use gpu for video to frames process?

ffmpeg format is very slow and on segment loses data

I am using imageio.get_writer('hls.m3u8', mode='I', format='FFMPEG', fps=30)
m3u8.append_data(frame)

I find that to upload appended data is very slow and if the file gets to a point that it segments then the playback is horrible
(I am using

Use pkg_resources to get binary

I would suggest to use pkg_resources to get the binary. This also enables getting the binary when the package is zipped (for some reason). I also read somewhere that this is the better approach.

import pathlib
from pkg_resources import resource_filename

binaries_dir = pathlib.Path(resource_filename("imageio", "binaries"))

I could create a PR.

Updating IMAGEIO_FFMPEG_EXE leads to `@lru_cache` related issues

I was trying to run some imageio_ffmpeg code with different versions of the ffmpeg executable for comparison purposes.

Setting IMAGEIO_FFMPEG_EXE was unfortunately not enough and it took me a while to track down the problem. It turns out that the ffmpeg executable is being non-consistently cached through a @lru_cache decorator. It would be nice to provide a more robust means of switching ffmpeg executable from python. Either removing the use of a cache or providing a setter function for IMAGEIO_FFMPEG_EXE that takes care of clearing all relevant caches.

The culprit in my case was here:

imageio-ffmpeg/imageio_ffmpeg/_io.py

Lines 135 to 137 in 7bfe71d

	@lru_cache()
	def _get_exe():
	return get_ffmpeg_exe()

with a workaround being to call imageio_ffmpeg._io._get_exe.cache_clear().

Steps to reproduce the problem:

# Uncomment the next line for google colab
#!pip install imageio_ffmpeg
import os
import imageio_ffmpeg
print("imageio_ffmpeg.__version__:", imageio_ffmpeg.__version__)

# The following try/catch is only relevant if this code is called several times
# e.g. in a notebook
try:
    init_ffmpeg
except NameError:
    init_ffmpeg = imageio_ffmpeg.get_ffmpeg_exe()
    init_utils_ffmpeg = imageio_ffmpeg._utils.get_ffmpeg_exe()
    init_io_ffmpeg = imageio_ffmpeg._io._get_exe()
    try:
      init_ffmpeg_version = imageio_ffmpeg.get_ffmpeg_version()
    except FileNotFoundError as e:
      print("Couldn't get ffmpeg executable version: ", e)
      init_ffmpeg_version = "not-found"

print("\ninit_ffmpeg:",init_ffmpeg)
print("init_utils_ffmpeg:",init_utils_ffmpeg)
print("init_io_ffmpeg:",init_io_ffmpeg)
print("init_ffmpeg version:",init_ffmpeg_version)

os.environ["IMAGEIO_FFMPEG_EXE"] = "ffmpeg"
print("\nSetting env IMAGEIO_FFMPEG_EXE to", os.environ["IMAGEIO_FFMPEG_EXE"])

print("imageio_ffmpeg.get_ffmpeg_exe():",imageio_ffmpeg.get_ffmpeg_exe())
print("imageio_ffmpeg._utils.get_ffmpeg_exe():",imageio_ffmpeg._utils.get_ffmpeg_exe())
print("imageio_ffmpeg._io._get_exe():",imageio_ffmpeg._io._get_exe())
imageio_ffmpeg._io._get_exe.cache_clear()
print("\nCache cleared - imageio_ffmpeg._io._get_exe():",imageio_ffmpeg._io._get_exe())
try:
  print("\nffmpeg version:",imageio_ffmpeg.get_ffmpeg_version())
except FileNotFoundError as e:
  print("\nCouldn't get ffmpeg executable version: ", e)

which leads to the following output on a current version of google colab:

imageio_ffmpeg.__version__: 0.4.7

init_ffmpeg: /usr/local/lib/python3.7/dist-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2
init_utils_ffmpeg: /usr/local/lib/python3.7/dist-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2
init_io_ffmpeg: /usr/local/lib/python3.7/dist-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2
init_ffmpeg version: 4.2.2-static

Setting env IMAGEIO_FFMPEG_EXE to ffmpeg
imageio_ffmpeg.get_ffmpeg_exe(): ffmpeg
imageio_ffmpeg._utils.get_ffmpeg_exe(): ffmpeg
imageio_ffmpeg._io._get_exe(): /usr/local/lib/python3.7/dist-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2

Cache cleared - imageio_ffmpeg._io._get_exe(): ffmpeg

ffmpeg version: 3.4.11-0ubuntu0.1

Aarch64 wheel is missing on PyPi

I have seen that the aarch64 wheel is missing in Pypi. It will be useful if the aarch64 wheel is released as -

• The Arm architecture has been available on AWS (Graviton2) for about 2 years now and has seen a lot of adoption by various organizations. The install base of Arm users is growing at a rapid pace.

• Oracle cloud will be releasing Arm based Compute Shapes in the future. Please see

• Generally, there is a lot of momentum behind various products switching to Arm based CPUs. For example, the Microsoft surface Pro X is an Arm based device.

@almarklein Please let me know your interest in releasing aarch64 wheels. I can help in this. To start with can you please suggest the steps you are using for building the wheel.

cannot convert each frame to numpy format

Each frame is a byte object, but I cannot convert each frame to numpy format or other image formats. Can you tell me how to do this，thank you!

This is my code:

import imageio.v2 as imageio
import imageio_ffmpeg
import ffmpeg

from io import BytesIO
from PIL import Image
import numpy
import cv2
import base64

def read_frame_as_jpg(in_filename, frame_num):
    out, err = (
        ffmpeg
        .input(in_filename)
        .filter_('select', 'gte(n,{})'.format(frame_num))
        .output('pipe:', vframes=1, format='image2', vcodec='h264_cuvid')
        .run(capture_stdout=True)
    )
    return out

def base64_to_image(base64_str): 
    image = base64.b64decode(base64_str, altchars=None, validate=False)
    image = BytesIO(image)
    image = Image.open(image)
    return image


video_path = "rtsp://admin:[email protected]:554/h264/ch1/child/av_stream"
# video_path = "/mnt/harddisk1/data/AICityChallenge/test/all_data/videos/001.mp4"

# frame = read_frame_as_jpg(video_path, 0)

# video_reader = imageio.get_reader(video_path, 'ffmpeg', ffmpeg_params=['-c:v', 'h264_cuvid'])

video_reader=imageio_ffmpeg.read_frames(video_path,input_params=['-c:v', 'h264_cuvid']) 
meta = video_reader.__next__()  

print(video_reader)

for frame in video_reader:
    # print(frame)

    # method 1
    # image=base64_to_image(frame)   #here is wrong,  Error: PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7f7a8accaae0>

    # method 2
    decoded = cv2.imdecode(numpy.frombuffer(frame, numpy.uint8), -1)
    print(decoded)  ##here is wrong,  The output content is all None
    
    # method 3
    # bytes_stream = BytesIO(frame)   #here is wrong,  Error: PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7f7a8accaae0>
    # image = Image.open(bytes_stream)

**This is my env:**
Ubuntu Version:20.04
Python Version: Python 3.8.0
FFmpeg Version:
ffmpeg version 5.1.git Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --prefix=/usr/local/ffmpeg --enable-shared --disable-static --disable-doc --enable-gpl --enable-libx264 --enable-cuda --enable-cuvid
  libavutil      58.  6.100 / 58.  6.100
  libavcodec     60.  9.100 / 60.  9.100
  libavformat    60.  4.101 / 60.  4.101
  libavdevice    60.  2.100 / 60.  2.100
  libavfilter     9.  5.100 /  9.  5.100
  libswscale      7.  2.100 /  7.  2.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100

Python Package:
Package                  Version
------------------------ --------------------
absl-py                  1.4.0
anyio                    3.6.2
asn1crypto               0.24.0
cachetools               5.3.0
certifi                  2022.12.7
charset-normalizer       3.1.0
click                    8.1.3
coloredlogs              15.0.1
contourpy                1.0.7
cryptography             2.1.4
cycler                   0.11.0
einops                   0.6.0
ffmpeg-python            0.2.0
Flask                    2.2.3
flatbuffers              23.3.3
fonttools                4.39.0
future                   0.18.3
google-auth              2.16.2
google-auth-oauthlib     0.4.6
grpcio                   1.51.3
h11                      0.14.0
httpcore                 0.16.3
httpx                    0.23.3
humanfriendly            10.0
idna                     3.4
imageio                  2.26.0
imageio-ffmpeg           0.4.8
imgaug                   0.4.0
importlib-metadata       6.0.0
importlib-resources      5.12.0
itsdangerous             2.1.2
Jinja2                   3.1.2
keyring                  10.6.0
keyrings.alt             3.0
kiwisolver               1.4.4
lazy_loader              0.1
lxml                     4.9.2
Markdown                 3.4.1
markdown-it-py           2.2.0
MarkupSafe               2.1.2
matplotlib               3.7.1
mdurl                    0.1.2
mpmath                   1.3.0
networkx                 3.0
numpy                    1.24.2
nvidia-cublas-cu11       11.10.3.66
nvidia-cuda-nvrtc-cu11   11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11        8.5.0.96
oauthlib                 3.2.2
onnx                     1.13.1
onnx-simplifier          0.4.17
onnxconverter-common     1.13.0
onnxruntime              1.14.1
onnxruntime-gpu          1.14.1
opencv-python            4.7.0.72
packaging                23.0
pafy                     0.5.5
pandas                   1.5.3
Pillow                   9.4.0
pip                      23.0.1
protobuf                 3.20.3
psutil                   5.9.4
pyasn1                   0.4.8
pyasn1-modules           0.2.8
pycrypto                 2.6.1
Pygments                 2.14.0
PyGObject                3.26.1
pyparsing                3.0.9
python-dateutil          2.8.2
python-dotenv            1.0.0
pytz                     2022.7.1
PyWavelets               1.4.1
PyWebDAV3                0.10.0
pyxdg                    0.25
PyYAML                   6.0
requests                 2.28.2
requests-oauthlib        1.3.1
rfc3986                  1.5.0
rich                     13.3.2
rsa                      4.9
scikit-image             0.20.0
scipy                    1.9.1
seaborn                  0.12.2
SecretStorage            2.3.1
sentry-sdk               1.16.0
setuptools               67.6.0
shapely                  2.0.1
six                      1.11.0
sniffio                  1.3.0
sympy                    1.11.1
tensorboard              2.12.0
tensorboard-data-server  0.7.0
tensorboard-plugin-wit   1.8.1
thop                     0.1.1.post2209072238
tifffile                 2023.3.15
torch                    1.13.1
torchvision              0.14.1
tqdm                     4.65.0
typing_extensions        4.5.0
ultralytics              8.0.53
urllib3                  1.26.15
webdavclient3            3.14.6
Werkzeug                 2.2.3
wheel                    0.30.0
zipp                     3.15.0

`Writer.append_data` hangs when starting ffmpeg process.

Probably related to #33.

Something very weird is happening when Popen is called with a preexec_fn in a complex program with many moving parts. The calling thread hangs trying to read from a pipe (presumbaly created by Popen, although setting stdout and stderr to None or DEVNULL has no effect on the behavior), while the new process blocks on a futex:

[pid 30794] set_robust_list(0x7f54ba3629e0, 24 <unfinished ...>
[pid 30794] <... set_robust_list resumed> ) = 0
[pid 30794] write(12, "\1\0\0\0\0\0\0\0", 8) = 8
[pid 30794] write(12, "\1\0\0\0\0\0\0\0", 8) = 8
[pid 30794] write(12, "\1\0\0\0\0\0\0\0", 8) = 8
[pid 30794] write(12, "\1\0\0\0\0\0\0\0", 8 <unfinished ...>
[pid 30794] <... write resumed> )       = 8
[pid 30794] write(12, "\1\0\0\0\0\0\0\0", 8 <unfinished ...>
[pid 30794] <... write resumed> )       = 8
[pid 30794] write(12, "\1\0\0\0\0\0\0\0", 8 <unfinished ...>
[pid 30794] <... write resumed> )       = 8
[pid 30794] write(12, "\1\0\0\0\0\0\0\0", 8) = 8
[pid 30794] poll([{fd=10, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=10, revents=POLLOUT}])
[pid 30794] writev(10, [{iov_base="\202\2\2\0\10\0\200\10+\0\1\0", iov_len=12}], 1) = 12
[pid 30794] futex(0x7f54ba35c338, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>

This isn't caused by imageio specifically, since the hang is reproducible with

def nop(): pass

Popen(["uname", '-a'], stdin=None, stdout=None, stderr=None, preexec_fn=nop)

but imageio seems to be the best place for a workaround.

Would you consider merging a pull request adding a prevent_sigint argument to write_frames (and read_frames for symmetry)? It would require changes to imageio too, since the arguments are passed explicitly in FfmpegFormat._initialize.

logger warnings are a bit much

There is a lot of warning output from the logger, and no apparent way to disable it.

There's a python library for warnings in the stdlib: https://docs.python.org/3/library/warnings.html

Using this instead of the root logger would make it easier to discard the logging warnings.

Why do you suggest to seed the generator when writing a video

In the example you suggest to do:

# Read a video file
reader = read_frames(path)
meta = reader.__next__()  # meta data, e.g. meta["size"] -> (width, height)
for frame in reader:
    ... # each frame is a bytes object
# Write a video file
writer = write_frames(path, size)  # size is (width, height)
writer.send(None)  # seed the generator
for frame in frames:
    writer.send(frame)
writer.close()  # don't forget this

However, it is unclear why one should "seed" the generator. Is this in the case that there are no frames?

Buffers not released on program shutdown

With the new release, we're getting errors in our CI workflow: pygfx/pygfx#287 (comment)

In one test, we try to run an example that uses imageio-ffmpeg in a subprocess, and that subprocess fails with:

Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedReader name=8> at interpreter shutdown, possibly due to daemon threads
E         Python runtime state: finalizing (tstate=0x5630592e3240)
E         
E         Current thread 0x00007f4ca6d0f740 (most recent call first):
E           File "/opt/hostedtoolcache/Python/3.9.12/x64/lib/python3.9/site-packages/imageio_ffmpeg/_io.py", line 365 in read_frames
E           File "/opt/hostedtoolcache/Python/3.9.12/x64/lib/python3.9/site-packages/imageio/plugins/ffmpeg.py", line 375 in _close
E           File "/opt/hostedtoolcache/Python/3.9.12/x64/lib/python3.9/site-packages/imageio/core/format.py", line 352 in close
E           File "/opt/hostedtoolcache/Python/3.9.12/x64/lib/python3.9/site-packages/imageio/core/format.py", line 341 in __del__

Reading video in local and docker-compose file works fine but in minikube below issue occured

Problem : based on my understanding ffmpeg-linux64-v4.2.2 needs some of depedancies which is i don't know
ffmpeg_reader.py used this ffmpeg-linux64-v4.2.2 for subprocess start more info : link line 257
minikube path : /usr/local/lib/python3.8/site-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2

Expected Behavior

>>> from moviepy.editor import *
>>> clip=VideoFileClip("https://videorenderingbucket.s3.ap-south-1.amazonaws.com/video_file/Pranali_Shukan_Wedding_Invite.mp4_video_2023-06-19_10%3A25%3A13.685572.mp4")
>>> clip
<moviepy.video.io.VideoFileClip.VideoFileClip object at 0x7f175c97ac40>

Actual Behavior

From moviepy. Editor import *
clip=videofileclip("https: //videorenderingbucket. S3. Ap-south-1. Amazonaws. Com/video_file/the+breathtaking+beauty+of+nature+-+hd. Mp4")`

### Logs
traceback (most recent call last):
file "/usr/local/lib/python3.8/site-packages/moviepy/video/io/ffmpeg_reader. Py", line 285, in ffmpeg_parse_infos
line = [l for l in lines if keyword in l][index]
indexerror: list index out of range

during handling of the above exception, another exception occurred:

traceback (most recent call last):
file "<stdin>", line 1, in <module>
file "/usr/local/lib/python3.8/site-packages/moviepy/video/io/videofileclip. Py", line 88, in __init__
self. Reader = ffmpeg_videoreader(filename, pix_fmt=pix_fmt,
file "/usr/local/lib/python3.8/site-packages/moviepy/video/io/ffmpeg_reader. Py", line 35, in __init__
infos = ffmpeg_parse_infos(filename, print_infos, check_duration,
file "/usr/local/lib/python3.8/site-packages/moviepy/video/io/ffmpeg_reader. Py", line 289, in ffmpeg_parse_infos
raise ioerror(("moviepy error: failed to read the duration of file %s. \n"
oserror: moviepy error: failed to read the duration of file https: //videorenderingbucket. S3. Ap-south-1. Amazonaws. Com/video_file/the+breathtaking+beauty+of+nature+-+hd. Mp4.
Here are the file infos returned by ffmpeg:

ffmpeg version 4.2.2-static https: //johnvansickle. Com/ffmpeg/ copyright (c) 2000-2019 the ffmpeg developers
built with gcc 8 (debian 8.3.0-6)
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100`

Specifications

• Python Version: 3.8.10
• MoviePy Version:1.0.3
• Platform Name: Minikube
• Platform Version: v1.30.1

Requirement.txt

• aiofiles==23.1.0
• autopep8==2.0.2
• bcrypt==4.0.1
• boto3==1.26.144
• chill==0.9.0
• cloudpickle==2.2.1
• cryptography==41.0.1
• fastapi==0.95.0
• ffmpeg-python==0.2.0
• ffpyplayer==4.4.0
• flower==1.2.0
• httpx==0.23.3
• ipykernel==6.22.0
• moviepy==1.0.3
• mysql-connector-python==8.0.33
• opencv-python==4.7.0.72
• pandas==1.5.3
• pip-chill==1.0.1
• pyjwt==2.7.0
• pymysql==1.0.3
• python-decouple==3.8
• python-dotenv==1.0.0
• python-multipart==0.0.6
• rabbitmq==0.2.0
• redis==4.5.4
• sqlalchemy==2.0.12
• uvicorn==0.21.1

Unable to find ffmpeg binary x86_64

Hello,

Is there a ffmpeg binary available for this architecture?

I'm running across what seems to be an executable issue(error below for reference) and it's possible that the ffmpeg binary that I'm using might not be correct. My program is being deployed using a lambda environment with a runtime setting of python3.9 and x86_64 architecture

I tried both the most recent snapshot and release from this link.

Thanks,
Zack

[ERROR] OSError: MoviePy error: failed to read the duration of file /tmp/magic-assets-public-drops---sample-video.mp4.
Here are the file infos returned by ffmpeg:

-i: /var/task/ffmpeg: cannot execute binary file

Traceback (most recent call last):
  File "/var/task/CreateThumbnail.py", line 99, in handler
    create_video_thumbnail(bucket, key, file_name, extension)
  File "/var/task/CreateThumbnail.py", line 72, in create_video_thumbnail
    resize_video(thumb_download_path, thumb_upload_path)
  File "/var/task/CreateThumbnail.py", line 30, in resize_video
    with VideoFileClip(video_path) as video:
  File "/var/task/moviepy/video/io/VideoFileClip.py", line 88, in init
    self.reader = FFMPEG_VideoReader(filename, pix_fmt=pix_fmt,
  File "/var/task/moviepy/video/io/ffmpeg_reader.py", line 35, in init
    infos = ffmpeg_parse_infos(filename, print_infos, check_duration,
  File "/var/task/moviepy/video/io/ffmpeg_reader.py", line 296, in ffmpeg_parse_infos
    raise IOError(("MoviePy error: failed to read the duration of file %s.\n"

Create manifest file

To include license file, for e.g. conda package.

Set ffmpeg_timeout to zero by default.

On an Ubuntu 18.04 VM, I can easily reproduce ffmpeg being killed using this script:

import imageio_ffmpeg
import numpy as np

ims = [np.random.uniform(0, 255, size=(1000, 1000, 3)).astype(np.uint8) for i in range(10)]

w = imageio_ffmpeg.write_frames("/home/almar/Desktop/foo.mp4", (1000, 1000), ffmpeg_timeout=0)
w.send(None)
for i in range(200):
    w.send(ims[i % 10])
    print(i)
w.close()
print("closed")

If we can assume ffmpeg to not hang doing nothing, it would be much better to set the default timeout to zero (i.e. wait until ffmpeg is done). However, we probably want to combine this with proper keyboardinterrupt handling #4.

Hang in subprocess communicate

I'm getting a weird hang when using this format in imageio (0.3.0).

py-spy --dump --pid 43792
Thread 0x7FA50A5B7700 (active)
         run (imageio_ffmpeg/_parsing.py:61)
         _bootstrap_inner (threading.py:917)
         _bootstrap (threading.py:885)
Thread 0x7FA57CB96700 (active)
         _communicate (subprocess.py:1701)
         communicate (subprocess.py:939)
         read_frames (imageio_ffmpeg/_io.py:193)
         _close (imageio/plugins/ffmpeg.py:342)
         close (imageio/core/format.py:252)
....

It looks like it's blocking forever trying to send "q" to the child process, should that have a timeout on it? I'm not sure what causes this issue but it seems to happen unreliably in my script. Creating a reduced version would be challenging because it is rare but the script runs for a long time.

I see that there is some code to just kill the subprocess, but that code is disabled.

threading.Thread.setDaemon has been deprecated in favor of setting daemon attribute directly in Python 3.10

Ref : python/cpython#25174

$ rg -t py -w 'currentThread|notifyAll|activeCount|isDaemon|setDaemon' 
imageio_ffmpeg/_parsing.py
21:        self.setDaemon(True)

Video rotation metadata

I am getting this warning when reading videos from mobile phones

imageio_ffmpeg - WARNING - The frame size for reading (480, 720) is different from the source frame size (720, 480).

This means I have to rotate frames either 90º or 270º.

ffmpeg -i provides such rotation info of videos, For example:

ffmpeg -i myrotatedMOV.mov
....

Duration: 00:00:14.31, start: 0.000000, bitrate: 778 kb/s
    Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 480x360, 702 kb/s, 29.98 fps, 30 tbr, 600 tbn, 1200 tbc
    Metadata:
      rotate          : 180
      creation_time   : 2013-01-09 12:47:36
      handler_name    : Core Media Data Handler
    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, s16, 62 kb/s
    Metadata:
      creation_time   : 2013-01-09 12:47:36
      handler_name    : Core Media Data Handler

How can I get such information with imageio_ffmpeg?
Tried get_meta_data() function but is not there

ffmpeg not included in wheel for MacOS M1

The readme on the main page states that ffmpeg is included with the install from pypi. However after installation I get the error:

RuntimeError: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.

Not sure if it may be affecting it, but I am installing the package to a virtual environment.

Clarification on minimum requirements in terms of ffmpeg features

I was interested in getting a hardware accelerated version of ffmpeg running on google colab that I could use through imageio-ffmpeg.

I followed the steps discussed here to compile a minimal ffmpeg version with hardware acceleration on colab:
https://pytorch.org/audio/stable/hw_acceleration_tutorial.html
https://colab.research.google.com/drive/1DDah_IaGULEO66CfQWltRqaVheBkiXdN#sandboxMode=true

Unfortunately, this led to "broken pipe" issues. It took me a bit of time to track this down to features that were missing for compatibility with imageio-ffmpeg in the minimal ffmpeg configure step. Adding the following seems to work so far but I am not sure if this what should be recommended:

• --enable-demuxer=rawvideo
• --enable-muxer=null
• --enable-filter=null
• --enable-filter=nullsrc
• --enable-protocol=pipe

Some documentation about this and/or a short test function that checks the output of ffmpeg -demuxers, ffmpeg -muxers, ffmpeg -filters, ffmpeg -protocols and the like for the minimum set of features would be great.

Setup requires pip > 19

We are building imageio-ffmpeg from source with Python 3.6, which comes bundled with Pip 18.
Imageio-ffmpeg requires pip > 19.

The problem is that imageio-ffmpeg won't build, even from source, because we don't meet the pip requirements. Technically, imageio-ffmpeg can be built and installed from source without pip.

We work in an offline environment, so I can work around this by downloading an updated version of Pip from a computer with network access, and then transferring it to the offline computers, but it would be nice to have that build requirement removed, or even downgraded, since it's not technically necessary.

Calling `imageio_ffmpeg.get_ffmpeg_exe()` under CygWin raises an exception

Calling imageio_ffmpeg.get_ffmpeg_exe() under CygWin raises an exception.

$ python3
Python 3.6.8 (default, Feb 14 2019, 22:09:48)
[GCC 7.4.0] on cygwin
>>> import imageio_ffmpeg
>>> imageio_ffmpeg.get_ffmpeg_exe()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/site-packages/imageio_ffmpeg/_utils.py", line 36, in get_ffmpeg_exe
    if plat.startswith("win"):
AttributeError: 'NoneType' object has no attribute 'startswith'
>>>

The get_platform function, which is used to set plat, seems to work fine with a normal import.

>>> import sysconfig
>>> sysconfig.get_platform()
'cygwin-3.0.6-x86_64'

Unable to find a suitable output format for 'temp-file'

I am trying to read a video file and write it to a socket using the code below and it is failing. I would appreciate your help:

async def write_video_to_stream(path_to_video, path_to_socket):
    video_reader = imageio.get_reader(path_to_video)
    sock = await create_socket(path_to_socket)
    _, socket_writer = await asyncio.open_unix_connection(sock=sock)
    img_writer = imageio.get_writer(socket_writer, format='FFMPEG', ffmpeg_log_level='debug')
    for frame in video_reader:
        img_writer.append_data(frame)
        #await socket_writer.drain()
    img_writer.close()
    socket_writer.close()

The code is failing with the stacktrace below (debug logging turned on):

(object-detection) mugo@eric-aspire:~/ai-object-detection/detection-visualization$ python socket_stream.py ~/Videos/dog-rescue-water.mp4 /tmp/sock2
ffmpeg version 4.1-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'rawvideo'.
Reading option '-s' ... matched as option 's' (set frame size (WxH or abbreviation)) with argument '400x400'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'rgb24'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '10.00'.
Reading option '-i' ... matched as input url with argument '-'.
Reading option '-an' ... matched as option 'an' (disable audio) with argument '1'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'libx264'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv420p'.
Reading option '-crf' ... matched as AVOption 'crf' with argument '25'.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'debug'.
Reading option '/tmp/imageio_8sv61nse' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option v (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url -.
Applying option f (force format) with argument rawvideo.
Applying option vcodec (force video codec ('copy' to copy stream)) with argument rawvideo.
Applying option s (set frame size (WxH or abbreviation)) with argument 400x400.
Applying option pix_fmt (set pixel format) with argument rgb24.
Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 10.00.
Successfully parsed a group of options.
Opening an input file: -.
[rawvideo @ 0x6584d00] Opening 'pipe:' for reading
[pipe @ 0x6585680] Setting default whitelist 'crypto'
[rawvideo @ 0x6584d00] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[rawvideo @ 0x6584d00] All info found
[rawvideo @ 0x6584d00] After avformat_find_stream_info() pos: 480000 bytes read:480000 seeks:0 frames:1
Input #0, rawvideo, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: 38400 kb/s
    Stream #0:0, 1, 1/10: Video: rawvideo, 1 reference frame (RGB[24] / 0x18424752), rgb24, 400x400, 0/1, 38400 kb/s, 10 tbr, 10 tbn, 10 tbc
Successfully opened the file.
Parsing a group of options: output url /tmp/imageio_8sv61nse.
Applying option an (disable audio) with argument 1.
Applying option vcodec (force video codec ('copy' to copy stream)) with argument libx264.
Applying option pix_fmt (set pixel format) with argument yuv420p.
Successfully parsed a group of options.
Opening an output file: /tmp/imageio_8sv61nse.
[NULL @ 0x65870c0] Unable to find a suitable output format for '/tmp/imageio_8sv61nse'
/tmp/imageio_8sv61nse: Invalid argument
[AVIOContext @ 0x6585500] Statistics: 480000 bytes read, 0 seeks
Fatal read error on socket transport
protocol: <asyncio.streams.StreamReaderProtocol object at 0x7fa09efcc2b0>
transport: <_SelectorSocketTransport fd=7 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/asyncio/selector_events.py", line 801, in _read_ready__data_received
    data = self._sock.recv(self.max_size)
OSError: [Errno 22] Invalid argument
Traceback (most recent call last):
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/site-packages/imageio_ffmpeg/_io.py", line 411, in write_frames
    p.stdin.write(bb)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "socket_stream.py", line 33, in <module>
    asyncio.run(write_video_to_stream(ns.path_to_video, ns.path_to_socket))
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "socket_stream.py", line 11, in write_video_to_stream
    img_writer.append_data(frame)
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/site-packages/imageio/core/format.py", line 492, in append_data
    return self._append_data(im, total_meta)
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/site-packages/imageio/plugins/ffmpeg.py", line 572, in _append_data
    self._write_gen.send(im)
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/site-packages/imageio_ffmpeg/_io.py", line 418, in write_frames
    raise IOError(msg)
OSError: [Errno 32] Broken pipe

FFMPEG COMMAND:
/home/mugo/miniconda3/envs/object-detection/lib/python3.7/site-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.1 -y -f rawvideo -vcodec rawvideo -s 400x400 -pix_fmt rgb24 -r 10.00 -i - -an -vcodec libx264 -pix_fmt yuv420p -crf 25 -v debug /tmp/imageio_8sv61nse

FFMPEG STDERR OUTPUT:

Fatal Python error: could not acquire lock for <_io.BufferedReader name=11> at interpreter shutdown, possibly due to daemon threads

Thread 0x00007fa0a65a0700 (most recent call first):
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/site-packages/imageio_ffmpeg/_parsing.py", line 61 in run
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/threading.py", line 917 in _bootstrap_inner
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/threading.py", line 885 in _bootstrap

Current thread 0x00007fa0aad6d740 (most recent call first):
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/subprocess.py", line 1704 in _communicate
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/subprocess.py", line 939 in communicate
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/site-packages/imageio_ffmpeg/_io.py", line 193 in read_frames
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/site-packages/imageio/plugins/ffmpeg.py", line 343 in _close
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/site-packages/imageio/core/format.py", line 252 in close
  File "/home/mugo/miniconda3/envs/object-detection/lib/python3.7/site-packages/imageio/core/format.py", line 241 in __del__
Aborted (core dumped)

imageio-ffmpeg depends on setuptools

ImageIO has an unspecified runtime dependency on setuptools.
Usage is here:

from pkg_resources import resource_filename

This is an issue for dependency managers like pdm that don't assume setuptools must be installed. This fix should be as simple as:

-     install_requires=[]
+     install_requires=['setuptools']

unbuffered reads from ffmpeg

I don't have time to test and to work on this much, but I think that the code to read in data from ffmpeg can be optimized slightly.

The inefficiencies probably stem from the call to read_n_bytes, which uses reads in a string (an immutable type), then converts to a numpy buffer (by copying the memory).

I found that for my 1048x1328x3 frames, it was able to speed things from reading in a tight loop at 283 fps to 293 fps.

Marginal gain, but maybe somebody needs it. Maybe this can make a bigger difference depending on the workload/hardware/decoding process.

Here is a rough sketch of the patch.
You need to set the input to unbuffered see bug report on numpy below.

Patch skeleton

diff --git a/imageio/plugins/ffmpeg.py b/imageio/plugins/ffmpeg.py
index 83f9a9f..5d81546 100644
--- a/imageio/plugins/ffmpeg.py
+++ b/imageio/plugins/ffmpeg.py
@@ -471,7 +471,8 @@ class FfmpegFormat(Format):
             # For Windows, set `shell=True` in sp.Popen to prevent popup
             # of a command line window in frozen applications.
             self._proc = sp.Popen(
-                cmd, stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.PIPE, shell=ISWIN
+                cmd, stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.PIPE, shell=ISWIN,
+                bufsize=0,
             )
 
             # Create thread that keeps reading from stderr
@@ -611,7 +612,9 @@ class FfmpegFormat(Format):
             w, h = self._meta["size"]
             framesize = self._depth * w * h * self._bytes_per_channel
             assert self._proc is not None
-
+            s = np.fromfile(self._proc.stdout, dtype=np.uint8,
+                count=framesize)
+            return s, True
             try:
                 # Read framesize bytes
                 if self._frame_catcher:  # pragma: no cover - camera thing
@@ -644,8 +647,9 @@ class FfmpegFormat(Format):
             w, h = self._meta["size"]
             # t0 = time.time()
             s, is_new = self._read_frame_data()
-            result = np.frombuffer(s, dtype=self._dtype).copy()
-            result = result.reshape((h, w, self._depth))
+            result = s.reshape(h, w, self._depth)
+            # result = np.frombuffer(s, dtype=self._dtype).copy()
+            # result = result.reshape((h, w, self._depth))
             # t1 = time.time()
             # print('etime', t1-t0)

I'm not too sure of the other performance implications of buffered vs unbuffered reads. Investigating that will take more time than I have. Maybe other parts of the code can benefit from this kind of stuff.

numpy/numpy#12309

ffprobe

I just happened across this again. Not sure how I lost it, but it would be handy to use FFPROBE for metadata

related: Zulko/moviepy#249

When building a pyinstaller package on Windows with --windowed option, the packaged app fails to start

After investigating the reasons for such behaviour (without --windowed everything works fine), I narrowed it down to this part of code in _io.py

p = subprocess.Popen(
    cmd,
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    stderr=None,
    **_popen_kwargs(prevent_sigint=True)
)

The comment that follows explains that stderr was deliberately set to None because if redirected to pipe on Windows, it causes ffmpeg to hang. On the other hand, stderr=None causes a problem with --windowed option on Windows (OSError: [WinError 6] The handle is invalid). My understanding is that Windows is trying to recreate stderr handle as it is set to None, but unable to do this due to --windowed mode.

A possible solution is to redirect stderr (and probably stdout too) to subprocess.DEVNULL instead of None, i.e.

stderr=subprocess.DEVNULL