vaida12345 / waifuextension

The waifu2x & Other image-enlargers on Mac

waifuExtension

The waifu2x on Mac.

The new version is capable of taking advantages of CPU, GPU, and ANE.

Usage

• Enlarge videos or images with machine learning on Mac.
• Interpolate frames for videos.

Install

Files and source code could be found in releases.

Note: If mac says the app was damaged / unknown developer, please go to Finder, right click on the App, and choose open.

Privacy

This app works completely offline and requires no internet connection. Nothing is collected or stored, expect for:

• Your settings stored in its containter.
• Temp images during processing in its containter, the existance would only last for three lines of code, after which it is deleted.

If the app crashes, please choose not to share crash log with Apple.

Models

The waifu2x models where obtained from waifu2x-caffe, and translated to coreML via coremltools.

Other models are:

• dain-ncnn-vulkan
• realsr-ncnn-vulkan
• cain-ncnn-vulkan
• realcugan-ncnn-vulkan
• Real-ESRGAN
• rife-ncnn-vulkan

Note

This app was based on the work of waifu2x-ios. Nearly all the files in the folder "waifu2x-mac" were created by him. However, modifications were done to improve speed.

Speed

When processing a standard 1080p image (1920 × 1080) using Waifu2x Caffe, MacBook Pro with the M1 Max chip took only 0.7 seconds.

Interface

This app was written with SwiftUI.

Preview

Credits

• waifu2x-ios for nearly all the algorithms used to enlarge images.
• waifu2x-caffe for all the models.
• stack overflow for all the solutions.
• dain-ncnn-vulkan for dain-ncnn-vulkan.
• realsr-ncnn-vulkan for realsr-ncnn-vulkan.
• cain-ncnn-vulkan for cain-ncnn-vulkan.
• realcugan-ncnn-vulkan for realcugan-ncnn-vulkan.
• Real-ESRGAN for Real-ESRGAN.
• rife-ncnn-vulkan for rife-ncnn-vulkan.

waifuextension's Issues

Allow option to store intermediate files in memory

A video that takes around 1 hour in waifu2x MAS to upsample (720p→1440p) is estimated to take 6-10 hours in waifuExtension. I guess one reason is that temp files are written to disk and read when needed. I have m1 max with 64 GB RAM, and the program say that it will need around 30 GB of disk space, so theoretically waifuExtension can just put all temp files in RAM and I suppose that will increase the speed. Is this practical to implement as an option?

Ver 4.1.3 App is not functioning properly after Real-ESRGAN model update

Macbook Pro M1 Max

Describe the bug



The Real-ESRGAN model has been loaded successfully, the Model Manager status is displayed as true, but the application fails to run, and none of images to be processed.

To Reproduce
I used the model linked below.
https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesrgan-ncnn-vulkan-20220424-macos.zip

However, if I change back to the old version of the model in the link below, the application will work properly.
https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.3.0/realesrgan-ncnn-vulkan-20211212-macos.zip

Suggestion

The 'Time Remaining' and 'ETA' have never been accurate, sometimes cause crash, and the progress bar blow ‘ETA' has never accurately expressed the actual processing progress. It is recommended to remove these items.

When the portable binary of the Real-ESRGAN model runs, the completion percentage will be displayed. I would suggest to display this percentage，or the progress bar (or other chart) created by these percentages in the app.

Looking forward to the next update of the app. Thank you.

Hope this prj is able to support Homebrew

As the title said, thanks anyway ~

[feature request]add homebrew support

Would you join homebrew?
It's easy way to help us install/uninstall/upgrade app.

Various crashes

I have no idea why they happened and don't have the time to reproduce as they occur 1~3 hours into an operation. By the way, is there a way to recover/resume a session?
waifuextension crash.zip

m

Describe the bug
Please give a description of what the bug is.

To Reproduce
Please provide the information about:

• Image Model
• Frame Interpolation Model (if used)
• Video or Image

Attachments
If you find providing attachments useful, please provide the images / videos you wished to enlarge, and the bug report.

Additional context
Add any other context about the problem here.

Custom model on realesrgan-ncnn-vulkan

Hi there,

Is there any chance let user add custom model (both *.param and *.bin) to realesrgan-ncnn-vulkan/models and selectable in dropdown list?

I'm giving a try, but when I build this project, it shows "No such module 'Support'". Looks like I have some dependency problem. :(

"done" button disappeared

Describe the bug
Please give a description of what the bug is.

To Reproduce
Please provide the information about:

• Image Model
• Frame Interpolation Model (if used)
• Video or Image

Attachments
If you find providing attachments useful, please provide the images/videos you wished to enlarge and the bug report.

Additional context
Add any other context about the problem here.
System: macOS 13.0 (22A380)

wish add realesrgan-ncnn-vulkan model in the app

wish add realesrgan-ncnn-vulkan model in the app

Crashes when starting to process

Describe the bug
This app crashes when trying to enlarge a video file on macOS Sonoma, M1 Max. The video file was mp4 format at 720p. The crash log is available.

To Reproduce
Please provide the information about:

• Image Model: any
• Frame Interpolation Model (if used): never was available
• Video or Image: video

Attachments
https://github.com/Vaida12345/waifuExtension/assets/48839145/d6091133-ab5e-41ef-adc5-d051635d2b27
crash.log

Additional context
Add any other context about the problem here.

Missing Support Framework

Describe the bug
When running the Xcode project, it says: ~/waifuExtension/waifuExtension.xcodeproj Missing package product 'Support'

Where could I find the Support framework or library?

TMP folder unable to be accessed

heylo!

i'm getting this error:

" You don’t have permission to save the file “test.m4v” in the folder “tmp”. "

unless i use Waifu... then i get this error:

"PixelBufferPool nil after starting session"

MacOS 13.4.1
Mac Mini M1 16GB

thanks!

Maybe we still need a older version for macOS 11.x

Added models don't show up available list

Hi there,

I have tried the models included by default in the App and it worked without problem.
I then decided to add the 3 other models listed in preferences: I downloaded the releases containing the excecutables and added them into the App in /Content/Ressources.

The models were marked as imported I guess since the check mark showed ✓.
However, when upscaling an image, only the 4 default ones are shown and I don't see the 3 new ones (CAIN, DAIN, RIFE).

5.0beta2 is not working properly on M1 Max Macbook Pro

Describe the bug
Everything is normal from importing the model to processing, but after processing, there is no result that should be output in the output folder. The same problem also exists in the previous version.

To Reproduce

• Image Model - Real ESRGAN model

Attachments

Additional context
Since other models are rarely used at ordinary times, whether other models also have the same problem has not been tested.

A little suggestion
Although the built-in model will cause the application size to be too large, I hope the release page has a full version of all built-in models and a lightweight version without built-in models, two versions to choose. I personally think that compared with not using this application well, the application size may be not the most important problem to me.

macOS 12.0.1 or later

Hello,
is there a reason for requiring macOS 12.0.1 or later? Could it be compiled to run on Big Sur?
Regards,
Bogdan

No output with any method other than waifu2x on latest (beta) release

Describe the bug
No output. Only a file Icon? is generated then deleted. Libraries seems properly linked as folder sizes are given.
Waifu2x is okay anyway.

To Reproduce

• Real-ESRGAN,Real-SR,Real-CUGAN, latest releases just downloaded fresh.
• Frame Interpolation Model (if used) Not used
• any picture
  Choose a pic, choose an algo other than waifu2x, and watch the output folder.

Additional context
MacOS 12.4, arm64 Macbook, 5.0b1, 4.1.3

chmod library executives to 777 won't help
toggling Parallel and TTA won't help

[Reminder] Don't use MIT license for a complete app

I'm the author of waifu2x-mac. I worked on this project back in 2017 but stopped later. Guess why? I also used MIT license and someone took my code and published the app almost as-is on the Mac App Store and charged for $3 (archived page). I'm not gonna to work on a project and let some random person make free money out of it.

If you are building a full app and publish under MIT license, it's almost guaranteed that someone will do the same and publish your app to MAS without even modifying the UI, then charge a premium. And it can be even worse: they can add much more features to the app and do not contribute a single line of code back to your project.

If you are not going to publish this app to MAS by yourself, please use GPL license.

In any case, if you don't feel like working for some random person for free and let them take all the income, please avoid MIT license.

hope the model will be updated to the coreML version of Apple silicon (someone provides a ready-made model)

https://github.com/john-rocky/CoreML-Models
This repository provides a coreML version of almost all models, hoping to produce a separate application version of waifuExtension for Apple silicon to improve efficiency by using the coreML models.

A suggestion about the app running on M1.

The Real-CUGAN model runs slowly on M1. Hope to use ANE on Real-CUGAN.
Thanks!

Miss support files

Describe the bug
The project depend on relative path "../../Swift/testRoom/testRoom/Source/Support" and XCode reports "Build input files cannot be found", should those files be upload please 😁

To Reproduce