Comments (14)
I have a fix for this, but I really can't figure out how to commit changes. The
problem appears to be that for some reason which I can't figure out, using the
Win32
API doesn't buffer, so each of the tiny reads goes back to the disk. What I
did was
incorporate a 4K buffer in the file_win32.cpp file. It makes it much, much
faster
in both reading and writing. I've attached my file. Trying building with it
and see
if it fixes the performance problem.
Dan
Original comment by [email protected]
on 8 Feb 2010 at 4:03
Attachments:
from mp4v2.
jeffreyloden,
Could you try this again with the most recent trunk? And is there some way you
could give me your test code so I could try and repeat it? There were some
more changes to the native windows functions (mostly to deal with unicode), but
I'd like to know if you still have the same issues.
One thought I had is to improve the file provider abstraction so users can more
easily create their own read/write routines (in particular, I'd like to write
one using Qt).
Thanks,
kidjan
Original comment by [email protected]
on 22 Sep 2010 at 5:34
from mp4v2.
Just to see, I tried this with the most current rev, and the problem still
exists. Not sure if there are any developers that run on Windows, but if so,
my fix greatly speeds up reads and writes. I would be glad to work with anyone
that would be interested in getting this into the sources.
Very easy to test, on Windows run a read and a write with and without the
changes and notice the speedup. I've got thousands of users using my version
and I've never had a problem with it.
Dan
Original comment by [email protected]
on 18 Mar 2012 at 10:54
from mp4v2.
What happens if you change the flags used? For example, see:
http://msdn.microsoft.com/en-us/library/aa363858.aspx#caching_behavior
...in particular,
"Specifying the FILE_FLAG_SEQUENTIAL_SCAN flag can increase performance for
applications that read large files using sequential access. Performance gains
can be even more noticeable for applications that read large files mostly
sequentially, but occasionally skip forward over small ranges of bytes. If an
application moves the file pointer for random access, optimum caching
performance most likely will not occur. However, correct operation is still
guaranteed."
Looks like the code is currently using FILE_ATTRIBUTE_NORMAL (see
http://code.google.com/p/mp4v2/issues/attachmentText?id=50&aid=-8024171265002390
40&name=File_win32.cpp&token=4idgdzVy-fM3qodZaF02-OC04Xs%3A1332118380013#46 );
might be interesting to see what happens if you put in
FILE_FLAG_SEQUENTIAL_SCAN since most MP4 file reading probably "read(s) large
files mostly sequentially, but occasionally skip forward over small ranges of
bytes"
Not opposed to the patch, but I'd prefer a solution that leveraged the existing
file API rather than caching in user land.
Original comment by [email protected]
on 19 Mar 2012 at 1:32
from mp4v2.
Also, this sort of sheds some light on this API:
http://blogs.msdn.com/b/oldnewthing/archive/2012/01/20/10258690.aspx
So, the weird thing is if an application passes in neither
FILE_FLAG_SEQUENTIAL_SCAN or FILE_FLAG_RANDOM_ACCESS, then the underlying API
goes into some sort of heuristic mode where it tries to optimize performance
based on the calls coming in. Also, it seems that this behavior may not be
consistent across Windows OSs.
It has to be said: what a *shit* API this is. The documentation alone really
shows you how convoluted it is. I wouldn't be opposed to going back to the
more portable way (fopen/fread/fseek) and doing away with this windows specific
file provider all together.
Original comment by [email protected]
on 19 Mar 2012 at 1:45
from mp4v2.
Going back to fopen/fread/fseek has some pretty big drawbacks too: can't read
filenames with non-ascii characters and can't read filenames that live in long
paths.
I've been super delinquent in not trying Dan's patch for ages. There may be
some light at the end of the tunnel for me but given my past performance, is
there someone else who can take a look at his patch and work through any
changes (if there are any) with him and get it committed?
Thanks much.
-DB
Original comment by [email protected]
on 19 Mar 2012 at 3:19
from mp4v2.
Before we do any patches, somebody needs to at least try
FILE_FLAG_SEQUENTIAL_SCAN to see if that resolves the performance issues.
Original comment by [email protected]
on 19 Mar 2012 at 3:59
from mp4v2.
@dbyron,
I was under the impression that fopen accepts UTF8? If that's true, seems like
you could read filenames with non-ascii characters, but maybe I'm misinformed
about this.
Original comment by [email protected]
on 21 May 2012 at 4:58
from mp4v2.
I don't think fopen accepts UTF-8. See the attached test program. I created
three text files corresponding to the three files it tries to open. fopen
opens the one with the ascii name, but not the ascii ones.
Original comment by [email protected]
on 21 May 2012 at 8:08
Attachments:
from mp4v2.
Hi,
I use _wfopen() to handle Unicode filenames.
_wfopen() can't handle very long paths (longer than MAX_PATH characters), but I
think it's acceptable because Explorer can not also handle such long paths.
--
Ken Takata
Original comment by [email protected]
on 10 Jul 2013 at 11:49
Attachments:
from mp4v2.
I just tested out using _wfopen, and it has as good a performance as my fix,
and since it's an OS API instead of my home rolled buffering, it seems like the
right answer. Can someone put this in the next release?
Dan
Original comment by [email protected]
on 5 Aug 2013 at 4:09
from mp4v2.
happily, if I can get it as a diff?
Original comment by [email protected]
on 16 Aug 2013 at 8:48
from mp4v2.
OK, I have attached the diff.
--
Ken Takata
Original comment by [email protected]
on 16 Aug 2013 at 9:47
Attachments:
from mp4v2.
Hi,
I have updated the patch for R507.
--
Ken Takata
Original comment by [email protected]
on 13 Jan 2015 at 12:52
Attachments:
from mp4v2.
Related Issues (20)
- MP4Info not showing information related to MJPEG Video
- Can't compile with clang
- mp4info fails to show info of the simple video taken by Android phone HOT 1
- Error when trying to compile the project mp4v2 - 2.0.0 HOT 1
- Writing tags corrupts DASH-encoded files
- Integer overflow/wraparound in \src\mp4track.cpp
- Better FileProvider API HOT 3
- Modifying tags with mp4tags -m "test" corrupts some specific MP4 files
- add compilation, gapless, and genretype tag support to mp4tags
- https://code.google.com/p/mp4v2/issues/list?can=1&q=http%3A%2F%2Flimelinx.com%2Fdswj4
- Enhancement: Fill Genre from GenreType
- "Progressive Rock" mistyped as "Progresive Rock" in src/itmf/type.cpp HOT 1
- Ability to free memory allocated by MP4GetTrackH264SeqPictHeaders function HOT 2
- Support for Track Reference Atoms HOT 1
- Import/Export mp4 tags from/to text file
- make dist fail because make doc fails HOT 2
- use of impl / private code in example utils; enumGenreType HOT 1
- Charset problem at filenames
- Generate mp4 file.Can't play by media player.
- Google Code will shut down
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mp4v2.