Unrecovered memory growth (leak?) when loading spine items. about sdklauncher-ios

[danielweck]

Hello @nodehack thank you for your detailed report.
What iOS version? (hardware device, or emulator?)

At any rate, it sounds like you may be affected by either ; or all ; of these known bugs (ongoing investigation):

1. LibXML parser -related memory leaks, try the Pull Request to see if it fixes the problem):
   readium/readium-sdk#161

2. WebCore::RenderSVGRoot issue? (probably not, as this seems to be SVG specific):
   https://groups.google.com/d/msg/readium-dev/OiY2iOtpC58/WGlIofzgISwJ
   CC @remipraud

3. UIWebview vs. WKWebView (apparently, the former leaks lots of memory, even iBooks crashes):
   https://groups.google.com/forum/#!msg/readium-dev/PQZDjeeNwsk/KfyxtNwPXnsJ
   CC @clabordrie

[nodehack]

Hi Daniel,
Both the iPad2 and simulator I see this on are running iOS 8.1 (8.1.3 on the iPad2 to be specific).
Thanks for the suggestions. I'll give that LibXML parser a shot and see if that fixes this and report back.

[nodehack]

I refactored my application over to use the new WKWebview and the memory leak is gone. I would suggest anyone who can write their app for iOS 8 or higher to use a WKWebview. It seems to load faster and any memory it allocates for spine changes is immediately recovered.

Note that if you are going to use a WKWebview that you'll need to serve both the epub files and the readium js/css/html files via the internal http server or you'll get CORS errors.

Hi Tom

You have been faster than me :)

Yes it is not possible to load files locally with WKWebView

Would it be possible to have your changes, i have started it but i was so busy i could not finished
It would save me times

Cyril

Cyril Labordrie

Arutua Studio
[email protected]
[phone]
[phone]
www.arutua.com

Le 6 mars 2015 à 18:09, Tom Feldmann [email protected] a écrit :

I refactored my application over to use the new WKWebview and the memory leak is gone. I would suggest anyone who can write their app for iOS 8 or higher to use a WKWebview. It seems to load faster and any memory it allocates for spine changes is immediately recovered.

Note that if you are going to use a WKWebview that you'll need to serve both the epub files and the readium js/css/html files via the internal http server or you'll get CORS errors.

—
Reply to this email directly or view it on GitHub.

[bluefirepatrick]

Is there a pull request for this refactoring? I would be interested in seeing it.

Thanks,
Patrick

Sent from my iPad

On Mar 6, 2015, at 9:09 AM, Tom Feldmann <[email protected]mailto:[email protected]> wrote:

I refactored my application over to use the new WKWebview and the memory leak is gone. I would suggest anyone who can write their app for iOS 8 or higher to use a WKWebview. It seems to load faster and any memory it allocates for spine changes is immediately recovered.

Note that if you are going to use a WKWebview that you'll need to serve both the epub files and the readium js/css/html files via the internal http server or you'll get CORS errors.

Reply to this email directly or view it on GitHubhttps://github.com//issues/48#issuecomment-77595766.

[nodehack]

The application I'm writing is part of an R&D effort Follett School Solutions is doing to evaluate new epub technologies..so that code I cannot share. However, I've been given approval for refactoring the SDKLauncher to use the WKWebView. I'm hoping to have a branch up with my changes sometime on Monday.

[danielweck]

Thanks @nodehack
Important note about "malicious content":
The nature of the cross-origin boundary between file:// and http:// allowed us to exploit some sort of "loophole" (in iOS, OSX and Android, not sure about Windows). That is to say: Readium's file://path/to/reader.html benefits from read and write credentials over the http:// EPUB HTML Content Documents, but not the other way around. This makes it possible to inject behaviours, annotations, etc. from the application level into spine items, whilst at the same time making it impossible for EPUB HTML files to hijack the parent context.

So, because of WKWebView's all http:// approach, EPUB HTML spine items can now access window.parent/top and do all sorts of nasty things, as demonstrated by this test EPUB:
https://github.com/danielweck/epub-reading-system-js-sandbox-test
More details in this Google shared doc (note that sandboxing the iframe is not an acceptable solution, as this prevents Readium from injecting behaviours into spine items):
https://docs.google.com/document/d/1GK1aVsrTv23WroBWMX-XiwYtXbq6huW_pK8QXRaY6XQ

The solution used in the Chrome extension and the cloud reader (both of which serve content from the same domain as the app itself) is to override window.parent/top before spine item scripts can use it:

window.parent = window.self;
window.top = window.self;

Chrome extension:
epubReadingSystem.js: https://github.com/readium/readium-js-viewer/blob/develop/chrome-app/epubReadingSystem.js
module.config().epubReadingSystemUrl: https://github.com/readium/readium-js-viewer/blob/develop/lib/workers/ContentTransformer.js#L49
https://github.com/readium/readium-js-viewer/blob/develop/chrome-app/extended-worker-config.js#L18

Cloud reader:
_contentDocumentTextPreprocessor: https://github.com/readium/readium-js/blob/develop/epub-modules/epub-fetch/src/models/iframe_zip_loader.js#L150
https://github.com/readium/readium-js/blob/develop/epub-modules/Readium.js#L34

Caveat: IIRC, window.top cannot be overridden, so the security hole remains. To be confirmed.

[nodehack]

Good info @danielweck
That would be a nasty issue for apps that don't have control over the content they display.

Sounds like we could use a WKUserScript as part of the WKWebViewConfiguration to override parent/top when readium loads. That might have to be another topic/issue if the SDKLauncher decides to adopt WKWebView.

[danielweck]

Thanks @nodehack for the tip about WKUserScript, that might come in handy!

[nodehack]

What is the best way to create a "feature" branch for this repo? I've got a git-flow feature branch created but I don't have permission to push to the repo.

[rkwright]

Hm. Unfortunately, github doesn't support branch-specific permissions, so I can't give you permission for a feature branch but not to develop or master. There seem to be two options:

1. I give you permissions to create a feature branch (and push permissions for develop and master) and you swear (on any volume from the Knuth set) that you won't touch develop or master
2. You create your feature branch in your own repo and push the PR to Readium

Thoughts?

[nodehack]

@rkwright I think I'll perhaps do the second option. I don't want to accidentally mess with develop or master. I'm not so knowledgable on submodules so that seems the safest way to me. Thanks.

[rkwright]

Sounds good. Thanks

[nodehack]

Need a little help with the submodule.
I've forked the SDKLauncher-iOS project and created my own feature branch but the feature requires a change to RDPackageResourceConnection which lives in readium-sdk submodule. When I go to push my feature branch to my repo, Sourcetree is asking me to update readium-sdk as well. However, it is pointing to readium/readium-sdk.git not nodehack/readium-sdk.git.

Thoughts?

[danielweck]

The .gitmodules configuration file must be modified to reference your own repository clones, I am afraid:

https://github.com/readium/SDKLauncher-iOS/blob/develop/.gitmodules

[submodule "readium-sdk"]
    path = readium-sdk
    url = https://github.com/readium/readium-sdk.git
[submodule "Resources/readium-shared-js"]
    path = Resources/readium-shared-js
    url = https://github.com/readium/readium-shared-js.git

[danielweck]

@nodehack another option is to preserve .gitmodules as-is to avoid "polluting" your Pull Request, and instead to change the remote inside the submodule folder itself (which is effectively a separate / standalone repository clone).

[nodehack]

@danielweck I don't see remote or a specific submodule folder anywhere in my project. Are you referring to the config file found inside the .git folder?

[danielweck]

Hi @nodehack thanks for your Pull Request(s).
As for Git remotes:
https://help.github.com/categories/managing-remotes/

[rkwright]

@nodehack : Are you interested in getting developer status on Readium? This would allow you to commit/push permissions. You don't have to formally join the Readium Foundation. All that is required is to sign and return (digitally) the ICLA here. I can also send you a PDF of the agreement.

Hi nodehack

I have just downloaded your fork where you changed UIWebWiew to WKWebView.
Congratulation for your great work !

The iOS Launcher is still leaking but definitively less than previously.

On my big EPub previously the memory usage was growing to 80meg right now when you finish the book it is only 14Meg
Loading time is very fast.

Then very great job.

I will the newt weeks work on the last leaks.

Daniel, and nodehack what do you think about putting the code in readium GitHub, this will be useful for everyone

I suggest ti use, #define like USE_WK_WEB_VIEW because WKWebView can be uses only on iOS 8 and later

Cyril

Cyril Labordrie

Arutua Studio
[email protected]
[phone]
[phone]
www.arutua.com

Le 11 mars 2015 à 18:50, Ric Wright [email protected] a écrit :

@nodehack : Are you interested in getting developer status on Readium? This would allow you to commit/push permissions. You don't have to formally join the Readium Foundation. All that is required is to sign and return (digitally) the ICLA here. I can also send you a PDF of the agreement.

—
Reply to this email directly or view it on GitHub.

[danielweck]

@clabordrie during this week's engineering conference call we briefly discussed the possibility of including WKWebView support in the ReadiumSDK iOS launcher, possibly in a feature branch until the feature has been thoroughly tested. Several developers had already started investigating the feasibility of migrating away from UIWebView (which implies either dropping support for iOS pre-v8, or baring the maintenance costs of two distinct platforms), so there is clearly some interest in working this out.

[agilshane]

@danielweck - migrating to WKWebView looks very interesting. Is this something you would have time to look at since you know the ins and outs of initialization, injection, and the security issues around cross origin access?

With formal support for script injection via WKUserScript, perhaps this would invite a new approach to initialization and injection in general, not just for the cross origin issue.

[danielweck]

@agilshane I agree, this is a good opportunity to try a new "injection" method. There's no feature branch yet, but there's a Pull Request:
#50
Unfortunately I am already fully committed to other tasks, so I won't be able to help in the foreseeable future. :(

[nodehack]

To those that are considering refactoring their app to use WKWebView note that I'm running into issues when readium-shared-js returns an object from a javascript call made from the native side. You get "A JavaScript exception occurred" error back from WKWebView when calling functions like ReadiumSDK.reader.addSelectionHighlight. You'll need to change the return type to a string for it to work. I'm really not sure why this is the case. I thought that's why Apple changed the interface to deal with id's instead of making everything NSString returns. /shrug

in annotations_manager.js

this.addSelectionHighlight = function(id, type) {
        for(spine in liveAnnotations) {
            var annotationsForView = liveAnnotations[spine]; 
            if (annotationsForView.getCurrentSelectionCFI()) {
                var annotation = annotationsForView.addSelectionHighlight(id, type);
                annotation.idref = spines[spine].idref;
                return annotation;  "<-- this will need to be changed to 'return JSON.stringify(annotation);' to function with iOS WKWebView."
            }
        }
        return undefined;
    };

[danielweck]

@nodehack (Tom Feldmann), this JavaScript exception occurred error looks pretty critical. Thanks!

I haven't looked at the details in @agilshane (Shane Meyer)'s feature branch yet:
https://github.com/readium/readium-sdk/commits/feature/wkwebview
https://github.com/readium/SDKLauncher-iOS/commits/feature/wkwebview
(this is a more thorough refactoring that includes an abstraction API to "inject" the Readium components into EPUB XHTML5 Content Documents, in a UIWebView / WKWebView -agnostic way)

Code diff:
develop...feature/wkwebview
readium/readium-sdk@develop...feature/wkwebview

[agilshane]

As @danielweck mentioned, last week I posted a feature/wkwebview branch to the iOS launcher and a matching branch for related changes in readium-sdk. @nodehack it was great to see the success you were having with WKWebView. I wanted to try a somewhat different approach and so created separate branches. A quick summary of feature/wkwebview:

1. It's backward compatible. UIWebView is still supported. When running on iOS 8 or greater, WKWebView is used automatically.
2. For sending messages from JavaScript to native code, rather than using our existing implementation of "set-the-window-href-but-not-really-then-cancel-the-epubobjc-nav-request-and-inspect-the-url", I added support for script messages, which is more straightforward. There is a "readium" script message handler that the launcher installs and host_app_feedback.js looks for and uses if present.
3. Rather than increase the coupling between readium-sdk (RDServices) and the launcher, I modified RDServices so that it can handle core resources without any assumptions about what they might be or where they might live. This allows CocoaHTTPServer to handle requests for things like readium-shared-js, which it now must since file URLs that reach directly into the app bundle are not supported with WKWebView. So RDServices continues to be agnostic about the particular web view type and the types and locations of core resources.

I also ran into the issue you described about needing to stringify, which does seem odd. Here's an example of how this is handled in feature/wkwebview, which calls stringify for both the UIWebView and WKWebView paths:

https://github.com/readium/SDKLauncher-iOS/blob/feature/wkwebview/Resources/host_app_feedback.js#L111

Still to do is handling access to window.parent/top and any related changes to iframe injection.

[nodehack]

@agilshane Excellent! I love the inclusion of webkit message handlers in your branch! It seems much more full featured than my example. Good stuff!

[danielweck]

@nodehack I hope you don't mind, I closed your pull request (routine maintenance task), given that we all seem to agree about focusing our joint efforts on the new feature branch. Thanks again for exploring possible solutions though, and hopefully you will be able to contribute further as well :)

PS: here is some additional information about the benefits of migrating to WKWebView:
http://blog.initlabs.com/post/100113463211/wkwebview-vs-uiwebview
(thanks Bill MC for the link)

[danielweck]

Regarding my comment above about security #48 (comment)
...here's a code checkin that addresses this:
readium/readium-sdk@41d0770

[bluefirepatrick]

@nodehack, can you check this again now that we have checked in a launcher that includes WKWebView? It would be nice to close this issue.

[nodehack]

@bluefirepatrick I'm assuming that I'm looking at the wkwebview branch and if so then yes, this memory issue is no longer present on that branch. Would this issue get closed when that is folded into master or do you need it closed before you merge?

[bluefirepatrick]

Thanks for checking that, @nodehack. We will close this after it gets folded into master.

[danielweck]

Just a reminder of @agilshane (Shane Meyer)'s feature branch:
https://github.com/readium/readium-sdk/commits/feature/wkwebview
https://github.com/readium/SDKLauncher-iOS/commits/feature/wkwebview

Code diff:
develop...feature/wkwebview
readium/readium-sdk@develop...feature/wkwebview

[rkwright]

Resolved so closing.