Comments (2)
Thanks for the report!
I did a brief investigation by compiling this program with -ddump-stg
and decompiling the generated class files as well. It turns out the optimizations are happening and the intermediate vectors have been fused away and there's a nice tight loop that calculates the sum via an accumulator variable.
What's interesting is that when I checked out VisualVM, the old generation for the GC was through the roof - 1GB! This means that there's a memory leak going on somewhere. The code is nice and small so it should be easy to investigate. Thank you for constructing a minimal example! I'll take a look at this.
from eta-hackage.
Some notes on the investigation for this:
-
I did memory profiling in VisualVM and discovered that the entire list from 1 to 100,000,000 is retained along with all the intermediate thunks used to construct it. This list was created as a result of fusing the vector operations and lifting out the resulting list to the top-level, creating a CAF (constant applicative form) or a globally shared thunk.
In Eta, globally shared thunks are referenced through a static field of the class which corresponds to the module for which that thunk was declared (either directly or indirectly). Thus, the head of the list is retained throughout the duration of the program, preventing the GC from collecting the front part of the list. The way to get around this is to make the static reference to the CAF weak so that it can be GC'd when it's unused and re-created again as necessary. See typelead/eta#554.
This problem doesn't affect GHC since its GC has special handling for top-level closures and thunks which clears out unused heap objects.
-
I turned on
-fno-full-laziness
to see if I could prevent the optimizer from lifting the list out as a CAF. The optimizer did not lift the CAF this time, but unfortunately, even in this case, it turns out there's head retention going on where the head of the list is retained by a local variable in the function that computes the initial [1..100,000,000] list. This problem can be fixed by typelead/eta#79.
Two bugs with one program, great work @rpeszek! :)
from eta-hackage.
Related Issues (20)
- dhall-to-etlas doesn't build due to ansi-terminal dependency HOT 2
- Issue with compiling language-javascript HOT 1
- Patch cborg-0.2.0.0 HOT 1
- cryptohash-md5 and cryptohash-sha1 large input test HOT 1
- Patch for cql-io HOT 1
- lens-aeson examples throw IllegalStateExceptions HOT 7
- Patch for pandoc 2.5 HOT 2
- failure on Text.Regex.mkRegex HOT 3
- Patch needed to install Reflex? Dependency on haskell-src-exts-1.20.3 HOT 2
- Get Duckling to work HOT 9
- "regex-compat" package behavior is inconsistent with GHCi
- Temporary library -- creation of temp files does not work on windows platform HOT 3
- request: Patch pretty-show-1.6.16 HOT 1
- Patch the Cabal library HOT 1
- directory-1.3.1.0 fix createDirectoryIfMissing HOT 4
- Patch megaparsec package to avoid StackOverflowException's HOT 14
- Patch criterion + support for JMH HOT 12
- Request: patch cryptohash-md5 and cryptohash-sha1 HOT 2
- [Patch Request] MissingH-1.4.0.1 HOT 3
- Patch text-show-3.7.3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eta-hackage.