Leaking memory on every compile #10

egbulmer · 2015-09-24T02:18:33Z

Hi there! I am seeing a memory leak after every garden compile of approximately 10-15mb each time. This eventually causes an OutOfMemoryException like this one:

Exception in thread "Thread-9" java.lang.OutOfMemoryError: PermGen space
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at clojure.core_deftype__init.load(Unknown Source)
    at clojure.core_deftype__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:274)
    at clojure.lang.RT.classForName(RT.java:2154)
    at clojure.lang.RT.classForName(RT.java:2163)
    at clojure.lang.RT.loadClassForName(RT.java:2182)
    at clojure.lang.RT.load(RT.java:436)
    at clojure.lang.RT.load(RT.java:412)
    at clojure.core$load$fn__5448.invoke(core.clj:5866)
    at clojure.core$load.doInvoke(core.clj:5865)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core__init.load(Unknown Source)
    at clojure.core__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:274)
    at clojure.lang.RT.classForName(RT.java:2154)
    at clojure.lang.RT.classForName(RT.java:2163)
    at clojure.lang.RT.loadClassForName(RT.java:2182)
    at clojure.lang.RT.load(RT.java:436)
    at clojure.lang.RT.load(RT.java:412)

I have increased my PermGen size as suggested here but this only delays the problem for a while. Since I'm making rapid changes to my styles and seeing the results reload in the browser, I very quickly run out of memory and have to re-run my Boot command, which is a pain!

I'm not sure if the leak is in the Boot task or in Garden itself. Let me know what I can do to help!

The text was updated successfully, but these errors were encountered:

martinklepsch · 2015-09-24T08:36:48Z

Try removing the garden task from your pipeline. If you can't reproduce the
problem anymore then it's most likely an issue with this task/garden.

Thanks!
On Thu 24 Sep 2015 at 04:18 Elliot Bulmer [email protected] wrote:

Hi there! I am seeing a memory leak after every garden compile of
approximately 10-15mb each time. This eventually causes an
OutOfMemoryException like this one:

Exception in thread "Thread-9" java.lang.OutOfMemoryError: PermGen space
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at clojure.core_deftype__init.load(Unknown Source)
at clojure.core_deftype__init.(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core__init.load(Unknown Source)
at clojure.core__init.(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)

I have increased my PermGen size as suggested here
https://github.com/boot-clj/boot/wiki/JVM-Options but this only delays
the problem for a while. Since I'm making rapid changes to my styles and
seeing the results reload in the browser, I very quickly run out of memory
and have to re-run my Boot command, which is a pain!

I'm not sure if the leak is in the Boot task or in Garden itself. Let me
know what I can do to help!

—
Reply to this email directly or view it on GitHub
#10.

egbulmer · 2015-09-24T08:57:48Z

Hi @martinklepsch. I've removed it from the pipeline. The memory footprint still increases when it re-compiles the JS code, but at a much smaller rate (200-300 KB). This makes me think that either boot-garden or garden itself are the culprits.

To work around the issue I'm now generating the CSS dynamically and inserting it like this:

(require '[goog.style] '[garden.core :refer [css]]))
(goog.style/installStyles (css [:body {:background "#ff0000"}]))

I know this isn't ideal but it is fine for quick-development right now. What other info or troubleshooting can I do to assist in narrowing down the problem for you?

martinklepsch · 2015-10-04T22:37:31Z

I experimented with some different ways to use the pods but that didn't seem to have an impact on the used memory. I guess it would be useful to test with lein-garden if a similar problem exists. Other than that it might be good to remove pods completely and see how that impacts memory usage.

How are you measuring these things btw, any tools that are particularly helpful?

martinklepsch · 2015-10-07T17:10:48Z

boot-clj/boot#268 — just discovered this issue in Boot which makes me assume it's a general issue.

egbulmer · 2015-10-09T02:52:03Z

How are you measuring these things btw, any tools that are particularly helpful?

I just used ps aux to check the memory footprint of the boot Java process. I found that whenever watch caused the style.clj to be re-compiled by boot-garden that the memory footprint increased by 8-10mb. I repeated this a number of times until I got the OutOfMemory exception.

boot-clj/boot#268 — just discovered this issue in Boot which makes me assume it's a general issue.

Interesting. I'll try lein-garden to see if it has the same problem, but it seems like it may be specific to using boot.

martinklepsch · 2015-10-22T20:10:41Z

@sonelliot Spent some time today looking into this problem. Few questions:

When the exception occurs:

What are your JVM options?
How much memory does the process consume?
How much memory does your machine have?

egbulmer · 2015-10-23T03:37:21Z

@martinklepsch I am currently using a different machine. I will try to jump back on the other computer tomorrow and get some more detailed stats for you. In regards to your questions:

1 . What are your JVM options?

By this, do you mean the value of the BOOT_JVM_OPTIONS environment variable? Are there other system-level JVM parameters you'd like as well?

My BOOT_JVM_OPTIONS is set to:

-XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled

2 . How much memory does the process consume?

From memory, it was approximately 1 GB before I hit the exception. I will get a more detailed measurement for you tomorrow.

3 . How much memory does your machine have?

It is a desktop PC with 12GB of DDR3 RAM.

martinklepsch · 2016-01-07T18:50:54Z

I believe this has been fixed with Boot v2.4.0. I recommend trying with a recent release of Boot.

martinklepsch closed this as completed Jan 7, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Leaking memory on every compile #10

Leaking memory on every compile #10

egbulmer commented Sep 24, 2015

martinklepsch commented Sep 24, 2015

egbulmer commented Sep 24, 2015

martinklepsch commented Oct 4, 2015

martinklepsch commented Oct 7, 2015

egbulmer commented Oct 9, 2015

martinklepsch commented Oct 22, 2015

egbulmer commented Oct 23, 2015

martinklepsch commented Jan 7, 2016

Leaking memory on every compile #10

Leaking memory on every compile #10

Comments

egbulmer commented Sep 24, 2015

martinklepsch commented Sep 24, 2015

egbulmer commented Sep 24, 2015

martinklepsch commented Oct 4, 2015

martinklepsch commented Oct 7, 2015

egbulmer commented Oct 9, 2015

martinklepsch commented Oct 22, 2015

egbulmer commented Oct 23, 2015

martinklepsch commented Jan 7, 2016