Hopbot log for 2007-11-02 - Helma IRC channel: #helma on irc.freenode.net

2007-11-02:

[0:01] <drbobb> ok i get it, free memory must be relative to current heap size, not the maximum
[0:03] <BSlivka> Ohhh
[0:03] <BSlivka> So you run gc a few times, the heap shrinks
[0:03] <BSlivka> and the free mem shrinks
[0:03] <BSlivka> good to know
[0:03] <drbobb> so it seems
[0:07] <drbobb> i guess the number i really want to know would be given by
[0:07] <drbobb> maxMemory - totalMemory + freeMemory
[0:07] <drbobb> (iiuc)
[0:08] <drbobb> (i'm trying to prevent heap overflows when handling large chunks of data)
[0:14] <drbobb> yes, this number does seem to behave reasonably
[0:15] <drbobb> i need an estimate of how much data i can handle in one go, for given jvm max heap size setting
[0:20] <drbobb> wow javascript data structures are really wasteful of memory
[0:21] <drbobb> for a big array of numbers, it seems i need an estimated 55.5 bytes * length
[0:24] <drbobb> by comparing maxMemory - totalMemory + freeMemory before and after deallocating the array & gc()
[0:27] <drbobb> just launching a rhino shell seems to require about 4824344 of the jvm's heap
[0:33] <drbobb> and 10 million numbers are too much at this value of max heap size
[0:33] <drbobb> (and in fact, i get an exception thrown)
[0:37] <drbobb> hmm i didn't imagine the overhead was so *huge*
[0:49] <drbobb> for a java array with the same set of values, i get an overhead of only about 2kB
[0:50] <drbobb> (measured the same way)
[0:51] <drbobb> memory in bytes recovered by deallocating the array - ( length of array * 8 )
[0:52] <BSlivka> If it's important to you, you can try wrapping a java array in a javascript object
[0:53] <drbobb> rhino does it mostly automatically
[0:54] <drbobb> yes, i can work around this and avoid copying the data into a javascript array (i think)
[0:58] <drbobb> it looks like rhino allows me to call javascript Array methods on a java array
[1:01] <drbobb> (so i can eg. get small slices of the array w/o the overhead of copying all the data)
[1:01] <drbobb> hey this is something the rhino guys did *really well*
[1:08] <drbobb> ok the bottom line is, i can't have js arrays of the sizes i need, with the value of max heap size i'm using
[1:09] <drbobb> which is basically the greatest value i can use w/o swapping a lot
[1:09] <drbobb> but, for numerics, i can use java arrays of doubles and pretend, most of the time, that they're javascript arrays
[1:10] <BSlivka> I almost thought for a second that Helma provided some mechanism for this, but then I remembered that it was actually WX_JS
[1:11] <BSlivka> Which allows you to directly allocate some low level memory from javascript.
[1:11] <BSlivka> but it's ontop of spidermonkey, so it may not be what you're after
[1:14] <drbobb> rhino seems to be very smart about this array handling
[1:15] <drbobb> the only time i see i'm dealing with a wrapped java array, not a javascript one
[1:16] <drbobb> is when calling arr.toString(), which invokes the java metod
[1:16] <drbobb> method
[1:17] <drbobb> and of course, i can't put a string (or whatever else) value into an array of doubles
[1:18] <drbobb> er, actually that doesn't even throw an error
[1:18] <drbobb> it assigns NaN instead
[1:18] <drbobb> (which isn't such a great thing)
[1:24] <drbobb> anyway, pretty cool stuff
[1:24] <drbobb> ok, goodnite

 

 

In the channel now:

Logs by date: