It’s fairly simple actually. If you’re in a large module (such as the
one I posted for another reason the other day), and you open the Windows
Task Manager and monitor the java.exe memory usage, the usage will jump
up each time you move your game map window to another section of the map
(i.e., bring new tiled sections into memory) or zoom in/out.
Eventually, it will consume the entire allocated heap (and a bit more)
and repainting the map images will bog down heavily (often taking 20+
seconds to repaint) and on occasion the game will crash with a message
saying “out of memory” or something like that.
I can’t reproduce it. Can you show me the errorLog from one of these
runs?
Speaking about tiles, how exactly do they work? I mean… are they saved everytime one loads the same module with a different build of Vassal?
It’s weird, because after wiping out completely the AppData/Roaming/VASSAL/tiles and starting MH 1.2 with build 8355 tiles were saved in a certain new folder:
is it normal they took 100 Mb of space when the original images themselves take only 40 Mb or so?
then I quit and reloaded the same module with build 8313
2) is it normal that this time no tiles were saved?
then I quit and reloaded the same module with beta 1
3) is it normal that this time tiles were saved again, and in the same folder as for the 8355 build, for a total of 200 Mb in use?
Finally, 4) shouldn’t there be an automatic tiles removal process when somebody does a “Remove Module”?
I am sure there is none at present because I removed MH 1.2 from the Vassal console (of beta 1) and, while the module didn’t show up anymore in any other build’s console window, the 200 Mb of tiles were still sitting there untouched.
It was strange to have to wipe out 1.2 GB of disk space with only maybe 4 modules still showing in the Vassal console (but also, I guess, multiple istances of the MH tiles whenever I tried a new build).
Maybe tile cleaning up should be included in 3.2.0 official release.
Speaking about tiles, how exactly do they work? I mean… are they saved
everytime one loads the same module with a different build of Vassal?
No, new tiles should be computed for a module only when the images in
a module have changed.
It’s weird, because after wiping out completely the
AppData/Roaming/VASSAL/tiles and starting MH 1.2 with build 8355 tiles
were saved in a certain new folder:
is it normal they took 100 Mb of space when the original images
themselves take only 40 Mb or so?
Yes, because the tiles are gzipped raw image data, so the compression
isn’t as tight as in the image files.
then I quit and reloaded the same module with build 8313
2) is it normal that this time no tiles were saved?
Yes.
then I quit and reloaded the same module with beta 1
3) is it normal that this time tiles were saved again, and in the same
folder as for the 8355 build, for a total of 200 Mb in use?
Yes, but this is due to some changes made after beta1 was released. The
behavior of beta1 wasn’t correct.
Finally, 4) shouldn’t there be an automatic tiles removal process when somebody
does a “Remove Module”?
I spend a good piece of the weekend trying to reproduce the problems we’d been having in earlier builds and couldn’t. Perhaps it was fixed indirectly by other changes? In general, the repaint of the large map seems more snappy than it use to be (its fully repainting a 1920x1080 monitor in a few seconds which is much quicker than it had been).
As we continue to test, if someone gets an error I will make sure to capture the errorLog.
Okay, so it happened again this morning. Attached is a screenshot of the memory error, the blank hexgrid waiting endlessly (or a really, really long time) to be re-tiled, and the resource manager showing java.exe over it’s 512MB heap limit. If I increase the heap to, say, 1024 MB it doesn’t help - the java.exe usage creeps up to about 1100MB as I scroll around and zoom in/out on the map.
That’s not a complete errorLog. It doesn’t have the OutOfMemoryError in it, for one thing. It’s also missing the info which should be written by the ModuleManager indicating how the Player was launched. Did you copy this from the errorLog file, or from the errorLog window in the ModuleManager?
I didn’t copy contents of the file at all. I simply renamed the file from errorLog to errorLog.txt (so the forums would allow upload - it won’t allow un-extensioned files to be uploaded). I never touched the contents.
I looked again at the errorLog you posted, and what it shows is that the Module Manager ran for about 4 seconds, and then exited. There’s nothing missing—it’s just not the error log from the run where you ran out of memory. The errorLog is overwritten each time you start VASSAL, so when there’s a problem, you have to grab it before starting VASSAL again.
If I scroll around, zoom in and out, until the java.exe has consumed it’s entire heap allocation, then I set the zoom level to 100% or higher (maybe this is related to the size of a tile vs the size of the visible image?) and set the map window to straddle the boundary between two large map images such that a portion of each map is visible, the tiles never stop re-painting themselves causing the map image tiles to blink and flash endlessly (and occasionally, after a while, throw an exception/eek bug). If I reduce the image size to less than 100% (75% in my case) and/or move the window off the map boundary, the flashing/blinking stops and the map repaints itself correctly and only once (at least that I can see).
I can reproduce the blinking-symptom at will now that I know root mechanism.
(The issue can probably be reproduced separately, but just in case I’ve posted the module that I used to cause the issue here: dl.dropbox.com/u/24119799/WAA.zip
It’s 83 MB, just FYI. Run it in v3.2 and set up at least two of the six map segments to reproduce the issue.)
I still think there’s a memory-leak problem in 3.2 related to
unloading image tiles and with large modules eventually causes
problems.
I think I’ve finally isolated this issue.
If I scroll around, zoom in and out, until the java.exe has consumed
it’s entire heap allocation, then I set the zoom level to 100% or higher
(maybe this is related to the size of a tile vs the size of the visible
image?) and set the map window to straddle the boundary between two
large map images such that a portion of each map is visible, the tiles
never stop re-painting themselves causing the map image tiles to blink
and flash endlessly (and occasionally, after a while, throw an
exception/eek bug). If I reduce the image size to less than 100% (75%
in my case) and/or move the window off the map boundary, the
flashing/blinking stops and the map repaints itself correctly and only
once (at least that I can see).
To what do you have your max heap set?
The most likely reason this is happening is that the image cache is being
cleared before what’s on your screen is finished painting, which triggers
reloading those same images during the next repaint cycle, which again
causes the cache to clear partway through.
I’ve tried setting the max heap to a bunch of different sizes from time to time. I had it at 512 MB for quite a while. It happens to be set at 1024 MB at the moment.
I’ve tried setting the max heap to a bunch of different sizes from time
to time. I had it at 512 MB for quite a while. It happens to be set at
1024 MB at the moment.
I found the cause of the problem. The ImageOp for the 1.0-scale board
image was getting the wrong path to the board image, which caused the
tile cache lookup to fail. When that happens, we check the images
directory in the module to see if we can find the full image there. (The
reason for doing this is that you might be using the editor and have
just added the image, so it wouldn’t yet be tiled.) The cache check
appends “image/” to the image path, while for historic reasons, the
in-module check does not, and takes care of it somewhere else. This a
bad state of affairs, and you found one of the cases where the
inconsistency between the two path conventions causes bad behavior.
What’s happening in your case is that we get the the correct tiles for
every scale factor other than 1.0. For 1.0, we don’t find the tiles,
but we do find your colossal images, and we also know how to tile
images in memory, so we try that, rather than failing outright. So 1.0
scale factor works, but slowly, so long as all the tiles on screen are
from the same image. What happens when you scroll to the boundary
between two images at 1.0 scale factor is that a request goes out to
load the second map image in order to make tiles from it in memory. Your
map images are so large that only one fits in the JVM heap at a time,
so as soon as the second image starts to load, the first one and its
tiles get kicked out of the in-memory image cache. If you don’t scroll
away, the next repaint send out a request for the tiles from the first
map, which causes the first map to be reloaded, which kicks the second
map and its tiles out of the cache. This will cycle forever if you let
it.
Try 3.2.0-svn8434, which I believe corrects the problem:
Gave it a quick go… feelin’ sweet! No evidence of the old problem at all. More, the repaint seems generally snappier - is that possible or just wishful thinking on my part?
Gave it a quick go… feelin’ sweet! No evidence of the old problem at
all. More, the repaint seems generally snappier - is that possible or
just wishful thinking on my part?
Repaint at 1:1 is definitely swifter, since it’s using the tile cache
now, whereas it inadvertantly wasn’t before.
