Page 1 of 2

Cleaning out the tiles directory

PostPosted: November 16th, 2012, 1:58 pm
by mroyer
A low priority issue that probably should be addressed someday for 3.2 and beyond.

It appears that the tiling does not clean up after itself when a new version of a module is released and retiled. The old, dormant tiles are still in the AppData/Roaming/VASSAL/tiles directory (at least on Windows 7). For most modules, this isn't a big deal as they are fairly stable and not modified very often. But for some modules, especially those in development, out of date tiles can accumulate to a large volume.

For example, I just discovered and removed over 800 MBytes of tiles.

Tiles.png
Tiles.png (40.51 KiB) Viewed 9540 times


Like I said, this is low priority and my case is probably unusually large - but, nevertheless, some mechanism for handling this or at least letting general users know that it may need to be cleaned out is probably appropriate.

-Mark R.

Re: Cleaning out the tiles directory

PostPosted: November 16th, 2012, 3:15 pm
by barbanera
You should just "remove" the old modules - with the corresponding command - and the associated tiles will go away.

>> Finally,
>> __4) shouldn't there be an automatic tiles removal process when somebody
>> does a "Remove Module"?__

>Yes. Fixed in 3.2.0-svn8370.

Re: Cleaning out the tiles directory

PostPosted: November 16th, 2012, 4:41 pm
by mroyer
?
Perhaps I'm missing something in my understanding.

When a module version is incremented, is there an "old module" left around that can be removed? I haven't noticed that behavior. I'll double check tonight, but I thought when a module version is incremented, it replaced the old version module, re-tiles images and leaves all the old version tiles as clutter, but with no old module to remove.

-Mark R.

Re: Cleaning out the tiles directory

PostPosted: November 16th, 2012, 7:05 pm
by barbanera
I am not sure if the tiling is triggered by a changed module version or by a changed module file name. Personally I never changed the module version (from 1.5 to 1.6, say) without saving the module at the same time with a different file name. Therefore, by doing that I always got multiple versions of the same module showing up in the Vassal console window. That's what I was referring to.

Anyway, I also noticed that when you open the same module under different versions of Vassal (beta1 vs beta2 etc), you will get a new tiling. Therefore, I would suggest to remove modules you don't need to use anymore from any version of Vassal currently installed. Presumably, uninstalling an older version of Vassal 3.2.0 should also remove all its associated tiles, too.

Re: Cleaning out the tiles directory

PostPosted: November 16th, 2012, 9:17 pm
by mroyer
Understood and agreed. Otherwise, there is a risk of outdated tiles accumulating indefinitely on the user's disk.
I guess I should add an entry to the bug tracker for this.

Re: Cleaning out the tiles directory

PostPosted: November 16th, 2012, 9:54 pm
by uckelman
Thus spake barbanera:
> I am not sure if the tiling is triggered by a changed module version or
> by a changed module file name.

Tile hashes are based on the module name and version, not the module
filename.
--
J.

Re: Cleaning out the tiles directory

PostPosted: November 16th, 2012, 10:00 pm
by uckelman
Thus spake mroyer:
>
> A low priority issue that probably should be addressed someday for 3.2
> and beyond.
>

There won't be any need for writing tiles to disk for V4---at least not
for the reference C++ client. (In the C++ demo, decoding even very large
images and slicing them into tiles in memory never takes more than two
or three seconds; writing them to disk would be a waste of time.) So
this is only an issue for 3.2.

> It appears that the tiling does not clean up after itself when a new
> version of a module is released and retiled. The old, dormant tiles are
> still in the AppData/Roaming/VASSAL/tiles directory (at least on Windows
> 7). For most modules, this isn't a big deal as they are fairly stable
> and not modified very often. But for some modules, especially those in
> development, out of date tiles can accumulate to a large volume.

Loading a newer version of a module does not guarantee that we won't
still also load the old version. So how can we detect that tiles can
be removed?

--
J.

Re: Cleaning out the tiles directory

PostPosted: November 17th, 2012, 5:37 am
by mroyer
uckelman wrote:Thus spake mroyer:
Loading a newer version of a module does not guarantee that we won't
still also load the old version. So how can we detect that tiles can
be removed?
J.


I see... so this is intended behavior then.
Fair enough.

-Mark R.

Re: Cleaning out the tiles directory

PostPosted: November 17th, 2012, 11:46 am
by uckelman
Thus spake mroyer:
>
> "uckelman" wrote:
> > Thus spake mroyer:
> > Loading a newer version of a module does not guarantee that we won't
> > still also load the old version. So how can we detect that tiles can
> > be removed?
> > J.
>
>
> I see... so this is intended behavior then.
> Fair enough.

It's not a rhetorical question. I'm asking you to propose a method
of detecting when tiles can be removed. I don't know of one myself.

--
J.

Re: Cleaning out the tiles directory

PostPosted: November 18th, 2012, 11:42 am
by mroyer
uckelman wrote:It's not a rhetorical question. I'm asking you to propose a method
of detecting when tiles can be removed. I don't know of one myself.

--
J.


If having multiple versions is sometimes desired and sometimes undesired, then I guess there's probably not an automated way to know. The information would need to come from the user regarding whether he/she wanted multiple versions to be active at once.

-Mark R.

Re: Cleaning out the tiles directory

PostPosted: November 18th, 2012, 3:00 pm
by uckelman
Thus spake mroyer:
>
> "uckelman" wrote:
> >
> > It's not a rhetorical question. I'm asking you to propose a method
> > of detecting when tiles can be removed. I don't know of one myself.
> >
> > --
> > J.
>
>
> If having multiple versions is sometimes desired and sometimes
> undesired, then I guess there's probably not an automated way to know.
> The information would need to come from the user regarding whether
> he/she wanted multiple versions to be active at once.

One thing I'd considered was setting a maximum tile cache size, where
least recently used modules would have their tiles expired when the
limit was reached. I didn't implement this for two reasons:

* It would add yet another setting (maximum tile cache size) that most
users would lack sufficeint context to set intelligently, and

* You can't rely on file atimes anymore now that people are using SSDs
with filesystems that don't set atimes, so we'd have to write code for
tracking the last time each tile directory was used.

--
J.

Re: Cleaning out the tiles directory

PostPosted: November 19th, 2012, 10:36 pm
by mroyer
The only other thing I can think of is if there were a "last played" date property of a module that got updated when the module is played. Tiles could expire after some long time since a last playing (whether hardcoded or user-preference settable). But, quite frankly, I'm not sure the issue is worth the effort especially where it's a v3.2 issue only. When I first reported it, I didn't realize I was running into a corner-case of design intent.

uckelman wrote:
There won't be any need for writing tiles to disk for V4---at least not
for the reference C++ client. (In the C++ demo, decoding even very large
images and slicing them into tiles in memory never takes more than two
or three seconds; writing them to disk would be a waste of time.) So
this is only an issue for 3.2.

J.


Has anyone tried this with my module? I'm going to go out on a limb and speculate that the module probably has the largest map images of any VASSAL module to date, so it would be a good stress test for V4 in-memory tiling. The module is unreleased, but if anyone wants to test how it "slices" in v4 I can make it available.

-Mark R.

Re: Cleaning out the tiles directory

PostPosted: November 20th, 2012, 8:50 pm
by uckelman
Thus spake mroyer:
>
> Has anyone tried this with my module? I'm going to go out on a limb and
> speculate that the module probably has the largest map images of any
> VASSAL module to date, so it would be a good stress test for V4
> in-memory tiling. The module is unreleased, but if anyone wants to test
> how it "slices" in v4 I can make it available.

How large is the map image, in pixels?

--
J.

Re: Cleaning out the tiles directory

PostPosted: November 21st, 2012, 2:13 am
by mroyer
uckelman wrote:How large is the map image, in pixels?


There are six map images that can be puzzled together depending on scenario (from one to all six at once).
Each map image is exactly 8845 x 6643 pixels. When VASSAL v3.2 tiles all six maps, it takes a few minutes, which I find surprisingly fast given the size.

-Mark R.

Re: Cleaning out the tiles directory

PostPosted: November 21st, 2012, 9:06 am
by uckelman
Thus spake mroyer:
>
> "uckelman" wrote:
> >
> > How large is the map image, in pixels?
> >
>
>
> There are six map images that can be puzzled together depending on
> scenario (from one to all six at once).
> Each map image is exactly 8845 x 6643 pixels. When VASSAL v3.2 tiles
> all six maps, it takes a few minutes, which I find surprisingly fast
> given the size.

Six of those goes a bit beyond the amount of video RAM I'd expect
machines to have, if they're loaded uncompressed. In order to accomodate
that, I think we'd have to turn on texture compression.

--
J.