Create account / Log in

Cleaning out the tiles directory

Discussion area for the development team.

Moderators: uckelman, Tim M

Cleaning out the tiles directory

Postby mroyer » November 16th, 2012, 1:58 pm

A low priority issue that probably should be addressed someday for 3.2 and beyond.

It appears that the tiling does not clean up after itself when a new version of a module is released and retiled. The old, dormant tiles are still in the AppData/Roaming/VASSAL/tiles directory (at least on Windows 7). For most modules, this isn't a big deal as they are fairly stable and not modified very often. But for some modules, especially those in development, out of date tiles can accumulate to a large volume.

For example, I just discovered and removed over 800 MBytes of tiles.

Tiles.png
Tiles.png (40.51 KiB) Viewed 9548 times


Like I said, this is low priority and my case is probably unusually large - but, nevertheless, some mechanism for handling this or at least letting general users know that it may need to be cleaned out is probably appropriate.

-Mark R.
mroyer
 
Posts: 166
Joined: March 4th, 2011, 5:06 pm
Location: Massachusetts

Re: Cleaning out the tiles directory

Postby barbanera » November 16th, 2012, 3:15 pm

You should just "remove" the old modules - with the corresponding command - and the associated tiles will go away.

>> Finally,
>> __4) shouldn't there be an automatic tiles removal process when somebody
>> does a "Remove Module"?__

>Yes. Fixed in 3.2.0-svn8370.
barbanera
 
Posts: 395
Joined: January 12th, 2012, 2:27 pm

Re: Cleaning out the tiles directory

Postby mroyer » November 16th, 2012, 4:41 pm

?
Perhaps I'm missing something in my understanding.

When a module version is incremented, is there an "old module" left around that can be removed? I haven't noticed that behavior. I'll double check tonight, but I thought when a module version is incremented, it replaced the old version module, re-tiles images and leaves all the old version tiles as clutter, but with no old module to remove.

-Mark R.
mroyer
 
Posts: 166
Joined: March 4th, 2011, 5:06 pm
Location: Massachusetts

Re: Cleaning out the tiles directory

Postby barbanera » November 16th, 2012, 7:05 pm

I am not sure if the tiling is triggered by a changed module version or by a changed module file name. Personally I never changed the module version (from 1.5 to 1.6, say) without saving the module at the same time with a different file name. Therefore, by doing that I always got multiple versions of the same module showing up in the Vassal console window. That's what I was referring to.

Anyway, I also noticed that when you open the same module under different versions of Vassal (beta1 vs beta2 etc), you will get a new tiling. Therefore, I would suggest to remove modules you don't need to use anymore from any version of Vassal currently installed. Presumably, uninstalling an older version of Vassal 3.2.0 should also remove all its associated tiles, too.
barbanera
 
Posts: 395
Joined: January 12th, 2012, 2:27 pm

Re: Cleaning out the tiles directory

Postby mroyer » November 16th, 2012, 9:17 pm

Understood and agreed. Otherwise, there is a risk of outdated tiles accumulating indefinitely on the user's disk.
I guess I should add an entry to the bug tracker for this.
mroyer
 
Posts: 166
Joined: March 4th, 2011, 5:06 pm
Location: Massachusetts

Re: Cleaning out the tiles directory

Postby uckelman » November 16th, 2012, 9:54 pm

Thus spake barbanera:
> I am not sure if the tiling is triggered by a changed module version or
> by a changed module file name.

Tile hashes are based on the module name and version, not the module
filename.
--
J.
User avatar
uckelman
Site Admin
 
Posts: 8981
Joined: December 10th, 2007, 9:48 am
Location: Durham, England

Re: Cleaning out the tiles directory

Postby uckelman » November 16th, 2012, 10:00 pm

Thus spake mroyer:
>
> A low priority issue that probably should be addressed someday for 3.2
> and beyond.
>

There won't be any need for writing tiles to disk for V4---at least not
for the reference C++ client. (In the C++ demo, decoding even very large
images and slicing them into tiles in memory never takes more than two
or three seconds; writing them to disk would be a waste of time.) So
this is only an issue for 3.2.

> It appears that the tiling does not clean up after itself when a new
> version of a module is released and retiled. The old, dormant tiles are
> still in the AppData/Roaming/VASSAL/tiles directory (at least on Windows
> 7). For most modules, this isn't a big deal as they are fairly stable
> and not modified very often. But for some modules, especially those in
> development, out of date tiles can accumulate to a large volume.

Loading a newer version of a module does not guarantee that we won't
still also load the old version. So how can we detect that tiles can
be removed?

--
J.
User avatar
uckelman
Site Admin
 
Posts: 8981
Joined: December 10th, 2007, 9:48 am
Location: Durham, England

Re: Cleaning out the tiles directory

Postby mroyer » November 17th, 2012, 5:37 am

uckelman wrote:Thus spake mroyer:
Loading a newer version of a module does not guarantee that we won't
still also load the old version. So how can we detect that tiles can
be removed?
J.


I see... so this is intended behavior then.
Fair enough.

-Mark R.
mroyer
 
Posts: 166
Joined: March 4th, 2011, 5:06 pm
Location: Massachusetts

Re: Cleaning out the tiles directory

Postby uckelman » November 17th, 2012, 11:46 am

Thus spake mroyer:
>
> "uckelman" wrote:
> > Thus spake mroyer:
> > Loading a newer version of a module does not guarantee that we won't
> > still also load the old version. So how can we detect that tiles can
> > be removed?
> > J.
>
>
> I see... so this is intended behavior then.
> Fair enough.

It's not a rhetorical question. I'm asking you to propose a method
of detecting when tiles can be removed. I don't know of one myself.

--
J.
User avatar
uckelman
Site Admin
 
Posts: 8981
Joined: December 10th, 2007, 9:48 am
Location: Durham, England

Re: Cleaning out the tiles directory

Postby mroyer » November 18th, 2012, 11:42 am

uckelman wrote:It's not a rhetorical question. I'm asking you to propose a method
of detecting when tiles can be removed. I don't know of one myself.

--
J.


If having multiple versions is sometimes desired and sometimes undesired, then I guess there's probably not an automated way to know. The information would need to come from the user regarding whether he/she wanted multiple versions to be active at once.

-Mark R.
mroyer
 
Posts: 166
Joined: March 4th, 2011, 5:06 pm
Location: Massachusetts

Re: Cleaning out the tiles directory

Postby uckelman » November 18th, 2012, 3:00 pm

Thus spake mroyer:
>
> "uckelman" wrote:
> >
> > It's not a rhetorical question. I'm asking you to propose a method
> > of detecting when tiles can be removed. I don't know of one myself.
> >
> > --
> > J.
>
>
> If having multiple versions is sometimes desired and sometimes
> undesired, then I guess there's probably not an automated way to know.
> The information would need to come from the user regarding whether
> he/she wanted multiple versions to be active at once.

One thing I'd considered was setting a maximum tile cache size, where
least recently used modules would have their tiles expired when the
limit was reached. I didn't implement this for two reasons:

* It would add yet another setting (maximum tile cache size) that most
users would lack sufficeint context to set intelligently, and

* You can't rely on file atimes anymore now that people are using SSDs
with filesystems that don't set atimes, so we'd have to write code for
tracking the last time each tile directory was used.

--
J.
User avatar
uckelman
Site Admin
 
Posts: 8981
Joined: December 10th, 2007, 9:48 am
Location: Durham, England

Re: Cleaning out the tiles directory

Postby mroyer » November 19th, 2012, 10:36 pm

The only other thing I can think of is if there were a "last played" date property of a module that got updated when the module is played. Tiles could expire after some long time since a last playing (whether hardcoded or user-preference settable). But, quite frankly, I'm not sure the issue is worth the effort especially where it's a v3.2 issue only. When I first reported it, I didn't realize I was running into a corner-case of design intent.

uckelman wrote:
There won't be any need for writing tiles to disk for V4---at least not
for the reference C++ client. (In the C++ demo, decoding even very large
images and slicing them into tiles in memory never takes more than two
or three seconds; writing them to disk would be a waste of time.) So
this is only an issue for 3.2.

J.


Has anyone tried this with my module? I'm going to go out on a limb and speculate that the module probably has the largest map images of any VASSAL module to date, so it would be a good stress test for V4 in-memory tiling. The module is unreleased, but if anyone wants to test how it "slices" in v4 I can make it available.

-Mark R.
mroyer
 
Posts: 166
Joined: March 4th, 2011, 5:06 pm
Location: Massachusetts

Re: Cleaning out the tiles directory

Postby uckelman » November 20th, 2012, 8:50 pm

Thus spake mroyer:
>
> Has anyone tried this with my module? I'm going to go out on a limb and
> speculate that the module probably has the largest map images of any
> VASSAL module to date, so it would be a good stress test for V4
> in-memory tiling. The module is unreleased, but if anyone wants to test
> how it "slices" in v4 I can make it available.

How large is the map image, in pixels?

--
J.
User avatar
uckelman
Site Admin
 
Posts: 8981
Joined: December 10th, 2007, 9:48 am
Location: Durham, England

Re: Cleaning out the tiles directory

Postby mroyer » November 21st, 2012, 2:13 am

uckelman wrote:How large is the map image, in pixels?


There are six map images that can be puzzled together depending on scenario (from one to all six at once).
Each map image is exactly 8845 x 6643 pixels. When VASSAL v3.2 tiles all six maps, it takes a few minutes, which I find surprisingly fast given the size.

-Mark R.
mroyer
 
Posts: 166
Joined: March 4th, 2011, 5:06 pm
Location: Massachusetts

Re: Cleaning out the tiles directory

Postby uckelman » November 21st, 2012, 9:06 am

Thus spake mroyer:
>
> "uckelman" wrote:
> >
> > How large is the map image, in pixels?
> >
>
>
> There are six map images that can be puzzled together depending on
> scenario (from one to all six at once).
> Each map image is exactly 8845 x 6643 pixels. When VASSAL v3.2 tiles
> all six maps, it takes a few minutes, which I find surprisingly fast
> given the size.

Six of those goes a bit beyond the amount of video RAM I'd expect
machines to have, if they're loaded uncompressed. In order to accomodate
that, I think we'd have to turn on texture compression.

--
J.
User avatar
uckelman
Site Admin
 
Posts: 8981
Joined: December 10th, 2007, 9:48 am
Location: Durham, England

Next

Return to Developers

Who is online

Users browsing this forum: No registered users and 2 guests