Technical questions

[uckelman]

Thus spake Flint1b:
> Is there some way I can reproduce this memory issue so I can see for
> myself?

You'd need access to the machines where it occurs, which I've never had.
What I can tell you is that I've given users commands to try and they've
reported back that Java won't start with an initial heap far, far less
than the amount of free RAM they have. (IIRC, I once had someone who
was consistently unable to get Java to run with a 1GB heap when they had
8GB RAM and Windows was telling them they had something like 4GB free...)

--
J.

[Brent Easton]

I expect there will be a 3.3.1 as bug fixes accumulate. I'm not keen on adding new features to 3.3. I will happily review bug fixes, however. In the next few days, I'd like to put together a list of significant bugs to which I can point people interested in helping. Fixing bugs is the single biggest contribution anyone could make at this point.

I'd also like to include changes in the following areas:

• Consistency - Description fields that should be called name, missing descriptions fields in traits, Fields missing the Calculated Property button that should have them etc.
• Documentation - Changes to the on-line reference manual. Errors, typos, out of date info, missing information, missing links, clarifications etc.

This is where we can really use some help and non-technical folks can make a big contribution.

[Tim M]

assuming that as much work will be done for that as was done for Vassal 3 over the last 10-15 years.

Very little work has been done over the last 10 years and most of that has been remedial. The bulk of the work before that was trying to fit features in that should have been designed in from the start but weren't. Most of development time that has gone into Vassal over the last 15 years has been wasted dealing with crap. I can't tell you how many months of my life I lost trying to retrofit Internationalisation into modules.

I remember adding lots of /i18n to lines and reading the code to know where to put it - like a dry book (The Silmarillion anyone?) but at least we gave it a go

[Flint1b]

Thus spake Flint1b:
> Is there some way I can reproduce this memory issue so I can see for
> myself?

You'd need access to the machines where it occurs, which I've never had.
What I can tell you is that I've given users commands to try and they've
reported back that Java won't start with an initial heap far, far less
than the amount of free RAM they have. (IIRC, I once had someone who
was consistently unable to get Java to run with a 1GB heap when they had
8GB RAM and Windows was telling them they had something like 4GB free...)

--
J.

I wonder if it is this issue:
- https://www.oracle.com/java/technologie ... heap_32bit
- https://stackoverflow.com/questions/171 ... windows-xp

A quick search given the symptoms found these, they all seem to stem from 32bit JVMs, and solvable by switching to 64bit. In 2020, with most if not all CPUs of the last 10-20 years being 64bit, and some OSes dropping 32bit support altogether, does Vassal support 32bit or would it be possible to tell the users they need a 64bit JVM?

And:

Well I'll nominate 12554 as one that would probably help a decent number of users (based on marktb's saying "it is known" in the community that Undo screws up a lot), but HOOOO BOY it's a bear. I put some research posts into the bug entry. I *hypothesize* that if we made every PieceMover-type method that currently generates MovePiece sequences for Stacked units do a RemovePiece-then-MovePiece-then-AddPiece for every single move, that the problem would be fixed... but sheesh.

This looks like a good one to get to learn the code from the bottom up, I will look into it.

[uckelman]

Thus spake Flint1b:
>
> I wonder if it is this issue:
> - https://www.oracle.com/java/technologie ... heap_32bit[1]
> - https://stackoverflow.com/questions/171 ... windows-xp[2]
>
> A quick search given the symptoms found these, they all seem to stem
> from 32bit JVMs, and solvable by switching to 64bit. In 2020, with most
> if not all CPUs of the last 10-20 years being 64bit, and some OSes
> dropping 32bit support altogether, does Vassal support 32bit or would it
> be possible to tell the users they need a 64bit JVM?

Years ago, people were running 32-bit Windows and thus having only 4GB of
address space. So I know in some cases that was the problem: there
simiply was no block of address space large enough. I haven't encountered
that in a while, as it seems like all the 32-bit Windows systems are
retired now.

The JVM which will be bundled with the Windows package is 64-bit, but
there's nothing in the code which would keep VASSAL from working with
a 32-bit JVM. Given that we're bundling on Windows from 3.3.0 onward
and I don't turn up any places to download a 32-bit JVM in a minute of
looking, I expect we'll see basically no one running a 32-bit JVM
anyway, so there won't be any practical difference between saying we
support it and saying we don't.

That said, I know I've seen the problem I was describing many times
when the user had a 64-bit JVM.

> "Cattlesquat" wrote:
> > Well I'll nominate 12554 as one that would probably help a decent
> > number of users (based on marktb's saying "it is known" in the
> > community that Undo screws up a lot), but HOOOO BOY it's a bear. I put
> > some research posts into the bug entry. I *hypothesize* that if we
> > made every PieceMover-type method that currently generates MovePiece
> > sequences for Stacked units do a
> > RemovePiece-then-MovePiece-then-AddPiece for every single move, that
> > the problem would be fixed... but sheesh.
>
>
> This looks like a good one to get to learn the code from the bottom up,
> I will look into it.

Go for it.

--
J.

[Flint1b]

Yet another question from yours truly.

I see a lot of class fields with "protected" visibility. I have not counted but it feels like the majority of the fields are protected. This is very uncommon for me, I have learned long ago that composition is preferable to inheritance, and that fields should be marked protected if and only if the class has been designed with inheritance in mind. In todays corporate Java world I would say 99% of the fields are private, the methods are either public, private, or sometimes with package visibility so they can be accessed by unit tests, and even if there is inheritance, the subclasses often access the private fields of the superclass using the public getters/setters. Might seem like an overhead but the JVM can optimize this away at runtime by inlining these accessor methods, modern JVMs can even turn often used code hotspots into native code. The general idea is to hide as much as possible from the outside world and from subclasses, and to only allow access to fields through accessor methods, never directly.

Are all these protected fields in Vassal a kind of public API for the modules, to allow them to inherit from Vassal's core classes and add/overwrite behaviour?

Or is this a legacy of the C++ developers and the 80ies/90ies style of OOP where inheritance was always a good thing and let's make everything protected by default in case we want to inherit from it later on?

[Cattlesquat]

However/whyever they once arose, I can definitely tell you that having most of the stuff be protected is absolutely critical for creating custom classes for modules (so in other words, yes, effectively a public API).

(And yes, I'm old enough to have worked professionally in the all-inheritance-all-the-time world, in C++)

Brian

[Brent Easton]

Are all these protected fields in Vassal a kind of public API for the modules, to allow them to inherit from Vassal's core classes and add/overwrite behaviour?

Mostly this.

Early on, little or no thought was given to the design of the classes to allow Module Developers to develop custom code. Many classes had private members without even protected accessibility. It was next to impossible to write properly functioning custom classes except in the simplest of cases.

Like many other parts of Vassal, custom module code was a neat demonstration of capability, but with no formal underpinning.

The primary Vassal 'API' is the module editor, I don't think the designer ever envisaged or wanted people to be writing large amounts of Java to implement their modules.

[Flint1b]

I understand. So things are even worse than if these fields were due to weird OOP practices.

Another question, I have tried several modules, deleted some of them afterwards, now I have 1.3GB in my ~/.VASSAL/tiles directory. I guess each of its subdirectories belongs to a module? Is there any way of telling which of them belongs to which module? Is there any cleanup procedure? Can I delete all of them and just reopen the modules that I kept to rebuild the tiles?

[JoelCFC25]

Can I delete all of them and just reopen the modules that I kept to rebuild the tiles?

I don't know about other questions, but the answer to this one is definitely "yes".

[Brent Easton]

And you can delete them from the Module Manager Tools -> Clear Tile Cache

[Flint1b]

A couple more, this time concerning custom code.

I have looked into what I thought was a quite sophisticated module and simply had to have custom code, but only found images in it, several premade savegames which seem to be the scenarios, and the buildfile XML, no custom code, no Java classes or anything. I'm curious how a module with custom code works, is it subclasses only or are original vassal classes sometimes overridden, perhaps anyone could show me a module that has this so I can dissect it.

Another question is, can an estimation be made as to how many or what percentage of modules use custom code, and also what parts of the "public API" is actually in use by modules? And several other questions that are deducted from this one, are all modules known, are 100% of vassal modules hosted on this site's wiki or are there rogue modules that are not found on this site? And if it is not known which parts of the public API are actually in use, would it make sense to write a program that parses all the modules and spits out a list of which module uses exactly which class/method of the public API? And another question, probably more for module developers than for vassal developers, are module developers generally aware of the possibilities to extend vassal by writing custom code, or are they happy and content with the module editor and these simple beanshell scripts they can write in the editor?

[Cattlesquat]

Some modules that have custom code. The first two on the list are mine, so I include the source code in the modules as well.
* For the People 3.2 - whole bunch of stuff, including Chatter w/ HTML/colors, some UI-code to interpret commands a different way, a "flare map", a visually tweaked Turn Tracker, and a few miscellaneous other things.
* Paths of Glory 9.7 - Chatter w/ HTML/colors, a "flare map", and a couple tweaks to Turn Tracker.
* Twilight Struggle - a "chess clock" among other things

[uckelman]

Thus spake Flint1b:
> Another question is, can an estimation be made as to how many or what
> percentage of modules use custom code, and also what parts of the
> "public API" is actually in use by modules?

Sort of.

I can get a list of VASSAL classes which are used by custom code in a
module using jdeps and a bit of shell scripting. I can run that on
all the modules on our site and get a list of all the classes which
are in use by all the modules we host. I did this a few weeks ago
to check if it was safe to remove a few long-deprecated classes.

Unfortunately, jdeps doesn't give you function or field information,
only class dependencies---at least not that I can see---so I can't
tell if anything uses particular deprecated public or protected member
functions, which would also be nice to remove. (On that point, I'm not
entirely certain what will happen with subclasses in custom code if
one removes fields or functions they don't use. I know it wouldn't
hinder recompilation, but we're not in a position to recompile those
classes. So I don't know if this is a viable option.)

This is what you can do with jdeps:

jdeps -verbose:class -e 'VASSAL.*' x.jar

Note that it throws an exception if you try to give it a filename which
does not end in '.jar', even though a JAR is just a ZIP archive, and so
is a module. Apparently they did this to be irritating, because it had
to be extra work to reject files which it can otherwise process just fine.

> And several other questions
> that are deducted from this one, are all modules known, are 100% of
> vassal modules hosted on this site's wiki or are there rogue modules
> that are not found on this site?

There are definitely modules we don't host. It's hard to know what
proportion of all modules in existence that is, but my guess is that
we host the vast majority.

> And if it is not known which parts of
> the public API are actually in use, would it make sense to write a
> program that parses all the modules and spits out a list of which module
> uses exactly which class/method of the public API?

Yes. It would be a nice thing to have, as then we could do what I said
above---find any deprecated functions nothing we hosts uses, and remove
them. (Better might be to remove deprecated functions which aren't used
by _current_ versions of modules, but that's trickier, as you can't tell
just from looking at a module whether it's the current version.)

--
J.

[Flint1b]

Thanks for taking the time to answer.

Jdeps is indeed a very high-level tool and only works on package- and class-level dependencies, and also seems to have issues with generics and it's type erasure. There are other good tools though that can find field-level and method-level dependencies, there are also bytecode manipulation frameworks like asm or cglib that can be used to analyze existing bytecode. Some dependencies sadly cannot be detected at all, references to static final fields (constants) of type String for instance are optimized away by the javac compiler and inlined into the bytecode (a glorious "optimization" e.g. when you have few files with huge static final SQL statements for tables with 150+ columns that are referenced from all over the codebase, leading to megabytes worth of strings being duplicated).

Removing unused fields or methods should not break existing bytecode, they should keep working if the fields and methods they reference keep the same interface.

I will look some more into this and try to find a decent solution.