3.3.3 release plan

[Cattlesquat]

We do need something describing at least the new features in enough detail
that people who are interested can discover what they are.

Okay here are the remaining items from my "documentation sprint" that I would ideally get done in time for 3.3.3, in rough order of my current prioritization:
(1) [DONE] All screenshots that needed updating because of 3.3.3
(2) [DONE] Full "edit pass" on existing articles; many articles expanded
(3) [DONE] Basic alphabetical Table-of-Contents of subcomponents for Module, Map, and Game Piece pages
(4) [DONE] HTML Chatter documentation - own article, plus mentions in Report Action, Message Format, and Module articles.
(5) Finish my half-done "Toolbar" article (Mommy, where do buttons on the Toolbar come from?)
(6) Updates to Expression & Property Match Expression articles to account for Brent's latest improvements.
(7) "New Features List" per above, for announcement of the various designer-facing and player-facing improvements in 3.3.3
(8) If there's time, I'll start getting together a little "quick links" navigatio section to go on the bottom of every page.

It's not *that* many more days of work. Of course as soon as I see the Chatter get merged I'm gonna be going "crap! crap! crap!" and trying to get it all typed in quick

Stuff that could fall "below the line" but I could do sooner if desired:
(9) A decent "Landing Page" for the online Help -- something that helps you Browse Your Way To Goodness.
(10) Remove the Editor's "Help" link that leads to the PDF file and point it at the new Landing Page (small code change)
(11) A pass through the UI (i.e. code) to consistently refer to "Key Command" rather than e.g. "Keystroke", "Keyboard command", "Keyboard shortcut", "Shortcut", etc, etc, etc. Obviously "Hotkey" is still a qualitatively Different Thing. Obviously this would also involve re-taking a bunch of screenshots, lucky me. But I've definitely got THAT down to a science now.

Brian

[Cattlesquat]

Does it make sense to pick some date in the near future and aim for that as "beta 1 date" as a kind of "milestone"? Friday? Next Monday? Some other day?

[Flint1b]

Let's do it like other modern projects and bring out a beta every week, or every other week or something like that. Rolling releases. Whatever gets merged until then gets into the beta.

And for the hardcore Vassal enthusiasts, a nightly build every day. @Joel do you think you could automate the build so we can give it to Travis'es naked ubuntu, have it download all prerequisites and run the full release build? Shouldn't be too much left, we already have automated JDK download, some of the tools can be installed by ubuntu's apt, just these exotic ones that need to be compiled manually seem tricky but it's no rocket science to setup an ubuntu to build C stuff. Then we'd have to handle automatic versioning, tagging and deployment of the built packages, shouldn't be too complicated.

[uckelman]

Thus spake Cattlesquat:
> Does it make sense to pick some date in the near future and aim for that
> as "beta 1 date" as a kind of "milestone"? Friday? Next Monday? Some
> other day?

I see two PRs which should still be merged---the ones for 13129. I've
been trying to plow through my message backlog this evening rather
than tend to PRs. If you're satisfied with those two, I'll get them
merged soonish.

Do the docs actually need to be updated for beta1?

I'm expecting that with this quantity of changes, we'll have a period of
fixing bugs between beta1 and the subsequent release, and even if there
aren't bug reports we need to have beta1 out for at least 10 days to
afford sufficient opportunity to try it. So I expect that you'll have
ample time to finish the doc update before 3.3.3 even if you're not done
by the time we put out beta1.

Does that sound ok? If so, we can release beta1 once those two PRs are
merged, maybe tomorrow evening? Or would you prefer to have the updated
docs go out with it?

--
J.

[Cattlesquat]

Okay yeah, if we're okay with some docs getting finished up during beta then we're probably getting close to a point that we could pull the trigger. Give me an "aim point" date for beta 1 and I'll try to have things at least be "clean" by then. And I'll also try to get a "new feature page" together though again that can improve over several betas.

[uckelman]

Thus spake Flint1b:
> Let's do it like other modern projects and bring out a beta every week,
> or every other week or something like that. Rolling releases. Whatever
> gets merged until then gets into the beta.

Uptake of new versions has always struck me as slow. Would a faster release
cycle encourage users to upgrade faster? I'm not sure why I should expect
that over having more versions in use simultaneously.

Have you ever played an American Civil War games where movement can
generate stragglers? (The Gamers' CWBS is like this.) The faster you march
the more stragglers you get...

What benefit you foresee of having more frequent (beta) releases?

> And for the hardcore Vassal enthusiasts, a nightly build every day.
> @Joel do you think you could automate the build so we can give it to
> Travis'es naked ubuntu, have it download all prerequisites and run the
> full release build? Shouldn't be too much left, we already have
> automated JDK download, some of the tools can be installed by ubuntu's
> apt, just these exotic ones that need to be compiled manually seem
> tricky but it's no rocket science to setup an ubuntu to build C stuff.
> Then we'd have to handle automatic versioning, tagging and deployment of
> the built packages, shouldn't be too complicated.

Yes, I'm sure I could automate the build. Can we actually download builds
from Travis?

--
J.

[Flint1b]

I think if we made it bi-weekly, it wouldn't be much different from we currently have, the betas already get released roughly every two weeks, except if there is a regular release.

And the nightly releases, you already do those custom builds almost every day, it would be the same just instead of having to build them manually you would tell the people "hey try this nightly from yesterday, see if your bug is fixed, here is the link".

Can we actually download builds from Travis?

No Travis is just a "dumb" build server without a filesystem where things get saved, it's the other way round, we write a script that tells Travis to upload the artifacts somewhere, I think we could upload them to GitHub, to our own server, or to some 3rd party site like Bintray.

We could write the code for that in the Makefile, or an extra shell script we call from the Makefile, setup Travis to do an automatic build of current master every 2 weeks / every night, add some logic which detects that and runs a full release build instead of the usual "mvn clean test". At the end of the day Travis is just a regular ubuntu machine and can do anything that we tell it to do in our shell script.

[uckelman]

Thus spake Cattlesquat:
> Okay yeah, if we're okay with some docs getting finished up during beta
> then we're probably getting close to a point that we could pull the
> trigger. Give me an "aim point" date for beta 1 and I'll try to have
> things at least be "clean" by then. And I'll also try to get a "new
> feature page" together though again that can improve over several betas.

I didn't have as much time as I expected today, as I fixed a floating point
bug in FreeRotator that's likely been there since it was written.

--
J.

[Brent Easton]

Prior to the 3.3.3 release, I would like to try and handle the Deprecated method removal more gracefully.

There is also an issue with the SNAPSHOT property Map change that breaks all of my modules, and possibly quite a few others. I'm in the process of updating my custom code, but we will be inundated with bug reports if we don't do something at the Vassal end.

Yes Yan, I know, we need to get rid of the crap and get module owners to update and all that, but rather than break all the modules, I would like to go the route of the annoying popup messages that will drive the users of such modules batty. I propose the following implementation of Joel's 'stupid simple' approach that I am happy to work on and implement for 3.3.3. This can also be applied to any new methods made Deprecated in the future.

1. Replace all recently removed Deprecated methods.
2. Add a Call to popup a Disableable ProblemDialog each time a Deprecated method is called. Each different Deprecated routine called will popup a new Dialog, even if you disabled the last one.
3. Include a Deprecated date in the popup method call, to generate something like 'This routine will be removed after 1-Aug-2021' which will be 1 year after Deprecated date.

In future cleanups, anyone updating a class with a method that has a Deprecated date more than a year in the past can remove that method without guilt.

The Problem with the SNAPSHOT change is that ReportState fails with a Class Cast Exception if any module has custom code that includes the setProperty of SNAPSHOT using the PieceCloner. We have no idea how many modules have custom traits that save the SNAPSHOT as a GamePiece as no Deprecated methods are called in this case. My modules do because I built custom versions of existing classes that did this, so I copied that in my code, I am sure others will as well

The simplest fix is for ReportState to check the type of the SNAPSHOT returned, and if it is a GamePiece, then just export the properties at that point into a map. This works, at the expense of a performance hit for the custom code as it is basically doing both the old and the new method. I propose that at this point we also popup an appropriate 'Deprecated Code' dialog when we see this.

I think this is a good compromise. We still get to remove the cruft (after a year), but we get the message in the face of the players and designers and give them a change to fix the problem before we get the support hassles of dealing with it.

Thoughts?

[Flint1b]

In that case I propose some kind of automatism, since manual handling of all these deprecation notes and dialogs will be way too much work. Something like a special format for the deprecation comment and the date in it, and a code scanner that regularly scans the codebase, finds these comments and reports which of them are due for deletion.

Maybe even a custom annotation, something like

Code: Select all

@Deprecated
@VassalDeprecated(since = "2020-08-05")
void method() {
}


This actually looks cool, I've never written custom annotations but they are powerful, I think we could hide the whole code for showing the dialog behind them. Then a custom maven plugin that compares the date in these annotations with current date and spits out a warning if a year has passed.

One question that remains is, how to handle deprecated fields? In a method call we can simply add the code for showing the dialog but how do we do that for access of deprecated fields? Could custom annotations handle that? I don't know.

Another question that I have right away if we do this, can I go on and mark 99% of all protected fields and methods as deprecated? Just in case we might want to delete them in a year. Many (or even most?) of the protected fields and methods are not even meant to be called by subclasses, many classes that have these fields are not even meant to be subclassed, they were just made protected due to ... I don't know, old-school object-oriented approach where everything is per default protected? We have this spotbugs plugin now for instance and it detected a lot of "protected static" fields that should be final but are not, they are constants, we could and should make them final but we don't know for certain whether modules assign new values to them. If they were private it would be easy going but thanks to this "protected by default" way of writing code we have a lot of trouble managing this code, and a lot of ways for custom modules to break Vassal by simply assigning new values to core constants.

Small example to make things more clear:
- here https://github.com/vassalengine/vassal/ ... .java#L423
- and here https://github.com/vassalengine/vassal/ ... .java#L444

So in a module I write this class and break core functionality, thanks to glorious protected visibility:

Code: Select all

class VassalBreaker extends ImageIOImageLoader {
  public VassalBreaker() {
    readImage = null;
    readSize = null;
  }
}


Now some will answer "but that's highly hypothetical, no module should and would do such a thing" but I ask, why then allow this in the first place, why the F make these things protected, why open doors instead of hiding core code behind safe walls?

Overall, I am ambivalent on this issue. Generally I am for marking much more as deprecated and deleting it ASAP, or hiding it from the public API by changing protected to private. But if we do this one-year thing I am for investing some more time and thought up front and building a decent "deprecation system", and then having a smooth ride in the years to come where we just get notified by the build server of what can be deleted.

[Brent Easton]

But if we do this one-year thing I
am for investing some more time and thought up front and building a
decent "deprecation system", and then having a smooth ride in the years
to come where we just get notified by the build server of what can be
deleted.

I agree, let's do it properly. I'm not sure that we can build the Dialog display into the Annotation though.

Let's start by building that and implementing it for the recently departed @Deprecated methods. I think we should also apply it to the other @Deprecated routines that you didn't remove because they are used by modules hosted on Vassal. That will get the process rolling.

Then we can discuss a plan to make more methods @Deprecated. The problem with Deprecating every protected method in a class is you have to provide alternate names for every Method, so it will be a painful process.

And yes, @Deprecated fields are a problem.

[Flint1b]

About hiding protected fields/methods, I wouldn't see a "deprecated" thing as one that will have to be necessarily deleted. I think it's legitimate to mark something as deprecated and later on change it from protected to private and remove the deprecated mark.

I looked a bit into custom annotations, they are not as powerful as I thought, we cannot hide any implementation behind them. But we could use them as markers that get picked up by the build server, we would need a separate project and the structure would be something like this:

Code: Select all

- vassal-deprecation-parent
-- vassal-deprecation-core: contains the annotations
-- vassal-deprecation-maven-plugin: depends on vassal-deprecation-core, scans code during build, looks for the annotations and reports

- vassal-parent
-- vassal-app: depends on vassal-deprecation-core, adds annotations to code, uses vassal-deprecation-maven-plugin during the build


[uckelman]

Thus spake Flint1b:
>
> I looked a bit into custom annotations, they are not as powerful as I
> thought, we cannot hide any implementation behind them.

Yes, unfortunately they're not decorators like they are in Python.

--
J.

[Brent Easton]

I was thinking of something simple like this using the existing @Deprecated tags as follows.

I'm not quite sure what you are suggesting Yan. Are you saying we auto-generate the 'ProblemDialog.showDeprecated ("2020-08-05");' line based on the existence of the @Deprecated tag?

Map:

Code: Select all

 
  @Deprecated(since = "2020-08-05", forRemoval = true)
  public Rectangle componentRectangle(Rectangle r) {
    ProblemDialog.showDeprecated ("2020-08-05");
    return mapToComponent(r);
  }

ProblemDialog:

Code: Select all

  public static Future<?> showDeprecated(String date) {
    final StackWalker.StackFrame frame =
      StackWalker.getInstance(Set.of(StackWalker.Option.RETAIN_CLASS_REFERENCE), 2)
        .walk(f -> f.skip(1).limit(1).collect(Collectors.toList())).get(0);
    final String method = frame.getClassName() + "." + frame.getMethodName();
    String expiry;
    try {
      expiry = LocalDate.parse(date, DateTimeFormatter.ofPattern("uuuu-M-d")).plusYears(1)
        .format(DateTimeFormatter.ofPattern("d-MMM-uuuu", Resources.getLocale()));
    }
    catch (Exception ignored){
      expiry = "some later date";
    }

    // FIXME I18N
    return showDisableable(JOptionPane.WARNING_MESSAGE,
      null,
      null, method,
      "Out of date Custom Code",
      "Out of date Custom Code",
      "This module contains Custom code that is calling the VASSAL function " + method +
        ". This function is scheduled to be removed from VASSAL after " + expiry +
        " and this module will cease to function.\n\nPlease check whether there is an updated version of this module. If not, please contact the maintainer of this module and request that it be fixed."
    );
  }

This produces

[Flint1b]

I wasn't sure myself, but I looked into it and while it would have been cool if we could get these things to auto-generate, we can't. We can still use annotations though to get the build server to notify us when the one year has passed, otherwise it would be more like a chore, having to manually scan through the code in a regular fashion and fishing out these deprecated tags. We would need a) a custom annotation for that and b) a custom maven plugin that we plug into our build and tell to find our custom tags, calculate the due date and once the date comes spit out a warning or even build error in the build log.