Test builds for 3.3.0

I added a module from a new location. VASSAL asked for permission to access that location. And then VASSAL worked again like I thought it did before. It saves the Module Manager list, and has access to the modules.

When I manually restore the RecentModules in V_Global then it somehow changes the signature of VASSAL and it launches it in a new sandbox. Hands Off thinks it’s a new app, and it lost file system access again. It doesn’t re-ask for access, so it still has it, but Module Manager says it doesn’t when checking the files.

This makes it even more important that the Module Manager doesn’t reset the list as the only way to restore it is by manually one-by-one re-adding the modules you had.

Once the permissions are messed up for Module Manager they are messed up. Also for 3.2.17. :frowning:

Thus spake bdgza:

[This message has been edited.]

I added a module from a new location. VASSAL asked for permission to
access that location. And then VASSAL worked again like I thought it did
before. It saves the Module Manager list, and has access to the modules.

When I manually restore the RecentModules in V_Global then it somehow
changes the signature of VASSAL and it launches it in a new sandbox.
Hands Off thinks it’s a new app, and it lost file system access again.
It doesn’t re-ask for access, so it still has it, but Module Manager
says it doesn’t when checking the files.

This makes it even more important that the Module Manager doesn’t reset
the list as the only way to restore it is by manually one-by-one
re-adding the modules you had.

Once the permissions are messed up for Module Manager they are messed
up. Also for 3.2.17. :frowning:

So… what you you suggest as a solution?


J.

It would detect a difference between “operation not permitted” and “file is not there”, and maybe make an assumption that the former on Mac means a permissions problem due to Catalina. The file should then not be removed from the Module Manager, but can be re-read (to ask the user for Permission to access), or this is retried when you try to launch the module (as if you added it). As long as you don’t lose your entire Module Manager list of modules each time there is a minor issue, and the user can rectify it, perhaps with a manual action, that would be less bad.

The biggest problem I have with VASSAL at the moment is the losing of sync in the Memoir '44 module. When moving pieces in a live internet game the sync is lost and it’s not seen on the other side. This happens constantly, not consistently, but in every game at least once. I haven’t been able to track down anything in my VSAV game files that could be causing this. I haven’t seen any errors in the log. Playing back a vlog doesn’t provide any useful clues, it just loses sync. So far I don’t know if it’s something I do to my VSAV files that I can’t figure out, a problem with my module I can’t figure out (too much in it?), or a bug in VASSAL. I have been trying to find the problem by myself, but I am stumped.

I am playing now a VASSAL 3.3 vs 3.2.17 game and it happened. I want to try to see if it happens with 3.3 vs 3.3.

Is the only change to 3.3.0 a Java platform update? Or, are there other feature additions/changes in the pipeline?

Thus spake eljayplay:

Is the only change to 3.3.0 a Java platform update? Or, are there other
feature additions/changes in the pipeline?

Yes, that’s all. Upgrading to a current version of Java fixes some
problems people are having presently, and that should keep VASSAL usable
while we work on V4.


J.

I might have fixed the remaining issue with HDPI support just now. Test builds will follow later today.

Found another snag. Maybe no test builds for a few more days.

Map, Board, and StackMetrics are adjusted for HDPI support now. Still to go are GlobalMap, MenuDisplayer, PieceMover, and CounterDetailViewer.

CounterDetailViewer is adjusted now.

Current state of things:

  1. I have set up a VASSAL repo on github: github.com/uckelman/vassal

I had not been intending to do this for V3, on the assumption that by now we’d be working on V4; however, the work for V3.3 has reminded me of just how painful Subversion is in comparison with Git and I can’t stand it anymore.

I’m not abandoning our svn repo, but the rest of the work I do for V3.3 is going to happen using git and then be pushed back to svn later.

The two branches in the git repo are master, which is the same as master in svn at present, and hdpi, which is where I’md going the HiDPI work.

  1. Traditionally, one screen pixel has equaled one “user space” pixel. This changed when Apple introduced their “Retina” displays some years ago. On a Retina screen, each user-space pixel is twice as wide and twice as tall as a screen pixel—i.e., each user-space pixel corresponds to four screen pixels. Screens where user-space pixels correspond to multiple screen pixels are HiDPI screens.

Through Java 8, Java treated HiDPI displays as regular displays, which is why Java applications running in those versions of Java looked very small on HiDPI displays. Java 9 has HiDPI support for Swing components—a Swing application (as VASSAL is) running on Java 9 or later on a HiDPI display will look the right size, since the Swing components in Java 9 are drawn to allow for a difference betwen user and screen pixels. In order to accomplish this, how all the standard Swing components are drawn had to be modified. (Specifically, each component has to be aware that its user-space dimensions differ from its screen dimensions by a constant factor.) We get all this for free with the standard components when moving from Java 8 to Java 9—but we’re stuck undertaking that work for any custom components we have which draw themselves. One such component is the map. There are no standard Swing components which are even remotely close to being able to handle displaying game maps.

  1. With Java 8 on a HiDPI display, Swing applications are drawn at half scale. With Java 9 (or later) on a HiDPI display, Swing applications which haven’t been updated for HiDPI have their custom components drawn at user-space size but then upscaled to fill the pixels, with predictably crappy looking results. This has the virtue of keeping the applications nominally functional, but makes anything your application draws quite blurry. The correct solution is to paint on the basis of how many screen pixels you have to fill, so that no upscaling occurs. To take a concrete example: If a map is being displayed at 25% zoom on a HiDPI display where each user-space pixel is four screen pixels, then we need to treat the map’s zoom as 50% when drawing (25% map zoom * screen scale factor of 2).

That sounds simple, but in practice it is not—the reason being that drawing and component interaction had previously always taken place in the same coordinate space, so there are no indicators in the code for which are which, and coordinates originating from component interaction necesasrily get fed into drawing when doing things like dragging pieces.

  1. So, if you look at Map.java, you’ll find Map.View, which is the Swing component which displays the map. Map.View.paint(Graphics g) is the function which paints the map, and that’s where we have to start.

That Graphics is actually a Graphics2D, so you can cast it to one and get an AffineTransform from it:

  final Graphics2D g2d = (Graphics2D) g;
  final AffineTransform orig_t = g2d.getTransform();

If you’re running Java 8 or using a display which is not HiDPI and you check the scale elements of the AffineTransform (using getScaleX(), getScaleY()), you’ll find that they’re 1.0. If you’re using Java 9 or later on a HiDPI system, however (or setting the sun.java2d.uiScale property to simulate a HiDPI system—more on that later), you’ll find that your scale elements are 2.0. Now we see how exactly Java 9 handles upscaling custom Swing components which haven’t been modified for HiDPI support—it’s the scale factor on the AffineTransform which does it.

So, how do we adjust this to avoid upscaling and get crisp drawing back? What we need to do is save the AffineTransform we’re given and replace it with an indentity transform but then draw everything with an effective zoom that’s our actual map zoom multiplied by the scale factor the original AffineTransform had.

In service of this, I’ve expanded the coordinate space transformation functions in Map to handle conversions to and from drawing coordinates, because we have to deal with three coordinate spaces now—map, component, AND drawing, whereas previously component and drawing coordinates were always identical. These functions all have names of the form “aToB”, where “a” and “B” are coordinate spaces. E.g., componentToDrawing() converts component coordinates to drawing coordinates, while drawingToMap() converts drawing coordinates to map coordinates.)

Map.View.paint() doesn’t do much itself, but there we convert the visible rectangle from component space to drawing space before passing it on to Map.paintRegion():

  final Rectangle r = map.componentToDrawing(getVisibleRect());

Map.paintRegion() calls in succession the functions which clear the map border, paint the boards, and draw the pieces. What I’ve done in each of these is similar, so I’ll take Map.drawBoardsInRegion() as an exmaple and omit discussion of the others.

  public void drawBoardsInRegion(Graphics g,
                                 Rectangle visibleRect,
                                 Component c) {
    final double dzoom = getZoom() * os_scale;
    for (Board b : boards) {
      b.drawRegion(g, getLocation(b, dzoom), visibleRect, dzoom, c);
    }
  }

The change I’ve made is simply to the scale factor which gets passed on: It used to be Map.getZoom(), but is now getZoom() * os_scale (which is the HiDPI scale factor). That’s it. Everything further down this call tree is expecting to be told a scale factor and to draw at that scale factor, so by adjusting the scale factor here, it does the right thing. (*This is not 100% true, I had to make a small modification in Board.java as well, but for most things we draw it is true.)

  1. I’ve made a bunch of changes as in #4 which get us most of the way back to correct rendering. However, there are a few difficult components which involve both rendering and user interaction which are not yet done: GlobalMap, MenuDisplayer, PieceMover for sure; possibly also LOS_Thread, FreeRotator, MapCenterer.

In order to troubleshoot these, you need to run with Java 9 or later, and either use a HiDPI display OR set the sun.java2d.uiScale property to force Java to upscale your UI. The following has been my test setup:

  java -classpath lib/Vengine.jar --add-exports java.desktop/sun.java2d.cmm=ALL-UNNAMED -Dsun.java2d.uiScale=2 VASSAL.launch.Player --standalone ../mods/The_caucasus_campaign_1_2_1.vmod

(Note that 2 is likely the only value which makese sense for uiScale. 1 is just normal, so you won’t be able to troubleshoot that way. Non-integers seem not to work, and anyhow would be awful for aliasing if they did. 3 is way too huge.)

Particular problems:

  • GlobalMap is very broken. It needs a complete read-through.
  • MenuDisplayer handles showing context menus when you right-click. The problem to solve here is that the map jumps when you right-click under HiDPI.
  • PieceMover handles piece drags, among other things. The problem here is incorrect repainting behind drags.

There are likely to be other problems I have yet to find. I’d appreciate being made aware of those and shown how to reproduce them.

  1. If you’d like to help with this, clone the git repo and start reading through the relevant code.

Baseline : git commit id 29c901de8cfd8a93cd144a3d509d1f021b55e57c

I could not do a proper compile of ImageIOImageLoader due to a missing class named sun.java2d.cmm.ProfileDeferralMgr.

Well I ran the tests on my Windows Box with different JDKs and I noticed these things

TileToImageTest :: testTileToImage - differs but image looks the same
→ maybe read in.png and out.png again and compare the loaded Image ?

ProcessCallableTest :: testNormal differs by 1 byte (longer on windows) and the test contains ‘\n’ sequences so most likely OS dependent test.

VASSAL.test.AllTests

  • may be required due to using make.
  • IDE integration (here Eclipse) complains about its presence.

Baseline : git commit id 7255ef7cadb977f0d3b86ab6499def27ec224760

Situation is basically the same, but when running the tests on Java 14
TileToImageTest :: testTileToImage is green.

Here is an observation on the code:

A lot of felt weight comes from dispersed explicit low-level math.
Basically we have a super tight coupling with Framework code.
So these calculations are treated 2nd class compared to libary code.

Decoupling stuff and treating 2d-Math as the core-domain of Vassal-the-UI-Library will help.
We are talking about Vassal specific representations of Point, Rectangle or Dimenions, etc.

I would start with Vassal.internal.math.Rectangle2D class as there is a lot of drawing / collission code that can be moved to a dedicated spot. Once that is out of the way a clearer picture will emerge.

Seamless integration is key. So being able to “cast” between Vassal / AWT / Swing types is crucial.
So getting back to AWT / Swing should be a simple .toAWT() or .toSwing() call on the Vassal-Object.
Active refactorings will quickly make the actual functionality emerge - so don’t worry too much about it.

Unless we’re talking about FULL images the memory offset is effectively neglectible.
However there are significant effects to development speed due to readability, testability and comfort.

I had multiple instances where performing such changes had multiple positive effects at the same time that sound contradicting in the first place:

  • reduced technical complexity of an application code
  • extending feature-complexity drastically
  • performance gains - probably because the Hotspot JVM Compiler kicked in for the first time

I’ve fixed the problems with display of context menus and repainting while dragging and scrolling (which I thought were with MenuDisplayer and PieceMover, but turned out to be due to an oversight in handling the AffineTransform in Map.View.paint()).

Left to fix are GlobalMap, LOS_Thread, FreeRotator.

GlobalMap is fixed now.

FreeRotator is fixed now.

Started looking at PieceMover, however it is quite a beast to understand.

Thus spake AlisterMcLane via messages:

Started looking at PieceMover, however it is quite a beast to
understand.

No need. It turned out that there was nothing to fix in PieceMover.

What’s on the hdpi branch presently has fixes for every problem I’ve
found save for the repaint problem in LOS_Thread I’m working on at
the moment.


J.

I’ve now fixed every HiDPI problem I’ve seen.

Please try the 3.3.0-svn9303 builds here: vassalengine.org/~uckelman/tmp/