Test builds for 3.3.0

Hi Joel

I have also tested the latest version (hdpitest2), and also experience some weird artifacts when scrolling or dragging selection boxes. The entire map, unlike the previous version, in drawn though.

I have emailed you a video capture to give you better understanding of what is going on.

The selection lasso had been relying on the mouse coordinates being in the same space as what’s being drawn in the JPanel for the map, which is no longer the case. There’s a “hidpitest3” build now which should fix the selection lasso.

Hi Joel

I have tested the latest version you provided. Unfortunately the errors with the selection lasso and scrolling issue persists. I have tested with both 100% and 150% desktop scaling, and the issue seems to be the same in both cases. Note that the selection lasso correctly selects counters within its boundaries. The issue is that is top left corner of the map is always drawn inside the lasso.

On a side note, Java 14 will have official support for jpackage. This should make it possible to simplify the distribution build pipeline. A talk was held about it on the Devoxx conference this week.

https://www.youtube.com/watch?v=ZGW9AalZLN4

Thought you might find it interesting.

Thus spake bolabola:

I have tested the latest version you provided. Unfortunately the errors
with the selection lasso and scrolling issue persists. I have tested
with both 100% and 150% desktop scaling, and the issue seems to be the
same in both cases. Note that the selection lasso correctly selects
counters within its boundaries. The issue is that is top left corner of
the map is always drawn inside the lasso.

I’m not surprised there’s still a scrolling problem; I haven’t made any
attempt to address that yet.

I think I know why the map is being redrawn incorrectly inside the
selection lasso, on the basis of your description. Thanks for that.

Getting this right will involve going through all the code in Map and
all the classes which implement Drawable and ensuring that we have proper
separation between UI coordinates and painting coordinates.

This may take some time, but I don’t see any other options—we’re already
committed to upgrading to Java 9 or later, and that forces dealing with
HiDPI on us. Neither a release that renders blurry images nor one which
is illegibly tiny is a something I want to go out the door.

I’d welcome help with going through this code and making adjustments, but
this is a place in VASSAL where some of the grottiest old code collides
with some of the most highly optimized newer code, so it’s likely to be
rather hairy.

On a side note, Java 14 will have official support for jpackage. This
should make it possible to simplify the distribution build pipeline. A
talk was held about it on the Devoxx conference this week.

Given that I’ve already solved that problem without jpackage, it is, like
everything else about Java at this point, a day late and a dollar short.
I sincerely hope that 3.3.0 is the last release we do using Java.


J.

Yes, this was also what I initially feared. That this would be hard and very time consuming to fix.

Again, I think it is worth considering just using the -Dsun.java2d.uiScale=1.0 system property. This should make 3.3.3 work in exactly the same way as 3.2.17 with regards to scaling. Everything should work, but with no HDPI support as in 3.2.17 though.

That would allow you to make a new version running on a recent version of Java, and decouple the JVM from whatever the enduser has installed on his or her system.

I would love to help out on the coding side, but my knowledge of the codebase is extremely limited.

I wouldn’t write off Java completely for version 4, though. I don’t know if you have given JavaFX a look. It is a really impressive UI platform, with amazing performance (entire rendering pipeline is pushed to the graphics card) and the UI programming model that is a really good match for an application like Vassal (node based scene graph). And the UI components has a modern look-and-feel.

I created a demo application some years ago using JavaFX, which I have just updated to Java 13 / JavaFX 13.

With a couple of hundred of lines, you get amazingly far.

* Scrollable boardgame map
* Zooming
* Counter dragging
* Counter rotation
* Counter flipping
* Counter dropdown menus
* Animated transitions
* Property binding
* Selection box
* Complete HDPI support
* Dependency injection on controllers
* UI testable with TestFX.
* Modularized application.

You can find the demo app here: https://github.com/boleer/tabula-demo

You can find descriptions under the /site/apt:

https://github.com/boleer/tabula-demo/blob/master/src/site/apt/index.apt.vm
https://github.com/boleer/tabula-demo/blob/master/src/site/apt/why-javafx.apt.vm

The project uses Maven, so it should be relative easy to get up and running, you just need to install Maven 3.6.0 and JDK 13, and then run the following command:

  • mvn package -DskipTests
  • mvn exec:java

I have attached a screen dump of what the demo app looks like.

You can try to use the zooming slider to get an impression of how amazing the rendering performance is.

Thus spake bolabola:

Yes, this was also what I initially feared. That this would be hard and
very time consuming to fix.

I think it won’t be trivial, but it’s a problem where all the relevant code
is ours, which means there’s far less uncertainty involved than with, say,
determining how to bundle Java into our DMG. (We got it, but I could not
predict when or even if we would.)

Again, I think it is worth considering just using the
-Dsun.java2d.uiScale=1.0 system property. This should make 3.3.3 work in
exactly the same way as 3.2.17 with regards to scaling. Everything
should work, but with no HDPI support as in 3.2.17 though.

Are you sure? Even for people with HiDPI screens? I was under the
impression that this would make 3.3.0 very small for people with HiDIP
screens. Am I wrong here? If I’ve misunderstood the situation, I’d like
to know.

With a couple of hundred of lines, you get amazingly far.

It’s an interesting looking demo. I managed something similar with very
little code a few years ago in C++ using SDL. We have a huge amount of
code which would not have needed to be written if we’d had a different
design model, but I think this also shows that it doesn’t take much to
get halfway there, and that it’s the second half which is far more work.

I have a long list of complaints about Java at this point; after working
in Java for 15 years, I feel I’ve come by them honestly. One category of
these complaints concerns image decoding. Anything you do in JavaFX is
going to be afflicted by these as well, so far as I can see.

I have put hundreds of hours into investigating and working around image
decoding bugs in Java over the years, and hardly any of those bugs have
been fixed during that time. There’s one PNG loading bug in Oracle’s
tracker which has had an obviously correct patch sitting there for more
than a decade now. If you submitted a bug to libpng demonstrating that a
correct PNG was misdecoded, you might wait a few weeks tops for a release
that fixed the bug. It’s not as though there are only one or two such
bugs, either. I count workarounds for 13 different image decoding bugs in
our image loader right now.

And it’s not just problems with unusual but standards-compliant images.
The JPEG loader class in ImageIO isn’t thread-safe. That is not a typo,
the class is not thread safe. The whole class is not thread safe. You
literally cannot safely load JPEGS on two different threads
simultaneously even when each thread has its own JPEG decoder; if you do,
you risk an unchecked exception being thrown. It’s blindingly obvious why,
too, if you look at the code, and it’s shocking that they let it out the
door in the first place. This bug’s been around for ages as well. No joke,
the sole acknowledgement the bug report got from someone at Oracle was:
“We don’t have plans to fix it”.

Correct image decoding is vital to us; most of what we do is display
images in various arrangements. It’s been infuriating to have to devote
so much time to a problem that exists only because Java neither uses
the standard image loading libraries nor properly maintains its own.

I won’t willingly go down that road a second time.


J.

I’m not sure what info you need for HighDPI feedback but this is my setup:

iMac with Retina 5K display running Windows10 in BootCamp. Fairly recent Radeon drivers (not the BootCamp ones).
Display set to 5120x2800
“Change the size of text, apps, and other items” says 450%. If I click advanced scaling settings it says I have a custom scaling size of 200 and “Let Windows try to fix apps so they’re not blurry” is checked.

VASL 6.4.4 with VASSAL 3.2.17 and Java8 looks “normal” (compared to my time on the Mac side).
VASL 6.4.4 with VASSAL 3.3.0 svn9291 and Java13 from the installer looks “normal”.

Trying to go through the items described above, scrolling the VASL boards seems to work as expected.
I can drag the selection square around the counter fine.

From my limited time on Windows pre 3.3.0 I know that VASSAL/VASL would show up super tiny in Java8 but that would render “correctly” with later JVMs when the libraries were dropped manually. Or at least that was my impression.

Either way, I can provide whatever diagnostics or run any test you like if you need more feedback or more specific info.

No, I am not 100% sure, but I would think so. If you can provide me we a build with the system property set, I can test whether it works in exactly the same way as 3.2.17, (that is no scaling).

The image decoding is a completely different implementation in JavaFX, and doesn’t use or require usage of ImageIO in any way. It is also thread safe. It supports PNG, BMP, GIF and JPEG images out of the box. Custom decoding implementations (for SVG for example) can be provided to JavaFX by implementing custom ImageLoaders, but without the need to implement complex transform algorithms, as this be done by the JavaFX core (on the GPU).

All decoder implementations are pure java implementations, except for the JPEG implementation, which uses the latest version of the libjpeg library.

Looking at the code in the ImageIOImageLoader class in Vassal and all the workarounds you implemented to correctly decode JPEG images, I fully understand your frustration with the ImageIO, and Oracle’s unwillingness to fix the issues. It is amazing that you had the patience to go to such lengths to implement workarounds for something that Oracle should have fixed. And I agree, it doesn’t reflect well on Sun, that they released an Image IO library that you cannot use in the thread safe manner.

Again though, none of this would not be an issue in JavaFX, due to the completely different implementation. All the complex code in the image and imageop packages for image transforms, tiling and image decoding, which I assume was originally written to overcome limitations in AWT / Swing due to functionality limitations and poor performance. None of this is necessary in JavaFX. You simply load the images, and JavaFX handles all the logic around affine transformations (scaling, rotating, filtering, animations) by pushing the images to the GPU and executing the transforms there.

This luckily this is not the case with JavaFX. As you surely know Java (OpenJDK) and JavaFX (OpenJFX) has been open sourced, and has a very vibrant community, allowing for much faster turnaround on fixes, in part because of the new 1/2 release cycle, as well as a very active open source community around the OpenJDK and OpenJFX.

Using JavaFX also has the added advantage, that some parts of the Vassal codebase might be reuseable, and making old module plugins work would be a lot easier. I also think it would be a lot easier to find contributors.

Another of the main pain points with Vassal and Java during the years has been, as I understand it, that Vassal had to work on many different versions Java, due to the lack of control of the runtime environment on the end user machine. This is luckily no longer an issue, due to jlink and the option to package the Java runtime with the application.

I hope that this answers at least some of your concerns about using JavaFX.

If you need to me to test anything related to scaling, please to don’t hesitate to let me know. I’ll be more than happy to help out with the testing of Vassal 3.3.0. Zgrose can properly also be helpful in this regard, as he is also using a HDPI monitor.

Thus spake bolabola:

No, I am not 100% sure, but I would think so. If you can provide me we a
build with the system property set, I can test whether it works in
exactly the same way as 3.2.17, (that is no scaling).

I’ve uploaded a set of builds which launch with -Dsun.java2d.uiScale=1.0
set, named svn9292-uiscale, here:

vassalengine.org/~uckelman/tmp/

The image decoding is a completely different implementation in JavaFX,
and doesn’t use or require usage of ImageIO in any way. It is also
thread safe. It supports PNG, BMP, GIF and JPEG images out of the box.
Custom decoding implementations (for SVG for example) can be provided to
JavaFX by implementing custom ImageLoaders, but without the need to
implement complex transform algorithms, as this be done by the JavaFX
core (on the GPU).

All decoder implementations are pure java implementations, except for
the JPEG implementation, which uses the latest version of the libjpeg
library.

It sounds like they’re repeating the same mistake made with ImageIO,
namely that they’re not using the standard image libraries that the
entire rest of the software world uses. Nor does leaving out SVG
inspire confidence.

I hope that this answers at least some of your concerns about using
JavaFX.

Thanks for the explanation, but I’m not interested in continuing to use
Java in any form for VASSAL 4. Even if all my concerns regarding images
were satisfied, I still have an overwhelming collection of other reasons
for using somehting else. I don’t see much future in Java at this point,
and I don’t want us tied to it.

If you need to me to test anything related to scaling, please to don’t
hesitate to let me know. I’ll be more than happy to help out with the
testing of Vassal 3.3.0. Zgrose can properly also be helpful in this
regard, as he is also using a HDPI monitor.

Thanks. Give the build above a try. What I’m interested in particular is
if the UI scaling differs from 3.2.17. (If the scaling behavior is the
same as 3.2.17, then we can release 3.3.0 this way and fix HiDPI scaling
for 3.3.1.)


J.

svn9292 is horrible on my screen. 3.2.17 and svn9291 much better. Hard to tell in isolation but hard for me to resample the image at the moment. In 3.2.17 and svn9291, the two module lines are as big as the GitHub icon.

[attachment=1]Annotation 2019-11-14 161318.png[/attachment]

[attachment=0]Annotation 2019-11-14 162045.png[/attachment]

Wow, that is awful. I was worried that svn9292-uiscale would be very small, not upscaled. What version of Windows is that you’re using?

Windows 10 Pro, v 1903
build 18362.476

Ok, I am sorry you feel that way about Java. Seems there is no point in discussing it further.

I have tested the latest version you provided with the ui scale system property set, and compared it to version 3.1.17. It works in exactly the same way as 3.1.17 and none of the rendering artifacts are present in this version.

Seems to me that this a possible way forward with regards to a 3.3.0 release.

Thus spake bolabola:

I have tested the latest version you provided with the ui scale system
property set, and compared it to version 3.1.17. It works in exactly the
same way as 3.1.17 and none of the rendering artifacts are present in
this version.

What do you make of the rendering issue zgrose posted about earlier today?


J.

I have retested both versions just make I didn’t miss anything. Both versions render the same on my 4K screen, and I am also running Windows 10. Both versions render relative small, because neither scale to desktop settings.

I thought that it might be something with the window title rendering, due to AWT using native windows, but that doesn’t seems to be the case. I can’t discern any difference between the to versions.

Seems that zgrose gets HDPI scaling on his 3.2.17 version, which I simply don’t understand.

I bit surprised myself because I remember VASSAL being small on my son’s computer before. But things do constantly change.

[attachment=0]Annotation 2019-11-16 093905.png[/attachment]

[quote=“zgrose”]
I bit surprised myself because I remember VASSAL being small on my son’s computer before. But things do constantly change.

Ok, I think I might have an explanation for what is going on.

When I run 3.2.17, and look at the error log, I get:

2019-11-16 18:20:17,983 [0-main] INFO VASSAL.launch.StartUp - Starting
2019-11-16 18:20:17,984 [0-main] INFO VASSAL.launch.StartUp - OS Windows 8 6.2
2019-11-16 18:20:17,984 [0-main] INFO VASSAL.launch.StartUp - Java version 1.6.0_45
2019-11-16 18:20:17,985 [0-main] INFO VASSAL.launch.StartUp - VASSAL version 3.2.17
2019-11-16 18:20:18,010 [0-AWT-EventQueue-0] INFO VASSAL.launch.ModuleManager - Manager

Seems that I am running Vassal with Java 1.6, which I didn’t even know I had installed.

When running Vassal 3.3.0 I get,

2019-11-16 18:21:38,360 [0-main] INFO VASSAL.launch.StartUp - Starting
2019-11-16 18:21:38,363 [0-main] INFO VASSAL.launch.StartUp - OS Windows 10 10.0
2019-11-16 18:21:38,363 [0-main] INFO VASSAL.launch.StartUp - Java version 13
2019-11-16 18:21:38,363 [0-main] INFO VASSAL.launch.StartUp - VASSAL version 3.3.0-svn9292
2019-11-16 18:21:38,433 [0-AWT-EventQueue-0] INFO VASSAL.launch.ModuleManager - Manager

which makes sense.

Turns out that Java 1.8 actually had some, but flawed, HDPI support. See https://bugs.openjdk.java.net/browse/JDK-8180815. If you look at Pardeep Sharma last comment, it all makes sense.

I think the reason your sons version might have looked small is most likely that he has been running an older version of Java, properly Java 6 or 7.

So apparently Java 8 has some HDPI support, though not perfect (blurred fonts), that doesn’t affect Vassals rendering of images.

I’ll try and fix my local setup to force Vassal 3.2.17 to run against Java 8 and see if I get the same result as you.

I have tried installing a Java 8:

2019-11-16 18:38:43,641 [0-main] INFO VASSAL.launch.StartUp - Starting
2019-11-16 18:38:43,641 [0-main] INFO VASSAL.launch.StartUp - OS Windows 10 10.0
2019-11-16 18:38:43,641 [0-main] INFO VASSAL.launch.StartUp - Java version 1.8.0_231
2019-11-16 18:38:43,641 [0-main] INFO VASSAL.launch.StartUp - VASSAL version 3.2.17

I know I get the same result as you, zgrose. 3.2.17 is now scaled, but with somewhat (not to bad) blurred texts.

So unfortunately it seems that with this in mind, using the -Dsun.java2d.uiScale=1.0 system property in Vassal 3.3.0 would be a regression for users who has been running Vassal on Java 8.

I only just found out about 3.3.0, I am testing on macOS. It feels nice and responsive.

2020-01-08 09:44:43,581 [0-main] INFO VASSAL.launch.StartUp - Starting 2020-01-08 09:44:43,586 [0-main] INFO VASSAL.launch.StartUp - OS Mac OS X 10.15.2 2020-01-08 09:44:43,586 [0-main] INFO VASSAL.launch.StartUp - Java version 13.0.1 2020-01-08 09:44:43,586 [0-main] INFO VASSAL.launch.StartUp - VASSAL version 3.3.0-svn9291

I have modules registered in the Module Manager that are on an external volume (disk), when opening 3.3 it doesn’t have file system permissions to read these files, which means they get erased from the Module Manager – this is a Catalina problem. It would be great if the Module Manager would ask the user for permission to the file system when reading the V_Global and encountering this error, instead of assuming the files are gone and throwing all the entries away. VASSAL pops up a permission dialog if you add a new module from this volume, after which it gets access. But the next time VASSAL launches it throws out all these modules again with not permitted.

java.io.FileNotFoundException: /Users/myuser/Downloads/Normandy_ver094.vmod (Operation not permitted) at java.base/java.io.RandomAccessFile.open0(Native Method) at java.base/java.io.RandomAccessFile.open(Unknown Source) at java.base/java.io.RandomAccessFile.<init>(Unknown Source) at java.base/java.io.RandomAccessFile.<init>(Unknown Source) at java.base/java.util.zip.ZipFile$Source.<init>(Unknown Source) at java.base/java.util.zip.ZipFile$Source.get(Unknown Source) at java.base/java.util.zip.ZipFile$CleanableResource.<init>(Unknown Source) at java.base/java.util.zip.ZipFile.<init>(Unknown Source) at java.base/java.util.zip.ZipFile.<init>(Unknown Source) at java.base/java.util.zip.ZipFile.<init>(Unknown Source) at VASSAL.build.module.metadata.MetaDataFactory.buildMetaData(MetaDataFactory.java:62) at VASSAL.launch.ModuleManagerWindow$ModuleInfo.loadMetaData(ModuleManagerWindow.java:1231) at VASSAL.launch.ModuleManagerWindow$ModuleInfo.<init>(ModuleManagerWindow.java:1259) at VASSAL.launch.ModuleManagerWindow.buildTree(ModuleManagerWindow.java:481) at VASSAL.launch.ModuleManagerWindow.<init>(ModuleManagerWindow.java:376) at VASSAL.launch.ModuleManagerWindow.<clinit>(ModuleManagerWindow.java:178) at VASSAL.launch.ModuleManager.launch(ModuleManager.java:455) at VASSAL.launch.ModuleManager$2.run(ModuleManager.java:342) at java.desktop/java.awt.event.InvocationEvent.dispatch(Unknown Source) at java.desktop/java.awt.EventQueue.dispatchEventImpl(Unknown Source) at java.desktop/java.awt.EventQueue$4.run(Unknown Source) at java.desktop/java.awt.EventQueue$4.run(Unknown Source) at java.base/java.security.AccessController.doPrivileged(Unknown Source) at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.desktop/java.awt.EventQueue.dispatchEvent(Unknown Source) at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source) at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source) at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source) at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.desktop/java.awt.EventDispatchThread.run(Unknown Source) 2020-01-08 09:44:44,914 [0-AWT-EventQueue-0] INFO VASSAL.launch.ModuleManagerWindow - Module /Users/myuser/Downloads/Normandy_ver094.vmod not found - Removed.

VASSAL asks for permission to access the Remote Volume after which the module loads, but then on the next launch the Module Manager again doesn’t have the permissions. I feel like this worked before I tried to fiddle with it for the screenshots, but now I can’t get it work.

If permission is not granted or withdrawn later VASSAL fails silently, it’ll just close the module or get stuck on the tile loader dialog.

Unarchiver asks for access before extracting a zip file after which it will work for this location forever:

VASSAL Files and Folders permission:


When I drag a card from the Deck it still also creates a selection rectangle as well (which also selects all cards in the Deck). This bug already existed previously, but has not been fixed in 3.3.

I added a module from a new location. VASSAL asked for permission to access that location. And then VASSAL worked again like I thought it did before. It saves the Module Manager list, and has access to the modules.

When I manually restore the RecentModules in V_Global then it somehow changes the signature of VASSAL and it launches it in a new sandbox. Hands Off thinks it’s a new app, and it lost file system access again. It doesn’t re-ask for access, so it still has it, but Module Manager says it doesn’t when checking the files.

This makes it even more important that the Module Manager doesn’t reset the list as the only way to restore it is by manually one-by-one re-adding the modules you had.

Once the permissions are messed up for Module Manager they are messed up. Also for 3.2.17. :frowning: