Test builds for 3.3.0

bola · November 3, 2019, 9:16pm

It is not a bug in Vassal, it is a bug in the JDK image rendering engine. In JDK 9 HDPI monitor support was added. This scales the UI in the Java application according to the scaling settings in the operating system. This works fine for UI elements (button, labels, text), but causes issues with Image rendering.

No matter which zoom level I use in Vassal when having the scaling in the operating system set to anything other than 100%, images (map, counters, UI icons) gets blurred and jagged, because the Image / BufferedImage has a bug. Images are run through the image scaling transformation twice (once by the JDK, and once by the operation system (D3D), resulting in a blurred image).

Note that if I set my operating system scaling to 100%, and zoom to 150% in Vassal, maps and images still look very smooth.

As I wrote in my previous post, the issue is known to the JDK developers: bugs.openjdk.java.net/browse/JD … 22image%22

Most users (except users running on HDPI laptops) will have the operating system scaling set to 100%, and will not experience the issue at all.

I think the fix is relatively easy. If HDPI support is disabled by the system property (-Dsun.java2d.uiScale=1), Vassal will work exactly as it did on HDPI screen as it did before the upgrade to JDK 9.

I hope this makes it a bit clearer. If not, please let me know.

/Bo

CloClooo · November 4, 2019, 9:52am

Hello Joel, file is too big to attach. (forum limits me to 256kB). Can you send me your email so that I can send you the file? Or tell me how to proceed.
Kr
Claudio

CloClooo · November 4, 2019, 10:38am

Hello Joel,
I tried to make a change in a game module, and saved it to a new vmod file.
When I tried to open the module with 9287, I got this error

“The module you are trying to use (E:\Vassal\TLD\Tld-0.1.9.019.vmod) was created with VASSAL 3.3.0-svn9287. This version of VASSAL (3.3.0-svn9287) cannot edit or play modules which are that new.”

Kr
Claudio

"

uckelman · November 4, 2019, 12:18pm

Thus spake ClaudioC:

Hello Joel, file is too big to attach. (forum limits me to 256kB). Can
you send me your email so that I can send you the file? Or tell me how
to proceed.
Kr
Claudio

There’s a little icon with an envelope and an @ symbol under my avatar
to the right of my posts. That has a link with my email address. (Everyone
has one of those.)

–
J.

zgrose · November 4, 2019, 3:02pm

I don’t see that icon on anyone:

[attachment=0]Annotation 2019-11-04 090051.png[/attachment]

uckelman · November 4, 2019, 7:21pm

Thus spake zgrose via messages:

“uckelman” wrote:

Thus spake ClaudioC:
There’s a little icon with an envelope and an @ symbol under my avatar
to the right of my posts. That has a link with my email address.
(Everyone
has one of those.)

I don’t see that icon on anyone:

Weird. My email address is uckelman@nomic.net.

–
J.

uckelman · November 4, 2019, 7:45pm

Thanks for sending me the vlog. The problem is that a very long time ago, someone unwisely set the internal date format for logs to the default date format that Java provided for the US locale, and the default changed sometime between the version of Java you’re using for 3.2.17 and the version of Java we bundle with the 3.3.0 test builds. I’ve fixed the problem by explicitly specifying the date format so that it can’t be changed out from under us again.

Try the svn9290 build I just uploaded.

CloClooo · November 4, 2019, 9:19pm

uckelman:

ClaudioC:

Joel, I am now trying to play all my ongoing games with the latest build.
Yesterday I opened the log of a Vietnam game and got the following module errors.

VG Vietnam version 4.3.4

Loading Claudio_vs_Petri_Vietnam_01236.vlog …

Bad Data in Module: Illegal date format 7/6/19 8:15 PM

Thanks for sending me the vlog. The problem is that a very long time ago, someone unwisely set the internal date format for logs to the default date format that Java provided for the US locale, and the default changed sometime between the version of Java you’re using for 3.2.17 and the version of Java we bundle with the 3.3.0 test builds. I’ve fixed the problem by explicitly specifying the date format so that it can’t be changed out from under us again.

Try the svn9290 build I just uploaded.

Hello Joel,
It works correctly now. Thank you.
Claudio

uckelman · November 4, 2019, 9:39pm

Thank you for pointing out the problem and supplying a test case for me.

uckelman · November 4, 2019, 10:05pm

I understand what’s happening now. Since Java 9, the paint() member function of Swing UI components gets called with a Graphics2D that has an AffineTransform set to the scale factor that’s being used for your desktop—but the getWidth() and getHeight() member functions still return unscaled dimensions. So, if you have a component which is 800x600 in terms of user pixels but the OS has a scale factor of 200%, then you have 1600x1200 real pixels on which to paint. If you go ahead and run paint() with the AffineTransform you’re handed with the Graphics2D, you will paint at 800x600 but every user pixel you paint will get blown up into four physical pixels on the screen (and look like crap).

But, this only happens with (1) components where you’ve implemented paint() yourself, and (2) components which paint Images. The Swing components which paint text and other UI widgets are smart enough to spot that they’re being scaled, undo the scaling, and paint at full physical size. This is why in your screenshots, the text, button edges, scroll bars, etc. look sharp (well, as sharp as Java UI elements ever look) but the icons and map look upscaled and blurry.

The solution is to do the same thing for custom components as Swing components do for themselves. It turns out that this isn’t so difficult to do for the map, so I made a test build for you to try, so we can judge the feasibility of doing this for the remaining components which draw images. The test builds are, here, labeled “hidpitest”:

vassalengine.org/~uckelman/tmp/

If this works for you like it did for me, you should get a smoothly scaled map no matter what your desktop scale factor is. Anything else which was poorly scaled before, such as icons, should still be that way, as I haven’t attempted to modify drawing of anything but the map yet.

I’m interested to know how well this works for you.

bola · November 5, 2019, 8:35pm

uckelman:

bolabola:

It is not a bug in Vassal, it is a bug in the JDK image rendering engine. In JDK 9 HDPI monitor support was added. This scales the UI in the Java application according to the scaling settings in the operating system. This works fine for UI elements (button, labels, text), but causes issues with Image rendering.

I understand what’s happening now. Since Java 9, the paint() member function of Swing UI components gets called with a Graphics2D that has an AffineTransform set to the scale factor that’s being used for your desktop—but the getWidth() and getHeight() member functions still return unscaled dimensions. So, if you have a component which is 800x600 in terms of user pixels but the OS has a scale factor of 200%, then you have 1600x1200 real pixels on which to paint. If you go ahead and run paint() with the AffineTransform you’re handed with the Graphics2D, you will paint at 800x600 but every user pixel you paint will get blown up into four physical pixels on the screen (and look like crap).

But, this only happens with (1) components where you’ve implemented paint() yourself, and (2) components which paint Images. The Swing components which paint text and other UI widgets are smart enough to spot that they’re being scaled, undo the scaling, and paint at full physical size. This is why in your screenshots, the text, button edges, scroll bars, etc. look sharp (well, as sharp as Java UI elements ever look) but the icons and map look upscaled and blurry.

The solution is to do the same thing for custom components as Swing components do for themselves. It turns out that this isn’t so difficult to do for the map, so I made a test build for you to try, so we can judge the feasibility of doing this for the remaining components which draw images. The test builds are, here, labeled “hidpitest”:

vassalengine.org/~uckelman/tmp/

If this works for you like it did for me, you should get a smoothly scaled map no matter what your desktop scale factor is. Anything else which was poorly scaled before, such as icons, should still be that way, as I haven’t attempted to modify drawing of anything but the map yet.

I’m interested to know how well this works for you.

Hi Joel

I have tested your hdpi version. The map is now rendered smoothly as expected, but only some of the map. I am by no means an expert, but it seems that the change introduces a viewport issue. Only 1/4 of the map is rendered. If I drag the scollbars, a different part of the map is rendered, but on 1/4 it seems. If I drag a selection square on the map, the same part of the map is rendered inside the dragged square.

I’ll email you a screendump, so you can get a better idea of what it looks like.

bola · November 5, 2019, 8:55pm

bolabola:

uckelman:

bolabola:

It is not a bug in Vassal, it is a bug in the JDK image rendering engine. In JDK 9 HDPI monitor support was added. This scales the UI in the Java application according to the scaling settings in the operating system. This works fine for UI elements (button, labels, text), but causes issues with Image rendering.

I understand what’s happening now. Since Java 9, the paint() member function of Swing UI components gets called with a Graphics2D that has an AffineTransform set to the scale factor that’s being used for your desktop—but the getWidth() and getHeight() member functions still return unscaled dimensions. So, if you have a component which is 800x600 in terms of user pixels but the OS has a scale factor of 200%, then you have 1600x1200 real pixels on which to paint. If you go ahead and run paint() with the AffineTransform you’re handed with the Graphics2D, you will paint at 800x600 but every user pixel you paint will get blown up into four physical pixels on the screen (and look like crap).

But, this only happens with (1) components where you’ve implemented paint() yourself, and (2) components which paint Images. The Swing components which paint text and other UI widgets are smart enough to spot that they’re being scaled, undo the scaling, and paint at full physical size. This is why in your screenshots, the text, button edges, scroll bars, etc. look sharp (well, as sharp as Java UI elements ever look) but the icons and map look upscaled and blurry.

The solution is to do the same thing for custom components as Swing components do for themselves. It turns out that this isn’t so difficult to do for the map, so I made a test build for you to try, so we can judge the feasibility of doing this for the remaining components which draw images. The test builds are, here, labeled “hidpitest”:

vassalengine.org/~uckelman/tmp/

If this works for you like it did for me, you should get a smoothly scaled map no matter what your desktop scale factor is. Anything else which was poorly scaled before, such as icons, should still be that way, as I haven’t attempted to modify drawing of anything but the map yet.

I’m interested to know how well this works for you.

Hi Joel

I have tested your hdpi version. The map is now rendered smoothly as expected, but only some of the map. I am by no means an expert, but it seems that the change introduces a viewport issue. Only 1/4 of the map is rendered. If I drag the scollbars, a different part of the map is rendered, but on 1/4 it seems. If I drag a selection square on the map, the same part of the map is rendered inside the dragged square.

I’ll email you a screendump, so you can get a better idea of what it looks like.

I have done a bit more testing. When I set my desktop scale to 100%, the map initially renders smoothly, but if I drag a selection square, the rendering artifact persists. Also, when using the scollbars, weird rendering artifacts appear.

I hope this is helpful.

Let me know if there is anything I can do to help, or if you need more tests.

I can also test a version with the system property -Dsun.java2d.uiScale=1.0 set, if you want. This might be a solution for now, if adding HDPI support to Vassal turns out to be a big task. Then possibly HDPI can be added to Vassal 3.3.1 as a feature.

/Bo

CloClooo · November 6, 2019, 1:43pm

uckelman:

bolabola:

It is not a bug in Vassal, it is a bug in the JDK image rendering engine. In JDK 9 HDPI monitor support was added. This scales the UI in the Java application according to the scaling settings in the operating system. This works fine for UI elements (button, labels, text), but causes issues with Image rendering.

I understand what’s happening now. Since Java 9, the paint() member function of Swing UI components gets called with a Graphics2D that has an AffineTransform set to the scale factor that’s being used for your desktop—but the getWidth() and getHeight() member functions still return unscaled dimensions. So, if you have a component which is 800x600 in terms of user pixels but the OS has a scale factor of 200%, then you have 1600x1200 real pixels on which to paint. If you go ahead and run paint() with the AffineTransform you’re handed with the Graphics2D, you will paint at 800x600 but every user pixel you paint will get blown up into four physical pixels on the screen (and look like crap).

But, this only happens with (1) components where you’ve implemented paint() yourself, and (2) components which paint Images. The Swing components which paint text and other UI widgets are smart enough to spot that they’re being scaled, undo the scaling, and paint at full physical size. This is why in your screenshots, the text, button edges, scroll bars, etc. look sharp (well, as sharp as Java UI elements ever look) but the icons and map look upscaled and blurry.

The solution is to do the same thing for custom components as Swing components do for themselves. It turns out that this isn’t so difficult to do for the map, so I made a test build for you to try, so we can judge the feasibility of doing this for the remaining components which draw images. The test builds are, here, labeled “hidpitest”:

vassalengine.org/~uckelman/tmp/

If this works for you like it did for me, you should get a smoothly scaled map no matter what your desktop scale factor is. Anything else which was poorly scaled before, such as icons, should still be that way, as I haven’t attempted to modify drawing of anything but the map yet.

I’m interested to know how well this works for you.

Hello Joel,
The map paints correctly whatever the vassal zoom + windows zoom I select. So I suppose it confirms what you say. Note that onthe flipside the map drawing seems to behave very strangely. If I scroll down to see the 'south" of the map (arrow downs or elevator bar down) the map picture does not go up, instead a white area scrolls down.

If you need more infos on this I will have to record a short video.
Bye

uckelman · November 6, 2019, 11:54pm

There’s a simple explanation for the odd map painting: I adjusted the scaling of the map, but missed adjusting the scaling of the visible rectangle, which was computed just above where I was working.

Try the “hidpitest2” build, from the same location.

CloClooo · November 7, 2019, 4:57pm

Hello Joel,
Not sure you fixed the painting issue.
I tried several modules and strange things occur on the map area every time.
For instance Vietnam 1965 1975
Hex grid (generated by Vassal) is not aligned with map, sometimes painted outside paint area, sometimes limited to part of the paint area (if I scroll the map down)

Changing vassal zoom level also causes strange behaviors.
I sent some screenshots by mail.

Kr
Claudio

bola · November 7, 2019, 8:03pm

Hi Joel

I have also tested the latest version (hdpitest2), and also experience some weird artifacts when scrolling or dragging selection boxes. The entire map, unlike the previous version, in drawn though.

I have emailed you a video capture to give you better understanding of what is going on.

uckelman · November 9, 2019, 12:56am

The selection lasso had been relying on the mouse coordinates being in the same space as what’s being drawn in the JPanel for the map, which is no longer the case. There’s a “hidpitest3” build now which should fix the selection lasso.

bola · November 9, 2019, 5:44pm

uckelman:

bolabola:

uckelman:

There’s a simple explanation for the odd map painting: I adjusted the scaling of the map, but missed adjusting the scaling of the visible rectangle, which was computed just above where I was working.

Try the “hidpitest2” build, from the same location.

Hi Joel

I have also tested the latest version (hdpitest2), and also experience some weird artifacts when scrolling or dragging selection boxes. The entire map, unlike the previous version, in drawn though.

I have emailed you a video capture to give you better understanding of what is going on.

The selection lasso had been relying on the mouse coordinates being in the same space as what’s being drawn in the JPanel for the map, which is no longer the case. There’s a “hidpitest3” build now which should fix the selection lasso.

Hi Joel

I have tested the latest version you provided. Unfortunately the errors with the selection lasso and scrolling issue persists. I have tested with both 100% and 150% desktop scaling, and the issue seems to be the same in both cases. Note that the selection lasso correctly selects counters within its boundaries. The issue is that is top left corner of the map is always drawn inside the lasso.

On a side note, Java 14 will have official support for jpackage. This should make it possible to simplify the distribution build pipeline. A talk was held about it on the Devoxx conference this week.

https://www.youtube.com/watch?v=ZGW9AalZLN4

Thought you might find it interesting.

uckelman · November 10, 2019, 12:27am

Thus spake bolabola:

I have tested the latest version you provided. Unfortunately the errors
with the selection lasso and scrolling issue persists. I have tested
with both 100% and 150% desktop scaling, and the issue seems to be the
same in both cases. Note that the selection lasso correctly selects
counters within its boundaries. The issue is that is top left corner of
the map is always drawn inside the lasso.

I’m not surprised there’s still a scrolling problem; I haven’t made any
attempt to address that yet.

I think I know why the map is being redrawn incorrectly inside the
selection lasso, on the basis of your description. Thanks for that.

Getting this right will involve going through all the code in Map and
all the classes which implement Drawable and ensuring that we have proper
separation between UI coordinates and painting coordinates.

This may take some time, but I don’t see any other options—we’re already
committed to upgrading to Java 9 or later, and that forces dealing with
HiDPI on us. Neither a release that renders blurry images nor one which
is illegibly tiny is a something I want to go out the door.

I’d welcome help with going through this code and making adjustments, but
this is a place in VASSAL where some of the grottiest old code collides
with some of the most highly optimized newer code, so it’s likely to be
rather hairy.

On a side note, Java 14 will have official support for jpackage. This
should make it possible to simplify the distribution build pipeline. A
talk was held about it on the Devoxx conference this week.

Given that I’ve already solved that problem without jpackage, it is, like
everything else about Java at this point, a day late and a dollar short.
I sincerely hope that 3.3.0 is the last release we do using Java.

–
J.

bola · November 10, 2019, 2:32am

uckelman:

Thus spake bolabola:

I have tested the latest version you provided. Unfortunately the errors
with the selection lasso and scrolling issue persists. I have tested
with both 100% and 150% desktop scaling, and the issue seems to be the
same in both cases. Note that the selection lasso correctly selects
counters within its boundaries. The issue is that is top left corner of
the map is always drawn inside the lasso.

I’m not surprised there’s still a scrolling problem; I haven’t made any
attempt to address that yet.

I think I know why the map is being redrawn incorrectly inside the
selection lasso, on the basis of your description. Thanks for that.

Getting this right will involve going through all the code in Map and
all the classes which implement Drawable and ensuring that we have proper
separation between UI coordinates and painting coordinates.

This may take some time, but I don’t see any other options—we’re already
committed to upgrading to Java 9 or later, and that forces dealing with
HiDPI on us. Neither a release that renders blurry images nor one which
is illegibly tiny is a something I want to go out the door.

I’d welcome help with going through this code and making adjustments, but
this is a place in VASSAL where some of the grottiest old code collides
with some of the most highly optimized newer code, so it’s likely to be
rather hairy.

On a side note, Java 14 will have official support for jpackage. This
should make it possible to simplify the distribution build pipeline. A
talk was held about it on the Devoxx conference this week.

Given that I’ve already solved that problem without jpackage, it is, like
everything else about Java at this point, a day late and a dollar short.
I sincerely hope that 3.3.0 is the last release we do using Java.

–
J.

Yes, this was also what I initially feared. That this would be hard and very time consuming to fix.

Again, I think it is worth considering just using the -Dsun.java2d.uiScale=1.0 system property. This should make 3.3.3 work in exactly the same way as 3.2.17 with regards to scaling. Everything should work, but with no HDPI support as in 3.2.17 though.

That would allow you to make a new version running on a recent version of Java, and decouple the JVM from whatever the enduser has installed on his or her system.

I would love to help out on the coding side, but my knowledge of the codebase is extremely limited.

I wouldn’t write off Java completely for version 4, though. I don’t know if you have given JavaFX a look. It is a really impressive UI platform, with amazing performance (entire rendering pipeline is pushed to the graphics card) and the UI programming model that is a really good match for an application like Vassal (node based scene graph). And the UI components has a modern look-and-feel.

I created a demo application some years ago using JavaFX, which I have just updated to Java 13 / JavaFX 13.

With a couple of hundred of lines, you get amazingly far.

* Scrollable boardgame map
* Zooming
* Counter dragging
* Counter rotation
* Counter flipping
* Counter dropdown menus
* Animated transitions
* Property binding
* Selection box
* Complete HDPI support
* Dependency injection on controllers
* UI testable with TestFX.
* Modularized application.

You can find the demo app here: https://github.com/boleer/tabula-demo

You can find descriptions under the /site/apt:

https://github.com/boleer/tabula-demo/blob/master/src/site/apt/index.apt.vm
https://github.com/boleer/tabula-demo/blob/master/src/site/apt/why-javafx.apt.vm

The project uses Maven, so it should be relative easy to get up and running, you just need to install Maven 3.6.0 and JDK 13, and then run the following command:

mvn package -DskipTests
mvn exec:java

I have attached a screen dump of what the demo app looks like.

You can try to use the zooming slider to get an impression of how amazing the rendering performance is.