Test builds for 3.3.0

[ClaudioC]

There's a simple explanation for the odd map painting: I adjusted the scaling of the map, but missed adjusting the scaling of the visible rectangle, which was computed just above where I was working.

Try the "hidpitest2" build, from the same location.

Hello Joel,
Not sure you fixed the painting issue.
I tried several modules and strange things occur on the map area every time.
For instance Vietnam 1965 1975
Hex grid (generated by Vassal) is not aligned with map, sometimes painted outside paint area, sometimes limited to part of the paint area (if I scroll the map down)

Changing vassal zoom level also causes strange behaviors.
I sent some screenshots by mail.

Kr
Claudio

[bolabola]

There's a simple explanation for the odd map painting: I adjusted the scaling of the map, but missed adjusting the scaling of the visible rectangle, which was computed just above where I was working.

Try the "hidpitest2" build, from the same location.

Hi Joel

I have also tested the latest version (hdpitest2), and also experience some weird artifacts when scrolling or dragging selection boxes. The entire map, unlike the previous version, in drawn though.

I have emailed you a video capture to give you better understanding of what is going on.

[uckelman]

There's a simple explanation for the odd map painting: I adjusted the scaling of the map, but missed adjusting the scaling of the visible rectangle, which was computed just above where I was working.

Try the "hidpitest2" build, from the same location.

Hi Joel

I have also tested the latest version (hdpitest2), and also experience some weird artifacts when scrolling or dragging selection boxes. The entire map, unlike the previous version, in drawn though.

I have emailed you a video capture to give you better understanding of what is going on.

The selection lasso had been relying on the mouse coordinates being in the same space as what's being drawn in the JPanel for the map, which is no longer the case. There's a "hidpitest3" build now which should fix the selection lasso.

[bolabola]

There's a simple explanation for the odd map painting: I adjusted the scaling of the map, but missed adjusting the scaling of the visible rectangle, which was computed just above where I was working.

Try the "hidpitest2" build, from the same location.

Hi Joel

I have also tested the latest version (hdpitest2), and also experience some weird artifacts when scrolling or dragging selection boxes. The entire map, unlike the previous version, in drawn though.

I have emailed you a video capture to give you better understanding of what is going on.

The selection lasso had been relying on the mouse coordinates being in the same space as what's being drawn in the JPanel for the map, which is no longer the case. There's a "hidpitest3" build now which should fix the selection lasso.

Hi Joel

I have tested the latest version you provided. Unfortunately the errors with the selection lasso and scrolling issue persists. I have tested with both 100% and 150% desktop scaling, and the issue seems to be the same in both cases. Note that the selection lasso correctly selects counters within its boundaries. The issue is that is top left corner of the map is always drawn inside the lasso.

On a side note, Java 14 will have official support for jpackage. This should make it possible to simplify the distribution build pipeline. A talk was held about it on the Devoxx conference this week.

https://www.youtube.com/watch?v=ZGW9AalZLN4

Thought you might find it interesting.

[uckelman]

Thus spake bolabola:
> I have tested the latest version you provided. Unfortunately the errors
> with the selection lasso and scrolling issue persists. I have tested
> with both 100% and 150% desktop scaling, and the issue seems to be the
> same in both cases. Note that the selection lasso correctly selects
> counters within its boundaries. The issue is that is top left corner of
> the map is always drawn inside the lasso.

I'm not surprised there's still a scrolling problem; I haven't made any
attempt to address that yet.

I think I know why the map is being redrawn incorrectly inside the
selection lasso, on the basis of your description. Thanks for that.

Getting this right will involve going through all the code in Map and
all the classes which implement Drawable and ensuring that we have proper
separation between UI coordinates and painting coordinates.

This may take some time, but I don't see any other options---we're already
committed to upgrading to Java 9 or later, and that forces dealing with
HiDPI on us. Neither a release that renders blurry images nor one which
is illegibly tiny is a something I want to go out the door.

I'd welcome help with going through this code and making adjustments, but
this is a place in VASSAL where some of the grottiest old code collides
with some of the most highly optimized newer code, so it's likely to be
rather hairy.

> On a side note, Java 14 will have official support for jpackage. This
> should make it possible to simplify the distribution build pipeline. A
> talk was held about it on the Devoxx conference this week.

Given that I've already solved that problem without jpackage, it is, like
everything else about Java at this point, a day late and a dollar short.
I sincerely hope that 3.3.0 is the last release we do using Java.

--
J.

[bolabola]

Thus spake bolabola:
> I have tested the latest version you provided. Unfortunately the errors
> with the selection lasso and scrolling issue persists. I have tested
> with both 100% and 150% desktop scaling, and the issue seems to be the
> same in both cases. Note that the selection lasso correctly selects
> counters within its boundaries. The issue is that is top left corner of
> the map is always drawn inside the lasso.

I'm not surprised there's still a scrolling problem; I haven't made any
attempt to address that yet.

I think I know why the map is being redrawn incorrectly inside the
selection lasso, on the basis of your description. Thanks for that.

Getting this right will involve going through all the code in Map and
all the classes which implement Drawable and ensuring that we have proper
separation between UI coordinates and painting coordinates.

This may take some time, but I don't see any other options---we're already
committed to upgrading to Java 9 or later, and that forces dealing with
HiDPI on us. Neither a release that renders blurry images nor one which
is illegibly tiny is a something I want to go out the door.

I'd welcome help with going through this code and making adjustments, but
this is a place in VASSAL where some of the grottiest old code collides
with some of the most highly optimized newer code, so it's likely to be
rather hairy.

> On a side note, Java 14 will have official support for jpackage. This
> should make it possible to simplify the distribution build pipeline. A
> talk was held about it on the Devoxx conference this week.

Given that I've already solved that problem without jpackage, it is, like
everything else about Java at this point, a day late and a dollar short.
I sincerely hope that 3.3.0 is the last release we do using Java.

--
J.

Yes, this was also what I initially feared. That this would be hard and very time consuming to fix.

Again, I think it is worth considering just using the -Dsun.java2d.uiScale=1.0 system property. This should make 3.3.3 work in exactly the same way as 3.2.17 with regards to scaling. Everything should work, but with no HDPI support as in 3.2.17 though.

That would allow you to make a new version running on a recent version of Java, and decouple the JVM from whatever the enduser has installed on his or her system.

I would love to help out on the coding side, but my knowledge of the codebase is extremely limited.

I wouldn't write off Java completely for version 4, though. I don't know if you have given JavaFX a look. It is a really impressive UI platform, with amazing performance (entire rendering pipeline is pushed to the graphics card) and the UI programming model that is a really good match for an application like Vassal (node based scene graph). And the UI components has a modern look-and-feel.

I created a demo application some years ago using JavaFX, which I have just updated to Java 13 / JavaFX 13.

With a couple of hundred of lines, you get amazingly far.

* Scrollable boardgame map
* Zooming
* Counter dragging
* Counter rotation
* Counter flipping
* Counter dropdown menus
* Animated transitions
* Property binding
* Selection box
* Complete HDPI support
* Dependency injection on controllers
* UI testable with TestFX.
* Modularized application.

You can find the demo app here: https://github.com/boleer/tabula-demo

You can find descriptions under the /site/apt:

https://github.com/boleer/tabula-demo/blob/master/src/site/apt/index.apt.vm
https://github.com/boleer/tabula-demo/blob/master/src/site/apt/why-javafx.apt.vm

The project uses Maven, so it should be relative easy to get up and running, you just need to install Maven 3.6.0 and JDK 13, and then run the following command:

- mvn package -DskipTests
- mvn exec:java

I have attached a screen dump of what the demo app looks like.

You can try to use the zooming slider to get an impression of how amazing the rendering performance is.

[uckelman]

Thus spake bolabola:
> Yes, this was also what I initially feared. That this would be hard and
> very time consuming to fix.

I think it won't be trivial, but it's a problem where all the relevant code
is ours, which means there's far less uncertainty involved than with, say,
determining how to bundle Java into our DMG. (We got it, but I could not
predict when or even if we would.)

> Again, I think it is worth considering just using the
> -Dsun.java2d.uiScale=1.0 system property. This should make 3.3.3 work in
> exactly the same way as 3.2.17 with regards to scaling. Everything
> should work, but with no HDPI support as in 3.2.17 though.

Are you sure? Even for people with HiDPI screens? I was under the
impression that this would make 3.3.0 very small for people with HiDIP
screens. Am I wrong here? If I've misunderstood the situation, I'd like
to know.

> With a couple of hundred of lines, you get amazingly far.

It's an interesting looking demo. I managed something similar with very
little code a few years ago in C++ using SDL. We have a huge amount of
code which would not have needed to be written if we'd had a different
design model, but I think this also shows that it doesn't take much to
get halfway there, and that it's the second half which is far more work.

I have a long list of complaints about Java at this point; after working
in Java for 15 years, I feel I've come by them honestly. One category of
these complaints concerns image decoding. Anything you do in JavaFX is
going to be afflicted by these as well, so far as I can see.

I have put hundreds of hours into investigating and working around image
decoding bugs in Java over the years, and hardly any of those bugs have
been fixed during that time. There's one PNG loading bug in Oracle's
tracker which has had an obviously correct patch sitting there for more
than a decade now. If you submitted a bug to libpng demonstrating that a
correct PNG was misdecoded, you might wait a few weeks tops for a release
that fixed the bug. It's not as though there are only one or two such
bugs, either. I count workarounds for 13 different image decoding bugs in
our image loader right now.

And it's not just problems with unusual but standards-compliant images.
The JPEG loader _class_ in ImageIO isn't thread-safe. That is not a typo,
the _class_ is not thread safe. The whole class is not thread safe. You
literally cannot safely load JPEGS on two different threads
simultaneously even when each thread has its own JPEG decoder; if you do,
you risk an unchecked exception being thrown. It's blindingly obvious why,
too, if you look at the code, and it's shocking that they let it out the
door in the first place. This bug's been around for ages as well. No joke,
the sole acknowledgement the bug report got from someone at Oracle was:
"We don't have plans to fix it".

Correct image decoding is vital to us; most of what we do is display
images in various arrangements. It's been infuriating to have to devote
so much time to a problem that exists only because Java neither uses
the standard image loading libraries nor properly maintains its own.

I won't willingly go down that road a second time.

--
J.

[zgrose]

I'm not sure what info you need for HighDPI feedback but this is my setup:

iMac with Retina 5K display running Windows10 in BootCamp. Fairly recent Radeon drivers (not the BootCamp ones).
Display set to 5120x2800
"Change the size of text, apps, and other items" says 450%. If I click advanced scaling settings it says I have a custom scaling size of 200 and "Let Windows try to fix apps so they're not blurry" is checked.

VASL 6.4.4 with VASSAL 3.2.17 and Java8 looks "normal" (compared to my time on the Mac side).
VASL 6.4.4 with VASSAL 3.3.0 svn9291 and Java13 from the installer looks "normal".

Trying to go through the items described above, scrolling the VASL boards seems to work as expected.
I can drag the selection square around the counter fine.

From my limited time on Windows pre 3.3.0 I know that VASSAL/VASL would show up super tiny in Java8 but that would render "correctly" with later JVMs when the libraries were dropped manually. Or at least that was my impression.

Either way, I can provide whatever diagnostics or run any test you like if you need more feedback or more specific info.

[bolabola]

> Are you sure? Even for people with HiDPI screens? I was under the
> impression that this would make 3.3.0 very small for people with HiDIP
> screens. Am I wrong here? If I've misunderstood the situation, I'd like
> to know.

No, I am not 100% sure, but I would think so. If you can provide me we a build with the system property set, I can test whether it works in exactly the same way as 3.2.17, (that is no scaling).

> I have a long list of complaints about Java at this point; after working
> in Java for 15 years, I feel I've come by them honestly. One category of
> these complaints concerns image decoding. Anything you do in JavaFX is
> going to be afflicted by these as well, so far as I can see.

The image decoding is a completely different implementation in JavaFX, and doesn't use or require usage of ImageIO in any way. It is also thread safe. It supports PNG, BMP, GIF and JPEG images out of the box. Custom decoding implementations (for SVG for example) can be provided to JavaFX by implementing custom ImageLoaders, but without the need to implement complex transform algorithms, as this be done by the JavaFX core (on the GPU).

All decoder implementations are pure java implementations, except for the JPEG implementation, which uses the latest version of the libjpeg library.

> I have put hundreds of hours into investigating and working around image
> decoding bugs in Java over the years, and hardly any of those bugs have
> been fixed during that time. There's one PNG loading bug in Oracle's
> tracker which has had an obviously correct patch sitting there for more
> than a decade now. If you submitted a bug to libpng demonstrating that a
> correct PNG was misdecoded, you might wait a few weeks tops for a release
> that fixed the bug. It's not as though there are only one or two such
> bugs, either. I count workarounds for 13 different image decoding bugs in
> our image loader right now.

> And it's not just problems with unusual but standards-compliant images.
> The JPEG loader _class_ in ImageIO isn't thread-safe. That is not a typo,
> the _class_ is not thread safe. The whole class is not thread safe. You
> literally cannot safely load JPEGS on two different threads
> simultaneously even when each thread has its own JPEG decoder; if you do,
> you risk an unchecked exception being thrown. It's blindingly obvious why,
> too, if you look at the code, and it's shocking that they let it out the
> door in the first place. This bug's been around for ages as well. No joke,
> the sole acknowledgement the bug report got from someone at Oracle was:
> "We don't have plans to fix it".

Looking at the code in the ImageIOImageLoader class in Vassal and all the workarounds you implemented to correctly decode JPEG images, I fully understand your frustration with the ImageIO, and Oracle's unwillingness to fix the issues. It is amazing that you had the patience to go to such lengths to implement workarounds for something that Oracle should have fixed. And I agree, it doesn't reflect well on Sun, that they released an Image IO library that you cannot use in the thread safe manner.

Again though, none of this would not be an issue in JavaFX, due to the completely different implementation. All the complex code in the image and imageop packages for image transforms, tiling and image decoding, which I assume was originally written to overcome limitations in AWT / Swing due to functionality limitations and poor performance. None of this is necessary in JavaFX. You simply load the images, and JavaFX handles all the logic around affine transformations (scaling, rotating, filtering, animations) by pushing the images to the GPU and executing the transforms there.

Correct image decoding is vital to us; most of what we do is display
images in various arrangements. It's been infuriating to have to devote
so much time to a problem that exists only because Java neither uses
the standard image loading libraries nor properly maintains its own.

This luckily this is not the case with JavaFX. As you surely know Java (OpenJDK) and JavaFX (OpenJFX) has been open sourced, and has a very vibrant community, allowing for much faster turnaround on fixes, in part because of the new 1/2 release cycle, as well as a very active open source community around the OpenJDK and OpenJFX.

Using JavaFX also has the added advantage, that some parts of the Vassal codebase might be reuseable, and making old module plugins work would be a lot easier. I also think it would be a lot easier to find contributors.

Another of the main pain points with Vassal and Java during the years has been, as I understand it, that Vassal had to work on many different versions Java, due to the lack of control of the runtime environment on the end user machine. This is luckily no longer an issue, due to jlink and the option to package the Java runtime with the application.

I won't willingly go down that road a second time.

I hope that this answers at least some of your concerns about using JavaFX.

If you need to me to test anything related to scaling, please to don't hesitate to let me know. I'll be more than happy to help out with the testing of Vassal 3.3.0. Zgrose can properly also be helpful in this regard, as he is also using a HDPI monitor.

[uckelman]

Thus spake bolabola:
>
> No, I am not 100% sure, but I would think so. If you can provide me we a
> build with the system property set, I can test whether it works in
> exactly the same way as 3.2.17, (that is no scaling).

I've uploaded a set of builds which launch with -Dsun.java2d.uiScale=1.0
set, named svn9292-uiscale, here:

http://www.vassalengine.org/~uckelman/tmp/

> The image decoding is a completely different implementation in JavaFX,
> and doesn't use or require usage of ImageIO in any way. It is also
> thread safe. It supports PNG, BMP, GIF and JPEG images out of the box.
> Custom decoding implementations (for SVG for example) can be provided to
> JavaFX by implementing custom ImageLoaders, but without the need to
> implement complex transform algorithms, as this be done by the JavaFX
> core (on the GPU).
>
> All decoder implementations are pure java implementations, except for
> the JPEG implementation, which uses the latest version of the libjpeg
> library.

It sounds like they're repeating the same mistake made with ImageIO,
namely that they're not using the standard image libraries that the
entire rest of the software world uses. Nor does leaving out SVG
inspire confidence.

> I hope that this answers at least some of your concerns about using
> JavaFX.

Thanks for the explanation, but I'm not interested in continuing to use
Java in any form for VASSAL 4. Even if all my concerns regarding images
were satisfied, I still have an overwhelming collection of other reasons
for using somehting else. I don't see much future in Java at this point,
and I don't want us tied to it.

> If you need to me to test anything related to scaling, please to don't
> hesitate to let me know. I'll be more than happy to help out with the
> testing of Vassal 3.3.0. Zgrose can properly also be helpful in this
> regard, as he is also using a HDPI monitor.

Thanks. Give the build above a try. What I'm interested in particular is
if the UI scaling differs from 3.2.17. (If the scaling behavior is the
same as 3.2.17, then we can release 3.3.0 this way and fix HiDPI scaling
for 3.3.1.)

--
J.

[zgrose]

svn9292 is horrible on my screen. 3.2.17 and svn9291 much better. Hard to tell in isolation but hard for me to resample the image at the moment. In 3.2.17 and svn9291, the two module lines are as big as the GitHub icon.

Annotation 2019-11-14 161318.png (125.09 KiB) Viewed 2910 times

Annotation 2019-11-14 162045.png (75.06 KiB) Viewed 2910 times

[uckelman]

svn9292 is horrible on my screen. 3.2.17 and svn9291 much better. Hard to tell in isolation but hard for me to resample the image at the moment. In 3.2.17 and svn9291, the two module lines are as big as the GitHub icon.

Wow, that is awful. I was worried that svn9292-uiscale would be very small, not upscaled. What version of Windows is that you're using?

[zgrose]

Windows 10 Pro, v 1903
build 18362.476

[bolabola]

Thanks for the explanation, but I'm not interested in continuing to use
Java in any form for VASSAL 4. Even if all my concerns regarding images
were satisfied, I still have an overwhelming collection of other reasons
for using somehting else. I don't see much future in Java at this point,
and I don't want us tied to it.

Ok, I am sorry you feel that way about Java. Seems there is no point in discussing it further.

I have tested the latest version you provided with the ui scale system property set, and compared it to version 3.1.17. It works in exactly the same way as 3.1.17 and none of the rendering artifacts are present in this version.

Seems to me that this a possible way forward with regards to a 3.3.0 release.

[uckelman]

Thus spake bolabola:
>
> I have tested the latest version you provided with the ui scale system
> property set, and compared it to version 3.1.17. It works in exactly the
> same way as 3.1.17 and none of the rendering artifacts are present in
> this version.

What do you make of the rendering issue zgrose posted about earlier today?

--
J.