Roadmap for VASSAL 4

Sorry to bump in in the middle without a clue, but are you really planning to rewrite the (core) vassal in c++?! If so, please do count me in.

Yes, I have ridiculous schedule and a pile of kids to look after but this is something I have been looking after for so long. Or even if you are just planning to rework the current engine so that it could run on Android system… count me in on that too! :slight_smile:

MVC model is a good for responsibility separation and doing at least the Model part in C++ might be a viable solution. Adding a scripting language support such as Python or Lua should not be a big issue and with JNI Java/Android interface should not be a problem either. IPad etc. on the other hand might be an issue: I really don’t know how Objective-C and C++ plays out, but I have heard rumors that it should be OK.


Rami

Thus spake rami:

Sorry to bump in in the middle without a clue, but are you really
planning to rewrite the (core) vassal in c++?!

Yes.

If so, please do count me in.

We’d like the help. We need to do a bit of design work before diving
in; criticizsm of the proposed design would be helpful at the moment.

MVC model is a good for responsibility separation and doing at least the
Model part in C++ might be a viable solution.

I already have C++ demo of the GUI View. It doesn’t handle much yet, but
it can draw pieces and a map and handles user interaction.


J.

I have been reading this thread with great interest. I can see you that you have decided to develop Vassal 4.0 in C++.

All though I love Java, I fully understand why you want to make the move to a C++ implementation, and I’m very happy to see that you are so focused on making a cross-platform implementation.

You have been looking at different libraries, such as wkWidget and boost. I would suggest that you take a look at Qt. Qt has a large range of advantages compared to most other GUI libraries. I’ll list some of them here:

  1. Qt is a complete GUI and application framework library (cross platform network, file handling, XML, threading, and so forth). Its very feature rich, and hence there is no need for other libraries like boost.
  2. The Qt GUI model is very similar to the Swing programming model, and can be learned with minimal effort by Swing developers.
  3. The Qt libraries are extremely well documented (by far the best documentation I have seen). There lots of books available on the Qt libraries, and tons of information can be found on the web. For example: http://doc.qt.nokia.com/4.7/index.html
  4. The Qt libraries are cross-platform (Windows, Mac and Linux). Qt apps can also be compiled to Android devices.
  5. The Qt rendering pipeline is based on OpenGL, and hence has very good performance.
  6. Qt has a huge developer community (250.000 plus developers) and is being actively developed. And Qt is properly one of the most used cross-platform C++ libraries out there today. See: http://www.google.com/trends?q=qt%2C+wxwidgets&ctab=0&geo=all&date=all&sort=0
  7. Qt is open source (GPL and LGPL). Backed by Nokia.
    8 ) Forms and GUI’s is easy to design with the Qt Creator (form designer). GUI’s can naturally also be created progmatically like in Swing if preferred.
  8. Very good IDE support (Qt creator witch is a complete C++ editor with form designer), or Eclipse CDT integration.
  9. There are lots a 3. party widgets available.

You can find the Qt homepage here: http://qt.nokia.com/products/

Instructional videos can be found here: http://developer.qt.nokia.com/elearning

If you want a quick overview of Qt, here is a good introductional video: http://developer.qt.nokia.com/elearning/watch/qt_programming_essentials_by_ics_part_1

I haven’t been programming in C++ for the last 10 years (mainly Java), but I would like to contribute to the Vassal 4.0 project, if possible (Vassal is somehow very dear to my heart :stuck_out_tongue:, me being a passionate board gamer ). I will need to brush up on my C++ though :stuck_out_tongue:

Anyhow, I think it is worth giving Qt a look. I think it would be a perfect fit for a new Vassal 4.0 C++ engine.

QT was already brought up earlier although I can’t say why it is no longer mentioned anymore…

Thus spake Tim M:

“bolabola” wrote:

Anyhow, I think it is worth giving Qt a look. I think it would be a
perfect fit for a new Vassal 4.0 C++ engine.

QT was already brought up earlier although I can’t say why it is no
longer mentioned anymore…

I think the reason is: We’ve been discussing high-level design issues
recently, and which GUI toolkit (or which libraries of any sort) we use
isn’t so relevant at this point.

Regarding Qt, though:

I used Qt some years ago. I found it to be quite irritating that it
had poor integration with the Standard Template Library, and the degree
to which it was generating code using its special preprocessor made
some problems a nightmare to debug. One of the things which people point
to as an advantage of Qt—namely, that it is a giant, all-encompasing
framework providing everything you might need—is also touted as an
advantage of Java, and it one of the reasons why I don’t ever want to
use Java again if I can avoid it. Using a monolithic library means that
it’s harder to swap out parts which are unsuitable or broken, and such
libraries will tend to worm their way into every part of your design. I
want to keep our dependencies on third-party libraries as confined as
possible, so that we’re free to replace them as needed.


J.

I understand your concern, but most if not all cross-platform GUI toolkits are all-encompasing. Primarily due to the sheer complexity of constructing a cross-platform C++ GUI toolkit. Other cross-platform GUI toolkits, like for example wxWidgets and Juce are also all-encompasing, as in so far as they use their own container types (like QString, QList in Qt). STL is not used in any GUI toolkit I have been able to find. The reason cross-platform GUI toolkits don’t use STL, I think, is that STL is not available on all platforms, and that STL is heavily dependent on templates, witch not all compilers handle adequately.

Qt has some support for STL in its container classes, allowing for relatively easy bridging between the container types. Using Boost with Qt shouldn’t be a problem, if this is preferred to Qt utility libraries (networking, threading, file handling and so forth).

Decoupling the toolkit libraries from the application model / architecture can be ensured to some degree by a proper design using a layer of indirection between the application model and the GUI toolkit libraries, though a complete decoupling (toolkit having no effect on the design of the application architecture), would be hard to achieve. But this would be true for any GUI toolkit.

So short of developing your own cross-platform GUI toolkit (witch of cource isn’t feasible :stuck_out_tongue:), you’ll always end up with some dependcies to the GUI toolkit. Changing the GUI toolkit once implemented, would be a huge task no matter witch toolkit is choosen. So choosing a toolkit with proper cross-platform support, proper documentation, proper design, good rendering performance, vibrant community and so forth, is in my eyes key to succes.

But I can gather from the previous post, that you at the current stage primarily are looking at the model design and module file format, and haven’t really decided on a GUI toolkit yet. I just wanted to make sure that you where aware of the Qt toolkit.

By the way, I love your C++ test demo. Clearly out performs the current Java rendering performance :smiley:

Bo

QT was already brought up earlier although I can’t say why it is no
longer mentioned anymore…

I think the main reason was the preprocessor.

  • M.

Interesting thread. I’ll talk about converting to c++ in next message, just want to say that we seem to be losing track of the most important priority for most of Vassal Users:

Number 1: Vassal 4.0 should run all of the 992 existing modules without a hitch (Well, ideally - in practice at least 99% of them).

If the planned rewrite doesn’t do this then it’s not Vassal 4.0 but NewVassal 1.0, a system with few modules and even fewer users and of very little interest to our customers - the Vassal users.

uckelman wrote:

Editing and Designing may be difficult but that would be made easier by properly documenting the piece traits (with an advanced section of the very clever things people have done with them) rather than producing a new set of advanced traits which work differently (hence existing module designers must start from scratch) and won’t be documented (because the Authors think they’re too easy to need it, hate doing it - like all of us and it’s left for later = never)

The game server is a problem but probably 90% of Vassal users use it for pbem so aren’t worried. If any client can be a server then there is no need for a central server anyway.

The interface may be clunky and dated but it works and most Vassal players know how to use it and will probably scream very loudly if it changes in a new version. It’s easy to update the software but much harder to update the users :smiley:

The codebase does need fixing. But we don’t need to jump into a different language etc to do it. Just slog away rewriting it.

Just to reiterate the main point: you don’t have a blank sheet of paper on which to design the perfect game system. You’ve got the legacy of the current system and lots of users to satisfy. Your options are limited.

A lot of modules are going to break with the next version – there is no question about that. For example, the Twilight Struggle module is guaranteed to not work in the next major version because of the custom code therein. There are a number of other modules with custom code that also will not work.

This is to be expected. As a rule, major version changes break existing code. Hopefully, the developers of those modules will be on top of it! (I know the designer of the Twilight Struggle module will be.)

It’s also not guaranteed that the next platform will be based on C++. I’m not sure that it’s my top choice. Right now, my top choice would be wxPython or even just pyOpenGL on top of tkinter. It runs surprisingly fast and is exceedingly easy to code.

But I’m fickle.

  • M.

On 2011-09-05, at 12:23 PM, george973 george973@btinternet.com wrote:

Interesting thread. I’ll talk about converting to c++ in next message,
just want to say that we seem to be losing track of the most important
priority for most of Vassal Users:

Number 1: Vassal 4.0 should run all of the 992 existing modules without
a hitch (Well, ideally - in practice at least 99% of them).

If the planned rewrite doesn’t do this then it’s not Vassal 4.0 but
NewVassal 1.0, a system with few modules and even fewer users and of
very little interest to our customers - the Vassal users.

uckelman wrote:

I’ve identified four problems which I see at the most serious ones
we’re facing:

  • Editing and designing modules is unnecessarily difficult.
  • The game server is a single point of failure and will not scale.
  • The interface is clunky and dated.
  • The codebase is tangly, hard to modify and debug.

Editing and Designing may be difficult but that would be made easier by
properly documenting the piece traits (with an advanced section of the
very clever things people have done with them) rather than producing a
new set of advanced traits which work differently (hence existing module
designers must start from scratch) and won’t be documented (because the
Authors think they’re too easy to need it, hate doing it - like all of
us and it’s left for later = never)

The game server is a problem but probably 90% of Vassal users use it for
pbem so aren’t worried. If any client can be a server then there is no
need for a central server anyway.

The interface may be clunky and dated but it works and most Vassal
players know how to use it and will probably scream very loudly if it
changes in a new version. It’s easy to update the software but much
harder to update the users :smiley:
The codebase does need fixing. But we don’t need to jump into a
different language etc to do it. Just slog away rewriting it.

Just to reiterate the main point: you don’t have a blank sheet of paper
on which to design the perfect game system. You’ve got the legacy of the
current system and lots of users to satisfy. Your options are limited.


Read this topic online here:
Roadmap for VASSAL 4 - #209 by george973


messages mailing list
messages@vassalengine.org
vassalengine.org/mailman/listinfo/messages

Thus spake Michael Kiefte:

It’s also not guaranteed that the next platform will be based on C++. I’m not
sure that it’s my top choice. Right now, my top choice would be wxPython or
even just pyOpenGL on top of tkinter. It runs surprisingly fast and is exce
edingly easy to code.

The problem I ran into in the Python demo I wrote was that the parts
which were not being handled by the video hardware were surprisingly
slow. Loading large images, for example, was much, much slower than
Java, while the same images loaded instantly in C++. I also have
serious doubts about the maintainability of large projects writen in
dynamic languages. No compile-time checking means more bugs to find
at runtime. I think Python is a good choice for our scripting language,
but not a good choice for the engine itself.


J.

Had you compiled the python code to bytecode?

Your other arguments are valid though.

  • M.

On 2011-09-05, at 1:47 PM, Joel Uckelman uckelman@nomic.net wrote:

Thus spake Michael Kiefte:

It’s also not guaranteed that the next platform will be based on C++. I’m not
sure that it’s my top choice. Right now, my top choice would be wxPython or
even just pyOpenGL on top of tkinter. It runs surprisingly fast and is exce
edingly easy to code.

The problem I ran into in the Python demo I wrote was that the parts
which were not being handled by the video hardware were surprisingly
slow. Loading large images, for example, was much, much slower than
Java, while the same images loaded instantly in C++. I also have
serious doubts about the maintainability of large projects writen in
dynamic languages. No compile-time checking means more bugs to find
at runtime. I think Python is a good choice for our scripting language,
but not a good choice for the engine itself.


J.


messages mailing list
messages@vassalengine.org
vassalengine.org/mailman/listinfo/messages

postby mkiefte » 05 Sep 2011 17:25

Custom class modules breaking might be acceptable depending how many there are. Is there anyway to look at the modules on Vassal site to see how many have custom classes (contain .class files)?

I don’t think breaking other modules would be especially as the developers will often have moved on and no longer be interested. If too many modules break then users will remain on Vassal 3.x

Actually major versions don’t break the code, normally. Every new version of OpenGL library doesn’t mean all the programs using it must be re-written. Every new version of c or c++ compiler doesn’t mean rewriting all your source code. Every new version of windows or linux doesn’t mean throwing away all your programs and buying a new set. (Although I’m sure MS would make you do this if it thought it could get away with it!)

If the new version can’t read most of the modules then it is not a version of Vassal. It’s something else.

I hope not. The delivery of software that runs on multiple platforms is a very large piece of work. I did this professionally for many years and the amount of time spent working out why the system doesn’t work on build X of system Y when it does everywhere else can take more time than developing the system in the first place.

Consider just the actual deliverables you need to create: binaries for Windows XP, Vista, Windows 7, linux (RedHat, Ubuntu, Debian), Mac OSx Lion, Leopard, … With 64 bit and 32 bit varieties all with installation packages to ensure all the libraries you need are also installed (in the correct version). For a release you need to build all these, test them, apply any fixes, rebuild , test and so on. It used to take a team of 4 or 5 working full time most of a month to get all our builds for a particular release together and ready to go and someone would always come up with an urgent fix to be squeezed in at the last moment with predictable results for the quality of the release.

Going with C++ or anything that needs platform dependent deliverables is making yourselves a lot of work. Stick with Java. Fix the Java bugs in OpenJDK (which is being ported to Mac).

Thus spake Michael Kiefte:

Had you compiled the python code to bytecode?

I believe so, yes. But it wasn’t my code that was slow, it was the
image loading code I was calling that was slow.


J.

I dont think there is but having gone through most of the modules maintaining the library I would say the number of modules with custom classes is less than 100 modules or 10% there abouts.

Breaking these modules should not be a major issue for most of them as the majority contain custom classes that get incorporated into the core at some point (such as the turn counter for example) down the road and only need to be updated to reflect this after the class is adapted into the core engine.

The remaining few such as your OCS code or Mike’s TS code are the exceptions and will be the problem modules that you’ll have to handle yourselves unless you can make it generic enough to include in Vassal if you think it could be useful for others ( hint - cards in hand count shows in toolbar, looking at you Mike! :smiley: )

Thus spake george973:

A lot of modules are going to break with the next version – there is
no question about that. For example, the Twilight Struggle module is
guaranteed to not work in the next major version because of the custom
code therein. There are a number of other modules with custom code
that also will not work.

Custom class modules breaking might be acceptable depending how many
there are. Is there anyway to look at the modules on Vassal site to see
how many have custom classes (contain .class files)?

I could write a script to run on the server to check on this, but I’d
rather not spend time doing that unless we need to know precisely. My
experience with examining random modules has been that the answer is
“few”.

I don’t think breaking other modules would be especially as the
developers will often have moved on and no longer be interested. If too
many modules break then users will remain on Vassal 3.x

Writing a module converter is one of the projects which I expect we will
pursue nearly from the start with V4. All of the built-in stuff in V3
will map straightforwardly to something in V4.

I hope not. The delivery of software that runs on multiple platforms is
a very large piece of work. I did this professionally for many years and
the amount of time spent working out why the system doesn’t work on
build X of system Y when it does everywhere else can take more time than
developing the system in the first place.

Consider just the actual deliverables you need to create: binaries for
Windows XP, Vista, Windows 7, linux (RedHat, Ubuntu, Debian), Mac OSx
Lion, Leopard, … With 64 bit and 32 bit varieties all with
installation packages to ensure all the libraries you need are also
installed (in the correct version). For a release you need to build all
these, test them, apply any fixes, rebuild , test and so on. It used to
take a team of 4 or 5 working full time most of a month to get all our
builds for a particular release together and ready to go and someone
would always come up with an urgent fix to be squeezed in at the last
moment with predictable results for the quality of the release.

Going with C++ or anything that needs platform dependent deliverables is
making yourselves a lot of work. Stick with Java. Fix the Java bugs in
OpenJDK (which is being ported to Mac).

We have platform-dependent deliverables and platform-specific code with
Java, plus a load of decade-old bugs for which we’ve had to write
workaround code. Java is not platform-independent for nontrival things,
despite what Sun and now Oracle claim. I already spend quite a lot of
time determining why things work on one platform but not another (by
my count, something like 20 hours in the past two weeks alone), a lot
of which is due to Java bugs. I no longer have any faith that these bugs
will be fixed. With C++, we can rely on higher-quality libraries, ones
where you don’t typically encounter bugs which are nearly old enough to
drive.

For more detail, I refer you to points 1, 3, and 5 in this post:


J.

The remaining few such as your OCS code or Mike’s TS code are the
exceptions and will be the problem modules that you’ll have to handle
yourselves unless you can make it generic enough to include in Vassal if
you think it could be useful for others ( hint - cards in hand count
shows in toolbar, looking at you Mike! :smiley: )

It’s in my inbox! (i.e., my todo list)

  • M.

Python Imaging Library?

  • M.

On 5 September 2011 15:29, Joel Uckelman uckelman@nomic.net wrote:

Thus spake Michael Kiefte:

Had you compiled the python code to bytecode?

I believe so, yes. But it wasn’t my code that was slow, it was the
image loading code I was calling that was slow.


J.


messages mailing list
messages@vassalengine.org
vassalengine.org/mailman/listinfo/messages

Thus spake Michael Kiefte:

Python Imaging Library?

I think it might have been, yes. I’ll see about getting you the code
as soon as I can.


J.

Thus spake george973:

The codebase does need fixing. But we don’t need to jump into a
different language etc to do it. Just slog away rewriting it.

I’ve been trying to refactor parts of the current codebase for literally
years now. This experience has led me to conclude that:

  1. This cannot be done with any reasonable amount of effort. E.g., the
    current codebase is so messed up w/r/t thread safety, that it’s just not
    feasible to fix it. It is, I believe, feasible to do it correctly from
    the start, if one is trying to do that. Our problem is that nobody paid
    any attention to these things until the 3.0 release or thereabouts.

  2. This cannot be done while maintaining existing class hierarchies in
    a backwards-compatible way. E.g., doing something as simple as adding
    the capacity for components to report on what images they use will
    break every piece of custom code in existence.

Just to reiterate the main point: you don’t have a blank sheet of paper
on which to design the perfect game system. You’ve got the legacy of the
current system and lots of users to satisfy. Your options are limited.

I don’t intend to abandon existing modules. A working module converter
will be one of the V4 milestones.


J.

by uckelman » 01 Aug 2011 21:03:

Hmmm. It all depends on which part of a multi-platform project you work on. We always estimated that 90% would be platform independent and 10% platform dependent but the effort would be 50-50 for both parts. We were writing in objective-C so had ported the GNU compiler to each platform (so avoiding problems where system X’s compiler optimises your code to not working).

The basic problem is that nothing worth doing ports easy and anything that ports easily is trivial.

Don’t forget the Java implementation on any system stands on the image i/o libraries for that system. The implementors of Java didn’t re-invent the wheel. Any code you write will need to sit on the same libraries and hit the same bugs possibly.

Don’t forget the major testing headache. Now you can assume that Vassal will run on 90% of machines which can install JRE but if you write in C++ you’ll have to build it and test it on all possible environments including old versions of MacOS and Windows and Linux.

You know it works on whatever you’ve built it on. Don’t forget most users are not at the leading edge of Windows, Linux or MaxOS and their machines have old libraries which may have old bugs. You’ll not be able to carry your new libraries around with you as they may not work on older software and/or installing them on older systems may break other programs on the machine. Not to mention 64-bit and 32-bit version of the libraries.

If you don’t want to wait for a bug to be fixed join openJDK and fix the bugs there. They’ll get into mainstream Java and openJDK is porting to Mac so you’ll have 1.6 on the Mac plus bug fixes.

I would hope so!! But if just speed was the goal would it be better to go for Java Native classes for the bits that need speeding up

Don’t forget that no one writes a post on the forum to say his JRE installation worked perfectly! You’ll get a very distorted view if you take the forum posts to represent the general experience. There are thousands using Vassal who’ve had little or no problem.

Do our users care if they are in a code ghetto? They just want their modules to work. As they do now for the most part. The number of bug reports is minimal compared to the number of games played on Vassal.

Hmm, to be a little cynical but perhaps realistics, I suspect someone will be saying the same about your new design in five years time. A design is only clean if you’ve thought everything out beforehand and never extend it. In reality any program grows by accretion after its initial release as users request thing never thought of which don’t quite fit the design or do things with your product which the designers never imagined - and with scripts the module developers will be able to really go to town. Consider what has been done with traits which never crossed anyone’s mind.

But I do agree with the major rewrite, just not with abandoning the Java platform to do it.

by uckelman » 05 Sep 2011 22:58 :

Looking at the code even just quickly shows that it wasn’t written to be multi-threaded and the layout of the packages etc is all wrong. So a major rewrite is called for but this does not mean leaping into something totally different.

Yes, I accept all my custom code will be broken, reluctantly but I do accept it. I suspect that mkiefte and I account for most of it anyway.

I suspect the users would just like the new version to load existing modules and save files without them having to re download new modules (assuming you convert all those on the web site).

Conversion in Java would be easiest as you could just replace existing classes in the buildFile by stubs which collected the data and then created the new classes from that. You could do something like that in C++ but would have to re-write the stubs rather than just stripping surplus code out of existing classes.

I think you are making yourselves a lot of work going to C++ and I don’t think you’ll reap as many benefits as you think especially for portability.