Create account / Log in

Lua

Discussion area for the development team.

Moderators: uckelman, Tim M

Lua

Postby Rhett » April 24th, 2020, 12:46 pm

In this post I will give information about Lua. I will tell how to install it. I will give my personal opinions about Lua, what I consider its pros and cons as I understand it right now.

I stress that I have never worked with Lua. Therefore there is a lot to learn.

I installed Lua on Linux Mint 19.3.

I recommend that everyone first installs Lua on a virtual machine. VirtualBox is excellent for this use. Download the Linux Mint 19.3 iso and install it on VirtualBox.

Lua itself is found on

https://www.lua.org/download.html

as lua-5.3.5.tar.gz

Before building Lua install this:

Code: Select all
sudo apt-get install build-essential
sudo apt-get install libreadline-dev

Then extract and:

Code: Select all
cd ~/Downloads/lua-5.3.5
make linux


Install Lua with:

Code: Select all
sudo make linux install


lua and luac will now be in /usr/local/bin
luac is the 'compiler' of Lua scripts
lua both runs Lua scripts and compiled Lua scripts

In Geany:

luac -o "%e" "%f" as Build command
lua "%e" as Run command


As I understand it, the most used libray for making Lua GUI componets is wxWidgets.

https://wxwidgets.org/downloads/

Latest Stable Release: 3.0.4

Before building install this:

Code: Select all
sudo apt-get install libgtk2.0-dev
sudo apt-get install freeglut3-dev
sudo apt-get install libcppunit-dev
sudo apt-get install libwebkitgtk-dev


Then extract to wxWidgets-3.0.4 and:

Code: Select all
cd  wxWidgets-3.0.4
mkdir buildgtk
cd buildgtk


First you need to configure the Make. There are many options and for the time being I chose these:

Code: Select all
../configure --with-opengl --with-gtk –with-regex

The option –enable-debug will of course be of use (and many others). Then:

Code: Select all
make
sudo make install
sudo ldconfig


The 'make' step will take some time.


wxLua is a “Lua scripting language wrapper around the wxWidgets cross-platform C++ GUI library”.

http://wxlua.sourceforge.net/download.php

wxLua-2.8.12.3-src.tar.gz


First install Cmake-qt-gui in Software Manager. You also need to install this:

Code: Select all
sudo apt-get install libncurses5-dev
sudo apt-get install libgtk-3-dev
sudo apt-get install doxygen


Then (in any location)

Code: Select all
mkdir wxLua-build


Start Cmake and locate extracted wxLua-2.8.12.3-src folder and the wxLua-build folder. I recommend

Options->Warning Messages->Suppress Warnings->Developer Warnings

Remove media from wxLuaBind_COMPONENTS and xWidgets_COMPONENTS


Click Configure several times. Finally press Generate. Go to wxLua-build and

Code: Select all
make



There is a tool for building self-running Lua programs called srlua.

http://webserver2.tecgraf.puc-rio.br/~l ... lua/#srlua

Extract to folder srlua-102. In folder:

Code: Select all
make
sudo make install



I have found that srlua works well for non-wx scripts.

Code: Select all
srglue /usr/local/bin/srlua test.lua test
./test


For wx scripts add this at the top:

Code: Select all
package.cpath = '../wxLua-build/lib/Debug/lib?.so'
require('wx')

It compiles without errors but running it (as lua ./wxtest) gives a Segmentation fault (core dumped).

But this works for the same script:

Code: Select all
../wxLua-build/bin/Debug/wxLua -c wxtest.lua


It would be nice if a compiled Lua script could be distributed without the user having to download anything else (like we have to download the Java run-time).



Pros and cons for Lua in my opinion:


Pros:
  • Sandboxing for security, see http://lua-users.org/wiki/SandBoxes. Obviously Lua must have the ability to save games on the host file system.
  • Lua has its own IDE with debugging, see f.ex https://studio.zerobrane.com/
  • The huge flexibility of a scripting language. In Vassal all modules are really just scripts. Instead of inventing a meta-language (or making java custom classes) the modules can use the script Vassal itself is made of.

Cons:
  • Lua is unknown to many (ulike Java) and needs to be learned. It is a different language from, say, Javascript. It does not have the wide knowledge-base as Java has.
  • Lua programs need to be distributed in different variants, for Windows, Linux, Mac and Andriod. In Java the work is already done for us in the Java runtime.
  • Lua is not strictly an object-orientated language though it has object orientation features,
    read here: http://lua-users.org/wiki/ObjectOrientationTutorial. What impact this will have on the development of a large program I don't know. A strong-typed language is always an advantage.


As I said, this is just based on my very superficial knowledge of Lua. I hope others can expand on it. The ability and proficiently of manipulating images in the GUI (scaling and rotating) will be most important.
Rhett
 
Posts: 41
Joined: March 18th, 2014, 9:32 am

Re: Lua

Postby Brent Easton » April 24th, 2020, 1:04 pm

Hi Rhett,

You may be interested in the series of blog articles I wrote here http://www.vassalengine.org/wiki/Vassal_4 evaluating potential scripting languages for Vassal 4 and the results of a test implementation I did building a Lua scripting framework for a dummy C++ Vassal 4 implementation.

While Lua may not be well-known to professional programmers, it is probably the most likely language to be known by non-programmers interested in gaming as it has become the default language of choice for writing mods for video games.

Cheers,
Brent.
User avatar
Brent Easton
 
Posts: 3168
Joined: December 21st, 2007, 3:06 am
Location: Berry, NSW, Australia

Re: Lua

Postby Rhett » April 25th, 2020, 1:58 pm

Yes, I have read your blog. It was interesting. It shows (among other things) the flexibility we have with scripting.

It is important to find out if Lua meets all our needs, how it meets them and how efficient it does it. With my post I hope to start this gathering of information. We can of course not start a huge software project without knowing if the tools we use are sufficient.

I was disappointed that srlua did not work for a wx-script, but the reasons for that may be many. Do we at all need to distribute Lua Vassal as a stand-alone package?

I hope my post inspires others to find the answers. It is important to listen to the input from those who make modules, what needs and wishes they have.
Rhett
 
Posts: 41
Joined: March 18th, 2014, 9:32 am

Re: Lua

Postby Brent Easton » April 26th, 2020, 12:06 am

Hi Rhett,

It is important to find out if Lua meets all our needs, how it meets them and how efficient it does it. With my post I hope to start this gathering of information. We can of course not start a huge software project without knowing if the tools we use are sufficient.


I did a quite extensive review of the of the different options available, how a scripting language can meet our needs and some efficiency tests. For a scripting Language, we only need a basic, fast language that allows for calls back into C++ to do any heavy lifting. Lua gives us this. Lua was designed from the start as an efficient scripting language and is pretty much as efficient as a scripting language can be. I did some benchmarks against a couple of Javascript implementations and Lua was 6-10 times faster. It is 30 times faster than the current Beanshell scripting language we are using in Vassal 3.

There are overheads in the C++ to Lua to C++ interfaces, but I have tried to make these as efficient as possible. I do not know yet whether we will be able to support custom C++ code in modules, or whether they will be needed. Some of my modules use custom dialogs that may be difficult to do in Lua, but we will see. We may be able to build support for Lua to call out to our widgets library.

However, the sandboxing capabilities in the Lua environment is the real clincher. There is no point in having a screaming fast scripting language if it can be used to crash modules, either deliberately or accidentally.

At this stage, I would expect that the Lua scripting implementation I have written will be built into the initial test releases of Vassal 4.

I was disappointed that srlua did not work for a wx-script, but the reasons for that may be many. Do we at all need to distribute Lua Vassal as a stand-alone package?


At this stage, no widgets library has been selected for Vassal 4. Wx-widgets and Qt have been suggested. The Lua source I chose to use is the stock standard vanilla 5.3 Lua source from lua.org. The recommendation from Lua.org is that for a project like ours, we take a fork of their source and maintain that seperately as part of our own project. I had to make a couple of changes to the standard Lua source to patch a couple of security vulnerabilities that could be taken advantage of by malicious scripts.

As Lua is stock standard ANSI C, it can be compiled and linked directly into the Vassal executable on any platform we choose to support.

I hope my post inspires others to find the answers. It is important to listen to the input from those who make modules, what needs and wishes they have.


Absolutely agree, the more input we get, the better Vassal 4 will be. My background is that I am a long-time Vassal developer and also have built a library of modules make extensive use of custom Java code. My goal would be to try and bring all of that custom code 'into' the Vassal eco-system via Lua.

Regards,
Brent.
User avatar
Brent Easton
 
Posts: 3168
Joined: December 21st, 2007, 3:06 am
Location: Berry, NSW, Australia

Re: Lua

Postby Rhett » April 27th, 2020, 11:55 am

Hi Brent.

The recommendation from Lua.org is that for a project like ours, we take a fork of their source and maintain that separately as part of our own project. I had to make a couple of changes to the standard Lua source to patch a couple of security vulnerabilities that could be taken advantage of by malicious scripts.

Ok, this means that we can make our own Lua. This is the advantage we have with open source. We are free to customize the code to our needs and correct bugs.

I suppose the downside is that we both have to develop and maintain the tools, while using the tools to develop Vassal.


However, the sandboxing capabilities in the Lua environment is the real clincher. There is no point in having a screaming fast scripting language if it can be used to crash modules, either deliberately or accidentally.

Yes, this is always a worry. Today we take it for granted that the Java custom classes in a module are safe. I guess the administrators oversee that they are so (that they inspect the source).

When a module-script has more access to the data structures of Vassal, there is a risk that the module will compromise Vassal. One way of dealing with this is to limit what a module-script can do. We create an API to Vassal, where data can only be read or written in a very controlled way through this API. It will also involve changing Lua itself, for example not allowing it to write directly to the host file system.

Javascript is a good example of a well defined sandbox, which also allows add-ons to further limit accress to the Javascript API (for example canvas blockers).


I do not know yet whether we will be able to support custom C++ code in modules, or whether they will be needed. Some of my modules use custom dialogs that may be difficult to do in Lua, but we will see. We may be able to build support for Lua to call out to our widgets library.

In my opinion, if we need C++ code in a module then a little bit of the point with scripting is gone. Lua and its GUI should have the necessary features for module-makers. Since we have access to the source code (of say wxWidgets or what package we chose) I suppose we should be able to make the GUI that fits our needs.

C++ has many advantages for developing huge software projects, i.e. object-orientation and type-checking. I don't know to what extent Lua can directly access C++ datastructues. We could develop Vassal in C++ and let modules be developed with our custom Lua with access to the Vassal C++ datastructures.


Many modules are static. Today a module is just a huge XML file. I have found it useful many times to edit this file directly. Unfortunately, if you make one small error then the module will not load in the editor. The whole procedure of defining counters and other static content has to be done in a more straightforward way. The XML file(s) that make up a module have to report what is wrong with them (by "compiling them" and reporting "compile errors").


Rhett
Rhett
 
Posts: 41
Joined: March 18th, 2014, 9:32 am

Re: Lua

Postby Brent Easton » April 27th, 2020, 12:53 pm

Ok, this means that we can make our own Lua. This is the advantage we have with open source. We are free to customize the code to our needs and correct bugs.

I suppose the downside is that we both have to develop and maintain the tools, while using the tools to develop Vassal.


True, but I think the Lua C code itself will be completely stable. There should be no need to ever update it again once we take a fork. Anything new or interesting we need to do, we can just do in C++ and call out from Lua. I only needed to make a couple of small changes to the Lua code itself to resolve a couple of potential security issues that could allow a malicious script to crash out of Lua. Most of the sandboxing I was able to implement in the Lua side of the interface.

Yes, this is always a worry. Today we take it for granted that the Java custom classes in a module are safe. I guess the administrators oversee that they are so (that they inspect the source).


Ah, no :)

When a module-script has more access to the data structures of Vassal, there is a risk that the module will compromise Vassal. One way of dealing with this is to limit what a module-script can do. We create an API to Vassal, where data can only be read or written in a very controlled way through this API.


Already defined and implemented as part of my demo. Lua scripts are supplied with a pointer to a proxy Vassal object, not the real thing. The proxy object defines a strict API that the Lua code has access to.

In my opinion, if we need C++ code in a module then a little bit of the point with scripting is gone. Lua and its GUI should have the necessary features for module-makers. Since we have access to the source code (of say wxWidgets or what package we chose) I suppose we should be able to make the GUI that fits our needs.


I mostly agree. Lua has no native GUI itself, we would need to provide functionality in the C++ interface for it to be able to build and use GUI elements like dialog boxes.

However, some of the custom code I have implemented for my modules would be difficult to build in Lua. Mostly for efficiency reasons, but also because of the scope of the code required. Having said that, I am hoping that some of what I have built can be be built into Vassal 4 as a standard feature.

Lua will be there as a scripting language to automate, tweak and extend Vassal. It isn't really suitable to re-write large parts of Vassal in it. Not that will stop people from trying!

I don't know to what extent Lua can directly access C++ datastructues. We could develop Vassal in C++ and let modules be developed with our custom Lua with access to the Vassal C++ datastructures.


Lua is extremely simple, but very flexible. Apart from the usual primitives, it has a single data structure that can be programmed to be an array, a structure, a list or whatever is required.

However, I envisage that most data access will be via function calls through the API on proxies, not by mapping C++ structures on to Lua structures as such.

Brent.
User avatar
Brent Easton
 
Posts: 3168
Joined: December 21st, 2007, 3:06 am
Location: Berry, NSW, Australia

Re: Lua

Postby Cattlesquat » April 27th, 2020, 10:27 pm

What sorts of things do you envision WON'T be able to be done any more in Vassal 4, that you can presently do with custom classes?

The fact that you've got stuff you built up that you found useful in modules that you think you might not be able to port is actually the first twinge of concern I've had.

Brian
User avatar
Cattlesquat
 
Posts: 890
Joined: December 2nd, 2019, 4:57 pm
Location: Baltimore, Maryland, USA

Re: Lua

Postby Brent Easton » April 27th, 2020, 11:04 pm

The two things that come to mind that I have done in my GTS modules:

    * A custom LOS of thread that highlights blocking terrain in real time as you drag the end of the LOS thread about the map. 'Blocking terrain' types may occupy a whole hex, a hex edge, or be caused by a particular counter type being present in a hex.
    * Complex custom interactive dialog boxes implementing the games combat resolution system.

The custom LOS thread I would not want implement in Lua for efficiency reasons. However, I have hope of building a configurable version into core Vassal.

Potentially, we could build a library for Lua to call and build/handle custom interactive dialogs. Efficiency wouldn't be a concern here, but it would be complex.

Everything else I believe would be handled easily within Lua, if not the expanded feature set of Vassal.

Regards.
User avatar
Brent Easton
 
Posts: 3168
Joined: December 21st, 2007, 3:06 am
Location: Berry, NSW, Australia

Re: Lua

Postby Rhett » April 28th, 2020, 11:04 am

Does Lua handle events? With my limited knowledge of Lua I have not seen it do so ... or ?

Javascript does of course handle events, both user initiated events (click a button) and browser initiated events (page loaded).

I can imagine Vassal C++ catching an event and sending it to the module-script for handling. If it is not handled by the script Vassal C++ will (maybe) do a default handling. The ability to handle events in a script will be important for implementing GUI features.

An example is the CTRL+click event in the Vasl module. It draws a red circle for 1-2 seconds on the map and is a good way of telling your opponent where something is going to happen. To achieve this in a script you need to handle the click event and have a GUI widget that paints the circle on the map and erases it after a certain time.


I suppose many Java custom classes today involve extending Vassal classes to add functionality. How can this be done with a script? Maybe certain useful classes have to be implemented in Lua rather than in C++. If Lua can access C++ datastructures (though an API) can the opposite also be the case ? That Vassal C++ access Lua classes ? Lua classes could be data structures that hold information about a map, a hex and a counter.

Classes in Vassal C++ should never need to be extended. This will define a clear difference between what a script can do and what Vassal must do. Obviously saving a game is something that only Vassal can do. A script should never be able to override the core functions (classes) of Vassal.


I think a lot of work has to be done with the design of Vassal 4. This will later save us a lot of work, both in developing and later updating Vassal 4, and when module-makers start making modules with it. There is much to be said about using a strong-typed, object-orientated language like C++ for developing Vassal, but it must also be flexible enough for a large variety of modern games.
Rhett
 
Posts: 41
Joined: March 18th, 2014, 9:32 am

Re: Lua

Postby Brent Easton » April 28th, 2020, 11:33 am

I suppose many Java custom classes today involve extending Vassal classes to add functionality. How can this be done with a script?


By adding sufficient hooks into the standard Vassal features where you can (optionally) call out to a script to modify, enhance or even inhibit the standard Vassal operation. Each hook will be called with the necessary arguments to access the functionality and a well documented API to 'do stuff'.

Maybe certain useful classes have to be implemented in Lua rather than in C++.
.

It may turn out that way, but is not something I am planning on at the start. Doing anything in Lua will be an order of magnitude slower than doing it in C++, so I would like to try and avoid a heavy dependence on Lua for 'standard' processing. That is not going to stop people from writing epic Lua scripts, I'm sure.

If Lua can access C++ datastructures (though an API) can the opposite also be the case ? That Vassal C++ access Lua classes ? Lua classes could be data structures that hold information about a map, a hex and a counter.


While that is possible, it doesn't make sense to call from a super fast environment into a slow environment to do work.

A script should never be able to override the core functions (classes) of Vassal.


Yes, but what is 'core' is probably quite small. Everything else should be accessible.

I think a lot of work has to be done with the design of Vassal 4.


Amazingly, Vassal 4 was first mooted nearly 9 years ago. How the time has flown!
User avatar
Brent Easton
 
Posts: 3168
Joined: December 21st, 2007, 3:06 am
Location: Berry, NSW, Australia

Re: Lua

Postby Rhett » May 4th, 2020, 2:13 pm

For your information, it is possible to access Lua classes in C++. Below an example:


Code: Select all
-- script.lua

Counter = {
hex = 0,
move = function(self, v)
          self.hex = self.hex + v
       end
}

function Counter:position(v)
   self.hex = v
end

Counter.position(Counter, 2)
Counter:move(3)



Code: Select all
// vassal.cpp

#include <iostream>
#include <string>
 
extern "C" {
# include "lua.h"
# include "lauxlib.h"
# include "lualib.h"
}

 
int main()
{
    lua_State* L = luaL_newstate();
 
    luaL_loadfile(L, "script.lua");
    lua_pcall(L, 0, 0, 0);
   
   
    lua_getglobal(L, "Counter");
    int top = lua_gettop( L );
    lua_getfield(L, top, "move");
    lua_pushvalue(L, -2);
    lua_pushinteger(L, 4);
   
    lua_pcall(L, 2, 0, 0);
    lua_pop(L, 1);

   
    lua_getglobal(L, "Counter");
    lua_pushstring(L, "hex");
    lua_gettable(L, -2);
    int balance = (int)lua_tonumber(L, -1);
    lua_pop(L, 1);
   
    // prints 2 + 3 + 4 = 9
    std::cout << "Counter position = " << balance << std::endl;
   

 
    lua_close(L);
}


Vassal C++ starts and maintains a Lua State, and may later get information from the Lua classes made, for example the hex coordinates of counters on board which it uses to save games.
Rhett
 
Posts: 41
Joined: March 18th, 2014, 9:32 am

Re: Lua

Postby Rhett » May 12th, 2020, 3:37 pm

I have been testing wxLua a little more lately. wxLua does have its own event handling.

I tried to draw a background image (a map) on a panel with an OnPaint event like this:


Code: Select all
package.cpath = '/home/test/Documents/wxLua-build/lib/Debug/lib?.so'
require('wx')

frame = wx.wxFrame(wx.NULL, wx.wxID_ANY, "wxLua Minimal Demo",
                   wx.wxDefaultPosition, wx.wxSize(450, 450),
                   wx.wxDEFAULT_FRAME_STYLE)                   
                   
panel = wx.wxPanel(frame, wx.wxID_ANY, wx.wxDefaultPosition, wx.wxSize(300, 300))

function OnPaint(event)
    local dc = wx.wxPaintDC(panel)
    local bitmap = wx.wxBitmap("/home/test/projects/vassal/Map.png")
    dc.DrawRectangle(dc, 10, 10, 50, 50)
    dc:DrawBitMap(bitmap, 0, 0)
    dc:delete()
end
                 
panel:Connect(wx.wxEVT_PAINT, OnPaint)

frame:Show(true)


When I run this, the DrawRectangle works but the DrawBitMap gives this error message:

Code: Select all
Lua: Error while running chunk
/home/test/projects/vassal/wxtest.lua:18: wxLua: Unable to call an unknown method 'DrawBitMap' on a 'wxPaintDC' type.
stack traceback:
   [C]: ?
   /home/test/projects/vassal/wxtest.lua:18: in function </home/test/projects/vassal/wxtest.lua:14>


Does wxLua not wrap DrawBitMap from wxWidgets? Why wrap DrawRectangle and not DrawBitMap? If this is true (and not some bug in my code) it will really restrict what one can do with wxLua.
Rhett
 
Posts: 41
Joined: March 18th, 2014, 9:32 am

Re: Lua

Postby Rhett » May 12th, 2020, 3:59 pm

lol ... it was a bug in my code.

This works:

Code: Select all
package.cpath = '/home/test/Documents/wxLua-build/lib/Debug/lib?.so'
require('wx')

frame = wx.wxFrame(wx.NULL, wx.wxID_ANY, "wxLua Minimal Demo",
                   wx.wxDefaultPosition, wx.wxSize(450, 450),
                   wx.wxDEFAULT_FRAME_STYLE)                   
                   
panel = wx.wxPanel(frame, wx.wxID_ANY, wx.wxDefaultPosition, wx.wxSize(300, 300))

function OnPaint(event)
    local dc = wx.wxPaintDC(panel)
    local bitmap = wx.wxBitmap("/home/test/projects/vassal/Map.png")
    dc.DrawRectangle(dc, 10, 10, 50, 50)
    dc.DrawBitmap(dc, bitmap, 0, 0, false)
    dc:delete()
end
                 
panel:Connect(wx.wxEVT_PAINT, OnPaint)

frame:Show(true)
Rhett
 
Posts: 41
Joined: March 18th, 2014, 9:32 am

Re: Lua

Postby Brent Easton » May 15th, 2020, 11:05 am

We never did get to the point of choosing a development platform. It is unlikely to be wx if we ever want to support IOS or Android.
User avatar
Brent Easton
 
Posts: 3168
Joined: December 21st, 2007, 3:06 am
Location: Berry, NSW, Australia

Re: Lua

Postby Rhett » May 16th, 2020, 3:32 pm

Not being able to support iOS and Android is a real problem. Even if I personally can not imagine playing VASSAL modules on a small screen, the use of various tablets with iOS/Android will likely increase in the years to come.

A little about WxAndroid here:

https://wiki.wxwidgets.org/WxAndroid

It says the "native Android GUI API is in Java and we need some way to call it from C++". I don't know much about this. Andriod itself is a modified Linux. I know it is possible to download the Andriod Native Development Kit (NDK) for C++ if you develop apps with Andriod IDE:

https://developer.android.com/studio/pr ... ative-code


Broadly speaking, it's not a good idea as an application developer to spend much time getting your application to run on all platforms (Windows, Linux, OS X, iOS and Android). This should be taken care of by the tools one uses. The application itself should not need to be modified to suite the different platforms (this is how it ideally should be).


Here they talk about methods for using Lua to build iOS apps.

https://luanova.org/ioswithlua/

It mentions Corona (no joke)

https://coronalabs.com/

Corona is Lua based, cross-platform, can use C++ and is free (no hidden fees, charges, royalties). Sounds fine, but it's not possible to gauge this until a pilot-Vassal with key features is implemented and runs smothly.


Two issues seem very important:

1) The script language (and GUI) must not in any way be able to read or modify the host file system (must be a true sandbox).
2) The GIU must allow scaling and rotating of images to be done correcty and efficiently.

Besides this, the script must have event handling and a minimum of object-orientation. You can not create a large script without encapsulation. Anyone who has developed plain Javascript knows that a file larger that 1000-1500 lines becomes unwieldly.



Other GUI platforms for Lua besides wxWidgets and Corona:

Qt

https://www.qt.io/
https://github.com/lqt5


Gideros

http://giderosmobile.com/

See also here:

http://lua-users.org/wiki/GraphicalUser ... ceToolkits
Rhett
 
Posts: 41
Joined: March 18th, 2014, 9:32 am

Next

Return to Developers

Who is online

Users browsing this forum: No registered users and 1 guest