Using Lua In Delphi¶

For our new game, the current plan is to use Lua to provide content and control, with Delphi doing most of the heavy lifting (rendering, game state saving etc.). To achieve this, it's necessary to establish an interface to Lua (for this, I'm using the excellent interface by Dennis Spreen - VerySimple.Lua - this is specifically for Lua 5.3) and then to get the data out of it. This article is going to focus on the later as VerySimple.Lua is very simple to use (excellent job Dennis ).

Let's start with an introduction to Lua tables with a sample table that contains two tile definitions.

local tiledefs = {
        {
            type = "tile",
            sourceFilename = "_basegame_/graphics/tiles/grassland/grass1.png",
            tileWidth = 32,
            tileHeight = 32,
            tileType = _TTTile,
            sourceFrames = {
                rect(0,0,31,31),
                rect(32,0,63,31),
                rect(64,0,95,31),
                rect(96,0,127,31)
            },
            frameBrushMode = _TBMRandom,
            frameChance = {0.25, 0.25, 0.25, 0.25},
            frameAnimated = false,
            availableForTopDown = true,
            availableForSizeView = false,
            name = "Grass 1",
            groups = "Biome|Temperate|Grass,Grass|Green",
            globalTint = color3f(1,1,1)
        },
        {
            type = "tile",
            sourceFilename = "_basegame_/graphics/tiles/grassland/bog1.png",
            tileWidth = 32,
            tileHeight = 32;
            tileType = _TTTile,
            sourceFrames = {
                rect(0,0,31,31),
                rect(32,0,63,31),
                rect(64,0,95,31),
                rect(96,0,127,31)
            },
            frameAnimated = true,
            animationSequences = {
                {
                    frameSequence = {0,1,2,3},
                    frameTimes = {100,100,100,100},
                    frameTints = {color3f(1,1,1),color3f(1,1,1),color3f(1,1,1),color3f(1,1,1)},
                    animationMode = _TAMLoopForward
                },
                {
                    frameSequence = {2,3,1,0},
                    frameTimes = {50,100,75,25},
                    frameTints = {color3f(1,0,0),color3f(0,1,0),color3f(0,1,1),color3f(0.5,0.5,0.5)},
                    animationMode = _TAMPingPongStartFoward
                }
            },
            animationSynchronized = true,
            availableForTopDown = true,
            availableForSideView = false,
            name = "Bog 1",
            groups = "Biome|Temperate|Grass,Grass|Green"
        }
    }

Just a quick note about some of the things you'll see in these sample definitions. These form part of my test data and as such they are using functions and variables defined elsewhere within the Lua that make my life easier. rect(x1,y1,x2,y2) for example generates the structure {left=, top=, right=, bottom=}, clearly writing rect(0,0,31,31) is much easier than that lengthy table.

At this point, how this data finds it's way into Lua isn't really important, but this data is collected in a global variable called 'datacollector' which is itself a table. There is a structure to this and it looks something like this:-

datacollector = {
    container = {
        tile = {
            [<TILENAME>] = { <TILEDATA> },
            [<TILENAME>] = { <TILEDATA> }
        }
    }
}

Now the critical thing about the Lua API... once you've executed a script, the state is maintained, so if you execute another script it will have access to everything that's gone before it, global variables, functions, etc. providing their scope allows. Once the script has completed, you can gain access to the state the Lua instance was left in using the low level API functions in VerySimple.Lua.Lib, most of which involve manipulating the Lua stack (access as TVerySimpleLua.luastate).

In the example code that follows, it is assumed you are simply using an instance of TVerySimpleLua (in VerySimple.Lua.pas), we'll call it 'vsl'. TVerySimpleLua is a helper class that wraps a lot of functionality and uses VerySimple.Lua.Lib.pas as this unit contains the actual Lua API implementation. It is also assumed that you've executed a script that defines the data outlined above.

So let's look at how, in Delphi, we might begin accessing our data. The first thing we need to do is have Lua put the data we're interested in on the top of the stack. Using the example above, we would do this:-

lua_getglobal(vsl.luastate, 'datacollector');

Note, you will have to do some type casting or buffering when passing strings into and getting them out of Lua as the string type is ansistring.

If you were to look at the top item of the Lua stack, the item type would be LUA_TTABLE. The next step is to process that table using lua_next(), a function that iterates over the elements of the table.

lua_pushnil(vsl.luastate);
while (lua_next(vsl.luastate, -2) <> 0) do
begin
  {We have the next item here}

  lua_pop(vsl.luastate, 1);
end;

Let's get the obvious out of the way first... what are we doing pushing nil onto the Lua stack? Well, lua_next() expects either NIL or a start point to be on the top of the stack. If it finds NIL it will start at the beginning of the table and go through every item, if it finds a name (or a number, but more on that later) it will give you the next item after the one you have named. But beware, do not assume you know the order of the items because in my experience they can come out of the table in a seemingly random order (the exception is arrays, these always come back in the order in which they are defined).

lua_next() expects two parameters, the first is the stack upon which you wish to operate (yes you can have multiple Lua instances in the same application, each with their own state) and the second is the stack index containing the table on which you wish to operate.

You should hopefully start seeing a familiar pattern to the Lua API calls. Most of them expect an instance of lua_state and then they have an index which is a reference to the stack item they are to work on, with -1 being the top item, -2 the next one down etc.

So after our first call to lua_next() the stack might look something like this (the types of the item are shown in <>):-

[-1] [<LUA_TTABLE>]
[-2] [<LUA_TSTRING> "container"]
[-3] [<LUA_TTABLE>]

At this point, we need to descend into the container table and then the tile table. To descend into a table you need to create an inner loop just the same as the example above (I'm adopting a recursive approach for this). At which point the stack should look something like this:-

[-1] [<LUA_TTABLE>]
[-2] [<LUA_TSTRING> "Grass 1"]
[-3] [<LUA_TTABLE>]
[-4] [<LUA_TSTRING> "tile"]
[-5] [<LUA_TTABLE>]
[-6] [<LUA_TSTRING> "container"]
[-7] [<LUA_TTABLE>]

If we descend into the table at the top of the stack now, things will get a little more interesting, so let's assume we've done that and look at the top two items on the stack.

[-1] [<LUA_TSTRING> "_basegame_/graphics/tiles/grassland/grass1.png"]
[-2] [<LUA_TSTRING> "sourceFilename"]

There is the data we want... or some of it at least. From here it's a short hop to loading the data into an object using RTTI which for the simple types LUA_TSTRING, LUA_TNUMBER (this can represent either a float or an integer so you should use lua_isinteger() and/or lua_isnumber() to establish the sub-type) and LUA_TBOOLEAN is pretty straight forward. And I dare say, if you're an RTTI expert you'll be able to do a lot better than me, but I've resorted to detecting the types of the target properties and using custom loader classes that handle that particular property (for example, I have a property called tileAnimationSequences which in Delphi is defined as being TList - for this I have a custom loader dedicated to the property name which is instantiated dynamically using RTTI and then called to load that properties data which is largely composed of much simpler types).

This is a stack dump from my actual code for a real world example.

You'll notice that my stack indices are all positive. These are the actual indices for the items and the negative numbers are offsets, with -1 being the top (in the example above, item 18) of the stack. This is where the negative indexing is much easier to get on with once you remember -1 is the top. You can see the starting point is in item 6, this is the datacollector table and was loaded onto the stack using lua_gettable(). Above that you can see the descent into the structure.

So what is the lua_pop() all about? Well that removes the data for the last item returned by lua_next(), leaving the key on the stack (remember we pushed NIL on at the beginning?) which allows lua_next() to get the next value or return 0 (indicating false, no more data - this is an API that was geared towards use with C and C++, so functions that return boolean values do so as 0 = false, any other value = true, these can be typecast as boolean or simply compared with 0).

Now let's talk about a little gotcha ¶

If like me, you're new to this it's likely you'll write yourself a stack dump function that just dumps the current Lua stack so you can see what's going on. Part of this will be converting the content of the stack to a printable format. You absolutely MUST query the item type for each element of the stack and use the appropriate lua_toXX() function. You can use lua_tostring() on every item, but for LUA_TNUMBER at least this has the unwanted side effect of changing the item type to LUA_TSTRING on the stack (this side effect is caused by lua_tolstring() which is used by lua_tostring()).

This issue becomes a problem when you are processing arrays (such as the sourceFrames field in the example tables). When you encounter an array, it will present like this on the stack:-

[-1] [LUA_TSTRING "the data"]
[-2] [LUA_TNUMBER 1] <- This is the non-zero based index of the item in the array

If you call lua_tostring(vsl.lustate, -2) the stack will end up looking like this:-

[-1] [LUA_TSTRING "the data"]
[-2] [LUA_TSTRING "1"]

All will be well until you call the lua_next() that is iterating over the array. If that happens, your application is likely to make the most ungraceful of exits. This is a known limitation and is documented in the Lua documentation:-

While traversing a table, do not call lua_tolstring directly on a key, unless you know that the key is actually a string. Recall that lua_tolstring may change the value at the given index; this confuses the next call to lua_next. Refer to the Lua reference manual for more information.

You have been warned. One work around for this is to write yourself a function that handles all of the type checking etc. for you and then using the appropriate API functions gets the data before converting it into a string.

I hope someone finds this information useful, I'm sure there will be more articles coming as I continue my deep dive into the Lua API. A big thanks again to Dennis Spreen for his excellent interface.

Using Lua In Delphi¶

Now let's talk about a little gotcha ¶

Comments