Intermediate Scripting: Metatables Tutorial

zoonior · December 16, 2021, 5:53am

GUIDE TITLE: Intermediate Scripting: Metatables Tutorial
ESTIMATED COMPLETION TIME: 30 Minutes
CORE VERSION: 1.0.226

SUGGESTED PREREQUISITES: basic knowledge of Lua

EXPECT TO LEARN:

what metatables and metamethods are;
how to use the __index metamethod to define Lua classes;
how to define arithmetic operators;

Companion code is available in Community Content under the name: "Metatables Tutorial Scripts".

Introduction

Lua is a very well-designed programming language. It’s compact, fast, simple but easily extensible. In fact, extensibility is a key feature of Lua. The main mechanism of Lua extensibility is metatables. It’s metatables that make it possible that Vector3 has (*) and (..) (dot product) operators or that our scripts have classes and methods (called with (:) syntax). If you are planning to do actual programming in Lua, not just use methods of Core API, you will have to figure out metatables. Their concept is not that complicated but grasping it may pose somewhat of a challenge - perhaps because other languages don’t have them; and even if you are a seasoned C# programmer or know every way to say false in Javascript, it may require significant effort.

There are quite a few tutorials on metatables, and, preparing to write this one, I have look through about a dozen of them. They all share the same structure of recapping Lua Reference Manual. If you are able to understand metatables and metamethods just by reading the manual — then congratulations, you don’t need a tutorial! If, on the other hand, you haven’t even made an attempt yet, then it’s not more complicated than trying to get Core Contexts .

We will show how and when metatables work in several small examples.

INFO: you may use Core Editor or any other Lua environment — for example, this simple online Lua playground. Each block of code in this tutorial is self-sufficient and may be run individually.

NOTE: Core uses Lua 5.3, but for security reasons some built-in Lua functions, such as getmetatable, setmetatable, rawget, rawset etc. have been restricted to tables. That’s why we will be talking only about tables in this tutorial.

NOTE: There are 2 operators that you can use to get value from a Lua table: indexing operator table[key] and dot operator table.key. A key is often being called index, especially in the context of arrays, but I will only be using the term key, as per Lua Reference Manual convention.

What is a metatable?

Let’s create a script, define a simple table and place a string at key 1.
When we write the expression simpleTable[1], we get our string; when we write simpleTable[0], we get nil, right?

-- SimpleTable_01.lua

local simpleTable = {}
simpleTable[1] = "string at key 1"

-- Demo:
print("\n# SimpleTable_01 demo:")

-- index table with key `1`
print(simpleTable[1]) --> "string at key 1"
-- index table with key `0`
print(simpleTable[0]) --> nil

That is absolutely correct in the first case, but in the second, where we get nil, it’s not all true. Lua returns nil when there is no such key in the table (it is important to understand that Lua doesn’t think that the table has a nil value at the missing key; it thinks that such key is not part of the table) and doesn’t consider such situation OK; it considers it exceptional but doesn’t know what to do with it, so it returns nil. But before that, it tries to find a way to avoid it - so it checks whether our simpleTable has a metatable.

Metatable is a regular table that is attached to another table by the setmetatable(table, metatable) function. Let’s add a metatable to our script.

-- SimpleTable_02.lua

local simpleTable = {}
simpleTable[1] = "string at key 1"

local simpleMetatable = {}
-- now simpleTable will have metatable: simpleMetatable
setmetatable(simpleTable, simpleMetatable)

-- Demo:
print("\n# SimpleTable_02 demo:")
print(simpleTable[0]) --> still nil ¯\_(ツ)_/¯

And nothing happened! The thing is that in order to resolve such situation, not only Lua has to find a metatable, it also has to find a metamethod __index inside of it. What is a metamethod? It is usually a function placed into a metatable field, name of which starts with __.

There are over twenty metamethods in Lua that cover more or less all operations you can do on a table. Some of them, like __index, are used in exceptional situations (key not found in the table), some are called on an illegal operation (like adding two tables with a (+) operator), some override the default behavior (metamethod __tostring).

Let’s get back to our example and add the __index metamethod to simpleMetatable. It will fire when we attempt to index simpleTable with a missing 0 key.

--- SimpleTable_03.lua

local simpleTable = {}
simpleTable[1] = "string at key 1"

local simpleMetatable = {}

-- now simpleTable will have metatable
setmetatable(simpleTable, simpleMetatable)

-- (!) metamethod `__index` will be called with a table and key as arguments
simpleMetatable.__index = function(table, key)
    return "I'm the __index metamethod."
end

-- Demo:
print("\n# SimpleTable_03 demo:")

-- index table with key `0`, that is present in simpleTable
print(simpleTable[1]) --> string at key 1

-- index table with key `0`, that is missing in simpleTable
print(simpleTable[0]) --> I'm the __index metamethod.

Great. Now if the key that we are trying to access is present in the table, the index operator behaves as it did before, but if the key is missing, it returns the I'm the __index metamethod. string instead of nil.

Let’s move on to a more practical example.

Metamethod __index is mostly used for writing classes. Lua doesn’t have a built-in concept of classes, so usually we create a class by using the popular pattern where a class is defined as a metatable and its __index metamethod recursively refers to that metatable (i.e., itself). Despite its name, metamethod is not necessarily a function; __index can be a table as well.

Practical use of `__index` metamethod

Lua class that doesn't work

For starters, let’s write a class that looks like a real one but doesn’t work! Let’s find out why.

-- DysfunctionalChest.lua

-- (NOTE!) Intentionally incorrectly written class
local DysfunctionalChest = {}

-- constructor of a chest instance
function DysfunctionalChest.New(gold)
    local chest = {gold = gold or 0}
    return chest
end

-- the only method
function DysfunctionalChest:AddGoldToChest(gold)
    self.gold = self.gold + gold
end

-- Demo:
print("\n# DysfunctionalChest demo:")

-- let's print out all fields of the DysfunctionalChest
for key, value in pairs(DysfunctionalChest) do
    print("DysfunctionalChest:", key, value)
end
-- prints:
-- DysfunctionalChest: New function: ...
-- DysfunctionalChest: AddGoldToChest function: ...

-- create new chest
local dysfunctionalChest = DysfunctionalChest.New(100)
-- let's print out all fields of the instance
for key, value in pairs(dysfunctionalChest) do
    print("dysfunctionalChest:", key, value)
end
-- prints:
-- dysfunctionalChest: gold 100

-- (!) there will be an error: attempt to call a nil value (method 'AddGoldToChest')
print(pcall(function()
    dysfunctionalChest:AddGoldToChest(11)
end))

INFO: if you are not yet acquainted with object-oriented programming, just remember that a method is a function that takes an instance of the class as an implicit first argument and thus can access its data. That’s what happens when you use Lua (:) syntax: you get an implicit self as the first parameter.

The last line throws an error, have you already guessed why?

The functions that we defined for the class were placed into the DysfunctionalChest table. There is only one field gold in the dysfunctionalChest. And DysfunctionalChest is in no way connected to its instance, dysfunctionalChest.

To make our class work, we don’t need to copy our methods into every new chest instance (it is wasteful - both in terms of occupied memory and time the constructor spends on copying them), we can simply attach the metatable to a chest instance. The metatable should contain the __index metamethod that is in fact a table with methods. We already have such a table (DysfunctionalChest), we just need to connect it to the __index metamethod.

Simple lua class with one method

-- SimpleChest.lua

-- SimpleChest is a metatable for all our chests
local SimpleChest = {}

-- (!) SimpleChest holds all our methods, so we set __index metamethod to
-- SimpleChest!
SimpleChest.__index = SimpleChest

-- constructor of a chest instance
function SimpleChest.New(gold)
    local chest = {gold = gold or 0}
    -- (!) set SimpleChest as a metatable of the chest instance
    setmetatable(chest, SimpleChest)
    return chest
end

-- the only method
function SimpleChest:AddGoldToChest(gold)
    self.gold = self.gold + gold
end

-- Demo:
print("\n# SimpleChest demo:")

-- create new chest
local simpleChest = SimpleChest.New(100)

-- (!) method works!
simpleChest:AddGoldToChest(11)

print("simpleChest.gold:", simpleChest.gold) --> simpleChest.gold: 111

Now let’s use the __add metamethod to replace the AddGoldToChest method with the addition operator (+).

But first we bring in the programmer’s best friend: the __tostring metamethod.

Metamethod `__tostring`.

The easiest way to debug a Lua program is the print function. But if you just use the print function to print out a table, Lua first calls the built-in function tostring to convert this table to string, then prints out the result. And the result would look like this: "table: 00000206D21F56C0" which is not very helpful debugging-wise. Surely we can print a lua table with a for cycle, but it is rather tedious and somewhat ugly.

Luckily, we have the __tostring metamethod. If it is defined in the metatable, Lua will use it instead of the built-in tostring method.

So let’s get on to it.

-- PrintableChest.lua

local PrintableChest = {}
PrintableChest.__index = PrintableChest
-- constructor of a chest instance
function PrintableChest.New(gold)
    -- Note that in this class constructor is a little shorter; that is
    -- possible because `setmetatable` returns its first argument (table) with
    -- metatable attached.
    return setmetatable({gold = gold or 0}, PrintableChest)
end

-- overrides tostring function
function PrintableChest:__tostring()
    return "PrintableChest.gold: " .. self.gold
end

-- Demo:
print("\n# PrintableChest demo:")

local printableChest = PrintableChest.New(100)
print(printableChest) --> PrintableChest.gold: 100")

Addition Operator `(+)`

The __add metamethod is different from __index and __tostring because we have to define it as a function with 2 operands. If either one of the operands is not a number, Lua searches for the __add metamethod — first it checks the first operand, then the second. If the metamethod is found, Lua calls it with the operands as arguments.

This is the reason we shouldn’t use (:) syntax to define __add — it will only cover half of the cases; self can be a second argument as well.

Let’s agree that when we add a chest and a number (in any order) we add the number to the gold field of the chest. If we add two chests, we return a brand new chest with field gold equal to the sum of gold fields of two other chests.

-- AdditiveChest.lua
local AdditiveChest = {}
AdditiveChest.__index = AdditiveChest

-- constructor of a chest instance
function AdditiveChest.New(gold)
    return setmetatable({gold = gold or 0}, AdditiveChest)
end

function AdditiveChest:__tostring() return "AdditiveChest.gold: " .. self.gold end

-- overrides operator (+)
-- Note the syntax (.) instead of the syntax (:)
function AdditiveChest.__add(left, right)
    if type(right) == "number" then
        -- right is a number and left is a chest
        left.gold = left.gold + right
        return left -- return chest (self)
    elseif type(left) == "number" then
        -- left is a number and right is a chest
        right.gold = right.gold + left
        return right -- return chest (self)
    else
        -- left and right are chests, so we return the new chest with
        -- the sum of their gold
        return AdditiveChest.New(left.gold + right.gold)
    end
end

-- Demo:
print("\n# AdditiveChest demo:")

local newAdditiveChest = AdditiveChest.New(100)
print(newAdditiveChest + 11) --> AdditiveChest.gold: 111
print(22 + newAdditiveChest) --> AdditiveChest.gold: 133
print(newAdditiveChest + newAdditiveChest) --> AdditiveChest.gold: 266

Lua built-in functions: `rawget` and `getmetatable`

You may be wondering if it is possible to index a table without firing metamethods — and whether it is possible to know where the value is taken from: from the table or from the __index metamethod. The answer is yes! There is a built-in Lua function rawget(table, key) that works the same way as the index operator (table[key]), but never triggers metamethods.

INFO: There are 4 raw-access functions in the Lua standard library:

rawget(table, key)to get values from the table without triggering __index metamethod
rawset(table, key, value) to set values in the table without triggering __newindex metamethod
rawequal(left, right) to test for equality without triggering __eq metamethod
next(table, key) to iterate the table without triggering __pairs metamethod

You may also need the getmetatable(table) function to get the metatable from the table.

Anatomy of `__index` metamethod (optional)

What I suggest we do next is write a ourIndexOperator function. We will use pure Lua but semantically it will be identical to the built-in index operator []. Let’s insert a bunch of debug prints to give us a clearer notion of what’s going on.

Disclaimer: It is a toy function: Lua Virtual Machine is written in C; we will merely try to reproduce its behavior in Lua.

-- OurIndexOperator.lua

-- Let's write a substitution for Lua indexing operator `[]`; behavior will be
-- identical to `table[key]`, including firing the metamethod `__index` when
-- appropriate.
local function ourIndexOperator(table, key)
    -- actual value of key in table
    local value = rawget(table, key)
    -- Case 1: value is `nil`, look for metamethod `__index`
    if value == nil then
        -- looking for the metatable
        local metatable = getmetatable(table)
        -- Case 1.1: table has a metatable
        if metatable ~= nil then
            -- looking for the `__index` metamethod in the metatable
            -- Note that we are not using `rawget` here: a metatable may have a
            -- different metatable, this is legal!
            local metamethod = metatable.__index
            -- Case 1.1.1: metatable has an `__index` metamethod
            if metamethod ~= nil then
                -- Case 1.1.1.1: __index metamethod is a table
                if type(metamethod) == "table" then
                -- Note that we are not using `rawget` here: a table-metamethod
                -- may have a different metatable, this is also legal!
                    value = metamethod[key]
                    print("! fire metamethod `__index` as a table")
                    return value
                -- Case 1.1.1.2: __index metamethod is a function
                elseif type(metamethod) == "function" then
                    value = metamethod(table, key)
                    print("! fire metamethod `__index` as a function")
                    return value
                    -- Note that there is no `else` here: Lua ignores
                    -- metamethods with wrong types.
                end
                print("no *valid* metamethod `__index` found")
            -- Case 1.1.2: metatable has no `__index` metamethod
            else
                print("no metamethod `__index` found")
            end
        -- Case 1.2: table has no metatable
        else
            print("no metatable found")
        end
    -- Case 2: this key is present in the table (value not `nil`), return value
    else
        return value
    end
    -- We get here if:
    -- 1. metatable of the table == nil
    -- 2. there is metatable, but there is no __index field
    -- 3. there is __index field, but it is a wrong type: not (table or function)
    return nil
end

-- Demo:
print("\n# OurIndexOperator demo:")

local ourMetatable = {
    __index = function(table, key)
        return "table has no key: " .. key
    end
}
local ourTable = setmetatable({"string at key 1"}, ourMetatable)

print("== table access without triggering `__index` metamethod")
print(ourIndexOperator(ourTable, 1))
print()

print("== table access with triggering `__index` metamethod")
print(ourIndexOperator(ourTable, 0))
print()

print("== remove metatable")
setmetatable(ourTable, nil)
print(ourIndexOperator(ourTable, 0))
print()

print("== set metatable to empty table")
setmetatable(ourTable, {})
print(ourIndexOperator(ourTable, 0))

--[[ Demo output:
== table access without triggering `__index` metamethod
string at key 1

== table access with triggering `__index` metamethod
! fire metamethod `__index` as a function
table has no key: 0

== remove metatable
no metatable found
nil

== set metatable to empty table
no metamethod `__index` found
nil
]]

Intermediate Scripting: Metatables Tutorial

Introduction

What is a metatable?

Practical use of __index metamethod

Lua class that doesn't work

Simple lua class with one method

Metamethod __tostring.

Addition Operator (+)

Lua built-in functions: rawget and getmetatable

Anatomy of __index metamethod (optional)

Practical use of `__index` metamethod

Metamethod `__tostring`.

Addition Operator `(+)`

Lua built-in functions: `rawget` and `getmetatable`

Anatomy of `__index` metamethod (optional)