Introduction

There are a lot of templating engines to choose from. The vast majority of which are primarily geared toward the web. Meaning they’re specially designed for outputting HTML/XML documents. Lua provides more flexibility and can easily be used as a general templating engine. Also, Lua is very easy to embed and use from a C application.

Lua being flexible and easily embeddable means that it can be used to extend a program in a variety of ways. Email notifications, report creation, and receipt generation are some common uses for templating.

Being embeddable means instead of writing a custom template engine using Lua you can use an existing one written in Lua. Lua on my OS X system is 160K in size. Adding Lua and a templating engine (custom or generic library) increases the overall application size by a minuscule amount.

Template Engine

A little bit of work is necessary to use Lua for templating. In any template format a template syntax is necessary as well. The template needs to be parsed and different actions taken depending on what the parser encounters. Lua doesn’t have this built in but it’s not hard to add. Here is a basic template engine I wrote:

template.lua

--- Template renderer.
--
-- Takes a string with embedded Lua code and renders
-- it based on the Lua code.
--
--  All template blocks end with }}. Lua blocks start
--  with { + a modifier specifying the operation.
--
-- Supports:
--  * {{ var }} for printing variables.
--  * {% func }} for running Lua functions.
--
--  Use \{ to use a literal { in the template.
--
--  Multi-line strings in Lua blocks are supported but
--  [[ is not allowed. Use [=[ or some other variation.
--
--  Both compile and compile_file can take an optional
--  env table which when provided will be used as the
--  env for the Lua code in the template. This allows
--  a level of sandboxing. Note that any globals including
--  libraries that the template needs to access must be
--  provided by env if used.

local M = {}

-- Append text or code to the builder.
local function appender(builder, text, code)
    if code then
        builder[#builder+1] = code
    else
    	-- [[ has a \n immediately after it. Lua will strip
    	-- the first \n so we add one knowing it will be
    	-- removed to ensure that if text starts with a \n
    	-- it won't be lost.
        builder[#builder+1] = "_ret[#_ret+1] = [[\n" .. text .. "]]"
    end
end

--- Takes a string and determines what kind of block it
-- is and takes the appropriate action.
--
-- The text should be something like:
-- "{{ ... }}"
--
-- If the block is supported the begin and end tags will
-- be stripped and the associated action will be taken.
-- If the tag isn't supported the block will be output
-- as is.
local function run_block(builder, text)
    local func
    local tag

    tag = text:sub(1, 2)

    if tag == "{{" then
        func = function(code)
            return ('_ret[#_ret+1] = %s'):format(code)
        end
    elseif tag == "{%" then
        func = function(code)
            return code
        end
    end
    if func then
        appender(builder, nil, func(text:sub(3, #text-3)))
    else
        appender(builder, text)
    end
end

--- Compile a Lua template into a string.
--
-- @param      tmpl The template.
-- @param[opt] env  Environment table to use for sandboxing.
--
-- return Compiled template.
function M.compile(tmpl, env)
    -- Turn the template into a string that can be run though
    -- Lua. Builder will be used to efficiently build the string
    -- we'll run. The string will use it's own builder (_ret). Each
    -- part that comprises _ret will be the various pieces of the
    -- template. Strings, variables that should be printed and
    -- functions that should be run.
    local builder = { "_ret = {}\n" }
    local pos     = 1
    local b
    local func
    local err

    if #tmpl == 0 then
    	return ""
    end

    while pos < #tmpl do
    	-- Look for start of a Lua block.
    	b = tmpl:find("{", pos)
    	if not b then
    		break
        end

        -- Check if this is a block or escaped {.
        if tmpl:sub(b-1, b-1) == "\\" then
            appender(builder, tmpl:sub(pos, b-2))
            appender(builder, "{")
            pos = b+1
        else
        	-- Add all text up until this block.
            appender(builder, tmpl:sub(pos, b-1))
            -- Find the end of the block.
            pos = tmpl:find("}}", b)
            if not pos then
                appender(builder, "End tag ('}}') missing")
                break
            end
            run_block(builder, tmpl:sub(b, pos+2))
            -- Skip back the }} (pos points to the start of }}).
            pos = pos+2
        end
    end
    -- Add any text after the last block. Or all of it if there
    -- are no blocks.
    if pos then
        appender(builder, tmpl:sub(pos, #tmpl))
    end

    builder[#builder+1] = "return table.concat(_ret)"
    -- Run the Lua code we built though Lua and get the result.
    func, err = load(table.concat(builder, "\n"), "template", "t", env)
    if not func then
    	return err
    end
    return func()
end

function M.compile_file(name, env)
    local f, err = io.open(name, "rb")
    if not f then
    	return err
    end
    local t = f:read("*all")
    f:close()
    return M.compile(t, env)
end

return M

Without the comments this template engine comes out to about 88 lines. That’s not bad for what it can do. It can print Lua variables and run Lua code. Within the template blocks {{ and {% the syntax is Lua; thats the beauty of this engine.

This engine only supports two features. Printing Lua variables and running Lua code. It can easily be extended to support additional features if necessary. For example:

Includes for other template files to stack them
Includes for env data such as a function library (instead of using require)
Escaping content (HTML/XML tag builder)

Security and Arbitrary (User Provided) Templates

Using a full featured scripting langue will probably make the security conscious a bit worried. Lua can do a lot of things like run arbitrary programs installed on the system, delete files and all sorts of other things that, while useful, might not be the best thing for a template to be able to do. Not to worry this template engine supports running the template in a restricted environment. You’ll notice that the compile functions take an env argument (optional).

The env argument is provided as a table which will be the environment the template runs in. This means you can provide a limited and safe subset of features. Such as not allowing access to the io library. You can even leave out require so the template can’t load other Lua scripts or modules (such as luasocket). This means you can have user provided templates without have to worry about malicious content.

Sample Template

Lets look at a complete example template that uses all of the supported features. This should make the syntax easier to understand. Anything in a template block ({{ {% }}) is taken from the Lua environment.

If there is an error the template engine will return the error instead of the templated text. This is by design. Returning nil, error is a way the engine could be changed to behave on error. Errors can occur for a number of reasons, such as calling a function that doesn’t exist.

template.txt

Lua template file example:

{{ name }}

Foods:{% for _,v in ipairs(foods) do }}
* {{ v }}{% end }}

Here is an \{ and \{\{ which are escaped. Also
}} won't hurt because there is not opening {
Don't forget, {= unsupported blocks are kept }}

{{ var_a }} + {{ var_b }} = {{ var_a + var_b }}
{{ var_a }} + {{ var_b }} = adder({{ adder(var_a, var_b) }})

look we can still do lua in the template:
{% l = "I'm a string" }}{{ l }}
{% l = { "index 1 of a table" } }}{{ type(l) }} {{ l[1] }}
{% l = { z="element z of a table" } }}{{ type(l) }} {{ l["z"] }}

The count is {{ count.get_a() }} and {{ count.get_b() }}

All done

Using the Engine with Lua

The engine and template are only useful if they can be put together. Here is a basic Lua script that sets the variables and implements the function that the above template uses. The script also sets a restricted environment so we can safely run templates.

main.lua

local t = require("template")

if #arg ~= 1 then
    print("usage:ntprog template")
    os.exit(1)
end

-- Variables
var_a = 7
var_b = 19
name  = "John"
foods = { "Apple", "Pear", "Banana" }

-- Count "library"
count = {}
function count.get_a()
    return 4
end
function count.get_b()
    return 5
end

-- Some function
function adder(a, b)
    return a+b
end

-- Safe environmet to prevent templates
-- from doing things like running arbitary
-- commands with os.execute.
local env = {
    pairs  = pairs,
    ipairs = ipairs,
    type   = type,
    table  = table,
    string = string,
    date   = os.date,
    math   = math,
    adder  = adder,
    count  = count,
    var_a  = var_a,
    var_b  = var_b,
    name   = name,
    foods  = foods
}

print(t.compile_file(arg[1], env))

Using Engine From C

Using the template engine with Lua is all well and good but one major advantage of using Lua for this task is Lua’s ability to be embedded into a C program. Everything we did in main.lua we can do in C.

main.c

#include <stdio.h>
#include <string.h>

#include <lua.h>
#include <lauxlib.h>
#include <lualib.h>

typedef struct {
    int a;
    int b;
} counter_data_t;

static int get_a(lua_State *L)
{
    counter_data_t *data;

    data = (counter_data_t *)lua_touserdata(L, lua_upvalueindex(1));
    lua_pushinteger(L, data->a);
    return 1;
}

static int get_b(lua_State *L)
{
    counter_data_t *data;

    data = (counter_data_t *)lua_touserdata(L, lua_upvalueindex(1));
    lua_pushinteger(L, data->b);
    return 1;
}

static int adder(lua_State *L)
{
    int a;
    int b;

    a = luaL_checkint(L, -2);
    b = luaL_checkint(L, -1);

    a += b;

    lua_pushinteger(L, a);
    return 1;
}

static struct {
    char *name;
} globals[] = {
    { "pairs"  },
    { "ipairs" },
    { "type"   },
    { "table"  },
    { "string" },
    { "math"   },
    { NULL     }
};

int main(int argc, char **argv)
{
    lua_State      *L;
    int             count_a  = 4;
    int             count_b  = 5;
    int             var_a    = 7;
    int             var_b    = 19;
    const char     *name     = "John";
    counter_data_t  cd;
    size_t          i;

    if (argc != 2) {
        printf("usage:nt%s templaten", argv[0]);
        return 1;
    }

    memset(&cd, 0, sizeof(cd));
    cd.a = count_a;
    cd.b = count_b;

    L = luaL_newstate();
    luaL_openlibs(L);

    /* Load the template module which we will run our template through. */
    if (luaL_dofile(L, "template.lua")) {
        printf("Could not load file: %sn", lua_tostring(L, -1));
        lua_close(L);
        return 0;
    }
    /* Put the function we're going to run on the stack. */
    lua_getfield(L, -1, "compile_file");
    /* Put the template filename on the stack that we will use. */
    lua_pushstring(L, argv[1]);

    /* Push a table which will be our sandboxed env onto the stack. */
    lua_newtable(L);

    /* Push each of the system libs we want available into the table */
    for (i = 0; globals[i].name != NULL; i++) {
        lua_getglobal(L, globals[i].name);
        lua_setfield(L, -2, globals[i].name);
    }
    /* os.date needs to be pushed a bit differently since we only want
     * to push parts of it. */
    lua_getglobal(L, "os");
    lua_getfield(L, -1, "date");
    /* Set os.date to date in our table. */
    lua_setfield(L, -3, "date");
    /* Remove os from the stack. */
    lua_pop(L, 1);

    /* Push our internal function and variables */
    lua_pushcfunction(L, adder);
    lua_setfield(L, -2, "adder");

    lua_pushinteger(L, var_a);
    lua_setfield(L, -2, "var_a");

    lua_pushinteger(L, var_b);
    lua_setfield(L, -2, "var_b");

    lua_pushstring(L, name);
    lua_setfield(L, -2, "name");

    /* Create a table for count and put its functions into it using the
     * reference to the objects they use as a closure. */
    lua_newtable(L);
    lua_pushlightuserdata(L, &cd);
    lua_pushcclosure(L, get_a, 1);
    lua_setfield(L, -2, "get_a");
    lua_pushlightuserdata(L, &cd);
    lua_pushcclosure(L, get_b, 1);
    lua_setfield(L, -2, "get_b");
    lua_setfield(L, -2, "count");

    /* Create a table and push our food items. */
    i = 1;
    lua_newtable(L);
    lua_pushstring(L, "Apple");
    lua_rawseti(L, -2, i++);
    lua_pushstring(L, "Pear");
    lua_rawseti(L, -2, i++);
    lua_pushstring(L, "Banana");
    lua_rawseti(L, -2, i++);
    lua_setfield(L, -2, "foods");

    /* Right now we have the stack as follows:
     * -1 env table
     * -2 template filename
     * -3 template function
     * -4 template Module
     */
    if (lua_pcall(L, 2, LUA_MULTRET, 0) != 0) {
        printf("Error: %sn", lua_tostring(L, -1));
        lua_close(L);
        return 0;
    }

    /* Print the template (or error message) */
    printf("%sn", lua_tostring(L, -1));

    lua_close(L);
    return 0;
}

C Functionality Explained

The Environment

The second argument to the template engine’s compile function is the environment to use. We create a table and add it to the stack. Then we load each global into the table that we want accessible.

..static struct {
    char *name;
} globals[] = {
    { "pairs"  },
    { "ipairs" },
    { "type"   },
    { "table"  },
    { "string" },
    { "math"   },
    { NULL     }
};
...
    /* Push each of the system libs we want available into the table */
    for (i = 0; globals[i].name != NULL; i++) {
        lua_getglobal(L, globals[i].name);
        lua_setfield(L, -2, globals[i].name);
    }
    /* os.date needs to be pushed a bit differently since we only want
     * to push parts of it. */
    lua_getglobal(L, "os");
    lua_getfield(L, -1, "date");
    /* Set os.date to date in our table. */
    lua_setfield(L, -3, "date");
    /* Remove os from the stack. */
    lua_pop(L, 1);
...

Global libraries are easy because they’re handled by name. Get and set are the only operations we need to worry about. Specific features (functions) within a library are a bit harder. We need to get the global, then pull out the function we want, add it to our env table then remove the library from the stack.

In this case the built in Lua libraries are entirely loaded in C code. This is more to demonstrate the concept than anything else. I’d recommend creating the library environment in Lua instead of C. Basically, create a table and load the application provided functionality, pass it to Lua which will then augment the table with a restricted set of built in Lua libraries. I’m recommending this hybrid approach simply because this is an operation that would be easier in Lua.

Variables

Variables are also easy. We use a name and push their value.

Functions

There are three functions that are made avalaible to the Lua environment. adder, get_a, and get_b. Of these adder is the easy one because it’s just a normal function.

Both get_a and get_b are added to a count table. This is to demonstrate accessing objects from C. Both of these functions use a counter_data_t struct which stores pointers to objects (they’re really ints for a simpler example but pretend they’re opaque pointers to objects). The function retrieves the object it’s associated with an performs and action on it (returns the value but it could be passed to a function that uses the object).

...
    lua_newtable(L);
    lua_pushlightuserdata(L, &cd);
    lua_pushcclosure(L, get_a, 1);
    lua_setfield(L, -2, "get_a");
...

Light user data and C closures are the key here. We store our struct of objects that the functions will need in light user data. By using light user data we can fully manage the object outside of Lua and we don’t have to worry about Lua’s garbage collector. As long as the object is created before and isn’t destroyed until after the Lua state there is nothing to worry about.

As you might know light user data (unlike full user data) can’t be used with luaL_checkudata to verify the data type. This is because light user data can’t have a metatable associated with it (it’s just a C pointer). In this case it’s not a problem because we’re using C closures. The user data is set with lua_pushcclosure which is only accessible from C. This means we can be assured the user data type is correct and we aren’t receiving a random memory address from the calling Lua code (malicious intent, misuse, or mistake).

Tables with Values

In addition to tables with functions we can also push tables with values. In this case a simple array.

...
    /* Create a table and push our food items. */
    i = 1;
    lua_newtable(L);
    lua_pushstring(L, "Apple");
    lua_rawseti(L, -2, i++);
    lua_pushstring(L, "Pear");
    lua_rawseti(L, -2, i++);
    lua_pushstring(L, "Banana");
    lua_rawseti(L, -2, i++);
    lua_setfield(L, -2, "foods");
...

Again, this isn’t a real situation but it suffices for demonstration purposes. Typically you’ll iterate over something that returns multiple values and add them to the table. For values that may not be used in every template providing a function that creates the table and returns it is a better option. That will copy the values into Lua only when necessary instead of always.

Embedding

Right now the template.lua is read from disk and must be located in the same directory as the C binary. This works but another approach (especially for the security conscious) is to compile the Lua code and embed it directly into the C binary. See this for details about how to do so.

Building

CMakeLists.txt

cmake_minimum_required (VERSION 3.0)
project (lua_templates C)

find_package(Lua REQUIRED)

include_directories (
    ${CMAKE_CURRENT_BINARY_DIR}
    ${CMAKE_CURRENT_SOURCE_DIR}
    ${LUA_INCLUDE_DIR}
)

set (SOURCES
    main.c
)

add_executable (${PROJECT_NAME} ${SOURCES} ${LUA_LIBRARIES})
target_link_libraries (${PROJECT_NAME} lua)

Building:

$ mkdir build
$ cd build
$ cmake ..
$ cp lua_templates ../
$ cd ..

Run

Both the Lua and C applications will produce the same output. Which they should because they setup the same environment and pass the same template to the same template engine.

$ ./lua_templates template.txt
or
$ lua main.lua template.txt
Lua template file example:

John

Foods:
* Apple
* Pear
* Banana

Here is an { and {{ which are escaped. Also
}} won't hurt because there is not opening {
Don't forget, {= unsupported blocks are kept }}


7 + 19 = 26
7 + 19 = adder(26)

look we can still do lua in the template:
I'm a string
table index 1 of a table
table element z of a table

The count is 4 and 5

All done

Introduction#

Template Engine#

Security and Arbitrary (User Provided) Templates#

Sample Template#

Using the Engine with Lua#

Using Engine From C#

C Functionality Explained#

The Environment#

Variables#

Functions#

Tables with Values#

Embedding#

Building#

Run#