Introduction
A while ago I wrote a template engine in Lua. This engine was fairly basic and I wrote it for work as a proof of concept. It was meant as a way to see this was a viable solution. After moving forward with the project I found that the engine was usable but it needed some enhancements.
One big difference with the new engine(s) vs the old one is new line handling. The old one would keep all new lines. Since logic statements are easier to read on their own line, there would be lots of new lines that you might not want or expect. Instead these ones require a space (’ ‘) after a block close to keep the new line. Otherwise, any line ending with a block close will strip that lines new line character.
Engine 1
I ended up writing a more maintainable engine with a cleaner syntax. As well as some additional features like comment blocks. The line handling was made more manageable for laying out the template to get expected output. For this block ends that are at the end of a line have the new line removed. Adding a space will keep the new line.
template_engine1.lua
--- Template Engine.
--
-- Takes a string with embedded Lua code block and renders
-- it based on the content of the blocks.
--
-- All template blocks start with '{ + start modifier' and
-- end with 'end modifier + }'.
--
-- Supports:
-- * {# text #} for comments.
-- * {% func %} for running Lua code.
-- * {{ var }} for printing variables.
--
-- Use \{ to use a literal { in the template if you need a literal {.
-- Also a { without an end modifier following will be treated as literal {.
--
-- Template block ends that end a line (whether they are part of a valid
-- block or not) will not create a new line. Use a space ' ' at the end
-- of the line if you want a new line preserved. The space will be removed.
-- So use two if you want the newline and the space preserved.
--
-- Multi-line strings in Lua blocks are supported but
-- [[ is not allowed. Use [=[ or some other variation.
--
-- Both compile and compile_file can take an optional
-- env table which when provided will be used as the
-- env for the Lua code in the template. This allows
-- a level of sandboxing. Note that any globals including
-- libraries that the template needs to access must be
-- provided by env if used.
local M = {}
-- Note: Modifiers and end modifiers must be symbols.
--- Map of start block modifiers to their end block modifier.
local END_MODIFIER = {
["#"] = "#",
["%"] = "%",
["{"] = "}",
}
--- Actions that should be taken when a block is encountered.
local MODIFIER_FUNC = {
["#"] = function(code)
return ""
end,
["%"] = function(code)
return code
end,
["{"] = function(code)
return ("_ret[#_ret+1] = %s"):format(code)
end,
}
--- Handle newline rules for blocks that end a line.
-- Blocks ending with a space keep their newline and blocks that
-- do not lose their newline.
local function handle_block_ends(text)
local modifier_set = ""
-- Build up the set of end modifiers.
-- Prefix each modifier with % to ensure they are escaped properly for gsub.
-- Block ends are and must always be symbols.
for _,v in pairs(END_MODIFIER) do
modifier_set = modifier_set.."%"..v
end
text = text:gsub("(["..modifier_set.."])} \n", "%1}\n\n")
text = text:gsub("(["..modifier_set.."])}\n", "%1}")
return text
end
--- Append text or code to the builder.
local function appender(builder, text, code)
if code then
builder[#builder+1] = code
elseif text then
-- [[ has a \n immediately after it. Lua will strip
-- the first \n so we add one knowing it will be
-- removed to ensure that if text starts with a \n
-- it won't be lost.
builder[#builder+1] = "_ret[#_ret+1] = [[\n" .. text .. "]]"
end
end
--- Takes a string and determines what kind of block it
-- is and takes the appropriate action.
--
-- The text should be something like:
-- "{{ ... }}"
--
-- If the block is supported the begin and end tags will
-- be stripped and the associated action will be taken.
-- If the tag isn't supported the block will be output
-- as is.
local function run_block(builder, text)
local func
local modifier
-- Text is {...
-- Pull out the character after { to determine if we
-- have a modifier and what action needs to be taken.
modifier = text:sub(2, 2)
func = MODIFIER_FUNC[modifier]
if func then
appender(builder, nil, func(text:sub(3, #text-3)))
else
appender(builder, text)
end
end
--- Compile a Lua template into a string.
--
-- @param tmpl The template.
-- @param[opt] env Environment table to use for sandboxing.
--
-- return Compiled template.
function M.compile(tmpl, env)
-- Turn the template into a string that can be run though
-- Lua. Builder will be used to efficiently build the string
-- we'll run. The string will use it's own builder (_ret). Each
-- part that comprises _ret will be the various pieces of the
-- template. Strings, variables that should be printed and
-- functions that should be run.
local builder = { "_ret = {}\n" }
local pos = 1
local b
local modifier
local ret
local func
local err
local out
if #tmpl == 0 then
return ""
end
-- Handle the new line rules for block ends.
tmpl = handle_block_ends(tmpl)
while pos < #tmpl do
-- Look for start of a block.
b = tmpl:find("{", pos)
if not b then
break
end
-- Check if this is a block or escaped { or not followed by block modifier.
-- We store the next character as the modifier to help us determine if
-- we have encountered a block or not.
modifier = tmpl:sub(b+1, b+1)
if tmpl:sub(b-1, b-1) == "\\" then
appender(builder, tmpl:sub(pos, b-2))
appender(builder, "{")
pos = b+1
elseif not END_MODIFIER[modifier] then
appender(builder, tmpl:sub(pos, b+1))
pos = b+2
else
-- Some modifiers for block ends aren't the same as the block start modifier.
modifier = END_MODIFIER[modifier]
-- Add all text up until this block.
appender(builder, tmpl:sub(pos, b-1))
-- Find the end of the block.
if modifier == "%" then
pos = tmpl:find("[^\\]?%%}", b)
else
pos = tmpl:find(("[^\\]?%s}"):format(modifier), b)
end
if pos then
-- If we captured a character before the modifier move past it.
if tmpl:sub(pos, pos) ~= modifier then
pos = pos+1
end
run_block(builder, tmpl:sub(b, pos+2))
-- Skip past the *} (pos points to the start of *}).
pos = pos+2
else
-- Add back the { because we don't have an end block.
-- We want to keep any text that isn't in a real block.
appender(builder, "{")
pos = b+1
end
end
end
-- Add any text after the last block. Or all of it if there
-- are no blocks.
if pos then
appender(builder, tmpl:sub(pos, #tmpl))
end
-- Create the compiled template.
builder[#builder+1] = "return table.concat(_ret)"
--print(table.concat(builder, "\n"))
-- Run the Lua code we built though Lua and get the result.
ret, func, err = pcall(load, table.concat(builder, "\n"), "template", "t", env)
if not ret then
return nil, func
end
if not func then
return nil, err
end
ret, out, err = pcall(func)
if not ret then
return nil, out
end
if not out then
return nil, err
end
return out
end
--- Compile a Lua template into a string by
-- reading the template from a file.
--
-- @param name The file name to read from.
-- @param[opt] env Environment table to use for sandboxing.
--
-- return Compiled template.
function M.compile_file(name, env)
local f, err = io.open(name, "rb")
if not f then
return err
end
local t = f:read("*all")
f:close()
return M.compile(t, env)
end
return M
This engine while better has a major flaw. If you want to specify variables for
the template to use you have to specify a full env
. If you don’t need sand
boxing then you’re out of luck. You must use sand boxing if you want to specify
variables for the template to use.
Engine 2
Let’s correct the shot comings of engine 1 by adding a flag that determines if
the env
should be append to _ENV
. This allows the same functionality as not
passing env
(or passing as nil
) while still allowing variables to be specified.
template_engine2.lua
--- Template Engine.
--
-- Takes a string with embedded Lua code block and renders
-- it based on the content of the blocks.
--
-- All template blocks start with '{ + start modifier' and
-- end with 'end modifier + }'.
--
-- Supports:
-- * {# text #} for comments.
-- * {% func %} for running Lua code.
-- * {{ var }} for printing variables.
--
-- Use \{ to use a literal { in the template if you need a literal {.
-- Also a { without an end modifier following will be treated as literal {.
--
-- Template block ends that end a line (whether they are part of a valid
-- block or not) will not create a new line. Use a space ' ' at the end
-- of the line if you want a new line preserved. The space will be removed.
-- So use two if you want the newline and the space preserved.
--
-- Multi-line strings in Lua blocks are supported but
-- [[ is not allowed. Use [=[ or some other variation.
--
-- Both compile and compile_file can take an optional
-- env table which when provided will be used as the
-- env for the Lua code in the template. This allows
-- a level of sandboxing. Note that any globals including
-- libraries that the template needs to access must be
-- provided by env if used.
local M = {}
-- Note: Modifiers and end modifiers must be symbols.
--- Map of start block modifiers to their end block modifier.
local END_MODIFIER = {
["#"] = "#",
["%"] = "%",
["{"] = "}",
}
--- Actions that should be taken when a block is encountered.
local MODIFIER_FUNC = {
["#"] = function(code)
return ""
end,
["%"] = function(code)
return code
end,
["{"] = function(code)
return ("_ret[#_ret+1] = %s"):format(code)
end,
}
--- Handle newline rules for blocks that end a line.
-- Blocks ending with a space keep their newline and blocks that
-- do not lose their newline.
local function handle_block_ends(text)
local modifier_set = ""
-- Build up the set of end modifiers.
-- Prefix each modifier with % to ensure they are escaped properly for gsub.
-- Block ends are and must always be symbols.
for _,v in pairs(END_MODIFIER) do
modifier_set = modifier_set.."%"..v
end
text = text:gsub("(["..modifier_set.."])} \n", "%1}\n\n")
text = text:gsub("(["..modifier_set.."])}\n", "%1}")
return text
end
--- Append text or code to the builder.
local function appender(builder, text, code)
if code then
builder[#builder+1] = code
elseif text then
-- [[ has a \n immediately after it. Lua will strip
-- the first \n so we add one knowing it will be
-- removed to ensure that if text starts with a \n
-- it won't be lost.
builder[#builder+1] = "_ret[#_ret+1] = [[\n" .. text .. "]]"
end
end
--- Takes a string and determines what kind of block it
-- is and takes the appropriate action.
--
-- The text should be something like:
-- "{{ ... }}"
--
-- If the block is supported the begin and end tags will
-- be stripped and the associated action will be taken.
-- If the tag isn't supported the block will be output
-- as is.
local function run_block(builder, text)
local func
local modifier
-- Text is {...
-- Pull out the character after { to determine if we
-- have a modifier and what action needs to be taken.
modifier = text:sub(2, 2)
func = MODIFIER_FUNC[modifier]
if func then
appender(builder, nil, func(text:sub(3, #text-3)))
else
appender(builder, text)
end
end
--- Compile a Lua template into a string.
--
-- @param tmpl The template.
-- @param[opt] env Environment table to use for sandboxing.
-- @param[opt] env_append Should the provided env be appended to the global env.
-- This allows adding variables that the template can use
-- while simulating passing nil as the env.
--
-- return Compiled template.
function M.compile(tmpl, env, env_append)
-- Turn the template into a string that can be run though
-- Lua. Builder will be used to efficiently build the string
-- we'll run. The string will use it's own builder (_ret). Each
-- part that comprises _ret will be the various pieces of the
-- template. Strings, variables that should be printed and
-- functions that should be run.
local builder = { "_ret = {}\n" }
local pos = 1
local b
local modifier
local ret
local func
local err
local out
local final_env
if #tmpl == 0 then
return ""
end
-- Handle the new line rules for block ends.
tmpl = handle_block_ends(tmpl)
while pos < #tmpl do
-- Look for start of a block.
b = tmpl:find("{", pos)
if not b then
break
end
-- Check if this is a block or escaped { or not followed by block modifier.
-- We store the next character as the modifier to help us determine if
-- we have encountered a block or not.
modifier = tmpl:sub(b+1, b+1)
if tmpl:sub(b-1, b-1) == "\\" then
appender(builder, tmpl:sub(pos, b-2))
appender(builder, "{")
pos = b+1
elseif not END_MODIFIER[modifier] then
appender(builder, tmpl:sub(pos, b+1))
pos = b+2
else
-- Some modifiers for block ends aren't the same as the block start modifier.
modifier = END_MODIFIER[modifier]
-- Add all text up until this block.
appender(builder, tmpl:sub(pos, b-1))
-- Find the end of the block.
if modifier == "%" then
pos = tmpl:find("[^\\]?%%}", b)
else
pos = tmpl:find(("[^\\]?%s}"):format(modifier), b)
end
if pos then
-- If we captured a character before the modifier move past it.
if tmpl:sub(pos, pos) ~= modifier then
pos = pos+1
end
run_block(builder, tmpl:sub(b, pos+2))
-- Skip past the *} (pos points to the start of *}).
pos = pos+2
else
-- Add back the { because we don't have an end block.
-- We want to keep any text that isn't in a real block.
appender(builder, "{")
pos = b+1
end
end
end
-- Add any text after the last block. Or all of it if there
-- are no blocks.
if pos then
appender(builder, tmpl:sub(pos, #tmpl))
end
-- Create the compiled template.
builder[#builder+1] = "return table.concat(_ret)"
if env then
final_env = {}
if env_append then
for k,v in pairs(_ENV) do final_env[k] = v end
end
for k,v in pairs(env) do final_env[k] = v end
end
-- Run the Lua code we built though Lua and get the result.
ret, func, err = pcall(load, table.concat(builder, "\n"), "template", "t", final_env)
if not ret then
return nil, func
end
if not func then
return nil, err
end
ret, out, err = pcall(func)
if not ret then
return nil, out
end
if not out then
return nil, err
end
return out
end
--- Compile a Lua template into a string by
-- reading the template from a file.
--
-- @param name The file name to read from.
-- @param[opt] env Environment table to use for sandboxing.
-- @param[opt] env_append Should the provided env be appended to the global env.
-- This allows adding variables that the template can use
-- while simulating passing nil as the env.
--
-- return Compiled template.
function M.compile_file(name, env, env_append)
local f, err = io.open(name, "rb")
if not f then
return err
end
local t = f:read("*all")
f:close()
return M.compile(t, env, env_append)
end
return M
Engine 3
Both of the above engines work very well. That said I ended up needing easy escaping. Especially when dealing with web content. This entailed quite a rework. I ended up using Lua 5.3 because of the very handy utf8 library.
template_engine3.lua
--- Template Engine.
--
-- Takes a string with embedded Lua code block and renders
-- it based on the content of the blocks.
--
-- All template blocks start with '{ + start modifier' and
-- end with 'end modifier + }'.
--
-- Supports:
-- * {# text #} for comments.
-- * {= mode =} for setting a global escaping mode.
-- * {% expression %} for running Lua code.
-- * {{ var }} for printing.
--
-- Use \{ to use a literal { in the template if you need a literal {.
-- Also a { without an end modifier following will be treated as literal {.
--
-- The following escaping modes are supported:
-- * raw - No escaping is performed.
-- * attribute - Attribute escaping is performed.
-- * html - html/xml escaping is performed.
-- * js - JavaScript escaping is performed.
-- * css - CSS escaping is performed.
-- * url - URL escaping is performed.
-- * url_plus - URL escaping with spaces as + is performed.
-- Escaping rules (seem overboard) came from OWASP's
-- "Output Encoding Rules Summary" for XSS prevention at
-- https://www.owasp.org/index.php/XSS_%28Cross_Site_Scripting%29_Prevention_Cheat_Sheet#Output_Encoding_Rules_Summary
--
-- {= mode =} and {{ var|e("mode") }} can be used to specify an escape mode.
-- {= =} is global and |e is specific to the variable block. |e will override
-- any global mode that is set. |e must come at the end of a {{ }} block.
--
-- The global mode will be changed each time {= mode =}.
--
-- If the global mode given is invalid the global mode will not be changed.
-- If |e has an invalid mode set then the global mode will be used.
--
-- Template block ends that end a line (whether they are part of a valid
-- block or not) will not create a new line. Use a space ' ' at the end
-- of the line if you want a new line preserved. The space will be removed.
-- So use two if you want the newline and the space preserved.
--
-- Multi-line strings in Lua blocks are supported but
-- [[ is not allowed. Use [=[ or some other variation.
--
-- Internal variables and functions that are run in the same environment
-- as the template are prefixed with an underscore '_'. Any functions
-- functions or variables created in the template should not use this prefix.
--
-- The template will be run in a sand box with a set of safe globals configured.
-- The sand box can be extended by proving an env with additional globals that
-- should be available. Globals can be added but not removed. Additionally the
-- env should contain any variables that the template will need to access such
-- as setting a username for display in the template.
local M = {}
local function trim(text)
local from = text:match("^%s*()")
return from > #text and "" or text:match(".*%S", from)
end
-- The following string based code is going to be embedded into the
-- builder so it's accessible and can be used within the sand boxed
-- environment.
local url_encode = [[
local function _url_encode(text, quote_plus)
local c
local builder = {}
if not text then
return ""
end
for i=1, #text do
c = text:sub(i, i)
if c == " " and quote_plus then
builder[#builder+1] = "+"
elseif c:find("%w") then
builder[#builder+1] = c
else
for j=1, #c do
builder[#builder+1] = ("%%%02x"):format(string.byte(c:sub(j, j)))
end
end
end
return table.concat(builder, "")
end
]]
local html_escape_table = [[
local _html_escape_table = {
["&"] = "&",
["<"] = "<",
[">"] = ">",
['"'] = ""e;",
["'"] = "'",
["/"] = "/",
["\\"] = "\",
}
]]
local ESCAPE_FUNC = [[
local _ESCAPE_FUNC = {
raw = function(text)
return text
end,
html = function(text)
return (text:gsub("[&<>\"'/\\]", _html_escape_table))
end,
attribute = function(text)
local c
local builder = {}
if not text then
return ""
end
for i=1, #text do
c = text:sub(i, i)
if c:find("%w") then
builder[#builder+1] = c
else
builder[#builder+1] = ("&#x%02X;"):format(utf8.codepoint(c))
end
end
return table.concat(builder, "")
end,
js = function(text)
local c
local builder = {}
if not text then
return ""
end
for i=1, #text do
c = text:sub(i, i)
if c:find("%w") then
builder[#builder+1] = c
else
builder[#builder+1] = ("\\u%04d"):format(utf8.codepoint(c))
end
end
return table.concat(builder, "")
end,
css = function(text)
local c
local builder = {}
if not text then
return ""
end
for i=1, #text do
c = text:sub(i, i)
if c:find("%w") then
builder[#builder+1] = c
else
builder[#builder+1] = ("\\%06d"):format(utf8.codepoint(c))
end
end
return table.concat(builder, "")
end,
url = function(text)
return _url_encode(text)
end,
url_plus = function(text)
return _url_encode(text, true)
end,
}
]]
local do_escape = [[
local function _do_escape(text, escape)
local escape_func
if escape then
escape_func = _ESCAPE_FUNC[escape]
end
if not escape_func then
escape_func = _ESCAPE_FUNC[_global_escape]
end
if escape_func then
return escape_func(text)
end
return text
end
]]
-- Note: Modifiers and end modifiers must be symbols.
--- Map of start block modifiers to their end block modifier.
local END_MODIFIER = {
["#"] = "#",
["="] = "=",
["%"] = "%",
["{"] = "}",
}
--- Actions that should be taken when a block is encountered.
local MODIFIER_FUNC = {
["="] = function(code)
return ("_global_escape = \"%s\""):format(trim(code))
end,
["#"] = function(code)
return ""
end,
["%"] = function(code)
return code
end,
["{"] = function(code)
local e
local escape = "nil"
-- Check for and pull out an escape modifier
e = code:match("%|e%([\"'].-[\"']%) *$")
if e then
escape = trim(e:match("[\"'](.-)[\"']"))
code = code:sub(1, #code-#e)
end
return ("_ret[#_ret+1] = _do_escape(%s, \"%s\")"):format(code, escape)
end,
}
--- Handle newline rules for blocks that end a line.
-- Blocks ending with a space keep their newline and blocks that
-- do not lose their newline.
local function handle_block_ends(text)
local modifier_set = ""
-- Build up the set of end modifiers.
-- Prefix each modifier with % to ensure they are escaped properly for gsub.
-- Block ends are and must always be symbols.
for _,v in pairs(END_MODIFIER) do
modifier_set = modifier_set.."%"..v
end
text = text:gsub("(["..modifier_set.."])} \n", "%1}\n\n")
text = text:gsub("(["..modifier_set.."])}\n", "%1}")
return text
end
--- Append text or code to the builder.
local function appender(builder, text, code)
if code then
builder[#builder+1] = code
elseif text then
-- [[ has a \n immediately after it. Lua will strip
-- the first \n so we add one knowing it will be
-- removed to ensure that if text starts with a \n
-- it won't be lost.
builder[#builder+1] = "_ret[#_ret+1] = [[\n".. text .."]]"
end
end
--- Takes a string and determines what kind of block it
-- is and takes the appropriate action.
--
-- The text should be something like:
-- "{{ ... }}"
--
-- If the block is supported the begin and end tags will
-- be stripped and the associated action will be taken.
-- If the tag isn't supported the block will be output
-- as is.
local function run_block(builder, text)
local func
local modifier
-- Text is {...
-- Pull out the character after { to determine if we
-- have a modifier and what action needs to be taken.
modifier = text:sub(2, 2)
func = MODIFIER_FUNC[modifier]
if func then
appender(builder, nil, func(text:sub(3, #text-3)))
else
appender(builder, text)
end
end
--- Compile a Lua template into a string.
--
-- @param tmpl The template.
-- @param[opt] env Environment table to use for sandboxing.
--
-- return Compiled template.
function M.compile(tmpl, env)
-- Turn the template into a string that can be run though
-- Lua. Builder will be used to efficiently build the string
-- we'll run. The string will use it's own builder (_ret). Each
-- part that comprises _ret will be the various pieces of the
-- template. Strings, variables that should be printed and
-- functions that should be run.
local builder = { "_ret = {}\n" }
local pos = 1
local b
local modifier
local func
local err
local ret
local out
if #tmpl == 0 then
return ""
end
env = env or {}
-- Add some globals to the env that restricts what the template can do.
-- Some of the globals are required for the internal (escape functions) to operate.
env["ipairs"] = ipairs
env["next"] = next
env["pairs"] = pairs
env["pcall"] = pcall
env["tonumber"] = tonumber
env["tostring"] = tostring
env["type"] = type
env["utf8"] = utf8
env["math"] = math
env["string"] = string
env["table"] = {
concat = table.concat,
insert = table.insert,
move = table.move,
remove = table.remove,
sort = table.sort,
}
env["os"] = {
clock = os.clock,
date = os.date,
difftime = os.difftime,
time = os.time,
}
-- Add the escaping functions.
builder[#builder+1] = "_global_escape = \"raw\""
builder[#builder+1] = url_encode
builder[#builder+1] = html_escape_table
builder[#builder+1] = ESCAPE_FUNC
builder[#builder+1] = do_escape
-- Handle the new line rules for block ends.
tmpl = handle_block_ends(tmpl)
while pos < #tmpl do
-- Look for start of a block.
b = tmpl:find("{", pos)
if not b then
break
end
-- Check if this is a block or escaped { or not followed by block modifier.
-- We store the next character as the modifier to help us determine if
-- we have encountered a block or not.
modifier = tmpl:sub(b+1, b+1)
if tmpl:sub(b-1, b-1) == "\\" then
appender(builder, tmpl:sub(pos, b-2))
appender(builder, "{")
pos = b+1
elseif not END_MODIFIER[modifier] then
appender(builder, tmpl:sub(pos, b+1))
pos = b+2
else
-- Some modifiers for block ends aren't the same as the block start modifier.
modifier = END_MODIFIER[modifier]
-- Add all text up until this block.
appender(builder, tmpl:sub(pos, b-1))
-- Find the end of the block.
if modifier == "%" then
pos = tmpl:find("[^\\]?%%}", b)
else
pos = tmpl:find(("[^\\]?%s}"):format(modifier), b)
end
if pos then
-- If we captured a character before the modifier move past it.
if tmpl:sub(pos, pos) ~= modifier then
pos = pos+1
end
run_block(builder, tmpl:sub(b, pos+2))
-- Skip past the *} (pos points to the start of *}).
pos = pos+2
else
-- Add back the { because we don't have an end block.
-- We want to keep any text that isn't in a real block.
appender(builder, "{")
pos = b+1
end
end
end
-- Add any text after the last block. Or all of it if there
-- are no blocks.
if pos then
appender(builder, tmpl:sub(pos, #tmpl))
end
-- Create the compiled template.
builder[#builder+1] = "return table.concat(_ret)"
-- Run the Lua code we built though Lua and get the result.
ret, func, err = pcall(load, table.concat(builder, "\n"), "template", "t", env)
if not ret then
return nil, func
end
if not func then
return nil, err
end
ret, out, err = pcall(func)
if not ret then
return nil, out
end
if not out then
return nil, err
end
return out
end
--- Compile a Lua template into a string by
-- reading the template from a file.
--
-- @param name The file name to read from.
-- @param[opt] env Environment table to use for sandboxing.
--
-- return Compiled template.
function M.compile_file(name, env)
local f, err = io.open(name, "rb")
if not f then
return err
end
local t = f:read("*all")
f:close()
return M.compile(t, env)
end
return M
This engine is very similar to engine 2 but with always appending to the env
.
That said this does not use _ENV
. It uses a sand box but explicitly sets it.
os
for example is limited quite a bit. If necessary it can be enabled by
passing it in. Variables as well as additional features (like require) can be
passed in via env
.
One thing to note about the features added to the environment is there is some information that math, string, and table by themselves are not secure. That said all the information about this is for Lua 5.1. I haven’t found an explanation as to why it’s insecure. Nor have I found an updated list for 5.2 or 5.3. If you’re really paranoid you can change it to something else.
There are four (five but one’s a variation) escaping modes supported as well as a ‘raw’ or no mode. The modes are: html (also works for xml), js (Javascript), css, url, url_plus (like url but uses + instead of %20 for spaces). The escaping seems overly paranoid but that’s what OWASP says. If the escaping is just too much it can be changed to something more manageable.
This engine handles escaping by putting the escaping functions into the compiled template string. Due to this the environment is pre-setup with all required functionality (as well as a few useful additions).
Escaping can be used in two ways. Setting a global escape using {= mode =} which can be changed each time it’s set or removed if set to raw. Additionally you can set escaping on a per variable ({{ … }}) basis using |e(“mode”). If the |e syntax looks familiar it’s because it’s inspired by Twig. |e will override any global mode.
Example
Since the templates work differently with new lines after block close, we need an updated example file. This one has a space (’ ‘) after each block close that ends a line.
{{ name }}
Foods:{% for _,v in ipairs(foods) do %}
* {{ v }}{% end %}
Here is an \{ and \{\{ which are escaped. Also
}} won't hurt because there is not opening \{
Don't forget, {= unsupported blocks are kept }}
{{ var_a }} + {{ var_b }} = {{ var_a + var_b }}
{{ var_a }} + {{ var_b }} = adder({{ adder(var_a, var_b) }})
look we can still do lua in the template:
{% l = "I'm a string" %}{{ l }}
{% l = { "index 1 of a table" } %}{{ type(l) }} {{ l[1] }}
{% l = { z="element z of a table" } %}{{ type(l) }} {{ l["z"] }}
The count is {{ count.get_a() }} and {{ count.get_b() }}
All done
Using this and any of the new template engines with the previous example apps will produce the same output as the old engine.
Embedding
For the template engine I needed to embed it into a C application. I wrote about how to do this already. That said, my instructions have one major flaw. I said to use luac (good) then xxd with -i (bad) to create a C header file that can be included to have access to the compiled template engine.
The big issue is xxd isn’t available on all the system I need this to work on. There are a few Lua scripts out there that allow turning Lua (compiled) into a header. I don’t like the ones out there because I really just want the data compiled into an array (with a length). Most of the ones out there do a bit more like automatically calling luaL_dostring. So to fulfill my need I wrote my own Lua script that emulates xxd -i to the point of creating identical output.
bin2header.lua
--- Output binary data as a C header file.
-- This is a Lua implementation of xxd -i.
-- It is intended for use on systems that
-- do not support or have xxd natively.
local function main()
local in_filename
local out_filename
local fi
local fo
local data
local var_name
local builder = {}
local pos = 0
local block
local c
if not arg or #arg < 2 then
print("usage: "..arg[0].." in out var_name")
return 1
end
in_filename = arg[1]
fi = io.open(in_filename, "rb")
if not fi then
print("Failure: could not open "..in_filename.." for reading")
return 1
end
out_filename = arg[2]
fo = io.open(out_filename, "w")
if not fo then
print("Failure: could not open "..out_filename.." for writing")
fi:close()
return 1
end
data = fi:read("*a")
if not data then
print("Failure: file is empty")
fo:close()
fi:close()
return 1
end
fi:close()
var_name = #arg > 2 and arg[3] or "out"
builder[#builder+1] = "unsigned char "..var_name.."[] = {"
builder[#builder+1] = "\n"
pos = 1
while pos < #data do
block = data:sub(pos, pos+11)
for i=1,#block do
c = block:sub(i, i)
builder[#builder+1] = ("%s0x%02x%s%s"):format(i==1 and " " or "", string.byte(c), pos+i<=#data and "," or "", i<12 and pos+i<=#data and " " or "")
end
pos = pos+12
builder[#builder+1] = "\n"
end
builder[#builder+1] = "};"
builder[#builder+1] = "\n"
builder[#builder+1] = "unsigned int "..var_name.."_len = "..#data..";"
builder[#builder+1] = "\n"
fo:write(table.concat(builder, ""))
fo:close()
return 0
end
os.exit(main())
Conclusion
Overall I’m very happy with all three engines and I can see them being useful in various contexts. I’m also happy to have a cross platform solution for embedding. The best part is all of this is built with pure Lua!