Generating MapServer Mapfiles from Templates with Jinja2

TL;DR: Describe your layers as a list of Python dicts, render a Jinja2 template into the MAP/WEB/LAYER/CLASS/STYLE blocks of a .map file, and write the result to disk. Then validate it in-process with mappyfile, which parses the mapfile against a formal grammar and schema-checks it before it ever reaches a running server. Route every DATA and CONNECTION value through a quoting filter, keep credentials in a PostgreSQL service file rather than the template, and finish with a mapserv syntax check so MapServer’s own parser has the final word.

The Core Challenge

A MapServer mapfile is a strict, nested, keyword-driven configuration format: MAP contains WEB and one or more LAYER blocks, each LAYER contains CLASS blocks, and each CLASS contains STYLE blocks — every one closed by its own END. Hand-editing this for a choropleth with a dozen layers and five class breaks each is tedious and error-prone, and the errors are unforgiving: a single unbalanced END or a stray unescaped quote inside a DATA string produces an opaque msLoadMap() failure at request time, long after the change was deployed.

Templating solves the repetition but introduces two hazards of its own. First, string escaping: DATA and CONNECTION values routinely contain SQL fragments, spaces, and quotation marks that must be quoted the way MapServer’s parser expects, or the token boundary shifts and the whole block is misread. Second, verification: a template that renders without a Python exception can still emit a structurally invalid mapfile. Both hazards are addressed the same way — a disciplined quoting filter on the way in, and a grammar-and-schema validation pass on the way out. This page treats MapServer configuration as versioned, testable code, the theme running through the wider MapServer Configuration as Code guide.

Production-Ready Code

The script below renders a choropleth mapfile from a single context dictionary and validates it two ways. The only third-party dependencies are Jinja2 and mappyfile, both pip-installable; the mapserv binary is used later, in the verification step, and is optional at render time.

"""
render_mapfile.py — generate and validate a MapServer .map file from Jinja2.

Dependencies (pip install):
    Jinja2>=3.1, mappyfile>=1.0
"""
from __future__ import annotations

from pathlib import Path
from typing import Any

import mappyfile
from jinja2 import DictLoader, Environment, StrictUndefined

# ---------------------------------------------------------------------------
# 1. The template. Inlined here for readability; in a real project load it
#    from disk with FileSystemLoader so it can be linted and diffed on its own.
# ---------------------------------------------------------------------------
MAPFILE_TEMPLATE = """\
MAP
  NAME {{ map.name | mftext }}
  STATUS ON
  SIZE {{ map.width }} {{ map.height }}
  EXTENT {{ map.extent | join(" ") }}
  UNITS {{ map.units }}
  IMAGETYPE {{ map.imagetype }}
  PROJECTION
    "init=epsg:{{ map.epsg }}"
  END

  WEB
    METADATA
      "wms_title"          {{ map.title | mftext }}
      "wms_onlineresource" {{ map.online_resource | mftext }}
      "wms_srs"            {{ ("EPSG:%d" % map.epsg) | mftext }}
      "wms_enable_request" "*"
    END
  END
{% for layer in layers %}
  LAYER
    NAME {{ layer.name | mftext }}
    TYPE {{ layer.type }}
    STATUS ON
    CONNECTIONTYPE POSTGIS
    CONNECTION {{ layer.connection | mftext }}
    DATA {{ layer.data | mftext }}
    PROJECTION
      "init=epsg:{{ layer.epsg | default(map.epsg) }}"
    END
{% for cls in layer.classes %}
    CLASS
      NAME {{ cls.label | mftext }}
{% if cls.min is not none and cls.max is not none %}
      EXPRESSION ( [{{ layer.field }}] >= {{ cls.min }} AND [{{ layer.field }}] < {{ cls.max }} )
{% elif cls.max is not none %}
      EXPRESSION ( [{{ layer.field }}] < {{ cls.max }} )
{% else %}
      EXPRESSION ( [{{ layer.field }}] >= {{ cls.min }} )
{% endif %}
      STYLE
        COLOR {{ cls.colour | mftext }}
        OUTLINECOLOR "#3a3a3a"
        WIDTH 0.4
      END
    END
{% endfor %}
  END
{% endfor %}
END
"""


def mftext(value: Any) -> str:
    """
    Quote and escape a value as a MapServer mapfile string token.

    MapServer strings are double-quoted; an embedded backslash or double
    quote must be backslash-escaped or the parser mis-reads the token
    boundary — the classic cause of silent DATA/CONNECTION corruption.
    """
    text = str(value).replace("\\", "\\\\").replace('"', '\\"')
    return f'"{text}"'


def render_mapfile(context: dict[str, Any]) -> str:
    """Render the mapfile template with a strict, non-HTML environment."""
    env = Environment(
        loader=DictLoader({"mapfile.j2": MAPFILE_TEMPLATE}),
        autoescape=False,           # a mapfile is NOT HTML — never HTML-escape
        trim_blocks=True,           # drop the newline after a block tag
        lstrip_blocks=True,         # strip leading whitespace before a block tag
        undefined=StrictUndefined,  # fail loudly on a missing template variable
    )
    env.filters["mftext"] = mftext
    return env.get_template("mapfile.j2").render(**context)


def validate_mapfile(text: str) -> dict[str, Any]:
    """Grammar-check with mappyfile.loads, then schema-check with validate."""
    parsed = mappyfile.loads(text)       # raises on any grammar error
    errors = mappyfile.validate(parsed)  # returns [] when schema-valid
    if errors:
        raise ValueError(f"mapfile failed schema validation: {errors}")
    return parsed


if __name__ == "__main__":
    context: dict[str, Any] = {
        "map": {
            "name": "census_choropleth",
            "title": "Population Density (2021 Census)",
            "online_resource": "https://maps.example.org/cgi-bin/mapserv",
            "width": 800,
            "height": 600,
            "extent": [-8.65, 49.86, 1.77, 60.86],
            "units": "DD",
            "imagetype": "PNG",
            "epsg": 4326,
        },
        "layers": [
            {
                "name": "population_density",
                "type": "POLYGON",
                # No password here: 'service=gisdb' makes PostgreSQL resolve
                # host, db, user and password from pg_service.conf + .pgpass,
                # so the rendered mapfile carries no secret.
                "connection": "service=gisdb",
                "data": (
                    "geom FROM (SELECT gid, geom, pop_density "
                    "FROM census.blocks) AS q USING UNIQUE gid USING SRID=4326"
                ),
                "field": "pop_density",
                "classes": [
                    {"label": "< 100",    "min": None, "max": 100,  "colour": "#fee5d9"},
                    {"label": "100-500",  "min": 100,  "max": 500,  "colour": "#fcae91"},
                    {"label": "500-2000", "min": 500,  "max": 2000, "colour": "#fb6a4a"},
                    {"label": ">= 2000",  "min": 2000, "max": None, "colour": "#cb181d"},
                ],
            },
        ],
    }

    rendered = render_mapfile(context)
    parsed = validate_mapfile(rendered)

    out_path = Path("build/census_choropleth.map")
    out_path.parent.mkdir(parents=True, exist_ok=True)
    # Write mappyfile's normalised round-trip, not the raw render, so the file
    # on disk is guaranteed to be exactly what the grammar accepts.
    out_path.write_text(mappyfile.dumps(parsed), encoding="utf-8")
    print(f"Wrote and validated {out_path} ({len(parsed['layers'])} layer(s))")

Step-by-Step Walkthrough

Modelling layers as data. The context dictionary is the single source of truth: a global map block plus a layers list where each entry carries a name, geometry type, a connection, a data query, the field to classify on, and a classes list of colour breaks. Because this is plain data, it can come from a YAML file, a database query, or a REST call — the same rendering path serves a hand-written fixture and a fully dynamic catalogue. Building layer configuration from live sources is exactly the workflow explored in Using gsconfig-py3 for MapServer Layer Configuration.

The template and its loops. MAPFILE_TEMPLATE emits the fixed MAP, PROJECTION, and WEB blocks once, then loops over layers to produce a LAYER block each, and a nested loop over classes to produce the CLASS/STYLE pairs. Each class break becomes an EXPRESSION comparing the layer’s field against the break bounds; the {% if %} ladder handles the open-ended first class (max only) and last class (min only) so the lowest and highest buckets are not accidentally excluded.

Quoting with mftext. Every value that MapServer treats as a quoted string — NAME, wms_title, CONNECTION, DATA, and the class labels — passes through the mftext filter. It wraps the value in double quotes and backslash-escapes any embedded backslash or double quote. This is the load-bearing detail: a DATA subquery containing a quoted identifier, or a Windows-style path with backslashes, would otherwise terminate the token early and shift every subsequent keyword into the wrong block. Doing the escaping in one filter, rather than by hand in the template, means it cannot be forgotten on one field out of twenty.

The Environment settings. autoescape=False is deliberate and important — Jinja2’s HTML autoescaping would turn a < or & inside a SQL DATA clause into </&, silently breaking the query. trim_blocks and lstrip_blocks keep the rendered mapfile cleanly indented instead of riddled with blank lines from the loop tags. StrictUndefined converts a typo like layer.naem into an immediate UndefinedError at render time rather than an empty NAME in the output.

Keeping secrets out. The connection value is service=gisdb, not a full DSN with a password. PostgreSQL resolves service=gisdb to a host, database, user, and password from pg_service.conf and .pgpass on the server. The rendered mapfile therefore contains no credentials, which is what makes it safe to commit to version control and to schema-validate in CI. This mirrors the environment-separation approach used across the MapServer Configuration as Code cluster.

Validating and writing. validate_mapfile first calls mappyfile.loads, which parses the text with MapServer’s grammar and raises on any structural fault, then mappyfile.validate, which schema-checks keyword names and value types and returns a list of problems. Only a clean parse reaches disk — and the file written is mappyfile.dumps(parsed), the normalised round-trip, so the on-disk mapfile is guaranteed to be something the grammar already accepted.

Verification

Run the script. A clean render, parse, and schema-check prints a one-line confirmation:

python render_mapfile.py

Wrote and validated build/census_choropleth.map (1 layer(s))

For a final, authoritative check, let MapServer’s own parser load the file. A mapserv invocation that requests GetCapabilities forces a full msLoadMap(); a healthy mapfile returns an HTTP content-type header, while a broken one prints a parse error and a line number:

mapserv -conf /etc/mapserver/mapserver.conf \
        QUERY_STRING="map=$(pwd)/build/census_choropleth.map&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities" \
  | head -n 1

Content-Type: application/vnd.ogc.wms_xml; charset=UTF-8

If you deliberately corrupt the mapfile — delete an END, for instance — the same command reports the fault instead:

msLoadMap(): Symbol definition error. Parsing error near (END):(line 41)

Gotchas & Edge Cases

Back to MapServer Configuration as Code

Related

• MapServer Configuration as Code — the version-controlled, testable approach to mapfile management this page builds on
• Using gsconfig-py3 for MapServer Layer Configuration — when to reach for mapscript instead of pure templating
• GeoServer vs MapServer: A Feature Matrix — how declarative mapfiles compare with GeoServer’s REST-driven catalogue