Querying a CSW Catalog with OWSLib in Python

TL;DR: Instantiate owslib.csw.CatalogueServiceWeb against the catalogue endpoint, then call getrecords2() with a list of constraints — a PropertyIsLike on csw:AnyText combined with a BBox spatial filter — plus esn='full', maxrecords, and startposition. Read paging state from csw.results['nextrecord'], iterate the csw.records dict to pull title, subjects, and bbox, then call getrecordbyid() with an ISO outputschema to retrieve the full ISO 19139 document.

An OGC Catalogue Service for the Web (CSW) exposes a search interface over spatial metadata records. Talking to it by hand means assembling an OGC Filter Encoding XML document, POSTing it, and parsing a namespaced response. OWSLib collapses all of that into a handful of Python calls, but the abstraction leaks in two places that trip up most integrations: paging is manual and stateful, and the queryable property names change with both the CSW version and the record profile the server advertises.

The Core Challenge

CSW is a search protocol, not a bulk-download protocol. GetRecords — the operation that runs a query — returns a page of matching records plus a cursor, never the whole result set. You must combine three concerns in every call and reconcile them across two incompatible protocol generations:

• Filter composition. A useful query is rarely a single predicate. You typically want a free-text match and a spatial constraint at once, which means wrapping a text filter and a BBox filter in a logical And. OWSLib models these as owslib.fes objects (Filter Encoding Specification) that it serialises to the OGC Filter XML the server expects.
• Paging. maxrecords bounds the page size; startposition selects the page. The server reports how many records matched, how many it returned, and the 1-based index of the next record in csw.results. You loop until that nextrecord value is 0.
• Version and profile drift. CSW 2.0.2 uses Filter Encoding 1.1 and the Dublin Core csw:AnyText queryable by default; CSW 3.0.0 uses Filter Encoding 2.0 and, under the ISO Application Profile, apiso:AnyText. Sending the wrong property name against a given endpoint yields an OWS ExceptionReport rather than an empty result.

The diagram below traces the full request cycle, from connection through the paging loop to the per-record ISO fetch.

Production-Ready Code

The script below connects to a catalogue, runs a combined text-and-spatial query, walks every page of the result set, and then retrieves the complete ISO 19139 record for the first hit. The only third-party dependency is OWSLib (pip install OWSLib).

from owslib.csw import CatalogueServiceWeb
from owslib.fes import PropertyIsLike, BBox, And
from owslib.util import ServiceException
from requests.exceptions import RequestException

# ISO 19139 output schema — servers with the ISO Application Profile return
# a full gmd:MD_Metadata document when this outputschema is requested.
ISO_SCHEMA = "http://www.isotc211.org/2005/gmd"

CSW_ENDPOINT = "https://demo.pycsw.org/cite/csw"
PAGE_SIZE = 10


def connect(url: str, version: str = "2.0.2") -> CatalogueServiceWeb:
    """
    Open a CSW connection. `version` pins the protocol generation:
    '2.0.2' (Filter Encoding 1.1) or '3.0.0' (Filter Encoding 2.0).
    Instantiation issues GetCapabilities, so wrap it for transport errors.
    """
    try:
        return CatalogueServiceWeb(url, version=version, timeout=30)
    except (ServiceException, RequestException) as exc:
        raise SystemExit(f"Could not reach CSW endpoint {url}: {exc}")


def build_constraints(text: str, bbox: list[float]) -> list:
    """
    Combine a free-text predicate and a spatial predicate into a single
    logical AND. OWSLib serialises these owslib.fes objects to OGC Filter XML.

    - PropertyIsLike on csw:AnyText is the Dublin Core full-text queryable.
      For a CSW 3.0 ISO endpoint, use 'apiso:AnyText' instead.
    - BBox expects [minx, miny, maxx, maxy] in the given CRS (default 4326).
    """
    text_filter = PropertyIsLike("csw:AnyText", f"%{text}%")
    spatial_filter = BBox(bbox, crs="urn:ogc:def:crs:EPSG::4326")
    # A list wrapping a single And([...]) means "match every predicate".
    return [And([text_filter, spatial_filter])]


def search(csw: CatalogueServiceWeb, constraints: list) -> None:
    """
    Run getrecords2 and page through the entire matched set.

    csw.results is a dict reporting paging state:
      matches   – total records that satisfied the query
      returned  – records in this page
      nextrecord – 1-based index of the next record, or 0 when exhausted
    """
    start = 1  # CSW positions are 1-based, not 0-based.
    while True:
        csw.getrecords2(
            constraints=constraints,
            esn="full",              # element set name: brief | summary | full
            maxrecords=PAGE_SIZE,
            startposition=start,
        )
        results = csw.results
        print(
            f"page @ {start}: {results['returned']} of "
            f"{results['matches']} total match(es)"
        )

        # csw.records maps record id -> parsed record object.
        for record_id, record in csw.records.items():
            subjects = ", ".join(record.subjects) if record.subjects else "—"
            extent = record.bbox
            box = (
                f"[{extent.minx}, {extent.miny}, {extent.maxx}, {extent.maxy}]"
                if extent else "no bbox"
            )
            print(f"  {record_id}: {record.title} | subjects: {subjects} | {box}")

        nextrecord = results["nextrecord"]
        # A 0 (or a value past the total) signals the final page.
        if not nextrecord or nextrecord <= start:
            break
        start = nextrecord


def fetch_iso(csw: CatalogueServiceWeb, record_id: str) -> None:
    """Retrieve the complete ISO 19139 record for a single identifier."""
    csw.getrecordbyid(id=[record_id], outputschema=ISO_SCHEMA)
    record = csw.records.get(record_id)
    if record is None:
        print(f"No ISO record returned for {record_id}")
        return
    # Under the ISO schema, OWSLib parses into an MD_Metadata object exposing
    # identification, contact, and referenceSystem members.
    ident = record.identification[0] if record.identification else None
    title = ident.title if ident else "(untitled)"
    print(f"ISO 19139 record {record_id}: {title}")


def main() -> None:
    csw = connect(CSW_ENDPOINT, version="2.0.2")
    constraints = build_constraints(text="lake", bbox=[-10.0, 35.0, 30.0, 60.0])

    try:
        search(csw, constraints)
    except ServiceException as exc:
        # OWS ExceptionReport: invalid property name, unsupported filter, etc.
        raise SystemExit(f"CSW rejected the query: {exc}")

    if csw.records:
        first_id = next(iter(csw.records))
        fetch_iso(csw, first_id)


if __name__ == "__main__":
    main()

This procedure fits into the broader catalogue tooling described in the CSW Catalog Service Integration guide, and the same query pattern underpins the unattended pipeline in Scheduling Recurring CSW Harvests with Python.

Step-by-Step Walkthrough

Connecting. connect() instantiates CatalogueServiceWeb, which issues a GetCapabilities request during construction. That single round trip populates csw.identification, csw.operations, and the per-operation queryables — so a failure here means the endpoint is unreachable or not a CSW at all, which is why the constructor is wrapped for ServiceException and RequestException. The version argument pins the protocol generation; leave it at '2.0.2' unless the server’s capabilities advertise 3.0.0.

Composing the filter. build_constraints() creates two owslib.fes predicates. PropertyIsLike("csw:AnyText", "%lake%") performs a wildcard full-text match against every indexed field — csw:AnyText is the Dublin Core catch-all queryable, and the % characters are the OGC Filter wildcard, not SQL. BBox([minx, miny, maxx, maxy]) adds a spatial constraint; passing an explicit crs avoids the axis-order ambiguity discussed in the coordinate-handling material. Wrapping both in And([...]) and returning that inside a one-element list tells getrecords2 to require every predicate. To OR predicates instead, pass a list of several constraint objects at the top level.

Running the query and paging. search() calls getrecords2() with the constraints, esn="full" to request the full element set (rather than the sparser brief or summary), maxrecords=PAGE_SIZE, and startposition=start. After each call, csw.results carries the paging cursor: matches is the total hit count, returned is the size of this page, and nextrecord is the 1-based index where the next page begins. The loop advances start to nextrecord and terminates when the server reports 0 — the signal that the result set is exhausted. Guarding with nextrecord <= start prevents an infinite loop against servers that report a stale cursor.

Iterating records. After every getrecords2 call, csw.records is a dict mapping each record identifier to a parsed record object. Reading record.title, record.subjects (a list of keyword strings), and record.bbox (an object exposing minx, miny, maxx, maxy) is enough to build a search-results listing or feed a downstream index. Because csw.records is replaced on each call, extract what you need inside the loop rather than accumulating references across pages.

Fetching the full ISO record. The Dublin Core view returned by getrecords2 is deliberately shallow. fetch_iso() calls getrecordbyid() with outputschema=ISO_SCHEMA, prompting an ISO-profile server to return a complete gmd:MD_Metadata document. OWSLib then parses it into an object exposing identification, contact, and referenceSystem members — the structured detail defined by Implementing ISO 19115 Metadata Standards.

Verification

Run the script against a public pycsw demo endpoint:

pip install OWSLib
python query_csw.py

The output confirms that filtering, paging, and the ISO fetch all fired:

page @ 1: 10 of 23 total match(es)
  urn:uuid:19887a8a-... : Lakes of Scandinavia | subjects: hydrography, lake | [4.0, 55.0, 30.0, 60.0]
  urn:uuid:2b3c...      : Alpine Lake Survey    | subjects: lake, bathymetry  | [6.0, 45.0, 11.0, 48.0]
  ...
page @ 11: 10 of 23 total match(es)
  ...
page @ 21: 3 of 23 total match(es)
  ...
ISO 19139 record urn:uuid:19887a8a-...: Lakes of Scandinavia

Confirm that total match(es) stays constant across pages, that the reported page origins step by maxrecords, and that the final page returns fewer than PAGE_SIZE records before the loop exits. The trailing ISO line proves getrecordbyid resolved the full record.

Gotchas & Edge Cases

Back to CSW Catalog Service Integration

Related

• CSW Catalog Service Integration — the GetRecords, GetRecordById, and Filter Encoding fundamentals behind this query
• Scheduling Recurring CSW Harvests with Python — turning a one-off query into an unattended, incremental harvest
• Implementing ISO 19115 Metadata Standards — the structure of the full ISO 19139 record returned by getrecordbyid