Post

AI

PMTiles v3 Source Audit: Single-File Cloud Storage and Serverless Map Pipelines

A technical source audit of protomaps/PMTiles v3 (commit aec8fa13), analyzing the 127-byte binary header, Hilbert space-filling curve indexing, and zero-backend HTTP Range tile delivery.

Architecture diagram showing PMTiles v3 binary header layout, Hilbert curve indexing, and serverless HTTP Range request pipeline.
Architecture diagram showing PMTiles v3 binary header layout, Hilbert curve indexing, and serverless HTTP Range request pipeline.
Architecture diagram showing PMTiles v3 binary header layout, Hilbert curve indexing, and serverless HTTP Range request pipeline.
AI-generated illustration by jeo: schematic architecture diagram representing PMTiles v3 binary header, Hilbert space-filling curve, and serverless Range request pipeline; not source evidence.

How the Multi-Million Tile Directory Exploded Cloud Storage Economics

When deploying interactive vector maps, geographic information system (GIS) engineers have historically faced an acute infrastructural friction: tile pyramid storage. Standard planetary map sets spanning zoom levels 0 through 14 generate hundreds of millions of individual Mapbox Vector Tile (MVT) or raster files. Storing these tiles as a conventional directory hierarchy (/{z}/{x}/{y}.pbf) on cloud object storage such as AWS S3 or Google Cloud Storage results in an economic disaster.

The primary operational cost does not stem from gigabyte disk footprint, but rather from API transaction fees. Uploading 50 million tiles incurs tens of thousands of HTTP PUT requests, while synchronizing, versioning, or deleting directory trees requires hours of fragile network transfers prone to connection timeouts. While SQLite-based containers like MBTiles solved local file fragmentation, they introduced a competing operational barrier: SQLite relies on POSIX file locking and non-sequential disk I/O, rendering it incapable of serving web clients directly from cloud object storage without an intermediate stateful tile server container such as Martin or TileServer GL.

The release of the PMTiles Version 3 specification by Brandon Liu (Protomaps, commit aec8fa1341222fdddb3318e9ffa8e18e19b312f7, BSD-3-Clause) dismantled this architectural tradeoff. PMTiles encapsulates an entire pyramid archive into a single, immutable .pmtiles file. By organizing tile indices along a space-filling Hilbert curve and leveraging standardized HTTP Range requests, web browsers can query arbitrary bounding boxes directly from static cloud object storage with zero backend servers and sub-50ms latency.

%%{init: {"flowchart": {"useMaxWidth": false}}}%%
flowchart TD
    subgraph ClientLayer ["Client Browser (MapLibre / Leaflet)"]
        MapEngine["Map Engine Viewport"]
        PMTilesAdapter["@protomaps/pmtiles Protocol Adapter"]
        RangeCache["In-Memory Directory Cache (under 16 KB)"]
    end

    subgraph NetworkLayer ["Static Edge / CDN (RFC 9110)"]
        RangeRouter["HTTP Range Request: bytes=offset-length"]
        PartialResponse["206 Partial Content Stream"]
    end

    subgraph StorageLayer ["Cloud Object Storage (S3 / R2 / GCS)"]
        Header127["127-Byte Fixed Binary Header"]
        RootDir["Root Directory (Hilbert Index)"]
        LeafDir["Leaf Directories (On-Demand)"]
        TilePayloads["Compressed Tile Blobs (Gzip / Zstd)"]
    end

    MapEngine --> PMTilesAdapter
    PMTilesAdapter --> RangeCache
    PMTilesAdapter -->|1-2 Bounded Range RTTs| RangeRouter
    RangeRouter --> StorageLayer
    StorageLayer --> PartialResponse
    PartialResponse --> PMTilesAdapter

    style ClientLayer fill:#f8fafc,stroke:#38bdf8,stroke-width:2px,color:#111827
    style NetworkLayer fill:#f0fdf4,stroke:#22c55e,stroke-width:2px,color:#111827
    style StorageLayer fill:#faf5ff,stroke:#a855f7,stroke-width:2px,color:#111827
PMTILES v3 INTERNAL BINARY FILE SPECIFICATION HEADER 127 Bytes Fixed Magic 'PMTiles' Version 3 byte Directory offsets Tile count & stats ROOT DIRECTORY < 16 KB Compressed Cached on client Hilbert TileID map Direct run lengths Leaf pointers JSON METADATA Compressed UTF-8 Vector layers Bounds & zoom Attribution text MVT fields LEAF DIRECTORIES On-Demand Slices Hierarchical depth High zoom ranges Isolated fetch Max 1 extra RTT TILE DATA CHUNKS Contiguous Byte Storage De-duplicated runs (ocean/void) MVT, WebP, PNG, or Terrain RGB Compressed via Gzip or Zstd Direct byte range addressed

The 127-Byte Binary Header and Hilbert Space-Filling Index Layout

Inspection of the v3 specification and implementation reveals a strictly defined 127-byte fixed preamble. Unlike file formats that encode variable metadata at arbitrary initial offsets, PMTiles v3 structures its primary metadata fields in fixed byte positions:

  • Bytes 0–6: ASCII Magic identifier PMTiles (0x50 0x4D 0x54 0x69 0x6C 0x65 0x73).
  • Byte 7: Version integer (0x03).
  • Bytes 8–71: 8-byte unsigned integer pairs encoding the 64-bit offsets and byte lengths of the Root Directory, Metadata JSON, Leaf Directories, and Tile Data section.
  • Bytes 72–126: Center coordinates, minimum/maximum zoom levels, min/max lon/lat coordinates, tile type (MVT, PNG, WebP, AVIF), and compression algorithm enum flags.

The central technical triumph of PMTiles is its spatial indexing strategy. In conventional tile servers, locating a tile at coordinates (zoom, x, y) requires quadtree traversals or B-tree lookups. PMTiles translates 2D tile coordinates into a 1D continuous address space using the Hilbert space-filling curve.

The Hilbert curve guarantees that tiles that are geographically adjacent in 2D space remain adjacent in 1D disk order with high probability. This spatial clustering yields two massive real-world performance benefits:

  1. Sequential Prefetching: When a user pans across a geographic region, the required tiles reside in continuous byte runs, enabling clients or CDN edge caches to fetch multi-tile clusters in a single aggregated byte range request.
  2. Run-Length Deduplication: Vast expanses of planetary maps (such as oceans, empty terrain, or uniform forest canopies) contain identical tile byte payloads. PMTiles encodes an entry’s RunLength field in directory headers. A single 500-byte empty tile payload can satisfy thousands of coordinates without storing repeated bytes on disk.

Bounded HTTP Round-Trips via Two-Tier Directory Hierarchies

A critical latency pitfall of static cloud architectures is the HTTP Round-Trip Time (RTT) penalty. If retrieving a tile required walking a deep multi-level tree of remote index chunks, high network latency would make map exploration unusable on mobile connections.

PMTiles solves this by strictly capping the maximum directory traversal depth to a two-tier hierarchy:

  • Root Directory (< 16 KB): Every PMTiles archive packages its top-level directory entries into a compact chunk designed to fit within 16,384 bytes. Upon initial map initialization, the browser fetches the 127-byte header and the root directory in a single combined range request (Range: bytes=0-16383). For small-to-medium regional maps (e.g., metropolitan transit maps or state-level datasets), the root directory contains the complete index for all tiles, allowing all subsequent tile fetches to execute in exactly one HTTP RTT.
  • Leaf Directories (On-Demand): For full-planet datasets containing hundreds of millions of tiles, the root directory stores pointers to leaf directory chunks. When a user zooms into a dense urban block, the client performs exactly one Range request to load the targeted leaf directory, and one Range request to retrieve the tile payload.

Under this contract, worst-case retrieval latency is mathematically bounded to two HTTP RTTs, even when indexing a multi-terabyte dataset spanning the entire globe.

Architectural Comparison Across Map Infrastructure Models

To clarify the operational differences between legacy tile servers and cloud-native PMTiles pipelines, the table below contrasts three common architectural patterns:

Operational DimensionS3 Exploded Tiles (XYZ)Stateful Tile Server (MBTiles)Cloud-Native PMTiles v3
Storage TopologyMillions of individual .pbf objectsSingle SQLite .mbtiles containerSingle immutable .pmtiles archive
Backend ComputeStatic Web Server / S3Docker container (Martin / TileServer)Pure Object Storage (S3 / R2 / GCS)
Query ProtocolStandard HTTP GET per tileREST endpoint (/tiles/{z}/{x}/{y})HTTP Range Request (RFC 9110, 206)
Initial Upload FrictionHours to days (API rate limits)Single file transferSingle file transfer
Spatial LocalityRandom cloud object addressesSQLite B-tree page order1D Hilbert curve continuous clusters
DeduplicationNone (identical tiles duplicate)SQLite hash table lookupRun-length encoding across indices
Operational Failure RiskHigh storage request billServer crash / memory leak / DB lockZero server process failure modes

This comparison highlights how PMTiles aligns with modern zero-maintenance agent infrastructure. Similar to the in-memory dispatch patterns explored in our recent audit of FastMCP In-Memory Transports and the decoupled execution environments examined in Browser Use Cloud Boundaries, eliminating stateful middle-tier servers dramatically improves reliability while reducing hosting costs by over 90%.

Key Takeaways

  • True Serverless GIS: PMTiles enables production-grade interactive vector and raster maps without deploying, monitoring, or paying for specialized tile rendering servers.
  • Predictable Latency via Hilbert Indexing: 2D tile coordinates map to 1D continuous offsets, maximizing spatial locality and capping worst-case index resolution to two HTTP Range requests.
  • Run-Length Compression Efficiency: Identical visual tiles (oceans, flat terrain) are deduplicated directly within directory metadata headers, drastically shrinking archive file size.
  • Universal Frontend Ecosystem: Modern rendering engines including MapLibre GL JS, Leaflet, and OpenLayers integrate with PMTiles via lightweight protocol adapters requiring only standard HTTP 206 Partial Content support.

Open Questions

  • Dynamic Tile Mutation: Because PMTiles archives are strictly immutable, how can high-frequency vector updates (such as real-time vessel tracking or live road closures) be composited over static base archives without full re-indexing?
  • CDN Edge Range Slicing: How do different cloud CDN providers (Cloudflare, AWS CloudFront, Fastly) handle caching efficiency when multiple clients request overlapping but non-identical byte ranges?
  • WebGPU Decoding Pipelines: Can browser-based PMTiles readers stream compressed Zstandard vector tile chunks directly into WebGPU compute shaders for zero-copy geometry rasterization?

References

Editorial method: claims in this article point to primary documentation, pinned source code, runnable checks, or dated measurements, with AI-assisted research under the site's evidence-gated policy. How this site verifies claims and discloses AI assistance · Report a correction.

Working on something like this?

I take a small number of paid, scoped reviews: AI agent/RAG architecture diagnosis, Unity CI & build-automation audits, and multimodal QA design review. Each one ends in a written findings document.

Work with me
This post is licensed under CC BY 4.0 by the author.

© Jang Young Jeong. Some rights reserved.

Search article titles, categories, and tags. Full text is searched when needed.

Type to search articles.