Merge pull request #5 from geographika/gdalg

Add GDALG example

Author: geographika

workshop/content/docs/advanced/arcgis.md

@@ -183,7 +183,7 @@ labelsCheckbox.addEventListener('change', (event) => {
     --8<-- "arcgis.js"
     ```
 
-??? Mapfile "stac.map"
+??? Mapfile "arcgis.map"
 
     ``` scala
     --8<-- "arcgis.map"

workshop/content/docs/advanced/gdalg.md

@@ -0,0 +1,138 @@
+# Working with GDAL Vector Pipelines
+
+## Overview
+
+MapServer can dynamically run a [GDAL Vector Pipeline](https://gdal.org/en/latest/programs/gdal_vector_pipeline.html), and render its output -
+all through a simple Mapfile.
+
+In this workshop we'll use the Tartu roads dataset used in the [Line Styling](../mapfile/lines.md) exercise, dynamically
+buffer it using GDAL, and display the result in OpenLayers using a MapServer WMS.
+
+This is a simple example of a pipeline, but additional steps can be chained together to create more complex workflows.
+MapServer reads a vector pipeline using the [GDALG: GDAL Streamed Algorithm](https://gdal.org/en/latest/drivers/vector/gdalg.html) driver.
+
+<div class="map">
+  <iframe src="https://mapserver.github.io/getting-started-with-mapserver-demo/gdalg.html"></iframe>
+</div>
+
+## Checking the Pipelines with GDAL
+
+Before configuring MapServer, it is often easier to test your pipelines directly with GDAL, to ensure they run correctly.
+Run the commands below to connect to the MapServer Docker container and use GDAL to get information about the pipelines.
+
+```bash
+# open a shell inside the MapServer container
+docker exec -it mapserver /bin/bash
+
+# check the dataset used in the pipeline
+gdal vector info data/osm/roads.fgb
+
+# inspect the a pipeline JSON file included in the container
+gdal vector info roads.gdalg.json
+
+# test an inline pipeline using a single-quoted string (recommended)
+gdal vector info '{"type": "gdal_streamed_alg","command_line": "gdal vector pipeline ! read /etc/mapserver/data/osm/roads.fgb ! geom buffer --distance=0.0001"}'
+
+# alternatively escape the double quotes
+gdal vector info "{\"type\": \"gdal_streamed_alg\",\"command_line\": \"gdal vector pipeline ! read data/osm/roads.fgb ! geom buffer --distance=0.0001\"}"
+```
+
+## The Mapfile
+
+### Embedding Pipelines in a Mapfile
+
+The pipeline `LAYER` in this example uses `CONNECTIONTYPE OGR` and includes the GDAL pipeline "inline" - meaning the pipeline is defined
+directly in the Mapfile. Our example pipeline looks like this:
+
+```scala
+CONNECTION '{"type": "gdal_streamed_alg","command_line": "gdal vector pipeline ! read /etc/mapserver/data/osm/roads.fgb ! geom buffer --distance=0.0001"}'
+DATA "0"
+```
+
+Key points to note:
+
+- `DATA "0"` tells MapServer to use the first (index 0) layer in the connection. Since the pipeline returns a single dataset, this will correspond
+  to the buffered roads.
+- When using an inline GDAL pipeline, you must provide the absolute path to any datasets used by the pipeline.
+- GDAL requires a valid JSON string for the pipeline. All property names and string values must use double quotes.
+
+In a Mapfile, you have two options for embedding JSON:
+
+1. Wrap the JSON string in single quotes (as in the example above) - this is simpler and easier to read or copy/paste.
+2. Escape the double quotes using `\"` as in the example below:
+
+  ```scala
+  CONNECTION "{\"type\": \"gdal_streamed_alg\",\"command_line\": \"gdal vector pipeline ! read /etc/mapserver/data/osm/roads.fgb ! geom buffer --distance=0.0001\"}"
+  ```
+
+### Referencing Pipelines in a JSON File
+
+MapServer can also reference a JSON file containing a pipeline, which makes it easy to test and reuse the pipeline with GDAL.
+By convention GDALG files should use the `.gdalg.json` extension.
+
+```scala
+CONNECTION "roads.gdalg.json"
+DATA "0"
+```
+
+The contents of pipeline JSON file `roads.gdalg.json` are shown below. Notice that the dataset path `data/osm/roads.fgb` is relative to the JSON file.
+
+Using relative paths in a JSON file makes the pipeline more portable, because it can be moved to a different folder or system without changing the dataset paths.
+
+```json
+{
+    "type": "gdal_streamed_alg",
+    "command_line": "gdal vector pipeline ! read data/osm/roads.fgb ! geom buffer --distance=0.0001"
+}
+```
+
+### Hatch Styling
+
+Finally, we use a [hatch symbol](https://mapserver.org/mapfile/symbol.html#mapfile-symbol-type) to style the buffered roads.
+Hatch symbols allow you to add patterned fills, and you can adjust their angle, width, and size in the [STYLE](https://mapserver.org/mapfile/style.html) block.
+
+```scala
+SYMBOL
+  NAME "hatchsymbol"
+  TYPE hatch
+END
+...
+LAYER
+  CLASS
+    STYLE
+      SYMBOL "hatchsymbol"
+      COLOR "#78C8FF"
+      WIDTH 0.1
+      ANGLE 45
+      SIZE 8
+    END
+```
+
+## Code
+
+!!! example
+
+    - Direct MapServer request: <http://localhost:7000/?map=/etc/mapserver/gdalg.map&REQUEST=GetMap&SERVICE=WMS&VERSION=1.3.0&FORMAT=image%2Fpng&STYLES=&TRANSPARENT=TRUE&LAYERS=buffered_roads%2Croads&WIDTH=1707&HEIGHT=848&CRS=EPSG%3A3857&BBOX=2974643.6269619283%2C8046226.818245997%2C2976682.478528896%2C8047239.608870775>
+    - Local OpenLayers example: <http://localhost:7001/gdalg.html>
+
+??? JavaScript "gdalg.js"
+
+    ``` js
+    --8<-- "gdalg.js"
+    ```
+
+??? Mapfile "gdalg.map"
+
+    ``` scala
+    --8<-- "gdalg.map"
+    ```
+
+## Exercises
+
+1. Switch to using the JSON file - replace the inline GDAL pipeline with `roads.gdalg.json` in your Mapfile.
+
+2. Extend the GDAL pipeline - add another processing step to the pipeline.
+   Refer to the [GDAL Vector Pipeline documentation](https://gdal.org/en/latest/programs/gdal_vector_pipeline.html) for examples of available operations.
+
+3. Experiment with hatch styling - adjust properties such as `ANGLE` and `SIZE` in your `STYLE` block
+   to see how the hatch pattern changes.

workshop/content/mkdocs.yml

@@ -30,6 +30,7 @@ nav:
         - OGC API - Features: outputs/ogcapi-features.md
     - Advanced:
         - ArcGIS Feature Server: advanced/arcgis.md
+        - GDAL Pipeline: advanced/gdalg.md
         - Vector Symbols: advanced/symbols.md
         - Clusters: advanced/clusters.md
         - SLD: advanced/sld.md

workshop/exercises/app/gdalg.html

@@ -0,0 +1,14 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8" />
+    <link rel="icon" type="image/x-icon" href="https://openlayers.org/favicon.ico" />
+    <link rel="stylesheet" href="node_modules/ol/ol.css">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>GDAL Vector Pipeline</title>
+</head>
+<body>
+    <div id="map" style="background-color: #FFFAF0"></div>
+    <script type="module" src="./js/gdalg.js"></script>
+</body>
+</html>

workshop/exercises/app/index.html

@@ -30,9 +30,10 @@ <h2>Outputs</h2>
         <li><a href="vector-tiles.html">Vector Tiles</a></li>
         <li><a href="ogcapi-features.html">OGC API - Features</a></li>
     </ul>
-    <h2>Advanced</h2>    
+    <h2>Advanced</h2>
     <ul>
         <li><a href="arcgis.html">ArcGIS Feature Server</a></li>
+        <li><a href="gdalg.html">GDAL Pipelines</a></li>
         <li><a href="railways.html">Vector Symbols (Railways)</a></li>
         <li><a href="clusters.html">Clusters</a></li>
         <li><a href="landuse.html">Landuse</a></li>

workshop/exercises/app/js/gdalg.js

@@ -0,0 +1,26 @@
+import '../css/style.css';
+import ImageWMS from 'ol/source/ImageWMS.js';
+import Map from 'ol/Map.js';
+import View from 'ol/View.js';
+import { Image as ImageLayer } from 'ol/layer.js';
+
+const mapserverUrl = import.meta.env.VITE_MAPSERVER_BASE_URL;
+const mapfilesPath = import.meta.env.VITE_MAPFILES_PATH;
+
+const layers = [
+    new ImageLayer({
+        source: new ImageWMS({
+            url: mapserverUrl + mapfilesPath + 'gdalg.map&',
+            params: { 'LAYERS': 'buffered_roads,roads' },
+            ratio: 1
+        }),
+    }),
+];
+const map = new Map({
+    layers: layers,
+    target: 'map',
+    view: new View({
+        center: [2975862.75916499, 8046369.8646329],
+        zoom: 17,
+    }),
+});

workshop/exercises/mapfiles/gdalg.map

@@ -0,0 +1,61 @@
+MAP
+  NAME "GDALG"
+  EXTENT 26.668678 58.339241 26.796582 58.409410
+  SIZE 800 600
+  IMAGECOLOR "#0A1E50"
+  PROJECTION
+    "init=epsg:4326"
+  END
+  WEB
+    METADATA
+      "ows_enable_request" "*"
+      "ows_srs" "EPSG:4326 EPSG:3857"
+    END
+  END
+
+  SYMBOL
+    NAME "hatchsymbol"
+    TYPE hatch
+  END
+
+  LAYER
+    NAME "buffered_roads"
+    TYPE POLYGON
+    STATUS OFF
+    CONNECTIONTYPE OGR
+    # note we need to use the full path to the dataset using the inline pipeline
+    CONNECTION '{"type": "gdal_streamed_alg","command_line": "gdal vector pipeline ! read /etc/mapserver/data/osm/roads.fgb ! geom buffer --distance=0.0001"}'
+    DATA "0"
+    # CONNECTION "roads.gdalg.json"
+    # DATA "0"
+    CLASS
+      STYLE
+        COLOR 50 100 180
+      END
+      STYLE
+        SYMBOL "hatchsymbol"
+        COLOR "#78C8FF"
+        WIDTH 0.1
+        ANGLE 45
+        SIZE 8
+      END
+    END
+  END
+
+  # original unbuffered roads dataset
+  LAYER
+    NAME "roads"
+    TYPE LINE
+    STATUS OFF
+    CONNECTIONTYPE FLATGEOBUF
+    DATA "data/osm/roads.fgb"
+    CLASS
+      STYLE
+        WIDTH 0.8
+        COLOR "#DCF0FF"
+      END
+    END
+  END
+
+END
+

workshop/exercises/mapfiles/roads.gdalg.json

@@ -0,0 +1,4 @@
+{
+    "type": "gdal_streamed_alg",
+    "command_line": "gdal vector pipeline ! read data/osm/roads.fgb ! geom buffer --distance=0.0001"
+}