Ground Generation

Ground Generation System in Arnis

This document provides a comprehensive technical reference for Arnis's ground generation system, which handles terrain elevation data and transforms it into Minecraft's block-based terrain.

Overview

The ground generation system is responsible for:

Fetching real-world elevation data from AWS Terrain Tiles (free S3 access)
Processing and normalizing this data with statistical outlier filtering
Converting it to Minecraft-compatible height values with adaptive scaling
Providing terrain height information to the WorldEditor during block placement
Caching tiles locally for improved performance

The system enables the creation of realistic terrain features that match the real-world topography of the selected area using high-quality elevation data from multiple sources (SRTM, 3DEP, EUDEM, ArcticDEM), enhancing the immersion and realism of the generated worlds.

Architecture

The ground generation system is encapsulated in the Ground struct and its associated methods. The core components include:

Ground
├── elevation_enabled: bool
├── ground_level: i32
└── elevation_data: Option<ElevationData>
    ├── heights: Vec<Vec<i32>>
    ├── width: usize
    └── height: usize

Key Components

1. Ground Struct

The main interface for terrain elevation, providing:

Height queries for specific coordinates
Min/max elevation calculations
Default flat terrain when elevation data is unavailable

2. ElevationData Struct

A private struct that stores:

A 2D grid of height values mapped to Minecraft Y coordinates
The dimensions of the elevation grid
Associated metadata for interpolation and scaling

Technical Process Flow

The ground generation process follows these key steps:

1. Initialization

pub fn generate_ground_data(args: &Args) -> Ground {
    if args.terrain {
        emit_gui_progress_update(5.0, "Fetching elevation...");
    }
    Ground::new(args)
}

The system begins by checking if terrain generation is enabled via the args.terrain flag and initializes the progress reporting.

2. Elevation Data Acquisition

The fetch_elevation_data method handles the complex process of retrieving real-world elevation data:

Calculate Scale Factors:

let (scale_factor_z, scale_factor_x) = 
    crate::osm_parser::geo_distance(args.bbox.min(), args.bbox.max());
let scale_factor_x: f64 = scale_factor_x * args.scale;
let scale_factor_z: f64 = scale_factor_z * args.scale;

Determine Appropriate Zoom Level:
```
let zoom: u8 = Self::calculate_zoom_level(args.bbox);
```
The zoom level is dynamically calculated based on the bounding box size, ranging from MIN_ZOOM (10) to MAX_ZOOM (15).
Calculate Required Map Tiles:
```
let tiles: Vec<(u32, u32)> = Self::get_tile_coordinates(args.bbox, zoom);
```
This determines all the map tiles needed to cover the requested area.
Fetch Elevation Tiles via AWS Terrain Tiles:
```
let url: String = format!(
    "https://s3.amazonaws.com/elevation-tiles-prod/terrarium/{}/{}/{}.png",
    zoom, tile_x, tile_y
);
```
Each tile is fetched as a PNG image from AWS S3 where RGB values encode elevation data using the Terrarium format. No API key is required.

Transform to Height Values:

let height: f64 = (pixel[0] as f64 * 256.0 + pixel[1] as f64 + pixel[2] as f64 / 256.0) - 32768.0;

The RGB values are decoded according to the AWS Terrarium elevation encoding format.

3. Data Processing

Once the raw elevation data is collected, it undergoes several processing steps:

Fill Missing Values:
```
Self::fill_nan_values(&mut height_grid);
```
Any gaps in the elevation data (water bodies, etc.) are filled using iterative interpolation from neighboring cells.
Statistical Outlier Filtering:
```
Self::filter_elevation_outliers(&mut height_grid);
```
Statistical outlier detection removes corrupt elevation data using percentile analysis (1st/99th percentiles), preventing extreme spikes that could break terrain generation.

Adaptive Gaussian Blur Smoothing:

let sigma = if grid_size <= SMALL_GRID_REF {
    SMALL_SIGMA_REF * (grid_size / SMALL_GRID_REF)
} else {
    SMALL_SIGMA_REF + t * (LARGE_SIGMA_REF - SMALL_SIGMA_REF)
};
let blurred_heights: Vec<Vec<f64>> = Self::apply_gaussian_blur(&height_grid, sigma);

Adaptive Gaussian blur with dynamic sigma calculation based on grid size smooths terrain and reduces tile boundary artifacts.

Dynamic Height Scaling with Safety Margins:
```
let height_scale: f64 = BASE_HEIGHT_SCALE * args.scale.sqrt();
let available_y_range = (MAX_Y - ground_level) as f64;
let safety_margin = 0.9; // Use 90% of available range
```
The height values are scaled with a base factor of 0.6, modified by the square root of the user's scale factor, with safety margins to prevent exceeding Minecraft's build limits.

Enhanced Minecraft Coordinate Conversion:

let mc_row: Vec<i32> = row.iter().map(|&h| {
    let relative_height: f64 = (h - min_height) / height_range;
    let scaled_height: f64 = relative_height * scaled_range;
    ((args.ground_level as f64 + scaled_height).round() as i32)
        .clamp(args.ground_level, MAX_Y)
}).collect();

Heights are mapped to Minecraft's Y coordinates with improved clamping and range validation.

4. Runtime Height Queries

During world generation, element processors request ground heights via the level method:

pub fn level(&self, coord: XZPoint) -> i32 {
    if !self.elevation_enabled || self.elevation_data.is_none() {
        return self.ground_level;
    }

    let data: &ElevationData = self.elevation_data.as_ref().unwrap();
    let (x_ratio, z_ratio) = self.get_data_coordinates(coord, data);
    self.interpolate_height(x_ratio, z_ratio, data)
}

This method:

Checks if elevation is enabled (returning a flat ground level if not)
Converts coordinates to elevation data space
Performs bilinear interpolation to get precise height values

Algorithm Details

Adaptive Gaussian Blur Implementation

The terrain smoothing uses an adaptive separable 2D Gaussian blur implementation with dynamic sigma calculation:

fn apply_gaussian_blur(heights: &[Vec<f64>], sigma: f64) -> Vec<Vec<f64>> {
    let kernel_size: usize = (sigma * 3.0).ceil() as usize * 2 + 1;
    let kernel: Vec<f64> = Self::create_gaussian_kernel(kernel_size, sigma);
    
    // Two-pass separable filter: horizontal then vertical
}

// Adaptive sigma calculation based on grid size
const SMALL_GRID_REF: f64 = 100.0;   // Reference: 100x100 grid
const SMALL_SIGMA_REF: f64 = 15.0;   // Sigma for small grids
const LARGE_GRID_REF: f64 = 1000.0;  // Reference: 1000x1000 grid  
const LARGE_SIGMA_REF: f64 = 7.0;    // Sigma for large grids

Key improvements:

Adaptive Sigma: Dynamic blur strength based on grid size for optimal quality
Separability: O(n) complexity via horizontal + vertical passes instead of O(n²)
Boundary Handling: Enhanced edge processing to prevent artifacts
Scale Optimization: Better smoothing for both small and large terrain areas

Coordinate Transformation

The system handles multiple coordinate spaces with enhanced precision:

Geographic Coordinates (lat/lng): Used for API requests
Tile Coordinates: Used for AWS S3 API (x, y, zoom)
Elevation Grid Coordinates: Internal representation (height_grid[y][x])
Minecraft Coordinates: Final XZ coordinates with Y elevation

The transformation chain uses Web Mercator projection (EPSG:3857):

Geographic → Tile (Web Mercator) → Elevation Grid → Minecraft Coordinates

Distance calculations use the Haversine formula optimized for accuracy:

fn lat_distance(lat1: f64, lat2: f64) -> f64  // Earth curvature in NS direction
fn lon_distance(lat: f64, lon1: f64, lon2: f64) -> f64  // East-West at given latitude

Enhanced Interpolation System

Height values use an improved interpolation approach with boundary safety:

fn interpolate_height(&self, x_ratio: f64, z_ratio: f64, data: &ElevationData) -> i32 {
    let x: usize = ((x_ratio * (data.width - 1) as f64).round() as usize).min(data.width - 1);
    let z: usize = ((z_ratio * (data.height - 1) as f64).round() as usize).min(data.height - 1);
    data.heights[z][x]
}

Features:

Uses normalized coordinates (0.0 to 1.0) for position within the grid
Enhanced boundary checking to prevent index out-of-bounds errors
Efficient nearest-neighbor lookup optimized for real-time use
Compatible with statistical outlier filtering for robust data access

Technical Implementation Details

Constants and Parameters

The system uses several important constants updated for AWS Terrain Tiles:

const MAX_Y: i32 = 319;                          // Minecraft build height limit
const BASE_HEIGHT_SCALE: f64 = 0.6;              // Enhanced elevation scaling factor
const AWS_TERRARIUM_URL: &str = "https://s3.amazonaws.com/elevation-tiles-prod/terrarium/{z}/{x}/{y}.png";
const TERRARIUM_OFFSET: f64 = 32768.0;           // Terrarium format offset for height decoding
const MIN_ZOOM: u8 = 10;                         // Minimum tile zoom level
const MAX_ZOOM: u8 = 15;                         // Maximum tile zoom level

// Adaptive Gaussian blur parameters
const SMALL_GRID_REF: f64 = 100.0;               // Reference: 100x100 grid
const SMALL_SIGMA_REF: f64 = 15.0;               // Sigma for small grids
const LARGE_GRID_REF: f64 = 1000.0;              // Reference: 1000x1000 grid  
const LARGE_SIGMA_REF: f64 = 7.0;                // Sigma for large grids

These constants define the physical limits, data source configuration, and quality parameters. No API key is required for AWS S3 access.

Enhanced Error Handling and Debugging

Statistical Reporting: Detailed elevation range and outlier statistics
Extreme Value Detection: Warns about elevations outside reasonable ranges (-1000m to 10000m)
Tile Processing Progress: Logs download progress and cache hit/miss ratios
Data Quality Metrics: Reports number of outliers filtered and interpolated
Graceful Degradation: Falls back to flat terrain when elevation data fails
Debug Image Output: Visual elevation maps for quality verification

Performance Optimizations

Several optimizations enhance performance and efficiency:

#[inline(always)] Attributes:
```
#[inline(always)]
pub fn level(&self, coord: XZPoint) -> i32 { ... }
```
Critical methods are marked for inline expansion to reduce function call overhead.
Intelligent Tile Caching:
- Location: ./arnis-tile-cache/
- Format: z{zoom}_x{tile_x}_y{tile_y}.png
- Persistence: Tiles cached permanently until manually deleted
- Benefits: Eliminates redundant network requests for overlapping regions
- Performance: Significantly faster processing for repeated areas
Direct S3 Access:
- No API Gateway Overhead: Direct access to AWS S3 eliminates latency
- No Rate Limiting: Unlimited concurrent requests
- No Authentication: Eliminates token validation overhead
- CDN Acceleration: Global edge locations for faster downloads
Memory Optimization:
- Efficient Array Processing: Uses iterators and direct indexing
- Separable Filter Implementation: O(n) vs O(n²) complexity for Gaussian blur
- Statistical Filtering: Reduces memory corruption from outlier data

Enhanced Bounds Checking:

.clamp(0.0, 1.0)  // Normalized coordinate clamping
.min(data.width - 1)  // Array bounds protection

Enhanced Debug Visualization

For development and quality assurance, the system provides comprehensive debugging tools:

fn save_debug_image(heights: &Vec<Vec<i32>>, filename: &str) { ... }

Debug Features:

Visual Elevation Maps: Grayscale PNG images showing terrain height distribution
Statistical Analysis: Min/max elevation ranges and distribution metrics
Outlier Detection Reports: Number and location of filtered extreme values
Tile Processing Logs: Download progress, cache hits/misses, and timing data
Data Quality Metrics: Source attribution and resolution information

Enhanced Debugging Output:

Height data range: -12.3 to 2847.1 m
WARNING: Found 23 pixels with extremely high elevations (> 10000m)
Filtering outliers outside range: -8.2m to 2834.7m  
Filtered 156 elevation outliers, interpolating replacements...
Minecraft height data range: 64 to 234 blocks
Processed 42 tiles (38 cached, 4 downloaded) in 3.2s

This comprehensive debugging system helps verify elevation data correctness and identify processing artifacts for quality assurance.

Integration with Other Systems

WorldEditor Integration

The Ground system integrates with the WorldEditor via the get_absolute_y method:

pub fn get_absolute_y(&self, x: i32, y_offset: i32, z: i32) -> i32 {
    if let Some(ground) = &self.ground {
        ground.level(XZPoint::new(
            x - self.xzbbox.min_x(),
            z - self.xzbbox.min_z(),
        )) + y_offset
    } else {
        y_offset // If no ground reference, use y_offset as absolute Y
    }
}

This translation layer allows element processors to place blocks relative to the ground surface, rather than requiring absolute Y coordinates.

User Configuration

Ground generation respects several user-configurable parameters:

terrain: Boolean flag to enable/disable terrain elevation
ground_level: Base Y-level for flat terrain or minimum elevation
scale: Affects both horizontal and vertical scaling
bbox: Determines the geographic area for elevation data

System Limitations and Considerations

Current Limitations

Memory Usage: For large areas, the elevation grid can consume significant memory (O(n²) where n is the area dimension). No streaming or chunked processing is currently implemented.
Single Provider Dependency: While AWS S3 has excellent uptime, the system relies entirely on AWS Terrain Tiles availability. No fallback data sources are implemented.
Height Constraints: Minecraft's build height limit (319) constrains the vertical scale of terrain features, particularly for mountainous regions.
Resolution Limits: The maximum zoom level (15) provides 4.8m ground resolution, which may be insufficient for very detailed terrain features.

Resolved Limitations (Previously Fixed)

✅ API Costs: Eliminated $0.50/1000 requests cost through migration to free AWS S3 access
✅ Rate Limiting: No longer applicable with direct S3 access
✅ API Key Management: No authentication required
✅ Data Quality: Statistical outlier filtering removes corrupt elevation data
✅ Processing Artifacts: Enhanced Gaussian blur with adaptive parameters
✅ Debugging Capabilities: Comprehensive diagnostic output and visualization tools

Example Usage

Within element processors, the Ground system is typically used as follows:

// Get the height at a specific point
let ground_level = editor.get_ground().unwrap().level(XZPoint::new(x, z));

// Place a block 3 units above ground level
editor.set_block(STONE, x, 3, z, None, None);

// Create a structure with a foundation at ground level
for y in 0..5 {
    editor.set_block(STONE, x, y, z, None, None);
}

// Terrain-aware building placement with foundation pillars
if args.terrain && min_level == 0 {
    let local_ground_level = if let Some(ground) = editor.get_ground() {
        ground.level(XZPoint::new(x, z))
    } else {
        args.ground_level
    };
    
    // Create foundation from ground to building base
    editor.fill_blocks_y_absolute(
        foundation_block, 
        x, 
        local_ground_level, 
        z, 
        start_y_offset - 1, 
        None, 
        None
    );
}

Advanced Features

Building Integration

Buildings automatically adapt to terrain when enabled:

let start_y_offset = if args.terrain {
    // Find maximum ground level across building footprint  
    max_ground_level + min_level_offset
} else {
    args.ground_level + min_level_offset
};

Conclusion

The Ground generation system in Arnis provides a sophisticated and cost-effective solution for incorporating real-world terrain data into Minecraft worlds. With the recent migration to AWS Terrain Tiles, the system now offers:

Technical Excellence:

High-quality elevation data from multiple authoritative sources (SRTM, 3DEP, EUDEM, ArcticDEM)
Advanced statistical processing with outlier filtering and adaptive smoothing
Optimal 4.8m ground resolution with intelligent zoom level selection
Robust error handling and comprehensive debugging capabilities

Operational Benefits:

Zero ongoing costs through free AWS S3 access
No API key management or rate limiting concerns
Persistent tile caching for improved performance
Simplified deployment and configuration

Quality Assurance:

Statistical outlier detection removes corrupt elevation data
Enhanced debugging provides detailed quality metrics
Perfect coordinate alignment with OSM data processing
Comprehensive test coverage and validation

The system's efficient implementation balances accuracy with performance, enabling practical application in large-scale world generation projects. Its seamless integration with the WorldEditor makes terrain-aware construction intuitive for all element processors, while the migration to AWS Terrain Tiles ensures long-term sustainability and cost-effectiveness.

The ground generation system now operates at maximum theoretical quality with perfect spatial alignment, making it an ideal foundation for realistic Minecraft world generation from real-world geographic data.

Uh oh!

Ground Generation

Ground Generation System in Arnis

Overview

Architecture

Key Components

1. Ground Struct

2. ElevationData Struct

Technical Process Flow

1. Initialization

2. Elevation Data Acquisition

3. Data Processing

4. Runtime Height Queries

Algorithm Details

Adaptive Gaussian Blur Implementation

Coordinate Transformation

Enhanced Interpolation System

Technical Implementation Details

Constants and Parameters

Enhanced Error Handling and Debugging

Performance Optimizations

Enhanced Debug Visualization

Integration with Other Systems

WorldEditor Integration

User Configuration

System Limitations and Considerations

Current Limitations

Resolved Limitations (Previously Fixed)

Example Usage

Advanced Features

Building Integration

Conclusion

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally