Spatial Narrative

spatial narrative

Extract geographic narratives from text.

What It Does

spatial-narrative extracts locations and events from unstructured text, turning documents into structured geospatial data.

Input:  "The summit in Paris brought together leaders from Berlin and Tokyo.
         Negotiations continued through the week before concluding in Geneva."

Output: [
  { location: Paris (48.86°, 2.35°), text: "summit" },
  { location: Berlin (52.52°, 13.41°), text: "leaders" },
  { location: Tokyo (35.68°, 139.65°), text: "leaders" },
  { location: Geneva (46.20°, 6.14°), text: "concluding" }
]

Core Workflow

┌──────────────┐      ┌──────────────┐      ┌──────────────┐      ┌──────────────┐
│     Text     │  →   │  Geoparser   │  →   │   Narrative  │  →   │    Export    │
│  (documents, │      │  (extract    │      │   (events,   │      │  (GeoJSON,   │
│   articles)  │      │   locations) │      │   analysis)  │      │   mapping)   │
└──────────────┘      └──────────────┘      └──────────────┘      └──────────────┘

Key Features

FeatureDescription
GeoparsingExtract place names from text and resolve to coordinates
Built-in Gazetteer2,500+ world cities with coordinates, population, aliases
Coordinate DetectionParse decimal degrees, DMS, and other coordinate formats
Online GazetteersOptional Nominatim, GeoNames, Wikidata integration
Event ModelingStructure extracted locations into events with timestamps
AnalysisClustering, spatial metrics, trajectory detection
ExportGeoJSON, CSV, JSON for mapping tools

Quick Example

use spatial_narrative::parser::{GeoParser, BuiltinGazetteer};

// Create parser with built-in gazetteer (2500+ cities, no API needed)
let gazetteer = BuiltinGazetteer::new();
let parser = GeoParser::with_gazetteer(gazetteer);

// Extract locations from text
let text = "Fighting broke out near Kyiv before spreading to Kharkiv and Odesa.";
let mentions = parser.extract(text);

for mention in &mentions {
    if let Some(loc) = &mention.location {
        println!("{}: ({:.2}°, {:.2}°)", mention.text, loc.lat, loc.lon);
    }
}
// Kyiv: (50.45°, 30.52°)
// Kharkiv: (49.99°, 36.23°)
// Odesa: (46.48°, 30.73°)

Use Cases

  • Journalism: Extract locations from news articles to map story development
  • Intelligence: Geolocate events from reports and social media
  • Historical Research: Map events from historical documents
  • Disaster Response: Extract affected locations from situation reports
  • Academic Research: Ground qualitative text data in geography

Modules

ModulePurpose
parserGeoparsing: extract locations from text
coreData types: Event, Location, Timestamp, Narrative
analysisClustering, metrics, trajectory analysis
indexSpatial/temporal indexing for large datasets
graphEvent relationship networks
ioGeoJSON, CSV, JSON export

Getting Started

Ready to extract locations from text? Start with the Installation guide!

Installation

Requirements

  • Rust: 1.70 or later
  • Cargo: Included with Rust

Adding to Your Project

Add spatial-narrative to your Cargo.toml:

[dependencies]
spatial-narrative = "0.1"

Or use cargo add:

cargo add spatial-narrative

Features

The library comes with sensible defaults. All core features are included by default.

Default Features

[dependencies]
spatial-narrative = "0.1"  # Includes all standard features

Optional Features

Enable additional functionality by specifying features:

[dependencies]
spatial-narrative = { version = "0.1", features = ["geocoding", "ml-ner-download"] }
FeatureDescriptionDefault
serdeSerialization/deserialization support
parallelParallel processing with rayon
geocodingExternal geocoding APIs (Nominatim, GeoNames, Wikidata)
gpx-supportGPX file format support
databaseDatabase persistence (PostgreSQL, SQLite)
projectionsCoordinate system transformations
nlpEnhanced text processing with NLP
ml-nerMachine learning NER with ONNX Runtime
ml-ner-downloadAuto-download ML models from HuggingFace Hub
cliCommand-line interface tools
fullAll features enabled

ML-NER Requirements

The ml-ner and ml-ner-download features require ONNX Runtime:

macOS (Homebrew):

brew install onnxruntime
export ORT_DYLIB_PATH=$(brew --prefix onnxruntime)/lib/libonnxruntime.dylib

Linux (Ubuntu/Debian):

sudo apt install libonnxruntime
export ORT_DYLIB_PATH=/usr/lib/libonnxruntime.so

Manual Download: Download from ONNX Runtime releases and set ORT_DYLIB_PATH to the library path.

See ML-NER documentation for detailed setup instructions.

Verifying Installation

Create a simple test file:

// src/main.rs
use spatial_narrative::core::{Location, Timestamp, Event};

fn main() {
    let location = Location::new(40.7128, -74.0060);
    let timestamp = Timestamp::now();
    let event = Event::new(location, timestamp, "Hello, spatial-narrative!");
    
    println!("Created event: {}", event.text);
    println!("Location: ({}, {})", event.location.lat, event.location.lon);
}

Run it:

cargo run

You should see:

Created event: Hello, spatial-narrative!
Location: (40.7128, -74.006)

Building from Source

# Clone the repository
git clone https://github.com/yourusername/spatial-narrative.git
cd spatial-narrative

# Build
cargo build --release

# Run tests
cargo test

# Generate documentation
cargo doc --open

Minimum Supported Rust Version (MSRV)

The current MSRV is Rust 1.70.

This version is tested in CI and will be maintained according to our compatibility policy.

Next Steps

Quick Start

This guide will get you up and running with spatial-narrative in under 5 minutes.

Create Your First Narrative

Step 1: Create Events

Events are the fundamental building blocks. Each event has a location, timestamp, and text description:

use spatial_narrative::core::{Event, Location, Timestamp};

// Create locations
let nyc = Location::new(40.7128, -74.0060);
let boston = Location::new(42.3601, -71.0589);

// Create timestamps
let morning = Timestamp::parse("2024-06-15T09:00:00Z").unwrap();
let afternoon = Timestamp::parse("2024-06-15T14:00:00Z").unwrap();

// Create events
let event1 = Event::new(nyc, morning, "Departure from New York City");
let event2 = Event::new(boston, afternoon, "Arrival in Boston");

println!("Event 1: {} at {}", event1.text, event1.timestamp.to_rfc3339());
println!("Event 2: {} at {}", event2.text, event2.timestamp.to_rfc3339());

Step 2: Build a Narrative

A Narrative collects events into a coherent story:

use spatial_narrative::core::{Event, Location, Timestamp, NarrativeBuilder};

let events = vec![
    Event::new(
        Location::new(40.7128, -74.0060),
        Timestamp::parse("2024-06-15T09:00:00Z").unwrap(),
        "Departure from NYC"
    ),
    Event::new(
        Location::new(42.3601, -71.0589),
        Timestamp::parse("2024-06-15T14:00:00Z").unwrap(),
        "Arrival in Boston"
    ),
];

let narrative = NarrativeBuilder::new()
    .title("Road Trip to Boston")
    .description("A day trip from NYC to Boston")
    .tag("travel")
    .tag("road-trip")
    .events(events)
    .build();

println!("Narrative: {}", narrative.title.as_deref().unwrap_or("Untitled"));
println!("Events: {}", narrative.events.len());

Step 3: Query Events

Filter events spatially and temporally:

use spatial_narrative::core::{GeoBounds, TimeRange, Timestamp};

// Get events in chronological order
let ordered = narrative.events_chronological();
for event in &ordered {
    println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}

// Get time range
if let Some(range) = narrative.time_range() {
    println!("Duration: {} hours", range.duration().num_hours());
}

// Get geographic bounds
if let Some(bounds) = narrative.bounds() {
    println!("Area: ({:.2}, {:.2}) to ({:.2}, {:.2})",
        bounds.min_lat, bounds.min_lon,
        bounds.max_lat, bounds.max_lon
    );
}

Step 4: Index for Fast Queries

For large datasets, use indexes:

use spatial_narrative::index::SpatiotemporalIndex;
use spatial_narrative::core::{GeoBounds, TimeRange, Timestamp};

// Create an index
let mut index = SpatiotemporalIndex::new();

// Insert events
for event in &narrative.events {
    index.insert(event.clone(), &event.location, &event.timestamp);
}

// Query by location and time
let bounds = GeoBounds::new(40.0, -75.0, 43.0, -70.0);
let time_range = TimeRange::new(
    Timestamp::parse("2024-06-15T00:00:00Z").unwrap(),
    Timestamp::parse("2024-06-15T23:59:59Z").unwrap(),
);

let results = index.query(&bounds, &time_range);
println!("Found {} events in region during time range", results.len());

Step 5: Build a Graph

Connect events into a relationship graph:

use spatial_narrative::graph::{NarrativeGraph, EdgeType};

// Create graph from events
let mut graph = NarrativeGraph::from_events(narrative.events.clone());

// Auto-connect by temporal sequence
graph.connect_temporal();

// Connect spatially close events (within 50km)
graph.connect_spatial(50.0);

println!("Graph: {} nodes, {} edges", graph.node_count(), graph.edge_count());

// Export to DOT format for visualization
let dot = graph.to_dot();
println!("DOT output:\n{}", dot);

Step 6: Export to GeoJSON

Export for use in mapping tools:

use spatial_narrative::io::{Format, GeoJsonFormat};

let geojson_format = GeoJsonFormat::default();
let geojson_string = geojson_format.export(&narrative.events).unwrap();

// Save to file
std::fs::write("narrative.geojson", &geojson_string).unwrap();

println!("Exported to narrative.geojson");

Complete Example

Here's everything together:

use spatial_narrative::core::{Event, Location, Timestamp, NarrativeBuilder};
use spatial_narrative::index::SpatiotemporalIndex;
use spatial_narrative::graph::{NarrativeGraph, EdgeType};
use spatial_narrative::io::{Format, GeoJsonFormat};

fn main() {
    // Create events
    let events = vec![
        Event::new(
            Location::builder().lat(40.7128).lon(-74.0060).name("NYC").build().unwrap(),
            Timestamp::parse("2024-06-15T09:00:00Z").unwrap(),
            "Departure from New York City"
        ),
        Event::new(
            Location::builder().lat(41.2033).lon(-73.1975).name("New Haven").build().unwrap(),
            Timestamp::parse("2024-06-15T11:00:00Z").unwrap(),
            "Quick stop in New Haven"
        ),
        Event::new(
            Location::builder().lat(42.3601).lon(-71.0589).name("Boston").build().unwrap(),
            Timestamp::parse("2024-06-15T14:00:00Z").unwrap(),
            "Arrival in Boston"
        ),
    ];

    // Build narrative
    let narrative = NarrativeBuilder::new()
        .title("Road Trip to Boston")
        .events(events)
        .build();

    // Index events
    let mut index = SpatiotemporalIndex::new();
    for event in &narrative.events {
        index.insert(event.clone(), &event.location, &event.timestamp);
    }

    // Build graph
    let mut graph = NarrativeGraph::from_events(narrative.events.clone());
    graph.connect_temporal();

    // Export
    let geojson = GeoJsonFormat::default().export(&narrative.events).unwrap();

    println!("Created narrative with {} events", narrative.events.len());
    println!("Graph has {} connections", graph.edge_count());
    println!("GeoJSON: {} bytes", geojson.len());
}

Next Steps

Core Concepts

Understanding these concepts will help you use spatial-narrative effectively.

The Data Model

┌─────────────────────────────────────────────────────────────────┐
│                         NARRATIVE                                │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │ Title, Description, Tags, Metadata                           ││
│  └─────────────────────────────────────────────────────────────┘│
│                              │                                   │
│                              ▼                                   │
│  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐               │
│  │ Event 1 │ │ Event 2 │ │ Event 3 │ │ Event N │ ...           │
│  └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘               │
│       │           │           │           │                      │
│       ▼           ▼           ▼           ▼                      │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │  Location    │  Timestamp   │  Text      │  Metadata        ││
│  │  (lat, lon)  │  (datetime)  │  (string)  │  (tags, source)  ││
│  └─────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────┘

Core Types

Event

The Event is the atomic unit. It represents something that happened:

  • Where: A Location (latitude, longitude, optional elevation)
  • When: A Timestamp (timezone-aware datetime with precision)
  • What: Text description
  • Context: Tags, source references, custom metadata
// Simple event
let event = Event::new(location, timestamp, "Something happened");

// Rich event with builder
let event = EventBuilder::new()
    .location(location)
    .timestamp(timestamp)
    .text("Something happened")
    .tag("important")
    .source(source_ref)
    .build()
    .unwrap();

Location

A Location represents a point on Earth:

  • Required: lat (-90 to 90), lon (-180 to 180)
  • Optional: elevation (meters), uncertainty (meters), name
// Minimal
let loc = Location::new(40.7128, -74.0060);

// With details
let loc = Location::builder()
    .lat(40.7128)
    .lon(-74.0060)
    .elevation(10.5)
    .uncertainty_meters(5.0)
    .name("Empire State Building")
    .build()
    .unwrap();

Timestamp

A Timestamp represents a moment in time:

  • Timezone-aware (stored as UTC)
  • Precision levels: Year, Month, Day, Hour, Minute, Second
  • Flexible parsing (ISO 8601, dates, partial dates)
// Parse various formats
Timestamp::parse("2024-01-15T10:30:00Z")?;     // Full ISO 8601
Timestamp::parse("2024-01-15")?;                // Date only
Timestamp::parse("2024-01")?;                   // Month only
Timestamp::parse("2024")?;                      // Year only

// Current time
Timestamp::now();

Narrative

A Narrative is a collection of events that form a coherent story:

  • Has a title, description, and tags
  • Contains zero or more events
  • Provides aggregate operations (bounds, time range, filtering)
let narrative = NarrativeBuilder::new()
    .title("My Story")
    .description("A series of events")
    .tag("category")
    .events(vec![event1, event2, event3])
    .build();

Spatial Concepts

Coordinates

We use WGS84 (standard GPS) coordinates:

  • Latitude: -90 (South Pole) to +90 (North Pole)
  • Longitude: -180 to +180 (Prime Meridian at 0)
                    +90° (North Pole)
                         │
    -180° ───────────────┼───────────────► +180°
         (International  │  (Date Line)
          Date Line)     │
                         │
                    -90° (South Pole)

GeoBounds

A GeoBounds represents a rectangular region:

  • Defined by min/max latitude and longitude
  • Used for spatial queries and filtering
// NYC metropolitan area
let bounds = GeoBounds::new(
    40.4,   // min_lat (south)
    -74.3,  // min_lon (west)
    41.0,   // max_lat (north)
    -73.7   // max_lon (east)
);

// From a collection of locations
let bounds = GeoBounds::from_locations(&locations);

Temporal Concepts

TimeRange

A TimeRange represents a period between two timestamps:

  • Has a start and end timestamp
  • Supports duration calculations and overlap checks
// Explicit range
let range = TimeRange::new(start_timestamp, end_timestamp);

// From year/month
let year_2024 = TimeRange::year(2024);
let june_2024 = TimeRange::month(2024, 6);

Temporal Precision

Events can have different temporal precision:

PrecisionExampleUse Case
Year"2024"Historical events
Month"2024-01"Approximate dates
Day"2024-01-15"Calendar events
Hour"2024-01-15T10:00"Scheduled events
Minute"2024-01-15T10:30"Meetings
Second"2024-01-15T10:30:45"Precise logging

Graph Concepts

NarrativeGraph

Events can be connected into a directed graph:

  • Nodes = Events
  • Edges = Relationships between events

Edge Types

TypeMeaningAuto-Connect
TemporalA happens before Bconnect_temporal()
SpatialA is near Bconnect_spatial(km)
ThematicA and B share tagsconnect_thematic()
CausalA causes BManual
ReferenceA references BManual
let mut graph = NarrativeGraph::from_events(events);

// Automatic connections
graph.connect_temporal();     // Time sequence
graph.connect_spatial(10.0);  // Within 10km
graph.connect_thematic();     // Shared tags

// Manual connections
graph.connect(node1, node2, EdgeType::Causal);

Indexing Concepts

Spatial Index (R-tree)

For efficient geographic queries:

  • Bounding box queries: "Find all events in this region"
  • Radius queries: "Find all events within X km of this point"
  • Nearest neighbor: "Find the closest N events to this location"

Temporal Index (B-tree)

For efficient time-based queries:

  • Range queries: "Find all events between date A and date B"
  • Before/After: "Find all events before/after this time"
  • Ordering: "Iterate events in chronological order"

Spatiotemporal Index

Combines both for powerful queries:

  • "Find events in NYC during June 2024"
  • "Find nearest events to this location within the last hour"

Next Steps

  • Location - Deep dive into Location type
  • Timestamp - Deep dive into Timestamp type
  • Event - Deep dive into Event type

Core Types Overview

The core module provides the fundamental types for spatial narratives.

Type Hierarchy

spatial_narrative::core
├── Location         # Geographic point
├── Timestamp        # Point in time
├── Event            # Something that happened somewhere, somewhen
├── Narrative        # Collection of events
├── GeoBounds        # Geographic bounding box
├── TimeRange        # Temporal range
├── SourceRef        # Source attribution
└── EventId          # Unique event identifier

Quick Reference

Location

use spatial_narrative::core::Location;

// Simple
let loc = Location::new(40.7128, -74.0060);

// With builder
let loc = Location::builder()
    .lat(40.7128)
    .lon(-74.0060)
    .name("New York City")
    .build()
    .unwrap();

Timestamp

use spatial_narrative::core::Timestamp;

// Parse from string
let ts = Timestamp::parse("2024-01-15T10:30:00Z").unwrap();

// Current time
let now = Timestamp::now();

// From Unix timestamp
let ts = Timestamp::from_unix(1705315800).unwrap();

Event

use spatial_narrative::core::{Event, Location, Timestamp};

// Simple
let event = Event::new(
    Location::new(40.7128, -74.0060),
    Timestamp::now(),
    "Event description"
);

// With builder
let event = EventBuilder::new()
    .location(location)
    .timestamp(timestamp)
    .text("Description")
    .tag("category")
    .build()
    .unwrap();

Narrative

use spatial_narrative::core::NarrativeBuilder;

let narrative = NarrativeBuilder::new()
    .title("My Narrative")
    .description("A collection of events")
    .events(vec![event1, event2])
    .build();

Common Operations

Serialization

All core types implement Serialize and Deserialize:

use serde_json;

// Serialize
let json = serde_json::to_string(&event)?;

// Deserialize
let event: Event = serde_json::from_str(&json)?;

Comparison

Events, Timestamps, and Locations implement comparison traits:

// Timestamps are ordered
if timestamp1 < timestamp2 {
    println!("Event 1 happened first");
}

// Events can be compared by time
let events = events.into_iter()
    .sorted_by_key(|e| e.timestamp)
    .collect();

Validation

Types validate on construction:

// This will fail - invalid latitude
let result = Location::new(200.0, 0.0);  // Returns error

// Use builders for detailed error handling
match Location::builder().lat(200.0).lon(0.0).build() {
    Ok(loc) => println!("Valid"),
    Err(e) => println!("Error: {}", e),
}

Location

The Location type represents a geographic point on Earth.

Creating Locations

Simple Constructor

use spatial_narrative::core::Location;

// Latitude, Longitude (WGS84)
let nyc = Location::new(40.7128, -74.0060);
let tokyo = Location::new(35.6762, 139.6503);
let sydney = Location::new(-33.8688, 151.2093);

Builder Pattern

For locations with additional attributes:

let location = Location::builder()
    .lat(40.7484)
    .lon(-73.9857)
    .elevation(443.0)           // meters above sea level
    .uncertainty_meters(10.0)   // GPS accuracy
    .name("Empire State Building")
    .build()
    .unwrap();

From Tuple

let loc: Location = (40.7128, -74.0060).into();

Properties

PropertyTypeDescription
latf64Latitude (-90 to 90)
lonf64Longitude (-180 to 180)
elevationOption<f64>Meters above sea level
uncertainty_metersOption<f64>Location accuracy
nameOption<String>Human-readable name

Methods

distance_to

Calculate great-circle distance to another location:

let nyc = Location::new(40.7128, -74.0060);
let london = Location::new(51.5074, -0.1278);

let distance_km = nyc.distance_to(&london);
println!("NYC to London: {:.0} km", distance_km);  // ~5570 km

to_geo_point

Convert to geo crate's Point type:

use geo::Point;

let location = Location::new(40.7128, -74.0060);
let point: Point<f64> = location.to_geo_point();

Validation

Coordinates are validated on construction:

// Valid
Location::new(0.0, 0.0);       // Null Island
Location::new(90.0, 180.0);    // North Pole, Date Line
Location::new(-90.0, -180.0);  // South Pole, Date Line

// Invalid - will return Error
Location::builder().lat(91.0).lon(0.0).build();   // Lat out of range
Location::builder().lat(0.0).lon(181.0).build();  // Lon out of range

Serialization

use serde_json;

let loc = Location::new(40.7128, -74.0060);

// To JSON
let json = serde_json::to_string(&loc)?;
// {"lat":40.7128,"lon":-74.006,"elevation":null,"uncertainty_meters":null,"name":null}

// From JSON
let loc: Location = serde_json::from_str(&json)?;

Examples

City Locations

let cities = vec![
    Location::builder().lat(40.7128).lon(-74.0060).name("New York").build()?,
    Location::builder().lat(51.5074).lon(-0.1278).name("London").build()?,
    Location::builder().lat(35.6762).lon(139.6503).name("Tokyo").build()?,
    Location::builder().lat(-33.8688).lon(151.2093).name("Sydney").build()?,
];

With Elevation

// Mountain peaks
let everest = Location::builder()
    .lat(27.9881)
    .lon(86.9250)
    .elevation(8848.86)
    .name("Mount Everest")
    .build()?;

let denali = Location::builder()
    .lat(63.0692)
    .lon(-151.0070)
    .elevation(6190.5)
    .name("Denali")
    .build()?;

GPS Tracking

// GPS readings with uncertainty
let readings = vec![
    Location::builder()
        .lat(40.7128).lon(-74.0060)
        .uncertainty_meters(5.0)
        .build()?,
    Location::builder()
        .lat(40.7130).lon(-74.0058)
        .uncertainty_meters(3.0)
        .build()?,
];

Timestamp

The Timestamp type represents a point in time with timezone awareness and configurable precision.

Creating Timestamps

Parse from String

use spatial_narrative::core::Timestamp;

// Full ISO 8601
let ts = Timestamp::parse("2024-01-15T10:30:00Z")?;

// Date only
let ts = Timestamp::parse("2024-01-15")?;

// Year-month
let ts = Timestamp::parse("2024-01")?;

// Year only
let ts = Timestamp::parse("2024")?;

Current Time

let now = Timestamp::now();

From Unix Timestamp

// Seconds since epoch
let ts = Timestamp::from_unix(1705315800)?;

// Milliseconds since epoch
let ts = Timestamp::from_unix_millis(1705315800000)?;

Properties

PropertyTypeDescription
datetimeDateTime<Utc>Underlying chrono datetime
precisionTemporalPrecisionYear, Month, Day, Hour, Minute, Second

Temporal Precision

Timestamps track their precision level:

use spatial_narrative::core::TemporalPrecision;

let ts = Timestamp::parse("2024")?;
assert_eq!(ts.precision, TemporalPrecision::Year);

let ts = Timestamp::parse("2024-01-15")?;
assert_eq!(ts.precision, TemporalPrecision::Day);

let ts = Timestamp::parse("2024-01-15T10:30:00Z")?;
assert_eq!(ts.precision, TemporalPrecision::Second);

Methods

Formatting

let ts = Timestamp::parse("2024-01-15T10:30:00Z")?;

// RFC 3339 format
ts.to_rfc3339();  // "2024-01-15T10:30:00+00:00"

// Format with precision
ts.format_with_precision();  // Respects the timestamp's precision

Conversion

// To Unix timestamp
let seconds = ts.to_unix();
let millis = ts.to_unix_millis();

Comparison

let ts1 = Timestamp::parse("2024-01-01T00:00:00Z")?;
let ts2 = Timestamp::parse("2024-01-02T00:00:00Z")?;

if ts1 < ts2 {
    println!("ts1 is earlier");
}

// Duration between timestamps
let duration = ts2.duration_since(&ts1);
println!("Difference: {} hours", duration.num_hours());

Serialization

use serde_json;

let ts = Timestamp::parse("2024-01-15T10:30:00Z")?;

// To JSON (serializes as string)
let json = serde_json::to_string(&ts)?;
// "2024-01-15T10:30:00+00:00"

// From JSON
let ts: Timestamp = serde_json::from_str(&json)?;

Examples

Historical Events

// Historical dates with appropriate precision
let ww1_start = Timestamp::parse("1914-07-28")?;       // Day precision
let moon_landing = Timestamp::parse("1969-07-20T20:17:40Z")?;  // Second precision
let renaissance = Timestamp::parse("1400")?;            // Year precision

Time Ranges

use spatial_narrative::core::TimeRange;

let start = Timestamp::parse("2024-01-01T00:00:00Z")?;
let end = Timestamp::parse("2024-12-31T23:59:59Z")?;

let year_2024 = TimeRange::new(start, end);
println!("Duration: {} days", year_2024.duration().num_days());

Sorting Events

let mut events = vec![event3, event1, event2];

// Sort by timestamp
events.sort_by_key(|e| e.timestamp.clone());

// Now in chronological order
for event in &events {
    println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}

Event

The Event type represents something that happened at a specific place and time.

Creating Events

Simple Constructor

use spatial_narrative::core::{Event, Location, Timestamp};

let event = Event::new(
    Location::new(40.7128, -74.0060),
    Timestamp::parse("2024-01-15T10:00:00Z").unwrap(),
    "Conference begins"
);

Builder Pattern

use spatial_narrative::core::{EventBuilder, Location, Timestamp, SourceRef};

let event = EventBuilder::new()
    .location(Location::new(40.7128, -74.0060))
    .timestamp(Timestamp::parse("2024-01-15T10:00:00Z").unwrap())
    .text("Conference begins in Manhattan")
    .tag("conference")
    .tag("technology")
    .source(SourceRef::builder()
        .title("Event Calendar")
        .url("https://example.com/events")
        .build())
    .build()
    .unwrap();

Properties

PropertyTypeDescription
idEventIdUnique identifier (UUID)
locationLocationWhere it happened
timestampTimestampWhen it happened
textStringDescription
tagsHashSet<String>Categories/labels
sourceOption<SourceRef>Source attribution
metadataHashMap<String, Value>Custom metadata

Methods

Tags

// Add tags
event.add_tag("important");
event.add_tag("verified");

// Check tags
if event.has_tag("important") {
    println!("This is important!");
}

// Get all tags
for tag in &event.tags {
    println!("Tag: {}", tag);
}

Metadata

use serde_json::json;

// Add metadata
event.set_metadata("priority", json!(1));
event.set_metadata("verified", json!(true));

// Get metadata
if let Some(priority) = event.get_metadata("priority") {
    println!("Priority: {}", priority);
}

Traits

Events implement spatial and temporal traits:

use spatial_narrative::core::traits::{SpatialEntity, TemporalEntity};

// Spatial trait
let coords = event.coordinates();  // (lat, lon)
let bounds = event.bounds();       // GeoBounds

// Temporal trait  
let time = event.time();           // Timestamp
let range = event.time_range();    // TimeRange

Examples

News Event

let event = EventBuilder::new()
    .location(Location::builder()
        .lat(48.8566).lon(2.3522)
        .name("Paris, France")
        .build()?)
    .timestamp(Timestamp::parse("2024-07-14T10:00:00Z")?)
    .text("Bastille Day celebrations commence")
    .tag("celebration")
    .tag("national-holiday")
    .source(SourceRef::builder()
        .title("Le Monde")
        .source_type(SourceType::Article)
        .url("https://lemonde.fr/article/123")
        .build())
    .build()?;

Sensor Reading

let reading = EventBuilder::new()
    .location(Location::builder()
        .lat(37.7749).lon(-122.4194)
        .uncertainty_meters(1.0)
        .build()?)
    .timestamp(Timestamp::now())
    .text("Temperature reading")
    .tag("sensor")
    .tag("temperature")
    .build()?;

reading.set_metadata("temperature_c", json!(22.5));
reading.set_metadata("humidity_pct", json!(65));

Narrative

The Narrative type represents a collection of related events that form a coherent story.

Creating Narratives

Builder Pattern

use spatial_narrative::core::{NarrativeBuilder, Event};

let narrative = NarrativeBuilder::new()
    .title("Road Trip to Boston")
    .description("A day trip from NYC to Boston with stops along the way")
    .tag("travel")
    .tag("road-trip")
    .events(vec![event1, event2, event3])
    .build();

Empty Narrative

let mut narrative = NarrativeBuilder::new()
    .title("My Story")
    .build();

// Add events later
narrative.add_event(event1);
narrative.add_event(event2);

Properties

PropertyTypeDescription
idNarrativeIdUnique identifier
titleOption<String>Narrative title
descriptionOption<String>Description
eventsVec<Event>Collection of events
tagsHashSet<String>Categories/labels
metadataNarrativeMetadataAdditional metadata

Methods

Events

// Add events
narrative.add_event(event);

// Get event count
println!("Events: {}", narrative.events.len());

// Iterate events
for event in &narrative.events {
    println!("{}", event.text);
}

Chronological Order

// Get events sorted by time
let ordered = narrative.events_chronological();

for event in ordered {
    println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}

Time Range

// Get overall time span
if let Some(range) = narrative.time_range() {
    println!("Start: {}", range.start.to_rfc3339());
    println!("End: {}", range.end.to_rfc3339());
    println!("Duration: {} days", range.duration().num_days());
}

Geographic Bounds

// Get bounding box of all events
if let Some(bounds) = narrative.bounds() {
    println!("Lat: {} to {}", bounds.min_lat, bounds.max_lat);
    println!("Lon: {} to {}", bounds.min_lon, bounds.max_lon);
    
    let (center_lat, center_lon) = bounds.center();
    println!("Center: ({}, {})", center_lat, center_lon);
}

Filtering

// Filter by geographic bounds
let paris_events = narrative.filter_spatial(&paris_bounds);

// Filter by time range
let june_events = narrative.filter_temporal(&june_2024);

Examples

Historical Timeline

let ww1 = NarrativeBuilder::new()
    .title("World War I Timeline")
    .description("Key events of the Great War")
    .tag("history")
    .tag("world-war")
    .events(vec![
        Event::new(
            Location::new(43.8563, 18.4131),  // Sarajevo
            Timestamp::parse("1914-06-28")?,
            "Assassination of Archduke Franz Ferdinand"
        ),
        Event::new(
            Location::new(48.8566, 2.3522),   // Paris
            Timestamp::parse("1919-06-28")?,
            "Treaty of Versailles signed"
        ),
    ])
    .build();

println!("Timeline: {}", ww1.title.as_deref().unwrap_or("Untitled"));
println!("Duration: {} years", ww1.time_range().unwrap().duration().num_days() / 365);

Travel Journal

let trip = NarrativeBuilder::new()
    .title("European Adventure 2024")
    .tag("travel")
    .tag("europe")
    .events(cities_visited)
    .build();

// Get geographic extent
if let Some(bounds) = trip.bounds() {
    println!("Trip covered {} degrees of latitude", 
        bounds.max_lat - bounds.min_lat);
}

Bounds

The core module provides types for representing geographic and temporal bounds.

GeoBounds

Represents a rectangular geographic region.

Creating GeoBounds

use spatial_narrative::core::GeoBounds;

// Explicit bounds
let bounds = GeoBounds::new(
    40.4,   // min_lat (south)
    -74.3,  // min_lon (west)
    41.0,   // max_lat (north)
    -73.7   // max_lon (east)
);

// From locations
let bounds = GeoBounds::from_locations(&[
    Location::new(40.7128, -74.0060),  // NYC
    Location::new(42.3601, -71.0589),  // Boston
]);

Methods

// Check if a location is within bounds
if bounds.contains(40.75, -73.99) {
    println!("Location is within bounds");
}

// Get center point
let (lat, lon) = bounds.center();

// Check intersection
if bounds1.intersects(&bounds2) {
    println!("Regions overlap");
}

TimeRange

Represents a period between two timestamps.

Creating TimeRange

use spatial_narrative::core::{TimeRange, Timestamp};

// From timestamps
let range = TimeRange::new(
    Timestamp::parse("2024-01-01T00:00:00Z")?,
    Timestamp::parse("2024-12-31T23:59:59Z")?
);

// Helper methods
let year_2024 = TimeRange::year(2024);
let june_2024 = TimeRange::month(2024, 6);

Methods

// Get duration
let days = range.duration().num_days();

// Check if timestamp is within range
if range.contains(&timestamp) {
    println!("Time is within range");
}

// Check overlap
if range1.overlaps(&range2) {
    println!("Ranges overlap");
}

Examples

Regional Analysis

// Define regions
let east_coast = GeoBounds::new(25.0, -82.0, 45.0, -66.0);
let west_coast = GeoBounds::new(32.0, -125.0, 49.0, -114.0);

// Filter events by region
let east_events = narrative.filter_spatial(&east_coast);
let west_events = narrative.filter_spatial(&west_coast);

Temporal Analysis

// Define periods
let q1_2024 = TimeRange::new(
    Timestamp::parse("2024-01-01")?,
    Timestamp::parse("2024-03-31")?
);

// Filter events by period
let q1_events = narrative.filter_temporal(&q1_2024);

Sources

The SourceRef type provides source attribution for events.

Creating Source References

use spatial_narrative::core::{SourceRef, SourceType};

let source = SourceRef::builder()
    .title("The New York Times")
    .source_type(SourceType::Article)
    .url("https://nytimes.com/article/123")
    .author("Jane Reporter")
    .date("2024-01-15")
    .build();

Source Types

TypeDescription
ArticleNews article or blog post
ReportOfficial report or document
WitnessEyewitness account
SensorAutomated sensor data
ArchiveHistorical archive
OtherOther source type

Properties

PropertyTypeDescription
source_typeSourceTypeCategory of source
titleOption<String>Source title
urlOption<String>URL reference
authorOption<String>Author/creator
dateOption<String>Publication date
notesOption<String>Additional notes

Examples

News Article

let source = SourceRef::builder()
    .source_type(SourceType::Article)
    .title("Breaking: Event Occurs")
    .url("https://news.example.com/article")
    .author("John Journalist")
    .date("2024-01-15")
    .build();

Sensor Data

let source = SourceRef::builder()
    .source_type(SourceType::Sensor)
    .title("Weather Station #42")
    .notes("Automated reading every 5 minutes")
    .build();

Historical Archive

let source = SourceRef::builder()
    .source_type(SourceType::Archive)
    .title("National Archives Collection")
    .url("https://archives.gov/document/123")
    .notes("Declassified 2020")
    .build();

I/O Overview

The io module provides import and export functionality for narratives in various formats.

Supported Formats

FormatTypeBest For
JSONNativeFull fidelity, all metadata preserved
GeoJSONStandardWeb mapping, GIS tools, Leaflet/Mapbox
CSVTabularSpreadsheets, data analysis, pandas

The Format Trait

All formats implement the Format trait:

use spatial_narrative::io::Format;

pub trait Format {
    fn import<R: Read>(&self, reader: &mut R) -> Result<Narrative>;
    fn export<W: Write>(&self, narrative: &Narrative, writer: &mut W) -> Result<()>;
    fn import_str(&self, s: &str) -> Result<Narrative>;
    fn export_str(&self, narrative: &Narrative) -> Result<String>;
}

Quick Examples

Export to Multiple Formats

use spatial_narrative::io::{GeoJsonFormat, CsvFormat, JsonFormat, Format};
use spatial_narrative::core::{Narrative, NarrativeBuilder, Event, Location, Timestamp};

let narrative = NarrativeBuilder::new()
    .title("My Story")
    .event(Event::new(
        Location::new(40.7128, -74.0060),
        Timestamp::now(),
        "Something happened"
    ))
    .build();

// Export to GeoJSON (for web maps)
let geojson = GeoJsonFormat::new().export_str(&narrative)?;

// Export to CSV (for spreadsheets)
let csv = CsvFormat::new().export_str(&narrative)?;

// Export to JSON (for full fidelity)
let json = JsonFormat::new().export_str(&narrative)?;

Import from File

use std::fs::File;
use std::io::BufReader;

// Import from GeoJSON file
let file = File::open("data.geojson")?;
let mut reader = BufReader::new(file);
let narrative = GeoJsonFormat::new().import(&mut reader)?;

println!("Loaded {} events", narrative.events.len());

Export to File

use std::fs::File;
use std::io::BufWriter;

// Export to CSV file
let file = File::create("output.csv")?;
let mut writer = BufWriter::new(file);
CsvFormat::new().export(&narrative, &mut writer)?;

Format Comparison

FeatureJSONGeoJSONCSV
All metadata⚠️ Partial⚠️ Limited
Tags
Sources⚠️ Optional
Custom metadata⚠️ Properties
Human readable⚠️
GIS compatible⚠️
Spreadsheet ready
File sizeMediumLargeSmall

Round-Trip Fidelity

For lossless round-trips, use JsonFormat:

let json = JsonFormat::new();

// Export
let exported = json.export_str(&narrative)?;

// Import
let imported = json.import_str(&exported)?;

// Verify
assert_eq!(narrative.events.len(), imported.events.len());

Next Steps

JSON Format

The native JSON format provides full fidelity for narratives, preserving all metadata.

Basic Usage

use spatial_narrative::io::{JsonFormat, Format};
use spatial_narrative::core::{Narrative, NarrativeBuilder, Event, Location, Timestamp};

let narrative = NarrativeBuilder::new()
    .title("My Narrative")
    .event(Event::new(
        Location::new(40.7128, -74.0060),
        Timestamp::now(),
        "Event description"
    ))
    .build();

// Export
let json = JsonFormat::new().export_str(&narrative)?;

// Import
let imported = JsonFormat::new().import_str(&json)?;

Pretty Printing

For human-readable output:

let format = JsonFormat::pretty();
let json = format.export_str(&narrative)?;

// Output is indented and formatted
println!("{}", json);

Output Structure

The JSON format produces:

{
  "version": "1.0",
  "narrative": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "title": "My Narrative",
    "description": null,
    "author": null,
    "created": "2024-01-15T10:00:00Z",
    "modified": "2024-01-15T10:00:00Z",
    "tags": ["example"],
    "metadata": {}
  },
  "events": [
    {
      "id": "660e8400-e29b-41d4-a716-446655440001",
      "location": {
        "lat": 40.7128,
        "lon": -74.006,
        "elevation": null,
        "uncertainty_meters": null,
        "name": "New York City"
      },
      "timestamp": "2024-01-15T10:00:00Z",
      "text": "Event description",
      "tags": ["conference"],
      "source": {
        "title": "Source Name",
        "source_type": "article",
        "url": "https://example.com"
      },
      "metadata": {}
    }
  ]
}

Version Compatibility

The format includes version information for forward compatibility:

// Current version is "1.0"
// Future versions will maintain backward compatibility

let json = r#"{"version": "1.0", "narrative": {...}, "events": [...]}"#;
let narrative = JsonFormat::new().import_str(json)?;

File Operations

Export to File

use std::fs::File;
use std::io::BufWriter;

let file = File::create("narrative.json")?;
let mut writer = BufWriter::new(file);
JsonFormat::pretty().export(&narrative, &mut writer)?;

Import from File

use std::fs::File;
use std::io::BufReader;

let file = File::open("narrative.json")?;
let mut reader = BufReader::new(file);
let narrative = JsonFormat::new().import(&mut reader)?;

When to Use JSON

Use JSON when:

  • Archiving narratives for later use
  • Transferring between systems using this library
  • You need to preserve all metadata
  • Round-trip fidelity is important

Consider other formats when:

  • Integrating with GIS tools → use GeoJSON
  • Importing to spreadsheets → use CSV
  • Minimizing file size is critical

Metadata Preservation

JSON preserves all custom metadata:

use serde_json::json;

let mut event = Event::new(location, timestamp, "Description");
event.set_metadata("custom_field", json!({"nested": "data"}));
event.set_metadata("priority", json!(1));

// All metadata is preserved in JSON export
let json = JsonFormat::new().export_str(&narrative)?;
let imported = JsonFormat::new().import_str(&json)?;

assert_eq!(
    imported.events[0].get_metadata("custom_field"),
    Some(&json!({"nested": "data"}))
);

GeoJSON Format

Export narratives as GeoJSON FeatureCollections for use with mapping libraries.

Basic Usage

use spatial_narrative::io::{GeoJsonFormat, Format};
use spatial_narrative::core::{Narrative, Event, Location, Timestamp};

// Export to GeoJSON
let geojson = GeoJsonFormat::new().export_str(&narrative)?;

// Import from GeoJSON
let narrative = GeoJsonFormat::new().import_str(&geojson)?;

Output Structure

Events are exported as Point features:

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [-74.006, 40.7128]
      },
      "properties": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "timestamp": "2024-01-15T10:00:00Z",
        "text": "Event description",
        "tags": ["conference", "technology"]
      }
    }
  ]
}

Note: GeoJSON uses [longitude, latitude] order per the specification.

Configuration Options

Customize the export with GeoJsonOptions:

use spatial_narrative::io::{GeoJsonFormat, GeoJsonOptions};

let options = GeoJsonOptions {
    include_ids: true,           // Include event IDs
    include_tags: true,          // Include tags array
    include_sources: true,       // Include source information
    timestamp_property: "time".to_string(),      // Property name for timestamp
    text_property: "description".to_string(),    // Property name for text
};

let format = GeoJsonFormat::with_options(options);
let geojson = format.export_str(&narrative)?;

Default Options

GeoJsonOptions {
    include_ids: true,
    include_tags: true,
    include_sources: false,
    timestamp_property: "timestamp".to_string(),
    text_property: "text".to_string(),
}

Web Mapping Integration

Leaflet

// Load exported GeoJSON
fetch('narrative.geojson')
  .then(res => res.json())
  .then(geojson => {
    L.geoJSON(geojson, {
      pointToLayer: (feature, latlng) => {
        return L.circleMarker(latlng, {
          radius: 8,
          fillColor: '#ff7800',
          color: '#000',
          weight: 1,
          opacity: 1,
          fillOpacity: 0.8
        });
      },
      onEachFeature: (feature, layer) => {
        layer.bindPopup(`
          <b>${feature.properties.text}</b><br>
          ${feature.properties.timestamp}
        `);
      }
    }).addTo(map);
  });

Mapbox GL JS

map.on('load', () => {
  map.addSource('narrative', {
    type: 'geojson',
    data: 'narrative.geojson'
  });
  
  map.addLayer({
    id: 'events',
    type: 'circle',
    source: 'narrative',
    paint: {
      'circle-radius': 8,
      'circle-color': '#ff7800'
    }
  });
});

Importing GeoJSON

Import existing GeoJSON data:

let geojson = r#"{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {"type": "Point", "coordinates": [-74.006, 40.7128]},
      "properties": {
        "timestamp": "2024-01-15T10:00:00Z",
        "text": "Event description"
      }
    }
  ]
}"#;

let narrative = GeoJsonFormat::new().import_str(geojson)?;
println!("Imported {} events", narrative.events.len());

Property Mapping

The importer looks for these properties:

PropertyMaps ToRequired
timestamp, time, datetimeEvent.timestampYes
text, description, nameEvent.textNo
tagsEvent.tagsNo
idEvent.idNo (generates UUID)

File Operations

use std::fs::File;
use std::io::{BufReader, BufWriter};

// Export
let file = File::create("events.geojson")?;
GeoJsonFormat::new().export(&narrative, &mut BufWriter::new(file))?;

// Import
let file = File::open("events.geojson")?;
let narrative = GeoJsonFormat::new().import(&mut BufReader::new(file))?;

When to Use GeoJSON

Use GeoJSON when:

  • Displaying events on web maps (Leaflet, Mapbox, Google Maps)
  • Importing into GIS software (QGIS, ArcGIS)
  • Sharing with systems that expect standard geographic formats
  • Building map visualizations

Consider other formats when:

  • You need to preserve all metadata → use JSON
  • Importing to spreadsheets → use CSV

CSV Format

Export narratives as CSV for spreadsheet analysis and data science workflows.

Basic Usage

use spatial_narrative::io::{CsvFormat, Format};

// Export to CSV
let csv = CsvFormat::new().export_str(&narrative)?;

// Import from CSV
let narrative = CsvFormat::new().import_str(&csv)?;

Output Structure

Default CSV output:

latitude,longitude,timestamp,text,tags
40.7128,-74.006,2024-01-15T10:00:00Z,"Conference begins","conference,technology"
40.758,-73.9855,2024-01-15T14:00:00Z,"Press conference","press,media"

Configuration Options

Customize column names and delimiter:

use spatial_narrative::io::{CsvFormat, CsvOptions};

let options = CsvOptions {
    lat_column: "lat".to_string(),
    lon_column: "lng".to_string(),
    timestamp_column: "datetime".to_string(),
    text_column: Some("description".to_string()),
    tags_column: Some("categories".to_string()),
    elevation_column: Some("altitude".to_string()),
    source_title_column: None,
    source_url_column: None,
    delimiter: b',',
};

let format = CsvFormat::with_options(options);
let csv = format.export_str(&narrative)?;

Column Options

OptionDefaultDescription
lat_column"latitude"Latitude column name
lon_column"longitude"Longitude column name
timestamp_column"timestamp"Timestamp column name
text_columnSome("text")Event text column
tags_columnSome("tags")Tags column (comma-separated)
elevation_columnNoneElevation column
source_title_columnNoneSource title column
source_url_columnNoneSource URL column
delimiterb','Field delimiter

TSV (Tab-Separated)

let options = CsvOptions {
    delimiter: b'\t',
    ..Default::default()
};

let tsv = CsvFormat::with_options(options).export_str(&narrative)?;

Importing CSV

Import from existing CSV data:

let csv = r#"lat,lon,time,description
40.7128,-74.006,2024-01-15T10:00:00Z,Conference begins
40.758,-73.9855,2024-01-15T14:00:00Z,Press conference"#;

let options = CsvOptions {
    lat_column: "lat".to_string(),
    lon_column: "lon".to_string(),
    timestamp_column: "time".to_string(),
    text_column: Some("description".to_string()),
    ..Default::default()
};

let narrative = CsvFormat::with_options(options).import_str(csv)?;

Handling Tags

Tags are stored as comma-separated values:

// Export: tags become "tag1,tag2,tag3"
// Import: "tag1,tag2,tag3" becomes HashSet{"tag1", "tag2", "tag3"}

let csv = "latitude,longitude,timestamp,text,tags
40.7128,-74.006,2024-01-15T10:00:00Z,Event,\"conference,technology,important\"";

let narrative = CsvFormat::new().import_str(csv)?;
assert!(narrative.events[0].has_tag("conference"));

File Operations

use std::fs::File;
use std::io::{BufReader, BufWriter};

// Export to file
let file = File::create("events.csv")?;
CsvFormat::new().export(&narrative, &mut BufWriter::new(file))?;

// Import from file
let file = File::open("events.csv")?;
let narrative = CsvFormat::new().import(&mut BufReader::new(file))?;

Integration with Data Tools

Python/Pandas

import pandas as pd

# Read exported CSV
df = pd.read_csv('events.csv', parse_dates=['timestamp'])

# Analyze
print(f"Events: {len(df)}")
print(f"Date range: {df['timestamp'].min()} to {df['timestamp'].max()}")
print(f"Locations: {df[['latitude', 'longitude']].nunique()}")

Excel

CSV files open directly in Excel. For best results:

  • Use UTF-8 encoding
  • Quote text fields containing commas
  • Use ISO 8601 timestamps

R

library(tidyverse)

events <- read_csv("events.csv")

# Plot on map
library(sf)
events_sf <- st_as_sf(events, coords = c("longitude", "latitude"), crs = 4326)
plot(events_sf)

When to Use CSV

Use CSV when:

  • Analyzing data in spreadsheets (Excel, Google Sheets)
  • Using with data science tools (pandas, R)
  • Simple data exchange with minimal overhead
  • Data pipelines that expect tabular formats

Consider other formats when:

  • You need complex metadata → use JSON
  • Mapping in web applications → use GeoJSON
  • Source attribution is important → use JSON

Limitations

CSV has some inherent limitations:

FeatureSupport
Location coordinates✅ Full
Timestamps✅ Full
Event text✅ Full
Tags✅ As comma-separated string
Basic source info⚠️ Optional columns
Custom metadata❌ Not supported
Nested data❌ Not supported

Custom Formats

Implement the Format trait to create custom import/export formats.

The Format Trait

use spatial_narrative::io::Format;
use spatial_narrative::core::Narrative;
use spatial_narrative::error::Result;
use std::io::{Read, Write};

pub trait Format {
    /// Import a narrative from a reader
    fn import<R: Read>(&self, reader: &mut R) -> Result<Narrative>;
    
    /// Export a narrative to a writer
    fn export<W: Write>(&self, narrative: &Narrative, writer: &mut W) -> Result<()>;
    
    /// Import from a string (has default implementation)
    fn import_str(&self, s: &str) -> Result<Narrative> {
        self.import(&mut s.as_bytes())
    }
    
    /// Export to a string (has default implementation)
    fn export_str(&self, narrative: &Narrative) -> Result<String> {
        let mut buf = Vec::new();
        self.export(narrative, &mut buf)?;
        Ok(String::from_utf8(buf)?)
    }
}

Example: XML Format

use spatial_narrative::io::Format;
use spatial_narrative::core::{Narrative, NarrativeBuilder, Event, Location, Timestamp};
use spatial_narrative::error::{Error, Result};
use std::io::{Read, Write};

pub struct XmlFormat;

impl XmlFormat {
    pub fn new() -> Self {
        Self
    }
}

impl Format for XmlFormat {
    fn import<R: Read>(&self, reader: &mut R) -> Result<Narrative> {
        let mut content = String::new();
        reader.read_to_string(&mut content)?;
        
        // Parse XML (using quick-xml or similar)
        // This is a simplified example
        let mut builder = NarrativeBuilder::new();
        
        // ... parse events from XML ...
        
        Ok(builder.build())
    }
    
    fn export<W: Write>(&self, narrative: &Narrative, writer: &mut W) -> Result<()> {
        writeln!(writer, r#"<?xml version="1.0" encoding="UTF-8"?>"#)?;
        writeln!(writer, "<narrative>")?;
        
        if let Some(title) = &narrative.title {
            writeln!(writer, "  <title>{}</title>", escape_xml(title))?;
        }
        
        writeln!(writer, "  <events>")?;
        for event in &narrative.events {
            writeln!(writer, "    <event>")?;
            writeln!(writer, "      <lat>{}</lat>", event.location.lat)?;
            writeln!(writer, "      <lon>{}</lon>", event.location.lon)?;
            writeln!(writer, "      <timestamp>{}</timestamp>", 
                event.timestamp.to_rfc3339())?;
            writeln!(writer, "      <text>{}</text>", escape_xml(&event.text))?;
            writeln!(writer, "    </event>")?;
        }
        writeln!(writer, "  </events>")?;
        writeln!(writer, "</narrative>")?;
        
        Ok(())
    }
}

fn escape_xml(s: &str) -> String {
    s.replace('&', "&amp;")
     .replace('<', "&lt;")
     .replace('>', "&gt;")
     .replace('"', "&quot;")
}

Example: KML Format

For Google Earth compatibility:

pub struct KmlFormat;

impl Format for KmlFormat {
    fn export<W: Write>(&self, narrative: &Narrative, writer: &mut W) -> Result<()> {
        writeln!(writer, r#"<?xml version="1.0" encoding="UTF-8"?>"#)?;
        writeln!(writer, r#"<kml xmlns="http://www.opengis.net/kml/2.2">"#)?;
        writeln!(writer, "<Document>")?;
        
        if let Some(title) = &narrative.title {
            writeln!(writer, "  <name>{}</name>", title)?;
        }
        
        for event in &narrative.events {
            writeln!(writer, "  <Placemark>")?;
            writeln!(writer, "    <name>{}</name>", escape_xml(&event.text))?;
            writeln!(writer, "    <TimeStamp>")?;
            writeln!(writer, "      <when>{}</when>", event.timestamp.to_rfc3339())?;
            writeln!(writer, "    </TimeStamp>")?;
            writeln!(writer, "    <Point>")?;
            writeln!(writer, "      <coordinates>{},{}</coordinates>",
                event.location.lon, event.location.lat)?;
            writeln!(writer, "    </Point>")?;
            writeln!(writer, "  </Placemark>")?;
        }
        
        writeln!(writer, "</Document>")?;
        writeln!(writer, "</kml>")?;
        Ok(())
    }
    
    fn import<R: Read>(&self, _reader: &mut R) -> Result<Narrative> {
        // KML import implementation
        todo!("KML import not yet implemented")
    }
}

Using Custom Formats

let narrative = create_narrative();

// Use custom XML format
let xml_format = XmlFormat::new();
let xml = xml_format.export_str(&narrative)?;

// Use custom KML format
let kml_format = KmlFormat::new();
let kml = kml_format.export_str(&narrative)?;

// Save to file
let mut file = File::create("narrative.kml")?;
kml_format.export(&narrative, &mut file)?;

Best Practices

  1. Handle errors gracefully: Return Result with descriptive errors
  2. Use buffered I/O: Wrap readers/writers in BufReader/BufWriter
  3. Support round-trips: Ensure import(export(n)) == n where possible
  4. Document limitations: Note what metadata is lost in conversion
  5. Validate on import: Check for required fields and valid values

Error Handling

Use the library's error types:

use spatial_narrative::error::{Error, Result};

impl Format for MyFormat {
    fn import<R: Read>(&self, reader: &mut R) -> Result<Narrative> {
        let mut content = String::new();
        reader.read_to_string(&mut content)?;
        
        if content.is_empty() {
            return Err(Error::Parse("Empty input".to_string()));
        }
        
        // Parse and validate...
        
        Ok(narrative)
    }
}

Indexing Overview

The index module provides efficient data structures for spatial and temporal queries.

Index Types

IndexData StructureBest For
SpatialIndexR-treeGeographic queries (bbox, radius, k-nearest)
TemporalIndexB-treeTime range queries
SpatiotemporalIndexCombinedSpace + time queries together

Quick Example

use spatial_narrative::index::{SpatialIndex, TemporalIndex, SpatiotemporalIndex};
use spatial_narrative::core::{Event, Location, Timestamp, GeoBounds, TimeRange};

// Create events
let events = vec![
    Event::new(Location::new(40.7128, -74.0060), 
        Timestamp::parse("2024-01-15T10:00:00Z").unwrap(), "NYC Event"),
    Event::new(Location::new(34.0522, -118.2437), 
        Timestamp::parse("2024-01-15T14:00:00Z").unwrap(), "LA Event"),
];

// Spatial-only queries
let mut spatial = SpatialIndex::new();
for e in &events {
    spatial.insert(e.clone(), &e.location);
}
let nearby = spatial.query_radius(40.7, -74.0, 0.5);

// Temporal-only queries
let mut temporal = TemporalIndex::new();
for e in &events {
    temporal.insert(e.clone(), &e.timestamp);
}
let range = TimeRange::new(
    Timestamp::parse("2024-01-15T09:00:00Z").unwrap(),
    Timestamp::parse("2024-01-15T12:00:00Z").unwrap(),
);
let morning = temporal.query_range(&range);

// Combined queries
let mut combined = SpatiotemporalIndex::new();
for e in &events {
    combined.insert(e.clone(), &e.location, &e.timestamp);
}
let bounds = GeoBounds::new(40.0, -75.0, 41.0, -73.0);
let results = combined.query(&bounds, &range);

Performance Characteristics

OperationSpatialIndexTemporalIndexSpatiotemporalIndex
InsertO(log n)O(log n)O(log n)
Point queryO(log n)O(log n)O(log n)
Range queryO(log n + k)O(log n + k)O(log n + k)
K-nearestO(k log n)--

Where n is the number of items and k is the number of results.

Choosing an Index

Use SpatialIndex when:

  • You only care about location
  • Finding events near a point
  • Bounding box searches
  • K-nearest neighbor queries

Use TemporalIndex when:

  • You only care about time
  • Finding events in a time range
  • Before/after queries
  • Chronological iteration

Use SpatiotemporalIndex when:

  • You need to filter by both space AND time
  • Events have meaningful locations and timestamps
  • Generating heatmaps

Generic Types

All indexes are generic over the stored item type:

// Index of events
let event_index: SpatialIndex<Event> = SpatialIndex::new();

// Index of custom structs
struct MyData { name: String, location: Location }
let my_index: SpatialIndex<MyData> = SpatialIndex::new();

// Index of references
let ref_index: SpatialIndex<&Event> = SpatialIndex::new();

// Index of IDs (lookup in separate collection)
let id_index: SpatialIndex<usize> = SpatialIndex::new();

Building from Iterators

Efficiently build indexes from collections:

// From events with extractors
let spatial = SpatialIndex::from_iter(
    events.iter().cloned(),
    |e| &e.location
);

let temporal = TemporalIndex::from_iter(
    events.iter().cloned(),
    |e| &e.timestamp
);

let combined = SpatiotemporalIndex::from_iter(
    events.iter().cloned(),
    |e| &e.location,
    |e| &e.timestamp
);

Next Steps

Spatial Index (R-tree)

The SpatialIndex uses an R-tree for efficient geographic queries.

Creating an Index

use spatial_narrative::index::SpatialIndex;
use spatial_narrative::core::{Event, Location, Timestamp};

// Empty index
let mut index: SpatialIndex<Event> = SpatialIndex::new();

// Insert items
let event = Event::new(
    Location::new(40.7128, -74.0060),
    Timestamp::now(),
    "NYC Event"
);
index.insert(event, &Location::new(40.7128, -74.0060));

From Iterator

Build an index efficiently from a collection:

let index = SpatialIndex::from_iter(
    events.iter().cloned(),
    |event| &event.location
);

Query Types

Bounding Box Query

Find all items within a rectangular region:

// Query by lat/lon bounds
let results = index.query_bbox(
    40.0,   // min_lat
    -75.0,  // min_lon
    41.0,   // max_lat
    -73.0   // max_lon
);

println!("Found {} events in region", results.len());

Using GeoBounds

use spatial_narrative::core::GeoBounds;

let nyc_area = GeoBounds::new(40.4, -74.3, 41.0, -73.7);
let results = index.query_bounds(&nyc_area);

Radius Query (Degrees)

Find items within a radius (in degrees):

// Approximate radius query using degrees
// Note: This is Euclidean distance in degrees, not great-circle distance
let results = index.query_radius(
    40.7128,  // center lat
    -74.0060, // center lon
    0.1       // radius in degrees (~11km at this latitude)
);

Radius Query (Meters)

For accurate distance-based queries:

// Great-circle distance using Haversine formula
let results = index.query_radius_meters(
    40.7128,  // center lat
    -74.0060, // center lon
    5000.0    // radius in meters (5km)
);

K-Nearest Neighbors

Find the K closest items to a point:

// Find 10 nearest events
let nearest = index.nearest(40.7128, -74.0060, 10);

for event in nearest {
    println!("Near event: {}", event.text);
}

Methods Reference

MethodDescription
new()Create empty index
from_iter()Build from iterator with location extractor
insert(item, location)Add an item
query_bbox(min_lat, min_lon, max_lat, max_lon)Bounding box query
query_bounds(bounds)Query using GeoBounds
query_radius(lat, lon, radius_deg)Radius query (degrees)
query_radius_meters(lat, lon, radius_m)Radius query (meters)
nearest(lat, lon, k)K-nearest neighbors
len()Number of items
is_empty()Check if empty

Performance Tips

Bulk Loading

For large datasets, use from_iter instead of repeated insert:

// Efficient: bulk load
let index = SpatialIndex::from_iter(events, |e| &e.location);

// Less efficient: repeated inserts
let mut index = SpatialIndex::new();
for event in events {
    index.insert(event.clone(), &event.location);  // O(log n) each
}

Query Optimization

Start with broad queries, then filter:

// Get candidates from index
let candidates = index.query_bbox(39.0, -76.0, 42.0, -72.0);

// Apply additional filters
let filtered: Vec<_> = candidates
    .into_iter()
    .filter(|e| e.has_tag("important"))
    .filter(|e| e.timestamp.year() == 2024)
    .collect();

Use Cases

Finding Nearby Events

let my_location = Location::new(40.7580, -73.9855);  // Times Square

// Find events within 1km
let nearby = index.query_radius_meters(
    my_location.lat, 
    my_location.lon, 
    1000.0
);

println!("Found {} events within 1km", nearby.len());

Regional Analysis

// Define regions
let regions = vec![
    ("NYC", GeoBounds::new(40.4, -74.3, 41.0, -73.7)),
    ("LA", GeoBounds::new(33.7, -118.7, 34.4, -117.9)),
    ("Chicago", GeoBounds::new(41.6, -88.0, 42.1, -87.5)),
];

// Count events per region
for (name, bounds) in &regions {
    let count = index.query_bounds(bounds).len();
    println!("{}: {} events", name, count);
}

Clustering Preprocessing

// Use spatial index to speed up DBSCAN-style clustering
let candidates = index.query_radius_meters(point.lat, point.lon, eps);

// Only check distances for nearby points
for candidate in candidates {
    let distance = haversine_distance(&point, &candidate.location);
    if distance <= eps {
        // Add to cluster
    }
}

Temporal Index (B-tree)

The TemporalIndex uses a B-tree for efficient time-based queries.

Creating an Index

use spatial_narrative::index::TemporalIndex;
use spatial_narrative::core::{Event, Location, Timestamp};

// Empty index
let mut index: TemporalIndex<Event> = TemporalIndex::new();

// Insert items
let event = Event::new(
    Location::new(40.7128, -74.0060),
    Timestamp::parse("2024-01-15T10:00:00Z").unwrap(),
    "Morning meeting"
);
index.insert(event, &Timestamp::parse("2024-01-15T10:00:00Z").unwrap());

From Iterator

let index = TemporalIndex::from_iter(
    events.iter().cloned(),
    |event| &event.timestamp
);

Query Types

Time Range Query

Find items within a time range:

use spatial_narrative::core::TimeRange;

let range = TimeRange::new(
    Timestamp::parse("2024-01-01T00:00:00Z").unwrap(),
    Timestamp::parse("2024-01-31T23:59:59Z").unwrap(),
);

let january_events = index.query_range(&range);
println!("Found {} events in January", january_events.len());

Convenience Ranges

// Entire year
let results = index.query_range(&TimeRange::year(2024));

// Specific month
let results = index.query_range(&TimeRange::month(2024, 6));  // June 2024

// Specific day
let results = index.query_range(&TimeRange::day(2024, 7, 4));  // July 4th

Before/After Queries

let cutoff = Timestamp::parse("2024-06-01T00:00:00Z").unwrap();

// All events before June
let before = index.before(&cutoff);

// All events after June
let after = index.after(&cutoff);

// Including the cutoff time
let at_or_before = index.at_or_before(&cutoff);
let at_or_after = index.at_or_after(&cutoff);

First and Last

// Get earliest event
if let Some(first) = index.first() {
    println!("First event: {}", first.text);
}

// Get latest event
if let Some(last) = index.last() {
    println!("Last event: {}", last.text);
}

Chronological Iteration

// Iterate in time order
for event in index.chronological() {
    println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}

Reverse Chronological

// Most recent first
for event in index.reverse_chronological() {
    println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}

Sliding Window

Iterate with a sliding time window:

use spatial_narrative::index::SlidingWindowIter;
use std::time::Duration;

let window_size = Duration::from_secs(3600);  // 1 hour
let step = Duration::from_secs(1800);         // 30 minute steps

for window in index.sliding_window(window_size, step) {
    println!("Window {}: {} events", 
        window.start.to_rfc3339(),
        window.events.len());
}

Methods Reference

MethodDescription
new()Create empty index
from_iter()Build from iterator
insert(item, timestamp)Add an item
query_range(range)Query time range
before(timestamp)Items before (exclusive)
after(timestamp)Items after (exclusive)
at_or_before(timestamp)Items at or before (inclusive)
at_or_after(timestamp)Items at or after (inclusive)
first()Earliest item
last()Latest item
chronological()Time-ordered iterator
reverse_chronological()Reverse time iterator
sliding_window(size, step)Sliding window iterator
time_range()Get overall time range
len()Number of items

Use Cases

Timeline Visualization

// Get events in order for display
let timeline: Vec<_> = index.chronological().collect();

for event in timeline {
    println!("{}: {}", 
        event.timestamp.format("%Y-%m-%d %H:%M"),
        event.text);
}

Activity Analysis

use std::time::Duration;

// Count events per hour
let hour = Duration::from_secs(3600);
let mut hourly_counts = Vec::new();

for window in index.sliding_window(hour, hour) {
    hourly_counts.push((window.start, window.events.len()));
}

// Find busiest hour
if let Some((time, count)) = hourly_counts.iter().max_by_key(|(_, c)| c) {
    println!("Busiest hour: {} with {} events", 
        time.to_rfc3339(), count);
}

Recent Events

use std::time::Duration;

// Get events from the last 24 hours
let now = Timestamp::now();
let yesterday = now.subtract(Duration::from_secs(86400));
let range = TimeRange::new(yesterday, now);

let recent = index.query_range(&range);
println!("{} events in the last 24 hours", recent.len());

Gap Detection

// Find gaps between events
let ordered: Vec<_> = index.chronological().collect();

for window in ordered.windows(2) {
    let gap = window[1].timestamp.duration_since(&window[0].timestamp);
    if gap > Duration::from_secs(3600 * 6) {  // 6+ hour gap
        println!("Gap from {} to {}", 
            window[0].timestamp.to_rfc3339(),
            window[1].timestamp.to_rfc3339());
    }
}

Spatiotemporal Index

The SpatiotemporalIndex combines spatial and temporal indexing for efficient queries on both dimensions.

Creating an Index

use spatial_narrative::index::SpatiotemporalIndex;
use spatial_narrative::core::{Event, Location, Timestamp};

// Empty index
let mut index: SpatiotemporalIndex<Event> = SpatiotemporalIndex::new();

// Insert items
let event = Event::new(
    Location::new(40.7128, -74.0060),
    Timestamp::parse("2024-01-15T10:00:00Z").unwrap(),
    "NYC Event"
);
index.insert(event, 
    &Location::new(40.7128, -74.0060),
    &Timestamp::parse("2024-01-15T10:00:00Z").unwrap()
);

From Iterator

let index = SpatiotemporalIndex::from_iter(
    events.iter().cloned(),
    |e| &e.location,
    |e| &e.timestamp
);

Combined Queries

Query by both space and time:

use spatial_narrative::core::{GeoBounds, TimeRange};

let bounds = GeoBounds::new(40.0, -75.0, 41.0, -73.0);  // NYC area
let range = TimeRange::month(2024, 1);                   // January 2024

// Events in NYC during January
let results = index.query(&bounds, &range);
println!("Found {} events", results.len());

Individual Dimension Queries

Spatial Only

let bounds = GeoBounds::new(40.0, -75.0, 41.0, -73.0);
let in_area = index.query_spatial(&bounds);

Temporal Only

let range = TimeRange::year(2024);
let in_2024 = index.query_temporal(&range);

Nearest in Time Range

Find spatially nearest items within a time range:

let range = TimeRange::day(2024, 1, 15);

// 5 nearest events on January 15th
let nearest = index.nearest_in_range(40.7128, -74.0060, 5, &range);

for event in nearest {
    println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}

Index Properties

// Number of items
println!("Items: {}", index.len());

// Check if empty
if !index.is_empty() {
    // Get bounds
    if let Some(bounds) = index.bounds() {
        println!("Geographic extent: {:.2}° x {:.2}°", 
            bounds.lat_span(), bounds.lon_span());
    }
    
    // Get time range
    if let Some(range) = index.time_range() {
        println!("Time span: {} to {}", 
            range.start.to_rfc3339(), 
            range.end.to_rfc3339());
    }
}

// Access all items
for item in index.items() {
    println!("{}", item.text);
}

Methods Reference

MethodDescription
new()Create empty index
from_iter()Build from iterator
insert(item, location, timestamp)Add an item
query(bounds, range)Query by space AND time
query_spatial(bounds)Query by space only
query_temporal(range)Query by time only
nearest_in_range(lat, lon, k, range)K-nearest in time window
bounds()Geographic extent
time_range()Temporal extent
len()Number of items
is_empty()Check if empty
items()Access all items

Use Cases

Incident Analysis

// Find incidents in downtown during business hours
let downtown = GeoBounds::new(40.70, -74.02, 40.75, -73.98);

let business_hours = TimeRange::new(
    Timestamp::parse("2024-01-15T09:00:00Z").unwrap(),
    Timestamp::parse("2024-01-15T17:00:00Z").unwrap(),
);

let incidents = index.query(&downtown, &business_hours);
println!("Found {} downtown incidents during business hours", incidents.len());

Event Correlation

// Find events near a location around a specific time
let location = (40.7580, -73.9855);  // Times Square
let time = Timestamp::parse("2024-01-01T00:00:00Z").unwrap();

// 2 hours around midnight on NYE
let window = TimeRange::new(
    time.subtract(std::time::Duration::from_secs(3600)),
    time.add(std::time::Duration::from_secs(3600)),
);

let times_square = GeoBounds::new(40.755, -73.990, 40.760, -73.982);
let nye_events = index.query(&times_square, &window);

Coverage Analysis

// Check coverage across regions and time periods
let regions = vec![
    ("North", GeoBounds::new(41.0, -74.5, 42.0, -73.5)),
    ("Central", GeoBounds::new(40.5, -74.5, 41.0, -73.5)),
    ("South", GeoBounds::new(40.0, -74.5, 40.5, -73.5)),
];

let months = (1..=12).map(|m| TimeRange::month(2024, m));

println!("Coverage by region and month:");
for (name, bounds) in &regions {
    print!("{}: ", name);
    for month in months.clone() {
        let count = index.query(bounds, &month).len();
        print!("{} ", count);
    }
    println!();
}

Anomaly Detection

// Find unusual activity spikes
let normal_bounds = GeoBounds::new(40.0, -75.0, 42.0, -73.0);

// Compare weekday vs weekend activity
let weekday = TimeRange::new(
    Timestamp::parse("2024-01-15T00:00:00Z").unwrap(),  // Monday
    Timestamp::parse("2024-01-15T23:59:59Z").unwrap(),
);

let weekend = TimeRange::new(
    Timestamp::parse("2024-01-13T00:00:00Z").unwrap(),  // Saturday
    Timestamp::parse("2024-01-13T23:59:59Z").unwrap(),
);

let weekday_count = index.query(&normal_bounds, &weekday).len();
let weekend_count = index.query(&normal_bounds, &weekend).len();

println!("Weekday: {} events, Weekend: {} events", weekday_count, weekend_count);

Heatmaps

Generate spatial density data for visualization.

Overview

The Heatmap struct represents event density across a grid, useful for:

  • Visualizing activity hotspots
  • Identifying high-density areas
  • Creating choropleth maps

Generating Heatmaps

use spatial_narrative::index::{SpatiotemporalIndex, GridSpec, Heatmap};
use spatial_narrative::core::GeoBounds;

// Build index from events
let index = SpatiotemporalIndex::from_iter(
    events.iter().cloned(),
    |e| &e.location,
    |e| &e.timestamp
);

// Define grid
let bounds = GeoBounds::new(40.0, -75.0, 42.0, -73.0);
let grid = GridSpec::new(bounds, 50, 50);  // 50x50 cells

// Generate heatmap
let heatmap = index.heatmap(grid);

GridSpec

Configure the heatmap grid:

use spatial_narrative::index::GridSpec;
use spatial_narrative::core::GeoBounds;

let grid = GridSpec::new(
    GeoBounds::new(40.0, -75.0, 42.0, -73.0),  // bounds
    100,  // lat_cells (rows)
    100,  // lon_cells (columns)
);

// Get cell dimensions
let (lat_size, lon_size) = grid.cell_size();
println!("Cell size: {:.4}° lat x {:.4}° lon", lat_size, lon_size);

Accessing Heatmap Data

Raw Counts

// Get count at specific cell
let count = heatmap.get(25, 30);  // row 25, column 30
println!("Cell (25, 30): {} events", count);

// Maximum count
let max = heatmap.max_count();
println!("Maximum density: {} events", max);

Normalized Values

// Get normalized value (0.0 to 1.0)
let intensity = heatmap.get_normalized(25, 30);
println!("Cell (25, 30) intensity: {:.2}", intensity);

Cell Centers

// Get the center coordinates of a cell
let (lat, lon) = heatmap.cell_center(25, 30);
println!("Cell center: ({:.4}, {:.4})", lat, lon);

Iterating Over Cells

// Iterate over all cells
for lat_idx in 0..heatmap.grid.lat_cells {
    for lon_idx in 0..heatmap.grid.lon_cells {
        let count = heatmap.get(lat_idx, lon_idx);
        if count > 0 {
            let (lat, lon) = heatmap.cell_center(lat_idx, lon_idx);
            println!("({:.4}, {:.4}): {} events", lat, lon, count);
        }
    }
}

Exporting for Visualization

As GeoJSON Grid

use serde_json::json;

let mut features = Vec::new();

for lat_idx in 0..heatmap.grid.lat_cells {
    for lon_idx in 0..heatmap.grid.lon_cells {
        let count = heatmap.get(lat_idx, lon_idx);
        if count > 0 {
            let (lat_size, lon_size) = heatmap.grid.cell_size();
            let min_lat = heatmap.grid.bounds.min_lat + lat_idx as f64 * lat_size;
            let min_lon = heatmap.grid.bounds.min_lon + lon_idx as f64 * lon_size;
            
            features.push(json!({
                "type": "Feature",
                "properties": {
                    "count": count,
                    "intensity": heatmap.get_normalized(lat_idx, lon_idx)
                },
                "geometry": {
                    "type": "Polygon",
                    "coordinates": [[
                        [min_lon, min_lat],
                        [min_lon + lon_size, min_lat],
                        [min_lon + lon_size, min_lat + lat_size],
                        [min_lon, min_lat + lat_size],
                        [min_lon, min_lat]
                    ]]
                }
            }));
        }
    }
}

let geojson = json!({
    "type": "FeatureCollection",
    "features": features
});

As CSV

use std::fs::File;
use std::io::Write;

let mut file = File::create("heatmap.csv")?;
writeln!(file, "lat,lon,count,intensity")?;

for lat_idx in 0..heatmap.grid.lat_cells {
    for lon_idx in 0..heatmap.grid.lon_cells {
        let count = heatmap.get(lat_idx, lon_idx);
        let (lat, lon) = heatmap.cell_center(lat_idx, lon_idx);
        let intensity = heatmap.get_normalized(lat_idx, lon_idx);
        writeln!(file, "{},{},{},{}", lat, lon, count, intensity)?;
    }
}

Visualization Integration

Leaflet.heat

// Load heatmap data
const heatData = [];
for (const feature of geojson.features) {
    const coords = feature.geometry.coordinates[0][0];
    heatData.push([
        (coords[1] + feature.geometry.coordinates[0][2][1]) / 2,  // center lat
        (coords[0] + feature.geometry.coordinates[0][1][0]) / 2,  // center lon
        feature.properties.intensity  // intensity
    ]);
}

L.heatLayer(heatData, {radius: 25}).addTo(map);

Mapbox GL JS

map.addSource('heatmap', {
    type: 'geojson',
    data: geojson
});

map.addLayer({
    id: 'heat',
    type: 'fill',
    source: 'heatmap',
    paint: {
        'fill-color': [
            'interpolate',
            ['linear'],
            ['get', 'intensity'],
            0, 'rgba(0, 0, 255, 0)',
            0.5, 'rgba(255, 255, 0, 0.5)',
            1, 'rgba(255, 0, 0, 0.8)'
        ]
    }
});

Use Cases

Finding Hotspots

// Find cells with highest activity
let threshold = heatmap.max_count() / 2;
let mut hotspots = Vec::new();

for lat_idx in 0..heatmap.grid.lat_cells {
    for lon_idx in 0..heatmap.grid.lon_cells {
        if heatmap.get(lat_idx, lon_idx) >= threshold {
            hotspots.push(heatmap.cell_center(lat_idx, lon_idx));
        }
    }
}

println!("Found {} hotspot cells", hotspots.len());

Comparing Time Periods

// Generate heatmaps for different periods
let morning = TimeRange::new(
    Timestamp::parse("2024-01-15T06:00:00Z").unwrap(),
    Timestamp::parse("2024-01-15T12:00:00Z").unwrap(),
);

let evening = TimeRange::new(
    Timestamp::parse("2024-01-15T18:00:00Z").unwrap(),
    Timestamp::parse("2024-01-16T00:00:00Z").unwrap(),
);

// Filter and generate separate heatmaps
let morning_events = index.query_temporal(&morning);
let evening_events = index.query_temporal(&evening);

// Compare distributions...

Graph Overview

The graph module provides tools for representing narratives as directed graphs.

Concepts

In a narrative graph:

  • Nodes are events
  • Edges are relationships between events
  • Edge types describe the nature of relationships

NarrativeGraph

use spatial_narrative::graph::{NarrativeGraph, EdgeType};
use spatial_narrative::core::{Event, Location, Timestamp};

// Create events
let e1 = Event::new(Location::new(40.7, -74.0), Timestamp::now(), "Event 1");
let e2 = Event::new(Location::new(40.7, -74.0), Timestamp::now(), "Event 2");

// Build graph
let mut graph = NarrativeGraph::new();
let n1 = graph.add_event(e1);
let n2 = graph.add_event(e2);

// Connect events
graph.connect(n1, n2, EdgeType::Temporal);

println!("Nodes: {}, Edges: {}", graph.node_count(), graph.edge_count());

Edge Types

TypeDescriptionUse Case
TemporalTime sequenceA happens before B
SpatialGeographic proximityA and B are near each other
CausalCause and effectA causes B
ThematicShared themes/tagsA and B cover same topic
ReferenceCitation/mentionA references B
CustomUser-definedDomain-specific relationships

Quick Example

use spatial_narrative::graph::{NarrativeGraph, EdgeType};
use spatial_narrative::core::Narrative;

// Create from narrative
let mut graph = NarrativeGraph::from_events(narrative.events.clone());

// Auto-connect by different strategies
graph.connect_temporal();      // Connect events in time order
graph.connect_spatial(10.0);   // Connect events within 10km
graph.connect_thematic();      // Connect events sharing tags

// Query the graph
println!("Connected events: {}", graph.edge_count());

// Find paths
if let Some(path) = graph.shortest_path(node_a, node_b) {
    println!("Path length: {} nodes", path.nodes.len());
}

Why Use Graphs?

Graphs enable powerful analysis:

  • Path finding: How are two events connected?
  • Clustering: Which events form natural groups?
  • Centrality: Which events are most connected?
  • Subgraphs: Extract related event clusters
  • Visualization: Export to Graphviz, D3.js, etc.

Module Contents

Visualization Preview

Export to DOT and render with Graphviz:

let dot = graph.to_dot();
std::fs::write("graph.dot", dot)?;
// Run: dot -Tpng graph.dot -o graph.png
┌─────────┐  temporal  ┌─────────┐  spatial  ┌─────────┐
│ Event 1 │ ──────────▶│ Event 2 │ ──────────▶│ Event 3 │
└─────────┘            └─────────┘            └─────────┘
     │                      │                      
     │ thematic             │ causal              
     ▼                      ▼                     
┌─────────┐            ┌─────────┐               
│ Event 4 │            │ Event 5 │               
└─────────┘            └─────────┘               

Building Graphs

Create and populate narrative graphs.

Creating a Graph

Empty Graph

use spatial_narrative::graph::NarrativeGraph;

let mut graph = NarrativeGraph::new();

From Events

use spatial_narrative::graph::NarrativeGraph;
use spatial_narrative::core::{Event, Location, Timestamp};

let events = vec![
    Event::new(Location::new(40.7, -74.0), Timestamp::now(), "Event 1"),
    Event::new(Location::new(40.8, -74.1), Timestamp::now(), "Event 2"),
];

let graph = NarrativeGraph::from_events(events);
println!("Graph has {} nodes", graph.node_count());

From Narrative

let graph = NarrativeGraph::from_events(narrative.events.clone());

Adding Events

use spatial_narrative::graph::NarrativeGraph;

let mut graph = NarrativeGraph::new();

// Add returns NodeId for later reference
let node1 = graph.add_event(event1);
let node2 = graph.add_event(event2);

// Look up node by event ID
let node = graph.get_node(&event.id);

Connecting Events

Basic Connection

use spatial_narrative::graph::EdgeType;

graph.connect(node1, node2, EdgeType::Temporal);

Weighted Connection

use spatial_narrative::graph::EdgeWeight;

// Create weighted edge
let weight = EdgeWeight::with_weight(EdgeType::Spatial, 0.8);
graph.connect_weighted(node1, node2, weight);

// With label
let weight = EdgeWeight::new(EdgeType::Causal)
    .with_label("caused by");
graph.connect_weighted(node1, node2, weight);

Accessing Nodes and Edges

Get Event by Node

if let Some(event) = graph.event(node_id) {
    println!("Event: {}", event.text);
}

// Mutable access
if let Some(event) = graph.event_mut(node_id) {
    event.add_tag("processed");
}

Iterate Nodes

for (node_id, event) in graph.nodes() {
    println!("Node {}: {}", node_id.index(), event.text);
}

Iterate Edges

for (from, to, weight) in graph.edges() {
    println!("{} -> {} ({:?})", 
        from.index(), to.index(), weight.edge_type);
}

Filter Edges by Type

let temporal_edges = graph.edges_of_type(EdgeType::Temporal);
println!("Temporal connections: {}", temporal_edges.len());

Graph Properties

// Counts
println!("Nodes: {}", graph.node_count());
println!("Edges: {}", graph.edge_count());
println!("Empty: {}", graph.is_empty());

// Degree analysis
let in_degree = graph.in_degree(node);    // Incoming edges
let out_degree = graph.out_degree(node);  // Outgoing edges
println!("Node has {} in, {} out", in_degree, out_degree);

Neighbors

// Get connected nodes
let successors = graph.successors(node);   // Outgoing connections
let predecessors = graph.predecessors(node);  // Incoming connections

for successor in successors {
    if let Some(event) = graph.event(successor) {
        println!("Leads to: {}", event.text);
    }
}

Checking Connectivity

// Direct connection
if graph.are_connected(node1, node2) {
    println!("Directly connected");
}

// Any path exists
if graph.has_path(node1, node2) {
    println!("Path exists");
}

Special Nodes

// Entry points (no predecessors)
let roots = graph.roots();
println!("Entry points: {}", roots.len());

// End points (no successors)
let leaves = graph.leaves();
println!("End points: {}", leaves.len());

Subgraphs

Extract portions of the graph:

use spatial_narrative::core::{TimeRange, GeoBounds};

// By time
let january = TimeRange::month(2024, 1);
let subgraph = graph.subgraph_temporal(&january);
println!("January subgraph: {} nodes", subgraph.graph.node_count());

// By location
let nyc = GeoBounds::new(40.4, -74.3, 41.0, -73.7);
let subgraph = graph.subgraph_spatial(&nyc);
println!("NYC subgraph: {} nodes", subgraph.graph.node_count());

SubgraphResult

let result = graph.subgraph_temporal(&range);

// The new graph
let new_graph = result.graph;

// Mapping from old to new node IDs
for (old_id, new_id) in result.node_mapping {
    println!("Node {} -> {}", old_id.index(), new_id.index());
}

Example: Building a Story Graph

// Create graph from news events
let mut graph = NarrativeGraph::from_events(news_events);

// Connect by time (earlier → later)
graph.connect_temporal();

// Connect nearby events (within 1km)
graph.connect_spatial(1.0);

// Connect events sharing topics
graph.connect_thematic();

// Add manual causal links
let cause = graph.get_node(&cause_event.id).unwrap();
let effect = graph.get_node(&effect_event.id).unwrap();
graph.connect(cause, effect, EdgeType::Causal);

println!("Story graph: {} events, {} connections",
    graph.node_count(), graph.edge_count());

Connection Strategies

Automatically connect events based on different criteria.

Temporal Connection

Connect events in chronological order:

let mut graph = NarrativeGraph::from_events(events);

// Connect A → B if A happens before B
graph.connect_temporal();

This creates a chain:

Event 1 ─temporal→ Event 2 ─temporal→ Event 3 ─temporal→ ...

How It Works

  • Events are sorted by timestamp
  • Each event connects to the next in sequence
  • Creates EdgeType::Temporal edges
  • Existing edges are not duplicated

Spatial Connection

Connect events that are geographically close:

// Connect events within 10km of each other
graph.connect_spatial(10.0);

// Closer threshold for dense areas
graph.connect_spatial(1.0);  // 1km

How It Works

  • Compares all pairs of events
  • Uses Haversine distance (great-circle)
  • Creates bidirectional EdgeType::Spatial edges
  • Edge weight = 1.0 - (distance / max_distance)
// Edge weight indicates proximity
// Weight 1.0 = same location
// Weight 0.5 = half the max distance apart
// Weight 0.0 = at max distance

Thematic Connection

Connect events that share tags:

graph.connect_thematic();

How It Works

  • Events sharing one or more tags are connected
  • Creates bidirectional EdgeType::Thematic edges
  • Edge weight = shared_tags / max_tags
// Example: Event A has tags ["news", "politics"]
//          Event B has tags ["politics", "election"]
// Shared: 1 tag, Max: 2 tags
// Weight: 0.5

Manual Connections

For relationships that can't be auto-detected:

use spatial_narrative::graph::{EdgeType, EdgeWeight};

// Get nodes
let cause = graph.get_node(&event1.id).unwrap();
let effect = graph.get_node(&event2.id).unwrap();

// Simple connection
graph.connect(cause, effect, EdgeType::Causal);

// With weight and label
let weight = EdgeWeight::with_weight(EdgeType::Reference, 0.9)
    .with_label("cites");
graph.connect_weighted(cause, effect, weight);

Edge Types Reference

TypeAuto-ConnectDirectionWeight Meaning
Temporalconnect_temporal()UnidirectionalAlways 1.0
Spatialconnect_spatial(km)BidirectionalProximity
Thematicconnect_thematic()BidirectionalTag overlap
CausalManualUnidirectionalStrength
ReferenceManualUnidirectionalRelevance
CustomManualEitherUser-defined

Combining Strategies

Apply multiple connection strategies:

let mut graph = NarrativeGraph::from_events(events);

// Build a rich connection graph
graph.connect_temporal();      // Time sequence
graph.connect_spatial(5.0);    // 5km proximity
graph.connect_thematic();      // Shared topics

println!("Created {} connections", graph.edge_count());

// Analyze by type
let temporal = graph.edges_of_type(EdgeType::Temporal).len();
let spatial = graph.edges_of_type(EdgeType::Spatial).len();
let thematic = graph.edges_of_type(EdgeType::Thematic).len();

println!("  Temporal: {}", temporal);
println!("  Spatial: {}", spatial);
println!("  Thematic: {}", thematic);

Selective Connection

Connect only certain events:

// First, add all events
let mut graph = NarrativeGraph::from_events(events);

// Connect only important events temporally
let important: Vec<_> = graph.nodes()
    .filter(|(_, e)| e.has_tag("important"))
    .map(|(id, _)| id)
    .collect();

// Sort by time and connect
let mut sorted: Vec<_> = important.iter()
    .filter_map(|&id| graph.event(id).map(|e| (id, e.timestamp.clone())))
    .collect();
sorted.sort_by(|a, b| a.1.cmp(&b.1));

for window in sorted.windows(2) {
    graph.connect(window[0].0, window[1].0, EdgeType::Temporal);
}

Use Cases

News Story Tracking

// Connect news events
graph.connect_temporal();     // Story progression
graph.connect_thematic();     // Related topics

// Find story threads
for root in graph.roots() {
    println!("Story starts: {}", graph.event(root).unwrap().text);
    let thread: Vec<_> = std::iter::successors(Some(root), |&n| {
        graph.successors(n).into_iter().next()
    }).collect();
    println!("  {} events in thread", thread.len());
}

Location-Based Analysis

// Focus on spatial relationships
graph.connect_spatial(0.5);  // 500m - same block
graph.connect_spatial(2.0);  // 2km - neighborhood

// Find location clusters
let high_degree: Vec<_> = graph.nodes()
    .filter(|(id, _)| graph.in_degree(*id) + graph.out_degree(*id) > 5)
    .collect();

println!("Highly connected locations: {}", high_degree.len());

Topic Networks

// Build topic network
graph.connect_thematic();

// Find central topics (most connected)
let mut connections: Vec<_> = graph.nodes()
    .map(|(id, _)| (id, graph.in_degree(id) + graph.out_degree(id)))
    .collect();
connections.sort_by(|a, b| b.1.cmp(&a.1));

println!("Most connected events:");
for (id, degree) in connections.iter().take(5) {
    let event = graph.event(*id).unwrap();
    println!("  {} ({} connections)", event.text, degree);
}

Path Finding

Find paths and analyze connectivity in narrative graphs.

Connectivity Checks

Direct Connection

// Are two nodes directly connected?
if graph.are_connected(node_a, node_b) {
    println!("A connects directly to B");
}

Path Exists

// Is there any path from A to B?
if graph.has_path(node_a, node_b) {
    println!("A can reach B (directly or indirectly)");
} else {
    println!("No path from A to B");
}

Shortest Path

Find the shortest path between two nodes:

if let Some(path) = graph.shortest_path(start, end) {
    println!("Path found:");
    println!("  Nodes: {}", path.nodes.len());
    println!("  Total weight: {:.2}", path.total_weight);
    
    // Print path
    for node_id in &path.nodes {
        let event = graph.event(*node_id).unwrap();
        println!("  → {}", event.text);
    }
} else {
    println!("No path exists");
}

PathInfo

pub struct PathInfo {
    pub nodes: Vec<NodeId>,    // Nodes in order
    pub total_weight: f64,     // Sum of edge weights
}

impl PathInfo {
    pub fn len(&self) -> usize {
        self.nodes.len()
    }
}

Neighborhood Analysis

Successors (Outgoing)

// Events this event leads to
let following = graph.successors(node);
for next in following {
    let event = graph.event(next).unwrap();
    println!("Leads to: {}", event.text);
}

Predecessors (Incoming)

// Events that lead to this event
let previous = graph.predecessors(node);
for prev in previous {
    let event = graph.event(prev).unwrap();
    println!("Preceded by: {}", event.text);
}

Graph Structure

Entry Points (Roots)

Events with no predecessors - story starting points:

let roots = graph.roots();
println!("Story begins at {} events", roots.len());

for root in roots {
    let event = graph.event(root).unwrap();
    println!("  Start: {}", event.text);
}

End Points (Leaves)

Events with no successors - story endings:

let leaves = graph.leaves();
println!("Story ends at {} events", leaves.len());

for leaf in leaves {
    let event = graph.event(leaf).unwrap();
    println!("  End: {}", event.text);
}

Degree Analysis

// Incoming connections
let in_deg = graph.in_degree(node);

// Outgoing connections
let out_deg = graph.out_degree(node);

println!("Node has {} incoming, {} outgoing connections", in_deg, out_deg);

// Find highly connected nodes
for (node_id, event) in graph.nodes() {
    let total_degree = graph.in_degree(node_id) + graph.out_degree(node_id);
    if total_degree > 5 {
        println!("Hub: {} ({} connections)", event.text, total_degree);
    }
}

Traversal Patterns

Follow Timeline

// Follow temporal connections from a starting point
fn follow_timeline(graph: &NarrativeGraph, start: NodeId) -> Vec<NodeId> {
    let mut path = vec![start];
    let mut current = start;
    
    while let Some(next) = graph.successors(current)
        .into_iter()
        .filter(|&n| {
            graph.edges()
                .any(|(from, to, w)| from == current && to == n 
                    && w.edge_type == EdgeType::Temporal)
        })
        .next()
    {
        path.push(next);
        current = next;
    }
    
    path
}

Find All Paths

// Find all paths between two nodes (BFS)
fn find_all_paths(
    graph: &NarrativeGraph, 
    start: NodeId, 
    end: NodeId,
    max_depth: usize
) -> Vec<Vec<NodeId>> {
    let mut paths = Vec::new();
    let mut queue = vec![(vec![start], 0)];
    
    while let Some((path, depth)) = queue.pop() {
        if depth > max_depth {
            continue;
        }
        
        let current = *path.last().unwrap();
        if current == end {
            paths.push(path);
            continue;
        }
        
        for next in graph.successors(current) {
            if !path.contains(&next) {
                let mut new_path = path.clone();
                new_path.push(next);
                queue.push((new_path, depth + 1));
            }
        }
    }
    
    paths
}

Use Cases

Story Thread Extraction

// Extract complete story threads from roots to leaves
for root in graph.roots() {
    let thread = follow_timeline(&graph, root);
    
    println!("Story thread ({} events):", thread.len());
    for node_id in &thread {
        let event = graph.event(*node_id).unwrap();
        println!("  {}: {}", event.timestamp.to_rfc3339(), event.text);
    }
}

Finding Connection Between Events

// How are two events connected?
let event_a = graph.get_node(&event1.id).unwrap();
let event_b = graph.get_node(&event2.id).unwrap();

if let Some(path) = graph.shortest_path(event_a, event_b) {
    println!("Connection found via {} intermediate events:", path.len() - 2);
    
    for window in path.nodes.windows(2) {
        let from = graph.event(window[0]).unwrap();
        let to = graph.event(window[1]).unwrap();
        
        // Find edge type
        let edge_type = graph.edges()
            .find(|(f, t, _)| *f == window[0] && *t == window[1])
            .map(|(_, _, w)| w.edge_type);
        
        println!("  {} →[{:?}]→ {}", from.text, edge_type, to.text);
    }
}

Hub Detection

// Find events that connect many other events
let mut hubs: Vec<_> = graph.nodes()
    .map(|(id, event)| {
        let degree = graph.in_degree(id) + graph.out_degree(id);
        (id, event, degree)
    })
    .collect();

hubs.sort_by(|a, b| b.2.cmp(&a.2));

println!("Top 5 hub events:");
for (_, event, degree) in hubs.iter().take(5) {
    println!("  {} ({} connections)", event.text, degree);
}

Graph Export & Visualization

Export your narrative graphs for visualization in external tools.

DOT Format (Graphviz)

Export to DOT format for rendering with Graphviz.

Basic Export

use spatial_narrative::graph::NarrativeGraph;

let graph = NarrativeGraph::from_events(events);
graph.connect_temporal();

// Export to DOT
let dot = graph.to_dot();

// Save to file
std::fs::write("narrative.dot", &dot)?;

Rendering with Graphviz

# Install Graphviz
# Ubuntu/Debian: sudo apt install graphviz
# macOS: brew install graphviz
# Windows: choco install graphviz

# Render to PNG
dot -Tpng narrative.dot -o narrative.png

# Render to SVG (better for web)
dot -Tsvg narrative.dot -o narrative.svg

# Render to PDF
dot -Tpdf narrative.dot -o narrative.pdf

Custom Options

use spatial_narrative::graph::DotOptions;

// Timeline layout (left-to-right)
let timeline_dot = graph.to_dot_with_options(DotOptions::timeline());

// Hierarchical layout (top-to-bottom)
let hier_dot = graph.to_dot_with_options(DotOptions::hierarchical());

// Custom options
let options = DotOptions {
    rank_direction: "LR".to_string(),
    node_shape: "ellipse".to_string(),
    font_name: "Helvetica".to_string(),
};
let custom_dot = graph.to_dot_with_options(options);

Online Visualization

Paste DOT output into online tools:

JSON Format

Export for web visualization libraries.

Basic Export

// Compact JSON
let json = graph.to_json();

// Pretty-printed JSON
let json = graph.to_json_pretty();

// Save to file
std::fs::write("narrative.json", &json)?;

JSON Structure

{
  "nodes": [
    {
      "id": 0,
      "event_id": "550e8400-e29b-41d4-a716-446655440000",
      "text": "Event description",
      "location": { "lat": 40.7128, "lon": -74.006 },
      "timestamp": "2024-01-15T10:00:00+00:00",
      "tags": ["conference", "technology"]
    }
  ],
  "edges": [
    {
      "source": 0,
      "target": 1,
      "type": "Temporal",
      "weight": 1.0,
      "label": null
    }
  ],
  "metadata": {
    "node_count": 5,
    "edge_count": 8
  }
}

Web Visualization Libraries

D3.js

// Load the JSON
fetch('narrative.json')
  .then(response => response.json())
  .then(data => {
    // Create force-directed graph
    const simulation = d3.forceSimulation(data.nodes)
      .force("link", d3.forceLink(data.edges).id(d => d.id))
      .force("charge", d3.forceManyBody())
      .force("center", d3.forceCenter(width / 2, height / 2));
  });

Cytoscape.js

const cy = cytoscape({
  container: document.getElementById('cy'),
  elements: {
    nodes: data.nodes.map(n => ({ data: { id: n.id, label: n.text } })),
    edges: data.edges.map(e => ({ data: { source: e.source, target: e.target } }))
  }
});

Sigma.js

const graph = new graphology.Graph();
data.nodes.forEach(n => graph.addNode(n.id, { label: n.text }));
data.edges.forEach(e => graph.addEdge(e.source, e.target));

const sigma = new Sigma(graph, container);

Node Colors

The DOT export uses automatic node coloring:

ColorMeaning
🟢 Light GreenRoot nodes (no incoming edges)
🩷 Light PinkLeaf nodes (no outgoing edges)
🔵 Light BlueHub nodes (high connectivity)
🟡 Light YellowRegular nodes

Edge Styles

Edges are styled by type:

TypeColorStyle
TemporalBlueSolid
SpatialMagentaDashed
CausalOrangeBold
ThematicRedDotted
ReferenceOliveSolid
CustomGraySolid

Analysis Overview

The analysis module provides tools for extracting insights from spatial narratives.

Features

FeatureDescriptionTypes
Spatial MetricsGeographic extent, distances, dispersion[SpatialMetrics]
Temporal MetricsDuration, rates, gaps, bursts[TemporalMetrics]
MovementTrajectory extraction and stop detection[Trajectory], [Stop]
ClusteringDensity and centroid-based clustering[DBSCAN], [KMeans]
ComparisonSimilarity scoring between narratives[compare_narratives]

Quick Examples

Spatial Metrics

use spatial_narrative::analysis::SpatialMetrics;
use spatial_narrative::core::{Event, Location, Timestamp};

let events = vec![
    Event::new(Location::new(40.7128, -74.0060), Timestamp::now(), "NYC"),
    Event::new(Location::new(34.0522, -118.2437), Timestamp::now(), "LA"),
    Event::new(Location::new(41.8781, -87.6298), Timestamp::now(), "Chicago"),
];

let metrics = SpatialMetrics::from_events(&events);

println!("Total distance: {:.0} km", metrics.total_distance / 1000.0);
println!("Geographic extent: {:.0} km", metrics.max_extent / 1000.0);
println!("Centroid: ({:.4}, {:.4})", metrics.centroid.0, metrics.centroid.1);

Temporal Metrics

use spatial_narrative::analysis::TemporalMetrics;

let metrics = TemporalMetrics::from_events(&events);

println!("Duration: {:?}", metrics.duration);
println!("Event count: {}", metrics.event_count);
println!("Average interval: {:?}", metrics.avg_interval);

DBSCAN Clustering

use spatial_narrative::analysis::DBSCAN;

// Cluster events within 5km, requiring 2+ points per cluster
let dbscan = DBSCAN::new(5000.0, 2);
let result = dbscan.cluster(&events);

println!("Found {} clusters", result.num_clusters());
println!("Noise points: {}", result.noise.len());

K-Means Clustering

use spatial_narrative::analysis::KMeans;

// Partition events into 3 clusters
let kmeans = KMeans::new(3);
let result = kmeans.cluster(&events);

for (i, cluster) in result.clusters.iter().enumerate() {
    println!("Cluster {}: {} events", i, cluster.events.len());
}

Trajectory Analysis

use spatial_narrative::analysis::{Trajectory, detect_stops, StopThreshold};

// Extract trajectory from events
let trajectory = Trajectory::from_events(&events);

println!("Trajectory length: {:.0} m", trajectory.total_distance());
println!("Duration: {:?}", trajectory.duration());

// Detect stops (stationary periods)
let threshold = StopThreshold::new(50.0, std::time::Duration::from_secs(300));
let stops = detect_stops(&events, &threshold);

for stop in stops {
    println!("Stop at ({:.4}, {:.4}) for {:?}", 
        stop.location.lat, stop.location.lon, stop.duration);
}

Comparing Narratives

use spatial_narrative::analysis::{compare_narratives, ComparisonConfig};
use spatial_narrative::core::Narrative;

let similarity = compare_narratives(&narrative1, &narrative2, &ComparisonConfig::default());

println!("Spatial similarity: {:.2}", similarity.spatial);
println!("Temporal similarity: {:.2}", similarity.temporal);
println!("Thematic similarity: {:.2}", similarity.thematic);
println!("Overall similarity: {:.2}", similarity.overall);

Module Structure

spatial_narrative::analysis
├── SpatialMetrics      # Geographic analysis
├── TemporalMetrics     # Time-based analysis
├── Trajectory          # Movement paths
├── DBSCAN              # Density clustering
├── KMeans              # K-means clustering
├── detect_stops        # Stop detection
├── detect_gaps         # Gap detection
├── detect_bursts       # Burst detection
└── compare_narratives  # Narrative comparison

Next Steps

Spatial Metrics

Analyze the geographic characteristics of events.

SpatialMetrics

The SpatialMetrics struct provides geographic measurements for a set of events.

use spatial_narrative::analysis::SpatialMetrics;
use spatial_narrative::core::{Event, Location, Timestamp};

let events = vec![
    Event::new(Location::new(40.7128, -74.0060), Timestamp::now(), "NYC"),
    Event::new(Location::new(34.0522, -118.2437), Timestamp::now(), "LA"),
    Event::new(Location::new(41.8781, -87.6298), Timestamp::now(), "Chicago"),
];

let metrics = SpatialMetrics::from_events(&events);

Available Metrics

MetricTypeDescription
total_distancef64Sum of distances between consecutive events (meters)
max_extentf64Maximum distance between any two events (meters)
centroid(f64, f64)Geographic center (lat, lon)
boundsGeoBoundsBounding box containing all events
dispersionf64Standard distance from centroid (meters)

Example Output

println!("Metrics for {} events:", events.len());
println!("  Total distance: {:.1} km", metrics.total_distance / 1000.0);
println!("  Maximum extent: {:.1} km", metrics.max_extent / 1000.0);
println!("  Centroid: ({:.4}°, {:.4}°)", metrics.centroid.0, metrics.centroid.1);
println!("  Dispersion: {:.1} km", metrics.dispersion / 1000.0);
println!("  Bounds: {:.2}°N to {:.2}°N, {:.2}°W to {:.2}°W",
    metrics.bounds.min_lat, metrics.bounds.max_lat,
    metrics.bounds.min_lon.abs(), metrics.bounds.max_lon.abs());

Distance Functions

Haversine Distance

Calculate great-circle distance between two points:

use spatial_narrative::analysis::haversine_distance;
use spatial_narrative::core::Location;

let nyc = Location::new(40.7128, -74.0060);
let london = Location::new(51.5074, -0.1278);

let distance_m = haversine_distance(&nyc, &london);
println!("NYC to London: {:.0} km", distance_m / 1000.0);  // ~5570 km

Bearing

Calculate initial bearing between two points:

use spatial_narrative::analysis::bearing;

let from = Location::new(40.7128, -74.0060);  // NYC
let to = Location::new(51.5074, -0.1278);     // London

let degrees = bearing(&from, &to);
println!("Bearing: {:.1}°", degrees);  // ~51° (northeast)

Destination Point

Calculate destination given start, bearing, and distance:

use spatial_narrative::analysis::destination_point;

let start = Location::new(40.7128, -74.0060);
let bearing_deg = 45.0;   // Northeast
let distance_m = 100_000.0;  // 100 km

let dest = destination_point(&start, bearing_deg, distance_m);
println!("Destination: ({:.4}, {:.4})", dest.lat, dest.lon);

Density Mapping

Generate a density map of event locations:

use spatial_narrative::analysis::{density_map, DensityCell};
use spatial_narrative::core::GeoBounds;

// Define grid
let bounds = GeoBounds::new(34.0, -118.5, 41.0, -73.5);
let lat_cells = 50;
let lon_cells = 50;

// Generate density map
let cells = density_map(&events, &bounds, lat_cells, lon_cells);

// Find hotspots
let max_density = cells.iter().map(|c| c.count).max().unwrap_or(0);
let hotspots: Vec<_> = cells.iter()
    .filter(|c| c.count > max_density / 2)
    .collect();

println!("Found {} hotspot cells", hotspots.len());

DensityCell

Each cell in the density map contains:

FieldTypeDescription
lat_idxusizeRow index
lon_idxusizeColumn index
countusizeNumber of events in cell
center(f64, f64)Cell center coordinates

Use Cases

Analyzing Geographic Spread

let metrics = SpatialMetrics::from_events(&events);

if metrics.max_extent > 1_000_000.0 {
    println!("Events span over 1000 km - continental scale");
} else if metrics.max_extent > 100_000.0 {
    println!("Events span over 100 km - regional scale");
} else {
    println!("Events are localized - city scale");
}

Finding the Geographic Center

let metrics = SpatialMetrics::from_events(&events);
let center = Location::new(metrics.centroid.0, metrics.centroid.1);

// Find events nearest to center
for event in &events {
    let dist = haversine_distance(&event.location, &center);
    if dist < 10_000.0 {  // Within 10km of center
        println!("Near center: {}", event.text);
    }
}

Temporal Metrics

Analyze the time-based characteristics of events.

TemporalMetrics

The TemporalMetrics struct provides time-based measurements for a set of events.

use spatial_narrative::analysis::TemporalMetrics;
use spatial_narrative::core::{Event, Location, Timestamp};

let events = vec![
    Event::new(Location::new(40.7, -74.0), 
        Timestamp::parse("2024-01-01T10:00:00Z").unwrap(), "Morning"),
    Event::new(Location::new(40.7, -74.0), 
        Timestamp::parse("2024-01-01T14:00:00Z").unwrap(), "Afternoon"),
    Event::new(Location::new(40.7, -74.0), 
        Timestamp::parse("2024-01-02T09:00:00Z").unwrap(), "Next day"),
];

let metrics = TemporalMetrics::from_events(&events);

Available Metrics

MetricTypeDescription
durationDurationTime span from first to last event
event_countusizeNumber of events
avg_intervalDurationAverage time between events
min_intervalDurationShortest time between events
max_intervalDurationLongest time between events
startTimestampFirst event timestamp
endTimestampLast event timestamp

Example Output

println!("Temporal metrics for {} events:", metrics.event_count);
println!("  Duration: {:?}", metrics.duration);
println!("  Average interval: {:?}", metrics.avg_interval);
println!("  Min interval: {:?}", metrics.min_interval);
println!("  Max interval: {:?}", metrics.max_interval);
println!("  Time range: {} to {}", 
    metrics.start.to_rfc3339(), 
    metrics.end.to_rfc3339());

Event Rate

Calculate the rate of events over time:

use spatial_narrative::analysis::event_rate;
use std::time::Duration;

// Events per hour
let hourly_rate = event_rate(&events, Duration::from_secs(3600));
println!("Events per hour: {:.2}", hourly_rate);

// Events per day
let daily_rate = event_rate(&events, Duration::from_secs(86400));
println!("Events per day: {:.2}", daily_rate);

Gap Detection

Find significant gaps in the event timeline:

use spatial_narrative::analysis::detect_gaps;
use std::time::Duration;

// Find gaps longer than 1 hour
let min_gap = Duration::from_secs(3600);
let gaps = detect_gaps(&events, min_gap);

for gap in gaps {
    println!("Gap from {} to {} ({:?})",
        gap.start.to_rfc3339(),
        gap.end.to_rfc3339(),
        gap.duration);
}

Gap Struct

Each detected gap contains:

FieldTypeDescription
startTimestampEnd of previous event
endTimestampStart of next event
durationDurationLength of the gap

Burst Detection

Identify periods of high event activity:

use spatial_narrative::analysis::detect_bursts;
use std::time::Duration;

// Find bursts with 5+ events in 1-hour windows
let window = Duration::from_secs(3600);
let min_events = 5;
let bursts = detect_bursts(&events, window, min_events);

for burst in bursts {
    println!("Burst: {} events from {} to {}",
        burst.count,
        burst.start.to_rfc3339(),
        burst.end.to_rfc3339());
}

Time Binning

Group events into time bins:

use spatial_narrative::analysis::TimeBin;
use std::time::Duration;

// Group by hour
let bins = TimeBin::from_events(&events, Duration::from_secs(3600));

for bin in bins {
    println!("{}: {} events", bin.start.to_rfc3339(), bin.count);
}

Use Cases

Activity Pattern Analysis

let metrics = TemporalMetrics::from_events(&events);

// Calculate activity intensity
let events_per_hour = metrics.event_count as f64 
    / metrics.duration.as_secs_f64() * 3600.0;

if events_per_hour > 10.0 {
    println!("High activity: {:.1} events/hour", events_per_hour);
} else if events_per_hour > 1.0 {
    println!("Moderate activity: {:.1} events/hour", events_per_hour);
} else {
    println!("Low activity: {:.2} events/hour", events_per_hour);
}

Finding Quiet Periods

use std::time::Duration;

// Gaps longer than 6 hours
let quiet_periods = detect_gaps(&events, Duration::from_secs(6 * 3600));

println!("Found {} quiet periods (>6 hours):", quiet_periods.len());
for gap in quiet_periods {
    println!("  {:?} gap starting {}", 
        gap.duration, 
        gap.start.to_rfc3339());
}

Rush Hour Detection

use std::time::Duration;

// Detect 30-minute windows with 10+ events
let rush_periods = detect_bursts(&events, Duration::from_secs(1800), 10);

if !rush_periods.is_empty() {
    println!("Detected {} rush periods", rush_periods.len());
    for rush in rush_periods {
        println!("  {} events in 30min window at {}", 
            rush.count, 
            rush.start.to_rfc3339());
    }
}

Movement Analysis

Analyze movement patterns and detect significant locations.

Trajectory

A Trajectory represents a path through space and time.

use spatial_narrative::analysis::Trajectory;
use spatial_narrative::core::{Event, Location, Timestamp};

let events = vec![
    Event::new(Location::new(40.7128, -74.0060), 
        Timestamp::parse("2024-01-01T10:00:00Z").unwrap(), "Start"),
    Event::new(Location::new(40.7480, -73.9850), 
        Timestamp::parse("2024-01-01T10:30:00Z").unwrap(), "Midtown"),
    Event::new(Location::new(40.7580, -73.9855), 
        Timestamp::parse("2024-01-01T11:00:00Z").unwrap(), "Times Square"),
];

let trajectory = Trajectory::from_events(&events);

Trajectory Methods

MethodReturn TypeDescription
total_distance()f64Total path length in meters
duration()DurationTime from start to end
average_speed()f64Average speed in m/s
points()&[TrajectoryPoint]All points on the path
bounds()GeoBoundsBounding box of path

Example Usage

println!("Trajectory analysis:");
println!("  Distance: {:.2} km", trajectory.total_distance() / 1000.0);
println!("  Duration: {:?}", trajectory.duration());
println!("  Avg speed: {:.1} km/h", trajectory.average_speed() * 3.6);

// Access individual segments
for (i, segment) in trajectory.segments().enumerate() {
    println!("  Segment {}: {:.0}m in {:?}", 
        i, segment.distance, segment.duration);
}

Stop Detection

Identify locations where movement paused.

use spatial_narrative::analysis::{detect_stops, StopThreshold};
use std::time::Duration;

// Configure stop detection
let threshold = StopThreshold::new(
    50.0,                            // Max radius in meters
    Duration::from_secs(300),        // Min duration (5 minutes)
);

let stops = detect_stops(&events, &threshold);

for stop in &stops {
    println!("Stop at ({:.4}, {:.4})", stop.location.lat, stop.location.lon);
    println!("  Duration: {:?}", stop.duration);
    println!("  Events: {}", stop.events.len());
}

StopThreshold

Configure stop detection sensitivity:

FieldTypeDescription
radius_metersf64Maximum spread to consider a stop
min_durationDurationMinimum time to qualify as a stop

Stop Struct

Each detected stop contains:

FieldTypeDescription
locationLocationCenter of the stop
startTimestampWhen the stop began
endTimestampWhen the stop ended
durationDurationHow long the stop lasted
eventsVec<&Event>Events during the stop

Movement Analyzer

For more advanced movement analysis:

use spatial_narrative::analysis::MovementAnalyzer;

let analyzer = MovementAnalyzer::new(&events);

// Get movement statistics
let stats = analyzer.statistics();
println!("Movement statistics:");
println!("  Total distance: {:.2} km", stats.total_distance / 1000.0);
println!("  Moving time: {:?}", stats.moving_time);
println!("  Stopped time: {:?}", stats.stopped_time);
println!("  Max speed: {:.1} km/h", stats.max_speed * 3.6);

// Detect mode changes
let segments = analyzer.segment_by_speed(5.0); // 5 m/s threshold
for segment in segments {
    let mode = if segment.avg_speed > 5.0 { "vehicle" } else { "walking" };
    println!("  {} segment: {:.0}m", mode, segment.distance);
}

Use Cases

GPS Track Analysis

// Analyze a GPS track
let trajectory = Trajectory::from_events(&gps_points);
let stops = detect_stops(&gps_points, &StopThreshold::new(25.0, Duration::from_secs(120)));

println!("Track summary:");
println!("  Total distance: {:.2} km", trajectory.total_distance() / 1000.0);
println!("  Duration: {:?}", trajectory.duration());
println!("  Stops: {}", stops.len());

// Calculate moving vs stopped time
let stopped_time: Duration = stops.iter().map(|s| s.duration).sum();
let moving_time = trajectory.duration() - stopped_time;
println!("  Moving time: {:?}", moving_time);
println!("  Stopped time: {:?}", stopped_time);

Delivery Route Analysis

// Identify delivery stops
let threshold = StopThreshold::new(
    30.0,                          // 30m radius (parking area)
    Duration::from_secs(60),       // At least 1 minute
);

let delivery_stops = detect_stops(&route_events, &threshold);

println!("Delivery route analysis:");
println!("  Total stops: {}", delivery_stops.len());
println!("  Route distance: {:.1} km", 
    Trajectory::from_events(&route_events).total_distance() / 1000.0);

// Average time per stop
if !delivery_stops.is_empty() {
    let avg_stop_time: Duration = delivery_stops.iter()
        .map(|s| s.duration)
        .sum::<Duration>() / delivery_stops.len() as u32;
    println!("  Avg stop time: {:?}", avg_stop_time);
}

Anomaly Detection

let trajectory = Trajectory::from_events(&events);

// Find unusually fast segments (potential GPS errors)
for segment in trajectory.segments() {
    let speed_kmh = segment.speed() * 3.6;
    if speed_kmh > 200.0 {
        println!("Warning: Unrealistic speed {:.0} km/h detected", speed_kmh);
        println!("  From {} to {}", 
            segment.start.to_rfc3339(), 
            segment.end.to_rfc3339());
    }
}

Clustering

Group events based on spatial proximity.

DBSCAN

Density-Based Spatial Clustering of Applications with Noise.

DBSCAN finds clusters of arbitrary shape and identifies outliers (noise).

use spatial_narrative::analysis::DBSCAN;
use spatial_narrative::core::{Event, Location, Timestamp};

let events = vec![
    // Cluster 1: NYC area
    Event::new(Location::new(40.7128, -74.0060), Timestamp::now(), "NYC 1"),
    Event::new(Location::new(40.7138, -74.0050), Timestamp::now(), "NYC 2"),
    Event::new(Location::new(40.7118, -74.0070), Timestamp::now(), "NYC 3"),
    // Cluster 2: LA area  
    Event::new(Location::new(34.0522, -118.2437), Timestamp::now(), "LA 1"),
    Event::new(Location::new(34.0532, -118.2427), Timestamp::now(), "LA 2"),
    // Noise: isolated point
    Event::new(Location::new(41.8781, -87.6298), Timestamp::now(), "Chicago"),
];

// Create DBSCAN with 1km radius and minimum 2 points per cluster
let dbscan = DBSCAN::new(1000.0, 2);
let result = dbscan.cluster(&events);

println!("Found {} clusters", result.num_clusters());
println!("Noise points: {}", result.noise.len());

DBSCAN Parameters

ParameterDescription
epsMaximum distance (meters) between points in a cluster
min_pointsMinimum points required to form a cluster

Choosing Parameters

// Dense urban data: small radius, more points
let urban = DBSCAN::new(500.0, 5);    // 500m, 5+ points

// Sparse regional data: larger radius, fewer points  
let regional = DBSCAN::new(10_000.0, 3);  // 10km, 3+ points

// Very dense data (GPS tracks): tight clustering
let gps = DBSCAN::new(50.0, 10);      // 50m, 10+ points

ClusteringResult

// Iterate over clusters
for (i, cluster) in result.clusters.iter().enumerate() {
    println!("Cluster {}:", i);
    println!("  Events: {}", cluster.events.len());
    println!("  Center: ({:.4}, {:.4})", 
        cluster.centroid.0, cluster.centroid.1);
    
    for event in &cluster.events {
        println!("    - {}", event.text);
    }
}

// Handle noise points
if !result.noise.is_empty() {
    println!("Noise points (unclustered):");
    for event in &result.noise {
        println!("  - {} at ({:.4}, {:.4})", 
            event.text, event.location.lat, event.location.lon);
    }
}

K-Means

Partition events into exactly K clusters.

use spatial_narrative::analysis::KMeans;

// Partition into 3 clusters
let kmeans = KMeans::new(3);
let result = kmeans.cluster(&events);

for (i, cluster) in result.clusters.iter().enumerate() {
    println!("Cluster {} ({} events):", i, cluster.events.len());
    println!("  Centroid: ({:.4}, {:.4})", 
        cluster.centroid.0, cluster.centroid.1);
}

K-Means Parameters

ParameterDescription
kNumber of clusters to create
max_iterationsMaximum iterations (default: 100)

Choosing K

// Try different values of K and evaluate
for k in 2..=5 {
    let kmeans = KMeans::new(k);
    let result = kmeans.cluster(&events);
    
    // Calculate average cluster size
    let avg_size = events.len() as f64 / k as f64;
    println!("K={}: avg cluster size = {:.1}", k, avg_size);
}

Cluster Struct

Both algorithms return Cluster objects:

FieldTypeDescription
eventsVec<&Event>Events in the cluster
centroid(f64, f64)Geographic center (lat, lon)
boundsGeoBoundsBounding box

When to Use Which

AlgorithmBest For
DBSCANUnknown number of clusters, irregular shapes, noise handling
K-MeansKnown number of clusters, roughly equal-sized groups

Use Cases

Finding Activity Hotspots

let dbscan = DBSCAN::new(1000.0, 5);  // 1km, 5+ events
let result = dbscan.cluster(&events);

// Sort clusters by size
let mut clusters: Vec<_> = result.clusters.iter().collect();
clusters.sort_by(|a, b| b.events.len().cmp(&a.events.len()));

println!("Top activity hotspots:");
for (i, cluster) in clusters.iter().take(5).enumerate() {
    println!("  {}. {} events at ({:.4}, {:.4})",
        i + 1, cluster.events.len(),
        cluster.centroid.0, cluster.centroid.1);
}

Regional Grouping

// Group events into regions for separate analysis
let kmeans = KMeans::new(4);  // 4 regions
let result = kmeans.cluster(&events);

for (i, cluster) in result.clusters.iter().enumerate() {
    let region_name = match i {
        0 => "Northeast",
        1 => "Southeast", 
        2 => "Midwest",
        3 => "West",
        _ => "Unknown",
    };
    
    println!("{} region: {} events", region_name, cluster.events.len());
}

Outlier Detection

let dbscan = DBSCAN::new(5000.0, 2);
let result = dbscan.cluster(&events);

// Noise points are potential outliers or unique events
if !result.noise.is_empty() {
    println!("Potential outliers ({}):", result.noise.len());
    for event in &result.noise {
        println!("  - {} at ({:.4}, {:.4})", 
            event.text, 
            event.location.lat, 
            event.location.lon);
    }
}

Narrative Comparison

Compare narratives to find similarities and differences.

Basic Comparison

use spatial_narrative::analysis::{compare_narratives, ComparisonConfig};
use spatial_narrative::core::Narrative;

let similarity = compare_narratives(&narrative1, &narrative2, &ComparisonConfig::default());

println!("Similarity scores:");
println!("  Spatial: {:.2}", similarity.spatial);
println!("  Temporal: {:.2}", similarity.temporal);
println!("  Thematic: {:.2}", similarity.thematic);
println!("  Overall: {:.2}", similarity.overall);

NarrativeSimilarity

The comparison returns a NarrativeSimilarity struct:

FieldTypeDescription
spatialf64Geographic overlap (0.0 to 1.0)
temporalf64Time overlap (0.0 to 1.0)
thematicf64Tag/topic similarity (0.0 to 1.0)
overallf64Weighted average score

Scores range from 0.0 (no similarity) to 1.0 (identical).

ComparisonConfig

Customize how similarity is calculated:

use spatial_narrative::analysis::ComparisonConfig;

let config = ComparisonConfig {
    spatial_weight: 0.4,    // 40% weight on geography
    temporal_weight: 0.3,   // 30% weight on time
    thematic_weight: 0.3,   // 30% weight on themes
    spatial_threshold_km: 10.0,  // Events within 10km considered "same place"
    temporal_threshold_hours: 24.0,  // Events within 24h considered "same time"
};

let similarity = compare_narratives(&n1, &n2, &config);

Spatial Comparison Functions

Spatial Similarity

Calculate geographic overlap:

use spatial_narrative::analysis::spatial_similarity;

let score = spatial_similarity(&narrative1, &narrative2);
println!("Spatial similarity: {:.2}", score);  // 0.0 to 1.0

Spatial Intersection

Find events that occur in similar locations:

use spatial_narrative::analysis::spatial_intersection;

// Events within 5km of each other
let matching_pairs = spatial_intersection(&narrative1, &narrative2, 5000.0);

for (e1, e2) in matching_pairs {
    println!("Match: '{}' and '{}'", e1.text, e2.text);
    println!("  Distance: {:.0}m apart", 
        e1.location.distance_to(&e2.location));
}

Spatial Union

Get combined geographic bounds:

use spatial_narrative::analysis::spatial_union;

let combined_bounds = spatial_union(&narrative1, &narrative2);
println!("Combined area: {:.2}° x {:.2}°",
    combined_bounds.lat_span(),
    combined_bounds.lon_span());

Temporal Comparison

Temporal Similarity

use spatial_narrative::analysis::temporal_similarity;

let score = temporal_similarity(&narrative1, &narrative2);
println!("Temporal similarity: {:.2}", score);

Thematic Comparison

Thematic Similarity

Compare based on shared tags:

use spatial_narrative::analysis::thematic_similarity;

let score = thematic_similarity(&narrative1, &narrative2);
println!("Thematic similarity: {:.2}", score);

Common Locations

Find locations that appear in both narratives:

use spatial_narrative::analysis::common_locations;

// Locations within 1km considered the same
let shared = common_locations(&narrative1, &narrative2, 1000.0);

println!("Common locations ({}):", shared.len());
for loc in shared {
    println!("  ({:.4}, {:.4})", loc.lat, loc.lon);
}

Use Cases

let threshold = 0.5;  // 50% similarity threshold
let mut related = Vec::new();

for candidate in &all_narratives {
    let sim = compare_narratives(&target, candidate, &ComparisonConfig::default());
    if sim.overall > threshold {
        related.push((candidate, sim.overall));
    }
}

// Sort by similarity
related.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap());

println!("Related narratives:");
for (narrative, score) in related.iter().take(5) {
    println!("  {}: {:.0}% similar", 
        narrative.title.as_deref().unwrap_or("Untitled"),
        score * 100.0);
}

Detecting Duplicate Reports

let config = ComparisonConfig {
    spatial_threshold_km: 0.1,      // 100m
    temporal_threshold_hours: 1.0,   // 1 hour
    ..Default::default()
};

for i in 0..narratives.len() {
    for j in (i + 1)..narratives.len() {
        let sim = compare_narratives(&narratives[i], &narratives[j], &config);
        
        if sim.overall > 0.9 {
            println!("Potential duplicate:");
            println!("  '{}' and '{}'",
                narratives[i].title.as_deref().unwrap_or("?"),
                narratives[j].title.as_deref().unwrap_or("?"));
            println!("  Similarity: {:.0}%", sim.overall * 100.0);
        }
    }
}

Clustering Narratives by Topic

// Group narratives by thematic similarity
let mut groups: Vec<Vec<&Narrative>> = Vec::new();

for narrative in &narratives {
    let mut found_group = false;
    
    for group in &mut groups {
        // Check if similar to group representative
        let sim = thematic_similarity(narrative, group[0]);
        if sim > 0.6 {
            group.push(narrative);
            found_group = true;
            break;
        }
    }
    
    if !found_group {
        groups.push(vec![narrative]);
    }
}

println!("Found {} thematic groups", groups.len());
for (i, group) in groups.iter().enumerate() {
    println!("  Group {}: {} narratives", i + 1, group.len());
}

Text Processing Overview

The spatial_narrative library provides comprehensive text processing capabilities for extracting geographic information from unstructured text.

Modules

Text processing is split across two modules:

  • text - Named Entity Recognition (NER), ML-NER, and keyword extraction
  • parser - Geoparsing and coordinate detection

Key Features

Geoparsing (parser module)

Extract locations from text using multiple strategies:

  • Coordinate Detection: Decimal degrees, degrees with symbols, DMS format
  • Place Name Resolution: Built-in gazetteer with 2500+ world cities from GeoNames
  • Custom Gazetteers: Plug in your own place name databases or external APIs
use spatial_narrative::parser::{GeoParser, BuiltinGazetteer};

let gazetteer = BuiltinGazetteer::new();
let parser = GeoParser::with_gazetteer(Box::new(gazetteer));

let text = "The conference in Paris started at 48.8566°N, 2.3522°E.";
let mentions = parser.extract(text);

for mention in mentions {
    println!("Found: {} ({:?})", mention.text, mention.mention_type);
    if let Some(loc) = mention.location {
        println!("  -> {}, {}", loc.lat, loc.lon);
    }
}

Named Entity Recognition (text module)

Extract entities from narrative text using rule-based patterns:

use spatial_narrative::text::TextAnalyzer;

let analyzer = TextAnalyzer::new();
let text = "Dr. Smith visited Google headquarters in Mountain View on January 15, 2024.";
let entities = analyzer.entities(text);

for entity in entities {
    println!("{}: {} (confidence: {:.2})", entity.entity_type, entity.text, entity.confidence);
}

Keyword Extraction

Identify key terms and phrases:

use spatial_narrative::text::KeywordExtractor;

let extractor = KeywordExtractor::new();
let text = "Climate change affects coastal cities. Rising sea levels threaten coastal communities.";
let keywords = extractor.extract(text, 5);

for kw in keywords {
    println!("{}: {:.3}", kw.word, kw.score);
}

ML-NER (Advanced, requires ml-ner feature)

Use transformer-based models for high-accuracy entity extraction:

use spatial_narrative::text::{MlNerModel, NerModel};

// Auto-download and cache model (~65MB)
let model = MlNerModel::download_blocking(NerModel::DistilBertQuantized)?;

let text = "Dr. Chen presented her findings in Paris on March 15, 2024.";
let entities = model.extract(text)?;

for entity in entities {
    println!("{}: \"{}\" (confidence: {:.2})", entity.label, entity.text, entity.score);
}
// Output:
// PER: "Dr. Chen" (confidence: 0.99)
// LOC: "Paris" (confidence: 0.98)
// MISC: "March 15, 2024" (confidence: 0.95)

When to Use Each Module

TaskModuleKey Type
Extract coordinates from textparserGeoParser
Resolve place names to coordinatesparserBuiltinGazetteer
Extract entities (rule-based)textTextAnalyzer
Extract entities (ML, high accuracy)textMlNerModel
Find important keywordstextKeywordExtractor

Next Steps

Geoparsing

The GeoParser extracts geographic locations from text, detecting both coordinates and place names.

Basic Usage

use spatial_narrative::parser::GeoParser;

let parser = GeoParser::new();
let mentions = parser.extract("Meeting at 40.7128, -74.0060");

assert_eq!(mentions.len(), 1);
let loc = mentions[0].location.as_ref().unwrap();
println!("Latitude: {}, Longitude: {}", loc.lat, loc.lon);

Coordinate Formats

The parser detects three coordinate formats:

Decimal Degrees

40.7128, -74.0060
51.5074 -0.1278
-33.8688, 151.2093

Degrees with Symbols

40.7128°N, 74.0060°W
48.8566N 2.3522E
33.8688°S, 151.2093°E

DMS (Degrees, Minutes, Seconds)

40°42'46"N, 74°0'22"W
51°30'26"N, 0°7'40"W

Place Name Resolution

Use a gazetteer to resolve place names to coordinates:

use spatial_narrative::parser::{GeoParser, BuiltinGazetteer, MentionType};

let gazetteer = BuiltinGazetteer::new();
let parser = GeoParser::with_gazetteer(Box::new(gazetteer));

let text = "The conference was held in Tokyo and participants came from London.";
let mentions = parser.extract(text);

for mention in &mentions {
    if matches!(mention.mention_type, MentionType::PlaceName) {
        println!("Place: {}", mention.text);
    }
}

Built-in Gazetteer

The BuiltinGazetteer includes 2500+ world cities:

  • Major world cities selected by population from GeoNames
  • Cities across 150+ countries with precise coordinates
  • Common aliases (NYC → New York City, SHA → Shanghai, etc.)
  • Data licensed under CC BY 4.0
use spatial_narrative::parser::{BuiltinGazetteer, Gazetteer};

let gazetteer = BuiltinGazetteer::new();

// Direct lookup
if let Some(loc) = gazetteer.lookup("Paris") {
    println!("Paris: {}, {}", loc.lat, loc.lon);
}

// Aliases work too
assert!(gazetteer.contains("NYC"));
assert!(gazetteer.contains("New York City"));

Custom Gazetteer

Implement the Gazetteer trait for custom place databases:

use spatial_narrative::parser::Gazetteer;
use spatial_narrative::core::Location;
use std::collections::HashMap;

struct MyGazetteer {
    places: HashMap<String, (f64, f64)>,
}

impl Gazetteer for MyGazetteer {
    fn lookup(&self, name: &str) -> Option<Location> {
        self.places.get(&name.to_lowercase())
            .map(|(lat, lon)| Location::new(*lat, *lon))
    }

    fn contains(&self, name: &str) -> bool {
        self.places.contains_key(&name.to_lowercase())
    }

    fn all_names(&self) -> Vec<&str> {
        self.places.keys().map(|s| s.as_str()).collect()
    }
}

Configuration

Use LocationPattern to control what the parser detects:

use spatial_narrative::parser::{GeoParser, LocationPattern, BuiltinGazetteer};

// Only detect coordinates, not place names
let mut parser = GeoParser::new();
parser.set_pattern(LocationPattern::coordinates_only());

// Or customize fully
let pattern = LocationPattern {
    detect_decimal: true,
    detect_symbols: true,
    detect_dms: false,
    detect_places: false,
    min_confidence: 0.8,
};
parser.set_pattern(pattern);

LocationMention

Each detected location is returned as a LocationMention:

use spatial_narrative::parser::{GeoParser, MentionType};

let parser = GeoParser::new();
let mentions = parser.extract("Coordinates: 40.7128, -74.0060");

for mention in mentions {
    println!("Text: '{}'", mention.text);
    println!("Position: {}-{}", mention.start, mention.end);
    println!("Type: {:?}", mention.mention_type);
    println!("Confidence: {:.2}", mention.confidence);
    
    if let Some(loc) = mention.location {
        println!("Location: {}, {}", loc.lat, loc.lon);
    }
}

MentionType

  • DecimalDegrees - "40.7128, -74.0060"
  • DegreesWithSymbols - "40.7128°N, 74.0060°W"
  • DMS - "40°42'46"N, 74°0'22"W"
  • PlaceName - "Paris", "New York City"

Confidence Scores

Each mention has a confidence score (0.0 to 1.0):

TypeTypical Confidence
DMS0.99
DegreesWithSymbols0.98
DecimalDegrees0.95
PlaceName0.85

Coordinate formats have higher confidence because they're unambiguous. Place names are lower due to potential ambiguity (e.g., "Paris" could be Paris, France or Paris, Texas).

Named Entity Recognition

The TextAnalyzer provides basic Named Entity Recognition (NER) for extracting structured information from narrative text.

Basic Usage

use spatial_narrative::text::TextAnalyzer;

let analyzer = TextAnalyzer::new();
let text = "Dr. Smith met with CEO Johnson at Google headquarters.";
let entities = analyzer.entities(text);

for entity in entities {
    println!("{:?}: {}", entity.entity_type, entity.text);
}

Entity Types

The analyzer detects six entity types:

TypeDescriptionExamples
PersonPeople's names"Dr. Smith", "Mr. Johnson"
OrganizationCompanies, institutions"Google Inc.", "MIT", "NASA"
LocationPlace names"New York", "Mount Everest"
DateTimeDates and times"January 15, 2024", "March 2023"
NumericNumbers with units"$5.5 million", "100 km"
EventNamed events(custom additions)
OtherUnclassified entities-

Person Detection

Detects names with common titles:

use spatial_narrative::text::{TextAnalyzer, EntityType};

let analyzer = TextAnalyzer::new();
let text = "Dr. Jane Smith and Prof. Bob Johnson attended the meeting.";
let entities = analyzer.entities(text);

let people: Vec<_> = entities.iter()
    .filter(|e| matches!(e.entity_type, EntityType::Person))
    .collect();

for person in people {
    println!("Person: {}", person.text);
}

Recognized titles: Dr., Mr., Mrs., Ms., Miss, Prof., Professor, President, Senator, Governor, Mayor, Chief, Captain, General, Admiral, etc.

Organization Detection

Detects organizations by suffix patterns:

use spatial_narrative::text::{TextAnalyzer, EntityType};

let analyzer = TextAnalyzer::new();
let text = "Apple Inc. partnered with MIT and the World Health Organization.";
let entities = analyzer.entities(text);

let orgs: Vec<_> = entities.iter()
    .filter(|e| matches!(e.entity_type, EntityType::Organization))
    .collect();
// Found: "Apple Inc.", "MIT", "World Health Organization"

Recognized patterns:

  • Suffixes: Inc., Corp., LLC, Ltd., Co., Foundation, Institute, University, Organization
  • Acronyms: NASA, FBI, CIA, NATO, WHO, UN, EU, etc.

Date Detection

Extracts various date formats:

use spatial_narrative::text::{TextAnalyzer, EntityType};

let analyzer = TextAnalyzer::new();
let text = "The event on January 15, 2024 was rescheduled to March 2024.";
let entities = analyzer.entities(text);

let dates: Vec<_> = entities.iter()
    .filter(|e| matches!(e.entity_type, EntityType::DateTime))
    .collect();

Recognized formats:

  • Full dates: "January 15, 2024", "15 January 2024"
  • Month-year: "March 2024", "Jan 2024"
  • Abbreviated months: "Jan", "Feb", "Mar", etc.

Location Detection

Detects common location patterns and major places:

use spatial_narrative::text::{TextAnalyzer, EntityType};

let analyzer = TextAnalyzer::new();
let text = "The company has offices in New York, London, and Tokyo.";
let entities = analyzer.entities(text);

let locations: Vec<_> = entities.iter()
    .filter(|e| matches!(e.entity_type, EntityType::Location))
    .collect();

Built-in location database includes major world cities and countries.

Numeric Detection

Extracts numbers with units:

use spatial_narrative::text::{TextAnalyzer, EntityType};

let analyzer = TextAnalyzer::new();
let text = "The project cost $5.5 million and covered 100 kilometers.";
let entities = analyzer.entities(text);

let numerics: Vec<_> = entities.iter()
    .filter(|e| matches!(e.entity_type, EntityType::Numeric))
    .collect();
// Found: "$5.5 million", "100 kilometers"

Recognized patterns:

  • Currency: "$5.5 million", "€100"
  • Distance: "100 km", "50 miles"
  • Percentages: "25%", "75 percent"
  • General: "1,000", "3.14"

Tokenization

The analyzer also provides text tokenization:

use spatial_narrative::text::TextAnalyzer;

let analyzer = TextAnalyzer::new();
let text = "Hello, world! This is a test.";

// All tokens
let tokens = analyzer.tokenize(text);
// ["Hello", ",", "world", "!", "This", "is", "a", "test", "."]

// Words only (no punctuation)
let words = analyzer.tokenize_words(text);
// ["Hello", "world", "This", "is", "a", "test"]

Sentence Splitting

Split text into sentences:

use spatial_narrative::text::TextAnalyzer;

let analyzer = TextAnalyzer::new();
let text = "First sentence. Second sentence! Third sentence?";
let sentences = analyzer.sentences(text);

assert_eq!(sentences.len(), 3);

Custom Locations

Add custom location names:

use spatial_narrative::text::TextAnalyzer;

let mut analyzer = TextAnalyzer::new();
analyzer.add_location("Springfield");
analyzer.add_location("Gotham City");

let text = "The hero saved Gotham City.";
let entities = analyzer.entities(text);
// Now detects "Gotham City" as a location

Confidence Scores

Each entity has a confidence score:

use spatial_narrative::text::TextAnalyzer;

let analyzer = TextAnalyzer::new();
let entities = analyzer.entities("Dr. Smith works at NASA.");

for entity in entities {
    println!("{}: {} (confidence: {:.2})", 
        entity.entity_type, 
        entity.text, 
        entity.confidence
    );
}

Confidence levels:

  • 0.9+: High confidence (clear patterns like "Dr. Smith")
  • 0.7-0.9: Medium confidence
  • 0.5-0.7: Lower confidence (may need verification)

Limitations

This is a rule-based NER system, not a machine learning model:

  • ✅ Fast and deterministic
  • ✅ No external dependencies
  • ✅ Works offline
  • ❌ May miss unconventional patterns
  • ❌ Limited to English
  • ❌ No context-aware disambiguation

For production NLP tasks requiring high accuracy, consider integrating with external NLP services or ML models.

Machine Learning Named Entity Recognition

The ml-ner feature provides state-of-the-art Named Entity Recognition using transformer models (BERT, RoBERTa, DistilBERT) via ONNX Runtime.

Features

  • High Accuracy: Transformer-based models trained on CoNLL-2003
  • Auto-Download: Automatically fetch models from HuggingFace Hub
  • Caching: Models are cached locally after first download
  • Multiple Models: Choose from 5 pre-trained models or bring your own
  • Multi-language: Support for 40+ languages with the multilingual model

Entity Types

All pre-trained models recognize four entity types:

  • LOC - Locations (cities, countries, regions)
  • PER - Persons (names of people)
  • ORG - Organizations (companies, institutions)
  • MISC - Miscellaneous (dates, events, products)

Installation

Add the ml-ner-download feature to enable both ML-NER and auto-download:

[dependencies]
spatial-narrative = { version = "0.1", features = ["ml-ner-download"] }

Or use just ml-ner if you want to provide your own models:

[dependencies]
spatial-narrative = { version = "0.1", features = ["ml-ner"] }

ONNX Runtime Setup

ML-NER requires ONNX Runtime to be installed. You have several options:

Option 1: Environment Variable

Set ORT_DYLIB_PATH to point to your ONNX Runtime library:

# macOS
export ORT_DYLIB_PATH=/path/to/libonnxruntime.dylib

# Linux
export ORT_DYLIB_PATH=/path/to/libonnxruntime.so

# Windows
set ORT_DYLIB_PATH=C:\path\to\onnxruntime.dll

Option 2: Install via Package Manager

macOS (Homebrew):

brew install onnxruntime
export ORT_DYLIB_PATH=$(brew --prefix onnxruntime)/lib/libonnxruntime.dylib

Linux (Ubuntu/Debian):

sudo apt install libonnxruntime
export ORT_DYLIB_PATH=/usr/lib/libonnxruntime.so

Option 3: Manual Download

Download from ONNX Runtime releases:

  1. Download the appropriate package for your platform
  2. Extract the archive
  3. Set ORT_DYLIB_PATH to the library file

Available Models

ModelSizeF1 ScoreSpeedLanguages
DistilBertQuantized~65MB~90%FastEnglish
DistilBert~250MB~90%FastEnglish
BertBase~400MB~91%MediumEnglish
BertLarge~1.2GB~93%SlowEnglish
Multilingual~700MB~90%Medium40+ languages

The DistilBertQuantized model is recommended for most use cases, offering the best balance of size, speed, and accuracy.

Basic Usage

The simplest way to get started:

use spatial_narrative::text::{MlNerModel, NerModel};

// First run downloads ~65MB, subsequent runs load from cache
let model = MlNerModel::download_blocking(NerModel::DistilBertQuantized)?;

let text = "Dr. Sarah Chen presented her research in Paris on March 15, 2024.";
let entities = model.extract(text)?;

for entity in entities {
    println!("{}: \"{}\" (confidence: {:.2})", 
        entity.label, entity.text, entity.score);
}
// Output:
// PER: "Dr. Sarah Chen" (confidence: 0.99)
// LOC: "Paris" (confidence: 0.98)
// MISC: "March 15, 2024" (confidence: 0.95)

With Progress Reporting

Show download progress for large models:

use spatial_narrative::text::{MlNerModel, NerModel};

let model = MlNerModel::download_blocking_with_progress(
    NerModel::DistilBertQuantized,
    |downloaded, total| {
        if total > 0 {
            let pct = (downloaded as f64 / total as f64) * 100.0;
            println!("Downloading: {:.1}%", pct);
        }
    }
)?;

Using Different Models

use spatial_narrative::text::{MlNerModel, NerModel};

// For best accuracy (larger download)
let model = MlNerModel::download_blocking(NerModel::BertLarge)?;

// For multilingual text
let model = MlNerModel::download_blocking(NerModel::Multilingual)?;

// For custom HuggingFace models
let model = MlNerModel::download_blocking(
    NerModel::Custom("my-org/my-ner-model".into())
)?;

Advanced Usage

Manual Model Loading

If you have pre-downloaded ONNX models:

use spatial_narrative::text::MlNerModel;

let model = MlNerModel::from_directory("./my-ner-model/")?;
// Directory should contain: model.onnx, tokenizer.json, config.json

Cache Management

use spatial_narrative::text::{
    model_cache_dir,
    model_cache_path,
    is_model_cached,
    cache_size_bytes,
    clear_model_cache,
    NerModel,
};

// Check cache location
println!("Cache dir: {:?}", model_cache_dir());

// Check if a model is cached
let model = NerModel::DistilBertQuantized;
if is_model_cached(&model) {
    println!("Model already cached at: {:?}", model_cache_path(&model));
}

// Get total cache size
if let Ok(size) = cache_size_bytes() {
    println!("Cache size: {:.2} MB", size as f64 / 1024.0 / 1024.0);
}

// Clear cache for a specific model
clear_model_cache(Some(&model))?;

// Clear all cached models
clear_model_cache(None)?;

Async API

For async applications, use the async API:

use spatial_narrative::text::{MlNerModel, NerModel};

let model = MlNerModel::download(NerModel::DistilBertQuantized).await?;
let entities = model.extract("Text to analyze")?;

Integration with Geoparsing

Combine ML-NER with gazetteer lookup for comprehensive geoparsing:

use spatial_narrative::text::{MlNerModel, NerModel};
use spatial_narrative::parser::{BuiltinGazetteer, Gazetteer};

// Extract entities with ML
let ml_model = MlNerModel::download_blocking(NerModel::DistilBertQuantized)?;
let text = "The summit was held in Geneva, Switzerland.";
let ml_entities = ml_model.extract(text)?;

// Resolve locations with gazetteer
let gazetteer = BuiltinGazetteer::new();

for ml_entity in ml_entities {
    if ml_entity.label == "LOC" {
        // Convert to standard Entity and lookup coordinates
        let entity = ml_entity.to_entity();
        
        if let Some(location) = gazetteer.lookup(&entity.text) {
            println!("{} is at {}, {}", 
                entity.text, location.lat, location.lon);
        }
    }
}

Entity Structure

The MlEntity struct provides detailed extraction results:

pub struct MlEntity {
    /// Entity type: "LOC", "PER", "ORG", or "MISC"
    pub label: String,
    
    /// The extracted text
    pub text: String,
    
    /// Confidence score (0.0 to 1.0)
    pub score: f64,
    
    /// Character position in original text
    pub start: usize,
    
    /// End position in original text
    pub end: usize,
}

Convert to standard Entity for use with other components:

let entity = ml_entity.to_entity();
// Returns Entity with appropriate EntityType enum

Example Application

See the complete example:

cargo run --example ml_ner_download --features ml-ner-download

This example demonstrates:

  • Checking cache status
  • Auto-downloading models
  • Extracting entities from various texts
  • Integration with geoparsing workflow

Exporting Custom Models

To use your own fine-tuned models:

  1. Train or fine-tune a token classification model on HuggingFace
  2. Export to ONNX using Optimum:
pip install optimum[exporters]
optimum-cli export onnx --model your-model-name ./output-dir/
  1. Load in spatial-narrative:
let model = MlNerModel::from_directory("./output-dir/")?;

Or host on HuggingFace Hub and use:

let model = MlNerModel::download_blocking(
    NerModel::Custom("your-org/your-model".into())
)?;

Performance Tips

  1. Choose the right model: Use DistilBertQuantized for most applications
  2. Cache models: First download takes time, but subsequent loads are fast
  3. Batch processing: Process multiple texts in sequence after loading once
  4. Model lifecycle: Keep the model in memory for repeated extractions
  5. Async for I/O: Use async API when downloading in web servers

Troubleshooting

ONNX Runtime Not Found

If you see errors about ONNX Runtime:

  1. Install ONNX Runtime (see setup section above)
  2. Set ORT_DYLIB_PATH environment variable
  3. Verify the path points to the correct library file

Model Download Fails

  • Check internet connection
  • Verify HuggingFace Hub is accessible
  • Try clearing cache: clear_model_cache(None)?
  • Check cache directory permissions

Low Accuracy

  • Try a larger model (BertBase or BertLarge)
  • For non-English text, use the Multilingual model
  • Consider fine-tuning a custom model on your domain

Cache Locations

Models are cached in platform-specific directories:

  • Linux: ~/.cache/spatial-narrative/models/
  • macOS: ~/Library/Caches/spatial-narrative/models/
  • Windows: %LOCALAPPDATA%\spatial-narrative\models\

Each model has its own subdirectory containing:

  • model.onnx - The neural network model
  • tokenizer.json - Text tokenization configuration
  • config.json - Label mappings and metadata

License Notes

  • DistilBERT models: Apache 2.0 License
  • BERT models: Apache 2.0 License
  • Multilingual model: CC BY-NC-SA 4.0 License (non-commercial)
  • ONNX Runtime: MIT License

Check individual model licenses on HuggingFace Hub before commercial use.

Keyword Extraction

The KeywordExtractor identifies important terms and phrases from text using term frequency analysis.

Basic Usage

use spatial_narrative::text::KeywordExtractor;

let extractor = KeywordExtractor::new();
let text = "Climate change affects coastal cities. Rising sea levels threaten coastal communities worldwide.";
let keywords = extractor.extract(text, 5);

for kw in keywords {
    println!("{}: {:.3}", kw.text, kw.score);
}

How It Works

The extractor uses a TF (Term Frequency) based approach:

  1. Tokenization: Text is split into words
  2. Normalization: Words are lowercased
  3. Filtering: Stop words and short words are removed
  4. Scoring: Words are scored by frequency
  5. Ranking: Top N keywords are returned

Keyword Struct

Each extracted keyword contains:

use spatial_narrative::text::{KeywordExtractor, Keyword};

let extractor = KeywordExtractor::new();
let keywords = extractor.extract("Machine learning is transforming technology.", 3);

for kw in keywords {
    println!("Word: {}", kw.text);
    println!("Score: {:.3}", kw.score);
    println!("Frequency: {}", kw.frequency);
}
FieldTypeDescription
textStringThe keyword text
scoref64Normalized score (0.0 to 1.0)
frequencyusizeRaw occurrence count

Configuration

Minimum Word Length

Skip short words:

use spatial_narrative::text::KeywordExtractor;

let extractor = KeywordExtractor::new().min_length(4); // Skip words shorter than 4 chars

let text = "The big cat sat on the mat.";
let keywords = extractor.extract(text, 10);
// Only extracts words with 4+ characters

Custom Stop Words

Add domain-specific stop words:

use spatial_narrative::text::KeywordExtractor;

let extractor = KeywordExtractor::new();

let text = "Data analysis reveals important data patterns in analysis.";
let keywords = extractor.extract_with_stopwords(text, 5, &["data", "analysis"]);
// "data" and "analysis" are now filtered out

Phrase Extraction

Extract multi-word phrases (n-grams):

use spatial_narrative::text::KeywordExtractor;

// Configure max phrase length (includes bigrams and trigrams)
let extractor = KeywordExtractor::new().max_phrase_length(2);
let text = "Machine learning and deep learning are subfields of artificial intelligence.";
let phrases = extractor.extract(text, 3); // Top 3 terms (including bigrams)

for phrase in phrases {
    println!("{}: {:.3}", phrase.text, phrase.score);
}
// Example: "machine learning", "deep learning", "artificial intelligence"

The max_phrase_length parameter controls n-gram extraction (2 for bigrams, 3 for trigrams).

Built-in Stop Words

The default stop word list includes common English words:

the, a, an, is, are, was, were, be, been, being,
have, has, had, do, does, did, will, would, could,
should, may, might, must, shall, can, need, dare,
and, or, but, if, then, else, when, where, why,
how, all, each, every, both, few, more, most, other,
some, such, no, not, only, same, so, than, too, very,
just, also, now, here, there, this, that, these, those,
i, you, he, she, it, we, they, me, him, her, us, them,
my, your, his, its, our, their, mine, yours, hers, ours,
what, which, who, whom, whose, of, in, to, for, with,
on, at, by, from, as, into, through, during, before,
after, above, below, between, under, again, further,
once, about, up, down, out, off, over, own, because

Use Cases

Document Summarization

Extract key themes from a document:

use spatial_narrative::text::KeywordExtractor;

let extractor = KeywordExtractor::new();
let document = "Long document text...";
let themes = extractor.extract(document, 10);

println!("Key themes:");
for theme in themes {
    println!("  - {}", theme.text);
}

Narrative Tagging

Auto-generate tags for narratives:

use spatial_narrative::text::KeywordExtractor;
use spatial_narrative::core::Narrative;

let extractor = KeywordExtractor::new();

fn auto_tag(narrative: &Narrative, extractor: &KeywordExtractor) -> Vec<String> {
    // Combine all event descriptions
    let text: String = narrative.events()
        .filter_map(|e| e.description.clone())
        .collect::<Vec<_>>()
        .join(" ");
    
    // Extract top keywords as tags
    extractor.extract(&text, 5)
        .into_iter()
        .map(|kw| kw.text)
        .collect()
}

Topic Comparison

Compare topics between documents:

use spatial_narrative::text::KeywordExtractor;
use std::collections::HashSet;

let extractor = KeywordExtractor::new();

let doc1_keywords: HashSet<_> = extractor.extract(doc1, 20)
    .into_iter()
    .map(|kw| kw.word)
    .collect();

let doc2_keywords: HashSet<_> = extractor.extract(doc2, 20)
    .into_iter()
    .map(|kw| kw.word)
    .collect();

let common: HashSet<_> = doc1_keywords.intersection(&doc2_keywords).collect();
println!("Common topics: {:?}", common);

Performance

The extractor is optimized for speed:

  • O(n) tokenization
  • O(n log n) sorting for top-k selection
  • Minimal memory allocation

Typical performance: ~10,000 words/ms on modern hardware.

Limitations

  • English-focused (stop word list is English)
  • No stemming or lemmatization
  • TF-only (no IDF weighting)
  • Case-insensitive matching only

For advanced keyword extraction with TF-IDF or semantic analysis, consider using specialized NLP libraries.

Common Patterns

Practical patterns for working with spatial narratives.

Creating Events

From Raw Data

use spatial_narrative::core::{Event, EventBuilder, Location, Timestamp};

// Simple event
let event = Event::new(
    Location::new(40.7128, -74.0060),
    Timestamp::parse("2024-01-15T10:00:00Z").unwrap(),
    "Event description"
);

// Rich event with builder
let event = EventBuilder::new()
    .location(Location::builder()
        .lat(40.7128)
        .lon(-74.0060)
        .name("New York City")
        .build()
        .unwrap())
    .timestamp(Timestamp::parse("2024-01-15T10:00:00Z").unwrap())
    .text("Conference begins")
    .tag("conference")
    .tag("technology")
    .build();

From Database Records

fn event_from_row(row: &Row) -> Result<Event> {
    let event = EventBuilder::new()
        .location(Location::new(
            row.get::<f64>("latitude")?,
            row.get::<f64>("longitude")?
        ))
        .timestamp(Timestamp::parse(&row.get::<String>("datetime")?)?)
        .text(row.get::<String>("description")?)
        .build();
    Ok(event)
}

From CSV Line

fn event_from_csv(line: &str) -> Result<Event> {
    let parts: Vec<&str> = line.split(',').collect();
    let event = Event::new(
        Location::new(parts[0].parse()?, parts[1].parse()?),
        Timestamp::parse(parts[2])?,
        parts[3]
    );
    Ok(event)
}

Building Narratives

From Event Collection

use spatial_narrative::core::{Narrative, NarrativeBuilder};

let narrative = NarrativeBuilder::new()
    .title("My Narrative")
    .description("A collection of events")
    .author("Research Team")
    .events(events)
    .tag("research")
    .build();

Incremental Building

let mut builder = NarrativeBuilder::new()
    .title("Growing Narrative");

for data in data_stream {
    if let Ok(event) = process_data(data) {
        builder = builder.event(event);
    }
}

let narrative = builder.build();

Filtering Patterns

By Location

use spatial_narrative::core::GeoBounds;

let nyc_bounds = GeoBounds::new(40.4, -74.3, 41.0, -73.7);
let nyc_events = narrative.filter_spatial(&nyc_bounds);

By Time

use spatial_narrative::core::TimeRange;

let january = TimeRange::month(2024, 1);
let january_events = narrative.filter_temporal(&january);

By Tags

let important: Vec<_> = narrative.events.iter()
    .filter(|e| e.has_tag("important"))
    .collect();

Combined Filters

let filtered: Vec<_> = narrative.events.iter()
    .filter(|e| nyc_bounds.contains(&e.location))
    .filter(|e| january.contains(&e.timestamp))
    .filter(|e| e.has_tag("verified"))
    .collect();

Index Usage Patterns

Build Once, Query Many

use spatial_narrative::index::SpatiotemporalIndex;

// Build index once
let index = SpatiotemporalIndex::from_iter(
    events.iter().cloned(),
    |e| &e.location,
    |e| &e.timestamp
);

// Query multiple times
let q1 = index.query(&bounds1, &range1);
let q2 = index.query(&bounds2, &range2);
let q3 = index.query_spatial(&bounds3);
use spatial_narrative::index::SpatialIndex;

let index = SpatialIndex::from_iter(events, |e| &e.location);

// Find 5 nearest events to a point
let nearest = index.nearest(user_lat, user_lon, 5);

Graph Patterns

Complete Analysis Pipeline

use spatial_narrative::graph::{NarrativeGraph, EdgeType};

// Build graph
let mut graph = NarrativeGraph::from_events(events);

// Apply connection strategies
graph.connect_temporal();
graph.connect_spatial(10.0);
graph.connect_thematic();

// Analyze
let roots = graph.roots();      // Starting points
let leaves = graph.leaves();    // End points
let hub_count = graph.nodes()
    .filter(|(id, _)| graph.in_degree(*id) + graph.out_degree(*id) > 5)
    .count();

println!("Graph: {} roots, {} leaves, {} hubs", 
    roots.len(), leaves.len(), hub_count);

Path Analysis

// Find how events are connected
let start = graph.get_node(&event1.id).unwrap();
let end = graph.get_node(&event2.id).unwrap();

if let Some(path) = graph.shortest_path(start, end) {
    println!("Connected via {} events", path.nodes.len());
}

IO Patterns

Multi-Format Export

use spatial_narrative::io::{Format, GeoJsonFormat, CsvFormat, JsonFormat};

// Export to all formats
let geojson = GeoJsonFormat::new().export_str(&narrative)?;
let csv = CsvFormat::new().export_str(&narrative)?;
let json = JsonFormat::pretty().export_str(&narrative)?;

std::fs::write("data.geojson", &geojson)?;
std::fs::write("data.csv", &csv)?;
std::fs::write("data.json", &json)?;

Round-Trip Preservation

// Use JSON for lossless round-trips
let json = JsonFormat::new();
let exported = json.export_str(&narrative)?;
let imported = json.import_str(&exported)?;

assert_eq!(narrative.events.len(), imported.events.len());

Analysis Patterns

Quick Metrics

use spatial_narrative::analysis::{SpatialMetrics, TemporalMetrics};

let spatial = SpatialMetrics::from_events(&events);
let temporal = TemporalMetrics::from_events(&events);

println!("Spatial extent: {:.0} km", spatial.max_extent / 1000.0);
println!("Time span: {:?}", temporal.duration);
println!("Event rate: {:.2}/hour", 
    events.len() as f64 / temporal.duration.as_secs_f64() * 3600.0);

Clustering

use spatial_narrative::analysis::DBSCAN;

let dbscan = DBSCAN::new(5000.0, 3);  // 5km radius, min 3 points
let result = dbscan.cluster(&events);

println!("Found {} clusters, {} noise points", 
    result.num_clusters(), 
    result.noise.len());

Error Handling

Graceful Parsing

let events: Vec<Event> = raw_data.iter()
    .filter_map(|row| {
        match parse_event(row) {
            Ok(event) => Some(event),
            Err(e) => {
                eprintln!("Skipping invalid row: {}", e);
                None
            }
        }
    })
    .collect();

Validation

fn validate_event(event: &Event) -> Result<()> {
    if event.text.is_empty() {
        return Err(Error::Validation("Event text is empty".into()));
    }
    if event.location.lat < -90.0 || event.location.lat > 90.0 {
        return Err(Error::Validation("Invalid latitude".into()));
    }
    Ok(())
}

Filtering Events

Techniques for filtering and selecting events.

Spatial Filtering

By Bounding Box

use spatial_narrative::core::GeoBounds;

// Define bounds
let nyc = GeoBounds::new(40.4, -74.3, 41.0, -73.7);

// Filter narrative
let nyc_narrative = narrative.filter_spatial(&nyc);

// Or filter events directly
let nyc_events: Vec<_> = events.iter()
    .filter(|e| nyc.contains(&e.location))
    .collect();

By Radius

use spatial_narrative::analysis::haversine_distance;
use spatial_narrative::core::Location;

let center = Location::new(40.7128, -74.0060);
let radius_m = 5000.0;  // 5km

let nearby: Vec<_> = events.iter()
    .filter(|e| haversine_distance(&center, &e.location) <= radius_m)
    .collect();

By Multiple Regions

let regions = vec![
    GeoBounds::new(40.4, -74.3, 41.0, -73.7),   // NYC
    GeoBounds::new(33.7, -118.7, 34.4, -117.9), // LA
    GeoBounds::new(41.6, -88.0, 42.1, -87.5),   // Chicago
];

let in_any_region: Vec<_> = events.iter()
    .filter(|e| regions.iter().any(|r| r.contains(&e.location)))
    .collect();

Temporal Filtering

By Time Range

use spatial_narrative::core::TimeRange;

// Specific range
let range = TimeRange::new(
    Timestamp::parse("2024-01-01T00:00:00Z").unwrap(),
    Timestamp::parse("2024-01-31T23:59:59Z").unwrap(),
);
let january = narrative.filter_temporal(&range);

// Convenience constructors
let q1 = events.iter()
    .filter(|e| TimeRange::month(2024, 1).contains(&e.timestamp)
             || TimeRange::month(2024, 2).contains(&e.timestamp)
             || TimeRange::month(2024, 3).contains(&e.timestamp));

Before/After

let cutoff = Timestamp::parse("2024-06-01T00:00:00Z").unwrap();

let before: Vec<_> = events.iter()
    .filter(|e| e.timestamp < cutoff)
    .collect();

let after: Vec<_> = events.iter()
    .filter(|e| e.timestamp >= cutoff)
    .collect();

By Day of Week

let weekends: Vec<_> = events.iter()
    .filter(|e| {
        let weekday = e.timestamp.weekday();
        weekday == 0 || weekday == 6  // Sunday or Saturday
    })
    .collect();

By Time of Day

// Business hours (9 AM to 5 PM)
let business_hours: Vec<_> = events.iter()
    .filter(|e| {
        let hour = e.timestamp.hour();
        hour >= 9 && hour < 17
    })
    .collect();

Tag Filtering

Single Tag

let important: Vec<_> = events.iter()
    .filter(|e| e.has_tag("important"))
    .collect();

Any of Tags

let priority_tags = ["urgent", "important", "critical"];

let priority: Vec<_> = events.iter()
    .filter(|e| priority_tags.iter().any(|t| e.has_tag(t)))
    .collect();

All of Tags

let required_tags = ["verified", "published"];

let complete: Vec<_> = events.iter()
    .filter(|e| required_tags.iter().all(|t| e.has_tag(t)))
    .collect();

Excluding Tags

let not_spam: Vec<_> = events.iter()
    .filter(|e| !e.has_tag("spam") && !e.has_tag("duplicate"))
    .collect();

Text Filtering

Contains Keyword

let mentions_storm: Vec<_> = events.iter()
    .filter(|e| e.text.to_lowercase().contains("storm"))
    .collect();

Regex Matching

use regex::Regex;

let phone_pattern = Regex::new(r"\d{3}-\d{3}-\d{4}").unwrap();

let with_phone: Vec<_> = events.iter()
    .filter(|e| phone_pattern.is_match(&e.text))
    .collect();

Minimum Length

let substantive: Vec<_> = events.iter()
    .filter(|e| e.text.len() >= 50)
    .collect();

Combined Filtering

Chained Filters

let filtered: Vec<_> = events.iter()
    .filter(|e| nyc_bounds.contains(&e.location))
    .filter(|e| january.contains(&e.timestamp))
    .filter(|e| e.has_tag("verified"))
    .filter(|e| !e.has_tag("duplicate"))
    .collect();

With Predicate Function

fn is_relevant(event: &Event) -> bool {
    // Complex filtering logic
    let in_region = NYC_BOUNDS.contains(&event.location);
    let in_timeframe = STUDY_PERIOD.contains(&event.timestamp);
    let has_content = event.text.len() >= 10;
    let is_verified = event.has_tag("verified");
    
    in_region && in_timeframe && has_content && is_verified
}

let relevant: Vec<_> = events.iter()
    .filter(|e| is_relevant(e))
    .collect();

Using Indexes for Filtering

Spatial Index

use spatial_narrative::index::SpatialIndex;

let index = SpatialIndex::from_iter(events.clone(), |e| &e.location);

// Fast bounding box query
let in_region = index.query_bounds(&bounds);

// Fast radius query
let nearby = index.query_radius_meters(lat, lon, radius);

Temporal Index

use spatial_narrative::index::TemporalIndex;

let index = TemporalIndex::from_iter(events.clone(), |e| &e.timestamp);

// Fast range query
let in_period = index.query_range(&time_range);

// Ordered access
let chronological = index.chronological();

Combined Index

use spatial_narrative::index::SpatiotemporalIndex;

let index = SpatiotemporalIndex::from_iter(
    events.clone(), 
    |e| &e.location, 
    |e| &e.timestamp
);

// Query both dimensions efficiently
let filtered = index.query(&bounds, &time_range);

Sampling

Random Sample

use rand::seq::SliceRandom;

let mut rng = rand::thread_rng();
let sample: Vec<_> = events.choose_multiple(&mut rng, 100).collect();

Systematic Sample

// Every 10th event
let systematic: Vec<_> = events.iter()
    .enumerate()
    .filter(|(i, _)| i % 10 == 0)
    .map(|(_, e)| e)
    .collect();

Stratified Sample

// Sample from each region
let mut samples = Vec::new();
for region in &regions {
    let in_region: Vec<_> = events.iter()
        .filter(|e| region.contains(&e.location))
        .collect();
    samples.extend(in_region.choose_multiple(&mut rng, 10));
}

Integration Examples

spatial-narrative is designed to fit into your existing data pipeline. Here's how to integrate it with common tools.

Data Ingestion

HTTP APIs (with reqwest)

Fetch data from any API and transform into Events:

use reqwest::blocking::Client;
use spatial_narrative::core::{Event, EventBuilder, Location, Timestamp, NarrativeBuilder};

fn fetch_and_process() -> Result<Narrative, Box<dyn std::error::Error>> {
    let client = Client::new();

    // Fetch from your data source
    let response: Vec<MyApiRecord> = client
        .get("https://api.example.com/events")
        .send()?
        .json()?;

    // Transform to Events
    let events: Vec<Event> = response
        .into_iter()
        .filter_map(|r| {
            Some(EventBuilder::new()
                .location(Location::new(r.lat, r.lon))
                .timestamp(Timestamp::parse(&r.date).ok()?)
                .text(&r.description)
                .build())
        })
        .collect();

    Ok(NarrativeBuilder::new()
        .title("API Data")
        .events(events)
        .build())
}

CSV Files

use csv::Reader;
use spatial_narrative::core::{Event, EventBuilder, Location, Timestamp};

fn load_csv(path: &str) -> Vec<Event> {
    let mut rdr = Reader::from_path(path).unwrap();

    rdr.deserialize()
        .filter_map(|result| {
            let record: MyRecord = result.ok()?;
            Some(EventBuilder::new()
                .location(Location::new(record.lat, record.lon))
                .timestamp(Timestamp::parse(&record.date).ok()?)
                .text(&record.text)
                .build())
        })
        .collect()
}

JSON Files

use serde_json;
use spatial_narrative::core::{Event, Location, Timestamp};

#[derive(serde::Deserialize)]
struct RawEvent {
    lat: f64,
    lon: f64,
    time: String,
    description: String,
}

fn load_json(path: &str) -> Vec<Event> {
    let data = std::fs::read_to_string(path).unwrap();
    let raw: Vec<RawEvent> = serde_json::from_str(&data).unwrap();

    raw.into_iter()
        .filter_map(|r| {
            Some(Event::new(
                Location::new(r.lat, r.lon),
                Timestamp::parse(&r.time).ok()?,
                &r.description
            ))
        })
        .collect()
}

Web Mapping

Leaflet

Export to GeoJSON and display with Leaflet:

use spatial_narrative::io::{GeoJsonFormat, Format};

let geojson = GeoJsonFormat::new().export_str(&narrative)?;
std::fs::write("public/events.geojson", &geojson)?;
<!DOCTYPE html>
<html>
<head>
    <link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css" />
    <script src="https://unpkg.com/leaflet@1.9.4/dist/leaflet.js"></script>
</head>
<body>
    <div id="map" style="height: 100vh;"></div>
    <script>
        const map = L.map('map').setView([40.7128, -74.0060], 10);
        
        L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
            attribution: '© OpenStreetMap'
        }).addTo(map);
        
        fetch('events.geojson')
            .then(res => res.json())
            .then(data => {
                L.geoJSON(data, {
                    pointToLayer: (feature, latlng) => {
                        return L.circleMarker(latlng, {
                            radius: 8,
                            fillColor: '#ff7800',
                            color: '#000',
                            weight: 1,
                            fillOpacity: 0.8
                        });
                    },
                    onEachFeature: (feature, layer) => {
                        layer.bindPopup(`
                            <strong>${feature.properties.text}</strong><br>
                            ${feature.properties.timestamp}
                        `);
                    }
                }).addTo(map);
            });
    </script>
</body>
</html>

Mapbox GL JS

mapboxgl.accessToken = 'your-token';
const map = new mapboxgl.Map({
    container: 'map',
    style: 'mapbox://styles/mapbox/streets-v11',
    center: [-74.0060, 40.7128],
    zoom: 10
});

map.on('load', () => {
    map.addSource('events', {
        type: 'geojson',
        data: 'events.geojson'
    });
    
    map.addLayer({
        id: 'events-layer',
        type: 'circle',
        source: 'events',
        paint: {
            'circle-radius': 8,
            'circle-color': '#ff7800'
        }
    });
});

Data Science

Python/Pandas

Export to CSV for pandas:

use spatial_narrative::io::{CsvFormat, Format};

let csv = CsvFormat::new().export_str(&narrative)?;
std::fs::write("events.csv", &csv)?;
import pandas as pd
import geopandas as gpd

# Read CSV
df = pd.read_csv('events.csv', parse_dates=['timestamp'])

# Basic analysis
print(f"Events: {len(df)}")
print(f"Date range: {df['timestamp'].min()} to {df['timestamp'].max()}")

# Convert to GeoDataFrame
gdf = gpd.GeoDataFrame(
    df, 
    geometry=gpd.points_from_xy(df.longitude, df.latitude),
    crs="EPSG:4326"
)

# Spatial operations
gdf.plot()

R

library(tidyverse)
library(sf)

# Read CSV
events <- read_csv("events.csv")

# Convert to sf object
events_sf <- st_as_sf(events, 
    coords = c("longitude", "latitude"), 
    crs = 4326)

# Plot
ggplot(events_sf) +
    geom_sf() +
    theme_minimal()

GIS Software

QGIS

  1. Export to GeoJSON:
let geojson = GeoJsonFormat::new().export_str(&narrative)?;
std::fs::write("events.geojson", &geojson)?;
  1. In QGIS: Layer → Add Layer → Add Vector Layer
  2. Select the GeoJSON file

ArcGIS

Export to GeoJSON, then import as feature class.

Graph Visualization

Graphviz

let dot = graph.to_dot();
std::fs::write("graph.dot", &dot)?;
# Render to PNG
dot -Tpng graph.dot -o graph.png

# Render to SVG
dot -Tsvg graph.dot -o graph.svg

# Different layouts
neato -Tpng graph.dot -o graph-neato.png
circo -Tpng graph.dot -o graph-circo.png

D3.js Force Graph

let json = graph.to_json();
std::fs::write("public/graph.json", &json)?;
<script src="https://d3js.org/d3.v7.min.js"></script>
<script>
    fetch('graph.json')
        .then(res => res.json())
        .then(data => {
            const svg = d3.select("svg");
            const width = +svg.attr("width");
            const height = +svg.attr("height");
            
            const simulation = d3.forceSimulation(data.nodes)
                .force("link", d3.forceLink(data.edges).id(d => d.id))
                .force("charge", d3.forceManyBody().strength(-100))
                .force("center", d3.forceCenter(width / 2, height / 2));
            
            // Draw edges
            const link = svg.selectAll(".link")
                .data(data.edges)
                .join("line")
                .attr("class", "link");
            
            // Draw nodes
            const node = svg.selectAll(".node")
                .data(data.nodes)
                .join("circle")
                .attr("class", "node")
                .attr("r", 8);
            
            simulation.on("tick", () => {
                link.attr("x1", d => d.source.x)
                    .attr("y1", d => d.source.y)
                    .attr("x2", d => d.target.x)
                    .attr("y2", d => d.target.y);
                
                node.attr("cx", d => d.x)
                    .attr("cy", d => d.y);
            });
        });
</script>

Database Integration

SQLite/SQLx

use sqlx::sqlite::SqlitePool;

async fn save_events(pool: &SqlitePool, events: &[Event]) -> Result<()> {
    for event in events {
        sqlx::query(
            "INSERT INTO events (id, lat, lon, timestamp, text) VALUES (?, ?, ?, ?, ?)"
        )
        .bind(event.id.to_string())
        .bind(event.location.lat)
        .bind(event.location.lon)
        .bind(event.timestamp.to_rfc3339())
        .bind(&event.text)
        .execute(pool)
        .await?;
    }
    Ok(())
}

async fn load_events(pool: &SqlitePool) -> Result<Vec<Event>> {
    let rows = sqlx::query("SELECT * FROM events")
        .fetch_all(pool)
        .await?;
    
    let events = rows.iter()
        .map(|row| Event::new(
            Location::new(row.get("lat"), row.get("lon")),
            Timestamp::parse(row.get::<String, _>("timestamp")).unwrap(),
            row.get("text")
        ))
        .collect();
    
    Ok(events)
}

PostgreSQL with PostGIS

CREATE TABLE events (
    id UUID PRIMARY KEY,
    location GEOGRAPHY(POINT, 4326),
    timestamp TIMESTAMPTZ,
    text TEXT,
    tags TEXT[]
);

-- Spatial index
CREATE INDEX events_location_idx ON events USING GIST (location);

REST API

Actix-web

use actix_web::{web, App, HttpServer, HttpResponse};
use spatial_narrative::io::{GeoJsonFormat, Format};

async fn get_events(data: web::Data<AppState>) -> HttpResponse {
    let geojson = GeoJsonFormat::new()
        .export_str(&data.narrative)
        .unwrap();
    
    HttpResponse::Ok()
        .content_type("application/geo+json")
        .body(geojson)
}

#[actix_web::main]
async fn main() -> std::io::Result<()> {
    HttpServer::new(|| {
        App::new()
            .route("/events", web::get().to(get_events))
    })
    .bind("127.0.0.1:8080")?
    .run()
    .await
}

CLI Tools

Process Pipeline

use clap::Parser;

#[derive(Parser)]
struct Args {
    #[arg(short, long)]
    input: PathBuf,
    
    #[arg(short, long)]
    output: PathBuf,
    
    #[arg(long)]
    format: String,
}

fn main() -> Result<()> {
    let args = Args::parse();
    
    // Load
    let input = std::fs::read_to_string(&args.input)?;
    let narrative = JsonFormat::new().import_str(&input)?;
    
    // Export
    let output = match args.format.as_str() {
        "geojson" => GeoJsonFormat::new().export_str(&narrative)?,
        "csv" => CsvFormat::new().export_str(&narrative)?,
        _ => JsonFormat::new().export_str(&narrative)?,
    };
    
    std::fs::write(&args.output, &output)?;
    Ok(())
}

API Reference

Full API documentation is available at docs.rs/spatial-narrative.

Generate local docs with:

cargo doc --open

Performance

Performance characteristics and optimization tips.

Complexity

Indexing Operations

OperationSpatialIndexTemporalIndexSpatiotemporalIndex
InsertO(log n)O(log n)O(log n)
Build (bulk)O(n log n)O(n log n)O(n log n)
Point queryO(log n)O(log n)O(log n)
Range queryO(log n + k)O(log n + k)O(log n + k)
K-nearestO(k log n)N/AN/A

Where n = number of items, k = number of results.

Graph Operations

OperationComplexity
Add nodeO(1)
Add edgeO(1)
Get neighborsO(degree)
Has path (BFS)O(V + E)
Shortest pathO((V + E) log V)
connect_temporal()O(n log n)
connect_spatial(d)O(n²)
connect_thematic()O(n² × tags)

Where V = vertices, E = edges.

Memory Usage

Per-Event Overhead

ComponentApproximate Size
Event base~200 bytes
Location~40 bytes
Timestamp~32 bytes
EventId (UUID)16 bytes
Text (per char)1 byte
Tag (each)~24 bytes + string

Index Overhead

IndexOverhead per item
SpatialIndex~64 bytes
TemporalIndex~48 bytes
SpatiotemporalIndex~112 bytes

Approximate Memory Formula

Memory (MB) ≈ events × (200 + avg_text_len + tags × 30) / 1_000_000
           + index_overhead

Optimization Tips

Bulk Loading

Use from_iter instead of repeated insert:

// ✅ Fast: O(n log n) bulk load
let index = SpatialIndex::from_iter(events, |e| &e.location);

// ❌ Slow: O(n log n) per insert
let mut index = SpatialIndex::new();
for event in events {
    index.insert(event, &event.location);
}

Query Then Filter

Use indexes for initial filtering, then apply additional criteria:

// Get spatial candidates (fast)
let candidates = spatial_index.query_bounds(&bounds);

// Apply additional filters (linear)
let filtered: Vec<_> = candidates.into_iter()
    .filter(|e| e.has_tag("important"))
    .filter(|e| time_range.contains(&e.timestamp))
    .collect();

Avoid O(n²) Operations

The connect_spatial and connect_thematic methods are O(n²):

// For large graphs, consider:

// 1. Sample events
let sample: Vec<_> = events.choose_multiple(&mut rng, 1000).collect();
let mut graph = NarrativeGraph::from_events(sample);
graph.connect_spatial(10.0);  // Now O(sample²)

// 2. Use spatial index for proximity
let index = SpatialIndex::from_iter(events.clone(), |e| &e.location);
for event in &events {
    let nearby = index.query_radius_meters(event.location.lat, event.location.lon, 1000.0);
    // Only connect to nearby events
}

// 3. Skip if graph is too large
if events.len() > 10_000 {
    println!("Skipping spatial connections (too large)");
} else {
    graph.connect_spatial(10.0);
}

Lazy Evaluation

Build indexes only when needed:

struct LazyNarrative {
    events: Vec<Event>,
    spatial_index: Option<SpatialIndex<Event>>,
}

impl LazyNarrative {
    fn get_spatial_index(&mut self) -> &SpatialIndex<Event> {
        if self.spatial_index.is_none() {
            self.spatial_index = Some(SpatialIndex::from_iter(
                self.events.iter().cloned(),
                |e| &e.location
            ));
        }
        self.spatial_index.as_ref().unwrap()
    }
}

Parallel Processing

Use rayon for parallel operations:

use rayon::prelude::*;

// Parallel filtering
let filtered: Vec<_> = events.par_iter()
    .filter(|e| expensive_check(e))
    .collect();

// Parallel analysis
let metrics: Vec<_> = narratives.par_iter()
    .map(|n| SpatialMetrics::from_events(&n.events))
    .collect();

Benchmarks

Index Query Performance

Tested on 100,000 events:

Query TypeTime
Spatial bbox (1% area)~50 μs
Spatial radius (1km)~80 μs
Temporal range (1 month)~40 μs
Combined (1% × 1 month)~100 μs
K-nearest (k=10)~120 μs

Graph Operations

Operation1K events10K events100K events
from_events1 ms10 ms100 ms
connect_temporal2 ms25 ms300 ms
connect_spatial(10km)50 ms5 stoo slow
shortest_path< 1 ms5 ms50 ms
to_dot5 ms50 ms500 ms

I/O Performance

FormatExport 10K eventsImport 10K events
JSON15 ms20 ms
GeoJSON20 ms25 ms
CSV10 ms15 ms

Memory Optimization

String Interning

For repeated tags, consider interning:

use std::collections::HashSet;
use std::sync::Arc;

// Share common strings
let tag_pool: HashSet<Arc<str>> = HashSet::new();

fn intern_tag(pool: &mut HashSet<Arc<str>>, tag: &str) -> Arc<str> {
    if let Some(existing) = pool.get(tag) {
        existing.clone()
    } else {
        let new = Arc::from(tag);
        pool.insert(new.clone());
        new
    }
}

Streaming Processing

For very large datasets, process in chunks:

use std::io::{BufRead, BufReader};

let file = File::open("huge_file.csv")?;
let reader = BufReader::new(file);

for chunk in reader.lines().chunks(10_000) {
    let events: Vec<Event> = chunk
        .filter_map(|line| parse_event(&line.ok()?).ok())
        .collect();
    
    // Process chunk
    process_events(&events);
}

FAQ

Frequently asked questions about spatial-narrative.

General

What is a spatial narrative?

A spatial narrative is a sequence of events that are anchored in both space (geographic location) and time. Think of it as a story that unfolds across a map.

Examples:

  • A news story with events happening in different cities over time
  • GPS tracks showing a journey
  • Historical events with known locations and dates
  • Sensor readings with coordinates and timestamps

When should I use this library?

Use spatial-narrative when you need to:

  • Store events with both location AND time information
  • Query events by geography (bounding box, radius, nearest)
  • Query events by time range
  • Analyze relationships between events
  • Export to mapping formats (GeoJSON) or data formats (CSV)
  • Build graphs showing how events connect

What makes this different from just using a Vec?

The library provides:

  • Efficient indexing: O(log n) spatial and temporal queries
  • Rich types: Location with validation, Timestamp with timezone handling
  • Relationship graphs: Connect events and find paths
  • Analysis tools: Clustering, metrics, movement detection
  • I/O formats: GeoJSON, CSV, JSON with proper handling

Core Types

How precise are coordinates?

Coordinates use f64 (64-bit floating point), which provides about 15-16 significant digits. For geographic coordinates, this is sub-millimeter precision — far more than any real-world application needs.

What timezone does Timestamp use?

Timestamp is timezone-aware and stores time in UTC internally. When parsing:

// Explicit UTC
Timestamp::parse("2024-01-15T10:00:00Z")

// With offset
Timestamp::parse("2024-01-15T10:00:00+05:00")

// Local time (uses system timezone)
Timestamp::parse("2024-01-15T10:00:00")

Can I use custom IDs for events?

Events get auto-generated UUIDs, but you can store custom IDs in metadata:

event.set_metadata("custom_id", json!("MY-001"));

Indexing

Which index should I use?

Use CaseIndex
Only location queriesSpatialIndex
Only time queriesTemporalIndex
Both location AND timeSpatiotemporalIndex

Are indexes updated automatically?

No, indexes are immutable after creation. If you add events, rebuild the index:

// Add new events
events.push(new_event);

// Rebuild index
let index = SpatialIndex::from_iter(events.iter().cloned(), |e| &e.location);

How accurate is the radius query?

query_radius_meters uses the Haversine formula for great-circle distance, which is accurate for the Earth as a sphere. Error is typically < 0.5% vs a full ellipsoid model.

Graphs

What's the difference between EdgeType variants?

TypeMeaningAuto-Generated
TemporalA happens before Bconnect_temporal()
SpatialA is near Bconnect_spatial(km)
ThematicA and B share tagsconnect_thematic()
CausalA causes BManual only
ReferenceA mentions BManual only
CustomUser-definedManual only

Why is connect_spatial slow?

connect_spatial compares all pairs of events (O(n²)). For 10,000 events, that's 100 million comparisons.

Workarounds:

  • Use a spatial index to pre-filter candidates
  • Sample events before connecting
  • Skip for large datasets

I/O

Which format preserves all data?

JsonFormat preserves everything including custom metadata. GeoJsonFormat and CsvFormat may lose some fields.

Can I import from other GeoJSON sources?

Yes, as long as the GeoJSON has Point features with timestamp information:

{
  "type": "Feature",
  "geometry": {"type": "Point", "coordinates": [-74.0, 40.7]},
  "properties": {"timestamp": "2024-01-15T10:00:00Z"}
}

How do I handle large files?

Use streaming I/O:

let file = File::open("large.geojson")?;
let reader = BufReader::new(file);
let narrative = GeoJsonFormat::new().import(&mut reader)?;

Analysis

What distance unit does DBSCAN use?

DBSCAN::new(eps, min_points) takes eps in meters.

// 500 meter radius
let dbscan = DBSCAN::new(500.0, 3);

// 5 kilometer radius
let dbscan = DBSCAN::new(5000.0, 3);

How does clustering work with timestamps?

Current clustering is spatial only. For spatiotemporal clustering, filter by time first:

// Cluster events from January only
let january_events: Vec<_> = events.iter()
    .filter(|e| TimeRange::month(2024, 1).contains(&e.timestamp))
    .collect();

let dbscan = DBSCAN::new(1000.0, 3);
let result = dbscan.cluster(&january_events);

Troubleshooting

"No events found" after import

Check that:

  1. File path is correct
  2. Format matches file content
  3. Required fields exist (coordinates, timestamp)
// Debug: print what was imported
let narrative = GeoJsonFormat::new().import_str(&content)?;
println!("Imported {} events", narrative.events.len());
if let Some(first) = narrative.events.first() {
    println!("First event: {:?}", first);
}

Timestamp parse errors

Ensure ISO 8601 format:

  • 2024-01-15T10:00:00Z
  • 2024-01-15T10:00:00+00:00
  • 2024/01/15 10:00 (wrong separators)
  • Jan 15, 2024 (not ISO format)

Graph visualization looks wrong

Try different layout options:

// Timeline (left-to-right)
let dot = graph.to_dot_with_options(DotOptions::timeline());

// Hierarchical (top-to-bottom)
let dot = graph.to_dot_with_options(DotOptions::hierarchical());

Different Graphviz engines:

neato -Tpng graph.dot -o graph.png   # Force-directed
circo -Tpng graph.dot -o graph.png   # Circular
fdp -Tpng graph.dot -o graph.png     # Spring model

Contributing

See CONTRIBUTING.md for contribution guidelines.

Changelog

See CHANGELOG.md for version history.