Spatial Narrative
Extract geographic narratives from text.
What It Does
spatial-narrative extracts locations and events from unstructured text, turning documents into structured geospatial data.
Input: "The summit in Paris brought together leaders from Berlin and Tokyo.
Negotiations continued through the week before concluding in Geneva."
Output: [
{ location: Paris (48.86°, 2.35°), text: "summit" },
{ location: Berlin (52.52°, 13.41°), text: "leaders" },
{ location: Tokyo (35.68°, 139.65°), text: "leaders" },
{ location: Geneva (46.20°, 6.14°), text: "concluding" }
]
Core Workflow
┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Text │ → │ Geoparser │ → │ Narrative │ → │ Export │
│ (documents, │ │ (extract │ │ (events, │ │ (GeoJSON, │
│ articles) │ │ locations) │ │ analysis) │ │ mapping) │
└──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘
Key Features
| Feature | Description |
|---|---|
| Geoparsing | Extract place names from text and resolve to coordinates |
| Built-in Gazetteer | 2,500+ world cities with coordinates, population, aliases |
| Coordinate Detection | Parse decimal degrees, DMS, and other coordinate formats |
| Online Gazetteers | Optional Nominatim, GeoNames, Wikidata integration |
| Event Modeling | Structure extracted locations into events with timestamps |
| Analysis | Clustering, spatial metrics, trajectory detection |
| Export | GeoJSON, CSV, JSON for mapping tools |
Quick Example
use spatial_narrative::parser::{GeoParser, BuiltinGazetteer};
// Create parser with built-in gazetteer (2500+ cities, no API needed)
let gazetteer = BuiltinGazetteer::new();
let parser = GeoParser::with_gazetteer(gazetteer);
// Extract locations from text
let text = "Fighting broke out near Kyiv before spreading to Kharkiv and Odesa.";
let mentions = parser.extract(text);
for mention in &mentions {
if let Some(loc) = &mention.location {
println!("{}: ({:.2}°, {:.2}°)", mention.text, loc.lat, loc.lon);
}
}
// Kyiv: (50.45°, 30.52°)
// Kharkiv: (49.99°, 36.23°)
// Odesa: (46.48°, 30.73°)
Use Cases
- Journalism: Extract locations from news articles to map story development
- Intelligence: Geolocate events from reports and social media
- Historical Research: Map events from historical documents
- Disaster Response: Extract affected locations from situation reports
- Academic Research: Ground qualitative text data in geography
Modules
| Module | Purpose |
|---|---|
parser | Geoparsing: extract locations from text |
core | Data types: Event, Location, Timestamp, Narrative |
analysis | Clustering, metrics, trajectory analysis |
index | Spatial/temporal indexing for large datasets |
graph | Event relationship networks |
io | GeoJSON, CSV, JSON export |
Getting Started
Ready to extract locations from text? Start with the Installation guide!
Links
Installation
Requirements
- Rust: 1.70 or later
- Cargo: Included with Rust
Adding to Your Project
Add spatial-narrative to your Cargo.toml:
[dependencies]
spatial-narrative = "0.1"
Or use cargo add:
cargo add spatial-narrative
Features
The library comes with sensible defaults. All core features are included by default.
Default Features
[dependencies]
spatial-narrative = "0.1" # Includes all standard features
Optional Features
Enable additional functionality by specifying features:
[dependencies]
spatial-narrative = { version = "0.1", features = ["geocoding", "ml-ner-download"] }
| Feature | Description | Default |
|---|---|---|
serde | Serialization/deserialization support | ✅ |
parallel | Parallel processing with rayon | ❌ |
geocoding | External geocoding APIs (Nominatim, GeoNames, Wikidata) | ❌ |
gpx-support | GPX file format support | ❌ |
database | Database persistence (PostgreSQL, SQLite) | ❌ |
projections | Coordinate system transformations | ❌ |
nlp | Enhanced text processing with NLP | ❌ |
ml-ner | Machine learning NER with ONNX Runtime | ❌ |
ml-ner-download | Auto-download ML models from HuggingFace Hub | ❌ |
cli | Command-line interface tools | ❌ |
full | All features enabled | ❌ |
ML-NER Requirements
The ml-ner and ml-ner-download features require ONNX Runtime:
macOS (Homebrew):
brew install onnxruntime
export ORT_DYLIB_PATH=$(brew --prefix onnxruntime)/lib/libonnxruntime.dylib
Linux (Ubuntu/Debian):
sudo apt install libonnxruntime
export ORT_DYLIB_PATH=/usr/lib/libonnxruntime.so
Manual Download:
Download from ONNX Runtime releases and set ORT_DYLIB_PATH to the library path.
See ML-NER documentation for detailed setup instructions.
Verifying Installation
Create a simple test file:
// src/main.rs
use spatial_narrative::core::{Location, Timestamp, Event};
fn main() {
let location = Location::new(40.7128, -74.0060);
let timestamp = Timestamp::now();
let event = Event::new(location, timestamp, "Hello, spatial-narrative!");
println!("Created event: {}", event.text);
println!("Location: ({}, {})", event.location.lat, event.location.lon);
}
Run it:
cargo run
You should see:
Created event: Hello, spatial-narrative!
Location: (40.7128, -74.006)
Building from Source
# Clone the repository
git clone https://github.com/yourusername/spatial-narrative.git
cd spatial-narrative
# Build
cargo build --release
# Run tests
cargo test
# Generate documentation
cargo doc --open
Minimum Supported Rust Version (MSRV)
The current MSRV is Rust 1.70.
This version is tested in CI and will be maintained according to our compatibility policy.
Next Steps
- Quick Start - Get up and running quickly
- Concepts - Understand the core ideas
Quick Start
This guide will get you up and running with spatial-narrative in under 5 minutes.
Create Your First Narrative
Step 1: Create Events
Events are the fundamental building blocks. Each event has a location, timestamp, and text description:
use spatial_narrative::core::{Event, Location, Timestamp};
// Create locations
let nyc = Location::new(40.7128, -74.0060);
let boston = Location::new(42.3601, -71.0589);
// Create timestamps
let morning = Timestamp::parse("2024-06-15T09:00:00Z").unwrap();
let afternoon = Timestamp::parse("2024-06-15T14:00:00Z").unwrap();
// Create events
let event1 = Event::new(nyc, morning, "Departure from New York City");
let event2 = Event::new(boston, afternoon, "Arrival in Boston");
println!("Event 1: {} at {}", event1.text, event1.timestamp.to_rfc3339());
println!("Event 2: {} at {}", event2.text, event2.timestamp.to_rfc3339());
Step 2: Build a Narrative
A Narrative collects events into a coherent story:
use spatial_narrative::core::{Event, Location, Timestamp, NarrativeBuilder};
let events = vec![
Event::new(
Location::new(40.7128, -74.0060),
Timestamp::parse("2024-06-15T09:00:00Z").unwrap(),
"Departure from NYC"
),
Event::new(
Location::new(42.3601, -71.0589),
Timestamp::parse("2024-06-15T14:00:00Z").unwrap(),
"Arrival in Boston"
),
];
let narrative = NarrativeBuilder::new()
.title("Road Trip to Boston")
.description("A day trip from NYC to Boston")
.tag("travel")
.tag("road-trip")
.events(events)
.build();
println!("Narrative: {}", narrative.title.as_deref().unwrap_or("Untitled"));
println!("Events: {}", narrative.events.len());
Step 3: Query Events
Filter events spatially and temporally:
use spatial_narrative::core::{GeoBounds, TimeRange, Timestamp};
// Get events in chronological order
let ordered = narrative.events_chronological();
for event in &ordered {
println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}
// Get time range
if let Some(range) = narrative.time_range() {
println!("Duration: {} hours", range.duration().num_hours());
}
// Get geographic bounds
if let Some(bounds) = narrative.bounds() {
println!("Area: ({:.2}, {:.2}) to ({:.2}, {:.2})",
bounds.min_lat, bounds.min_lon,
bounds.max_lat, bounds.max_lon
);
}
Step 4: Index for Fast Queries
For large datasets, use indexes:
use spatial_narrative::index::SpatiotemporalIndex;
use spatial_narrative::core::{GeoBounds, TimeRange, Timestamp};
// Create an index
let mut index = SpatiotemporalIndex::new();
// Insert events
for event in &narrative.events {
index.insert(event.clone(), &event.location, &event.timestamp);
}
// Query by location and time
let bounds = GeoBounds::new(40.0, -75.0, 43.0, -70.0);
let time_range = TimeRange::new(
Timestamp::parse("2024-06-15T00:00:00Z").unwrap(),
Timestamp::parse("2024-06-15T23:59:59Z").unwrap(),
);
let results = index.query(&bounds, &time_range);
println!("Found {} events in region during time range", results.len());
Step 5: Build a Graph
Connect events into a relationship graph:
use spatial_narrative::graph::{NarrativeGraph, EdgeType};
// Create graph from events
let mut graph = NarrativeGraph::from_events(narrative.events.clone());
// Auto-connect by temporal sequence
graph.connect_temporal();
// Connect spatially close events (within 50km)
graph.connect_spatial(50.0);
println!("Graph: {} nodes, {} edges", graph.node_count(), graph.edge_count());
// Export to DOT format for visualization
let dot = graph.to_dot();
println!("DOT output:\n{}", dot);
Step 6: Export to GeoJSON
Export for use in mapping tools:
use spatial_narrative::io::{Format, GeoJsonFormat};
let geojson_format = GeoJsonFormat::default();
let geojson_string = geojson_format.export(&narrative.events).unwrap();
// Save to file
std::fs::write("narrative.geojson", &geojson_string).unwrap();
println!("Exported to narrative.geojson");
Complete Example
Here's everything together:
use spatial_narrative::core::{Event, Location, Timestamp, NarrativeBuilder};
use spatial_narrative::index::SpatiotemporalIndex;
use spatial_narrative::graph::{NarrativeGraph, EdgeType};
use spatial_narrative::io::{Format, GeoJsonFormat};
fn main() {
// Create events
let events = vec![
Event::new(
Location::builder().lat(40.7128).lon(-74.0060).name("NYC").build().unwrap(),
Timestamp::parse("2024-06-15T09:00:00Z").unwrap(),
"Departure from New York City"
),
Event::new(
Location::builder().lat(41.2033).lon(-73.1975).name("New Haven").build().unwrap(),
Timestamp::parse("2024-06-15T11:00:00Z").unwrap(),
"Quick stop in New Haven"
),
Event::new(
Location::builder().lat(42.3601).lon(-71.0589).name("Boston").build().unwrap(),
Timestamp::parse("2024-06-15T14:00:00Z").unwrap(),
"Arrival in Boston"
),
];
// Build narrative
let narrative = NarrativeBuilder::new()
.title("Road Trip to Boston")
.events(events)
.build();
// Index events
let mut index = SpatiotemporalIndex::new();
for event in &narrative.events {
index.insert(event.clone(), &event.location, &event.timestamp);
}
// Build graph
let mut graph = NarrativeGraph::from_events(narrative.events.clone());
graph.connect_temporal();
// Export
let geojson = GeoJsonFormat::default().export(&narrative.events).unwrap();
println!("Created narrative with {} events", narrative.events.len());
println!("Graph has {} connections", graph.edge_count());
println!("GeoJSON: {} bytes", geojson.len());
}
Next Steps
- Concepts - Understand the architecture
- Core Types - Deep dive into types
- Cookbook - Common patterns and recipes
Core Concepts
Understanding these concepts will help you use spatial-narrative effectively.
The Data Model
┌─────────────────────────────────────────────────────────────────┐
│ NARRATIVE │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ Title, Description, Tags, Metadata ││
│ └─────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ Event 1 │ │ Event 2 │ │ Event 3 │ │ Event N │ ... │
│ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ │
│ │ │ │ │ │
│ ▼ ▼ ▼ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ Location │ Timestamp │ Text │ Metadata ││
│ │ (lat, lon) │ (datetime) │ (string) │ (tags, source) ││
│ └─────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────┘
Core Types
Event
The Event is the atomic unit. It represents something that happened:
- Where: A
Location(latitude, longitude, optional elevation) - When: A
Timestamp(timezone-aware datetime with precision) - What: Text description
- Context: Tags, source references, custom metadata
// Simple event
let event = Event::new(location, timestamp, "Something happened");
// Rich event with builder
let event = EventBuilder::new()
.location(location)
.timestamp(timestamp)
.text("Something happened")
.tag("important")
.source(source_ref)
.build()
.unwrap();
Location
A Location represents a point on Earth:
- Required:
lat(-90 to 90),lon(-180 to 180) - Optional:
elevation(meters),uncertainty(meters),name
// Minimal
let loc = Location::new(40.7128, -74.0060);
// With details
let loc = Location::builder()
.lat(40.7128)
.lon(-74.0060)
.elevation(10.5)
.uncertainty_meters(5.0)
.name("Empire State Building")
.build()
.unwrap();
Timestamp
A Timestamp represents a moment in time:
- Timezone-aware (stored as UTC)
- Precision levels: Year, Month, Day, Hour, Minute, Second
- Flexible parsing (ISO 8601, dates, partial dates)
// Parse various formats
Timestamp::parse("2024-01-15T10:30:00Z")?; // Full ISO 8601
Timestamp::parse("2024-01-15")?; // Date only
Timestamp::parse("2024-01")?; // Month only
Timestamp::parse("2024")?; // Year only
// Current time
Timestamp::now();
Narrative
A Narrative is a collection of events that form a coherent story:
- Has a title, description, and tags
- Contains zero or more events
- Provides aggregate operations (bounds, time range, filtering)
let narrative = NarrativeBuilder::new()
.title("My Story")
.description("A series of events")
.tag("category")
.events(vec![event1, event2, event3])
.build();
Spatial Concepts
Coordinates
We use WGS84 (standard GPS) coordinates:
- Latitude: -90 (South Pole) to +90 (North Pole)
- Longitude: -180 to +180 (Prime Meridian at 0)
+90° (North Pole)
│
-180° ───────────────┼───────────────► +180°
(International │ (Date Line)
Date Line) │
│
-90° (South Pole)
GeoBounds
A GeoBounds represents a rectangular region:
- Defined by min/max latitude and longitude
- Used for spatial queries and filtering
// NYC metropolitan area
let bounds = GeoBounds::new(
40.4, // min_lat (south)
-74.3, // min_lon (west)
41.0, // max_lat (north)
-73.7 // max_lon (east)
);
// From a collection of locations
let bounds = GeoBounds::from_locations(&locations);
Temporal Concepts
TimeRange
A TimeRange represents a period between two timestamps:
- Has a start and end timestamp
- Supports duration calculations and overlap checks
// Explicit range
let range = TimeRange::new(start_timestamp, end_timestamp);
// From year/month
let year_2024 = TimeRange::year(2024);
let june_2024 = TimeRange::month(2024, 6);
Temporal Precision
Events can have different temporal precision:
| Precision | Example | Use Case |
|---|---|---|
Year | "2024" | Historical events |
Month | "2024-01" | Approximate dates |
Day | "2024-01-15" | Calendar events |
Hour | "2024-01-15T10:00" | Scheduled events |
Minute | "2024-01-15T10:30" | Meetings |
Second | "2024-01-15T10:30:45" | Precise logging |
Graph Concepts
NarrativeGraph
Events can be connected into a directed graph:
- Nodes = Events
- Edges = Relationships between events
Edge Types
| Type | Meaning | Auto-Connect |
|---|---|---|
Temporal | A happens before B | connect_temporal() |
Spatial | A is near B | connect_spatial(km) |
Thematic | A and B share tags | connect_thematic() |
Causal | A causes B | Manual |
Reference | A references B | Manual |
let mut graph = NarrativeGraph::from_events(events);
// Automatic connections
graph.connect_temporal(); // Time sequence
graph.connect_spatial(10.0); // Within 10km
graph.connect_thematic(); // Shared tags
// Manual connections
graph.connect(node1, node2, EdgeType::Causal);
Indexing Concepts
Spatial Index (R-tree)
For efficient geographic queries:
- Bounding box queries: "Find all events in this region"
- Radius queries: "Find all events within X km of this point"
- Nearest neighbor: "Find the closest N events to this location"
Temporal Index (B-tree)
For efficient time-based queries:
- Range queries: "Find all events between date A and date B"
- Before/After: "Find all events before/after this time"
- Ordering: "Iterate events in chronological order"
Spatiotemporal Index
Combines both for powerful queries:
- "Find events in NYC during June 2024"
- "Find nearest events to this location within the last hour"
Next Steps
- Location - Deep dive into Location type
- Timestamp - Deep dive into Timestamp type
- Event - Deep dive into Event type
Core Types Overview
The core module provides the fundamental types for spatial narratives.
Type Hierarchy
spatial_narrative::core
├── Location # Geographic point
├── Timestamp # Point in time
├── Event # Something that happened somewhere, somewhen
├── Narrative # Collection of events
├── GeoBounds # Geographic bounding box
├── TimeRange # Temporal range
├── SourceRef # Source attribution
└── EventId # Unique event identifier
Quick Reference
Location
use spatial_narrative::core::Location;
// Simple
let loc = Location::new(40.7128, -74.0060);
// With builder
let loc = Location::builder()
.lat(40.7128)
.lon(-74.0060)
.name("New York City")
.build()
.unwrap();
Timestamp
use spatial_narrative::core::Timestamp;
// Parse from string
let ts = Timestamp::parse("2024-01-15T10:30:00Z").unwrap();
// Current time
let now = Timestamp::now();
// From Unix timestamp
let ts = Timestamp::from_unix(1705315800).unwrap();
Event
use spatial_narrative::core::{Event, Location, Timestamp};
// Simple
let event = Event::new(
Location::new(40.7128, -74.0060),
Timestamp::now(),
"Event description"
);
// With builder
let event = EventBuilder::new()
.location(location)
.timestamp(timestamp)
.text("Description")
.tag("category")
.build()
.unwrap();
Narrative
use spatial_narrative::core::NarrativeBuilder;
let narrative = NarrativeBuilder::new()
.title("My Narrative")
.description("A collection of events")
.events(vec![event1, event2])
.build();
Common Operations
Serialization
All core types implement Serialize and Deserialize:
use serde_json;
// Serialize
let json = serde_json::to_string(&event)?;
// Deserialize
let event: Event = serde_json::from_str(&json)?;
Comparison
Events, Timestamps, and Locations implement comparison traits:
// Timestamps are ordered
if timestamp1 < timestamp2 {
println!("Event 1 happened first");
}
// Events can be compared by time
let events = events.into_iter()
.sorted_by_key(|e| e.timestamp)
.collect();
Validation
Types validate on construction:
// This will fail - invalid latitude
let result = Location::new(200.0, 0.0); // Returns error
// Use builders for detailed error handling
match Location::builder().lat(200.0).lon(0.0).build() {
Ok(loc) => println!("Valid"),
Err(e) => println!("Error: {}", e),
}
Module Links
- Location - Geographic points
- Timestamp - Points in time
- Event - Events with location and time
- Narrative - Event collections
- Bounds - Geographic and temporal bounds
- Sources - Source attribution
Location
The Location type represents a geographic point on Earth.
Creating Locations
Simple Constructor
use spatial_narrative::core::Location;
// Latitude, Longitude (WGS84)
let nyc = Location::new(40.7128, -74.0060);
let tokyo = Location::new(35.6762, 139.6503);
let sydney = Location::new(-33.8688, 151.2093);
Builder Pattern
For locations with additional attributes:
let location = Location::builder()
.lat(40.7484)
.lon(-73.9857)
.elevation(443.0) // meters above sea level
.uncertainty_meters(10.0) // GPS accuracy
.name("Empire State Building")
.build()
.unwrap();
From Tuple
let loc: Location = (40.7128, -74.0060).into();
Properties
| Property | Type | Description |
|---|---|---|
lat | f64 | Latitude (-90 to 90) |
lon | f64 | Longitude (-180 to 180) |
elevation | Option<f64> | Meters above sea level |
uncertainty_meters | Option<f64> | Location accuracy |
name | Option<String> | Human-readable name |
Methods
distance_to
Calculate great-circle distance to another location:
let nyc = Location::new(40.7128, -74.0060);
let london = Location::new(51.5074, -0.1278);
let distance_km = nyc.distance_to(&london);
println!("NYC to London: {:.0} km", distance_km); // ~5570 km
to_geo_point
Convert to geo crate's Point type:
use geo::Point;
let location = Location::new(40.7128, -74.0060);
let point: Point<f64> = location.to_geo_point();
Validation
Coordinates are validated on construction:
// Valid
Location::new(0.0, 0.0); // Null Island
Location::new(90.0, 180.0); // North Pole, Date Line
Location::new(-90.0, -180.0); // South Pole, Date Line
// Invalid - will return Error
Location::builder().lat(91.0).lon(0.0).build(); // Lat out of range
Location::builder().lat(0.0).lon(181.0).build(); // Lon out of range
Serialization
use serde_json;
let loc = Location::new(40.7128, -74.0060);
// To JSON
let json = serde_json::to_string(&loc)?;
// {"lat":40.7128,"lon":-74.006,"elevation":null,"uncertainty_meters":null,"name":null}
// From JSON
let loc: Location = serde_json::from_str(&json)?;
Examples
City Locations
let cities = vec![
Location::builder().lat(40.7128).lon(-74.0060).name("New York").build()?,
Location::builder().lat(51.5074).lon(-0.1278).name("London").build()?,
Location::builder().lat(35.6762).lon(139.6503).name("Tokyo").build()?,
Location::builder().lat(-33.8688).lon(151.2093).name("Sydney").build()?,
];
With Elevation
// Mountain peaks
let everest = Location::builder()
.lat(27.9881)
.lon(86.9250)
.elevation(8848.86)
.name("Mount Everest")
.build()?;
let denali = Location::builder()
.lat(63.0692)
.lon(-151.0070)
.elevation(6190.5)
.name("Denali")
.build()?;
GPS Tracking
// GPS readings with uncertainty
let readings = vec![
Location::builder()
.lat(40.7128).lon(-74.0060)
.uncertainty_meters(5.0)
.build()?,
Location::builder()
.lat(40.7130).lon(-74.0058)
.uncertainty_meters(3.0)
.build()?,
];
Timestamp
The Timestamp type represents a point in time with timezone awareness and configurable precision.
Creating Timestamps
Parse from String
use spatial_narrative::core::Timestamp;
// Full ISO 8601
let ts = Timestamp::parse("2024-01-15T10:30:00Z")?;
// Date only
let ts = Timestamp::parse("2024-01-15")?;
// Year-month
let ts = Timestamp::parse("2024-01")?;
// Year only
let ts = Timestamp::parse("2024")?;
Current Time
let now = Timestamp::now();
From Unix Timestamp
// Seconds since epoch
let ts = Timestamp::from_unix(1705315800)?;
// Milliseconds since epoch
let ts = Timestamp::from_unix_millis(1705315800000)?;
Properties
| Property | Type | Description |
|---|---|---|
datetime | DateTime<Utc> | Underlying chrono datetime |
precision | TemporalPrecision | Year, Month, Day, Hour, Minute, Second |
Temporal Precision
Timestamps track their precision level:
use spatial_narrative::core::TemporalPrecision;
let ts = Timestamp::parse("2024")?;
assert_eq!(ts.precision, TemporalPrecision::Year);
let ts = Timestamp::parse("2024-01-15")?;
assert_eq!(ts.precision, TemporalPrecision::Day);
let ts = Timestamp::parse("2024-01-15T10:30:00Z")?;
assert_eq!(ts.precision, TemporalPrecision::Second);
Methods
Formatting
let ts = Timestamp::parse("2024-01-15T10:30:00Z")?;
// RFC 3339 format
ts.to_rfc3339(); // "2024-01-15T10:30:00+00:00"
// Format with precision
ts.format_with_precision(); // Respects the timestamp's precision
Conversion
// To Unix timestamp
let seconds = ts.to_unix();
let millis = ts.to_unix_millis();
Comparison
let ts1 = Timestamp::parse("2024-01-01T00:00:00Z")?;
let ts2 = Timestamp::parse("2024-01-02T00:00:00Z")?;
if ts1 < ts2 {
println!("ts1 is earlier");
}
// Duration between timestamps
let duration = ts2.duration_since(&ts1);
println!("Difference: {} hours", duration.num_hours());
Serialization
use serde_json;
let ts = Timestamp::parse("2024-01-15T10:30:00Z")?;
// To JSON (serializes as string)
let json = serde_json::to_string(&ts)?;
// "2024-01-15T10:30:00+00:00"
// From JSON
let ts: Timestamp = serde_json::from_str(&json)?;
Examples
Historical Events
// Historical dates with appropriate precision
let ww1_start = Timestamp::parse("1914-07-28")?; // Day precision
let moon_landing = Timestamp::parse("1969-07-20T20:17:40Z")?; // Second precision
let renaissance = Timestamp::parse("1400")?; // Year precision
Time Ranges
use spatial_narrative::core::TimeRange;
let start = Timestamp::parse("2024-01-01T00:00:00Z")?;
let end = Timestamp::parse("2024-12-31T23:59:59Z")?;
let year_2024 = TimeRange::new(start, end);
println!("Duration: {} days", year_2024.duration().num_days());
Sorting Events
let mut events = vec![event3, event1, event2];
// Sort by timestamp
events.sort_by_key(|e| e.timestamp.clone());
// Now in chronological order
for event in &events {
println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}
Event
The Event type represents something that happened at a specific place and time.
Creating Events
Simple Constructor
use spatial_narrative::core::{Event, Location, Timestamp};
let event = Event::new(
Location::new(40.7128, -74.0060),
Timestamp::parse("2024-01-15T10:00:00Z").unwrap(),
"Conference begins"
);
Builder Pattern
use spatial_narrative::core::{EventBuilder, Location, Timestamp, SourceRef};
let event = EventBuilder::new()
.location(Location::new(40.7128, -74.0060))
.timestamp(Timestamp::parse("2024-01-15T10:00:00Z").unwrap())
.text("Conference begins in Manhattan")
.tag("conference")
.tag("technology")
.source(SourceRef::builder()
.title("Event Calendar")
.url("https://example.com/events")
.build())
.build()
.unwrap();
Properties
| Property | Type | Description |
|---|---|---|
id | EventId | Unique identifier (UUID) |
location | Location | Where it happened |
timestamp | Timestamp | When it happened |
text | String | Description |
tags | HashSet<String> | Categories/labels |
source | Option<SourceRef> | Source attribution |
metadata | HashMap<String, Value> | Custom metadata |
Methods
Tags
// Add tags
event.add_tag("important");
event.add_tag("verified");
// Check tags
if event.has_tag("important") {
println!("This is important!");
}
// Get all tags
for tag in &event.tags {
println!("Tag: {}", tag);
}
Metadata
use serde_json::json;
// Add metadata
event.set_metadata("priority", json!(1));
event.set_metadata("verified", json!(true));
// Get metadata
if let Some(priority) = event.get_metadata("priority") {
println!("Priority: {}", priority);
}
Traits
Events implement spatial and temporal traits:
use spatial_narrative::core::traits::{SpatialEntity, TemporalEntity};
// Spatial trait
let coords = event.coordinates(); // (lat, lon)
let bounds = event.bounds(); // GeoBounds
// Temporal trait
let time = event.time(); // Timestamp
let range = event.time_range(); // TimeRange
Examples
News Event
let event = EventBuilder::new()
.location(Location::builder()
.lat(48.8566).lon(2.3522)
.name("Paris, France")
.build()?)
.timestamp(Timestamp::parse("2024-07-14T10:00:00Z")?)
.text("Bastille Day celebrations commence")
.tag("celebration")
.tag("national-holiday")
.source(SourceRef::builder()
.title("Le Monde")
.source_type(SourceType::Article)
.url("https://lemonde.fr/article/123")
.build())
.build()?;
Sensor Reading
let reading = EventBuilder::new()
.location(Location::builder()
.lat(37.7749).lon(-122.4194)
.uncertainty_meters(1.0)
.build()?)
.timestamp(Timestamp::now())
.text("Temperature reading")
.tag("sensor")
.tag("temperature")
.build()?;
reading.set_metadata("temperature_c", json!(22.5));
reading.set_metadata("humidity_pct", json!(65));
Narrative
The Narrative type represents a collection of related events that form a coherent story.
Creating Narratives
Builder Pattern
use spatial_narrative::core::{NarrativeBuilder, Event};
let narrative = NarrativeBuilder::new()
.title("Road Trip to Boston")
.description("A day trip from NYC to Boston with stops along the way")
.tag("travel")
.tag("road-trip")
.events(vec![event1, event2, event3])
.build();
Empty Narrative
let mut narrative = NarrativeBuilder::new()
.title("My Story")
.build();
// Add events later
narrative.add_event(event1);
narrative.add_event(event2);
Properties
| Property | Type | Description |
|---|---|---|
id | NarrativeId | Unique identifier |
title | Option<String> | Narrative title |
description | Option<String> | Description |
events | Vec<Event> | Collection of events |
tags | HashSet<String> | Categories/labels |
metadata | NarrativeMetadata | Additional metadata |
Methods
Events
// Add events
narrative.add_event(event);
// Get event count
println!("Events: {}", narrative.events.len());
// Iterate events
for event in &narrative.events {
println!("{}", event.text);
}
Chronological Order
// Get events sorted by time
let ordered = narrative.events_chronological();
for event in ordered {
println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}
Time Range
// Get overall time span
if let Some(range) = narrative.time_range() {
println!("Start: {}", range.start.to_rfc3339());
println!("End: {}", range.end.to_rfc3339());
println!("Duration: {} days", range.duration().num_days());
}
Geographic Bounds
// Get bounding box of all events
if let Some(bounds) = narrative.bounds() {
println!("Lat: {} to {}", bounds.min_lat, bounds.max_lat);
println!("Lon: {} to {}", bounds.min_lon, bounds.max_lon);
let (center_lat, center_lon) = bounds.center();
println!("Center: ({}, {})", center_lat, center_lon);
}
Filtering
// Filter by geographic bounds
let paris_events = narrative.filter_spatial(&paris_bounds);
// Filter by time range
let june_events = narrative.filter_temporal(&june_2024);
Examples
Historical Timeline
let ww1 = NarrativeBuilder::new()
.title("World War I Timeline")
.description("Key events of the Great War")
.tag("history")
.tag("world-war")
.events(vec![
Event::new(
Location::new(43.8563, 18.4131), // Sarajevo
Timestamp::parse("1914-06-28")?,
"Assassination of Archduke Franz Ferdinand"
),
Event::new(
Location::new(48.8566, 2.3522), // Paris
Timestamp::parse("1919-06-28")?,
"Treaty of Versailles signed"
),
])
.build();
println!("Timeline: {}", ww1.title.as_deref().unwrap_or("Untitled"));
println!("Duration: {} years", ww1.time_range().unwrap().duration().num_days() / 365);
Travel Journal
let trip = NarrativeBuilder::new()
.title("European Adventure 2024")
.tag("travel")
.tag("europe")
.events(cities_visited)
.build();
// Get geographic extent
if let Some(bounds) = trip.bounds() {
println!("Trip covered {} degrees of latitude",
bounds.max_lat - bounds.min_lat);
}
Bounds
The core module provides types for representing geographic and temporal bounds.
GeoBounds
Represents a rectangular geographic region.
Creating GeoBounds
use spatial_narrative::core::GeoBounds;
// Explicit bounds
let bounds = GeoBounds::new(
40.4, // min_lat (south)
-74.3, // min_lon (west)
41.0, // max_lat (north)
-73.7 // max_lon (east)
);
// From locations
let bounds = GeoBounds::from_locations(&[
Location::new(40.7128, -74.0060), // NYC
Location::new(42.3601, -71.0589), // Boston
]);
Methods
// Check if a location is within bounds
if bounds.contains(40.75, -73.99) {
println!("Location is within bounds");
}
// Get center point
let (lat, lon) = bounds.center();
// Check intersection
if bounds1.intersects(&bounds2) {
println!("Regions overlap");
}
TimeRange
Represents a period between two timestamps.
Creating TimeRange
use spatial_narrative::core::{TimeRange, Timestamp};
// From timestamps
let range = TimeRange::new(
Timestamp::parse("2024-01-01T00:00:00Z")?,
Timestamp::parse("2024-12-31T23:59:59Z")?
);
// Helper methods
let year_2024 = TimeRange::year(2024);
let june_2024 = TimeRange::month(2024, 6);
Methods
// Get duration
let days = range.duration().num_days();
// Check if timestamp is within range
if range.contains(×tamp) {
println!("Time is within range");
}
// Check overlap
if range1.overlaps(&range2) {
println!("Ranges overlap");
}
Examples
Regional Analysis
// Define regions
let east_coast = GeoBounds::new(25.0, -82.0, 45.0, -66.0);
let west_coast = GeoBounds::new(32.0, -125.0, 49.0, -114.0);
// Filter events by region
let east_events = narrative.filter_spatial(&east_coast);
let west_events = narrative.filter_spatial(&west_coast);
Temporal Analysis
// Define periods
let q1_2024 = TimeRange::new(
Timestamp::parse("2024-01-01")?,
Timestamp::parse("2024-03-31")?
);
// Filter events by period
let q1_events = narrative.filter_temporal(&q1_2024);
Sources
The SourceRef type provides source attribution for events.
Creating Source References
use spatial_narrative::core::{SourceRef, SourceType};
let source = SourceRef::builder()
.title("The New York Times")
.source_type(SourceType::Article)
.url("https://nytimes.com/article/123")
.author("Jane Reporter")
.date("2024-01-15")
.build();
Source Types
| Type | Description |
|---|---|
Article | News article or blog post |
Report | Official report or document |
Witness | Eyewitness account |
Sensor | Automated sensor data |
Archive | Historical archive |
Other | Other source type |
Properties
| Property | Type | Description |
|---|---|---|
source_type | SourceType | Category of source |
title | Option<String> | Source title |
url | Option<String> | URL reference |
author | Option<String> | Author/creator |
date | Option<String> | Publication date |
notes | Option<String> | Additional notes |
Examples
News Article
let source = SourceRef::builder()
.source_type(SourceType::Article)
.title("Breaking: Event Occurs")
.url("https://news.example.com/article")
.author("John Journalist")
.date("2024-01-15")
.build();
Sensor Data
let source = SourceRef::builder()
.source_type(SourceType::Sensor)
.title("Weather Station #42")
.notes("Automated reading every 5 minutes")
.build();
Historical Archive
let source = SourceRef::builder()
.source_type(SourceType::Archive)
.title("National Archives Collection")
.url("https://archives.gov/document/123")
.notes("Declassified 2020")
.build();
I/O Overview
The io module provides import and export functionality for narratives in various formats.
Supported Formats
| Format | Type | Best For |
|---|---|---|
| JSON | Native | Full fidelity, all metadata preserved |
| GeoJSON | Standard | Web mapping, GIS tools, Leaflet/Mapbox |
| CSV | Tabular | Spreadsheets, data analysis, pandas |
The Format Trait
All formats implement the Format trait:
use spatial_narrative::io::Format;
pub trait Format {
fn import<R: Read>(&self, reader: &mut R) -> Result<Narrative>;
fn export<W: Write>(&self, narrative: &Narrative, writer: &mut W) -> Result<()>;
fn import_str(&self, s: &str) -> Result<Narrative>;
fn export_str(&self, narrative: &Narrative) -> Result<String>;
}
Quick Examples
Export to Multiple Formats
use spatial_narrative::io::{GeoJsonFormat, CsvFormat, JsonFormat, Format};
use spatial_narrative::core::{Narrative, NarrativeBuilder, Event, Location, Timestamp};
let narrative = NarrativeBuilder::new()
.title("My Story")
.event(Event::new(
Location::new(40.7128, -74.0060),
Timestamp::now(),
"Something happened"
))
.build();
// Export to GeoJSON (for web maps)
let geojson = GeoJsonFormat::new().export_str(&narrative)?;
// Export to CSV (for spreadsheets)
let csv = CsvFormat::new().export_str(&narrative)?;
// Export to JSON (for full fidelity)
let json = JsonFormat::new().export_str(&narrative)?;
Import from File
use std::fs::File;
use std::io::BufReader;
// Import from GeoJSON file
let file = File::open("data.geojson")?;
let mut reader = BufReader::new(file);
let narrative = GeoJsonFormat::new().import(&mut reader)?;
println!("Loaded {} events", narrative.events.len());
Export to File
use std::fs::File;
use std::io::BufWriter;
// Export to CSV file
let file = File::create("output.csv")?;
let mut writer = BufWriter::new(file);
CsvFormat::new().export(&narrative, &mut writer)?;
Format Comparison
| Feature | JSON | GeoJSON | CSV |
|---|---|---|---|
| All metadata | ✅ | ⚠️ Partial | ⚠️ Limited |
| Tags | ✅ | ✅ | ✅ |
| Sources | ✅ | ✅ | ⚠️ Optional |
| Custom metadata | ✅ | ⚠️ Properties | ❌ |
| Human readable | ⚠️ | ✅ | ✅ |
| GIS compatible | ❌ | ✅ | ⚠️ |
| Spreadsheet ready | ❌ | ❌ | ✅ |
| File size | Medium | Large | Small |
Round-Trip Fidelity
For lossless round-trips, use JsonFormat:
let json = JsonFormat::new();
// Export
let exported = json.export_str(&narrative)?;
// Import
let imported = json.import_str(&exported)?;
// Verify
assert_eq!(narrative.events.len(), imported.events.len());
Next Steps
- JSON Format - Native format with full fidelity
- GeoJSON Format - Standard geographic format
- CSV Format - Tabular export for spreadsheets
- Custom Formats - Implement your own format
JSON Format
The native JSON format provides full fidelity for narratives, preserving all metadata.
Basic Usage
use spatial_narrative::io::{JsonFormat, Format};
use spatial_narrative::core::{Narrative, NarrativeBuilder, Event, Location, Timestamp};
let narrative = NarrativeBuilder::new()
.title("My Narrative")
.event(Event::new(
Location::new(40.7128, -74.0060),
Timestamp::now(),
"Event description"
))
.build();
// Export
let json = JsonFormat::new().export_str(&narrative)?;
// Import
let imported = JsonFormat::new().import_str(&json)?;
Pretty Printing
For human-readable output:
let format = JsonFormat::pretty();
let json = format.export_str(&narrative)?;
// Output is indented and formatted
println!("{}", json);
Output Structure
The JSON format produces:
{
"version": "1.0",
"narrative": {
"id": "550e8400-e29b-41d4-a716-446655440000",
"title": "My Narrative",
"description": null,
"author": null,
"created": "2024-01-15T10:00:00Z",
"modified": "2024-01-15T10:00:00Z",
"tags": ["example"],
"metadata": {}
},
"events": [
{
"id": "660e8400-e29b-41d4-a716-446655440001",
"location": {
"lat": 40.7128,
"lon": -74.006,
"elevation": null,
"uncertainty_meters": null,
"name": "New York City"
},
"timestamp": "2024-01-15T10:00:00Z",
"text": "Event description",
"tags": ["conference"],
"source": {
"title": "Source Name",
"source_type": "article",
"url": "https://example.com"
},
"metadata": {}
}
]
}
Version Compatibility
The format includes version information for forward compatibility:
// Current version is "1.0"
// Future versions will maintain backward compatibility
let json = r#"{"version": "1.0", "narrative": {...}, "events": [...]}"#;
let narrative = JsonFormat::new().import_str(json)?;
File Operations
Export to File
use std::fs::File;
use std::io::BufWriter;
let file = File::create("narrative.json")?;
let mut writer = BufWriter::new(file);
JsonFormat::pretty().export(&narrative, &mut writer)?;
Import from File
use std::fs::File;
use std::io::BufReader;
let file = File::open("narrative.json")?;
let mut reader = BufReader::new(file);
let narrative = JsonFormat::new().import(&mut reader)?;
When to Use JSON
Use JSON when:
- Archiving narratives for later use
- Transferring between systems using this library
- You need to preserve all metadata
- Round-trip fidelity is important
Consider other formats when:
- Integrating with GIS tools → use GeoJSON
- Importing to spreadsheets → use CSV
- Minimizing file size is critical
Metadata Preservation
JSON preserves all custom metadata:
use serde_json::json;
let mut event = Event::new(location, timestamp, "Description");
event.set_metadata("custom_field", json!({"nested": "data"}));
event.set_metadata("priority", json!(1));
// All metadata is preserved in JSON export
let json = JsonFormat::new().export_str(&narrative)?;
let imported = JsonFormat::new().import_str(&json)?;
assert_eq!(
imported.events[0].get_metadata("custom_field"),
Some(&json!({"nested": "data"}))
);
GeoJSON Format
Export narratives as GeoJSON FeatureCollections for use with mapping libraries.
Basic Usage
use spatial_narrative::io::{GeoJsonFormat, Format};
use spatial_narrative::core::{Narrative, Event, Location, Timestamp};
// Export to GeoJSON
let geojson = GeoJsonFormat::new().export_str(&narrative)?;
// Import from GeoJSON
let narrative = GeoJsonFormat::new().import_str(&geojson)?;
Output Structure
Events are exported as Point features:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [-74.006, 40.7128]
},
"properties": {
"id": "550e8400-e29b-41d4-a716-446655440000",
"timestamp": "2024-01-15T10:00:00Z",
"text": "Event description",
"tags": ["conference", "technology"]
}
}
]
}
Note: GeoJSON uses [longitude, latitude] order per the specification.
Configuration Options
Customize the export with GeoJsonOptions:
use spatial_narrative::io::{GeoJsonFormat, GeoJsonOptions};
let options = GeoJsonOptions {
include_ids: true, // Include event IDs
include_tags: true, // Include tags array
include_sources: true, // Include source information
timestamp_property: "time".to_string(), // Property name for timestamp
text_property: "description".to_string(), // Property name for text
};
let format = GeoJsonFormat::with_options(options);
let geojson = format.export_str(&narrative)?;
Default Options
GeoJsonOptions {
include_ids: true,
include_tags: true,
include_sources: false,
timestamp_property: "timestamp".to_string(),
text_property: "text".to_string(),
}
Web Mapping Integration
Leaflet
// Load exported GeoJSON
fetch('narrative.geojson')
.then(res => res.json())
.then(geojson => {
L.geoJSON(geojson, {
pointToLayer: (feature, latlng) => {
return L.circleMarker(latlng, {
radius: 8,
fillColor: '#ff7800',
color: '#000',
weight: 1,
opacity: 1,
fillOpacity: 0.8
});
},
onEachFeature: (feature, layer) => {
layer.bindPopup(`
<b>${feature.properties.text}</b><br>
${feature.properties.timestamp}
`);
}
}).addTo(map);
});
Mapbox GL JS
map.on('load', () => {
map.addSource('narrative', {
type: 'geojson',
data: 'narrative.geojson'
});
map.addLayer({
id: 'events',
type: 'circle',
source: 'narrative',
paint: {
'circle-radius': 8,
'circle-color': '#ff7800'
}
});
});
Importing GeoJSON
Import existing GeoJSON data:
let geojson = r#"{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {"type": "Point", "coordinates": [-74.006, 40.7128]},
"properties": {
"timestamp": "2024-01-15T10:00:00Z",
"text": "Event description"
}
}
]
}"#;
let narrative = GeoJsonFormat::new().import_str(geojson)?;
println!("Imported {} events", narrative.events.len());
Property Mapping
The importer looks for these properties:
| Property | Maps To | Required |
|---|---|---|
timestamp, time, datetime | Event.timestamp | Yes |
text, description, name | Event.text | No |
tags | Event.tags | No |
id | Event.id | No (generates UUID) |
File Operations
use std::fs::File;
use std::io::{BufReader, BufWriter};
// Export
let file = File::create("events.geojson")?;
GeoJsonFormat::new().export(&narrative, &mut BufWriter::new(file))?;
// Import
let file = File::open("events.geojson")?;
let narrative = GeoJsonFormat::new().import(&mut BufReader::new(file))?;
When to Use GeoJSON
Use GeoJSON when:
- Displaying events on web maps (Leaflet, Mapbox, Google Maps)
- Importing into GIS software (QGIS, ArcGIS)
- Sharing with systems that expect standard geographic formats
- Building map visualizations
Consider other formats when:
CSV Format
Export narratives as CSV for spreadsheet analysis and data science workflows.
Basic Usage
use spatial_narrative::io::{CsvFormat, Format};
// Export to CSV
let csv = CsvFormat::new().export_str(&narrative)?;
// Import from CSV
let narrative = CsvFormat::new().import_str(&csv)?;
Output Structure
Default CSV output:
latitude,longitude,timestamp,text,tags
40.7128,-74.006,2024-01-15T10:00:00Z,"Conference begins","conference,technology"
40.758,-73.9855,2024-01-15T14:00:00Z,"Press conference","press,media"
Configuration Options
Customize column names and delimiter:
use spatial_narrative::io::{CsvFormat, CsvOptions};
let options = CsvOptions {
lat_column: "lat".to_string(),
lon_column: "lng".to_string(),
timestamp_column: "datetime".to_string(),
text_column: Some("description".to_string()),
tags_column: Some("categories".to_string()),
elevation_column: Some("altitude".to_string()),
source_title_column: None,
source_url_column: None,
delimiter: b',',
};
let format = CsvFormat::with_options(options);
let csv = format.export_str(&narrative)?;
Column Options
| Option | Default | Description |
|---|---|---|
lat_column | "latitude" | Latitude column name |
lon_column | "longitude" | Longitude column name |
timestamp_column | "timestamp" | Timestamp column name |
text_column | Some("text") | Event text column |
tags_column | Some("tags") | Tags column (comma-separated) |
elevation_column | None | Elevation column |
source_title_column | None | Source title column |
source_url_column | None | Source URL column |
delimiter | b',' | Field delimiter |
TSV (Tab-Separated)
let options = CsvOptions {
delimiter: b'\t',
..Default::default()
};
let tsv = CsvFormat::with_options(options).export_str(&narrative)?;
Importing CSV
Import from existing CSV data:
let csv = r#"lat,lon,time,description
40.7128,-74.006,2024-01-15T10:00:00Z,Conference begins
40.758,-73.9855,2024-01-15T14:00:00Z,Press conference"#;
let options = CsvOptions {
lat_column: "lat".to_string(),
lon_column: "lon".to_string(),
timestamp_column: "time".to_string(),
text_column: Some("description".to_string()),
..Default::default()
};
let narrative = CsvFormat::with_options(options).import_str(csv)?;
Handling Tags
Tags are stored as comma-separated values:
// Export: tags become "tag1,tag2,tag3"
// Import: "tag1,tag2,tag3" becomes HashSet{"tag1", "tag2", "tag3"}
let csv = "latitude,longitude,timestamp,text,tags
40.7128,-74.006,2024-01-15T10:00:00Z,Event,\"conference,technology,important\"";
let narrative = CsvFormat::new().import_str(csv)?;
assert!(narrative.events[0].has_tag("conference"));
File Operations
use std::fs::File;
use std::io::{BufReader, BufWriter};
// Export to file
let file = File::create("events.csv")?;
CsvFormat::new().export(&narrative, &mut BufWriter::new(file))?;
// Import from file
let file = File::open("events.csv")?;
let narrative = CsvFormat::new().import(&mut BufReader::new(file))?;
Integration with Data Tools
Python/Pandas
import pandas as pd
# Read exported CSV
df = pd.read_csv('events.csv', parse_dates=['timestamp'])
# Analyze
print(f"Events: {len(df)}")
print(f"Date range: {df['timestamp'].min()} to {df['timestamp'].max()}")
print(f"Locations: {df[['latitude', 'longitude']].nunique()}")
Excel
CSV files open directly in Excel. For best results:
- Use UTF-8 encoding
- Quote text fields containing commas
- Use ISO 8601 timestamps
R
library(tidyverse)
events <- read_csv("events.csv")
# Plot on map
library(sf)
events_sf <- st_as_sf(events, coords = c("longitude", "latitude"), crs = 4326)
plot(events_sf)
When to Use CSV
Use CSV when:
- Analyzing data in spreadsheets (Excel, Google Sheets)
- Using with data science tools (pandas, R)
- Simple data exchange with minimal overhead
- Data pipelines that expect tabular formats
Consider other formats when:
- You need complex metadata → use JSON
- Mapping in web applications → use GeoJSON
- Source attribution is important → use JSON
Limitations
CSV has some inherent limitations:
| Feature | Support |
|---|---|
| Location coordinates | ✅ Full |
| Timestamps | ✅ Full |
| Event text | ✅ Full |
| Tags | ✅ As comma-separated string |
| Basic source info | ⚠️ Optional columns |
| Custom metadata | ❌ Not supported |
| Nested data | ❌ Not supported |
Custom Formats
Implement the Format trait to create custom import/export formats.
The Format Trait
use spatial_narrative::io::Format;
use spatial_narrative::core::Narrative;
use spatial_narrative::error::Result;
use std::io::{Read, Write};
pub trait Format {
/// Import a narrative from a reader
fn import<R: Read>(&self, reader: &mut R) -> Result<Narrative>;
/// Export a narrative to a writer
fn export<W: Write>(&self, narrative: &Narrative, writer: &mut W) -> Result<()>;
/// Import from a string (has default implementation)
fn import_str(&self, s: &str) -> Result<Narrative> {
self.import(&mut s.as_bytes())
}
/// Export to a string (has default implementation)
fn export_str(&self, narrative: &Narrative) -> Result<String> {
let mut buf = Vec::new();
self.export(narrative, &mut buf)?;
Ok(String::from_utf8(buf)?)
}
}
Example: XML Format
use spatial_narrative::io::Format;
use spatial_narrative::core::{Narrative, NarrativeBuilder, Event, Location, Timestamp};
use spatial_narrative::error::{Error, Result};
use std::io::{Read, Write};
pub struct XmlFormat;
impl XmlFormat {
pub fn new() -> Self {
Self
}
}
impl Format for XmlFormat {
fn import<R: Read>(&self, reader: &mut R) -> Result<Narrative> {
let mut content = String::new();
reader.read_to_string(&mut content)?;
// Parse XML (using quick-xml or similar)
// This is a simplified example
let mut builder = NarrativeBuilder::new();
// ... parse events from XML ...
Ok(builder.build())
}
fn export<W: Write>(&self, narrative: &Narrative, writer: &mut W) -> Result<()> {
writeln!(writer, r#"<?xml version="1.0" encoding="UTF-8"?>"#)?;
writeln!(writer, "<narrative>")?;
if let Some(title) = &narrative.title {
writeln!(writer, " <title>{}</title>", escape_xml(title))?;
}
writeln!(writer, " <events>")?;
for event in &narrative.events {
writeln!(writer, " <event>")?;
writeln!(writer, " <lat>{}</lat>", event.location.lat)?;
writeln!(writer, " <lon>{}</lon>", event.location.lon)?;
writeln!(writer, " <timestamp>{}</timestamp>",
event.timestamp.to_rfc3339())?;
writeln!(writer, " <text>{}</text>", escape_xml(&event.text))?;
writeln!(writer, " </event>")?;
}
writeln!(writer, " </events>")?;
writeln!(writer, "</narrative>")?;
Ok(())
}
}
fn escape_xml(s: &str) -> String {
s.replace('&', "&")
.replace('<', "<")
.replace('>', ">")
.replace('"', """)
}
Example: KML Format
For Google Earth compatibility:
pub struct KmlFormat;
impl Format for KmlFormat {
fn export<W: Write>(&self, narrative: &Narrative, writer: &mut W) -> Result<()> {
writeln!(writer, r#"<?xml version="1.0" encoding="UTF-8"?>"#)?;
writeln!(writer, r#"<kml xmlns="http://www.opengis.net/kml/2.2">"#)?;
writeln!(writer, "<Document>")?;
if let Some(title) = &narrative.title {
writeln!(writer, " <name>{}</name>", title)?;
}
for event in &narrative.events {
writeln!(writer, " <Placemark>")?;
writeln!(writer, " <name>{}</name>", escape_xml(&event.text))?;
writeln!(writer, " <TimeStamp>")?;
writeln!(writer, " <when>{}</when>", event.timestamp.to_rfc3339())?;
writeln!(writer, " </TimeStamp>")?;
writeln!(writer, " <Point>")?;
writeln!(writer, " <coordinates>{},{}</coordinates>",
event.location.lon, event.location.lat)?;
writeln!(writer, " </Point>")?;
writeln!(writer, " </Placemark>")?;
}
writeln!(writer, "</Document>")?;
writeln!(writer, "</kml>")?;
Ok(())
}
fn import<R: Read>(&self, _reader: &mut R) -> Result<Narrative> {
// KML import implementation
todo!("KML import not yet implemented")
}
}
Using Custom Formats
let narrative = create_narrative();
// Use custom XML format
let xml_format = XmlFormat::new();
let xml = xml_format.export_str(&narrative)?;
// Use custom KML format
let kml_format = KmlFormat::new();
let kml = kml_format.export_str(&narrative)?;
// Save to file
let mut file = File::create("narrative.kml")?;
kml_format.export(&narrative, &mut file)?;
Best Practices
- Handle errors gracefully: Return
Resultwith descriptive errors - Use buffered I/O: Wrap readers/writers in
BufReader/BufWriter - Support round-trips: Ensure
import(export(n)) == nwhere possible - Document limitations: Note what metadata is lost in conversion
- Validate on import: Check for required fields and valid values
Error Handling
Use the library's error types:
use spatial_narrative::error::{Error, Result};
impl Format for MyFormat {
fn import<R: Read>(&self, reader: &mut R) -> Result<Narrative> {
let mut content = String::new();
reader.read_to_string(&mut content)?;
if content.is_empty() {
return Err(Error::Parse("Empty input".to_string()));
}
// Parse and validate...
Ok(narrative)
}
}
Indexing Overview
The index module provides efficient data structures for spatial and temporal queries.
Index Types
| Index | Data Structure | Best For |
|---|---|---|
SpatialIndex | R-tree | Geographic queries (bbox, radius, k-nearest) |
TemporalIndex | B-tree | Time range queries |
SpatiotemporalIndex | Combined | Space + time queries together |
Quick Example
use spatial_narrative::index::{SpatialIndex, TemporalIndex, SpatiotemporalIndex};
use spatial_narrative::core::{Event, Location, Timestamp, GeoBounds, TimeRange};
// Create events
let events = vec![
Event::new(Location::new(40.7128, -74.0060),
Timestamp::parse("2024-01-15T10:00:00Z").unwrap(), "NYC Event"),
Event::new(Location::new(34.0522, -118.2437),
Timestamp::parse("2024-01-15T14:00:00Z").unwrap(), "LA Event"),
];
// Spatial-only queries
let mut spatial = SpatialIndex::new();
for e in &events {
spatial.insert(e.clone(), &e.location);
}
let nearby = spatial.query_radius(40.7, -74.0, 0.5);
// Temporal-only queries
let mut temporal = TemporalIndex::new();
for e in &events {
temporal.insert(e.clone(), &e.timestamp);
}
let range = TimeRange::new(
Timestamp::parse("2024-01-15T09:00:00Z").unwrap(),
Timestamp::parse("2024-01-15T12:00:00Z").unwrap(),
);
let morning = temporal.query_range(&range);
// Combined queries
let mut combined = SpatiotemporalIndex::new();
for e in &events {
combined.insert(e.clone(), &e.location, &e.timestamp);
}
let bounds = GeoBounds::new(40.0, -75.0, 41.0, -73.0);
let results = combined.query(&bounds, &range);
Performance Characteristics
| Operation | SpatialIndex | TemporalIndex | SpatiotemporalIndex |
|---|---|---|---|
| Insert | O(log n) | O(log n) | O(log n) |
| Point query | O(log n) | O(log n) | O(log n) |
| Range query | O(log n + k) | O(log n + k) | O(log n + k) |
| K-nearest | O(k log n) | - | - |
Where n is the number of items and k is the number of results.
Choosing an Index
Use SpatialIndex when:
- You only care about location
- Finding events near a point
- Bounding box searches
- K-nearest neighbor queries
Use TemporalIndex when:
- You only care about time
- Finding events in a time range
- Before/after queries
- Chronological iteration
Use SpatiotemporalIndex when:
- You need to filter by both space AND time
- Events have meaningful locations and timestamps
- Generating heatmaps
Generic Types
All indexes are generic over the stored item type:
// Index of events
let event_index: SpatialIndex<Event> = SpatialIndex::new();
// Index of custom structs
struct MyData { name: String, location: Location }
let my_index: SpatialIndex<MyData> = SpatialIndex::new();
// Index of references
let ref_index: SpatialIndex<&Event> = SpatialIndex::new();
// Index of IDs (lookup in separate collection)
let id_index: SpatialIndex<usize> = SpatialIndex::new();
Building from Iterators
Efficiently build indexes from collections:
// From events with extractors
let spatial = SpatialIndex::from_iter(
events.iter().cloned(),
|e| &e.location
);
let temporal = TemporalIndex::from_iter(
events.iter().cloned(),
|e| &e.timestamp
);
let combined = SpatiotemporalIndex::from_iter(
events.iter().cloned(),
|e| &e.location,
|e| &e.timestamp
);
Next Steps
- Spatial Index - R-tree geographic queries
- Temporal Index - B-tree time queries
- Spatiotemporal Index - Combined queries
- Heatmaps - Density visualization
Spatial Index (R-tree)
The SpatialIndex uses an R-tree for efficient geographic queries.
Creating an Index
use spatial_narrative::index::SpatialIndex;
use spatial_narrative::core::{Event, Location, Timestamp};
// Empty index
let mut index: SpatialIndex<Event> = SpatialIndex::new();
// Insert items
let event = Event::new(
Location::new(40.7128, -74.0060),
Timestamp::now(),
"NYC Event"
);
index.insert(event, &Location::new(40.7128, -74.0060));
From Iterator
Build an index efficiently from a collection:
let index = SpatialIndex::from_iter(
events.iter().cloned(),
|event| &event.location
);
Query Types
Bounding Box Query
Find all items within a rectangular region:
// Query by lat/lon bounds
let results = index.query_bbox(
40.0, // min_lat
-75.0, // min_lon
41.0, // max_lat
-73.0 // max_lon
);
println!("Found {} events in region", results.len());
Using GeoBounds
use spatial_narrative::core::GeoBounds;
let nyc_area = GeoBounds::new(40.4, -74.3, 41.0, -73.7);
let results = index.query_bounds(&nyc_area);
Radius Query (Degrees)
Find items within a radius (in degrees):
// Approximate radius query using degrees
// Note: This is Euclidean distance in degrees, not great-circle distance
let results = index.query_radius(
40.7128, // center lat
-74.0060, // center lon
0.1 // radius in degrees (~11km at this latitude)
);
Radius Query (Meters)
For accurate distance-based queries:
// Great-circle distance using Haversine formula
let results = index.query_radius_meters(
40.7128, // center lat
-74.0060, // center lon
5000.0 // radius in meters (5km)
);
K-Nearest Neighbors
Find the K closest items to a point:
// Find 10 nearest events
let nearest = index.nearest(40.7128, -74.0060, 10);
for event in nearest {
println!("Near event: {}", event.text);
}
Methods Reference
| Method | Description |
|---|---|
new() | Create empty index |
from_iter() | Build from iterator with location extractor |
insert(item, location) | Add an item |
query_bbox(min_lat, min_lon, max_lat, max_lon) | Bounding box query |
query_bounds(bounds) | Query using GeoBounds |
query_radius(lat, lon, radius_deg) | Radius query (degrees) |
query_radius_meters(lat, lon, radius_m) | Radius query (meters) |
nearest(lat, lon, k) | K-nearest neighbors |
len() | Number of items |
is_empty() | Check if empty |
Performance Tips
Bulk Loading
For large datasets, use from_iter instead of repeated insert:
// Efficient: bulk load
let index = SpatialIndex::from_iter(events, |e| &e.location);
// Less efficient: repeated inserts
let mut index = SpatialIndex::new();
for event in events {
index.insert(event.clone(), &event.location); // O(log n) each
}
Query Optimization
Start with broad queries, then filter:
// Get candidates from index
let candidates = index.query_bbox(39.0, -76.0, 42.0, -72.0);
// Apply additional filters
let filtered: Vec<_> = candidates
.into_iter()
.filter(|e| e.has_tag("important"))
.filter(|e| e.timestamp.year() == 2024)
.collect();
Use Cases
Finding Nearby Events
let my_location = Location::new(40.7580, -73.9855); // Times Square
// Find events within 1km
let nearby = index.query_radius_meters(
my_location.lat,
my_location.lon,
1000.0
);
println!("Found {} events within 1km", nearby.len());
Regional Analysis
// Define regions
let regions = vec![
("NYC", GeoBounds::new(40.4, -74.3, 41.0, -73.7)),
("LA", GeoBounds::new(33.7, -118.7, 34.4, -117.9)),
("Chicago", GeoBounds::new(41.6, -88.0, 42.1, -87.5)),
];
// Count events per region
for (name, bounds) in ®ions {
let count = index.query_bounds(bounds).len();
println!("{}: {} events", name, count);
}
Clustering Preprocessing
// Use spatial index to speed up DBSCAN-style clustering
let candidates = index.query_radius_meters(point.lat, point.lon, eps);
// Only check distances for nearby points
for candidate in candidates {
let distance = haversine_distance(&point, &candidate.location);
if distance <= eps {
// Add to cluster
}
}
Temporal Index (B-tree)
The TemporalIndex uses a B-tree for efficient time-based queries.
Creating an Index
use spatial_narrative::index::TemporalIndex;
use spatial_narrative::core::{Event, Location, Timestamp};
// Empty index
let mut index: TemporalIndex<Event> = TemporalIndex::new();
// Insert items
let event = Event::new(
Location::new(40.7128, -74.0060),
Timestamp::parse("2024-01-15T10:00:00Z").unwrap(),
"Morning meeting"
);
index.insert(event, &Timestamp::parse("2024-01-15T10:00:00Z").unwrap());
From Iterator
let index = TemporalIndex::from_iter(
events.iter().cloned(),
|event| &event.timestamp
);
Query Types
Time Range Query
Find items within a time range:
use spatial_narrative::core::TimeRange;
let range = TimeRange::new(
Timestamp::parse("2024-01-01T00:00:00Z").unwrap(),
Timestamp::parse("2024-01-31T23:59:59Z").unwrap(),
);
let january_events = index.query_range(&range);
println!("Found {} events in January", january_events.len());
Convenience Ranges
// Entire year
let results = index.query_range(&TimeRange::year(2024));
// Specific month
let results = index.query_range(&TimeRange::month(2024, 6)); // June 2024
// Specific day
let results = index.query_range(&TimeRange::day(2024, 7, 4)); // July 4th
Before/After Queries
let cutoff = Timestamp::parse("2024-06-01T00:00:00Z").unwrap();
// All events before June
let before = index.before(&cutoff);
// All events after June
let after = index.after(&cutoff);
// Including the cutoff time
let at_or_before = index.at_or_before(&cutoff);
let at_or_after = index.at_or_after(&cutoff);
First and Last
// Get earliest event
if let Some(first) = index.first() {
println!("First event: {}", first.text);
}
// Get latest event
if let Some(last) = index.last() {
println!("Last event: {}", last.text);
}
Chronological Iteration
// Iterate in time order
for event in index.chronological() {
println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}
Reverse Chronological
// Most recent first
for event in index.reverse_chronological() {
println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}
Sliding Window
Iterate with a sliding time window:
use spatial_narrative::index::SlidingWindowIter;
use std::time::Duration;
let window_size = Duration::from_secs(3600); // 1 hour
let step = Duration::from_secs(1800); // 30 minute steps
for window in index.sliding_window(window_size, step) {
println!("Window {}: {} events",
window.start.to_rfc3339(),
window.events.len());
}
Methods Reference
| Method | Description |
|---|---|
new() | Create empty index |
from_iter() | Build from iterator |
insert(item, timestamp) | Add an item |
query_range(range) | Query time range |
before(timestamp) | Items before (exclusive) |
after(timestamp) | Items after (exclusive) |
at_or_before(timestamp) | Items at or before (inclusive) |
at_or_after(timestamp) | Items at or after (inclusive) |
first() | Earliest item |
last() | Latest item |
chronological() | Time-ordered iterator |
reverse_chronological() | Reverse time iterator |
sliding_window(size, step) | Sliding window iterator |
time_range() | Get overall time range |
len() | Number of items |
Use Cases
Timeline Visualization
// Get events in order for display
let timeline: Vec<_> = index.chronological().collect();
for event in timeline {
println!("{}: {}",
event.timestamp.format("%Y-%m-%d %H:%M"),
event.text);
}
Activity Analysis
use std::time::Duration;
// Count events per hour
let hour = Duration::from_secs(3600);
let mut hourly_counts = Vec::new();
for window in index.sliding_window(hour, hour) {
hourly_counts.push((window.start, window.events.len()));
}
// Find busiest hour
if let Some((time, count)) = hourly_counts.iter().max_by_key(|(_, c)| c) {
println!("Busiest hour: {} with {} events",
time.to_rfc3339(), count);
}
Recent Events
use std::time::Duration;
// Get events from the last 24 hours
let now = Timestamp::now();
let yesterday = now.subtract(Duration::from_secs(86400));
let range = TimeRange::new(yesterday, now);
let recent = index.query_range(&range);
println!("{} events in the last 24 hours", recent.len());
Gap Detection
// Find gaps between events
let ordered: Vec<_> = index.chronological().collect();
for window in ordered.windows(2) {
let gap = window[1].timestamp.duration_since(&window[0].timestamp);
if gap > Duration::from_secs(3600 * 6) { // 6+ hour gap
println!("Gap from {} to {}",
window[0].timestamp.to_rfc3339(),
window[1].timestamp.to_rfc3339());
}
}
Spatiotemporal Index
The SpatiotemporalIndex combines spatial and temporal indexing for efficient queries on both dimensions.
Creating an Index
use spatial_narrative::index::SpatiotemporalIndex;
use spatial_narrative::core::{Event, Location, Timestamp};
// Empty index
let mut index: SpatiotemporalIndex<Event> = SpatiotemporalIndex::new();
// Insert items
let event = Event::new(
Location::new(40.7128, -74.0060),
Timestamp::parse("2024-01-15T10:00:00Z").unwrap(),
"NYC Event"
);
index.insert(event,
&Location::new(40.7128, -74.0060),
&Timestamp::parse("2024-01-15T10:00:00Z").unwrap()
);
From Iterator
let index = SpatiotemporalIndex::from_iter(
events.iter().cloned(),
|e| &e.location,
|e| &e.timestamp
);
Combined Queries
Query by both space and time:
use spatial_narrative::core::{GeoBounds, TimeRange};
let bounds = GeoBounds::new(40.0, -75.0, 41.0, -73.0); // NYC area
let range = TimeRange::month(2024, 1); // January 2024
// Events in NYC during January
let results = index.query(&bounds, &range);
println!("Found {} events", results.len());
Individual Dimension Queries
Spatial Only
let bounds = GeoBounds::new(40.0, -75.0, 41.0, -73.0);
let in_area = index.query_spatial(&bounds);
Temporal Only
let range = TimeRange::year(2024);
let in_2024 = index.query_temporal(&range);
Nearest in Time Range
Find spatially nearest items within a time range:
let range = TimeRange::day(2024, 1, 15);
// 5 nearest events on January 15th
let nearest = index.nearest_in_range(40.7128, -74.0060, 5, &range);
for event in nearest {
println!("{}: {}", event.timestamp.to_rfc3339(), event.text);
}
Index Properties
// Number of items
println!("Items: {}", index.len());
// Check if empty
if !index.is_empty() {
// Get bounds
if let Some(bounds) = index.bounds() {
println!("Geographic extent: {:.2}° x {:.2}°",
bounds.lat_span(), bounds.lon_span());
}
// Get time range
if let Some(range) = index.time_range() {
println!("Time span: {} to {}",
range.start.to_rfc3339(),
range.end.to_rfc3339());
}
}
// Access all items
for item in index.items() {
println!("{}", item.text);
}
Methods Reference
| Method | Description |
|---|---|
new() | Create empty index |
from_iter() | Build from iterator |
insert(item, location, timestamp) | Add an item |
query(bounds, range) | Query by space AND time |
query_spatial(bounds) | Query by space only |
query_temporal(range) | Query by time only |
nearest_in_range(lat, lon, k, range) | K-nearest in time window |
bounds() | Geographic extent |
time_range() | Temporal extent |
len() | Number of items |
is_empty() | Check if empty |
items() | Access all items |
Use Cases
Incident Analysis
// Find incidents in downtown during business hours
let downtown = GeoBounds::new(40.70, -74.02, 40.75, -73.98);
let business_hours = TimeRange::new(
Timestamp::parse("2024-01-15T09:00:00Z").unwrap(),
Timestamp::parse("2024-01-15T17:00:00Z").unwrap(),
);
let incidents = index.query(&downtown, &business_hours);
println!("Found {} downtown incidents during business hours", incidents.len());
Event Correlation
// Find events near a location around a specific time
let location = (40.7580, -73.9855); // Times Square
let time = Timestamp::parse("2024-01-01T00:00:00Z").unwrap();
// 2 hours around midnight on NYE
let window = TimeRange::new(
time.subtract(std::time::Duration::from_secs(3600)),
time.add(std::time::Duration::from_secs(3600)),
);
let times_square = GeoBounds::new(40.755, -73.990, 40.760, -73.982);
let nye_events = index.query(×_square, &window);
Coverage Analysis
// Check coverage across regions and time periods
let regions = vec![
("North", GeoBounds::new(41.0, -74.5, 42.0, -73.5)),
("Central", GeoBounds::new(40.5, -74.5, 41.0, -73.5)),
("South", GeoBounds::new(40.0, -74.5, 40.5, -73.5)),
];
let months = (1..=12).map(|m| TimeRange::month(2024, m));
println!("Coverage by region and month:");
for (name, bounds) in ®ions {
print!("{}: ", name);
for month in months.clone() {
let count = index.query(bounds, &month).len();
print!("{} ", count);
}
println!();
}
Anomaly Detection
// Find unusual activity spikes
let normal_bounds = GeoBounds::new(40.0, -75.0, 42.0, -73.0);
// Compare weekday vs weekend activity
let weekday = TimeRange::new(
Timestamp::parse("2024-01-15T00:00:00Z").unwrap(), // Monday
Timestamp::parse("2024-01-15T23:59:59Z").unwrap(),
);
let weekend = TimeRange::new(
Timestamp::parse("2024-01-13T00:00:00Z").unwrap(), // Saturday
Timestamp::parse("2024-01-13T23:59:59Z").unwrap(),
);
let weekday_count = index.query(&normal_bounds, &weekday).len();
let weekend_count = index.query(&normal_bounds, &weekend).len();
println!("Weekday: {} events, Weekend: {} events", weekday_count, weekend_count);
Heatmaps
Generate spatial density data for visualization.
Overview
The Heatmap struct represents event density across a grid, useful for:
- Visualizing activity hotspots
- Identifying high-density areas
- Creating choropleth maps
Generating Heatmaps
use spatial_narrative::index::{SpatiotemporalIndex, GridSpec, Heatmap};
use spatial_narrative::core::GeoBounds;
// Build index from events
let index = SpatiotemporalIndex::from_iter(
events.iter().cloned(),
|e| &e.location,
|e| &e.timestamp
);
// Define grid
let bounds = GeoBounds::new(40.0, -75.0, 42.0, -73.0);
let grid = GridSpec::new(bounds, 50, 50); // 50x50 cells
// Generate heatmap
let heatmap = index.heatmap(grid);
GridSpec
Configure the heatmap grid:
use spatial_narrative::index::GridSpec;
use spatial_narrative::core::GeoBounds;
let grid = GridSpec::new(
GeoBounds::new(40.0, -75.0, 42.0, -73.0), // bounds
100, // lat_cells (rows)
100, // lon_cells (columns)
);
// Get cell dimensions
let (lat_size, lon_size) = grid.cell_size();
println!("Cell size: {:.4}° lat x {:.4}° lon", lat_size, lon_size);
Accessing Heatmap Data
Raw Counts
// Get count at specific cell
let count = heatmap.get(25, 30); // row 25, column 30
println!("Cell (25, 30): {} events", count);
// Maximum count
let max = heatmap.max_count();
println!("Maximum density: {} events", max);
Normalized Values
// Get normalized value (0.0 to 1.0)
let intensity = heatmap.get_normalized(25, 30);
println!("Cell (25, 30) intensity: {:.2}", intensity);
Cell Centers
// Get the center coordinates of a cell
let (lat, lon) = heatmap.cell_center(25, 30);
println!("Cell center: ({:.4}, {:.4})", lat, lon);
Iterating Over Cells
// Iterate over all cells
for lat_idx in 0..heatmap.grid.lat_cells {
for lon_idx in 0..heatmap.grid.lon_cells {
let count = heatmap.get(lat_idx, lon_idx);
if count > 0 {
let (lat, lon) = heatmap.cell_center(lat_idx, lon_idx);
println!("({:.4}, {:.4}): {} events", lat, lon, count);
}
}
}
Exporting for Visualization
As GeoJSON Grid
use serde_json::json;
let mut features = Vec::new();
for lat_idx in 0..heatmap.grid.lat_cells {
for lon_idx in 0..heatmap.grid.lon_cells {
let count = heatmap.get(lat_idx, lon_idx);
if count > 0 {
let (lat_size, lon_size) = heatmap.grid.cell_size();
let min_lat = heatmap.grid.bounds.min_lat + lat_idx as f64 * lat_size;
let min_lon = heatmap.grid.bounds.min_lon + lon_idx as f64 * lon_size;
features.push(json!({
"type": "Feature",
"properties": {
"count": count,
"intensity": heatmap.get_normalized(lat_idx, lon_idx)
},
"geometry": {
"type": "Polygon",
"coordinates": [[
[min_lon, min_lat],
[min_lon + lon_size, min_lat],
[min_lon + lon_size, min_lat + lat_size],
[min_lon, min_lat + lat_size],
[min_lon, min_lat]
]]
}
}));
}
}
}
let geojson = json!({
"type": "FeatureCollection",
"features": features
});
As CSV
use std::fs::File;
use std::io::Write;
let mut file = File::create("heatmap.csv")?;
writeln!(file, "lat,lon,count,intensity")?;
for lat_idx in 0..heatmap.grid.lat_cells {
for lon_idx in 0..heatmap.grid.lon_cells {
let count = heatmap.get(lat_idx, lon_idx);
let (lat, lon) = heatmap.cell_center(lat_idx, lon_idx);
let intensity = heatmap.get_normalized(lat_idx, lon_idx);
writeln!(file, "{},{},{},{}", lat, lon, count, intensity)?;
}
}
Visualization Integration
Leaflet.heat
// Load heatmap data
const heatData = [];
for (const feature of geojson.features) {
const coords = feature.geometry.coordinates[0][0];
heatData.push([
(coords[1] + feature.geometry.coordinates[0][2][1]) / 2, // center lat
(coords[0] + feature.geometry.coordinates[0][1][0]) / 2, // center lon
feature.properties.intensity // intensity
]);
}
L.heatLayer(heatData, {radius: 25}).addTo(map);
Mapbox GL JS
map.addSource('heatmap', {
type: 'geojson',
data: geojson
});
map.addLayer({
id: 'heat',
type: 'fill',
source: 'heatmap',
paint: {
'fill-color': [
'interpolate',
['linear'],
['get', 'intensity'],
0, 'rgba(0, 0, 255, 0)',
0.5, 'rgba(255, 255, 0, 0.5)',
1, 'rgba(255, 0, 0, 0.8)'
]
}
});
Use Cases
Finding Hotspots
// Find cells with highest activity
let threshold = heatmap.max_count() / 2;
let mut hotspots = Vec::new();
for lat_idx in 0..heatmap.grid.lat_cells {
for lon_idx in 0..heatmap.grid.lon_cells {
if heatmap.get(lat_idx, lon_idx) >= threshold {
hotspots.push(heatmap.cell_center(lat_idx, lon_idx));
}
}
}
println!("Found {} hotspot cells", hotspots.len());
Comparing Time Periods
// Generate heatmaps for different periods
let morning = TimeRange::new(
Timestamp::parse("2024-01-15T06:00:00Z").unwrap(),
Timestamp::parse("2024-01-15T12:00:00Z").unwrap(),
);
let evening = TimeRange::new(
Timestamp::parse("2024-01-15T18:00:00Z").unwrap(),
Timestamp::parse("2024-01-16T00:00:00Z").unwrap(),
);
// Filter and generate separate heatmaps
let morning_events = index.query_temporal(&morning);
let evening_events = index.query_temporal(&evening);
// Compare distributions...
Graph Overview
The graph module provides tools for representing narratives as directed graphs.
Concepts
In a narrative graph:
- Nodes are events
- Edges are relationships between events
- Edge types describe the nature of relationships
NarrativeGraph
use spatial_narrative::graph::{NarrativeGraph, EdgeType};
use spatial_narrative::core::{Event, Location, Timestamp};
// Create events
let e1 = Event::new(Location::new(40.7, -74.0), Timestamp::now(), "Event 1");
let e2 = Event::new(Location::new(40.7, -74.0), Timestamp::now(), "Event 2");
// Build graph
let mut graph = NarrativeGraph::new();
let n1 = graph.add_event(e1);
let n2 = graph.add_event(e2);
// Connect events
graph.connect(n1, n2, EdgeType::Temporal);
println!("Nodes: {}, Edges: {}", graph.node_count(), graph.edge_count());
Edge Types
| Type | Description | Use Case |
|---|---|---|
Temporal | Time sequence | A happens before B |
Spatial | Geographic proximity | A and B are near each other |
Causal | Cause and effect | A causes B |
Thematic | Shared themes/tags | A and B cover same topic |
Reference | Citation/mention | A references B |
Custom | User-defined | Domain-specific relationships |
Quick Example
use spatial_narrative::graph::{NarrativeGraph, EdgeType};
use spatial_narrative::core::Narrative;
// Create from narrative
let mut graph = NarrativeGraph::from_events(narrative.events.clone());
// Auto-connect by different strategies
graph.connect_temporal(); // Connect events in time order
graph.connect_spatial(10.0); // Connect events within 10km
graph.connect_thematic(); // Connect events sharing tags
// Query the graph
println!("Connected events: {}", graph.edge_count());
// Find paths
if let Some(path) = graph.shortest_path(node_a, node_b) {
println!("Path length: {} nodes", path.nodes.len());
}
Why Use Graphs?
Graphs enable powerful analysis:
- Path finding: How are two events connected?
- Clustering: Which events form natural groups?
- Centrality: Which events are most connected?
- Subgraphs: Extract related event clusters
- Visualization: Export to Graphviz, D3.js, etc.
Module Contents
NarrativeGraph- Main graph structure- Connection strategies - Auto-connecting events
- Path finding - Shortest paths and connectivity
- Export - DOT format for visualization
Visualization Preview
Export to DOT and render with Graphviz:
let dot = graph.to_dot();
std::fs::write("graph.dot", dot)?;
// Run: dot -Tpng graph.dot -o graph.png
┌─────────┐ temporal ┌─────────┐ spatial ┌─────────┐
│ Event 1 │ ──────────▶│ Event 2 │ ──────────▶│ Event 3 │
└─────────┘ └─────────┘ └─────────┘
│ │
│ thematic │ causal
▼ ▼
┌─────────┐ ┌─────────┐
│ Event 4 │ │ Event 5 │
└─────────┘ └─────────┘
Building Graphs
Create and populate narrative graphs.
Creating a Graph
Empty Graph
use spatial_narrative::graph::NarrativeGraph;
let mut graph = NarrativeGraph::new();
From Events
use spatial_narrative::graph::NarrativeGraph;
use spatial_narrative::core::{Event, Location, Timestamp};
let events = vec![
Event::new(Location::new(40.7, -74.0), Timestamp::now(), "Event 1"),
Event::new(Location::new(40.8, -74.1), Timestamp::now(), "Event 2"),
];
let graph = NarrativeGraph::from_events(events);
println!("Graph has {} nodes", graph.node_count());
From Narrative
let graph = NarrativeGraph::from_events(narrative.events.clone());
Adding Events
use spatial_narrative::graph::NarrativeGraph;
let mut graph = NarrativeGraph::new();
// Add returns NodeId for later reference
let node1 = graph.add_event(event1);
let node2 = graph.add_event(event2);
// Look up node by event ID
let node = graph.get_node(&event.id);
Connecting Events
Basic Connection
use spatial_narrative::graph::EdgeType;
graph.connect(node1, node2, EdgeType::Temporal);
Weighted Connection
use spatial_narrative::graph::EdgeWeight;
// Create weighted edge
let weight = EdgeWeight::with_weight(EdgeType::Spatial, 0.8);
graph.connect_weighted(node1, node2, weight);
// With label
let weight = EdgeWeight::new(EdgeType::Causal)
.with_label("caused by");
graph.connect_weighted(node1, node2, weight);
Accessing Nodes and Edges
Get Event by Node
if let Some(event) = graph.event(node_id) {
println!("Event: {}", event.text);
}
// Mutable access
if let Some(event) = graph.event_mut(node_id) {
event.add_tag("processed");
}
Iterate Nodes
for (node_id, event) in graph.nodes() {
println!("Node {}: {}", node_id.index(), event.text);
}
Iterate Edges
for (from, to, weight) in graph.edges() {
println!("{} -> {} ({:?})",
from.index(), to.index(), weight.edge_type);
}
Filter Edges by Type
let temporal_edges = graph.edges_of_type(EdgeType::Temporal);
println!("Temporal connections: {}", temporal_edges.len());
Graph Properties
// Counts
println!("Nodes: {}", graph.node_count());
println!("Edges: {}", graph.edge_count());
println!("Empty: {}", graph.is_empty());
// Degree analysis
let in_degree = graph.in_degree(node); // Incoming edges
let out_degree = graph.out_degree(node); // Outgoing edges
println!("Node has {} in, {} out", in_degree, out_degree);
Neighbors
// Get connected nodes
let successors = graph.successors(node); // Outgoing connections
let predecessors = graph.predecessors(node); // Incoming connections
for successor in successors {
if let Some(event) = graph.event(successor) {
println!("Leads to: {}", event.text);
}
}
Checking Connectivity
// Direct connection
if graph.are_connected(node1, node2) {
println!("Directly connected");
}
// Any path exists
if graph.has_path(node1, node2) {
println!("Path exists");
}
Special Nodes
// Entry points (no predecessors)
let roots = graph.roots();
println!("Entry points: {}", roots.len());
// End points (no successors)
let leaves = graph.leaves();
println!("End points: {}", leaves.len());
Subgraphs
Extract portions of the graph:
use spatial_narrative::core::{TimeRange, GeoBounds};
// By time
let january = TimeRange::month(2024, 1);
let subgraph = graph.subgraph_temporal(&january);
println!("January subgraph: {} nodes", subgraph.graph.node_count());
// By location
let nyc = GeoBounds::new(40.4, -74.3, 41.0, -73.7);
let subgraph = graph.subgraph_spatial(&nyc);
println!("NYC subgraph: {} nodes", subgraph.graph.node_count());
SubgraphResult
let result = graph.subgraph_temporal(&range);
// The new graph
let new_graph = result.graph;
// Mapping from old to new node IDs
for (old_id, new_id) in result.node_mapping {
println!("Node {} -> {}", old_id.index(), new_id.index());
}
Example: Building a Story Graph
// Create graph from news events
let mut graph = NarrativeGraph::from_events(news_events);
// Connect by time (earlier → later)
graph.connect_temporal();
// Connect nearby events (within 1km)
graph.connect_spatial(1.0);
// Connect events sharing topics
graph.connect_thematic();
// Add manual causal links
let cause = graph.get_node(&cause_event.id).unwrap();
let effect = graph.get_node(&effect_event.id).unwrap();
graph.connect(cause, effect, EdgeType::Causal);
println!("Story graph: {} events, {} connections",
graph.node_count(), graph.edge_count());
Connection Strategies
Automatically connect events based on different criteria.
Temporal Connection
Connect events in chronological order:
let mut graph = NarrativeGraph::from_events(events);
// Connect A → B if A happens before B
graph.connect_temporal();
This creates a chain:
Event 1 ─temporal→ Event 2 ─temporal→ Event 3 ─temporal→ ...
How It Works
- Events are sorted by timestamp
- Each event connects to the next in sequence
- Creates
EdgeType::Temporaledges - Existing edges are not duplicated
Spatial Connection
Connect events that are geographically close:
// Connect events within 10km of each other
graph.connect_spatial(10.0);
// Closer threshold for dense areas
graph.connect_spatial(1.0); // 1km
How It Works
- Compares all pairs of events
- Uses Haversine distance (great-circle)
- Creates bidirectional
EdgeType::Spatialedges - Edge weight = 1.0 - (distance / max_distance)
// Edge weight indicates proximity
// Weight 1.0 = same location
// Weight 0.5 = half the max distance apart
// Weight 0.0 = at max distance
Thematic Connection
Connect events that share tags:
graph.connect_thematic();
How It Works
- Events sharing one or more tags are connected
- Creates bidirectional
EdgeType::Thematicedges - Edge weight = shared_tags / max_tags
// Example: Event A has tags ["news", "politics"]
// Event B has tags ["politics", "election"]
// Shared: 1 tag, Max: 2 tags
// Weight: 0.5
Manual Connections
For relationships that can't be auto-detected:
use spatial_narrative::graph::{EdgeType, EdgeWeight};
// Get nodes
let cause = graph.get_node(&event1.id).unwrap();
let effect = graph.get_node(&event2.id).unwrap();
// Simple connection
graph.connect(cause, effect, EdgeType::Causal);
// With weight and label
let weight = EdgeWeight::with_weight(EdgeType::Reference, 0.9)
.with_label("cites");
graph.connect_weighted(cause, effect, weight);
Edge Types Reference
| Type | Auto-Connect | Direction | Weight Meaning |
|---|---|---|---|
Temporal | connect_temporal() | Unidirectional | Always 1.0 |
Spatial | connect_spatial(km) | Bidirectional | Proximity |
Thematic | connect_thematic() | Bidirectional | Tag overlap |
Causal | Manual | Unidirectional | Strength |
Reference | Manual | Unidirectional | Relevance |
Custom | Manual | Either | User-defined |
Combining Strategies
Apply multiple connection strategies:
let mut graph = NarrativeGraph::from_events(events);
// Build a rich connection graph
graph.connect_temporal(); // Time sequence
graph.connect_spatial(5.0); // 5km proximity
graph.connect_thematic(); // Shared topics
println!("Created {} connections", graph.edge_count());
// Analyze by type
let temporal = graph.edges_of_type(EdgeType::Temporal).len();
let spatial = graph.edges_of_type(EdgeType::Spatial).len();
let thematic = graph.edges_of_type(EdgeType::Thematic).len();
println!(" Temporal: {}", temporal);
println!(" Spatial: {}", spatial);
println!(" Thematic: {}", thematic);
Selective Connection
Connect only certain events:
// First, add all events
let mut graph = NarrativeGraph::from_events(events);
// Connect only important events temporally
let important: Vec<_> = graph.nodes()
.filter(|(_, e)| e.has_tag("important"))
.map(|(id, _)| id)
.collect();
// Sort by time and connect
let mut sorted: Vec<_> = important.iter()
.filter_map(|&id| graph.event(id).map(|e| (id, e.timestamp.clone())))
.collect();
sorted.sort_by(|a, b| a.1.cmp(&b.1));
for window in sorted.windows(2) {
graph.connect(window[0].0, window[1].0, EdgeType::Temporal);
}
Use Cases
News Story Tracking
// Connect news events
graph.connect_temporal(); // Story progression
graph.connect_thematic(); // Related topics
// Find story threads
for root in graph.roots() {
println!("Story starts: {}", graph.event(root).unwrap().text);
let thread: Vec<_> = std::iter::successors(Some(root), |&n| {
graph.successors(n).into_iter().next()
}).collect();
println!(" {} events in thread", thread.len());
}
Location-Based Analysis
// Focus on spatial relationships
graph.connect_spatial(0.5); // 500m - same block
graph.connect_spatial(2.0); // 2km - neighborhood
// Find location clusters
let high_degree: Vec<_> = graph.nodes()
.filter(|(id, _)| graph.in_degree(*id) + graph.out_degree(*id) > 5)
.collect();
println!("Highly connected locations: {}", high_degree.len());
Topic Networks
// Build topic network
graph.connect_thematic();
// Find central topics (most connected)
let mut connections: Vec<_> = graph.nodes()
.map(|(id, _)| (id, graph.in_degree(id) + graph.out_degree(id)))
.collect();
connections.sort_by(|a, b| b.1.cmp(&a.1));
println!("Most connected events:");
for (id, degree) in connections.iter().take(5) {
let event = graph.event(*id).unwrap();
println!(" {} ({} connections)", event.text, degree);
}
Path Finding
Find paths and analyze connectivity in narrative graphs.
Connectivity Checks
Direct Connection
// Are two nodes directly connected?
if graph.are_connected(node_a, node_b) {
println!("A connects directly to B");
}
Path Exists
// Is there any path from A to B?
if graph.has_path(node_a, node_b) {
println!("A can reach B (directly or indirectly)");
} else {
println!("No path from A to B");
}
Shortest Path
Find the shortest path between two nodes:
if let Some(path) = graph.shortest_path(start, end) {
println!("Path found:");
println!(" Nodes: {}", path.nodes.len());
println!(" Total weight: {:.2}", path.total_weight);
// Print path
for node_id in &path.nodes {
let event = graph.event(*node_id).unwrap();
println!(" → {}", event.text);
}
} else {
println!("No path exists");
}
PathInfo
pub struct PathInfo {
pub nodes: Vec<NodeId>, // Nodes in order
pub total_weight: f64, // Sum of edge weights
}
impl PathInfo {
pub fn len(&self) -> usize {
self.nodes.len()
}
}
Neighborhood Analysis
Successors (Outgoing)
// Events this event leads to
let following = graph.successors(node);
for next in following {
let event = graph.event(next).unwrap();
println!("Leads to: {}", event.text);
}
Predecessors (Incoming)
// Events that lead to this event
let previous = graph.predecessors(node);
for prev in previous {
let event = graph.event(prev).unwrap();
println!("Preceded by: {}", event.text);
}
Graph Structure
Entry Points (Roots)
Events with no predecessors - story starting points:
let roots = graph.roots();
println!("Story begins at {} events", roots.len());
for root in roots {
let event = graph.event(root).unwrap();
println!(" Start: {}", event.text);
}
End Points (Leaves)
Events with no successors - story endings:
let leaves = graph.leaves();
println!("Story ends at {} events", leaves.len());
for leaf in leaves {
let event = graph.event(leaf).unwrap();
println!(" End: {}", event.text);
}
Degree Analysis
// Incoming connections
let in_deg = graph.in_degree(node);
// Outgoing connections
let out_deg = graph.out_degree(node);
println!("Node has {} incoming, {} outgoing connections", in_deg, out_deg);
// Find highly connected nodes
for (node_id, event) in graph.nodes() {
let total_degree = graph.in_degree(node_id) + graph.out_degree(node_id);
if total_degree > 5 {
println!("Hub: {} ({} connections)", event.text, total_degree);
}
}
Traversal Patterns
Follow Timeline
// Follow temporal connections from a starting point
fn follow_timeline(graph: &NarrativeGraph, start: NodeId) -> Vec<NodeId> {
let mut path = vec![start];
let mut current = start;
while let Some(next) = graph.successors(current)
.into_iter()
.filter(|&n| {
graph.edges()
.any(|(from, to, w)| from == current && to == n
&& w.edge_type == EdgeType::Temporal)
})
.next()
{
path.push(next);
current = next;
}
path
}
Find All Paths
// Find all paths between two nodes (BFS)
fn find_all_paths(
graph: &NarrativeGraph,
start: NodeId,
end: NodeId,
max_depth: usize
) -> Vec<Vec<NodeId>> {
let mut paths = Vec::new();
let mut queue = vec![(vec![start], 0)];
while let Some((path, depth)) = queue.pop() {
if depth > max_depth {
continue;
}
let current = *path.last().unwrap();
if current == end {
paths.push(path);
continue;
}
for next in graph.successors(current) {
if !path.contains(&next) {
let mut new_path = path.clone();
new_path.push(next);
queue.push((new_path, depth + 1));
}
}
}
paths
}
Use Cases
Story Thread Extraction
// Extract complete story threads from roots to leaves
for root in graph.roots() {
let thread = follow_timeline(&graph, root);
println!("Story thread ({} events):", thread.len());
for node_id in &thread {
let event = graph.event(*node_id).unwrap();
println!(" {}: {}", event.timestamp.to_rfc3339(), event.text);
}
}
Finding Connection Between Events
// How are two events connected?
let event_a = graph.get_node(&event1.id).unwrap();
let event_b = graph.get_node(&event2.id).unwrap();
if let Some(path) = graph.shortest_path(event_a, event_b) {
println!("Connection found via {} intermediate events:", path.len() - 2);
for window in path.nodes.windows(2) {
let from = graph.event(window[0]).unwrap();
let to = graph.event(window[1]).unwrap();
// Find edge type
let edge_type = graph.edges()
.find(|(f, t, _)| *f == window[0] && *t == window[1])
.map(|(_, _, w)| w.edge_type);
println!(" {} →[{:?}]→ {}", from.text, edge_type, to.text);
}
}
Hub Detection
// Find events that connect many other events
let mut hubs: Vec<_> = graph.nodes()
.map(|(id, event)| {
let degree = graph.in_degree(id) + graph.out_degree(id);
(id, event, degree)
})
.collect();
hubs.sort_by(|a, b| b.2.cmp(&a.2));
println!("Top 5 hub events:");
for (_, event, degree) in hubs.iter().take(5) {
println!(" {} ({} connections)", event.text, degree);
}
Graph Export & Visualization
Export your narrative graphs for visualization in external tools.
DOT Format (Graphviz)
Export to DOT format for rendering with Graphviz.
Basic Export
use spatial_narrative::graph::NarrativeGraph;
let graph = NarrativeGraph::from_events(events);
graph.connect_temporal();
// Export to DOT
let dot = graph.to_dot();
// Save to file
std::fs::write("narrative.dot", &dot)?;
Rendering with Graphviz
# Install Graphviz
# Ubuntu/Debian: sudo apt install graphviz
# macOS: brew install graphviz
# Windows: choco install graphviz
# Render to PNG
dot -Tpng narrative.dot -o narrative.png
# Render to SVG (better for web)
dot -Tsvg narrative.dot -o narrative.svg
# Render to PDF
dot -Tpdf narrative.dot -o narrative.pdf
Custom Options
use spatial_narrative::graph::DotOptions;
// Timeline layout (left-to-right)
let timeline_dot = graph.to_dot_with_options(DotOptions::timeline());
// Hierarchical layout (top-to-bottom)
let hier_dot = graph.to_dot_with_options(DotOptions::hierarchical());
// Custom options
let options = DotOptions {
rank_direction: "LR".to_string(),
node_shape: "ellipse".to_string(),
font_name: "Helvetica".to_string(),
};
let custom_dot = graph.to_dot_with_options(options);
Online Visualization
Paste DOT output into online tools:
JSON Format
Export for web visualization libraries.
Basic Export
// Compact JSON
let json = graph.to_json();
// Pretty-printed JSON
let json = graph.to_json_pretty();
// Save to file
std::fs::write("narrative.json", &json)?;
JSON Structure
{
"nodes": [
{
"id": 0,
"event_id": "550e8400-e29b-41d4-a716-446655440000",
"text": "Event description",
"location": { "lat": 40.7128, "lon": -74.006 },
"timestamp": "2024-01-15T10:00:00+00:00",
"tags": ["conference", "technology"]
}
],
"edges": [
{
"source": 0,
"target": 1,
"type": "Temporal",
"weight": 1.0,
"label": null
}
],
"metadata": {
"node_count": 5,
"edge_count": 8
}
}
Web Visualization Libraries
D3.js
// Load the JSON
fetch('narrative.json')
.then(response => response.json())
.then(data => {
// Create force-directed graph
const simulation = d3.forceSimulation(data.nodes)
.force("link", d3.forceLink(data.edges).id(d => d.id))
.force("charge", d3.forceManyBody())
.force("center", d3.forceCenter(width / 2, height / 2));
});
Cytoscape.js
const cy = cytoscape({
container: document.getElementById('cy'),
elements: {
nodes: data.nodes.map(n => ({ data: { id: n.id, label: n.text } })),
edges: data.edges.map(e => ({ data: { source: e.source, target: e.target } }))
}
});
Sigma.js
const graph = new graphology.Graph();
data.nodes.forEach(n => graph.addNode(n.id, { label: n.text }));
data.edges.forEach(e => graph.addEdge(e.source, e.target));
const sigma = new Sigma(graph, container);
Node Colors
The DOT export uses automatic node coloring:
| Color | Meaning |
|---|---|
| 🟢 Light Green | Root nodes (no incoming edges) |
| 🩷 Light Pink | Leaf nodes (no outgoing edges) |
| 🔵 Light Blue | Hub nodes (high connectivity) |
| 🟡 Light Yellow | Regular nodes |
Edge Styles
Edges are styled by type:
| Type | Color | Style |
|---|---|---|
| Temporal | Blue | Solid |
| Spatial | Magenta | Dashed |
| Causal | Orange | Bold |
| Thematic | Red | Dotted |
| Reference | Olive | Solid |
| Custom | Gray | Solid |
Analysis Overview
The analysis module provides tools for extracting insights from spatial narratives.
Features
| Feature | Description | Types |
|---|---|---|
| Spatial Metrics | Geographic extent, distances, dispersion | [SpatialMetrics] |
| Temporal Metrics | Duration, rates, gaps, bursts | [TemporalMetrics] |
| Movement | Trajectory extraction and stop detection | [Trajectory], [Stop] |
| Clustering | Density and centroid-based clustering | [DBSCAN], [KMeans] |
| Comparison | Similarity scoring between narratives | [compare_narratives] |
Quick Examples
Spatial Metrics
use spatial_narrative::analysis::SpatialMetrics;
use spatial_narrative::core::{Event, Location, Timestamp};
let events = vec![
Event::new(Location::new(40.7128, -74.0060), Timestamp::now(), "NYC"),
Event::new(Location::new(34.0522, -118.2437), Timestamp::now(), "LA"),
Event::new(Location::new(41.8781, -87.6298), Timestamp::now(), "Chicago"),
];
let metrics = SpatialMetrics::from_events(&events);
println!("Total distance: {:.0} km", metrics.total_distance / 1000.0);
println!("Geographic extent: {:.0} km", metrics.max_extent / 1000.0);
println!("Centroid: ({:.4}, {:.4})", metrics.centroid.0, metrics.centroid.1);
Temporal Metrics
use spatial_narrative::analysis::TemporalMetrics;
let metrics = TemporalMetrics::from_events(&events);
println!("Duration: {:?}", metrics.duration);
println!("Event count: {}", metrics.event_count);
println!("Average interval: {:?}", metrics.avg_interval);
DBSCAN Clustering
use spatial_narrative::analysis::DBSCAN;
// Cluster events within 5km, requiring 2+ points per cluster
let dbscan = DBSCAN::new(5000.0, 2);
let result = dbscan.cluster(&events);
println!("Found {} clusters", result.num_clusters());
println!("Noise points: {}", result.noise.len());
K-Means Clustering
use spatial_narrative::analysis::KMeans;
// Partition events into 3 clusters
let kmeans = KMeans::new(3);
let result = kmeans.cluster(&events);
for (i, cluster) in result.clusters.iter().enumerate() {
println!("Cluster {}: {} events", i, cluster.events.len());
}
Trajectory Analysis
use spatial_narrative::analysis::{Trajectory, detect_stops, StopThreshold};
// Extract trajectory from events
let trajectory = Trajectory::from_events(&events);
println!("Trajectory length: {:.0} m", trajectory.total_distance());
println!("Duration: {:?}", trajectory.duration());
// Detect stops (stationary periods)
let threshold = StopThreshold::new(50.0, std::time::Duration::from_secs(300));
let stops = detect_stops(&events, &threshold);
for stop in stops {
println!("Stop at ({:.4}, {:.4}) for {:?}",
stop.location.lat, stop.location.lon, stop.duration);
}
Comparing Narratives
use spatial_narrative::analysis::{compare_narratives, ComparisonConfig};
use spatial_narrative::core::Narrative;
let similarity = compare_narratives(&narrative1, &narrative2, &ComparisonConfig::default());
println!("Spatial similarity: {:.2}", similarity.spatial);
println!("Temporal similarity: {:.2}", similarity.temporal);
println!("Thematic similarity: {:.2}", similarity.thematic);
println!("Overall similarity: {:.2}", similarity.overall);
Module Structure
spatial_narrative::analysis
├── SpatialMetrics # Geographic analysis
├── TemporalMetrics # Time-based analysis
├── Trajectory # Movement paths
├── DBSCAN # Density clustering
├── KMeans # K-means clustering
├── detect_stops # Stop detection
├── detect_gaps # Gap detection
├── detect_bursts # Burst detection
└── compare_narratives # Narrative comparison
Next Steps
- Spatial Metrics - Geographic analysis
- Temporal Metrics - Time-based analysis
- Movement Analysis - Trajectories and stops
- Clustering - Event grouping
- Comparison - Narrative similarity
Spatial Metrics
Analyze the geographic characteristics of events.
SpatialMetrics
The SpatialMetrics struct provides geographic measurements for a set of events.
use spatial_narrative::analysis::SpatialMetrics;
use spatial_narrative::core::{Event, Location, Timestamp};
let events = vec![
Event::new(Location::new(40.7128, -74.0060), Timestamp::now(), "NYC"),
Event::new(Location::new(34.0522, -118.2437), Timestamp::now(), "LA"),
Event::new(Location::new(41.8781, -87.6298), Timestamp::now(), "Chicago"),
];
let metrics = SpatialMetrics::from_events(&events);
Available Metrics
| Metric | Type | Description |
|---|---|---|
total_distance | f64 | Sum of distances between consecutive events (meters) |
max_extent | f64 | Maximum distance between any two events (meters) |
centroid | (f64, f64) | Geographic center (lat, lon) |
bounds | GeoBounds | Bounding box containing all events |
dispersion | f64 | Standard distance from centroid (meters) |
Example Output
println!("Metrics for {} events:", events.len());
println!(" Total distance: {:.1} km", metrics.total_distance / 1000.0);
println!(" Maximum extent: {:.1} km", metrics.max_extent / 1000.0);
println!(" Centroid: ({:.4}°, {:.4}°)", metrics.centroid.0, metrics.centroid.1);
println!(" Dispersion: {:.1} km", metrics.dispersion / 1000.0);
println!(" Bounds: {:.2}°N to {:.2}°N, {:.2}°W to {:.2}°W",
metrics.bounds.min_lat, metrics.bounds.max_lat,
metrics.bounds.min_lon.abs(), metrics.bounds.max_lon.abs());
Distance Functions
Haversine Distance
Calculate great-circle distance between two points:
use spatial_narrative::analysis::haversine_distance;
use spatial_narrative::core::Location;
let nyc = Location::new(40.7128, -74.0060);
let london = Location::new(51.5074, -0.1278);
let distance_m = haversine_distance(&nyc, &london);
println!("NYC to London: {:.0} km", distance_m / 1000.0); // ~5570 km
Bearing
Calculate initial bearing between two points:
use spatial_narrative::analysis::bearing;
let from = Location::new(40.7128, -74.0060); // NYC
let to = Location::new(51.5074, -0.1278); // London
let degrees = bearing(&from, &to);
println!("Bearing: {:.1}°", degrees); // ~51° (northeast)
Destination Point
Calculate destination given start, bearing, and distance:
use spatial_narrative::analysis::destination_point;
let start = Location::new(40.7128, -74.0060);
let bearing_deg = 45.0; // Northeast
let distance_m = 100_000.0; // 100 km
let dest = destination_point(&start, bearing_deg, distance_m);
println!("Destination: ({:.4}, {:.4})", dest.lat, dest.lon);
Density Mapping
Generate a density map of event locations:
use spatial_narrative::analysis::{density_map, DensityCell};
use spatial_narrative::core::GeoBounds;
// Define grid
let bounds = GeoBounds::new(34.0, -118.5, 41.0, -73.5);
let lat_cells = 50;
let lon_cells = 50;
// Generate density map
let cells = density_map(&events, &bounds, lat_cells, lon_cells);
// Find hotspots
let max_density = cells.iter().map(|c| c.count).max().unwrap_or(0);
let hotspots: Vec<_> = cells.iter()
.filter(|c| c.count > max_density / 2)
.collect();
println!("Found {} hotspot cells", hotspots.len());
DensityCell
Each cell in the density map contains:
| Field | Type | Description |
|---|---|---|
lat_idx | usize | Row index |
lon_idx | usize | Column index |
count | usize | Number of events in cell |
center | (f64, f64) | Cell center coordinates |
Use Cases
Analyzing Geographic Spread
let metrics = SpatialMetrics::from_events(&events);
if metrics.max_extent > 1_000_000.0 {
println!("Events span over 1000 km - continental scale");
} else if metrics.max_extent > 100_000.0 {
println!("Events span over 100 km - regional scale");
} else {
println!("Events are localized - city scale");
}
Finding the Geographic Center
let metrics = SpatialMetrics::from_events(&events);
let center = Location::new(metrics.centroid.0, metrics.centroid.1);
// Find events nearest to center
for event in &events {
let dist = haversine_distance(&event.location, ¢er);
if dist < 10_000.0 { // Within 10km of center
println!("Near center: {}", event.text);
}
}
Temporal Metrics
Analyze the time-based characteristics of events.
TemporalMetrics
The TemporalMetrics struct provides time-based measurements for a set of events.
use spatial_narrative::analysis::TemporalMetrics;
use spatial_narrative::core::{Event, Location, Timestamp};
let events = vec![
Event::new(Location::new(40.7, -74.0),
Timestamp::parse("2024-01-01T10:00:00Z").unwrap(), "Morning"),
Event::new(Location::new(40.7, -74.0),
Timestamp::parse("2024-01-01T14:00:00Z").unwrap(), "Afternoon"),
Event::new(Location::new(40.7, -74.0),
Timestamp::parse("2024-01-02T09:00:00Z").unwrap(), "Next day"),
];
let metrics = TemporalMetrics::from_events(&events);
Available Metrics
| Metric | Type | Description |
|---|---|---|
duration | Duration | Time span from first to last event |
event_count | usize | Number of events |
avg_interval | Duration | Average time between events |
min_interval | Duration | Shortest time between events |
max_interval | Duration | Longest time between events |
start | Timestamp | First event timestamp |
end | Timestamp | Last event timestamp |
Example Output
println!("Temporal metrics for {} events:", metrics.event_count);
println!(" Duration: {:?}", metrics.duration);
println!(" Average interval: {:?}", metrics.avg_interval);
println!(" Min interval: {:?}", metrics.min_interval);
println!(" Max interval: {:?}", metrics.max_interval);
println!(" Time range: {} to {}",
metrics.start.to_rfc3339(),
metrics.end.to_rfc3339());
Event Rate
Calculate the rate of events over time:
use spatial_narrative::analysis::event_rate;
use std::time::Duration;
// Events per hour
let hourly_rate = event_rate(&events, Duration::from_secs(3600));
println!("Events per hour: {:.2}", hourly_rate);
// Events per day
let daily_rate = event_rate(&events, Duration::from_secs(86400));
println!("Events per day: {:.2}", daily_rate);
Gap Detection
Find significant gaps in the event timeline:
use spatial_narrative::analysis::detect_gaps;
use std::time::Duration;
// Find gaps longer than 1 hour
let min_gap = Duration::from_secs(3600);
let gaps = detect_gaps(&events, min_gap);
for gap in gaps {
println!("Gap from {} to {} ({:?})",
gap.start.to_rfc3339(),
gap.end.to_rfc3339(),
gap.duration);
}
Gap Struct
Each detected gap contains:
| Field | Type | Description |
|---|---|---|
start | Timestamp | End of previous event |
end | Timestamp | Start of next event |
duration | Duration | Length of the gap |
Burst Detection
Identify periods of high event activity:
use spatial_narrative::analysis::detect_bursts;
use std::time::Duration;
// Find bursts with 5+ events in 1-hour windows
let window = Duration::from_secs(3600);
let min_events = 5;
let bursts = detect_bursts(&events, window, min_events);
for burst in bursts {
println!("Burst: {} events from {} to {}",
burst.count,
burst.start.to_rfc3339(),
burst.end.to_rfc3339());
}
Time Binning
Group events into time bins:
use spatial_narrative::analysis::TimeBin;
use std::time::Duration;
// Group by hour
let bins = TimeBin::from_events(&events, Duration::from_secs(3600));
for bin in bins {
println!("{}: {} events", bin.start.to_rfc3339(), bin.count);
}
Use Cases
Activity Pattern Analysis
let metrics = TemporalMetrics::from_events(&events);
// Calculate activity intensity
let events_per_hour = metrics.event_count as f64
/ metrics.duration.as_secs_f64() * 3600.0;
if events_per_hour > 10.0 {
println!("High activity: {:.1} events/hour", events_per_hour);
} else if events_per_hour > 1.0 {
println!("Moderate activity: {:.1} events/hour", events_per_hour);
} else {
println!("Low activity: {:.2} events/hour", events_per_hour);
}
Finding Quiet Periods
use std::time::Duration;
// Gaps longer than 6 hours
let quiet_periods = detect_gaps(&events, Duration::from_secs(6 * 3600));
println!("Found {} quiet periods (>6 hours):", quiet_periods.len());
for gap in quiet_periods {
println!(" {:?} gap starting {}",
gap.duration,
gap.start.to_rfc3339());
}
Rush Hour Detection
use std::time::Duration;
// Detect 30-minute windows with 10+ events
let rush_periods = detect_bursts(&events, Duration::from_secs(1800), 10);
if !rush_periods.is_empty() {
println!("Detected {} rush periods", rush_periods.len());
for rush in rush_periods {
println!(" {} events in 30min window at {}",
rush.count,
rush.start.to_rfc3339());
}
}
Movement Analysis
Analyze movement patterns and detect significant locations.
Trajectory
A Trajectory represents a path through space and time.
use spatial_narrative::analysis::Trajectory;
use spatial_narrative::core::{Event, Location, Timestamp};
let events = vec![
Event::new(Location::new(40.7128, -74.0060),
Timestamp::parse("2024-01-01T10:00:00Z").unwrap(), "Start"),
Event::new(Location::new(40.7480, -73.9850),
Timestamp::parse("2024-01-01T10:30:00Z").unwrap(), "Midtown"),
Event::new(Location::new(40.7580, -73.9855),
Timestamp::parse("2024-01-01T11:00:00Z").unwrap(), "Times Square"),
];
let trajectory = Trajectory::from_events(&events);
Trajectory Methods
| Method | Return Type | Description |
|---|---|---|
total_distance() | f64 | Total path length in meters |
duration() | Duration | Time from start to end |
average_speed() | f64 | Average speed in m/s |
points() | &[TrajectoryPoint] | All points on the path |
bounds() | GeoBounds | Bounding box of path |
Example Usage
println!("Trajectory analysis:");
println!(" Distance: {:.2} km", trajectory.total_distance() / 1000.0);
println!(" Duration: {:?}", trajectory.duration());
println!(" Avg speed: {:.1} km/h", trajectory.average_speed() * 3.6);
// Access individual segments
for (i, segment) in trajectory.segments().enumerate() {
println!(" Segment {}: {:.0}m in {:?}",
i, segment.distance, segment.duration);
}
Stop Detection
Identify locations where movement paused.
use spatial_narrative::analysis::{detect_stops, StopThreshold};
use std::time::Duration;
// Configure stop detection
let threshold = StopThreshold::new(
50.0, // Max radius in meters
Duration::from_secs(300), // Min duration (5 minutes)
);
let stops = detect_stops(&events, &threshold);
for stop in &stops {
println!("Stop at ({:.4}, {:.4})", stop.location.lat, stop.location.lon);
println!(" Duration: {:?}", stop.duration);
println!(" Events: {}", stop.events.len());
}
StopThreshold
Configure stop detection sensitivity:
| Field | Type | Description |
|---|---|---|
radius_meters | f64 | Maximum spread to consider a stop |
min_duration | Duration | Minimum time to qualify as a stop |
Stop Struct
Each detected stop contains:
| Field | Type | Description |
|---|---|---|
location | Location | Center of the stop |
start | Timestamp | When the stop began |
end | Timestamp | When the stop ended |
duration | Duration | How long the stop lasted |
events | Vec<&Event> | Events during the stop |
Movement Analyzer
For more advanced movement analysis:
use spatial_narrative::analysis::MovementAnalyzer;
let analyzer = MovementAnalyzer::new(&events);
// Get movement statistics
let stats = analyzer.statistics();
println!("Movement statistics:");
println!(" Total distance: {:.2} km", stats.total_distance / 1000.0);
println!(" Moving time: {:?}", stats.moving_time);
println!(" Stopped time: {:?}", stats.stopped_time);
println!(" Max speed: {:.1} km/h", stats.max_speed * 3.6);
// Detect mode changes
let segments = analyzer.segment_by_speed(5.0); // 5 m/s threshold
for segment in segments {
let mode = if segment.avg_speed > 5.0 { "vehicle" } else { "walking" };
println!(" {} segment: {:.0}m", mode, segment.distance);
}
Use Cases
GPS Track Analysis
// Analyze a GPS track
let trajectory = Trajectory::from_events(&gps_points);
let stops = detect_stops(&gps_points, &StopThreshold::new(25.0, Duration::from_secs(120)));
println!("Track summary:");
println!(" Total distance: {:.2} km", trajectory.total_distance() / 1000.0);
println!(" Duration: {:?}", trajectory.duration());
println!(" Stops: {}", stops.len());
// Calculate moving vs stopped time
let stopped_time: Duration = stops.iter().map(|s| s.duration).sum();
let moving_time = trajectory.duration() - stopped_time;
println!(" Moving time: {:?}", moving_time);
println!(" Stopped time: {:?}", stopped_time);
Delivery Route Analysis
// Identify delivery stops
let threshold = StopThreshold::new(
30.0, // 30m radius (parking area)
Duration::from_secs(60), // At least 1 minute
);
let delivery_stops = detect_stops(&route_events, &threshold);
println!("Delivery route analysis:");
println!(" Total stops: {}", delivery_stops.len());
println!(" Route distance: {:.1} km",
Trajectory::from_events(&route_events).total_distance() / 1000.0);
// Average time per stop
if !delivery_stops.is_empty() {
let avg_stop_time: Duration = delivery_stops.iter()
.map(|s| s.duration)
.sum::<Duration>() / delivery_stops.len() as u32;
println!(" Avg stop time: {:?}", avg_stop_time);
}
Anomaly Detection
let trajectory = Trajectory::from_events(&events);
// Find unusually fast segments (potential GPS errors)
for segment in trajectory.segments() {
let speed_kmh = segment.speed() * 3.6;
if speed_kmh > 200.0 {
println!("Warning: Unrealistic speed {:.0} km/h detected", speed_kmh);
println!(" From {} to {}",
segment.start.to_rfc3339(),
segment.end.to_rfc3339());
}
}
Clustering
Group events based on spatial proximity.
DBSCAN
Density-Based Spatial Clustering of Applications with Noise.
DBSCAN finds clusters of arbitrary shape and identifies outliers (noise).
use spatial_narrative::analysis::DBSCAN;
use spatial_narrative::core::{Event, Location, Timestamp};
let events = vec![
// Cluster 1: NYC area
Event::new(Location::new(40.7128, -74.0060), Timestamp::now(), "NYC 1"),
Event::new(Location::new(40.7138, -74.0050), Timestamp::now(), "NYC 2"),
Event::new(Location::new(40.7118, -74.0070), Timestamp::now(), "NYC 3"),
// Cluster 2: LA area
Event::new(Location::new(34.0522, -118.2437), Timestamp::now(), "LA 1"),
Event::new(Location::new(34.0532, -118.2427), Timestamp::now(), "LA 2"),
// Noise: isolated point
Event::new(Location::new(41.8781, -87.6298), Timestamp::now(), "Chicago"),
];
// Create DBSCAN with 1km radius and minimum 2 points per cluster
let dbscan = DBSCAN::new(1000.0, 2);
let result = dbscan.cluster(&events);
println!("Found {} clusters", result.num_clusters());
println!("Noise points: {}", result.noise.len());
DBSCAN Parameters
| Parameter | Description |
|---|---|
eps | Maximum distance (meters) between points in a cluster |
min_points | Minimum points required to form a cluster |
Choosing Parameters
// Dense urban data: small radius, more points
let urban = DBSCAN::new(500.0, 5); // 500m, 5+ points
// Sparse regional data: larger radius, fewer points
let regional = DBSCAN::new(10_000.0, 3); // 10km, 3+ points
// Very dense data (GPS tracks): tight clustering
let gps = DBSCAN::new(50.0, 10); // 50m, 10+ points
ClusteringResult
// Iterate over clusters
for (i, cluster) in result.clusters.iter().enumerate() {
println!("Cluster {}:", i);
println!(" Events: {}", cluster.events.len());
println!(" Center: ({:.4}, {:.4})",
cluster.centroid.0, cluster.centroid.1);
for event in &cluster.events {
println!(" - {}", event.text);
}
}
// Handle noise points
if !result.noise.is_empty() {
println!("Noise points (unclustered):");
for event in &result.noise {
println!(" - {} at ({:.4}, {:.4})",
event.text, event.location.lat, event.location.lon);
}
}
K-Means
Partition events into exactly K clusters.
use spatial_narrative::analysis::KMeans;
// Partition into 3 clusters
let kmeans = KMeans::new(3);
let result = kmeans.cluster(&events);
for (i, cluster) in result.clusters.iter().enumerate() {
println!("Cluster {} ({} events):", i, cluster.events.len());
println!(" Centroid: ({:.4}, {:.4})",
cluster.centroid.0, cluster.centroid.1);
}
K-Means Parameters
| Parameter | Description |
|---|---|
k | Number of clusters to create |
max_iterations | Maximum iterations (default: 100) |
Choosing K
// Try different values of K and evaluate
for k in 2..=5 {
let kmeans = KMeans::new(k);
let result = kmeans.cluster(&events);
// Calculate average cluster size
let avg_size = events.len() as f64 / k as f64;
println!("K={}: avg cluster size = {:.1}", k, avg_size);
}
Cluster Struct
Both algorithms return Cluster objects:
| Field | Type | Description |
|---|---|---|
events | Vec<&Event> | Events in the cluster |
centroid | (f64, f64) | Geographic center (lat, lon) |
bounds | GeoBounds | Bounding box |
When to Use Which
| Algorithm | Best For |
|---|---|
| DBSCAN | Unknown number of clusters, irregular shapes, noise handling |
| K-Means | Known number of clusters, roughly equal-sized groups |
Use Cases
Finding Activity Hotspots
let dbscan = DBSCAN::new(1000.0, 5); // 1km, 5+ events
let result = dbscan.cluster(&events);
// Sort clusters by size
let mut clusters: Vec<_> = result.clusters.iter().collect();
clusters.sort_by(|a, b| b.events.len().cmp(&a.events.len()));
println!("Top activity hotspots:");
for (i, cluster) in clusters.iter().take(5).enumerate() {
println!(" {}. {} events at ({:.4}, {:.4})",
i + 1, cluster.events.len(),
cluster.centroid.0, cluster.centroid.1);
}
Regional Grouping
// Group events into regions for separate analysis
let kmeans = KMeans::new(4); // 4 regions
let result = kmeans.cluster(&events);
for (i, cluster) in result.clusters.iter().enumerate() {
let region_name = match i {
0 => "Northeast",
1 => "Southeast",
2 => "Midwest",
3 => "West",
_ => "Unknown",
};
println!("{} region: {} events", region_name, cluster.events.len());
}
Outlier Detection
let dbscan = DBSCAN::new(5000.0, 2);
let result = dbscan.cluster(&events);
// Noise points are potential outliers or unique events
if !result.noise.is_empty() {
println!("Potential outliers ({}):", result.noise.len());
for event in &result.noise {
println!(" - {} at ({:.4}, {:.4})",
event.text,
event.location.lat,
event.location.lon);
}
}
Narrative Comparison
Compare narratives to find similarities and differences.
Basic Comparison
use spatial_narrative::analysis::{compare_narratives, ComparisonConfig};
use spatial_narrative::core::Narrative;
let similarity = compare_narratives(&narrative1, &narrative2, &ComparisonConfig::default());
println!("Similarity scores:");
println!(" Spatial: {:.2}", similarity.spatial);
println!(" Temporal: {:.2}", similarity.temporal);
println!(" Thematic: {:.2}", similarity.thematic);
println!(" Overall: {:.2}", similarity.overall);
NarrativeSimilarity
The comparison returns a NarrativeSimilarity struct:
| Field | Type | Description |
|---|---|---|
spatial | f64 | Geographic overlap (0.0 to 1.0) |
temporal | f64 | Time overlap (0.0 to 1.0) |
thematic | f64 | Tag/topic similarity (0.0 to 1.0) |
overall | f64 | Weighted average score |
Scores range from 0.0 (no similarity) to 1.0 (identical).
ComparisonConfig
Customize how similarity is calculated:
use spatial_narrative::analysis::ComparisonConfig;
let config = ComparisonConfig {
spatial_weight: 0.4, // 40% weight on geography
temporal_weight: 0.3, // 30% weight on time
thematic_weight: 0.3, // 30% weight on themes
spatial_threshold_km: 10.0, // Events within 10km considered "same place"
temporal_threshold_hours: 24.0, // Events within 24h considered "same time"
};
let similarity = compare_narratives(&n1, &n2, &config);
Spatial Comparison Functions
Spatial Similarity
Calculate geographic overlap:
use spatial_narrative::analysis::spatial_similarity;
let score = spatial_similarity(&narrative1, &narrative2);
println!("Spatial similarity: {:.2}", score); // 0.0 to 1.0
Spatial Intersection
Find events that occur in similar locations:
use spatial_narrative::analysis::spatial_intersection;
// Events within 5km of each other
let matching_pairs = spatial_intersection(&narrative1, &narrative2, 5000.0);
for (e1, e2) in matching_pairs {
println!("Match: '{}' and '{}'", e1.text, e2.text);
println!(" Distance: {:.0}m apart",
e1.location.distance_to(&e2.location));
}
Spatial Union
Get combined geographic bounds:
use spatial_narrative::analysis::spatial_union;
let combined_bounds = spatial_union(&narrative1, &narrative2);
println!("Combined area: {:.2}° x {:.2}°",
combined_bounds.lat_span(),
combined_bounds.lon_span());
Temporal Comparison
Temporal Similarity
use spatial_narrative::analysis::temporal_similarity;
let score = temporal_similarity(&narrative1, &narrative2);
println!("Temporal similarity: {:.2}", score);
Thematic Comparison
Thematic Similarity
Compare based on shared tags:
use spatial_narrative::analysis::thematic_similarity;
let score = thematic_similarity(&narrative1, &narrative2);
println!("Thematic similarity: {:.2}", score);
Common Locations
Find locations that appear in both narratives:
use spatial_narrative::analysis::common_locations;
// Locations within 1km considered the same
let shared = common_locations(&narrative1, &narrative2, 1000.0);
println!("Common locations ({}):", shared.len());
for loc in shared {
println!(" ({:.4}, {:.4})", loc.lat, loc.lon);
}
Use Cases
Finding Related Stories
let threshold = 0.5; // 50% similarity threshold
let mut related = Vec::new();
for candidate in &all_narratives {
let sim = compare_narratives(&target, candidate, &ComparisonConfig::default());
if sim.overall > threshold {
related.push((candidate, sim.overall));
}
}
// Sort by similarity
related.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap());
println!("Related narratives:");
for (narrative, score) in related.iter().take(5) {
println!(" {}: {:.0}% similar",
narrative.title.as_deref().unwrap_or("Untitled"),
score * 100.0);
}
Detecting Duplicate Reports
let config = ComparisonConfig {
spatial_threshold_km: 0.1, // 100m
temporal_threshold_hours: 1.0, // 1 hour
..Default::default()
};
for i in 0..narratives.len() {
for j in (i + 1)..narratives.len() {
let sim = compare_narratives(&narratives[i], &narratives[j], &config);
if sim.overall > 0.9 {
println!("Potential duplicate:");
println!(" '{}' and '{}'",
narratives[i].title.as_deref().unwrap_or("?"),
narratives[j].title.as_deref().unwrap_or("?"));
println!(" Similarity: {:.0}%", sim.overall * 100.0);
}
}
}
Clustering Narratives by Topic
// Group narratives by thematic similarity
let mut groups: Vec<Vec<&Narrative>> = Vec::new();
for narrative in &narratives {
let mut found_group = false;
for group in &mut groups {
// Check if similar to group representative
let sim = thematic_similarity(narrative, group[0]);
if sim > 0.6 {
group.push(narrative);
found_group = true;
break;
}
}
if !found_group {
groups.push(vec![narrative]);
}
}
println!("Found {} thematic groups", groups.len());
for (i, group) in groups.iter().enumerate() {
println!(" Group {}: {} narratives", i + 1, group.len());
}
Text Processing Overview
The spatial_narrative library provides comprehensive text processing capabilities for extracting geographic information from unstructured text.
Modules
Text processing is split across two modules:
text- Named Entity Recognition (NER), ML-NER, and keyword extractionparser- Geoparsing and coordinate detection
Key Features
Geoparsing (parser module)
Extract locations from text using multiple strategies:
- Coordinate Detection: Decimal degrees, degrees with symbols, DMS format
- Place Name Resolution: Built-in gazetteer with 2500+ world cities from GeoNames
- Custom Gazetteers: Plug in your own place name databases or external APIs
use spatial_narrative::parser::{GeoParser, BuiltinGazetteer};
let gazetteer = BuiltinGazetteer::new();
let parser = GeoParser::with_gazetteer(Box::new(gazetteer));
let text = "The conference in Paris started at 48.8566°N, 2.3522°E.";
let mentions = parser.extract(text);
for mention in mentions {
println!("Found: {} ({:?})", mention.text, mention.mention_type);
if let Some(loc) = mention.location {
println!(" -> {}, {}", loc.lat, loc.lon);
}
}
Named Entity Recognition (text module)
Extract entities from narrative text using rule-based patterns:
use spatial_narrative::text::TextAnalyzer;
let analyzer = TextAnalyzer::new();
let text = "Dr. Smith visited Google headquarters in Mountain View on January 15, 2024.";
let entities = analyzer.entities(text);
for entity in entities {
println!("{}: {} (confidence: {:.2})", entity.entity_type, entity.text, entity.confidence);
}
Keyword Extraction
Identify key terms and phrases:
use spatial_narrative::text::KeywordExtractor;
let extractor = KeywordExtractor::new();
let text = "Climate change affects coastal cities. Rising sea levels threaten coastal communities.";
let keywords = extractor.extract(text, 5);
for kw in keywords {
println!("{}: {:.3}", kw.word, kw.score);
}
ML-NER (Advanced, requires ml-ner feature)
Use transformer-based models for high-accuracy entity extraction:
use spatial_narrative::text::{MlNerModel, NerModel};
// Auto-download and cache model (~65MB)
let model = MlNerModel::download_blocking(NerModel::DistilBertQuantized)?;
let text = "Dr. Chen presented her findings in Paris on March 15, 2024.";
let entities = model.extract(text)?;
for entity in entities {
println!("{}: \"{}\" (confidence: {:.2})", entity.label, entity.text, entity.score);
}
// Output:
// PER: "Dr. Chen" (confidence: 0.99)
// LOC: "Paris" (confidence: 0.98)
// MISC: "March 15, 2024" (confidence: 0.95)
When to Use Each Module
| Task | Module | Key Type |
|---|---|---|
| Extract coordinates from text | parser | GeoParser |
| Resolve place names to coordinates | parser | BuiltinGazetteer |
| Extract entities (rule-based) | text | TextAnalyzer |
| Extract entities (ML, high accuracy) | text | MlNerModel |
| Find important keywords | text | KeywordExtractor |
Next Steps
- Geoparsing - Coordinate detection and place name resolution
- Named Entity Recognition - Rule-based entity extraction
- ML-NER (Advanced) - Machine learning-powered NER with auto-download
- Keyword Extraction - Identify key terms
Geoparsing
The GeoParser extracts geographic locations from text, detecting both coordinates and place names.
Basic Usage
use spatial_narrative::parser::GeoParser;
let parser = GeoParser::new();
let mentions = parser.extract("Meeting at 40.7128, -74.0060");
assert_eq!(mentions.len(), 1);
let loc = mentions[0].location.as_ref().unwrap();
println!("Latitude: {}, Longitude: {}", loc.lat, loc.lon);
Coordinate Formats
The parser detects three coordinate formats:
Decimal Degrees
40.7128, -74.0060
51.5074 -0.1278
-33.8688, 151.2093
Degrees with Symbols
40.7128°N, 74.0060°W
48.8566N 2.3522E
33.8688°S, 151.2093°E
DMS (Degrees, Minutes, Seconds)
40°42'46"N, 74°0'22"W
51°30'26"N, 0°7'40"W
Place Name Resolution
Use a gazetteer to resolve place names to coordinates:
use spatial_narrative::parser::{GeoParser, BuiltinGazetteer, MentionType};
let gazetteer = BuiltinGazetteer::new();
let parser = GeoParser::with_gazetteer(Box::new(gazetteer));
let text = "The conference was held in Tokyo and participants came from London.";
let mentions = parser.extract(text);
for mention in &mentions {
if matches!(mention.mention_type, MentionType::PlaceName) {
println!("Place: {}", mention.text);
}
}
Built-in Gazetteer
The BuiltinGazetteer includes 2500+ world cities:
- Major world cities selected by population from GeoNames
- Cities across 150+ countries with precise coordinates
- Common aliases (NYC → New York City, SHA → Shanghai, etc.)
- Data licensed under CC BY 4.0
use spatial_narrative::parser::{BuiltinGazetteer, Gazetteer};
let gazetteer = BuiltinGazetteer::new();
// Direct lookup
if let Some(loc) = gazetteer.lookup("Paris") {
println!("Paris: {}, {}", loc.lat, loc.lon);
}
// Aliases work too
assert!(gazetteer.contains("NYC"));
assert!(gazetteer.contains("New York City"));
Custom Gazetteer
Implement the Gazetteer trait for custom place databases:
use spatial_narrative::parser::Gazetteer;
use spatial_narrative::core::Location;
use std::collections::HashMap;
struct MyGazetteer {
places: HashMap<String, (f64, f64)>,
}
impl Gazetteer for MyGazetteer {
fn lookup(&self, name: &str) -> Option<Location> {
self.places.get(&name.to_lowercase())
.map(|(lat, lon)| Location::new(*lat, *lon))
}
fn contains(&self, name: &str) -> bool {
self.places.contains_key(&name.to_lowercase())
}
fn all_names(&self) -> Vec<&str> {
self.places.keys().map(|s| s.as_str()).collect()
}
}
Configuration
Use LocationPattern to control what the parser detects:
use spatial_narrative::parser::{GeoParser, LocationPattern, BuiltinGazetteer};
// Only detect coordinates, not place names
let mut parser = GeoParser::new();
parser.set_pattern(LocationPattern::coordinates_only());
// Or customize fully
let pattern = LocationPattern {
detect_decimal: true,
detect_symbols: true,
detect_dms: false,
detect_places: false,
min_confidence: 0.8,
};
parser.set_pattern(pattern);
LocationMention
Each detected location is returned as a LocationMention:
use spatial_narrative::parser::{GeoParser, MentionType};
let parser = GeoParser::new();
let mentions = parser.extract("Coordinates: 40.7128, -74.0060");
for mention in mentions {
println!("Text: '{}'", mention.text);
println!("Position: {}-{}", mention.start, mention.end);
println!("Type: {:?}", mention.mention_type);
println!("Confidence: {:.2}", mention.confidence);
if let Some(loc) = mention.location {
println!("Location: {}, {}", loc.lat, loc.lon);
}
}
MentionType
DecimalDegrees- "40.7128, -74.0060"DegreesWithSymbols- "40.7128°N, 74.0060°W"DMS- "40°42'46"N, 74°0'22"W"PlaceName- "Paris", "New York City"
Confidence Scores
Each mention has a confidence score (0.0 to 1.0):
| Type | Typical Confidence |
|---|---|
| DMS | 0.99 |
| DegreesWithSymbols | 0.98 |
| DecimalDegrees | 0.95 |
| PlaceName | 0.85 |
Coordinate formats have higher confidence because they're unambiguous. Place names are lower due to potential ambiguity (e.g., "Paris" could be Paris, France or Paris, Texas).
Named Entity Recognition
The TextAnalyzer provides basic Named Entity Recognition (NER) for extracting structured information from narrative text.
Basic Usage
use spatial_narrative::text::TextAnalyzer;
let analyzer = TextAnalyzer::new();
let text = "Dr. Smith met with CEO Johnson at Google headquarters.";
let entities = analyzer.entities(text);
for entity in entities {
println!("{:?}: {}", entity.entity_type, entity.text);
}
Entity Types
The analyzer detects six entity types:
| Type | Description | Examples |
|---|---|---|
Person | People's names | "Dr. Smith", "Mr. Johnson" |
Organization | Companies, institutions | "Google Inc.", "MIT", "NASA" |
Location | Place names | "New York", "Mount Everest" |
DateTime | Dates and times | "January 15, 2024", "March 2023" |
Numeric | Numbers with units | "$5.5 million", "100 km" |
Event | Named events | (custom additions) |
Other | Unclassified entities | - |
Person Detection
Detects names with common titles:
use spatial_narrative::text::{TextAnalyzer, EntityType};
let analyzer = TextAnalyzer::new();
let text = "Dr. Jane Smith and Prof. Bob Johnson attended the meeting.";
let entities = analyzer.entities(text);
let people: Vec<_> = entities.iter()
.filter(|e| matches!(e.entity_type, EntityType::Person))
.collect();
for person in people {
println!("Person: {}", person.text);
}
Recognized titles: Dr., Mr., Mrs., Ms., Miss, Prof., Professor, President, Senator, Governor, Mayor, Chief, Captain, General, Admiral, etc.
Organization Detection
Detects organizations by suffix patterns:
use spatial_narrative::text::{TextAnalyzer, EntityType};
let analyzer = TextAnalyzer::new();
let text = "Apple Inc. partnered with MIT and the World Health Organization.";
let entities = analyzer.entities(text);
let orgs: Vec<_> = entities.iter()
.filter(|e| matches!(e.entity_type, EntityType::Organization))
.collect();
// Found: "Apple Inc.", "MIT", "World Health Organization"
Recognized patterns:
- Suffixes: Inc., Corp., LLC, Ltd., Co., Foundation, Institute, University, Organization
- Acronyms: NASA, FBI, CIA, NATO, WHO, UN, EU, etc.
Date Detection
Extracts various date formats:
use spatial_narrative::text::{TextAnalyzer, EntityType};
let analyzer = TextAnalyzer::new();
let text = "The event on January 15, 2024 was rescheduled to March 2024.";
let entities = analyzer.entities(text);
let dates: Vec<_> = entities.iter()
.filter(|e| matches!(e.entity_type, EntityType::DateTime))
.collect();
Recognized formats:
- Full dates: "January 15, 2024", "15 January 2024"
- Month-year: "March 2024", "Jan 2024"
- Abbreviated months: "Jan", "Feb", "Mar", etc.
Location Detection
Detects common location patterns and major places:
use spatial_narrative::text::{TextAnalyzer, EntityType};
let analyzer = TextAnalyzer::new();
let text = "The company has offices in New York, London, and Tokyo.";
let entities = analyzer.entities(text);
let locations: Vec<_> = entities.iter()
.filter(|e| matches!(e.entity_type, EntityType::Location))
.collect();
Built-in location database includes major world cities and countries.
Numeric Detection
Extracts numbers with units:
use spatial_narrative::text::{TextAnalyzer, EntityType};
let analyzer = TextAnalyzer::new();
let text = "The project cost $5.5 million and covered 100 kilometers.";
let entities = analyzer.entities(text);
let numerics: Vec<_> = entities.iter()
.filter(|e| matches!(e.entity_type, EntityType::Numeric))
.collect();
// Found: "$5.5 million", "100 kilometers"
Recognized patterns:
- Currency: "$5.5 million", "€100"
- Distance: "100 km", "50 miles"
- Percentages: "25%", "75 percent"
- General: "1,000", "3.14"
Tokenization
The analyzer also provides text tokenization:
use spatial_narrative::text::TextAnalyzer;
let analyzer = TextAnalyzer::new();
let text = "Hello, world! This is a test.";
// All tokens
let tokens = analyzer.tokenize(text);
// ["Hello", ",", "world", "!", "This", "is", "a", "test", "."]
// Words only (no punctuation)
let words = analyzer.tokenize_words(text);
// ["Hello", "world", "This", "is", "a", "test"]
Sentence Splitting
Split text into sentences:
use spatial_narrative::text::TextAnalyzer;
let analyzer = TextAnalyzer::new();
let text = "First sentence. Second sentence! Third sentence?";
let sentences = analyzer.sentences(text);
assert_eq!(sentences.len(), 3);
Custom Locations
Add custom location names:
use spatial_narrative::text::TextAnalyzer;
let mut analyzer = TextAnalyzer::new();
analyzer.add_location("Springfield");
analyzer.add_location("Gotham City");
let text = "The hero saved Gotham City.";
let entities = analyzer.entities(text);
// Now detects "Gotham City" as a location
Confidence Scores
Each entity has a confidence score:
use spatial_narrative::text::TextAnalyzer;
let analyzer = TextAnalyzer::new();
let entities = analyzer.entities("Dr. Smith works at NASA.");
for entity in entities {
println!("{}: {} (confidence: {:.2})",
entity.entity_type,
entity.text,
entity.confidence
);
}
Confidence levels:
- 0.9+: High confidence (clear patterns like "Dr. Smith")
- 0.7-0.9: Medium confidence
- 0.5-0.7: Lower confidence (may need verification)
Limitations
This is a rule-based NER system, not a machine learning model:
- ✅ Fast and deterministic
- ✅ No external dependencies
- ✅ Works offline
- ❌ May miss unconventional patterns
- ❌ Limited to English
- ❌ No context-aware disambiguation
For production NLP tasks requiring high accuracy, consider integrating with external NLP services or ML models.
Machine Learning Named Entity Recognition
The ml-ner feature provides state-of-the-art Named Entity Recognition using transformer models (BERT, RoBERTa, DistilBERT) via ONNX Runtime.
Features
- High Accuracy: Transformer-based models trained on CoNLL-2003
- Auto-Download: Automatically fetch models from HuggingFace Hub
- Caching: Models are cached locally after first download
- Multiple Models: Choose from 5 pre-trained models or bring your own
- Multi-language: Support for 40+ languages with the multilingual model
Entity Types
All pre-trained models recognize four entity types:
- LOC - Locations (cities, countries, regions)
- PER - Persons (names of people)
- ORG - Organizations (companies, institutions)
- MISC - Miscellaneous (dates, events, products)
Installation
Add the ml-ner-download feature to enable both ML-NER and auto-download:
[dependencies]
spatial-narrative = { version = "0.1", features = ["ml-ner-download"] }
Or use just ml-ner if you want to provide your own models:
[dependencies]
spatial-narrative = { version = "0.1", features = ["ml-ner"] }
ONNX Runtime Setup
ML-NER requires ONNX Runtime to be installed. You have several options:
Option 1: Environment Variable
Set ORT_DYLIB_PATH to point to your ONNX Runtime library:
# macOS
export ORT_DYLIB_PATH=/path/to/libonnxruntime.dylib
# Linux
export ORT_DYLIB_PATH=/path/to/libonnxruntime.so
# Windows
set ORT_DYLIB_PATH=C:\path\to\onnxruntime.dll
Option 2: Install via Package Manager
macOS (Homebrew):
brew install onnxruntime
export ORT_DYLIB_PATH=$(brew --prefix onnxruntime)/lib/libonnxruntime.dylib
Linux (Ubuntu/Debian):
sudo apt install libonnxruntime
export ORT_DYLIB_PATH=/usr/lib/libonnxruntime.so
Option 3: Manual Download
Download from ONNX Runtime releases:
- Download the appropriate package for your platform
- Extract the archive
- Set
ORT_DYLIB_PATHto the library file
Available Models
| Model | Size | F1 Score | Speed | Languages |
|---|---|---|---|---|
| DistilBertQuantized | ~65MB | ~90% | Fast | English |
| DistilBert | ~250MB | ~90% | Fast | English |
| BertBase | ~400MB | ~91% | Medium | English |
| BertLarge | ~1.2GB | ~93% | Slow | English |
| Multilingual | ~700MB | ~90% | Medium | 40+ languages |
The DistilBertQuantized model is recommended for most use cases, offering the best balance of size, speed, and accuracy.
Basic Usage
Auto-Download (Recommended)
The simplest way to get started:
use spatial_narrative::text::{MlNerModel, NerModel};
// First run downloads ~65MB, subsequent runs load from cache
let model = MlNerModel::download_blocking(NerModel::DistilBertQuantized)?;
let text = "Dr. Sarah Chen presented her research in Paris on March 15, 2024.";
let entities = model.extract(text)?;
for entity in entities {
println!("{}: \"{}\" (confidence: {:.2})",
entity.label, entity.text, entity.score);
}
// Output:
// PER: "Dr. Sarah Chen" (confidence: 0.99)
// LOC: "Paris" (confidence: 0.98)
// MISC: "March 15, 2024" (confidence: 0.95)
With Progress Reporting
Show download progress for large models:
use spatial_narrative::text::{MlNerModel, NerModel};
let model = MlNerModel::download_blocking_with_progress(
NerModel::DistilBertQuantized,
|downloaded, total| {
if total > 0 {
let pct = (downloaded as f64 / total as f64) * 100.0;
println!("Downloading: {:.1}%", pct);
}
}
)?;
Using Different Models
use spatial_narrative::text::{MlNerModel, NerModel};
// For best accuracy (larger download)
let model = MlNerModel::download_blocking(NerModel::BertLarge)?;
// For multilingual text
let model = MlNerModel::download_blocking(NerModel::Multilingual)?;
// For custom HuggingFace models
let model = MlNerModel::download_blocking(
NerModel::Custom("my-org/my-ner-model".into())
)?;
Advanced Usage
Manual Model Loading
If you have pre-downloaded ONNX models:
use spatial_narrative::text::MlNerModel;
let model = MlNerModel::from_directory("./my-ner-model/")?;
// Directory should contain: model.onnx, tokenizer.json, config.json
Cache Management
use spatial_narrative::text::{
model_cache_dir,
model_cache_path,
is_model_cached,
cache_size_bytes,
clear_model_cache,
NerModel,
};
// Check cache location
println!("Cache dir: {:?}", model_cache_dir());
// Check if a model is cached
let model = NerModel::DistilBertQuantized;
if is_model_cached(&model) {
println!("Model already cached at: {:?}", model_cache_path(&model));
}
// Get total cache size
if let Ok(size) = cache_size_bytes() {
println!("Cache size: {:.2} MB", size as f64 / 1024.0 / 1024.0);
}
// Clear cache for a specific model
clear_model_cache(Some(&model))?;
// Clear all cached models
clear_model_cache(None)?;
Async API
For async applications, use the async API:
use spatial_narrative::text::{MlNerModel, NerModel};
let model = MlNerModel::download(NerModel::DistilBertQuantized).await?;
let entities = model.extract("Text to analyze")?;
Integration with Geoparsing
Combine ML-NER with gazetteer lookup for comprehensive geoparsing:
use spatial_narrative::text::{MlNerModel, NerModel};
use spatial_narrative::parser::{BuiltinGazetteer, Gazetteer};
// Extract entities with ML
let ml_model = MlNerModel::download_blocking(NerModel::DistilBertQuantized)?;
let text = "The summit was held in Geneva, Switzerland.";
let ml_entities = ml_model.extract(text)?;
// Resolve locations with gazetteer
let gazetteer = BuiltinGazetteer::new();
for ml_entity in ml_entities {
if ml_entity.label == "LOC" {
// Convert to standard Entity and lookup coordinates
let entity = ml_entity.to_entity();
if let Some(location) = gazetteer.lookup(&entity.text) {
println!("{} is at {}, {}",
entity.text, location.lat, location.lon);
}
}
}
Entity Structure
The MlEntity struct provides detailed extraction results:
pub struct MlEntity {
/// Entity type: "LOC", "PER", "ORG", or "MISC"
pub label: String,
/// The extracted text
pub text: String,
/// Confidence score (0.0 to 1.0)
pub score: f64,
/// Character position in original text
pub start: usize,
/// End position in original text
pub end: usize,
}
Convert to standard Entity for use with other components:
let entity = ml_entity.to_entity();
// Returns Entity with appropriate EntityType enum
Example Application
See the complete example:
cargo run --example ml_ner_download --features ml-ner-download
This example demonstrates:
- Checking cache status
- Auto-downloading models
- Extracting entities from various texts
- Integration with geoparsing workflow
Exporting Custom Models
To use your own fine-tuned models:
- Train or fine-tune a token classification model on HuggingFace
- Export to ONNX using Optimum:
pip install optimum[exporters]
optimum-cli export onnx --model your-model-name ./output-dir/
- Load in spatial-narrative:
let model = MlNerModel::from_directory("./output-dir/")?;
Or host on HuggingFace Hub and use:
let model = MlNerModel::download_blocking(
NerModel::Custom("your-org/your-model".into())
)?;
Performance Tips
- Choose the right model: Use DistilBertQuantized for most applications
- Cache models: First download takes time, but subsequent loads are fast
- Batch processing: Process multiple texts in sequence after loading once
- Model lifecycle: Keep the model in memory for repeated extractions
- Async for I/O: Use async API when downloading in web servers
Troubleshooting
ONNX Runtime Not Found
If you see errors about ONNX Runtime:
- Install ONNX Runtime (see setup section above)
- Set
ORT_DYLIB_PATHenvironment variable - Verify the path points to the correct library file
Model Download Fails
- Check internet connection
- Verify HuggingFace Hub is accessible
- Try clearing cache:
clear_model_cache(None)? - Check cache directory permissions
Low Accuracy
- Try a larger model (BertBase or BertLarge)
- For non-English text, use the Multilingual model
- Consider fine-tuning a custom model on your domain
Cache Locations
Models are cached in platform-specific directories:
- Linux:
~/.cache/spatial-narrative/models/ - macOS:
~/Library/Caches/spatial-narrative/models/ - Windows:
%LOCALAPPDATA%\spatial-narrative\models\
Each model has its own subdirectory containing:
model.onnx- The neural network modeltokenizer.json- Text tokenization configurationconfig.json- Label mappings and metadata
License Notes
- DistilBERT models: Apache 2.0 License
- BERT models: Apache 2.0 License
- Multilingual model: CC BY-NC-SA 4.0 License (non-commercial)
- ONNX Runtime: MIT License
Check individual model licenses on HuggingFace Hub before commercial use.
Keyword Extraction
The KeywordExtractor identifies important terms and phrases from text using term frequency analysis.
Basic Usage
use spatial_narrative::text::KeywordExtractor;
let extractor = KeywordExtractor::new();
let text = "Climate change affects coastal cities. Rising sea levels threaten coastal communities worldwide.";
let keywords = extractor.extract(text, 5);
for kw in keywords {
println!("{}: {:.3}", kw.text, kw.score);
}
How It Works
The extractor uses a TF (Term Frequency) based approach:
- Tokenization: Text is split into words
- Normalization: Words are lowercased
- Filtering: Stop words and short words are removed
- Scoring: Words are scored by frequency
- Ranking: Top N keywords are returned
Keyword Struct
Each extracted keyword contains:
use spatial_narrative::text::{KeywordExtractor, Keyword};
let extractor = KeywordExtractor::new();
let keywords = extractor.extract("Machine learning is transforming technology.", 3);
for kw in keywords {
println!("Word: {}", kw.text);
println!("Score: {:.3}", kw.score);
println!("Frequency: {}", kw.frequency);
}
| Field | Type | Description |
|---|---|---|
text | String | The keyword text |
score | f64 | Normalized score (0.0 to 1.0) |
frequency | usize | Raw occurrence count |
Configuration
Minimum Word Length
Skip short words:
use spatial_narrative::text::KeywordExtractor;
let extractor = KeywordExtractor::new().min_length(4); // Skip words shorter than 4 chars
let text = "The big cat sat on the mat.";
let keywords = extractor.extract(text, 10);
// Only extracts words with 4+ characters
Custom Stop Words
Add domain-specific stop words:
use spatial_narrative::text::KeywordExtractor;
let extractor = KeywordExtractor::new();
let text = "Data analysis reveals important data patterns in analysis.";
let keywords = extractor.extract_with_stopwords(text, 5, &["data", "analysis"]);
// "data" and "analysis" are now filtered out
Phrase Extraction
Extract multi-word phrases (n-grams):
use spatial_narrative::text::KeywordExtractor;
// Configure max phrase length (includes bigrams and trigrams)
let extractor = KeywordExtractor::new().max_phrase_length(2);
let text = "Machine learning and deep learning are subfields of artificial intelligence.";
let phrases = extractor.extract(text, 3); // Top 3 terms (including bigrams)
for phrase in phrases {
println!("{}: {:.3}", phrase.text, phrase.score);
}
// Example: "machine learning", "deep learning", "artificial intelligence"
The max_phrase_length parameter controls n-gram extraction (2 for bigrams, 3 for trigrams).
Built-in Stop Words
The default stop word list includes common English words:
the, a, an, is, are, was, were, be, been, being,
have, has, had, do, does, did, will, would, could,
should, may, might, must, shall, can, need, dare,
and, or, but, if, then, else, when, where, why,
how, all, each, every, both, few, more, most, other,
some, such, no, not, only, same, so, than, too, very,
just, also, now, here, there, this, that, these, those,
i, you, he, she, it, we, they, me, him, her, us, them,
my, your, his, its, our, their, mine, yours, hers, ours,
what, which, who, whom, whose, of, in, to, for, with,
on, at, by, from, as, into, through, during, before,
after, above, below, between, under, again, further,
once, about, up, down, out, off, over, own, because
Use Cases
Document Summarization
Extract key themes from a document:
use spatial_narrative::text::KeywordExtractor;
let extractor = KeywordExtractor::new();
let document = "Long document text...";
let themes = extractor.extract(document, 10);
println!("Key themes:");
for theme in themes {
println!(" - {}", theme.text);
}
Narrative Tagging
Auto-generate tags for narratives:
use spatial_narrative::text::KeywordExtractor;
use spatial_narrative::core::Narrative;
let extractor = KeywordExtractor::new();
fn auto_tag(narrative: &Narrative, extractor: &KeywordExtractor) -> Vec<String> {
// Combine all event descriptions
let text: String = narrative.events()
.filter_map(|e| e.description.clone())
.collect::<Vec<_>>()
.join(" ");
// Extract top keywords as tags
extractor.extract(&text, 5)
.into_iter()
.map(|kw| kw.text)
.collect()
}
Topic Comparison
Compare topics between documents:
use spatial_narrative::text::KeywordExtractor;
use std::collections::HashSet;
let extractor = KeywordExtractor::new();
let doc1_keywords: HashSet<_> = extractor.extract(doc1, 20)
.into_iter()
.map(|kw| kw.word)
.collect();
let doc2_keywords: HashSet<_> = extractor.extract(doc2, 20)
.into_iter()
.map(|kw| kw.word)
.collect();
let common: HashSet<_> = doc1_keywords.intersection(&doc2_keywords).collect();
println!("Common topics: {:?}", common);
Performance
The extractor is optimized for speed:
- O(n) tokenization
- O(n log n) sorting for top-k selection
- Minimal memory allocation
Typical performance: ~10,000 words/ms on modern hardware.
Limitations
- English-focused (stop word list is English)
- No stemming or lemmatization
- TF-only (no IDF weighting)
- Case-insensitive matching only
For advanced keyword extraction with TF-IDF or semantic analysis, consider using specialized NLP libraries.
Common Patterns
Practical patterns for working with spatial narratives.
Creating Events
From Raw Data
use spatial_narrative::core::{Event, EventBuilder, Location, Timestamp};
// Simple event
let event = Event::new(
Location::new(40.7128, -74.0060),
Timestamp::parse("2024-01-15T10:00:00Z").unwrap(),
"Event description"
);
// Rich event with builder
let event = EventBuilder::new()
.location(Location::builder()
.lat(40.7128)
.lon(-74.0060)
.name("New York City")
.build()
.unwrap())
.timestamp(Timestamp::parse("2024-01-15T10:00:00Z").unwrap())
.text("Conference begins")
.tag("conference")
.tag("technology")
.build();
From Database Records
fn event_from_row(row: &Row) -> Result<Event> {
let event = EventBuilder::new()
.location(Location::new(
row.get::<f64>("latitude")?,
row.get::<f64>("longitude")?
))
.timestamp(Timestamp::parse(&row.get::<String>("datetime")?)?)
.text(row.get::<String>("description")?)
.build();
Ok(event)
}
From CSV Line
fn event_from_csv(line: &str) -> Result<Event> {
let parts: Vec<&str> = line.split(',').collect();
let event = Event::new(
Location::new(parts[0].parse()?, parts[1].parse()?),
Timestamp::parse(parts[2])?,
parts[3]
);
Ok(event)
}
Building Narratives
From Event Collection
use spatial_narrative::core::{Narrative, NarrativeBuilder};
let narrative = NarrativeBuilder::new()
.title("My Narrative")
.description("A collection of events")
.author("Research Team")
.events(events)
.tag("research")
.build();
Incremental Building
let mut builder = NarrativeBuilder::new()
.title("Growing Narrative");
for data in data_stream {
if let Ok(event) = process_data(data) {
builder = builder.event(event);
}
}
let narrative = builder.build();
Filtering Patterns
By Location
use spatial_narrative::core::GeoBounds;
let nyc_bounds = GeoBounds::new(40.4, -74.3, 41.0, -73.7);
let nyc_events = narrative.filter_spatial(&nyc_bounds);
By Time
use spatial_narrative::core::TimeRange;
let january = TimeRange::month(2024, 1);
let january_events = narrative.filter_temporal(&january);
By Tags
let important: Vec<_> = narrative.events.iter()
.filter(|e| e.has_tag("important"))
.collect();
Combined Filters
let filtered: Vec<_> = narrative.events.iter()
.filter(|e| nyc_bounds.contains(&e.location))
.filter(|e| january.contains(&e.timestamp))
.filter(|e| e.has_tag("verified"))
.collect();
Index Usage Patterns
Build Once, Query Many
use spatial_narrative::index::SpatiotemporalIndex;
// Build index once
let index = SpatiotemporalIndex::from_iter(
events.iter().cloned(),
|e| &e.location,
|e| &e.timestamp
);
// Query multiple times
let q1 = index.query(&bounds1, &range1);
let q2 = index.query(&bounds2, &range2);
let q3 = index.query_spatial(&bounds3);
Nearest Neighbor Search
use spatial_narrative::index::SpatialIndex;
let index = SpatialIndex::from_iter(events, |e| &e.location);
// Find 5 nearest events to a point
let nearest = index.nearest(user_lat, user_lon, 5);
Graph Patterns
Complete Analysis Pipeline
use spatial_narrative::graph::{NarrativeGraph, EdgeType};
// Build graph
let mut graph = NarrativeGraph::from_events(events);
// Apply connection strategies
graph.connect_temporal();
graph.connect_spatial(10.0);
graph.connect_thematic();
// Analyze
let roots = graph.roots(); // Starting points
let leaves = graph.leaves(); // End points
let hub_count = graph.nodes()
.filter(|(id, _)| graph.in_degree(*id) + graph.out_degree(*id) > 5)
.count();
println!("Graph: {} roots, {} leaves, {} hubs",
roots.len(), leaves.len(), hub_count);
Path Analysis
// Find how events are connected
let start = graph.get_node(&event1.id).unwrap();
let end = graph.get_node(&event2.id).unwrap();
if let Some(path) = graph.shortest_path(start, end) {
println!("Connected via {} events", path.nodes.len());
}
IO Patterns
Multi-Format Export
use spatial_narrative::io::{Format, GeoJsonFormat, CsvFormat, JsonFormat};
// Export to all formats
let geojson = GeoJsonFormat::new().export_str(&narrative)?;
let csv = CsvFormat::new().export_str(&narrative)?;
let json = JsonFormat::pretty().export_str(&narrative)?;
std::fs::write("data.geojson", &geojson)?;
std::fs::write("data.csv", &csv)?;
std::fs::write("data.json", &json)?;
Round-Trip Preservation
// Use JSON for lossless round-trips
let json = JsonFormat::new();
let exported = json.export_str(&narrative)?;
let imported = json.import_str(&exported)?;
assert_eq!(narrative.events.len(), imported.events.len());
Analysis Patterns
Quick Metrics
use spatial_narrative::analysis::{SpatialMetrics, TemporalMetrics};
let spatial = SpatialMetrics::from_events(&events);
let temporal = TemporalMetrics::from_events(&events);
println!("Spatial extent: {:.0} km", spatial.max_extent / 1000.0);
println!("Time span: {:?}", temporal.duration);
println!("Event rate: {:.2}/hour",
events.len() as f64 / temporal.duration.as_secs_f64() * 3600.0);
Clustering
use spatial_narrative::analysis::DBSCAN;
let dbscan = DBSCAN::new(5000.0, 3); // 5km radius, min 3 points
let result = dbscan.cluster(&events);
println!("Found {} clusters, {} noise points",
result.num_clusters(),
result.noise.len());
Error Handling
Graceful Parsing
let events: Vec<Event> = raw_data.iter()
.filter_map(|row| {
match parse_event(row) {
Ok(event) => Some(event),
Err(e) => {
eprintln!("Skipping invalid row: {}", e);
None
}
}
})
.collect();
Validation
fn validate_event(event: &Event) -> Result<()> {
if event.text.is_empty() {
return Err(Error::Validation("Event text is empty".into()));
}
if event.location.lat < -90.0 || event.location.lat > 90.0 {
return Err(Error::Validation("Invalid latitude".into()));
}
Ok(())
}
Filtering Events
Techniques for filtering and selecting events.
Spatial Filtering
By Bounding Box
use spatial_narrative::core::GeoBounds;
// Define bounds
let nyc = GeoBounds::new(40.4, -74.3, 41.0, -73.7);
// Filter narrative
let nyc_narrative = narrative.filter_spatial(&nyc);
// Or filter events directly
let nyc_events: Vec<_> = events.iter()
.filter(|e| nyc.contains(&e.location))
.collect();
By Radius
use spatial_narrative::analysis::haversine_distance;
use spatial_narrative::core::Location;
let center = Location::new(40.7128, -74.0060);
let radius_m = 5000.0; // 5km
let nearby: Vec<_> = events.iter()
.filter(|e| haversine_distance(¢er, &e.location) <= radius_m)
.collect();
By Multiple Regions
let regions = vec![
GeoBounds::new(40.4, -74.3, 41.0, -73.7), // NYC
GeoBounds::new(33.7, -118.7, 34.4, -117.9), // LA
GeoBounds::new(41.6, -88.0, 42.1, -87.5), // Chicago
];
let in_any_region: Vec<_> = events.iter()
.filter(|e| regions.iter().any(|r| r.contains(&e.location)))
.collect();
Temporal Filtering
By Time Range
use spatial_narrative::core::TimeRange;
// Specific range
let range = TimeRange::new(
Timestamp::parse("2024-01-01T00:00:00Z").unwrap(),
Timestamp::parse("2024-01-31T23:59:59Z").unwrap(),
);
let january = narrative.filter_temporal(&range);
// Convenience constructors
let q1 = events.iter()
.filter(|e| TimeRange::month(2024, 1).contains(&e.timestamp)
|| TimeRange::month(2024, 2).contains(&e.timestamp)
|| TimeRange::month(2024, 3).contains(&e.timestamp));
Before/After
let cutoff = Timestamp::parse("2024-06-01T00:00:00Z").unwrap();
let before: Vec<_> = events.iter()
.filter(|e| e.timestamp < cutoff)
.collect();
let after: Vec<_> = events.iter()
.filter(|e| e.timestamp >= cutoff)
.collect();
By Day of Week
let weekends: Vec<_> = events.iter()
.filter(|e| {
let weekday = e.timestamp.weekday();
weekday == 0 || weekday == 6 // Sunday or Saturday
})
.collect();
By Time of Day
// Business hours (9 AM to 5 PM)
let business_hours: Vec<_> = events.iter()
.filter(|e| {
let hour = e.timestamp.hour();
hour >= 9 && hour < 17
})
.collect();
Tag Filtering
Single Tag
let important: Vec<_> = events.iter()
.filter(|e| e.has_tag("important"))
.collect();
Any of Tags
let priority_tags = ["urgent", "important", "critical"];
let priority: Vec<_> = events.iter()
.filter(|e| priority_tags.iter().any(|t| e.has_tag(t)))
.collect();
All of Tags
let required_tags = ["verified", "published"];
let complete: Vec<_> = events.iter()
.filter(|e| required_tags.iter().all(|t| e.has_tag(t)))
.collect();
Excluding Tags
let not_spam: Vec<_> = events.iter()
.filter(|e| !e.has_tag("spam") && !e.has_tag("duplicate"))
.collect();
Text Filtering
Contains Keyword
let mentions_storm: Vec<_> = events.iter()
.filter(|e| e.text.to_lowercase().contains("storm"))
.collect();
Regex Matching
use regex::Regex;
let phone_pattern = Regex::new(r"\d{3}-\d{3}-\d{4}").unwrap();
let with_phone: Vec<_> = events.iter()
.filter(|e| phone_pattern.is_match(&e.text))
.collect();
Minimum Length
let substantive: Vec<_> = events.iter()
.filter(|e| e.text.len() >= 50)
.collect();
Combined Filtering
Chained Filters
let filtered: Vec<_> = events.iter()
.filter(|e| nyc_bounds.contains(&e.location))
.filter(|e| january.contains(&e.timestamp))
.filter(|e| e.has_tag("verified"))
.filter(|e| !e.has_tag("duplicate"))
.collect();
With Predicate Function
fn is_relevant(event: &Event) -> bool {
// Complex filtering logic
let in_region = NYC_BOUNDS.contains(&event.location);
let in_timeframe = STUDY_PERIOD.contains(&event.timestamp);
let has_content = event.text.len() >= 10;
let is_verified = event.has_tag("verified");
in_region && in_timeframe && has_content && is_verified
}
let relevant: Vec<_> = events.iter()
.filter(|e| is_relevant(e))
.collect();
Using Indexes for Filtering
Spatial Index
use spatial_narrative::index::SpatialIndex;
let index = SpatialIndex::from_iter(events.clone(), |e| &e.location);
// Fast bounding box query
let in_region = index.query_bounds(&bounds);
// Fast radius query
let nearby = index.query_radius_meters(lat, lon, radius);
Temporal Index
use spatial_narrative::index::TemporalIndex;
let index = TemporalIndex::from_iter(events.clone(), |e| &e.timestamp);
// Fast range query
let in_period = index.query_range(&time_range);
// Ordered access
let chronological = index.chronological();
Combined Index
use spatial_narrative::index::SpatiotemporalIndex;
let index = SpatiotemporalIndex::from_iter(
events.clone(),
|e| &e.location,
|e| &e.timestamp
);
// Query both dimensions efficiently
let filtered = index.query(&bounds, &time_range);
Sampling
Random Sample
use rand::seq::SliceRandom;
let mut rng = rand::thread_rng();
let sample: Vec<_> = events.choose_multiple(&mut rng, 100).collect();
Systematic Sample
// Every 10th event
let systematic: Vec<_> = events.iter()
.enumerate()
.filter(|(i, _)| i % 10 == 0)
.map(|(_, e)| e)
.collect();
Stratified Sample
// Sample from each region
let mut samples = Vec::new();
for region in ®ions {
let in_region: Vec<_> = events.iter()
.filter(|e| region.contains(&e.location))
.collect();
samples.extend(in_region.choose_multiple(&mut rng, 10));
}
Integration Examples
spatial-narrative is designed to fit into your existing data pipeline. Here's how to integrate it with common tools.
Data Ingestion
HTTP APIs (with reqwest)
Fetch data from any API and transform into Events:
use reqwest::blocking::Client;
use spatial_narrative::core::{Event, EventBuilder, Location, Timestamp, NarrativeBuilder};
fn fetch_and_process() -> Result<Narrative, Box<dyn std::error::Error>> {
let client = Client::new();
// Fetch from your data source
let response: Vec<MyApiRecord> = client
.get("https://api.example.com/events")
.send()?
.json()?;
// Transform to Events
let events: Vec<Event> = response
.into_iter()
.filter_map(|r| {
Some(EventBuilder::new()
.location(Location::new(r.lat, r.lon))
.timestamp(Timestamp::parse(&r.date).ok()?)
.text(&r.description)
.build())
})
.collect();
Ok(NarrativeBuilder::new()
.title("API Data")
.events(events)
.build())
}
CSV Files
use csv::Reader;
use spatial_narrative::core::{Event, EventBuilder, Location, Timestamp};
fn load_csv(path: &str) -> Vec<Event> {
let mut rdr = Reader::from_path(path).unwrap();
rdr.deserialize()
.filter_map(|result| {
let record: MyRecord = result.ok()?;
Some(EventBuilder::new()
.location(Location::new(record.lat, record.lon))
.timestamp(Timestamp::parse(&record.date).ok()?)
.text(&record.text)
.build())
})
.collect()
}
JSON Files
use serde_json;
use spatial_narrative::core::{Event, Location, Timestamp};
#[derive(serde::Deserialize)]
struct RawEvent {
lat: f64,
lon: f64,
time: String,
description: String,
}
fn load_json(path: &str) -> Vec<Event> {
let data = std::fs::read_to_string(path).unwrap();
let raw: Vec<RawEvent> = serde_json::from_str(&data).unwrap();
raw.into_iter()
.filter_map(|r| {
Some(Event::new(
Location::new(r.lat, r.lon),
Timestamp::parse(&r.time).ok()?,
&r.description
))
})
.collect()
}
Web Mapping
Leaflet
Export to GeoJSON and display with Leaflet:
use spatial_narrative::io::{GeoJsonFormat, Format};
let geojson = GeoJsonFormat::new().export_str(&narrative)?;
std::fs::write("public/events.geojson", &geojson)?;
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css" />
<script src="https://unpkg.com/leaflet@1.9.4/dist/leaflet.js"></script>
</head>
<body>
<div id="map" style="height: 100vh;"></div>
<script>
const map = L.map('map').setView([40.7128, -74.0060], 10);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
attribution: '© OpenStreetMap'
}).addTo(map);
fetch('events.geojson')
.then(res => res.json())
.then(data => {
L.geoJSON(data, {
pointToLayer: (feature, latlng) => {
return L.circleMarker(latlng, {
radius: 8,
fillColor: '#ff7800',
color: '#000',
weight: 1,
fillOpacity: 0.8
});
},
onEachFeature: (feature, layer) => {
layer.bindPopup(`
<strong>${feature.properties.text}</strong><br>
${feature.properties.timestamp}
`);
}
}).addTo(map);
});
</script>
</body>
</html>
Mapbox GL JS
mapboxgl.accessToken = 'your-token';
const map = new mapboxgl.Map({
container: 'map',
style: 'mapbox://styles/mapbox/streets-v11',
center: [-74.0060, 40.7128],
zoom: 10
});
map.on('load', () => {
map.addSource('events', {
type: 'geojson',
data: 'events.geojson'
});
map.addLayer({
id: 'events-layer',
type: 'circle',
source: 'events',
paint: {
'circle-radius': 8,
'circle-color': '#ff7800'
}
});
});
Data Science
Python/Pandas
Export to CSV for pandas:
use spatial_narrative::io::{CsvFormat, Format};
let csv = CsvFormat::new().export_str(&narrative)?;
std::fs::write("events.csv", &csv)?;
import pandas as pd
import geopandas as gpd
# Read CSV
df = pd.read_csv('events.csv', parse_dates=['timestamp'])
# Basic analysis
print(f"Events: {len(df)}")
print(f"Date range: {df['timestamp'].min()} to {df['timestamp'].max()}")
# Convert to GeoDataFrame
gdf = gpd.GeoDataFrame(
df,
geometry=gpd.points_from_xy(df.longitude, df.latitude),
crs="EPSG:4326"
)
# Spatial operations
gdf.plot()
R
library(tidyverse)
library(sf)
# Read CSV
events <- read_csv("events.csv")
# Convert to sf object
events_sf <- st_as_sf(events,
coords = c("longitude", "latitude"),
crs = 4326)
# Plot
ggplot(events_sf) +
geom_sf() +
theme_minimal()
GIS Software
QGIS
- Export to GeoJSON:
let geojson = GeoJsonFormat::new().export_str(&narrative)?;
std::fs::write("events.geojson", &geojson)?;
- In QGIS: Layer → Add Layer → Add Vector Layer
- Select the GeoJSON file
ArcGIS
Export to GeoJSON, then import as feature class.
Graph Visualization
Graphviz
let dot = graph.to_dot();
std::fs::write("graph.dot", &dot)?;
# Render to PNG
dot -Tpng graph.dot -o graph.png
# Render to SVG
dot -Tsvg graph.dot -o graph.svg
# Different layouts
neato -Tpng graph.dot -o graph-neato.png
circo -Tpng graph.dot -o graph-circo.png
D3.js Force Graph
let json = graph.to_json();
std::fs::write("public/graph.json", &json)?;
<script src="https://d3js.org/d3.v7.min.js"></script>
<script>
fetch('graph.json')
.then(res => res.json())
.then(data => {
const svg = d3.select("svg");
const width = +svg.attr("width");
const height = +svg.attr("height");
const simulation = d3.forceSimulation(data.nodes)
.force("link", d3.forceLink(data.edges).id(d => d.id))
.force("charge", d3.forceManyBody().strength(-100))
.force("center", d3.forceCenter(width / 2, height / 2));
// Draw edges
const link = svg.selectAll(".link")
.data(data.edges)
.join("line")
.attr("class", "link");
// Draw nodes
const node = svg.selectAll(".node")
.data(data.nodes)
.join("circle")
.attr("class", "node")
.attr("r", 8);
simulation.on("tick", () => {
link.attr("x1", d => d.source.x)
.attr("y1", d => d.source.y)
.attr("x2", d => d.target.x)
.attr("y2", d => d.target.y);
node.attr("cx", d => d.x)
.attr("cy", d => d.y);
});
});
</script>
Database Integration
SQLite/SQLx
use sqlx::sqlite::SqlitePool;
async fn save_events(pool: &SqlitePool, events: &[Event]) -> Result<()> {
for event in events {
sqlx::query(
"INSERT INTO events (id, lat, lon, timestamp, text) VALUES (?, ?, ?, ?, ?)"
)
.bind(event.id.to_string())
.bind(event.location.lat)
.bind(event.location.lon)
.bind(event.timestamp.to_rfc3339())
.bind(&event.text)
.execute(pool)
.await?;
}
Ok(())
}
async fn load_events(pool: &SqlitePool) -> Result<Vec<Event>> {
let rows = sqlx::query("SELECT * FROM events")
.fetch_all(pool)
.await?;
let events = rows.iter()
.map(|row| Event::new(
Location::new(row.get("lat"), row.get("lon")),
Timestamp::parse(row.get::<String, _>("timestamp")).unwrap(),
row.get("text")
))
.collect();
Ok(events)
}
PostgreSQL with PostGIS
CREATE TABLE events (
id UUID PRIMARY KEY,
location GEOGRAPHY(POINT, 4326),
timestamp TIMESTAMPTZ,
text TEXT,
tags TEXT[]
);
-- Spatial index
CREATE INDEX events_location_idx ON events USING GIST (location);
REST API
Actix-web
use actix_web::{web, App, HttpServer, HttpResponse};
use spatial_narrative::io::{GeoJsonFormat, Format};
async fn get_events(data: web::Data<AppState>) -> HttpResponse {
let geojson = GeoJsonFormat::new()
.export_str(&data.narrative)
.unwrap();
HttpResponse::Ok()
.content_type("application/geo+json")
.body(geojson)
}
#[actix_web::main]
async fn main() -> std::io::Result<()> {
HttpServer::new(|| {
App::new()
.route("/events", web::get().to(get_events))
})
.bind("127.0.0.1:8080")?
.run()
.await
}
CLI Tools
Process Pipeline
use clap::Parser;
#[derive(Parser)]
struct Args {
#[arg(short, long)]
input: PathBuf,
#[arg(short, long)]
output: PathBuf,
#[arg(long)]
format: String,
}
fn main() -> Result<()> {
let args = Args::parse();
// Load
let input = std::fs::read_to_string(&args.input)?;
let narrative = JsonFormat::new().import_str(&input)?;
// Export
let output = match args.format.as_str() {
"geojson" => GeoJsonFormat::new().export_str(&narrative)?,
"csv" => CsvFormat::new().export_str(&narrative)?,
_ => JsonFormat::new().export_str(&narrative)?,
};
std::fs::write(&args.output, &output)?;
Ok(())
}
API Reference
Full API documentation is available at docs.rs/spatial-narrative.
Generate local docs with:
cargo doc --open
Performance
Performance characteristics and optimization tips.
Complexity
Indexing Operations
| Operation | SpatialIndex | TemporalIndex | SpatiotemporalIndex |
|---|---|---|---|
| Insert | O(log n) | O(log n) | O(log n) |
| Build (bulk) | O(n log n) | O(n log n) | O(n log n) |
| Point query | O(log n) | O(log n) | O(log n) |
| Range query | O(log n + k) | O(log n + k) | O(log n + k) |
| K-nearest | O(k log n) | N/A | N/A |
Where n = number of items, k = number of results.
Graph Operations
| Operation | Complexity |
|---|---|
| Add node | O(1) |
| Add edge | O(1) |
| Get neighbors | O(degree) |
| Has path (BFS) | O(V + E) |
| Shortest path | O((V + E) log V) |
| connect_temporal() | O(n log n) |
| connect_spatial(d) | O(n²) |
| connect_thematic() | O(n² × tags) |
Where V = vertices, E = edges.
Memory Usage
Per-Event Overhead
| Component | Approximate Size |
|---|---|
| Event base | ~200 bytes |
| Location | ~40 bytes |
| Timestamp | ~32 bytes |
| EventId (UUID) | 16 bytes |
| Text (per char) | 1 byte |
| Tag (each) | ~24 bytes + string |
Index Overhead
| Index | Overhead per item |
|---|---|
| SpatialIndex | ~64 bytes |
| TemporalIndex | ~48 bytes |
| SpatiotemporalIndex | ~112 bytes |
Approximate Memory Formula
Memory (MB) ≈ events × (200 + avg_text_len + tags × 30) / 1_000_000
+ index_overhead
Optimization Tips
Bulk Loading
Use from_iter instead of repeated insert:
// ✅ Fast: O(n log n) bulk load
let index = SpatialIndex::from_iter(events, |e| &e.location);
// ❌ Slow: O(n log n) per insert
let mut index = SpatialIndex::new();
for event in events {
index.insert(event, &event.location);
}
Query Then Filter
Use indexes for initial filtering, then apply additional criteria:
// Get spatial candidates (fast)
let candidates = spatial_index.query_bounds(&bounds);
// Apply additional filters (linear)
let filtered: Vec<_> = candidates.into_iter()
.filter(|e| e.has_tag("important"))
.filter(|e| time_range.contains(&e.timestamp))
.collect();
Avoid O(n²) Operations
The connect_spatial and connect_thematic methods are O(n²):
// For large graphs, consider:
// 1. Sample events
let sample: Vec<_> = events.choose_multiple(&mut rng, 1000).collect();
let mut graph = NarrativeGraph::from_events(sample);
graph.connect_spatial(10.0); // Now O(sample²)
// 2. Use spatial index for proximity
let index = SpatialIndex::from_iter(events.clone(), |e| &e.location);
for event in &events {
let nearby = index.query_radius_meters(event.location.lat, event.location.lon, 1000.0);
// Only connect to nearby events
}
// 3. Skip if graph is too large
if events.len() > 10_000 {
println!("Skipping spatial connections (too large)");
} else {
graph.connect_spatial(10.0);
}
Lazy Evaluation
Build indexes only when needed:
struct LazyNarrative {
events: Vec<Event>,
spatial_index: Option<SpatialIndex<Event>>,
}
impl LazyNarrative {
fn get_spatial_index(&mut self) -> &SpatialIndex<Event> {
if self.spatial_index.is_none() {
self.spatial_index = Some(SpatialIndex::from_iter(
self.events.iter().cloned(),
|e| &e.location
));
}
self.spatial_index.as_ref().unwrap()
}
}
Parallel Processing
Use rayon for parallel operations:
use rayon::prelude::*;
// Parallel filtering
let filtered: Vec<_> = events.par_iter()
.filter(|e| expensive_check(e))
.collect();
// Parallel analysis
let metrics: Vec<_> = narratives.par_iter()
.map(|n| SpatialMetrics::from_events(&n.events))
.collect();
Benchmarks
Index Query Performance
Tested on 100,000 events:
| Query Type | Time |
|---|---|
| Spatial bbox (1% area) | ~50 μs |
| Spatial radius (1km) | ~80 μs |
| Temporal range (1 month) | ~40 μs |
| Combined (1% × 1 month) | ~100 μs |
| K-nearest (k=10) | ~120 μs |
Graph Operations
| Operation | 1K events | 10K events | 100K events |
|---|---|---|---|
| from_events | 1 ms | 10 ms | 100 ms |
| connect_temporal | 2 ms | 25 ms | 300 ms |
| connect_spatial(10km) | 50 ms | 5 s | too slow |
| shortest_path | < 1 ms | 5 ms | 50 ms |
| to_dot | 5 ms | 50 ms | 500 ms |
I/O Performance
| Format | Export 10K events | Import 10K events |
|---|---|---|
| JSON | 15 ms | 20 ms |
| GeoJSON | 20 ms | 25 ms |
| CSV | 10 ms | 15 ms |
Memory Optimization
String Interning
For repeated tags, consider interning:
use std::collections::HashSet;
use std::sync::Arc;
// Share common strings
let tag_pool: HashSet<Arc<str>> = HashSet::new();
fn intern_tag(pool: &mut HashSet<Arc<str>>, tag: &str) -> Arc<str> {
if let Some(existing) = pool.get(tag) {
existing.clone()
} else {
let new = Arc::from(tag);
pool.insert(new.clone());
new
}
}
Streaming Processing
For very large datasets, process in chunks:
use std::io::{BufRead, BufReader};
let file = File::open("huge_file.csv")?;
let reader = BufReader::new(file);
for chunk in reader.lines().chunks(10_000) {
let events: Vec<Event> = chunk
.filter_map(|line| parse_event(&line.ok()?).ok())
.collect();
// Process chunk
process_events(&events);
}
FAQ
Frequently asked questions about spatial-narrative.
General
What is a spatial narrative?
A spatial narrative is a sequence of events that are anchored in both space (geographic location) and time. Think of it as a story that unfolds across a map.
Examples:
- A news story with events happening in different cities over time
- GPS tracks showing a journey
- Historical events with known locations and dates
- Sensor readings with coordinates and timestamps
When should I use this library?
Use spatial-narrative when you need to:
- Store events with both location AND time information
- Query events by geography (bounding box, radius, nearest)
- Query events by time range
- Analyze relationships between events
- Export to mapping formats (GeoJSON) or data formats (CSV)
- Build graphs showing how events connect
What makes this different from just using a Vec?
The library provides:
- Efficient indexing: O(log n) spatial and temporal queries
- Rich types: Location with validation, Timestamp with timezone handling
- Relationship graphs: Connect events and find paths
- Analysis tools: Clustering, metrics, movement detection
- I/O formats: GeoJSON, CSV, JSON with proper handling
Core Types
How precise are coordinates?
Coordinates use f64 (64-bit floating point), which provides about 15-16 significant digits. For geographic coordinates, this is sub-millimeter precision — far more than any real-world application needs.
What timezone does Timestamp use?
Timestamp is timezone-aware and stores time in UTC internally. When parsing:
// Explicit UTC
Timestamp::parse("2024-01-15T10:00:00Z")
// With offset
Timestamp::parse("2024-01-15T10:00:00+05:00")
// Local time (uses system timezone)
Timestamp::parse("2024-01-15T10:00:00")
Can I use custom IDs for events?
Events get auto-generated UUIDs, but you can store custom IDs in metadata:
event.set_metadata("custom_id", json!("MY-001"));
Indexing
Which index should I use?
| Use Case | Index |
|---|---|
| Only location queries | SpatialIndex |
| Only time queries | TemporalIndex |
| Both location AND time | SpatiotemporalIndex |
Are indexes updated automatically?
No, indexes are immutable after creation. If you add events, rebuild the index:
// Add new events
events.push(new_event);
// Rebuild index
let index = SpatialIndex::from_iter(events.iter().cloned(), |e| &e.location);
How accurate is the radius query?
query_radius_meters uses the Haversine formula for great-circle distance, which is accurate for the Earth as a sphere. Error is typically < 0.5% vs a full ellipsoid model.
Graphs
What's the difference between EdgeType variants?
| Type | Meaning | Auto-Generated |
|---|---|---|
Temporal | A happens before B | connect_temporal() |
Spatial | A is near B | connect_spatial(km) |
Thematic | A and B share tags | connect_thematic() |
Causal | A causes B | Manual only |
Reference | A mentions B | Manual only |
Custom | User-defined | Manual only |
Why is connect_spatial slow?
connect_spatial compares all pairs of events (O(n²)). For 10,000 events, that's 100 million comparisons.
Workarounds:
- Use a spatial index to pre-filter candidates
- Sample events before connecting
- Skip for large datasets
I/O
Which format preserves all data?
JsonFormat preserves everything including custom metadata. GeoJsonFormat and CsvFormat may lose some fields.
Can I import from other GeoJSON sources?
Yes, as long as the GeoJSON has Point features with timestamp information:
{
"type": "Feature",
"geometry": {"type": "Point", "coordinates": [-74.0, 40.7]},
"properties": {"timestamp": "2024-01-15T10:00:00Z"}
}
How do I handle large files?
Use streaming I/O:
let file = File::open("large.geojson")?;
let reader = BufReader::new(file);
let narrative = GeoJsonFormat::new().import(&mut reader)?;
Analysis
What distance unit does DBSCAN use?
DBSCAN::new(eps, min_points) takes eps in meters.
// 500 meter radius
let dbscan = DBSCAN::new(500.0, 3);
// 5 kilometer radius
let dbscan = DBSCAN::new(5000.0, 3);
How does clustering work with timestamps?
Current clustering is spatial only. For spatiotemporal clustering, filter by time first:
// Cluster events from January only
let january_events: Vec<_> = events.iter()
.filter(|e| TimeRange::month(2024, 1).contains(&e.timestamp))
.collect();
let dbscan = DBSCAN::new(1000.0, 3);
let result = dbscan.cluster(&january_events);
Troubleshooting
"No events found" after import
Check that:
- File path is correct
- Format matches file content
- Required fields exist (coordinates, timestamp)
// Debug: print what was imported
let narrative = GeoJsonFormat::new().import_str(&content)?;
println!("Imported {} events", narrative.events.len());
if let Some(first) = narrative.events.first() {
println!("First event: {:?}", first);
}
Timestamp parse errors
Ensure ISO 8601 format:
- ✅
2024-01-15T10:00:00Z - ✅
2024-01-15T10:00:00+00:00 - ❌
2024/01/15 10:00(wrong separators) - ❌
Jan 15, 2024(not ISO format)
Graph visualization looks wrong
Try different layout options:
// Timeline (left-to-right)
let dot = graph.to_dot_with_options(DotOptions::timeline());
// Hierarchical (top-to-bottom)
let dot = graph.to_dot_with_options(DotOptions::hierarchical());
Different Graphviz engines:
neato -Tpng graph.dot -o graph.png # Force-directed
circo -Tpng graph.dot -o graph.png # Circular
fdp -Tpng graph.dot -o graph.png # Spring model
Contributing
See CONTRIBUTING.md for contribution guidelines.
Changelog
See CHANGELOG.md for version history.