Geospatial Property Risk Scoring Engine (v2)

This repository contains consolidated research spreadsheets, high-performance database precalculation tools, scoring engines, interactive dark-theme dashboards, and automated presentation builders for UK property risk profiles.

The project is built around the Boundary-Matched, Severity-Weighted, and Population-Normalized Criminology Model (v2), providing ultra-local postcode safety indices alongside macro-outcode district safety percentiles.

---

📂 Project Directory Structure

All analytical models, scripts, and visual assets are located within the crime_score subdirectory:

e:/INTERN/RS/
├── .gitignore                                            # Prevents committing credentials & large files
├── .env                                                  # Environment variables (private, ignored)
└── crime_score/
    ├── dashboard/
    │   └── crime_dashboard.html                          # Interactive dark-theme glassmorphic client (Leaflet.js & Chart.js)
    ├── data/
    │   ├── PropMarker_UK_Property_Data_Intelligence.xlsx           # Original research workbook
    │   ├── PropMarker_UK_Property_Data_Intelligence_ENRICHED.xlsx  # Enriched worksheets with official gov APIs
    │   ├── PropMarker_Property_Data_Intelligence_CONSOLIDATED.xlsx # Consolidated sheets (Master & Guide)
    │   ├── PropMarker_Property_Data_Intelligence_CONSOLIDATED.csv  # Flat master dataset
    │   └── PropMarker_Crime_Severity_Weights.csv                   # Severity weights list
    ├── docs/
    │   ├── crime_score_methodology.md             # Theoretical mathematical methodology (decay, exponential curves)
    │   ├── crime_score_logic_correction.md        # Logic correction methodology brief for senior review
    │   ├── propmarker_research_and_calibration.md # Detailed national rates and case studies
    │   ├── crime_score_presentation_notes.md      # Slide-by-slide presenter scripts & speaker notes
    │   ├── url_results.txt                        # Bulk URL validation status report
    │   └── PropMarker_Crime_Score_Presentation.pptx# Widescreen PowerPoint presentation slide deck
    └── scripts/
        ├── precalculate_outcode_stats.py          # Parallel database-side outcode aggregator with timeout limits
        ├── calculate_crime_score.py               # CLI tool to run local calculations on any postcode (loads v2 cache)
        ├── server.py                              # Multi-threaded server hosting APIs & static dashboard files
        ├── add_detailed_methodology.py            # Excel sheet builder for the detailed methodology worksheets
        ├── consolidate_sheets.py                  # Main sheet consolidation & CSV export script
        ├── add_gov_sites_sheet.py                 # Government site sheet builder
        ├── validate_urls.py                       # Validates all unique URLs in spreadsheets
        ├── verify_consolidation.py                # Automated verifier for row/column counts
        └── create_presentation.py                 # Generates the widescreen PowerPoint slide deck

---

🧮 Localization & Calibration Model (v2)

PropMarker evaluates crime safety at two distinct scales to balance local micro-safety with district-level context:

1. Local Postcode Safety Index (1–100)

Assesses the immediate vicinity of a property within an 800m to 1,000m radius centered on the geocoded postcode.

• Proximity Distance Decay: Crimes closer to the home carry full weight, while crimes at the outer perimeter are discounted:
  • 0m to 250m: 1.0x impact multiplier
  • 251m to 500m: 0.6x impact multiplier
  • 501m to 1000m: 0.2x impact multiplier
  • 1000m+: 0.0x (Excluded)
• Severity Weighting: Crimes are multiplied by ONS-aligned severity weights (Violence = 10, ASB = 1) to prevent high-frequency petty crimes from skewing safety ratings.
• Normalization Curve: Maps the raw score onto an exponential decay curve to output a standardized index: $$\text{Index} = 100 \times \left(1 - e^{-0.00085 \times \text{Raw Score}}\right)$$

2. Outcode District National Percentile Rank (v2)

Evaluates macro-level safety within the outcode district boundaries (e.g. SW1A, E1, M1) using database-side parallel thread pools.

• ONS Severity-Weighted outcode score: Sum of all severity-weighted offences in the district.
• Population-Normalized Risk Rate: Calculates weighted points per 1,000 residents using ONS census population data: $$\text{Risk Rate} = \frac{\text{Weighted Outcode Crimes}}{\text{Population of Outcode}} \times 1,000$$
• National Percentile Ranking: Ranks the district's Risk Rate against all 3,096 UK outcodes to output the percentile (0% to 100%).
• Self-Balancing Tiers: Divides the percentiles into three equal, robust bands: Low Risk (0–33.3%), Medium Risk (33.4–66.6%), and High Risk (>66.6%).
• Geographic Crime Density: Contextual reference metric reporting crimes per square kilometer (crimes/km²).

---

⚡ Getting Started & Running Scripts

To set up your environment and run the calculations, follow these commands. First, navigate to the crime_score directory:

cd crime_score

1. Install Dependencies

Ensure you have Python 3.8+ installed, then run:

pip install pymongo openpyxl python-pptx

2. Setup Environment Variables

Create a .env file at the project root (e:/INTERN/RS/.env) containing your MongoDB URI:

MONGODB_URI=mongodb://username:password@ip_address:port/database_name

(Note: Do not commit the .env file to Git. It is already ignored in the .gitignore.)

3. Run the Database Precalculation Cache

Run the aggregator to precalculate populations, area densities, risk rates, and percentile ranks across all 3,095 outcodes. This saves a JSON cache locally:

python scripts/precalculate_outcode_stats.py

4. Run the Local Backend Server

Start the multi-threaded API and web server:

python scripts/server.py

This serves:

• Local scoring API: http://localhost:8888/api/crime-score?postcode=SW1A1AA&radius=800
• Available dates: http://localhost:8888/api/crimes-street-dates
• Local dashboard UI: http://localhost:8888

5. Programmatic PowerPoint & Spreadsheet Builders

• Update Excel Workbooks: Regenerates sheets with detailed v2 formulas and consolidates them:

  python scripts/add_detailed_methodology.py
  python scripts/consolidate_sheets.py

• Build Widescreen Slide Deck: Re-compiles Slide 3, Slide 6, and Slide 8 to showcase v2 rankings and case studies:

  python scripts/create_presentation.py