Measure building segmentation before claiming an improvement.

The problem

The historical validation score mixed source geography across splits and included missing image/mask pairs. It could not establish predictive quality. The newer study preserves the WHU publisher partitions and checks every paired file.

My contribution

I connected a frozen WHU checkpoint to real CPU inference, bounded image uploads and polygons, explicit readiness and failure states, and a local-upload interface. I added full-state recovery, a simple RGB comparator, and an external-city experiment that exposes weak transfer.

How it works

1. Frozen data and checkpoint
2. Bounded local upload + CPU prediction
3. Mask, contours and explicit failure states
4. Versioned evaluation and recoverable training

Decisions that shaped the release

• Measure the exact serving implementation in its locked CPU environment. Keep its results separate from the original CUDA AMP evaluation.
• Treat missing weights, corrupt images, oversized inputs and unavailable remote imagery as failures. Demo selection never fabricates predictions.
• Recover full optimizer, scheduler, AMP and RNG state at epoch boundaries with workers=0. Validate source, data, runtime and reviewed-label bindings before resuming.
• Register 32 SpaceNet tiles per city before downloading labels or scoring. Keep the original model and threshold frozen, including when transfer results are poor.

Measured results

The serving CLI and local HTTP API now share a manifest-bound prediction path. On one generated numerical image, their probability arrays and masks match exactly; an independent forward calculation agrees within the declared tolerance. This verifies a local serving transaction, not building accuracy or geographic transfer. Browser interaction and original-container startup are now verified in the separate browser acceptance record.

The original baseline remains the fixed reference; later training experiments did not improve it. The new CPU result scores every WHU test tile through the actual ModelService path. Reusing WHU is release verification, not a new independent holdout. The 64 preselected external tiles provide a separate, small transfer test.

A failure worth inspecting

A strong within-domain score does not transfer to Paris.

With the original checkpoint and threshold unchanged, IoU fell to 0.4016 on Las Vegas and 0.0474 on Paris, 32 tiles per city. Different acquisition and annotation domains matter. The examples show actual masks, predictions and missed/false-positive pixels, without selecting another model after seeing the results.

{
  "threshold": 0.5,
  "samples_per_city": 32,
  "regions": {
    "AOI_2_Vegas": {
      "samples": 32,
      "iou": 0.4015882263973643,
      "dice": 0.573047374162959,
      "precision": 0.6536702855301016,
      "recall": 0.5101287091779976,
      "f1": 0.5730473741629589,
      "pixel_accuracy": 0.8296798816568047,
      "tp": 1545338,
      "fp": 818756,
      "fn": 1483972,
      "tn": 9671934
    },
    "AOI_3_Paris": {
      "samples": 32,
      "iou": 0.04735199844592298,
      "dice": 0.09042231936576166,
      "precision": 0.12092433431310538,
      "recall": 0.07220842891447594,
      "f1": 0.09042231936576166,
      "pixel_accuracy": 0.8839291420118344,
      "tp": 78002,
      "fp": 567046,
      "fn": 1002232,
      "tn": 11872720
    }
  },
  "external_tuning": false
}

Working artifacts

Limits and next questions

• Binary building extent only. SpaceNet footprint labels and WHU aerial annotations differ; this small transfer experiment does not establish performance on arbitrary addresses or sensors.
• Exact epoch-boundary resume supports workers=0. Persistent-worker RNG and mid-batch recovery are not claimed; CPU and CUDA recovery remain separate verified profiles.
• Scoring used four independent CPU services for verification; this does not change the deployed API concurrency limit.
• The CUDA memory measurement covers one native 512px, batch-1 training step. It does not reconstruct peak memory during the historical full training runs.
• The three-seed factorial study is complete: all twelve runs finished their planned training. Its completed analysis separates sampling from augmentation. The later reused-test follow-up is development evidence, not a fresh holdout; the original baseline and browser acceptance remain separately dated.
• WHU imagery is not republished because its exact redistribution terms remain unclear. The SpaceNet-derived examples include attribution and CC BY-SA 4.0 terms.
• The supported application flow is local PNG/JPEG/WebP upload. Remote imagery/address workflows are unavailable. Contours omit holes and components too small to form a polygon; scores are uncalibrated.
• Actual-checkpoint browser acceptance verified local upload, probability preview, three hash-matched exports, failure recovery, keyboard use and mobile layout. This is a locally runnable application; no public inference service is claimed.