OSINT · Homelab · PowerShell

Digital
Footprint Finder

Local-first OSINT pipeline · Sherlock · HIBP v3 · Splunk JSON events · Diff engine

October 2026 PowerShell 5.1+ Windows 11 Homelab Study Project

A local-first OSINT and breach-monitoring pipeline that finds where a person's identity is exposed online, reports only what is new, and emits Splunk-ready JSON events.

OSINT PowerShell Splunk Threat Intel HIBP v3 Sherlock Diff Engine Windows 11

View on GitHub All Projects

The Problem

Exposure is hundreds of small leaks

A person's online identity is scattered across social platforms, data broker databases, and breach dumps — each exposure individually minor, collectively significant. Commercial removal services exist but operate as black boxes: they take a monthly fee, provide vague dashboards, and give no visibility into what they actually found or removed.

This project builds that visibility layer locally. No data leaves the machine, no subscription, no black box.

How It Works

Pipeline overview

Stage 1

Collect

Sherlock scans targets.txt usernames. HIBP v3 checks emails.txt for breaches.

Stage 2

Normalise

PowerShell orchestrator maps raw hits into consistent indicator objects with flat field names.

Stage 3

Diff

Baseline diff engine drops anything already seen. New indicators get appended to the baseline file.

Stage 4

Emit

One JSON event per new indicator written to splunk_exposures.json for Universal Forwarder pickup.

Stage	Input	Tool / Process	Output
Collect	`targets.txt`, `emails.txt`	Sherlock, HIBP v3 API	Raw hits (URLs, breach names)
Normalise	Raw hits	PowerShell orchestrator	Indicator objects (7 fields)
Diff	Indicators + baseline	Diff engine	Net-new indicators only
Emit	Net-new indicators	File writer	JSONL → `splunk_exposures.json`

Design Decisions

Why it works this way

Local-first: targets.txt and emails.txt never leave the machine. No cloud service sees the target list.
Noise control via diff engine: only net-new indicators trigger an event. Re-running the pipeline on the same data produces no output.
HIBP key in a Windows user environment variable: the API key is never stored in code or committed to the repository.
Allow-list .gitignore: only explicitly allowed file types can be committed — personal data files cannot be accidentally staged.
Flat, consistent field names: simple SPL queries require no field extraction or eval gymnastics.

Sample Event

Splunk-ready JSON output

Each new indicator is written as one line of JSON. The values below are synthetic — no real usernames, emails, or scan output are stored in this project.

// splunk_exposures.json — one line per new indicator (synthetic sample)
{
  "timestamp": "2026-10-15T01:42:25Z",
  "host":      "WORKSTATION",
  "event_type": "new_footprint_exposure",
  "status":    "Found",
  "source":    "Sherlock",
  "indicator": "https://example.com/exampleuser",
  "action_required": "Review / Remediate / Opt-Out"
}

Event fields:

Field	Description
timestamp	ISO 8601 UTC time the indicator was detected
host	Machine running the pipeline
event_type	Always `new_footprint_exposure` for net-new hits
status	`Found` (Sherlock match) or breach name (HIBP)
source	`Sherlock` or `HIBP`
indicator	URL (Sherlock) or breach name (HIBP)
action_required	Human-readable next step

Splunk Integration

Universal Forwarder + SPL

The Universal Forwarder monitors splunk_exposures.json and ingests each new line into the footprint index.

inputs.conf stanza

[monitor://C:\Users\<you>\footprint_system\splunk_exposures.json]
index      = footprint
sourcetype = _json
disabled   = 0

SPL — exposures by source

index=footprint event_type=new_footprint_exposure
| stats count by source
| sort - count

Limitations & Roadmap

Honest scope

Current Limitations

Detection only — no opt-out automation yet
Two sources: Sherlock and HIBP
Sherlock produces false positives
Windows-first; no Linux port yet
No Splunk dashboard shipped yet

Roadmap

Holehe integration (email → platform checks)
HTML / PDF run report
Splunk dashboard export
Opt-out templates (Australian Privacy Act APP 12/13, GDPR Art. 17)
Removal tracker
Bash port for Linux with scheduled re-scans
Unit tests for diff logic

Responsible Use

Ethical scope

Authorised use only. This tool is designed for use on identities you own or have written permission to assess. Running OSINT or breach lookups against individuals without their consent may violate privacy law.

Built with AI assistance (Claude); reviewed and run by the owner.

Back to all projects

DigitalFootprint Finder