The Role You own the data substrate and the batch runtime of a product we are building from scratch. This is a data-specialist seat, not a general backend seat. Routine application development is guided in weekly reviews; what cannot be substituted is your depth in data modeling, set-based computation, and pipeline correctness. You will work self-directed, writing your own specs and decisions, with your tests and documentation carrying the quality bar between reviews.
Stack: Python, PostgreSQL, Django, server-rendered frontend (htmx).
What You Will Own
Core Responsibilities / Engineering Focus
Deep third-party data ingestion
Build reliable ingestion pipelines for external data sources
Account for silently dropped webhooks through reconciliation with source-of-truth totals
Large-scale batch computation
Use set-based SQL for processing large datasets
Ensure jobs complete within fixed nightly processing windows
Multi-tenant reliability
Maintain strict tenant isolation
Ensure one tenant’s bad data does not impact another tenant’s processing
Backfill & replay capabilities
Design systems to support backfills and replays as standard capabilities
Avoid relying on one-off emergency scripts
Append-only & auditable data
Maintain immutable, auditable records
Ensure every automated decision can be reconstructed and traced
Resilient vendor API integrations
Build for failures and unreliable external systems
Implement retries, circuit breakers and reconciliation mechanisms
Operational reliability
Build strong alerting and monitoring
Handle database partitioning effectively
Design and maintain idempotent jobs

What We Are Looking For
Required
Ideal Candidate Profile

5–8 years of experience building and operating production systems — years are a proxy; the skills below are the actual bar.
Strong PostgreSQL expertise
Schema design
Query planning and optimization
Batch performance at real scale

Production data pipeline ownership
Ingestion
Transformation
Reconciliation
Data serving
Experience should go beyond application CRUD/ORM work.
Strong SQL & data thinking
Thinks in sets, not loops
Multi-step SQL transformations
Window functions
Statistical aggregates, percentiles & distributions
Incremental computation
Experience building cohort, retention or funnel metrics from raw event/order data
Production Python backend experience
Experience with any Python backend framework is fine
Able to keep business logic cleanly separated from framework-specific code
Strong testing mindset
Tests are part of the development process, not an afterthought
Especially experienced with testing batch jobs and reconciliation systems where data errors can remain unnoticed
Production problem-solving / “war stories”
Has dealt with real incidents such as:

Queues backing up
Webhooks silently dropping
Data loss or inconsistencies
Reconciliation catching issues that went unnoticed

Strong written communication
Comfortable with documentation-driven development
Can write clear specs, decisions and technical documentation
High ownership
Comfortable building systems from scratch
Can independently own outcomes with minimal supervision

Good to have
Production Django experience
E-commerce / D2C platform experience
Streaming & event-driven architecture
Kafka or similar systems
Familiarity with probabilistic data structures
HyperLogLog
Bloom Filters
t-digest
~ Comfort with server-rendered applications/screens when required

Data Engineer (FrontEnd)

Start with your email