Data Science, Machine Learning & AI Wiki

Feature Pipelines

Properties7
descriptionPipelines that compute model features with point-in-time training and serving semantics.
areadata-engineering
topicsfeature-pipelines
levelintermediate
statuscomplete
page_typesystem-design
aliases

2 min read

Feature Pipelines

Feature pipelines compute model inputs from operational or analytical data. Their special contract is point-in-time correctness: training features must include only information available before the label or prediction timestamp, while serving features must use the same definitions under lower latency.

Point-in-time correctness

This SQL example shows the leakage boundary. The transaction at 2026-01-10T02:00:00 happens after the label timestamp and must not be included:

WITH labels(entity_id, label_ts) AS (
  VALUES (7, '2026-01-10T00:00:00')
),
transactions(entity_id, event_ts, amount) AS (
  VALUES
    (7, '2026-01-08T12:00:00', 30),
    (7, '2026-01-09T20:00:00', 40),
    (7, '2026-01-10T02:00:00', 999)
)
SELECT
  l.entity_id,
  sum(t.amount) AS spend_before_label
FROM labels l
LEFT JOIN transactions t
  ON t.entity_id = l.entity_id
 AND t.event_ts < l.label_ts
GROUP BY l.entity_id;

Result:

entity_id  spend_before_label
7          70

The query joins transactions only when event_ts < label_ts, so the feature is . The later transaction for 999 is excluded because it occurs two hours after the label timestamp; without the timestamp predicate, the feature would be and would leak future behavior into training.

Architecture

Feature pipelines sit between data-pipelines and training-pipelines. Batch features can be built in a warehouse with dbt or SQL; low-latency features may require streaming updates from batch-versus-streaming systems into an online store. Dataset-versioning, data-lineage, and data-quality checks must travel with the training dataset.

The same feature definitions must feed both paths; when the offline and online paths diverge, the result is training-serving skew.

flowchart TD
  Data[Curated data from data pipelines] --> Defs[Shared feature definitions]
  Defs --> Batch[Batch path: warehouse or dbt features]
  Defs --> Stream[Streaming path: low-latency updates]
  Batch --> Offline[Offline store for training]
  Stream --> Online[Online store for serving]
  Offline --> Train[Training with point-in-time joins]
  Online --> Serve[Serving under low latency]

Failure modes

Training-serving skew appears when offline SQL uses a different join, window, or fill value than online serving code. Backfills can rewrite historical features unless snapshot identifiers are pinned. Aggregations over late events need explicit watermark behavior, not accidental dependence on ingestion order.

References

Nav

Section — Data Engineering

← Reproducibility

Graph View

Backlinks