Alpha 0.1.0

DataOps411

Data pipeline operations. Right now.

Latest milestones

Pipeline upgrade tracker

View tracker

Headlines

8 in this edition
  1. DuckDB 2.0.1 changes what happens when multiple rows match a MERGE

    A target row can be updated repeatedly. DuckDB recommends checking that each target has one matching source row or that all matches supply the same update values.

    DuckDB
    Options
  2. PostgreSQL 19 adds logical replication for sequence values

    A server already running with wal_level set to replica can also enable logical replication without restarting.

    PostgreSQL
    Options
  3. DuckDB 1.4 reaches the end of community support

    The project lists November 17 as the end date for the 1.4 LTS branch. Its 2.0 LTS branch has support scheduled through October 2027.

    DuckDB
    Options
  4. dbt Core 1.12.7 falls back to checking freshness one source at a time

    When a batch metadata check fails, dbt tries the selected sources individually so one failed batch does not take every check with it.

    dbt Labs
    Options
  5. PostgreSQL 14.25 closes out the 14 release line

    Teams maintaining 14.x can apply the final minor update and then plan the separate migration to a supported major version.

    PostgreSQL
    Options
  6. Airflow’s Databricks provider adds an operator for agent invocation

    Version 7.24.0 introduces a hook and operator for invoking Databricks agents from a workflow.

    Apache Airflow
    Options
  7. Airflow 3.4.0 adds bounded loops to task groups

    Workflow authors can set an iteration limit and decide at runtime whether a task group should run again.

    Apache Airflow
    Options
  8. Airflow’s OpenLineage provider changes SQL hook job names

    Version 2.21.0 adds hll_query to hook-generated names. Lineage dashboards that match the old naming pattern need an update.

    Apache Airflow
    Options