BigQuery and Apache Iceberg

dbt supports materializing Iceberg tables on BigQuery via the catalog integration, starting with the dbt-bigquery 1.10 release.

Creating Iceberg Tables

dbt supports creating Iceberg tables for two of the BigQuery materializations:

• Table
• Incremental

BigQuery Iceberg catalogs

BigQuery supports Iceberg tables via its built-in catalog BigLake Metastore today. No setup is needed to access the BigLake Metastore. However, you will need to have a storage bucket and the required BigQuery roles configured prior to creating an Iceberg table.

dbt catalog integration configurations

The following table outlines the configuration fields required to set up a catalog integration for Iceberg tables in Snowflake.

Field	Required	Accepted values
name	yes	Name of catalog integration
catalog_name	yes	The name of the catalog integration in BigQuery. For example, biglake_metastore.
external_volume	yes	gs://<bucket_name>
table_format	yes	iceberg
catalog_type	yes	biglake_metastore
file_format	optional	default,parquet

dbt has an additonal configuration: storage_uri that the user can use on the model configuration to override the catalog integration path to supply the entire storage_uri path directly.

Configure catalog integration for managed Iceberg tables

1. Create a catalogs.yml at the top level of your dbt project.
   
   An example:

catalogs:
  - name: my_bigquery_iceberg_catalog
    active_write_integration: biglake_metastore
    write_integrations:
      - name: biglake_metastore
        external_volume: 'gs://mydbtbucket'
        table_format: iceberg
        catalog_type: biglake_metastore

2. Apply the catalog configuration at either the model, folder, or project level:

iceberg_model.yml

{{
    config(
        materialized='table',
        catalog_name = my_bigquery_iceberg_catalog

    )
}}

select * from {{ ref('jaffle_shop_customers') }}

3. Execute the dbt model with a dbt run -s iceberg_model.

Limitations

BigQuery today does not support connecting to external Iceberg catalogs. In terms of SQL operations and table management features, please refer to the BigQuery docs for more information.