cassandra 如何使用DBT在与源数据库不同的目标数据库中生成物化视图?

myss37ts  于 2023-06-22  发布在  Cassandra
关注(0)|答案(1)|浏览(168)

我是个新手。我希望从提供商(特里诺这是连接到cassandra)读取数据,并在其他提供商(如postgres/clickhouse)中编写所有中间视图/表。特里诺不允许重命名表(它没有创建视图的权限,这也在官方文档中描述)。
有什么我错过了。

trino:
      target: dev
      outputs:
        dev:
          type: trino
          method: none
          user: admin
          host: trino.XX.adroot
          port: 80
          schema: demo
          catalog: cassandra
    
    postgres:
      target: dev
      outputs:
        dev:
          type: postgres
          host: localhost
          user: postgres
          password: postgres
          port: 5432
          dbname: demo # or database instead of dbname
          schema: public
  

# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'pmrush_elt'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'trino'

# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"  # directory which will store compiled SQL files
clean-targets:         # directories to be removed by `dbt clean`
  - "target"
  - "dbt_packages"

# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models

# In this example config, we tell dbt to build all models in the example/
# directory as views. These settings can be overridden in the individual model
# files using the `{{ config(...) }}` macro.
models:
  pmrush_elt:
    # Config indicated by + and applies to all files under models/example/
    staging:
      +materialized: table

源数据库

version: 2
sources:
  - name: demo
    catalog: cassandra
    schema: demo
    tables:
      - name: keyword_data
      - name: serp

特里诺的错误消息,解释了为什么我想在另一个提供程序中创建物化视图

15:11  Found 1 model, 0 tests, 0 snapshots, 0 analyses, 316 macros, 0 operations, 0 seed files, 2 sources, 0 exposures, 0 metrics
17:15:11  
17:15:12  Concurrency: 1 threads (target='dev')
17:15:12  
17:15:12  1 of 1 START sql table model demo.stg_keyword_data ....................... [RUN]
17:15:16  1 of 1 ERROR creating sql table model demo.stg_keyword_data .............. [ERROR in 4.87s]
17:15:16  
17:15:16  Finished running 1 table model in 0 hours 0 minutes and 5.11 seconds (5.11s).
17:15:16  
17:15:16  Completed with 1 error and 0 warnings:
17:15:16  
17:15:16  Database Error in model stg_keyword_data (models/staging/demo/stg_keyword_data.sql)
17:15:16    TrinoUserError(type=USER_ERROR, name=NOT_SUPPORTED, message="This connector does not support renaming tables", query_id=20230424_171516_00019_68nw3)
17:15:16    compiled Code at target/run/pmrush_elt/models/staging/demo/stg_keyword_data.sql
17:15:16  
17:15:16  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
jgwigjjp

jgwigjjp1#

看起来你所尝试的是冲突的基本思想dbt作为一个工具的转换工具在英语教学范式。
也许你可以通过将{{ config(database="database_name") }}添加到你的sql文件中来实现。

相关问题