Questions tagged [dbt]
Questions relating to dbt (data build tool) set up and usage. dbt is an open-source command line tool that enables data teams to transform data.
1,493
questions
0
votes
0
answers
23
views
DBT:How to get macros executed in overview.md which is used to overide the dbt docs main page
Requirement: To get datetimestamp in the main page of the dbt docs site , so that we know when it was last refreshed
Code:
Have created a macro and the macro is been called in the overview.md
macro
{% ...
- 966
0
votes
0
answers
49
views
DBT - Forcing dependencies dynamically when referencing a large number of tables
I have a set of 35 models corresponding to objects of different types in my staging layer which I union in my intermediate layer. For the sake of this post I have replaced the data source that I am ...
- 1,163
1
vote
1
answer
64
views
change column names using a dbt macro
Can I use a dbt macro to modify the column names in AWS redshift?
Example changes to be made:
"checkedAt" --> checked_at
"helloWorld" --> hello_world
"testKitchenNew&...
- 3,198
0
votes
1
answer
31
views
I can't connect dbt to two databases | cross-database
I have a problem regarding DBT. I created some tables in POSTGRESQL:
select * from libraria.main.usuario u;
select * from libraria.main.endereco and ;
select * from libraria.main.telefone t;
select * ...
0
votes
1
answer
14
views
How can I include a project's folder structure in my dbt Cloud project's DAG?
I'm looking for a way to include folder structure and other available metadata into my dbt Cloud project's DAG. While the lineage is clearly demonstrated in the DAG generated, I'm looking for a way to ...
0
votes
2
answers
51
views
DBT Duplicate row detected during DML action - Snowflake
Creating an incremental model using the MERGE strategy with a unique key. The unique key to merge on is a surrogate key of columns from the table.
When I run this incremental model I get the error: ...
- 21
1
vote
1
answer
60
views
Create a Sql Statement IN DBT macro and use the query in the dbt model
my_macro.sql
{macro my_macro(value)}
{% set query %}
select A from value
{{ return(query) }}
{% endset %}
{% endmacro %}
model.sql
select * from B
where A in ({{ my_macro(value) }}
when i ...
- 13
0
votes
1
answer
41
views
I want to use the condition union if exists with a table referred with {{this}} in snowflake SQL
In dbt, I want to append table if it exists. Basically, I am building an incremental model where I find out lagged values from previous day, for the first run, I want to union today's data along with {...
- 91
0
votes
1
answer
36
views
How do I utilize the fivetran/zendesk models from its dbt_package?
I am new to dbt packages and I am stuck trying to get the models from the dbt package 'fivetran/zendesk' to materialize.
https://hub.getdbt.com/fivetran/zendesk/latest/
As per the documentation, I ...
- 21
0
votes
1
answer
32
views
Issues with DBT Compilation and Jinja
I'm running into issues with my DBT set up. Aafter setting up some linters and formatting checks, it won't run, and the error is quite vague:
Unable to do partial parsing because of a version ...
0
votes
0
answers
24
views
ClassCastException in Spark SQL Incremental Load with DBT
I'm encountering a ClassCastException error when running an incremental load using DBT and Spark SQL. The error message indicates an issue with casting in the Spark execution plan:
org.apache.hive....
0
votes
0
answers
43
views
Fetching return value of dbt macro in airflow
I am having a dbt macro which is defined as follows :
As you can see, I am returning the result from the dbt macro.
`
{% macro return_hello() %}
{% set result = 'Hello, dbt!' %}
{{ return(...
0
votes
0
answers
58
views
Is it a good practice to use update_timestamp for incremental loading in DBT with insert_overwrite strategy?
Suppose I have a dbt/bigquery model that reads from a simple upstream model partitionned by day using "partition_date" and clustered by "update_timestamp" and that would look like
...
- 11
0
votes
1
answer
66
views
Is changing date partitionning granularity a breaking change?
In Bigquery, suppose I create a table and partition it by a date column "mydate" with a "DAY" granularity.
Using DBT, this can be done using :
partition_by = {
"...
- 11
0
votes
0
answers
46
views
Complex loop calc in dbt + postgres
I need to recursively/cyclically calculate data by months with intermediate saves to the database, as the next period will be calculated/saved based on these data.
Details:
We have data based on ...
