All Questions
Tagged with snowflake-cloud-data-platform azure
132
questions
0
votes
0
answers
54
views
401 Unauthorized Error with dbt Models on Snowflake After ADF Ingestion
Recently, we implemented a solution where data is ingested into Snowflake using Azure Data Factory (ADF). After ingestion, the ADF data pipeline triggers dbt models using programmatic invocations. We ...
- 501
0
votes
2
answers
98
views
Azure DevOps Pipeline Error: Password Not an Integer in snowsql Command
I'm encountering an issue with my Azure DevOps pipeline where I'm trying to run a snowsql command with a password retrieved from Azure Key Vault. The pipeline retrieves the secret correctly, but when ...
0
votes
1
answer
95
views
Using Azure Logic Apps to validate excel documents and move them to different folders in Sharepoint?
It's been a while since I used Azure Logic Apps but I've got a use case for a client where we need a (stage) landing place for excel data (probably sharepoint) but before we ingest that data into our ...
- 19
0
votes
0
answers
19
views
Spark connectors from Azure Databricks to Snowflake using AzureAD login [duplicate]
I am trying to connect below Spark connectors from Databricks to Snowflake. On the Snowflake side, I can access the account using "Login via AzureAD" which are my Azure account credentials.
...
- 195
0
votes
0
answers
90
views
Effective way to track source deleted records in incremental load in Azure Databricks
Scenario :
The source is snowflake and it is in readonly mode.
Implementing full load, saving maximum timestamp in databricks for incremental load.
But in source side apart from record updates, ...
- 195
0
votes
1
answer
162
views
Incremental Database Copy from Azure SQL DB to Snowflake
I would like to upload a table from Azure SQL DB to Snowflake using Azure Factory.
Right now I am using the pipeline "copy data" with source and destination. The entire Events table (about ...
- 169
0
votes
2
answers
137
views
ADF Script Creating a Table with Parquet format using INFER_SCHEMA in Snowflake
I'm trying to copy data from Salesforce to Snowflake using ADF avoiding use SAS Token, because of that I going to use integration storage on Snowflake.
I'm trying to follow this example: https://...
0
votes
0
answers
62
views
On-prem Postgresql DB to Snowflake Data Pipeline
I am tasked with building a data pipeline from an on-prem Postgresql DB that is leveraged to manage my OLTP transactions to Snowflake on Azure and I am busting my brain on choosing a couple of options ...
0
votes
1
answer
71
views
Sent table from Azure SQL to snowflake
I would like to configure sending data from azure SQL database to Snowflake (And also, as an option to send in the future only changes in the database, not the whole database).
I have created logical ...
- 169
0
votes
1
answer
76
views
What is the best way to take backup snowflake tables as in Azure blob?
I have a requirement to store all the snowflake tables as Azure blob storage(Parquet).
I have identified two ways,
1.Create a external stage and write a procedure to unload all tables one by one ...
0
votes
1
answer
182
views
Azure Data Factory - How to load recently loaded file from a blob into snowflake?
I have below notebook job within the ADF which loads a excell/csv file into a blob storage.
sheetId='33222424'
load_time = today.strftime("%Y%m%d")
path = 'my_path'
smartsheet_client.Sheets....
- 115
0
votes
1
answer
95
views
Azure Data Factory- How to create a table from snowflake view
Below is the scenario
I have two snowflake account - account-1 and account-2
we need to pull the view definition from account-1 and create the respective table in account-2 in snowflake.
Currently we ...
- 115
-1
votes
1
answer
179
views
Load Data from Api to Snowflake
I am trying to copy data from Rest API to Snowflake using data factory pipeline. There are no issues with the configuration. However, when I am trying to publish it getting below error.
any inputs?
- 4,144
0
votes
1
answer
256
views
Authenticating Snowflake Via OAuth from Azure DevOps Pipeline
Running Pipelines in Azure DevOps where we need to authenticate with Snowflake and execute on Snowflake. For different Pipeline there are different Users used to authenticate and run process on ...
0
votes
1
answer
200
views
Snowflake-Databricks- Failing to connect snowflake from Databricks as incoming connection is blocked
I am trying to connect snowflake from Azure Databricks using below code
import snowflake.connector
# Connection parameters
account = 'account'
user = 'user'
password = 'password'
warehouse = '...
- 115