All Questions
Tagged with snowflake-cloud-data-platform pandas
160
questions
0
votes
0
answers
39
views
Issues with SQLAlchemy
I'm trying to run the following code (some of the stuff I blind, like the passwords)to get some Snowflake data into a Jupyter notebook:
import math as math
import numpy as np;
import pandas as pd;
...
0
votes
0
answers
80
views
Error 100357 (P0000): Returned Series has a different number of elements than the input
Hi I am using a vectorized python udf - https://docs.snowflake.com/en/developer-guide/udf/python/udf-python-batch
I am making use of a snowflake app and have this code in the udf:
@vectorized(input=...
- 642
0
votes
1
answer
415
views
snowflake-connector-python[pandas] write_pandas creates duplicate records in table
I am attempting to copy data into snowflake on an AWS Lambda. I have a situation right now where I have a dataframe that has no duplicates in it. I verify this by checking my dataframe like so:
df....
- 3,524
1
vote
1
answer
670
views
Snowflake Pandas Error: AttributeError: 'Engine' object has no attribute 'cursor'
I'm trying to read data from a Snowflake table using Python (pandas). When I execute the following code:
from snowflake.sqlalchemy import URL
from sqlalchemy import create_engine
import pandas as pd
...
- 521
1
vote
1
answer
293
views
Using pandas.to_sql() and getting TypeError: not all arguments converted during string formatting
I am trying to load a pandas DataFrame to a Snowflake database. I am using sqlalchemy to create the engine. Based in what other StackOverflow answers have suggested I am using with engine.begin() to ...
- 11
0
votes
0
answers
89
views
Dataframe to Snowflake -> error with the library snowflake connector (AttributeError: 'NoneType' object has no attribute 'get')
I'm trying to uplaod a dataframe to snowflake, however I get an Attribute error when I'm uploading it.
My dataframe:
Type
VALUE
MATRICULE
CATEGORIE
LINE
DATE
0
A
100
1
Base
3
2023-11-01
1
B
Nan
1
...
- 270
0
votes
0
answers
25
views
Is it possible to tell Pandas to NOT escape single quotes in JSON data [duplicate]
I'm trying to use df.itertuples to loop over each row and insert in to Snowflake. I was originally trying to use df.to_sql(), but this wasn't creating the table properly so my VARIANT type columns ...
- 1,564
0
votes
1
answer
421
views
Casting None to snowflake table in a TIMESTAMP_NZT column giving expression type error
I am using snowpark and pandas to add a dataframe to snowflake table.
My table looks like
ID column2 .......... Timestamp_column
None
2023-09-...
- 93
0
votes
1
answer
125
views
Operations on groups in Snowpark
I'm applying a function on a grouped dataframe in Pandas.
Has anyone had to convert this into the Snowpark API. If so, what were the functions you used?
- 2,467
1
vote
1
answer
1k
views
How to remove double quotes in all Snowpark DataFrame column names?
I'm trying to save my Pandas DataFrame into Snowpark DataFrame with this code:
pdf = pd.DataFrame(...)
pdf.columns = pdf.columns.str.replace(' ', '')
sdf = session.createDataFrame(pdf)
sdf.write....
- 985
0
votes
1
answer
575
views
Error Ocurred while saving Snowpark DataFrame (Converted From Pandas DataFrame) using DataFrame.write.mode("append").save_as_table("TABLE1")
I have read csv file using streamlit app using pandas DataFrame, the data is in Pandas DataFrame and the objective is to save this Pandas DataFrame into Snowflake Table.
There are Timestamp columns in ...
0
votes
1
answer
318
views
snowflake unload to S3 as parquet has no column names nor correct datatypes
The following produces a parquet file in S3:
USE DATABASE SANDBOX;
USE SCHEMA SANDBOX;
CREATE OR REPLACE FILE FORMAT my_parquet_format
TYPE = parquet;
COPY INTO @bla/x_
FROM (
SELECT
...
- 17.3k
1
vote
0
answers
83
views
Loading int16 data with NULLs from snowflake into a pandas DataFrame
I am looking for an automated way to fetch data that doesn't fit into numpy datatypes (e.g. nullable ints) from a snowflake db (using the snowflake connector).
E.g. I have a query that loads a column ...
- 1,779
1
vote
1
answer
915
views
Failed to import ArrayResult. No Apache Arrow result set format can be used. ImportError: No module named 'snowflake.connector.snow_logging'
I generated one .exe file from my python script using pyinstaller --onefile file.py with Pycharm. By executing the generated .exe file I get the error:
Failed to import ArrayResult. No Apache Arrow ...
1
vote
1
answer
540
views
SQL compilation error invalid identifier 'COLUMN_NAME' when using df.to_sql()
I'm working on some code to push a dataframe to snowflake using the code:
import pandas as pd
from snowflake.connector.pandas_tools import pd_writer
from sqlalchemy.types import String
from snowflake....
- 47