Questions tagged [google-bigquery]
Google BigQuery is a Google Cloud Platform product providing serverless queries of petabyte-scale data sets using SQL. BigQuery provides multiple read-write pipelines, and enables data analytics that transform how businesses analyze data.
google-bigquery
26,209
questions
0
votes
0
answers
6
views
Using the Confluent BigQuery sink Kafka connector, why are my messages/records not being written to new BQ tables, but existing tables are?
I am running Kafka Connect locally using Docker, but also in AWS MSK. Both are producing the same results.
My connector configuration is as follows:
{
"name": "connector-name",
&...
- 440
1
vote
0
answers
5
views
Creating a table in BigQuery with mixed data types
I have a table in BigQuery I'm trying to create in BigQuery that looks as such:
state
action_date
Alabama
None
Alaska
10/21/24
Delaware
10/28/24
District of Columbia
10/25/24
Florida
10/26/24
...
- 656
0
votes
0
answers
11
views
Error Loading ORC Files into BigQuery: "Invalid timestamp value" after 17th July 2024
I've been successfully fetching ORC files from an Azure Blob Storage to a GCS bucket using the Storage Transfer Service. This process handles around 100 files. After loading the files into GCS, I use ...
- 37
0
votes
0
answers
9
views
bigquery data transfer service is not syncing with Teradata extraction in Python
The need is to extract the data from Teradata to bigquery using bigquery data transfer service BQ DTS in my python code.
what I faced the problem here is I am unable to pass teradata connection ...
- 1
0
votes
1
answer
25
views
BigQuery Error about SELECT list expression references column date which is neither grouped nor aggregated at []
Recently I have been learning SQL and when I tried the following code I got an error, I would like to know what's wrong with the code
SELECT
CAST(date AS date) AS date_only,
CAST(purchase_price AS ...
-1
votes
0
answers
20
views
Snowflake vs BigQuery – How to Choose the Right Cloud Platform [closed]
When deciding between Snowflake and BigQuery for your cloud data warehousing needs, it's essential to consider factors such as performance, scalability, and pricing. Snowflake offers a multi-cluster ...
- 1
0
votes
0
answers
11
views
Extract part of text from a variable declared Bigquery regexp_extract
I have a declared variable in bigquery with words that I want to identify in a table
DECLARE words_to_find DEFAULT r'(CASA|PERRO|CORAZON|GALLETA)';
WITH tabla AS
( select 1 as row_id, 'QUIERO IR ...
- 1
0
votes
0
answers
10
views
How to handle new fields in a STRUCT when dynamically adding using union on all tables in BigQuery
Process
In BigQuery, Google Analytics data is linked to a BigQuery project. Each site corresponds with a dataset, and each day an events_* table is added to that dataset. My query dynamically iterates ...
- 423
0
votes
0
answers
34
views
How to reduce decimal places to just 2?
I have this query in BigQuery
SELECT Id,
AVG(VeryActiveMinutes) AS avg_very_active,
SUM(VeryActiveMinutes) AS total_very_active,
SUM(SedentaryMinutes) AS total_sedentary
FROM `...
0
votes
0
answers
35
views
DETERMINISTIC_DECRYPT_STRING failed: Keyset deserialization failed: Error reading keyset data: Could not parse the input stream as a Keyset-proto
I have a data encryption key encrypted (DEK) that was used to encrypt some values using the algorithm AES256 and the encrypted values are in a BigQuery table.
I am using the KMS provided by Google to ...
- 59
0
votes
1
answer
12
views
Creating bigquery tables with expiration option leads to unexpected expiration time
I'm creating a table in GBQ with an expiration timestamp as following:
CREATE OR REPLACE TABLE
`myproject.mydataset.testtable`
OPTIONS (expiration_timestamp=TIMESTAMP(DATE_ADD(CURRENT_DATE(), ...
- 1,646
1
vote
1
answer
53
views
BigQuery Data transfer crashes with error INVALID_ARGUMENT: Creating table with tags is not supported
I have crashlytics being synchronized in BigQuery and I have a Data transfer job configured to copy its data from US to EU region. It was working great for years and now I got an error, since 1 week.
...
- 73
0
votes
1
answer
38
views
What would be the best strategy if you need to partition a table by a field, but the resulting number of partitions exceeds the limit?
Ok, so I have the following case:
I am using incremental strategy (using dbt and airflow) to feed data to a BigQuery table, BigQuery has a limit of 10,000 partitions, so I have the table partitioned ...
- 339
0
votes
1
answer
29
views
SQL - Czech character encoding from UTF-8 HEX
Does anyone know if there is a function that converts URF-8 HEX characters included in search queries back to the original encoding, for example:
https://www.bambule.cz/vyhledavani/?search=hasi%C4%8Di+...
- 11
0
votes
1
answer
18
views
Creating query for count of rows in a 24 hour window with window_start_dates
I have a table of rows in bigquery that has a userid and timestamp field createddate. I want the query to output userid, window_start_date, count(*) grouped by userid, window_start_date. The ...
