All Questions
710
questions
1
vote
2
answers
61
views
What is the most suitable data type for a column in a table?
Currently I'm trying to develop a PostgreSQL database schema, it has only 2 tables, one of which contains usernames. The difficulty is that for certain reasons I cannot store the username directly, so ...
- 1,245
0
votes
0
answers
33
views
which database is suitable
I have a postgres as main database. Couple tables are huge, they store approximately 5 million rows. So my query usually contains couple joins which are super expensive, so I decided to denormalize ...
- 23
0
votes
0
answers
30
views
How do I design data structures and choose technologies for reporting?
Problem Summary
I have multiple different applications which collate data and currently output different data files (CSV, JSON, XML). I have several reports that I am building in Power BI, which use ...
- 116
0
votes
0
answers
26
views
I have survey data with responses are encoded in numbers and the actual datamap file where the code number & actual value is present in another file
I have survey data with responses are encoded in numbers and the actual datamap file where the code number & actual value is present in another file. I want to analyse this data how to combine ...
0
votes
1
answer
48
views
Create a table in postgress on jupyter results in syntax error?
I try to create a table in postgress in my jupyter lab!
but there comes this bug!
%%bash
cat <<"EOF" | sudo docker exec -i dbis-postgres-db-1 psql -U postgres
DROP DATABASE IF ...
0
votes
0
answers
102
views
Best solution for storing 10,000,000+ lines of json objects which needs to be checked (new data inserted) daily
I have a big zip file which contains around 10,000,000+ lines of json objects. Every day I receive a new zip file which contains mostly exactly the same data but some of it (maybe around 2%) is ...
- 1
0
votes
1
answer
106
views
Common table expression (CTE) in Apache Cassandra
I had a SingleStore database and I decided to switch to Apache Cassandra, transferred all the tables with data, and decided to test the queries. Cassandra doesn't understand "WITH" Clause.
...
0
votes
0
answers
23
views
database design for react native application with in app purchase
I want to design database schemas for react native applications that have in app purchase and free with ads. Can you help me or give me a link to a document or book about that issue? I'm so grateful
...
0
votes
1
answer
41
views
Does a compound index where the prefix field is unique make any sense?
Let's take the field _id, which is unique across the collection.
Does it make any sense to have any compound index where _id is the prefix eg:
{
_id: 1,
A: 1
}
Would the index above be any more ...
0
votes
1
answer
94
views
Do transactions happen only in SQL database in Spring Boot projects?
I'm learning about SQL and NoSQL databases in Spring Boot projects. And I have a question about transactions in databases. I know that when you use @Transactional at a method, all the queries from ...
- 1,040
0
votes
2
answers
764
views
Is $lookup from MongoDB slower than a join in a SQL database, with indexes involve?
db.inventory.aggregate( [
{
$lookup:
{
from: "order",
localField: "_id",
foreignField: "item_id",
as: "...
0
votes
0
answers
18
views
Algorithm for finding top-k similar nodes in database
I'm trying to choose the right database and schemas for the following problem:
There are millions of type A nodes and type B nodes in the system. A and B nodes are disjoint and don't have direct ...
- 6,470
0
votes
1
answer
84
views
Data modelling - Nested object vs Separate DB Table
I'm designing Sticky notes Management Application (CRUD), where StickyNote has 3 attributes, heading, summary and task list. The task has 2 attributes - heading and summary.
A sticky note can have ...
- 1,096
0
votes
1
answer
574
views
How to write SQL Query in Azure cosmos DB to find search pattern of any user
Below is a sample item from azure cosmos DB container, I'm trying to write sql to get below expected output but unable to get it, can you help to write correct sql please. your help is greatly ...
- 29
0
votes
1
answer
228
views
Cosmos DB NoSQL - Make a SQL SELECT Query that nests documents, where documents are flat in the database
I'm looking for a SELECT query that, given a familyID, uses a Subquery (https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/query/subquery) that can get me an array result of Families with nested &...
- 525
-1
votes
1
answer
83
views
Querying JSON key with values as JSON in SQL Server
I have these JSON:
Application 1:
{
"business_industry": "Agriculture",
"docs": [
{
"internal": false,
"type": "Asset & ...
- 2,710
1
vote
2
answers
2k
views
The best choice for database for large datasets [closed]
I'm about to start a new project which should have a rather large database.
the records have roughly up to 10 fields , these datas will have following specifications :
Only insert and read records ...
- 1,429
0
votes
1
answer
73
views
what is the best way to store alerts where many users may subscribe to an alert?
I have a system which responds to different types of events:
Type1 events
Type2 events
Type3 events
...
The purpose of the system is to give users alerts for the types they subscribe to.
I also have ...
- 139
1
vote
1
answer
195
views
Couchbase n1q1 query to fetch map's specific key
Below is the couchbase doc
`
{
"docId": "1111212",
"user": {
"id": "5050017",
"name": "string",
"type"...
- 901
0
votes
0
answers
27
views
If ObjectId is stored in _id field, is there any reason to still have a timestamp field?
THe ObjectId captures the date and time the document is created. So what reason is there to still have the timestamp field?
0
votes
1
answer
86
views
Should I use NoSQL for a project management app like Citrix Podio?
I'm thinking about developing an app similar to Podio, where you can create different "Apps" with personalized fields. I understand that this couldn't work on a SQL database because the ...
- 59
0
votes
1
answer
578
views
Use of secondary indexes in a redis database in comparison with SQL statements
I'm working with a redis database. I have already implemented Python code to access the redis server. The problem is that the code implemented is complex and not easy maintainable.
Secondary indexes ...
- 2,076
0
votes
1
answer
361
views
Designing relations in Microservices architecture
I am new in microservices architecture and I am a bit confused about designing relations between data of each Microservice. So I think about design as its on picture bellow:
So my question is:
Is it ...
- 53
0
votes
1
answer
121
views
2 different not shared databases for the same microservice good approach?
Context:
Microservices architecture, DDD, CQRS, Event driven.
SQL database.
I have an use case, where I have to store a record when a entity state is updated. I'm afraid that the quantity of records ...
- 39
1
vote
1
answer
69
views
Architecture for a large number of entries with many columns searchable by range
The database consists of one main table with one million entries each with approximately 100 fields.
Most fields are numeric and searching will be by ranges, not exact values.
It will be possible to ...
- 351
0
votes
0
answers
238
views
How to retrieve jsonb data from db using spring jpa?
When interacting directly with a sql db (that contain a jsonb column), the query required is:
SELECT stuff FROM alerts WHERE id = '12345'
AND (stuff->'action'->>'actionId' = '123')
ORDER BY (...
- 217
1
vote
0
answers
148
views
DynamoDB Modelling: Find all entities that match search term against various entity attributes
I am new to DynamoDB and the world of NoSQL in general, and although I have purchased and read Alex DeBrie's DynamoDB book, I still couldn't figure this out.
Scenario:
I want to build an app with one ...
- 15k
0
votes
1
answer
343
views
dynamodb composite primary key design with multiple attributes
I am trying to design a dynamodb table to store data, in which multiple attributes constitutes as unique key. In this case how do I design a DynamoDBHashKey/primary key?
Currently I am thinking of ...
- 437
0
votes
1
answer
72
views
Database for substring search
This is not full text search!
I have approximately 1 billion of rows, the core column is a string with medium length of 13 chars and I want to search with a substring as filter in any positions (left,...
- 1
-1
votes
1
answer
138
views
Draw ERD for a project using SQL + NoSQL databases [closed]
For my graduation project, I need to draw a ERD to combine 2 databases,
SQL for authentication and NoSQL for other data like articles and posts and tasks.
How can I specifically display relationships ...
0
votes
0
answers
79
views
Struggling with SQL vs NoSQL
I've got years of experience with SQL. I'm starting on a side project with a fairly simple (to me) data model. I've read a number of posts, articles, guides, etc. over the last few weeks and am trying ...
- 440
0
votes
3
answers
111
views
Adequate database structure for mixed data
I have to create a database for sensor measurements. I have scalar data like date, serial number, firmware version, etc., but I also have array data, basically sensor data in the form of a 1000-...
- 13
0
votes
1
answer
132
views
NoSQL or SQL database for thousands of rows / items in array every day?
I would like to keep data about number of views of cca 2.000 videos per day in DB. But let's say I would like to keep the data from the whole year. It would then be 730.000 rows in SQL DB, which I ...
- 89
1
vote
1
answer
267
views
How can I find matching elements in an array using CouchDB's Mango selector?
How can I find selected only elements included value during array elements using CouchDB's Mango API?
I tried $elemMatch, but I didn't get the result I wanted
Document
{
"id": &...
- 29
1
vote
1
answer
135
views
How to get a table name inside a pg query with json?
I'm trying to get a json with a table name inside with a row, I've trying:
Create a table:
create table if not exists customer (
_id serial,
first_name varchar(50),
constraint pk_customer ...
-1
votes
1
answer
511
views
Redis sort by score and date
I'm trying to strategies how to calculate position/rank based on score and date
I have a two pieces of data with the same score shown in the image below.
If i run the command
$ZADD usersScores 11 ...
- 649
1
vote
1
answer
44
views
Cassandra table data modeling
I just started to learn about Cassandra and I have a pretty specific question.
After looking at some videos and tutorials in my understanding there is 1 primary that includes partition keys and ...
- 25
1
vote
2
answers
2k
views
Filtering a Snowflake SQL SELECT statement
I'm trying to find a good maybe subquery strategy to select a value within a document by matching another value, with multiple documents. I can execute a query, but not in a scalable methodology for ...
- 21
3
votes
1
answer
2k
views
How to handle DB connection in Go when using Repository pattern?
Let's assume I have a very simple repository interface that only reads from target database:
type UserRepository interface {
read(ctx context.Context, id WHAT_TYPE_I_SHOULD_USE_HERE) models.User
}
...
- 6,710
1
vote
1
answer
116
views
How to include the result of the calculated column in another calculation with Iforels
I build a business automation app with Iforels. They allow using of custom formulas in a "function" field. The question is how to use values from the related Units if the value of the ...
- 6,358
-1
votes
2
answers
67
views
Choose the right Database for IoT
I'm wondering if someone in the community could help me to choose the right database for my use case. I have IoT data also, I expect to have about 2000(min) or 20,000(max) lines per hour. Which ...
- 69
-2
votes
1
answer
123
views
What Would Be a Best Practice to Uniquely specify an Entry With a Boolean Flag in the Database?
Let's assume that we have a collection (or table) that is called students, and in our system, we need to persist the best student of all time, which is going to be one and only one among all the ...
- 11
1
vote
3
answers
3k
views
SQL vs NOSQL : Explanation for few points
After reading more than 100 documents on SQL vs NOSQL, I still have two questions.
1). How latency of NoSQL DB is better than SQL DB?
2). How NoSQL DB's are auto scalable?
3). How NoSQL DB stores ...
- 1,182
0
votes
1
answer
967
views
Designing tags system with nosql/elastic search
I have to design a system with this schema.
{
"documentId" : 123
"documentType" : "paper"
"tags" :["abc","xyz"]
//other meta data of ...
- 387
0
votes
1
answer
125
views
How do these database management systems practically behave during a network partition?
I am looking into deploying a database management system, replicated across regions (various data centers across a country). I am currently looking into the following candidates:
MongoDB (NoSQL, CP ...
- 6,570
0
votes
1
answer
54
views
How to create the following data structure in an SQL environment
I have a FireStore database and I need to convert it to SQL. My SeminarsAndTraining document looks like this:
{
"st_name": "SOL Level 1",
"attendance": [
{"...
- 33
0
votes
1
answer
123
views
What sql engine is best in terms of low db size?
I have a script that gathers some data every 5 mins and stores it directly into a sqlite table using pandas to_sql function. So far it gathered around 300k rows and the table size already exceeds 30 ...
- 55
0
votes
1
answer
34
views
How to extract customer and products ordered statistics in web site with python
To get the statistical behavior of buyers and visitors to a store site and process the numbers obtained from the number of visits, number of purchases and other similar items regardless of what ...
0
votes
1
answer
27
views
How to optimize this query in mongodb?
I have a query like below in MongoDB.
a = xml_db.find(
{
'crawl_id': crawl_id,
'low_performer': { '$nin': [source] }
}
).hint('crawl_id_low_performer_index')
print(a.count())
...
- 11k
0
votes
1
answer
33
views
Which database should I use to store in real time my exact location from an Android application and then display it in my website (tracking system)?
I'm kind of new at backend development, and I should to create a website (tracking) display the location of my phone, this location I have to retrieve from an android application.
My question is which ...
