Questions tagged [dataframe]
A data frame is a 2D tabular data structure. Usually, it contains data where rows are observations and columns are variables and are allowed to be of different types (as distinct from an array or matrix). While "data frame" or "dataframe" is the term used for this concept in several languages (R, Apache Spark, deedle, Maple, the pandas library in Python and the DataFrames library in Julia), "table" is the term used in MATLAB and SQL.
dataframe
146,897
questions
0
votes
0
answers
9
views
Dataframe replace columns and save to new df
I loaded in a .dat file to a pandas dataframe. Two of the columns are mean and error. I used the values in these two columns to create a randomized value for mean. I want to replace the mean column in ...
0
votes
0
answers
11
views
how to apply an expression from a column to another column in pyspark dataframe
I would like to know if this possible to apply.
for example i have this table:
new_feed_dt regex_to_apply expr_to_apply
053021 | _(\d+) | date_format(to_date(new_feed_dt, '...
0
votes
0
answers
19
views
How to calculate the Relative Strength Index (RSI) through record iterations in pandas dataframe
I have created a pandas dataframe as follows:
import pandas as pd
import numpy as np
ds = { 'trend' : [1,1,1,1,2,2,3,3,3,3,3,3,4,4,4,4,4], 'price' : [23,43,56,21,43,55,54,32,9,12,11,12,23,3,2,1,1]...
- 1,392
0
votes
0
answers
26
views
Using Polars expressions to apply `eval()` to a column
I would like to achieve the following via Polars expressions, as opposed to mapping the elements row-by-row, but I have not been able to figure out a way.
import polars
def foo():
return 1 + 1
...
- 63
1
vote
1
answer
35
views
How to group rows based on column ID in a pandas dataframe?
I have below the dataframe below df1:
ID Label Value
id_1 A
id_1 B
id_1 C
id_1 D
id_1 E
id_1 10
id_1 20
id_1 30
id_2 F
id_2 G
...
- 25
0
votes
1
answer
56
views
split pandas datafram based on given row string
I have a text file with a data set of the form
Line 1
Line 2
!
1.01499999 0.504999995 6.19969398E-7 5.38933136E-7 1.35450875E-6
1.74000001 0.220000029 7.92876381E-6 4.1831604E-6 6.61433387E-6
2....
- 2,019
1
vote
4
answers
66
views
String Manipulation based on Char Length in a dataframe
I wanted to do some string manipulation based on Char length condition.
I have this table, let's called it sample table.
RiskCode
A01
A02.999
I want to transform the RiskCode column in sample ...
3
votes
1
answer
34
views
Resample ohlc pandas
I have a problem with resample in pandas.
There is a dataframe with the data specified below.
Datetime as an index
I need to generate a resample of a 3-hour interval, even if there is only 2 hours of ...
- 31
1
vote
4
answers
60
views
Shift part of row in dataframe to new row
I have a dataframe (pandas) that I want to transform for displaying purposes.
Therefore I want to shift some parts of the dataframe to new rows like below :
col1 col2 col_to_shift col_not_to_shift1 ...
- 623
1
vote
2
answers
54
views
Interpolate dataframe in R subject to totals
I have the following dataset which has the number of A, B and C both in 2000 and 2005. I now need to inpolate the dataframe subject to the constraint that the sum of A, B and C must be consistent with ...
- 25
-2
votes
0
answers
13
views
i am in the process of using pandasai, but keep getting this error from pandas.compat import is_numpy_dev as _is_numpy_dev
from pandas.compat import is_numpy_dev as _is_numpy_dev # pyright: ignore # noqa:F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
i am running numpy version 2
python version 3.11
pandas 1....
- 71
0
votes
0
answers
31
views
Pyspark Filtering Array inside a Struct column
I have a column in my Spark DataFrame that has this schema:
root
|-- my_feature_name: struct (nullable = true)
| |-- first_profiles: map (nullable = true)
| | |-- key: string
| | |--...
- 392
-5
votes
0
answers
50
views
Transpose tab-delimited dataframe [closed]
I have a tab-delimited .txt file containing:
1 Julia aa bb
8 William dd ee ff gg
which I want to reform this as below:
1 Julia aa
1 Julia bb
8 William dd
8 William ee
8 ...
0
votes
1
answer
30
views
AMBIGUOUS_REFERENCE error when trying to aggregate a dataframe in azure
Have a dataframe with the following columns
year, month, loc_code, usg_type, id_code, usg
trying to aggregate
in SQL it would have been
select year, month, loc_code, usg_type, count(distinct id_code) ...
- 487
-1
votes
0
answers
20
views
`colorNumeric()`: ! Wasn't able to determine range of domain
I am trying to read a shapefile and assign colors using leaflet colorNumeric() to variable V4 which are numeric but discontinuous. It has values 1 to 16 and then 74 to 95 with some missing values in ...
- 508