Questions tagged [pandas]
Pandas is a Python library for data manipulation and analysis, e.g. dataframes, multidimensional time series and cross-sectional datasets commonly found in statistics, experimental science results, econometrics, or finance. Pandas is one of the main data science libraries in Python.
pandas
287,993
questions
0
votes
1
answer
31
views
Filter datetime column with object as data type in python
I have a df with a Timestamp and Value columns. Both have the 'object' dtype
| Timestamp | Value |
--------------------------------------
| 8/21/2023 12:00:00 AM | a |
| 11/...
1
vote
2
answers
33
views
Converting JSON list with multiple nested dictionaries to csv or excel
I have a JSON that I download from a website that has multiple nested dictionaries inside the main list. This is a very simplified version of it.
[
{
"id": 1,
"...
- 11
0
votes
1
answer
25
views
Series is empty when using .loc to slice
enter image description here
I'd like to get the item between Q1 to Q9. Q1 to Q9 exist and also they are sequential. As shown in the picture, I used .loc to slice the series object:
s.loc['Q1':'Q2']
...
1
vote
2
answers
34
views
Map Dataframe Column Values Based on Two Dictionaries Conditionally [duplicate]
I have a dataframe df_test. I want to map the column color conditionally:
if category is 'tv', then map using the tv_map dictionary
else map using the radio_map dictionary
I could split df_test by ...
- 727
3
votes
2
answers
56
views
How do I get variable length slices of values using Pandas?
I have data that includes a full name and first name, and I need to make a new column with the last name. I can assume full - first = last.
I've been trying to use slice with an index the length of ...
- 65
2
votes
0
answers
51
views
How to compare rows within the same csv file faster
I have a csv file containing 720,000 rows with and 10 columns, the columns that are relevant to the problem are ['timestamp_utc', 'looted_by__name', 'item_id', 'quantity']
This File is logs of items ...
- 31
0
votes
0
answers
20
views
Dataframe replace columns and save to new df
I loaded in a .dat file to a pandas dataframe. Two of the columns are mean and error. I used the values in these two columns to create a randomized value for mean. I want to replace the mean column in ...
0
votes
1
answer
26
views
Read/Write pipeline in pandas
I'm building a simple pipeline to pull a dataset from a database and write it to a csv so that I can access the data more quickly in the future.
Currently, I have this:
# data loading as a pipeline
# ...
- 305
0
votes
2
answers
56
views
How to use a for loop in storing multiple data frames with slightly different columns?
In my directory, I have a list of 9 txt files. Without manually running pd.read_table() for each of them, I would like to efficiently run a loop for example and save a data frame for each file. Note, ...
- 1,065
-1
votes
0
answers
33
views
How do i search two files to know if the values are similar or not
I have two CSV files that I want to compare their contents and print the contents that doesn't have the same data. Most contents will have the same name but not all their data are accurate. I want to ...
- 109
0
votes
0
answers
15
views
Darts (Python) TiDE model gives irrational results when switched to probabilistic mode
Let's consider this simple code:
import yfinance as yf
import pandas as pd
from darts import TimeSeries
import darts.models as dm
import darts.utils.likelihood_models as dl
import lightning.pytorch as ...
- 741
0
votes
0
answers
24
views
How to calculate the Relative Strength Index (RSI) through record iterations in pandas dataframe
I have created a pandas dataframe as follows:
import pandas as pd
import numpy as np
ds = { 'trend' : [1,1,1,1,2,2,3,3,3,3,3,3,4,4,4,4,4], 'price' : [23,43,56,21,43,55,54,32,9,12,11,12,23,3,2,1,1]...
- 1,392
0
votes
1
answer
28
views
Pandas apply is turning matrices into nan/None
I am running the following code on a dataset trying to tally the rows from one data set that match a varied set of criteria. I am using the apply function to store this tally inside of a matrix, where ...
1
vote
1
answer
33
views
Filtering a pandas series consisting of lists and NaN values if the elements contain a string
Let's consider this dataframe:
temp = pd.DataFrame({'x': [['ab', 'bc'], ['hg'], np.nan]})
temp
x
0 [ab, bc]
1 [hg]
2 NaN
I'd like to create a new column called dummy that takes the value ...
- 1,969
0
votes
0
answers
30
views
Importing xlsx data from Pandas to SQL Server, receiving string data, right truncation and text column data incomplete
I have an xlsx file that I'm trying to run through Python to edit and then upload to our database.
For vendor A, their data comes in through a CSV and I've been able to edit and upload that no issue.
...
