Questions tagged [pandas]
Pandas is a Python library for data manipulation and analysis, e.g. dataframes, multidimensional time series and cross-sectional datasets commonly found in statistics, experimental science results, econometrics, or finance. Pandas is one of the main data science libraries in Python.
pandas
287,995
questions
0
votes
0
answers
8
views
Identify starting row of actual data in Pandas DataFrame with merged header cells
My original df looks like this -
df
Note in the data frame:
The headers are there till row 3 & from row 4 onwards, the values for those headers are starting.
The numbers of rows & columns ...
0
votes
0
answers
10
views
Read Met Office Data Point JSON into Panda
I am using the MetOffice Datapoint API to download UK Weather data as a JSON. I would then like to read that JSON file into a pandas DataFrame. The format of the JSON file is as shown
{"SiteRep&...
0
votes
1
answer
37
views
Filter datetime column with object as data type in python
I have a df with a Timestamp and Value columns. Both have the 'object' dtype
| Timestamp | Value |
--------------------------------------
| 8/21/2023 12:00:00 AM | a |
| 11/...
1
vote
2
answers
34
views
Converting JSON list with multiple nested dictionaries to csv or excel
I have a JSON that I download from a website that has multiple nested dictionaries inside the main list. This is a very simplified version of it.
[
{
"id": 1,
"...
- 11
0
votes
1
answer
26
views
Series is empty when using .loc to slice
enter image description here
I'd like to get the item between Q1 to Q9. Q1 to Q9 exist and also they are sequential. As shown in the picture, I used .loc to slice the series object:
s.loc['Q1':'Q2']
...
1
vote
2
answers
35
views
Map Dataframe Column Values Based on Two Dictionaries Conditionally [duplicate]
I have a dataframe df_test. I want to map the column color conditionally:
if category is 'tv', then map using the tv_map dictionary
else map using the radio_map dictionary
I could split df_test by ...
- 727
3
votes
2
answers
58
views
How do I get variable length slices of values using Pandas?
I have data that includes a full name and first name, and I need to make a new column with the last name. I can assume full - first = last.
I've been trying to use slice with an index the length of ...
- 65
2
votes
0
answers
53
views
How to compare rows within the same csv file faster
I have a csv file containing 720,000 rows with and 10 columns, the columns that are relevant to the problem are ['timestamp_utc', 'looted_by__name', 'item_id', 'quantity']
This File is logs of items ...
- 31
0
votes
0
answers
23
views
Dataframe replace columns and save to new df
I loaded in a .dat file to a pandas dataframe. Two of the columns are mean and error. I used the values in these two columns to create a randomized value for mean. I want to replace the mean column in ...
0
votes
1
answer
27
views
Read/Write pipeline in pandas
I'm building a simple pipeline to pull a dataset from a database and write it to a csv so that I can access the data more quickly in the future.
Currently, I have this:
# data loading as a pipeline
# ...
- 305
0
votes
2
answers
56
views
How to use a for loop in storing multiple data frames with slightly different columns?
In my directory, I have a list of 9 txt files. Without manually running pd.read_table() for each of them, I would like to efficiently run a loop for example and save a data frame for each file. Note, ...
- 1,065
-1
votes
0
answers
35
views
How do i search two files to know if the values are similar or not
I have two CSV files that I want to compare their contents and print the contents that doesn't have the same data. Most contents will have the same name but not all their data are accurate. I want to ...
- 109
0
votes
0
answers
16
views
Darts (Python) TiDE model gives irrational results when switched to probabilistic mode
Let's consider this simple code:
import yfinance as yf
import pandas as pd
from darts import TimeSeries
import darts.models as dm
import darts.utils.likelihood_models as dl
import lightning.pytorch as ...
- 741
0
votes
0
answers
25
views
How to calculate the Relative Strength Index (RSI) through record iterations in pandas dataframe
I have created a pandas dataframe as follows:
import pandas as pd
import numpy as np
ds = { 'trend' : [1,1,1,1,2,2,3,3,3,3,3,3,4,4,4,4,4], 'price' : [23,43,56,21,43,55,54,32,9,12,11,12,23,3,2,1,1]...
- 1,392
0
votes
1
answer
29
views
Pandas apply is turning matrices into nan/None
I am running the following code on a dataset trying to tally the rows from one data set that match a varied set of criteria. I am using the apply function to store this tally inside of a matrix, where ...
