Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
-
Updated
Jul 16, 2024 - Python
Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
the portable Python dataframe library
Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Converts a whole subdirectory with a big (or small) volume of PDF documents to a dataset (pandas DataFrame) with error tracking and choice of features
db2ixf is a python package with a CLI that simplifies the parsing and processing of IBM Integration eXchange Format (IXF) files.
(PoC) A very memory-efficient way to read data from PostgreSQL
A web application for viewing Apache Parquet files . This is a Python + Flask application
Reading both XLSX and XLSB files, fast and memory-safe, with Python, into PyArrow
highspeed timeseries pandas dataframe database
Seamlessly switch Pandas DataFrame backend to PyArrow.
Python scripts to process, and analyze log files using PySpark.
Python scripts to download, process, and analyze NYC TLC trip data
Add a description, image, and links to the pyarrow topic page so that developers can more easily learn about it.
To associate your repository with the pyarrow topic, visit your repo's landing page and select "manage topics."