apache-arrow

Instant Kubernetes-Native Application Observability

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..

❄️ Coolest database around 🧊 Embeddable column database written in Go.

Manipulate JSON-like data with NumPy-like idioms.

Replacing MongoDB, Neo4J, and Elastic with 1 transactional database. Features: zero-copy semantics, swappable backends, bindings for C, C++, Python, Java, GoLang

Infrastructures™ for Machine Learning Training/Inference in Production.

A Rust DataFrame implementation, built on Apache Arrow

Geospatial extensions for Polars

Official Julia implementation of Apache Arrow

A SQLite vtable extension to read Parquet files

Manipulate arrays of complex data structures as easily as Numpy.

Fletcher: A framework to integrate FPGA accelerators with Apache Arrow

Rust-based WebAssembly bindings to read and write Apache Parquet data

Cross-filter millions (or even billions) of data entries with no interaction delay

Pod5: a high performance file format for nanopore reads.

Convert a CSV to a parquet file.

OSM planet dump high performance data loader. Transform OpenStreetMap World/Region PBF dump into partitioned by H3 regions PostGIS pgsnapshot (lossless) OSM schema representation and/or into ArrowIPC/Parquet dumps

A simple dataframe library for Ruby