avro

Apache Avro is a data serialization system.

Record Query - A tool for doing record analysis and transformation

Confluent Schema Registry for Kafka

Apache Kafka and Confluent Platform examples and demos

Avro for JavaScript ⚡

What's in your data? Extract schema, statistics and entities from datasets

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].

[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Avro schema generation and serialization / deserialization for Scala

Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Command Line Tool for managing Apache Kafka

Iceberg is a table format for large, slow-moving tabular data

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files

Web tool for Avro Schema Registry |

More than 2000+ Data engineer interview questions.

Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.

A tool for data sampling, data generation, and data diffing