datalake

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Apache Doris is an easy-to-use, high performance and unified analytics database.

Data Lake for Deep Learning. Build, manage, query, version, & visualize datasets. Stream data real-time to PyTorch/TensorFlow. https://activeloop.ai

Upserts, Deletes And Incremental Processing on Big Data.

lakeFS - Git-like capabilities for your object storage

Dinky is an out of the box one-stop real-time computing platform dedicated to the construction and practice of Unified Streaming & Batch and Unified Data Lake & Data Warehouse. Based on Apache Flink, Dinky provides the ability to connect many big data frameworks including OLAP and Data Lake.

The LeoFS Storage System

A Table Structure Storage on Data Lakes to Unify Batch and Streaming Data Processing

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

汇总Apache Hudi相关资料

Arctic is a streaming lake warehouse service open sourced by NetEase

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)

Use SQL to build ELT pipelines on a data lakehouse.

The Internals of Delta Lake

A Data Platform built for AWS, powered by Kubernetes.

Streaming application development and management system, based on Linkis and DSS, planning to provide the workflow-like graphical drag-and-drop development capability.

Apache Spark Course Material

Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi

This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.

anyscale roadmap