bigdata

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀

Python clone of Spark, a MapReduce alike framework in Python

🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

Google, Naver multiprocess image web crawler (Selenium)

学习记录的一些笔记，以及所看得一些电子书eBooks、视频资源和平常收纳的一些自己认为比较好的博客、网站、工具。涉及大数据几大组件、Python机器学习和数据分析、Linux、操作系统、算法、网络等

Datafaker is a large-scale test data and flow test data generation tool. Datafaker fakes data and inserts to varied data sources. 测试数据生成工具

ROOT I/O in pure Python and NumPy.

A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.

Simple-IT-English: smart wordbook from community for community

A collection of pentest tools and resources targeting Hadoop environments

大数据分析可视化实践

AthenaCLI is a CLI tool for AWS Athena service that can do auto-completion and syntax highlighting.

ROOT I/O in pure Python and NumPy.

Scalable Bloom Filter implemented in Python

O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian

🌲 Configuration flaws detector for Hadoop, MongoDB, MySQL, and more!

Educational notes,Hands on problems w/ solutions for hadoop ecosystem

Apache Spark 3 - Structured Streaming Course Material

重庆大学计算机学院“大数据课程群”实验及PPT

Amas is recursive acronym for “Amas, monitor alert system”.