Upserts, Deletes And Incremental Processing on Big Data.
type-class based data cleansing library for Apache Spark SQL
This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.
SparkSQL.jl enables Julia programs to work with Apache Spark data using just SQL.
Delta Lake Examples
Repository for Lab “Distributed Big Data Analytics” (MA-INF 4223), University of Bonn
This repository contains all the projects and labs I worked on while pursuing professional certificate programs, specializations, and bootcamp. [Areas: Deep Learning, Machine Learning, Applied Data Science].
Connect to SQL Server using Apache Spark
Trigger spark-submit in Golang. A Go implementation of famous SparkLauncher.java.
FLiP Stack Weekly covering Apache NiFi, Apache Flink, Apache Kafka, Apache Spark, Apache Iceberg, Apache Ozone, Apache Pulsar, and more...
Link Prediction is about predicting the future connections in a graph. In this project, Link Prediction is about predicting whether two authors will be collaborating for their future paper or not given the graph of authors who collaborated for atleast one paper together.
A Capstone Project that covers several aspects of Data Engineering (Data Exploration, Cleaning, Modeling, Pipelining, Processing)
Use this project to join data from multiple csv files. Currently in this project we support one to one and one to many join. Along with this you can find how to use kafka producer efficiently with spark.
This is a Jupyter Notebook to practice Apache Spark in Google Colab, especially for the exam CCA Spark and Hadoop Developer Exam (CCA175).
You will find here the demo codes for my Data+AI 2020 talk about customizing Apache Spark state store.
Add a description, image, and links to the apachespark topic page so that developers can more easily learn about it.
To associate your repository with the apachespark topic, visit your repo's landing page and select "manage topics."