virtual-clusters

vcluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.

Sorting of large dataset files(80GB) using Hadoop(Mapreduce) techniques and Apache Spark in Java and scheduled job on the virtual cluster(using 4 nodes) using a SLURM scheduler with bash scripting

Recipes for setting up a local virtual cluster using specific scheduling engines.

Creates psuedo hdfs and spark cluster