HPGO

Hybrid Parallelism Global Orchestration

This project is in Alpha Stage: The functionality provided should be considered research grade and the name "HPGO" is expected to change upon stable release

Get Started

Install

The binary package of this library is provided for every release.

From Rust Cargo, as a Rust crate

cargo install HPGO

From Python PyPI, as a Python3 package

pip3 install HPGO

Build from source

rustup default nightly
cargo build --release
# library produced under targets/release/libHPGO.so

Use

# Import HPGO Python API from HPGO.so
import HPGO
# Construct the Conductor object
c = HPGO.conductor_from_torch_graph_and_seps("./profiles/xlnet/graph.txt", 64, 512, [8, 16])
res = c.py_orchestrate()
print(res)

Contribution

Pull requests are welcomed!

License

This project is open sourced under the terms of BSD-3-Clause, details of which can be found in the LICENSE file

If you apply this library to any project and research, please cite this code:

@misc{HPGO,
  author       = {Yi Rong},
  title        = {HPGO, [{Hybrid Parallelism} Global Orchestration]},
  howpublished = {\url{https://github.com/LER0ever/HPGO}},
  year         = {2020}
}

The project contains source code from PipeDream, a Microsoft Research project licensed under the MIT License. Code taken from Pipedream includes a Rust file src/input/torch_graph_py.rs, several profiling data files under the profiles directory, and the contrib directory.

This project also contains profiling data provided by the AlibabaPAI/DAPPLE project. The development of the DAPPLE planner continues here.