I wanted to make 3 test suites where each would run a test class with a specific input. I figured that in order to do this I could:
- Make an abstract test suite with all the variable parts (
AbstractPerformanceTest
). - Make a subclass out of it (
SmallInputPerformanceTestSuite
). - Instantiate the member variables of the abstract class with a public
init()
method. - Use the instantiated variables in the classes that the test suite runs.
It should be noted that I have a main folder where all the datasets which the algorithm will act on are stored and one folder that contains what the results of that algorithm should be. My folder structure of the input data and expected results data is like this:
The abstract suite class that all suites inherit from:
public abstract class AbstractPerformanceTest {
private static SparkContextWrapper sparkContext;
private static List<File> inputFiles;
private static List<File> expectedResults;
private static SkylineAlgorithmFactory algorithmFactory;
private static int timesToRun;
public static void init(List<File> datasetsFiles, List<File> expectedResultsFiles,
SparkContextWrapper sc, int times) {
inputFiles = datasetsFiles;
expectedResults = expectedResultsFiles;
sparkContext = sc;
algorithmFactory = new SkylineAlgorithmFactory();
timesToRun = times;
}
public static List<File> getInputFiles() {
return inputFiles;
}
public static List<File> getExpectedResultsFiles() {
return expectedResults;
}
public static SparkContextWrapper getSparkContext() {
return sparkContext;
}
public static SkylineAlgorithmFactory getAlgorithmFactory() {
return algorithmFactory;
}
public static int getTimesToRun() {
return timesToRun;
}
}
An example of a test suite that will have a small input of data:
@RunWith(Suite.class)
@Suite.SuiteClasses({BlockNestedLoopPerformanceTest.class, SortFilterSkylinePerformanceTest.class})
public class SmallInputPerformanceTestSuite extends AbstractPerformanceTest {
private final static String DATASETS_FOLDER = "data/datasets/small";
private final static String EXPECTED_RESULTS_FOLDER = "data/expected results/small";
public SmallInputPerformanceTestSuite() {
}
@BeforeClass
public static void setUpClass() {
List<File> datasetsFilenames = getListOfFilesFromFolder(DATASETS_FOLDER);
List<File> expectedResultsFilenames = getListOfFilesFromFolder(EXPECTED_RESULTS_FOLDER);
SparkContextWrapper sparkContext =
new SparkContextWrapper("SmallInputPerformanceTestSuite", "local[4]");
int timesToRun = 10;
AbstractPerformanceTest.init(datasetsFilenames, expectedResultsFilenames,
sparkContext, timesToRun);
}
private static List<File> getListOfFilesFromFolder(String folderName) {
File folder = new File(folderName);
return Arrays.asList(folder.listFiles());
}
@AfterClass
public static void tearDownClass() {
AbstractPerformanceTest.getSparkContext().stop();
}
}
One of the classes that the test suite will run:
public class BlockNestedLoopPerformanceTest {
private static SparkContextWrapper sparkContext;
private static SkylineAlgorithmFactory algorithmFactory;
private static List<File> inputFiles;
private static List<File> expectedResultsFiles;
private static int timesToRun;
public BlockNestedLoopPerformanceTest() {
}
@BeforeClass
public static void setUpClass() {
sparkContext = AbstractPerformanceTest.getSparkContext();
algorithmFactory = AbstractPerformanceTest.getAlgorithmFactory();
inputFiles = AbstractPerformanceTest.getInputFiles();
expectedResultsFiles = AbstractPerformanceTest.getExpectedResultsFiles();
timesToRun = AbstractPerformanceTest.getTimesToRun();
}
@Test
public void shouldReturnCorrectUniformSkylines() throws FileNotFoundException {
System.out.println("Block-nested loop - Uniform skylines\n");
File inputFile = getFileContainingKeyword(inputFiles, "uniform");
File expResultFile = getFileContainingKeyword(expectedResultsFiles, "uniform");
long totalDuration = getTotalRuntime(inputFile, expResultFile);
System.out.println("Algorithm: BlockNestedLoop"
+ "\nFile: " + inputFile
+ "\nTimes to run: " + timesToRun
+ "\nTime (average): " + totalDuration / timesToRun + " ms\n");
}
@Test
public void shouldReturnCorrectCorrelatedSkylines() throws FileNotFoundException {
System.out.println("Block-nested loop - Correlated skylines\n");
File inputFile = getFileContainingKeyword(inputFiles, "correl");
File expResultFile = getFileContainingKeyword(expectedResultsFiles, "correl");
long totalDuration = getTotalRuntime(inputFile, expResultFile);
System.out.println("Algorithm: BlockNestedLoop"
+ "\nFile: " + inputFile
+ "\nTimes to run: " + timesToRun
+ "\nTime (average): " + totalDuration / timesToRun + " ms\n");
}
@Test
public void shouldReturnCorrectAnticorrelatedSkylines() throws FileNotFoundException {
System.out.println("Block-nested loop - Anticorrelated skylines\n");
File inputFile = getFileContainingKeyword(inputFiles, "anticor");
File expResultFile = getFileContainingKeyword(expectedResultsFiles, "anticor");
long totalDuration = getTotalRuntime(inputFile, expResultFile);
System.out.println("Algorithm: BlockNestedLoop"
+ "\nFile: " + inputFile
+ "\nTimes to run: " + timesToRun
+ "\nTime (average): " + totalDuration / timesToRun + " ms\n");
}
private File getFileContainingKeyword(List<File> filesToSearchIn, String keyword) throws FileNotFoundException {
String lowerCaseKeyword = keyword.toLowerCase();
for (File file : filesToSearchIn) {
String fileName = file.getName();
String lowerCaseFileName = fileName.toLowerCase();
if (lowerCaseFileName.contains(lowerCaseKeyword)) {
return file;
}
}
throw new FileNotFoundException("Keyword: " + keyword + " was not found in " + getFolderPath());
}
private String getFolderPath() {
return inputFiles.get(0).getParent();
}
private long getTotalRuntime(File inputFile, File expResultFile) {
String inputFilePath = inputFile.getAbsolutePath();
List<Point2D> expResult = getPointsFromFile(expResultFile);
SkylineAlgorithm bnl = algorithmFactory.getBlockNestedLoop(sparkContext);
long totalDuration = 0;
for (int i = 0; i < timesToRun; i++) {
long startTime = System.currentTimeMillis();
List<Point2D> result = bnl.getSkylinePoints(inputFilePath);
long endTime = System.currentTimeMillis();
totalDuration += endTime - startTime;
expResult.removeAll(result);
assertTrue(expResult.isEmpty());
}
return totalDuration;
}
private List<Point2D> getPointsFromFile(File file) {
TextFileToPointRDD txtToPoints = new TextFileToPointRDD(sparkContext);
JavaRDD<Point2D> pointsRDD = txtToPoints.getPointRDDFromTextFile(file.getAbsolutePath(), " ");
return pointsRDD.collect();
}
}
I would like critique on how to make this more clear/clean, readable, eliminating duplicate code (my other test class for example will be exactly the same except it will create an SFS algorithm) etc.
getTotalRuntime()
inBlockNestedLoopPerformanceTest
. My goal is to check that the algorithms return the correct results and measure their performance. – Aki K Dec 1 '14 at 13:47ConcreteTestClass() { super(arg1, arg2, etc); }
? – fge Dec 1 '14 at 14:05super()
has to be the first statement in the constructor it made the code unreadable hence why I used the static initializer. – Aki K Dec 1 '14 at 14:08