Take the 2-minute tour ×
Code Review Stack Exchange is a question and answer site for peer programmer code reviews. It's 100% free, no registration required.

I am very early in my journey of learning Scala, and I was doing challenges on hackerrank.com to get used to the syntax. As I finished one (very trivial) challenge, I noticed that the run times seem to be unusually slow.

Thinking that it might be environment-dependent (as the code runs on the HackerRank servers), I decided to quickly reimplement the same code in C#. Interestingly, the code runs 5x faster...

I know that the environment could affect a lot of things, but surely a 5x speed gain for such a trivial operation is strange? They are using Mono for compiling C# code, so both implementations run on Linux. You can get detailed info on their environment here.

The only other idea I have is that, as I am very new to Scala, it is my code which is the problem. So, please take a look at my two implementations below (first in Scala, second in C#) for this problem.

Scala implementation:

object Solution {
    def main(args: Array[String]) {
        println(if (readLine.toLowerCase.filter(x => x >= 'a' && x <= 'z').distinct.length == 26) "pangram" else "not pangram")
    }
}

C# implementation:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
class Solution {
    static void Main(String[] args) {
        string x = System.Console.ReadLine();
        System.Console.WriteLine((x.ToLowerInvariant().ToCharArray().Where(z => z >= 'a' && z <= 'z').Distinct().Count() == 26) ? "pangram" : "not pangram");
    }
}

and let me know how it is possible that the Scala implementation runs that much slower...

I can post screenshots of the run results if needed, but to give an idea - most C# test cases finish in 0.1s and Scala cases take 0.5s.

Can it be the startup time of the runtime? Or do I surrender to the idea that Scala is really that much slower by nature?

The timings were returned by HackerRank's environment (so I realize, of course, that this is nowhere near a benchmark - but nevertheless 5x is crazy).

Scala results:

Scala

C# results:

C#

share|improve this question
    
How did you made the test? How did you took the time? Understanding this may give a clue about the different. –  roterl Oct 18 at 22:17
    
@roterl Please see edit... The times were returned by Hackerrank's environment (so I realize it is not a 100% accurate comparison). –  Ruslan Oct 18 at 22:28
1  
By the way, if you extend App you don't have to put your code inside a main method. –  AmigoNico Oct 22 at 7:09
    
@AmigoNico That's awesome info, thanks! I never knew that. –  Ruslan Oct 22 at 17:07

2 Answers 2

up vote 3 down vote accepted

There is no reason for Scala code to be slower than C#.
The different is probably the time that it take to the JVM to load.
To verify that I suggest you to write simple code in C#, Scala & Java which just print "Hello World" and see the different. My guess is that the Java & Scala will take the same time while the C# will be faster.
Then just do something which do long calculations and see the different again.

share|improve this answer
    
I did run a few benchmarks on my own machine, and Scala definitely is faster than C# on Mono. So it turns out you were right - it was the JVM booting that took up the bulk of the execution time. I will admit, this came as a very pleasant surprise; I expected Scala to be slower (although not by 5x - hence this question). I will put together an article about my benchmarks. –  Ruslan Oct 19 at 3:16
1  
The JVM execute code very fast, and Scala provide about the same performances as Java. I'll glad to read your benchmarks article. –  roterl Oct 19 at 6:26

Benchmarking involves some subtleties that require a little more than just running the code once and measuring its execution time. There is indeed a startup cost when executing a Java VM (the Scala runtime) that includes, among other things, class loading.

The following are some basic tips for benchmarking with the JVM, and therefore they also apply to benchmarking Scala code:

  • Start with a warm up phase to make sure the runtime is finished initializing, any caches are warm, and the just-in-time compiler has compiled your code into native machine code. You can do this by running the code you intend to measure several thousand times before you start measuring time.
  • Take the average execution time for the code under benchmark over several thousand executions, if possible. This will statistically eliminate, for example, garbage collection pauses that may occur while the code is executing.
  • Use System.nanoTime() for measuring execution time, as this is generally more accurate than using System.currentTimeMillis()

Just these simple strategies should give you are more fair comparison. I would be interested in seeing your results.

For a more detailed analysis of issues encountered with benchmarking with a Java VM, although a bit dated at this point, see http://www.ibm.com/developerworks/library/j-benchmark1/.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.