GDC compiler settings on codechef

Currently codechef provides GDC 6.3 as the only implementation of the D programming language. This version had been released in 2016 and nowadays is missing some of the useful language features. Upgrade to a more recent version would be very much appreciated. But apparently this is not the only problem.

During the latest contest, doing binary search suffered from TLE and this forced me to do some investigation. I suspect that there may be something wrong with the used optimization settings for the GDC compiler on the codechef platform. The performance of the following reduced code snippet is very bad:

import std.algorithm, std.conv, std.range, std.stdio, std.string, std.stdint;

// calculate integer square root using binary search
static int64_t isqrt(int64_t x) {
  return iota(0, min(x, 3037000499) + 1).map!(v => (v * v > x)).assumeSorted.lowerBound(true).length - 1;
}

void main() {
  // calculate integer square roots for all numbers in the [0, 2000000) range and print their sum
  2000000.iota.map!isqrt.sum.writeln;
}

If I paste it to codechef ide and run, then the execution time is around 1.3 seconds. If I paste it to AtCoder custom test page and run, then the execution time is only around 0.15 seconds. Yes, the version of GDC is different and AtCoder is using different hardware, but almost an order of magnitude difference still looks abnormal.

So in order to do an additional test, I have installed an old version of Debian Linux (so that it has the same GDC 6.3 as codechef) on my old laptop with a 2GHz Intel Core2 Duo processor and also tried to benchmark this code snippet there:

  • If I compile this code without any optimizations by running “gdc test.d”, then the execution time is 2.2 seconds.
  • If I compiler this code with -O2 optimizations by running “gdc -O2 -frelease test.d” (the same optimization settings as used by AtCoder), then the execution time reduces to 0.36 seconds.

This makes me think that codechef probably doesn’t enable optimizations for GDC. And if this is the case, then it is very unfortunate because the performance difference may be gigantic. And writing code in functional style apparently takes the biggest hit. I already encountered a very strange performance issue earlier (TLE vs. AC), which fortunately could be workarounded by rewriting the array printing code in imperative style during that contest.

If the root cause is indeed just a missing “-O2” compiler option for GDC, then I believe that the problem can be resolved very easily if @admin takes a look at it.

All existing compiler flags are mentioned in Are any compiler flags set on the online judge?

Since D isn’t listed there, it does mean that we don’t pass any flags for it as of now. But we’ve noted your points, and in the next iteration of language updates, we’ll make it a point to introduce this.

Thanks! How long does it usually take until the next iteration of language updates? The comment about the status of programming languages and compilers was last updated in 2017. I can see that in 2019 there was a report about Rust compiler not enabling optimizations, but nobody replied in the forum. Running the test program from that forum post shows that the Rust optimizations issue is resolved now.

I have implemented a configuration sanity checker for D language too, which can be run in the codechef IDE mode:

import std.stdio, std.range, std.algorithm;

// Optimizing compilers should do tail call optimization here, so
// 0xBADF00D magic constant won't be found in stack if optimizations
// are enabled. Is there a cleaner way to detect optimizations?
bool detect_opt() {
  int[100] filler;
  bool detect_tail_call_opt(int depth, int magic) {
    if (depth > 100) {
      int x; foreach (i ; 20 .. 80) if (*(&x + i) == magic) return false;
      return true;
    }
    return detect_tail_call_opt(depth + 1, magic);
  }
  return filler[0] || detect_tail_call_opt(0, 0xBADF00D);
}

// GDC11 was the first version to start supporting getTargetInfo traits
bool detect_gdc11() {
  version(GNU) { return __traits(compiles, __traits(getTargetInfo, "cppStd")); }
  else return false;
}

void main() {
  bool assert_on = false, optimizations_on = detect_opt();
  version(assert) { assert_on = true; }
  string[] warnings;
  version(GNU) {
    writeln("Detected compiler: GDC");
    if (!optimizations_on)
      warnings ~= "Performance warning: '-O3' option was not used!";
    if (assert_on)
      warnings ~= "Performance warning: '-frelease' option was not used!";
    if (detect_gdc11())
      warnings ~= "Note: GDC11 or newer may need '-flto' option, see https://gcc.gnu.org/PR102765";
  } else version(LDC) {
    writefln("Detected compiler: LDC (optimizing for %s)", __traits(targetCPU));
    if (!optimizations_on)
      warnings ~= "Performance warning: '-O' option was not used!";
    if (assert_on)
      warnings ~= "Performance warning: '-release' option was not used!";
    if (__traits(targetCPU) == "pentium4")
      warnings ~= "Performance warning: '-mcpu=native' option was not used!";
  } else version(DigitalMars) {
    writeln("Detected compiler: DMD");
    if (!optimizations_on)
      warnings ~= "Performance warning: '-O' option was not used!";
    if (assert_on)
      warnings ~= "Performance warning: '-release' option was not used!";
    warnings ~= "Performance warning: DMD generates much slower code than GDC or LDC!";
  } else {
    warnings ~= "Unknown compiler";
  }
  if (size_t.sizeof < 8)
    warnings ~= "Performance warning: not a 64-bit compiler!";
  if (warnings.empty)
    writeln("Everything seems to be properly configured.");
  else
    writeln(warnings.joiner("\n"));
}

Would it be possible to just quickly add “-O3 -frelease” options to the command line of your GDC 6.3 compiler?

There is no time limit multiplier for D and it’s treated as a high performance optimizing compiler with the same time limits as C++. Also looks like I’m the only remaining user of the D language on the codechef platform at the moment (based on checking submissions for a few recent contests). So the risk of making the users unhappy by breaking something is pretty low.

We hope to have the next iteration in the next month or two.
Will check if we can make the -O2 change sooner, but not sure.