We have collected some guidlines here to make the submissions as comparable as possible. This does not ensure perfect fairness but that’s not possible. Especially since we want to ensure a big diversity and flexibility in terms of available languages. (everything from soldering your own circuits to building a solution in Minecraft or Excel)
The code you write will not be submitted to this website. Therefore you need to benchmark the code yourself. If your code is not executing faster than 5ms we recommend to use hyperfine.
This tool runs a binary multiple times and then analyzes the runtime. It even gives you the standard deviation and the improvement factor.
For faster programs you need to build a custom solution. Most popular programming languages have their own benchmark tool (e.g. Rust has criterion). They work great and if you don’t find any you can always fall back to using hyperfine or just taking the difference of the time at the start of your program and at the end.
Another question is what actually needs to be measured. Does the input loading and parsing count as well? Do you add both parts or submit them separately?
This year we want to provide more statistics and insights. Therefore, you can submit the runtimes of both parts separately. In the global leaderboard you will be ranked by the sum of both parts. This also means you have to solve both parts for each day to appear on the global daily leaderboard!
Since the input parsing is tightly integrated with solving the problem it has to be included in the measurement. Make sure that your benchmarking tool measures both the input parsing and the actual solving! Also ensure that you don’t embed your input file into the binary (as a static str, etc.). Your program has to be able to compute the solution for another input without being modified.
As you probably noticed, you are free to submit any total time you want. There is no automated checking going on to see if you actually wrote code which is that fast. If you plan on exploiting this system please think about all other participants for a second. We want to have a fun challenge this advent and want to compete a bit.
There is no price and no benefit to you if you enter wrong results. The only thing you achieve is to ruin the event for all other people. This leaderboard builds on trust and we believe that the community of Advent of Code is capable of being trustworthy. Nonetheless, if it does not work we will ban people from participating or even shut the whole website down.