For each Track, we distinguish General, Solution Generation and Adjudication Rules.
General
Rule 1
This competition seeks to encourage research into automated nurse rostering methods, and to offer prizes to the most successful methods in particular tracks. It is the spirit of these rules that is important, not the letter. With any set of rules for any competition it is possible to work within the letter of the rules but outside the spirit. The organisers ask that you please don’t do this. It’s not fair, it’s not good science, and it will result in disqualification.
Rule 2
The organisers reserve the right to disqualify any participant from the competition at any time if the participant is determined by the organisers to have worked outside the spirit of the competition rules. The organisers’ decision is final in any matter. Decisions will be made democratically by the organisers with the Chair having the casting vote in the case to ties.
Rule 3
The organisers reserve the right to change the rules at any time and without notice. Any change of rules will be accompanied by a general email to all participants.
Solution Generation
Rule 4
The competition has an opening day and a deadline when all submissions must be uploaded. These deadlines are absolute and no extensions will be given under any circumstances because to do so would be unfair to other participants.
Rule 5
Participants have to implement an algorithm to tackle the problem on a single processor machine; they can use any programming language. The use of third-party software is allowed under the following restrictions:
- it is free software
- it’s behaviour is (reasonably) documented
- it runs under a commonly-used operating system (Linux, Windows or Mac OS)
Rule 6
The goal is to produce rosters in which all hard constraints are satisfied (i.e. feasible rosters) and to minimize the number of broken soft constraints.
Rule 7
Instances of different size and type will appear on the web site from the opening day. Two weeks before the deadline more instances will be placed on the web. A third set of datasets will be used to internally rank the top competitors. The datasets are therefore classified as Early Instances, Late Instances and Hidden Instances. Competitors should refer to the information associated with each track for further specifics on datasets. The Hidden Instances will be released after the competition is closed.
Rule 8
Participants have to benchmark their machine with the program provided in order to know how much time they have available to run their program on their machines.
Rule 9
The algorithms should take as input a problem file in the format described, and produce as output a solution in the described format. It should do so within the allowed CPU time. Obviously the algorithm should not take account of additional hard coded knowledge about the instance (e.g. introducing instance specific heuristics). The same version of the algorithm must be used for all instances. That is, the algorithm should not “know” which instance it is solving – while your particular algorithm might analyse the problem instance and set parameters accordingly, it should not “recognise” the particular instance. The programmer should not set different parameters for different instances although of the program is doing this automatically then this is acceptable.
Rule 10
The algorithm can be either deterministic or stochastic. In both cases, participants must be prepared to show that these results are repeatable in the given computer time. In particular, the participants that use a stochastic algorithm should code their program in such a way that the exact run that produced each solution submitted can be repeated (by recording the random seed, etc..). They can try several runs to produce each submitted solution (each with the allowed computer time), but they must be able to repeat the run for any solution submitted.
Rule 11
Participants should submit for each instance (Early and Late) the best score found by their algorithm in the specified computer time, by uploading it onto the web site. Competitors should refer to the information associated with each track for further specifics.
Rule 12
Participants should also submit a concise and clear description of their algorithm, so that in principle others can implement it. A template will be made available one month before the end date for this purpose. This is a fundamental part of a competitors’ submission.
Adjudication
Rule 13
For each track, a set of 5 finalists will be chosen after the competition deadline. Ordering of competitors will be based on the scores provided on the early and late instances. The actual list will be based on the ranks of solvers on each single instance. The mean average of the ranks will produce the final place list. More details on how the orderings will be established can be found here.
Rule 14
The finalists will be asked to provide the executable that will be run and tested by the organisers. The finalists’ solver will be rerun by the organisers on all instances (including the Hidden ones). It is the responsibility of the competitor to ensure all information is provided to enable the organisers to recreate the solution.
The solver submitted by the finalist should require as command-line arguments, input and output file names and, for stochastic solvers only, the random seed. For example (stochastic solver):
> my_solver.exe dataset1.nrp dataset1.sln 1542955064
If appropriate information is not received or indeed the submitted solutions cannot be recreated, another finalist will be chosen from the original competitors.
Rule 15
Finalists’ eventual place listings will be based on the ranks on each single instance for a set of trials on all instances (including the hidden ones). As with Rule 10, an explanation of the procedures to be used can be found here.
Rule 16
In some circumstances, finalists may be required to show source code to the organisers. This is simply to check that they have stuck to the rules and will be treated in the strictest confidence.
Rule 17
Entries from participating organising partners will not be permitted. However, results from participants who choose to work on the problems will be presented for comparison.