Contribute

Everything here is someone else's published work. Corrections are the most valuable thing you can send, and a correction about your own paper gets acted on the same day.

Open an issue

All of it goes through GitHub issues. There is no form and no account to create beyond GitHub.

• A number is wrong, or attributed to the wrong paper. Name the instance and what it should be. Rows are corrected in place and the history stays in git.
• A missing result. The instance, the energy, its error bar, the method, and the paper it was published in. Anything missing renders as n/a rather than blocking the row (rules §3).
• An error bar we could not find. 31 sampled energies across 26 instances are listed but rank for nothing because no error bar was found in the source we read, and 5 of them sit below their instance's current record. One message closes that (rules §6).
• An objection to a row. Wrong symmetry sector, a mis-declared bound_type, an error bar with no autocorrelation correction: these are technical disputes with a process, rules §10. Rulings cite a clause rather than anyone's judgement about anyone's work.
• What a number cost. GPU-hours × device, parameter count, wall-clock. No row carries this yet, which is why there is no accuracy-versus-cost view (the format is specified).

The marks on a row are questions, not criticism

○ means we found no error metric in the source we read. It is a statement about our search, not about the authors: the figure may well be in a supplement, a companion paper, or your own records. The row stays in the table and stays in the ranking either way (error metrics).

Variance and checkpoints

Var(E) — the variance of the local energy, not the error bar — is what the V-score needs, and almost nobody reports it. It is a by-product of any sampling pass, so if you send an optimized checkpoint and the model code, the variance can be measured and credited on your row. A state we retrain ourselves is a new row rather than a variance for yours (rules §11).