The International Satisfiability Modulo Theories (SMT) Competition.
Home
Introduction
Benchmark Submission
Publications
SMT-LIB
Previous Editions
Competition results for the UFIDL division in the Single Query Track.
Page generated on 2019-07-23 17:56:09 +0000
Benchmarks: 20 Time Limit: 2400 seconds Memory Limit: 60 GB
| Sequential Performance | Parallel Performance | SAT Performance (parallel) | UNSAT Performance (parallel) | 24s Performance (parallel) |
|---|---|---|---|---|
| Par4 | Par4 | Par4 | Par4 | Par4 |
| Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Abstained | Timeout | Memout |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Par4 | 0 | 12 | 20236.029 | 19748.761 | 12 | 3 | 9 | 8 | 8 | 0 | |
| Z3n | 0 | 11 | 15689.359 | 15692.729 | 11 | 2 | 9 | 9 | 6 | 0 | |
| 2018-Z3n | 0 | 11 | 16071.642 | 16073.027 | 11 | 2 | 9 | 9 | 6 | 0 | |
| CVC4 | 0 | 10 | 19802.131 | 20279.155 | 10 | 1 | 9 | 10 | 8 | 0 | |
| veriT | 0 | 7 | 26302.614 | 26306.917 | 7 | 0 | 7 | 13 | 10 | 0 | |
| SMTInterpol | 0 | 7 | 28947.169 | 28907.764 | 7 | 1 | 6 | 13 | 12 | 0 | |
| Vampire | 0 | 7 | 31220.601 | 31206.391 | 7 | 0 | 7 | 13 | 13 | 0 | |
| UltimateEliminator+Yices-2.6.1 | 0 | 0 | 14449.287 | 14433.182 | 0 | 0 | 0 | 20 | 6 | 0 | |
| UltimateEliminator+MathSAT-5.5.4 | 0 | 0 | 14449.498 | 14433.097 | 0 | 0 | 0 | 20 | 6 | 0 | |
| UltimateEliminator+SMTInterpol | 0 | 0 | 14451.431 | 14429.896 | 0 | 0 | 0 | 20 | 6 | 0 |
| Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Abstained | Timeout | Memout |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Par4 | 0 | 12 | 20236.029 | 19748.761 | 12 | 3 | 9 | 8 | 8 | 0 | |
| Z3n | 0 | 11 | 15692.139 | 15692.639 | 11 | 2 | 9 | 9 | 6 | 0 | |
| 2018-Z3n | 0 | 11 | 16072.462 | 16072.907 | 11 | 2 | 9 | 9 | 6 | 0 | |
| CVC4 | 0 | 10 | 20175.351 | 20278.745 | 10 | 1 | 9 | 10 | 8 | 0 | |
| veriT | 0 | 7 | 26306.454 | 26306.637 | 7 | 0 | 7 | 13 | 10 | 0 | |
| SMTInterpol | 0 | 7 | 28947.169 | 28907.764 | 7 | 1 | 6 | 13 | 12 | 0 | |
| Vampire | 0 | 7 | 31220.601 | 31206.391 | 7 | 0 | 7 | 13 | 13 | 0 | |
| UltimateEliminator+SMTInterpol | 0 | 0 | 20655.421 | 14273.006 | 0 | 0 | 0 | 20 | 5 | 0 | |
| UltimateEliminator+Yices-2.6.1 | 0 | 0 | 20755.737 | 14308.432 | 0 | 0 | 0 | 20 | 5 | 0 | |
| UltimateEliminator+MathSAT-5.5.4 | 0 | 0 | 21032.878 | 14376.617 | 0 | 0 | 0 | 20 | 5 | 0 |
| Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Timeout | Memout |
|---|---|---|---|---|---|---|---|---|---|---|
| Par4 | 0 | 3 | 1035.971 | 548.098 | 3 | 3 | 0 | 17 | 8 | 0 |
| 2018-Z3n | 0 | 2 | 2400.109 | 2400.109 | 2 | 2 | 0 | 18 | 6 | 0 |
| Z3n | 0 | 2 | 2400.113 | 2400.113 | 2 | 2 | 0 | 18 | 6 | 0 |
| CVC4 | 0 | 1 | 974.039 | 1077.436 | 1 | 1 | 0 | 19 | 8 | 0 |
| SMTInterpol | 0 | 1 | 2401.044 | 2400.654 | 1 | 1 | 0 | 19 | 12 | 0 |
| UltimateEliminator+SMTInterpol | 0 | 0 | 10.148 | 6.357 | 0 | 0 | 0 | 20 | 5 | 0 |
| UltimateEliminator+MathSAT-5.5.4 | 0 | 0 | 10.076 | 7.01 | 0 | 0 | 0 | 20 | 5 | 0 |
| UltimateEliminator+Yices-2.6.1 | 0 | 0 | 9.989 | 7.109 | 0 | 0 | 0 | 20 | 5 | 0 |
| veriT | 0 | 0 | 2400.236 | 2400.212 | 0 | 0 | 0 | 20 | 10 | 0 |
| Vampire | 0 | 0 | 7200.0 | 7200.0 | 0 | 0 | 0 | 20 | 13 | 0 |
| Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Timeout | Memout |
|---|---|---|---|---|---|---|---|---|---|---|
| Par4 | 0 | 9 | 0.057 | 0.663 | 9 | 0 | 9 | 11 | 8 | 0 |
| 2018-Z3n | 0 | 9 | 0.753 | 0.753 | 9 | 0 | 9 | 11 | 6 | 0 |
| Z3n | 0 | 9 | 0.778 | 0.778 | 9 | 0 | 9 | 11 | 6 | 0 |
| CVC4 | 0 | 9 | 1.312 | 1.309 | 9 | 0 | 9 | 11 | 8 | 0 |
| veriT | 0 | 7 | 4706.218 | 4706.426 | 7 | 0 | 7 | 13 | 10 | 0 |
| Vampire | 0 | 7 | 4820.601 | 4806.391 | 7 | 0 | 7 | 13 | 13 | 0 |
| SMTInterpol | 0 | 6 | 7346.125 | 7307.11 | 6 | 0 | 6 | 14 | 12 | 0 |
| UltimateEliminator+SMTInterpol | 0 | 0 | 20614.607 | 14249.458 | 0 | 0 | 0 | 20 | 5 | 0 |
| UltimateEliminator+Yices-2.6.1 | 0 | 0 | 20717.944 | 14282.526 | 0 | 0 | 0 | 20 | 5 | 0 |
| UltimateEliminator+MathSAT-5.5.4 | 0 | 0 | 20994.753 | 14350.673 | 0 | 0 | 0 | 20 | 5 | 0 |
| Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Abstained | Timeout | Memout |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Par4 | 0 | 11 | 216.069 | 216.785 | 11 | 2 | 9 | 9 | 9 | 0 | |
| 2018-Z3n | 0 | 11 | 216.862 | 216.862 | 11 | 2 | 9 | 9 | 9 | 0 | |
| Z3n | 0 | 11 | 216.891 | 216.891 | 11 | 2 | 9 | 9 | 9 | 0 | |
| CVC4 | 0 | 9 | 217.903 | 217.895 | 9 | 0 | 9 | 11 | 9 | 0 | |
| veriT | 0 | 7 | 264.514 | 264.487 | 7 | 0 | 7 | 13 | 11 | 0 | |
| Vampire | 0 | 7 | 332.601 | 318.391 | 7 | 0 | 7 | 13 | 13 | 0 | |
| SMTInterpol | 0 | 6 | 366.146 | 337.063 | 6 | 1 | 5 | 14 | 13 | 0 | |
| UltimateEliminator+SMTInterpol | 0 | 0 | 195.431 | 173.896 | 0 | 0 | 0 | 20 | 6 | 0 | |
| UltimateEliminator+MathSAT-5.5.4 | 0 | 0 | 193.498 | 177.097 | 0 | 0 | 0 | 20 | 6 | 0 | |
| UltimateEliminator+Yices-2.6.1 | 0 | 0 | 193.287 | 177.182 | 0 | 0 | 0 | 20 | 6 | 0 |
n Non-competing.
Abstained: Total of benchmarks in logics in this division that solver chose to abstain from. For SAT/UNSAT scores, this column also includes benchmarks not known to be SAT/UNSAT.