The International Satisfiability Modulo Theories (SMT) Competition.
Home
Introduction
Benchmark Submission
Publications
SMT-LIB
Previous Editions
Competition results for the UFBV division in the Single Query Track.
Page generated on 2020-07-04 11:46:59 +0000
Benchmarks: 72 Time Limit: 1200 seconds Memory Limit: 60 GB
| Sequential Performance | Parallel Performance | SAT Performance (parallel) | UNSAT Performance (parallel) | 24s Performance (parallel) |
|---|---|---|---|---|
| CVC4 | CVC4 | — | CVC4 | CVC4 |
| Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Abstained | Timeout | Memout |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 2018-Z3n | 0 | 50 | 26465.515 | 26470.183 | 50 | 18 | 32 | 22 | 22 | 0 | |
| 2019-Par4n | 0 | 50 | 26527.12 | 26468.887 | 50 | 18 | 32 | 22 | 22 | 0 | |
| z3n | 0 | 49 | 27629.809 | 27634.255 | 49 | 18 | 31 | 23 | 23 | 0 | |
| CVC4 | 0 | 27 | 36606.133 | 37946.924 | 27 | 0 | 27 | 45 | 26 | 0 | |
| UltimateEliminator+MathSAT | 0 | 6 | 5022.737 | 4853.305 | 6 | 0 | 6 | 66 | 3 | 0 |
| Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Abstained | Timeout | Memout |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 2019-Par4n | 0 | 50 | 26527.12 | 26468.887 | 50 | 18 | 32 | 22 | 22 | 0 | |
| 2018-Z3n | 0 | 50 | 26469.425 | 26469.443 | 50 | 18 | 32 | 22 | 22 | 0 | |
| z3n | 0 | 49 | 27632.839 | 27632.845 | 49 | 18 | 31 | 23 | 23 | 0 | |
| CVC4 | 0 | 27 | 37777.534 | 37945.354 | 27 | 0 | 27 | 45 | 26 | 0 | |
| UltimateEliminator+MathSAT | 0 | 6 | 5022.737 | 4853.305 | 6 | 0 | 6 | 66 | 3 | 0 |
| Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Timeout | Memout |
|---|---|---|---|---|---|---|---|---|---|---|
| z3n | 0 | 18 | 2.163 | 2.164 | 18 | 18 | 0 | 54 | 23 | 0 |
| 2019-Par4n | 0 | 18 | 1.352 | 2.844 | 18 | 18 | 0 | 54 | 22 | 0 |
| 2018-Z3n | 0 | 18 | 3.006 | 3.007 | 18 | 18 | 0 | 54 | 22 | 0 |
| UltimateEliminator+MathSAT | 0 | 0 | 890.169 | 828.629 | 0 | 0 | 0 | 72 | 3 | 0 |
| CVC4 | 0 | 0 | 10802.007 | 10802.007 | 0 | 0 | 0 | 72 | 26 | 0 |
| Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Timeout | Memout |
|---|---|---|---|---|---|---|---|---|---|---|
| 2019-Par4n | 0 | 32 | 1325.767 | 1266.043 | 32 | 0 | 32 | 40 | 22 | 0 |
| 2018-Z3n | 0 | 32 | 1266.418 | 1266.436 | 32 | 0 | 32 | 40 | 22 | 0 |
| z3n | 0 | 31 | 2430.676 | 2430.681 | 31 | 0 | 31 | 41 | 23 | 0 |
| CVC4 | 0 | 27 | 5947.731 | 5968.374 | 27 | 0 | 27 | 45 | 26 | 0 |
| UltimateEliminator+MathSAT | 0 | 6 | 122.016 | 82.416 | 6 | 0 | 6 | 66 | 3 | 0 |
| Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Abstained | Timeout | Memout |
|---|---|---|---|---|---|---|---|---|---|---|---|
| z3n | 0 | 49 | 584.839 | 584.845 | 49 | 18 | 31 | 23 | 23 | 0 | |
| 2019-Par4n | 0 | 49 | 614.22 | 588.338 | 49 | 18 | 31 | 23 | 23 | 0 | |
| 2018-Z3n | 0 | 49 | 588.933 | 588.937 | 49 | 18 | 31 | 23 | 23 | 0 | |
| CVC4 | 0 | 15 | 1251.747 | 1251.761 | 15 | 0 | 15 | 57 | 47 | 0 | |
| UltimateEliminator+MathSAT | 0 | 6 | 451.845 | 346.457 | 6 | 0 | 6 | 66 | 7 | 0 |
n Non-competing.
Abstained: Total of benchmarks in logics in this division that solver chose to abstain from. For SAT/UNSAT scores, this column also includes benchmarks not known to be SAT/UNSAT.