The International Satisfiability Modulo Theories (SMT) Competition.
Home
Introduction
Benchmark Submission
Publications
SMT-LIB
Previous Editions
Competition results for the QF_BVFPLRA logic in the Single Query Track.
Page generated on 2022-08-10 11:17:45 +0000
Benchmarks: 69 Time Limit: 1200 seconds Memory Limit: 60 GB
Sequential Performance | Parallel Performance | SAT Performance (parallel) | UNSAT Performance (parallel) | 24s Performance (parallel) |
---|---|---|---|---|
Bitwuzla | Bitwuzla | Bitwuzla | cvc5 | Bitwuzla |
Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Timeout | Memout |
---|---|---|---|---|---|---|---|---|---|---|
Bitwuzla | 0 | 62 | 10814.904 | 10813.047 | 62 | 36 | 26 | 7 | 7 | 0 |
cvc5 | 0 | 62 | 15529.117 | 15530.993 | 62 | 35 | 27 | 7 | 7 | 0 |
2021-cvc5n | 0 | 61 | 15342.338 | 15344.127 | 61 | 35 | 26 | 8 | 8 | 0 |
z3-4.8.17n | 0 | 58 | 19662.083 | 19664.577 | 58 | 35 | 23 | 11 | 10 | 0 |
MathSATn | 0 | 56 | 20223.042 | 20226.152 | 56 | 35 | 21 | 13 | 13 | 0 |
COLIBRI | 0 | 49 | 10867.767 | 10867.791 | 49 | 25 | 24 | 20 | 9 | 0 |
Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Timeout | Memout |
---|---|---|---|---|---|---|---|---|---|---|
Bitwuzla | 0 | 62 | 10816.264 | 10812.797 | 62 | 36 | 26 | 7 | 7 | 0 |
cvc5 | 0 | 62 | 15529.737 | 15530.773 | 62 | 35 | 27 | 7 | 7 | 0 |
2021-cvc5n | 0 | 61 | 15342.978 | 15343.707 | 61 | 35 | 26 | 8 | 8 | 0 |
z3-4.8.17n | 0 | 58 | 19662.833 | 19664.197 | 58 | 35 | 23 | 11 | 10 | 0 |
MathSATn | 0 | 56 | 20224.312 | 20225.452 | 56 | 35 | 21 | 13 | 13 | 0 |
COLIBRI | 0 | 49 | 10867.767 | 10867.791 | 49 | 25 | 24 | 20 | 9 | 0 |
Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | N/A | Timeout | Memout |
---|---|---|---|---|---|---|---|---|---|---|---|
Bitwuzla | 0 | 36 | 407.572 | 403.826 | 36 | 36 | 0 | 0 | 33 | 7 | 0 |
z3-4.8.17n | 0 | 35 | 1022.457 | 1022.611 | 35 | 35 | 0 | 1 | 33 | 10 | 0 |
2021-cvc5n | 0 | 35 | 1546.904 | 1546.921 | 35 | 35 | 0 | 1 | 33 | 8 | 0 |
cvc5 | 0 | 35 | 1713.989 | 1714.033 | 35 | 35 | 0 | 1 | 33 | 7 | 0 |
MathSATn | 0 | 35 | 2585.168 | 2585.501 | 35 | 35 | 0 | 1 | 33 | 13 | 0 |
COLIBRI | 0 | 25 | 2436.793 | 2436.818 | 25 | 25 | 0 | 11 | 33 | 9 | 0 |
Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | N/A | Timeout | Memout |
---|---|---|---|---|---|---|---|---|---|---|---|
cvc5 | 0 | 27 | 13815.747 | 13816.74 | 27 | 0 | 27 | 6 | 36 | 7 | 0 |
Bitwuzla | 0 | 26 | 10408.693 | 10408.971 | 26 | 0 | 26 | 7 | 36 | 7 | 0 |
2021-cvc5n | 0 | 26 | 13796.073 | 13796.786 | 26 | 0 | 26 | 7 | 36 | 8 | 0 |
COLIBRI | 0 | 24 | 8430.975 | 8430.973 | 24 | 0 | 24 | 9 | 36 | 9 | 0 |
z3-4.8.17n | 0 | 23 | 18640.377 | 18641.587 | 23 | 0 | 23 | 10 | 36 | 10 | 0 |
MathSATn | 0 | 21 | 17639.145 | 17639.95 | 21 | 0 | 21 | 12 | 36 | 13 | 0 |
Solver | Error Score | Correct Score | CPU Time Score | Wall Time Score | Solved | Solved SAT | Solved UNSAT | Unsolved | Timeout | Memout |
---|---|---|---|---|---|---|---|---|---|---|
Bitwuzla | 0 | 51 | 540.962 | 537.118 | 51 | 34 | 17 | 18 | 18 | 0 |
COLIBRI | 0 | 49 | 283.767 | 283.791 | 49 | 25 | 24 | 20 | 9 | 0 |
2021-cvc5n | 0 | 46 | 749.399 | 749.408 | 46 | 30 | 16 | 23 | 23 | 0 |
cvc5 | 0 | 42 | 749.401 | 749.432 | 42 | 28 | 14 | 27 | 27 | 0 |
MathSATn | 0 | 41 | 786.724 | 786.742 | 41 | 27 | 14 | 28 | 28 | 0 |
z3-4.8.17n | 0 | 40 | 896.888 | 896.842 | 40 | 27 | 13 | 29 | 29 | 0 |
n Non-competing.
N/A: Benchmarks not known to be SAT/UNSAT, respectively.