SMT-COMP

The International Satisfiability Modulo Theories (SMT) Competition.

GitHub

Home
Introduction
Benchmark Submission
Publications
SMT-LIB
Previous Editions

SMT-COMP 2020

Rules
Benchmarks
Tools
Specs
Participants
Results
Slides

QF_AX (Single Query Track)

Competition results for the QF_AX division in the Single Query Track.

Page generated on 2020-07-04 11:46:59 +0000

Benchmarks: 300
Time Limit: 1200 seconds
Memory Limit: 60 GB

Winners

Sequential PerformanceParallel PerformanceSAT Performance (parallel)UNSAT Performance (parallel)24s Performance (parallel)
Yices2Yices2Yices2 Yices2 Yices2

Sequential Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Yices2-fixedn 0 300 3.741 6.30530016413600 0
Yices2 0 300 3.845 6.25630016413600 0
2019-Yices 2.6.2n 0 300 4.355 6.4130016413600 0
2018-Yicesn 0 300 4.526 6.61130016413600 0
MathSAT5n 0 300 19.961 20.03330016413600 0
z3n 0 300 23.208 23.24230016413600 0
CVC4 0 300 88.543 88.60330016413600 0
SMTInterpol 0 300 467.817 207.53130016413600 0
SMTInterpol-fixedn 0 300 470.203 208.15730016413600 0
Alt-Ergo 0 119 42135.437 27063.242119011918118 0

Parallel Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Yices2 0 300 3.845 6.25630016413600 0
Yices2-fixedn 0 300 3.741 6.30530016413600 0
2019-Yices 2.6.2n 0 300 4.355 6.4130016413600 0
2018-Yicesn 0 300 4.526 6.61130016413600 0
MathSAT5n 0 300 19.961 20.03330016413600 0
z3n 0 300 23.208 23.24230016413600 0
CVC4 0 300 88.543 88.60330016413600 0
SMTInterpol 0 300 467.817 207.53130016413600 0
SMTInterpol-fixedn 0 300 470.203 208.15730016413600 0
Alt-Ergo 0 128 52274.557 21535.03512801281729 0

SAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Yices2 0 164 0.924 2.66316416401360 0
Yices2-fixedn 0 164 0.883 2.70316416401360 0
2019-Yices 2.6.2n 0 164 1.313 2.73816416401360 0
2018-Yicesn 0 164 1.364 2.77516416401360 0
MathSAT5n 0 164 8.244 8.28816416401360 0
z3n 0 164 8.902 8.92316416401360 0
CVC4 0 164 10.113 10.18116416401360 0
SMTInterpol-fixedn 0 164 131.355 69.41216416401360 0
SMTInterpol 0 164 131.856 69.47316416401360 0
Alt-Ergo 0 0 8158.101 3254.7060003009 0

UNSAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Yices2 0 136 2.921 3.59313601361640 0
Yices2-fixedn 0 136 2.859 3.60213601361640 0
2019-Yices 2.6.2n 0 136 3.042 3.67113601361640 0
2018-Yicesn 0 136 3.162 3.83613601361640 0
MathSAT5n 0 136 11.717 11.74413601361640 0
z3n 0 136 14.306 14.31913601361640 0
CVC4 0 136 78.429 78.42113601361640 0
SMTInterpol 0 136 335.962 138.05713601361640 0
SMTInterpol-fixedn 0 136 338.848 138.74513601361640 0
Alt-Ergo 0 128 44116.455 18280.32912801281729 0

24 seconds Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Yices2 0 300 3.845 6.25630016413600 0
Yices2-fixedn 0 300 3.741 6.30530016413600 0
2019-Yices 2.6.2n 0 300 4.355 6.4130016413600 0
2018-Yicesn 0 300 4.526 6.61130016413600 0
MathSAT5n 0 300 19.961 20.03330016413600 0
z3n 0 300 23.208 23.24230016413600 0
CVC4 0 300 88.543 88.60330016413600 0
SMTInterpol 0 300 467.817 207.53130016413600 0
SMTInterpol-fixedn 0 300 470.203 208.15730016413600 0
Alt-Ergo 0 86 4173.79 2452.3758608621476 0

n Non-competing.