SMT-COMP

The International Satisfiability Modulo Theories (SMT) Competition.

GitHub

Home
Introduction
Benchmark Submission
Publications
SMT-LIB
Previous Editions

SMT-COMP 2020

Rules
Benchmarks
Tools
Specs
Participants
Results
Slides

QF_ABV (Single Query Track)

Competition results for the QF_ABV division in the Single Query Track.

Page generated on 2020-07-04 11:46:58 +0000

Benchmarks: 3385
Time Limit: 1200 seconds
Memory Limit: 60 GB

Winners

Sequential PerformanceParallel PerformanceSAT Performance (parallel)UNSAT Performance (parallel)24s Performance (parallel)
BitwuzlaBitwuzlaBitwuzla Bitwuzla Bitwuzla

Sequential Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedAbstainedTimeout Memout
Bitwuzla 0 3370 30617.418 30633.0173370231110591515 0
Bitwuzla-fixedn 0 3370 30645.93 30643.5493370231110591515 0
2019-Boolectorn 0 3369 30976.372 30972.0573369231110581616 0
2019-Par4n 0 3360 43289.226 34638.9063360230510552525 0
Yices2 0 3358 54245.844 54271.4573358230510532727 0
Yices2-fixedn 0 3358 54258.657 54295.5693358230510532727 0
MathSAT5n 0 3311 108307.431 108303.6993311227810337473 1
CVC4 0 3309 104284.532 106386.893309228310267676 0
z3n 0 3284 141860.837 141826.453328422611023101101 0

Parallel Performance

Solver Error Score Correct ScoreCPU Time ScoreWall Time ScoreSolvedSolved SATSolved UNSATUnsolvedAbstainedTimeout Memout
Bitwuzla 0 337030619.81830632.3973370231110591515 0
Bitwuzla-fixedn 0 337030647.8630643.0693370231110591515 0
2019-Boolectorn 0 336930977.89230971.4073369231110581616 0
2019-Par4n 0 336646773.20631145.953366230810581919 0
Yices2 0 335854249.13454270.3173358230510532727 0
Yices2-fixedn 0 335854262.25754294.4693358230510532727 0
MathSAT5n 0 3311108329.961108300.4093311227810337473 1
CVC4 0 3309106021.592106383.013309228310267676 0
z3n 0 3284141873.987141822.753328422611023101101 0

SAT Performance

Solver Error Score Correct ScoreCPU Time ScoreWall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2019-Boolectorn 0 23115497.6825487.168231123110107416 0
Bitwuzla-fixedn 0 23116018.9416005.27231123110107415 0
Bitwuzla 0 23116009.2536012.523231123110107415 0
2019-Par4n 0 230814271.0187450.359230823080107719 0
Yices2 0 230513990.20713999.721230523050108027 0
Yices2-fixedn 0 230514005.89414026.538230523050108027 0
CVC4 0 228343169.39343506.049228322830110276 0
MathSAT5n 0 227851014.54550988.625227822780110773 1
z3n 0 226172328.62372290.012261226101124101 0

UNSAT Performance

Solver Error Score Correct ScoreCPU Time ScoreWall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Bitwuzla 0 105924610.56424619.873105901059232615 0
Bitwuzla-fixedn 0 105924628.91824637.798105901059232615 0
2019-Par4n 0 105832502.18923695.591105801058232719 0
2019-Boolectorn 0 105825480.2125484.239105801058232716 0
Yices2-fixedn 0 105340256.36240267.931105301053233227 0
Yices2 0 105340258.92740270.596105301053233227 0
MathSAT5n 0 103357315.41657311.785103301033235273 1
CVC4 0 102662852.19962876.961102601026235976 0
z3n 0 102369545.36469532.7441023010232362101 0

24 seconds Performance

Solver Error Score Correct ScoreCPU Time ScoreWall Time ScoreSolvedSolved SATSolved UNSATUnsolvedAbstainedTimeout Memout
2019-Par4n 0 33283205.7562104.3793328229310355757 0
Bitwuzla 0 33102913.4592923.1843310227710337575 0
Bitwuzla-fixedn 0 33102936.4362929.1313310227710337575 0
2019-Boolectorn 0 33062991.1392982.3123306227510317979 0
Yices2-fixedn 0 32833528.3683557.15328322691014102102 0
Yices2 0 32833539.4393557.732328322691014102102 0
CVC4 0 32316248.4076218.17732312232999154154 0
z3n 0 32175664.535641.78832172222995168168 0
MathSAT5n 0 32165929.5655895.60732162218998169168 1

n Non-competing.
Abstained: Total of benchmarks in logics in this division that solver chose to abstain from. For SAT/UNSAT scores, this column also includes benchmarks not known to be SAT/UNSAT.