SMT-COMP

The International Satisfiability Modulo Theories (SMT) Competition.

GitHub

Home
Introduction
Benchmark Submission
Publications
SMT-LIB
Competitions by Year

SMT-COMP 2020

Rules
Benchmarks
Tools
Specs
Participants
Results
Slides

QF_ABV (Single Query Track)

Competition results for the QF_ABV division in the Single Query Track.

Page generated on 2020-07-04 11:46:58 +0000

Benchmarks: 3385
Time Limit: 1200 seconds
Memory Limit: 60 GB

Winners

Sequential PerformanceParallel PerformanceSAT Performance (parallel)UNSAT Performance (parallel)24s Performance (parallel)
BitwuzlaBitwuzlaBitwuzla Bitwuzla Bitwuzla

Sequential Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Bitwuzla 0 3370 30617.418 30633.0173370231110591515 0
Bitwuzla-fixedn 0 3370 30645.93 30643.5493370231110591515 0
2019-Boolectorn 0 3369 30976.372 30972.0573369231110581616 0
2019-Par4n 0 3360 43289.226 34638.9063360230510552525 0
Yices2 0 3358 54245.844 54271.4573358230510532727 0
Yices2-fixedn 0 3358 54258.657 54295.5693358230510532727 0
MathSAT5n 0 3311 108307.431 108303.6993311227810337473 1
CVC4 0 3309 104284.532 106386.893309228310267676 0
z3n 0 3284 141860.837 141826.453328422611023101101 0

Parallel Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Bitwuzla 0 3370 30619.818 30632.3973370231110591515 0
Bitwuzla-fixedn 0 3370 30647.86 30643.0693370231110591515 0
2019-Boolectorn 0 3369 30977.892 30971.4073369231110581616 0
2019-Par4n 0 3366 46773.206 31145.953366230810581919 0
Yices2 0 3358 54249.134 54270.3173358230510532727 0
Yices2-fixedn 0 3358 54262.257 54294.4693358230510532727 0
MathSAT5n 0 3311 108329.961 108300.4093311227810337473 1
CVC4 0 3309 106021.592 106383.013309228310267676 0
z3n 0 3284 141873.987 141822.753328422611023101101 0

SAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2019-Boolectorn 0 2311 5497.682 5487.168231123110107416 0
Bitwuzla-fixedn 0 2311 6018.941 6005.27231123110107415 0
Bitwuzla 0 2311 6009.253 6012.523231123110107415 0
2019-Par4n 0 2308 14271.018 7450.359230823080107719 0
Yices2 0 2305 13990.207 13999.721230523050108027 0
Yices2-fixedn 0 2305 14005.894 14026.538230523050108027 0
CVC4 0 2283 43169.393 43506.049228322830110276 0
MathSAT5n 0 2278 51014.545 50988.625227822780110773 1
z3n 0 2261 72328.623 72290.012261226101124101 0

UNSAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Bitwuzla 0 1059 24610.564 24619.873105901059232615 0
Bitwuzla-fixedn 0 1059 24628.918 24637.798105901059232615 0
2019-Par4n 0 1058 32502.189 23695.591105801058232719 0
2019-Boolectorn 0 1058 25480.21 25484.239105801058232716 0
Yices2-fixedn 0 1053 40256.362 40267.931105301053233227 0
Yices2 0 1053 40258.927 40270.596105301053233227 0
MathSAT5n 0 1033 57315.416 57311.785103301033235273 1
CVC4 0 1026 62852.199 62876.961102601026235976 0
z3n 0 1023 69545.364 69532.7441023010232362101 0

24 seconds Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2019-Par4n 0 3328 3205.756 2104.3793328229310355757 0
Bitwuzla 0 3310 2913.459 2923.1843310227710337575 0
Bitwuzla-fixedn 0 3310 2936.436 2929.1313310227710337575 0
2019-Boolectorn 0 3306 2991.139 2982.3123306227510317979 0
Yices2-fixedn 0 3283 3528.368 3557.15328322691014102102 0
Yices2 0 3283 3539.439 3557.732328322691014102102 0
CVC4 0 3231 6248.407 6218.17732312232999154154 0
z3n 0 3217 5664.53 5641.78832172222995168168 0
MathSAT5n 0 3216 5929.565 5895.60732162218998169168 1

n Non-competing.