SMT-COMP

The International Satisfiability Modulo Theories (SMT) Competition.

GitHub

Home
Introduction
Benchmark Submission
Publications
SMT-LIB
Previous Editions

SMT-COMP 2023

Rules
Benchmarks
Specs
Model Validation Track
Proof Exhibition Track
Parallel & Cloud Tracks
Participants
Results
Statistics
Comparisons
Slides

BV (Single Query Track)

Competition results for the BV logic in the Single Query Track.

Page generated on 2023-07-06 16:04:53 +0000

Benchmarks: 1040
Time Limit: 1200 seconds
Memory Limit: 60 GB

Winners

Sequential PerformanceParallel PerformanceSAT Performance (parallel)UNSAT Performance (parallel)24s Performance (parallel)
cvc5cvc5Bitwuzla cvc5 Bitwuzla

Sequential Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2019-Par4n 0 961 11999.273 3171.2279612527097979 0
cvc5 0 937 34547.172 35619.028937246691103103 0
Bitwuzla 0 838 10821.129 10804.53838255583202202 0
Bitwuzla Fixedn 0 838 10824.968 10799.985838255583202202 0
YicesQS 0 749 4938.411 4938.946749211538291290 0
UltimateEliminator+MathSAT 0 310 4241.81 3240.70131029281730138 12
UltimateIntBlastingWrapper+SMTInterpol 0 5 82.875 34.6155141035210 0
Q3B 4 889 10354.898 4366.962889243646151146 0

Parallel Performance

Solver Error Score Correct ScoreCPU Time ScoreWall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2019-Par4n 0 96825562.3236627.599682557137272 0
cvc5 0 93734547.17235619.028937246691103103 0
Bitwuzla Fixedn 0 83810824.96810799.985838255583202202 0
Bitwuzla 0 83810821.12910804.53838255583202202 0
YicesQS 0 7494938.4114938.946749211538291290 0
UltimateEliminator+MathSAT 0 3104241.813240.70131029281730137 12
UltimateIntBlastingWrapper+SMTInterpol 0 582.87534.6155141035208 0
Q3B 4 89624903.6389234.061896248648144139 0

SAT Performance

Solver Error Score Correct ScoreCPU Time ScoreWall Time ScoreSolvedSolved SATSolved UNSATUnsolvedN/ATimeout Memout
Bitwuzla 0 2552482.1042482.427255255017768202 0
Bitwuzla Fixedn 0 2552511.1532498.285255255017768202 0
2019-Par4n 0 25511604.9593019.36525525501776872 0
cvc5 0 24623401.59924066.333246246026768103 0
YicesQS 0 2112456.7632457.058211211061768290 0
UltimateEliminator+MathSAT 0 291836.5051561.6829290243768137 12
UltimateIntBlastingWrapper+SMTInterpol 0 111.3864.401110271768208 0
Q3B 4 24816618.0596415.328248248024768139 0

UNSAT Performance

Solver Error Score Correct ScoreCPU Time ScoreWall Time ScoreSolvedSolved SATSolved UNSATUnsolvedN/ATimeout Memout
2019-Par4n 0 71313957.3643608.22571307133729072 0
cvc5 0 69111145.57411552.695691069159290103 0
Q3B 0 6488285.5792818.7336480648102290139 0
Bitwuzla Fixedn 0 5838313.8158301.75830583167290202 0
Bitwuzla 0 5838339.0258322.1035830583167290202 0
YicesQS 0 5382481.6472481.8895380538212290290 0
UltimateEliminator+MathSAT 0 2812405.3051679.0222810281469290137 12
UltimateIntBlastingWrapper+SMTInterpol 0 471.48930.214404746290208 0

24 seconds Performance

Solver Error Score Correct ScoreCPU Time ScoreWall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2019-Par4n 0 9342168.49629.077934239695106106 0
Bitwuzla Fixedn 0 772885.443858.685772226546268268 0
Bitwuzla 0 772876.927858.945772226546268268 0
cvc5 0 7591207.0581178.819759139620281281 0
YicesQS 0 730311.585311.618730205525310310 0
UltimateEliminator+MathSAT 0 2911842.1931040.09729120271749192 12
UltimateIntBlastingWrapper+SMTInterpol 0 582.87534.6155141035288 0
Q3B 4 8562190.452831.491856225631184180 0

n Non-competing.
N/A: Benchmarks not known to be SAT/UNSAT, respectively.