SMT-COMP

The International Satisfiability Modulo Theories (SMT) Competition.

GitHub

Home
Introduction
Benchmark Submission
Publications
SMT-LIB
Competitions by Year

SMT-COMP 2020

Rules
Benchmarks
Tools
Specs
Participants
Results
Slides

FP (Single Query Track)

Competition results for the FP division in the Single Query Track.

Page generated on 2020-07-04 11:46:58 +0000

Benchmarks: 986
Time Limit: 1200 seconds
Memory Limit: 60 GB

Winners

Sequential PerformanceParallel PerformanceSAT Performance (parallel)UNSAT Performance (parallel)24s Performance (parallel)
CVC4CVC4CVC4 CVC4 CVC4

Note: the division has disagreements

Sequential Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2019-Z3n 0 813 218799.989 218835.14481325788173124 31
z3n 0 806 224884.275 224889.22680624782180163 0
CVC4 0 680 265293.673 265346.01768014666306213 0
UltimateEliminator+MathSAT 0 152 3352.29 2337.127152111418340 0

Parallel Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2019-Z3n 0 813 218824.549 218829.32481325788173124 31
z3n 0 806 224915.435 224881.63680624782180163 0
CVC4 0 680 265340.503 265336.90768014666306213 0
UltimateEliminator+MathSAT 0 152 3352.29 2337.127152111418340 0

SAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2019-Z3n 0 25 1437.935 1438.21825250961124 31
z3n 0 24 2180.437 2180.47824240962163 0
CVC4 0 14 13218.915 13218.91214140972213 0
UltimateEliminator+MathSAT 0 11 90.779 68.864111109750 0

UNSAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2019-Z3n 0 788 84142.153 84146.6447880788198124 31
z3n 0 782 91782.797 91748.9487820782204163 0
CVC4 0 666 109321.587 109317.9946660666320213 0
UltimateEliminator+MathSAT 0 141 2862.78 1989.83614101418450 0

24 seconds Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
z3n 0 699 7794.006 7783.35269921678287272 0
2019-Z3n 0 688 8210.327 8208.94568818670298249 31
CVC4 0 613 7785.67 7780.25261314599373280 0
UltimateEliminator+MathSAT 0 152 3352.29 2337.127152111418340 0

n Non-competing.