SMT-COMP 2019

The International Satisfiability Modulo Theories (SMT) Competition.

GitHub

Home
Introduction
Papers
Benchmark Submission
Rules
Benchmarks
Tools
Specs
Participants
Results
Slides

Previous Competitions

SMT-LIB

BVFP (Single Query Track)

Competition results for the BVFP division in the Single Query Track.

Page generated on 2019-07-23 17:56:09 +0000

Benchmarks: 24
Time Limit: 2400 seconds
Memory Limit: 60 GB

Winners

Sequential PerformanceParallel PerformanceSAT Performance (parallel)UNSAT Performance (parallel)24s Performance (parallel)
CVC4CVC4CVC4 CVC4 CVC4

Sequential Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Z3n 0 24 1416.267 1423.6382420400 0
CVC4 0 8 38402.852 38405.9818441616 0
2018-CVC4n 0 8 38403.088 38406.4078441616 0
UltimateEliminator+MathSAT-5.5.4 0 1 81.649 57.463110230 0

Parallel Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Z3n 0 24 1416.267 1423.6382420400 0
CVC4 0 8 38405.382 38405.3818441616 0
2018-CVC4n 0 8 38406.008 38406.0078441616 0
UltimateEliminator+MathSAT-5.5.4 0 1 81.649 57.463110230 0

SAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Z3n 0 20 1211.717 1219.0472020040 0
CVC4 0 4 38402.871 38402.8714402016 0
2018-CVC4n 0 4 38403.227 38403.2264402016 0
UltimateEliminator+MathSAT-5.5.4 0 1 68.185 47.71110230 0

UNSAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
CVC4 0 4 2.511 2.514042016 0
2018-CVC4n 0 4 2.78 2.7814042016 0
Z3n 0 4 204.55 204.591404200 0
UltimateEliminator+MathSAT-5.5.4 0 0 13.464 9.753000240 0

24 seconds Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Z3n 0 12 337.37 337.37212931212 0
CVC4 0 8 389.382 389.3818441616 0
2018-CVC4n 0 8 390.008 390.0078441616 0
UltimateEliminator+MathSAT-5.5.4 0 1 81.649 57.463110230 0

n Non-competing.