SMT-COMP

The International Satisfiability Modulo Theories (SMT) Competition.

GitHub

Home
Introduction
Benchmark Submission
Previous Competitions
SMT-LIB

SMT-COMP 2019

Rules
Benchmarks
Tools
Specs
Participants
Results
Slides

NIA (Single Query Track)

Competition results for the NIA division in the Single Query Track.

Page generated on 2019-07-23 17:56:09 +0000

Benchmarks: 11
Time Limit: 2400 seconds
Memory Limit: 60 GB

Winners

Sequential PerformanceParallel PerformanceSAT Performance (parallel)UNSAT Performance (parallel)24s Performance (parallel)
CVC4CVC4CVC4 CVC4 CVC4

Sequential Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2018-Z3n 0 11 3.127 3.128119200 0
Z3n 0 10 2400.433 2400.672109111 0
CVC4 0 9 0.754 0.7598120 0
UltimateEliminator+MathSAT-5.5.4 0 8 7494.058 7425.48487133 0
ProB 0 5 9800.212 9801.5155063 0
UltimateEliminator+Yices-2.6.1 0 1 702.906 655.183110100 0
Vampire 0 1 24161.63 24040.8391011010 0

Parallel Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2018-Z3n 0 11 3.127 3.128119200 0
Z3n 0 10 2400.593 2400.592109111 0
CVC4 0 9 0.754 0.7598120 0
UltimateEliminator+MathSAT-5.5.4 0 8 7494.058 7425.48487133 0
ProB 0 5 9800.942 9801.3655063 0
UltimateEliminator+Yices-2.6.1 0 1 702.906 655.183110100 0
Vampire 0 1 24161.63 24040.8391011010 0

SAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2018-Z3n 0 9 0.494 0.49499020 0
Z3n 0 9 0.532 0.53299021 0
CVC4 0 8 0.523 0.5288030 0
UltimateEliminator+MathSAT-5.5.4 0 7 4998.368 4961.82977043 0
ProB 0 5 6889.504 6889.92155063 0
UltimateEliminator+Yices-2.6.1 0 1 692.464 649.475110100 0
Vampire 0 0 21600.0 21600.00001110 0

UNSAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2018-Z3n 0 2 2.634 2.63420290 0
CVC4 0 1 0.231 0.23101100 0
Z3n 0 1 2400.06 2400.06101101 0
Vampire 0 1 2561.63 2440.8391011010 0
UltimateEliminator+MathSAT-5.5.4 0 1 2495.69 2463.655101103 0
UltimateEliminator+Yices-2.6.1 0 0 10.442 5.709000110 0
ProB 0 0 2911.438 2911.439000113 0

24 seconds Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
2018-Z3n 0 11 3.127 3.128119200 0
Z3n 0 10 24.593 24.592109111 0
CVC4 0 9 0.754 0.7598120 0
UltimateEliminator+MathSAT-5.5.4 0 6 139.781 133.84966055 0
ProB 0 5 128.674 128.78155065 0
UltimateEliminator+Yices-2.6.1 0 1 61.506 46.867110101 0
Vampire 0 0 264.0 264.00001111 0

n Non-competing.