SMT-COMP

The International Satisfiability Modulo Theories (SMT) Competition.

GitHub

Home
Introduction
Benchmark Submission
Previous Competitions
SMT-LIB

SMT-COMP 2019

Rules
Benchmarks
Tools
Specs
Participants
Results
Slides

FP (Single Query Track)

Competition results for the FP division in the Single Query Track.

Page generated on 2019-07-23 17:56:09 +0000

Benchmarks: 1234
Time Limit: 2400 seconds
Memory Limit: 60 GB

Winners

Sequential PerformanceParallel PerformanceSAT Performance (parallel)UNSAT Performance (parallel)24s Performance (parallel)
CVC4CVC4CVC4 CVC4 CVC4

Sequential Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Z3n 0 1037 506948.197 506921.041037201017197137 48
2018-CVC4n 0 880 722150.807 722220.70288010870354260 4
CVC4 0 865 643828.961 643879.55586510855369254 9
UltimateEliminator+MathSAT-5.5.4 0 179 4129.212 2878.785179017910550 0

Parallel Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Z3n 0 1037 506987.757 506914.631037201017197137 48
2018-CVC4n 0 880 722233.327 722210.34288010870354260 4
CVC4 0 865 643910.121 643867.34586510855369254 9
UltimateEliminator+MathSAT-5.5.4 0 179 4129.212 2878.785179017910550 0

SAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Z3n 0 20 1516.418 1516.735202001214137 48
2018-CVC4n 0 10 24020.056 24020.054101001224260 4
CVC4 0 10 24022.688 24022.688101001224254 9
UltimateEliminator+MathSAT-5.5.4 0 0 66.95 47.67600012340 0

UNSAT Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Z3n 0 1017 176651.752 176577.957101701017217137 48
2018-CVC4n 0 870 357413.271 357390.2878700870364260 4
CVC4 0 855 279087.432 279044.6578550855379254 9
UltimateEliminator+MathSAT-5.5.4 0 179 3589.085 2501.181179017910550 0

24 seconds Performance

Solver Error Score Correct Score CPU Time Score Wall Time ScoreSolvedSolved SATSolved UNSATUnsolvedTimeout Memout
Z3n 0 870 10130.353 10084.22287017853364305 48
2018-CVC4n 0 794 12016.346 11987.32379410784440436 4
CVC4 0 786 9666.161 9665.2678610776448333 9
UltimateEliminator+MathSAT-5.5.4 0 179 4129.212 2878.785179017910550 0

n Non-competing.