FP (Single Query Track)

Competition results for the FP logic in the Single Query Track.

Page generated on 2021-07-18 17:29:05 +0000

Benchmarks: 1340
Time Limit: 1200 seconds
Memory Limit: 60 GB

Winners

Sequential Performance	Parallel Performance	SAT Performance (parallel)	UNSAT Performance (parallel)	24s Performance (parallel)
cvc5	cvc5	cvc5	cvc5	cvc5

Sequential Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
cvc5	0	1188	217862.278	230857.823	1188	107	1081	152	152	0
z3n	0	1105	350499.091	350541.589	1105	90	1015	235	232	0
2020-CVC4n	0	948	370746.439	370806.684	948	95	853	392	276	0
UltimateEliminator+MathSAT	0	177	37601.704	35849.439	177	4	173	1163	24	0
2019-Z3n	0	0	335345.391	335326.613	0	0	0	1340	192	36

Parallel Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
cvc5	0	1188	229822.009	230850.003	1188	107	1081	152	152	0
z3n	0	1105	350581.881	350531.859	1105	90	1015	235	232	0
2020-CVC4n	0	948	370801.279	370793.854	948	95	853	392	276	0
UltimateEliminator+MathSAT	0	177	37601.704	35849.439	177	4	173	1163	24	0
2019-Z3n	0	0	335382.701	335318.433	0	0	0	1340	192	36

SAT Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	N/A	Timeout	Memout
cvc5	0	107	35750.714	35761.371	107	107	0	11	1222	152	0
2020-CVC4n	0	95	57598.248	57602.994	95	95	0	23	1222	276	0
z3n	0	90	67278.528	67304.999	90	90	0	28	1222	232	0
UltimateEliminator+MathSAT	0	4	537.19	832.268	4	4	0	114	1222	24	0
2019-Z3n	0	0	52414.578	52413.534	0	0	0	118	1222	192	36

UNSAT Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	N/A	Timeout	Memout
cvc5	0	1081	58471.295	59488.632	1081	0	1081	28	231	152	0
z3n	0	1015	151282.089	151205.592	1015	0	1015	94	231	232	0
2020-CVC4n	0	853	177603.031	177590.86	853	0	853	256	231	276	0
UltimateEliminator+MathSAT	0	173	6275.988	4297.752	173	0	173	936	231	24	0
2019-Z3n	0	0	151683.714	151620.298	0	0	0	1109	231	192	36

24 seconds Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
cvc5	0	1037	9083.521	9027.347	1037	46	991	303	303	0
z3n	0	877	12964.872	12936.664	877	5	872	463	460	0
2020-CVC4n	0	812	11650.923	11637.099	812	22	790	528	412	0
UltimateEliminator+MathSAT	0	177	7394.922	5652.63	177	4	173	1163	27	0
2019-Z3n	0	0	13025.605	12996.483	0	0	0	1340	430	36

n Non-competing.
N/A: Benchmarks not known to be SAT/UNSAT, respectively.