AUFNIRA (Single Query Track)

Competition results for the AUFNIRA logic in the Single Query Track.

Page generated on 2021-07-18 17:29:05 +0000

Benchmarks: 300
Time Limit: 1200 seconds
Memory Limit: 60 GB

Winners

Sequential Performance	Parallel Performance	SAT Performance (parallel)	UNSAT Performance (parallel)	24s Performance (parallel)
Vampire	Vampire	—	Vampire	Vampire

Sequential Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
2019-Par4n	0	56	300845.015	296862.697	56	2	54	244	244	0
2020-CVC4n	0	52	293203.95	298477.024	52	0	52	248	244	0
Vampire - fixedn	0	47	305562.238	303215.291	47	0	47	253	252	0
Vampire	0	46	307064.015	305390.802	46	0	46	254	254	0
2020-Vampiren	0	45	307222.047	305466.221	45	0	45	255	254	0
cvc5 - fixedn	0	42	153041.648	308466.954	42	0	42	258	255	0
cvc5	0	42	153805.642	308462.234	42	0	42	258	255	0
z3n	0	34	206382.214	207683.209	34	3	31	266	97	3
UltimateEliminator+MathSAT	0	1	1532.334	911.18	1	0	1	299	0	0

Parallel Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
2019-Par4n	0	56	300845.015	296862.697	56	2	54	244	244	0
2020-CVC4n	0	52	298328.58	298464.994	52	0	52	248	244	0
Vampire - fixedn	0	51	319457.668	300454.964	51	0	51	249	247	0
Vampire	0	51	312150.045	302215.106	51	0	51	249	249	0
2020-Vampiren	0	49	313550.197	302600.313	49	0	49	251	250	0
cvc5	0	42	308359.479	308449.594	42	0	42	258	255	0
cvc5 - fixedn	0	42	308379.173	308454.114	42	0	42	258	255	0
z3n	0	34	206398.814	207678.759	34	3	31	266	97	3
UltimateEliminator+MathSAT	0	1	1532.334	911.18	1	0	1	299	0	0

SAT Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	N/A	Timeout	Memout
z3n	0	3	0.059	0.06	3	3	0	0	297	97	3
2019-Par4n	0	2	1200.013	1200.036	2	2	0	1	297	244	0
UltimateEliminator+MathSAT	0	0	15.509	8.706	0	0	0	3	297	0	0
2020-CVC4n	0	0	1561.224	1567.465	0	0	0	3	297	244	0
cvc5 - fixedn	0	0	2400.306	2400.322	0	0	0	3	297	255	0
cvc5	0	0	2400.303	2401.116	0	0	0	3	297	255	0
2020-Vampiren	0	0	3600.0	3600.0	0	0	0	3	297	250	0
Vampire	0	0	3600.0	3600.0	0	0	0	3	297	249	0
Vampire - fixedn	0	0	3600.0	3600.0	0	0	0	3	297	247	0

UNSAT Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	N/A	Timeout	Memout
2019-Par4n	0	54	18845.002	14862.661	54	0	54	9	237	244	0
2020-CVC4n	0	52	18363.99	18495.151	52	0	52	11	237	244	0
Vampire - fixedn	0	51	31457.398	16055.214	51	0	51	12	237	247	0
Vampire	0	51	27750.045	17815.106	51	0	51	12	237	249	0
2020-Vampiren	0	49	25550.057	18202.113	49	0	49	14	237	250	0
cvc5	0	42	27555.893	27645.189	42	0	42	21	237	255	0
cvc5 - fixedn	0	42	27575.669	27650.592	42	0	42	21	237	255	0
z3n	0	31	26719.997	26720.913	31	0	31	32	237	97	3
UltimateEliminator+MathSAT	0	1	333.085	200.146	1	0	1	62	237	0	0

24 seconds Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
Vampire	0	42	6922.841	6383.378	42	0	42	258	258	0
Vampire - fixedn	0	41	6938.758	6377.935	41	0	41	259	258	0
2020-Vampiren	0	39	6674.974	6356.337	39	0	39	261	260	0
2019-Par4n	0	36	6403.169	6372.549	36	2	34	264	264	0
z3n	0	34	6377.622	6377.09	34	3	31	266	261	3
2020-CVC4n	0	28	6470.785	6469.826	28	0	28	272	269	0
cvc5 - fixedn	0	27	6520.988	6520.995	27	0	27	273	270	0
cvc5	0	27	6520.981	6521.796	27	0	27	273	270	0
UltimateEliminator+MathSAT	0	1	1532.334	911.18	1	0	1	299	0	0

n Non-competing.
N/A: Benchmarks not known to be SAT/UNSAT, respectively.