NIA (Single Query Track)

Competition results for the NIA logic in the Single Query Track.

Page generated on 2022-08-10 11:17:44 +0000

Benchmarks: 208
Time Limit: 1200 seconds
Memory Limit: 60 GB

Winners

Sequential Performance	Parallel Performance	SAT Performance (parallel)	UNSAT Performance (parallel)	24s Performance (parallel)
cvc5	cvc5	cvc5	cvc5	cvc5

Sequential Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
cvc5	0	190	25220.459	25260.87	190	68	122	18	18	0
UltimateEliminator+MathSAT	0	129	26326.053	25870.922	129	33	96	79	21	0
z3-4.8.17n	0	88	141326.125	141340.461	88	60	28	120	117	0
2021-z3n	0	87	48926.732	48934.827	87	54	33	121	39	0
YicesQS	0	80	153872.096	153895.272	80	55	25	128	128	0
Vampire	0	63	181844.456	175985.7	63	0	63	145	145	0

Parallel Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
cvc5	0	190	25242.469	25259.85	190	68	122	18	18	0
UltimateEliminator+MathSAT	0	129	26326.053	25870.922	129	33	96	79	21	0
z3-4.8.17n	0	88	141335.905	141335.961	88	60	28	120	117	0
2021-z3n	0	87	48935.932	48933.227	87	54	33	121	39	0
YicesQS	0	80	153889.946	153889.982	80	55	25	128	128	0
Vampire	0	66	202145.386	173845.181	66	0	66	142	142	0

SAT Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	N/A	Timeout	Memout
cvc5	0	68	4144.306	4147.203	68	68	0	2	138	18	0
z3-4.8.17n	0	60	12075.503	12075.404	60	60	0	10	138	117	0
YicesQS	0	55	18174.537	18174.567	55	55	0	15	138	128	0
2021-z3n	0	54	8448.436	8447.319	54	54	0	16	138	39	0
UltimateEliminator+MathSAT	0	33	11246.533	11081.465	33	33	0	37	138	21	0
Vampire	0	0	98400.89	83984.88	0	0	0	70	138	142	0

UNSAT Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	N/A	Timeout	Memout
cvc5	0	122	21098.162	21112.647	122	0	122	16	70	18	0
UltimateEliminator+MathSAT	0	96	15079.52	14789.458	96	0	96	42	70	21	0
Vampire	0	66	103744.496	89860.301	66	0	66	72	70	142	0
2021-z3n	0	33	40487.496	40485.908	33	0	33	105	70	39	0
z3-4.8.17n	0	28	129260.403	129260.557	28	0	28	110	70	117	0
YicesQS	0	25	135715.409	135715.415	25	0	25	113	70	128	0

24 seconds Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
cvc5	0	177	828.41	827.347	177	62	115	31	31	0
UltimateEliminator+MathSAT	0	129	1538.861	1102.878	129	33	96	79	22	0
2021-z3n	0	86	1252.805	1249.655	86	53	33	122	45	0
z3-4.8.17n	0	85	2969.507	2969.361	85	59	26	123	123	0
YicesQS	0	77	3253.732	3253.738	77	53	24	131	131	0
Vampire	0	38	4420.68	4171.558	38	0	38	170	170	0

n Non-competing.
N/A: Benchmarks not known to be SAT/UNSAT, respectively.