QF_FP (Single Query Track)

Competition results for the QF_FP logic in the Single Query Track.

Page generated on 2021-07-18 17:29:06 +0000

Benchmarks: 300
Time Limit: 1200 seconds
Memory Limit: 60 GB

Winners

Sequential Performance	Parallel Performance	SAT Performance (parallel)	UNSAT Performance (parallel)	24s Performance (parallel)
Bitwuzla	Bitwuzla	Bitwuzla	Bitwuzla	Bitwuzla

Sequential Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
2020-Bitwuzlan	0	274	51746.792	51752.84	274	188	86	26	26	0
2020-Bitwuzla-fixedn	0	272	51765.827	51722.77	272	188	84	28	28	0
Bitwuzla	0	266	56741.944	56744.906	266	184	82	34	34	0
COLIBRI - fixedn	0	260	49383.872	49389.879	260	176	84	40	40	0
2020-COLIBRIn	0	257	52837.077	52834.672	257	174	83	43	43	0
cvc5	0	257	69355.166	69310.956	257	184	73	43	43	0
MathSAT5n	0	244	80834.499	80828.752	244	179	65	56	56	0
2020-MathSAT5n	0	244	81443.832	81437.766	244	179	65	56	56	0
2020-CVC4n	0	237	88892.858	88907.525	237	174	63	63	63	0
z3n	0	211	121312.62	126631.855	211	155	56	89	72	5
COLIBRI	1	260	47662.963	47664.309	260	173	87	40	39	0

Parallel Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
2020-Bitwuzlan	0	274	51750.452	51751.85	274	188	86	26	26	0
2020-Bitwuzla-fixedn	0	272	51768.457	51722.04	272	188	84	28	28	0
Bitwuzla	0	266	56745.494	56744.086	266	184	82	34	34	0
COLIBRI - fixedn	0	260	49387.662	49388.689	260	176	84	40	40	0
2020-COLIBRIn	0	257	52841.427	52833.162	257	174	83	43	43	0
cvc5	0	257	69361.656	69309.336	257	184	73	43	43	0
MathSAT5n	0	244	80911.059	80826.262	244	179	65	56	56	0
2020-MathSAT5n	0	244	81454.442	81435.596	244	179	65	56	56	0
2020-CVC4n	0	237	88905.178	88904.525	237	174	63	63	63	0
z3n	0	211	121327.86	126628.535	211	155	56	89	72	5
COLIBRI	1	260	47665.323	47663.379	260	173	87	40	39	0

SAT Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	N/A	Timeout	Memout
2020-Bitwuzla-fixedn	0	188	10055.867	10007.835	188	188	0	2	110	28	0
2020-Bitwuzlan	0	188	10042.312	10041.701	188	188	0	2	110	26	0
Bitwuzla	0	184	12731.11	12732.25	184	184	0	6	110	34	0
cvc5	0	184	13953.9	13955.686	184	184	0	6	110	43	0
MathSAT5n	0	179	18966.835	18927.686	179	179	0	11	110	56	0
2020-MathSAT5n	0	179	19260.693	19247.669	179	179	0	11	110	56	0
COLIBRI - fixedn	0	176	17473.027	17473.672	176	176	0	14	110	40	0
2020-COLIBRIn	0	174	19782.071	19773.663	174	174	0	16	110	43	0
2020-CVC4n	0	174	27026.261	27027.64	174	174	0	16	110	63	0
COLIBRI	0	173	19697.546	19695.118	173	173	0	17	110	39	0
z3n	0	155	48370.721	53670.059	155	155	0	35	110	72	5

UNSAT Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	N/A	Timeout	Memout
2020-Bitwuzlan	0	86	33308.14	33310.149	86	0	86	17	197	26	0
COLIBRI - fixedn	0	84	23514.635	23515.018	84	0	84	19	197	40	0
2020-Bitwuzla-fixedn	0	84	33312.59	33314.205	84	0	84	19	197	28	0
2020-COLIBRIn	0	83	24659.356	24659.499	83	0	83	20	197	43	0
Bitwuzla	0	82	35614.384	35611.836	82	0	82	21	197	34	0
cvc5	0	73	47007.756	46953.65	73	0	73	30	197	43	0
MathSAT5n	0	65	53544.225	53498.576	65	0	65	38	197	56	0
2020-MathSAT5n	0	65	53793.749	53787.928	65	0	65	38	197	56	0
2020-CVC4n	0	63	53478.917	53476.885	63	0	63	40	197	63	0
z3n	0	56	64557.139	64558.476	56	0	56	47	197	72	5
COLIBRI	1	87	19567.777	19568.261	87	0	87	16	197	39	0

24 seconds Performance

Solver	Error Score	Correct Score	CPU Time Score	Wall Time Score	Solved	Solved SAT	Solved UNSAT	Unsolved	Timeout	Memout
COLIBRI - fixedn	0	246	1687.908	1688.737	246	171	75	54	54	0
2020-COLIBRIn	0	244	1791.281	1782.967	244	172	72	56	56	0
Bitwuzla	0	213	2498.805	2494.683	213	157	56	87	87	0
2020-Bitwuzla-fixedn	0	212	2508.75	2507.764	212	156	56	88	88	0
2020-Bitwuzlan	0	212	2510.738	2508.455	212	156	56	88	88	0
cvc5	0	191	3309.996	3310.125	191	143	48	109	109	0
2020-CVC4n	0	183	3344.2	3340.655	183	137	46	117	117	0
2020-MathSAT5n	0	175	3562.321	3540.729	175	137	38	125	125	0
MathSAT5n	0	174	3596.978	3576.929	174	139	35	126	126	0
z3n	0	129	4455.358	4455.417	129	107	22	171	166	5
COLIBRI	1	253	1530.592	1528.6	253	169	84	47	46	0

n Non-competing.
N/A: Benchmarks not known to be SAT/UNSAT, respectively.