Result Overview

Number of Run Configurations

190

Change of Run time

median 0% (min. -20%, max. 6%)

Performance Changes between Versions

awfy-startup

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
CD		5	283.74	0
Havlak		5	7191.24	1

macro-startup

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
DeltaBlue		5	18.67	0
GraphSearch		5	28.45	1
Json		5	37.01	-1
NBody		5	29.73	0
PageRank		5	42.65	0
Richards		5	130.33	-1

micro-startup

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
Bounce		5	31.97	0
BubbleSort		5	38.54	0
Dispatch		5	31.21	1
Fannkuch		5	25.38	0
Fibonacci		5	46.02	0
FieldLoop		5	15.07	0
IntegerLoop		5	25.37	0
List		5	7.15	0
Loop		5	136.26	2
Mandelbrot		5	48.08	0
Permute		5	55.11	0
Queens		5	43.74	1
QuickSort		5	35.23	2
Recurse		5	39.43	0
Sieve		5	55.07	0
Storage		5	24.67	-3
Sum		5	70.59	1
Test		10	50.13	-1
TestGC		10	38.69	0
Towers		5	19.66	1
TreeSort		5	25.84	0
WhileLoop		5	32.73	0

som-parse

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
SomInit		10	48.52	0
SomParse		10	20.18	0

awfy-startup

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
CD		5	255.48	1
Havlak		5	1270.82	0

awfy-steady

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
CD		130	406.83	1
Havlak		130	990.16	0

macro-startup

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
DeltaBlue		5	43.23	0
GraphSearch		5	32.04	0
Json		5	56.32	0
NBody		5	13.16	0
PageRank		5	15.22	0
Richards		5	73.30	0

macro-steady

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
DeltaBlue		120	82.37	1
GraphSearch		250	160.47	0
Json		120	139.85	0
NBody		120	95.03	-4
PageRank		120	140.97	0
Richards		130	102.75	-2

micro-startup

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
Bounce		5	20.88	1
BubbleSort		5	9.02	0
Dispatch		5	3.05	1
Fannkuch		5	16.50	1
Fibonacci		5	19.41	0
FieldLoop		5	3.47	0
IntegerLoop		5	3.48	0
List		5	6.43	0
Loop		5	17.27	0
Mandelbrot		5	11.24	0
Permute		5	26.60	0
Queens		5	30.58	0
QuickSort		5	19.71	0
Recurse		5	15.16	1
Sieve		5	9.40	0
Storage		5	23.72	0
Sum		5	8.81	0
Test		10	81.23	-2
TestGC		10	38.86	0
Towers		5	12.37	1
TreeSort		5	28.06	-1
WhileLoop		5	6.74	1

micro-steady

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
Bounce		60	290.11	0
BubbleSort		55	145.92	0
Dispatch		55	688.95	0
Fannkuch		55	196.95	0
Fibonacci		60	44.52	0
FieldLoop		55	312.20	0
IntegerLoop		55	999.91	0
List		65	553.57	1
Loop		55	868.71	0
Mandelbrot		110	521.85	0
Permute		55	303.73	-3
Queens		55	462.66	1
QuickSort		55	683.77	1
Recurse		65	24.14	0
Sieve		60	273.61	5
Storage		60	130.07	1
Sum		55	619.67	0
Towers		55	1537.54	4
TreeSort		60	553.54	0
WhileLoop		55	556.17	1

awfy-startup

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
CD		5	305.43	-9
Havlak		5	7379.59	-7

macro-startup

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
DeltaBlue		5	21.73	-5
GraphSearch		5	30.32	-6
Json		5	34.83	-9
NBody		5	33.51	-9
PageRank		5	52.50	-7
Richards		5	135.71	-5

micro-startup

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
Bounce		5	28.55	-8
BubbleSort		5	42.44	-10
Dispatch		5	42.10	-6
Fannkuch		5	29.98	-5
Fibonacci		5	60.85	-6
FieldLoop		5	12.70	4
IntegerLoop		5	34.36	-5
List		5	7.94	-9
Loop		5	194.94	-2
Mandelbrot		5	41.12	-13
Permute		5	55.95	-7
Queens		5	49.24	-4
QuickSort		5	36.32	-11
Recurse		5	34.47	-13
Sieve		5	54.03	-10
Storage		5	28.10	-3
Sum		5	96.95	-5
Test		10	33.99	-4
TestGC		10	10.36	-1
Towers		5	22.95	-5
TreeSort		5	22.73	-7
WhileLoop		5	35.66	-13

som-parse

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
SomInit		10	38.73	-12
SomParse		10	19.24	0

awfy-startup

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
CD		5	620.63	6
Havlak		5	951.33	-5

awfy-steady

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
CD		130	150.83	4
Havlak		130	622.13	1

macro-startup

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
DeltaBlue		5	39.48	-1
GraphSearch		5	38.98	-1
Json		5	55.88	-4
NBody		5	12.38	-1
PageRank		5	13.50	0
Richards		5	74.61	-10

macro-steady

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
DeltaBlue		120	73.47	1
GraphSearch		250	164.09	-3
Json		120	135.72	0
NBody		120	99.61	3
PageRank		120	134.88	1
Richards		130	98.00	-1

micro-startup

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
Bounce		5	19.87	-2
BubbleSort		5	12.32	-1
Dispatch		5	3.45	0
Fannkuch		5	21.55	-1
Fibonacci		5	28.09	-2
FieldLoop		5	3.61	-1
IntegerLoop		5	3.68	0
List		5	12.94	-3
Loop		5	16.53	0
Mandelbrot		5	12.02	-3
Permute		5	29.95	-1
Queens		5	30.21	-1
QuickSort		5	45.35	-3
Recurse		5	19.39	-2
Sieve		5	13.68	-1
Storage		5	26.12	-1
Sum		5	8.81	0
Test		10	57.94	-2
TestGC		10	10.44	-2
Towers		5	57.73	-5
TreeSort		5	43.69	-1
WhileLoop		5	8.64	-1

micro-steady

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
Bounce		60	147.68	0
BubbleSort		55	206.73	-20
Dispatch		55	690.61	0
Fannkuch		55	212.91	0
Fibonacci		60	214.86	1
FieldLoop		55	399.40	0
IntegerLoop		55	984.72	0
List		65	288.25	1
Loop		55	802.48	0
Mandelbrot		110	1000.71	0
Permute		55	326.09	0
Queens		55	218.45	0
QuickSort		55	456.06	0
Recurse		65	108.74	0
Sieve		60	490.21	0
Storage		60	139.22	1
Sum		55	612.11	-1
Towers		55	120.77	0
TreeSort		60	618.02	0
WhileLoop		55	1337.18	0

micro-somsom

Executor: SomSom-ast-interp

		#M	median time in ms	time diff %
List		1	9785.85	-1
Loop		1	6906.07	0
Mandelbrot		1	144.28	1
Queens		1	5892.11	1
Recurse		1	6403.85	1

micro-somsom

Executor: SomSom-bc-interp

		#M	median time in ms	time diff %
List		1	10846.11	-6
Loop		1	7551.35	-6
Mandelbrot		1	151.28	-6
Queens		1	6305.99	-4
Recurse		1	6880.91	-7

Executor Comparisons

awfy-startup

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
CD		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	283.74 255.48 305.43 620.63	0 -10 8 119
Havlak		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	7191.24 1270.82 7379.59 951.33	0 -82 3 -87

macro-startup

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
DeltaBlue		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	18.67 43.23 21.73 39.48	0 132 16 111
GraphSearch		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	28.45 32.04 30.32 38.98	0 13 7 37
Json		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	37.01 56.32 34.83 55.88	0 52 -6 51
NBody		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	29.73 13.16 33.51 12.38	0 -56 13 -58
PageRank		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	42.65 15.22 52.50 13.50	0 -64 23 -68
Richards		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	130.33 73.30 135.71 74.61	0 -44 4 -43

micro-startup

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
Bounce		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	31.97 20.88 28.55 19.87	0 -35 -11 -38
BubbleSort		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	38.54 9.02 42.44 12.32	0 -77 10 -68
Dispatch		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	31.21 3.05 42.10 3.45	0 -90 35 -89
Fannkuch		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	25.38 16.50 29.98 21.55	0 -35 18 -15
Fibonacci		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	46.02 19.41 60.85 28.09	0 -58 32 -39
FieldLoop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	15.07 3.47 12.70 3.61	0 -77 -16 -76
IntegerLoop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	25.37 3.48 34.36 3.68	0 -86 35 -85
List		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	7.15 6.43 7.94 12.94	0 -10 11 81
Loop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	136.26 17.27 194.94 16.53	0 -87 43 -88
Mandelbrot		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	48.08 11.24 41.12 12.02	0 -77 -14 -75
Permute		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	55.11 26.60 55.95 29.95	0 -52 2 -46
Queens		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	43.74 30.58 49.24 30.21	0 -30 13 -31
QuickSort		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	35.23 19.71 36.32 45.35	0 -44 3 29
Recurse		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	39.43 15.16 34.47 19.39	0 -62 -13 -51
Sieve		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	55.07 9.40 54.03 13.68	0 -83 -2 -75
Storage		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	24.67 23.72 28.10 26.12	0 -4 14 6
Sum		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	70.59 8.81 96.95 8.81	0 -88 37 -88
Test		ast-interp ast-jit bc-interp bc-jit	10 10 10 10	50.13 81.23 33.99 57.94	0 62 -32 16
TestGC		ast-interp ast-jit bc-interp bc-jit	10 10 10 10	38.69 38.86 10.36 10.44	0 0 -73 -73
Towers		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	19.66 12.37 22.95 57.73	0 -37 17 194
TreeSort		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	25.84 28.06 22.73 43.69	0 9 -12 69
WhileLoop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	32.73 6.74 35.66 8.64	0 -79 9 -74

som-parse

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
SomInit		ast-interp bc-interp	10 10	48.52 38.73	0 -20
SomParse		ast-interp bc-interp	10 10	20.18 19.24	0 -5

awfy-steady

Baseline: RPySOM-ast-jit

		Exe	#M	median time in ms	time diff %
CD		ast-jit bc-jit	130 130	406.83 150.83	0 -63
Havlak		ast-jit bc-jit	130 130	990.16 622.13	0 -37

macro-steady

Baseline: RPySOM-ast-jit

		Exe	#M	median time in ms	time diff %
DeltaBlue		ast-jit bc-jit	120 120	82.37 73.47	0 -11
GraphSearch		ast-jit bc-jit	250 250	160.47 164.09	0 2
Json		ast-jit bc-jit	120 120	139.85 135.72	0 -3
NBody		ast-jit bc-jit	120 120	95.03 99.61	0 5
PageRank		ast-jit bc-jit	120 120	140.97 134.88	0 -4
Richards		ast-jit bc-jit	130 130	102.75 98.00	0 -5

micro-steady

Baseline: RPySOM-ast-jit

		Exe	#M	median time in ms	time diff %
Bounce		ast-jit bc-jit	60 60	290.11 147.68	0 -49
BubbleSort		ast-jit bc-jit	55 55	145.92 206.73	0 42
Dispatch		ast-jit bc-jit	55 55	688.95 690.61	0 0
Fannkuch		ast-jit bc-jit	55 55	196.95 212.91	0 8
Fibonacci		ast-jit bc-jit	60 60	44.52 214.86	0 383
FieldLoop		ast-jit bc-jit	55 55	312.20 399.40	0 28
IntegerLoop		ast-jit bc-jit	55 55	999.91 984.72	0 -2
List		ast-jit bc-jit	65 65	553.57 288.25	0 -48
Loop		ast-jit bc-jit	55 55	868.71 802.48	0 -8
Mandelbrot		ast-jit bc-jit	110 110	521.85 1000.71	0 92
Permute		ast-jit bc-jit	55 55	303.73 326.09	0 7
Queens		ast-jit bc-jit	55 55	462.66 218.45	0 -53
QuickSort		ast-jit bc-jit	55 55	683.77 456.06	0 -33
Recurse		ast-jit bc-jit	65 65	24.14 108.74	0 350
Sieve		ast-jit bc-jit	60 60	273.61 490.21	0 79
Storage		ast-jit bc-jit	60 60	130.07 139.22	0 7
Sum		ast-jit bc-jit	55 55	619.67 612.11	0 -1
Towers		ast-jit bc-jit	55 55	1537.54 120.77	0 -92
TreeSort		ast-jit bc-jit	60 60	553.54 618.02	0 12
WhileLoop		ast-jit bc-jit	55 55	556.17 1337.18	0 140

micro-somsom

Baseline: SomSom-ast-interp

		Exe	#M	median time in ms	time diff %
List		ast-interp bc-interp	1 1	9785.85 10846.11	0 11
Loop		ast-interp bc-interp	1 1	6906.07 7551.35	0 9
Mandelbrot		ast-interp bc-interp	1 1	144.28 151.28	0 5
Queens		ast-interp bc-interp	1 1	5892.11 6305.99	0 7
Recurse		ast-interp bc-interp	1 1	6403.85 6880.91	0 7