Result Overview

Number of Run Configurations

178

Change of Run time

median 0% (min. -52%, max. 155%)

Performance Changes between Versions

macro-startup

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
DeltaBlue		5	16.68	-6
GraphSearch		5	24.20	-9
Json		5	36.35	0
NBody		5	25.77	-7
PageRank		5	34.59	-11
Richards		5	126.71	0

micro-startup

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
Bounce		5	31.30	1
BubbleSort		5	29.83	-16
Dispatch		5	22.65	-20
Fannkuch		5	20.84	-15
Fibonacci		5	43.02	-2
FieldLoop		5	14.27	0
IntegerLoop		5	24.75	-3
List		5	6.87	1
Loop		5	60.88	-52
Mandelbrot		5	47.41	1
Permute		5	52.38	1
Queens		5	38.67	-8
QuickSort		5	34.04	-3
Recurse		5	37.84	-3
Sieve		5	46.83	-10
Storage		5	24.75	1
Sum		5	40.59	-38
Test		10	49.21	-10
TestGC		10	36.40	-6
Towers		5	18.76	2
TreeSort		5	25.18	3
WhileLoop		5	32.77	-4

som-parse

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
SomInit		10	48.48	1
SomParse		10	19.19	0

macro-startup

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
DeltaBlue		5	38.99	-10
GraphSearch		5	29.33	-8
Json		5	59.48	1
NBody		5	11.54	-13
PageRank		5	13.06	-15
Richards		5	74.56	4

macro-steady

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
DeltaBlue		120	82.86	2
GraphSearch		250	158.88	0
Json		120	142.58	0
NBody		120	117.83	25
PageRank		120	116.49	-12
Richards		130	100.76	5

micro-startup

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
Bounce		5	20.30	-4
BubbleSort		5	10.57	14
Dispatch		5	3.20	9
Fannkuch		5	17.28	4
Fibonacci		5	19.84	0
FieldLoop		5	3.41	0
IntegerLoop		5	3.39	0
List		5	6.42	0
Loop		5	15.34	-6
Mandelbrot		5	11.13	1
Permute		5	27.35	0
Queens		5	28.61	-8
QuickSort		5	20.25	1
Recurse		5	15.99	1
Sieve		5	8.81	-7
Storage		5	26.02	0
Sum		5	8.58	1
Test		10	85.82	-3
TestGC		10	36.26	-7
Towers		5	12.40	-1
TreeSort		5	28.25	1
WhileLoop		5	6.64	0

micro-steady

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
Bounce		60	251.94	-13
BubbleSort		55	214.58	50
Dispatch		55	603.31	-2
Fannkuch		55	198.94	-1
Fibonacci		60	45.63	2
FieldLoop		55	303.79	0
IntegerLoop		55	992.52	0
List		65	555.35	2
Loop		55	721.37	-7
Mandelbrot		110	522.40	0
Permute		55	307.28	0
Queens		55	451.18	-1
QuickSort		55	675.23	0
Recurse		65	24.18	0
Sieve		60	242.13	-5
Storage		60	133.12	-1
Sum		55	621.74	0
Towers		55	1477.63	-10
TreeSort		60	555.24	1
WhileLoop		55	550.92	0

macro-startup

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
DeltaBlue		5	20.96	-2
GraphSearch		5	27.40	-6
Json		5	37.90	1
NBody		5	30.04	-8
PageRank		5	39.25	-20
Richards		5	147.16	0

micro-startup

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
Bounce		5	31.61	-3
BubbleSort		5	34.56	-18
Dispatch		5	33.39	-18
Fannkuch		5	23.90	-22
Fibonacci		5	58.70	1
FieldLoop		5	11.36	1
IntegerLoop		5	33.51	0
List		5	7.71	-2
Loop		5	133.84	-29
Mandelbrot		5	41.81	-4
Permute		5	55.33	2
Queens		5	46.48	-9
QuickSort		5	41.20	-6
Recurse		5	36.93	-3
Sieve		5	44.35	-15
Storage		5	28.55	2
Sum		5	66.03	-28
Test		10	33.54	0
TestGC		10	12.17	5
Towers		5	22.61	0
TreeSort		5	25.09	-1
WhileLoop		5	40.35	-21

som-parse

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
SomInit		10	40.40	-3
SomParse		10	18.92	2

macro-startup

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
DeltaBlue		5	30.73	-22
GraphSearch		5	38.20	1
Json		5	57.34	0
NBody		5	19.37	58
PageRank		5	13.54	4
Richards		5	83.10	3

macro-steady

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
DeltaBlue		120	66.27	-6
GraphSearch		250	170.97	4
Json		120	131.84	-1
NBody		120	116.51	19
PageRank		120	249.93	85
Richards		130	91.75	2

micro-startup

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
Bounce		5	20.30	-4
BubbleSort		5	15.68	29
Dispatch		5	5.55	66
Fannkuch		5	27.29	25
Fibonacci		5	29.00	0
FieldLoop		5	3.40	-1
IntegerLoop		5	3.56	0
List		5	13.37	1
Loop		5	23.96	46
Mandelbrot		5	11.64	0
Permute		5	30.73	0
Queens		5	59.48	97
QuickSort		5	50.50	1
Recurse		5	19.64	1
Sieve		5	8.90	-29
Storage		5	27.65	0
Sum		5	11.55	35
Test		10	56.69	-2
TestGC		10	12.06	1
Towers		5	62.51	1
TreeSort		5	43.08	0
WhileLoop		5	6.97	0

micro-steady

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
Bounce		60	143.50	-10
BubbleSort		55	412.03	61
Dispatch		55	1757.56	155
Fannkuch		55	147.56	-31
Fibonacci		60	211.45	-1
FieldLoop		55	334.32	0
IntegerLoop		55	991.89	0
List		65	229.01	-2
Loop		55	1610.25	99
Mandelbrot		110	936.23	0
Permute		55	315.13	-1
Queens		55	230.32	6
QuickSort		55	374.42	2
Recurse		65	107.38	0
Sieve		60	320.41	-7
Storage		60	140.19	-2
Sum		55	1396.83	127
Towers		55	114.21	0
TreeSort		60	602.48	0
WhileLoop		55	817.65	-1

micro-somsom

Executor: SomSom-ast-interp

		#M	median time in ms	time diff %
List		1	8512.14	-10
Loop		1	6091.22	-7
Mandelbrot		1	127.86	-6
Queens		1	5405.69	-5
Recurse		1	5661.48	-6

micro-somsom

Executor: SomSom-bc-interp

		#M	median time in ms	time diff %
List		1	9720.00	-8
Loop		1	6896.28	-8
Mandelbrot		1	143.17	-8
Queens		1	5944.80	-7
Recurse		1	6211.34	-8

Executor Comparisons

macro-startup

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
DeltaBlue		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	16.68 38.99 20.96 30.73	0 134 26 84
GraphSearch		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	24.20 29.33 27.40 38.20	0 21 13 58
Json		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	36.35 59.48 37.90 57.34	0 64 4 58
NBody		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	25.77 11.54 30.04 19.37	0 -55 17 -25
PageRank		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	34.59 13.06 39.25 13.54	0 -62 13 -61
Richards		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	126.71 74.56 147.16 83.10	0 -41 16 -34

micro-startup

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
Bounce		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	31.30 20.30 31.61 20.30	0 -35 1 -35
BubbleSort		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	29.83 10.57 34.56 15.68	0 -65 16 -47
Dispatch		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	22.65 3.20 33.39 5.55	0 -86 47 -75
Fannkuch		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	20.84 17.28 23.90 27.29	0 -17 15 31
Fibonacci		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	43.02 19.84 58.70 29.00	0 -54 36 -33
FieldLoop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	14.27 3.41 11.36 3.40	0 -76 -20 -76
IntegerLoop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	24.75 3.39 33.51 3.56	0 -86 35 -86
List		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	6.87 6.42 7.71 13.37	0 -7 12 95
Loop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	60.88 15.34 133.84 23.96	0 -75 120 -61
Mandelbrot		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	47.41 11.13 41.81 11.64	0 -77 -12 -75
Permute		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	52.38 27.35 55.33 30.73	0 -48 6 -41
Queens		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	38.67 28.61 46.48 59.48	0 -26 20 54
QuickSort		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	34.04 20.25 41.20 50.50	0 -41 21 48
Recurse		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	37.84 15.99 36.93 19.64	0 -58 -2 -48
Sieve		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	46.83 8.81 44.35 8.90	0 -81 -5 -81
Storage		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	24.75 26.02 28.55 27.65	0 5 15 12
Sum		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	40.59 8.58 66.03 11.55	0 -79 63 -72
Test		ast-interp ast-jit bc-interp bc-jit	10 10 10 10	49.21 85.82 33.54 56.69	0 74 -32 15
TestGC		ast-interp ast-jit bc-interp bc-jit	10 10 10 10	36.40 36.26 12.17 12.06	0 0 -67 -67
Towers		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	18.76 12.40 22.61 62.51	0 -34 20 233
TreeSort		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	25.18 28.25 25.09 43.08	0 12 0 71
WhileLoop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	32.77 6.64 40.35 6.97	0 -80 23 -79

som-parse

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
SomInit		ast-interp bc-interp	10 10	48.48 40.40	0 -17
SomParse		ast-interp bc-interp	10 10	19.19 18.92	0 -1

macro-steady

Baseline: RPySOM-ast-jit

		Exe	#M	median time in ms	time diff %
DeltaBlue		ast-jit bc-jit	120 120	82.86 66.27	0 -20
GraphSearch		ast-jit bc-jit	250 250	158.88 170.97	0 8
Json		ast-jit bc-jit	120 120	142.58 131.84	0 -8
NBody		ast-jit bc-jit	120 120	117.83 116.51	0 -1
PageRank		ast-jit bc-jit	120 120	116.49 249.93	0 115
Richards		ast-jit bc-jit	130 130	100.76 91.75	0 -9

micro-steady

Baseline: RPySOM-ast-jit

		Exe	#M	median time in ms	time diff %
Bounce		ast-jit bc-jit	60 60	251.94 143.50	0 -43
BubbleSort		ast-jit bc-jit	55 55	214.58 412.03	0 92
Dispatch		ast-jit bc-jit	55 55	603.31 1757.56	0 191
Fannkuch		ast-jit bc-jit	55 55	198.94 147.56	0 -26
Fibonacci		ast-jit bc-jit	60 60	45.63 211.45	0 363
FieldLoop		ast-jit bc-jit	55 55	303.79 334.32	0 10
IntegerLoop		ast-jit bc-jit	55 55	992.52 991.89	0 0
List		ast-jit bc-jit	65 65	555.35 229.01	0 -59
Loop		ast-jit bc-jit	55 55	721.37 1610.25	0 123
Mandelbrot		ast-jit bc-jit	110 110	522.40 936.23	0 79
Permute		ast-jit bc-jit	55 55	307.28 315.13	0 3
Queens		ast-jit bc-jit	55 55	451.18 230.32	0 -49
QuickSort		ast-jit bc-jit	55 55	675.23 374.42	0 -45
Recurse		ast-jit bc-jit	65 65	24.18 107.38	0 344
Sieve		ast-jit bc-jit	60 60	242.13 320.41	0 32
Storage		ast-jit bc-jit	60 60	133.12 140.19	0 5
Sum		ast-jit bc-jit	55 55	621.74 1396.83	0 125
Towers		ast-jit bc-jit	55 55	1477.63 114.21	0 -92
TreeSort		ast-jit bc-jit	60 60	555.24 602.48	0 9
WhileLoop		ast-jit bc-jit	55 55	550.92 817.65	0 48

micro-somsom

Baseline: SomSom-ast-interp

		Exe	#M	median time in ms	time diff %
List		ast-interp bc-interp	1 1	8512.14 9720.00	0 14
Loop		ast-interp bc-interp	1 1	6091.22 6896.28	0 13
Mandelbrot		ast-interp bc-interp	1 1	127.86 143.17	0 12
Queens		ast-interp bc-interp	1 1	5405.69 5944.80	0 10
Recurse		ast-interp bc-interp	1 1	5661.48 6211.34	0 10