Result Overview

Number of Run Configurations

190

Change of Run time

median 0% (min. -25%, max. 4%)

Performance Changes between Versions

awfy-startup

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
CD		5	286.00	-1
Havlak		5	7166.40	0

macro-startup

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
DeltaBlue		5	18.52	0
GraphSearch		5	27.98	0
Json		5	36.84	0
NBody		5	30.30	2
PageRank		5	41.59	1
Richards		5	129.62	-1

micro-startup

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
Bounce		5	31.74	-2
BubbleSort		5	37.52	1
Dispatch		5	30.33	0
Fannkuch		5	25.73	0
Fibonacci		5	45.38	0
FieldLoop		5	14.92	0
IntegerLoop		5	26.70	1
List		5	7.10	0
Loop		5	132.16	0
Mandelbrot		5	49.18	-2
Permute		5	54.76	1
Queens		5	42.57	0
QuickSort		5	34.54	-1
Recurse		5	38.43	0
Sieve		5	53.66	0
Storage		5	24.66	0
Sum		5	68.25	0
Test		10	54.78	1
TestGC		10	38.78	0
Towers		5	19.16	0
TreeSort		5	25.08	-1
WhileLoop		5	32.72	-3

som-parse

Executor: RPySOM-ast-interp

		#M	median time in ms	time diff %
SomInit		10	48.13	-1
SomParse		10	20.32	0

awfy-startup

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
CD		5	264.31	0
Havlak		5	1269.78	-1

awfy-steady

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
CD		130	409.10	1
Havlak		130	1038.54	0

macro-startup

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
DeltaBlue		5	43.67	0
GraphSearch		5	32.44	-1
Json		5	59.43	0
NBody		5	13.62	0
PageRank		5	15.64	1
Richards		5	72.04	-1

macro-steady

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
DeltaBlue		120	80.99	-1
GraphSearch		250	162.92	0
Json		120	141.00	0
NBody		120	95.95	0
PageRank		120	142.19	0
Richards		130	97.56	-3

micro-startup

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
Bounce		5	21.36	0
BubbleSort		5	9.46	1
Dispatch		5	3.10	1
Fannkuch		5	16.75	0
Fibonacci		5	20.54	0
FieldLoop		5	3.54	2
IntegerLoop		5	3.51	-1
List		5	6.66	0
Loop		5	17.29	-1
Mandelbrot		5	11.41	0
Permute		5	28.00	1
Queens		5	31.27	0
QuickSort		5	20.07	-1
Recurse		5	16.39	0
Sieve		5	9.70	0
Storage		5	26.09	2
Sum		5	8.92	0
Test		10	85.51	-3
TestGC		10	38.93	0
Towers		5	12.74	1
TreeSort		5	28.22	-1
WhileLoop		5	6.64	-4

micro-steady

Executor: RPySOM-ast-jit

		#M	median time in ms	time diff %
Bounce		60	287.28	-12
BubbleSort		55	145.95	0
Dispatch		55	688.41	0
Fannkuch		55	200.13	0
Fibonacci		60	44.97	1
FieldLoop		55	312.35	0
IntegerLoop		55	985.42	0
List		65	564.09	3
Loop		55	869.34	0
Mandelbrot		110	526.63	1
Permute		55	306.55	-3
Queens		55	456.50	0
QuickSort		55	677.09	0
Recurse		65	24.46	1
Sieve		60	245.23	0
Storage		60	129.36	-1
Sum		55	616.67	-1
Towers		55	1492.69	-1
TreeSort		60	550.37	0
WhileLoop		55	555.47	0

awfy-startup

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
CD		5	303.14	-9
Havlak		5	7291.52	-7

macro-startup

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
DeltaBlue		5	21.59	-6
GraphSearch		5	30.21	-6
Json		5	35.31	-7
NBody		5	32.91	-11
PageRank		5	51.40	-8
Richards		5	130.20	-10

micro-startup

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
Bounce		5	29.44	-6
BubbleSort		5	42.04	-11
Dispatch		5	42.77	-5
Fannkuch		5	30.25	-5
Fibonacci		5	61.15	-5
FieldLoop		5	12.11	3
IntegerLoop		5	33.76	-8
List		5	7.69	-10
Loop		5	189.99	-5
Mandelbrot		5	39.78	-22
Permute		5	56.03	-7
Queens		5	49.84	-3
QuickSort		5	35.53	-13
Recurse		5	34.76	-13
Sieve		5	53.61	-11
Storage		5	28.07	-3
Sum		5	95.32	-7
Test		10	35.16	-3
TestGC		10	10.43	0
Towers		5	22.68	-7
TreeSort		5	22.51	-7
WhileLoop		5	35.51	-13

som-parse

Executor: RPySOM-bc-interp

		#M	median time in ms	time diff %
SomInit		10	38.53	-13
SomParse		10	19.33	0

awfy-startup

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
CD		5	500.09	-15
Havlak		5	984.44	-3

awfy-steady

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
CD		130	141.03	-3
Havlak		130	635.60	1

macro-startup

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
DeltaBlue		5	39.42	-2
GraphSearch		5	38.29	-2
Json		5	56.22	-3
NBody		5	12.36	-2
PageRank		5	13.36	-2
Richards		5	77.34	-10

macro-steady

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
DeltaBlue		120	72.80	1
GraphSearch		250	168.92	1
Json		120	131.67	0
NBody		120	97.40	0
PageRank		120	144.45	0
Richards		130	91.82	0

micro-startup

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
Bounce		5	20.76	0
BubbleSort		5	12.49	0
Dispatch		5	3.45	0
Fannkuch		5	21.96	0
Fibonacci		5	28.95	-3
FieldLoop		5	3.51	-1
IntegerLoop		5	3.68	-1
List		5	12.74	-3
Loop		5	16.61	-1
Mandelbrot		5	11.74	-3
Permute		5	31.14	-1
Queens		5	30.79	-1
QuickSort		5	49.94	-2
Recurse		5	19.77	-2
Sieve		5	12.85	-1
Storage		5	27.70	1
Sum		5	8.82	-1
Test		10	58.99	-2
TestGC		10	10.50	1
Towers		5	57.97	-5
TreeSort		5	43.40	-2
WhileLoop		5	7.22	-3

micro-steady

Executor: RPySOM-bc-jit

		#M	median time in ms	time diff %
Bounce		60	180.20	-4
BubbleSort		55	205.33	-25
Dispatch		55	691.37	0
Fannkuch		55	211.40	1
Fibonacci		60	211.85	0
FieldLoop		55	334.58	0
IntegerLoop		55	996.89	0
List		65	254.30	1
Loop		55	797.13	-1
Mandelbrot		110	978.55	0
Permute		55	327.32	2
Queens		55	225.61	4
QuickSort		55	353.90	-3
Recurse		65	106.64	0
Sieve		60	361.30	-2
Storage		60	138.85	0
Sum		55	618.30	0
Towers		55	116.88	-9
TreeSort		60	603.17	0
WhileLoop		55	821.25	0

micro-somsom

Executor: SomSom-ast-interp

		#M	median time in ms	time diff %
List		1	9687.58	0
Loop		1	6921.66	2
Mandelbrot		1	139.84	0
Queens		1	5928.09	1
Recurse		1	6365.78	1

micro-somsom

Executor: SomSom-bc-interp

		#M	median time in ms	time diff %
List		1	10761.13	-7
Loop		1	7478.95	-7
Mandelbrot		1	151.17	-6
Queens		1	6279.43	-4
Recurse		1	6859.68	-7

Executor Comparisons

awfy-startup

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
CD		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	286.00 264.31 303.14 500.09	0 -8 6 75
Havlak		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	7166.40 1269.78 7291.52 984.44	0 -82 2 -86

macro-startup

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
DeltaBlue		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	18.52 43.67 21.59 39.42	0 136 17 113
GraphSearch		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	27.98 32.44 30.21 38.29	0 16 8 37
Json		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	36.84 59.43 35.31 56.22	0 61 -4 53
NBody		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	30.30 13.62 32.91 12.36	0 -55 9 -59
PageRank		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	41.59 15.64 51.40 13.36	0 -62 24 -68
Richards		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	129.62 72.04 130.20 77.34	0 -44 0 -40

micro-startup

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
Bounce		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	31.74 21.36 29.44 20.76	0 -33 -7 -35
BubbleSort		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	37.52 9.46 42.04 12.49	0 -75 12 -67
Dispatch		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	30.33 3.10 42.77 3.45	0 -90 41 -89
Fannkuch		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	25.73 16.75 30.25 21.96	0 -35 18 -15
Fibonacci		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	45.38 20.54 61.15 28.95	0 -55 35 -36
FieldLoop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	14.92 3.54 12.11 3.51	0 -76 -19 -76
IntegerLoop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	26.70 3.51 33.76 3.68	0 -87 26 -86
List		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	7.10 6.66 7.69 12.74	0 -6 8 79
Loop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	132.16 17.29 189.99 16.61	0 -87 44 -87
Mandelbrot		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	49.18 11.41 39.78 11.74	0 -77 -19 -76
Permute		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	54.76 28.00 56.03 31.14	0 -49 2 -43
Queens		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	42.57 31.27 49.84 30.79	0 -27 17 -28
QuickSort		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	34.54 20.07 35.53 49.94	0 -42 3 45
Recurse		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	38.43 16.39 34.76 19.77	0 -57 -10 -49
Sieve		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	53.66 9.70 53.61 12.85	0 -82 0 -76
Storage		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	24.66 26.09 28.07 27.70	0 6 14 12
Sum		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	68.25 8.92 95.32 8.82	0 -87 40 -87
Test		ast-interp ast-jit bc-interp bc-jit	10 10 10 10	54.78 85.51 35.16 58.99	0 56 -36 8
TestGC		ast-interp ast-jit bc-interp bc-jit	10 10 10 10	38.78 38.93 10.43 10.50	0 0 -73 -73
Towers		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	19.16 12.74 22.68 57.97	0 -34 18 203
TreeSort		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	25.08 28.22 22.51 43.40	0 13 -10 73
WhileLoop		ast-interp ast-jit bc-interp bc-jit	5 5 5 5	32.72 6.64 35.51 7.22	0 -80 9 -78

som-parse

Baseline: RPySOM-ast-interp

		Exe	#M	median time in ms	time diff %
SomInit		ast-interp bc-interp	10 10	48.13 38.53	0 -20
SomParse		ast-interp bc-interp	10 10	20.32 19.33	0 -5

awfy-steady

Baseline: RPySOM-ast-jit

		Exe	#M	median time in ms	time diff %
CD		ast-jit bc-jit	130 130	409.10 141.03	0 -66
Havlak		ast-jit bc-jit	130 130	1038.54 635.60	0 -39

macro-steady

Baseline: RPySOM-ast-jit

		Exe	#M	median time in ms	time diff %
DeltaBlue		ast-jit bc-jit	120 120	80.99 72.80	0 -10
GraphSearch		ast-jit bc-jit	250 250	162.92 168.92	0 4
Json		ast-jit bc-jit	120 120	141.00 131.67	0 -7
NBody		ast-jit bc-jit	120 120	95.95 97.40	0 2
PageRank		ast-jit bc-jit	120 120	142.19 144.45	0 2
Richards		ast-jit bc-jit	130 130	97.56 91.82	0 -6

micro-steady

Baseline: RPySOM-ast-jit

		Exe	#M	median time in ms	time diff %
Bounce		ast-jit bc-jit	60 60	287.28 180.20	0 -37
BubbleSort		ast-jit bc-jit	55 55	145.95 205.33	0 41
Dispatch		ast-jit bc-jit	55 55	688.41 691.37	0 0
Fannkuch		ast-jit bc-jit	55 55	200.13 211.40	0 6
Fibonacci		ast-jit bc-jit	60 60	44.97 211.85	0 371
FieldLoop		ast-jit bc-jit	55 55	312.35 334.58	0 7
IntegerLoop		ast-jit bc-jit	55 55	985.42 996.89	0 1
List		ast-jit bc-jit	65 65	564.09 254.30	0 -55
Loop		ast-jit bc-jit	55 55	869.34 797.13	0 -8
Mandelbrot		ast-jit bc-jit	110 110	526.63 978.55	0 86
Permute		ast-jit bc-jit	55 55	306.55 327.32	0 7
Queens		ast-jit bc-jit	55 55	456.50 225.61	0 -51
QuickSort		ast-jit bc-jit	55 55	677.09 353.90	0 -48
Recurse		ast-jit bc-jit	65 65	24.46 106.64	0 336
Sieve		ast-jit bc-jit	60 60	245.23 361.30	0 47
Storage		ast-jit bc-jit	60 60	129.36 138.85	0 7
Sum		ast-jit bc-jit	55 55	616.67 618.30	0 0
Towers		ast-jit bc-jit	55 55	1492.69 116.88	0 -92
TreeSort		ast-jit bc-jit	60 60	550.37 603.17	0 10
WhileLoop		ast-jit bc-jit	55 55	555.47 821.25	0 48

micro-somsom

Baseline: SomSom-ast-interp

		Exe	#M	median time in ms	time diff %
List		ast-interp bc-interp	1 1	9687.58 10761.13	0 11
Loop		ast-interp bc-interp	1 1	6921.66 7478.95	0 8
Mandelbrot		ast-interp bc-interp	1 1	139.84 151.17	0 8
Queens		ast-interp bc-interp	1 1	5928.09 6279.43	0 6
Recurse		ast-interp bc-interp	1 1	6365.78 6859.68	0 8