Result Overview

Number of Run Configurations

149

Change of Run time

median 0% (min. -15%, max. 72%)

Changes in Benchmark Set

Performance Changes between Versions

som-interp

Executor: PySOM-ast-int

		#M	median time in ms	time diff %
Bounce		15	4731.32	1
CD		15	7650.35	6
DeltaBlue		15	1364.12	0
Havlak		15	7907.33	5
Json		15	2062.03	0
List		15	5046.33	13
Mandelbrot		15	4362.07	2
NBody		15	5938.83	2
Permute		15	4399.56	3
Queens		15	4404.11	0
Richards		15	4407.25	0
Sieve		15	7456.06	-9
Storage		15	1276.92	2
Towers		15	4981.85	1

som-steady

Executor: PySOM-ast-jit

		#M	median time in ms	time diff %
Bounce		1000	144.71	0
CD		1000	840.13	-1
DeltaBlue		1000	98.77	-1
Havlak		1000	949.20	2
Json		1000	92.91	-1
List		1000	811.25	-4
Mandelbrot		1000	131.93	0
NBody		1000	118.72	0
Permute		1000	176.13	0
Queens		1000	527.58	1
Richards		1000	83.37	-1
Sieve		1000	307.74	-2
Storage		1000	65.79	0
Towers		1000	889.52	-2

som-interp

Executor: PySOM-bc-int

		#M	median time in ms	time diff %
Bounce		15	4552.61	1
CD		15	7072.96	4
DeltaBlue		15	1428.27	2
Havlak		15	8110.00	9
Json		15	2024.57	1
List		15	4930.03	23
Mandelbrot		15	3556.94	0
NBody		15	6732.58	-2
Permute		15	4178.78	2
Queens		15	5461.56	-1
Richards		15	4573.18	-1
Sieve		15	7162.69	0
Storage		15	1315.93	-1
Towers		15	5174.12	4

som-steady

Executor: PySOM-bc-jit

		#M	median time in ms	time diff %
Bounce		1000	82.28	0
CD		1000	305.40	-6
DeltaBlue		1000	79.05	-1
Havlak		1000	571.90	-3
Json		1000	66.91	0
List		1000	453.30	6
Mandelbrot		1000	130.76	4
NBody		1000	121.32	-1
Permute		1000	176.43	0
Queens		1000	298.60	0
Richards		1000	92.08	1
Sieve		1000	589.66	0
Storage		1000	70.51	-1
Towers		1000	69.58	-5

somrs-interp

Executor: SOM-RS-ast-int

		#M	median time in ms	time diff %
Bounce		1	6258.38	-2
CD		1	6685.03	2
DeltaBlue		1	1351.58	-1
Havlak	No matching configuration for a31e167bf1d6812a2251e10321d81abf0a3fab54: total
Json		1	2572.57	-1
List		1	5658.71	23
Mandelbrot		1	4430.51	-3
NBody		1	5344.15	-6
Permute		1	5678.73	0
Queens		1	4894.97	-5
Richards		1	3849.95	-4
Sieve		1	9709.76	-3
Storage		1	2485.55	3
Towers		1	4850.15	-2

somrs-interp

Executor: SOM-RS-bc-int

		#M	median time in ms	time diff %
Bounce		1	5530.20	-1
DeltaBlue	No matching configuration for 2e1a50989356d8beb32c55393c8c2087d7f77707: total
Havlak	No matching configuration for a31e167bf1d6812a2251e10321d81abf0a3fab54: total
Json		1	2207.31	-1
List		1	5298.28	25
Mandelbrot		1	3978.14	1
NBody		1	4867.56	-2
Permute		1	5064.55	1
Queens		1	4516.16	-3
Richards		1	3830.05	4
Sieve		1	7743.07	4
Towers		1	4389.59	10

sompp-interp

Executor: SOMpp-int

		#M	median time in ms	time diff %
Bounce		15	3373.93	4
CD		15	5063.89	4
DeltaBlue		15	861.14	2
Havlak		15	5095.13	8
Json		15	1887.53	2
List		15	4250.04	28
Mandelbrot		15	3458.73	1
NBody		15	4431.68	0
Permute		15	3327.04	-1
Queens		15	4617.36	3
Richards		15	3113.59	0
Sieve		15	6011.38	0
Storage		15	1186.29	-3
Towers		15	3281.98	-2

som-steady

Executor: TruffleSOM-ast-HotspotEE-jit-main

		#M	median time in ms	time diff %
Bounce		1000	45.66	0
CD		1000	110.37	-3
DeltaBlue		1000	41.04	0
Havlak		1000	291.04	6
Json		1000	107.01	-4
List		1000	134.18	0
Mandelbrot		1000	92.82	0
NBody		1000	85.71	0
Permute		1000	142.98	0
Queens		1000	127.38	0
Richards		1000	69.77	0
Sieve		1000	162.63	1
Storage		1000	55.04	0
Towers		1000	129.69	5

som-interp

Executor: TruffleSOM-ast-NativeEE-int-main

		#M	median time in ms	time diff %
Bounce		15	4296.40	1
CD		15	9103.06	1
DeltaBlue		15	1852.64	-2
Havlak		15	9904.90	6
Json		15	2604.97	-1
List		15	6329.17	-2
Mandelbrot		15	1853.40	0
NBody		15	5024.06	1
Permute		15	5000.10	-1
Queens		15	3601.04	2
Richards		15	5977.81	-1
Sieve		15	3677.94	2
Storage		15	1543.25	-1
Towers		15	6590.87	0

som-steady

Executor: TruffleSOM-bc-HotspotEE-jit-main

		#M	median time in ms	time diff %
Bounce		1000	65.84	0
CD		1000	134.84	11
DeltaBlue		1000	46.43	0
Havlak		1000	326.54	14
Json		1000	114.30	-1
List		1000	114.28	-9
Mandelbrot		1000	100.15	0
NBody		1000	93.57	0
Permute		1000	192.99	0
Queens		1000	126.08	0
Richards		1000	71.37	0
Sieve		1000	88.07	1
Storage		1000	54.80	0
Towers		1000	123.68	-9

som-interp

Executor: TruffleSOM-bc-NativeEE-int-main

		#M	median time in ms	time diff %
Bounce		15	6490.23	0
CD		15	9715.88	7
DeltaBlue		15	2320.76	-2
Havlak		15	13117.11	10
Json		15	3537.36	-1
List		15	10675.72	72
Mandelbrot		15	4235.01	-15
NBody		15	8922.38	-2
Permute		15	6787.49	0
Queens		15	6783.47	-1
Richards		15	7256.76	-3
Sieve		15	8134.21	1
Storage		15	1921.33	1
Towers		15	8635.85	9

Executor Comparisons

som-interp

Baseline: PySOM-ast-int

		Exe	#M	median time in ms	time diff %
Bounce		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4731.32 4552.61 4296.40 6490.23	0 -4 -9 37
CD		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	7650.35 7072.96 9103.06 9715.88	0 -8 19 27
DeltaBlue		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	1364.12 1428.27 1852.64 2320.76	0 5 36 70
Havlak		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	7907.33 8110.00 9904.90 13117.11	0 3 25 66
Json		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	2062.03 2024.57 2604.97 3537.36	0 -2 26 72
List		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	5046.33 4930.03 6329.17 10675.72	0 -2 25 112
Mandelbrot		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4362.07 3556.94 1853.40 4235.01	0 -18 -58 -3
NBody		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	5938.83 6732.58 5024.06 8922.38	0 13 -15 50
Permute		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4399.56 4178.78 5000.10 6787.49	0 -5 14 54
Queens		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4404.11 5461.56 3601.04 6783.47	0 24 -18 54
Richards		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4407.25 4573.18 5977.81 7256.76	0 4 36 65
Sieve		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	7456.06 7162.69 3677.94 8134.21	0 -4 -51 9
Storage		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	1276.92 1315.93 1543.25 1921.33	0 3 21 50
Towers		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4981.85 5174.12 6590.87 8635.85	0 4 32 73

som-steady

Baseline: PySOM-ast-jit

		Exe	#M	median time in ms	time diff %
Bounce		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	144.71 82.28 45.66 65.84	0 -43 -68 -55
CD		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	840.13 305.40 110.37 134.84	0 -64 -87 -84
DeltaBlue		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	98.77 79.05 41.04 46.43	0 -20 -58 -53
Havlak		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	949.20 571.90 291.04 326.54	0 -40 -69 -66
Json		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	92.91 66.91 107.01 114.30	0 -28 15 23
List		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	811.25 453.30 134.18 114.28	0 -44 -83 -86
Mandelbrot		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	131.93 130.76 92.82 100.15	0 -1 -30 -24
NBody		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	118.72 121.32 85.71 93.57	0 2 -28 -21
Permute		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	176.13 176.43 142.98 192.99	0 0 -19 10
Queens		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	527.58 298.60 127.38 126.08	0 -43 -76 -76
Richards		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	83.37 92.08 69.77 71.37	0 10 -16 -14
Sieve		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	307.74 589.66 162.63 88.07	0 92 -47 -71
Storage		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	65.79 70.51 55.04 54.80	0 7 -16 -17
Towers		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	889.52 69.58 129.69 123.68	0 -92 -85 -86

somrs-interp

Baseline: SOM-RS-ast-int

		Exe	#M	median time in ms	time diff %
Bounce		ast-int bc-int	1 1	6258.38 5530.20	0 -12
CD			1	6685.03	0
DeltaBlue		ast-int bc-int	1 1	1351.58 0.00	0 -100
Json		ast-int bc-int	1 1	2572.57 2207.31	0 -14
List		ast-int bc-int	1 1	5658.71 5298.28	0 -6
Mandelbrot		ast-int bc-int	1 1	4430.51 3978.14	0 -10
NBody		ast-int bc-int	1 1	5344.15 4867.56	0 -9
Permute		ast-int bc-int	1 1	5678.73 5064.55	0 -11
Queens		ast-int bc-int	1 1	4894.97 4516.16	0 -8
Richards		ast-int bc-int	1 1	3849.95 3830.05	0 -1
Sieve		ast-int bc-int	1 1	9709.76 7743.07	0 -20
Storage			1	2485.55	0
Towers		ast-int bc-int	1 1	4850.15 4389.59	0 -9