Result Overview

Number of Run Configurations

149

Change of Run time

median 0% (min. -18%, max. 26%)

Changes in Benchmark Set

Performance Changes between Versions

som-interp

Executor: PySOM-ast-int

		#M	median time in ms	time diff %
Bounce		15	4623.87	-1
CD		15	7273.67	1
DeltaBlue		15	1334.04	-2
Havlak		15	7406.60	-2
Json		15	1904.40	-7
List		15	4064.35	-9
Mandelbrot		15	4664.27	9
NBody		15	6145.82	6
Permute		15	4252.56	-1
Queens		15	4500.70	2
Richards		15	4460.43	2
Sieve		15	7579.35	-8
Storage		15	1267.78	1
Towers		15	4649.40	-6

som-steady

Executor: PySOM-ast-jit

		#M	median time in ms	time diff %
Bounce		1000	144.82	0
CD		1000	849.87	0
DeltaBlue		1000	99.96	1
Havlak		1000	933.21	0
Json		1000	95.52	2
List		1000	839.70	-1
Mandelbrot		1000	129.75	-2
NBody		1000	118.63	0
Permute		1000	178.13	1
Queens		1000	529.36	1
Richards		1000	83.16	-2
Sieve		1000	308.21	-2
Storage		1000	65.72	0
Towers		1000	961.88	6

som-interp

Executor: PySOM-bc-int

		#M	median time in ms	time diff %
Bounce		15	4393.96	-3
CD		15	7164.59	6
DeltaBlue		15	1413.88	1
Havlak		15	7330.28	-1
Json		15	1884.60	-6
List		15	3582.22	-11
Mandelbrot		15	3696.91	4
NBody		15	6837.29	0
Permute		15	4176.73	2
Queens		15	5474.45	-1
Richards		15	4643.31	1
Sieve		15	7391.71	3
Storage		15	1337.84	1
Towers		15	4893.27	-2

som-steady

Executor: PySOM-bc-jit

		#M	median time in ms	time diff %
Bounce		1000	82.08	0
CD		1000	324.60	0
DeltaBlue		1000	79.76	0
Havlak		1000	585.46	0
Json		1000	69.43	4
List		1000	456.24	7
Mandelbrot		1000	129.76	3
NBody		1000	121.56	-1
Permute		1000	176.64	0
Queens		1000	289.18	-3
Richards		1000	92.02	1
Sieve		1000	586.55	-1
Storage		1000	71.39	0
Towers		1000	84.21	15

somrs-interp

Executor: SOM-RS-ast-int

		#M	median time in ms	time diff %
Bounce		1	6268.70	-2
CD		1	6677.11	2
DeltaBlue		1	1363.72	0
Havlak	No matching configuration for 807f8d81edef384d50638b82b80da6cb0f9e4775: total
Json		1	2592.54	0
List		1	5605.51	21
Mandelbrot		1	4441.37	-2
NBody		1	5470.14	-3
Permute		1	5845.82	3
Queens		1	4901.06	-4
Richards		1	3932.63	-2
Sieve		1	9717.48	-3
Storage		1	2473.81	2
Towers		1	4847.79	-2

somrs-interp

Executor: SOM-RS-bc-int

		#M	median time in ms	time diff %
Bounce		1	5606.47	1
DeltaBlue	No matching configuration for 2e1a50989356d8beb32c55393c8c2087d7f77707: total
Havlak	No matching configuration for 807f8d81edef384d50638b82b80da6cb0f9e4775: total
Json		1	2208.64	-1
List		1	5339.37	26
Mandelbrot		1	3994.31	1
NBody		1	4911.33	-1
Permute		1	5100.97	2
Queens		1	4507.28	-3
Richards		1	3831.61	4
Sieve		1	7475.24	1
Towers		1	4332.31	8

sompp-interp

Executor: SOMpp-int

		#M	median time in ms	time diff %
Bounce		15	3360.36	3
CD		15	4811.03	-1
DeltaBlue		15	855.55	1
Havlak		15	4644.47	-2
Json		15	1692.38	-9
List		15	2743.73	-18
Mandelbrot		15	4079.39	19
NBody		15	4562.62	3
Permute		15	3339.20	-1
Queens		15	4572.69	2
Richards		15	3145.30	1
Sieve		15	5840.89	-2
Storage		15	1200.26	-2
Towers		15	2966.71	-12

som-steady

Executor: TruffleSOM-ast-HotspotEE-jit-main

		#M	median time in ms	time diff %
Bounce		1000	45.66	0
CD		1000	113.62	0
DeltaBlue		1000	41.00	0
Havlak		1000	266.05	-4
Json		1000	110.91	0
List		1000	134.29	0
Mandelbrot		1000	92.87	0
NBody		1000	85.72	0
Permute		1000	142.99	0
Queens		1000	127.42	0
Richards		1000	69.90	0
Sieve		1000	160.18	0
Storage		1000	55.13	0
Towers		1000	131.09	6

som-interp

Executor: TruffleSOM-ast-NativeEE-int-main

		#M	median time in ms	time diff %
Bounce		15	4234.41	0
CD		15	8961.79	0
DeltaBlue		15	1859.54	-1
Havlak		15	9452.64	1
Json		15	2589.03	-2
List		15	6197.66	-4
Mandelbrot		15	1698.17	-8
NBody		15	5247.39	6
Permute		15	4961.48	-2
Queens		15	3818.11	8
Richards		15	6099.73	1
Sieve		15	3341.12	-7
Storage		15	1542.72	-1
Towers		15	6397.36	-3

som-steady

Executor: TruffleSOM-bc-HotspotEE-jit-main

		#M	median time in ms	time diff %
Bounce		1000	65.95	0
CD		1000	121.53	0
DeltaBlue		1000	46.61	0
Havlak		1000	278.51	-3
Json		1000	111.72	-3
List		1000	125.65	0
Mandelbrot		1000	100.14	0
NBody		1000	93.53	0
Permute		1000	193.16	0
Queens		1000	126.19	0
Richards		1000	71.18	0
Sieve		1000	88.10	1
Storage		1000	55.25	1
Towers		1000	135.47	0

som-interp

Executor: TruffleSOM-bc-NativeEE-int-main

		#M	median time in ms	time diff %
Bounce		15	6339.80	-2
CD		15	9019.25	0
DeltaBlue		15	2321.80	-2
Havlak		15	11703.81	-2
Json		15	3210.44	-10
List		15	5637.64	-9
Mandelbrot		15	4643.50	-7
NBody		15	9257.25	2
Permute		15	6715.19	-1
Queens		15	6847.43	0
Richards		15	7409.49	-1
Sieve		15	8842.93	9
Storage		15	1884.35	-1
Towers		15	7716.80	-3

Executor Comparisons

som-interp

Baseline: PySOM-ast-int

		Exe	#M	median time in ms	time diff %
Bounce		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4623.87 4393.96 4234.41 6339.80	0 -5 -8 37
CD		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	7273.67 7164.59 8961.79 9019.25	0 -1 23 24
DeltaBlue		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	1334.04 1413.88 1859.54 2321.80	0 6 39 74
Havlak		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	7406.60 7330.28 9452.64 11703.81	0 -1 28 58
Json		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	1904.40 1884.60 2589.03 3210.44	0 -1 36 69
List		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4064.35 3582.22 6197.66 5637.64	0 -12 52 39
Mandelbrot		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4664.27 3696.91 1698.17 4643.50	0 -21 -64 0
NBody		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	6145.82 6837.29 5247.39 9257.25	0 11 -15 51
Permute		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4252.56 4176.73 4961.48 6715.19	0 -2 17 58
Queens		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4500.70 5474.45 3818.11 6847.43	0 22 -15 52
Richards		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4460.43 4643.31 6099.73 7409.49	0 4 37 66
Sieve		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	7579.35 7391.71 3341.12 8842.93	0 -2 -56 17
Storage		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	1267.78 1337.84 1542.72 1884.35	0 6 22 49
Towers		PySOM-ast-int PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main	15 15 15 15	4649.40 4893.27 6397.36 7716.80	0 5 38 66

som-steady

Baseline: PySOM-ast-jit

		Exe	#M	median time in ms	time diff %
Bounce		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	144.82 82.08 45.66 65.95	0 -43 -68 -54
CD		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	849.87 324.60 113.62 121.53	0 -62 -87 -86
DeltaBlue		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	99.96 79.76 41.00 46.61	0 -20 -59 -53
Havlak		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	933.21 585.46 266.05 278.51	0 -37 -71 -70
Json		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	95.52 69.43 110.91 111.72	0 -27 16 17
List		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	839.70 456.24 134.29 125.65	0 -46 -84 -85
Mandelbrot		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	129.75 129.76 92.87 100.14	0 0 -28 -23
NBody		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	118.63 121.56 85.72 93.53	0 2 -28 -21
Permute		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	178.13 176.64 142.99 193.16	0 -1 -20 8
Queens		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	529.36 289.18 127.42 126.19	0 -45 -76 -76
Richards		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	83.16 92.02 69.90 71.18	0 11 -16 -14
Sieve		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	308.21 586.55 160.18 88.10	0 90 -48 -71
Storage		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	65.72 71.39 55.13 55.25	0 9 -16 -16
Towers		PySOM-ast-jit PySOM-bc-jit TruffleSOM-ast-HotspotEE-jit-main TruffleSOM-bc-HotspotEE-jit-main	1000 1000 1000 1000	961.88 84.21 131.09 135.47	0 -91 -86 -86

somrs-interp

Baseline: SOM-RS-ast-int

		Exe	#M	median time in ms	time diff %
Bounce		ast-int bc-int	1 1	6268.70 5606.47	0 -11
CD			1	6677.11	0
DeltaBlue		ast-int bc-int	1 1	1363.72 0.00	0 -100
Json		ast-int bc-int	1 1	2592.54 2208.64	0 -15
List		ast-int bc-int	1 1	5605.51 5339.37	0 -5
Mandelbrot		ast-int bc-int	1 1	4441.37 3994.31	0 -10
NBody		ast-int bc-int	1 1	5470.14 4911.33	0 -10
Permute		ast-int bc-int	1 1	5845.82 5100.97	0 -13
Queens		ast-int bc-int	1 1	4901.06 4507.28	0 -8
Richards		ast-int bc-int	1 1	3932.63 3831.61	0 -3
Sieve		ast-int bc-int	1 1	9717.48 7475.24	0 -23
Storage			1	2473.81	0
Towers		ast-int bc-int	1 1	4847.79 4332.31	0 -11