v5 performance compare to som42
Here is the raw data.
small program : bubble, matmul, queen, queen2, quick
medium program: aes4, lexer, parser, pgen
to measure runtime, each program is repeated run: bubble x1000, matmul
x1000, queen x100, queen2 x100, quick x1000
aes4 x1000, lexer x10, parser x1, pgen x100.
The test machine is a notebook HP Compaq nc6400 Core 2 T7200 2GHz with
1G RAM.
som42
v5
noi runtime (ms)
average code size
noi runtime (ms) average code size (int)
bubble
5792 140 141
125 135.33 143
3385 109 109
125 114.33 134
matmul
9958 187 203
203 197.67 271
6009 188 188
188 188.00 224
queen
386965 532 531
515 526.00 237
265204 516 516
515 515.67 200
queen2 227309
328 328 344
333.33 273 131394
297 282 281
286.67 239
quick
26471 375 344
343 354.00 203
18338 281
343 344 322.67 185
aes4
17627 281 282
281 281.33 875
10101 234 234
234 234.00 752
lexer
381430 188 187
171 182.00 2001
212746 171 156
156 161.00 1650
parser 1088511
32 31 31
31.33 5313 593632
31 31
31 31.00 4819
pgen
87011 360 328
359 349.00 2983
47148 297 312
313 307.33 2601
noi ratio runtime ratio code size ratio som42
Mips v5 Mips
bubble 0.5844
0.8448
0.9371 42.80
29.61
matmul 0.6034
0.9511
0.8266 50.38
31.96
queen
0.6853 0.9804
0.8439
73.57 51.43
queen2 0.5780
0.8600
0.8755 68.19
45.84
quick
0.6928 0.9115
0.9113
74.78 61.94 56.83 43.13
aes4
0.5730 0.8318
0.8594
62.66 43.17
lexer
0.5578 0.8846
0.8246
20.96 13.21
parser 0.5454
0.9894
0.9070 34.74
19.15
pgen
0.5419 0.8806
0.8719
24.93 15.34
average 0.5958 0.9038
0.8730
50.33 34.06
------------------------------------------------
note parser gives errors when repeat, so run at x1 only, it
inputs itself
note pgen inputs from "parse-grammar.txt" of som42a
note lexer inputs itself
note code size is inspected from obj file, the size of CS in
int
Results
noi v5 is 60% of som42
runtime is 90%
code size is 87%
Mips small program (compare with Birthday benchmark) som42b 62,
v5 43
Mips all program som42b 50, v5 34
Discussion
The noi figure is as good as expected. noi is reduced 40%
compared to one-address som42. However, it does not translated into
runtime speedup. Only 10% speedup of v5 compared to som42. The
surprise result is the size of code, although v5 instruction format is
96-bit, it is more compact than som42 which is 64-bit. It means
the code has higher density. This should be investigate further.
The small program benchmark can be used to compare with Birthday
benchmark. som v19v design is guided by queen2 program. v5
vm is follows from v19v, in this respect it does quite well. For
queen2 runtime, this is the fastest runtime achieved to date.
5 December 2009
Long Live the King