MLCommons Inference 3.1 Results (4.0+ results will be added later)
Comparing Offline scenario for v3.0: L4x1_TRT and v3.1: L4x1_TRT
| Model |
v3.0: L4x1_TRT |
v3.1: L4x1_TRT |
Performance Delta |
| Model |
v3.0: L4x1_TRT |
v3.1: L4x1_TRT |
Performance Delta |
| resnet50 | 13157.8 | 12881.7 | -2.14 |
| 3d-unet-99.9 | 1.07674 | 1.0733 | -0.32 |
| rnnt | 3980.09 | 3899.48 | -2.07 |
| bert-99.9 | 630.644 | 631.457 | 0.13 |
| retinanet | 179.492 | 225.923 | 20.55 |
| bert-99 | 1032.08 | 1028.95 | -0.3 |
| 3d-unet-99 | 1.07674 | 1.0733 | -0.32 |
Comparing Server scenario for v3.0: L4x1_TRT and v3.1: L4x1_TRT
| Model |
v3.0: L4x1_TRT |
v3.1: L4x1_TRT |
Performance Delta |
| Model |
v3.0: L4x1_TRT |
v3.1: L4x1_TRT |
Performance Delta |
| resnet50 | 12198.5 | 12204.4 | 0.05 |
| rnnt | 3800.6 | 3754.56 | -1.23 |
| bert-99.9 | 529.892 | 539.238 | 1.73 |
| retinanet | 154.901 | 199.743 | 22.45 |
| bert-99 | 898.869 | 898.945 | 0.01 |
Comparing SingleStream scenario for v3.0: L4x1_TRT and v3.1: L4x1_TRT
| Model |
v3.0: L4x1_TRT |
v3.1: L4x1_TRT |
Performance Delta |
| Model |
v3.0: L4x1_TRT |
v3.1: L4x1_TRT |
Performance Delta |
| resnet50 | 0.361604 | 0.348745 | -3.69 |
| 3d-unet-99.9 | 1812.37 | 1817.09 | 0.26 |
| rnnt | 48.1187 | 19.4947 | -146.83 |
| retinanet | 6.18594 | 4.8076 | -28.67 |
| bert-99 | 2.53093 | 2.57906 | 1.87 |
| 3d-unet-99 | 1812.37 | 1817.09 | 0.26 |
Comparing MultiStream scenario for v3.0: L4x1_TRT and v3.1: L4x1_TRT
| Model |
v3.0: L4x1_TRT |
v3.1: L4x1_TRT |
Performance Delta |
| Model |
v3.0: L4x1_TRT |
v3.1: L4x1_TRT |
Performance Delta |
| resnet50 | 0.841997 | 0.846827 | 0.57 |
| retinanet | 49.94 | 40.7731 | -22.48 |
