...
- Only diffuse reflectors.
- We will disable any adaptive sampling so to make sure we have meaningful ray-counts.
- We will use only direct lighting to estimate the geometric area light contribution. In the statistics files for each renderer, one can see that we have only one path length.
- The light sources contain about 80K triangles.
- All Renders are done in Maya 2016.
The Renderers
| Warninginfo |
|---|
RenderMan/RIS 21 is about to be released with new sampling technology. We will complete the test as soon as the renderer is available. |
| Arnold | RenderMan/RIS | 3Delight OSL | |
|---|---|---|---|
| Version |
|
|
|
| Technology | Unidirectional path tracer. | Using unidirectional path tracer. Other options are available but not useful for this test. | Unidirectional path tracer. |
| Shaders | C++ | C++ | OSL |
...
We will start by creating a "ground truth" image for each renderer (encouragingly, images produced by both 3Delight and Arnold are almost exactly the same). This image is ; an image generated by using a very large amount of samples so there is no more apparent noise. We . Although it is normal for images generated by each renderer to vary slightly, encouragingly, images produced by both 3Delight and Arnold are almost exactly the same. We will then render several images with varying amount of samples and measure the RMSE between these images and the ground truth for each renderer. Timings and statistics will be collected at each render. Having this data will allow us to draw a conclusion about the convergence rate and general performance.
...
We use a 1x1 pixel sample in all renderers. Adaptivity is disabled as well as all additional bounces. In Arnold, the light samples are attributes of the geometry. In RenderMan/RIS the light samples are attributes of a custom shape. In 3Delight, the light attributes are on Maya's area light.
| Info |
|---|
Admittedly, those are not standard settings for any useful render. The goal here is to isolate one algorithm in order to understand its behaviour. Understanding an algorithm can shed light on its strengths and weaknesses and allows us to draw interesting conclusions for the more general case. |
...
Arnold — For light samples, Arnold uses effective sample counts that are proportional – within a constant – to – to the square of the user specified value. As we will see, this makes sense from a UI standpoint since the variance follows the inverse of the same rule in the case of Arnold. This makes the light samples slider linear in term of perceived noise. In the Arnold tables below, we will specify the effective samples per pixel along with the user samples. Those effective samples are gathered from Arnold's diagnostics files.
RenderMan/RIS – In While in Arnold , the light samples effectively control image quality when only direct lighting is needed. In , in RenderMan/RIS, we had to match light samples count with BxDF samples count to achieve acceptable quality and satisfactory convergence rates. Using light samples only, or BxDF samples only, produced noisy renders. In the RenderMan/RIS results betlowbelow, "N samples" means N samples for both light and BxDF. We did all the tests with the "advanced (mode 4)" light sampler since it produced the best results. The samples used by the renderer are the ones entered in the UI and are not squared as in Arnold. Note that we used the path tracer with one bounce instead of the "direct lighting" algorithm for one of the images because of a crash (quality and speed did not seem to suffer although there were some minor differences in the render).
3Delight – We have only one control for the general quality of the render. In the case of direct lighting, 3Delight "understands" that samples are best used for light sampling and that's what it does. As the tests will show, those samples have a linear impact on perceived noise levels.
...
| Samples (effective) | 2 (1.23) | 4 (4.91) | 8 (19.64) | 10 (30) | 16 (78.56) | 32 (314.29) | 64 (1257.18) |
|---|---|---|---|---|---|---|---|
| Image |  |
|
|
|
|
|
|
| Time (s) | 1 sec. | 2 sec. | 6 sec. | 9 sec. | 21 sec. | 81 sec. | 492 sec. |
| TTFP* (s) | 0 sec. | 0.35 sec. | 1.2 sec. | 2.0 sec. | 3.2 sec. | 11 sec. | 41 sec. |
| Shadow Rays | 0.678 M | 3.26 M | 10.8 M | 16.94 M | 43.4 M | 173.6 M | 694.5 M |
| RMSE | 0.15699 | 0.100115 | 0.0501787 | 0.0396399 | 0.0242515 | 0.0117413 | 0.00693426 |
...
| Samples | 2 | 4 | 8 | 16 | 32 | 64 | 128 | 256 |
|---|---|---|---|---|---|---|---|---|
| Image |
|
|
|
|
|
|
|
|
| Time (s) | 5.24 sec. | 7.18 sec. | 11.21 sec. | 20.68 sec. | 35.86 sec. | 69.48 sec. | 134.18 sec. | 265.05 sec. |
| TTFP (s) | 2 sec. | 2 sec. | 2 sec. | 2 sec. | 2 sec. | 2 sec. | 2 sec. | 2 sec. |
| Shadow Rays | 2.45 M | 3.13 M | 4.51 M | 7.27 M | 12.78 M | 23.8 M | 45.88 M | 90.1 M |
| RMSE | 0.0933142 | 0.0658266 | 0.0441248 | 0.0290439 | 0.0185575 | 0.0117566 | 0.00752546 | 0.00449892 |
...
| Samples | 1 | 2 | 4 | 8 | 16 | 32 | 64 | 256 | 512 | 1024 |
|---|---|---|---|---|---|---|---|---|---|---|
| Image |
|
|
|
|
|
|
|
|
|
|
| Time (s) | 6.74 sec. | 7.23 sec. | 7.99 sec. | 9.42 sec. | 12.12 sec. | 18.51 sec. | 29.40 sec. | 98.08 sec. | 195.77 sec. | 6:23.39 min:sec. |
| TTFP (s) | 3.1 sec. | 3.2 sec. | 4 sec. | 4.7 sec. | 6.7 sec. | 9 sec. | 15.5 sec. | 51.2 sec. | 97 sec. | |
| Rays | 1.47 M | 2.94 M | 5.88 M | 11.7 M | 21.1M | 47.02 M | 94.14 M | 376.3 M | 751.3 M | 1499 M |
| RMSE | 0.151125 | 0.121487 | 0.0953649 | 0.0728148 | 0.0520606 | 0.0373876 | 0.0265388 | 0.0138148 | 0.00854045 | 0.00396 |
Conclusions
- 3Delight generates light samples that are algorithmically better (in term of variance) than both Arnold and RenderMan. In short, for x effective samples:
- 3Delight Variance ~ 1/x
- Arnold Variance ~ 1/sqrt(x)
- RenderMan/RIS Variance ~ 1/sqrt(x) (possibly slightly worse but could be within a constant)
- 3Delight is slower to generate these samples. For draft renders (high variance), Arnold is fastest. For final renders (low variance) 3Delight is fastest and becomes increasingly faster with increasing samples.
- Arnold and RenderMan/RIS draw samples at about the same speed, but the quality of Arnold samples is better.
- Both Arnold and RenderMan/RIS produce biased images at low sample counts. More specifically: images are darker. 3Delight manages to keep the same energy overall independently of sample counts.
- Arnold, 3Delight and RenderMan/RIS rely on acceleration data structures to sample the geometric area lights. In Arnold and RenderMan/RIS, the algorithmic complexity to build those data structures is tied linearly to the number of samples (as well as the complexity of the light). In 3Delight, only the complexity of the light matters (time to first pixel for 3Delight was 2-3 seconds no matter how many samples there were).
...
| RenderMan/RIS | Arnold | 3Delight | |
|---|---|---|---|
| Images and Stats | arnold.tar.gz | 3delight.tar.gz | |
| Maya Scene | mal_prman.ma | mal_3delight.ma | |
| Remarks | The same scene works with both 3Delight and Arnold | ||