...
- Only diffuse reflectors. The original test scene had a diffuse+specular surface but that generated a considerable amount of additional noise in Arnold. Since we want to concentrate on light sampling here, we decided to go for a simpler setup.
- We will disable any adaptive sampling so to make sure we have very close ray-counts.
- We will use only direct lighting to estimate the geometric area light contribution. In the statistics files for each renderer, one can see that we have only one path lenghtlength.
- All Renders are done in Maya 2016.
...
We will start by creating a "ground truth" image for each renderer (this . This image is generated by using a very large amount of samples so there is no more apparent noise). We will then render several images with varying amount of samples and measure the RMSE between these images and ground truth. Timings will be taken at each rendererrender. Having this data will allow us to draw a conclusion about convergence rate and general performance.
The Renderers
| Arnold | RenderMan/RIS | 3Delight OSL | |
|---|---|---|---|
| Version |
|
| |
| Technology | Unidirectional path tracer. | Using unidirectional path tracer. Other options are available but not useful for this test. | Unidirectional path tracer. |
| Shaders | C++ | C++ | OSL |
Notes About Sampling Parameters
Arnold — For light samples, Arnold uses effective sample counts that are proportional – within a constant – to the square of the user specified value. As we will see, this makes perfect sense since the variance follows the inverse of the same rule in the case of Arnold. This makes the light samples slider linear in term of perceived noise (which is good UI practice). In the Arnold tables below, we will specify the effective samplers per pixel along with the user samples.
RenderMan – We had troubles extracting consistant quality from RenderMan. In Arnold and 3Delight, light samples is the single "go to" parameter to control image quality when only direct lighting is needed. In RenderMan we had to match light sample count with BxDF sample count to extract an achieve acceptable quality and convergence rates. Using light samples only (or BxDF samples only) produced slowly convergent renders. In RenderMan tables below, "N samples" means N samples for both light and BxDF. We did all the test with the "advanced (4)" light sampler — other samplers did not provide acceptable results for this test case. Noteworthy
3Delight – We have only one control for the general quality of the render. In the case of direct lighting, 3Delight "understands" that all samples are best used for light sampling and that's what it does. As tests will show, those samples have a linear impact on perceived noise levels.
Results
Arnold
| Samples (effective) | 2 (1.23) | 4 (4.91) | 8 (19.64) | 16 (78.56) | 32 (314.29) | 64 (1257.18) |
|---|---|---|---|---|---|---|
| Image |  |
|
|
|
|
|
| Time | 1s | 2s | 6s | 21s | 1:21 | 8:12 |
| Shadow Rays | 0.678 M | 3.26 M | 10.8 M | 43.4 M | 173.6 M | 694.5 M |
| RMSE | 0.15699 | 0.100115 | 0.0501787 | 0.0242515 | 0.0117413 | 0.00693426 |
3Delight
| Samples | 2 | 4 | 8 | 16 | 32 | 64 | 128 | 256 |
|---|---|---|---|---|---|---|---|---|
| Image |
|
|
|
|
|
|
|
|
| Time | 5.72s | 8s | 12.82 s | 22.18 s | 40.92 s | 86.91 | 165.69 s | 302.98 |
| Shadow Rays | 2.45 M | 3.13 M | 4.51 M | 7.27 M | 12.78 M | 23.8 M | 45.88 M | 90.1 M |
| RMSE | 0.0933142 | 0.0658266 | 0.0441248 | 0.0290439 | 0.0185575 | 0.0117566 | 0.00752546 | 0.00449892 |
RenderMan
| Samples | 1 | 2 | 4 | 8 | 32 | 64 | 256 | 512 | 1024 |
|---|---|---|---|---|---|---|---|---|---|
| Image |
|
|
|
|
|
|
|
|
|
| Time | 6.74s | 7.23s | 7.99s | 9.42s | 18.51s | 29.40 | 1:38.08 | 3:15.77 | 6:23.39 |
| Rays | 1.47 M | 2.94 M | 5.88 M | 11.7 M | 47.02 M | 94.14 M | 376.3 M | 751.3 M | 1499 M |
| RMSE | 0.151125 | 0.121487 | 0.0953649 | 0.0728148 | 0.0373876 | 0.0265388 | 0.0138148 | 0.00854045 | 0.00396 |
...
- 3Delight Varince ~ 1/N
- Arnold Variance ~ 1/sqrt(N)
- RenderMan/RIS Variance ~ 1/sqrt(N)
- 3Delight is slower to generate these samples. Meaning that for draft renders Arnold/PrMan are faster. For final renders 3Delight becomes increasingly faster with the number of samples.
- Arnold has predictable sampling control that behaves in a linear way.
- Both Arnold and RenderMan produce biased images at low sample counts. More specifically: images are darker. This explains higher RMSE with low sample counts. 3Delight manages to keep the same energy overall independent on sample counts.
- Arnold, 3Delight and RenderMan rely on acceleration data structures to sample the geometric area lights. In Arnold and RenderMan, the algorithmic complexity to build those data structures is tied to the number of samples as well as the complexity of the light. In 3Delight, only to the complexity of the light matters (time to first pixel for 3Delight was 3 seconds no matter how many samples there are).