In this test we will compare Geo Light sampling technology in 3Delight OSL, Arnold and RenderMan. We will use a test with a geometric area light that is composed of a relatively large number of faces. The test should allow us to find the convergence rate of each algorithm as well as general performance.

The Scene

The test scene allows for procedural generation of floating area lights using the "Tubes Per Step" PaintEffects attribute.


The scene is designed to procedurally generate a number of area lights floating on top of a "city".  

The Renderers



ArnoldRenderMan/RIS3Delight OSL
Version




TechnologyUnidirectional path tracer.Using unidirectional path tracer. Other options are available but not useful for this test.Unidirectional path tracer.
ShadersC++C++OSL

Methodology

We are interested to find out about:

We will start by creating a "ground truth" image for each renderer. This image is generated by using a very large amount of samples so there is no more apparent noise. We will then render several images with varying amount of samples and measure the RMSE between these images and ground truth. Timings will be taken at each render. Having this data will allow us to draw a conclusion about convergence rate and general performance.

The Setup


Notes About Sampling Parameters

Arnold — For light samples, Arnold uses effective sample counts that are proportional – within a constant –  to the square of the user specified value.  As we will see, this makes  sense from a UI standpoint since the variance follows the inverse of the same rule in the case of Arnold. This makes the light samples slider linear in term of perceived noise. In the Arnold tables below, we will specify the effective samplers per pixel along with the user samples.

RenderMan – We had troubles extracting consistent quality from RenderMan. In Arnold and 3Delight, light samples are the single "go to" parameter to control image quality when only direct lighting is considered. In RenderMan/RIS, we had to match light sample count with BxDF sample count to achieve acceptable quality and convergence rates. Using light samples only (or BxDF samples only) produced slowly convergent renders.  In RenderMan tables below, "N samples" means N samples for both light and BxDF. We did all the test with the "advanced (4)" light sampler —  other samplers did not provide acceptable results for this test case. Note that we used the path tracer with once bounce instead if the "direct lighting" algorithm since the latest would crash once in a while. 

3Delight – We have only one control for the general quality of the render. In the case of direct lighting, 3Delight "understands" that samples are best used for light sampling and that's what it does. As tests will show, those samples have a linear impact on perceived noise levels.

Results

Arnold

Samples (effective)2 (1.23)4 (4.91)8 (19.64)16 (78.56)32 (314.29)64 (1257.18)
Image


Time1s2s6s21s1:218:12
TTFP0s0.35s1.2s3.2s11s41s
Shadow Rays0.678 M

3.26 M

10.8 M43.4 M173.6 M694.5 M
RMSE0.156990.1001150.05017870.02425150.01174130.00693426

3Delight

Samples248163264128256
Image

Time5.72s8s12.82 s22.18 s40.92 s86.91165.69 s302.98
TTFP3333333
Shadow Rays2.45 M

3.13 M

4.51 M7.27 M12.78 M23.8 M45.88 M90.1 M
RMSE0.09331420.06582660.04412480.02904390.01855750.01175660.007525460.00449892


RenderMan

Samples124832642565121024
Image

Time6.74s7.23s7.99s9.42s18.51s29.401:38.083:15.776:23.39
TTFP3.1s3.24s4.7s9s15.5s51.2s

97s


Rays

1.47 M

2.94 M

5.88 M

11.7 M

47.02 M94.14 M376.3 M751.3 M1499 M
RMSE0.1511250.1214870.09536490.07281480.03738760.02653880.01381480.008540450.00396



The following plot gives us a good idea on the algorithm sophistication of the different light samplers. 

Rays2.453.134.517.2712.7823.845.8890.1
3Delight0.09331420.06582660.04412480.02904390.01855750.01175660.00752546

0.00449892

Rays0.6783.2610.843.4173.6694.5
Arnold0.156990.1001150.05017870.02425150.01174130.00693426
Rays1.472.945.8811.747.0294.14376.3751.3
RenderMan/RIS0.151125
0.121487
0.0953649

0.0728148

0.0373876





0.0265388

0.0138148

0.00854045


The following plot gives us a good idea the time required to achieve a certain quality. From the user perspective, this is an important quantity.

Time5.72s812.8222.1840.9286.91165.69302.98
3Delight0.09331420.06582660.04412480.02904390.01855750.01175660.00752546

0.00449892

Time1262181492
Arnold0.156990.1001150.05017870.02425150.01174130.00693426
Time6.747.237.999.4218.5129.4098.08383.39
RenderMan0.1511250.1214870.09536490.07281480.0373876



0.02653880.01381480.00854045


The following chart shows how much time it takes for each renderer to build the light acceleration data structure depending on sample count.

Rays2.453.134.517.2712.7823.845.8890.1
3Delight2222222

2

Rays0.6783.2610.843.4173.6694.5
Arnold00.351.23.21141
Rays1.472.945.8811.747.0294.14376.3751.3
RenderMan33.244.7915.551.297

Conclusions

    1. 3Delight Variance ~ 1/N
    2. Arnold Variance ~ 1/sqrt(N)
    3. RenderMan/RIS Variance ~ 1/sqrt(N)