Page History

It is not uncommon for production scenes to rely heavily on geo area lights. In this test we will compare Geo Light geo light sampling technology in 3Delight OSL, Arnold and RenderMan/RIS. We will use a test with a geometric area light that is composed of a relatively large number of faces. The test, although limited, should allow us to find the convergence rate of each algorithm as well as general performance.

The Scene

The test scene allows for procedural generation of floating area lights using the "Tubes Per Step" PaintEffects attribute.

...

The scene is designed to procedurally generate a number of area lights floating on top of a "city".  

• Only diffuse reflectors. The original test scene had a diffuse+specular surface but that generated a considerable amount of additional noise in Arnold. Since we want to concentrate on light sampling here, we decided to go for a simpler setup.
• We will disable any adaptive sampling so to make sure we have very close raymeaningful  ray-counts.
• We will use only direct lighting to estimate the geometric area light contribution. In the statistics files for each renderer, one can see that we have only one path length.
• The light sources contain about 80K triangles.
• All Renders are done in Maya 2016.

The Renderers

Methodology

We are interested to find out about:

• The quality of the geo light samples drawn by each renderer. How fast do we converge to ground truth by increasing the number of samples ?
• Performance. How fast is the sampler to produce an image of a given quality ?

We will start by creating a "ground truth" image for each renderer. This image is ; an image generated by using a very large amount of samples so there is no more apparent noise. We Although it is normal for images generated by each renderer to vary slightly, encouragingly, images produced by both 3Delight and Arnold are almost exactly the same.  We will then render several images with varying amount of samples and measure the RMSE between these images and the ground truth for each renderer. Timings and statistics will be taken collected at each render. Having this data will allow us to draw a conclusion about the convergence rate and general performance.

The Setup

We use a 1x1 pixel sample in all renderers. Adaptivity is disabled as well as all additional bounces. In Arnold, the light samples are attributes of the geometry. In RenderMan/RIS the light samples are attributes of a custom shape. In 3Delight, the light attributes are on Maya's area light. 

...

Notes

Arnold — For light samples, Arnold uses effective sample counts that are proportional – within a constant –  to – to the square of the user specified value.  As we will see, this makes  sense from a UI standpoint since the variance follows the inverse of the same rule in the case of Arnold. This makes the light samples slider linear in term of perceived noise. In the Arnold tables below, we will specify the effective samplers samples per pixel along with the user samples.RenderMan – We had . Those effective samples are gathered from Arnold's diagnostics files.

RenderMan/RIS – While in Arnold the light samples effectively troubles extracting consistent quality from RenderMan. In Arnold and 3Delight, light samples are the single "go to" parameter to control image quality when only direct lighting is considered. In needed, in RenderMan/RIS, we had to match light sample samples count with BxDF sample samples count to achieve acceptable quality and satisfactory convergence rates. Using light samples only (samples only, or BxDF samples only) produced slowly convergent samples only, produced noisy renders.  In the RenderMan tables /RIS results below, "N samples" means N samples for both light and BxDF.   We did all the test tests with the "advanced (mode 4)" light sampler —  other samplers did not provide acceptable results for this test casesince it produced the best results. The samples used by the renderer are the ones entered in the UI and are not squared as in Arnold. Note that we used the path tracer with once one bounce instead if of the "direct lighting" algorithm for one of the images because of a crash (quality and speed did not  seem to suffer although there were some minor differences in the render). 

3Delight – since the latest would crash once in a while. 3Delight – We have only one control for the general quality of the render. In the case of direct lighting, 3Delight "understands" that samples are best used for light sampling and that's what it does. As the tests will show, those samples have a linear impact on perceived noise levels.

Results

...

Arnold

The following graph gives a good idea on the convergence rate of the different light samplers.

Chart width 800 domainAxisUpperBound 300 title RMSE vs. RAYS type xyLine yLabel RMSE domainAxisLowerBound 0 xLabel Million Rays Rays 2.45 3.13 4.51 7.27 12.78 23.8 45.88 90.1 3Delight 0.0933142 0.0658266 0.0441248 0.0290439 0.0185575 0.0117566 0.00752546 0.00449892 Rays 0.678 3.26 10.8 16.94 43.4 173.6 694.5 Arnold

Samples (effective)2 (1.23)4 (4.91)8 (19.64)16 (78.56)32 (314.29)64 (1257.18)ImageImage Removed

Image Removed

Image Removed

Image Removed

Image Removed

Image Removed

Time1s2s6s21s1:218:12TTFP0s0.35s1.2s3.2s11s41sShadow Rays0.678 M

3.26 M

10.8 M43.4 M173.6 M694.5 MRMSE

0.15699 0.100115 0.0501787 0.0396399 0.0242515 0.0117413 0.00693426

...

Rays

Samples

1.47 2.94

48163264128256Image

Image Removed

Image Removed

Image Removed

Image Removed

Image Removed

Image Removed

Image Removed

Image Removed

Time5.72s8s12.82 s22.18 s40.92 s86.91165.69 s302.98TTFP3333333Shadow Rays2.45 M

3.13 M

4.51 M7.27 M12.78 M23.8 M45.88 M90.1 M

5.88 11.7 21.1 47.02 94.14 376.3 751.3 RenderMan/RIS 0.151125 0.121487 0.0953649 0.0728148 0.0520606 0.0373876 0.0265388 0.0138148 0.00854045
The following graph shows the time required to achieve a certain quality. From user's perspective, this is an important quantity.	Chart width 800 title RMSE vs. TIME type xyLine yLabel RMSE xLabel Time (seconds) Time 5.24 7.18 11.21 20.68 35.86 69.48 134.18 265.05 3Delight

RMSESamples

0.0933142 0.0658266 0.0441248 0.0290439 0.0185575 0.0117566 0.00752546 0.00449892

RenderMan

...

The following plot gives us a good idea the time required to achieve a certain quality. From the user perspective, this is an important quantity.

Arnold

*TTFP = Time to first pixel.

Samples (effective)	2 (1.23)	4 (4.91)	8 (19.64)	10 (30)	16 (78.56)	32 (314.29)	64 (1257.18)
Image	Image Added	Image Added	Image Added	Image Added	Image Added	Image Added	Image Added
Time	1 sec.	2 sec.	6 sec.	9 sec.	21 sec.	81 sec.	492 sec.
TTFP*	0 sec.	0.35 sec.	1.2 sec.	2.0 sec.	3.2 sec.	11 sec.	41 sec.
Shadow Rays	0.678 M	3.26 M	10.8 M	16.94 M	43.4 M	173.6 M	694.5 M
RMSE

Chart

width	600
title	RMSE vs. TIME
type	xyLine
yLabel	RMSE
xLabel	Time

Time5.72s812.8222.1840.9286.91165.69302.983Delight0.09331420.06582660.04412480.02904390.01855750.01175660.00752546

0.00449892

Time1262181492Arnold

	0.15699	0.100115	0.0501787	0.0396399	0.0242515	0.0117413	0.00693426

3Delight

...

RenderMan/RIS

...

The following chart shows how much time it takes to each renderer to build the light acceleration data structure depending on sample count.

Conclusions

• 3Delight generates light samples that are algorithmically better than (in term of variance) than both Arnold and RenderMan. In short, for N x effective samples:

1. 1. 3Delight Varince Variance ~ 1/Nx
   2. Arnold Variance ~ 1/sqrt(Nx)
   3. RenderMan/RIS Variance ~ 1/sqrt(N)(x) (possibly slightly worse but could be within a constant)
   
   Image Added

• 3Delight is slower to generate these samples. Meaning that for For draft renders Arnold/PrMan are faster(high variance), Arnold is fastest. For final renders (low variance) 3Delight is fastest and becomes increasingly faster with the number of samples.increasing samples.
• Arnold and RenderMan/RIS draw samples at about the same speed, but the quality of Arnold samples is better. 
• Both Arnold and RenderMan/RIS produce biased images at low sample counts. More specifically: images are darker. This explains higher RMSE with low sample counts. 3Delight manages to keep  the the same energy overall independent on independently of sample counts.
• Arnold, 3Delight and RenderMan/RIS rely on acceleration data structures to sample the geometric area lights. In Arnold and RenderMan/RIS, the algorithmic complexity to build those data structures is tied linearly to the number of samples (as well as the complexity of the light). In 3Delight, only to the complexity of the light matters (time to first pixel for 3Delight was 2-3 seconds no matter how many samples there arewere).  
  
  
  

Resources