Logikoma with simple outlines from "surface color" AOV (image credit: Paolo Berto Durante)

Outlining are typically used to define silhouettes and to outline features when doing toon/anime rendering or illustrations.

Outlines are often combined to CEL/toon shading, a type of non-photorealistic shading designed to make 3D computer graphics appear to be more "flat", for example in Maya the "Ramp Shader" offer this kind of look. However there are no artistic restrictions and outlines can also be combined with PBR-style rendering to develop more hybrid, photo-surreal and artistic looks

3Delight for Maya can perform outlining on any number of any desired Image Layers, including:

• shading component AOVs
• auxiliary AOVs
• "multi-matte" mask AOVs
• "multi-light" AOVs
• custom AOVs (any shading variable can be used for outline detection when output as a shading output)

Furthermore 3Delight powerful layer display subsets allow to isolate the detection on specific Maya sets only.

Advantages of using the 3Delight outlining system:

• Powerful imagelayer-based decoupling of the detection phase and the drawing phase
• Ability to isolate detection on a subset basis via layer display subsets
• Easy and logical to setup
• Ultra high quality outlines, due to state of the art sub-pixel anti-aliasing and filtering
• Fully working both in interactive IPR and final frame (progressive) rendering
• Outlines are drawn immediately (no need to wait for the end of the frame)
• Strong outline animation stability thanks to 3Delight analytic tessellation and rock solid geometry pipeline
• Supported in both Path Tracing (default, higher quality) and REYES (faster, lower quality)

The Outlining subgroup

Draw Outlines in this Layer

This attribute enables drawing of outlines for the selected layer. This can be seen as an indicator of which layer receives the outline. It is set to off by default with the exception of the "outline" auxiliary AOV which receives outlines by default.

Generate Outlines Using  this Layer

This attribute enables outline generation using the specified layer. Good image layers for detection purposes are the ones with sharp color discontinuities which facilitate the detection algorithm. When multiple image layers are used for generation (detection), they will be combined and drawn together and hiding will be resolved according to depth informations.

Usually (a combination of) the following auxiliary image layers, available in the AOV Selector or in custom shaders, are used to generate outlines:

• • surface color (auxiliary): this is the un-shaded and un-illuminated (textured) color of any object. It offers good stability, however detection will act on any eventual texturing, this means you can use ad-hoc textures to trigger detection of any detail you might want and for the same reason this might not be the ideal choice for detection in other situations.
  • root ID (auxiliary): the unique color-coded root ID (supports also instances). It offers goods stability across frames/shots being purely based on the maya full object name, and will detect an outilne between different hierarchies in the Maya DG
    
  • geometry ID (auxiliary): the unique color-coded geometry ID (supports also instances).It offers goods stability across frames/shots being purely based on the maya full object name, and will detect an outilne between different object/instances in the Maya DG
  • shader ID (auxiliary): the unique color-coded shader/material ID. It offers goods stability across frames/shots being purely based on the maya full material name, and will detect an outilne between different materials
  • face ID (auxiliary): the pseudo-unique color-coded IF for each face. This offers good stability. It detects a "wireframe" for the geometry. In order to export meshes with "face ID" information you need to add delightGeoAttributes> Geometry> Polygons> Export FaceID.
    
  • depth (auxiliary): this image layer is less stable, being view-dependent. It is often used to resolve, for example, facial features such as a jaw line. An outline will be detected between regions at different depth. Fine tuning with the outlining threshold is required.
  • world space normal (auxiliary): this image layer is less stable, being view-dependent and gradient colored. An outline will be detected between regions where surface normal, in world space, face away. Fine tuning with the outlining threshold is required. 
    
  • camera space normal (auxiliary): this image layer is less stable, being view-dependent and gradient colored. An outline will be detected between regions where surface normal, in camera space, face away. Fine tuning with the outlining threshold is required.
  • shadow (custom): this offers good stability if the shadow is sharp. Works for both delta-lights (spot/point/direct), area lights and environment. An outline will be detected around the shadow region.
  • curvature (custom): this image layer computes surface curvature based on several methods, it is pretty hard to detect stable curvature for the whole frame so it should be controlled on a shader basis, when controls are available. Fine tuning with the outlining threshold is required.
  • masks: any combination of geometry, materials or a mix of both, in a 3delightMaskSet (multi-masks): being black and white masks offer perfect contrast for detection.
  • outline texture (custom): this image layer outputs a specific texture in a separate AOV where detection can be executed. If the texture is properly painted it can offer good stability and can be useful to put outlines on areas with no geometric details. 
    
    

There is a wide range of artistic possibilities as any number of AOVs (including masks, auxiliary and custom ones) can be used to detect outlines, regardless of how stable they are it is up to you and your artistic purposes to define the look.

When this toggle is enabled, a part of the UI becomes sensitive to set the parameters of the outlines.

Outline parameters become sensitive for layers generating the outline.

Color

Specifies the color of the generated outline. It is set to white by default.

Outline Width

Specifies the width of the generated outline. It is set to ‘0.2’ by default. Values between ‘0’ and ‘1’ are allowed, producing very fine outlines. In this case it is recommended to raise the Pixel Samples attribute (in the Quality group or rendering attributes) in order to avoid aliasing of the outlines. Next to this attribute, there is an option menu to determine how this width should be interpreted. The available options are:

Threshold

Specifies the sensitivity of the outlining. The higher the threshold the more sensitive is the edge detection. For example, when detecting variations on ‘z’ (depth), a value of ‘0.1’ means that if there is a gap of ‘0.1’ between two surfaces (in the z direction) then an outline will be generated. This attribute is set to ‘0.1’ by default.

Depth Based Fadeout

Specifies the edge thickness fadeout with depth. It is off by default.

Min Z
Max Z

These two attributes specify the depth range on which the fadeout will be performed. By default, "Min Z" is set to ‘0.0’ and ‘Max Z’ is set to ‘1000.0’.

 Min Filter Width

The depth based fadeout will keep outlines with a minimum width specified by this attribute. Its default value is ‘1.0’.