Ray Depth Limit and You in Mental Ray

Posted 23 September, 2012 Anthony Tan (staring at yet another render. But still loving it for some reason..)

« previous | next »

Sparkles? Weird, unintuitive reflections? Lights doing crazyness? Raytracing? Bet you haven't set your ray depth limit deep enough (i.e. more than the default of 1..). Set it to four.

Ah, Mental Ray. I wll admit to having a major soft spot for you, but you can be trying what with strange defaults :P This one I sort of only really noticed while working on my 2012 reel's Tresemme bottle shot. It's a pretty bare piece and for about half a day I couldn't work out why I was getting some batshit-insane lighting results. I mean, it looked like light was going straight through one of the bottles and being reflected off another...that can't be right can it?

Oh wait. yes, it can.

Scene Setup

a groundplane, two cylinders, a wall and a light source. all in wireframea groundplane, two cylinders, a wall and a light source. all in wireframe
mr_raydepth_files.zip (402KB) - Maya 2012 ASCII file & PNGs

It's artificial, but basically, we have a groundplane (red), two cylinders (green and blue) acting as mirror items, a wall (white) which is used to shade the single lonely pointlight. The white dot is actually just something I stuck in during comp to show where the light is so it has no impact on the shot. Note from the orthographics that the pointlight should be completely, totally, utterly, unable to illuminate the two cylinders. What I particularly care about showing is the impact of the ray_depth_limit parameter so:

  • We have a simple pointlight set to have an intensity of 1, no decay, and pumping out white light;
  • Renderglobals have all the raytracing settings cranked up to stupid level (like, 10 reflection bounces); and
  • All the objects have a default MIA material applied, the only change is to addition of a green, blue, or red diffuse colour.

The baseline render (no shadows of any sort)

the base scene lit without shadows

Without shadows, we get this... you can see the two cylinders getting primary illumination from the pointlight. The white hotspot is illumination from the pointlight, obviously, and note particularly the reflections that the central cylinder is getting - it's seeing the cylinder to the right, and it's also reflecting in the groundplane.

Right. Flick on raytrace shadows at default settings (ray_depth_limit == 1). Hit the button.

Er, shadows?

the base scene lit with shadows on default

Immediately obvious - the light is being blocked correctly by the wall so the two cylinders behind receive no direct lighting, the black wedge is what you'd expect from the setup we have, but we have a couple areas of apparent madness. From the top:

  1. the floor is reflecting the objects as if they were directly lit;
  2. the central cylinder is reflecting the blue cylinder on it's right as if it was directly lit;
  3. both cylinders are reflecting the reflection of themselves, as if they were directly lit.

The reason that this looks weird, and wrong is due in some ways to an optimisation - the ray_depth_limit parameter. In essence, we're trying to set a threshold of bounces/traces at which we should stop calculating shadows and save ourselves some raytracing (optimisations like this are good things, they stop us from being here all weekend to render out a frame). Having an ray_depth_limit of 1 and means lights will stop spawning shadow rays after one bounce/trace which is only OK when you've not got anything reflective or transparent. Where there's proper shadowing (for example, on the floor in the black areas) the pathway would be as follows: eye ray into surface (trace 0), surface back to light (trace 1) at which point a shadow ray is detected, and all is good.

For the floor area where you can see incorrect reflections, the path is more like: eye ray into surface (trace 0), reflection in cylinder face (trace 1), cylinder face to light (trace 2), suggesting that you'll need to be tracing your shadow rays 2 deep in order to have them affecting that pathway.

Same exercise for the green cylinder's reflection of itself: eye ray into surface (trace 0), reflection of the wall's rear face (trace 1), wall's reflection of the cylinder (trace 2), cylinder face to light (trace 3), and yes, you guessed it. You'll need an ray_depth_limit of 3 here.

Here's the relevant renders, along with numbers on shadow ray volumes and for kicks, the render using depth mapped shadows which avoids this entire malarkey.

lighting results with an RDL of 2

lighting results with an RDL of 3 (Yep, there's still some multibounce reflections here..)

lighting results with an RDL of 4

lighting results with depth mapped shadows

Hope that all made sense?

In reality (har har. Reality.) this is a very artificial situation - particularly that four-trace reflection - other factors like decay rates and multiple light sources should be masking a lot of this as an issue (you wouldn't see that four-trace reflection with any sane light decay model applied for a start), but it's still a good thing to know about and really the default ray_depth_limit of 1 is just asking for trouble. I'd recommend setting it to 4, but at this stage I've not really done any performance impact testing - as always, YMMV. Best to really know your scene, rather than to follow any of my strange pronouncements.

Python scripts - setRDL

If it's of any use I've just thrown together a quick-y bit of python (relies on PyMEL, but that's a pretty trivial dependency) that'll set the RDL on all the lights or just for selected objects to a particular level. Essentially, if you want to just bulk run it, invoke it as follows:

  • setRDL() and it will set everything that's a light in your scene to 4.
  • setRDL(selected=True) is probably how you want to invoke it if you're doing it as a shelf button or something on the marking menu - this will target anything you've got selected
def setRDL(target_RDL=4,selected=False, override=False):
    """sets the Ray Depth Limit on all lights in scene to the target 
    selected -- whether or not to only work on selected items
    ignore_traps -- use with caution - this will set ALL lights to the target
                    RDL, irrespective of whether or not they have RT shadows on
                    or you've already set an RDL that's higher

    light_types_set = set(['pointLight', 'areaLight', 'directionalLight', 'volumeLight'])
    # select all the lights in scene 
    lights_in_scene = ls(sl=selected, type='light');
    # if we're doing selected == True, then we need to just check if you've
    # selected transforms, instead of the shapes (as I would do..)
    transforms_to_process = ls(sl=True, type = 'transform')
    for the_item in transforms_to_process:
        if the_item.getChildren()[0].nodeType() in light_types_set :
    print "setting the Ray Depth Limit to %d on all %d lights selected in scene" \
       % (target_RDL, len(lights_in_scene))

    success = 0    # how many set successfully
    bypassRDLover = 0    # how many skipping due to RDL being over 4
    bypassRToff = 0    # how many skipped because they're not RT enabled
    for the_light in lights_in_scene:
        # if the light has RT shadows enabled, and the RDL is less than or
        # equal to our target, set the RDL.
        # you can remove the 'if light has RT shadows' trap if you wanted

        if override == False :
            if the_light.getUseRayTraceShadows() == False :
                bypassRToff = bypassRToff + 1
            elif the_light.getRayDepthLimit() >= target_RDL :
                bypassRDLover = bypassRDLover + 1
            else :
                success = success + 1
        else :
            success = success + 1

    print "RDL not set due to raytraced shadows being off: %d / %d" \
          %(bypassRToff, len(lights_in_scene))
    print "RDL not set due to RDL already set higher: %d / %d" \
          %(bypassRDLover, len(lights_in_scene))
    print "success: %d / %d" % (success, len(lights_in_scene) )
    if override :
        print "TRAPS OVERIDDEN"
    print "done"
Accentuate the negative, eliminate the positives!
Good Cylinder, Bad Cylidner
  1. Mental Ray is a renderer that rewards the curious, and the person willing to dive into the guts of a system in order to make it jump hoops. MR is good like that, it'll do whatever tricks you want to teach it to do - kinda like Renderman, but just different.. don't let anyone blame MR for their rendering failures unless they can show you how they've also tried to mod things to reroute around the problem. Of course if they're complaining about it being complex.. that's another story (but we are dealing with a complex subject here..)
  2. I'm obviously not the first to write about this (see Master Zap over at http://mentalraytips.blogspot.com.au/2007/10/mayas-default-shadow-settings.html), nor is it particularly new. But heck, writing something and putting it online at least makes sure I understand what the heck is going on - it's like a science experiment, til you do it, you don't know if you really know what you're doing.