Announcement

Collapse
No announcement yet.

Understanding DMC Sampler

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Understanding DMC Sampler

    Hi,

    I was hoping someone can have a look over these two elements, both are from the same scene but using different DMC sampler settings, and perhaps give me an insight as to what is going on.

    This first one is 1/50, clr thresh 0.007

    Click image for larger version

Name:	samplerate_1-50.jpg
Views:	1
Size:	206.4 KB
ID:	877281

    This second one is 1/16 clr thresh 0.003

    Click image for larger version

Name:	samplerate_1-16.jpg
Views:	2
Size:	404.0 KB
ID:	877280

    I follow with interest various threads that touch on using this element to understand noise, (a lot of input from joconnell actually so hope he sees this!) and I've concluded that the second image is what should be expected if the dmc sampler is working most effciently, a nicely balanced 'heat map'.

    However, that image rendered in 4.5 hours, the first in 38 mins, with no real difference in the beauty passes! Now of course I can simply stick with the settings for the first image as I'm happy with the result, but I'm not understanding why these settings work, can anybody help?

    And I've read the demystifying dmc sampler article a number of times but am struggling to understand that too!!

    Cheers, Michael.
    Cheers, Michael.

  • #2
    At the end of each render, V-Ray will print some statistics about the number of camera/reflection/refraction/shadow/GI rays, as well as the total number of traced rays. Compare the numbers between the two renders - it should help to figure out where the difference comes from. I suppose that it might be good to put these into their own render elements too.

    Best regards,
    Vlado
    I only act like I know everything, Rogers.

    Comment


    • #3
      Hi Vlado,

      Where are those statistics, the job was rendered through backburner?

      Cheers, Michael.
      Cheers, Michael.

      Comment


      • #4
        AFAIK
        1. red in the second image means that 16 as max DMC is not enough for the required clr thresh (0,003)
        2. red for large areas for me it's no good; it should be confined to very small portion of the image
        Alessandro

        Comment


        • #5
          Vlado:

          I'm all for more statistics and predictability in the dmc sampler as you can guess by my stupid questions over the past while - The sample rate element is a really handy diagnostic tool to start with and a similar tool for reflection, refraction and lighting would be amazing too - I think you mentioned that it'd have a memory overhead and I'd imagine also a cpu overhead for the extra sums but during the tweaking stage I think I'd find it really useful

          Michael - beware, long post that has to be split in two for forum letter counts


          With your sample rate element, it only deals with your anti aliasing sampling and nothing else. In your first image of 1/50, you're getting no red values so it means that no part of your image requires your upper 50 value of aa sampling, nothing anywhere near that in fact so it means your aa is more than the scene needs. It looks like it's a geometrically and bump texture simple scene so it doesn't need massive aa. With the second image, your aa has gone down a lot so now you're getting way more red pixels as far more parts of the image are using your lowered ceiling of 16 aa samples. The lowered colour threshold is making vray use a fair finer level of quality before it can finish rendering too - I'd be interested to see your render times at 1/16 if you'd left the colour threshold at 1/16 and 0.007.

          If I was to take a guess, I'd say your scene is being heavily lit from outside, so all of the complicated sampling in your image will be down to a really wide range of brightness, your glossy reflections on the floor covering such a large area, and then the very soft shadows coming in from your window light. If you have a look at the red pixels, they're on the smooth curving lines of the chairs, and on the edges of the window frame. I'd say the wall lines at the window and around the skirting board aren't getting hit with very bright light, so they get medium green sampling - geometrically they're the exact same as the lines making up the window panes, the only difference is the pixels along the edges of the window panes are being blasted with whatever bright or complex light source is outside the window, so vray is putting additional AA samples there to deal with the difference in brightness values between the inside normal exposure and then the extremely bright rim pixels. So you probably don't need to use much aa sampling, but you need to up your light samples for your main window light source, and likely the reflection samples on the floor material. Could you pop up a Raw GI, reflection and raw lighting pass along with the beauty to have another look? I'd say what's happening with the bottom render is that your reflections are complicated but your floor material reflection samples aren't that high, thus vray is having to use aa sampling closer to it's max value to clean it up - hence the red pixels. If you open up your 1/16 scene, see what happens to the sample rate element and render times if you turn up the floor reflection samples by a factor of 4 and render, then set it back to what it was before, then turn up your external light source samples by 4 and render again. I think your scene is more complex in terms of material and light sampling rather than AA so it might make a difference.

          In terms of the DMC tutorial there's a lot in it! I've asked a lot of questions to the guys in chaos to try and clear up my understanding of DMC and Vlado and Svetlozar have been brilliant. I've to go back to Toni's tutorial again and re read it knowing what I know now. If you want what I've found as relates to it, it's as follows:

          Vray always tries to balance your various samplers to get the cleanest result with the smallest amount of sampling.
          Vray's AA sampler makes the biggest final difference to how it decides sampling.
          The sample settings of all of your materials get divided by your top AA setting, so the higher you go with your max AA, the more your material and light settings get divided.
          If your material or light sampling is too low, it'll force your AA to go to higher levels to try and clean up the noisy results that the light or material has returned to it (the cause of your red pixels in the 1/16 render).
          Just because you use insane sample values in your materials or lights, it doesn't mean that vray will actually use those numbers, it just gives it the option of using them.

          At the minute with sampling, I find the AA is a good place to start as it provides the most measurable response via the sample rate element - you get a visual reference of exactly what it's doing, and since the AA sampler is influencing everything else it's kind of a good "pivot". Now, the sample rate element WILL change as you tweak other parameters. For example in your 1/16 image above, if you were to either turn up your light samples, or turn up your material reflection samples on the floor, all of the red area on the sample rate element will start to drop off. Since your material is now doing more work, it'll return a far cleaner result to the aa sampler, and it'll have to use less of your max aa samples to reach the right level of cleanness for your final image, and this is the vray internal balancing act.

          As one method you could try doing this to get things semi predictable:
          - Turn on the default scene lights and hide all the others, turn your GI off. Stick on a grey override material so that you get a crap looking render, but the important thing is that you only want to see how vray is sampling your edges.
          - Tweak your aa until you get really clean edges and your sample rate is mainly blue and then red at the edges where you need all of the quality.
          - Next keep your middle grey material but turn on your proper lighting. With this you've got new levels of noise to deal with from the raytracing of your light sources and if you're using brute force, from the light sampling here too. As a result your sample rate element will change again, as your previously clean, boring surfaces in your grey render now have shadow and highlight areas. The shadows if they're particularly soft, will need more sampling and cause grain with the default light subdivs, so you'll find that your sample rate element will now get more red, as your light is causing grain that forces your aa sampler to do more work. What you can do is leave your AA where it is, and turn on your Raw GI and Raw lighting elements. If you've got grain in your raw lighting pass, that means you don't have enough samples in your light source, so gradually turn up the samples until you get something that's clean enough. Don't be afraid to use big changes in the value. If you start with 8, try going up to 25, 50 or 100. If I'm using a hdri in a domelight for example, I've ended up using values of 400 on the current film I'm working on. So ideally what you can do is maintain the AA settings you've had before, and alter your light samples until this is now clean also.
          - Last turn on your proper materials. Turning these on means you're now bringing in noise from glossy materials so again if they're using low samples, they'll likely cause some grain which again will cause the aa sampler to again do more work so your sample rate will again go red in the noisy areas. Again this means you're going to have to up your material samples in the materials so turn on your reflection, raw reflection, specular, refraction and raw refraction elements. The non raw versions are the most important as they're the level of reflections / refractions which finally end up added into your pixels. The reflection filter element can be useful here too as it'll tell you exactly how much your reflection is conributing to the final materials - pure white pixels mean 100% reflective, dark grey mean nearly no reflections. If it's noisy in the raw reflection for a material, but the reflection filter has almost black pixels then it's quite likely that vray has decided it isn't important enough to the final result to go up to any higher quality with. In terms of cleaning up reflections I've found that glossy reflections themselves are partially to do with their own material sample settings, and second of all the samples of the lighting solution that they're reflecting. If you take a glossy surface and light it with a hdri, you can clean up noise by either turning up the material samples, or turning up the samples of the light that hits the surface. If you're unsure which, take a look at your raw lighting channel. If the raw lighting of the surface is pretty clean but your reflections are noisy, up your material samples. If both your lighting and reflection channels are noisy, then up your light.

          This is my current method of thinking and understanding vray, if anyone has enough time to read the above nonsense and I've got gaps or misunderstanding then I'd love to get any feedback.
          Last edited by joconnell; 21-02-2013, 07:32 AM.

          Comment


          • #6
            -- OVERALL THOUGHTS


            With the universal method set up by vlado a while back he didn't intend the render settings to be the best or the fastest, just a clean and understandable method to get good results for everyone. We're starting to push Vray a lot harder since version 2.0 and the quality of work has come on hugely, so that method isn't as relevant in all cases, and where we need to get extremely clean results or fast results then the universal methods won't be the best option.

            Using the sample rate, raw light, raw Gi, raw and non raw reflect and refract and reflection filter passes what you should be able to do is identify what type of scene you have.

            If you have really simple geometry but it's something like an interior room lit by a hdri, then you probably need less AA but more light samples.
            If you have a scene where everything is glossy but again it's simple geometry, you can probably use less AA but more material samples.
            If you have a scene where you've got lots of complex geometry edges but a simple lighting setup such as a forest or a field with grass, you'll need lots of AA quality to keep the leaves from sparkling a lot, especially if the camera is moving, and then probably quite a few gi samples to get the right level of detail in all the small light bounces.
            If you have an object lit by a bright HDRI that has glossy reflections, you'll definitely need higher light samples in the dome light with the hdri, and then maybe more reflection samples in materials that have glossy reflections as they're reflection something with quite a bit of information too.
            If you have a scene that's mainly non reflective - take a room with walls, carpets and furniture but the off shiny object and some small interior lights. This will likely be overall quite clean with low aa, light and material samples save for a few parts. With this you're best off using your passes or turning lights on and off to find out what materials and what lights are causing the noise and then really heavily upping the samples for these small parts, but keeping the overall scene samples low.


            The main thing is to try and identify the cause of the noise and irradicate that, knowing that all the time your AA is reducing the amount of samples everthing else uses, so I like to get the AA roughly right first and then use that as a base to start changing the light and material things.

            Hope this helps and doesn't confuse you even more.

            Comment


            • #7
              WOW. Amazing input, joconnell, many thanks! I was really hoping you'd get involved as the workflow you've just described is kind of how I started this testing process, I'd read it in another thread.

              Scene is lit by one domelight, a peter guthrie hdri. Intention was to just use this as only light source as BB3viz did in his 'old vicarage' work.

              I then went the route of using grey material until I got a really clean rawlighting pass, domelight subdivs at 256, texmap resolution at 2048.

              Click image for larger version

Name:	1-16_rawgi.jpg
Views:	1
Size:	177.0 KB
ID:	846955Click image for larger version

Name:	1-16_rawlight.jpg
Views:	1
Size:	161.1 KB
ID:	846956Click image for larger version

Name:	1-16_reflect.jpg
Views:	1
Size:	140.1 KB
ID:	846957Click image for larger version

Name:	1-16_rgb.jpg
Views:	1
Size:	254.0 KB
ID:	846958Click image for larger version

Name:	samplerate_1-16.jpg
Views:	2
Size:	404.0 KB
ID:	846959

              Hopefully you agree that these passes are clean? I don't normally get this deep into noise, I have stock settings that work great in terms of deadlines and business, but I want to push things further!

              So I was happy with cleaness of light and then switched on my already setup materials and got the results. Ok and ready for post, but unacceptable render time of 4.5 hrs using our dr servers with lc secondary and imap primary 3200 x 1800. Normally accept about an hour a render.

              So I'm pleased with results and noise levels but not the times, and looked to the samplerate element and got to this point. I'll do some more tests based on your input tonight, however its worth noting the floor mat has ref subdivs 48, the red leather chair has 64, which I consider very high at the minute....

              What do you think to these passes, are they what you expected?
              Cheers, Michael.

              Comment


              • #8
                Double post
                Last edited by joconnell; 21-02-2013, 08:41 AM.

                Comment


                • #9
                  Righty. The passes look beautiful but let's try to get them quicker.

                  Looking at that, I think you'll need more samples on the floor and maybe the roof to make the aa sampling easier. I reckon you'll still get some red on the floor but it'll be more based around the detail in the bump map (which is considered the same as geometric detail, thus it's an aa thing). I'd be inclined to turn down the res of the hdri to make your light clean up quicker as all of the main surfaces in the scene aren't reflecting it. If you changed the angle of the camera so the window was in the background and the outside HDRI was then being reflected on the floor, it'd be a different case. Right now though I'd wonder if you're getting any major benefit from the extra sampling needed. I'd say your AA threshold is probably quite high so I'd experiment with bringing that up to 0.01 and trying to clean with material samples first. If you were still getting noise, you could also look at unlinking the aa sampler colour threshold from the "use dmc threshold" option. Again since the scene is quite simple geometrically, you could perhaps get away with a lower quality on your AA of 0.01, then maybe go lower on your dmc sampler noise threshold of 0.08 or slightly lower.

                  Vlado - I could be thinking too simply about the relationship between the aa noise threshold and the dmc noise threshold. Is it literally a case of being able to separate the noise levels of the final aa compared to the other fuzzy samplers? Or by unlinking the two is there any other more careful balance broken?

                  Comment


                  • #10
                    Originally posted by joconnell View Post
                    This is my current method of thinking and understanding vray, if anyone has enough time to read the above nonsense and I've got gaps or misunderstanding then I'd love to get any feedback.
                    this is very valuable insight, it's great from your part to take the time and share your method of work and findings on the forums. must admit I've learnt a lot from your posts on how the dmc sampler works, especially the idea of render elements as being meaningful in other ways than just compositing. it's been a real eye opener.

                    Comment


                    • #11
                      Cheers Rivoli,

                      The only thing I hope is that I'm not wrong on any of the things I'm writing - it can do as much harm to spread false information as to put up no information at all

                      Comment


                      • #12
                        Not sure I follow the last bit of advice joconnell, the passes Ive shown used these settings...

                        Click image for larger version

Name:	settings.jpg
Views:	1
Size:	142.2 KB
ID:	846960

                        By the way, I totally agree with rivoli, I've learned so much from your posts across the forum, it really is invaluable input, very much appreciated.
                        Cheers, Michael.

                        Comment


                        • #13
                          Ah yes.

                          So your Adaptive DMC clr threshold can go up to 0.01, and then if it brings in a huge amount of noise in your render, try using the noise threshold in the system tab > Vray Dmc sampler rollout. This will be the noise / quality threshold for the material and light sampling.

                          Comment


                          • #14
                            Just curious. Would a "heat map" type element be useful to show what buckets took how much time compared to the rest (normalised) to further debug this type of problem? or will the sample rate element cover this?
                            Signing out,
                            Christian

                            Comment


                            • #15
                              Hmm - The sample rate is showing you way more detail than the buckets would so you can still use it to chase things down to an object / material level - the more options we have to guide us though the better, once they aren't taking away development time from other important things. I found when working as a lecturer that people take in information in different fashions, so a way of thinking that works for me mightn't work for you and vice versa so the more ways we have of looking at a problem the better!

                              Comment

                              Working...
                              X