Correct workflow if CG object integration onto backplate... ?

[LudvikKoutny]

Hi,

i am having very hard time figuring out what is the correct workflow of integrating CG elements onto backplate in Vray.

In Vray, there is no matte/shadow material, just VrayMTLWrapper or Vray object properties.

So say i have an HDRI, backplate and a car sitting on a plane. I can not just plug camera mapped backplate into diffuse slot. Backplate contains final baked lighting information, and plugging it into diffuse slot would alter that information. So i usually plug it into the self illumination slot, and then feed this material into VrayMTLWrapper, which has enabled shadow catching.

Here is my problem:

I first rendered two reference images, one in Corona renderer, other one in Mental Ray. Both these renderers have very simple, well documented and strict workflow of the matte shadow workflow, so it was a nobrainer:

MentalRay:


Corona:


Now, i tried the workflow i mentioned above with Vray, and here is what i got:



As you can see, there are two problems:

1, There is a strange glow illuminating my car from underneath. It seems as if the self-illum material was not affected by shadows cast by the car. The issue is even more apparent, if i raise exposure of my backplate:


You can see very unrealistic glow appearing on the areas where tires touch the ground, as well as on the bottom edges of the car body.

2, There is a line on the edge of my plane. Vray for some reasons spreads shadows a lot further than other two renderers... a lot further than it should. You can see that in the alpha channel:



I have not yet found any solutions to these problems, and there are even more problems, that make CG integration in Vray very painful:

Vray does not have it's native camera mapping solution, so:

1, If i use Camera map per pixel, then camera projected texture tiles outside of the camera frustrum. So i very often get really weird reflections in my CG objects. For example where ground stops at the bottom of frame, sky of my backplate starts to tile right after it, creating weird bright reflection on the places where they should not be.

Other renderers offer an option to pass off-screen rays to environment, which makes sense, as if you have matching HDRI and backplate, and align them properly, returning what's on HDRI in given direction gives the most accurate data about what could be there.

2, With camera map per pixel, i am really limited only to rendering from camera. And i cannot render freely from perspective views.

So i would like to know what's official and proper way to integrate CG elements into backplate or footage with Vray, while avoiding problems described above.

[kosso_olli]

I often have the same problems. What bothers me most is that I can always see the reflection of the shadow catcher in my cars. What I would like to see in the reflection are the shadows being cast from the car, but not the whole ground plane the shadow falls onto.

[LudvikKoutny]

I often have the same problems. What bothers me most is that I can always see the reflection of the shadow catcher in my cars. What I would like to see in the reflection are the shadows being cast from the car, but not the whole ground plane the shadow falls onto.

Well, if you have properly camera-projected hi-res backplate, then i believe it is desired to see reflections of the shadow catcher plane in the car, so it looks like it is properly grounded in the environment. Like on this picture:

Notice how teapots appear to be sitting right on the white lines, making the correct impression of being there in contact with environment

The problem i have, is that it seems vray reflects and illuminates car by the ground before it has shadow applied on it.

[kosso_olli]

Looks good! Can you provide a stripped down sample scene? Because I seem to have a different problem. When I try to get the shadows reflecting in the teapots, I can always see my whole ground plane in the reflection, not just the shadow. I don't know, but I think I am doing something wrong.

[Jorq1H]

Here's an example of my own workflow. It's very basic, but it seems to give more or less accurate results. The reflective objects reflects the shadow catcher plus its actual shadows, as seen in the example below:


(Click image to view full size)

The material setup:


(Click image to view full size)

I often use my HDRI's as a backdrop as well, in which cases I leave the Generate GI on. But when using separate shots as backplates I turn that option off, or else the shadow catcher plane will emit the texture onto its surroundings as secondary rays.

[LudvikKoutny]

Now that is just completely wrong. As i said, backplate has already baked lighting into it. Lighting which you should not temper with unless you plan to add a light that was not there when HDRI and backplate was taken. MentalRay has an option called illuminators for that, but that requires some differential shading to be done as well.

The problem is following. If you simply plug the backplate into diffuse slot, then you have lighting from the location already there, but you apply it once more. And if you, say, put sphere light somewhere near the backplate, it will completely corrupt the lighting information there. The point of photographed backplate is that lighting is already there, the environment was illuminating the backplate when you were taking the photo, so you just add some CG objects, and occlude certain parts of the projected backplate from illumination of your HDRI by the CG object, but do not add any lighting or shading to the backplate.

To clarify - i am not looking for some arbitrary Bob's random workflow. I am looking for the one correct workflow, which any production renderer should be capable of.

kosso_olli: I will send you the stripped down scene once i get to work tomorrow

[Jorq1H]

If you simply plug the backplate into diffuse slot, then you have lighting from the location already there, but you apply it once more. And if you, say, put sphere light somewhere near the backplate, it will completely corrupt the lighting information there. The point of photographed backplate is that lighting is already there, the environment was illuminating the backplate when you were taking the photo, so you just add some CG objects, and occlude certain parts of the projected backplate from illumination of your HDRI by the CG object, but do not add any lighting or shading to the backplate.

I thought this was what a matte surface was supposed to do. Not receive light, only occlusion. Look at the alpha channel in my test render. This only adds the CG objects and their shadows (and reflections if need be). Where is the original "baked" lighting corruped? The ground is identical to the plain photography where there's no shadows.

I'm not trying to be difficult here, but I don't really see this problem. Although, I completely agree that there should exist one correct, universal way for vray and any render engine to blend fake shadows and a real backplate. If my approach is wrong, I hope we'll get some other, more correct answers in this thread soon.

[LudvikKoutny]

I will post some examples of why it is wrong tomorrow. Basically, matte mode in vray just hides primary visibility of the material, and replaces it just with caught shadows. But in reflections of your objects, you will still see the original material. And it will not be the unaltered backplate, it will be backplate altered by the illumination of the scene. It will react as a diffuse surface, not constant surface.

[Jason Stewart]

So it sounds to me that you are using a plane to project the backplate onto is that correct? I am not an expert on the subject, in fact I have never actually done it but Grant Warwick just did a tutorial on this (well not technically on this but it was included) a couple weeks back in his mastering vray course. I can't remember the exact details but I know that the backplate was not on geometry and it was projected on the "shadow catcher" plane via camera map. This way the backplate (the actual background) does not get changed by the scene, I can't exactly remember what he did with the shadow catcher I will have to look at it again.

One of the "issues" with that tutorial is that the backplate and spherical HDRI lighting the scene were taken about 50 feet away from one another so the use of the shadow catcher as a matte object would not have worked out well because the reflections would have been way off.

The illumination coming from the bottom of the vehicle could probably be stopped by turning off the full dome option in the vray dome light, that should avoid any illumination coming up from the ground but it will also eliminate any reflected light from coming up from the road if you understand what I am saying.

I will have a look at the tutorial again when I get home, can you post some screenshots of your scene setup just so I can figure out where they are different?

[Wilyman]

I will post some examples of why it is wrong tomorrow. Basically, matte mode in vray just hides primary visibility of the material, and replaces it just with caught shadows. But in reflections of your objects, you will still see the original material. And it will not be the unaltered backplate, it will be backplate altered by the illumination of the scene. It will react as a diffuse surface, not constant surface.

I don't think that's the case when you're using the 'matte for refl/refr' option, as Jorq1H is above.

I'm sure someone (Vlado?) will correct me if I'm wrong, but I think you'll get a reflection or refraction of the original backplate along with the shadow of the object.

The colour from any GI might still be slightly off, but the reflections should be exactly what you're after.

Cheers,

John

[LudvikKoutny]

Okay, here's the long post about why you can't just map diffuse:

I have chosen a nice image of a candle i found on wikipedia with free licence:



And i decided to integrate some CG elements into it.

- My first step was to recreate light sources apparent on the image, so i can use them to illuminate and cast shadows on my CG objects.

- My second step was to model a rough placeholder for the candle, so i could then camera project the image onto it as well, which ensures that i get the candle reflected in my CG objects and that candle casts shadows on my CG objects.

- My third step was adding a few teapots into the scene, as well as a mirror box to see what's going on in the reflections:


Here's my initial layout:



------------------------------------------------------------------------------------------------------------

What i did first, was just plugging my backplate into camera per pixel map, plugging my camera per pixel map into diffuse slot of VrayMTL, and assigning VrayMTL to the base scene geometry and candle placeholder.

This is what i got:


You can immediately see it is completely wrong. Objects that are already in the scene, and their representation is modeled (table, wall and candle) are receiving light even when the light is already photographed there, resulting it double light added, and there are shadows of these objects as well, resulting in double shadows. Proper shadow catcher solution should not add any light to the backplate, nor should it shadow itself.

You can also see how projected texture tiles in the mirror. That's because Vray doesn't have any capable camera projection tool of it's own, which would allow to pass of screen rays to environment in reflections.

------------------------------------------------------------------------------------------------------------

Now, i fed the material through VrayMTLWrapper, enabled matte object and enabled catching of shadows:


The directly visible portion of scene placeholders now looks fine, because it shows the screen mapped environment map behind it, but reflections are still completely wrong. Reflected objects have double lighting, double shadows, and thanks to the double shadows, bounce light on the bottom of teapots is a bit darker, than it should be, therefore incorrect.

Candle reflected in chrome teapot is a lot darker as well, because it does not reflect original light data on the backplate, but modified luminance caused by plugging the backplate into diffuse slot, which modifies shading.

You can see the difference especially on the directly visible candle vs reflected candle.

------------------------------------------------------------------------------------------------------------

So i went back, and assigned just VrayMTL to the placeholder geometry, and placed my camera mapped backplate into self-illumination slot, instead of diffuse. Diffuse was set to black:


Now, the camera mapped material looks correct. It does not alter lighting of backplate in any way, my CG objects still receive lighting, and are shadowed by the placeholder. BUT there are no shadows received on the projected material.


I though to myself that this may be handled by the MTLWrapper, so i plugged my self illuminated VrayMTL into the Wrapper, but sure enough, shadows still were not caught in reflection, or GI, only on directly visible surface. That's the source of glow i described in my original post:


------------------------------------------------------------------------------------------------------------

Now, here's MentalRay example of a correct setup that i am looking for. Matte objects do not shadow themselves, but shadow CG objects. Projected backplate luminance is not altered, but capable of catching shadows, and reflected elements propagate this behavior correctly:


I am also attaching both Vray scene and MentalRay scene for everyone to play with It's Max 2014 with Vray 3.00.06. Materials can be found in Slate Material editor view.



I really hope there is a simple and correct way to do it.

[LudvikKoutny]

And the winner is Wilyman!

Indeed, matte for refl/refr option indeed does the trick. It sounds quite confusing, so that's probably why i never touched it. Things like these should be well documented somewhere. Now, there is one more problem. Once i enabled it, i got this super strong, unrealastic GI spill coming from somewhere. I doubt what's happening there is physically correct.



And also, the problem of projected backplate being tiled off the screen still remains.

Thank you very much non the less !

[LudvikKoutny]

Actually, i take that back. The problem is still not resolved. There's a huge issue of matte objects self-shadowing. Matte objects should never ever be self-shadowing, yet they are. So i am getting double shadows. If i simply disable shadow casting of matte objects, then my matte placeholders won't be casting shadows on my CG objects.

[LudvikKoutny]

To display the issue:

Here's the original photo:


Here's Vray render with only base geometry, candle placeholder, and one teapot assigned with same matte material:

There's apparent issue of shadows from the candle being 2* darker, and teapot casting shadows on matte object even when teapot itself should be matte object.


Here's Mental Ray with only base geometry, candle placeholder, and one teapot assigned with same matte material:

As you can see, identic to original photo, as candle is not casting shadows which are already there, and teapot in the bottom right corner is correctly invisible, as it is a matte object which should not be casting shadows on other matte objects.

[Wilyman]

You can also see how projected texture tiles in the mirror. That's because Vray doesn't have any capable camera projection tool of it's own, which would allow to pass of screen rays to environment in reflections.

Just copy your camera map per pixel map into the opacity slot of the material, and stick a plain white image in there.
Then the part of the object that's in the frame will be opaque, everything else will be transparent, and your environment reflections will get through.

Actually, i take that back. The problem is still not resolved. There's a huge issue of matte objects self-shadowing. Matte objects should never ever be self-shadowing, yet they are. So i am getting double shadows. If i simply disable shadow casting of matte objects, then my matte placeholders won't be casting shadows on my CG objects.

I don't have VRay open at the moment, but isn't there a tick box in the properties that says something to the effect of 'no shadows on other mattes'?
Or am I dreaming it?

Cheers,

John