ALshader with SSS on VrayProxy results in increased raycasting (compared to normal maya geometry)

[Jimmi]

Hi Guys.

So, on our previous batch of small commercials we made extensive use of the ALshader because we found favorable speeds when using SSS.
Our look development and tests were done in our asset scenes with the shaders directly attached to Maya geometry and all was lovely.
Then we did experience some sudden slowdowns when we hit shot production, but the artists put it down to increased complexities and needed sampling changes and time was tight as usual

Then today they started prepping for the next show, and they now very clearly noticed that there was a very large rendertime difference between rending our assets in "maya geometry mode"(maya reference of the alembic file) and doing it though a VrayProxy node.
I had a look into it since it was very strange and very obvious that there were some kind of issue.
My investigation leads me to believe that that our rendertime increase were caused by the SSS on the ALshaders when assigned to Vray Proxy nodes.
There seems to be a crazy increase in raycasting when rendering through the vrayProxy node.

Ive made a tiny test scene(attached here, just hide one sphere and press render) using mostly the default settings of the ALshader.
A simple maya sphere with an AL shader( with SSS ) is rendered as maya geometry and then the same shader is attached to a proxy node loading an alembic file of same said sphere.

The rendering stats are

Maya Geometry
[2018/Apr/6|20:32:26] V-Ray: Number of raycasts: 1166864 (2.25 per pixel)
[2018/Apr/6|20:32:26] V-Ray: Camera rays: 1166864 (2.25 per pixel)
[2018/Apr/6|20:32:26] V-Ray: Shadow rays: 0 (0.00 per pixel)
[2018/Apr/6|20:32:26] V-Ray: GI rays: 0 (0.00 per pixel)
[2018/Apr/6|20:32:26] V-Ray: Reflection rays: 0 (0.00 per pixel)
[2018/Apr/6|20:32:26] V-Ray: Refraction rays: 0 (0.00 per pixel)
[2018/Apr/6|20:32:26] V-Ray: Unshaded rays: 0 (0.00 per pixel)
[2018/Apr/6|20:32:26] V-Ray: Number of light evaluations: 369431 (0.71 per pixel)

Vray Proxy
[2018/Apr/6|20:32:43] V-Ray: Number of raycasts: 12569455 (24.25 per pixel)
[2018/Apr/6|20:32:43] V-Ray: Camera rays: 3914793 (7.55 per pixel)
[2018/Apr/6|20:32:43] V-Ray: Shadow rays: 0 (0.00 per pixel)
[2018/Apr/6|20:32:43] V-Ray: GI rays: 0 (0.00 per pixel)
[2018/Apr/6|20:32:43] V-Ray: Reflection rays: 3388374 (6.54 per pixel)
[2018/Apr/6|20:32:43] V-Ray: Refraction rays: 0 (0.00 per pixel)
[2018/Apr/6|20:32:43] V-Ray: Unshaded rays: 5266288 (10.16 per pixel)
[2018/Apr/6|20:32:43] V-Ray: Number of light evaluations: 5266445 (10.16 per pixel)

That... seems a bit dodgy.
For some reason there are suddenly reflection rays and unshaded rays being introduced, together with a higher amount of camera rays for some odd reason.

The proxy geometry goes in as dynamic geometry and not static, but other than that I dont see any other particular differences in how things are handled (from my point of knowledge)
I tried skipping the VrayMeshMaterial though a direct assignment, but that still results in the same.
Ive also been looking for sampling overrides or other things on the vray proxy(subdivision, normals, whatever) that could in any way have a potential influence, but to no luck.

Rendertime goes from a second to a few seconds in this example, but on our larger production assets it becomes quite the increase in time.

We use Maya 2018 on Windows, vray 3.6004 and I've also tried the latest nightly build (vray_adv_36004_maya2018_x64_28160_zip) which gives exactly the same results in terms of raycasts and light evaluations.

Hope there is somebody out there who might know what the cause might be. My guys would love getting some speed back into the renders for this next batch

In case others might be using the ALshader together with vrayProxy nodes I figured I would bring it up here on the forum instead of doing a support email.

[yolov]

That's one of the most thorough bug reports I've seen lately, thanks

Unfortunately I can't reproduce this here. Both cases take the same amount of time with the same raycasts statistics. I had to assign the ALSurface to the referenced sphere, though, by default it had the default lambert shader which gives roughly the same statistics as yours and the same difference when compared to the ALSurface. Could you double-check if that's the case with the attached scene? Would it be possible to share another scene if that turns out to be the case?

Btw, the SSS portion of the shader seems to fall into the "unshaded rays" category in the stats (I'm almost certain about it). And so far we haven't noticed any differences favoring maya meshes over proxies when it comes to shading. Differences could come from loading the proxies or from exporting maya meshes (maya meshes would be slower) but time would be spent in the geometry phases and not shading. However, I'm still interested in any potential issues, so if there's a case where things seem odd, we'll have a look.

[Jimmi]

For f's sake.... "thorough" means absolute nothing if its inherently wrong. :-/
This is pretty damn embarrassing.

My test scene is wrong as you say, I forgot to assign the damn shader to the Maya geometry as you point out.
I had been testing on production assets all day which did show obvious differences between Maya and VrayProxy versions, so my simple test case was setup a bit to quickly and I missed the point with actually assigning the shader.
Sorry to have wasted your time on that, it was a late evening

I will see if I can fabricate another test scene and do some further debugging
(Unfortunately I cant share the production assets and scenes)

Once again.. sorry. I will be back with more(and proper) info, we are still pretty sure that there is some kind of issue somewhere

[yolov]

I could have been wrong
But the first clue was that there were only camera rays and nothing else in the first log, which was the same as with the purely diffuse lambert, so I had a strong bet on it.

In any case, even if sharing scenes is not an option, any clue on what we can do on our end to reproduce this is more than welcome, so thanks in advance.

[Jimmi]

Thanks Alex.
In hindsight I should have made that conclusion myself to be honest, its pretty obvious. I will probably be hitting my head on my desk over this blunder for a while.

So, after checking things again, I will have to say that our slowdowns does not seem to be directly related to the ALShader and there are no differences in raycasting counts between the two ways of submitting geometry.
It was just, way to fast, conclusions drawn on various "hints" I thought I saw in my initial tests.
We had just jumped over to ALshaders resently and had not noticed the differences on the previous rounds of shows/assets.
My other tests had been done with trying quite a few things and I thought some patterns had emerged, which then unfortunately got to quickly "confirmed" by my mistake in the "simple test scene".
(If the title of the forum post could somehow be changed and state that people should ignore this post then that would probably be smart so we dont cause confusion for people)

Nevertheless, there does seem to be significant slowdowns for us when comparing direct Maya geometry to VrayProxy with an alembic cache.
It doesn't seem to be part of the alembic loading as it seems to be proportional to the sampling/raycasting done (so the overheads that might be with the geometry submission to the render is not important)

Ive done some more tests this evening on a production asset (4588 geometries with about 6.4mill triangles) and it seems to be partly down to the different Embree acceleration structures used.
Maya geometry makes use of the static trees and Proxy gets put into the dynamic trees.
Seems that the dynamic trees are very slow in comparison to the static.

Also an odd/unintuitive thing in the stats is that with static trees the "Number of intersectable primitives" is the triangle count, whereas with dynamic its the actual object count.

Disabling embree seems to give similar times for maya-geometry and proxy renderings (Embree is faster in all cases though)

I will post some info and results from my tests tomorrow (need to tripple check things this time )

[vlado]

There should be no difference in the ray counts (unless there is a bug somewhere). You are right that proxies are treated as "dynamic" geometry since they are loaded on demand as needed and therefore traced slightly slower. They also use a more compact representation of the geometry which saves memory but might be a little slower to access in RAM.

Best regards,
Vlado

[Jimmi]

Hi guys.

So Im back on this issue.

My current "theory" is that the differences we saw with our production assets were due to "normals" being different between the proxy(alembic) and Maya geometry.

Ive used the standard maya alembic export. I have not turned off normals and would expect them to get exported and then later "used" in side the Vray Proxy if I disable "Compute Normals".
Compute Normals with a zero angle gives me the same result as if the setting was off, hinting that its not picking up the Maya normals from the alembic cache.

I have tried with both "creases" enabled or disabled in the export (which should not really matter in this case, but I tried it in case something funky was going on)

Ive attached my little test scene where i ruffed up the surface a bit and then setup the same AL shader on both the Maya spheres and the vray proxy node.
There is a visible shift in the reflections between the two setups.

The visual difference was more clear when adding GI and subsurface and the raycounts changed a fair bit in our tests scenes because of the change in direction of the raycasting and the fact that the surfaces got "brighter" and the sampling then changed.
So, sorry for the long confusion on this matter.

Is there any specific things we should do when exporting our geometry to make sure the normals are the same as the Maya geometry?
Is it even supported?
Am I just being daft in how I handle the geometry?

Jimmi

[yolov]

Interesting. Well, I haven't dug too deep into this, but here's what I can say at a first glance:

I've tried to eliminate as much of the variables as possible, so I've
* removed the dome light and used a Maya directional light instead
* disabled GI
* assigned a simple diffuse material
* disabled the AA filter (it adds a bit to the render time and I wanted to take it out of the equation)
* disabled adaptive sampling (by enabling "lock subdivs" for the bucket sampler and setting the value to 1, forcing V-Ray to take 1 primary sample per pixel)
* set min shading rate to 1 (forcing V-Ray to take a total of 1 secondary sample per pixel - well, in reality this controls how many secondary samples are taken per primary sample, but both being 1 means 1 primary and 1 secondary per pixel)
--> this gave me a more controlled environment, so to speak.

There was an actual difference in both the render time and the raycast statistics between the maya geo and the proxy. I can't really say much about normals, but having "compute normals" on or off didn't make any difference for me.
However, I could think of one thing to check on - the main difference in terms of how V-Ray loads the geometry (and where it loads it) is that by default, it treats the maya geo as static geometry (which can be changed) and the proxy as dynamic geometry (which cannot be changed). This means that the maya geo is kept in one place in memory (the V-Ray static memory pool), while the proxy is in another (the V-Ray dynamic memory pool). There's an option that controls this in render settings > Settings > System > Default geometry. The default "Auto" means that V-Ray will detect which is which and put in in a different place for the sake of memory usage (let's say). Changing this to "Dynamic" will force V-Ray to load the maya geo in (pretty much) the same way and in the same place where it keeps proxies in memories.
Using this for the maya geo gave me similar results with the simple shader when comparing the proxy to maya geo for both render time and raycasts. To be more specific, the stats and render time went (almost completely) up to the level observer with the proxy.

From then on, I started complexing up the scene gradually back to where it started (added light cache GI, dome light, sampling settings and alsurface again). Then I compared a few renders with maya geo for each step to a few with the proxy (to take the average - I'll explain why at the bottom - it has to do with some OS/hardware dependent variations of split-seconds that add up. Note1).

The result is that on each step the results were comparable, with the exception of when GI comes into play. It seems that the Light Cache takes a different number of samples between the 3 setups when 1000 subdivs for LC:
* proxy - 374 samples
* maya geo (dynamic) - 370 samples
* maya geo (static) - 362 samples

And it does so absolutely consistently, meaning that re-rendering the same setup will yield the exact same result each time.
Now, V-Ray is pretty deterministic, which means that re-rendering not only the LC setups, but all of the steps I've taken (disabling and re-enabling GI, sampling and so on) always gives the same result. With my initial "more controlled environment" setup I actually wanted to eliminate any adaptivity that might put off the stats in one way or the other.

A few (interesting) things to note, though:
* without GI, a "dynamic" maya geo requires more samples/rays
* with GI - the proxy requires more samples/rays than both the static and dynamic maya geo tests

Note1 - there are a few things that happen in a different order each time that are not entirely in V-Ray's control, here's an educated guess (please note that, cause I'm not a V-Ray developer) - the order in which CPU threads are initialized, the order in which different things that need to go in memory go into memory (well, from the OS perspective) and of course - whatever other tasks my machine was performing at the time. All of these combine can add up to a few tenths of the second, which for a test that renders for 1/2 and hour or even hours won't be that much, but for a test that renders under 10 seconds actually represent an impressive percentage of the total time. For example, this can mean 1.4s vs. 1.7 seconds for two consecutive but identical renders. Apart from the render time, however, the raycast stats will be completely identical.

So in conclusion:
* I don't think that there's something that happens differently in the shader.
* I'm pretty convinced that that normals don't play a large role in this (but I will double-check this with the devs to make sure).
* I am somewhat confident that there is a difference in the way GI is calculated for dynamic and static geometry (again - I'll need a confirmation from the devs)
* and I'm completely positive that (most) of the differences you're seeing come from way geometry is treated and loaded into memory. By "most" I mean the render time. The only thing that I can't explain myself is the raycast stats, which is yet another thing I'll check with the developers.

Sorry for the long post. It might not be as interesting to read as it was to write

[yolov]

And of course, feel free to doubt anything I've said that can be proven with more tests, as like I said, I don't know the V-Ray code inside and out (and even if I did, I would not fully understand it). In fact, you're more than welcome to doubt me.

[Jimmi]

Hi there Alex.

Thanks for your indepth checks.

I separated the "proxy" vs "native maya" ( dynamic vs static tree ) issue into a seperate thread since I managed to mess up this ticket by my initial bad run and to me it seemed that it could be a separate talk that would probably be relevant for more people.
https://forums.chaosgroup.com/forum/...-render-speeds
Im not really sure if Vlado managed to profile the things, but it was pretty clear that there was a significant difference in speed, even at "similar" amounts of samples.

In the latest example I posted here it does seem to me that there is a difference in normals between the two cases in the scene.
(I forgot to mention this, but if I take the Maya geometry and do "set to face" for the normals, then the proxy and maya-geo produce visually identical results )
So, it does seem that the proxy node is not caching the normals in the alembic cache, and fully relies on recalculating them based on the triangle faces (and then smooths it if enabled).

So, not getting "exactly" the same amount of samples in that case makes sense to me.
But your case where you have sample differences between dynamic and static trees, that does seem a bit odd. I would have expected it to give identical results.

[yolov]

I thought the username in that other thread looked familiar

Yes, it is odd and it concerns me too. Since I got to these conclusions rather late in the work day, I made a note to check in with the devs in the next few days (for the potential normals issue as well).
I'll also see if there's any progress with the issues in the other thread, but somehow it seems at least slightly related to me. I'll let you know when we have some news for you.

[Jimmi]

Hey there Alex.
Any luck on chasing up devs on this?
( and to be honest, the other thread is more important for us, so any luck on finding out if there has been any progress on that? )

[yolov]

Still nothing worth sharing, I'm afraid.