Multiple GPUs on DR message box question?

[biochemical_animations]

Is there an entry to look for in the message box for determining if the render node GPUs are being used using DR on active shade on RT GPU mode, like how the message box displays server (ip address) when doing a bucket production render?

I just added some GPUs to my render nodes this weekend and am testing to see if everything is working. I have a GPU monitor gadget displayed on a remote desktop connection (win 7), only problem is the gpu memory load doesn't get displayed...it says 0MB used out of 0MB available. I've tried re-installing the video drivers and such and same result.

However, the GPU load goes to max on the gadget.

I can switch to a monitor plugged in via hdmi, to see the gadget display the memory...it says the memory gets loaded when I press render, and I see the GPU load max out too. So it looks like it is working, but I just wanted to double check to see if there was an entry in the vray message window...thanks.

OK, so when I am on remote desktop, the memory usage will not display correctly, is there anyway I can check to see if a render node gpu is working properly via the message window?

[savage309]

I am not sure tbh, probably the best is to look at the render log of the render nodes. It says which GPUs are being used for rendering.
Using remote desktop can cause that troubles with monitoring tools - if all works correctly, I wouldn't pay much attention at the 0MB used.

Best,
Blago.

[biochemical_animations]

Thanks. Can you please point me in the direction of where the node log file would be?

I looked under C:\directory but I don't know what this is and cannot find anything intuitive.

[savage309]

It is in the console while rendering, but tbh I don't know where it gets saved. You can ask support@chaosgroup.com I guess.

Best,
Blago.

[biochemical_animations]

The console you are referring is on the node or ws?

[savage309]

Usually when a DR node is spawned on the node, it should have console. I am not really familiar with the DR, so there could be a ways to spawn it without it ... best to ask support (I am afraid that it is possible to confuse you with something, since I am not really familiar with that).

Best,
Blago.

[biochemical_animations]

OK, I found the log file, it is in under the users, app data, local data, temp directory. The only file that had recently dated information was vraylog.txt

I looked under the windows directory and this directory on the node and didn't find recent information regarding todays render tests...the node vraylog had info from 11/18 was the most recent, so I don't think the node is opening a console or writing a log file. I viewed the remote desktop and saw no console.

So my goal here is to see if the new GPUs on the render nodes help and to what degree. I posted a thread in the GPU forum regarding DR render times where higher, which I don't understand why.

Now these are just tests to understand how everything works. I probably will render animation frames to each node and not use DR, but it would be nice to have to see some frames DR rendered very quickly...so here is my log entry from my ws:

[2016/Dec/13|12:49:57] Tiled texture cache set to 4000 MB
[2016/Dec/13|12:49:58] Loading V-Ray plugins...
[2016/Dec/13|12:49:58] Translating scene for V-Ray...
[2016/Dec/13|12:49:58] Building scene command queue...
[2016/Dec/13|12:49:58] [VRayRTMax::renderThreadProc] Thread procedure entered
[2016/Dec/13|12:49:58] [VRayRTMax::renderThreadProc] sendClose=0
[2016/Dec/13|12:49:58] [VRayRTMax::renderThreadProc] Calling beginSequence()
[2016/Dec/13|12:49:58] Preparing renderer...
[2016/Dec/13|12:49:58] Preparing scene for rendering...
[2016/Dec/13|12:49:58] 10 % completed.
[2016/Dec/13|12:49:58] 20 % completed.
[2016/Dec/13|12:49:58] 31 % completed.
[2016/Dec/13|12:49:58] 40 % completed.
[2016/Dec/13|12:49:58] 50 % completed.
[2016/Dec/13|12:49:58] 61 % completed.
[2016/Dec/13|12:49:58] [RenderView] startCameraTime=0.000000, endCameraTime=2.000000

[2016/Dec/13|12:49:58] [RenderView] numCameraTMs=4, numFrames=2, frameSamples=2

[2016/Dec/13|12:49:58] 70 % completed.
[2016/Dec/13|12:49:58] 80 % completed.
[2016/Dec/13|12:49:58] 91 % completed.
[2016/Dec/13|12:49:58] 1 lights found.
[2016/Dec/13|12:49:58] Using probabilistic light sampling with 16 lights.
[2016/Dec/13|12:49:58] CUDA renderer requested.
[2016/Dec/13|12:49:58] CUDA renderer plugin already loaded.
[2016/Dec/13|12:49:58] CUDA renderer plugin instance successfully created.
[2016/Dec/13|12:49:58] EXT_RTOPENCL interface obtained successfully from CUDA renderer plugin instance.
[2016/Dec/13|12:49:58] Initializing built-in VFB.
[2016/Dec/13|12:49:58] [RenderView] startCameraTime=0.000000, endCameraTime=2.000000

[2016/Dec/13|12:49:58] [RenderView] numCameraTMs=4, numFrames=2, frameSamples=2

[2016/Dec/13|12:49:58] Starting DR 3
[2016/Dec/13|12:49:58] dr host: 192.168.0.11:20206
[2016/Dec/13|12:49:58] dr host: 192.168.0.12:20206
[2016/Dec/13|12:49:58] dr host: 192.168.0.13:20206
[2016/Dec/13|12:49:58] Connected to render host 192.168.0.11:20206
[2016/Dec/13|12:49:58] Connected to render host 192.168.0.12:20206
[2016/Dec/13|12:49:58] Connected to render host 192.168.0.13:20206
[2016/Dec/13|12:49:58] Using 3 hosts for distributed rendering.
[2016/Dec/13|12:49:58] Max ray intensity is enabled: rendered result may have incorrect brightness.
[2016/Dec/13|12:49:58] [VRayRTMax::renderThreadProc] Calling beginFrame()
[2016/Dec/13|12:49:58] ========================

[2016/Dec/13|12:49:58] Starting frame 0.

[2016/Dec/13|12:49:58] Preparing camera sampler.
[2016/Dec/13|12:49:58] Preparing scene for frame...
[2016/Dec/13|12:49:58] Dome light resolution increased from 512 to 2048 for light "VRayDome001@light_13"
[2016/Dec/13|12:49:58] Setting up 7 thread(s)
[2016/Dec/13|12:49:58] Threads completed
[2016/Dec/13|12:49:58] 10 % completed.
[2016/Dec/13|12:49:58] 20 % completed.
[2016/Dec/13|12:49:58] 30 % completed.
[2016/Dec/13|12:49:58] 41 % completed.
[2016/Dec/13|12:49:58] 50 % completed.
[2016/Dec/13|12:49:58] 60 % completed.
[2016/Dec/13|12:49:58] 70 % completed.
[2016/Dec/13|12:49:58] 80 % completed.
[2016/Dec/13|12:49:58] 91 % completed.
[2016/Dec/13|12:49:58] Compiling geometry...
[2016/Dec/13|12:49:58] 10 % completed.
[2016/Dec/13|12:49:58] 20 % completed.
[2016/Dec/13|12:49:58] 30 % completed.
[2016/Dec/13|12:49:58] 41 % completed.
[2016/Dec/13|12:49:58] 50 % completed.
[2016/Dec/13|12:49:58] 60 % completed.
[2016/Dec/13|12:49:58] 70 % completed.
[2016/Dec/13|12:49:58] 80 % completed.
[2016/Dec/13|12:49:58] 91 % completed.
[2016/Dec/13|12:49:58] Preparing ray server.
[2016/Dec/13|12:49:58] Using embree ray tracing.
[2016/Dec/13|12:49:58] Creating and initializing 7 thread(s).
[2016/Dec/13|12:49:58] Allocating memory for build data of 128749 faces (1029992 bytes, 1.0 MB).
[2016/Dec/13|12:49:58] Initializing face build data.
[2016/Dec/13|12:49:58] Creating 'done' event.
[2016/Dec/13|12:49:58] Starting first thread.
[2016/Dec/13|12:49:58] Waiting for thread completion.
[2016/Dec/13|12:49:58] Releasing thread resources.
[2016/Dec/13|12:49:58] SDTree statistics:
[2016/Dec/13|12:49:58] Total number of faces stored: 128749
[2016/Dec/13|12:49:58] Max tree depth: 35
[2016/Dec/13|12:49:58] Average tree depth: 22.8587
[2016/Dec/13|12:49:58] Number of tree nodes: 76801
[2016/Dec/13|12:49:58] Number of tree faces: 484369
[2016/Dec/13|12:49:58] Number of tree leafs: 34461
[2016/Dec/13|12:49:58] Average faces/leaf: 14.0556
[2016/Dec/13|12:49:58] Memory usage: 14.23 MB
[2016/Dec/13|12:49:58] Building static SD trees took 52 milliseconds
[2016/Dec/13|12:49:58] Full rayserver build. Took 53.034 ms.
[2016/Dec/13|12:49:58] buildFindAccelStructures. Took 7.347 ms.
[2016/Dec/13|12:49:58] Scene bounding box is [-98.4252,-98.4252,0]-[98.4252,98.4252,3.2767]
[2016/Dec/13|12:49:58] Preparing direct light manager.
[2016/Dec/13|12:49:58] 5 render channels in sequence data
[2016/Dec/13|12:49:58] Preparing global light manager.
[2016/Dec/13|12:49:58] Irradiance sample size is 84 bytes
[2016/Dec/13|12:49:58] Cannot use fast GI because secondary engine is not brute force or none
[2016/Dec/13|12:49:58] Waiting for image data...
[2016/Dec/13|12:49:58] [VRayRTMax::renderThreadProc] Calling renderImage()
[2016/Dec/13|12:49:58] Running RTEngine
[2016/Dec/13|12:49:58] Initializing CUDA renderer (single kernel version)...
[2016/Dec/13|12:49:59] Initializing environment kernel
[2016/Dec/13|12:49:59] Number of CUDA devices: 2
[2016/Dec/13|12:49:59] nVidia driver version: 376.19

continued on next reply

[biochemical_animations]

[2016/Dec/13|12:49:59] Transferring SD tree to CUDA devices...
[2016/Dec/13|12:49:59] SD tree transferred to device 0
[2016/Dec/13|12:49:59] SD tree transferred to device 1
[2016/Dec/13|12:49:59] Transferring Emb SD tree to CUDA devices...
[2016/Dec/13|12:49:59] Embree SD tree transferred to device 0
[2016/Dec/13|12:49:59] Embree SD tree transferred to device 1
[2016/Dec/13|12:49:59] Transferring MB tree to CUDA devices...
[2016/Dec/13|12:49:59] numNodeFaces=0
[2016/Dec/13|12:49:59] numNodes=0
[2016/Dec/13|12:49:59] numFaces=0
[2016/Dec/13|12:49:59] Successfuly transferred MB tree to CUDA devices...
[2016/Dec/13|12:49:59] Transferring Emb SD tree to CUDA devices...
[2016/Dec/13|12:49:59] Embree SD tree transferred to device 0
[2016/Dec/13|12:49:59] Embree SD tree transferred to device 1
[2016/Dec/13|12:49:59] Transferring geometric data
[2016/Dec/13|12:49:59] RTRayserver::getOCLGeomData. Took 7.356 ms.
[2016/Dec/13|12:49:59] Transferring vertex normals
[2016/Dec/13|12:49:59] 144357 vertex normals transferred to device 0
[2016/Dec/13|12:49:59] Transferring MOVING vertex normals
[2016/Dec/13|12:49:59] 0 MOVING vertex normals transferred to device 0
[2016/Dec/13|12:49:59] Transferring nodes and meshes
[2016/Dec/13|12:49:59] Nodes and meshes transferred to device 0
[2016/Dec/13|12:49:59] Transferring MOVING nodes, meshes and perface node indices
[2016/Dec/13|12:49:59] Successfuly transferred MOVING nodes and meshes to device 0
[2016/Dec/13|12:49:59] Transferring vertex normals
[2016/Dec/13|12:49:59] 144357 vertex normals transferred to device 1
[2016/Dec/13|12:49:59] Transferring MOVING vertex normals
[2016/Dec/13|12:49:59] 0 MOVING vertex normals transferred to device 1
[2016/Dec/13|12:49:59] Transferring nodes and meshes
[2016/Dec/13|12:49:59] Nodes and meshes transferred to device 1
[2016/Dec/13|12:49:59] Transferring MOVING nodes, meshes and perface node indices
[2016/Dec/13|12:49:59] Successfuly transferred MOVING nodes and meshes to device 1
[2016/Dec/13|12:49:59] Transferring map channels
[2016/Dec/13|12:49:59] Map channels transferred
[2016/Dec/13|12:49:59] Transferring moving map channels
[2016/Dec/13|12:49:59] Moving Map channels transferred
[2016/Dec/13|12:49:59] Transferring moving triangles
[2016/Dec/13|12:49:59] Moving triangles count = 0
[2016/Dec/13|12:49:59] Transfering geometry data to devices. Took 20.486 ms.
[2016/Dec/13|12:49:59] Transfering geometry trees to devices. Took 63.723 ms.
[2016/Dec/13|12:49:59] Building dynamic mesh trees
[2016/Dec/13|12:49:59] Setting up 7 thread(s)
[2016/Dec/13|12:49:59] Threads completed
[2016/Dec/13|12:49:59] BVH statistics:
[2016/Dec/13|12:49:59] Primitives count: 0 (Mem usage:0.00 MB)
[2016/Dec/13|12:49:59] Meshes count: 0 (Mem usage:0.00 MB)
[2016/Dec/13|12:49:59] Tree depth: 0
[2016/Dec/13|12:49:59] Number of tree nodes: 1
[2016/Dec/13|12:49:59] Tree takes 0.50 MB. Build time 0.04 ms.
[2016/Dec/13|12:49:59] Texture "Map #1459281765@tex_mono_0" is not supported and will be baked.
[2016/Dec/13|12:49:59] Transferring uvwBases channels for geom type 0
[2016/Dec/13|12:49:59] Transferring uvwBases channels for geom type 1
[2016/Dec/13|12:49:59] Transferring uvwBases channels for geom type 2
[2016/Dec/13|12:49:59] Number of lights: 1
[2016/Dec/13|12:49:59] Scene contains 1 lights, distributed in 1 groups. Avg lights per group is 1
[2016/Dec/13|12:49:59] 0 point lights transferred to device 0
[2016/Dec/13|12:49:59] 0 point lights transferred to device 1
[2016/Dec/13|12:49:59] Number of lights: 1
[2016/Dec/13|12:49:59] Total number of lights added by updateLights(): 0
[2016/Dec/13|12:49:59] Total memory allocated with oclbufman: 85047400
[2016/Dec/13|12:49:59] Lightmap successfuly loaded.
[2016/Dec/13|12:49:59] GPU light cache: 0 samples stored.
[2016/Dec/13|12:49:59] GPU light cache: 0 nodes stored.
[2016/Dec/13|12:49:59] GPU light cache: 0 leaves stored.
[2016/Dec/13|12:49:59] Mtls 7.79MB, Geo 14.09MB, Lights 80.09MB, LightCache 0.00MB, Other 64.75MB (data index 0)
[2016/Dec/13|12:49:59] Setting up 10 thread(s)
[2016/Dec/13|12:49:59] Rendering image...
[2016/Dec/13|12:49:59] Clearing ray server.
[2016/Dec/13|12:49:59] Clearing geometry.
[2016/Dec/13|12:49:59] Number of light evaluations: 0 (0.00 per pixel)
[2016/Dec/13|12:49:59] Preparing ray server.
[2016/Dec/13|12:49:59] Creating and initializing 7 thread(s).
[2016/Dec/13|12:49:59] Allocating memory for build data of 128749 faces (1029992 bytes, 1.0 MB).
[2016/Dec/13|12:49:59] Initializing face build data.
[2016/Dec/13|12:49:59] Creating 'done' event.
[2016/Dec/13|12:49:59] Starting first thread.
[2016/Dec/13|12:49:59] Waiting for thread completion.
[2016/Dec/13|12:49:59] Releasing thread resources.
[2016/Dec/13|12:49:59] SDTree statistics:
[2016/Dec/13|12:49:59] Total number of faces stored: 128749
[2016/Dec/13|12:49:59] Max tree depth: 35
[2016/Dec/13|12:49:59] Average tree depth: 22.8587
[2016/Dec/13|12:49:59] Number of tree nodes: 76701
[2016/Dec/13|12:49:59] Number of tree faces: 484369
[2016/Dec/13|12:49:59] Number of tree leafs: 34461
[2016/Dec/13|12:49:59] Average faces/leaf: 14.0556
[2016/Dec/13|12:49:59] Memory usage: 14.23 MB
[2016/Dec/13|12:49:59] Building static SD trees took 51 milliseconds
[2016/Dec/13|12:49:59] buildFindAccelStructures. Took 7.932 ms.
[2016/Dec/13|12:49:59] Partial rayserver rebuild. Took 60.506 ms.
[2016/Dec/13|12:49:59] Scene bounding box is [-98.4252,-98.4252,0]-[98.4252,98.4252,3.2767]
[2016/Dec/13|12:49:59] 1 lights found.
[2016/Dec/13|12:49:59] Using probabilistic light sampling with 16 lights.
[2016/Dec/13|12:49:59] Initializing built-in VFB.
[2016/Dec/13|12:49:59] Transferring SD tree to CUDA devices...
[2016/Dec/13|12:49:59] SD tree transferred to device 0
[2016/Dec/13|12:49:59] SD tree transferred to device 1
[2016/Dec/13|12:49:59] Transferring Emb SD tree to CUDA devices...
[2016/Dec/13|12:49:59] Embree SD tree transferred to device 0
[2016/Dec/13|12:49:59] Embree SD tree transferred to device 1
[2016/Dec/13|12:49:59] Transferring MB tree to CUDA devices...
[2016/Dec/13|12:49:59] numNodeFaces=0
[2016/Dec/13|12:49:59] numNodes=0
[2016/Dec/13|12:49:59] numFaces=0
[2016/Dec/13|12:49:59] Successfuly transferred MB tree to CUDA devices...
[2016/Dec/13|12:49:59] Transferring Emb SD tree to CUDA devices...
[2016/Dec/13|12:49:59] Embree SD tree transferred to device 0
[2016/Dec/13|12:49:59] Embree SD tree transferred to device 1
[2016/Dec/13|12:49:59] Transferring geometric data
[2016/Dec/13|12:49:59] RTRayserver::getOCLGeomData. Took 7.262 ms.
[2016/Dec/13|12:49:59] Transferring vertex normals
[2016/Dec/13|12:49:59] 144357 vertex normals transferred to device 0
[2016/Dec/13|12:49:59] Transferring MOVING vertex normals
[2016/Dec/13|12:49:59] 0 MOVING vertex normals transferred to device 0
[2016/Dec/13|12:49:59] Transferring nodes and meshes
[2016/Dec/13|12:49:59] Nodes and meshes transferred to device 0
[2016/Dec/13|12:49:59] Transferring MOVING nodes, meshes and perface node indices
[2016/Dec/13|12:49:59] Successfuly transferred MOVING nodes and meshes to device 0
[2016/Dec/13|12:49:59] Transferring vertex normals
[2016/Dec/13|12:49:59] 144357 vertex normals transferred to device 1
[2016/Dec/13|12:49:59] Transferring MOVING vertex normals
[2016/Dec/13|12:49:59] 0 MOVING vertex normals transferred to device 1
[2016/Dec/13|12:49:59] Transferring nodes and meshes
[2016/Dec/13|12:49:59] Nodes and meshes transferred to device 1
[2016/Dec/13|12:49:59] Transferring MOVING nodes, meshes and perface node indices
[2016/Dec/13|12:49:59] Successfuly transferred MOVING nodes and meshes to device 1
[2016/Dec/13|12:49:59] Transferring map channels
[2016/Dec/13|12:49:59] Map channels transferred
[2016/Dec/13|12:49:59] Transferring moving map channels
[2016/Dec/13|12:49:59] Moving Map channels transferred
[2016/Dec/13|12:49:59] Transferring moving triangles
[2016/Dec/13|12:49:59] Moving triangles count = 0
[2016/Dec/13|12:49:59] Transfering geometry data to devices. Took 14.429 ms.
[2016/Dec/13|12:49:59] Transfering geometry trees to devices. Took 59.496 ms.
[2016/Dec/13|12:49:59] Number of lights: 1
[2016/Dec/13|12:49:59] Scene contains 1 lights, distributed in 1 groups. Avg lights per group is 1
[2016/Dec/13|12:49:59] 0 point lights transferred to device 0
[2016/Dec/13|12:49:59] 0 point lights transferred to device 1
[2016/Dec/13|12:49:59] Number of lights: 1
[2016/Dec/13|12:49:59] Total number of lights added by updateLights(): 0
[2016/Dec/13|12:49:59] Total memory allocated with oclbufman: 85047400
[2016/Dec/13|12:49:59] Transfer updated data to devices. Took 0.000 ms.
[2016/Dec/13|12:49:59] Texture "Map #1459281765@tex_mono_0" is not supported and will be baked.
[2016/Dec/13|12:49:59] Transferring uvwBases channels for geom type 0
[2016/Dec/13|12:49:59] Transferring uvwBases channels for geom type 1
[2016/Dec/13|12:49:59] Transferring uvwBases channels for geom type 2
[2016/Dec/13|12:49:59] Photon size is 56 bytes.
[2016/Dec/13|12:49:59] Light cache sample size is 100 bytes.
[2016/Dec/13|12:49:59] Building light cache.
[2016/Dec/13|12:49:59] Building light cache...
[2016/Dec/13|12:49:59] Tracing 1000000 image samples for light cache in 64 passes.

continued on next reply

[biochemical_animations]

[2016/Dec/13|12:49:59] Using 35.16 MB for light cache preview buffer
[2016/Dec/13|12:49:59] Setting up 7 thread(s)
[2016/Dec/13|12:50:02] 10 % completed.
[2016/Dec/13|12:50:05] 20 % completed.
[2016/Dec/13|12:50:08] 30 % completed.
[2016/Dec/13|12:50:11] 40 % completed.
[2016/Dec/13|12:50:14] 50 % completed.
[2016/Dec/13|12:50:16] 60 % completed.
[2016/Dec/13|12:50:19] 70 % completed.
[2016/Dec/13|12:50:22] 80 % completed.
[2016/Dec/13|12:50:25] 90 % completed.
[2016/Dec/13|12:50:29] Threads completed
[2016/Dec/13|12:50:29] Merging light cache passes.
[2016/Dec/13|12:50:29] Merging light cache passes...
[2016/Dec/13|12:50:29] Setting up 7 thread(s)
[2016/Dec/13|12:50:29] 76 % completed.
[2016/Dec/13|12:50:29] Threads completed
[2016/Dec/13|12:50:31] Light cache contains 188015 samples.
[2016/Dec/13|12:50:31] Light cache samples collected; compiling lookup tree.
[2016/Dec/13|12:50:31] Light cache takes 32.2 MB.
[2016/Dec/13|12:50:31] Prefiltering light cache.
[2016/Dec/13|12:50:31] Prefiltering light cache...
[2016/Dec/13|12:50:31] 10 % completed.
[2016/Dec/13|12:50:31] 20 % completed.
[2016/Dec/13|12:50:31] 30 % completed.
[2016/Dec/13|12:50:31] 40 % completed.
[2016/Dec/13|12:50:31] 50 % completed.
[2016/Dec/13|12:50:31] 60 % completed.
[2016/Dec/13|12:50:31] 70 % completed.
[2016/Dec/13|12:50:31] 80 % completed.
[2016/Dec/13|12:50:31] 90 % completed.
[2016/Dec/13|12:50:31] Average rays per light cache sample: 22.72 (min 1, max 1653)
[2016/Dec/13|12:50:31] Using premultiplied light cache.
[2016/Dec/13|12:50:31] Lightmap successfuly loaded.
[2016/Dec/13|12:50:31] GPU light cache: 188015 samples stored.
[2016/Dec/13|12:50:31] GPU light cache: 33125 nodes stored.
[2016/Dec/13|12:50:31] GPU light cache: 33126 leaves stored.
[2016/Dec/13|12:50:31] Mtls 7.79MB, Geo 14.09MB, Lights 80.09MB, LightCache 13.24MB, Other 66.44MB (data index 3910)
[2016/Dec/13|12:50:31] End applyChangesToScene(). Number of changes 631. Took 31825.248 ms. Change processing 0.138 ms
[2016/Dec/13|12:50:31] Apply changes: Finished processing changes (size: 10200 kbytes; packets: 6, thread: 9)
[2016/Dec/13|12:50:31] Building optimized CUDA kernel (0) for comp cap 5.2 for device 0...
[2016/Dec/13|12:50:31] PTX file generated with CUDA Toolkit v7.5 for CUDA compute capability 2.0. CUDA runtime version 7050.
[2016/Dec/13|12:50:31] Optimized CUDA kernel assembled successfully
[2016/Dec/13|12:50:31] Total memory for device 0 : 6144 MB, free 4582 MB
[2016/Dec/13|12:50:31] Building optimized CUDA kernel (0) for comp cap 5.2 for device 1...
[2016/Dec/13|12:50:31] PTX file generated with CUDA Toolkit v7.5 for CUDA compute capability 2.0. CUDA runtime version 7050.
[2016/Dec/13|12:50:31] Progressive sampling base buffer memory usage: 123.20 MB
[2016/Dec/13|12:50:32] Optimized CUDA kernel assembled successfully
[2016/Dec/13|12:50:32] Total memory for device 1 : 6144 MB, free 4394 MB
[2016/Dec/13|12:50:32] Transfer updated data to devices. Took 0.001 ms.
[2016/Dec/13|12:50:32] Photon size is 56 bytes.
[2016/Dec/13|12:50:32] Light cache sample size is 100 bytes.
[2016/Dec/13|12:50:32] Building light cache.
[2016/Dec/13|12:50:32] Building light cache...
[2016/Dec/13|12:50:32] Tracing 1000000 image samples for light cache in 64 passes.

continued on next reply

[biochemical_animations]

[2016/Dec/13|12:50:32] Using 35.16 MB for light cache preview buffer
[2016/Dec/13|12:50:32] Setting up 7 thread(s)
[2016/Dec/13|12:50:35] 10 % completed.
[2016/Dec/13|12:50:38] 20 % completed.
[2016/Dec/13|12:50:41] 30 % completed.
[2016/Dec/13|12:50:44] 40 % completed.
[2016/Dec/13|12:50:47] 50 % completed.
[2016/Dec/13|12:50:49] 60 % completed.
[2016/Dec/13|12:50:52] 70 % completed.
[2016/Dec/13|12:50:55] 80 % completed.
[2016/Dec/13|12:50:58] 90 % completed.
[2016/Dec/13|12:51:00] 10200 % completed.
[2016/Dec/13|12:51:00] 10256 % completed.
[2016/Dec/13|12:51:00] 10311 % completed.
[2016/Dec/13|12:51:00] 10379 % completed.
[2016/Dec/13|12:51:00] 8948 % completed.
[2016/Dec/13|12:51:01] 7513 % completed.
[2016/Dec/13|12:51:01] 3049 % completed.
[2016/Dec/13|12:51:01] 1561 % completed.
[2016/Dec/13|12:51:01] 1578 % completed.
[2016/Dec/13|12:51:01] 1592 % completed.
[2016/Dec/13|12:51:02] 1607 % completed.
[2016/Dec/13|12:51:02] 1621 % completed.
[2016/Dec/13|12:51:02] 1636 % completed.
[2016/Dec/13|12:51:02] 1650 % completed.
[2016/Dec/13|12:51:02] 1665 % completed.
[2016/Dec/13|12:51:02] Threads completed
[2016/Dec/13|12:51:02] Merging light cache passes.
[2016/Dec/13|12:51:02] Merging light cache passes...
[2016/Dec/13|12:51:02] Setting up 7 thread(s)
[2016/Dec/13|12:51:03] Threads completed
[2016/Dec/13|12:51:03] Noise threshold lowered to 0.166667 with 46.6 percent active pixels.
[2016/Dec/13|12:51:04] Light cache contains 187976 samples.
[2016/Dec/13|12:51:04] Light cache samples collected; compiling lookup tree.
[2016/Dec/13|12:51:04] Light cache takes 32.2 MB.
[2016/Dec/13|12:51:04] Prefiltering light cache.
[2016/Dec/13|12:51:04] Prefiltering light cache...
[2016/Dec/13|12:51:04] 13 % completed.
[2016/Dec/13|12:51:04] 20 % completed.
[2016/Dec/13|12:51:04] 30 % completed.
[2016/Dec/13|12:51:04] 40 % completed.
[2016/Dec/13|12:51:04] 50 % completed.
[2016/Dec/13|12:51:04] 60 % completed.
[2016/Dec/13|12:51:04] 70 % completed.
[2016/Dec/13|12:51:04] 80 % completed.
[2016/Dec/13|12:51:04] 90 % completed.
[2016/Dec/13|12:51:04] 100 % completed.
[2016/Dec/13|12:51:04] 110 % completed.
[2016/Dec/13|12:51:04] 120 % completed.
[2016/Dec/13|12:51:04] 130 % completed.
[2016/Dec/13|12:51:04] 140 % completed.
[2016/Dec/13|12:51:04] 150 % completed.
[2016/Dec/13|12:51:04] 160 % completed.
[2016/Dec/13|12:51:04] 170 % completed.
[2016/Dec/13|12:51:04] 180 % completed.
[2016/Dec/13|12:51:04] 190 % completed.
[2016/Dec/13|12:51:04] 200 % completed.
[2016/Dec/13|12:51:04] 210 % completed.
[2016/Dec/13|12:51:04] 220 % completed.
[2016/Dec/13|12:51:04] 230 % completed.
[2016/Dec/13|12:51:04] 240 % completed.
[2016/Dec/13|12:51:04] 250 % completed.
[2016/Dec/13|12:51:04] 260 % completed.
[2016/Dec/13|12:51:04] 270 % completed.
[2016/Dec/13|12:51:04] 280 % completed.
[2016/Dec/13|12:51:04] 290 % completed.
[2016/Dec/13|12:51:04] 300 % completed.
[2016/Dec/13|12:51:04] 310 % completed.
[2016/Dec/13|12:51:04] 320 % completed.
[2016/Dec/13|12:51:04] 330 % completed.
[2016/Dec/13|12:51:04] 340 % completed.
[2016/Dec/13|12:51:04] 350 % completed.
[2016/Dec/13|12:51:04] 360 % completed.
[2016/Dec/13|12:51:04] 370 % completed.
[2016/Dec/13|12:51:04] 380 % completed.
[2016/Dec/13|12:51:04] 390 % completed.
[2016/Dec/13|12:51:04] 400 % completed.
[2016/Dec/13|12:51:04] 410 % completed.
[2016/Dec/13|12:51:04] 420 % completed.
[2016/Dec/13|12:51:04] 430 % completed.
[2016/Dec/13|12:51:04] 440 % completed.
[2016/Dec/13|12:51:04] 450 % completed.
[2016/Dec/13|12:51:04] 460 % completed.
[2016/Dec/13|12:51:04] 470 % completed.
[2016/Dec/13|12:51:04] 480 % completed.
[2016/Dec/13|12:51:04] 490 % completed.
[2016/Dec/13|12:51:04] 500 % completed.
[2016/Dec/13|12:51:04] 510 % completed.
[2016/Dec/13|12:51:04] 520 % completed.
[2016/Dec/13|12:51:04] 530 % completed.
[2016/Dec/13|12:51:04] 540 % completed.
[2016/Dec/13|12:51:04] 550 % completed.
[2016/Dec/13|12:51:04] 560 % completed.
[2016/Dec/13|12:51:04] 570 % completed.
[2016/Dec/13|12:51:04] 580 % completed.
[2016/Dec/13|12:51:04] 590 % completed.
[2016/Dec/13|12:51:04] 600 % completed.
[2016/Dec/13|12:51:04] 610 % completed.
[2016/Dec/13|12:51:04] 620 % completed.
[2016/Dec/13|12:51:04] 630 % completed.
[2016/Dec/13|12:51:04] 640 % completed.
[2016/Dec/13|12:51:04] 650 % completed.
[2016/Dec/13|12:51:04] 660 % completed.
[2016/Dec/13|12:51:04] 670 % completed.
[2016/Dec/13|12:51:04] 680 % completed.
[2016/Dec/13|12:51:04] 690 % completed.
[2016/Dec/13|12:51:04] 700 % completed.
[2016/Dec/13|12:51:04] 710 % completed.
[2016/Dec/13|12:51:04] 720 % completed.
[2016/Dec/13|12:51:04] 730 % completed.
[2016/Dec/13|12:51:04] 740 % completed.
[2016/Dec/13|12:51:04] 750 % completed.
[2016/Dec/13|12:51:04] 760 % completed.
[2016/Dec/13|12:51:04] 770 % completed.
[2016/Dec/13|12:51:04] 780 % completed.
[2016/Dec/13|12:51:04] 790 % completed.
[2016/Dec/13|12:51:04] 800 % completed.
[2016/Dec/13|12:51:04] 810 % completed.
[2016/Dec/13|12:51:04] 820 % completed.
[2016/Dec/13|12:51:04] 830 % completed.
[2016/Dec/13|12:51:04] 840 % completed.
[2016/Dec/13|12:51:04] 850 % completed.
[2016/Dec/13|12:51:04] 860 % completed.
[2016/Dec/13|12:51:04] 870 % completed.
[2016/Dec/13|12:51:04] 880 % completed.
[2016/Dec/13|12:51:04] 890 % completed.
[2016/Dec/13|12:51:04] 900 % completed.
[2016/Dec/13|12:51:04] 910 % completed.
[2016/Dec/13|12:51:04] 920 % completed.
[2016/Dec/13|12:51:04] 930 % completed.
[2016/Dec/13|12:51:04] 940 % completed.
[2016/Dec/13|12:51:04] 950 % completed.
[2016/Dec/13|12:51:04] 960 % completed.
[2016/Dec/13|12:51:04] 970 % completed.
[2016/Dec/13|12:51:04] 980 % completed.
[2016/Dec/13|12:51:04] 990 % completed.
[2016/Dec/13|12:51:04] 1000 % completed.
[2016/Dec/13|12:51:04] 1010 % completed.
[2016/Dec/13|12:51:04] 1020 % completed.
[2016/Dec/13|12:51:04] 1030 % completed.
[2016/Dec/13|12:51:04] 1040 % completed.
[2016/Dec/13|12:51:04] 1050 % completed.
[2016/Dec/13|12:51:04] 1060 % completed.
[2016/Dec/13|12:51:04] 1070 % completed.
[2016/Dec/13|12:51:04] 1080 % completed.
[2016/Dec/13|12:51:04] 1090 % completed.
[2016/Dec/13|12:51:04] 1100 % completed.
[2016/Dec/13|12:51:04] 1110 % completed.
[2016/Dec/13|12:51:04] 1120 % completed.
[2016/Dec/13|12:51:04] 1130 % completed.
[2016/Dec/13|12:51:04] 1140 % completed.
[2016/Dec/13|12:51:04] 1150 % completed.
[2016/Dec/13|12:51:04] 1160 % completed.
[2016/Dec/13|12:51:04] 1170 % completed.
[2016/Dec/13|12:51:04] 1180 % completed.
[2016/Dec/13|12:51:04] 1190 % completed.
[2016/Dec/13|12:51:04] 1200 % completed.
[2016/Dec/13|12:51:04] 1210 % completed.
[2016/Dec/13|12:51:04] 1220 % completed.
[2016/Dec/13|12:51:04] 1230 % completed.
[2016/Dec/13|12:51:04] 1240 % completed.
[2016/Dec/13|12:51:04] 1250 % completed.
[2016/Dec/13|12:51:04] 1260 % completed.
[2016/Dec/13|12:51:04] 1270 % completed.
[2016/Dec/13|12:51:04] 1280 % completed.
[2016/Dec/13|12:51:04] 1290 % completed.
[2016/Dec/13|12:51:04] 1300 % completed.
[2016/Dec/13|12:51:04] 1310 % completed.
[2016/Dec/13|12:51:04] 1320 % completed.
[2016/Dec/13|12:51:04] 1330 % completed.
[2016/Dec/13|12:51:04] 1340 % completed.
[2016/Dec/13|12:51:04] 1350 % completed.
[2016/Dec/13|12:51:04] 1360 % completed.
[2016/Dec/13|12:51:04] 1370 % completed.
[2016/Dec/13|12:51:04] 1380 % completed.
[2016/Dec/13|12:51:04] 1390 % completed.
[2016/Dec/13|12:51:04] 1400 % completed.
[2016/Dec/13|12:51:04] 1410 % completed.
[2016/Dec/13|12:51:04] 1420 % completed.
[2016/Dec/13|12:51:04] 1430 % completed.
[2016/Dec/13|12:51:04] 1440 % completed.
[2016/Dec/13|12:51:04] 1450 % completed.
[2016/Dec/13|12:51:04] 1460 % completed.
[2016/Dec/13|12:51:04] 1470 % completed.
[2016/Dec/13|12:51:04] 1480 % completed.
[2016/Dec/13|12:51:04] 1490 % completed.
[2016/Dec/13|12:51:04] 1500 % completed.
[2016/Dec/13|12:51:04] 1510 % completed.
[2016/Dec/13|12:51:04] 1520 % completed.
[2016/Dec/13|12:51:04] 1530 % completed.
[2016/Dec/13|12:51:04] 1540 % completed.
[2016/Dec/13|12:51:04] 1550 % completed.
[2016/Dec/13|12:51:04] 1560 % completed.
[2016/Dec/13|12:51:04] 1570 % completed.
[2016/Dec/13|12:51:04] 1580 % completed.
[2016/Dec/13|12:51:04] 1590 % completed.
[2016/Dec/13|12:51:04] 1600 % completed.
[2016/Dec/13|12:51:04] 1610 % completed.
[2016/Dec/13|12:51:04] 1620 % completed.
[2016/Dec/13|12:51:04] 1630 % completed.
[2016/Dec/13|12:51:04] 1640 % completed.
[2016/Dec/13|12:51:04] 1650 % completed.
[2016/Dec/13|12:51:04] 1660 % completed.
[2016/Dec/13|12:51:04] 1662 % completed.
[2016/Dec/13|12:51:04] 1670 % completed.
[2016/Dec/13|12:51:04] 1680 % completed.
[2016/Dec/13|12:51:04] 1690 % completed.
[2016/Dec/13|12:51:04] 1700 % completed.
[2016/Dec/13|12:51:04] 1710 % completed.
[2016/Dec/13|12:51:04] 1720 % completed.
[2016/Dec/13|12:51:04] 1730 % completed.
[2016/Dec/13|12:51:04] 1740 % completed.
[2016/Dec/13|12:51:04] 1750 % completed.
[2016/Dec/13|12:51:04] 1760 % completed.
[2016/Dec/13|12:51:04] 1770 % completed.
[2016/Dec/13|12:51:04] 1780 % completed.
[2016/Dec/13|12:51:04] 1790 % completed.
[2016/Dec/13|12:51:04] 1800 % completed.
[2016/Dec/13|12:51:04] 1810 % completed.
[2016/Dec/13|12:51:04] 1820 % completed.
[2016/Dec/13|12:51:04] 1830 % completed.
[2016/Dec/13|12:51:04] 1840 % completed.
[2016/Dec/13|12:51:04] 1850 % completed.
[2016/Dec/13|12:51:04] 1860 % completed.
[2016/Dec/13|12:51:04] 1870 % completed.

continued on next reply

[biochemical_animations]

[2016/Dec/13|12:51:04] Average rays per light cache sample: 22.72 (min 1, max 1653)
[2016/Dec/13|12:51:04] Using premultiplied light cache.
[2016/Dec/13|12:51:05] Lightmap successfuly loaded.
[2016/Dec/13|12:51:05] GPU light cache: 187976 samples stored.
[2016/Dec/13|12:51:05] GPU light cache: 33147 nodes stored.
[2016/Dec/13|12:51:05] GPU light cache: 33148 leaves stored.
[2016/Dec/13|12:51:05] Mtls 7.79MB, Geo 14.09MB, Lights 80.09MB, LightCache 26.49MB, Other 66.44MB (data index 3914)
[2016/Dec/13|12:51:05] End applyChangesToScene(). Number of changes 4. Took 32496.068 ms. Change processing 0.026 ms
[2016/Dec/13|12:51:20] Noise threshold lowered to 0.125000 with 16.5 percent active pixels.
[2016/Dec/13|12:51:30] Noise threshold lowered to 0.083333 with 8.2 percent active pixels.
[2016/Dec/13|12:51:39] Final noise threshold set: 0.050000
[2016/Dec/13|12:52:02] 12 % completed.
[2016/Dec/13|12:52:31] 20 % completed.
[2016/Dec/13|12:52:51] 30 % completed.
[2016/Dec/13|12:52:53] Number of rays per pixel changed to 109, total number of samples is 40330.
[2016/Dec/13|12:52:53] 40 % completed.
[2016/Dec/13|12:52:53] Number of rays per pixel changed to 159, total number of samples is 40227.
[2016/Dec/13|12:52:53] Number of rays per pixel changed to 225, total number of samples is 40050.
[2016/Dec/13|12:52:53] Number of rays per pixel changed to 512, total number of samples is 33280.
[2016/Dec/13|12:52:53] 55 % completed.
[2016/Dec/13|12:52:53] Number of rays per pixel changed to 64, total number of samples is 0.
[2016/Dec/13|12:52:53] 87 % completed.
[2016/Dec/13|12:52:54] Threads completed
[2016/Dec/13|12:52:54] Device 0 statistics:
[2016/Dec/13|12:52:54] GPU time: 78.481975 %
[2016/Dec/13|12:52:54] GPU ray trace time 0.211990 %
[2016/Dec/13|12:52:54] Transfer to GPU time 0.004770 %
[2016/Dec/13|12:52:54] Transfer from GPU time 99.783240 %
[2016/Dec/13|12:52:54] Device 1 statistics:
[2016/Dec/13|12:52:54] GPU time: 78.026826 %
[2016/Dec/13|12:52:54] GPU ray trace time 27.337319 %
[2016/Dec/13|12:52:54] Transfer to GPU time 0.005971 %
[2016/Dec/13|12:52:54] Transfer from GPU time 72.656711 %
[2016/Dec/13|12:52:54] Closing RTEngine (cancelled)
[2016/Dec/13|12:52:54] Max. Mpaths per second: 59.695576
[2016/Dec/13|12:52:54] Sampling level reached: 8755 samples per pixel
[2016/Dec/13|12:52:54] RT Frame took: 173.63s
[2016/Dec/13|12:52:54] [VRayRTMax::renderThreadProc] Calling endFrame()
[2016/Dec/13|12:52:54] Waiting for DR to finish
[2016/Dec/13|12:52:54] Number of raycasts: 145292076 (157.65 per pixel)
[2016/Dec/13|12:52:54] Camera rays: 2000052 (2.17 per pixel)
[2016/Dec/13|12:52:54] Shadow rays: 9826032 (10.66 per pixel)
[2016/Dec/13|12:52:54] GI rays: 18389802 (19.95 per pixel)
[2016/Dec/13|12:52:54] Reflection rays: 7789232 (8.45 per pixel)
[2016/Dec/13|12:52:54] Refraction rays: 7422090 (8.05 per pixel)
[2016/Dec/13|12:52:54] Unshaded rays: 111622712 (121.12 per pixel)
[2016/Dec/13|12:52:54] Clearing global light manager.
[2016/Dec/13|12:52:54] Clearing direct light manager.
[2016/Dec/13|12:52:54] Number of light evaluations: 19634460 (21.30 per pixel)
[2016/Dec/13|12:52:54] Clearing ray server.
[2016/Dec/13|12:52:54] Clearing geometry.
[2016/Dec/13|12:52:54] Number of intersectable primitives: 128750
[2016/Dec/13|12:52:54] SD triangles: 128749
[2016/Dec/13|12:52:54] MB triangles: 0
[2016/Dec/13|12:52:54] Static primitives: 0
[2016/Dec/13|12:52:54] Moving primitives: 0
[2016/Dec/13|12:52:54] Infinite primitives: 1
[2016/Dec/13|12:52:54] Static hair segments: 0
[2016/Dec/13|12:52:54] Moving hair segments: 0
[2016/Dec/13|12:52:54] Clearing camera image sampler.
[2016/Dec/13|12:52:54] Clearing camera sampler.
[2016/Dec/13|12:52:54] Clearing DMC sampler.
[2016/Dec/13|12:52:54] Clearing path sampler.
[2016/Dec/13|12:52:54] Clearing color mapper.
[2016/Dec/13|12:52:54] [VRayRTMax::renderThreadProc] Calling endSequence()
[2016/Dec/13|12:52:54] Closing DR
[2016/Dec/13|12:52:54] Setting up 1 thread(s)
[2016/Dec/13|12:52:54] Threads completed
[2016/Dec/13|12:52:54] [VRayRTMax::renderThreadProc] Rendering failed
[2016/Dec/13|12:52:54] [VRayRTMax::renderThreadProc] Exiting thread procedure
[2016/Dec/13|12:52:54] 0 error(s), 0 warning(s)
[2016/Dec/13|12:52:54] ================================

continued on next reply

[biochemical_animations]

Do you see anything that doesn't make sense?

Here are my test on my own scene:

*****************************
Comparison bench 2.0 (From google docs gpu rt benchmarks)
2x GTX 980 Ti 05m 24s 44.33% speed reduction
2x GTX 970 OC 09m 42s
*****************************

Here is my scene results:

Plate14.max
1280x720
RT GPU DR noise 0.05 5 GPUs used, 2x980Ti on ws, evga 970 on sys12, asus 970 on sys11, 980Ti on sys13
2:56s

Plate14.max
1280x720
RT GPU DR noise 0.05 evga 970 only on system12
10:45s

Plate14.max [duplicate render to see if time was consistant, different by 25s]
1280x720
RT GPU noise 0.05 2 cards on ws (2 x 980 ti) + DR EVGA 970 on Sys12
3:35s

Plate14.max
1280x720
RT GPU noise 0.05 2 cards on ws (2 x 980 ti)
3:22s
building light cache 29.3s
prefilter lc was immediate

Plate14.max
1280x720
RT GPU noise 0.05 2 cards on ws (2 x 980 ti) + DR EVGA 970 on Sys12
4:03s

Plate14.max [duplicate render to see if time was consistant]
1280x720
RT GPU noise 0.05 2 cards on ws (2 x 980 ti)
3:29s

*****************************

********** GPUs (970s only tests before I added 3x 980 ti above) **************

Plate14.max
1280x720
RT GPU noise 0.05 2 cards on ws (evga 970 and asus 970)
5:31.3s

Plate12.max
1280x720
RT GPU noise 0.05 evga on ws and asus on system13
6:5.8s

Plate12.max
1280x720
RT GPU noise 0.05 evga only on ws
9:39.7s


*****************************

plate12 and plate14 were identical scenes, just saved incrementally

[biochemical_animations]

Here are my comparison times with the google doc
plate14
min sec total sec
2x980Ti 3 29 209 36.86% speed reduction
2x970 5 31 331

this difference could be caused by the cpu and memory/mb differences


bench 2.0
min sec total sec
2x980Ti 5 24 324 44.33% speed reduction
2x970 OC 9 42 582

[savage309]

Logs seems fine.
I am not sure if I got correct what these performance numbers are / what they mean.

Best,
Blago.

[biochemical_animations]

Plate14.max (File name)
1280x720 (resolution)
RT GPU DR noise 0.05 evga 970 only on system12 (vray rt was used in gpu mode with dr, noice set to 0.05, evga 970 on system 12 stands for 1 graphics card, evga is the manufacturer, 970 is the card on system12 the render node)
10:45s (this is the render time)