Distribution ​

If you don't want to run the Simplygon processing on your local machine you can use distribution to offload the work to another machine.

Simplygon support three different distribution methods: Simplygon Grid, Incredibuild and FASTBuild and you can use distribution in our integrations or directly in the Simplygon API.

Simplygon process distribution works using pipelines and pipeline batches. Any pipeline and pipeline batch can be distributed directly through the API by using the run mode parameter in the run scene API call. Distribution is handled by the SimplygonBatch tool in a separate sandboxed process.

Paths ​

By default the batch tool is located using the SIMPLYGON_10_PATH environment variable. If you wish to place the executable in a custom location, use the SetSimplygonBatchPath setting on the pipeline to the path where the batch tool executable is located.

License ​

If you have a node locked license you must ensure that each machine that wants to act as a worker node in distribution must have its own license installed, since the license is unique for each machine. Otherwise the license will be included in the work data and the target machine does not need to have the license pre-installed.

Simplygon Grid ​

The easiest way to get distribution of pipelines and pipeline batches running is to use the Simplygon Grid. There is no need to setup coordinators or network details, just run the grid agent executable on any number of machines on the local network and everything is setup for distribution.

Running a distribution agent ​

To start a grid agent run the SimplygonGridAgent.exe tool on the machine. There is no coordinator or network setup needed, the pipeline execution locates available processing nodes using mDNS/DNS-SD.

Check Grid capabilities ​

Grid agent availability is currently based on compatible version, vulkan capabilities and load. If you want to check the status of your grid agents, run SimplygonBatch.exe -Discover. That will list all your agents that are discoverable and also print information about each agent, here is an example output:

❯ & "C:\Program Files\Simplygon\10\SimplygonBatch.exe" -Discover
SimplygonBatch discover remote agents
...
[2023-09-08 10:38:03]    [INFO] Found 6 agents:
[2023-09-08 10:38:03]    [INFO] =========================================
[2023-09-08 10:38:03]    [INFO] 9.9.9.120:49157 [10.2.2802, load 0]
[2023-09-08 10:38:03]    [INFO] =========================================
[2023-09-08 10:38:03]    [INFO] 9.9.9.203:49157 [10.2.2802, load 10]
[2023-09-08 10:38:03]    [INFO]   * Agent (10.2.2802) is not running the same version as client (10.2.3502)
[2023-09-08 10:38:03] [WARNING]   * No Vulkan capabilities. Pipelines requiring Vulkan cannot be distributed to this agent.
[2023-09-08 10:38:03]    [INFO] =========================================
[2023-09-08 10:38:03]    [INFO] 9.9.9.248:49157 [10.2.2802, load 43]
[2023-09-08 10:38:03] [WARNING]   * No Vulkan capabilities. Pipelines requiring Vulkan cannot be distributed to this agent.

In the above example we can see that two of the agents are missing vulkan capabilites and won't pick up pipelines that requires vulkan. Also, one of the agent is running an older version and needs an upgrade in order to be availabe and pick up distributed jobs.

Bridging subnets ​

If your network is segmented and multicast UDP packets cannot traverse subnet boundaries you can bridge these subnets by allowing the batch tool to connect to a Grid agent on a different subnet over TCP/IP and routing discovery of hosts from that subnet back to the batch tool over the TCP/IP connection. This is done by setting the environment variable SIMPLYGON_<MajorVersion>_GRID_HOSTS to a semicolon separated list of machines (either IP address or FQDN) running Grid agents on different subnets. This way you do not need to have every machine in the list, only one on each subnet that can act as the bridge during discovery. The bridge connection utilizes TCP/IP port 55001 which must be open in any firewalls. The environment variable needs to be set only on the machine that is the distributor.

Replace <MajorVersion> with the major version of Simplygon, e.g. SIMPLYGON_10_GRID_HOSTS.

Running as a Windows service ​

The grid agent can be started as a Windows service. To install the service, run the grid agent executable with the -InstallService argument. To uninstall the service, run the grid agent executable with the -UninstallService argument. Errors will be posted to the Windows event log. Note that services are run in session 0 and cannot display any UI to the logged in user.

Log to file ​

If you want the grid agent log output as a file, add the -LogFile <path-to-file> argument to the command line.

Distributed pipeline execution ​

Distribution is enabled by passing the appropriate run mode to the pipeline run scene API

spReductionPipeline reductionPipeline = sg->CreateReductionPipeline();

// Process a scene from file directly using distribution
reductionPipeline->RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode::RunDistributedUsingSimplygonGrid );

// Or if you have a scene object, process it using distribution
reductionPipeline->RunScene( scene, EPipelineRunMode::RunDistributedUsingSimplygonGrid )

spReductionPipeline reductionPipeline = sg.CreateReductionPipeline();

// Process a scene from file directly using distribution
reductionPipeline.RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode.RunDistributedUsingSimplygonGrid );

// Or if you have a scene object, process it using distribution
reductionPipeline.RunScene( scene, EPipelineRunMode.RunDistributedUsingSimplygonGrid )

reductionPipeline = sg.CreateReductionPipeline()

# Process a scene from file directly using distribution
reductionPipeline.RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode_RunDistributedUsingSimplygonGrid )

# Or if you have a scene object, process it using distribution
reductionPipeline.RunScene( scene, EPipelineRunMode_RunDistributedUsingSimplygonGrid )

Distributed batch tool ​

Distribution can be enabled when invoking the batch tool (SimplygonBatch.exe) by passing the -Distribute parameter on the command line

SimplygonBatch.exe -Distribute <path/to/pipeline.json> <input.scene> <output.scene>
SimplygonBatch.exe -Distribute <path/to/pipelinebatch.json>

If you wish to integrate the batch tool in an asset processing tool and read progress for an UI, you can pass and additional parameters to the batch tool executable, -Progress. It will suppress the command line progress bar and instead print progress as a number between 0 and 100, one line at a time, to the standard output.

SimplygonBatch.exe -Distribute -Progress <path/to/pipeline.json> <input.scene> <output.scene>
SimplygonBatch.exe -Distribute -Progress <path/to/pipelinebatch.json>

Here is an example how to do batch processing using Simplygon Grid.

How it works ​

The pipeline execution usees the batch tool (SimplygonBatch.exe) to do the distribution. The batch tool find a suitable node using mDNS/DNS-SD (multicast UDP port 5353), picks the node with the lowest overall load and sends/receives files using TCP/IP (by default using an ephemeral port). Progress and errors are propagated back to the originating host.

Cascaded distribution ​

If a pipeline execution is distributed, and the pipeline is cascaded with multiple child pipelines, the execution of the child pipeline will be re-distributed to parallelize work as much as possible. This is handled internally by the pipeline execution.

Fallback ​

If the pipeline execution is unable to locate a suitable node for distributed execution it will fall back to local processing.

Settings ​

BatchLogPath ​

Output SimplygonBatch.exe's log to file. This setting can be very useful when you want to troubleshoot pipelines executed using RunInNewProcess or RunDistributedUsingSimplygonGrid

sg->SetGlobalBatchLogPathSetting("path.log");

> SimplygonBatch.exe -Log "path.log"

GridDiscoverTimeout ​

The time in milliseconds SimplygonBatch waits before presenting agents found during -Discover

sg->SetGlobalGridDiscoverTimeoutSetting(1000);

> SimplygonBatch.exe -DiscoverTimeout 1000

GridAgentLoadLimit ​

The maximum allowed agent load between 1-100%

sg->SetGlobalGridAgentLoadLimitSetting(50);

> SimplygonBatch.exe -AgentLoadLimit 50

Network troubleshooting ​

In case the distribution batch tool has trouble distributing processing work you can run the SimplygonBatch.exe tool with a -Discover command line argument to make it list the remote grid agents it is able to find. If you are running agents on your local network and this call does not list any remote hosts, check that your network is not blocking multicast UDP packets between machines.

If the Grid agent reports the wrong IP address, it is possible to select which network interface the agent should listen on by setting the SIMPLYGON_<MajorVersion>_GRID_IFINDEX environment variable before starting the Grid agent.

See more information under Installation -> Environment variables.

FASTBuild ​

Simplygon can use FASTBuild to distribute pipeline and pipeline batch execution. Setup of FASTBuild is outside the scope of this documentation, please refer to the FASTBuild documentation. You must make sure FBuild.exe tool path must be set in the search paths and that the FASTBUILD_BROKERAGE_PATH environment variable is set in the environment where you execute the Simplygon processing or SimplygonBatch tool.

Distributed pipeline execution ​

Distribution using FASTBuild is enabled by passing the appropriate run mode to the pipeline run scene API.

spReductionPipeline reductionPipeline = sg->CreateReductionPipeline();

// Process a scene from file directly using distribution
reductionPipeline->RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode::RunDistributedUsingFastbuild );

// Or if you have a scene object, process it using distribution
reductionPipeline->RunScene( scene, EPipelineRunMode::RunDistributedUsingFastbuild )

spReductionPipeline reductionPipeline = sg.CreateReductionPipeline();

// Process a scene from file directly using distribution
reductionPipeline.RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode.RunDistributedUsingFastbuild );

// Or if you have a scene object, process it using distribution
reductionPipeline.RunScene( scene, EPipelineRunMode.RunDistributedUsingFastbuild )

reductionPipeline = sg.CreateReductionPipeline()

# Process a scene from file directly using distribution
reductionPipeline.RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode_RunDistributedUsingFastbuild )

# Or if you have a scene object, process it using distribution
reductionPipeline.RunScene( scene, EPipelineRunMode_RunDistributedUsingFastbuild )

Distributed using batch tool ​

Distribution using FASTBuild can be enabled when invoking the batch tool (SimplygonBatch.exe) by passing the -Fastbuild parameter on the command line

SimplygonBatch.exe -Fastbuild <path/to/pipeline.json> <input.scene> <output.scene>
SimplygonBatch.exe -Fastbuild <path/to/pipelinebatch.json>

Note that due to FASTBuild limitations, the batch tool cannot report any progress.

Incredibuild ​

Simplygon can use Incredibuild to distribute pipeline and pipeline batch execution. Setup of Incredibuild is outside the scope of this documentation, please refer to the Incredibuild documentation. The IBConsole.exe tool path must be set in the search paths in the environment where Simplygon is executing.

Distributed pipeline execution ​

Distribution using Incredibuild is enabled by passing the appropriate run mode to the pipeline run scene API.

spReductionPipeline reductionPipeline = sg->CreateReductionPipeline();

// Process a scene from file directly using distribution
reductionPipeline->RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode::RunDistributedUsingIncredibuild );

// Or if you have a scene object, process it using distribution
reductionPipeline->RunScene( scene, EPipelineRunMode::RunDistributedUsingIncredibuild )

spReductionPipeline reductionPipeline = sg.CreateReductionPipeline();

// Process a scene from file directly using distribution
reductionPipeline.RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode.RunDistributedUsingIncredibuild );

// Or if you have a scene object, process it using distribution
reductionPipeline.RunScene( scene, EPipelineRunMode.RunDistributedUsingIncredibuild )

reductionPipeline = sg.CreateReductionPipeline()

# Process a scene from file directly using distribution
reductionPipeline.RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode_RunDistributedUsingIncredibuild )

# Or if you have a scene object, process it using distribution
reductionPipeline.RunScene( scene, EPipelineRunMode_RunDistributedUsingIncredibuild )

Distributed batch tool ​

Distribution using Incredibuild can be enabled when invoking the batch tool (SimplygonBatch.exe) by passing the -Incredibuild parameter on the command line

SimplygonBatch.exe -Incredibuild <path/to/pipeline.json> <input.scene> <output.scene>
SimplygonBatch.exe -Incredibuild <path/to/pipelinebatch.json>

Note that due to Incredibuild limitations, the batch tool cannot report any progress. Also note that due to the limit of a single IBConsole active process, launching multiple SimplygonBatch instances with Incredibuild distribution will cause any second process to block until the first process has completed, effectively executing the pipelines in serial rather than parallel.