Grid ​

The easiest way to get distribution of pipelines and pipeline batches running is to use the Simplygon Grid. There is no need to setup coordinators or network details, just run the grid agent executable on any number of machines on the local network and everything is setup for distribution.

Running a distribution agent ​

To start a grid agent run the SimplygonGridAgent.exe tool on the machine. There is no coordinator or network setup needed, the pipeline execution locates available processing nodes using mDNS/DNS-SD.

As of version 10.2.5200 the agent must run the same Simplygon version as the client. The agent is always discoverable but the -Discover query will produce errors asking you to upgrade the agent to the compatible version displayed in the error message:

[2023-09-29 10:14:47]    [INFO] -------------------------------------------
[2023-09-29 10:14:47]    [INFO] 9.9.9.132:57644 (10.2.5600)
[2023-09-29 10:14:47]    [INFO] Load  |  Vulkan  |  Service
[2023-09-29 10:14:47]    [INFO] 0     |  Yes     |  No
[2023-09-29 10:14:47]   [ERROR] - This agent is unable to process any Grid jobs: agent (10.2.5600) is not running the same version as client (10.2.5803)
[2023-09-29 10:14:47]   [ERROR]   Please upgrade agent to the same version as client SDK: 10.2.5803
[2023-09-29 10:14:47]    [INFO] -------------------------------------------

Check Grid capabilities ​

Grid agent availability is currently based on compatible version, vulkan capabilities and load. If you want to check the status of your grid agents, run SimplygonBatch.exe -Discover. That will list all your agents that are discoverable and also print information about each agent, here is an example output:

❯ & "C:\Program Files\Simplygon\10\SimplygonBatch.exe" -Discover
SimplygonBatch discover remote agents
...
[2023-09-29 10:14:47]    [INFO] Found 3 agents:
[2023-09-29 10:14:47]    [INFO] -------------------------------------------
[2023-09-29 10:14:47]    [INFO] 9.9.9.120:49157 (10.2.5803)
[2023-09-29 10:14:47]    [INFO] Load  |  Vulkan  |  Service
[2023-09-29 10:14:47]    [INFO] 0     |  Yes     |  No
[2023-09-29 10:14:47]    [INFO] -------------------------------------------
[2023-09-29 10:14:47]    [INFO] 9.9.9.248:49157 (10.2.5600)
[2023-09-29 10:14:47]    [INFO] Load  |  Vulkan  |  Service
[2023-09-29 10:14:47]    [INFO] 10    |  Yes     |  No
[2023-09-29 10:14:47]   [ERROR] - This agent is unable to process any Grid jobs: agent (10.2.5600) is not running the same version as client (10.2.5803)
[2023-09-29 10:14:47]   [ERROR]   Please upgrade agent to the same version as client SDK: 10.2.5803
[2023-09-29 10:14:47]    [INFO] -------------------------------------------
[2023-09-29 10:14:47]    [INFO] 9.9.9.203:49157 (10.2.5803)
[2023-09-29 10:14:47]    [INFO] Load  |  Vulkan  |  Service
[2023-09-29 10:14:47]    [INFO] 65    |  Yes     |  Yes
[2023-09-29 10:14:47] [WARNING] - No Vulkan capabilities. Pipelines requiring Vulkan cannot be distributed to this agent.
[2023-09-29 10:14:47]    [INFO] -------------------------------------------

In the above example we can see that two of the agents are missing vulkan capabilites and won't pick up pipelines that requires vulkan. Also, one of the agent is running an older version and needs an upgrade in order to be availabe and pick up distributed jobs.

Bridging subnets ​

If your network is segmented and multicast UDP packets cannot traverse subnet boundaries you can bridge these subnets by allowing the batch tool to connect to a Grid agent on a different subnet over TCP/IP and routing discovery of hosts from that subnet back to the batch tool over the TCP/IP connection. This is done by setting the environment variable SIMPLYGON_<MajorVersion>_GRID_HOSTS to a semicolon separated list of machines (either IP address or FQDN) running Grid agents on different subnets. This way you do not need to have every machine in the list, only one on each subnet that can act as the bridge during discovery. The bridge connection utilizes TCP/IP port 55001 which must be open in any firewalls. The environment variable needs to be set only on the machine that is the distributor.

Replace <MajorVersion> with the major version of Simplygon, e.g. SIMPLYGON_10_GRID_HOSTS.

Running as a Windows service ​

The grid agent can be started as a Windows service. To install the service, run the grid agent executable with the -InstallService argument. To uninstall the service, run the grid agent executable with the -UninstallService argument. Errors will be posted to the Windows event log. Note that services are run in session 0 and cannot display any UI to the logged in user.

Log to file ​

If you want the grid agent log output as a file, add the -LogFile <path-to-file> argument to the command line.

Distributed pipeline execution ​

Distribution is enabled by passing the appropriate run mode to the pipeline run scene API

spReductionPipeline reductionPipeline = sg->CreateReductionPipeline();

// Process a scene from file directly using distribution
reductionPipeline->RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode::RunDistributedUsingSimplygonGrid );

// Or if you have a scene object, process it using distribution
reductionPipeline->RunScene( scene, EPipelineRunMode::RunDistributedUsingSimplygonGrid )

spReductionPipeline reductionPipeline = sg.CreateReductionPipeline();

// Process a scene from file directly using distribution
reductionPipeline.RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode.RunDistributedUsingSimplygonGrid );

// Or if you have a scene object, process it using distribution
reductionPipeline.RunScene( scene, EPipelineRunMode.RunDistributedUsingSimplygonGrid )

reductionPipeline = sg.CreateReductionPipeline()

# Process a scene from file directly using distribution
reductionPipeline.RunSceneFromFile( "input.obj", "output.obj", EPipelineRunMode_RunDistributedUsingSimplygonGrid )

# Or if you have a scene object, process it using distribution
reductionPipeline.RunScene( scene, EPipelineRunMode_RunDistributedUsingSimplygonGrid )

Distributed batch tool ​

Distribution can be enabled when invoking the batch tool (SimplygonBatch.exe) by passing the -Distribute parameter on the command line

SimplygonBatch.exe -Distribute <path/to/pipeline.json> <input.scene> <output.scene>
SimplygonBatch.exe -Distribute <path/to/pipelinebatch.json>

If you wish to integrate the batch tool in an asset processing tool and read progress for an UI, you can pass and additional parameters to the batch tool executable, -Progress. It will suppress the command line progress bar and instead print progress as a number between 0 and 100, one line at a time, to the standard output.

SimplygonBatch.exe -Distribute -Progress <path/to/pipeline.json> <input.scene> <output.scene>
SimplygonBatch.exe -Distribute -Progress <path/to/pipelinebatch.json>

Here is an example how to do batch processing using Simplygon Grid.

How it works ​

The pipeline execution usees the batch tool (SimplygonBatch.exe) to do the distribution. The batch tool find a suitable node using mDNS/DNS-SD (multicast UDP port 5353), picks the node with the lowest overall load and sends/receives files using TCP/IP (by default using an ephemeral port). Progress and errors are propagated back to the originating host.

Cascaded distribution ​

If a pipeline execution is distributed, and the pipeline is cascaded with multiple child pipelines, the execution of the child pipeline will be re-distributed to parallelize work as much as possible. This is handled internally by the pipeline execution.

Fallback ​

If the pipeline execution is unable to locate a suitable node for distributed execution it will fall back to local processing.

Settings ​

sg->SetGlobalBatchLogPathSetting("path.log");

> SimplygonBatch.exe -Log "path.log"

sg->SetGlobalGridDiscoverTimeoutSetting(1000);

> SimplygonBatch.exe -DiscoverTimeout 1000

sg->SetGlobalGridAgentLoadLimitSetting(50);

> SimplygonBatch.exe -AgentLoadLimit 50

Troubleshoot ​

Network troubleshooting ​

In case the distribution batch tool has trouble distributing processing work you can run the SimplygonBatch.exe tool with a -Discover command line argument to make it list the remote grid agents it is able to find. If you are running agents on your local network and this call does not list any remote hosts, check that your network is not blocking multicast UDP packets between machines.

If the Grid agent reports the wrong IP address, it is possible to select which network interface the agent should listen on by setting the SIMPLYGON_<MajorVersion>_GRID_IFINDEX environment variable before starting the Grid agent.

See more information under Installation -> Environment variables.

Known issues ​