What set of deep learning networks is well suited for realtime inference in OpenGL and/or DirectX?


I have a bit of experience with training and using networks using tensorflow and pytorch.

What I'd like to experiment with is to train networks for which the inference of the trained network can be implemented in pixel shaders, so that it can be applied as 'outputfilter' to a realtime game (for example, in a VR game this must happen like 180 times per second on 2k x 2k image buffers).

For that it's not practical to output each frame to some separate inference engine, apply it and then send it back to the OpenGL or DirectX context.

Which kind of networks are well suited for having the inference step directly implemented as GLSL or HLSL shaders?


Posted 2020-06-18T12:52:03.177

Reputation: 211

No answers