A technical experiment of real-time 3D fluid simulation and visual rendering capabilities of the TouchDesigner platform. Summer research for new media installation techniques at OUTPUT.

Background

The experiment originated from my interest in this topic and technical research on Vincent Houzé's interactive installation works. Unfortunately, there were very few tutorials about the rendering pipeline of similar kinds by the time of this project. Therefore, my research and implementation largely depended on Houzé's approach, which he partially shared on Github and the conventions in computer graphics.

Creating real-time interactive fluid visuals requires two parts - simulation and rendering.

Fluid Simulation

If to build from scratch, the simulation requires understanding the basics of fluid dynamics and its discrete implementation in GPU. The standard approach is to implement Lagrangian schemes in the GPGPU structure. The technique is commonly used for simulating liquid, smoke, fire, and fabric.

Houzé's approach is to integrate Nvidia FleX, a real-time particle solver by Nvidia, with TouchDesigner by building a C++ plugin. He released the plugin FlexChop opensource. The plugin provides an interface in TouchDesigner to specify a particle system and acquire later states of the simulation in real-time in a CHOP operator.

Fluid particles simulated in Nvidia FleX
Fluid particles simulated in Nvidia FleX

In addition, Nvidia FleX allows real-time input, such as geometries that collide with the fluid. I conveniently paired up a LeapMotion to translate hand gestures and positions into controls of the physical environment.

Gesture-controlled ball hitting fluid particles
Gesture-controlled ball hitting fluid particles

Screen-Space Rendering

To give these particles the look of fluid, there are several techniques. The traditional surface reconstruction algorithm is too computation-heavy and will not work in any real-time use case. In gaming graphics, the industry takes a screen-space approach. That is to infer the final graphics from several different renderings of the camera plane, regardless of the geometrical ground truth of the fluid.

Referencing Nvidia's talk at GDC 2010, the implementation includes several passes which render the particles as a sphere to generate depth map, normal map, and density map, to calculate the reflection and refraction. This is realized by coding custom shaders as a GLSL TOP in Touchdesigner.

Final result and different render passes from my experiment
Final result and different render passes from my experiment

Stylized Rendering

After building a foundation of the rendering framework, I started experimenting with different aesthetics by multiplying rendering techniques on Touchdesigner.

Acknowledgement

Special thanks to OUTPUT for providing the space, hardware, and opportunity.


↓ more projects ↓

Project NoCode 无码计划

An artist collective that engages with topical issues pertaining to urban life under the COVID-19 pandemic “new normal” in China.