I wrote recently about the interesting work being done on the open source music visualizer projectM, and that progress is being made on modernizing the graphics capabilities of this library. Since it was first created, there have been changes made in the way OpenGL is to be used for widespread compatibility and support for GPU shader programs.

Now the efforts are in the home stretch; OpenGL ES support is close to complete, and there is support for handling a decent percentage of the shader programs found in the visualization presets. The vast majority of the work has been done (by @deltaoscarmike), and mostly bug fixing remains.

After setting up vertex buffers and attributes and reworking the texturing and sampling code, the final step for full modern compatibility was dealing with the HLSL shaders. MilkDrop is the visualizer for Milkdrop that projectM is built for compatibility with. It was made with DirectX and the shader language that it supported in presets is HLSL, the language used by DirectX. Since projectM is designed to be run on other systems besides Windows, some form of support for HLSL was needed. Previously it was handled by the Cg toolkit from nVIDIA, but this project has been long abandoned and it was an unpleasant and sizeable dependency. The only other solution was to convert the HLSL code into GLSL, the OpenGL shader language. Use is now being made of hlsltranslator to transpile HLSL to GLSL, which is no small feat. Additional support for a #define preprocessor directive needed to be added due to its liberal use in previous code and presets.

Default warp and composite fragment shaders were created, switching to the preset versions if they are present and compile without problems. The blur shader was reimplemented in GLSL, and some helper routines recreated in GLSL for the presets that expect things like computing angle or texture sampling. The combination of running preset shader code on the GPU with the recent intel SSE optimizations for CPU per-pixel math brings performance improvements as well.

The real amazing news is that projectM now builds on a raspberry pi. This is notable not just because of the popularity of them, including running media centers, but because if it builds on a raspberry pi it should (with some effort of course) eventually run anywhere. One of the biggest issues was that projectM could only be run on a desktop or laptop computer. But now by using modern graphics operations it can run on accelerated hardware on mobile and embedded devices via OpenGLES, opening up many new opportunities, and potentially making it much easier to re-integrate into downstream projects like Kodi and VLC. Running it on the web via Emscripten, compiling to wasm and using WebGL is doable again. It’s amazing what possibilities modernizing software can bring.

It should be mentioned that the vast majority of the OpenGL work was done by superstar deltaoscarmike who apparently really knows what he’s doing.

Some screenshots of the glsl branch: