Nice work, the outputs look ethereal and quite beautiful. For some related work using genetic algorithms and evolution strategies, see[1].
Sketch synthesis is an area I'm pretty interested in lately; I'm currently exploring similar things with CLIP to guide fitness, natural evolution strategy to optimize the rendered results, and using an implicit neural representation to represent pen plotter paths (rather than a series of explicit curves/strokes).[2]
Awesome stuff :) The results are really impressive.
GPU acceleration is a really good idea; my CPU implementation of a similar idea with triangles (https://github.com/anematode/triangle-stacking) was compute constrained for smaller images (which was fixable with good SIMD optimizations) but became bandwidth constrained for very large images that don't fit in L2. I think a port to OpenCL would have been a good idea.
I made a simple web UI[1] for generating images like this. It uses a Go library by Michael Fogleman[2] for all the heavy lifting—hat’s off to him.
For Bézier curves in particular, iteratively constraining the search around initial matches seems key to retaining detail (see the “rep” argument in Fogleman’s work), for example in the eyes of the Vermeer portrait in the OP.
I wrote something similar years ago, which would instead convert an image into a mesh of polygons. The idea was to have a vector low-size SVG that could be used as an image placeholder or background for web pages.
I think I lost the code, but it was initially a genetic algorithm that randomly placed overlapping polygons, but the later improved method had connected polygons that shared points - which was far more computationally cheaper.
Another method I explored was to compose a representative image via a two-colour binarised bitmap, which provided a pixelated version of the image as a placeholder.
The core idea is that you drop the image as a small Data URI straight into the page, and then fetch the high-detail version later. From the user's perspective, they are getting a very usable web page early on, even on poor connections.
Back in the 00s a friend and I played with the idea of using genetic programming and random line segments to approximate images (ie. maintain a population of candidates, take the best n percent, "breed" them to create the next generation, add some "mutations", repeat). This implementation seems to use a basic hill-climbing algorithm to reach a local optimum – something like simulated annealing might improve both quality and speed of convergence.
Sketch synthesis is an area I'm pretty interested in lately; I'm currently exploring similar things with CLIP to guide fitness, natural evolution strategy to optimize the rendered results, and using an implicit neural representation to represent pen plotter paths (rather than a series of explicit curves/strokes).[2]
[1] https://es-clip.github.io/
[2] https://x.com/mattdesl/status/2011434166022476109
GPU acceleration is a really good idea; my CPU implementation of a similar idea with triangles (https://github.com/anematode/triangle-stacking) was compute constrained for smaller images (which was fixable with good SIMD optimizations) but became bandwidth constrained for very large images that don't fit in L2. I think a port to OpenCL would have been a good idea.
For Bézier curves in particular, iteratively constraining the search around initial matches seems key to retaining detail (see the “rep” argument in Fogleman’s work), for example in the eyes of the Vermeer portrait in the OP.
[1]: https://github.com/kevin-hanselman/primitive-web-ui
[2]: https://github.com/fogleman/primitive
I think I lost the code, but it was initially a genetic algorithm that randomly placed overlapping polygons, but the later improved method had connected polygons that shared points - which was far more computationally cheaper.
Another method I explored was to compose a representative image via a two-colour binarised bitmap, which provided a pixelated version of the image as a placeholder.
The core idea is that you drop the image as a small Data URI straight into the page, and then fetch the high-detail version later. From the user's perspective, they are getting a very usable web page early on, even on poor connections.
https://news.ycombinator.com/item?id=389727