An exciting usage of the so trendy generative AI would be to generate 3D models. But at this point there seem to be no reliable tool to do so by prompting around just yet.
This being said, CSM dropped a feature that gets us closer to it as you could now use AI to convert a 2D image to a 3D model which is easy to combine with images generated by AI.
Feature announcement: Any Image to 3D
Example of 3D models that have been created using CSM: CSM 3D Viewer
All in all, using generative AI to generate an image, to then use CSM to convert that image in 3D will fasten the process of generating 3D models in many cases.
And that process is documented below just for you lil wizards:
- Use MidJourney to create an image (png)
- Use Apple Photos to remove the background (png) or you can just go to Remove.bg if you don’t have an apple device
- Use CSM to convert the image to 3D (glb)
- Use Blender to fix any issues
- Use Blender to export with texture (obj)
- Use Mixamo to auto-rig and animate the 3D object (fbx)
And there you go, you can now use you model in AR, VR and various 3D environments using Unity and Unreal
This process has been shared by Ben X Tan on his linkedin together with a Google slides documenting his pipeline
Let him know if you used his process, there’s no shame in the crediting game, all but the contrary!
Originally published at https://lab.popul-ar.com on August 8, 2023.
