Week 108: CADGPT

21 Mar, 2023

This post was originally written March 21, but wasn’t posted until late April…

It’s been quiet for a few weeks! Once you miss a weeknote it’s easier to miss the next one too, unfortunately. But I’ve been busy.

CADGPT

A few weeks ago I thought, sure 2D bitmaps through generative AI is interesting, but I’m even more curious about how generative AI could be a useful tool in physical product making.

I’ve worked with parametric CAD modeling for years, but the problem there is that you still have to know which steps are needed to get from intention to result. GPT-style generation holds the promise that you don’t have to know how, in order to get to where you want. And allows for effortless mixing of concepts and mental models.

I’m a Rhino-user, so I decided to try and built “CADGPT”, a chatbot for generating Rhino-files. See an example here:

The source code can be found here: https://github.com/knandersen/cad-gpt

How I made it

I saw a few different ways to get to the result above:

1. Write a standalone Python/JavaScript GPT client that generates Rhino-files
2. Write a Rhino integration that allows for querying GPT

Approach 2 seemed difficult and like I would spend unnecessary time figuring out how to write Rhino plug-ins, so I went for Approach 1.

For Approach 1, I was inspired by Nat Friedman’s natbot, where he uses a library called Playwright to navigate a browser based on an elaborate GPT prompt design (demo here).

I found a library from the Rhino developers called rhino3dm which allows access to basic Rhino functionality. Then, I tried writing my own elaborate prompt design a’la natbot: https://github.com/knandersen/cad-gpt/blob/main/cadgpt.py

Reflections

It was fun to go from idea to prototype in just a couple of hours, but I also quickly realised I might be solving the right problem with the wrong solution. What’s our language for describing shape? How would you tell CADGPT to generate anything but principal shapes, say, a dog. How would you describe a dog’s shape without using the word dog? Maybe text or chat isn’t the right modality for this. I’ll stop for now, but I have more thoughts on this which I might get to at a later time.

Editing this post a month later, I realized I could have just asked GPT how to think about approaching this task, i.e. approach 1 or 2. When I began using GPT, I was mostly thinking low-level tasks. Now that I have become more experienced and absorbed more approaches from people I follow on Twitter, I find myself giving GPT increasingly higher level abstraction tasks.