Week 87: Paul Stretch

23 Oct, 2022

12 years ago, while I was studying, I came across an article that described an ambient Justin Bieber song. Sounded like an oxymoron to me, but also intriguing. A few seconds in, I was completely hooked.

I remember taking note that the song was made using something called PaulStretch, but I guess I never investigated further back then. I’ve thought about PaulStretch several times since then, and this week I decided to look into it. Apparently, it’s made by Paul Nasca, and he even put a C++ and python version on GitHub.

I’m always fascinated by people like Paul. He reminds me a bit of Tom Erbe. They seem like people who able to operate at the intersection of mathematics, code, sound in an aesthetic way. And that rather than building just for themselves, are able to provide beautiful sound tools.

One of my piano recordings before stretching:

After 8x stretching:

Next week #

Tuesday I’ll be doing a Shader Prototyping workshop hosted by Patricio González Vivo. Really excited to learn from Patricio. He’s the creator of Book of Shaders and I am perpetually in awe of the work he shares in his twitter feed.

Week 86: Diffusions and Dispersions

16 Oct, 2022

I started writing this weeknote almost a month ago - so much for weeknotes. But it’s fine, this was never meant to be a chore.

Work has been incredibly busy, and I’ve underestimated how difficult it is to build something truly new, while building an organization, a culture and being in a transition from individual contributer to leader. In a sense, I’m learning to be the diffuser of my way of thinking and doing, rather than doing the thinking or doing.

Stable Diffusion and Whisper #

It’s wild how much is happening every day with Stable Diffusion. I’m trying to keep up, and given my MacBook 2020’s limitations, I have been curious to create an efficient sandbox for myself to try out some of these new models.

I found Iulia Turc’s How To Run a Stable Diffusion Server on Google Cloud Platform (GCP) helpful, and I’ve set up VSCode to be able to SSH into my GCP server. I should probably use Google Colab like any other sane person, but I’m intrigued by having my own cloud computer. Feels like less of an abstraction somehow.

Now that I have the server, I have also been taking other models, like Whisper out for a spin. It’s fascinating, I can put an audio clip of me speaking non-sense Danish in, and it does a perfect translation in a few seconds.

Next, I’m curious to get DreamBooth up and running, though it’s proving difficult with the limited VRAM I have available on my GCP instance. By the time you’re reading this, I’m sure someone will have found a way to make it run on significantly less VRAM.

Reading #

I finished the Build book some weeks back. It’s good… really good. My leadership team has read it, but other departments have read it too, and it’s serving as a great common language and way of thinking.

I also finished the first book, A Wizard of Earthsea, from Ursula K. Le Guin’s Earthsea: The First Four Books. One thing I take away is the power of language and names. That a name is precious and reveals someone’s true nature, even forces one to surrender. Next is the second book, The Tombs of Atuan, and I’m curious to see if Ursula has another beautiful concept for me to digest.

I’m also reading Let My People Go Surfing by Yvon Chouinard as an audiobook. Not giving it my undivided attention, but it’s a good book filled with inspiring stories from a long life of a principled person.

Week 82: Stable Diffusion as a Creative Personal Assistant

4 Sep, 2022

Admittedly, the potential of missing the generative AI/ML train that’s running full speed these years gives me anxiety. I haven’t found the way to use it in my current job yet, but I’m curious.

This week I have been reading about Stable Diffusion, the open source latent text-to-image diffusion model capable of generating photo-realistic images given any text input. It being open source, malleable and run locally, is exciting.

Stable Diffusion Hello World #

I decided to take it out for a spin on my laptop, a 2020 Intel MacBook Pro. This Setting up Stable Diffusion for MacOS article by Craig Morten really helped. In less than 30 minutes I was generating my first images.

Three teddybears watching a sunset together

Three teddybears watching a sunset together, by Stable Diffusion

AI as Personal Assistant #

A few weeks ago, I was listening to an episode of Lex Friedman’s podcast featuring John Carmack. It’s a whopping 5 hour conversation, but I found it all kinds of interesting. Especially the part around Artificial General Intelligence (AGI) caught my ear. I’m really curious about the idea that, in the future, we will have a collection of AI’s as personal assistants.

Creative Personal Assistant #

So why not start developing a AGI as a personal assistant today? a Creative Personal Assistant using Stable Diffusion.

Basically the idea is to build a service you can email with a prompt, and receive a reply with the output from stable diffusion. It would look something like this:

  1. Send an email to personalassistant@domain.com with a prompt
  2. Wait a few minutes
  3. Receive a reply

I created an e-mail address for the purpose and registered the IMAP/SMTP connection and authentication info. Then I built a Node.js server that monitors the inbox of e-mail address. When it registers a new e-mail, it extracts the first line from the message body and defines that as the prompt. A text-to-image stable diffusion python-script is then spawned from my server, and when it detects a new output image, the image is e-mailed back to the sender.

  1. Server receives e-mail. Registers sender and message as prompt
  2. Server spawns the Stable Diffusion txt2img script
  3. Server registers image file output from txt2img
  4. Server sends e-mail back to sender containing image file

10 hours after thinking of the idea, I have a working proof-of-concept prototype. Incredible what’s possible with technology nowadays.

E-mail response from the Creative Personal Assistant

E-mail response from the Creative Personal Assistant

Week 81: Morephaweb and Forensic Architecture

28 Aug, 2022

It wasn’t until today Sunday that I found some time to work on side projects. Last week I added tracking to Morphaweb so I can see if people actually use the site to export reels - and it turns out people do! That gave me some motivation to work a bit more on morphaweb (yes the title pun was terrible).

Morphaweb #

One of the major challenges has been handling multiple files. After an hour or two trying to wrestle nested javascript promises, I also discovered the Crunker library which lets me concatenate audio files into a single audio file.

Next up I would like to automatically add a marker between each of the files uploaded for convenience. I should also start tagging my releases like the real software developers do.

Forensic Architecture #

Last Sunday I went to Louisiana Art Museum, and saw their Forensic Architecture exhibit.

The fifth exhibition Louisiana’s series The Architect’s Studio presents Forensic Architecture, an interdisciplinary research agency, based at Goldsmiths, University of London. Working in the intersection of architecture, law, journalism, human rights and the environment, Forensic Architecture investigates conflicts and crimes around the world.

The exhibition itself was really interesting, but even more so was that they actually keep a GitHub repository of all the models and tools they’ve created.

Week 80: Blender and a bit of reading

21 Aug, 2022

Another light week of side project work, but I did manage to finish the Blender x Three.js tutorials and there by also the last bit of Three.js Journey. It’s been a great course, and the best money I’ve ever spent on e-learning. Bruno is a great teacher, and he’s even added lessons since I bought the course, so maybe I’ll even get more for my money’s worth.

Blender x Three.js #

Since a couple of weeks ago I finished the UV unwrapping of the model, so the last part of the lesson was how to do model optimizations and exporting everything correctly from Blender.

Render of final blender model in three.js

Render of final blender model in three.js

Quite a lot of things to keep track of - somehow both easier and more difficult than I had imagined. Final result looks great though, and I’m excited and terrified to get started on my portfolio and working on the LEGO elements.

Moving around the scene

Moving around the scene

Reading #

Hacking around with the ScotRail audio announcements — Simon Willison found all the sound files of a Scottish train operator and wrote this great blog post on how he scraped it and built fun prototypes with it.

Physically Based is a database of physically based values for CG artists.

CSS Grid and Custom Shapes pt 1 — I keep being surprised how much is possible to do in pure CSS

Color and Contrast — A comprehensive guide for exploring and learning about the theory, science, and perception of color and contrast.