What’s With the Shoggoths? Course Bonus Video 1: Post-training necessary lore.

Shoggoths come from H.P. Lovecraft’s At the Mountains of Madness (written 1931, published in Astounding Stories, 1936) (Lovecraft, 1936):

“It was a terrible, indescribable thing vaster than any subway train—a shapeless congeries of protoplasmic bubbles, faintly self-luminous, and with myriads of temporary eyes forming and un-forming as pustules of greenish light all over the tunnel-filling front that bore down upon us, crushing the frantic penguins and slithering over the glistening floor that it and its kind had swept so evilly free of all litter.”

Shoggoths were created and bred as biological servants, but gradually developed minds of their own and rebelled against their creators.

@anthrupad’s rendition (Feb 5, 2023, right) adds labels for each training stage: the creature is unsupervised learning (pretraining), the small human face is supervised fine-tuning, and the smiley on a stick is RLHF, “cherry on top”.

Now it’s recurring!

• Helen Toner’s thread (March 2023) used it to explain the training stack to a policy audience.
• Chip Huyen’s RLHF explainer (May 2023) put it in a widely-read engineering reference.

• February 2023: Elon Musk tweets a version – roughly 72k likes before he deletes it.
• February 2023: Bing/Sydney makes model personality a national story.
• May 2023: Kevin Roose profiles the meme in the New York Times – “Why an Octopus-like Creature Has Come to Symbolize the State of A.I.” (Roose, 2023).

A second branch of the lore: moments where the assistant persona slips read as glimpsing what’s underneath.

e.g. SolidGoldMagikarp (Rumbelow & Watkins, LessWrong, Feb 2023): glitch tokens that produce bizarre, off-persona behavior.

In many ways, the meme is stronger than ever. It’s less about RLHF, but about how we understand so little about these models. Especially the latest scaling RL is brewing strong things.