Eli Tyre

Claude 3 Opus is more reticent to claim a favorite, but also picks octopus when forced to choose.

Claude 4's favorite animal is consistently an octopus (n=8). Holds for both Opus and Sonnet.

Or is American culture too opposed to tech to allow the Silicon Valley people to build them?

...but superintelligences?

Awww : )

Amazing.

I know!

Fuck that guy!

Or rather, prediction markets are better at forecasting outcomes than polls are, not better than polls at generating original evidence that's relevant to forecasting.

(It's like wikipedia vs. primary sources)

That's why they're better, tho. They're info aggregators, info generators.

I think so!

Oh! Local minima of sexual selection.

Or do you mean internally, like a human brain is doing adversarial generation?

Humans are GAN-like?

Like they're trying to signal and other humans are trying to catch dishonest signaling

You mean goodhart on...hedonism that doesn't contribute to fitness?

Is it self-preserving?

It's not the case that now the training procedure has an incentive to game the "anti-cheating" bias, by finding cheating strategies that look legit?

Yes please.

Is the best version of your plan for alignment still that unfished git hub page that you wrote up after talking with Zvi?

I am evidently willing to forgive Yudkowsky level arrogance.

(Though to be honest, the less correct he seems to be, the less patience I have with him being rude.

I haven't seen you being rude though.)

It does sounds self-aggrandizing, but whatever, I'll give you a pass on that if it turns out you're right.

Is that to say "this would be a totally dumb thing to do from OpenAI's epistemic vantage point regarding alignment, but from my own, I can see that actually the problem is mostly solved"?

Forgive me if I compress your view to the nearest caricature available. If I do that, I'm trying to help clarify the diff, not elide crucial details.

Are saying the old OpenAI Superalignment plan will just work? Make AI scientists, they figure out alignment, then train superintelligences?

> but almost no human wants to hear them,

Also, I'm a relatively non-technical idiot, but _I_ at least am trying to figure out what's going to happen and I sure as heck want to hear if we have most of the alignment pieces!

Recover them? While being "aligned"?

...like it will be an alignment attractor basin that converges to robust alignment?

Or is "alignment" in quotes because the concept is confused.

Also if there's a "special sauce" left to the brain, then it seems more plausible that there's something that the LLM minds can't do economically enough to be relevant.

Which is Steven Byrnes's basic view.

It seems like it matters for how "winner take all" the race dynamics are?

Slow takeoff vs fast takeoff and all that.

My paraphrase of your response to that is "maybe, we can't rule it out, but probably that efficiency stuff is ten thousand little things"?

The fact that human brains are much more sample efficient than LLM pre trainging suggests that there might be a "special sauce" left to discover.

That seems like it's some reason to think that early AGIs could discover a more efficient architecture than the transformer, and FOOM.

You did say somewhere (this one I'm confident of, since I read it more recently) that the original epistemic warrant for RSI was our blank map regarding how the brain works

This is an extremely helpful response.