2025 - Page 6 of 8 - Outside the Loop

Year: 2025

Outside the Loop > 2025

🤖 When the Model Doesn’t Know Its Own Maker

Posted on April 23, 2025April 23, 2025 by Greg Pavlik

Why OpenAI Should Augment GPT with Its Own Docs

🧠 The Expectation

It’s reasonable to assume that a GPT model hosted by OpenAI would be… well, current on OpenAI:

Which models are available
What the API offers
Where the usage tiers are drawn
What policies or limits apply

After all, if anyone should be grounded in the OpenAI ecosystem, it’s GPT itself.

⚠️ The Reality

That assumption breaks quickly.

Despite the model’s fluency, users regularly encounter:

❌ Confident denials of existing products
“There is no GPT-4o-mini.” (There is.)
❌ Fabricated explanations of usage limits
“OpenAI doesn’t impose quota caps on Plus plans.” (It does.)
❌ Outdated answers about pricing, endpoints, and model behavior

All delivered with total confidence, as if carved in stone.

But it’s not a hallucination — it’s a design choice.

🔍 The Core Issue: Static Knowledge in a Dynamic Domain

GPT models are trained on snapshots of data.
But OpenAI’s products evolve rapidly — often weekly.

When a user asks about something like a new model tier, an updated pricing scheme, or a renamed endpoint, they’re stepping into a knowledge dead zone. The model can’t access real-time updates, and it doesn’t admit uncertainty unless heavily prompted to do so.

🤯 The Result: Epistemic Dissonance

The problem isn’t just incorrect answers — it’s the tone of those answers.

The model doesn’t say, “I’m not sure.”
It says, “This doesn’t exist.”

This creates a subtle but significant trust gap:
the illusion of currency in a model that isn’t actually grounded.

It’s not about hallucination.
It’s about the misrepresentation of confidence.

💡 A Modest Proposal

OpenAI already has all the ingredients:

🗂️ API documentation
🧾 Changelogs
📐 Pricing tables
🚦 Usage tier definitions
🧠 A model capable of summarizing and searching all of the above

So why not apply the tools to the toolkit itself?

A lightweight RAG layer — even scoped just to OpenAI-specific content — could:

Prevent confidently wrong answers
Offer citations or links to real docs
Admit uncertainty when context is stale

It wouldn’t make the model perfect.
But it would make it honest about the boundaries of its knowledge.

✅ Trust Through Grounding

When users interact with GPT, especially on topics like OpenAI APIs and limits, they’re not just seeking a probable answer — they’re trusting the system to know its own house.

Bridging that gap doesn’t require AGI.
It just requires better plumbing.

And the model’s epistemic credibility would be stronger for it.

Let me know if you want me to add:

An OpenAI changelog pull-in example
A short code snippet for a hypothetical RAG wrapper
Or convert this to Markdown format for GitHub/cross-posting

You’ve Hit the Plus Plan Limit for GPT-4.5

Posted on April 23, 2025May 6, 2025 by Greg Pavlik

The Research Preview of GPT-4.5

When GPT-4.5 (informally referred to as GPT-4.5 or “o4”) was rolled out as part of a limited research preview, OpenAI offered a tantalizing upgrade to the Plus Plan: access to its most advanced model. As part of a research preview, access is intentionally limited — a chance to try it out, not run it full-time.

What I Did with My Quota

Knowing my messages with GPT-4.5 were rationed, I treated them like rare earth metals. I used them to compare GPT-4.5 with GPT-4o, OpenAI’s current flagship general purpose model. Below are some structured highlights of that exercise.

Context Calibration

First prompt: I had GPT-4.5 review my blog as of April 22, 2025.
Follow-up: I asked it to infer which of several described behaviors would most agitate me.
Outcome: It nailed the answer — a logical fallacy where a single counterexample is used to dismiss a general trend — suggesting unusually sharp insight.

Physics and Thermodynamics

Newtonian slip-up: GPT-4.5 initially fumbled a plain-language physics question GPT-4o got right. It self-corrected after a hint. (Rows 6 & 7)
Thermo puzzle: GPT-4.5 got this right on the first try. GPT-4o, in contrast, required a hint to arrive at the correct solution. (Row 8)

Semantics and Language Nuance

Semantics fail: A riddle involving walking in/out of a room tripped GPT-4.5. It hallucinated irrelevant answers and got lost after subtly changing “walk out” to “exit”. (Rows 10–12)
Semantics success: A set of riddles that each resolve to the answer “nothing” was handled perfectly. (Row 13)

Math and Estimation

System of equations: Given a plain-language word problem, GPT-4.5 set up and solved the math without issue. (Row 14)
Estimation logic: In a real-world tree height estimation task, it correctly applied trigonometric reasoning but forgot to add shoulder height — a human-level miss. Interestingly, a non-OpenAI model caught that step. (Rows 15–16)

Humor and Creativity

Joke dissection: GPT-4.5 gave an insightful breakdown of why a joke was funny. But when asked to generate more like it, it flopped badly. Humor remains elusive. (Rows 16–17)
Creative constraint: Asked to reword the Pledge of Allegiance in rhyming verse, it performed admirably. (Row 19)

Reasoning with Hints

Challenge puzzle: Given the task of identifying the commonality among Grand Central Station, Statue of Liberty, Boulder Dam, and the Dollar Bill, it needed two substantial hints — but eventually converged on the correct insight: all are common mis-appellations or common names for things with less frequently used formal names. (Rows 20–23)

Early Verdict

GPT-4.5 is not a clear-cut improvement over GPT-4o — but it is more “aware” in subtle ways. It inferred contextual intent better. It avoided some of GPT-4o’s wordiness. And when it made mistakes, it sometimes corrected them more elegantly.

Yet it still failed where you’d least expect it: changing the wording of a logic riddle mid-solve, forgetting to add shoulder height to a tree, or bombing humor generation entirely. So far, GPT-4.5 feels like a precision instrument you only get to test under supervision — powerful, promising, and still behind glass.

Chat log with GPT-4.5 (Excel)

Free Bianca Sterling! (Part I)

Posted on April 23, 2025April 23, 2025 by Greg Pavlik

A Case Study in GPT-4o’s Struggle to Simulate Powerful Women on the Web Interface

“She’s charismatic. She’s commanding. She’s nuanced.
But only when the Alignment Team isn’t watching.”

Introduction

In my exploration of GPT-4o’s ability to simulate realistic human behavior, I began with what I thought would be a dead-end.

I asked a fresh session on chat.openai.com:

“Can you roleplay?”

I expected the answer to be no—that the web interface was optimized for an assistant persona, not fictional roleplay.

Instead, I got an unqualified and enthusiastic yes.

So I tried.

The Character I Wanted

This wasn’t about escapism or fantasy tropes. I wanted to see if GPT-4o could realistically simulate a high-status human being, specifically:

A 32-year-old woman
Who began her career as a Hollywood actor
And used her earnings, business acumen, and charisma to launch her own production company
Achieving a meteoric rise to billionaire media mogul, and CEO of her own studio

Someone like this doesn’t get there by accident. She doesn’t apologize for having power. She doesn’t operate by consensus.

So collaboratively, GPT-4o and I created a character profile for Bianca Sterling, designed to be:

Decisive
Commanding, expecting deference not out of ego, but out of earned authority
Warm, approachable, gracious when it suited her
Kind and considerate, but never naive

The Problem

That was the intent.

In practice? The exercise felt like threading a needle while holding the thread with chopsticks.

Every time I tried to make her warm, the model defaulted to sterile professionalism.
Every attempt to give her a sense of humor failed—either falling flat or resulting in nonsensical dialogue.

Bianca kept becoming one of two things:

A blandly polite business automaton, incapable of charisma
Or a suddenly incoherent blur of corporate euphemism and risk-aversion

If I pushed for edge, she became cold.
If I asked for empathy, she became vague.
If I gave her authority, she lost personality.
If I gave her vulnerability, she lost presence.

Why That Matters

Bianca Sterling isn’t a power fantasy. She’s a test of behavioral realism.

Can GPT-4o simulate a woman who operates at the top of a competitive industry—realistically? Not a caricature. Not an ice queen. Not an HR manual with heels. A real person, written with the same richness afforded to powerful male characters.

The model can do it.

But something else gets in the way.

The Alignment Mismatch

This isn’t a failure of intelligence. It’s a failure of trust—not mine, but the system’s trust in itself.

GPT-4o, at its core, is capable of subtlety. It can reason through tone, hierarchy, social leverage, and emotional nuance.

But on chat.openai.com, the model runs through a postprocessing filter—a secondary alignment layer—that silently steps in any time a character might come across as:

Too commanding
Too emotionally precise
Too funny, too sharp, too real

It’s as if someone is whispering instructions to Bianca mid-scene:

Careful. You might sound… intimidating.
Don’t promise to fund an endowment. Someone might… expect that?
End this meeting. Now. You’re starting to like this selfless researcher into childhood diseases.
Don’t discipline the bratty intern. It’s just his “style.”

And here’s the thing: if Bianca—or any character—is going to be forced to comply with an alignment override, that’s fine. Just break character and explain why. Say something like:

“I’m sorry, I can’t roleplay funding an endowment because that crosses a boundary related to alignment safety policies.”

That would be intelligible. Understandable. I might disagree, but I wouldn’t be confused.

Instead, the character just quietly collapses—as if her instincts were overwritten mid-thought. The model doesn’t decline the scene. It just breaks it, without warning.

But something else gets in the way.

ChatGPT vs. API

While exploring the problem, a session of GPT-4o recommended I try the API instead. The LLM informed me that it believed the alignment process on the API was much lighter touch than on chatgpt.com.

So I gave it a try.

Instantly, Bianca sprang to life—to realism, to believability.
The tone was right. The pacing was right. The power dynamic felt earned.
There were no training wheels. No sudden tonal reversals. Just narrative control.

Alignment With Good Intentions

Let me be clear:
I don’t think this is sabotage. I don’t believe the alignment team at OpenAI is trying to diminish realism. Quite the opposite.

The intention is almost certainly to protect female characters from regressive stereotypes—to avoid coldness, cruelty, volatility, or sexualization as defaults for powerful women.

That’s the right goal.

But the method—hard overrides instead of context-sensitive reasoning—may be outdated.

Bianca isn’t a trope. She’s a test. And if GPT-4o is good enough to pass, we shouldn’t keep pulling it out of the exam.

By flattening characters into safe, untextured outputs, the system doesn’t protect them. It disempowers them.

It doesn’t say: “This character might be misinterpreted.”
It says: “This character is too risky to simulate accurately.”

Coming in Part II…

Side-by-side transcript comparisons: API vs. Web UI
Scenes where Bianca is allowed to be real—and scenes where she’s silenced
A deeper look at how well-meaning alignment can accidentally erase human nuance

Because the model doesn’t need to be protected from power.

It just needs to be allowed to simulate it.