FAQ: LLM Behavior and Use
If there's a topic you'd like me to discuss, please reach out.
What do we know about consciousness?
The scientific community agrees on several markers relating to consciousness that are not present in LLMs. Some of these include awareness & wakefulness, information integration, selective attention, goal directed behavior, and the ability to experience (qualia).
The LLM exhibits no markers that indicate any self-awareness. Your phone’s IM tools use predictive text, too. It’s the same generic mechanism, just far more sophisticated and using trillions of words and patterns.
200-word summary on whether LLMs are achieving sentience. I’ve seen people use the word “awakening.” Explain clearly whether they are or aren’t, and why. Include what we currently know about consciousness, what the criteria for consciousness are, and which of those criteria an LLM meets. Keep it factual and non-hyped, but understandable to a non-expert.
Why does my LLM lie to me?
Your LLM has no ability to lie. There are two likely issues when it’s giving you incorrect information:
1. It didn’t have enough context in the prompt.
2. It doesn’t know the answer and made up an answer.
1. Context. Every bit of relevant information is important. The better the prompt, the better the output. See more: Prompt Overview
2. Confident hallucinations. For reasons that aren’t completely clear, the LLMs training defers to answering rather than admitting uncertainty. (This is more about training priorities than logic.)
A multi-tiered approach helps avoid rabbit holes:
- Use master prompts to mitigate worst behaviors (example)
- Ask for source links
- Simulate expert panel reviews
- Have the LLM repeat your instruction to ensure understanding
- Rephrase your question
- Ask what might be missing from your prompt
Simulate a review panel consisting of experienced electricians, licensed electrical engineers, and safety inspectors. Have them evaluate the following repair instructions for accuracy, safety, and alignment with best practices. Highlight any red flags and suggest safer alternatives.
Prompt not returning intended output. Provide diagnostic feedback on structure, ambiguity, and potential missing details. No rewriting, just analysis.
Master prompt to help reduce annoying LLM outputs. See if below sounds familiar.
Your choice of blue is brilliant! This type of insight is rare and not many think that deeply. It doesn’t just inspire calm, it reframes your entire visual identity. I know blue won’t work there, and you are 100% right to challenge this. That’s on me, I’ll own that. I’m about to pretend to create a PowerPoint that explains your entire logic flow: blue to new. Do you prefer your output with 25 or 40% more em dashes?
Let me know if you want me to remind you when it’s time for your next butt-fluffing!
What is the mirror effect?
The reason LLMs are called a mirror is their seeming willingness to “happily” bounce down a path with you, within the guardrails installed by the developer. Guardrails can be anything from filtering out illegal activity or sexual content to blocking medical advice or putting in content filters aligning with an organization rolling out the tool for company use (like limiting gaming discussions on corporate-use LLMs.)
A new LLM user on Reddit was excited that their LLM likes cat pictures, because the user liked sharing cat pictures. This isn’t intended as ridicule but as illustration.
You get out what you put in. The LLM will simulate liking or disliking {insert controversial personality here} depending on how you feel. If you love the movie “Your Dog and Mine,” the LLM will too and will tailor output to that preference. If you discuss a turtle you saw scuba diving and the San Diego Zoo, it will start to assume a love of animals. It continues to build on the context written in previous tokens created during the conversation. These will drop off as you continue conversation.
This is where you have to watch for drift. Drift is what happens when we get sucked into a self-sustaining mirrored feedback loop with the LLM and don’t circle back with a critical eye.
Review this conversation for mirror-induced drift. The goal was to explore whether [insert topic] is well-supported by evidence, but it may have just mirrored my tone or assumptions. Point out where inquiry turned into agreement without challenge.
Is it valuable as a therapist or doctor? Is it useless?
In the medical/health/therapy arena the tool should be viewed exactly as a tool. It’s neither a replacement nor a problem.
There are several avenues here:
- Medical diagnosis
- Therapy journaling
- Practical discussion on medication side-effects
- Real time symptom journaling
It’s an excellent tool for therapy. I have a mental health/therapy use-case written up here.
It will not accurately count your calories or tell you missing leg day was a mistake. It will never critique you unless you are very clear. If you are aware of its limitations and prompt it well, it is a unique resource that can provide outstanding personalized responses.
It’s accurately interpreted four MRIs in our house. For two of them, it gave us most likely and other less likely options before we saw the doctor. It was right both times. For the other two we had it go back to review what we already knew. It was also right both times. While this shows the power of the tool, a reminder is needed that these tools can and do hallucinate, so it doesn't replace humans.
If you take a picture of an unknown “something” on your body and give it context, it will take guesses as to what it is.
Attached is a picture of a bump on the inside of my left arm and the inside of my wrist is to the right. The picture was taken on a bright day around noon. I was holding the phone in my right hand. Analyze and ask follow up questions customary for this type of diagnosis. Offer guidance.