• 4 Posts
  • 22 Comments
Joined 1 year ago
cake
Cake day: July 19th, 2023

help-circle
  • Even better, we can say that it’s the actual hard prompt: this is real text written by real OpenAI employees. GPTs are well-known to easily quote verbatim from their context, and OpenAI trains theirs to do it by teaching them to break down word problems into pieces which are manipulated and regurgitated. This is clownshoes prompt engineering done by manager-first principles like “not knowing what we want” and “being able to quickly change the behavior of our products with millions of customers in unpredictable ways”.


  • That’s the standard response from last decade. However, we now have a theory of soft prompting: start with a textual prompt, embed it, and then optimize the embedding with a round of fine-tuning. It would be obvious if OpenAI were using this technique, because we would only recover similar texts instead of verbatim texts when leaking the prompt (unless at zero temperature, perhaps.) This is a good example of how OpenAI’s offerings are behind the state of the art.


  • corbin@awful.systemstoTechTakes@awful.systemsChatGPT spills its prompt
    link
    fedilink
    English
    arrow-up
    6
    arrow-down
    1
    ·
    2 days ago

    Not with this framing. By adopting the first- and second-person pronouns immediately, the simulation is collapsed into a simple Turing-test scenario, and the computer’s only personality objective (in terms of what was optimized during RLHF) is to excel at that Turing test. The given personalities are all roles performed by a single underlying actor.

    As the saying goes, the best evidence for the shape-rotator/wordcel dichotomy is that techbros are terrible at words.

    NSFW

    The way to fix this is to embed the entire conversation into the simulation with third-person framing, as if it were a story, log, or transcript. This means that a personality would be simulated not by an actor in a Turing test, but directly by the token-predictor. In terms of narrative, it means strictly defining and enforcing a fourth wall. We can see elements of this in fine-tuning of many GPTs for RAG or conversation, but such fine-tuning only defines formatted acting rather than personality simulation.











  • This is some of the most corporate-brained reasoning I’ve ever seen. To recap:

    • NYC elects a cop as mayor
    • Cop-mayor decrees that NYC will be great again, because of businesses
    • Cops and other oinkers get extra cash even though they aren’t business
    • Commercial real estate is still cratering and cops can’t find anybody to stop/frisk/arrest/blame for it
    • Folks over in New Jersey are giggling at the cop-mayor, something must be done
    • NYC invites folks to become small-business owners, landlords, realtors, etc.
    • Cop-mayor doesn’t understand how to fund it (whaddaya mean, I can’t hire cops to give accounting advice!?)
    • Cop-mayor’s CTO (yes, the city has corporate officers) suggests a fancy chatbot instead of hiring people

    It’s a fucking pattern, ain’t it.






  • That’s a real possibility. At risk of going NSFW, HN seems to have a very predictable reaction to links to (English) WP; their comments are always tangents based on personal experiences. For example:

    But (at risk of invoking the shape-rotator stereotype) it seems like it’s hard for HN’s denizens to imagine a time when they personally were experiencing a memetic effect because memes are patterns rather than concretions. For analogy, an HN full of fish would not leave a single comment on the Fish WP article, “Water.” Edit: A fairer example would be an article like “Properties of Water”, because memetics is the study of memes, and memes are like water. (“Hydrodynamics” isn’t a standalone article, but it would be another good candidate.)







  • Going a bit on side venture here with this comment, but I’d so much prefer this to be well written up instead of a two and half hour video. And it’s so ubiquitous, watching videos over people reading, and I can’t help but think it’s extremely hurtful on a cultural/societal level.

    Side sneer: Nah, video is great. Watching an hour-long video is a standard part of research for the working software engineer. Lots of content from conferences like Strange Loop, DEFCON / Black Hat, and CCC is only available as video, and was intended to be watched or accompanied by slides.

    …But I may be biased, as I have no problem performing 2hr speedruns. Lately I’ve been running games with 3hr or 4hr routes, and I’ll readily agree that it’s not for everybody.