It’s not always easy to distinguish between existentialism and a bad mood.

  • 9 Posts
  • 566 Comments
Joined 3 years ago
cake
Cake day: July 2nd, 2023

help-circle


  • The post title vs your write-up had me going, I initially thought RFK was actually recommending better mental health support to counteract bad parenting habits instead of the usual run-of-the-mill moralising about screen time.

    One of my big hot takes is that being a new parent with a full time job and no assistance from an extended family can cause behavioural changes on par with neurological impairment and bad habits which can carry on even as the kids grow up and things get slightly more normal, so it’s nice to see a version of that in published paper form.




  • While I agree that separating instructions from data in LLM input is a fundamentally unsolvable problem, in this case that wasn’t the attack vector.

    It says the bot is forced by its security guardrails to write a small tool from scratch instead of using the one found in the downloaded zip, but due to the commonness of the task (decoding basexx text) the attacker predicts that the bot-created tool will try to import a specific dependency, so they’ve included a malicious version of that dependency in the zip, and because apparently python will prioritise locally available modules that’s what gets executed, making available all sorts of exploitation paths, including the attacker starting up a claude code instance in the users system.

    edit: Actually I’m starting to think you could classify the whole thing as a prompt injection, except the entire site was the malicious prompt, as in it declared itself (we are a site that archives notebooks in json form) in a way that would align with the bot wanting to write simple text decoding software to complete it’s mission of summarizing the available content.

    Interesting to think about.


  • I think it also depends on how all in you are on vibecoding. Like if you just prompt it and then occasionally check if the token carnage produced anything usable, then having an agents.md is probably just best practices cargo culting. But in other cases, having a Here’s How Thing Are Done In This Repo, You Fucker markdown file should make going through the more slop-happy merge requests a little less soul crushing.

    The +20% additional token churn is surprising, I would have though the markdown files get cached pretty early down the line, unless the agent file is really big compared to the chatbot output.


  • I mean it’s great if you want to automate some specific code generation in the most inefficient way possible while posing to the bosses as a forward thinking AI-first developer.

    The closest anyone can come to a use for the AGENTS.md file is so you clarify your own thinking on how the project works

    That just means that you are now supposed to write skill files instead of proper documentation to make your project even more inscrutable to anyone but a chatbots with a bottomless token budget.







  • I really like the self perpetuating optimism loop thing in the reply to Aella, it’s like a whole new cognitive bias I hadn’t thought applied to rationalists.

    I also still believe academia is a deeply broken culture. I believe this in large part because most academics have told me this.

    You seem to believe uncritically what they tell you here. The truth is, everyone LOVES to complain but if you tell them “THEN LEAVE??” they’d rather keep their position. And it’s very normal, actually. most systems (idk, liek law, healthcare, environmental protection) are “deeply broken.”

    Maybe you think that only systems worth engaging with are those with self-perpetuating optimism loops? where everyone is so happy and satisfied with the community and activities? If so, it matches really well the intellectual environment I think you are in.




  • In a world where the AGI/ASI discourse wasn’t so heavily poisoned by anthropomorphization a discussion on whether agency and self-determination are integral to the rise of disembodied consciousness might be worth having, i.e. if something is recognizably conscious should we immediately assume it knows how to want, or is consciousness meaningfully interchangeable with awareness?

    Some guy tried to write a scifi book about non-anthropomorphic consciousness and he ended up mainstreaming the discourse about how the self isn’t a thing in itself and consciousness as an evolutionary deadend. It’s called Blindsight and it’s got space vampires who optimized reading gauges by swapping the usual moving-spike-that-points-at-numbers design with depictions of human faces at various stages of suffering.