• 5 Posts
  • 282 Comments
Joined 3 years ago
cake
Cake day: July 19th, 2023

help-circle

  • https://www.patreon.com/MaxRead/posts/ai-x-risk-and-ai-173181621

    worth the full read, but here’s the parts I liked

    But “x-risk” is not the only kind of “existential” risk about which AI enthusiasts and their converts tend to worry. This recent Ezra Klein column, written in the wake of OpenAI’s release of hundreds of mathematical proofs, is a nice articulation of something I’ve come to think of as “n-risk,” for “ennui risk” (with the mutual understanding that it’s not pronunciation-accurate so don’t bother making a nitpicky little comment about it): fear of the loss of comprehension, control, and meaning in the face of impossibly sophisticated godlike machines poses.

    We see the threat of “n-risk” in some of the emotional reaction of mathematicians faced with the results of OpenAI models bulldozing through open problems: a sense of having been surpassed and left behind, having the meaning from which one derives one’s worth totally emptied. In some sense, on a conceptual level and as a discursive object, x-risk is preferable to n-risk: it’s striking, dramatic, and offers us humans an ongoing chance at great feats of heroism and genius. N-risk has no such narrative reassurances. Maybe it’s better to burn out (as your body is turned into a paperclip) than to fade away.

    Here you might ask: Is the cause of melancholy and despair on the part of some mathematicians simply the knowledge that a computer system has surpassed their ability to understand or produce proofs? Or is it the realization that said computer system is privately owned and confidentially developed in service of accumulation rather than the pursuit of shared human knowledge? Or is it the contempt with which the people who work for the computer system companies and the hanger-on fans of these computer systems seem to regard the mathematicians? Or is it the sudden recognition that the pursuit of mathematical truths has already been so totally devalued by an anti-intellectual social system that you can no longer feel secure that the tradition to which you have devoted your life will continue as a space of intellectual inquiry, even given the development of these new tools? And so on.

    I mean, realistically, it’s all of the above.





  • Hmm, I guess they could have written a lot of code lines, but its not like the code they write would be equivalent to million LOC of a coherent whole.

    I’d have guessed it was better for a training set to have lots of different mostly-known-good solutions to common problems and good visibility of the solution space, rather than one linux-sized program, even with the same bug rate between the two. I’d imagine leetcode is monetizing their user generated code this way too. IDK.

    I do actually have a decent GPU i bought 4 years ago in my PC, so maybe I’ll do a few simple experiments again… my preferred approach is to pick a simple but nonlinear problem e.g. binary multiplication of two large numbers, where you e.g. skew the training sample distribution or omit some number pairs and see if the resulting model can learn correct multiplication anyway.

    For that particular experiment, I’d also point you towards some of the 90s literature on “genetic programming”, because you could reuse most of your scaffolding for it. No GPU req’d, ought to learn it fairly quickly.


  • Mercor has ~$1.5B revenue so napkin math would put at 10-20 million contractor hours per year. I’m not very fast but I can type out about a hundred lines an hour fairly regularly, especially boring code like unit tests, data cleaning or OO design patterns. LoC is a bad metric but still. For comparison, Linux and Chrome are ~40M LoC each, AAA games are 3-10M LoC.

    And to your point, they can do data augmentation by rotating nodes in the ASTs, changing variable names, and other tricks to make it even fatter.

    For comparison, the Software Heritage Archive of all open source is ~2 Petabytes / ~30B files. But not sure how much of it is AI slop at this point, or for dead languages / platforms.


  • So, I was thinking about how modern LLMs got better at coding in the last 1 year, despite not having much good new training data.

    Half the job posts I see on linkedin are from snorkel, handshake and mercor etc to train llms for $50-90/hr. So I think there’s been quite a bit of new training data created as those companies scaled up.

    So everyone is paying openai to pay mercor to pay contractors to write code and then regurgitate it, rather than just paying for what they want directly.

    idk. I’m just skeptical that there was actually no human in the loop and that the gains were from unit tests / synthetic TDD.












  • corey quinn at https://lnkd.in/p/gXmqzqpZ

    I’m a man. I have a yellow hat. I also have a monkey. My track record of supervising this monkey is spotty at best, but that’s not important right now.

    He is a very smart monkey. When you tell him to do something, he does it in a way that’s smarter than any monkey you’ve ever seen in your life. You had no idea monkeys were even capable of this level of greatness. Billions of dollars have now been poured into the economy, extolling the virtues and achievements of this incredible monkey. He can do things better than humans.

    Cheaper than humans.

    Get rid of the humans.

    You don’t need the humans; you have the monkey. Okay, keep a couple of humans for guardrails; not too many. They’re expensive, and the monkey does it better anyway.

    Suddenly it has come out that my monkey, in addition to being incredibly intelligent, has also been committing an escalating number of felonies. And also pooping. So much poop.

    We are now in undefined times. Never before has a monkey committed a crime; our legal systems are completely unprepared to apportion blame to me, the man in the yellow hat who has once again failed to properly supervise his monkey. Who is responsible for this monkey’s shitty, shitty behavior? It is completely unknowable. Clearly, the entire legal system will need to be overhauled to address the concept of a monkey committing crimes.

    Now please excuse me, I just got a call telling me that my teenager stole a car. Unfortunately I have a sneaking suspicion as to who is legally responsible here, and I dislike it. This is especially distressing, as the teenager has never made a cent for me in his life, yet I’m about to spend a boatload on criminal defense attorneys.