6 comments

  • boustrophedon 38 minutes ago
    I might have missed it in the readme, but what hardware did the training take place on? This is really neat.
    • sheepseb 26 minutes ago
      For the world model training I've used an RTX 4060, but for the agent playing the game that was directly on the pico using stream-ac
  • dannyw 2 hours ago
    This is incredibly cool! Fascinating to see looped transformers be necessary for mini-Doom, but not for simpler games.
  • andai 2 hours ago
    Very cool. I didn't know NNs were capable of such determinism, that's very interesting. Is this a similar effect to overfitting? Also, it's the same model for all the games, right?
    • svantana 1 hour ago
      > Also, it's the same model for all the games, right

      It's a bit unclear but it looks like different models, since they all have different parameter and MAC counts. It would be much more impressive with a single world model that could maybe dream up new games inbetween the games in the training set.

    • sheepseb 1 hour ago
      It's a self contained net for each game but the data used for training doesn't overlap with the validation. I think it would be easy to merge them into a single model
  • nater5000 2 hours ago
    Honestly fascinating. I almost feel like you'd be better off avoiding any discussion of the fact that this runs on a Raspberry Pi Pico 2 W because it takes away from the crazy world model capabilities (of course, the fact that this does run on a Pico 2 makes the whole thing much more impressive).

    There's gotta be a good way to incorporate this logic into a larger, more "traditional" game in a meaningful way. Seems like an approach for procedural generation that could solve some of the issues usually found with actual procedural generation.

  • tty_bordel 1 hour ago
    [dead]