Backups Aren't Simple

(filipovski.net)

31 points | by afilipovski 3 hours ago

5 comments

  • innocent_name 2 minutes ago
    >Motivated by our success in developing this solution, we try to use it to backup the homelab with its 10 Docker containers. But later we find out from logs on the individual machines that backups are failing. The reason being that many Docker containers like to create root-owned files, and if you’re not careful you can create a cronjob running as the default user.

    and a privesc also.

  • bloomingeek 38 minutes ago
    "encrypted, chunk-level deduplicated, GFS-rotated, point-in-time archived, cloud, 3-2-1 backup solution" is now my newest password, no commas. (Don't tell anyone!)
    • rileymat2 30 minutes ago
      Change the o in rotated to a 0 and you might have a secure password.
  • readthenotes1 41 minutes ago
    "Obviously, none of this is worth anything if you don’t actually test restores."

    One of my teachers claimed that contrary to common behavior, no one actually valued backups; it was restores that were worth paying for.

    • gerdesj 22 minutes ago
      Try running a business on that basis - where do the restores come from? 8)

      We charge our customers for backup storage by volume and that includes an annual demonstration of recovery. We fire up some or all of their systems on our gear in isolation and show that they are reasonably functional.

      Your teacher's comment is riffing on the well trod lines of: "heights don't kill people, it's depths that kill people", which is all about perspective. This is is not related to "guns don't kill people ..." which is about agency.

      • bitwize 10 minutes ago
        "It's not the fall that kills you, it's the sudden stop at the end."
    • TheCondor 26 minutes ago
      It is a special sensation when you go to your hand rolled backup "solution" to make some simple recoveries and start finding problems. It is a complex mix of not dread but disgust, sick to your stomach, self-anger, and loss. I can only image what that's like with like all of your children's baby pictures being the lost material. Borg and restic are fantastic, it still seems like some sort of dummy proof mode might be desirable, all the features in the world and they have some some reasonably sane policy or something that includes verifications.

      On the flip side of that though, really and truly, I've restored a laptop from a timemachine backup and it came back damn nearly identical and that was magical.

  • PunchyHamster 39 minutes ago
    So is apparently making readable site.

    Software like https://restic.net/ does a good job. Few choices of backend

    Accepts stuff from pipe too so you can just pipe mysqldump or pg_dumpall without intermediate files

    Very decent options for checking repo integrity, personally (well, at work too, we use it on few hundred user machines and servers) I also added "roll a dice for backup and try to restore it" test script to make sure it is working

    Decent deduplication too so "store year's worth of weekly snapshot" is very sensible strategy once you exclude the "constantly changing and useless" (caches etc.) files out of it

    can mount FUSE directory with all backups on sensible OSes.

    https://kopia.io/ does the same +GUI but a bit worse on CLI front (it insists on keeping local config which makes scripting a bit more involved, but not much)

    • bt1a 9 minutes ago
      importantly, does your die have a side that corresponds to a multi-continental cataclysmic natural disaster of magnitude only known unknown to pre-recorded-history
  • pizzafeelsright 9 minutes ago
    I don't want backups. I want synced duplicated data in geographically separated by shared credentials.