An alternative to faster blob storage is to represent the blob's schema in SQL directly.
If most blob updates are incremental you can eliminate a significant amount of storage IO and garbage collection this way.
One SQLite database per blob type with a metadata SQLite db that tracks each. We could leverage powerful AI reasoning models to produce SQLite schemas and virtual table adapters for each encountered JSON schema type.
I'm curious too. The only backup I found on GitHub uses cp; they may use filesystem snapshots or Litestream in production. If it’s Litestream, litestream sync -wait can force a sync after a write and wait for remote replication to complete.
An alternative to faster blob storage is to represent the blob's schema in SQL directly.
If most blob updates are incremental you can eliminate a significant amount of storage IO and garbage collection this way.
One SQLite database per blob type with a metadata SQLite db that tracks each. We could leverage powerful AI reasoning models to produce SQLite schemas and virtual table adapters for each encountered JSON schema type.