hckrnws
> no matter how small the edit was, the entire file gets rewritten
SQLite doesn't fix this, because you would still need to encrypt the whole file (at least with standard sqlite). If you just encrypted the data in the cells of the table, then you would expose metadata in plaintext.
SQLCipher does provide that, but as mentioned by others, it isn't quite the same thing as sqlite, and is maintained by a different entity.
> The primary issue is that new features cannot be added natively to the XML tree without causing breaking changes for older clients or third-party clients which have not adopted the change yet.
That isn't a limitation of xml, and could also be an issue with sqlite. The real problem here is if clients fail if they encounter tags or attributes they don't recognize. The fix here is for clients to just ignore data it doesn't know about, whether that is xml or sqlite.
The complaints about compatibility between different implementations would be just as bad with sqlite. You would still have some implementations storing data in custom attributes, and others using builtin fields. In fact it could be even worse if separate implementations have diverging schemas for the tables.
> Governance Issues
None of this has anything to do with sqlite vs xml. It is a social issue that could be solved without switching the underlying format, or remain an issue even if it was changed.
SQLite has its own closed-source page-level cipher format, so I don't think this argument makes sense.
https://www.sqlite.org/see/doc/trunk/www/readme.wiki
A weakness though, again, is that this is closed source...
The biggest weakness is the cost. Each client would have to purchase an expensive license. The source code is provided upon purchase though, but essentially destroys the ability to build a client from source due to the compiled binary distribution.
Comment was deleted :(
yeah, that was the point that I was making. Although I do wonder if encrypting the whole file is necessary.
I really doubt it. I have not seen any evidence to suggest that there are irreconcilable issues with SQLCipher's page level encryption over a flat file. Codebook, Enpass, Signal, and a ton of other important clients use it just fine.
That isn't really an option for an open source project like keepass(xc)
I'm not sure why it's a concern that the whole file needs a rewrite.
Naiively perhaps I thought that was helpful with solid state storage because it means that old data is trimmed faster?
It mentions it near the entire file being in memory but that seems a dubious concern. If the key is in memory the entire file can be comprised either way. Nothing can really stop that if you have access to the programs memory.
It's an issue if you have a really big password database, for example because you are storing large attachments in there. Especially if you are also syncing the file over the network. Also if your file is multiple GB, having it all in memory is an issue because of the amount of memory used.
That isn't really how keepass is meant to be used, but apparently people do use it this way.
The schema issues aren't solved by moving to sqlite, or the proposed solution is doable with XML too. I can see the same thing with the attributes (some described it as a shadow schema) happening in an attributes table just as easily. And in my experience relational schemas are a lot harder to modify than a document schema like XML.
EDIT: also you don't need to have just one password vault and I'd say you probably shouldn't, separate entries also assist with separation of concerns. This last adds a little overhead but is a reasonable workaround.
However on the whole I like sqlite for app persistence. It can, however, leak data (forensically) if not managed properly.
> Devising a new schema based on SQLite would allow for current features that are being jerry-rigged into the attributes to have their own real place in the database
Perfectly possible with XML too
> An SQLite based store is one of the most tested and optimal formats for document and application storage
It's optimized for things that largely don't matter for password storage. The testing is admirable, but there's no issue of keepass clients crashing or corrupting data so again, not very relevant (probably because of low concurrency, simple writes etc).
> A switch this big is a major chance to fix the governance structure and align it more with a democratic consortium than a benevolent-dictator-for-life style of project management
You don't need a technical change to solve this. In fact, a fork that would fracture clients is the last thing you need when making governance changes.
> So many quality of life features can be added where the old schema disallowed it
All of the features they list can be achieved with an XML format. The format isn't what's holding them back.
I don't know if this could count as "corrupting": I made the mistake of syncing my keepassxc database to my macbook with finder webdav client (nextcloud backend) it read the file alright but when I tried to write a new secret it helpfully wrote an empty file in place, wiping nextcloud file versions in the process. Thankfully, Nextcloud was smart enough to move the previous file in the trash bin and I could restore it. It seems keepassxc "save" procedure here was to delete the old file and replace it with a new one and something went catastrophically wrong in the process ? Looking at the settings there's is a parameter for this method for particular circumstances but I didn't enable it back then. Now I just have a second database only on this mac synced to icloud and never letting it near my nextcloud again.
I actually had a similar thing happen with gvfs-fuse (Google drive). It was a bug in gvfs using the quota usage of a file as it's file size (because libgdata didn't provide a method to get file size), but I was using a file shared to me so it had zero quota usage.
All of which is to say I would bet on something in the webdav-nextcloud line being at fault instead of keepassxc.
I sync'd this file using various means on different platforms, nextcloud apps on windows using virtual files and android with Storage Access Framework cloud integration, and on linux with rclone and somehow never had such a catastrophic issue
> All of the features they list can be achieved with an XML format.
Not writing the entire database on every save?
Not a problem for XML per se (you can work with byte positions, and with fixed-size blocks to avoid resizing/relocation), but in the case of KDBX there is the issue that it is encrypted as a whole. Not encrypting as a whole, on the other hand, risks leaking more information about the contents, like you can see which parts/how much changed between one update and the next.
Whole-file encryption with authentication is also more tamper-resistant. Basically the only thing an adversary can get away with there is rolling back the entire file to a previous version.
Whereas, any incrementally encrypted format has the additional risk of piecewise manipulations. For example, while SQLCipher authenticates each page, it doesn't authenticate the entire file, allowing for pages to be deleted, reordered, or duplicated (though duplication is easy to detect since each page has its own IV). The end result will generally just be a corrupted database, which will probably get detected by PRAGMA integrity_check, but compared to KDBX, this will not be detected by default nor is it guaranteed to be detectable at all.
Another in place option is AES-encrypted ZIP. ZIP has the benefits that the Directory at the end of the file can also include piecewise metadata for full file validation.
A part of me wonders if the only real upgrade needed for the next "large file" KDBX relative format is from a GZIP stream of the XML plus attachments to a ZIP container of the XML with attachments in some folder structure combined with the choice of a good piecewise (modern) encryption for the ZIP container. (That is taking more cues from 7zip than from classic, now broken password-encrypted ZIP files.)
(Though as someone who tries to keep my KDBX files small, I think I'd still prefer the option of a whole-file encrypted format.)
I actually prefer this. It's how most user-facing file formats work. KDBX in particular is often used in conjunction with syncing software, and I don't want a half changed file to sync and then the connection to be lost. The usual paradigm of "write new file then move and replace over old file" works more safely for atomic changes.
That's not a feature, that's an implementation detail.
Is kdbx broken or has it been causing data loss? I've been using KeepassXC as long as it has existed and no issues for me thus far. If kdbx is not problematic for it's intended use then I think moving to SQLite just makes it more hacker friendly which I have no need for. I have no need for other applications extending the use of my passwords.
If anything, maybe give people the option to export to SQLite and then use that going forward but keep it entirely optional.
Exporting to a relational db is beside the point. You can very easily just export a CSV file and double click it in DB Browser and have something working. Or you could export the XML document and write a quick importer for it. But genuinely what benefit does it provide other than allowing me to hack at my unencrypted db with sql statements? All the benefits that I was referring to are for daily usage of a KeePass client, not for a one off export.
I've glanced over the article - I think using SQLCypher makes sense if it benefits Keepass project internally, meaning makes it easier to implement new features or fix bugs.
I do not see how it would benefits end users in any significant way. Sure, you can look inside the database, how many people need that?
Dumping the database to CSV is not a good backup, schema changes over time, what was dumped from one version of the app would not work for importing into another version if schema changes. Backup it needs a versioned schema format, which would actually look like KDBX format if implemented in XML :-)
But genuinely what benefit does it provide other than allowing me to hack at my unencrypted db with sql statements?
None at all. That is why I am saying it should be entirely optional so most people can just ignore it and those wanting the alternate schema can get their feature request fulfilled.
I don't want to assume you didn't read the article, but this isn't really about the database engine. It's about the shadow schema that has grown up around the format. The database switch would serve as a flag day to unify things. It won't be a permanent fix, nothing at this scale ever is, and we'll probably need another migration in a few decades. Still worth doing.
I read it and I get it but I have never run into problems with KP using its existing schema. The only reason I could see this debate making sense would be if third parties want to integrate with it otherwise it works perfectly fine. And sure if it's worth doing then make it optional like I said. Have an export as SQLite, MySQL, Postgres, Oracle, DB2 but keep it optional given there is no need otherwise. This is solving a perceived problem that does not exist or creating a problem where there isn't one.
As for scale this app is for one person doing one query at a time or saving one password at a time. One person will not be saving millions of passwords or if for some reason they are then they can export to a format that an enterprise solution could take it's place. People have written team / company password managers that can use Oracle, MySQL and Postgres that can track and audit user and team password changes. This is not something that should ever be expected of an individual personal password manager like KeepassXC.
This is just my take but I would rather have the maintainers of KP focus on bugs, usability issues, quality of life enhancements which they have been doing great thus far in my opinion. Forcing a schema migration is just asking for trouble and potentially causing bugs that turn some people away from using it or cause people to lose data if they do not have snapshots of their database. Or if forcing a schema change make for damn sure there are many backups in each format and encourage the users to back up all the files to many external encrypted drives and store some of the encrypted devices off-site. Something everyone should be doing regardless.
I've been syncing my kdbx files with syncthing for many years and I have had a couple of inexplicable conflicts.
Most probable reason would be that I ended up editing on the wrong machine while being offline or having it open on another one, so user error. But I don't know.
In any case, the only 2 moving parts were KeePassXC (I think, not sure when I moved on from KeePass) and syncthing. But most importantly, it was easily fixed every time, no broken file - which points me more into a syncthing diverge.
Otherwise, no, apparently no issues with kdx itself, or I wouldn't have been able to merge these problems. (which brings me to my pet peeve, a good working merge is missing...)
Most probable reason would be that I ended up editing on the wrong machine while being offline or having it open on another one, so user error. But I don't know.
File locks or attempting to overwrite and open database would be a problem regardless of schema format or what application is holding the database file open for writing. This should be expected behavior and one would hope the app would detect this and seqfault / core dump without corrupting the original / previous version of the database file.
Yes, I just felt I had to specify further besides "I encountered problems that didn't seem to be inherent problems" but "no problems" would have been dishonest.
I think that's where I see the most concerns with this proposal: XML is already an extensible format (that's what the X stands for) and XML schema changes should be simpler than needing to run SQL migrations (alter/drop/etc). A switch to SQL doesn't guarantee that schema extensions better align as standards or allow for easier schema modifications. I think it more extends the risk that schema components get ossified and harder to extend.
That problem that the current schema doesn't have enough ways to declare custom memory-protected fields outside of user-facing attributes seems just as likely to be replicated and maybe worsened in an SQL schema.
Changing the database engine doesn't fix the architecture problems nor schema migration problems. It's certainly a good time to reevaluate the architecture problems and the schema migration problems. But the huge caution I'd suggest here would be look at the ossification complaints about the current XML schema and expect SQL migrations to be worse and plan for worse multi-schema operations and intenser ossification. (Especially because these files are expected to live in a large multi-vendor ecosystem, SQLite schema migration management is going to be much worse than any XML schema management.)
Maybe it's irrational, and I cannot actually justify it (and of course safe writing is of primary importance), but somehow rewriting the whole file feels like a good thing for a secrets storage. Updating only part of a file obviously reveals something, even though it probably shouldn't matter if it doesn't reveal anything useful. But the default mode of thinking is we can never assume the leaked information cannot be used somehow.
KDBX seems unique in this threat model out of the major providers. The cloud ones all use a relational DB, while major local ones like Enpass & Codebook just use a SQLCipher store. I wish someone with some real experience here would chime in: What metadata does a SQLCipher DB leak that a KDBX file does not? I mention that both of them obviously reveal the size of your vault to an attacker (w/ KDBX reporting the size more accurately, ultimately irrelevant), which is pragmatically unavoidable information leakage.
I do use it, and rewriting the whole file annoys me especially when the storage is not local and the database contains sizable blobs. For storing passwords and short secrets, it makes little to no difference but if I have 10 1MB blobs stored in there, it becomes upsetting.
Well, yes, this is what OP is saying, and I'm not arguing against that. However, this is not what *.kdbx was designed for. And I am only talking about what cryptographically changes for the intended use case if we encrypt every page separately.
10 1MB blobs is nothing on modern hardware.
The actual encryption itself is relatively quick, I don't mind that. It is the re-upload of the whole file that is my concern.
> So rather than risking sync issues uploading your 20MiB KDBX file on every minor change, you can upload just the 4KiB or so comprising that data.
Why is your KDBX file 20 MiB? It seems you are storing something that is not actually a good fit for a password manager, and expecting the entire world to change around you instead of storing those files in a more appropriate place.
We have a single kdbx with nearly 7000 entries. It is about 45MB at the moment with very few text docs in it. It once got to over 100MB when I found people using it as a doc store but it had a bit of a clear out a few years back.
"The reasonable man adapts himself to the world; the unreasonable man persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man."
My kdbx is only 173 KB, but I don't hate this idea.
I don't use KP, but I have a pdf for my floor safe in my password manager. I only open it a few times per year and I need more than just the combination, I need instructions on whether the first number is cw or ccw. While I could no doubt look it up on the internet every time, I was fearful that the user's manual might some day disappear from the internet. Some things that aren't obviously passwords still belong in a password manager.
In my mental model, the PDF is not a secret and can be stored anywhere -- encrypted, if desired, but it sounds like a public document.
The safe combination is a secret, and obviously belongs in secret storage.
In this specific example, if I had trouble remembering whether the first number of the combination was reached via cw or ccw rotations, I'd include that in the secret, e.g. "cw34-12-22-45".
(Some safe combinations require multiple rotations. I unintentionally became the owner of one that is something like "cw3x34-ccw2x12-cw5x22-ccw2x45". I still can't open it actually, but that pattern is what the Internet tells me. :)
>In my mental model, the PDF is not a secret and can be stored anywhere -- encrypted, if desired, but it sounds like a public document.
Sure, but I will need it at the same time and for the same reason as the combination which does belong in the password manager. To store it separately would be more difficult.
In any event, it's not large. I seem to remember it is only in the low hundreds of kilobytes. But there is occasion for such things.
>I unintentionally became the owner of one that is something like "cw3x34-ccw2x12-cw5x22-ccw2x45"
Mine is similar. I even have the cw/ccw in with the numbers, but that alone never seems to get it for me... sometimes I do need to look at the pdf. I simply don't get enough practice to do it from memory.
With cases like these, is it possible for you to simply copy the important text data into a note, or do you absolutely need the full pdf? Most attachments can be reduced down to their barest text form to avoid bloating the db.
Were that me (I used KP), it would be in a different kdbx file. This is one of the benefits of KP, I have about 8 different vaults for various things. I don't like putting my eggs in one basket.
You lose the convenience of one file though. In this case you might as well use a purpose built encryption tool rather than force KeePass into this usecase. A VeraCrypt container or encrypted overlay filesystem are a significant performance and UX upgrade since you are already willing to concede managing one file.
It's also possible to create dedicate entries for each of the other KeePass vaults and set the URL field and password to the respective paths (i.e., "kdbx://PATH/TO/OTHER/DATABASE.kdbx") and passwords, then you can simply double click on the URL field to automatically open and unlock the other vaults.
The URL field in KeePass has lots of convenient features [0], but unfortunately they're quite "obscure" and not very discoverable.
That's true regarding the one file convenience but from another angle it's a separation of concerns, especially considering it's a pdf accessed very occasionally.
I maybe half agree with you about the encrypted overlay filesystem but only in respect to files, not passwords though. I tend not to keep files in KP, if I need a singular encrypted file I'd probably 7zip it (7z format) with a password and encrypt the filenames. The password goes in KP as does the location.
Yes we agree. Keep the passwords in the password manager and keep the files in the EOFS. If you need true seperation between those files, just make different containers or FS for them.
I'm working on an alternative to KeePass/KeepassXC called Lockstep - it is local-first password manager that supports sync natively.
It uses SQLCypher as local data store and keeps KeePass-compatible data model - supports import of .kdbx database.
SQLCypher and different schema used in Lockstep alone did not solve any problems that I have with Keepass. Those problems are sync and sharing.
Solving sync and sharing cannot be done on whole database file level, as it implemented now in KeePass. Changes need to be tracked at the password record level, all changes need to persisted as operations log and that log needs to be distributed across devices.
The above means writing a whole protocol, and that's a lot more work than changing local storage.
The project is pre-alfa https://github.com/lockstepvault-hq/lockstep
Just create a new password manager using SQLCipher then? If it is good enough people will use it.
It is weird when people wants to change something that works just great.
KeePass have served me well for years on Windows, Mac, Android, Linux using Dropbox and Syncthing as storage. Don't mess with it.
SQLite seems like an odd dependency for a system which ultimately just journals events like "on 2026-02-24T19:36Z, entry 791 was created with username larry7 and password letmin" or "on 2026-02-24T20:51Z, the password for entry 791 became letmein2".
This is a reductionist take on what password managers actually need to do. A journal log is actually worse than the XML in this regard in that the number of events far exceeds the actual relevant relations. Also, adding an attachment/icon his no longer possible. Simple group associations and tags also become a complex nightmare in a log based system, whereas in SQLite they are super easy relationships. Relating passkeys, totp strings, icons, extensive history, etc all to one entry becomes too much of a task for that kind of format to handle.
It's actually very simple to read a journal to determine the full current state of the database. It's potentially even faster than parsing XML, yielding exactly the same in-memory structure. Your computer will do it faster than you can cancel the operation. This also gives you history for free, rather than as nested entries (though you might have to write a log flattener to prune old journal entries). The operations required to parse log entries like "icon 4 is now this data" and "entry 791 now uses icon 4" are minuscule. There is no complexity, no nightmare, no loss of relations.
I am not advocating for KeePass switching to a journal, however, because the current ecosystem is fine.
My problems with KeePass are rather that its development model is much like SQLite: a pretty closed model, with source published regularly, but no clear way of code contribution, which is sad because it has some UI regressions recently which I'd be happy to fix, if contribution was lower friction.
> many users have databases that fall in the range of 10-100MiB
> use one of the many other ways to operate the database as a single file and be on your way
Don't you still have to download and upload that 100MB pretty often? I feel like password databases get modified every few days, not every few months.
Moreover, if an app really wants to optimize the file-not-modified case for people who don't modify it often, can't it just internally cache an encrypted version of the database in another format -- in SQLite, even! -- and use that when the checksum/file time/whatever matches what they expect?
I feel like a client that actually cares about the user should tell them how much more expensive the file gets with each entry being added: "Hey, based on your recent download/upload speeds on this device, embedding this 1MiB file would add a 5-second delay." Switching the entire world from XML to SQLite seems like it completely misses the fundamental issue.
My KeePass database is pretty stable at this point. I would say edits happen every few weeks, if that. My edit date is ~2 weeks ago, and it was because I was logging into an account I hadn't touched in a few years. Nothing really changes much in my personal database. I add stuff occasionally, but new accounts are few and far between, and so are password changes. I'm not sure what would be changed every few days for an individual.
Without being simplistic - isn't this a case of "right tool for right job"?
You can pay me to build you SAP in Excel, but you really shouldn't!
If you have a 100mb password database, should you be moving to a more scalable solution?
I think a relational DB is the best tool for the job!
I argue more for the use case of "secrets manager" than solely password manager. People store TOTP seeds and Passkeys and a myriad attachments. A SQLite store would better facilitate this, as not everything has to be in memory all the time. This is only with regard to the size constraint, there are so many other benefits to be had from a formal specification of the schema to truly future-proof the standard.
SQLCipher is not SQLite.
A key aspect of SQLite's development model is its proprietary test suite. As far as I can tell, the SQLCipher developers do not have access to those tests.
This is not to say they aren't doing the best with what they've got, but SQLCipher is a fork of SQLite, and the scope of the changes they must make, no matter how conservatively they try to make them, should lead to a full re-test of the entire product, which they cannot do.
EDIT: I don't want this to come off as spreading FUD. The SQLCipher developers do a good job of laying out their development methodology, the relative (un-)testability of their product vs. SQLite, and other tradeoffs pretty well in the repository's README: https://github.com/sqlcipher/sqlcipher?tab=readme-ov-file#sq...
The keepass ecosystem is comprised of a dozen implementations of the KDB(X) file spec. Some are better than others.
I built KeePass Tusk back in 2018, for example. This would kill the project and abandon 30K users without a rewrite of the JS engine (there are several now!)
I agree with you that KDBX sucks, but at this point a keepass based on SQLite would be keepass in name only, a new password manager to migrate to.
Breaking format changes is not such a major issue, they happened before: kdb → pre-2.08 kdbx → kdbx3 → kdbx4. If the new format is worth it, popular apps will adopt it within a few years — while still supporting older formats. Users would just stick with their current format until the ecosystem catches up, as it happened with KDBX and KDBX4.
Good to see you in here! You make a great point, historically the breaking changes have not really affected users. You kept your db as is, and it would get migrated if you wanted to use new features. A friendly warning on open with a prompt to migrate to unlock new features (after gaining ecosystem traction) would be reassuring to users. On a more technical note, is there anything on your end with KeePassium that would be greatly improved, especially regarding potential improvements to auto-fill memory usage?
Nothing major, mostly UX improvements that could be defined as part of the new format. For instance, custom ordering of entry fields is not possible now because existing apps just sort them alphabetically on save. Multi-URL storage is basically KP2A's workaround adopted as-is by other apps.
That said, most of the concerns raised by the article — outdated schema, inefficiencies, governance issues — call for a new iteration of database format, but not necessarily SQLite. However, we would still be debating how to represent entry templates and how to accommodate features that stretch format's initial assumptions (be it multi-URLs or smart groups). We may still discover that passkeys need more fields than initially foreseen. Then someone would come up with item-level access rights scheme. Then something else.
All of these are already possible with XML+Gzip, just as much as with SQLite/SQLCipher. The main advantage of the latter is the standard, multi-platform library with a permissive license, instead of KDBX' specialized parsing. Switching to SQLite would probably lower the entry barrier for new apps. Which would be a good thing on the surface (more choice), but could end up with the same devil-in-details bedlam as the status quo.
I really appreciate your commentary here! I understand that SQLite would not automatically solve the politics issue. As ctoth mentioned, it would serve as a "flag day" for the KeePass community to hopefully cover everyone's bases in a formal manner. Schema versioning and evolution is way more up SQLite's alley. An SQLite schema has referential integrity which an XML schema lacks, making it harder to misuse and contort. It is also far simpler to modify a query in the event passkey storage need to be changed than it is to modify the parser.
The lower barrier to entry probably also reduces the number of catastrophic parsing mistakes a developer can make. This is a net positive gain for the wider ecosystem of external tools which do not have to re-implement the whole parser. Every language has a great SQLite library, the same cannot be said of KDBX.
As for autofill memory limitations, this is largely an implementation detail: just process data stream in small chunks, that's it.
Unfortunately, KeePassium's data layer was designed in the times of iOS 11, before AutoFill became a thing. So I chose the easier path of loading and processing the whole file at once. This made sense for 10-20 MB databases on iPhones with 2 GB of RAM. By the time the mistake became obvious, it was much harder to switch to streamed processing, especially with a long queue of lower-hanging feature requests.
This class of mistake could have been wholly avoided if the format was SQLite based because there is no assumption that everything must be loaded into memory. Of course I defer expertise to you, but I've had a far better time with SQLite than I've ever had with XML parsing (OFX/QBO still bothers me...)
Hey I’ve seen your project before! You bring up a super good point that I was thinking of when I brought up the idea that the extension should be renamed to .kp . Really the only reason to keep the KeePass name would be branding, people know and trust it. Honestly my dream password manager is essentially something that uses the CodeBook (by SQLCipher authors) storage format, but with the nice trustworthy, FOSS KeePass ecosystem chrome on top of it (keepassxc<-browser>, keepassium, etc).
I think that if things are bursting at the seams this is a good idea. But we’ve added Passkeys already, and the custom metadata ship has sailed. This is the kind of initiative I could see taking off as a solution to Passkeys, but it doesn’t represent a worthwhile investment for me now.
I’m at 1.6Mb, and with the frequency by which I update entries, the cost of data migration is relatively high compared to the data cost.
I'd be glad if someone could enlighten me on why the whole file needs to be encrypted.
What issues does storing an encrypted value (password, metadata, etc) associated with a particular key (let's say website name) have? (apart from leaking the fact that that file has a entry associated with that site)
> apart from leaking the fact that that file has a entry associated with that site
That is a big fat leak right there
If you are serious about this proposal, one way to move forwards is to make tool that converts kdbx <-> sqlite. If you can't roundtrip that conversion perfectly then the idea is dead on arrival.
Per the article:
> The migration process would also be frictionless for users, it is a simple data map between probably the two easiest formats of all time.
I cannot imagine how you could mess this up. The developers already implement numerous export formats. The migration is the easiest part. The actual implementation of a new data format into the codebase and all the new security and robustness testing is the difficult part.
>Important to note is that every time a KDBX file is updated, no matter how small the edit was, the entire file gets rewritten.
This seems like an implementation issue rather than a format/spec issue.
Because KDBX is a gzipped and encrypted stream, this is actually fundamentally an issue with the spec itself. A client must re-encrypt and compress the file prior to writing because a mere append operation is not possible. SQLite solves this issue by allowing you to write with page level granularity rather than being forced to dump the whole file for a single tiny change!
Not encrypting everything as a whole however leaks information about the structure and what/how much was changed between two versions of the file.
Couldn't they simply switch to zip files? Those have an index and allow opening individual files within the archive without reading the whole thing.
Also, I don't understand how using XML makes for a brittle schema and how SQL would solve it. If clients choke on unexpected XML elements, they could also do a "SELECT *" in SQL and choke on unexpected columns. And the problem with people adding different attributes seems like just the thing XML namespaces was designed for.
It's a single XML file. Zip sounds like the worst of both worlds. You would need a new schema that had individual files at some level (probably at the "row level.") The article mentions SQLCipher which allows encrypting individual values separately with different keys. Using different keys for different parts of a kdbx sounds ridiculous, but I could totally imagine each row being encrypted with a compound key - a database-level key and a row-level key, or using PKI with a hardware token so that you don't need to decrypt the whole row to read a single field, and a passive observer with access to the machine's memory can't gain access to secrets the user didn't explicitly request.
ZIP files can have block-like relatives to the SQLite page. It could still be a single XML file and have piecewise encryption in a way that change saving doesn't require an entire file rewrite, just the blocks that changed and the updated "File Directory" at the end of the ZIP file.
Though there would be opportunity to use more of the ZIP "folder structure" especially for binary attachments and icons, it wouldn't necessarily be "required", especially not for a first pass.
(That said there are security benefits to whole file encryption over piecewise encryption and it should probably be an option whether or not you want in-place saves with piecewise encryption or whole file replacement with whole file encryption.)
A ZIP file with solid encryption (i.e., the archive is encrypted as a single whole) has all of the same tradeoffs as a KDBX file as far as incremental updates are concerned.
A ZIP file with incremental encryption (i.e., each file is individually encrypted as a separate item) has its own problems. Notably: the file names are exposed (though this can be mitigated), the file metadata is not authenticated, and the central directory is not authenticated. So sure, you can read that index, but you can't trust it, so what good is it doing? Also, to support incremental updates, you'd either have to keep all the old versions of a file around, or else remove them and end up rewriting most/all of the archive anyway. It's frankly just not a very good format.
> SQLite solves this issue by allowing you to write with page level granularity rather than being forced to dump the whole file for a single tiny change!
Smaller ideas that would address this: add support for non-CBC encryption modes, tweak/disable the compression so that small changes require less rewriting.
I see, thanks. Have you considered moving to Bitwarden?
Yes actually, I have personally onboarded several people to BitWarden! (Including manually migrating their accounts, setting passwords and passkeys, etc) However, I dislike the centralization and lack of control. There was a paper discussed within the past couple days on HN about what a malicious BitWarden server was capable of, despite BitWarden's marketing. I believe BitWarden's team responded promptly and appropriately to the research.
VaultWarden is a good compromise (offers the choice of SQLite or Postgres under the hood), putting you in charge of the primary server, but it is honestly overkill for a single personal user compared to a kdbx file on a webdav share.
> overkill for a single personal user compared to a kdbx file on a webdav share.
Maybe. I'm looking into VaultWarden for my personal passwords because keeping a KBDX file up to date on iOS is painful (without a corporate cloud backing).
Hey I’m with you here actually. Synctrain on iOS makes it bearable, and actually wakes itself up periodically in the background to do a sync. It’s not as good as it could be, but far better than the alternatives. Otherwise you can spin up WebDAV and direct connect via keepassium. Both work well in my usage.
> Synctrain
Good tip, I'm going to check that out!
It's a file in the 10-500kB and passwords are read way more often than added.
If it's even tracked as an implementation issue, it probably ranks very low and fixing this requires a lot of care not to screw up things with the safety and feature rollout.
That's probably a security feature.
i would say it’s the correct implementation as you can’t edit atomically. write and move is what you have to do
Exactly. As such, people in the thread with huge dbs have a poor UX when they really do not need to. Also, people who have experienced corruption issues on network storage due to the default saving method (I personally have never experienced this).
It's not an issue, write-new-and-swap is probably the best approach for this type of application.
You need format supporting append/edit first before you can implement it "right"
Just checking the size of my .kdbx file: under 100K.
130KB, I don't store anything in it but username/passwords and a couple of passkeys.
> My cursory (lol, get it?) understanding may be incorrect.
I don't get it. Is this a reference to database cursors? Or is it implying that the blog post was AI generated?
Fortunately for everyone, database cursors. Took me a second to even realize how this could be related to AI. (I’ve never used cursor!)
Total meh from me, an end user. User of KeePass since at least 2015, I've written end-user guides, contributed to the main documentation, evangelize it to my family and friends when they have security questions.
I store every single important piece of info in my KeePass database. It stores ALL of my passwords, my SSN, credit cards, my health information, even some weird stuff like my vehicle maintainence records and whatnot. My KDBX file currently sits at 466K. Size is not a particularly compelling reason. Hate to be that guy, but if your database is much larger than that - you're probably doing it wrong.
Newer features like TOTP and passkeys are likewise not a concern for me. What did KeePassXC do when TOTP came around? They stored the relevant data in the attributes, and added a UI around it. It even works with my Steam TOTP, which is a nonstandard implementation. I haven't looked into it, but I imagine they did the same thing with passkeys. I don't see why this couldn't continue to be the paradigm they use. I don't use attributes at all - I haven't needed to, the notes section work great - but I do appreciate being able to look into the "raw data" of attributes quite easily, from within the UI.
If KeePass were being developed from scratch today, or if the developers of the various projects collectively really, really want to switch to a SQLite system of their own volition. Then sure, SQLite. I'm not going to ask them to do that now though.
---
On a separate note, an unfufilled niche that I have though, if anyone's looking for ideas. My secure password storage is a solved problem, KeePass is cross platform, easy to use, and very secure. What remains a problem is secure notes. I want to be able to write markdown (`.md`) documents, add photos and PDFs, then save it to a secure, encrypted folder somewhere. Doesn't need the same security posture as KeePass, but I don't want to leak metadata like file names.
Obsidian - my current notes app - is good from a usability standpoint, but it's not exactly secure. I could pair it up with Veracrypt, but that's a pain from a usability standpoint, and I don't trust my OS to keep the mounted Veracrypt volume contents a secret. Whatever the solution is, it must have a GPL license, or else I'm not going to trust it - from a long-term viability standpoint more than anything else.
If anyone has any suggestions here, would love to hear them.
Attributes are meant to be user facing, and are super useful for all the assorted info that you can use during autofill steps. I primarily use this to autofill card information with autotype.
Cryptomator checks all the requirements. It is a cross platform, GPL'd, encrypted overlay filesystem which you can put anything in, not just markdown docs. Just unlock it and point your notes app to it. Lock when done.
> ...my KeePass database. It stores ALL of my passwords, my SSN, credit cards, my health information, even some weird stuff like my vehicle maintainence records and whatnot. My KDBX file currently sits at 466K.
I thought I had a lot of info in my *.kdbx file - not just passwords - mine is a mere 80kb, though i do keep medical in a 'note to self' on signal.
Note to self is quite a bit more ephemeral in practice than keeping it in your kdbx file. Any number of things could cause you to lose your chats with Signal.
Some decent arguments for development concerns, but the users will certainly never notice.
For 10 years I've managed a family of 4 in a single KeePass db. Unique passwords across all accounts, random passwords instead of PII for "security questions", fake DOBs, and all other random security related stuff. Never had a single issue. Everything happens instantly as far as I can tell. And the file is 67kb.
My work dbs, separate only because they should be, are much smaller and simpler.
I love KeePass. Changing everything under the hood probably only has potential to make pain for the user. Best case is that nothing is made noticeably worse. Doesn't seem worth the risk.
Of course, the technical backbone of a product is rarely a user concern. But, changing the store allows for new design decisions when making the schema that are not tied to the old (IMO) technical debt of the KDBX format.
Users have wanted proper custom entry templates for years in order to input their card details into the manager. BitWarden, CodeBook, and others all have this built in without needing to fiddle with attributes and autofill to make it work.
I think a major change like this is worth it when we consider the KeePass of the next decade or two. This is genuinely my most important file by a longshot. Basing it on SQLite is just the right move for future proofing.
> KeePass has long been the gold standard and darling of the tech world, earned through its unrelenting commitment to security, stability, and data sovereignty.
Eh? I always thought of pass[1] in that role.
> Devising a new schema based on SQLite would allow for current features that are being jerry-rigged into the attributes to have their own real place in the database, rather than clogging the user-facing fields. It also ensures that if in the future, some weird authentication method were to come out, no breaking changes would be needed. You simply would add a table to accommodate it, and old clients would simply not support the feature and just load the database without it. Of course, a warning would be shown to the user if somehow their database uses new features on an old client.
Using a relational database does not solve this problem at all. It doesn’t even address it at all.
The original problem is you have multiple implementations defining their own data model. Whether the backend is a file, a database, or a post-it note, that doesn’t work.
Just as you can ignore tables in a database, you can ignore attributes in XML.
My current issue with pass is my difficulty with migrating my private GPG keys to new devices. Makes the experience so much more worse IMO. (I've been using pass for 6 years at this point)
KeePass is for sure better suited for this usecase. There is far less to keep track of, and the unlock mechanism and data are tied together. I've also had inexplicable issues migrating GPG keys cross-platform to where I just do not bother anymore. Ssh/age/minisign just work for my use cases.
I don't think given the file size it is all that relevant.
I mean if I wanted to start new password manager right now it would be a good choice to "just use SQLite" but for existing solution backward compatibility is far more important
Crafted by Rajat
Source Code