mirror of
https://github.com/ankitects/anki.git
synced 2025-09-20 06:52:21 -04:00
5 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Damien Elmes
|
4515c41d2c
|
Backup improvements (#1728)
* Collection needs to be closed prior to backup even when not downgrading * Backups -> BackupLimits * Some improvements to backup_task - backup_inner now returns the error instead of logging it, so that the frontend can discover the issue when they await a backup (or create another one) - start_backup() was acquiring backup_task twice, and if another thread started a backup between the two locks, the task could have been accidentally overwritten without awaiting it * Backups no longer require a collection close - Instead of closing the collection, we ensure there is no active transaction, and flush the WAL to disk. This means the undo history is no longer lost on backup, which will be particularly useful if we add a periodic backup in the future. - Because a close is no longer required, backups are now achieved with a separate command, instead of being included in CloseCollection(). - Full sync no longer requires an extra close+reopen step, and we now wait for the backup to complete before proceeding. - Create a backup before 'check db' * Add File>Create Backup https://forums.ankiweb.net/t/anki-mac-os-no-backup-on-sync/6157 * Defer checkpoint until we know we need it When running periodic backups on a timer, we don't want to be fsync()ing unnecessarily. * Skip backup if modification time has not changed We don't want the user leaving Anki open overnight, and coming back to lots of identical backups. * Periodic backups Creates an automatic backup every 30 minutes if the collection has been modified. If there's a legacy checkpoint active, tries again 5 minutes later. * Switch to a user-configurable backup duration CreateBackup() now uses a simple force argument to determine whether the user's limits should be respected or not, and only potentially destructive ops (full download, check DB) override the user's configured limit. I considered having a separate limit for collection close and automatic backups (eg keeping the previous 5 minute limit for collection close), but that had two downsides: - When the user closes their collection at the end of the day, they'd get a recent backup. When they open the collection the next day, it would get backed up again within 5 minutes, even though not much had changed. - Multiple limits are harder to communicate to users in the UI Some remaining decisions I wasn't 100% sure about: - If force is true but the collection has not been modified, the backup will be skipped. If the user manually deleted their backups without closing Anki, they wouldn't get a new one if the mtime hadn't changed. - Force takes preference over the configured backup interval - should we be ignored the user here, or take no backups at all? Did a sneaky edit of the existing ftl string, as it hasn't been live long. * Move maybe_backup() into Collection * Use a single method for manual and periodic backups When manually creating a backup via the File menu, we no longer make the user wait until the backup completes. As we continue waiting for the backup in the background, if any errors occur, the user will get notified about it fairly quickly. * Show message to user if backup was skipped due to no changes + Don't incorrectly assert a backup will be created on force * Add "automatic" to description * Ensure we backup prior to importing colpkg if collection open The backup doesn't happen when invoked from 'open backup' in the profile screen, which matches Anki's previous behaviour. The user could potentially clobber up to 30 minutes of their work if they exited to the profile screen and restored a backup, but the alternative is we create backups every time a backup is restored, which may happen a number of times if the user is trying various ones. Or we could go back to a separate throttle amount for this case, at the cost of more complexity. * Remove the 0 special case on backup interval; minimum of 5 minutes https://github.com/ankitects/anki/pull/1728#discussion_r830876833 |
||
|
Damien Elmes
|
c2e8d89fc6
|
Colpkg fixes (#1722)
* Fix legacy colpkg import; disable v3 import/export; add roundtrip test The test has revealed we weren't decompressing the media files on v3 import. That's easy to fix, but means all files need decompressing even when they already exist, which is not ideal - it would be better to store size/checksum in the metadata instead. * Switch media and meta to protobuf; re-enable v3 import/export - Fixed media not being decompressed on import - The uncompressed size and checksum is now included for each media entry, so that we can quickly check if a given file needs to be extracted. We're still just doing a naive size comparison on colpkg import at the moment, but we may want to use a checksum in the future, and will need a checksum for apkg imports. - Checksums can't be efficiently encoded in JSON, so the media list has been switched to protobuf to reduce the the space requirements. - The meta file has been switched to protobuf as well, for consistency. This will mean any colpkg files exported with beta7 will be unreadable. * Avoid integer version comparisons * Re-enable v3 test * Apply suggestions from code review Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com> * Add export_colpkg() method to Collection More discoverable, and easier to call from unit tests * Split import/export code out into separate folders Currently colpkg/*.rs contain some routines that will be useful for apkg import/export as well; in the future we can refactor them into a separate file in the parent module. * Return a proper error when media import fails This tripped me up when writing the earlier unit test - I had called the equivalent of import_colpkg()?, and it was returning a string error that I didn't notice. In practice this should result in the same text being shown in the UI, but just skips the tooltip. * Automatically create media folder on import * Move roundtrip test into separate file; check collection too * Remove zstd version suffix Prevents a warning shown each time Rust Analyzer is used to check the code. Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com> |
||
|
RumovZ
|
e759885734
|
Backend colpkg exporting (#1719)
* Implement colpkg exporting on backend * Use exporting logic in backup.rs * Refactor exporting.rs * Add backend function to export collection * Refactor backend/collection.rs * Use backend for colpkg exporting * Don't use default zip compression for media * Add exporting progress * Refactor media file writing * Write dummy collections * Localize dummy collection note * Minimize dummy db size * Use `NamedTempFile::new()` instead of `new_in` * Drop redundant v2 dummy collection * COLLECTION_VERSION -> PACKAGE_VERSION * Split `lock_collection()` into two to drop flag * Expose new colpkg in GUI * Improve dummy collection message * Please type checker * importing-colpkg-too-new -> exporting-... * Compress the media map in the v3 package (dae) On collections with lots of media, it can grow into megabytes. Also return an error in extract_media_file_names(), instead of masking it as an optional. * Store media map as a vector in the v3 package (dae) This compresses better (eg 280kb original, 100kb hashmap, 42kb vec) In the colpkg import case we don't need random access. When importing an apkg, we will need to be able to fetch file data for a given media filename, but the existing map doesn't help us there, as we need filename->index, not index->filename. * Ensure folders in the media dir don't break the file mapping (dae) |
||
|
RumovZ
|
f3c8857421
|
Backups (#1685)
* Add zstd dep
* Implement backend backup with zstd
* Implement backup thinning
* Write backup meta
* Use new file ending anki21b
* Asynchronously backup on collection close in Rust
* Revert "Add zstd dep"
This reverts commit
|
||
|
Damien Elmes
|
18851ace47 | split out cards and collection |