Commit graph

54 commits

Author SHA1 Message Date
RumovZ
4c79a1d969 Merge branch 'main' into apkg 2022-04-13 09:50:27 +02:00
RumovZ
71e6292f8d Maybe upgrade scheduler of apkg 2022-04-11 09:10:08 +02:00
RumovZ
251bfc4920 Factor out media importing 2022-04-11 08:44:18 +02:00
RumovZ
fe878dcdc8 Factor out note importing 2022-04-11 08:28:10 +02:00
RumovZ
a7278836d4 Factor out card importing
Also handle missing parents .
2022-04-10 16:07:35 +02:00
RumovZ
a402879e72 Refactor card and revlog importing 2022-04-10 11:03:46 +02:00
RumovZ
8348240f80 Factor out card importing 2022-04-10 10:36:42 +02:00
RumovZ
e1899152be Fix importing cards/revlog
Card ids are manually uniquified.
2022-04-10 09:22:41 +02:00
RumovZ
126a92c239 Handle card due dates and original due/did 2022-04-09 23:42:05 +02:00
RumovZ
a528106af0 Keep source ids of imported deck configs (or skip) 2022-04-09 15:52:23 +02:00
RumovZ
7798b78c7d Wrap apkg import in transaction 2022-04-09 15:35:16 +02:00
RumovZ
1ba5444d7d Make adding notetype with id undoable 2022-04-09 12:31:23 +02:00
RumovZ
2a15ba5404 Try to keep source ids of imported notes 2022-04-09 11:40:36 +02:00
RumovZ
5a76d2211a Keep source ids of imported revlog (or skip) 2022-04-09 10:17:28 +02:00
RumovZ
a0604a2e51 Keep source id of imported cards (or skip) 2022-04-09 09:49:36 +02:00
RumovZ
726438d1c5 Add roundtrip test for apkg (stub) 2022-04-08 22:09:45 +02:00
RumovZ
d6134748f2 Fix signature of import_apkg() 2022-04-08 18:57:01 +02:00
RumovZ
468c0286f9 Use dyn instead of generic for media_fn
Otherwise, would have to pass None with type annotation in the default
case.
2022-04-08 18:46:40 +02:00
RumovZ
08791e24fc Add import_cards(), import_revlog() 2022-04-08 18:12:37 +02:00
RumovZ
ffdd8b741e Add import_deck_configs() 2022-04-08 10:52:12 +02:00
RumovZ
7520afd9c5 decks_configs → deck_configs 2022-04-08 10:41:33 +02:00
RumovZ
5cf1705784 Add import_decks() 2022-04-08 10:34:49 +02:00
RumovZ
48bc6627b1 Note id, not guid of conflicting notes 2022-04-07 21:08:17 +02:00
RumovZ
94a6cdd6ed Add notes immediately instaed of preparing 2022-04-07 20:49:42 +02:00
RumovZ
b9c19273fc Move target_col into Context 2022-04-07 19:04:24 +02:00
RumovZ
9527322231 Add notetypes immediately instaed of preparing 2022-04-07 18:58:11 +02:00
RumovZ
7583a7e6b3 Prepare media based on checksums
- Ensure all existing media files are hashed.
- Hash incoming files during preparation to detect conflicts.
- Uniquify names of conflicting files with hash (not notetype id).
- Mark media files as used while importing notes.
- Finally copy used media.
2022-04-07 12:45:02 +02:00
RumovZ
a0085e7fd4 Add SafeMediaEntry for deserialized MediaEntries 2022-04-06 23:21:32 +02:00
RumovZ
efde7c7acc Refactor import-export/package
- Move media and meta code into appropriate modules.
- Normalize/check for normalization when deserializing media entries.
2022-04-06 18:02:02 +02:00
RumovZ
80dc3ae99e Fix due dates in days for apkg export 2022-04-06 12:31:29 +02:00
RumovZ
6836da07ec Start on apkg importing on backend 2022-04-05 18:15:51 +02:00
Damien Elmes
4673fb5fe4 Colpkg import now marks collection as modified
Addresses a corner case where a backup is imported into an
unchanged collection, resulting in the sync indicator showing a full
sync is required, which doesn't go away as syncing thinks no changes
are required.

An alternative way to solve this would be to reverse the order of checks
in the syncing code. It would have the advantage of retaining the
modification time of the backup, but any action like clicking on a deck
would cause it to be updated anyway, so I'm not sure that buys us much.
2022-04-03 16:07:39 +10:00
RumovZ
fce797cb47 search_notes_cards_into_table →
search_cards_of_notes_into_table
2022-03-31 18:23:18 +02:00
RumovZ
4aa5ee5fb4 Ignore ascii case when filtering system tags 2022-03-31 18:20:11 +02:00
RumovZ
fbc21da913 ExportData → ExchangeData 2022-03-31 12:53:36 +02:00
RumovZ
71670a065d Use schedule_as_new() to reset cards 2022-03-31 10:39:52 +02:00
RumovZ
bd14ccf2a3 Merge remote-tracking branch 'upstream/HEAD' into apkg 2022-03-31 10:28:16 +02:00
RumovZ
97c9dd4c18 Use temporary tables for gathering
export_apkg() now accepts a search instead of a deck id. Decks are
gathered according to the matched notes' cards.
2022-03-31 10:24:09 +02:00
Damien Elmes
9d64afc7bc Handle gaps in media in colpkg imports
Our old Python code was also skipping numbers when it encountered a
directory, leading to a colpkg that couldn't be imported with our new
code.
2022-03-31 13:32:38 +10:00
RumovZ
d1dd0586bd Only store file folder once in MediaIter 2022-03-30 21:37:02 +02:00
RumovZ
a46026f283 Accept func to produce MediaIter in export_apkg() 2022-03-30 21:20:34 +02:00
RumovZ
112ad118ab Reuse filename_is_safe/check_filename_safe() 2022-03-30 19:33:53 +02:00
RumovZ
84c5ac8a0a Gather export data instead of copying directly 2022-03-29 16:47:57 +02:00
RumovZ
6269a88cd3 Filter out missing media-paths at write time 2022-03-28 12:42:02 +02:00
RumovZ
566973146f Add apkg export on backend 2022-03-27 10:42:26 +02:00
Damien Elmes
6735b23e5b Force a schema change on colpkg import
If we don't force a full sync when restoring, any items that were added
since the backup may have already been sent to AnkiWeb, and they
won't have deletion records. After the user restores from a backup,
they'll end up in a state where their local and AnkiWeb collections
differ, and the changes will not sync. The old backup code forced a schema
change, but we weren't previously doing it via File>Import.
2022-03-22 13:10:32 +10:00
Damien Elmes
d100f7a2c8 Don't fsync media files on import
I was seeing import speeds of only 10-20 files a second before this
change.
2022-03-19 23:31:11 +10:00
RumovZ
16fe18d033
Refactor export-import code and resolve fixmes (#1723)
* Write media files in chunks

* Test media file writing

* Add iter `ReadDirFiles`

* Remove ImportMediaError, fail fatally instead

Partially reverts commit f8ed4d89ba.

* Compare hashes of media files to be restored

* Improve `MediaCopier::copy()`

* Restore media files atomically with tempfile

* Make downgrade flag an enum

* Remove SchemaVersion::Latest in favour of Option

* Remove sha1 comparison again

* Remove unnecessary repr(u8) (dae)
2022-03-18 19:31:55 +10:00
Damien Elmes
bf8e70c70f Ensure partial colpkg file removed if export fails 2022-03-17 20:58:36 +10:00
Damien Elmes
f8ed4d89ba Add new error variant to frontend; ensure errors are mapped appropriately 2022-03-17 20:58:35 +10:00