Anki/pylib/anki
Damien Elmes c2e8d89fc6
Colpkg fixes (#1722)
* Fix legacy colpkg import; disable v3 import/export; add roundtrip test

The test has revealed we weren't decompressing the media files on v3
import. That's easy to fix, but means all files need decompressing
even when they already exist, which is not ideal - it would be better
to store size/checksum in the metadata instead.

* Switch media and meta to protobuf; re-enable v3 import/export

- Fixed media not being decompressed on import
- The uncompressed size and checksum is now included for each media
entry, so that we can quickly check if a given file needs to be extracted.
We're still just doing a naive size comparison on colpkg import at the
moment, but we may want to use a checksum in the future, and will need
a checksum for apkg imports.
- Checksums can't be efficiently encoded in JSON, so the media list
has been switched to protobuf to reduce the the space requirements.
- The meta file has been switched to protobuf as well, for consistency.
This will mean any colpkg files exported with beta7 will be
unreadable.

* Avoid integer version comparisons

* Re-enable v3 test

* Apply suggestions from code review

Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com>

* Add export_colpkg() method to Collection

More discoverable, and easier to call from unit tests

* Split import/export code out into separate folders

Currently colpkg/*.rs contain some routines that will be useful for
apkg import/export as well; in the future we can refactor them into a
separate file in the parent module.

* Return a proper error when media import fails

This tripped me up when writing the earlier unit test - I had called
the equivalent of import_colpkg()?, and it was returning a string error
that I didn't notice. In practice this should result in the same text
being shown in the UI, but just skips the tooltip.

* Automatically create media folder on import

* Move roundtrip test into separate file; check collection too

* Remove zstd version suffix

Prevents a warning shown each time Rust Analyzer is used to check the
code.

Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com>
2022-03-17 15:11:23 +10:00
..
_backend Colpkg fixes (#1722) 2022-03-17 15:11:23 +10:00
_vendor PEP8 for rest of pylib (#1451) 2021-10-25 14:50:13 +10:00
importing The importer list have a Hook (#1543) 2021-12-09 11:20:29 +10:00
scheduler Move custom study tag and limit gathering+saving into the backend 2022-03-10 16:23:03 +10:00
syncserver updated package scripts 2021-10-28 18:46:45 +10:00
__init__.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
_legacy.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
backend_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
browser.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
BUILD.bazel add missing wheel/helper for darwin-arm64 2021-10-28 18:29:18 +10:00
buildinfo.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
card_rendering_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
cards.py Switch card.original_position to a proto3 optional 2022-03-10 20:48:26 +10:00
cards_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
collection.py Colpkg fixes (#1722) 2022-03-17 15:11:23 +10:00
collection_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
config.py Import submodules directly (#1662) 2022-02-13 13:40:47 +10:00
config_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
consts.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
db.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
dbproxy.py fix cyclic import when importing anki.collection directly 2022-02-15 19:08:29 +10:00
deckconfig_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
decks.py Remove top_deck_id arg in deck_tree() (#1702) 2022-03-02 15:30:32 +10:00
decks_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
errors.py Backend Custom Study (#1600) 2022-01-20 14:25:22 +10:00
exporting.py Backend colpkg exporting (#1719) 2022-03-15 16:48:02 +10:00
find.py Add py3.9 to hooks (#1542) 2021-12-09 09:11:22 +10:00
generic_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
hooks.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
hooks_gen.py fix generated code symlinks again 2022-01-24 16:59:37 +10:00
httpclient.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
i18n_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
lang.py add Odia to language list 2022-02-07 22:31:41 +10:00
latex.py Import submodules directly (#1662) 2022-02-13 13:40:47 +10:00
links_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
media.py Fix html_media_regexps (#1710) 2022-03-07 11:02:30 +10:00
media_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
models.py Import submodules directly (#1662) 2022-02-13 13:40:47 +10:00
notes.py Import submodules directly (#1662) 2022-02-13 13:40:47 +10:00
notes_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
notetypes_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
py.typed mark anki and aqt modules as having typing info 2020-01-13 13:03:37 +10:00
rsbackend.py PEP8 for rest of pylib (#1451) 2021-10-25 14:50:13 +10:00
scheduler_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
search_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
sound.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
stats.py update mypy and mypy-protobuf 2022-01-15 16:04:04 +10:00
stats_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
statsbg.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
stdmodels.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
storage.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
sync.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
sync_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
tags.py Move custom study tag and limit gathering+saving into the backend 2022-03-10 16:23:03 +10:00
tags_pb2.pyi fix generated code symlinks again 2022-01-24 16:59:37 +10:00
template.py Import submodules directly (#1662) 2022-02-13 13:40:47 +10:00
types.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
utils.py Update Python deps, including stable Black 2022-02-25 15:26:16 +10:00
wheel_description.txt move aqt_data into source folder; implement wheel building 2020-11-04 12:14:03 +10:00