Anki/pylib/anki
Damien Elmes cf45cbf429
Rework syncing code, and replace local sync server (#2329)
This PR replaces the existing Python-driven sync server with a new one in Rust.
The new server supports both collection and media syncing, and is compatible
with both the new protocol mentioned below, and older clients. A setting has
been added to the preferences screen to point Anki to a local server, and a
similar setting is likely to come to AnkiMobile soon.

Documentation is available here: <https://docs.ankiweb.net/sync-server.html>

In addition to the new server and refactoring, this PR also makes changes to the
sync protocol. The existing sync protocol places payloads and metadata inside a
multipart POST body, which causes a few headaches:

- Legacy clients build the request in a non-deterministic order, meaning the
entire request needs to be scanned to extract the metadata.
- Reqwest's multipart API directly writes the multipart body, without exposing
the resulting stream to us, making it harder to track the progress of the
transfer. We've been relying on a patched version of reqwest for timeouts,
which is a pain to keep up to date.

To address these issues, the metadata is now sent in a HTTP header, with the
data payload sent directly in the body. Instead of the slower gzip, we now
use zstd. The old timeout handling code has been replaced with a new implementation
that wraps the request and response body streams to track progress, allowing us
to drop the git dependencies for reqwest, hyper-timeout and tokio-io-timeout.

The main other change to the protocol is that one-way syncs no longer need to
downgrade the collection to schema 11 prior to sending.
2023-01-18 12:43:46 +10:00
..
_vendor Move away from Bazel (#2202) 2022-11-27 15:24:20 +10:00
foreign_data Plaintext import/export (#1850) 2022-06-01 20:26:16 +10:00
importing Fix typos (#2210) 2022-11-24 20:18:57 +10:00
scheduler Enable state-dependent custom scheduling data (#2049) 2022-09-05 16:48:01 +10:00
_backend.py Rework syncing code, and replace local sync server (#2329) 2023-01-18 12:43:46 +10:00
_legacy.py Move away from Bazel (#2202) 2022-11-27 15:24:20 +10:00
_rsbridge.pyi Rework syncing code, and replace local sync server (#2329) 2023-01-18 12:43:46 +10:00
browser.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
cards.py Add card meta for persisting custom scheduling state (#2040) 2022-09-02 11:22:49 +10:00
collection.py Rework syncing code, and replace local sync server (#2329) 2023-01-18 12:43:46 +10:00
config.py Import submodules directly (#1662) 2022-02-13 13:40:47 +10:00
consts.py Fix mypy not picking up on missing attributes 2022-11-04 14:56:36 +10:00
db.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
dbproxy.py fix cyclic import when importing anki.collection directly 2022-02-15 19:08:29 +10:00
decks.py Fix mypy not picking up on missing attributes 2022-11-04 14:56:36 +10:00
errors.py Refactor error handling (#2136) 2022-10-21 18:02:12 +10:00
exporting.py Narrow typehints within pylib (#2003) 2022-08-05 14:48:07 +10:00
find.py Add py3.9 to hooks (#1542) 2021-12-09 09:11:22 +10:00
hooks.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
httpclient.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
lang.py Move away from Bazel (#2202) 2022-11-27 15:24:20 +10:00
latex.py Make dvipng use transparent background by default (#2036) 2022-08-31 18:49:22 +10:00
media.py Narrow typehints within pylib (#2003) 2022-08-05 14:48:07 +10:00
models.py Import submodules directly (#1662) 2022-02-13 13:40:47 +10:00
notes.py Narrow typehints within pylib (#2003) 2022-08-05 14:48:07 +10:00
py.typed mark anki and aqt modules as having typing info 2020-01-13 13:03:37 +10:00
rsbackend.py PEP8 for rest of pylib (#1451) 2021-10-25 14:50:13 +10:00
sound.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
stats.py Fix some more RTL issues (#2244) 2022-12-08 22:29:56 +10:00
statsbg.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
stdmodels.py Fix mypy not picking up on missing attributes 2022-11-04 14:56:36 +10:00
storage.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
sync.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
syncserver.py Rework syncing code, and replace local sync server (#2329) 2023-01-18 12:43:46 +10:00
tags.py Move custom study tag and limit gathering+saving into the backend 2022-03-10 16:23:03 +10:00
template.py Store rendered side in TemplateRenderContext (#1852) 2022-05-10 11:44:44 +10:00
types.py Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
utils.py Move away from Bazel (#2202) 2022-11-27 15:24:20 +10:00