I've corrected some obvious issues, and we can fix others over time.
Mypy tests are currently broken, as adding the type hints has increased
mypy's testing surface.
APFS is normalization-preserving, which results in spurious changes
being detected if we assume we'll receive NFD from file listings
this needs to be ported to 2.0.x as well
Experiment with ignoring dirMod and scanning the folder for changes on
each sync. Use scandir() for better performance. Please let me know if
this causes a noticeable slowdown for you.
media.py sets CWD to the media directory of the collection
(collection.media), and relies on that directory being
maintained as CWD throughout execution. The original CWD
is restored in the close() function.
Remove reliance on CWD being set and maintained throughout
execution of media.py. Improves portability and usability
in different codebases.
Before sqlite 3.15, the parameter to vacuum was ignored. Since sqlite
3.15, it became a database name parameter. The "vacuum analyze" syntax
was never supported by sqlite, seems to be just a psql thing.
The error for this is "OperationalError: unknown database analyze", and
happens when doing a media sync.
we need to switch to autocommit mode before executing vacuums &
certain pragmas
also removed the pysqlite version workaround; we just use the
python-provided versions in all cases now as it is new enough
deletes in particular take some time for the server to process, but
don't require much bandwidth, leading to the progress appearing to
have pause when content is actually being processed
this also gives the user an idea of how long the process will take to
complete
- media syncing no longer locks the account, so it can be done
in the background in the future, and multiple clients can safely
sync media at the same time
- all operations are now idempotent, so they can be repeatedly safely
in the event of a connection error
- whether it's a normal incremental sync, an initial sync,
or the media database has been deleted, no files will be uploaded
or downloaded if they already exist on the other side
- file removals are now chunked like additions & updates, preventing
timeouts due to large requests
- if the server can't process a chunk in time, it will return a count
of what it did process, so the client can retry the rest
Notes for AnkiDroid:
- when porting this, recommend you pick a different name for the
.media.db2 file, so users don't accidentally copy the AD version to
the desktop or vice versa
- please make sure filenames are added to the zip in NFC form
