Commit graph

38 commits

Author SHA1 Message Date
Damien Elmes
014787c4cb log mediaChangesZip() 2014-07-04 18:14:45 +09:00
Damien Elmes
eba3aa583c if media.db import fails, start anew 2014-06-28 05:06:43 +09:00
dae
2aa7714f87 new media syncing protocol
- media syncing no longer locks the account, so it can be done
  in the background in the future, and multiple clients can safely
  sync media at the same time
- all operations are now idempotent, so they can be repeatedly safely
  in the event of a connection error
- whether it's a normal incremental sync, an initial sync,
  or the media database has been deleted, no files will be uploaded
  or downloaded if they already exist on the other side
- file removals are now chunked like additions & updates, preventing
  timeouts due to large requests
- if the server can't process a chunk in time, it will return a count
  of what it did process, so the client can retry the rest

Notes for AnkiDroid:

- when porting this, recommend you pick a different name for the
.media.db2 file, so users don't accidentally copy the AD version to
the desktop or vice versa
- please make sure filenames are added to the zip in NFC form
2014-06-26 09:00:38 +09:00
Damien Elmes
de8adfecff disallow \r and \n in media filenames 2014-03-06 09:00:47 +09:00
Damien Elmes
f0525a76fb if we renamed any files to nfc, need to rerun check 2013-12-13 20:24:39 +09:00
Damien Elmes
54e2d42fa6 off by one in zip count comparison 2013-12-09 17:37:43 +09:00
Damien Elmes
ba589ac8cc need to compare mediaList with nfd on macs
otherwise the server says we have file x and local code
doesn't notice it already exists
2013-12-04 11:43:57 +09:00
Damien Elmes
1c35a590e3 fix regression in unused media check 2013-11-29 02:07:31 +09:00
Damien Elmes
0372f30220 adjust media regexp to not trigger on mce_src
otherwise pasting the following will cause an error:

<img src="..." mce_src="/foo.png">
2013-11-27 19:24:41 +09:00
Damien Elmes
99d82c1f2d fix hasIllegal check, and associated unit test 2013-11-26 03:19:11 +09:00
Damien Elmes
f6b9dadf13 catch invalid file encodings in media check & sync 2013-11-13 17:19:25 +09:00
Damien Elmes
5f0ef85044 fix reversed check in media 2013-10-30 21:10:25 +09:00
Damien Elmes
82a54c780f catch invalid temp folder and other tweaks
- tweak sync code so that a failure in loading the collection
  won't leave the app with an unopen collection
- don't show corrupt collection message when the error is not
  a db error
- catch the temp folder issue when loading the collection. i suspect
  this was the issue that was causing some people to end up with
  an open anki instance with no collection loaded
2013-10-22 15:30:53 +09:00
Damien Elmes
da1672d42f don't rename/unlink files for non-local check 2013-10-17 17:59:26 +09:00
Damien Elmes
6be7503543 normalize to nfc when sending media in sync 2013-10-16 08:37:34 +09:00
Damien Elmes
b42c0c725b Revert "tolerate a str arg to writeData()"
This reverts commit 2d4e88afbd.

On second thought, this change is likely to mask the error in most cases, and
we want add-on authors to update their add-ons anyway as they would not be
handling non-latin text in 2.0.12.
2013-10-10 11:08:39 +09:00
Damien Elmes
2d4e88afbd tolerate a str arg to writeData() 2013-10-09 13:13:48 +09:00
Damien Elmes
6c304ebe2f to normalize unicode we need to have unicode 2013-10-06 04:53:11 +09:00
Damien Elmes
0d1d8c5bf9 define standard encoding for unicode (#893)
- always store media references in fields in NFC form
- always encode filenames on disk in NFC form on machines other than macs
- use relevant encoding when placing files in the media folder during syncs
  and apkg imports as well
- rename 'unused media' back to 'check media'
- check media can now automatically change media references and filename
  encodings to the correct format
2013-09-20 19:28:44 +09:00
Damien Elmes
a538e29480 refactor illegal char handling and file adding for media
- add writeData() so we can add files to media folder without having to
  save them in an intermediate folder first
- we were stripping or checking for illegal filename characters in
  multiple places; factor those checks out into separate routines
- add * to invalid char list, and disallow both forward and backslash
- remove size checks in syncAdd(); is handled server-side
2013-07-11 15:33:24 +09:00
Damien Elmes
f2d538d546 catch errors trying to determine fs type 2013-06-14 13:15:05 +09:00
Damien Elmes
a6b44cff01 skip fat32 check on unmapped network drive (#902) 2013-06-10 15:32:21 +09:00
Damien Elmes
ec888eb2aa fix bug in media sync where we weren't fetching needed files 2013-05-31 14:03:34 +09:00
Damien Elmes
9714143af9 remove dropbox image escape hack 2013-05-29 09:34:12 +09:00
Damien Elmes
dceacff47c tell server which files we need 2013-05-27 15:29:24 +09:00
Damien Elmes
0a81df60dc automatically resync media, and don't send media server already has (#583) 2013-05-27 14:25:30 +09:00
Damien Elmes
97a342ae23 when media deleted during sync, move to trash instead of deleting 2013-05-23 13:38:51 +09:00
Damien Elmes
918694a096 more media regexp fixes
- split quoted and unquoted image case into separate regexps, since we can't
  include a group reference inside a character set
- disallow spaces in the non-quoted case
- this should fix matching on images with other attributes again
2013-05-22 09:45:58 +09:00
Damien Elmes
85cb7ef47e ignore _ in field references (#803) 2013-05-18 18:57:24 +09:00
Damien Elmes
4aa634bc44 need greedy match on images 2013-05-17 16:20:27 +09:00
Damien Elmes
8f36841281 ensure we match on identical closing quote (#769) 2013-05-17 12:23:56 +09:00
Damien Elmes
13a2943667 full media scan on fat32 drives (#844) 2013-05-16 17:58:30 +09:00
Damien Elmes
c8f4d3a582 check checksum of all filename alternatives when adding media
patch thanks to Julien Baley
2013-05-16 17:24:20 +09:00
ospalh
d806f44484 Only the Mac file system stores file names NFD-normalized. So do that normalization only there when looking for files. 2013-04-11 17:37:06 +09:00
Damien Elmes
f3f7835032 add upper bound to # of media files in single zip 2013-04-11 14:13:11 +09:00
Damien Elmes
bf1af126b0 properly generate clozed latex in unused media check (#775) 2013-04-11 13:26:52 +09:00
Damien Elmes
b2457811a2 optimize imports 2012-12-22 09:17:10 +09:00
Damien Elmes
d477df8c28 merged libanki and ankiqt into single distro 2012-12-21 16:51:59 +09:00