Commit graph

126 commits

Author SHA1 Message Date
Damien Elmes
4bf334c6b3 strip some old code 2011-04-28 09:21:06 +09:00
Damien Elmes
1aafbd02f3 use env vars for host/port 2011-04-28 09:19:57 +09:00
Damien Elmes
c4e045463b set rd=2 in subscriptions 2011-03-04 14:32:17 +09:00
Damien Elmes
07db17be88 off by one in relativeDelay sync code 2011-02-07 00:04:39 +09:00
Damien Elmes
c1d15b8a9e clearer message when facts missing after sync 2011-01-21 11:02:07 +09:00
Damien Elmes
0b07707e68 make sure we don't try to send the queues when bundling deck 2011-01-13 07:55:39 +09:00
Damien Elmes
2ca27d389f fix local syncing 2011-01-07 13:35:15 +09:00
Damien Elmes
1f34abc003 more fixes for skewed clocks
if a client with a clock greater than server time synced a deck, the modified
time ended up higher than lastSync when the deck was modified on the server.
instead we force the modified time to be <= the server time, which is known
correct.
2011-01-04 15:51:24 +09:00
Damien Elmes
ff5bc72121 pass in a 0 timediff if using stock sync() 2010-12-25 12:43:35 +09:00
Damien Elmes
ee7da2bd65 update comment 2010-12-21 06:41:42 +09:00
Damien Elmes
7c45bab35a rate-limit sync progress messages for win32 installs with huge net bufs 2010-12-17 21:04:47 +09:00
Damien Elmes
400ca9a8a2 factor in time difference when determining common point 2010-12-17 03:55:04 +09:00
Damien Elmes
0c9672e7b8 rewrite media support
- media is no longer hashed, and instead stored in the db using its original
  name
- when adding media, its checksum is calculated and used to look for
  duplicates
- duplicate filenames will result in a number tacked on the file
- the size column is used to count card references to media. If media is
  referenced in a fact but not the question or answer, the count will be zero.
- there is no guarantee media will be listed in the media db if it is unused
  on the question & answer
- if rebuildMediaDir(delete=True), then entries with zero references are
  deleted, along with any unused files in the media dir.
- rebuildMediaDir() will update the internal checksums, and set the checksum
  to "" if a file can't be found
- rebuildMediaDir() is a lot less destructive now, and will leave alone
  directories it finds in the media folder (but not look in them either)
- rebuildMediaDir() returns more information about the state of media now
- the online and mobile clients will need to to make sure that when
  downloading media, entries with no checksum are non-fatal and should not
  abort the download process.
- the ref count is updated every time the q/a is updated - so the db should be
  up to date after every add/edit/import
- since we look for media on the q/a now, card templates like '<img
  src="{{{field}}}">' will work now
- export original files as gone as it is not needed anymore
- move from per-model media URL to deckVar. downloadMissingMedia() uses this
  now. Deck subscriptions will have to be updated to share media another way.
- pass deck in formatQA, as latex support is going to change
2010-12-11 01:19:31 +09:00
Damien Elmes
a383223e02 provide more info in sync error messages; catch zlib decode errors 2010-12-07 16:48:49 +09:00
Damien Elmes
2013e7e4ff conditional delete of css 2010-12-07 14:35:16 +09:00
Damien Elmes
15763f8f3c make sure we don't commit during a sync
updateDynamicIndices() is done on next deck load anyway
2010-12-07 11:59:02 +09:00
Damien Elmes
039af66a9d don't rebuild counts in applyPayload(), as the deck will be reopened 2010-12-07 11:56:43 +09:00
Damien Elmes
4c8f2d3b47 add finish() command to sync protocol 2010-12-07 11:02:23 +09:00
Damien Elmes
0af8da9cb8 sync updates
- set lastSync on successful upload, not before it
- make sure source file is closed
- use v2 sync protocol
2010-12-07 09:20:31 +09:00
Damien Elmes
da97701b2d disable lastSync fudging again 2010-12-07 09:19:09 +09:00
Damien Elmes
9259718fd5 set syncName after full download 2010-12-02 07:23:54 +09:00
Damien Elmes
6ec898ca4b Require explicit reset for most queue-modifying functions
When you call operations like deleteCards(), suspendCards() and so on, it is
now necessary to call deck.reset() afterwards. This allows the calling code to
delay a reset if necessary. If the calling code calls a function that says the
caller must reset, the caller should be sure to call .reset() and fetch the
current card again. Failure to do the latter will result in answerCard()
attempting to remove the card from the queue, when the queue has been cleared.
2010-11-23 17:41:36 +09:00
Damien Elmes
b69fd48768 more type handling updates; don't munge counts on sync
In various parts of the code we need to get all cards of a given category
(new, failed, etc) regardless of whether they're suspended, buried, etc. So we
store the true type in the obsolete relativeDelay column and add in index for
it, because it's cheaper than putting indices on reps & successive.
2010-11-13 18:39:24 +09:00
Damien Elmes
6ed0bc91bb update sync url 2010-11-08 09:22:36 +09:00
Damien Elmes
e0d46f0f12 be resilient if spaceUntil sent wrong 2010-11-06 07:50:05 +09:00
Damien Elmes
46790f2e92 remove (incorrect) code in sync, is covered on deck load anyway 2010-11-02 22:34:10 +09:00
Damien Elmes
2c5ac66083 type/priority changes, cram/rev early refactor, more
* Adjust type to remove cards from the queues, so we don't have to rebuild
  priorities to restore them:

Type -= 3 when suspending
Type += 3 when burying
Type += 6 when cramming / reviewing early

We still need to adjust priorities for backwards compatibility, but this can
be removed in the future.

* Factor out scheduler-specific code in answerCard(), so the different
  schedulers are now fully modular

* Differentiate between a card's current queue and its type

* Make sure dueCutoff cuts off at the chosen offset instead of midnight
2010-11-02 01:59:20 +09:00
Damien Elmes
ad743d850d start work on scheduling refactor
Previously we used getCard() to fetch a card at the time. This required a
number of indices to perform efficiently, and the indices were expensive in
terms of disk space and time required to keep them up to date. Instead we now
gather a bunch of cards at once.

- Drop checkDue()/isDue so writes are not necessary to the DB when checking
for due cards
- Due counts checked on deck load, and only updated once a day or at the end
of a session. This prevents cards from expiring during reviews, leading to
confusing undo behaviour and due counts that go up instead of down as you
review. The default will be to only expire cards once a day, which represents
a change from the way things were done previously.
- Set deck var defaults on deck load/create instead of upgrade, which should
fix upgrade issues
- The scheduling code can now have bits and pieces switched out, which should
make review early / cram etc easier to integrate
- Cards with priority <= 0 now have their type incremented by three, so we can
get access to schedulable cards with a single column.
- rebuildQueue() -> reset()
- refresh() -> refreshSession()
- Views and many of the indices on the cards table are now obsolete and will
  be removed in the future. I won't remove them straight away, so as to not
  break backward compatibility.
- Use bigger intervals between successive card templates, as the previous
intervals were too small to represent in doubles in some circumstances

Still to do:

- review early
- learn more
- failing mature cards where delay1 > delay0
2010-10-18 14:35:11 +09:00
Damien Elmes
8df9111b50 only copy used media on import/export 2010-10-16 10:29:39 +09:00
Damien Elmes
e010ef8062 add clock skew compensation again
this has the negative effect of causing multiple full syncs if syncing
multiple times within a 5 minute period of the previous full sync, but it
makes it much less likely that people's due counts will fall out of sync
2010-10-02 14:47:33 +09:00
Damien Elmes
99ba3f09c8 catch missing facts at end of sync 2010-09-12 12:21:39 +09:00
Damien Elmes
5e7c62bca5 don't compensate for clock skew 2010-07-30 18:15:24 +09:00
Damien Elmes
e956aa9afb remove obsolete function 2010-07-27 22:54:51 +09:00
Damien Elmes
d3fb189a72 improved lastSync/modified handling
- never bump deck mod while syncing
- set lastSync to current time, not deck modified time
- don't update lastSync until the final part of the sync
- lower clock skew allowance to ~5 minutes
- bump full sync threshold to 1000 modified items
2010-07-27 22:46:04 +09:00
Damien Elmes
55194f8aa7 ensure cardmodel/fieldmodels work when given a string too 2010-07-26 17:15:05 +09:00
Damien Elmes
61a7d6d79e make sure we match a given model even when given a string 2010-07-24 14:28:24 +09:00
Damien Elmes
6852b0acda bump mod time on full sync to server, ensure lastSync matches 2010-07-21 18:37:05 +09:00
Damien Elmes
f03000d27b remove string exceptions for python2.6 2010-06-10 13:24:46 +09:00
Damien Elmes
5616e679f5 cache the css as a deck var, don't accidentally send it in sync 2010-05-10 21:32:36 +09:00
Damien Elmes
fd1953bfb5 convert to a list, not tuples, so we can modify on the fly 2010-05-07 16:05:35 +09:00
Damien Elmes
c8d9bac5df clarify one way sync error 2010-03-05 09:27:11 +09:00
Damien Elmes
7c8e612704 use a constant for chunk size 2010-02-18 17:36:54 +09:00
Damien Elmes
33ec7ce133 clarify comment 2009-12-02 03:38:46 +09:00
Damien Elmes
093395b9e0 Revert "add 30 second timeout to all sync ops"
This reverts commit cbc23e5231.
2009-11-24 23:35:34 +09:00
Damien Elmes
cbc23e5231 add 30 second timeout to all sync ops 2009-10-04 19:53:12 +09:00
Damien Elmes
164b0583c3 unlink tmp file after full up 2009-10-04 19:33:18 +09:00
Damien Elmes
75f56d13e2 decrease chunk size to 32k due to crappy win32 network cards 2009-09-26 05:51:22 +09:00
Damien Elmes
27732e3553 catch large # of reviews in full sync, reduce limit to 500 2009-06-25 05:47:14 +09:00
Damien Elmes
6b7c0d7997 fix missing media problem, fix json decode float 2009-06-21 04:36:36 +09:00
Damien Elmes
1f0a8edfa4 strip out mediaSupported 2009-06-20 01:32:20 +09:00