The undo code was using triggers and a temporary table to write out all changed rows before making a change. This made for powerful undo/redo support, but had some problems:
- creating the tables and triggers wasn't cheap, especially on mobile devices
- likewise, every data modification required writing into two separate databases, almost doubling the amount of writes required
- it was possible to leave the DB in an inconsistent state if an undoable operation is followed by a non-undoable operation that references the undoable operation, and the user then rolls back the undoable operation.
To address these issues, we simplify undo by integrating it with the autosave changes:
- .save() can be passed a name to mark a rollback point. If the user undoes the change, any changes since the last save are lost
- autosaves happen every 5 minutes, and are pushed back on a .save(), so the maximum work a user can lose is 5 minutes.
- reviews are handled separately, so we can let the user undo multiple reviews at once
- if necessary, special cases could be added for other operations like marking
This means that if a user does two damaging operations in a row they won't be able to restore the first one, but such an event is both unlikely, and is also covered by the backups made each time a deck is opened.
Previously cloze deletions were handled by copying the contents of one field
into another and applying transforms to it. This had a number of problems:
- after you add a card, you can't undo the cloze deletion
- if you spot a mistake, you have to edit it twice (or more if you have more
than one cloze for a sentence)
- making multiple clozes requires copying & pasting the sentence multiple
times
- this also lead to much bigger decks if the sentences being cloze-deleted are
large
- related clozes can't be spaced apart as siblings
To address these issues, we introduce the idea of cloze tags in the card
template and fields. If the template has the text:
{{cloze:1:field}}
And a field has the following contents:
{{c1::hello}}
Then the template will automatically replace that part of the text with either
occluded text, or a highlighted answer. All other clozes in the field are
displayed normally.
At the same time, we add support for text: into the template library, instead
of manually creating text: fields in the dict for every field.
Finally, add a forecast routine to get the due counts for the next week, which
is used in the GUI.
The media table was originally introduced when Anki hashed media filenames,
and needed a way to remember the original filename. It also helped with:
1) getting a quick list of all media used in the deck, or the media added
since the last sync, for mobile clients
2) merging identical files with different names
But had some drawbacks:
- every operation that modifies templates, models or facts meant generating
the q/a and checking if any new media had appeared
- each entry is about 70 bytes, and some decks have 100k+ media files
So we remove the media table. We address 1) by being more intelligent about
media downloads on the mobile platform. We ask the user after a full sync if
they want to look for missing media, and they can choose not to if they know
they haven't added any. And on a partial sync, we can scan the contents of the
incoming facts for media references, and download any references we find. This
also avoids all the issues people had with media not downloading because it
was in their media folder but not in the media database.
For 2), when copying media to the media folder, if we have a duplicate
filename, we check if that file has the same md5, and avoid copying if so.
This won't merge identical content that has separate names, but instances
where users need that are rare.
The 'entry due' is the due time of a failed card before it enters the learning
queue. When the card graduates or is removed, it has its old due time
restored. We could pull this from the revlog, but it's cheaper to do it this
way.
We originally were triggering on 100 opcodes, because at the time we were
doing write-heavy alterations to the DB for inactive tags, and a higher level
of opcodes would pause the interface for a long time. The query structure is
different now, so we can afford to save the overhead of more frequent calls.
With the change, a .reset() triggers the handler 3 times; fixIntegrity()
triggers it 30 times over a period of 4.5 seconds.
A lot of the old checks in fixIntegrity() are no longer relevant, and some of
the others may no longer be required. They can be added back in as the need
arises.
We want to ensure that we never recycle ids from deleted cards. We could do
this with an autoincrement column in sqlite, but it's cheaper for us to handle
the ids ourselves, as the deck object is always in memory.
- remove revlog.py and move code into scheduler
- add a routine to log a learn repetition
- rename flags to type and set type=0 for learn mode
- add to unit test
Because the cards table is small now, loading it in and doing a table scan
isn't necessary. The facts table is bigger now, so we still need that index.
When adding facts, you can now pass in a group id which the GUI should support
editing. Templates will have an optional group id which overrides the provided
id, so users can automatically put certain card types in a different group (or
all of them, if desired). Greying out the group box in the GUI in that case
would be a good idea.
This means that the default learn queue sort order doesn't need another column
in the index, but it also means that generated cards will have a higher id,
and would appear later even if they have a lower ordinal. This is probably an
infrequent issue, and a plugin which rewrites ids would probably be an
adequate solution.
Our goal is to allow decks created on one platform to use similar fonts (even
if named differently) when moving to another platform. The old solution wasn't
useful for the web version or mobile versions. Instead, we store a mapping in
the deck, and when generating the CSS, we list all possible fonts. An option
in the interface for the user to add extra fonts might be nice.
The model now has a css column, and when it's flushed, it generates the css
for the fields and templates. This means we don't have to generate the CSS on
deck load anymore.
The hex cache has also been removed. Javascript couldn't handle big ints, but
since ints are small numbers now, we no longer need a cache to efficiently
convert an id to hex.
- convert checksums to int
- add bulk update & update on upgrade
- add indices pending performance testing. The fsum table & indices add about
2MB to a deck with 50k unique fields
