Commit graph

306 commits

Author SHA1 Message Date
Damien Elmes
b64f7a9456 fix burying in test scheduler
The bury new/review flags are now pulled from each card's home deck,
instead of using a global setting that had not been hooked up. This
unfortunately means we need to fetch the map of all decks up front, as
we need to be able to look up a deck configuration for cards that are
in filtered decks.

Fixes a "card was modified" error caused by cards being buried during
review, when they weren't removed up-front.
2021-05-12 12:00:15 +10:00
Damien Elmes
d878de54c0 default to the v2 scheduler in new collections
ported from 53d9433d94
2021-05-06 19:09:28 +10:00
Damien Elmes
2ff8c20686 update backend to support undoing of notetype changes 2021-04-30 12:54:59 +10:00
Damien Elmes
2902b64e82 move note/card removal for notetype out of storage layer 2021-04-29 19:44:09 +10:00
Damien Elmes
a95cbb8515 DeckConfId -> DeckConfigId 2021-04-28 21:09:26 +10:00
Damien Elmes
fd81700679 deckconf -> deckconfig 2021-04-20 21:54:24 +10:00
Damien Elmes
64ebc32b3d tidy up Rust imports
rustfmt can do this automatically, but only when run with a nightly
toolchain, so it needs to be manually done for now - see rslib/rusfmt.toml
2021-04-18 18:38:54 +10:00
Damien Elmes
262b50445c start on making deck config and schema/mod changes undoable
+ move timestamps into a struct in a separate file for convenience
2021-04-18 17:33:12 +10:00
Damien Elmes
76eb119870 add schema change prompt to removal, tweak return struct 2021-04-18 17:33:12 +10:00
RumovZ
5e3e194289 Remove redundant imports 2021-04-18 08:43:46 +02:00
Damien Elmes
e71f7714ad as_str() -> as_native_str() 2021-04-18 09:33:39 +10:00
Damien Elmes
1acc679e8f hide NativeName inner value, and require explicit accessors 2021-04-18 09:29:35 +10:00
RumovZ
20bd207f00 Give deck.name the newtype NativeDeckName
The deck name must be constructed by calling associated functions of
NativeDeckName, unless the name is guaranteed to be valid machine
name (like "Default").
NativeDeckName exposes methods to mutate the deck name and return
the human name.
The storage routines take &strs, but those should be slices of
NativeDeckNames to ensure machine form and normalization.
2021-04-17 22:47:04 +02:00
Damien Elmes
84fe309583 update scheduling ops
- migrate to CollectionOp()
- return actual change count when suspending/burying
- add helper to convert vec to vec of newtype
2021-04-06 16:38:42 +10:00
Damien Elmes
ebf7cc61d4 switch next_day_at to a newtype 2021-04-05 16:17:26 +10:00
Damien Elmes
996d9f9bbc undo support for tag collapse; expand->collapse for consistency w/ decks 2021-04-05 11:47:12 +10:00
Damien Elmes
c4b3ab62c8 embed deck messages 2021-04-04 21:41:16 +10:00
Damien Elmes
a250464309 switch DbError to tuple type 2021-04-01 17:34:03 +10:00
Damien Elmes
af37164fba move sync/network errors into separate file 2021-04-01 17:02:54 +10:00
Damien Elmes
094e4ad461 crate::err -> crate::error 2021-04-01 16:07:13 +10:00
Damien Elmes
cfac40febc switch NoteType to Notetype
When used as a variable, we were typically calling it a 'notetype', not
a 'note type'.
2021-03-27 22:03:19 +10:00
Damien Elmes
1055acb9f2 fix more issues uncovered by the latest clippy 2021-03-27 20:25:34 +10:00
Damien Elmes
dc81a7fed0 use mixed case for abbreviations in Rust code
So, this is fun. Apparently "DeckId" is considered preferable to the
"DeckID" were were using until now, and the latest clippy will start
warning about it. We could of course disable the warning, but probably
better to bite the bullet and switch to the naming that's generally
considered best.
2021-03-27 19:53:33 +10:00
Damien Elmes
3433c02242 i18n->tr in rslib/ to match Python/TS code 2021-03-27 12:09:51 +10:00
Damien Elmes
dc5fdd30d4 update no-arg TR references in rslib/ 2021-03-26 23:16:08 +10:00
Damien Elmes
9aece2a7b8 rework translation handling
Instead of generating a fluent.proto file with a giant enum, create
a .json file representing the translations that downstream consumers
can use for code generation.

This enables the generation of a separate method for each translation,
with a docstring that shows the actual text, and any required arguments
listed in the function signature.

The codebase is still using the old enum for now; updating it will need
to come in future commits, and the old enum will need to be kept
around, as add-ons are referencing it.

Other changes:

- move translation code into a separate crate
- store the translations on a per-file/module basis, which will allow
us to avoid sending 1000+ strings on each JS page load in the future
- drop the undocumented support for external .ftl files, that we weren't
using
- duplicate strings in translation files are now checked for at build
time
- fix i18n test failing when run outside Bazel
- drop slog dependency in i18n module
2021-03-26 09:41:32 +10:00
Damien Elmes
d382b33585 rework filtered deck screen & search errors
- Filtered deck creation now happens as an atomic operation, and is
undoable.
- The logic for initial search text, normalizing searches and so on
has been pushed into the backend.
- Use protobuf to pass the filtered deck to the updated dialog, so
we don't need to deal with untyped JSON.
- Change the "revise your search?" prompt to be a simple info box -
user has access to cancel and build buttons, and doesn't need a separate
prompt. Tweak the wording so the 'show excluded' button should be more
obvious.
- Filtered decks have a time appended to them instead of a number,
primarily because it's easier to implement. No objections going back to
the old behaviour if someone wants to contribute a clean patch.
The standard de-duplication will happen if two decks are created in the
same minute with the same name.
- Tweak the default sort order, and start with two searches. The UI
will still hide the second search by default, but by starting with two,
the frontend doesn't need logic for creating the starting text.
- Search errors now have their own error type, instead of using
InvalidInput, as that was intended mainly for bad API calls. The markdown
conversion is done when the error is converted from the backend, allowing
errors to printed as a string without any special handling by the calling
code.

TODO: when building a new filtered deck, update_active() is clobbering
the undo log when the overview is refreshed
2021-03-24 22:04:35 +10:00
Damien Elmes
12597e1094 support undo of filtered deck build/empty 2021-03-24 12:56:06 +10:00
RumovZ
1823c0dda4 Add get_note_without_fields() from storage 2021-03-20 11:59:45 +01:00
Damien Elmes
f7e4b6d822 fix spurious warning in db check for v1 scheduler+non-zero lapse ivl%
https://forums.ankiweb.net/t/invalid-property-on-card-not-resolving-via-recommended-check-database/8430
2021-03-19 22:57:43 +10:00
Damien Elmes
4c61c92806 speed up tag drag&drop and finish tag tidyup
approx 4x speedup when reparenting 10-15 tags and their children at once
2021-03-19 19:45:21 +10:00
Damien Elmes
08895c58d9 introduce separate routine to remove tags from specific notes
We were (ab)using the bulk update routine to do deletions, but that
code was really intended to be used for finding&replacing, where an
exact match is not a requirement.
2021-03-19 19:45:21 +10:00
Damien Elmes
09076da937 make tag deletion undoable, and speed it up
- ~4x faster than before on tag tree with 30k notes
- remove the separate clear_tag() backend method
2021-03-19 19:45:21 +10:00
Damien Elmes
157b74b671 make tag renaming undoable, and speed it up
~3x speedup when renaming a tag that's on 25k notes
2021-03-19 19:45:21 +10:00
Damien Elmes
8fc43956c2 move collection mtime bump into backend
Fixes the following issue:
- some code directly modifies the database, causing modified_in_python
to be set to true
- an undoable operation is run, which calls autosave() at the end
- autosave() notices there's an undoable operation, and commits immediately
- because modified_in_python was true, col.mtime was bumped in Python
- that invalidated the undo queue, preventing the operation from being
undone
2021-03-19 19:45:21 +10:00
Damien Elmes
28994458e9 add indexes to graves table to speed up undo 2021-03-12 18:59:24 +10:00
RumovZ
f4c2fe6485 Merge branch 'master' into sidebar-tools 2021-03-11 12:08:32 +01:00
Damien Elmes
e122f8ae0d undo support for deck adding/removing
Work in progress - still to do:
- renames appear as 'Update Deck' - easiest way to solve it would
be to have a separate backend method for renames
- drag&drop of decks not yet undoable
- since the undo status is updated after the backend method ends,
the older checkpoint() calls need to be replaced with an
update_undo_status() at the end of the call - if we just remove the
checkpoint, then the menu doesn't get updated
2021-03-10 23:50:11 +10:00
RumovZ
0c2ac4ba04 Merge branch 'master' into sidebar-tools 2021-03-10 10:34:36 +01:00
Damien Elmes
96940f0527 undo support for config entries 2021-03-10 11:53:27 +10:00
Damien Elmes
ce243c2cae Simplify note adding and the deck/notetype choosers
The existing code was really difficult to reason about:

- The default notetype depended on the selected deck, and vice versa,
and this logic was buried in the deck and notetype choosing screens,
and models.py.
- Changes to the notetype were not passed back directly, but were fired
via a hook, which changed any screen in the app that had a notetype
selector.

It also wasn't great for performance, as the most recent deck and tags
were embedded in the notetype, which can be expensive to save and sync
for large notetypes.

To address these points:

- The current deck for a notetype, and notetype for a deck, are now
stored in separate config variables, instead of directly in the deck
or notetype. These are cheap to read and write, and we'll be able to
sync them individually in the future once config syncing is updated in
the future. I seem to recall some users not wanting the tag saving
behaviour, so I've dropped that for now, but if people end up missing
it, it would be simple to add as an extra auxiliary config variable.
- The logic for getting the starting deck and notetype has been moved
into the backend. It should be the same as the older Python code, with
one exception: when "change deck depending on notetype" is enabled in
the preferences, it will start with the current notetype ("curModel"),
instead of first trying to get a deck-specific notetype.
- ModelChooser has been duplicated into notetypechooser.py, and it
has been updated to solely be concerned with keeping track of a selected
notetype - it no longer alters global state.
2021-03-10 11:53:27 +10:00
Damien Elmes
c0e939ceb6 split config.rs up 2021-03-10 11:53:27 +10:00
Damien Elmes
f4d931131b undo support for note removing 2021-03-10 11:47:53 +10:00
Damien Elmes
40aff4447a undo support for note adding 2021-03-10 11:47:53 +10:00
Damien Elmes
67c490a8dc support undoing deck mutations
This required refactoring the deck code a bit to split up the 'update'
and 'add' cases better.
2021-03-10 11:47:53 +10:00
Damien Elmes
c9eeb91e0a initial work on undoing reviews+burying siblings
- fetch sfld and csum when fetching notes, to make it cheaper
to write them back out unmodified
- make `fields` private, and access it via accessors, so we can
still catch when fields have been mutated without calling
prepare_for_update()
- fix python importing code passing a string in as the checksum
2021-03-10 11:47:53 +10:00
Damien Elmes
59ec485852 handle duplicate keys in schema 11 deckconf 2021-03-04 11:52:15 +10:00
Damien Elmes
07989afa38 fix infinite loop on card answer when parent is missing deck 2021-03-04 11:03:09 +10:00
RumovZ
25d57574c9 Enable removal of multiple tags from the sidebar 2021-03-02 11:05:16 +01:00
Damien Elmes
c74a71a6d7 interval sorting 2021-03-01 21:48:14 +10:00
Damien Elmes
2c6b6734b5 experimental queue building
Still a work in progress, and hidden behind a feature flag.
2021-03-01 12:18:21 +10:00
Damien Elmes
6a44269280 sched->scheduler 2021-02-23 17:35:20 +10:00
Damien Elmes
53d9433d94 Revert "use v2 scheduler+new timezone handling by default for new users"
This reverts commit 8372931b9b.

I fear this will be too disruptive - let's give AnkiDroid a bit more
time to catch up. Reverting this will mean new users are presented with
an upgrade notice on first startup, which looks a bit silly, but it's
probably the lesser of two evils.
2021-02-21 19:03:46 +10:00
Damien Elmes
8372931b9b use v2 scheduler+new timezone handling by default for new users
- In corner cases, enabling the new timezone handling later can cause
reviews to shift forward or back a day, so it's best to have it on
by default.
- https://github.com/ankidroid/Anki-Android/issues/5805 has not landed
in a stable release yet, but will hopefully not be too far off by the
time 2.1.41 is released.
- Existing users will be unaffected, as the upgrade prompt in the previous
commit asks them if they use AnkiDroid.
- Users starting on AnkiDroid will be unaffected, as their collections
will still be on V1.
- The error message AnkiWeb gives when syncing an older AnkiDroid
with the new timezone enabled has been updated to direct users to the
preferences screen.
2021-02-21 17:18:08 +10:00
Damien Elmes
5ae66af5d2 rework v2 scheduler upgrade; drop downgrade
- Rework V2 upgrade so that it no longer resets cards in learning,
or empties filtered decks.
- V1 users will receive a message at the top of the deck list
encouraging them to upgrade, and they can upgrade directly from that
screen.
- The setting in the preferences screen has been removed, so users
will need to use an older Anki version if they wish to switch back to
V1.
- Prevent V2 exports with scheduling from being importable into a V1
collection - the code was previously allowing this when it shouldn't
have been.
- New collections still default to v1 at the moment.

Also add helper to get map of decks and deck configs, as there were
a few places in the codebase where that was required.
2021-02-21 15:50:41 +10:00
Damien Elmes
a50601ed46 add tag drag & drop support 2021-02-02 20:14:04 +10:00
Damien Elmes
467064f873 collapsed->expanded in other tag uses for consistency 2021-02-02 18:52:57 +10:00
Damien Elmes
52b256663f collapsed->expanded in TagTreeNode 2021-02-02 18:52:57 +10:00
Damien Elmes
7d8448a321 decks start collapsed 2021-02-02 17:14:11 +10:00
Damien Elmes
163a9df886 avoid bumping mtime when correcting card eases on upgrade
The USN is still set, which should cause the cards to sync in the
non-conflict case, but if changes have been made on other devices
the ease fix will not take priority, as we could be overwriting the
reviews of someone who has not synced in a while.
2021-01-28 08:11:35 +10:00
Kerrick Staley
1eac36c5d7 Add comment explaining why sfld column has type integer
The `notes.sfld` column in the `collection.anki2` database has type `integer`, even though it contains text. This is [something SQLite allows](https://dba.stackexchange.com/questions/106364/text-string-stored-in-sqlite-integer-column) and was done to allow integer values to sort numerically. Add a comment explaining this.
2021-01-25 18:39:59 +08:00
Damien Elmes
ef9f6b022b coerce some more invalid decimal values to ints during sync 2021-01-25 09:43:50 +10:00
abdo
0ac97cf358 clear_tag() should be case-insensitive 2021-01-18 03:52:28 +03:00
Damien Elmes
9f964916ab simplify unused tags and DB check
- backend routines should contain minimal logic, and should call
into a routine on the collection
- instead of copying the giant-string approach the Python code was taking,
we use a HashSet to keep track of seen tags as we loop through the
notes, which should be more efficient
2021-01-16 20:38:16 +10:00
Damien Elmes
d80a5c56e3 no need for separate all_tags_sorted()
tag is the primary key, so sqlite will give it back to us in
sorted order already.
2021-01-16 19:46:58 +10:00
abdo
e018ea94b6 Merge branch 'master' of https://github.com/ankitects/anki into tagtree 2021-01-15 01:12:01 +03:00
abdo
ee3c019804 Remove Default impl of Tag 2021-01-14 07:04:55 +03:00
abdo
9c1d7c522a Refactor code for clearing unused tags and saving collapse state 2021-01-14 07:04:14 +03:00
abdo
9a68d84483 Keep tags in human form and update the tags table structure
See https://github.com/ankitects/anki/pull/900#issuecomment-758284016

- Leave tag names alone and add the collapsed and config columns to the tags table.
- Update The DB check code to preserve the collapse state of used tags.
- Add a simple test for clearing tags and their children
2021-01-12 23:12:35 +03:00
Damien Elmes
09dfa9ced6 allow normal sync tests to run offline 2021-01-12 18:47:08 +10:00
abdo
97b4c2124c sql formatting 2021-01-09 17:38:16 +03:00
abdo
1be789f25f Move sql code for upgrading to schema 17 to a separate file 2021-01-09 17:10:16 +03:00
abdo
c6e3d55400 fill_missing_tags's input should be sorted
I assumed that fill_missing_tags will work correctly with un unsorted
tag list previously so I replaced the all_tags_sorted call, but take the following the list for example:

["foo::bar", "foo"]

This will cause "foo" to be counted like a missing tag, since it's
encountered the first time when looking at "foo::bar"", and its config
and other associated data will be lost.
2021-01-09 17:10:16 +03:00
abdo
b33267f754 Do not check for missing tag parents at registration time 2021-01-09 17:10:16 +03:00
abdo
b276ce3dd5 Hierarchical tags 2021-01-09 17:10:13 +03:00
Damien Elmes
56e4e2d518 reformat .sql files 2021-01-09 14:08:55 +10:00
Damien Elmes
dee0a2fc18 fix reposition not being applied in browser sort order
https://forums.ankiweb.net/t/reposition-function-not-working/5817

Also changed id->cid in the non-sorting case, as otherwise when
using all_searched_cards() on a sorted list, the results will be wrong.
2020-12-16 10:58:46 +10:00
Damien Elmes
a9ea8e11a2 fix filtered decks not honoring sort order
https://forums.ankiweb.net/t/2-1-36-filtered-decks-bug/5649/
2020-12-09 22:50:49 +10:00
Damien Elmes
009906d0c2 fix duplicate search not checking first field
Thanks to abdo for the initial solution. Closes #838
2020-11-30 09:27:10 +10:00
Damien Elmes
c157ccb3f5 handle notes with invalid utf8 2020-11-06 10:21:51 +10:00
Damien Elmes
90fa38495e reindex when checking database
Came across a user with a corrupt index:

sqlite> pragma integrity_check;
 integrity_check = wrong # of entries in index ix_revlog_cid
 integrity_check = wrong # of entries in index ix_cards_sched

This is not picked up by a quick check, and a vacuum does not
fix it, but a reindex does.

AnkiWeb currently performs a full check when a file is uploaded
to it, so this was leading to "corrupt" to show up when syncing
the collection, with a local DB check not reporting/fixing the issue.
2020-10-05 17:40:43 +10:00
Damien Elmes
cc3fdb2f37 rename downgrade method for clarity 2020-09-22 15:53:26 +10:00
Damien Elmes
76acf04dc0 update schema to fix default initial ease in deck configs
Closes #766

- changes the on-disk representation from % to a multiplier,
eg 250 -> 2.5, as this is consistent with the other options
- resets deck configs at or below 1.3 to 2.5
- for any cards that were using a reset deck config, reset their
current factor if it's at or below 2.0x. The cutoff is arbitrary,
and just intended to make sure we catch cards the user has rated
Easy on multiple times. The existing due dates are left alone.
2020-09-22 08:16:39 +10:00
Damien Elmes
755a58b96c fix incorrect SQL in previous change 2020-09-19 12:27:00 +10:00
Damien Elmes
91759694a1 congrats_info() failed if learning card had negative due
https://forums.ankiweb.net/t/integralvalueoutofrange-for-filtered-deck/3353
2020-09-18 20:14:35 +10:00
Damien Elmes
56ceb6ba76 set_deck() 2020-09-03 17:42:46 +10:00
Damien Elmes
b65174a026 move card sorting and resetting to backend 2020-09-03 15:54:15 +10:00
Damien Elmes
ce49ca9401 log manual reschedule, but ignore the log entry in the stats 2020-09-02 17:56:23 +10:00
Damien Elmes
a834df60ce rename some card+note fields in backend 2020-09-01 10:24:38 +10:00
Damien Elmes
d3dede057a move bury/suspend into backend 2020-09-01 10:24:38 +10:00
Damien Elmes
ccfa989c62 move unbury/unsuspend routines into backend 2020-09-01 10:24:38 +10:00
Damien Elmes
b0a3861734 handle null return from sum() in other columns 2020-09-01 10:24:38 +10:00
Damien Elmes
e5685254c6 reimplement congrats screen in Rust+Typescript 2020-09-01 10:24:38 +10:00
Damien Elmes
4662a9fe1a check if we need to unbury at the start of the sync process
https://forums.ankiweb.net/t/bug-buried-cards-in-filtered-decks-not-being-unburied-next-day/2541/24
2020-08-28 21:09:07 +10:00
Damien Elmes
134765036c don't crash when upgrading collection with ntid of 0 2020-08-23 14:14:44 +10:00
Damien Elmes
9334272a27 generate extra templates when recovering notetype 2020-08-10 13:42:37 +10:00
Damien Elmes
0180868b60 fix negative review times in DB check 2020-08-07 14:37:44 +10:00
Damien Elmes
a1866e72fd handle preview repeat in deck list 2020-07-30 09:11:13 +10:00
Damien Elmes
5574c0dfb3 set schema on collection creation 2020-07-03 11:55:27 +10:00
Damien Elmes
0f1f80aebc handle revlog entries with negative review times 2020-06-29 15:48:01 +10:00
Damien Elmes
0cab26d40c cards added graph 2020-06-29 15:48:01 +10:00
Damien Elmes
82568a1f3e experiment with exposing raw card/revlog data to frontend 2020-06-29 15:48:00 +10:00
Damien Elmes
510f8b86cb some initial work on updating the graphs 2020-06-29 15:48:00 +10:00
Damien Elmes
df009d142c formatting fix 2020-06-22 11:58:31 +10:00
Damien Elmes
26fc6609a7 move revlog out of sync code, and add enums
and remove type=4, which does not appear to be used
2020-06-16 16:31:36 +10:00
Damien Elmes
b51f03085e migrate card stats to backend
Currently this renders the HTML directly like the previous Python
implementation - doing it in JS would probably make more sense in the
future.
2020-06-15 17:22:16 +10:00
Damien Elmes
263bfb2e8e on upgrade, mark decks modified that are renamed 2020-06-14 14:48:58 +10:00
Damien Elmes
816cec4a58 fix review cards with high intervals not being reset properly
would lead to a "fixed .. with invalid card property" message that
didn't go away
2020-06-09 20:11:26 +10:00
Damien Elmes
7c444b4d35 add progress to db check 2020-06-08 21:07:36 +10:00
Damien Elmes
1fe18718f7 add daily count updating to backend 2020-06-05 19:49:53 +10:00
Damien Elmes
4a69b55a90 add note/card removal to backend 2020-06-04 18:21:04 +10:00
Damien Elmes
4d7e23111e change sync label to indicate sync state
- blue for normal sync, red for full sync required
- refactor status fetching code so we don't hold a collection lock
during the network request, which slows things down
- fix sync spinner restarting when returning to deck list
2020-06-02 13:23:01 +10:00
Damien Elmes
529152aef6 rework usn handling in sync 2020-06-01 13:57:10 +10:00
Damien Elmes
0e5b7da62a login/full up/full down plugged in 2020-06-01 13:57:10 +10:00
Damien Elmes
b254b1f722 another snapshot of sync work
progress handler is going to need to be reworked to function across
threads
2020-06-01 13:57:10 +10:00
Damien Elmes
f10d0ee0cb more syncing work
no checks yet
2020-06-01 13:57:10 +10:00
Damien Elmes
c8d13209cd move dupe check to backend 2020-05-24 19:48:56 +10:00
Damien Elmes
cdbe7346ab bytes->blob in sql 2020-05-18 13:36:45 +10:00
Damien Elmes
b9b837e7bd update before_upload() 2020-05-17 14:13:21 +10:00
Damien Elmes
6114836484 move deck config to protobuf 2020-05-17 13:28:41 +10:00
Damien Elmes
54670580ad add option to limit deck tree counts to a particular node 2020-05-17 08:38:49 +10:00
Damien Elmes
da43d22aa5 update rusqlite 2020-05-16 19:49:08 +10:00
Damien Elmes
964a69e54e handle default deck and filtered deck suppression in the backend 2020-05-15 21:21:10 +10:00
Damien Elmes
826cbb0108 fetch template and note fields in backend during normal card render
Saves having to serialize the note fields and q/a templates, which
is particularly a win when rendering question/answer in the browse
screen.

Also some work towards being able to preview notes without having to
commit them to the database.
2020-05-13 10:38:49 +10:00
Damien Elmes
9317cee9ba handle scheduling preferences in the backend 2020-05-12 21:13:34 +10:00
Damien Elmes
2c7900989c fix default rollover 2020-05-12 21:13:34 +10:00
Damien Elmes
8af475ac6f add tests for the other checks, and fix new card due limiting 2020-05-12 21:13:34 +10:00
Damien Elmes
2b47e544a8 add tests for card fixes; implement deck recovery 2020-05-12 21:13:34 +10:00
Damien Elmes
5ec7251f6e stop using memory for sqlite temp store
On large collections this can end up consuming hundreds of megabytes
of RAM on operations like a vacuum.
2020-05-12 21:13:34 +10:00
Damien Elmes
70cc1699a6 rewrite DB check
- notes with wrong field count are now recovered instead of
being deleted
- notes with missing note types are now recovered
- notes with missing cards are now recovered
- recover_missing_deck() still needs implementing
- checks required
2020-05-12 21:13:34 +10:00
Damien Elmes
4c9b6be832 fetch notes in nid order
May reduce the amount of seeking on conventional disks. Chunking and
fetching batches of notes at once would improve things further, at the
cost of more memory use.
2020-05-12 21:13:34 +10:00
Damien Elmes
2413f286b1 bulk tag add/remove/update; canonify on note save
also remove the tag list updated hook - we'll need a better solution in
the future than having the library code call back into the GUI code
2020-05-12 21:13:34 +10:00
Damien Elmes
8b557ec382 move find&replace to backend 2020-05-12 21:13:34 +10:00
Damien Elmes
2308b136fd add an index on notes(mid) so we can avoid the tablescan 2020-05-12 21:13:34 +10:00
Damien Elmes
238441f2d9 use the backend for the deck due tree
- approx 3x faster on a large test deck
- counts are no longer capped to 1000 in the tree
2020-05-12 21:13:33 +10:00
Damien Elmes
67421e02ec add browserCollapsed as part of tree build 2020-05-12 21:13:33 +10:00
Damien Elmes
5cefece264 switch primary key to deck id 2020-05-12 21:13:33 +10:00
Damien Elmes
3ffb37270d add partial index on odid, and search odid as well
the odid != 0 check in cards_for_deck.sql is necessary for sqlite
to be able to take advantage of the index
2020-05-12 21:13:33 +10:00
Damien Elmes
f592672fa9 add separate decks table, and start on moving deck handling to Rust
The Python tests are passing, but there are still a number of issues
to work through, and the table/protobuf schema is not yet finalized.
2020-05-12 21:13:33 +10:00
Damien Elmes
36531ea96d add (currently disabled) check for missing/empty decks on card add 2020-05-12 21:13:33 +10:00
Damien Elmes
5ac3fb5514 handle negative due numbers when gathering existing cards 2020-05-12 21:13:33 +10:00
Damien Elmes
87cd119216 fix use counts not including unused note types 2020-05-12 21:13:33 +10:00
Damien Elmes
a181d9aa02 note type removal 2020-05-12 21:13:33 +10:00
Damien Elmes
6cc2bdbf87 start on exposing notes and individual note type methods
changes to note:

- add_note() now takes a provided deck id instead of looking it up
in the notetype
- note type use counts fetched using a single table scan
- make sure note type changes are persisted
- expose optionalness of ords in templates and fields json
2020-05-12 21:13:33 +10:00
Damien Elmes
b89dd32f78 mod schema if field/templates changed
Instead of throwing an error if schema not marked as changed,
just mark it changed, as that way it can be included as part of
the same transaction.
2020-05-12 21:13:33 +10:00
Damien Elmes
83bcb084fe template changes and card generation
Cloze cards are not yet supported, missing decks are not handled,
and more testing is still required.
2020-05-12 21:13:33 +10:00
Damien Elmes
a226f70733 cache notetypes 2020-05-12 21:13:33 +10:00
Damien Elmes
e8f4a75bd5 always fetch full notetype 2020-05-12 21:13:33 +10:00
Damien Elmes
f75fd5335d more work towards note adding
Still a prototype at this stage - we'll likely want a caching layer
for note types, and I'm not sure of the merit of having fields in
a separate table, since they're almost always required.
2020-05-12 21:13:33 +10:00
Damien Elmes
f24dc05c8d the start of note adding, and note type changes 2020-05-12 21:13:33 +10:00