GitHub/Anki
1
0
Fork 0
mirror of https://github.com/ankitects/anki.git synced 2026-01-15 14:59:00 -05:00
Code Issues Projects Releases Packages 1 Wiki Activity Actions
7243 commits 11 branches 166 tags 122 MiB
Commit graph

1212 commits

Author SHA1 Message Date
Damien Elmes
71dbbd9fa8 set schema on collection creation 2020-07-03 11:55:27 +10:00
Damien Elmes
9bb45673a8 show clearer error when media db locked 
should not happen frequently during a sync, but may happen during
the initial change checking stage

https://forums.ankiweb.net/t/error-when-adding-audio-file/495/2
 2020-07-01 11:19:06 +10:00
Damien Elmes
f43aa54ed9 fix media db conflict aborting immediately instead of after 5s 
https://forums.ankiweb.net/t/error-when-adding-audio-file/495/2
 2020-07-01 11:17:33 +10:00
Damien Elmes
8cfafdbc32 supported edited:n 
https://forums.ankiweb.net/t/feature-request-advanced-broswer-search-by-edited/268
 2020-07-01 10:17:57 +10:00
Damien Elmes
33317d41b3 update edited/changed column labels 
https://github.com/ankitects/anki/pull/674/files
 2020-06-29 18:34:03 +10:00
Damien Elmes
06d9f64dea lint fixes 2020-06-29 15:53:56 +10:00
Damien Elmes
24cf78b68b group strings by graph 2020-06-29 15:48:01 +10:00
Damien Elmes
275817a558 add remaining tooltip i18n 2020-06-29 15:48:01 +10:00
Damien Elmes
a7ce130941 i18n some tooltips 2020-06-29 15:48:01 +10:00
Damien Elmes
35e706d53a drop axis labels + add subtitles 
This leaves more room for the graphs and avoids issues with large values
overlapping the labels. We can improve the subtitle wording to make
things clearer in the future, and the hover tooltips should also
clarify what the axes represent.
 2020-06-29 15:48:01 +10:00
Damien Elmes
2994cadcb0 i18n some axis labels, and support vertical CJK text 2020-06-29 15:48:01 +10:00
Damien Elmes
042a9ff68b i18n graph titles 2020-06-29 15:48:01 +10:00
Damien Elmes
1a16e2306f i18n ranges 2020-06-29 15:48:01 +10:00
Damien Elmes
693fcfce7d i18n card counts 2020-06-29 15:48:01 +10:00
Damien Elmes
cebf22d749 i18n today section 2020-06-29 15:48:01 +10:00
Damien Elmes
349eca843b get i18n working in typescript 2020-06-29 15:48:01 +10:00
Damien Elmes
618fc65282 handle revlog entries with negative review times 2020-06-29 15:48:01 +10:00
Damien Elmes
4be8e7923a fix partially quoted text not handling escaped quotes 
https://forums.ankiweb.net/t/how-to-search-for-cards-containing-quotes/500/10
 2020-06-29 15:48:01 +10:00
Damien Elmes
fefc6dbdc2 bump rust nightly version 2020-06-29 15:48:01 +10:00
Damien Elmes
0293f2220c qt's js code now shares ts/node_modules; more graph work 2020-06-29 15:48:01 +10:00
Damien Elmes
4d88b6521d cards added graph 2020-06-29 15:48:01 +10:00
Damien Elmes
1e9901e9bd experiment with exposing raw card/revlog data to frontend 2020-06-29 15:48:00 +10:00
Damien Elmes
fd258d15c5 some initial work on updating the graphs 2020-06-29 15:48:00 +10:00
Damien Elmes
df8be64da3 protobuf package should be in PascalCase 2020-06-29 15:48:00 +10:00
Damien Elmes
61efce295e fix v2 scheduler cutoff being used if previously activated 2020-06-23 12:46:40 +09:00
Damien Elmes
710e5ecce2 formatting fix 2020-06-22 11:58:31 +10:00
Damien Elmes
cec1fdece5 move revlog out of sync code, and add enums 
and remove type=4, which does not appear to be used
 2020-06-16 16:31:36 +10:00
Damien Elmes
f5694a7e4a migrate card stats to backend 
Currently this renders the HTML directly like the previous Python
implementation - doing it in JS would probably make more sense in the
future.
 2020-06-15 17:22:16 +10:00
Damien Elmes
965fd848d1 on upgrade, mark decks modified that are renamed 2020-06-14 14:48:58 +10:00
Damien Elmes
07874f8328 handle db errors the same way as other backend requests 2020-06-12 20:39:02 +10:00
Damien Elmes
dc8297644b pass db json back as bytes 2020-06-12 20:20:29 +10:00
Damien Elmes
1773c6eec9 don't exclude the default deck from the normal name list 
https://forums.ankiweb.net/t/issue-backend-get-deck-names-doesnt-return-default-deck/52
 2020-06-11 14:36:00 +10:00
Damien Elmes
4092da9dbe use vendored hyper for panic fix 2020-06-11 13:30:35 +10:00
Damien Elmes
8f9d2f592e extra strings for empty cards 2020-06-10 19:29:25 +10:00
Damien Elmes
5bd66d485f add check database title 2020-06-10 08:44:57 +10:00
Damien Elmes
d5cd9af53b fix review cards with high intervals not being reset properly 
would lead to a "fixed .. with invalid card property" message that
didn't go away
 2020-06-09 20:11:26 +10:00
Damien Elmes
a11d803030 switch default leech action to tag only 2020-06-09 12:03:47 +10:00
Damien Elmes
6d7be475e6 catch invalid regex in search 
https://github.com/ankitects/anki/pull/652
 2020-06-09 11:46:07 +10:00
Damien Elmes
c6e5dc0ef6 check full upload return text 2020-06-09 08:50:16 +10:00
Damien Elmes
6207b1012f Revert "use trust-dns in reqwest" 
This reverts commit f15abb5c58.

trust-dns has introduced its own issues, such as when resolv.conf is
inaccessible, or contains hostnames with certain characters
 2020-06-08 22:03:24 +10:00
Damien Elmes
75471224fa add progress to db check 2020-06-08 21:07:36 +10:00
Damien Elmes
e74e5c604b add daily count updating to backend 2020-06-05 19:49:53 +10:00
Damien Elmes
55f8b87548 support generating a due tree for a different date 2020-06-05 09:38:31 +10:00
Damien Elmes
c5e65d02e1 add note/card removal to backend 2020-06-04 18:21:04 +10:00
Damien Elmes
f15abb5c58 use trust-dns in reqwest 
The default DNS resolution, if interrupted by an Abortable, will
cause a panic:

57ef271500/src/client/connect/dns.rs (L144)
 2020-06-03 15:47:54 +10:00
Damien Elmes
cb7d78dde5 "Template" special field should have been called "Card" 2020-06-02 16:47:30 +10:00
Damien Elmes
c8b585b3e4 automatically abort the media sync on full sync 2020-06-02 16:22:53 +10:00
Damien Elmes
7dc8df748b ensure sync indicator updates after download 2020-06-02 15:10:41 +10:00
Damien Elmes
7536a3b899 fix full download on Windows 2020-06-02 15:06:23 +10:00
Damien Elmes
ca704187c6 skip note update if unchanged 2020-06-02 13:32:36 +10:00
Damien Elmes
d7a2e78c0b change sync label to indicate sync state 
- blue for normal sync, red for full sync required
- refactor status fetching code so we don't hold a collection lock
during the network request, which slows things down
- fix sync spinner restarting when returning to deck list
 2020-06-02 13:23:01 +10:00
Damien Elmes
42c34b60c0 use a shared async runtime instead of recreating each time 2020-06-02 09:35:27 +10:00
Damien Elmes
a3e5068952 add some syncing tests - please see alpha2 notes before syncing 2020-06-01 19:44:25 +10:00
Damien Elmes
c6712cbf2c fix \n in buildhash 2020-06-01 19:38:45 +10:00
Damien Elmes
80210ab6bd handle missing counts in sanity check 2020-06-01 19:38:41 +10:00
Damien Elmes
300769aac7 if both sides empty, allow either 2020-06-01 18:11:43 +10:00
Damien Elmes
2ca3108bee don't put dyn=False in schema11 dconf's "other" 2020-06-01 18:11:40 +10:00
Damien Elmes
da2dc7bd8c tell AnkiWeb to abort on interrupt 2020-06-01 13:57:10 +10:00
Damien Elmes
ad3082d1c8 handle aborting normal sync 
- Use a separate abort handle, as the media sync is running
in the background and we need to be able to target them separately.
The current progress handling is going to need a rethink if we introduce
any other background tasks in the future.
- Roll back the transaction when interrupting.
 2020-06-01 13:57:10 +10:00
Damien Elmes
da1e0527ca report normal sync progress 
Also:
- provide a way for the progress handler to skip the throttling so that
we can ensure progress is updated at the end of a stage
- show 'checking' at the end of full sync
 2020-06-01 13:57:10 +10:00
Damien Elmes
1c52d11133 rework usn handling in sync 2020-06-01 13:57:10 +10:00
Damien Elmes
1303013ad6 add due counts to sanity check 2020-06-01 13:57:10 +10:00
Damien Elmes
0787381a99 login/full up/full down plugged in 2020-06-01 13:57:10 +10:00
Damien Elmes
aaed0464e5 rework progress handling 
- client now polls status instead of backend pushing it
- supports multiple threads
- update throttling happens in one place
 2020-06-01 13:57:10 +10:00
Damien Elmes
c4032c0176 another snapshot of sync work 
progress handler is going to need to be reworked to function across
threads
 2020-06-01 13:57:10 +10:00
Damien Elmes
b5284e8993 make sure we invalidate cache after direct update 2020-06-01 13:57:10 +10:00
Damien Elmes
2114fdf6f5 more syncing work 
no checks yet
 2020-06-01 13:57:10 +10:00
Damien Elmes
4d1e8d678d full upload/download HTTP code 2020-06-01 13:57:10 +10:00
Damien Elmes
f17dec9f2a start on HTTP client for syncing 2020-06-01 13:57:10 +10:00
Damien Elmes
91e3256e6a move dupe check to backend 2020-05-24 19:48:56 +10:00
Damien Elmes
0606bee2c5 fix rebuilding being done unnecessarily 2020-05-24 18:41:25 +10:00
Damien Elmes
e9fea26d1c use keyword args for calls with more than one argument 2020-05-24 09:12:47 +10:00
Damien Elmes
bbcc566e6d migrate the remaining methods 2020-05-24 08:36:50 +10:00
Damien Elmes
93f18c14d4 migrate more scheduling/media/etc 
almost there
 2020-05-23 21:34:19 +10:00
Damien Elmes
8b93a0c19d migrate notetypes, update GIL list 2020-05-23 20:43:55 +10:00
Damien Elmes
35489d5fa7 add some more newtypes to the RPC defs 2020-05-23 16:58:01 +10:00
Damien Elmes
8df2ab65fb migrate cards and notes 2020-05-23 16:19:48 +10:00
Damien Elmes
32a071cf79 migrate decks and dconf methods 2020-05-23 15:09:16 +10:00
Damien Elmes
927ed25b02 more methods 2020-05-23 14:01:36 +10:00
Damien Elmes
d57ded706f migrate more methods to service 2020-05-22 22:09:33 +10:00
Damien Elmes
36024ed086 start reworking protobuf handling 
Will allow us to cut down on boilerplate by automatically generating
code from RPC service definitions
 2020-05-22 20:56:15 +10:00
Damien Elmes
3f8a3afb78 fix extending limits in custom study 
https://anki.tenderapp.com/discussions/beta-testing/1967-2126-189-g7384df8f-crash-at-note-types-fields-save
 2020-05-20 14:13:40 +10:00
Damien Elmes
c78e8425aa automatically omit notes or cards table when possible 2020-05-19 11:27:02 +10:00
Damien Elmes
5e1583df59 fix unit test bug triggered by low resolution win32 timer 
We don't yet have a remove_note(), so just remove all cards for now.
 2020-05-18 19:02:54 +10:00
Damien Elmes
facbe53649 fix cutoff adjustment test in other timezones 2020-05-18 14:08:02 +10:00
Damien Elmes
01ebe964e9 bytes->blob in sql 2020-05-18 13:36:45 +10:00
Damien Elmes
3e156fdb49 bump version 2020-05-18 12:50:20 +10:00
Damien Elmes
4438be2c8d protobuf tidyups 2020-05-18 09:26:14 +10:00
Damien Elmes
e894125620 use empty buffer if the other map is empty in dconf 2020-05-18 08:12:27 +10:00
Damien Elmes
ef88cb59e6 fix syncing 2020-05-17 20:52:02 +10:00
Damien Elmes
e6379bc4d0 update current note type in backend 2020-05-17 20:01:16 +10:00
Damien Elmes
e230f360e9 ensure notetype name unique on add as well 2020-05-17 19:41:47 +10:00
Damien Elmes
b676522e5c fix renaming corner cases and decks.update() 
- .update() should update a single deck and preserve usn by default,
as that's what existing code expects
- decks are automatically renamed when they conflict with an existing
name
 2020-05-17 19:07:15 +10:00
Damien Elmes
2c19e171ed handle deletion of default deck in backend; use + instead of _ 2020-05-17 15:22:19 +10:00
Damien Elmes
336349d8d1 update before_upload() 2020-05-17 14:13:21 +10:00
Damien Elmes
a682865169 move deck config to protobuf 2020-05-17 13:28:41 +10:00
Damien Elmes
e85911c4e4 add option to limit deck tree counts to a particular node 2020-05-17 08:38:49 +10:00
Damien Elmes
17efa85d75 update rusqlite 2020-05-16 19:49:08 +10:00
Damien Elmes
e36e23b563 when merging day# in deck, make sure to reset older values 2020-05-16 15:58:25 +10:00
Damien Elmes
2e5f745b85 use the latest day when converting from legacy deck 2020-05-16 14:58:14 +10:00
Damien Elmes
70afc782ca add a unit test for deck counts 2020-05-16 14:07:48 +10:00
Damien Elmes
c60229779c switch to new deck tree in deck browser 
Saves us having to look up collapsed/filtered as we render, and gives
us type completion.
 2020-05-16 10:52:14 +10:00
Damien Elmes
cbfc9d5a87 handle default deck and filtered deck suppression in the backend 2020-05-15 21:21:10 +10:00
Damien Elmes
dc6471ad40 fetch stock notetypes from backend 2020-05-15 17:08:24 +10:00
Damien Elmes
91e71ae044 show (Deck) when card hasn't been added 2020-05-15 15:01:28 +10:00
Damien Elmes
8dcb7fd2ba check for empty cloze cards when rendering as well 2020-05-14 21:56:45 +10:00
Damien Elmes
6dd31341f8 add "fill empty" checkbox 2020-05-14 20:58:45 +10:00
Damien Elmes
4c52d41632 use group box instead of tabs in card layout screen 2020-05-14 17:26:40 +10:00
Damien Elmes
4a00ad075c drop availOrds(), and use backend for getting cloze numbers 2020-05-14 12:14:00 +10:00
Damien Elmes
257e893d95 update the card layout screen 
- changes are now committed in bulk when closing the dialog,
and can be canceled
- it's not necessary to save the note to the database to preview it
- duplicate fields are now shown as duplicates in the top list
- redraw preview more quickly
- use + instead of _ when deduplicating names, as the latter is a
glob character
 2020-05-13 17:24:49 +10:00
Damien Elmes
4815010da0 don't hide static template text when card is empty 2020-05-13 11:17:44 +10:00
Damien Elmes
14c0a5891b fetch template and note fields in backend during normal card render 
Saves having to serialize the note fields and q/a templates, which
is particularly a win when rendering question/answer in the browse
screen.

Also some work towards being able to preview notes without having to
commit them to the database.
 2020-05-13 10:38:49 +10:00
Damien Elmes
a95fbe73c3 handle decks set to random new order 
It probably makes more sense to randomize on queue build in the future,
but for now this imitates the previous Anki behaviour.
 2020-05-12 21:13:34 +10:00
Damien Elmes
9c3f982c3b return resolved deck to facilitate random order mode 2020-05-12 21:13:34 +10:00
Damien Elmes
21ac2c14a7 handle scheduling preferences in the backend 2020-05-12 21:13:34 +10:00
Damien Elmes
2eaa759478 fetch timing_today() params in backend 2020-05-12 21:13:34 +10:00
Damien Elmes
1018204e4f switch server back into a bool and rely on config 2020-05-12 21:13:34 +10:00
Damien Elmes
befa66fb27 fix default rollover 2020-05-12 21:13:34 +10:00
Damien Elmes
dc9200e639 add tests for the other checks, and fix new card due limiting 2020-05-12 21:13:34 +10:00
Damien Elmes
c77bc74a2b rename SortMode arg to match enum 2020-05-12 21:13:34 +10:00
Damien Elmes
bdf5daa422 add tests for card fixes; implement deck recovery 2020-05-12 21:13:34 +10:00
Damien Elmes
c227298565 stop using memory for sqlite temp store 
On large collections this can end up consuming hundreds of megabytes
of RAM on operations like a vacuum.
 2020-05-12 21:13:34 +10:00
Damien Elmes
d7f9bccd1e rewrite DB check 
- notes with wrong field count are now recovered instead of
being deleted
- notes with missing note types are now recovered
- notes with missing cards are now recovered
- recover_missing_deck() still needs implementing
- checks required
 2020-05-12 21:13:34 +10:00
Damien Elmes
b03dcd2643 fetch notes in nid order 
May reduce the amount of seeking on conventional disks. Chunking and
fetching batches of notes at once would improve things further, at the
cost of more memory use.
 2020-05-12 21:13:34 +10:00
Damien Elmes
667f9e6554 bulk tag add/remove/update; canonify on note save 
also remove the tag list updated hook - we'll need a better solution in
the future than having the library code call back into the GUI code
 2020-05-12 21:13:34 +10:00
Damien Elmes
5d493b6ab8 add the Extra field back to cloze deletions as "Back Extra" 
Originally removed because some users were adding cloze deletions to
it, but removing it just replaced that problem with a new problem where
users add the field but don't add it to their card templates.
 2020-05-12 21:13:34 +10:00
Damien Elmes
0a87b1ef2c support disabling unicode normalization in notes 2020-05-12 21:13:34 +10:00
Damien Elmes
85528cabc5 use backend for genCards() and updateFieldCache() 2020-05-12 21:13:34 +10:00
Damien Elmes
8c04718541 update find&replace in browser 
sadly the UI still stutters on large selections - the calls
to get the selected rows from Qt are really slow.
 2020-05-12 21:13:34 +10:00
Damien Elmes
ad1c519d1c move find&replace to backend 2020-05-12 21:13:34 +10:00
Damien Elmes
2e971da79e properly handle negated conditionals outside of req generation 2020-05-12 21:13:34 +10:00
Damien Elmes
2d0a6283e4 add an index on notes(mid) so we can avoid the tablescan 2020-05-12 21:13:34 +10:00
Damien Elmes
3abd72564a fix sort field check on save 2020-05-12 21:13:33 +10:00
Damien Elmes
e32be05451 catch attempts to overwrite notetype with older version 2020-05-12 21:13:33 +10:00
Damien Elmes
7ef72aba04 use the backend for the deck due tree 
- approx 3x faster on a large test deck
- counts are no longer capped to 1000 in the tree
 2020-05-12 21:13:33 +10:00
Damien Elmes
267be1b050 add missing decks in backend 
- need to compare parents with unicode case folding
- duplicate check enforced by the DB
 2020-05-12 21:13:33 +10:00
Damien Elmes
97e857a41c add browserCollapsed as part of tree build 2020-05-12 21:13:33 +10:00
Damien Elmes
13d23cd24b speed up browser load by rendering deck tree in Rust and skipping counts 2020-05-12 21:13:33 +10:00
Damien Elmes
611959e5a8 switch primary key to deck id 2020-05-12 21:13:33 +10:00
Damien Elmes
46e0469991 rework aux table sorting 
Instead of inserting text into the aux table and sorting on the fly
(which does a binary sort), we insert the ids in unicase-sorted order,
and then sort based on the automatically assigned rowids - giving
us faster sorts, and proper unicase folding.
 2020-05-12 21:13:33 +10:00
Damien Elmes
c727a9e3f1 fix case sensitivity of notetype/template searches 2020-05-12 21:13:33 +10:00
Damien Elmes
c9db36bc82 update searching code to use decks table 2020-05-12 21:13:33 +10:00
Damien Elmes
87551574c0 add partial index on odid, and search odid as well 
the odid != 0 check in cards_for_deck.sql is necessary for sqlite
to be able to take advantage of the index
 2020-05-12 21:13:33 +10:00
Damien Elmes
24ef0cc6f1 add separate decks table, and start on moving deck handling to Rust 
The Python tests are passing, but there are still a number of issues
to work through, and the table/protobuf schema is not yet finalized.
 2020-05-12 21:13:33 +10:00
Damien Elmes
cbfe20712e add (currently disabled) check for missing/empty decks on card add 2020-05-12 21:13:33 +10:00
Damien Elmes
082f9c95fd split decks into module 2020-05-12 21:13:33 +10:00
Damien Elmes
8481d224d6 remove notetype from cache when removing 2020-05-12 21:13:33 +10:00
Damien Elmes
2a17f99eaa remove unused _updateRequired and associated code 2020-05-12 21:13:33 +10:00
Damien Elmes
22440f32d7 remove :{} and leading/trailing whitespace from field names on save 2020-05-12 21:13:33 +10:00
Damien Elmes
d1cc4684d2 hook the empty cards code up to the GUI 2020-05-12 21:13:33 +10:00
Damien Elmes
d812853656 sort empty cards by notetype name 2020-05-12 21:13:33 +10:00
Damien Elmes
ed4c75c6c1 don't include non-empty notes in the list 2020-05-12 21:13:33 +10:00
Damien Elmes
c989e23012 handle negative due numbers when gathering existing cards 2020-05-12 21:13:33 +10:00
Damien Elmes
21d7df9473 empty card handling 2020-05-12 21:13:33 +10:00
Damien Elmes
4edeca6ed9 fix use counts not including unused note types 2020-05-12 21:13:33 +10:00
Damien Elmes
4cc070a240 note type removal 2020-05-12 21:13:33 +10:00
Damien Elmes
7106af61ff update template on field removals as well 2020-05-12 21:13:33 +10:00
Damien Elmes
3b07885911 handle changed sort field index 2020-05-12 21:13:33 +10:00
Damien Elmes
549f98340a update template when fields renamed 2020-05-12 21:13:33 +10:00
Damien Elmes
8dad5cc3ab switch to owned strings in ParsedTemplate 
will make it easier to cache the parsed results in the future,
and handle field renames & other transformations
 2020-05-12 21:13:33 +10:00
Damien Elmes
6578767c4e only compare to previous note type if it exists 2020-05-12 21:13:33 +10:00
Damien Elmes
b7bfe57de1 adjust sort index on save 2020-05-12 21:13:33 +10:00
Damien Elmes
c8cbc13bba start on exposing notes and individual note type methods 
changes to note:

- add_note() now takes a provided deck id instead of looking it up
in the notetype
- note type use counts fetched using a single table scan
- make sure note type changes are persisted
- expose optionalness of ords in templates and fields json
 2020-05-12 21:13:33 +10:00
Damien Elmes
f95fe72795 mod schema if field/templates changed 
Instead of throwing an error if schema not marked as changed,
just mark it changed, as that way it can be included as part of
the same transaction.
 2020-05-12 21:13:33 +10:00
Damien Elmes
eaecfca7d1 normal note types now generate a dummy card if required 
In the cloze deletion case, we already created a dummy card 0 when
no cloze deletions were found. This change makes normal note types
behave the same way - if no cards would be generated, a dummy card
0 is added to allow the note to be added.

This also applies when modifying note types - it is now possible
to delete card templates even if some notes only use that template,
as a dummy card 0 will be generated for notes that end up with no
cards left.
 2020-05-12 21:13:33 +10:00
Damien Elmes
db09122ec5 use default deck for new note types 2020-05-12 21:13:33 +10:00
Damien Elmes
9451d0e9ef cloze generation 2020-05-12 21:13:33 +10:00
Damien Elmes
d810ba5db9 template changes and card generation 
Cloze cards are not yet supported, missing decks are not handled,
and more testing is still required.
 2020-05-12 21:13:33 +10:00
Damien Elmes
3af6905ad0 fix infinite loop on unclosed {{/<% 2020-05-12 21:13:33 +10:00
Damien Elmes
0c8763edf3 cache notetypes 2020-05-12 21:13:33 +10:00
Damien Elmes
7ff6226a0e always fetch full notetype 2020-05-12 21:13:33 +10:00
Damien Elmes
d47e83c989 embed enum to avoid conflicts 2020-05-12 21:13:33 +10:00
Damien Elmes
6b8a45bcb6 more work towards note adding 
Still a prototype at this stage - we'll likely want a caching layer
for note types, and I'm not sure of the merit of having fields in
a separate table, since they're almost always required.
 2020-05-12 21:13:33 +10:00
Damien Elmes
da09c7781e handle legacy syntax in parser instead of modifying original template 
Simplifies things for the caller, and ensures legacy handling doesn't
get accidentally forgotten
 2020-05-12 21:13:33 +10:00
Damien Elmes
4a2256aecb the start of note adding, and note type changes 2020-05-12 21:13:33 +10:00
Damien Elmes
6e554e6585 add separate search_notes_only() 2020-05-12 21:13:33 +10:00
Damien Elmes
cbb76290c2 start work on field changes, and add search_cards to col 2020-05-12 21:13:33 +10:00
Damien Elmes
638e69af91 move note db code to storage 2020-05-12 21:13:33 +10:00
Damien Elmes
53fc3c2f9a move latex header and styling into separate files 2020-05-12 21:13:33 +10:00
Damien Elmes
3807dcacd5 native struct for CardTemplate 2020-05-12 21:13:33 +10:00
Damien Elmes
91767fc87b native struct for NoteField 
ord is made optional so we can keep track of changes in the future
 2020-05-12 21:13:33 +10:00
Damien Elmes
9c41210cf4 use native struct for NoteType 2020-05-12 21:13:33 +10:00
Damien Elmes
4566a35085 expose stock notetypes 2020-05-12 21:13:33 +10:00
Damien Elmes
65f12eeda2 add stock note types in backend 2020-05-12 21:13:33 +10:00
Damien Elmes
5e49ace907 add default note type translations to ftl 2020-05-12 21:13:33 +10:00
Damien Elmes
96300f5858 migrate the schema11 usages to new structs/sql queries 2020-05-12 21:13:33 +10:00
Damien Elmes
4b39a8b727 move schema11 note type code into separate file 2020-05-12 21:13:33 +10:00
Damien Elmes
475478cec5 split note types into separate tables 
- store the config in protobuf instead of json
- still loading+saving in bulk for now
- code using the schema11 structs needs to be migrated
 2020-05-12 21:13:33 +10:00
Damien Elmes
8a423e05fe ignore empty sound tags 
https://github.com/ankitects/anki/pull/612
 2020-05-12 20:53:50 +10:00
evandrocoan
49bb585338 Also bump rslib version on make_check 
f2b1796e22
 2020-05-11 13:29:12 -03:00
Damien Elmes
f2b1796e22 update crate version 
make check was missing this
 2020-05-11 20:41:16 +10:00
Damien Elmes
36ac3244c2 fix a unit test 2020-05-08 17:27:41 +10:00
Damien Elmes
a1e7690f35 fix dupe search with included formatting 
https://anki.tenderapp.com/discussions/ankidesktop/41286-2125-doesnt-show-duplicates
 2020-05-08 15:26:45 +10:00
evandrocoan
d62787aa49 Fixed rspy check not being called 
# Conflicts:
#	Makefile
 2020-05-01 22:51:17 -03:00
Damien Elmes
2c102f87c4 update rslib version 2020-05-02 10:46:58 +10:00
Damien Elmes
4c0d80ceae tolerate decks with missing mtime exported by third party software 2020-05-02 10:18:08 +10:00
Damien Elmes
722df62452 fix some clippy lints 2020-05-02 08:55:27 +10:00
Damien Elmes
e2c57b54a2 reset due on load if outside i32 bounds 
Some third party clients that create apkg files appear to be saving
a millisecond timestamp into this column. While this could also be
solved by increasing this field to a 64 bit number, some older clients
expect this column to be a 32 bit number.
 2020-04-30 11:57:35 +10:00
Damien Elmes
f2876cbd42 preserve entities when stripping HTML for MathJax 
https://anki.tenderapp.com/discussions/ankidesktop/40987-how-to-render-angled-brackets
 2020-04-30 11:17:38 +10:00
Damien Elmes
65a9eaa418 expose clock hack via env var 
closes #594
 2020-04-30 09:33:02 +10:00
Damien Elmes
76d60430d9 report filename when unable to read data 2020-04-30 08:02:08 +10:00
Damien Elmes
07b9671012 don't throw an error when a file is greater than MAXPATH 
https://anki.tenderapp.com/discussions/ankidesktop/40931-check-media-error
 2020-04-30 08:01:45 +10:00
Damien Elmes
39070abb89 drop type answer requirement special case 
While it makes no sense to have the typing box on the front without
another non-empty field, "the front of this card is empty" when
a typing box is on the template seems to be confusing some users, so
it is clearer to show them the box.
 2020-04-29 21:29:16 +10:00
Damien Elmes
6d404f1999 AnkiDroid expects timer to be an integer 
https://github.com/ankidroid/Anki-Android/issues/6089#issuecomment-621110598
 2020-04-29 20:52:00 +10:00
evandrocoan
15deb17737 Added an warning that rslib/build.rs is automatically generated file 2020-04-29 01:21:49 -03:00
Damien Elmes
91248aa8cf bump version 2020-04-28 08:57:56 +10:00
evandrocoan
e85c48298c Fixed make clean rule not working on Windows 2020-04-26 03:02:02 -03:00
Damien Elmes
811bcda14c support globbing chars inside word boundary search 2020-04-25 09:43:08 +10:00
Damien Elmes
2f8396a951 add w: shortcut for searching on word boundaries 2020-04-25 08:48:01 +10:00
Damien Elmes
93152043e1 report the missing filename in media check 
https://anki.tenderapp.com/discussions/ankidesktop/40578-check-media-error
 2020-04-17 17:17:42 +10:00
Damien Elmes
e051d8ab75 older clients require 'separate' on filtered decks 
the deckconf['new']['separate' property is unused, but older code
assumes filtered['separate'] exists, even though it doesn't do anything
with it

https://anki.tenderapp.com/discussions/beta-testing/1868-anki-2124-beta#comment_48240796
 2020-04-15 15:02:12 +10:00
Damien Elmes
f5f2782e3b embed sort kind enum to avoid future namespace conflicts 2020-04-14 20:31:10 +10:00
Damien Elmes
351c7b6a82 fix dyn being serialized as bool 
If you were running from git in the last ~30 hours, please load
your collection using the latest git, review one card, then force
a full upload to fix compatibility with other clients.
 2020-04-10 18:07:59 +10:00
Damien Elmes
b7696e6381 merge the 12-14 upgrade code 2020-04-10 17:46:57 +10:00
Damien Elmes
9b66ecd12d add socks support 2020-04-09 14:28:36 +10:00
Damien Elmes
76ecc9739e update rusqlite 2020-04-09 12:45:51 +10:00
Damien Elmes
0538d9b764 (de)serialize decks in backend 2020-04-09 12:41:59 +10:00
Damien Elmes
bca49bd054 load/save note types in backend 
This allows us to normalize bad data, and is the first step towards
splitting note types into separate tables.
 2020-04-08 10:05:07 +10:00
Damien Elmes
29d66cb717 skip terminal logging unless LOGTERM defined 
seems to be quite slow on Windows
 2020-04-07 10:25:47 +10:00
Damien Elmes
68142ed32e Cow<str> workaround no longer required 2020-04-07 10:08:03 +10:00
Damien Elmes
b0ea1c24d0 update rusqlite for param count assert fix 2020-04-07 08:54:57 +10:00
Damien Elmes
f4a6a12c6f avoid warning on int sortBackwards from older Anki versions 2020-04-06 17:11:03 +10:00
Damien Elmes
02d0d9a670 add separate json error 2020-04-06 16:29:31 +10:00
Damien Elmes
a9fd2da243 log to the terminal as well 2020-04-06 15:57:13 +10:00
Damien Elmes
913c5b9854 use bytes for deck config proto 2020-04-06 15:39:47 +10:00
Damien Elmes
6471adafcc use bytes for config proto 2020-04-06 15:39:47 +10:00
Damien Elmes
3ef959856f display warning when config load fails 2020-04-06 15:39:47 +10:00
Damien Elmes
d05722aa82 use blobs for config 2020-04-06 15:39:47 +10:00
Damien Elmes
b5022ad354 store config in separate DB table 
- mtime is tracked on each key individually, which will allow
merging of config changes when syncing in the future
- added col.(get|set|remove)_config()
- in order to support existing code that was mutating returned
values (eg col.conf["something"]["another"] = 5), the returned list/dict
will be automatically wrapped so that when the value is dropped, it
will save the mutated item back to the DB if it's changed. Code that
is fetching lists/dicts from the config like so:

col.conf["foo"]["bar"] = baz
col.setMod()

will continue to work in most case, but should be gradually updated to:

conf = col.get_config("foo")
conf["bar"] = baz
col.set_config("foo", conf)
 2020-04-06 15:39:47 +10:00
Damien Elmes
9bce844df7 fix duplicates search 2020-04-04 17:38:35 +10:00
Damien Elmes
3c7eabc51e add flag to skip downgrade on collection close 
Disabled for now; when enabled it will allow faster collection
open and close in the normal case, while continuing to downgrade
when exporting or doing a full sync.

Also, when downgrading is disabled, the journal mode is no longer
changed back to delete.
 2020-04-04 17:21:45 +10:00
Damien Elmes
ea2a3d7586 use separate schema version for tag changes 
Will prevent issues if user upgrades after an unclean shutdown
 2020-04-04 10:14:51 +10:00
Damien Elmes
6c9a6cde2b use case folding when sorting in canonify 2020-04-03 19:34:46 +10:00
Damien Elmes
ac54f85840 update tag handling 
- tag list stored in a separate DB table
- non-wildcard searches now do full unicode case folding
(eg tag:masse matches 'Maße')
- wildcard matches do simple unicode case folding
- some functions haven't been updated yet, so ascii folding will
continue to be used in some operations
 2020-04-03 19:34:46 +10:00
Damien Elmes
07c414692e preserve mtime/usn when syncing deck config, and add snake_case names 2020-04-03 19:34:46 +10:00
Damien Elmes
83c50488fd use support URL and point to source repo 2020-04-03 08:44:59 +10:00
evandrocoan
30107ab09d Created the GitHub Actions step Upload to PyPi to run only on 
tagged releases of anki:
https://anki.tenderapp.com/discussions/ankidesktop/40025-anki-in-python-package-index
 2020-04-02 02:57:34 -03:00
Damien Elmes
001226050d fix is:due search 2020-04-02 09:33:05 +10:00
Damien Elmes
ddefffceda return interrupted err when media sync aborted 2020-04-01 21:40:14 +10:00
Damien Elmes
f195296106 fix push-i18n-ftl 2020-04-01 18:20:31 +10:00
Damien Elmes
8f427dbbd7 FString -> TR 2020-04-01 18:16:43 +10:00
Damien Elmes
15ba839003 translate default deck config name 2020-04-01 17:49:25 +10:00
Damien Elmes
7e7ac42384 add fixme 2020-04-01 17:36:33 +10:00
Damien Elmes
7b201fe7e6 ignore .DS_Store in trash 
https://anki.tenderapp.com/discussions/ankidesktop/40035-anki-2123-on-macos-ds_store-files-in-new-anki-trash
 2020-04-01 13:37:57 +10:00
Damien Elmes
5bb4d798c1 add some timeouts 
I'm waiting to hear back from the reqwests author on the best way to
handle this. For now, this change just adds short timeouts
to the quick requests, and will wait for up to an hour when sending/
receiving files to allow for the slow connection+large file case.
 2020-04-01 10:03:54 +10:00
Damien Elmes
284ee0860b allow aborting a media sync while http req in flight 
The progress handling code needs a rethink, as we now have two separate
ways to flag that the media sync should abort. In the future, it may
make sense to switch to polling the backend for progress, instead of
passing a callback in.
 2020-04-01 09:49:25 +10:00
Damien Elmes
cffe146397 split deck config into separate SQL table 
- on collection load, the schema is upgraded to 12
- on collection close, the changes are reversed so older clients
can continue to open the collection
- in the future, we could potentially skip the reversal except
when exporting/doing a full sync
- the same approach should work for decks, note types and tags in the
future too
- the deck list code needs updating to cache the deck confs for the
life of the call
 2020-03-30 20:01:16 +10:00
Damien Elmes
089a68eb43 move deck conf handling to backend 2020-03-30 14:39:46 +10:00
Damien Elmes
d41124343d initial work on undo support 2020-03-29 17:52:16 +10:00
Damien Elmes
3924dedbb7 remove usn cache 
SqliteStorage no longer contains any mutable state
 2020-03-29 12:50:38 +10:00
Damien Elmes
cbbbced59e cache timing_today in collection, update it when cutover reached 2020-03-29 12:26:24 +10:00
Damien Elmes
0451a4d4e0 drop separate RequestContext/StorageContext 2020-03-29 12:12:35 +10:00
Damien Elmes
c4a78ffd82 roll back the vec cache changes 
prepare_cached() is sufficiently fast, and allowing the vec cache
to persist across calls is complicated due to lifetime restrictions
 2020-03-29 09:26:09 +10:00
Damien Elmes
cb03c87f8b Revert "add benchmark for vec cache" 
This reverts commit 0247d65ccd.
 2020-03-29 09:16:11 +10:00
Damien Elmes
36314f3b6f Merge pull request #528 from evandroforks/create_actions_for_windows_macos 
Create actions for Windows and Mac OS
 2020-03-27 15:26:22 +10:00
Damien Elmes
fc846ac4e4 use add_card() 2020-03-27 15:11:07 +10:00
Damien Elmes
1112fc8e8b add (unused) add_card, and move sql to separate files 2020-03-27 14:48:25 +10:00
Damien Elmes
0247d65ccd add benchmark for vec cache 
test storage::sqlite::bench::bench_hash_cache ... bench:         399 ns/iter (+/- 27)
test storage::sqlite::bench::bench_no_cache   ... bench:       4,854 ns/iter (+/- 499)
test storage::sqlite::bench::bench_vec_cache  ... bench:           0 ns/iter (+/- 0)
 2020-03-27 09:59:48 +10:00
Damien Elmes
d1c44ec06d store cached statements in a vec instead of separate optionals 2020-03-27 09:51:19 +10:00
evandrocoan
f1fe0209f6 Merge remote-tracking branch 'danielelmes/master' into create_actions_for_windows_macos 2020-03-26 19:31:01 -03:00
Damien Elmes
62be1b2688 update card in transaction 2020-03-27 07:58:25 +10:00
Damien Elmes
e28fdde984 handle mtime/usn bump in backend, and tweak integer sizes 2020-03-26 20:55:04 +10:00
Damien Elmes
aa33bf85ef report invalid type/queue 2020-03-26 19:32:35 +10:00
Damien Elmes
0510ab7c9c update_card() 2020-03-26 18:54:20 +10:00
Damien Elmes
b2fba4e1ff use backend to get card 2020-03-26 17:47:43 +10:00
Damien Elmes
1af3de6f70 DeckID, CardID 2020-03-26 15:10:40 +10:00
Damien Elmes
4150480709 NoteTypeID 2020-03-26 15:00:24 +10:00
Damien Elmes
3395a565a6 newtype NoteID 2020-03-26 14:42:43 +10:00
Damien Elmes
bb0b817d9c use a macro for newtype defs 2020-03-26 13:50:20 +10:00
Damien Elmes
d95cb93d7a usn newtype 2020-03-26 13:06:02 +10:00
Damien Elmes
bfc0287e51 use newtypes for distinguishing between second and millisecond stamps 2020-03-26 12:59:51 +10:00
evandrocoan
964cc1a11a Merge remote-tracking branch 'danielelmes/master' into create_actions_for_windows_macos 
# Conflicts:
#	.github/scripts/trailing-newlines.sh
#	.github/workflows/checks.yml
#	Makefile
#	pylib/Makefile
#	qt/Makefile
#	qt/tools/typecheck-setup.sh
#	rspy/Makefile
 2020-03-24 22:49:49 -03:00
Damien Elmes
2bd99127e9 add newline to mear clearly separate different ftl files 2020-03-25 08:53:42 +10:00
Damien Elmes
6aced5f125 handle fields with embedded nuls 2020-03-24 14:53:33 +10:00
evandrocoan
4fcecaed9f Created the .github/workflows/windows.yml 
# Conflicts:
#	.github/scripts/trailing-newlines.sh
#	Makefile
#	qt/tools/typecheck-setup.sh
#	react/Makefile
#	rspy/Makefile
#	svelte/Makefile
#	tslib/Makefile
 2020-03-23 19:51:03 -03:00
Damien Elmes
7a67f73183 convert asc to desc instead of appending desc to the end of the order 
as the latter doesn't work when sorting on more than one column

https://anki.tenderapp.com/discussions/beta-testing/1868-anki-2124-beta#comment_48174812
 2020-03-23 19:53:57 +10:00
Damien Elmes
4563fe36bc rename long filenames in fields if files renamed in a previous sync 2020-03-23 18:38:45 +10:00
Damien Elmes
b7fdb8aeb4 fix setting of wal 2020-03-23 13:52:57 +10:00
Damien Elmes
17f7a69830 handle trailing whitespace inside group 2020-03-22 18:52:44 +10:00
Damien Elmes
7c91d3e414 accept now_mins_west for v2 legacy timing as well 2020-03-22 17:33:14 +10:00
Damien Elmes
e02f032a47 use backend for v1 and v2 cutoff calculations 
this should also fix the CI failures, which were happening because
the datetime module wasn't matching the shifted time.time()
 2020-03-22 14:28:26 +10:00
Damien Elmes
01f9845b27 exclude autogenerated src from build deps 
prevents unnecessary rebuilds
 2020-03-22 03:39:38 +00:00
Damien Elmes
c0a4a1cb66 possible fix for CI failure 2020-03-22 13:17:00 +10:00
Damien Elmes
b8aaa8fba4 add enum for controlling sort order 
eg col.find_cards("", order=BuiltinSortKind.CARD_DUE)
 2020-03-22 12:59:24 +10:00
Damien Elmes
2060577ede handle collections with sortBackwards set to 0 instead of a bool 2020-03-22 10:26:09 +10:00
Damien Elmes
e22c3f6723 remove debugging line 2020-03-21 18:29:04 +10:00
Damien Elmes
3392fe8708 increase the rated search cap to 365, and allow searches for ease 0 
An add-on appears to use ease 0 when rescheduling cards, and it may
make sense for Anki to do the same in the future as well.
 2020-03-21 16:05:17 +10:00
Damien Elmes
cf50821d3b add search that ignores combining chars 
On a test of a ~40k card collection, the 'ignore accents' add-on
takes about 1150ms, and this code takes about 70ms.
 2020-03-21 15:15:59 +10:00
Damien Elmes
32efd1789b support regexp search in single field 2020-03-21 13:06:46 +10:00
Damien Elmes
d53c8f18ad support wildcard in field*:val search 2020-03-21 12:45:25 +10:00
Damien Elmes
0a6c5dd240 don't require wildcard for unicode case folding in search 2020-03-21 12:44:56 +10:00
Damien Elmes
982e1a811d add unicase collation 
sqlite's like is hard-coded to use ASCII comparisons, so we can't
take advantage of this yet
 2020-03-21 12:40:20 +10:00
Damien Elmes
9dd5fef0a1 support regex searches 2020-03-21 12:00:48 +10:00
Damien Elmes
e0e7560d2b add regexp() to sqlite 2020-03-21 10:23:11 +10:00
Damien Elmes
861139ea9e support deck:"foo bar" style searches 2020-03-21 09:34:24 +10:00
Damien Elmes
0089f0149a add note searching 2020-03-21 09:00:05 +10:00
Damien Elmes
a1853db3d8 fix formatting and unit test 2020-03-21 08:17:24 +10:00
Damien Elmes
e060e0daee handle escaped tag searches and tag:* special case 2020-03-21 08:09:28 +10:00
Damien Elmes
551aa135d6 check for child decks case-insensitively 2020-03-21 07:57:07 +10:00
Damien Elmes
798ea78a37 don't crash when card:0 passed in 2020-03-21 07:56:55 +10:00
Damien Elmes
25c4b8466e handle * in single-field search 2020-03-21 07:56:32 +10:00
Damien Elmes
c173e4a175 fix ease search 2020-03-21 07:56:09 +10:00
Damien Elmes
23717089e9 add the ability to provide a custom sort order; use backend for find 2020-03-21 07:55:21 +10:00
Damien Elmes
46851d7aa9 support sorting on tags 
I don't personally understand it, but some users seem to want it.
 2020-03-20 21:15:23 +10:00
Damien Elmes
9810b584d5 support sorting on note type, card template and decks 2020-03-20 21:15:23 +10:00
Damien Elmes
a004563d19 search order 2020-03-20 21:15:23 +10:00
Damien Elmes
13f726fe8f fix is:new/is:review 2020-03-20 21:15:23 +10:00
Damien Elmes
46cd7f23f5 added needs to use milliseconds 2020-03-20 21:15:23 +10:00
Damien Elmes
089e059951 tolerate some string IDs 2020-03-20 21:15:23 +10:00
Damien Elmes
1e21fa325b handle empty searches and leading/trailing whitespace 2020-03-20 21:15:23 +10:00
Damien Elmes
61391a760c add remaining tests and fix some clippy lints 2020-03-20 21:15:23 +10:00
Damien Elmes
ed009e4fbb fix rated and state searches 2020-03-20 21:15:23 +10:00
Damien Elmes
986f6c4437 fix tags, more tests 2020-03-20 21:15:23 +10:00
Damien Elmes
274e0ef315 fix cards 2020-03-20 21:15:23 +10:00
Damien Elmes
c3c56d7cf2 fix decks 2020-03-20 21:15:23 +10:00
Damien Elmes
518836e434 fix added 2020-03-20 21:15:23 +10:00
Damien Elmes
3fb5066152 fix qualified search 2020-03-20 21:15:23 +10:00
Damien Elmes
ac8534f4ee fix escape handling, and handle sql wildcards 2020-03-20 21:15:23 +10:00
Damien Elmes
8f38ad8aa2 avoid extra sql binding in unqualified search 2020-03-20 21:15:23 +10:00
Damien Elmes
b087ebc845 start on search tests 2020-03-20 21:15:23 +10:00
Damien Elmes
f36857ae27 prototype of integration 
no ordering yet, and no tests
 2020-03-20 21:15:23 +10:00
Damien Elmes
6c469a96eb finish the remaining searches 
Searches that require multiple deck or note type lookups won't perform
very well at the moment - it either needs caching or to be split up
at the DB level.

Nothing tested yet.
 2020-03-20 21:15:23 +10:00
Damien Elmes
a85646bb05 ctx->req 2020-03-20 21:15:23 +10:00
Damien Elmes
3227fc1b9b add timing to search 2020-03-20 21:15:23 +10:00
Damien Elmes
4f1c85bf8a add v1 and v2 legacy timing code 2020-03-20 21:15:23 +10:00
Damien Elmes
1b2ebfcf6f deck searching 
A bit more complicated than it needs to be, as we don't have the
full deck manager infrastructure yet.
 2020-03-20 21:15:23 +10:00
Damien Elmes
91270251bf address some clippy lints 2020-03-20 21:15:23 +10:00
Damien Elmes
490e2b3cd3 move html stripping out of field_checksum into caller 2020-03-20 21:15:23 +10:00
Damien Elmes
eef158b20c add field_at_index() sql func 2020-03-20 21:15:23 +10:00
Damien Elmes
445fa4876a more searching work 2020-03-20 21:15:23 +10:00
Damien Elmes
3b6fef5613 add card queue/type enums 2020-03-20 21:15:23 +10:00
Damien Elmes
4217031780 separate out template ordinal and name search 2020-03-20 21:15:23 +10:00
Damien Elmes
0b33131971 use .eq_ignore_ascii_case() to avoid allocating 2020-03-20 21:15:23 +10:00
Damien Elmes
1b2af8054a field_checksum needs to strip HTML 2020-03-20 21:15:23 +10:00
Damien Elmes
819d432e12 start of searching sql 2020-03-20 21:15:23 +10:00
Damien Elmes
4014e054a9 ensure rated ease in range 2020-03-20 21:15:23 +10:00
Damien Elmes
28defd2b3c ensure id list not empty 2020-03-20 21:15:23 +10:00
Damien Elmes
02af7b2ab1 decode search terms in parser 2020-03-20 21:15:23 +10:00
Damien Elmes
c12ebc6124 split up searches with a qualifier 2020-03-20 21:15:23 +10:00
Damien Elmes
43f21b8135 basic search parsing 2020-03-20 21:15:23 +10:00
Damien Elmes
82dbbe7211 add string about waiting for completion 2020-03-20 21:15:23 +10:00
Damien Elmes
5a20d9da79 release mutex before beginning media sync 
And check media sync is not running on close
 2020-03-20 21:15:23 +10:00
Damien Elmes
a321efe9db fix media sync being logged to console 2020-03-20 21:15:23 +10:00
Damien Elmes
5aba5c3ab9 drop the separate i18n backend 2020-03-20 21:15:23 +10:00
Damien Elmes
e5f883d1c3 support opening and closing the DB while keeping backend alive 
This is safer than just dropping the backend, as .close() will
block if something else is holding the mutex. Also means we can
drop the extra I18nBackend code.

Media syncing still needs fixing.
 2020-03-20 21:15:23 +10:00
Damien Elmes
c710580604 move media folder/db paths into collection 
this breaks background media syncing for now
 2020-03-20 21:15:23 +10:00
Damien Elmes
8c8674f57b drop unused col_path 2020-03-20 21:15:23 +10:00
Damien Elmes
3cacc00fc3 make the collection optional 2020-03-20 21:15:23 +10:00
Damien Elmes
116f7fa3f1 support first_row_only 2020-03-20 21:15:23 +10:00
Damien Elmes
7e5f713f05 proper implementation of executemany(); drop executescript() 2020-03-20 21:15:23 +10:00
Damien Elmes
121687ea91 wrap the collection in a mutex so DB access is thread safe 2020-03-20 21:15:23 +10:00
Damien Elmes
e72d21e09e move note code into notes.rs, add ability to rollback when unchanged 2020-03-20 21:15:23 +10:00
Damien Elmes
3f3a45e2ba add Collection struct, and get media check working again 
- media check no longer needs collection to be closed
- use savepoints for operations initiated by Rust, so they are
atomic without forcing a commit
 2020-03-20 21:15:23 +10:00
Damien Elmes
700d713f80 add begin/commit/rollback, and support creating collections 
all but one unit test is now passing
 2020-03-20 21:15:23 +10:00
Damien Elmes
bd7ded18c3 drop the protobuf prototype 2020-03-20 21:15:23 +10:00
Damien Elmes
38d4ea5311 use (or)json for DB bridge 
Some initial testing with orjson indicates performance varies from
slightly better than pysqlite to about 2x slower depending on the type
of query.

Performance could be improved by building the Python list in rspy
instead of sending back json that needs to be decoded, but it may make
more sense to rewrite the hotspots in Rust instead. More testing is
required in any case.
 2020-03-20 21:15:23 +10:00
Damien Elmes
a129b8f8cf initial work on DB handling in Rust 
committing the Protobuf implementation for posterity, but will replace
it with json, as Protobuf measures about 6x slower for some workloads
like 'select * from notes'
 2020-03-20 21:15:23 +10:00
Damien Elmes
1d6c084c2b fix clippy lint on other platforms 2020-03-19 10:58:52 +10:00
Damien Elmes
0d9ea33389 bump version 2020-03-19 10:46:01 +10:00
Damien Elmes
eae07af576 correctly handle NFD content in media DB from older Anki versions 2020-03-19 10:20:42 +10:00
Damien Elmes
29f625b61e add helper to duplicate a fluent string 2020-03-12 19:53:44 +10:00
Damien Elmes
0651aaa229 add media check window title, use consistent formatting in template 2020-03-12 18:02:22 +10:00
Damien Elmes
28edff2a5e fix fallback preferences being used instead of English 
Check needs to be done regardless of if we found a regional variant
or not.
 2020-03-12 18:02:15 +10:00
Damien Elmes
b198ba8073 ignore remote file links in media check 2020-03-10 13:46:52 +10:00
Damien Elmes
e512199ace add restore media action 2020-03-10 13:35:09 +10:00
Damien Elmes
cac9ccfa0d add empty trash action in aqt 2020-03-10 12:49:40 +10:00
Damien Elmes
a6b1a32378 don't clean up the trash folder automatically 
The media check now reports if any files are in the trash folder,
and a method has been exposed to clear the trash.
 2020-03-10 12:02:38 +10:00
Damien Elmes
005a8831e1 add basic log rotation 2020-03-10 10:33:14 +10:00
Damien Elmes
387207cd86 log sync errors 2020-03-10 09:43:48 +10:00
Damien Elmes
80065f3ce2 tweak logging 2020-03-10 09:30:05 +10:00
Damien Elmes
6f9079a341 don't try to read file data when file marked as a deletion 
If the user hadn't used the media check to clean up files with
invalid names on disk,  the long filename would break the 'automatically
rename long filenames when syncing' functionality, causing the original
long filename to be uploaded to AnkiWeb as an addition instead of
a deletion
 2020-03-10 09:29:36 +10:00
Damien Elmes
0a8fe2959c fix missing newline in ftl files 
https://anki.tenderapp.com/discussions/beta-testing/1836-problem-with-new-translations
 2020-03-09 18:58:41 +10:00
Damien Elmes
f6d5d4ea26 rspy version 2020-03-06 19:15:02 +10:00
Damien Elmes
0aa0c42d20 update to failure 0.1.7 2020-03-06 15:08:58 +10:00
Damien Elmes
2e4fb4401f fix incorrect mark_collection_modified() 
- usn shouldn't be changed
- mtime is in milliseconds
 2020-03-05 14:55:03 +10:00
Damien Elmes
0b4a7c1d10 work around bug in failure crate 2020-03-05 11:06:44 +10:00
Damien Elmes
ba97457e8a add nfd tag fix to db check 
we already normalize in the editor and importing, so perhaps these
tags were from an old version

https://anki.tenderapp.com/discussions/ankidesktop/39120-bug-filter-do-not-work-with-tags-which-contain-german-letters-like
 2020-03-01 11:21:25 +10:00
Damien Elmes
c97b37b7ae make logging explicit, and support logging to a file 2020-02-29 20:44:01 +10:00
Damien Elmes
36372991f7 bump rust toolchain for 1.41.1 fixes 2020-02-29 19:42:30 +10:00
Damien Elmes
42781a696f remove some unused imports 2020-02-29 15:21:11 +10:00
Damien Elmes
e26860ad04 don't require a media scan after media deleted inside Anki 2020-02-29 14:45:16 +10:00
Damien Elmes
0810d9d5e0 fix display of years 2020-02-28 17:14:35 +10:00
Damien Elmes
7cfe33a099 fix rerun-if-changed paths 2020-02-27 20:52:17 +10:00
Damien Elmes
2f92c9a803 add support for embedding the qt translations 2020-02-27 20:38:34 +10:00
Damien Elmes
2de865d7f0 migrate strings with trailing newlines to fluent 2020-02-27 20:36:40 +10:00
Damien Elmes
3c304927c8 Merge pull request #465 from alanhdu/rust-deps 
Upgrade some rust deps
 2020-02-27 14:13:26 +10:00
Alan Du
a6751c893b Update blake3 to 0.2 2020-02-26 22:50:48 -05:00
Alan Du
a6441dd4e0 Update prost-build version to 0.6 
This is the same version as Prost, and lets us reduce the number of deps
we have to compile
 2020-02-26 22:48:16 -05:00
evandrocoan
4a2456f62f Merge remote-tracking branch 'danielelmes/master' into fix_windows_build 
# Conflicts:
#	Makefile
 2020-02-27 00:20:34 -03:00
Damien Elmes
3dc0567f89 add string for media log title, and add button in prefs to access it 2020-02-27 12:22:24 +10:00
Damien Elmes
5337ea457a extra newline in network details 2020-02-27 11:22:48 +10:00
Damien Elmes
f20a23b1a9 use debug description for untranslated errors; fix template error 2020-02-27 11:07:18 +10:00
Damien Elmes
b030f2b40b info is superfluous 2020-02-27 10:53:49 +10:00
Damien Elmes
83d894ac87 return a localized error for all error kinds 
some errors are not yet localized, but now the Python code
doesn't need to think about which property to use
 2020-02-27 10:46:24 +10:00
Damien Elmes
21c94c623c add comment to sync.ftl 2020-02-27 09:18:49 +10:00
Damien Elmes
59328d9ade tweak rounding 
- avoid rounding minutes
- round the seconds taken in the card info screen
- provide different precise and imprecise modes, since we need
to display seconds to multiple decimals in some areas
 2020-02-25 17:29:06 +10:00
evandrocoan
2d316748a7 Merge remote-tracking branch 'danielelmes/master' into fix_windows_build 
# Conflicts:
#	Makefile
 2020-02-25 03:16:57 -03:00
evandrocoan
9e690f80e7 Renamed FIND_EXEC to FIND, replaced IS_WINDOWS by specific commands 
as PYTHON_BIN, ACTIVE_SCRIPT and INSTALL_PYAUDIO.
Fixed echo statements not using @ to suppress double message output.
Deprecated the usage of ECHOCMD := /bin/echo -e because it has no
effect: https://stackoverflow.com/questions/60387684/how-to-make-the-makefile-echos-to-use-bin-echo-e

# Conflicts:
#	Makefile
 2020-02-25 03:15:11 -03:00
Damien Elmes
64851afa5e cap answer buttons to 1 decimal place 
we can switch to NUMBER() instead in the future, but will need
to update all the translations at the same time
 2020-02-25 13:24:29 +10:00
Damien Elmes
c2cfe70b9b use Fluent's number formatting; don't show trailing zeros 
We can add NUMBER() in the future for more control, but this will
do for the time being.
 2020-02-25 13:22:41 +10:00
Damien Elmes
1f6e7805ec add workaround for panic on 32 bit systems 
https://anki.tenderapp.com/discussions/beta-testing/1817-failed-to-generate-operands-out-of-fluentnumber-could-not-convert-string-to-integer
 2020-02-25 13:22:41 +10:00
evandrocoan
5be9a702cb Fixed rslib/Makefile using find from Windows, instead from POSIX 2020-02-24 21:44:44 -03:00
Damien Elmes
db8da7d4b7 sort FStrings 
- easier to read
- ensures things don't break when ankirspy and anki wheels built
on separate machines due to mismatched directory order
 2020-02-25 08:26:20 +10:00
evandrocoan
6d16298254 Fixed development compilation with Windows 10 
1. Documented on README.development how to setup the environment
   for Windows.
2. Fixed qt/ts/package.json not working due usage of ; instead
   of &&
3. Fixed copy-qt-files rsync using Windows paths instead of Unix
   ones
4. Fixed Makefile's using Windows Linux Subsystem bash instead of
    the Cygwin one.
5. Ensured running the correct pip module by using python -m pip
    instead of just pip.
6. Fixed Makefiles using Windows `find` command, instead of the
    Cygwin's one (POSIX find).
7. Fixed pyenv sourcing/activate using /pyevn/bin/ instead of
    /python/Scripts/ on Windows.
8. Fixed pyaudio not installing/linking with portaudio on Windows
    by installing for a patched fork at evandroforks/pyaudio
9. Forked and fixed portaudio not building with Visual Studio 2017
    or superior and added the reference for the patched fork on
    README.development at evandroforks/portaudio.
 2020-02-24 14:59:11 -03:00
Damien Elmes
ea22296d4c expand clozes before rendering LaTeX 
https://anki.tenderapp.com/discussions/ankidesktop/39071-latex-media-windows-version-2121-0cd1ee92-fail-rendering-latex
 2020-02-24 18:34:08 +10:00
Damien Elmes
4704dcc061 check depends on repo 2020-02-23 19:31:17 +10:00
Damien Elmes
024126b3be support aqt-specific FTL files 2020-02-23 18:00:13 +10:00
Damien Elmes
c98247fa0f translations no longer require an open collection 2020-02-23 14:57:02 +10:00
Damien Elmes
0d1a30c6d8 rework Fluent handling 
- all .ftl files for a language are concatenated into a single file
at build time
- all languages are included in the binary
- external ftl files placed in the ftl folder can override the
built-in definitions
- constants are automatically generated for each string key
- dropped the separate StringsGroup enum
 2020-02-23 13:22:50 +10:00
Damien Elmes
fba8a6ffed fix message references 2020-02-22 21:48:14 +10:00
Damien Elmes
09e4e50751 add file prefix to fluent keys so we can merge the files at build 2020-02-22 20:26:15 +10:00
Damien Elmes
bff36141f0 make sure English takes priority if it's listed first 2020-02-21 22:01:35 +10:00
Damien Elmes
ecc055d2f4 fluent-rs git no longer required 2020-02-21 21:30:11 +10:00
Damien Elmes
72528ee41c add deck config category 2020-02-21 20:53:20 +10:00
Damien Elmes
abc260b7c3 reviews 2020-02-21 20:36:22 +10:00
Damien Elmes
258a8a04f0 fix comment 2020-02-21 20:10:58 +10:00
Damien Elmes
e84fa7dee3 add next learning due time + remaining count to congrats screen 
https://anki.tenderapp.com/discussions/ankidesktop/38000-v2-scheduler-learning-cards
 2020-02-21 19:54:50 +10:00
Damien Elmes
3a83d9844a add studied_today(), move to statistics.ftl 2020-02-21 18:01:15 +10:00
Damien Elmes
e3e376036f add average-answer-time to Fluent 
drop the 0.1 special case; we're using two decimals now
 2020-02-21 17:42:55 +10:00
Damien Elmes
65f2131340 remove unused import 2020-02-21 15:29:38 +10:00
Damien Elmes
b851577063 fix revision pin 2020-02-21 15:29:38 +10:00
Damien Elmes
4e612cdd3b add 'studied-today' to Fluent 
The string has been constructed in parts so the old translations
could be preserved
 2020-02-21 15:29:38 +10:00
Damien Elmes
74bff450e2 add non-abbreviated timespan translation; update existing short=True calls 
- drop the '5m3s' special casing done in the card stats screen, and
just use decimals
- change alignment of the review log so that the non-abbreviated
spans are easier to read
 2020-02-21 15:29:38 +10:00
Damien Elmes
11647daec1 move answer_button_time to the backend, split sched into separate module 2020-02-21 15:29:38 +10:00
Damien Elmes
2633d6db4b move answer button labels into fluent 2020-02-21 15:29:38 +10:00
Damien Elmes
cf1824dac5 update to latest fluent-rs and add basic locale-aware decimals 
- git version pinned at the moment until the concurrency fix
lands in 0.10.2
- currently float values are hard-coded at 2 decimal places;
we should switch to using NUMBER() in the future
 2020-02-21 15:29:38 +10:00
Damien Elmes
5bb81a95c2 add 'new #' prefix to new cards in the due column 2020-02-18 11:16:15 +10:00
Damien Elmes
0cd1ee92ee pin coarsetime for now, as .12 requires Sierra 2020-02-17 17:55:39 +10:00
Damien Elmes
302fc2b24e fix deletion notices being sent unnecessarily 2020-02-17 17:21:58 +10:00
Damien Elmes
fd01cac4d7 split "Due" into three different contexts for translators 2020-02-17 14:41:01 +10:00
Damien Elmes
957a0fae75 media check required message is no longer required 2020-02-17 13:51:33 +10:00
Damien Elmes
b554ad8c1d match older string 2020-02-17 13:41:02 +10:00
Damien Elmes
e898bd82f1 add some more comments 2020-02-17 11:43:18 +10:00
Damien Elmes
f40a00f0cb localize some error messages 2020-02-17 11:38:22 +10:00
Damien Elmes
f6643eda53 use new i18n infrastructure for more media check / media sync strings 2020-02-17 10:18:20 +10:00
Damien Elmes
60fafd0457 add some comments to card-template-rendering.ftl 2020-02-17 09:06:54 +10:00
Damien Elmes
4b63a07c99 expose translations to Python 2020-02-17 08:40:17 +10:00
Damien Elmes
0260343c69 cache i18n categories 2020-02-17 08:40:17 +10:00
Damien Elmes
342c53a1e4 don't hard-code available ftl languages 
Instead of trying to define which languages we support, just check
if an appropriate folder is available on disk. This allows users
to drop their own translations into the locale folder and have things
just work.
 2020-02-17 08:40:17 +10:00
Damien Elmes
998b2c0f13 wrap i18n struct in a shared mutex so we can start caching 2020-02-17 08:40:17 +10:00
Damien Elmes
1f736239cd pass progress back as translated string(s) 2020-02-17 08:40:17 +10:00
Damien Elmes
8450214cda add ftl to the build process, and a sync-git script 
also
- ensure po files are checked when updated
- add start of sync.ftl
 2020-02-17 08:40:17 +10:00
Damien Elmes
78ffde32e1 expose StringsGroup enum in protobuf 2020-02-17 08:40:17 +10:00
Damien Elmes
43005f00a5 bump version 2020-02-17 08:40:17 +10:00
Damien Elmes
0869a78668 more frequent progress updates 2020-02-17 08:40:17 +10:00
Damien Elmes
dd635afc29 rename card-templates.ftl 2020-02-17 08:40:17 +10:00
Damien Elmes
ba0bc3086c move empty card check into template code 2020-02-17 08:40:17 +10:00
Damien Elmes
0738ff46b6 make template errors translatable 2020-02-17 08:40:17 +10:00
Damien Elmes
834989a019 include report in MediaCheckOutput 2020-02-17 08:40:17 +10:00
Damien Elmes
36d2536262 add i18n support to Rust code using Fluent 2020-02-17 08:40:17 +10:00
Damien Elmes
8f34c720b8 remove redundant test_ prefix 2020-02-17 08:40:17 +10:00
Damien Elmes
afca40c83f mark deletions in media DB when files are deleted 2020-02-17 08:40:17 +10:00
Damien Elmes
5e500775d4 ignore errors when file to delete is already gone 
May be marked as pending upload or in media check screen, then removed
by user.
 2020-02-17 08:40:17 +10:00
Damien Elmes
90e5bfcb0c make sure we don't leave a trailing dot or space when truncating 2020-02-17 08:40:17 +10:00
Damien Elmes
0869fac03a make sure renames generated during sync don't get immediately removed 2020-02-17 08:40:17 +10:00
Damien Elmes
311fb524a4 don't filter out invalid filenames when we're sending them as a deletion 2020-02-17 08:40:17 +10:00
Damien Elmes
56931467a9 fix the wrong named being returned when renaming in sync 2020-02-17 08:40:17 +10:00
Damien Elmes
0d1549c5bd pt->pb for consistency with rsbackend.py 2020-02-17 08:40:17 +10:00
Damien Elmes
1383973753 shorten add_file_to_media_folder 2020-02-17 08:40:17 +10:00
Damien Elmes
89626c1a72 use media.trash for unused media deletion as well 2020-02-17 08:40:17 +10:00
Damien Elmes
d32e2fe6eb add extract_latex to backend; use it for latex build 2020-02-17 08:40:17 +10:00
Damien Elmes
d557312f73 include LaTeX png/svg files when checking for unused media 2020-02-17 08:40:17 +10:00
Damien Elmes
e3bd06ec85 files with leading underscore are ignored 2020-02-17 08:40:17 +10:00
Damien Elmes
9700acbf18 include normalized filenames in the rename list 
Since they'll need to be uploaded on the next sync, better not to
hide them from the list
 2020-02-17 08:40:17 +10:00
Damien Elmes
790f3c9f72 plug new media check in 2020-02-17 08:40:17 +10:00
Damien Elmes
3d5f6a40b2 when adding hash suffix, need length in hex, not bytes 2020-02-17 08:40:17 +10:00
Damien Elmes
92b76a5ba7 gather field references in Rust; media check now mostly complete 2020-02-17 08:40:17 +10:00
Damien Elmes
edf60b10d4 backend stores media folder/db locations; don't need to pass them in 2020-02-17 08:40:17 +10:00
Damien Elmes
46bb6f1b92 catch trailing space/period as well 2020-02-17 08:40:17 +10:00
Damien Elmes
09aecc12e8 check [ instead of con in unit test, so test works on Windows as well 2020-02-17 08:40:17 +10:00
Damien Elmes
64e017da9a run the TZ test only on Macs 2020-02-17 08:40:17 +10:00
Damien Elmes
e1f817a258 test shouldn't depend on dentry order 2020-02-17 08:40:17 +10:00
Damien Elmes
0da393da81 expire media trash after a week 2020-02-17 08:40:17 +10:00
Damien Elmes
a0680553ea use our own trash folder instead of using the system trash 
the trash crate was invoking external commands on Macs and Linux
which is slow and likely to fall over if a large number of files need
to be deleted at once.
 2020-02-17 08:40:17 +10:00
Damien Elmes
27b783ac72 tidying 2020-02-17 08:40:17 +10:00
Damien Elmes
76a61b8adf hold the DB open for the duration of the check 2020-02-17 08:40:17 +10:00
Damien Elmes
f1b0431cbd DB check 
Working, but should be refactored so media DB not re-opened
each time a file is renamed.
 2020-02-17 08:40:17 +10:00
Damien Elmes
5ec977c74b factor entry code out for later 2020-02-17 08:40:17 +10:00
Damien Elmes
1ca15c10d6 mgr didn't need to be mutable 2020-02-17 08:40:17 +10:00
Damien Elmes
2d859df88f nfc helper 2020-02-17 08:40:17 +10:00
Damien Elmes
7079063855 ignore 0 byte files when picking up changes 2020-02-17 08:40:17 +10:00
Damien Elmes
62924237d3 don't add non-normalized files to media DB 2020-02-17 08:40:17 +10:00
Damien Elmes
88ac38fc55 media table doesn't need rowid 2020-02-17 08:40:17 +10:00
Damien Elmes
51e5d4b792 dirty index doesn't need to cover false case 2020-02-17 08:40:17 +10:00
Damien Elmes
742ab59edb handle read errors during zip build 2020-02-17 08:40:17 +10:00
Damien Elmes
ef522b235a clean up invalid media DB entries on the fly, instead of requiring DB check 2020-02-17 08:40:17 +10:00
Damien Elmes
9c60f2c77e log media DB change registration progress, and allow aborting it 2020-02-17 08:40:17 +10:00
Damien Elmes
8e31068f0a move change tracking into separate file 2020-02-17 08:40:17 +10:00
Damien Elmes
ef14b1a96b simplify code by accumulating stats at the source 2020-02-17 08:40:17 +10:00
Damien Elmes
b21d24f6e8 move sync_media() into MediaManager 2020-02-17 08:40:17 +10:00
Damien Elmes
a769fb6172 update rslib version automatically 2020-02-17 08:40:17 +10:00
Damien Elmes
b62819092a remove duplicate method 2020-02-17 08:40:17 +10:00
Damien Elmes
e352a6b95c tidy up sync.rs 2020-02-17 08:40:17 +10:00
Damien Elmes
08eb9d70c8 handle concurrent modifications and ankiweb terminating early 2020-02-17 08:40:17 +10:00
Damien Elmes
3ee1d3f0da update dirmod as files added during sync 2020-02-17 08:40:17 +10:00
Damien Elmes
775c83e89a add debug line 2020-02-17 08:40:17 +10:00
Damien Elmes
00cd3be1a0 fix force_resync() 2020-02-17 08:40:17 +10:00
Damien Elmes
c0e7975fe7 rename non-normalized filenames when downloading 2020-02-17 08:40:17 +10:00
Damien Elmes
2c27c11584 comply with manylinux wheel requirements, and vendor sqlite on Windows 2020-02-17 08:40:17 +10:00
Damien Elmes
1f931a5b78 defer errors on upload to media check 2020-02-17 08:40:17 +10:00
Damien Elmes
2cecae4ca7 add proper version 2020-02-17 08:40:16 +10:00
Damien Elmes
77158760ff use enums for some common errors 2020-02-17 08:40:16 +10:00
Damien Elmes
81262ffd95 defer media DB load until media action performed 
This means a corrupt media DB will not prevent collection load.
 2020-02-17 08:40:16 +10:00
Damien Elmes
b883b09414 pass in endpoint 2020-02-17 08:40:16 +10:00
Damien Elmes
738e415463 connect media sync progress to Python, add a progress dialog 2020-02-17 08:40:16 +10:00
Damien Elmes
50d8a40e23 add progress hook to media sync 2020-02-17 08:40:16 +10:00
Damien Elmes
e26ab9073b send removed files to the trash 
The way the trash crate implements deletion on a Mac is ugly, and
we may need to look into alternatives.
 2020-02-17 08:40:16 +10:00
Damien Elmes
e764e68324 store sync state in a struct, and reuse ctx across methods 2020-02-17 08:40:16 +10:00
Damien Elmes
78bfbef863 media sync working, but unpolished 2020-02-17 08:40:16 +10:00
Damien Elmes
b2b8370477 bump nightly for the unwrap source line fix 2020-02-17 08:40:16 +10:00
Damien Elmes
973d8c56d6 more refactoring 2020-02-17 08:40:16 +10:00
Damien Elmes
7430d3c64f rearrange some methods to make structure clearer 2020-02-17 08:40:16 +10:00
Damien Elmes
1acb3b86b9 backend init can now fail, and update media db when file is added 
- Adding files inside Anki now updates the media DB, so a full file
scan at sync time is no longer required if no other changes have been
made.
- Use a protobuf message for backend initialization, and return a
string error if initialization fails.
 2020-02-17 08:40:16 +10:00
Damien Elmes
6bf4b9e83e port change tracking 2020-02-17 08:40:16 +10:00
Damien Elmes
4f08ae9e27 make sure results are checked 2020-02-17 08:40:16 +10:00
Damien Elmes
9bc38647e6 initial rs implementation of media database 2020-02-17 08:40:16 +10:00
Damien Elmes
c9a5ed3cc6 add add_file() and write_data() 2020-02-17 08:40:16 +10:00
Damien Elmes
9ec6d26db7 add another implementation of media.addFile() and cleanFilename() et al 
Instead of adding an incrementing number in the conflict case,
the file hash is appended.
 2020-02-17 08:40:16 +10:00
Damien Elmes
391a58077a show which side the template error occurred on 2020-02-08 11:54:08 +10:00
Damien Elmes
fc3570c949 clearer error when closing tags transposed 2020-02-08 10:32:34 +10:00
Damien Elmes
86a580d820 reuse reveal_cloze_text() for LaTeX cloze expansion 2020-01-28 07:40:44 +10:00
Damien Elmes
d598ce45fe move cloze-related code into a separate file 2020-01-27 20:41:23 +10:00
Damien Elmes
47bcb3845c support speed control in tts tags 2020-01-26 14:31:07 +10:00
Damien Elmes
46c6a7f7e4 extract and flag AV tags as part of the render process 
We can now show replay buttons for the audio contained in {{FrontSide}}
without having to play it again when the answer is shown.

The template code now always defers FrontSide rendering, as it wasn't
a big saving, and meant the logic had to be implemented twice.
 2020-01-24 11:06:11 +10:00
Damien Elmes
faaecdb2c7 look through negated conditionals instead of just ignoring them 
https://anki.tenderapp.com/discussions/ankidesktop/38220-bug-in-2119-regarding-computation-of-req
 2020-01-23 17:27:07 +10:00
Damien Elmes
2ee8902bd9 added flag_av_tags() for replay button support 2020-01-21 14:44:27 +10:00
Damien Elmes
b0d2958721 add rank handling to TTS; parse TTS args in get_av_tags() 2020-01-21 12:41:37 +10:00
Damien Elmes
a85ad8b257 more TTS and sound work 
- use provided language and voices when playing on Mac
- fix hang in waiting for termination
- allow players to return a rank for a given tag,
which will allow for the best matching player to be chosen
depending on the context (eg, prioritize one player for videos,
one tts player for certain voices, etc)
 2020-01-21 11:34:25 +10:00
Damien Elmes
548ed31ef7 add a simple TTS field filter 
This simply wraps the field in extra text that the frontend will
deal with. Also added some helpers for extracting and stripping
audio and TTS tags from the rendered text.
 2020-01-20 19:16:24 +10:00
Damien Elmes
97dc39d0e0 workflow fix 2020-01-19 15:02:08 +10:00
Damien Elmes
c525a75fa3 drop plus_one test 2020-01-17 18:54:38 -07:00
Damien Elmes
a9378b1e99 _localOffsetForDate() was broken 
It was including the elapsed time of day when calculating
the offset, leading to incorrect results
 2020-01-17 18:52:36 -07:00
Damien Elmes
21098c43b5 save filters and field name separately 2020-01-16 18:47:21 +10:00
Damien Elmes
2e8bdb580a show missing field errors in the same way as the other errors 2020-01-16 18:47:21 +10:00
Damien Elmes
52bd10063e more useful template error message 2020-01-16 18:47:21 +10:00
Damien Elmes
ce77759177 allow an empty field name if filters are provided 
This is to make it easier for add-ons to provide extra content that
degrades nicely on platforms that don't have the add-on.
 2020-01-15 18:48:23 +10:00
Damien Elmes
c84c3bce72 fix overlapping clozes 2020-01-14 07:07:21 +10:00
Damien Elmes
8eceedd2e5 more template rendering tweaks 
- The front and back are rendered in one call now. If the front
side contains no custom filters, we can bake {{FrontSide}} into the
rear side. If it did contain custom filters, we return the partially
complete rear template instead, and the calling code can inject
the FrontSide in after it has been fully rendered.

- Instead of modifying "cloze" into something like "cq-2", the card
ordinal and whether we're rendering the question or answer are now
passed in to the rendering filters as context.

- The Rust code doesn't need to support filter names split on '-'
anymore.

- Drop the "Show" part of hint descriptions so i18n support can be
deferred.

- Ignore blank filter names caused by user using two colons instead
of one.

- Fixed hint field and text transposition.
 2020-01-12 21:34:07 +10:00
Damien Elmes
c486a4bf6d fix type:cloze handling 2020-01-11 19:38:41 +10:00
Damien Elmes
524a27dd08 ignore extra { chars in replacement 
This will still show a trailing } with {{{field}}}, but will at
least show the field content instead of 'unknown field'.
 2020-01-11 18:02:24 +10:00
Damien Elmes
9028018277 static->const 2020-01-11 13:06:48 +10:00
Damien Elmes
41d155a6ee drop unnecessary capitalized cloze handling 2020-01-11 13:03:43 +10:00
Damien Elmes
e7c3218724 handle mathjax+cloze case 
instead of trying to selectively change clozes from c to C, just
strip HTML from inside the mathjax sections instead
 2020-01-11 12:57:32 +10:00
Damien Elmes
cf5457da25 cloze: support 
MathJax still to do
 2020-01-11 12:07:06 +10:00
Damien Elmes
d671f091ff type: and hint: support 
We may need to keep handling hints in the Python code for now until
i18n is sorted out.
 2020-01-11 09:35:41 +10:00
Damien Elmes
3510eb1861 add ruby filters 2020-01-10 21:04:52 +10:00
Damien Elmes
ab841d9ebe add some text helpers 2020-01-10 21:01:23 +10:00
Damien Elmes
ca6793c940 add copyright headers to some files 2020-01-10 21:00:48 +10:00
Damien Elmes
f101c34df7 skip rslib doctests, we don't use them and they're slower 2020-01-10 18:06:21 +10:00
Damien Elmes
3a14c247c8 replace fields and apply standard filters in rslib 
The filters still need to be implemented.
 2020-01-10 18:02:26 +10:00
Damien Elmes
8f3df55da2 flatten->render; pass field content in 
This is paving the way to move the standard filters into Rust.

Non-empty fields are now determined in Rust, using a single regex
instead of the overkill stripHTMLMedia(). The old implementation
has been moved into the Pystache code for now.
 2020-01-10 14:59:29 +10:00
Damien Elmes
f3071df403 fix next_day_at being off by a day before the rollover 2020-01-10 08:21:52 +10:00
Damien Elmes
31ebf0473f rework new timezone code 
We now store the UTC offset that was in effect at creation time,
and use that to determine the starting date.
 2020-01-09 16:58:27 +10:00
Damien Elmes
c0604c596b ignore template closing characters outside of a tag 2020-01-08 20:29:04 +10:00
Damien Elmes
520275bea3 handle conditional replacement in Rust 
This extends the existing Rust code to handle conditional
replacement. The replacement of field names and filters to text
remains in Python, so that add-ons can still define their own
field modifiers.

The code is currently running the old Pystache rendering and the
new implementation in parallel, and will print a message to the
console if they don't match. If you notice any problems, please
let me know.
 2020-01-08 20:28:04 +10:00
Damien Elmes
3a27b2481f fix days_elapsed moving backwards 2020-01-07 09:29:43 +10:00
Damien Elmes
2c5ee82dcf add failing test for days_elapsed calculation 
the current code was causing the day to move backwards:

at day 7 hour 23:59, elap is 1
at day 8 hour 0:59, elap is 0
at day 8 hour 1:59, elap is 0
at day 8 hour 2:59, elap is 1
at day 8 hour 3:59, elap is 1
at day 8 hour 4:59, elap is 2

as mentioned in https://github.com/ankitects/anki/pull/416
 2020-01-07 09:29:43 +10:00
Damien Elmes
9057c8ec85 make sure rust tests rerun on source change; don't hide output 2020-01-07 09:29:43 +10:00
Damien Elmes
ddb304aad2 add some more protobuf messages 2020-01-06 12:18:20 +10:00
Damien Elmes
c88694333b update readme, remove submodule readmes 2020-01-04 12:21:58 +10:00
Damien Elmes
c0438373dd move license info into one place 2020-01-03 09:41:30 +10:00
Damien Elmes
2a00e0a6b0 tweaking the folder names again 
hopefully that's the last of it
 2020-01-03 07:48:38 +10:00