Commit graph

387 commits

Author SHA1 Message Date
Damien Elmes
f5f2782e3b embed sort kind enum to avoid future namespace conflicts 2020-04-14 20:31:10 +10:00
Damien Elmes
0538d9b764 (de)serialize decks in backend 2020-04-09 12:41:59 +10:00
Damien Elmes
bca49bd054 load/save note types in backend
This allows us to normalize bad data, and is the first step towards
splitting note types into separate tables.
2020-04-08 10:05:07 +10:00
Damien Elmes
02d0d9a670 add separate json error 2020-04-06 16:29:31 +10:00
Damien Elmes
913c5b9854 use bytes for deck config proto 2020-04-06 15:39:47 +10:00
Damien Elmes
6471adafcc use bytes for config proto 2020-04-06 15:39:47 +10:00
Damien Elmes
b5022ad354 store config in separate DB table
- mtime is tracked on each key individually, which will allow
merging of config changes when syncing in the future
- added col.(get|set|remove)_config()
- in order to support existing code that was mutating returned
values (eg col.conf["something"]["another"] = 5), the returned list/dict
will be automatically wrapped so that when the value is dropped, it
will save the mutated item back to the DB if it's changed. Code that
is fetching lists/dicts from the config like so:

col.conf["foo"]["bar"] = baz
col.setMod()

will continue to work in most case, but should be gradually updated to:

conf = col.get_config("foo")
conf["bar"] = baz
col.set_config("foo", conf)
2020-04-06 15:39:47 +10:00
Damien Elmes
ac54f85840 update tag handling
- tag list stored in a separate DB table
- non-wildcard searches now do full unicode case folding
(eg tag:masse matches 'Maße')
- wildcard matches do simple unicode case folding
- some functions haven't been updated yet, so ascii folding will
continue to be used in some operations
2020-04-03 19:34:46 +10:00
Damien Elmes
07c414692e preserve mtime/usn when syncing deck config, and add snake_case names 2020-04-03 19:34:46 +10:00
Damien Elmes
284ee0860b allow aborting a media sync while http req in flight
The progress handling code needs a rethink, as we now have two separate
ways to flag that the media sync should abort. In the future, it may
make sense to switch to polling the backend for progress, instead of
passing a callback in.
2020-04-01 09:49:25 +10:00
Damien Elmes
cffe146397 split deck config into separate SQL table
- on collection load, the schema is upgraded to 12
- on collection close, the changes are reversed so older clients
can continue to open the collection
- in the future, we could potentially skip the reversal except
when exporting/doing a full sync
- the same approach should work for decks, note types and tags in the
future too
- the deck list code needs updating to cache the deck confs for the
life of the call
2020-03-30 20:01:16 +10:00
Damien Elmes
089a68eb43 move deck conf handling to backend 2020-03-30 14:39:46 +10:00
Damien Elmes
fc846ac4e4 use add_card() 2020-03-27 15:11:07 +10:00
Damien Elmes
e28fdde984 handle mtime/usn bump in backend, and tweak integer sizes 2020-03-26 20:55:04 +10:00
Damien Elmes
0510ab7c9c update_card() 2020-03-26 18:54:20 +10:00
Damien Elmes
b2fba4e1ff use backend to get card 2020-03-26 17:47:43 +10:00
Damien Elmes
e02f032a47 use backend for v1 and v2 cutoff calculations
this should also fix the CI failures, which were happening because
the datetime module wasn't matching the shifted time.time()
2020-03-22 14:28:26 +10:00
Damien Elmes
b8aaa8fba4 add enum for controlling sort order
eg col.find_cards("", order=BuiltinSortKind.CARD_DUE)
2020-03-22 12:59:24 +10:00
Damien Elmes
0089f0149a add note searching 2020-03-21 09:00:05 +10:00
Damien Elmes
23717089e9 add the ability to provide a custom sort order; use backend for find 2020-03-21 07:55:21 +10:00
Damien Elmes
f36857ae27 prototype of integration
no ordering yet, and no tests
2020-03-20 21:15:23 +10:00
Damien Elmes
e5f883d1c3 support opening and closing the DB while keeping backend alive
This is safer than just dropping the backend, as .close() will
block if something else is holding the mutex. Also means we can
drop the extra I18nBackend code.

Media syncing still needs fixing.
2020-03-20 21:15:23 +10:00
Damien Elmes
3cacc00fc3 make the collection optional 2020-03-20 21:15:23 +10:00
Damien Elmes
bd7ded18c3 drop the protobuf prototype 2020-03-20 21:15:23 +10:00
Damien Elmes
a129b8f8cf initial work on DB handling in Rust
committing the Protobuf implementation for posterity, but will replace
it with json, as Protobuf measures about 6x slower for some workloads
like 'select * from notes'
2020-03-20 21:15:23 +10:00
Damien Elmes
e512199ace add restore media action 2020-03-10 13:35:09 +10:00
Damien Elmes
cac9ccfa0d add empty trash action in aqt 2020-03-10 12:49:40 +10:00
Damien Elmes
67a101add1 add missing newline to .proto 2020-03-09 18:58:49 +10:00
Damien Elmes
c97b37b7ae make logging explicit, and support logging to a file 2020-02-29 20:44:01 +10:00
Damien Elmes
b030f2b40b info is superfluous 2020-02-27 10:53:49 +10:00
Damien Elmes
83d894ac87 return a localized error for all error kinds
some errors are not yet localized, but now the Python code
doesn't need to think about which property to use
2020-02-27 10:46:24 +10:00
Damien Elmes
94953cf6b5 separate out infallible and fallible commands in .proto 2020-02-27 08:46:09 +10:00
Damien Elmes
59328d9ade tweak rounding
- avoid rounding minutes
- round the seconds taken in the card info screen
- provide different precise and imprecise modes, since we need
to display seconds to multiple decimals in some areas
2020-02-25 17:29:06 +10:00
Damien Elmes
ea22296d4c expand clozes before rendering LaTeX
https://anki.tenderapp.com/discussions/ankidesktop/39071-latex-media-windows-version-2121-0cd1ee92-fail-rendering-latex
2020-02-24 18:34:08 +10:00
Damien Elmes
c98247fa0f translations no longer require an open collection 2020-02-23 14:57:02 +10:00
Damien Elmes
0d1a30c6d8 rework Fluent handling
- all .ftl files for a language are concatenated into a single file
at build time
- all languages are included in the binary
- external ftl files placed in the ftl folder can override the
built-in definitions
- constants are automatically generated for each string key
- dropped the separate StringsGroup enum
2020-02-23 13:22:50 +10:00
Damien Elmes
72528ee41c add deck config category 2020-02-21 20:53:20 +10:00
Damien Elmes
e84fa7dee3 add next learning due time + remaining count to congrats screen
https://anki.tenderapp.com/discussions/ankidesktop/38000-v2-scheduler-learning-cards
2020-02-21 19:54:50 +10:00
Damien Elmes
3a83d9844a add studied_today(), move to statistics.ftl 2020-02-21 18:01:15 +10:00
Damien Elmes
74bff450e2 add non-abbreviated timespan translation; update existing short=True calls
- drop the '5m3s' special casing done in the card stats screen, and
just use decimals
- change alignment of the review log so that the non-abbreviated
spans are easier to read
2020-02-21 15:29:38 +10:00
Damien Elmes
11647daec1 move answer_button_time to the backend, split sched into separate module 2020-02-21 15:29:38 +10:00
Damien Elmes
2633d6db4b move answer button labels into fluent 2020-02-21 15:29:38 +10:00
Damien Elmes
cf1824dac5 update to latest fluent-rs and add basic locale-aware decimals
- git version pinned at the moment until the concurrency fix
lands in 0.10.2
- currently float values are hard-coded at 2 decimal places;
we should switch to using NUMBER() in the future
2020-02-21 15:29:38 +10:00
Damien Elmes
fd01cac4d7 split "Due" into three different contexts for translators 2020-02-17 14:41:01 +10:00
Damien Elmes
f40a00f0cb localize some error messages 2020-02-17 11:38:22 +10:00
Damien Elmes
4b63a07c99 expose translations to Python 2020-02-17 08:40:17 +10:00
Damien Elmes
1f736239cd pass progress back as translated string(s) 2020-02-17 08:40:17 +10:00
Damien Elmes
78ffde32e1 expose StringsGroup enum in protobuf 2020-02-17 08:40:17 +10:00
Damien Elmes
0738ff46b6 make template errors translatable 2020-02-17 08:40:17 +10:00
Damien Elmes
834989a019 include report in MediaCheckOutput 2020-02-17 08:40:17 +10:00
Damien Elmes
36d2536262 add i18n support to Rust code using Fluent 2020-02-17 08:40:17 +10:00
Damien Elmes
1383973753 shorten add_file_to_media_folder 2020-02-17 08:40:17 +10:00
Damien Elmes
89626c1a72 use media.trash for unused media deletion as well 2020-02-17 08:40:17 +10:00
Damien Elmes
d32e2fe6eb add extract_latex to backend; use it for latex build 2020-02-17 08:40:17 +10:00
Damien Elmes
790f3c9f72 plug new media check in 2020-02-17 08:40:17 +10:00
Damien Elmes
edf60b10d4 backend stores media folder/db locations; don't need to pass them in 2020-02-17 08:40:17 +10:00
Damien Elmes
9c60f2c77e log media DB change registration progress, and allow aborting it 2020-02-17 08:40:17 +10:00
Damien Elmes
ef14b1a96b simplify code by accumulating stats at the source 2020-02-17 08:40:17 +10:00
Damien Elmes
1f931a5b78 defer errors on upload to media check 2020-02-17 08:40:17 +10:00
Damien Elmes
77158760ff use enums for some common errors 2020-02-17 08:40:16 +10:00
Damien Elmes
738e415463 connect media sync progress to Python, add a progress dialog 2020-02-17 08:40:16 +10:00
Damien Elmes
50d8a40e23 add progress hook to media sync 2020-02-17 08:40:16 +10:00
Damien Elmes
78bfbef863 media sync working, but unpolished 2020-02-17 08:40:16 +10:00
Damien Elmes
1acb3b86b9 backend init can now fail, and update media db when file is added
- Adding files inside Anki now updates the media DB, so a full file
scan at sync time is no longer required if no other changes have been
made.
- Use a protobuf message for backend initialization, and return a
string error if initialization fails.
2020-02-17 08:40:16 +10:00
Damien Elmes
9bc38647e6 initial rs implementation of media database 2020-02-17 08:40:16 +10:00
Damien Elmes
c9a5ed3cc6 add add_file() and write_data() 2020-02-17 08:40:16 +10:00
Damien Elmes
391a58077a show which side the template error occurred on 2020-02-08 11:54:08 +10:00
Damien Elmes
86a580d820 reuse reveal_cloze_text() for LaTeX cloze expansion 2020-01-28 07:40:44 +10:00
Damien Elmes
47bcb3845c support speed control in tts tags 2020-01-26 14:31:07 +10:00
Damien Elmes
46c6a7f7e4 extract and flag AV tags as part of the render process
We can now show replay buttons for the audio contained in {{FrontSide}}
without having to play it again when the answer is shown.

The template code now always defers FrontSide rendering, as it wasn't
a big saving, and meant the logic had to be implemented twice.
2020-01-24 11:06:11 +10:00
Damien Elmes
2ee8902bd9 added flag_av_tags() for replay button support 2020-01-21 14:44:27 +10:00
Damien Elmes
b0d2958721 add rank handling to TTS; parse TTS args in get_av_tags() 2020-01-21 12:41:37 +10:00
Damien Elmes
548ed31ef7 add a simple TTS field filter
This simply wraps the field in extra text that the frontend will
deal with. Also added some helpers for extracting and stripping
audio and TTS tags from the rendered text.
2020-01-20 19:16:24 +10:00
Damien Elmes
c525a75fa3 drop plus_one test 2020-01-17 18:54:38 -07:00
Damien Elmes
a9378b1e99 _localOffsetForDate() was broken
It was including the elapsed time of day when calculating
the offset, leading to incorrect results
2020-01-17 18:52:36 -07:00
Damien Elmes
8eceedd2e5 more template rendering tweaks
- The front and back are rendered in one call now. If the front
side contains no custom filters, we can bake {{FrontSide}} into the
rear side. If it did contain custom filters, we return the partially
complete rear template instead, and the calling code can inject
the FrontSide in after it has been fully rendered.

- Instead of modifying "cloze" into something like "cq-2", the card
ordinal and whether we're rendering the question or answer are now
passed in to the rendering filters as context.

- The Rust code doesn't need to support filter names split on '-'
anymore.

- Drop the "Show" part of hint descriptions so i18n support can be
deferred.

- Ignore blank filter names caused by user using two colons instead
of one.

- Fixed hint field and text transposition.
2020-01-12 21:34:07 +10:00
Damien Elmes
3a14c247c8 replace fields and apply standard filters in rslib
The filters still need to be implemented.
2020-01-10 18:02:26 +10:00
Damien Elmes
8f3df55da2 flatten->render; pass field content in
This is paving the way to move the standard filters into Rust.

Non-empty fields are now determined in Rust, using a single regex
instead of the overkill stripHTMLMedia(). The old implementation
has been moved into the Pystache code for now.
2020-01-10 14:59:29 +10:00
Damien Elmes
31ebf0473f rework new timezone code
We now store the UTC offset that was in effect at creation time,
and use that to determine the starting date.
2020-01-09 16:58:27 +10:00
Damien Elmes
520275bea3 handle conditional replacement in Rust
This extends the existing Rust code to handle conditional
replacement. The replacement of field names and filters to text
remains in Python, so that add-ons can still define their own
field modifiers.

The code is currently running the old Pystache rendering and the
new implementation in parallel, and will print a message to the
console if they don't match. If you notice any problems, please
let me know.
2020-01-08 20:28:04 +10:00
Damien Elmes
ddb304aad2 add some more protobuf messages 2020-01-06 12:18:20 +10:00
Damien Elmes
c88694333b update readme, remove submodule readmes 2020-01-04 12:21:58 +10:00
Damien Elmes
3ca39bfda3 move the separate components back into this monorepo
Earlier today I pushed a change that split this code up into multiple
repos, but that has proved to complicate things too much. So we're
back to a single repo, except the individual submodules are better
separated than they were before.

The README files need updating again; I will push them out soon.

Aside from splitting out the different modules, the sound code has
moved from from anki to aqt.
2020-01-02 19:43:19 +10:00
Damien Elmes
7fceffa890 rework the new scheduler's rollover/day counting code
The previous implementation interpreted the creation date as a local
time, and applied the rollover to that. If the initial creation date
was around midnight local time, even a one hour change due to daylight
savings could result in Anki skipping or doubling up on a day. To
address this, the rollover is now applied to the current time instead
of the creation date.

The new code needs the current time passed into it. This makes it
easier to unit test, and for AnkiWeb to be able to use the user's local
timezone.

The new timezone code is currently disabled, as this code needs to be
ported to all clients before it can be activated.
2019-12-27 21:53:20 +10:00
Damien Elmes
8ef1fa7ed1 bridge->backend 2019-12-25 08:59:33 +10:00
Damien Elmes
4347f10bcc compute template requirements in Rust
on a 100 field template, what took ~75 seconds now takes ~3 seconds.
2019-12-24 19:57:54 +10:00
Damien Elmes
e05b77b083 use protobuf for python/rust bridge
If we go ahead with betterproto on the Python side, it will mean
bumping the minimum Python dependency to 3.7.
2019-12-24 19:57:54 +10:00