Commit graph

2053 commits

Author SHA1 Message Date
Damien Elmes
9537646604 Rework error dialog
- Hide traceback
- Include full add-on info in 'copy debug info' button, like about
screen
- Link to troubleshooting page
- Use non-modal pop-up in the common case, to avoid potential conflicts
with other modals.

Closes #2830
2023-11-29 10:25:32 +10:00
Damien Elmes
cb39d40f9f Remove some v3 checks 2023-11-28 14:06:49 +10:00
Damien Elmes
910d864ee7 Update to Rust 1.74, and update most Rust deps
hyper has been held back, as neither reqwest nor axum support 1.0 yet
2023-11-27 13:34:42 +10:00
Damien Elmes
ff3cdaa4cf Suppress manual revlog entry if the previous entry was also manual
Means we lose some detail in the history, but should reduce the
collection bloat caused by a user experimenting with reschedule multiple
times in a short period, when they don't restore from backup afterwards.

https://forums.ankiweb.net/t/possible-syncing-limitation-by-fsrs-manual-scheduling-data-accumulation/37610
2023-11-27 11:24:31 +10:00
Damien Elmes
aea95ee9ae Switch FSRS reschedule to a global option; don't persist
A global is easier to use in conjunction with the 'optimize all' action.
The value is no longer persisted, as doing so makes it too easy for users
to generate a lot of revlog entries when playing with different FSRS
weights/retention settings, such as in https://forums.ankiweb.net/t/possible-syncing-limitation-by-fsrs-manual-scheduling-data-accumulation/37610
2023-11-27 11:24:31 +10:00
Damien Elmes
1021502377 Add option to calculate all weights at once 2023-11-27 11:24:31 +10:00
Damien Elmes
91d2822358 Update to latest FSRS 2023-11-27 11:24:31 +10:00
Jarrett Ye
e88866acc3 don't convert from sm2 when ivl = 0 (#2849) 2023-11-26 13:14:31 +10:00
Abdo
b5f1c6a53c Fix sorting of due timestamps in the browser (#2840)
* Fix sorting of due timestamps in the browser

* Fix due sorting in notes mode

* Drop initial ctype sorting

* Fix new card positions being treated as due days
2023-11-24 18:31:14 +10:00
Jarrett Ye
dcd729374d ignore filtered revlog in time of last review (#2845)
* ignore filtered revlog in time of last review

* fmt SQL
2023-11-24 15:52:00 +10:00
Damien Elmes
a64624a3d6 Use note order when previewing
https://forums.ankiweb.net/t/card-in-disorder/37534/2
2023-11-24 15:01:14 +10:00
Mani
b8d9e77984 fixes: remove unfinished shapes, remove selectable and make shapes remain inside canvas (#2809)
* remove unfinished polygon and remove selectable for shapes in polygon mode

* make group and polygon position remain inside canvas area

* click through transparent area in grouped object

* add some shortcuts for basic usages

* tools button icon in center & switch mode border

* fix load svg image

* basic rtl support, panzoom have issues in rtl mode

* better zoom option both in ltr and rtl

* handle zoom event in mask editor

* add h button to handle toggle mask

* add more mime type

* use capital M (shift+m) for toggle mask

* allow io shortcuts in mask editor only

* make other shapes also remain in canvas bound area

* better zoom implementation, zoom from center
add zoom to resize event listener

* add a border to corner to handle blend of control

* add refresh button to go to  selection menu

* add tooltip to shortcuts and also add shortcut for other tools

* make opacity remain in same state when toggled on

* opacity for group/ungroup objects

* update shortcuts implementation
2023-11-24 14:06:40 +10:00
Damien Elmes
89fe5f710c Include next_day timestamp in revlog export 2023-11-23 14:46:48 +10:00
Damien Elmes
d4b5d85ea7 Add remove_dir_all() 2023-11-22 17:41:45 +10:00
Damien Elmes
70900e91d0 Add a method for exporting revlogs in binary 2023-11-22 11:57:40 +10:00
Damien Elmes
e6a7a8ff4a When training with the whole collection, include deleted card history 2023-11-22 10:53:08 +10:00
Damien Elmes
c97205a535 Fix FSRS retrievability sorting issues
- We must use interval, not stability to infer days_elapsed
- We must use original due date in a filtered deck
- Use retrievability in filtered deck sorting, not just regular sorting
2023-11-20 13:59:45 +10:00
Damien Elmes
2da97e4720 Fix cards showing up due in the past in a filtered deck
Caused by using due instead of original_due when card was in learning.
I think the original goal of that code was to ignore the learning timestamp
and show the next review date instead, but it's both simpler and more
intuitive to show the learning date instead.
2023-11-20 10:52:13 +10:00
Damien Elmes
353d1ac003 Ignore invalid I/O notetypes when deciding whether to add
https://forums.ankiweb.net/t/can-t-add-new-cards/37155
2023-11-15 13:32:02 +10:00
Abdo
fc5b38567b Fix filtered deck ordering of intraday learning cards and reviews (#2828)
* Fix filtered deck ordering of intraday learning cards and reviews

* Handle rollover correctly
2023-11-14 15:08:36 +10:00
Abdo
7c2007ccad Persist FSRS weights search in preset (#2827) 2023-11-14 11:47:08 +10:00
RumovZ
134793835f Allow applying limits of inactive parents (#2824)
* Allow applying limits of inactive parents

* Tweak label/help text (dae)
2023-11-13 14:30:19 +10:00
RumovZ
cf28ec1437 Allow im-/exporting with or without deck configs (#2804)
* Allow im-/exporting with or without deck configs

Closes #2777.

* Enable webengine remote debugging in launch.json

* Reset deck limits and counts based on scheduling

Also:
- Fix `deck.common` not being reset.
- Apply all logic only depending on the source collection in the
gathering stage.
- Skip checking for scheduling and only act based on whether the call
wants scheduling. Preservation of filtered decks also depends on all
original decks being included.
- Fix check_ids() not covering revlog.

* Fix importing legacy filtered decks w/o scheduling

* Disable 'include deck options' by default, and fix tab order (dae)

* deck options > deck presets (dae)
2023-11-13 13:54:41 +10:00
Damien Elmes
e5153fde78 Fix some build steps being re-run a second time unnecessarily 2023-11-13 13:35:54 +10:00
Abdo
cb9d5218f7 Add auto-advance options to deck preset (#2765)
* Move stop-timer-on-answer strings to correct section

* Add auto-advance options to deck preset

* Implement answer actions

* Fix error when last card is answered before timeout

* Fix deserialization of answerAction

* Add answerAction to reserved key list

* Fix inverted boolean

* Add option to wait for audio to finish

* Add auto-advance toggle

* Add shortcut

* Disable auto-advance when main window state changes

* Start auto-advance timer after option is toggled

* Disable auto-advance when main window loses focus

* Use existing translations

* Add Answer Hard and Show Reminder
2023-11-13 10:41:51 +10:00
Abdo
48d38bb9af Show due date of buried/suspended cards in card info (#2820)
* Show due date of buried/suspended cards in card info

* Simplify match and handle relearning cards

* Omit ctype checks

* Test due, not card.due (dae)
2023-11-13 10:23:46 +10:00
Damien Elmes
b989c98639 Update fsrs-rs 2023-11-13 10:13:20 +10:00
Abdo
92b88fc3e2 Move anki.utils.html_to_text_line() to backend (#2816) 2023-11-09 09:57:23 +10:00
Damien Elmes
7bcb31b65b Check video tags in media check
https://forums.ankiweb.net/t/video-tag-does-not-recognize-the-source-file-in-check-media/36850/2
2023-11-08 14:00:41 +10:00
Damien Elmes
3c31386c1f Change color of info box depending on retention; increase limit to 0.99
Closes #2803
2023-11-07 13:05:43 +10:00
Damien Elmes
a2adbf0bf2 Expose fuzz delta for FSRS add-on 2023-11-06 12:27:53 +10:00
Damien Elmes
4cbb9a36d2 Remove redundant FSRS revlog code
Closes https://github.com/open-spaced-repetition/fsrs-rs/issues/113
2023-11-06 09:44:07 +10:00
Damien Elmes
c22cf9fc50 Handle stability < 0.5 in stability graph 2023-11-05 13:44:39 +10:00
Gustaf-C
5ff618b7ac Allow creation of empty filtered decks (#2788)
* Add new button to UI

* Add bool to allow creating empty filtered in back end

* Implement logic into front end for passing on bool

* Hide option on old decks

* Show option again if any settings are changed

* Revert "Show option again if any settings are changed"

This reverts commit 094acd9c65936823fa206594da5c1f3e4eb09248.

* Revert "Hide option on old decks"

This reverts commit d20a9a240b4fd85d080e8cc52d94318416ca753f.

* Update string

* Update ftl/core/decks.ftl

---------

Co-authored-by: Damien Elmes <dae@users.noreply.github.com>
2023-11-05 12:23:14 +10:00
Damien Elmes
aa5ccad26e Fix panic when enabling FSRS with add-on-rescheduled cards
https://forums.ankiweb.net/t/error-upon-fsrs-activation-on-anki-23-10/36488
2023-11-03 10:07:47 +10:00
RumovZ
2d03f6a55c Time out after 60s when checking URLs (#2785) 2023-11-01 07:46:49 +10:00
Damien Elmes
2b07464ca4 Fix D/R graphs excluding values of 1.0
Closes #2783
2023-10-29 07:45:10 +10:00
Damien Elmes
dbfbd356f1 Fix sm2_retention getting reset on downgrade/sync 2023-10-28 14:45:24 +10:00
Damien Elmes
04b5f53896 Expose method to check for v2 as well 2023-10-27 02:47:04 +10:00
Damien Elmes
9b496816b2 Don't lock collection while generating TTS 2023-10-26 11:45:17 +10:00
Damien Elmes
301405da5c Revert "Use scheduling flag instead of checking revlog when importing decks (#2730)"
This reverts commit 068d79329e.

The previous change breaks the assumption on this line:

                DeckKind::Filtered(_) => unreachable!(),

This will likely need a bigger refactor to handle this properly, and separate
out importing of deck configs from other scheduling data.
2023-10-26 11:23:24 +10:00
Damien Elmes
4b05cda8e2 Add public method to determine if v3 is enabled 2023-10-25 11:41:05 +10:00
Abdo
ebcbed1b83 Fix IO groups breaking upon editing (#2766)
* Fix IO groups breaking upon editing

* Emit change signal after group/ungroup
2023-10-23 09:43:31 +10:00
Damien Elmes
c9eb57ec7a Add image occlusion to stock notetypes 2023-10-22 11:00:39 +10:00
Aristotelis
e4d904444d Add ordinals to IO clozes (#2763)
Co-authored-by: Glutanimate <glutanimate@users.noreply.github.com>
2023-10-22 10:52:22 +10:00
Damien Elmes
cdf8fcaaaf Fix browser query not matching stats
- Round instead of floor when binning
- Account for rounding in search

https://forums.ankiweb.net/t/anki-23-10-release-candidate/35967/4
2023-10-21 12:00:43 +10:00
Damien Elmes
9e218e115b "Notetype" -> "Note Type" in browser column
While we may want to update things in the other direction in the future
(https://github.com/ankitects/anki/pull/1456#issuecomment-951767922),
this is more consistent with the existing other strings

Closes #2754
2023-10-21 11:35:30 +10:00
Gustaf-C
5c8bcd577d Tweak browser column labels to card type, note type (#2751)
* Tweak browser column labels to card type, note type

* Apply suggestions from code review
2023-10-20 12:46:09 +10:00
Aristotelis
8726ef8fff Add APIs for IO card rendering (#2739)
* Refactor: Add index to shapes package

* Add shape draw callback API to setupImageCloze

* Expose IO drawing API, switch away from image cloze naming

We currently use "image occlusion" in most places, but some references to "image cloze" still remain. For consistency's sake and to make it easier to quickly find IO-related code, this commit replaces all remaining references to "image cloze", only maintaining those required for backwards compatibility with existing note types.

* Add cloze ordinal to shapes

* Do not mutate original shapes during (de)normalization

Mutating shapes would be a recipe for trouble when combined with IO API use by external consumers.

(makeNormal(makeAbsolute(makeNormal())) is not idempotent,
and keeping track of the original state would introduce
additional complexity with no discernible performance benefit
or otherwise.)

* Tweak IO API, allowing modifications to ShapeProperties

* Tweak drawShape parameters

* Switch method order

For consistency with previous implementation

* Run Rust formatters

* Simplify position (de)normalization

---------

Co-authored-by: Glutanimate <glutanimate@users.noreply.github.com>
2023-10-20 09:36:46 +10:00
Sam Waechter
6547376845 Trim extra colons on conversion to NativeDeckName (#2748)
This fixes issue #2692. Leading/trailing colons are remove along with whitespace so " : : :foo:bar: : : " becomes "foo:bar"
2023-10-19 13:42:57 +10:00
Abdo
b09bf31748 Fix misleading warning when changing notetype from/to cloze (#2744) 2023-10-18 16:50:32 +10:00
Damien Elmes
695bb9e291 Update to latest fsrs 2023-10-18 09:03:23 +10:00
Damien Elmes
e41e7cc508 Fix stuttering in FSRS progress display 2023-10-14 14:03:20 +10:00
Abdo
10966394fd Remove v1/v2 support from the backend (#2727)
* Remove v1/v2 support from deck list

* Remove v1/v2 support from most routines and show error

* Remove scheduler_version from preferences

* Fix formatting

* Remove v1/v2 conditionals from Python code

* Fix legacy importer

* Remove legacy hooks

* Add missing scheduler checks

* Remove V2 logic from deck options screen

* Remove the review_did_undo hook

* Restore ability to open old options with shift (dae)
2023-10-14 10:50:59 +10:00
Damien Elmes
068d79329e Use scheduling flag instead of checking revlog when importing decks (#2730)
Closes #2714
2023-10-14 10:30:20 +10:00
Damien Elmes
ec8990dc11 Use sm2 retention when deriving memory state
Closes #2702
2023-10-13 10:37:35 +10:00
Abdo
495dc4d87c Add text tool to image occlusion (#2705)
* Add text tool to IO

* Remove unnecessary parentheses

* Fix text objects always grouped

* Remove log

* Fix text objects hidden on back side

* Implement text scaling

* Add inverse text outline

* Warn about IO notes with only text objects

This will result in a different error message than the case where no
objects are added at all though, and the user can bypass the warning.
Maybe this is better to avoid discarding the user's work if they have
spent some time adding text.

* Add isValidType

* Use matches!

* Lock aspect ratio of text objects

* Reword misleading comment

The confusion probably comes from the Fabric docs, which apparently need updating: http://fabricjs.com/docs/fabric.Canvas.html#uniformScaling

* Do not count text objects when calculating current index

* Make text objects respond to size changes

* Fix uniform scaling not working when editing

* Use Arial font

* Escape colons and unify parsing

* Handle scale factor when restricting shape to view

* Use 'cloned'

* Add text background

* Tweak drawShape's params
2023-10-12 13:40:11 +10:00
Abdo
3a52080eeb Add a backend method to extract static media references (#2716)
* Add a backend method to extract static media references

* Extract into Notetype.gather_media_names()
2023-10-11 14:10:02 +10:00
Damien Elmes
eaff6b466b Rust 1.73 2023-10-09 19:22:43 +10:00
Damien Elmes
bd6f0d2d92 Fix 'PanicException: command requires weights to be set' 2023-10-09 18:39:18 +10:00
Damien Elmes
6394a449bc Fix 'Expression tree is too large' when user has many decks
+ Allow 'did:' searches to match multiple decks at once
2023-10-09 18:39:18 +10:00
Damien Elmes
3738d9ce28 Apply tag fix from Abdo
https://forums.ankiweb.net/t/drag-and-drop-tag-groups-crashes-anki/35505/3
2023-10-09 13:55:37 +11:00
RumovZ
fd94ad49a0 Invalid sorting (#2709)
* Rollback if toggling state fails

Previously, if the search triggered by a state toggle failed, the switch
and the model would move to the new state, while the table would remain
in the previous state.

* Fix reversed sort orders of FSRS columns

* Add sep. default sort orders for notes and cards

* Add test for consistent default sort orders

* Add launch config for debugging in VSC

* Extend launch config for macOS and Linux
2023-10-07 06:36:15 +00:00
RumovZ
c7d058aa0b Merge all conflicting notetypes (#2707)
* Refactor import apkg tests

* Merge conflicting notetypes regardless of id match

Original ids are a new thing, and we need to handle previous remappings.
This is done separately from the conflict resolution for notetypes with
matching ids, because 1) we need to look at the notes to determine
conflicts, and 2) we don't want to change the notetype of *all* existing
notes with the conflicting notetype. The main reason is that for 2
existing notes with the same noteype, their incoming counterparts could
have *different* notetypes. So to get rid of all conflicts, they must be
resolved on a note-by-note basis.

* Delete merged, now unused notetypes
2023-10-05 06:18:10 +00:00
Damien Elmes
5977937800 Use 0 when there are no relearning steps; ensure not NaN 2023-10-05 09:55:05 +10:00
Damien Elmes
c4d00bce05 When the user has no relearning steps, synthesize the value 2023-10-03 13:50:29 +10:00
Damien Elmes
e8154d8678 Ensure fuzz doesn't go backwards
Closes #2694
2023-10-03 13:23:27 +10:00
Damien Elmes
32ca7d237f When learning steps are missing, start from the SM-2 state
Closes https://github.com/open-spaced-repetition/fsrs-rs/issues/87
2023-10-03 12:20:10 +10:00
Damien Elmes
9168e33ac6 Update FSRS crate 2023-10-03 11:28:56 +10:00
Damien Elmes
8731a1f7ce Add stability graph 2023-10-01 15:44:33 +10:00
Damien Elmes
49fcbe8f8f Support rescheduling on weight/retention change 2023-10-01 15:20:58 +10:00
Damien Elmes
cf69ca38ad Allow cards with no learning history when not training 2023-09-30 16:10:23 +10:00
Damien Elmes
6b0bc2ce71 Display the default weights as a placeholder 2023-09-30 16:10:23 +10:00
Damien Elmes
63088b0caf Update FSRS
- up to 10x performance increase in optimal retention
- expose loss aversion
- use SpinBoxes
2023-09-30 16:10:22 +10:00
Damien Elmes
a0e32286bb Update incompatible Rust deps 2023-09-28 10:00:47 +10:00
Damien Elmes
d631a7fb9c Shift weight calculation to backend so it can be run in parallel 2023-09-28 09:10:54 +10:00
Damien Elmes
00825d015c Expose the ability to train weights from items (#2687) 2023-09-28 08:28:24 +10:00
Jarrett Ye
742b7a7daa Fix first_rating_count & learn_cost (#2688)
* fix first_rating_count & learn_cost

* Update CONTRIBUTORS

* cargo clippy --fix

* cargo fmt
2023-09-27 18:31:40 +10:00
Damien Elmes
170d0c737d Use revlog to determine days_elapsed when studying/for card info
Currently prop searches and the retrievability column will continue to
derive the days from the card only, as it's difficulty to integrate revlog
upgrade lookups into those code paths, especially in a performant way.
One possible way we could solve this in the future is to store last_review_day
in the card data, so we can know it even if the due date has been shifted.
Check DB could fill it in for existing cards.
2023-09-27 16:17:47 +10:00
Damien Elmes
68d4878d55 Fix days_since_last_review() when used in filtered deck 2023-09-27 16:17:47 +10:00
Abdo
97bd06dc49 Implement "stop timer on answer" as a preset option (#2686)
* Implement "stop timer on answer" as a preset option

* Hide timer setting on AnkiMobile (dae)
2023-09-27 16:10:14 +10:00
Damien Elmes
c9b35c4fc6 Update FSRS; fix handling of invalid revlogs
State is now inferred from SM-2 data when the revlog is not suitable
2023-09-27 13:13:10 +10:00
Damien Elmes
a82021cb05 Fix invalid utf-8 in tags 2023-09-27 10:06:31 +10:00
Damien Elmes
676c5bb3ce Fix optimal retention caps 2023-09-26 14:17:55 +10:00
Abdo
da4d6f547c Match IO extensions in a case-insensitive manner (#2683) 2023-09-26 13:14:59 +10:00
Damien Elmes
3a6dd627d7 Allow desired retention to be set to 0.7
https://github.com/open-spaced-repetition/fsrs-rs/issues/79#issuecomment-1733779005
2023-09-26 13:03:49 +10:00
Damien Elmes
189b4f51d6 Fix changes_since_open() 2023-09-26 11:02:16 +10:00
Damien Elmes
35bcfa8679 Fix cancel signal being ignored if sent quickly 2023-09-25 16:32:12 +10:00
Damien Elmes
e9f6d181ed Allow cards with no learning history when not training 2023-09-25 15:57:35 +10:00
Damien Elmes
790644de66 UI tweaks
https://forums.ankiweb.net/t/anki-23-10-beta-1/34912/19
2023-09-25 15:54:18 +10:00
Damien Elmes
620a55c6e5 Support retrievability calculation for (re)learning cards 2023-09-25 14:58:39 +10:00
Damien Elmes
105addfa0d Fix extract_fsrs_retrievabilty()
- Need to use fuzzed interval to accurately determine elapsed days
- saturating_sub on an i32 is not useful
2023-09-25 14:40:44 +10:00
Damien Elmes
681d561a98 Expose memory state computation to Python
Closes #2676
2023-09-25 11:05:47 +10:00
Damien Elmes
48f643485c Use memory state to calculate relative overdueness 2023-09-24 13:07:11 +10:00
Damien Elmes
bdeca36234 When calculating retrievability, use stability instead of fuzzed interval 2023-09-24 12:53:05 +10:00
Damien Elmes
fc83f8d3cc Support fetching new cards by deck then random note
https://forums.ankiweb.net/t/feature-request-option-for-new-card-gather-order-that-prioritizes-subdecks-closer-to-top-but-gathers-cards-randomly-from-each-subdeck/23178
2023-09-24 11:54:10 +10:00
Damien Elmes
d2a00948ef Use FSRS difficulty when sorting by ease 2023-09-23 16:20:36 +10:00
Damien Elmes
57941e499a Allow extracting desired retention 2023-09-23 16:11:35 +10:00
Damien Elmes
af171c96d7 Gate graph display on fsrs status 2023-09-23 15:59:02 +10:00
Damien Elmes
0b84905c85 Avoid excessive floating point precision when serializing 2023-09-23 15:50:10 +10:00
Damien Elmes
3d59a0eebe Store desired retention in card data
If we want to be able to factor the desired retention into a sort based
on relative overdueness, having the values accessible on the card makes
things easier.
2023-09-23 15:42:42 +10:00
Damien Elmes
0f899efea7 Convert FSRS to a global option
Allowing some decks to be FSRS and some SM-2 will lead to confusing
behavior when sorting on SM-2 or FSRS-specific fields, or when moving
cards between decks.
2023-09-23 14:41:55 +10:00
Abdo
a3c7a07a96 Add an option to stop the timer on answer (#2673)
* Add an option to stop the timer on answer

* Fix tab order
2023-09-23 14:01:03 +10:00
Abdo
df8b439200 Skip template checks in Fields screen (#2670) 2023-09-20 16:09:54 +10:00
Damien Elmes
be725dca11 Fix crash when using FSRS with a low max interval setting 2023-09-20 13:14:03 +10:00
Damien Elmes
cafba25f4c Calculate parameters automatically
Logic derived from d8e2f6a0ff

Closes #2661
2023-09-18 16:43:36 +10:00
Sam Waechter
2e99c71464 Fix unable to save field dialog if certain fields are deleted (#2663)
* Fix unable to save field dialog if certain fields are deleted

Implemented solution suggested in issue #2556

* Fix unable to save field dialog if certain fields are deleted

fixed code formating

* Fix unable to save field dialog if certain fields are deleted

Made new functions to check referencelessness.  Added unit test.
2023-09-18 14:33:56 +10:00
Damien Elmes
41220ea61b Use field tags instead of hard-coding occlusion fields
+ Don't protect the comments field

It's not required by our current code. We can remove the protection
from Header and Back Extra in the future too, once we no longer depend
on them.

Closes #2621
2023-09-18 10:10:11 +10:00
Damien Elmes
125d0a2ad0 Add tags in DB check 2023-09-17 15:00:28 +10:00
Damien Elmes
b201feea69 Protect image occlusion fields and cloze field 2023-09-17 14:22:25 +10:00
Damien Elmes
581c7c87f1 Add support for tagging + protecting fields 2023-09-17 14:03:42 +10:00
Damien Elmes
a1756ea771 Start on a 'get params' button 2023-09-17 12:58:13 +10:00
Damien Elmes
e04c4957d7 Fix preset searches, and expose in text search 2023-09-17 12:51:22 +10:00
Damien Elmes
02a823c846 Clear memory state when user disables FSRS 2023-09-17 11:50:38 +10:00
Damien Elmes
608769ff75 Record FSRS difficulty in the review log
Will allow user to see a record of difficulty changes, and allows us
to identify reviews that have been done with FSRS vs SM-2, since the
valid range is different.
2023-09-17 11:50:38 +10:00
Damien Elmes
551de34f73 fsrs_memory_state -> memory_state 2023-09-17 11:50:38 +10:00
Damien Elmes
41bcb20f1e Integrate FSRS into Anki (#2654)
* Pack FSRS data into card.data

* Update FSRS card data when preset or weights change

+ Show FSRS stats in card stats

* Show a warning when there's a limited review history

* Add some translations; tweak UI

* Fix default requested retention

* Add browser columns, fix calculation of R

* Property searches

eg prop:d>0.1

* Integrate FSRS into reviewer

* Warn about long learning steps

* Hide minimum interval when FSRS is on

* Don't apply interval multiplier to FSRS intervals

* Expose memory state to Python

* Don't set memory state on new cards

* Port Jarret's new tests; add some helpers to make tests more compact

https://github.com/open-spaced-repetition/fsrs-rs/pull/64

* Fix learning cards not being given memory state

* Require update to v3 scheduler

* Don't exclude single learning step when calculating memory state

* Use relearning step when learning steps unavailable

* Update docstring

* fix single_card_revlog_to_items (#2656)

* not need check the review_kind for unique_dates

* add email address to CONTRIBUTORS

* fix last first learn & keep early review

* cargo fmt

* cargo clippy --fix

* Add Jarrett to about screen

* Fix fsrs_memory_state being initialized to default in get_card()

* Set initial memory state on graduate

* Update to latest FSRS

* Fix experiment.log being empty

* Fix broken colpkg imports

Introduced by "Update FSRS card data when preset or weights change"

* Update memory state during (re)learning; use FSRS for graduating intervals

* Reset memory state when cards are manually rescheduled as new

* Add difficulty graph; hide eases when FSRS enabled

* Add retrievability graph

* Derive memory_state from revlog when it's missing and shouldn't be

---------

Co-authored-by: Jarrett Ye <jarrett.ye@outlook.com>
2023-09-16 16:09:26 +10:00
Abdo
85314f263f Add a backend method to add notes in bulk (#2659)
* Add a backend method to add notes in bulk

* i -> idx

* Remove duplicate assignment

* Allow add_notes to work with multiple deck IDs

* Rename note_deck_id to requests
2023-09-16 13:51:32 +10:00
Damien Elmes
cafa99f239 Fix merge_undoable_ops() not updating counter 2023-09-15 16:53:00 +10:00
Abdo
c4cfc79d61 Fix unescaped HTML in type-in-the-answer (#2658) 2023-09-14 11:34:22 +10:00
RumovZ
15e3b5e3ab Tooltips for CSV import and import page refactoring (#2655)
* Make enum selector generic

* Refactor ImportCsvPage to support tooltips

* Improve csv import defaults

* Unify import pages

* Improve import page styling

* Fix life cycle issue with import properties

* Remove size constraints to fix scrollbar styling

* Add help strings and urls to csv import page

* Show ErrorPage on ImportPage error

* Fix escaping of import path

* Unify ImportPage and ImportLogPage

* Apply suggestions from code review (dae)

* Fix import progress

* Fix preview overflowing container

* Don't include <br> in FileIoErrors (dae)

e.g. 500: Failed to read '/home/dae/foo2.csv':<br>stream did not contain valid UTF-8

I thought about using {@html ...} here, but that's a potential security issue,
as the filename is not something we control.
2023-09-14 09:06:15 +10:00
Damien Elmes
f37d35943a Refactor media sync handling (#2647)
* Refactor media sync handling

- The media USN is now returned in sync/meta, which avoids an extra
round-trip.
- Media syncing is now automatically started by the syncing code at
the end of a normal or full sync, which avoids it competing for bandwidth
and resources, and avoids duplicate invalid login messages when the auth
token is invalid.
- Added a new media_sync_progress() method to both check if media is
syncing, and get access to the latest progress.
- Updated the sync log screen to only show the latest line, like AnkiMobile.
- Show media sync errors in a pop-up, so they don't get missed. Use a non-modal
pop-up to avoid potential conflicts with other modals.

* Remove print statement
2023-09-10 13:22:20 +10:00
RumovZ
6ac2d5eb66 Merging Notetypes on Import (#2612)
* Remember original id when importing notetype

* Reuse notetypes with matching original id

* Add field and template ids

* Enable merging imported notetypes

* Fix test

Note should be updated if the incoming note's notetype is
remapped to the existing note's notetype.
On the other hand, it should be skipped if its notetype id is mapped
to some new notetype.

* Change field and template ids to i32

* Add merge notetypes flag to proto message

* Add dialog for apkg import

* Move HelpModal into components

* Generalize import dialog

* Move SettingTitle into components

* Add help modal to ImportAnkiPackagePage

* Move SwitchRow into components

* Fix backend method import

* Make testable in browser

* Fix broken modal

* Wrap in container and fix margins

* Update commented Anki version of new proto fields

* Check ids when comparing notetype schemas

* Add tooltip for merging notetypes.

* Allow updating notes regardless of mtime

* Gitignore yarn-error.log

* Allow updating notetypes regardless of mtime

* Fix apkg help carousel

* Use i64s for template and field ids

* Add option to omit importing scheduling info

* Restore last settings in apkg import dialog

* Display error when getting metadata in webview

* Update manual links for apkg importing

* Apply suggestions from code review

Co-authored-by: Damien Elmes <dae@users.noreply.github.com>

* Omit schduling -> Import all cards as new cards

* Tweak importing-update-notes-help

* UpdateCondition → ImportAnkiPackageUpdateCondition

* Load keyboard.ftl

* Skip updating dupes in 'update alwyas' case

* Explain more when merging notetypes is required

* "omit scheduling" → "with scheduling"

* Skip updating notetype dupes if 'update always'

* Merge duplicated notetypes from previous imports

* Fix rebase aftermath

* Fix panic when merging

* Clarify 'update notetypes' help

* Mention 'merge notetypes' in the log

* Add a test which covers the previously panicking path

* Use nested ftl messages to ensure consistency

* Make order of merged fields deterministic

* Rewrite test to trigger panic

* Update version comment on new fields
2023-09-09 09:00:55 +10:00
Damien Elmes
93a4eea2d4 Wrap backend in an outer Arc
Necessary to allow backend commands that run in the background instead
of blocking.
2023-09-07 14:10:50 +10:00
Damien Elmes
d6f8ac8a68 Change Anki's version scheme; bump to 23.09 (#2640)
* Accept iterables as inputs to backend methods

* Shift add-on check to backend; use new endpoint

The new endpoint will return info on a suitable branch if found,
instead of returning all branches. This simplifies the frontend code,
and means that you can now drop support for certain versions without
it also remotely disabling the add-on for people who are running one of
the excluded versions, like in

https://forums.ankiweb.net/t/prevent-add-ons-from-being-disabled-remote-stealthily-surreptitiously/33427

* Bump version to 23.09

This changes Anki's version numbering system to year.month.patch, as
previously mentioned on https://forums.ankiweb.net/t/use-a-different-versioning-system-semver-perhaps/20046/5

This is shaping up to be a big release, with the introduction of FSRS and
image occlusion, and it seems like a good time to be finally updating the
version scheme as well. AnkiWeb has been updated to understand the new
format, and add-on authors will now specify version compatibility using
the full version number, as can be seen here:

https://ankiweb.net/shared/info/3918629684

* Shift update check to backend, and tidy up update.py

* Use the shared client for sync connections too
2023-09-07 12:37:15 +10:00
Damien Elmes
a5a92977fd Drop legacy schema option 2023-09-06 15:49:14 +10:00
Damien Elmes
fff60936bc Integrate the FSRS optimizer (#2633)
* Support searching for deck configs by name

* Integrate FSRS optimizer into Anki

* Hack in a rough implementation of evaluate_weights()

* Interrupt calculation if user closes dialog

* Fix interrupted error check

* log_loss/rmse

* Update to latest fsrs commit; add progress info to weight evaluation

* Fix progress not appearing when pretrain takes a while

* Update to latest commit
2023-09-05 18:45:05 +10:00
Damien Elmes
5059b60d80 Read csv text in eagerly
This allows us to display a more useful "not utf-8" message, instead of a
generic "invalid csv" message when the metadata can't be fetched.
2023-09-05 18:33:23 +10:00
Damien Elmes
793c031f82 Fix stack overflow in request::Error mapping 2023-09-05 15:27:17 +10:00
Abdo
9ab711d9b8 Support searching for custom data strings (#2634)
* Add extract_custom_data

* Add tests for has-cd

* Add `prop:cds` query
2023-09-04 11:17:13 +10:00
Damien Elmes
e24a84fe0d Update nightly formatter
Rustfmt is now capable of formatting let Some(..) else {} blocks
2023-09-02 16:13:50 +10:00
Abdo
fc8b4a68e1 Allow searching for missing custom data properties (#2626)
* Allow searching for missing custom data properties

* Add has-cd query
2023-09-01 14:13:31 +10:00
Nil Admirari
343f58b414 Support AVIF (#2630) 2023-08-31 08:44:10 +10:00
Damien Elmes
ac8e185ac0 Fix importing link 2023-08-31 07:52:21 +10:00
Damien Elmes
c78180b590 Fix clippy issues in Rust 1.72 2023-08-25 07:56:38 +10:00
Damien Elmes
6e64b2328e New collections now start with the v3 scheduler 2023-08-19 13:31:01 +10:00
Damien Elmes
7adc404cb8 Avoid allocating in extract_latex() if possible; make public 2023-08-19 08:05:39 +10:00
Damien Elmes
ed8c04b0da Take another approach to dealing with conflicting flattened keys
The approach in #2542 unfortunately introduced a regression, as whilst
it ensured that duplicate keys are removed when downgrading, it no longer
prevented the duplicates from being removed when converting to a legacy
Schema11 object. This resulted in things like backend.get_notetype_legacy()
returning duplicate keys, and could break syncing:

https://forums.ankiweb.net/t/windows-desktop-sync-error/33128

As syncing and schema11 object usage is quite common compared to downgrading,
the extra Value deserialization seemed a bit expensive, so I've switched
back to explicitly removing the problem keys. To ensure we don't forget to
add new keys in the future, I've added some new tests that should alert us
whenever a newly-added key is missing from the reserved list.
2023-08-15 11:25:53 +10:00
Damien Elmes
d3e06caa29 Support include_filtered=false, skip_default=true case 2023-08-08 15:34:43 +10:00
Damien Elmes
de42af74e3 Support importing AnkiDroid's legacy backups
They were missing the media map, and were not importable by the desktop.
2023-08-03 19:33:16 +10:00
Abdo
4b1d11163f Improve presentation of importing results (#2568)
* Implement import log screen in Svelte

* Show filename in import log screen title

* Remove unused NoteRow property

* Show number of imported notes

* Use a single nid expression

* Use 'count' as variable name for consistency

* Import from @tslib/backend instead

* Fix summary_template typing

* Fix clippy warning

* Apply suggestions from code review

* Fix imports

* Contents -> Fields

* Increase max length of browser search bar

https://github.com/ankitects/anki/pull/2568/files#r1255227035

* Fix race condition in Bootstrap tooltip destruction

https://github.com/twbs/bootstrap/issues/37474

* summary_template -> summaryTemplate

* Make show link a button

* Run import ops on Svelte side

* Fix geometry not being restored in CSV Import page

* Make VirtualTable fill available height

* Keep CSV dialog modal

* Reword importing-existing-notes-skipped

* Avoid mentioning matching based on first field

* Change tick and cross icons

* List skipped notes last

* Pure CSS spinner

* Move set_wants_abort() call to relevant dialogs

* Show number of imported cards

* Remove bold from first sentence and indent summaries

* Update UI after import operations

* Add close button to import log page

Also make virtual table react to resize event.

* Fix typing

* Make CSV dialog non-modal again

Otherwise user can't interact with browser window.

* Update window modality after import

* Commit DB and update undo actions after import op

* Split frontend proto into separate file, so backend can ignore it

Currently the automatically-generated frontend RPC methods get placed in
'backend.js' with all the backend methods; we could optionally split them
into a separate 'frontend.js' file in the future.

* Migrate import_done from a bridgecmd to a HTTP request

* Update plural form of importing-notes-added

* Move import response handling to mediasrv

* Move task callback to script section

* Avoid unnecessary :global()

* .log cannot be missing if result exists

* Move import log search handling to mediasrv

* Type common params of ImportLogDialog

* Use else if

* Remove console.log()

* Add way to test apkg imports in new log screen

* Remove unused import

* Get actual card count for CSV imports

* Use import type

* Fix typing error

* Ignore import log when checking for changes in Python layer

* Apply suggestions from code review

* Remove imported card count for now

* Avoid non-null assertion in assignment

* Change showInBrowser to take an array of notes

* Use dataclasses for import log args

* Simplify ResultWithChanges in TS

* Only abort import when window is modal

* Fix ResultWithChanges typing

* Fix Rust warnings

* Only log one duplicate per incoming note

* Update wording about note updates

* Remove caveat about found_notes

* Reduce font size

* Remove redundant map

* Give credit to loading.io

* Remove unused line

---------

Co-authored-by: RumovZ <gp5glkw78@relay.firefox.com>
2023-08-02 20:29:44 +10:00
Damien Elmes
a1b4edc442 Extract inline images as part of media check
We also need to get to the bottom of what's causing this:
https://forums.ankiweb.net/t/anki-browse-extremely-laggy/32533
2023-07-31 12:23:16 +10:00
Damien Elmes
70bf2729e6 Tolerate {{FrontSide}} on the front side
https://forums.ankiweb.net/t/display-the-template-for-the-front-has-a-problem/32035
2023-07-13 21:39:16 +10:00
Damien Elmes
dcc19e0326 Improve performance of invalid ID fix in DB check
Fixes https://forums.ankiweb.net/t/check-database-is-stuck-on-checking-history/31891
2023-07-07 22:53:41 +10:00
Damien Elmes
b133de00cd Add updated script for transforming ftl text 2023-07-06 22:27:09 +10:00
Damien Elmes
8452de0c5a Stop including Qt translations when EXTRA_FTL_ROOT provided
This was causing index mismatches when switching between release and
non-release builds, presumably as the build script wasn't being rerun
in that case. We could add a rerun-if-changed=RELEASE, but easier to
strip this out, as it's rare to run the checks from an external repo.
2023-07-04 18:26:27 +10:00
Damien Elmes
c44cf61673 Fix i18n build script not responding to env var changes
Not much use declaring the cargo flag when we bake the env var into
the binary! Also treat an empty variable as missing.
2023-07-04 17:17:41 +10:00
Damien Elmes
39393d3520 Move i18n helpers into ftl/, with a single main.rs
Clap gives us a nice help message and better arg parsing
2023-07-04 10:47:15 +10:00
Damien Elmes
942b3b2eff Fix some ftl args not being passed correctly in TypeScript 2023-07-04 00:47:59 +10:00
Damien Elmes
c873b28d15 Move .py i18n method generation to Rust 2023-07-03 15:58:46 +10:00
Damien Elmes
c83f577601 Move .ts i18n method generation to Rust
Based on a similar approach I used for AnkiDroid. The separate modules
file has been integrated into ftl.js.
2023-07-03 14:36:09 +10:00
Damien Elmes
c25c00e411 Combine all backend methods into a single js/d.ts file, like in Python
Easier to import from, and allows us to declare the output of the build
action without having to iterate over all the proto filenames. Have
confirmed it doesn't break esbuild's tree shaking.
2023-07-03 13:46:38 +10:00
Damien Elmes
5623523a16 deckconfig.proto -> deck_config.proto
Makes it consistent with our other proto files, and matches the service
name.
2023-07-03 13:44:54 +10:00
Damien Elmes
03d4f24ff5 Support creating a standalone sync server 2023-07-02 18:22:44 +10:00
Damien Elmes
d7f6a9e972 Fix building outside workspace
rslib/build.rs needs descriptors, even if env var not set
2023-07-02 18:22:44 +10:00
Damien Elmes
7956eee8b7 Fix progress not updating after one-way sync 2023-07-02 15:47:09 +10:00
Damien Elmes
1d79f684bd Fix newly-added image occlusion notetype not syncing
https://forums.ankiweb.net/t/synchronisation-issue/31505/7
2023-07-01 22:10:29 +10:00
Damien Elmes
03ec72ea19 Update Rust and Python deps (#2567)
* Update Python deps

* Update semver-compat Rust deps

* Update most crates to latest semver

* Update to latest axum-client-ip
2023-07-01 18:26:43 +10:00
Damien Elmes
3e2d71b81a Add must_use annotations to generated protobufs 2023-06-30 17:02:36 +10:00
Damien Elmes
59fd40928a Fix full template render choking on empty fields 2023-06-27 16:30:41 +10:00
Damien Elmes
82f608af24 Rework RenderCardOutput::question/answer
Instead of flattening the output (which was missing FrontSide), alter
the behaviour of render() instead. The non-partial output is now exposed
via Protobuf, so the non-Python clients can take advantage of it.
2023-06-27 00:37:41 +10:00
Damien Elmes
99b479c191 Rename sanity check fields avoid conflict with outer span 2023-06-26 15:04:58 +10:00
Damien Elmes
684e46b2a7 Handle invalid float mtimes in DB check
Likely an add-on or third-party tool created them, and they were breaking
DB queries for a user.

https://sqlite.org/stricttables.html could help us avoid this class of
issue in the future, though we'd need to check what client versions we'd
break with this change, and would need to change the 'sfld is an integer'
hack.
2023-06-26 13:40:54 +10:00
Damien Elmes
8646d48d93 Remove some unused Rust dependencies 2023-06-24 19:30:29 +10:00
Damien Elmes
a36ab85ded Drop workspace-hack in favor of workspace deps
Workspace deps were introduced in Rust 1.64. They don't cover all the
cases that Hakari did unfortunately, but they are simpler to maintain,
and they avoid a couple of issues that Hakari had:

- It sometimes made updating dependencies harder due to the locked versions,
so you had to disable Hakari, do the updates, and then re-generate (
e.g. b396313fe5)
- The current Hakari config was breaking AnkiDroid's build, as it was
stopping a cross-compile from functioning correctly.
2023-06-23 17:41:31 +10:00
Damien Elmes
f1a3d06522 Make Command::run() accept a single string for convenience
We can fall back on the standard constructor when we have dynamic input
2023-06-23 11:31:39 +10:00
Damien Elmes
2feca80c20 Add copy_file to anki_io, and ensure strings parent folder is created 2023-06-22 19:33:34 +10:00
Damien Elmes
ef454b8859 Fix ts interface being output to wrong filenames 2023-06-22 09:58:12 +10:00
Damien Elmes
4dd8fa7577 Fix lack of progress/error when importing a .colpkg
We should be accessing the backend progress state, instead of the
collection's state.
2023-06-22 09:46:09 +10:00
Damien Elmes
f6d1fddf85 More service generation refactoring
- Dropped the protobuf extensions in favor of explicitly listing out
methods in both services if we want to implement both, as it's clearer.
- Move Service/Method wrappers into a separate crate that the various
clients can import, to easily get at the list of backend services and
their correct indices and comments.
2023-06-22 09:46:09 +10:00
Damien Elmes
a1b3d4f07e Drop the generated method map
I'd been thinking it might be useful for a future API service, but
I think that's better implemented with more codegen, so we have a
statically-typed interface.
2023-06-22 09:46:09 +10:00
Damien Elmes
258cf95924 Fix total note count not being logged in apkg import
https://forums.ankiweb.net/t/2-1-65-conflicting-import-message-when-importing-decks-with-v1-scheduler/31352/3
2023-06-22 09:46:09 +10:00
Damien Elmes
2b2c3036f7 Make some more fields/methods public
Continuation of https://github.com/ankitects/anki/issues/2494
2023-06-20 21:59:49 +10:00
Damien Elmes
cb14b057ae Report actual error when DbError occurs 2023-06-20 21:58:27 +10:00
Damien Elmes
ff73376fb5 Use a separate service definition for backend-only services
Realised this is clearer than tagging each method individually. The
enum has been retained for the case where we want to implement the backend
method separately from the collection one.
2023-06-19 22:06:50 +10:00
Damien Elmes
3e093cc849 Handle case where cards are imported into a collection with a newer start
https://forums.ankiweb.net/t/bug-report-when-the-original-due-is-less-than-0-the-due-will-be-weird-when-empty-the-filter-deck/31279

The 0 day case isn't fixed by this, but I don't think we can fix that
with the current schema.
2023-06-19 18:39:00 +10:00
Damien Elmes
d2089bc00e Refactor service generation (#2552)
* Automatically elide empty inputs and outputs to backend methods

* Refactor service generation

Despite the fact that the majority of our Protobuf service methods require
an open collection, they were not accessible with just a Collection
object. To access the methods (e.g. because we haven't gotten around to
exposing the correct API in Collection yet), you had to wrap the collection
in a Backend object, and pay a mutex-acquisition cost for each call, even
if you have exclusive access to the object.

This commit migrates the majority of service methods to the Collection, so
they can now be used directly, and improves the ergonomics a bit at the
same time.

The approach taken:

- The service generation now happens in rslib instead of anki_proto, which
avoids the need for trait constraints and associated types.
- Service methods are assumed to be collection-based by default. Instead of
implementing the service on Backend, we now implement it on Collection, which
means our methods no longer need to use self.with_col(...).
- We automatically generate methods in Backend which use self.with_col() to
delegate to the Collection method.
- For methods that are only appropriate for the backend, we add a flag in
the .proto file. The codegen uses this flag to write the method into a
BackendFooService instead of FooService, which the backend implements.
- The flag can also allows us to define separate implementations for collection
and backend, so we can e.g. skip the collection mutex in the i18n service
while also providing the service on a collection.
2023-06-19 15:33:40 +10:00
Damien Elmes
45af4e3d9e Refactor progress handling (#2549)
Previously it was Backend's responsibility to store the last progress,
and when calling routines in Collection, one had to construct and pass
in a Fn, which wasn't the most ergonomic. This PR adds the last progress
state to the collection, so that the routines no longer need a separate
progress arg, and makes some other tweaks to improve ergonomics.

ThrottlingProgressHandler has been tweaked so that it now stores the
current state, so that callers don't need to store it separately. When
a long-running routine starts, it calls col.new_progress_handler(),
which automatically initializes the data to defaults, and updates the
shared UI state, so we no longer need to manually update the state at
the start of an operation.

The backend shares the Arc<Mutex<>> with the collection, so it can get
at the current state, and so we can update the state when importing a
backup.

Other tweaks:

- The current Incrementor was awkward to use in the media check, which
uses a single incrementing value across multiple method calls, so I've
added a simpler alternative for such cases. The old incrementor method
has been kept, but implemented directly on ThrottlingProgressHandler.
- The full sync code was passing the progress handler in a complicated
way that may once have been required, but no longer is.
- On the Qt side, timers are now stopped before deletion, or they keep
running for a few seconds.
- I left the ChangeTracker using a closure, as it's used for both importing
and syncing.
2023-06-19 13:48:32 +10:00
Damien Elmes
e0841de73a Include cmdline in error display; show cargo install progress 2023-06-17 14:55:55 +10:00
Damien Elmes
63fcafb8c7 Check for stale licenses.json in minilints
+ Add an anki_process library with some helpers for command running.
2023-06-17 14:01:27 +10:00
Damien Elmes
cd52b10fa2 Fix rsbridge build when 1.61 snafu feature enabled 2023-06-17 12:44:25 +10:00
Damien Elmes
c411b32f9c Drop num_integer crate 2023-06-17 00:09:41 +10:00
Damien Elmes
05c06148a0 Bump Chrono now that Rumo's TZ parser has been merged
https://github.com/chronotope/chrono/pull/978

+ Update workspace hack/licenses
2023-06-16 11:43:12 +10:00
Damien Elmes
59fcbd9e88 Include backend comments in Python and Rust codegen
This is of limited usefulness at the moment, as it doesn't help consumers
of the public API.

Also removed detached comments from the included comments.
2023-06-16 10:59:00 +10:00
Damien Elmes
a1f2f75f24 Don't require DESCRIPTORS_BIN in an external workspace
Consumers may not require it
2023-06-15 22:32:16 +10:00
Damien Elmes
2db9de4f23 Don't fail silently when an empty csv is imported 2023-06-15 17:47:46 +10:00
Damien Elmes
f76778608f Skip writing descriptors if unchanged 2023-06-15 17:17:56 +10:00
Damien Elmes
bbd70603a7 Add support for using n2 instead of ninja
Provides better visibility into what the build is currently doing.
Motivated by slow node.js downloads making the build appear stuck.

You can test this out by running ./tools/install-n2 then building
normally. Please report any problems, and 'cargo uninstall n2' to get
back to the old behaviour. It works on Windows, but prints a new line
each second instead of redrawing the same area.

A couple of changes were required for compatibility:

- n2 doesn't resolve $variable names inside other variables, so the
resolution needs to be done by our build generator.
- Our inputs and outputs in build.ninja need to be listed in a deterministic
order to avoid unwanted rebuilds. I've made a few other tweaks so the
build file should now be fully-deterministic.
2023-06-15 17:17:56 +10:00
Damien Elmes
ce73b960d4 Migrate to protobuf-es (#2547)
* Fix .no-reduce-motion missing from graphs spinner, and not being honored

* Begin migration from protobuf.js -> protobuf-es

Motivation:

- Protobuf-es has a nicer API: messages are represented as classes, and
fields which should exist are not marked as nullable.
- As it uses modules, only the proto messages we actually use get included
in our bundle output. Protobuf.js put everything in a namespace, which
prevented tree-shaking, and made it awkward to access inner messages.
- ./run after touching a proto file drops from about 8s to 6s on my machine. The tradeoff
is slower decoding/encoding (#2043), but that was mainly a concern for the
graphs page, and was unblocked by
778e02415b

Approach/notes:

- We generate the new protobuf-es interface in addition to existing
protobuf.js interface, so we can migrate a module at a time, starting
with the graphs module.
- rslib:proto now generates RPC methods for TS in addition to the Python
interface. The input-arg-unrolling behaviour of the Python generation is
not required here, as we declare the input arg as a PlainMessage<T>, which
marks it as requiring all fields to be provided.
- i64 is represented as bigint in protobuf-es. We were using a patch to
protobuf.js to get it to output Javascript numbers instead of long.js
types, but now that our supported browser versions support bigint, it's
probably worth biting the bullet and migrating to bigint use. Our IDs
fit comfortably within MAX_SAFE_INTEGER, but that may not hold for future
fields we add.
- Oneofs are handled differently in protobuf-es, and are going to need
some refactoring.

Other notable changes:

- Added a --mkdir arg to our build runner, so we can create a dir easily
during the build on Windows.
- Simplified the preference handling code, by wrapping the preferences
in an outer store, instead of a separate store for each individual
preference. This means a change to one preference will trigger a redraw
of all components that depend on the preference store, but the redrawing
is cheap after moving the data processing to Rust, and it makes the code
easier to follow.
- Drop async(Reactive).ts in favour of more explicit handling with await
blocks/updating.
- Renamed add_inputs_to_group() -> add_dependency(), and fixed it not adding
dependencies to parent groups. Renamed add() -> add_action() for clarity.

* Remove a couple of unused proto imports

* Migrate card info

* Migrate congrats, image occlusion, and tag editor

+ Fix imports for multi-word proto files.

* Migrate change-notetype

* Migrate deck options

* Bump target to es2020; simplify ts lib list

Have used caniuse.com to confirm Chromium 77, iOS 14.5 and the Chrome
on Android support the full es2017-es2020 features.

* Migrate import-csv

* Migrate i18n and fix missing output types in .js

* Migrate custom scheduling, and remove protobuf.js

To mostly maintain our old API contract, we make use of protobuf-es's
ability to convert to JSON, which follows the same format as protobuf.js
did. It doesn't cover all case: users who were previously changing the
variant of a type will need to update their code, as assigning to a new
variant no longer automatically removes the old one, which will cause an
error when we try to convert back from JSON. But I suspect the large majority
of users are adjusting the current variant rather than creating a new one,
and this saves us having to write proxy wrappers, so it seems like a
reasonable compromise.

One other change I made at the same time was to rename value->kind for
the oneofs in our custom study protos, as 'value' was easily confused
with the 'case/value' output that protobuf-es has.

With protobuf.js codegen removed, touching a proto file and invoking
./run drops from about 8s to 6s.

This closes #2043.

* Allow tree-shaking on protobuf types

* Display backend error messages in our ts alert()

* Make sourcemap generation opt-in for ts-run

Considerably slows down build, and not used most of the time.
2023-06-14 22:47:37 +10:00
Damien Elmes
387bb89333 Fix build failure in fresh checkout 2023-06-12 17:10:22 +10:00
Damien Elmes
50107d25f3 Fix build failure on Windows 2023-06-12 16:06:40 +10:00
Damien Elmes
dbe51f3802 Split io.rs into separate crate, and use it in proto build
Will be handy to use it in our other scripts in the future too - thanks
Rumo!

Results of benchmarking ./run before and after these crate splits:

- Touching a proto file leads to a slight increase: about +90ms
- Touching an rslib file leads to a bigger decrease, as there's less to
recompile: about -700ms

And ./ninja test is even better: about +200ms and -3800ms.
2023-06-12 15:47:51 +10:00
Damien Elmes
632c95c177 Move generated protobuf into anki_proto
Due to the orphan rule, this meant removing our usages of impl ProtoStruct,
or converting them to a trait when they were used commonly.

rslib now directly references anki_proto and anki_i18n, instead of
'pub use'-ing them, and we can put the generated files back in OUT_DIR.
2023-06-12 15:47:51 +10:00
RumovZ
43b11a6e18 Automate schema 11 other duplicates clearing (#2542)
* Skip linting target folder

Contains build files not passing the copyright header check.

* Implicitly clear duplicate keys when serializing

Fixes `originalStockKind` not being cleared from `other`, as it had
mistakenly been added to the field list for `NoteFieldSchema11`.
2023-06-12 11:14:14 +10:00
Damien Elmes
a4b6cc7b59 Move protobuf generation into a separate crate; write .py interface in Rust
A couple of motivations for this:

- genbackend.py was somewhat messy, and difficult to change with the
lack of types. The mobile clients used it as a base for their generation,
so improving it will make life easier for them too, once they're ported.
- It will make it easier to write a .ts generator in the future
- We currently implement a bunch of helper methods on protobuf types
which don't allow us to compile the protobuf types until we compile
the Anki crate. If we change this in the future, we will be able to
do more of the compilation up-front.

We no longer need to record the services in the proto file, as we can
extract the service order from the compiled protos. Support for map types
has also been added.
2023-06-12 09:52:00 +10:00
Damien Elmes
7317397917 When cards are intraday learning queue, don't use original due
https://forums.ankiweb.net/t/backlog-with-no-backlo/30832
2023-06-07 16:20:44 +10:00
Damien Elmes
86479ad924 Default to current deck in csv import if settings allow it (#2527)
* Default to current deck in csv import if settings allow it

Reuses defaults_for_adding(). In the future we might also want to update
the last deck/notetype on successful completion, if entries weren't
specified in the file.

https://forums.ankiweb.net/t/importing-new-notes-to-wrong-deck-in-anki-2-1-63/30598

* Address review feedback from Rumo
2023-05-31 13:47:12 +10:00
Damien Elmes
d793fc0ace Various changes to I/O handling (#2513)
* Store coordinates as ratios of full size

* Use single definition for cappedCanvasSize()

* Move I/O review code into ts/image-occlusion

A bit simpler when it's all in one place.

* Reduce number precision, and round to whole pixels

>>> n=10000
>>> for i in range(1, int(n)): assert i == round(float("%0.4f" % (i/n))*n)

* Minor typing tweak

So, it turns out that typing is mostly broken in ts/image-occlusion.
We're importing from fabric which is a js file without types, so types
like fabric.Canvas are resolving to any.

I first tried switching to `@types/fabric`, which introduced a slew of
typing errors. Wasted a few hours trying to address them, before deciding
to give up on it, since the types were not complete. Then found fabric
has a 6.0 beta that introduces typing, and spent some time with that, but
ran into some new issues as it still seems to be a work in progress.
I think we're probably best off waiting until it's out and stabilized
before sinking more effort into this.

* Refactor (de)serialization of occlusions

To make the code easier to follow/maintain, cloze deletions are now decoded/
encoded into simple data classes, which can then be converted to Fabric objects
and back. The data objects handle converting from absolute/normal positions, and
producing values suitable for writing to text (eg truncated floats).

Various other changes:

- Polygon points are now stored as 'x,y x2,y2 ...' instead of JSON in cloze
divs, as that makes the handling consistent with reading from cloze deletion
text.
- Fixed the reviewer not showing updated placement when a polygon was moved.
- Disabled rotation controls in the editor, since we don't support rotation during
review.
- Renamed hideInactive to occludeInactive, as it wasn't clear whether the former
meant to hide the occlusions, or keep them (hiding the content). It's stored
as 'oi=1' in the cloze text.

* Increase canvas size limit, and double pixels when required.

* Size canvas based on container size

This results in sharper masks when the intrinsic image size is smaller
than the container, and more legible ones when the container is smaller than
the intrinsic image size.

By using the container instead of the viewport, we account for margins,
and when the pixel ratio is 1x, the canvas size and container size should
match.

* Disable zoom animation on editor load

* Default to rectangle when adding new occlusions

* Allow users to add/update notes directly from mask editing page

* The mask editor needs to work with css pixels, not actual pixels

The canvas and image were being scaled too large, which impacted
performance.
2023-05-31 13:45:12 +10:00
Damien Elmes
f5e66a042a percent-encoding -> percent-encoding-iri
Closes #2526
2023-05-29 14:46:02 +10:00
Damien Elmes
db4936d87e Detect incorrect usage of triple slash in TypeScript (#2524)
* Migrate check_copyright to Rust

* Add a new lint to check accidental usages of /// in ts/svelte comments

* Fix a bunch of incorrect jdoc comments

* Move contributor check into minilints

Will allow users to detect the issue locally with './ninja check'
before pushing to CI.

* Make Cargo.toml consistent with other crates
2023-05-26 12:49:44 +10:00