Commit graph

1679 commits

Author SHA1 Message Date
Abdo
5096626822 Add option to exclude fields from search (#2394)
* Add option to exclude fields from unqualified searches

* Use temp tables instead

This is slightly faster according to my (very rough) tests.

* Make query a bit more readable

* exclude_from_search -> excludeFromSearch

* Remove superfluous notetypes table from query

* Rework to use field search logic

Thanks to Rumo for the suggestion: https://github.com/ankitects/anki/pull/2394#issuecomment-1446702402

* Exclude fields from field searches too

* Fix error on notetypes with no included fields

* Add back the exclude_fields function

This approach seems to perform better on average than the previously
benchmarked ones.

* Use pure-SQL approach to excluding fields

* Change single field search to use new approach

* Fix flawed any_excluded/sortf_excluded logic

* Support field exclusion in the nc operator

Also fix search text being wrapped in % in the any_excluded=true case.

* Support field exclusion in the re and w operators

* Label field exclusion as being slower

* Unqualified search should be wrapped in % in all cases

I was under the impression that it shouldn't be wrapped with the new
field exclusion logic.

* Remove unnecessary .collect()

* Refactor some complex return types into structs

* Do not exclude fields in field searches

* Add a test and docstring for CollectRanges

* Avoid destructuring in closures

* Remove the exclude_fields function

Minor wording tweaks by dae:
* num_fields -> total_fields_in_note
* fields -> field_ranges_to_search
* fields -> fields_to_search
* SingleField -> FieldQualified
* mid -> ntid
2023-03-20 07:46:03 +10:00
RumovZ
80d9e8bc66 Fix invalid ids on import (#2449) 2023-03-20 07:04:59 +10:00
RumovZ
8c86e742be Fix invalid ids on db check (#2445)
* Move open_test_collection into Collection test impl

* Fix invalid ids when checking database

* Report fixed invalid ids

* Improve message when trying to export invalid ids

Also move ImportError due to namespace conflicts with snafu macro.

* Take a human name in DeckAdder::new

* Mention timestamps in the db check message (dae)

Will help to correlate the fix with the message shown when importing/
exporting.
2023-03-19 10:58:35 +10:00
RumovZ
331ef50670 Fix new limit being decremented unduly (#2447) 2023-03-18 11:48:50 +10:00
RumovZ
7b916f1c7d Ensure state mutator runs after card is rendered (#2421)
* Ensure state mutator runs after card is rendered

* Ensure ease buttons only show when states are ready

* Pass context into states mutator

* Revert queuing of state mutator hook

Now that context data is exposed users shouldn't rely on the question
having been rendered anymore.

* Use callbacks instead of signals and timeout

... to track whether the states mutator ran or failed.

* Make mutator async

* Remove State enum

* Reduce requests and compute seed on backend
2023-03-16 16:31:00 +10:00
RumovZ
189b122f55 Check URLs in TS code (#2436)
* Move help page URLs in ts to new file

* Unnest linkchecker test module

* Check TS help pages

* Add a comment (dae)
2023-03-15 15:46:03 +10:00
RumovZ
ae843943bd Revert bury restriction (#2432)
* Remove outdated comment.

* Revert removal of independent bury rules

* Revert 'hierarchical bury modes'

It's now again allowed to bury new, but not review cards e.g., but
siblings of previously gathered card queues will not be buried.

* Tweak docs (dae)

* Add missing Learn and PreviewRepeat queues
2023-03-11 17:49:18 +10:00
RumovZ
6af418775e Fix backup creation for collections > 1 GiB (#2423)
Co-authored-by: Damien Elmes <dae@users.noreply.github.com>
2023-03-06 19:56:27 +10:00
RumovZ
e7890c7033 Export static references from sound tags (#2420) 2023-03-06 19:29:19 +10:00
RumovZ
f6fa73f4b3 Global new ignore review limit (#2417)
* Add CardAdder test helper

* Add option to have new cards ignore the review limit

Also entails a lot of refactoring because the old code was deeply
coupled to the previous behaviour.

* Add global option to ignore review limit

* Refactor decrementation

* Unify testing
2023-03-06 19:06:12 +10:00
Daniel Tang
1e6686fcd8 Fix panic with invalid sync server URL with port (#2412) 2023-03-03 20:28:11 +10:00
Damien Elmes
dba1fd49b8 Ensure good is at least previous+1 even if low hard factor set
https://forums.ankiweb.net/t/anki-scheduler-broken/27987/8
2023-03-03 15:34:08 +10:00
Damien Elmes
bc684ecaf4 Make some methods/fields public 2023-03-01 19:46:06 +10:00
Damien Elmes
8c340ed1ea Add custom display for SyncError 2023-03-01 19:46:06 +10:00
Damien Elmes
bae48285dc Implement HttpError directly so that sources can be extracted properly
When disabling the default snafu source handling, <dyn Error>.source()
doesn't work.
2023-03-01 19:46:06 +10:00
Damien Elmes
4759343f25 Refactor MediaIter
Removes the dependency on a local path, and allows the unicode checks
to be skipped if appropriate.
2023-03-01 18:57:48 +10:00
Damien Elmes
119dc0ea14 Allow .or_http_err() to work with anyhow 2023-03-01 18:54:01 +10:00
RumovZ
fe8995e1a6 Add option to tag notes with missing media (#2379)
* Keep track of notes with missing media files

* Add option to tag notes with missing media

* Update ftl/core/media-check.ftl (dae)
2023-02-20 18:48:09 +10:00
Damien Elmes
002fc7af38 Trigger rebuild on STRINGS_JSON change 2023-02-17 19:03:33 +10:00
Damien Elmes
02428fdb98 Fix card ease being reset on schema upgrade
9783f4c46a introduced a serious regression,
causing cards to be reset to the default ease when upgrading to the latest
schema version. This could also be triggered when exporting a colpkg with
legacy support.
2023-02-17 12:27:19 +10:00
RumovZ
0b57e92397 Implement TTS using windows crate (#2371)
* Implement TTS using windows crate

* Use API calls instead of SSML

* Properly stop player in case of TTS error

* Add context to WindowsErrors

* Validate available voices

* Remove TTS text from synthesize error

* Limit maximum buffer size

* Make validation optional and list it in tts filter

* We no longer need the winrt module (dae)

* Use a separate request object so the meaning of the bool is clear (dae)

* Slightly shorten runtime error message (dae)

The default message appears to clip slightly.

* Alternate buffer implementation (dae)

* Use array instead of vec

* Drop the max buffer size to 128k (dae)
2023-02-17 12:26:07 +10:00
RumovZ
42f9fdea8b Deck scoped dupe check (#2372)
* Support limiting dupe check to deck

* Expose deck limiting dupe check on frontend

* Make CSV dupe options configurable with headers

* Rename duplicate file headers

* Change dupe check limit to enum
2023-02-16 17:53:36 +10:00
Damien Elmes
804bccfbe7 Fix excess missed markers in non-Latin text 2023-02-10 15:11:36 +10:00
Damien Elmes
4e58281a70 Allow the network timeout to be customized
https://forums.ankiweb.net/t/local-sync-server-collection-exceeds-size-limit/27183/7
2023-02-08 14:33:02 +10:00
RumovZ
92742a1e74 Improve HttpError mapping (#2370)
* Use reqwest From impl in HttpError From impl

* Map timeouts to NetworkError (dae)
2023-02-08 12:54:42 +10:00
Damien Elmes
fa603db3f7 Show custom data in stats screen
https://forums.ankiweb.net/t/feature-request-display-custom-data-in-card-info/27187
2023-02-07 12:27:43 +10:00
Damien Elmes
3308ccd438 Don't enforce download size on client
https://forums.ankiweb.net/t/local-sync-server-collection-exceeds-size-limit/27183/6
2023-02-07 11:58:07 +10:00
RumovZ
6f1e55d1ef Add Rust bin to deprecate unused ftl entries (#2364)
* Add Rust bin to deprecate unused ftl entries

* Align function names with bin names

* Support passing in multiple ftl roots

* Use source instead of jsons for deprecating

* Fix CargoRun not working more than once (dae)

* Add ftl:deprecate (dae)

* Deprecate some strings (dae)

This is not all of the strings that are currently unused

* Check json files before deprecating; add allowlist (dae)

The scheduler messages we'll probably want to reuse for the v2->v3
transition, so I'd prefer to keep them undeprecated for now.

* Deprecate old bury options (dae)

* Support gathering usages from Kotlin files for AnkiDroid (dae)

* Update json scripts (dae)

* Remove old deprecation headers

* Parameterize JSON roots to keep

* Tweak deprecation message (dae)
2023-02-07 11:56:14 +10:00
RumovZ
682bf8ce46 Disable burying of previously gathered cards (#2361)
* Enforce hierarchical bury modes

Interday learning burying is only allowed if review burying is enabled
and review burying is only allowed if new burying is enabled.
Closes #2352.

* Switch front end to new bury modes

* Wording tweaks (dae)

* Hide interday option if using v2 scheduler (dae)
2023-02-06 12:02:27 +10:00
Damien Elmes
3f304dccab Deck-specific limits should be ignored when the v2 scheduler is enabled
Closes #2365
2023-02-06 11:02:26 +10:00
RumovZ
c2dbf486c7 Support UTF-8-BOM when importing CSV (#2360)
The csv crate already supports it, but the meta line parsing didn't.
2023-02-05 11:53:21 +10:00
Damien Elmes
fa322988e8 Fix comparison being collapsed when extra characters in input 2023-02-02 19:13:12 +10:00
Damien Elmes
81ad253225 Revert to blanking out missed chars in provided text
Quite a few users seem surprised to have missed characters appear in
their input text
2023-02-02 19:13:11 +10:00
Damien Elmes
47707e8042 Wrap expected text in code block when no answer provided 2023-02-02 18:01:23 +10:00
Kaben Nanlohy
d1cc124446 Allow burying cards in browser (#2351)
* Allow burying cards in browser

This code is based on existing "toggle suspend" command in browser.

- Adds "toggle bury" command to browser cards menu.
- Adds "browsing-toggle-bury" to core translation. Only english-language.
- Adds "buried" coloring to rows for buried cards in browser table.

Not yet done:

- Keyboard shortcut for "toggle bury" action.
- Non-english translations.

* Add contributor as requested in CONTRIBUTORS.

* Fix formatting in browser_table.rs.

* Add keyboard shortcut to "toggle bury" command.

This adds keyboard shortcut "ctrl-shift-j" to "toggle bury" command in
browser cards menu.

* Simplify logic for color of buried-card rows.
2023-01-30 19:21:06 +10:00
Damien Elmes
9783f4c46a Validate and clamp deck config when loading
Fixes #2353
2023-01-28 21:51:47 +10:00
Damien Elmes
becffc213d Use HTTP/1.1 for syncing
HTTP2 + a request body size not being declared up front was creating extra
work for AnkiWeb.
2023-01-28 21:51:47 +10:00
Damien Elmes
d84ef8181e Fix clean build failure due to protoc change
191b27e88a changed the name of the env
var in .cargo/config.toml, causing the check in setup_protoc() to think
a custom path had been provided, which skipped the download and extract
step.
2023-01-26 09:33:39 +10:00
Damien Elmes
05d6ca987d Use Rust nightly for formatting (#2348)
* Support specifying a working dir to a build command

* Use nightly for formatting

* Pass valid TERM in from environment

Rustfmt depends on a valid setting, and not just the var to be non-empty.

* Wrap comment
2023-01-25 23:35:53 +10:00
Mani
191b27e88a Use a ninja variable for Protoc binary (#2345)
* Use a ninja variable for Protoc binary

* fix whitespace
2023-01-23 20:44:47 +10:00
Damien Elmes
f89de8316c Fix outer whitespace not being trimmed in type answer 2023-01-23 17:20:35 +10:00
Damien Elmes
7c8e8a50f5 Probable fix for future due graph
Cards due earlier today will have a negative offset like -78000(secs).
The old typescript code was using floating point division to yield -1;
with integer division we get 0 instead.

https://forums.ankiweb.net/t/wrong-info-when-hovering-over-future-due-graph/26522
2023-01-20 00:18:13 +10:00
Damien Elmes
a66d2472dd Use backend for extracting cloze text to type
Closes #2311
2023-01-18 23:05:28 +10:00
Damien Elmes
9d3ccb88c4 Add some extra info to the 'modified without updating' message
For #2315
2023-01-18 22:38:28 +10:00
Damien Elmes
b396313fe5 Update Rust deps (#2332)
* Temporarily disable hakari

* Upgrade compatible deps except Chrono

* Update semver-incompatible crates

* Re-enable hakari

* Update licenses & cargo-deny

* Fix new clippy lints

* Update to latest Rust
2023-01-18 22:24:29 +10:00
Damien Elmes
7216032e82 Switch Rust import style (#2330)
* Prepare to switch Rust import style

* Run nightly format

Closes #2320

* Clean up a few imports

* Enable comment wrapping

* Wrap comments
2023-01-18 21:39:55 +10:00
Damien Elmes
ba595c25ce Probable fix for flaky test 2023-01-18 21:32:08 +10:00
Damien Elmes
8a89bfb328 Replace newlines with linebreaks when importing without HTML
https://forums.ankiweb.net/t/line-breaks-are-removed-when-importing-csv-on-linux/26451
2023-01-18 21:00:33 +10:00
Damien Elmes
3707e54ffa Rework syncing code, and replace local sync server (#2329)
This PR replaces the existing Python-driven sync server with a new one in Rust.
The new server supports both collection and media syncing, and is compatible
with both the new protocol mentioned below, and older clients. A setting has
been added to the preferences screen to point Anki to a local server, and a
similar setting is likely to come to AnkiMobile soon.

Documentation is available here: <https://docs.ankiweb.net/sync-server.html>

In addition to the new server and refactoring, this PR also makes changes to the
sync protocol. The existing sync protocol places payloads and metadata inside a
multipart POST body, which causes a few headaches:

- Legacy clients build the request in a non-deterministic order, meaning the
entire request needs to be scanned to extract the metadata.
- Reqwest's multipart API directly writes the multipart body, without exposing
the resulting stream to us, making it harder to track the progress of the
transfer. We've been relying on a patched version of reqwest for timeouts,
which is a pain to keep up to date.

To address these issues, the metadata is now sent in a HTTP header, with the
data payload sent directly in the body. Instead of the slower gzip, we now
use zstd. The old timeout handling code has been replaced with a new implementation
that wraps the request and response body streams to track progress, allowing us
to drop the git dependencies for reqwest, hyper-timeout and tokio-io-timeout.

The main other change to the protocol is that one-way syncs no longer need to
downgrade the collection to schema 11 prior to sending.
2023-01-18 12:43:46 +10:00
RumovZ
5da523a9eb Replace dissimilar crate with difflib (#2322)
This also inserts the expected text if it's missing at the very
beginning of the provided text.
2023-01-16 09:49:34 +10:00