Commit graph

53 commits

Author SHA1 Message Date
RumovZ
3e0c9dc866
New TTS/AV tag handling (#1559)
* Add new `card_rendering` mod

Parses a text with av/tts tags and strips or extracts tags.

* Replace old `extract_av_tags` and `strip_av_tags`

... with new `card_rendering` mod

* ressource -> resource

* Add AV prettifier for use in browser table

* Accept String in av tag routines

... and avoid redundant writes if no changes need to be made.

* add benchmarking with criterion; make links test optional (dae)

cargo install cargo-criterion, then run ./bench.sh

* performance comparison: creating HashMap up front (dae)

the previous solution:

anki_tag_parse          time:   [1.8401 us 1.8437 us 1.8476 us]

this solution:

anki_tag_parse          time:   [2.2420 us 2.2447 us 2.2477 us]
                        change: [+21.477% +21.770% +22.066%] (p = 0.00 < 0.05)
                        Performance has regressed.

* Revert "performance comparison: creating HashMap up front" (dae)

This reverts commit f19126a2f1.

* add missing header

* Write error message if tts lang is missing

* `Tag` -> `Directive`
2021-12-17 19:04:42 +10:00
RumovZ
3ab9712c18
Stop trimming filename references before encoding (#1462)
Closes  #1430
2021-10-28 19:22:51 +10:00
Damien Elmes
53fe7e574e handle ampersand entities in image filenames
In the old HTML editor, filenames were % escaped before feeding them to
beautifulsoup, causing bare ampersands to be left alone. The new HTML
editor reads content from the DOM, where a bare ampersand has been
transformed into an &amp;, and that gets saved back into the field,
so the media check now needs to deal with it for images as well.

https://forums.ankiweb.net/t/causing-problems-with-image-names/12171
2021-08-19 23:43:40 +10:00
Damien Elmes
2f434dd74d fix comment + copy/paste error 2021-07-17 09:02:14 +10:00
Damien Elmes
bf507cca98 move from Python's URI escaping to IRI escaping in Rust
Should make non-Latin text readable in the HTML editor, without the
breakages reverted in the previous change.
2021-07-16 10:38:00 +10:00
Damien Elmes
b392020798 fix clippy lints for latest Rust 2021-06-21 13:09:36 +10:00
Henrik Giesel
0f658de702 Add escape_anki_wildcards_for_search_node 2021-06-16 09:25:27 +02:00
Henrik Giesel
c5faf39d7c Make Browser root nodes use "_*" uniformly 2021-06-16 17:19:21 +10:00
Damien Elmes
220fca9a1d handle <br/> when rendering a single line
+ case-insensitive matching

https://forums.ankiweb.net/t/html-editor-modifies-note-when-a-field-with-break-tags-is-opened/10772
2021-06-14 13:05:48 +10:00
Damien Elmes
64ebc32b3d tidy up Rust imports
rustfmt can do this automatically, but only when run with a nightly
toolchain, so it needs to be manually done for now - see rslib/rusfmt.toml
2021-04-18 18:38:54 +10:00
Damien Elmes
dc81a7fed0 use mixed case for abbreviations in Rust code
So, this is fun. Apparently "DeckId" is considered preferable to the
"DeckID" were were using until now, and the latest clippy will start
warning about it. We could of course disable the warning, but probably
better to bite the bullet and switch to the naming that's generally
considered best.
2021-03-27 19:53:33 +10:00
Shaun Ren
1f3751d191 Fix extraneous whitespaces from strip_html_for_tts 2021-03-25 11:44:42 -04:00
RumovZ
9151bfb53e Return input if decode_entities() encounters error 2021-03-22 12:08:22 +01:00
RumovZ
e931a429b3 Add html_to_text_line() on backend 2021-03-20 12:00:45 +01:00
Damien Elmes
ab790c1d14 initial work on moving v2 card answering into backend
Not plugged into the Python code yet. Still a work in progress.

Other changes:

- move a bunch of From implementations out of the giant backend/mod.rs
file into separate submodules.
- reorder backend methods to match proto order
- fix some clippy lints
2021-02-20 14:48:07 +10:00
Damien Elmes
242b4ea505 switch search parser to using owned values
I was a bit too enthusiastic with using borrowed values in structs
earlier on in the Rust porting. In this case any performance gains are
dwarfed by the cost of querying the DB, and using owned values here
simplifies the code, and will make it easier to parse a fragment in
the From<SearchTerm> impl.
2021-02-11 12:19:36 +10:00
Damien Elmes
ded626f0b9 render deck description with markdown; strip images
To support images on that screen, we'll first need to adjust the base url
for each platform, or rewrite the local image URLs, as otherwise they
are resolved to _anki/pages/...
2021-02-06 15:02:40 +10:00
abdo
72e8f9d640 Merge branch 'master' of https://github.com/ankitects/anki into tagtree 2021-01-12 23:31:58 +03:00
abdo
b276ce3dd5 Hierarchical tags 2021-01-09 17:10:13 +03:00
RumovZ
9ef691c06f Provide filter searches through backend 2021-01-09 10:50:08 +01:00
abdo
197d665de8 Fix duplicate check not decoding entities
This is a regression introduced in 358d0f957e

See https://forums.ankiweb.net/t/bug-duplicates-not-detecting-on-paste/5753
2020-12-14 15:13:00 +03:00
Damien Elmes
3923f56cbf fix clippy lints 2020-11-24 20:13:05 +10:00
RumovZ
347c547e10 Add tests for conversion functions in text.rs 2020-11-20 09:45:53 +01:00
RumovZ
0fc84d19b2 Replace text.rs/text_to_re with text.rs/to_re 2020-11-20 09:23:25 +01:00
RumovZ
785540bddc Revert changes to normalisation handling
Handle norm calls individually in write_search_node_to_sql again.
2020-11-18 23:46:27 +01:00
RumovZ
c185fb966b Merge branch 'master' into rework-search-parser
Conflicts:
    rslib/src/search/sqlwriter.rs
2020-11-18 09:04:04 +01:00
RumovZ
91873d68eb Fix RE in to_custom_re of text.rs
Match every single (potentially escaped) character of the string, so
they can be escaped properly.
2020-11-17 15:39:54 +01:00
RumovZ
8c02c6e205 Split unescaping between parser and writer
* Unescape wildcards in writer instead of parser.
* Move text conversion functions to text.rs.
* Implicitly norm when converting text.
* Revert to using collection when comparing tags but add escape support.
2020-11-17 12:49:37 +01:00
RumovZ
0cff65e5a8 Fix bugs and inconsistencies in the search parser 2020-11-12 17:27:50 +01:00
Andreas Reis
e68a40f13e cleanup / renames
・ soundRegexps →  sound_regexps

・ htmlRegexps →  html_media_regexps

・ HTML_TAGS →  HTML_MEDIA_TAGS

・ escapeImages →  escape_media_filenames + alias

・ strip_html_preserving_image_filenames →  strip_html_preserving_media_filenames
2020-11-10 14:53:04 +01:00
Andreas Reis
6e9aaad11e Add audio & object tags to media check
Makes the media check recognize files in <audio> and <object> tags as used.

They've been observed/supported by the WebView (checked: Anki, AnkiDroid) since just about forever already and are extremely useful if one knows a thing about web dev.
2020-10-25 13:09:57 +01:00
Damien Elmes
c82a084edf handle quoted html chars in media check
https://forums.ankiweb.net/t/unable-to-play-longer-audio-on-cards/1313/30
2020-09-04 09:36:38 +10:00
Damien Elmes
9fcd6c66f4 fix nonbreaking spaces breaking media
https://forums.ankiweb.net/t/unable-to-play-longer-audio-on-cards/1313
2020-08-30 11:23:12 +10:00
Damien Elmes
9e53c84a35 fix globs not working in bulk tag add/remove 2020-08-17 18:14:00 +10:00
Damien Elmes
805a3a710e split note types into separate tables
- store the config in protobuf instead of json
- still loading+saving in bulk for now
- code using the schema11 structs needs to be migrated
2020-05-12 21:13:33 +10:00
Damien Elmes
66809dd8a3 ignore empty sound tags
https://github.com/ankitects/anki/pull/612
2020-05-12 20:53:50 +10:00
Damien Elmes
c95983ac1f preserve entities when stripping HTML for MathJax
https://anki.tenderapp.com/discussions/ankidesktop/40987-how-to-render-angled-brackets
2020-04-30 11:17:38 +10:00
Damien Elmes
51a379de23 add search that ignores combining chars
On a test of a ~40k card collection, the 'ignore accents' add-on
takes about 1150ms, and this code takes about 70ms.
2020-03-21 15:15:59 +10:00
Damien Elmes
08e64d246d don't require wildcard for unicode case folding in search 2020-03-21 12:44:56 +10:00
Damien Elmes
9f3cc0982d deck searching
A bit more complicated than it needs to be, as we don't have the
full deck manager infrastructure yet.
2020-03-20 21:15:23 +10:00
Damien Elmes
3a1fc74ec3 remove some unused imports 2020-02-29 15:21:11 +10:00
Damien Elmes
ee27711b65 remove redundant test_ prefix 2020-02-17 08:40:17 +10:00
Damien Elmes
c890ef871e include LaTeX png/svg files when checking for unused media 2020-02-17 08:40:17 +10:00
Damien Elmes
fabfcb0338 gather field references in Rust; media check now mostly complete 2020-02-17 08:40:17 +10:00
Damien Elmes
f7c26724f3 nfc helper 2020-02-17 08:40:17 +10:00
Damien Elmes
c075191697 reuse reveal_cloze_text() for LaTeX cloze expansion 2020-01-28 07:40:44 +10:00
Damien Elmes
9ad80f4d2c move cloze-related code into a separate file 2020-01-27 20:41:23 +10:00
Damien Elmes
21cbb5a766 support speed control in tts tags 2020-01-26 14:31:07 +10:00
Damien Elmes
17ebdfc099 extract and flag AV tags as part of the render process
We can now show replay buttons for the audio contained in {{FrontSide}}
without having to play it again when the answer is shown.

The template code now always defers FrontSide rendering, as it wasn't
a big saving, and meant the logic had to be implemented twice.
2020-01-24 11:06:11 +10:00
Damien Elmes
4fb227ca4c added flag_av_tags() for replay button support 2020-01-21 14:44:27 +10:00