Commit graph

66 commits

Author SHA1 Message Date
abdo
4d3f2c0232 Strip isolation chars from filenames in Check Media screen 2021-09-27 22:15:37 +03:00
Damien Elmes
827f3e073a handle ampersand entities in image filenames
In the old HTML editor, filenames were % escaped before feeding them to
beautifulsoup, causing bare ampersands to be left alone. The new HTML
editor reads content from the DOM, where a bare ampersand has been
transformed into an &, and that gets saved back into the field,
so the media check now needs to deal with it for images as well.

https://forums.ankiweb.net/t/causing-problems-with-image-names/12171
2021-08-19 23:43:40 +10:00
Damien Elmes
1e2ebd1c29 move from Python's URI escaping to IRI escaping in Rust
Should make non-Latin text readable in the HTML editor, without the
breakages reverted in the previous change.
2021-07-16 10:38:00 +10:00
Damien Elmes
ff095ed57b fix clippy lints for latest Rust 2021-06-21 13:09:36 +10:00
Damien Elmes
994d26937f remove some unneeded transactions from tests 2021-04-30 19:18:02 +10:00
Damien Elmes
03ca227fd4 make it more ergonomic to search directly via nodes in Rust 2021-04-30 11:37:55 +10:00
Damien Elmes
363a843d07 tidy up Rust imports
rustfmt can do this automatically, but only when run with a nightly
toolchain, so it needs to be manually done for now - see rslib/rusfmt.toml
2021-04-18 18:38:54 +10:00
Damien Elmes
cfe02d5df4 switch DbError to tuple type 2021-04-01 17:34:03 +10:00
Damien Elmes
cc54a9251e crate::err -> crate::error 2021-04-01 16:07:13 +10:00
Damien Elmes
d793c30526 switch NoteType to Notetype
When used as a variable, we were typically calling it a 'notetype', not
a 'note type'.
2021-03-27 22:03:19 +10:00
Damien Elmes
a90d5aa359 use mixed case for abbreviations in Rust code
So, this is fun. Apparently "DeckId" is considered preferable to the
"DeckID" were were using until now, and the latest clippy will start
warning about it. We could of course disable the warning, but probably
better to bite the bullet and switch to the naming that's generally
considered best.
2021-03-27 19:53:33 +10:00
Damien Elmes
9a7f0f9f5d i18n->tr in rslib/ to match Python/TS code 2021-03-27 12:09:51 +10:00
Damien Elmes
27c35cf2c5 update multi-arg TR references, where some needed reordering 2021-03-27 10:49:40 +10:00
Damien Elmes
d7005803bc update 1 arg tr strings in rslib 2021-03-27 10:39:53 +10:00
Damien Elmes
51f5df600f update no-arg TR references in rslib/ 2021-03-26 23:16:08 +10:00
Damien Elmes
0c63a7c8c4 rework translation handling
Instead of generating a fluent.proto file with a giant enum, create
a .json file representing the translations that downstream consumers
can use for code generation.

This enables the generation of a separate method for each translation,
with a docstring that shows the actual text, and any required arguments
listed in the function signature.

The codebase is still using the old enum for now; updating it will need
to come in future commits, and the old enum will need to be kept
around, as add-ons are referencing it.

Other changes:

- move translation code into a separate crate
- store the translations on a per-file/module basis, which will allow
us to avoid sending 1000+ strings on each JS page load in the future
- drop the undocumented support for external .ftl files, that we weren't
using
- duplicate strings in translation files are now checked for at build
time
- fix i18n test failing when run outside Bazel
- drop slog dependency in i18n module
2021-03-26 09:41:32 +10:00
Damien Elmes
42e20461c0 undoable ops now return changes directly; add new *_ops.py files
- Introduced a new transact() method that wraps the return value
in a separate struct that describes the changes that were made.
- Changes are now gathered from the undo log, so we don't need to
guess at what was changed - eg if update_note() is called with identical
note contents, no changes are returned. Card changes will only be set
if cards were actually generated by the update_note() call, and tag
will only be set if a new tag was added.
- mw.perform_op() has been updated to expect the op to return the changes,
or a structure with the changes in it, and it will use them to fire the
change hook, instead of fetching the changes from undo_status(), so there
is no risk of race conditions.
- the various calls to mw.perform_op() have been split into separate
files like card_ops.py. Aside from making the code cleaner, this works
around a rather annoying issue with mypy. Because we run it with
no_strict_optional, mypy is happy to accept an operation that returns None,
despite the type signature saying it requires changes to be returned.
Turning no_strict_optional on for the whole codebase is not practical
at the moment, but we can enable it for individual files.

Still todo:
- The cursor keeps moving back to the start of a field when typing -
we need to ignore the refresh hook when we are the initiator.
- The busy cursor icon should probably be delayed a few hundreds ms.
- Still need to think about a nicer way of handling saveNow()
- op_made_changes(), op_affects_study_queue() might be better embedded
as properties in the object instead
2021-03-19 19:45:21 +10:00
Damien Elmes
cce0ee2b03 handle quoted html chars in media check
https://forums.ankiweb.net/t/unable-to-play-longer-audio-on-cards/1313/30
2020-09-04 09:36:38 +10:00
Damien Elmes
7f42500e53 rename some card+note fields in backend 2020-09-01 10:24:38 +10:00
Damien Elmes
41a47fcf01 fix some clippy lints 2020-08-26 19:02:56 +10:00
Damien Elmes
aaed0464e5 rework progress handling
- client now polls status instead of backend pushing it
- supports multiple threads
- update throttling happens in one place
2020-06-01 13:57:10 +10:00
Damien Elmes
0a87b1ef2c support disabling unicode normalization in notes 2020-05-12 21:13:34 +10:00
Damien Elmes
85528cabc5 use backend for genCards() and updateFieldCache() 2020-05-12 21:13:34 +10:00
Damien Elmes
3b07885911 handle changed sort field index 2020-05-12 21:13:33 +10:00
Damien Elmes
0c8763edf3 cache notetypes 2020-05-12 21:13:33 +10:00
Damien Elmes
6b8a45bcb6 more work towards note adding
Still a prototype at this stage - we'll likely want a caching layer
for note types, and I'm not sure of the merit of having fields in
a separate table, since they're almost always required.
2020-05-12 21:13:33 +10:00
Damien Elmes
638e69af91 move note db code to storage 2020-05-12 21:13:33 +10:00
Damien Elmes
9c41210cf4 use native struct for NoteType 2020-05-12 21:13:33 +10:00
Damien Elmes
96300f5858 migrate the schema11 usages to new structs/sql queries 2020-05-12 21:13:33 +10:00
Damien Elmes
475478cec5 split note types into separate tables
- store the config in protobuf instead of json
- still loading+saving in bulk for now
- code using the schema11 structs needs to be migrated
2020-05-12 21:13:33 +10:00
Damien Elmes
07b9671012 don't throw an error when a file is greater than MAXPATH
https://anki.tenderapp.com/discussions/ankidesktop/40931-check-media-error
2020-04-30 08:01:45 +10:00
Damien Elmes
93152043e1 report the missing filename in media check
https://anki.tenderapp.com/discussions/ankidesktop/40578-check-media-error
2020-04-17 17:17:42 +10:00
Damien Elmes
bca49bd054 load/save note types in backend
This allows us to normalize bad data, and is the first step towards
splitting note types into separate tables.
2020-04-08 10:05:07 +10:00
Damien Elmes
8f427dbbd7 FString -> TR 2020-04-01 18:16:43 +10:00
Damien Elmes
7b201fe7e6 ignore .DS_Store in trash
https://anki.tenderapp.com/discussions/ankidesktop/40035-anki-2123-on-macos-ds_store-files-in-new-anki-trash
2020-04-01 13:37:57 +10:00
Damien Elmes
0451a4d4e0 drop separate RequestContext/StorageContext 2020-03-29 12:12:35 +10:00
Damien Elmes
4150480709 NoteTypeID 2020-03-26 15:00:24 +10:00
Damien Elmes
4563fe36bc rename long filenames in fields if files renamed in a previous sync 2020-03-23 18:38:45 +10:00
Damien Elmes
b087ebc845 start on search tests 2020-03-20 21:15:23 +10:00
Damien Elmes
6c469a96eb finish the remaining searches
Searches that require multiple deck or note type lookups won't perform
very well at the moment - it either needs caching or to be split up
at the DB level.

Nothing tested yet.
2020-03-20 21:15:23 +10:00
Damien Elmes
c710580604 move media folder/db paths into collection
this breaks background media syncing for now
2020-03-20 21:15:23 +10:00
Damien Elmes
e72d21e09e move note code into notes.rs, add ability to rollback when unchanged 2020-03-20 21:15:23 +10:00
Damien Elmes
3f3a45e2ba add Collection struct, and get media check working again
- media check no longer needs collection to be closed
- use savepoints for operations initiated by Rust, so they are
atomic without forcing a commit
2020-03-20 21:15:23 +10:00
Damien Elmes
a129b8f8cf initial work on DB handling in Rust
committing the Protobuf implementation for posterity, but will replace
it with json, as Protobuf measures about 6x slower for some workloads
like 'select * from notes'
2020-03-20 21:15:23 +10:00
Damien Elmes
b198ba8073 ignore remote file links in media check 2020-03-10 13:46:52 +10:00
Damien Elmes
e512199ace add restore media action 2020-03-10 13:35:09 +10:00
Damien Elmes
a6b1a32378 don't clean up the trash folder automatically
The media check now reports if any files are in the trash folder,
and a method has been exposed to clear the trash.
2020-03-10 12:02:38 +10:00
Damien Elmes
c97b37b7ae make logging explicit, and support logging to a file 2020-02-29 20:44:01 +10:00
Damien Elmes
ea22296d4c expand clozes before rendering LaTeX
https://anki.tenderapp.com/discussions/ankidesktop/39071-latex-media-windows-version-2121-0cd1ee92-fail-rendering-latex
2020-02-24 18:34:08 +10:00
Damien Elmes
0d1a30c6d8 rework Fluent handling
- all .ftl files for a language are concatenated into a single file
at build time
- all languages are included in the binary
- external ftl files placed in the ftl folder can override the
built-in definitions
- constants are automatically generated for each string key
- dropped the separate StringsGroup enum
2020-02-23 13:22:50 +10:00