Commit graph

46 commits

Author SHA1 Message Date
RumovZ
a765ab95ec
Enable invalid-name globally in pylib (#1454) 2021-10-25 19:10:56 +10:00
RumovZ
9dc3cf216a
PEP8 for rest of pylib (#1451)
* PEP8 dbproxy.py

* PEP8 errors.py

* PEP8 httpclient.py

* PEP8 lang.py

* PEP8 latex.py

* Add decorator to deprectate key words

* Make replacement for deprecated attribute optional

* Use new helper `_print_replacement_warning()`

* PEP8 media.py

* PEP8 rsbackend.py

* PEP8 sound.py

* PEP8 stdmodels.py

* PEP8 storage.py

* PEP8 sync.py

* PEP8 tags.py

* PEP8 template.py

* PEP8 types.py

* Fix DeprecatedNamesMixinForModule

The class methods need to be overridden with instance methods, so every
module has its own dicts.

* Use `# pylint: disable=invalid-name` instead of id

* PEP8 utils.py

* Only decorate `__getattr__` with `@no_type_check`

* Fix mypy issue with snakecase

Importing it from `anki._vendor` raises attribute errors.

* Format

* Remove inheritance of DeprecatedNamesMixin

There's almost no shared code now and overriding classmethods with
instance methods raises mypy issues.

* Fix traceback frames of deprecation warnings

* remove fn/TimedLog (dae)

Neither Anki nor add-ons appear to have been using it

* fix some issues with stringcase use (dae)

- the wheel was depending on the PyPI version instead of our vendored
version
- _vendor:stringcase should not have been listed in the anki py_library.
We already include the sources in py_srcs, and need to refer to them
directly. By listing _vendor:stringcase as well, we were making a
top-level stringcase library available, which would have only worked for
distributing because the wheel definition was also incorrect.
- mypy errors are what caused me to mistakenly add the above - they
were because the type: ignore at the top of stringcase.py was causing
mypy to completely ignore the file, so it was not aware of any attributes
it contained.
2021-10-25 14:50:13 +10:00
RumovZ
d665dbc9a7
PEP8 pylib (#1443)
* PEP8 scheduler/base.py

* PEP8 _backend/__init__.py

* PEP8 _backend/genbackend.py

* PEP8 _backend/genfluent.py

* PEP8 scheduler/__init__.py

* PEP8 __init__.py

* PEP8 _legacy.py

* PEP8 syncserver/__init__.py

- Make 'ip' a good name
- Overrule `global col` being identified as a constant

* PEP8 syncserver/__main__.py

* PEP8 buildinfo.py

* Implement `DeprecatedNamesMixin` for modules

* PEP8 browser.py

* PEP8 config.py

* PEP8 consts.py

* PEP8 db.py

* Format

* Improve AttributeError for DeprecatedNamesMixin

* print the line that imported/referenced the legacy module attr (dae)

* DeprecatedNamesMixinStandalone -> ...ForModule
2021-10-22 20:39:49 +10:00
Damien Elmes
048a9a2b60 vendor stringcase
It's a tiny library that has not been updated in years, and it was
leading to a warning on startup:

 DeprecationWarning: invalid escape sequence \W
  return re.sub("\W+", "", string)
2021-10-22 08:56:44 +10:00
RumovZ
3672b0fe73
Switch CardInfoDialog to ts page (#1414)
* Only collect card stats on the backend ...

... instead of rendering an HTML string using askama.

* Add ts page Card Info

* Update test for new `col.card_stats()`

* Remove obsolete CardStats code

* Use new ts page in `CardInfoDialog`

* Align start and end instead of left and right

Curiously, `text-align: start` does not work for `th` tags if assigned
via classes.

* Adopt ts refactorings after rebase

#1405 and #1409

* Clean up `ts/card-info/BUILD.bazel`

* Port card info logic from Rust to TS

* Move repeated field to the top

https://github.com/ankitects/anki/pull/1414#discussion_r725402730

* Convert pseudo classes to interfaces

* CardInfoPage -> CardInfo

* Make revlog in card info optional

* Add legacy support for old card stats

* Check for undefined instead of falsy

* Make Revlog separate component

* drop askama dependency (dae)

* Fix nightmode for legacy card stats
2021-10-14 19:22:47 +10:00
Damien Elmes
b9251290ca run pyupgrade over codebase [python upgrade required]
This adds Python 3.9 and 3.10 typing syntax to files that import
attributions from __future___. Python 3.9 should be able to cope with
the 3.10 syntax, but Python 3.8 will no longer work.

On Windows/Mac, install the latest Python 3.9 version from python.org.
There are currently no orjson wheels for Python 3.10 on Windows/Mac,
which will break the build unless you have Rust installed separately.

On Linux, modern distros should have Python 3.9 available already. If
you're on an older distro, you'll need to build Python from source first.
2021-10-04 15:05:48 +10:00
Damien Elmes
57f4d6588f fix uncaught error when int over 64 bits passed into db_command() 2021-09-30 16:46:33 +10:00
Damien Elmes
4da1c77220 add basic tag completion to backend
Matches should arrive in alphabetical order. Currently results are not
capped (JS should be able to handle ~1k tags without too much hassle),
and no reordering based on match location is done. Matches are substring
based, and multiple can be provided, eg "foo::bar" will match
"foof::baz::abbar".

This is not hooked up properly on the frontend at the moment -
updateSuggestions() seems to be missing the most recently typed character,
and is not updating the list of completions half the time.
2021-09-09 15:38:08 +02:00
RumovZ
d8d69a3810 Switch frontend to use backend links 2021-07-22 10:07:13 +02:00
Damien Elmes
0e7411188b configs.proto plural workaround no longer necessary 2021-07-11 19:35:18 +10:00
Damien Elmes
7ac1fa17e6 move proto files into separate py_library in same namespace
Will allow importing the Protobuf without pulling in the rest of
the library. This is not a full PEP420 namespace, and the wheel still
bundles everything - it just makes things easier in a Bazel workspace.
I originally tried with PEP420, but it required more invasive changes,
and I ran into issues with mypy.
2021-07-11 14:51:25 +10:00
Damien Elmes
185e9acd22 split out remaining tags, stats, media and rendering 2021-07-10 23:16:18 +10:00
Damien Elmes
35b059ecdb split out sync, search, scheduler & config 2021-07-10 21:33:12 +10:00
Damien Elmes
9e0a295ab9 split out decks, deckconfig, notes, notetypes 2021-07-10 20:44:22 +10:00
Damien Elmes
18851ace47 split out cards and collection 2021-07-10 19:52:31 +10:00
Damien Elmes
616db33c0e refactor protobuf handling for split/import
In order to split backend.proto into a more manageable size, the protobuf
handling needed to be updated. This took more time than I would have
liked, as each language handles protobuf differently:

- The Python Protobuf code ignores "package" directives, and relies
solely on how the files are laid out on disk. While it would have been
nice to keep the generated files in a private subpackage, Protobuf gets
confused if the files are located in a location that does not match
their original .proto layout, so the old approach of storing them in
_backend/ will not work. They now clutter up pylib/anki instead. I'm
rather annoyed by that, but alternatives seem to be having to add an extra
level to the Protobuf path, making the other languages suffer, or trying
to hack around the issue by munging sys.modules.
- Protobufjs fails to expose packages if they don't start with a capital
letter, despite the fact that lowercase packages are the norm in most
languages :-( This required a patch to fix.
- Rust was the easiest, as Prost is relatively straightforward compared
to Google's tools.

The Protobuf files are now stored in /proto/anki, with a separate package
for each file. I've split backend.proto into a few files as a test, but
the majority of that work is still to come.

The Python Protobuf building is a bit of a hack at the moment, hard-coding
"proto" as the top level folder, but it seems to get the job done for now.

Also changed the workspace name, as there seems to be a number of Bazel
repos moving away from the more awkward reverse DNS naming style.
2021-07-10 19:17:05 +10:00
Damien Elmes
80b98e0db8 move protobuf into separate folder in preparation for multiple files 2021-07-09 21:02:40 +10:00
Damien Elmes
c79f8ba88f in/out -> request/response
The saved characters weren't worth the increased difficulty when
reading, and the fact that we were deviating from protobuf norms.
2021-06-20 15:49:20 +10:00
Damien Elmes
968bd1b27a specific encoding of strings.json
https://forums.ankiweb.net/t/win10-build-error-unicodedecodeerror-gbk-codec-cant-decode-byte-0x91/10714
2021-06-11 20:12:38 +10:00
Damien Elmes
61e86cc29d new change notetype implementation for the frontend
- changes can now be undone
- the same field can now be mapped to multiple target fields, allowing
fields to be cloned
- the old Qt dialog has been removed
- the old col.models.change() API calls the new code, to avoid
breaking existing consumers. It requires the field map to always
be passed in, but that appears to have been the common case.
- closes #1175
2021-06-10 22:19:24 +10:00
Damien Elmes
49a1580566 use new API for test scheduler
Avoids duplicate work, and is a step towards allowing the next
states to be modified by third-party code.

Also:

- fixed incorrect underlined count, due to reviews being labeled as
learning cards
- fixed reviewer not refreshing when undoing a test review, by splitting
up backend queue rebuilding from frontend reviewer refresh
- moved answering into a CollectionOp
2021-05-11 13:06:03 +10:00
Damien Elmes
55277aa90a implement deck config saving on JS end 2021-04-20 19:50:05 +10:00
Damien Elmes
c49d6ce49f run black/isort on Python scripts 2021-04-14 18:22:02 +10:00
Damien Elmes
2168dfe63d add routine to set deck collapse state
Updating a deck via protobuf is now exposed on the backend, but not
currently on the frontend - I suspect we'll be better off writing
separate routines for the actions we need instead, and we get a better
undo description for free.

This is currently causing an ugly redraw in the browse screen, which
will need fixing.
2021-04-05 11:19:04 +10:00
Damien Elmes
41c5a25dc8 simplify errors
- use a flat enum instead of oneof messages, most of which were empty
- tidy up the Python side
2021-04-03 16:06:46 +10:00
Damien Elmes
89d249b3b6 update to the latest rules_rust + security framework update 2021-03-27 19:28:19 +10:00
Damien Elmes
d6b9cc4d9b drop the legacy enum from rslib, and pass separate module/message idx 2021-03-27 11:56:31 +10:00
Damien Elmes
3d366d5264 add types to some more Fluent variables 2021-03-26 16:52:54 +10:00
Damien Elmes
8cc6758eb1 declare variables with some common names as int instead of a union 2021-03-26 16:33:53 +10:00
Damien Elmes
0c338bfd53 update no-arg tr references in qt/ 2021-03-26 13:48:26 +10:00
Damien Elmes
ebe655975c update some more TR references in pylib; update tr_legacyglobal 2021-03-26 13:33:46 +10:00
Damien Elmes
48354931da update some no-arg TR constants 2021-03-26 12:37:18 +10:00
Damien Elmes
9aece2a7b8 rework translation handling
Instead of generating a fluent.proto file with a giant enum, create
a .json file representing the translations that downstream consumers
can use for code generation.

This enables the generation of a separate method for each translation,
with a docstring that shows the actual text, and any required arguments
listed in the function signature.

The codebase is still using the old enum for now; updating it will need
to come in future commits, and the old enum will need to be kept
around, as add-ons are referencing it.

Other changes:

- move translation code into a separate crate
- store the translations on a per-file/module basis, which will allow
us to avoid sending 1000+ strings on each JS page load in the future
- drop the undocumented support for external .ftl files, that we weren't
using
- duplicate strings in translation files are now checked for at build
time
- fix i18n test failing when run outside Bazel
- drop slog dependency in i18n module
2021-03-26 09:41:32 +10:00
Damien Elmes
4bd120cc4b split out remaining rpc methods
@david-allison-1 note this also changes the method index to start at
0 instead of 1
2021-03-11 17:04:32 +10:00
Damien Elmes
cd14987812 split out tags, deck config and card rendering 2021-03-11 16:05:06 +10:00
Damien Elmes
5df684fa6b rework backend codegen to support multiple services; split out sched
Rust requires all methods of impl Trait to be in a single file, which
means we had a giant backend/mod.rs covering all exposed methods. By
using separate service definitions for the separate areas, and updating
the code generation, we can split it into more manageable chunks -
this commit starts with the scheduling code.

In the long run, we'll probably want to split up the protobuf file into
multiple files as well.

Also dropped want_release_gil() from rsbridge, and the associated method
enum. While it allows us to skip the thread save/restore and mutex unlock/
lock, it looks to only be buying about 2.5% extra performance in the
best case (tested with timeit+format_timespan), and the majority of
the backend methods deal with I/O, and thus were already releasing the
GIL.
2021-03-11 14:51:29 +10:00
Damien Elmes
bb29ce88f3 minor code cleanups with pyupgrade
- pyupgrade --py38-plus --keep-runtime-typing --keep-percent-format
- third-party mpv and winpaths excluded
2021-02-11 09:43:40 +10:00
Damien Elmes
dfe3c457e5 use top level defs for protobuf enum cases
While mypy can understand nested references like ConfigBool.Key.COLLAPSE_RECENT,
PyCharm doesn't understand the metaclass syntax, and shows the definitions
as invalid.
2021-02-05 19:26:13 +10:00
Damien Elmes
168963460f update to latest mypy_protobuf
The handling of enum types has improved - we no longer need to import
separate types at typechecking time.
2021-02-03 13:31:52 +10:00
Damien Elmes
7fda601aef add some typehints, and remove some unused code 2021-01-31 20:56:21 +10:00
Damien Elmes
6c483bb577 add public wrappers for remaining backend functions 2021-01-31 18:56:16 +10:00
Damien Elmes
ea31e8ca3e move the remaining exports from _backend 2021-01-31 18:56:16 +10:00
Damien Elmes
260a270eb0 embed BuiltinSortKind 2021-01-31 18:56:16 +10:00
Damien Elmes
5974163343 embed kind enum in StockNoteType and remove prefix 2021-01-31 18:56:16 +10:00
Damien Elmes
67cb147493 move rsbridge into _backend 2021-01-31 18:55:45 +10:00
Damien Elmes
9d853bbb03 start work on more clearly defining backend/protobuf boundaries
- anki._backend stores the protobuf files and rsbackend.py code
- pylib modules import protobuf messages directly from the
_pb2 files, and explicitly export any will be returned or consumed
by public pylib functions, so that calling code can import from pylib
- the "rsbackend" no longer imports and re-exports protobuf messages
- pylib can just consume them directly.
- move errors to errors.py

Still todo:

- rsbridge
- finishing the work on rsbackend, and check what we need to add
back to the original file location to avoid breaking add-ons
2021-01-31 18:55:45 +10:00