This reverts commit fa4fc3e15a.
Issue turned out to be a packaging problem, and this should not be
required as the socket should be held open even if removed.
I was able to reproduce the crashes fairly reliably by opening the
prefs screen on startup and shutting down the app after 600ms; after
this change the crashes no longer seem to occur.
(for upgrading users, please see the notes at the bottom)
Bazel brought a lot of nice things to the table, such as rebuilds based on
content changes instead of modification times, caching of build products,
detection of incorrect build rules via a sandbox, and so on. Rewriting the build
in Bazel was also an opportunity to improve on the Makefile-based build we had
prior, which was pretty poor: most dependencies were external or not pinned, and
the build graph was poorly defined and mostly serialized. It was not uncommon
for fresh checkouts to fail due to floating dependencies, or for things to break
when trying to switch to an older commit.
For day-to-day development, I think Bazel served us reasonably well - we could
generally switch between branches while being confident that builds would be
correct and reasonably fast, and not require full rebuilds (except on Windows,
where the lack of a sandbox and the TS rules would cause build breakages when TS
files were renamed/removed).
Bazel achieves that reliability by defining rules for each programming language
that define how source files should be turned into outputs. For the rules to
work with Bazel's sandboxing approach, they often have to reimplement or
partially bypass the standard tools that each programming language provides. The
Rust rules call Rust's compiler directly for example, instead of using Cargo,
and the Python rules extract each PyPi package into a separate folder that gets
added to sys.path.
These separate language rules allow proper declaration of inputs and outputs,
and offer some advantages such as caching of build products and fine-grained
dependency installation. But they also bring some downsides:
- The rules don't always support use-cases/platforms that the standard language
tools do, meaning they need to be patched to be used. I've had to contribute a
number of patches to the Rust, Python and JS rules to unblock various issues.
- The dependencies we use with each language sometimes make assumptions that do
not hold in Bazel, meaning they either need to be pinned or patched, or the
language rules need to be adjusted to accommodate them.
I was hopeful that after the initial setup work, things would be relatively
smooth-sailing. Unfortunately, that has not proved to be the case. Things
frequently broke when dependencies or the language rules were updated, and I
began to get frustrated at the amount of Anki development time I was instead
spending on build system upkeep. It's now about 2 years since switching to
Bazel, and I think it's time to cut losses, and switch to something else that's
a better fit.
The new build system is based on a small build tool called Ninja, and some
custom Rust code in build/. This means that to build Anki, Bazel is no longer
required, but Ninja and Rust need to be installed on your system. Python and
Node toolchains are automatically downloaded like in Bazel.
This new build system should result in faster builds in some cases:
- Because we're using cargo to build now, Rust builds are able to take advantage
of pipelining and incremental debug builds, which we didn't have with Bazel.
It's also easier to override the default linker on Linux/macOS, which can
further improve speeds.
- External Rust crates are now built with opt=1, which improves performance
of debug builds.
- Esbuild is now used to transpile TypeScript, instead of invoking the TypeScript
compiler. This results in faster builds, by deferring typechecking to test/check
time, and by allowing more work to happen in parallel.
As an example of the differences, when testing with the mold linker on Linux,
adding a new message to tags.proto (which triggers a recompile of the bulk of
the Rust and TypeScript code) results in a compile that goes from about 22s on
Bazel to about 7s in the new system. With the standard linker, it's about 9s.
Some other changes of note:
- Our Rust workspace now uses cargo-hakari to ensure all packages agree on
available features, preventing unnecessary rebuilds.
- pylib/anki is now a PEP420 implicit namespace, avoiding the need to merge
source files and generated files into a single folder for running. By telling
VSCode about the extra search path, code completion now works with generated
files without needing to symlink them into the source folder.
- qt/aqt can't use PEP420 as it's difficult to get rid of aqt/__init__.py.
Instead, the generated files are now placed in a separate _aqt package that's
added to the path.
- ts/lib is now exposed as @tslib, so the source code and generated code can be
provided under the same namespace without a merging step.
- MyPy and PyLint are now invoked once for the entire codebase.
- dprint will be used to format TypeScript/json files in the future instead of
the slower prettier (currently turned off to avoid causing conflicts). It can
automatically defer to prettier when formatting Svelte files.
- svelte-check is now used for typechecking our Svelte code, which revealed a
few typing issues that went undetected with the old system.
- The Jest unit tests now work on Windows as well.
If you're upgrading from Bazel, updated usage instructions are in docs/development.md and docs/build.md. A summary of the changes:
- please remove node_modules and .bazel
- install rustup (https://rustup.rs/)
- install rsync if not already installed (on windows, use pacman - see docs/windows.md)
- install Ninja (unzip from https://github.com/ninja-build/ninja/releases/tag/v1.11.1 and
place on your path, or from your distro/homebrew if it's 1.10+)
- update .vscode/settings.json from .vscode.dist
Otherwise when user returns to profiles screen, they'll be unable to
open a different profile, as the collection is still open.
Encountered when opening the collection that triggered
https://github.com/ankitects/anki/issues/2123
* call_add-on_update_after_initial_sync
* Add safeMode restriction
* check for anki update after other modal dialogs
* fix failing test
* make changes suggested in review
* added option to load the last loaded profile
* add formatting changes for tests
* fix comments - integrate as new functionality rather than as an option, fix type hinting and remove bug if profile is deleted
In v3, it's more informative to show the count of child decks separately,
since increasing the limit of the current deck does not increase the limits
of child decks. When we rework the decks list in the future, a tooltip
will hopefully provide an easier way for users to see where cards are
available, and where limits are being applied.
Closes#1868
* Add apkg export on backend
* Filter out missing media-paths at write time
* Make TagMatcher::new() infallible
* Gather export data instead of copying directly
* Revert changes to rslib/src/tags/
* Reuse filename_is_safe/check_filename_safe()
* Accept func to produce MediaIter in export_apkg()
* Only store file folder once in MediaIter
* Use temporary tables for gathering
export_apkg() now accepts a search instead of a deck id. Decks are
gathered according to the matched notes' cards.
* Use schedule_as_new() to reset cards
* ExportData → ExchangeData
* Ignore ascii case when filtering system tags
* search_notes_cards_into_table →
search_cards_of_notes_into_table
* Start on apkg importing on backend
* Fix due dates in days for apkg export
* Refactor import-export/package
- Move media and meta code into appropriate modules.
- Normalize/check for normalization when deserializing media entries.
* Add SafeMediaEntry for deserialized MediaEntries
* Prepare media based on checksums
- Ensure all existing media files are hashed.
- Hash incoming files during preparation to detect conflicts.
- Uniquify names of conflicting files with hash (not notetype id).
- Mark media files as used while importing notes.
- Finally copy used media.
* Handle encoding in `replace_media_refs()`
* Add trait to keep down cow boilerplate
* Add notetypes immediately instaed of preparing
* Move target_col into Context
* Add notes immediately instaed of preparing
* Note id, not guid of conflicting notes
* Add import_decks()
* decks_configs → deck_configs
* Add import_deck_configs()
* Add import_cards(), import_revlog()
* Use dyn instead of generic for media_fn
Otherwise, would have to pass None with type annotation in the default
case.
* Fix signature of import_apkg()
* Fix search_cards_of_notes_into_table()
* Test new functions in text.rs
* Add roundtrip test for apkg (stub)
* Keep source id of imported cards (or skip)
* Keep source ids of imported revlog (or skip)
* Try to keep source ids of imported notes
* Make adding notetype with id undoable
* Wrap apkg import in transaction
* Keep source ids of imported deck configs (or skip)
* Handle card due dates and original due/did
* Fix importing cards/revlog
Card ids are manually uniquified.
* Factor out card importing
* Refactor card and revlog importing
* Factor out card importing
Also handle missing parents .
* Factor out note importing
* Factor out media importing
* Maybe upgrade scheduler of apkg
* Fix parent deck gathering
* Unconditionally import static media
* Fix deck importing edge cases
Test those edge cases, and add some global test helpers.
* Test note importing
* Let import_apkg() take a progress func
* Expand roundtrip apkg test
* Use fat pointer to avoid propogating generics
* Fix progress_fn type
* Expose apkg export/import on backend
* Return note log when importing apkg
* Fix archived collection name on apkg import
* Add CollectionOpWithBackendProgress
* Fix wrong Interrupted Exception being checked
* Add ClosedCollectionOp
* Add note ids to log and strip HTML
* Update progress when checking incoming media too
* Conditionally enable new importing in GUI
* Fix all_checksums() for media import
Entries of deleted files are nulled, not removed.
* Make apkg exporting on backend abortable
* Return number of notes imported from apkg
* Fix exception printing for QueryOp as well
* Add QueryOpWithBackendProgress
Also support backend exporting progress.
* Expose new apkg and colpkg exporting
* Open transaction in insert_data()
Was slowing down exporting by several orders of magnitude.
* Handle zstd-compressed apkg
* Add legacy arg to ExportAnkiPackage
Currently not exposed on the frontend
* Remove unused import in proto file
* Add symlink for typechecking of import_export_pb2
* Avoid kwargs in pb message creation, so typechecking is not lost
Protobuf's behaviour is rather subtle and I had to dig through the docs
to figure it out: set a field on a submessage to automatically assign
the submessage to the parent, or call SetInParent() to persist a default
version of the field you specified.
* Avoid re-exporting protobuf msgs we only use internally
* Stop after one test failure
mypy often fails much faster than pylint
* Avoid an extra allocation when extracting media checksums
* Update progress after prepare_media() finishes
Otherwise the bulk of the import ends up being shown as "Checked: 0"
in the progress window.
* Show progress of note imports
Note import is the slowest part, so showing progress here makes the UI
feel more responsive.
* Reset filtered decks at import time
Before this change, filtered decks exported with scheduling remained
filtered on import, and maybe_remove_from_filtered_deck() moved cards
into them as their home deck, leading to errors during review.
We may still want to provide a way to preserve filtered decks on import,
but to do that we'll need to ensure we don't rewrite the home decks of
cards, and we'll need to ensure the home decks are included as part of
the import (or give an error if they're not).
https://github.com/ankitects/anki/pull/1743/files#r839346423
* Fix a corner-case where due dates were shifted by a day
This issue existed in the old Python code as well. We need to include
the user's UTC offset in the exported file, or days_elapsed falls back
on the v1 cutoff calculation, which may be a day earlier or later than
the v2 calculation.
* Log conflicting note in remapped nt case
* take_fields() → into_fields()
* Alias `[u8; 20]` with `Sha1Hash`
* Truncate logged fields
* Rework apkg note import tests
- Use macros for more helpful errors.
- Split monolith into unit tests.
- Fix some unknown error with the previous test along the way.
(Was failing after 969484de4388d225c9f17d94534b3ba0094c3568.)
* Fix sorting of imported decks
Also adjust the test, so it fails without the patch. It was only passing
before, because the parent deck happened to come before the
inconsistently capitalised child alphabetically. But we want all parent
decks to be imported before their child decks, so their children can
adopt their capitalisation.
* target[_id]s → existing_card[_id]s
* export_collection_extracting_media() → ...
export_into_collection_file()
* target_already_exists→card_ordinal_already_exists
* Add search_cards_of_notes_into_table.sql
* Imrove type of apkg export selector/limit
* Remove redundant call to mod_schema()
* Parent tooltips to mw
* Fix a crash when truncating note text
String::truncate() is a bit of a footgun, and I've hit this before
too :-)
* Remove ExportLimit in favour of separate classes
* Remove OpWithBackendProgress and ClosedCollectionOp
Backend progress logic is now in ProgressManager. QueryOp can be used
for running on closed collection.
Also fix aborting of colpkg exports, which slipped through in #1817.
* Tidy up import log
* Avoid QDialog.exec()
* Default to excluding scheuling for deck list deck
* Use IncrementalProgress in whole import_export code
* Compare checksums when importing colpkgs
* Avoid registering changes if hashes are not needed
* ImportProgress::Collection → ImportProgress::File
* Make downgrading apkgs depend on meta version
* Generalise IncrementableProgress
And use it in entire import_export code instead.
* Fix type complexity lint
* Take count_map for IncrementableProgress::get_inner
* Replace import/export env with Shift click
* Accept all args from update() for backend progress
* Pass fields of ProgressUpdate explicitly
* Move update_interval into IncrementableProgress
* Outsource incrementing into Incrementor
* Mutate ProgressUpdate in progress_update callback
* Switch import/export legacy toggle to profile setting
Shift would have been nice, but the existing shortcuts complicate things.
If the user triggers an import with ctrl+shift+i, shift is unlikely to
have been released by the time our code runs, meaning the user accidentally
triggers the new code. We could potentially wait a while before bringing
up the dialog, but then we're forced to guess at how long it will take the
user to release the key.
One alternative would be to use alt instead of shift, but then we need to
trigger our shortcut when that key is pressed as well, and it could
potentially cause a conflict with an add-on that already uses that
combination.
* Show extension in export dialog
* Continue to provide separate options for schema 11+18 colpkg export
* Default to colpkg export when using File>Export
* Improve appearance of combo boxes when switching between apkg/colpkg
+ Deal with long deck names
* Convert newlines to spaces when showing fields from import
Ensures each imported note appears on a separate line
* Don't separate total note count from the other summary lines
This may come down to personal preference, but I feel the other counts
are equally as important, and separating them feels like it makes it
a bit easier to ignore them.
* Fix 'deck not normal' error when importing a filtered deck for the 2nd time
* Fix [Identical] being shown on first import
* Revert "Continue to provide separate options for schema 11+18 colpkg export"
This reverts commit 8f0b2c175f.
Will use a different approach
* Move legacy support into a separate exporter option; add to apkg export
* Adjust 'too new' message to also apply to .apkg import case
* Show a better message when attempting to import new apkg into old code
Previously the user could end seeing a message like:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte
Unfortunately we can't retroactively fix this for older clients.
* Hide legacy support option in older exporting screen
* Reflect change from paths to fnames in type & name
* Make imported decks normal at once
Then skip special casing in update_deck(). Also skip updating
description if new one is empty.
Co-authored-by: Damien Elmes <gpg@ankiweb.net>
* Always enable manually installed add-ons
Ensures that manually installed add-ons are enabled after the installation, even if previously disabled.
Prevents scenarios where users could end up with no active add-on build (e.g. when switching between stable add-on builds distributed via AnkiWeb and betas distributed via GitHub).
* Improve type annotations
* Also enable disabled AnkiWeb add-ons upon interactive installation
Applies to add-ons that users actively install via their AnkiWeb ID. Updates are exempt, preserving whatever status add-ons were in.
* Prevent disabled add-ons from triggering conflicts
* Fix download_addons() not passing on force_enable argument (dae)
* Collection needs to be closed prior to backup even when not downgrading
* Backups -> BackupLimits
* Some improvements to backup_task
- backup_inner now returns the error instead of logging it, so that
the frontend can discover the issue when they await a backup (or create
another one)
- start_backup() was acquiring backup_task twice, and if another thread
started a backup between the two locks, the task could have been accidentally
overwritten without awaiting it
* Backups no longer require a collection close
- Instead of closing the collection, we ensure there is no active
transaction, and flush the WAL to disk. This means the undo history
is no longer lost on backup, which will be particularly useful if we
add a periodic backup in the future.
- Because a close is no longer required, backups are now achieved with
a separate command, instead of being included in CloseCollection().
- Full sync no longer requires an extra close+reopen step, and we now
wait for the backup to complete before proceeding.
- Create a backup before 'check db'
* Add File>Create Backup
https://forums.ankiweb.net/t/anki-mac-os-no-backup-on-sync/6157
* Defer checkpoint until we know we need it
When running periodic backups on a timer, we don't want to be fsync()ing
unnecessarily.
* Skip backup if modification time has not changed
We don't want the user leaving Anki open overnight, and coming back
to lots of identical backups.
* Periodic backups
Creates an automatic backup every 30 minutes if the collection has been
modified.
If there's a legacy checkpoint active, tries again 5 minutes later.
* Switch to a user-configurable backup duration
CreateBackup() now uses a simple force argument to determine whether
the user's limits should be respected or not, and only potentially
destructive ops (full download, check DB) override the user's configured
limit.
I considered having a separate limit for collection close and automatic
backups (eg keeping the previous 5 minute limit for collection close),
but that had two downsides:
- When the user closes their collection at the end of the day, they'd
get a recent backup. When they open the collection the next day, it
would get backed up again within 5 minutes, even though not much had
changed.
- Multiple limits are harder to communicate to users in the UI
Some remaining decisions I wasn't 100% sure about:
- If force is true but the collection has not been modified, the backup
will be skipped. If the user manually deleted their backups without
closing Anki, they wouldn't get a new one if the mtime hadn't changed.
- Force takes preference over the configured backup interval - should
we be ignored the user here, or take no backups at all?
Did a sneaky edit of the existing ftl string, as it hasn't been live
long.
* Move maybe_backup() into Collection
* Use a single method for manual and periodic backups
When manually creating a backup via the File menu, we no longer make
the user wait until the backup completes. As we continue waiting for
the backup in the background, if any errors occur, the user will get
notified about it fairly quickly.
* Show message to user if backup was skipped due to no changes
+ Don't incorrectly assert a backup will be created on force
* Add "automatic" to description
* Ensure we backup prior to importing colpkg if collection open
The backup doesn't happen when invoked from 'open backup' in the profile
screen, which matches Anki's previous behaviour. The user could
potentially clobber up to 30 minutes of their work if they exited to
the profile screen and restored a backup, but the alternative is we
create backups every time a backup is restored, which may happen a number
of times if the user is trying various ones. Or we could go back to a
separate throttle amount for this case, at the cost of more complexity.
* Remove the 0 special case on backup interval; minimum of 5 minutes
https://github.com/ankitects/anki/pull/1728#discussion_r830876833
* Add zstd dep
* Implement backend backup with zstd
* Implement backup thinning
* Write backup meta
* Use new file ending anki21b
* Asynchronously backup on collection close in Rust
* Revert "Add zstd dep"
This reverts commit 3fcb2141d2.
* Add zstd again
* Take backup col path from col struct
* Fix formatting
* Implement backup restoring on backend
* Normalize restored media file names
* Refactor `extract_legacy_data()`
A bit cumbersome due to borrowing rules.
* Refactor
* Make thinning calendar-based and gradual
* Consider last kept backups of previous stages
* Import full apkgs and colpkgs with backend
* Expose new backup settings
* Test `BackupThinner` and make it deterministic
* Mark backup_path when closing optional
* Delete leaky timer
* Add progress updates for restoring media
* Write restored collection to tempfile first
* Do collection compression in the background thread
This has us currently storing an uncompressed and compressed copy of
the collection in memory (not ideal), but means the collection can be
closed without waiting for compression to complete. On a large collection,
this takes a close and reopen from about 0.55s to about 0.07s. The old
backup code for comparison: about 0.35s for compression off, about
8.5s for zip compression.
* Use multithreading in zstd compression
On my system, this reduces the compression time of a large collection
from about 0.55s to 0.08s.
* Stream compressed collection data into zip file
* Tweak backup explanation
+ Fix incorrect tab order for ignore accents option
* Decouple restoring backup and full import
In the first case, no profile is opened, unless the new collection
succeeds to load.
In the second case, either the old collection is reloaded or the new one
is loaded.
* Fix number gap in Progress message
* Don't revert backup when media fails but report it
* Tweak error flow
* Remove native BackupLimits enum
* Fix type annotation
* Add thinning test for whole year
* Satisfy linter
* Await async backup to finish
* Move restart disclaimer out of backup tab
Should be visible regardless of the current tab.
* Write restored collection in chunks
* Refactor
* Write media in chunks and refactor
* Log error if removing file fails
* join_backup_task -> await_backup_completion
* Refactor backup.rs
* Refactor backup meta and collection extraction
* Fix wrong error being returned
* Call sync_all() on new collection
* Add ImportError
* Store logger in Backend, instead of creating one on demand
init_backend() accepts a Logger rather than a log file, to allow other
callers to customize the logger if they wish.
In the future we may want to explore using the tracing crate as an
alternative; it's a bit more ergonomic, as a logger doesn't need to be
passed around, and it plays more nicely with async code.
* Sync file contents prior to rename; sync folder after rename.
* Limit backup creation to once per 30 min
* Use zstd::stream::copy_decode
* Make importing abortable
* Don't revert if backup media is aborted
* Set throttle implicitly
* Change force flag to minimum_backup_interval
* Don't attempt to open folders on Windows
* Join last backup thread before starting new one
Also refactor.
* Disable auto sync and backup when restoring again
* Force backup on full download
* Include the reason why a media file import failed, and the file path
- Introduce a FileIoError that contains a string representation of
the underlying I/O error, and an associated path. There are a few
places in the code where we're currently manually including the filename
in a custom error message, and this is a step towards a more consistent
approach (but we may be better served with a more general approach in
the future similar to Anyhow's .context())
- Move the error message into importing.ftl, as it's a bit neater
when error messages live in the same file as the rest of the messages
associated with some functionality.
* Fix importing of media files
* Minor wording tweaks
* Save an allocation
I18n strings with replacements are already strings, so we can skip the
extra allocation. Not that it matters here at all.
* Terminate import if file missing from archive
If a third-party tool is creating invalid archives, the user should know
about it. This should be rare, so I did not attempt to make it
translatable.
* Skip multithreaded compression on small collections
Co-authored-by: Damien Elmes <gpg@ankiweb.net>
* Add progress.single_shot()
* Fix periodic garbage collection
* Properly cleanup mediasync timers
* Revert some replacements with `single_shot()`
These timers shouldn't fire if their widget is destroyed.
* Add timer docs explaining issues and alternatives
* Apply suggestions from code review
* Tweak docstrings
* Fix wrong hook being torn down
* Fix item models not being destroyed
* Add missing gc for FilteredDeckConfigDialog
* Add missing type annotation
* Pass calling widget as parent to QTimer
Implicitly passing `self.mw` as the parent means that the QTimer won't
get destroyed before quitting the app, which also thwarts garbage
collection of any data captured by a passed closure.
* Make `Editor._links` an instance variable
Browser is inserting a closure into this dict capturing itself. As a class
variable, it won't get destroyed, so neither will the browser.
* Make `Editor._links` funcs take instance again
* Deprecate calling progress.timer() without parent
* show caller location when printing deprecation warning (dae)
* Add main view menu
* Add browser view menu
* Use standard keys for zooming and full screen
* Capitalise menu item names
* Toggle Showing Cards/Notes -> Toggle Cards/Notes
* Explicitly set linux full screen key
on_toggle_fullscreen -> on_toggle_full_screen
* Use submodule imports in aqt
* Use submodule imports in pylib
* More submodule imports in pylib
These required removing some direct imports to get rid of import cycles.
* Call StudyDeck with callback
* StudyDeck w/ callback, remove redundant assignment
* Replace exec() with show() for various dialogs
* Update super init args for Models.__init__
* Make StudyDialog ApplicationModal
When backups were moved into a separate thread 5 years ago, it improved
performance when switching between different profiles, as the backup
happened in the background. But when closing Anki, we wait on the
background thread to complete, so Anki hangs until the backup finishes.
The performance difference on a large collection is considerable:
- 0.45s without compression
- 7.9s with compression
Given that the majority of users probably aren't using multiple profiles,
I think the speed increase is probably worth the extra disk usage. In
the future, we may want to look into using zstd to compress the backups,
which may even be a performance win over the uncompressed version on
some devices.
* Make webview zoom optional
Also suppress mouse wheel zooming.
* Disable zoom for top and bottom bars in main view
* Factor in macos zoom by scrolling and refactor
* Support drag & drop in main window to import files
* Pass drag & drop events to super when not in deck browser
Seems this is required so that JS in the reviewer receives dragover
* Remove unused allowDrops
Kept the favicon, but have reverted the rest, as it unfortunately did
not seem to prevent the issue from occurring.
Original discussion: https://github.com/ankitects/anki/pull/1369
This reverts commit 6d0f7e7f05.
The packaged builds of 2.1.50 use python -OO, which means our assertion
statements won't be run. This is not an issue for unit tests (as we
don't run them from a packaged build), or for type assertions (which are
added for mypy's benefit), but we do need to ensure that invariant checks
are still run.
* Allow theme change at runtime and add hook
* Save or restore default palette on theme change
* Update aqt widget styles on theme change
* styling fixes
- drop _light_palette, as default_palette serves the same purpose
- save default platform theme, and restore it when switching away
from nightmode
- update macOS light/dark mode on theme switch
- fix unreadable menus on Windows
* update night-mode classes on theme change
This is the easy part - CSS styling that uses standard_css or our
css variables should update automatically. The main remaining issue
is JS code that sets colors based on the theme at the time it's run -
eg the graph code, and the editor.
* switch night mode value on toggle
* expose current theme via a store; switch graphs to use it
https://github.com/ankitects/anki/issues/1471#issuecomment-972402492
* start using currentTheme in editor/components
This fixes basic editing - there are still components that need updating.
* add simple xcodeproj for code completion
* add helper to get currently-active system theme on macOS
* fix setCurrentTheme not being immediately available
* live update tag color
* style().name() doesn't work on Qt5
* automatic theme switching on Windows/Mac
* currentTheme -> pageTheme
* Replace `nightModeKey` with `pageTheme`
Co-authored-by: Damien Elmes <gpg@ankiweb.net>
* PEP8 dbproxy.py
* PEP8 errors.py
* PEP8 httpclient.py
* PEP8 lang.py
* PEP8 latex.py
* Add decorator to deprectate key words
* Make replacement for deprecated attribute optional
* Use new helper `_print_replacement_warning()`
* PEP8 media.py
* PEP8 rsbackend.py
* PEP8 sound.py
* PEP8 stdmodels.py
* PEP8 storage.py
* PEP8 sync.py
* PEP8 tags.py
* PEP8 template.py
* PEP8 types.py
* Fix DeprecatedNamesMixinForModule
The class methods need to be overridden with instance methods, so every
module has its own dicts.
* Use `# pylint: disable=invalid-name` instead of id
* PEP8 utils.py
* Only decorate `__getattr__` with `@no_type_check`
* Fix mypy issue with snakecase
Importing it from `anki._vendor` raises attribute errors.
* Format
* Remove inheritance of DeprecatedNamesMixin
There's almost no shared code now and overriding classmethods with
instance methods raises mypy issues.
* Fix traceback frames of deprecation warnings
* remove fn/TimedLog (dae)
Neither Anki nor add-ons appear to have been using it
* fix some issues with stringcase use (dae)
- the wheel was depending on the PyPI version instead of our vendored
version
- _vendor:stringcase should not have been listed in the anki py_library.
We already include the sources in py_srcs, and need to refer to them
directly. By listing _vendor:stringcase as well, we were making a
top-level stringcase library available, which would have only worked for
distributing because the wheel definition was also incorrect.
- mypy errors are what caused me to mistakenly add the above - they
were because the type: ignore at the top of stringcase.py was causing
mypy to completely ignore the file, so it was not aware of any attributes
it contained.
The enum changes should work on PyQt 5.x, and are required in PyQt 6.x.
They are not supported by the PyQt5 typings however, so we need to run
our tests with PyQt6.
This adds Python 3.9 and 3.10 typing syntax to files that import
attributions from __future___. Python 3.9 should be able to cope with
the 3.10 syntax, but Python 3.8 will no longer work.
On Windows/Mac, install the latest Python 3.9 version from python.org.
There are currently no orjson wheels for Python 3.10 on Windows/Mac,
which will break the build unless you have Rust installed separately.
On Linux, modern distros should have Python 3.9 available already. If
you're on an older distro, you'll need to build Python from source first.
This was flawed - while non-Latin text is now acceptable
in an IRI, we still need to be concerned with reserved characters
such as spaces, and Anki unfortunately has been storing the filenames
in unencoded form in the DB, meaning we must encode them at display
time. We won't be able to move away from this until existing notes
are rewritten, and it will probably require breaking compatibility with
older clients.
https://en.wikipedia.org/wiki/Internationalized_Resource_Identifier
This reverts commit 14110add55.
Back in the WebKit days, images with Unicode filenames would fail to
appear if they weren't percent-escaped. This no longer seems to be the
case - with this patch, images appear correctly on the Mac and Windows
platforms I tested with.
Fixes https://forums.ankiweb.net/t/anki-2-1-45-beta/10664/96Fixes#1219
'FlagManager' allows cached access to the flag objects, takes care of
renaming flags and notifies GUI components with the hook
'flag_label_did_change'.
When a modal was created with another window as its parent, the other
window was being returned, when it was the current window that we
actually wanted. This caused nextCard() to be called again when it
presented the timebox modal, leading to a stack overflow.
https://forums.ankiweb.net/t/anki-2-1-45-alpha/10061/71
Also:
- fix issues where the Undo action in the Browse screen was not
consistent with the main window. The existing hook signature has been
changed; from a snapshot of the add-on code from a few months ago, it
was not a hook that was being used by anyone.
- change the undo shortcut in the Browse window to match the main
window. It was different because undoing a change in the editing area
could accidentally trigger an undo of an operation, but the damage is
limited now that (most) operations can be redone. If it still proves to
be a problem, perhaps we should just always swallow ctrl+z when an
editing field is focused.
- The old options are still available with a shift+click
- Translations still need updating
- See b7747b6a38 for some examples
of extending the screen in an add-on
- make sure we set flag in changes when config var changed
- move current deck get/set into backend
- set_config() now returns a bool indicating whether a change was
made, so other operations can be gated off it
- active decks generation is deferred until sched.reset()
By passing back the builder to the calling code to run, we don't need
to plumb extra arguments like success= and handler= through each
operation, and the ability to override the default tooltip behaviour
comes free on all operations
By calling refresh() manually after performing an op, we were refreshing
twice, and the selection was being lost when changes were made outside
of the sidebar.
Also drop the after_hooks arg to perform_op(), since nothing is using
it now.
- pass the handler directly
- reviewer special-cases for flags and notes are now applied at
call site
- drop the kind attribute on OpChanges which is not needed
Instances can pass handled_by=self to more easily ignore events they
initiate.
Fixes ugly refresh when expanding/collapsing decks, but we're still
refreshing the card/notes area unnecessarily in that case.
- Filtered deck creation now happens as an atomic operation, and is
undoable.
- The logic for initial search text, normalizing searches and so on
has been pushed into the backend.
- Use protobuf to pass the filtered deck to the updated dialog, so
we don't need to deal with untyped JSON.
- Change the "revise your search?" prompt to be a simple info box -
user has access to cancel and build buttons, and doesn't need a separate
prompt. Tweak the wording so the 'show excluded' button should be more
obvious.
- Filtered decks have a time appended to them instead of a number,
primarily because it's easier to implement. No objections going back to
the old behaviour if someone wants to contribute a clean patch.
The standard de-duplication will happen if two decks are created in the
same minute with the same name.
- Tweak the default sort order, and start with two searches. The UI
will still hide the second search by default, but by starting with two,
the frontend doesn't need logic for creating the starting text.
- Search errors now have their own error type, instead of using
InvalidInput, as that was intended mainly for bad API calls. The markdown
conversion is done when the error is converted from the backend, allowing
errors to printed as a string without any special handling by the calling
code.
TODO: when building a new filtered deck, update_active() is clobbering
the undo log when the overview is refreshed
- QueueConfig is only used by the scheduler
- DeckConfig was being used in places that Config should have been used
- Add "Dict" to the name so that the bare name is free for use with a
stronger type.
Qt is pretty enthusiastic about redrawing the card list when any sort
of activity occurs, and by serving blank cells while the DB was busy,
we were getting ugly flashes, and cells getting stuck blank.
Resolve the issue by calculating a row up front and caching it, then
serving stale content when updates are blocked.
- clear_unused_tags() is now undoable, and returns the number of removed
notes
- add a new mw.query_op() helper for immutable queries
- decouple "freeze/unfreeze ui state" hooks from the "interface update
required" hook, so that the former is fired even on error, and can be
made re-entrant
- use a 'block_updates' flag in Python, instead of setUpdatesEnabled(),
as the latter has the side-effect of preventing child windows like
tooltips from appearing, and forces a full redrawn when updates are
enabled again. The new behaviour leads to the card list blanking out
when a long-running op is running, but in the future if we cache the
cell values we can just display them from the cache instead.
- we were indiscriminately saving the note with saveNow(), due to the
call to saveTags(). Changed so that it only saves when the tags field
is focused.
- drain the "on_done" queue on main before launching a new background
task, to lower the chances of something in on_done making a small query
to the DB and hanging until a long op finishes
- the duplicate check in the editor was executed after the webview loads,
leading to it hanging until the sidebar finishes loading. Run it at
set_note() time instead, so that the editor loads first.
- don't throw an error when a long-running op started with with_progress()
finishes after the window it was launched from has closed
- don't throw an error when the browser is closed before the sidebar
has finished loading
- Introduced a new transact() method that wraps the return value
in a separate struct that describes the changes that were made.
- Changes are now gathered from the undo log, so we don't need to
guess at what was changed - eg if update_note() is called with identical
note contents, no changes are returned. Card changes will only be set
if cards were actually generated by the update_note() call, and tag
will only be set if a new tag was added.
- mw.perform_op() has been updated to expect the op to return the changes,
or a structure with the changes in it, and it will use them to fire the
change hook, instead of fetching the changes from undo_status(), so there
is no risk of race conditions.
- the various calls to mw.perform_op() have been split into separate
files like card_ops.py. Aside from making the code cleaner, this works
around a rather annoying issue with mypy. Because we run it with
no_strict_optional, mypy is happy to accept an operation that returns None,
despite the type signature saying it requires changes to be returned.
Turning no_strict_optional on for the whole codebase is not practical
at the moment, but we can enable it for individual files.
Still todo:
- The cursor keeps moving back to the start of a field when typing -
we need to ignore the refresh hook when we are the initiator.
- The busy cursor icon should probably be delayed a few hundreds ms.
- Still need to think about a nicer way of handling saveNow()
- op_made_changes(), op_affects_study_queue() might be better embedded
as properties in the object instead
Issues that need fixing:
- when the editor saves the note with perform_op(), if it isn't modified,
no new undo entry is created, and perform_op then returns the changes
made by the previous operation instead
- the approach of fetching the last action in a subsequent backend
method is unsound, as another queued operation may sneak in first before
we have a chance to query the result - it would be better if it were
returned in a single atomic action
- redrawing the current card while editing is likely to make sound
autoplay annoyingly, and it has an unpleasant redraw. We may be better off
fading it out instead
Side note: the editor cursor moves to the start of the field when the
note is updated in another window - it might be nicer to have it move
the cursor to the end instead.
- This avoids the need for a separate screen, though we may want to
slightly fade out the display when information is stale.
- Means the browser can delay updates just like the main window does.
'card modified' covers the common case where we need to rebuild the
study queue, but is also set when changing the card flags. We want to
avoid a queue rebuild in that case, as it causes UI flicker, and may
result in a different card being shown. Note marking doesn't trigger
a queue build, but still causes flicker, and may return the user back
to the front side when they were looking at the answer.
I still think entity-based change tracking is the simplest in the
common case, but to solve the above, I've introduced an enum describing
the last operation that was taken. This currently is not trying to list
out all possible operations, and just describes the ones we want to
special-case.
Other changes:
- Fire the old 'state_did_reset' hook after an operation is performed,
so legacy code can refresh itself after an operation is performed.
- Fire the new `operation_did_execute` hook when mw.reset() is called,
so that as the UI is updated to the use the new hook, it will still
be able to refresh after legacy code calls mw.reset()
- Update the deck browser, overview and review screens to listen to
the new hook, instead of relying on the main window to call moveToState()
- Add a 'set flag' backend action, so we can distinguish it from a
normal card update.
- Drop the separate added/modified entries in the change list in
favour of a single entry per entity.
- Add typing to mw.state
- Tweak perform_op()
- Convert a few more actions to use perform_op()
Basic proof of concept, where the 'delete note' operation in the
reviewer has been updated to use mw.perform_op(). Instead of manually
calling .reset() afterwards, a summary of the changes is returned as
part of the undo status query, and various parts of the GUI can listen
to gui_hooks.operation_did_execute and decide whether they want to
redraw based on the scope of the changes. This should allow the sidebar
to selectively redraw just the tags area in the future for example.
Currently we're just listing out all possible areas that might be changed;
in the future we could theoretically inspect the specific changes in the
undo log to provide a more accurate report (avoiding refreshing the tags
list when no tags were added for example).
You can test it out by opening the browse screen while studying, and
then deleting the current card - the browser should update to show (deleted)
on the cards due the earlier change.
If going ahead with this, aside from updating all the screens that currently
listen for resets, some thought will be required on how we can integrate
it with legacy code that expects to called when resets are made, and expects
to call .reset() when it makes changes.
Thoughts?
