* Migrate build system to uv
Closes#3787, and is a step towards #3081 and #4022
This change breaks our PyOxidizer bundling process. While we probably
could update it to work with the new venvs & lockfile, my intention
is to use this as a base to try out a uv-based packager/installer.
Some notes about the changes:
- Use uv for python download + venv installation
- Drop python/requirements* in favour of pyproject files / uv.lock
- Bumped to latest Python 3.9 version. The move to 3.13 should be
a fairly trivial change when we're ready.
- Dropped the old write_wheel.py in favour of uv/hatchling. This has
the unfortunate side-effect of dropping leading zeros in our wheels,
which we could try hack around in the future.
- Switch to Qt 6.7 for the dev repo, as it's the first PyQt version
with a Linux/ARM WebEngine wheel.
- Unified our macOS deployment target with minimum required for ARM.
- Dropped unused fluent python files
- Dropped unused python license generation
- Dropped helpers to run under Qt 5, as our wheels were already
requiring Qt 6 to install.
* Build action to create universal uv binary
* Drop some PyOxidizer-related files
* Use Windows ARM64 cargo/node binaries during build
We can't provide ARM64 wheels to users yet due to #4079, but we can
at least speed up the build.
The rustls -> native-tls change on Windows is because ring requires
clang to compile for ARM64, and I figured it's best to keep our Windows
deps consistent. We already built the wheels with native-tls.
* Make libankihelper a universal library
We were shipping a single arch library in a purelib, leading to
breakages when running on a different platform.
* Use Python wheel for mpv/lame on Windows/Mac
This is convenient, but suboptimal on a Mac at the moment. The first
run of mpv will take a number of seconds for security checks to run,
and our mpv code ends up timing out, repeating the process each time.
Our installer stub will need to invoke mpv once first to get it validated.
We could address this by distributing the audio with the installer/stub,
or perhaps by putting the binaries in a .pkg file that's notarized+stapled
and then included in the wheel.
* Add some helper scripts to build a fully-locked wheel
* Initial macOS launcher prototype
* Add a hidden env var to preload our libs and audio helpers on macOS
* qt/bundle -> qt/launcher
- remove more of the old bundling code
- handle app icon
* Fat binary, notarization & dmg
* Publish wheels on testpypi for testing
* Use our Python pin for the launcher too
* Python cleanups
* Extend launcher to other platforms + more
- Switch to Qt 6.8 for repo default, as 6.7 depends on an older
libwebp/tiff which is unavailable on newer installs
- Drop tools/mac-x86, as we no longer need to test against Qt 5
- Add flags to cross compile wheels on Mac and Linux
- Bump glibc target to 2_36, building on Debian Stable
- Increase mpv timeout on macOS to allow for initial gatekeeper checks
- Ship both arm64 and amd64 uv on Linux, with a bash stub to pick
the appropriate arch.
* Fix pylint on Linux
* Fix failure to run from /usr/local/bin
* Remove remaining pyoxidizer refs, and clean up duplicate release folder
* Rust dep updates
- Rust 1.87 for now (1.88 due out in around a week)
- Nom looks involved, so I left it for now
- prost-reflect depends on a new prost version that got yanked
* Python 3.13 + dep updates
Updated protoc binaries + add helper in order to try fix build breakage.
Ended up being due to an AI-generated update to pip-system-certs that
was not reviewed carefully enough:
https://gitlab.com/alelec/pip-system-certs/-/issues/36
The updated mypy/black needed some tweaks to our files.
* Windows compilation fixes
* Automatically run Anki after installing on Windows
* Touch pyproject.toml upon install, so we check for updates
* Update Python deps
- urllib3 for CVE
- pip-system-certs got fixed
- markdown/pytest also updated
* Treat play_file() and co as internal routines without protection
Our code and add-ons need a way to play audio from arbitrary locations. I propose we treat the _tag API as suitable for user input, and the _file API for internal use.
* Mention basename in the *_file() paths
We were (partially) doing this for MpvManager, but not for
Windows' SimpleMpvPlayer. By passing a media file starting
with a special scheme, a malicious actor could have caused a file to
be written to the filesystem on Windows.
Thanks once again to Michael Lappas for the report.
mpv looks for ytdl on the path, which includes the CWD on Windows.
A malicious shared deck could place an executable called yt-dlp.exe in the
media folder, which mpv would then helpfully invoke the first time
a YouTube link was encountered.
A big thank you to Michael Lappas for the report.
* Sanitize field content in editor
The editor already strips script tags from fields, but was allowing
through Javascript in things like onclick handlers. We block this now,
as the editor context has access to internal APIs that we don't want to
expose to untrusted third-party code.
* Require an auth token for API access
We were previously inspecting the referrer, but that is spoofable,
and doesn't guard against other processes on the machine.
To accomplish this, we use a request interceptor to automatically
add an auth token to webviews with the right context. Some related
changes were required:
- We avoid storing _page, which was leading to leaks & warning on exit
- At webview creation (or set_kind() invocation), we assign either
an authenticated or unauthenticated web profile.
- Some of our screens initialize the AnkiWebView when calling, e.g.,
aqt.forms.stats.Ui_Dialog(). They then immediately call .set_kind().
This reveals a race condition in our DOM handling code: the webview
initialization creates an empty page with the injected script, which
causes a domDone signal to be sent back. This signal arrives after
we've created another page with .set_kind(), causing our code to think
the DOM is ready when it's not. Then when we try to inject the dynamic
styling, we get an error, as the DOM is not ready yet. In the absence
of better solutions, I've added a hack to set_kind() to deal with this
for now.
* Provide AnkiWebPage init defaults for existing add-on callers
* Inject bridge script when profile set-up skipped
Some add-ons fully override AnkiWebPage.__init__ and thus depend on _setupBridge injecting the JS bridge script.
With this change we account for these cases, while giving add-ons the opportunity to look for solutions that do not require overriding AnkiWebPage.__init__ completely.
* Add some missed pages/endpoints (thanks to iamllama)
* Avoid sending API key for remote resources
Thanks to Abdo for the report
---------
Co-authored-by: Aristotelis P <201596065+aps-amboss@users.noreply.github.com>
* Add myself to CONTRIBUTORS
* Set draggable="false" attribute on .replay-button
Because currently if a user drags slightly (even unintentionally) upon clicking a play button, play does not happen
* Prevent dragging hint links
Because if a user moves cursor a little after `mousedown`, action (expanding the hint) does not occur. Which might cause issues from accessibility standpoint
* stop audio playback on browser close
* revert fix
* add caller-aware versions of play_file and stop_and_clear_queue
* stop editor's audio autoplay on close
* remove superfluous stop_and_clear_queue from addcards
Message arrived on a background non-Qt thread, and called run_in_background(),
which assumes it's running on the GUI thread. This resulted in single_shot()
failing to run in reviewer's on_av_player_did_end_playing on Linux/macOS.
* Move stop-timer-on-answer strings to correct section
* Add auto-advance options to deck preset
* Implement answer actions
* Fix error when last card is answered before timeout
* Fix deserialization of answerAction
* Add answerAction to reserved key list
* Fix inverted boolean
* Add option to wait for audio to finish
* Add auto-advance toggle
* Add shortcut
* Disable auto-advance when main window state changes
* Start auto-advance timer after option is toggled
* Disable auto-advance when main window loses focus
* Use existing translations
* Add Answer Hard and Show Reminder
This reverts commit fa4fc3e15a.
Issue turned out to be a packaging problem, and this should not be
required as the socket should be held open even if removed.
* Use submodule imports in aqt
* Use submodule imports in pylib
* More submodule imports in pylib
These required removing some direct imports to get rid of import cycles.
This was motivated by the fact that recording was crashing on the native
M1 build. That ended up being mostly a PEBKAC problem - turns out the
Mac Mini has no built-in microphone 🤦.
I still thinks this has some value though - it doesn't crash in such
cases, and probably doesn't suffer from the problem shown in this thread
either:
https://forums.ankiweb.net/t/anki-crashes-when-trying-to-record-on-mac/14764
For now, this is only enabled when running on arm64. If it turns out to
be reliable, it could be offered as an option on amd64 as well.
* Alias PyQt5 to PyQt6 on PyQt6 builds
Restores basic compatibility with PyQt5 add-ons
* Register QtCore early to work around sip error
* Monkey-patch unscoped enums that are in use by add-ons back in
Enums whose namespace moved with PyQt6 were determined using the tooling in https://github.com/qutebrowser/qutebrowser/issues/5904
Relevant enums for the Anki add-on ecosystem were found by grepping through all AnkiWeb add-ons and a selection of GitHub-released add-ons.
* Add full Qt.Key namespace
Maintains compatibility with add-ons that allow specifying key bindings via Qt.Key enums
* Reintroduce PyQt6.Qt as an alias for QtCore.Qt
* Alias classes shifted from QtWidgets to QtGui
* Add missing enums
Adds ≈200 enums that were missed during the initial grep
* Map exec_ calls to exec
* Tweak section headers
* Fix QtWebEngineWidgets imports failing due to delayed import
Addesses: "QtWebEngineWidgets must be imported before a QCoreApplication instance is created"
* Register additional aliases for top-level Qt modules
Given how we have had to deal with side-effects when not registering other aliased imports ahead of time, it seems safer to also register the remaining few with sys.modules.
* Handle calls to deprecated PyQt resource API graciously
* Create QtWebEngineWidgets aliases for classes moved to QtWebEngineCore
* Alias QShortcut
* Restore QWebEnginePage.view()
* Alias sip to PyQt6.sip
* Alias QtCore.QRegExp to QtCore.QRegularExpression
* Restructure aqt.qt into package
Pre-requirement for aliasing the PyQt5.Qt namespace correctly.
Should hopefully also make it easier to keep an overview as Qt-compat-related modules were proliferating.
* Properly alias PyQt5.Qt
PyQt5.Qt used to serve as a common namespace for all Qt classes, not just QtCore.Qt.*
While this changes does not make all classes accessible via PyQt5.Qt, it does so for the most important Qt submodules, which should cover most add-on breakages.
* Simplify Qt resource system legacy handling
* Also alias PyQt6.Qt
Covers imports of the form `from PyQt5 import import Qt` (due to previous aliasing of PyQt5 to PyQt6)
* Add missing enums
Better approach to grepping through add-ons yielded additional hits
* Run formatters
* Satisfy pylint
I do not recall anyone reporting that it worked better than the Qt
implementation for them, and the lack of recent wheels on PyPI is a pain.
We can always add it back in the future if enough people come out of
the woodwork to report they were using it.
The enum changes should work on PyQt 5.x, and are required in PyQt 6.x.
They are not supported by the PyQt5 typings however, so we need to run
our tests with PyQt6.
Means URLs like :/icons/foo.jpg should become icons:foo.jpg
This is part of the prep work for a PyQt6 update. PyQt6 has dropped
pyrcc, so we can longer generate the icons_qrc.py file we did previously.
Qt Designer expects us to use the resource system, so we continue to
generate the icons.qrc file to make editing the UI files easier. But at
runtime, we no longer use that file.
This adds Python 3.9 and 3.10 typing syntax to files that import
attributions from __future___. Python 3.9 should be able to cope with
the 3.10 syntax, but Python 3.8 will no longer work.
On Windows/Mac, install the latest Python 3.9 version from python.org.
There are currently no orjson wheels for Python 3.10 on Windows/Mac,
which will break the build unless you have Rust installed separately.
On Linux, modern distros should have Python 3.9 available already. If
you're on an older distro, you'll need to build Python from source first.
