- Update nextest (not required)
- Build nextest without self-update, which pulls in ring
- Disable running of tests in rsbridge, as it has no tests, and
requires host arch's python.lib to execute
- A double \ in CARGO_TARGET_DIR was breaking update_* tests
* Migrate build system to uv
Closes#3787, and is a step towards #3081 and #4022
This change breaks our PyOxidizer bundling process. While we probably
could update it to work with the new venvs & lockfile, my intention
is to use this as a base to try out a uv-based packager/installer.
Some notes about the changes:
- Use uv for python download + venv installation
- Drop python/requirements* in favour of pyproject files / uv.lock
- Bumped to latest Python 3.9 version. The move to 3.13 should be
a fairly trivial change when we're ready.
- Dropped the old write_wheel.py in favour of uv/hatchling. This has
the unfortunate side-effect of dropping leading zeros in our wheels,
which we could try hack around in the future.
- Switch to Qt 6.7 for the dev repo, as it's the first PyQt version
with a Linux/ARM WebEngine wheel.
- Unified our macOS deployment target with minimum required for ARM.
- Dropped unused fluent python files
- Dropped unused python license generation
- Dropped helpers to run under Qt 5, as our wheels were already
requiring Qt 6 to install.
* Build action to create universal uv binary
* Drop some PyOxidizer-related files
* Use Windows ARM64 cargo/node binaries during build
We can't provide ARM64 wheels to users yet due to #4079, but we can
at least speed up the build.
The rustls -> native-tls change on Windows is because ring requires
clang to compile for ARM64, and I figured it's best to keep our Windows
deps consistent. We already built the wheels with native-tls.
* Make libankihelper a universal library
We were shipping a single arch library in a purelib, leading to
breakages when running on a different platform.
* Use Python wheel for mpv/lame on Windows/Mac
This is convenient, but suboptimal on a Mac at the moment. The first
run of mpv will take a number of seconds for security checks to run,
and our mpv code ends up timing out, repeating the process each time.
Our installer stub will need to invoke mpv once first to get it validated.
We could address this by distributing the audio with the installer/stub,
or perhaps by putting the binaries in a .pkg file that's notarized+stapled
and then included in the wheel.
* Add some helper scripts to build a fully-locked wheel
* Initial macOS launcher prototype
* Add a hidden env var to preload our libs and audio helpers on macOS
* qt/bundle -> qt/launcher
- remove more of the old bundling code
- handle app icon
* Fat binary, notarization & dmg
* Publish wheels on testpypi for testing
* Use our Python pin for the launcher too
* Python cleanups
* Extend launcher to other platforms + more
- Switch to Qt 6.8 for repo default, as 6.7 depends on an older
libwebp/tiff which is unavailable on newer installs
- Drop tools/mac-x86, as we no longer need to test against Qt 5
- Add flags to cross compile wheels on Mac and Linux
- Bump glibc target to 2_36, building on Debian Stable
- Increase mpv timeout on macOS to allow for initial gatekeeper checks
- Ship both arm64 and amd64 uv on Linux, with a bash stub to pick
the appropriate arch.
* Fix pylint on Linux
* Fix failure to run from /usr/local/bin
* Remove remaining pyoxidizer refs, and clean up duplicate release folder
* Rust dep updates
- Rust 1.87 for now (1.88 due out in around a week)
- Nom looks involved, so I left it for now
- prost-reflect depends on a new prost version that got yanked
* Python 3.13 + dep updates
Updated protoc binaries + add helper in order to try fix build breakage.
Ended up being due to an AI-generated update to pip-system-certs that
was not reviewed carefully enough:
https://gitlab.com/alelec/pip-system-certs/-/issues/36
The updated mypy/black needed some tweaks to our files.
* Windows compilation fixes
* Automatically run Anki after installing on Windows
* Touch pyproject.toml upon install, so we check for updates
* Update Python deps
- urllib3 for CVE
- pip-system-certs got fixed
- markdown/pytest also updated
Make sure to run tools/install-n2 after updating to this commit.
n2 have merged in some changes we were previously hosting in a fork,
but the parsing of the flags was altered.
Prior to this change, ./run fails out of the box on ARM systems, as Qt
wasn't available on PyPI until the 6.8 release.
Also added a script in tools/ for testing Qt6.8 issues on other platforms.
* Feat/FSRS-5
* adapt the SimulatorConfig of FSRS-5
* update parameters from FSRS-4.5
* udpate to FSRS-rs v1.1.0
* ./ninja fix:minilints
* pass ci
* update cargo-deny to 0.14.24
* udpate to FSRS-rs v1.1.1
* update to fsrs-rs v1.1.2
* Update to latest Node LTS
* Add sveltekit
* Split tslib into separate @generated and @tslib components
SvelteKit's path aliases don't support multiple locations, so our old
approach of using @tslib to refer to both ts/lib and out/ts/lib will no
longer work. Instead, all generated sources and their includes are
placed in a separate out/ts/generated folder, and imported via @generated
instead. This also allows us to generate .ts files, instead of needing
to output separate .d.ts and .js files.
* Switch package.json to module type
* Avoid usage of baseUrl
Incompatible with SvelteKit
* Move sass into ts; use relative links
SvelteKit's default sass support doesn't allow overriding loadPaths
* jest->vitest, graphs example working with yarn dev
* most pages working in dev mode
* Some fixes after rebasing
* Fix/silence some svelte-check errors
* Get image-occlusion working with Fabric types
* Post-rebase lock changes
* Editor is now checked
* SvelteKit build integrated into ninja
* Use the new SvelteKit entrypoint for pages like congrats/deck options/etc
* Run eslint once for ts/**; fix some tests
* Fix a bunch of issues introduced when rebasing over latest main
* Run eslint fix
* Fix remaining eslint+pylint issues; tests now all pass
* Fix some issues with a clean build
* Latest bufbuild no longer requires @__PURE__ hack
* Add a few missed dependencies
* Add yarn.bat to fix Windows build
* Fix pages failing to show when ANKI_API_PORT not defined
* Fix svelte-check and vitest on Windows
* Set node path in ./yarn
* Move svelte-kit output to ts/.svelte-kit
Sadly, I couldn't figure out a way to store it in out/ if out/ is
a symlink, as it breaks module resolution when SvelteKit is run.
* Allow HMR inside Anki
* Skip SvelteKit build when HMR is defined
* Fix some post-rebase issues
I should have done a normal merge instead.
The 0.14.12 release appears to have broken "-A duplicate". Fix by
updating our checks to use the latest release/format.
Also update iana-time-zone, which was yanked, and ignore safemem,
which is only used when bundling.
- Fixes an issue where tasks would continue to appear active for a while
after they had finished on Unix platforms
- The latest n2 now behaves the same way as ninja when substituting
variables, so we no longer need to do the substitution ourselves.
* Add `extra` directory as a designated ignored folder
Excludes `extra/` from version tracking, file formatters, and file checks.
* Remove pytest cache from exclusion rules
Python test discovery is easy enough to disable for the workspace in VS Code's settings and pytest does not serve any purpose in the context of the project anyway.
- Fix warning on Linux about conflicting args
- Use clear instead of printing a control char
- Print the rebuild time
- Perform a rebuild on initial invocation
Also fix minilints declaring a stamp it wasn't creating. The same
approach is necessary with archives now too, as it no longer executes
under a standard "runner run".
For now, rustls is hard-coded - we could pass the desired TLS impl in
from the ./ninja script, but the runner is not recompiled frequently
anyway.
Workspace deps were introduced in Rust 1.64. They don't cover all the
cases that Hakari did unfortunately, but they are simpler to maintain,
and they avoid a couple of issues that Hakari had:
- It sometimes made updating dependencies harder due to the locked versions,
so you had to disable Hakari, do the updates, and then re-generate (
e.g. 943dddf28f)
- The current Hakari config was breaking AnkiDroid's build, as it was
stopping a cross-compile from functioning correctly.
- Dropped the protobuf extensions in favor of explicitly listing out
methods in both services if we want to implement both, as it's clearer.
- Move Service/Method wrappers into a separate crate that the various
clients can import, to easily get at the list of backend services and
their correct indices and comments.
Provides better visibility into what the build is currently doing.
Motivated by slow node.js downloads making the build appear stuck.
You can test this out by running ./tools/install-n2 then building
normally. Please report any problems, and 'cargo uninstall n2' to get
back to the old behaviour. It works on Windows, but prints a new line
each second instead of redrawing the same area.
A couple of changes were required for compatibility:
- n2 doesn't resolve $variable names inside other variables, so the
resolution needs to be done by our build generator.
- Our inputs and outputs in build.ninja need to be listed in a deterministic
order to avoid unwanted rebuilds. I've made a few other tweaks so the
build file should now be fully-deterministic.
* Fix .no-reduce-motion missing from graphs spinner, and not being honored
* Begin migration from protobuf.js -> protobuf-es
Motivation:
- Protobuf-es has a nicer API: messages are represented as classes, and
fields which should exist are not marked as nullable.
- As it uses modules, only the proto messages we actually use get included
in our bundle output. Protobuf.js put everything in a namespace, which
prevented tree-shaking, and made it awkward to access inner messages.
- ./run after touching a proto file drops from about 8s to 6s on my machine. The tradeoff
is slower decoding/encoding (#2043), but that was mainly a concern for the
graphs page, and was unblocked by
37151213cd
Approach/notes:
- We generate the new protobuf-es interface in addition to existing
protobuf.js interface, so we can migrate a module at a time, starting
with the graphs module.
- rslib:proto now generates RPC methods for TS in addition to the Python
interface. The input-arg-unrolling behaviour of the Python generation is
not required here, as we declare the input arg as a PlainMessage<T>, which
marks it as requiring all fields to be provided.
- i64 is represented as bigint in protobuf-es. We were using a patch to
protobuf.js to get it to output Javascript numbers instead of long.js
types, but now that our supported browser versions support bigint, it's
probably worth biting the bullet and migrating to bigint use. Our IDs
fit comfortably within MAX_SAFE_INTEGER, but that may not hold for future
fields we add.
- Oneofs are handled differently in protobuf-es, and are going to need
some refactoring.
Other notable changes:
- Added a --mkdir arg to our build runner, so we can create a dir easily
during the build on Windows.
- Simplified the preference handling code, by wrapping the preferences
in an outer store, instead of a separate store for each individual
preference. This means a change to one preference will trigger a redraw
of all components that depend on the preference store, but the redrawing
is cheap after moving the data processing to Rust, and it makes the code
easier to follow.
- Drop async(Reactive).ts in favour of more explicit handling with await
blocks/updating.
- Renamed add_inputs_to_group() -> add_dependency(), and fixed it not adding
dependencies to parent groups. Renamed add() -> add_action() for clarity.
* Remove a couple of unused proto imports
* Migrate card info
* Migrate congrats, image occlusion, and tag editor
+ Fix imports for multi-word proto files.
* Migrate change-notetype
* Migrate deck options
* Bump target to es2020; simplify ts lib list
Have used caniuse.com to confirm Chromium 77, iOS 14.5 and the Chrome
on Android support the full es2017-es2020 features.
* Migrate import-csv
* Migrate i18n and fix missing output types in .js
* Migrate custom scheduling, and remove protobuf.js
To mostly maintain our old API contract, we make use of protobuf-es's
ability to convert to JSON, which follows the same format as protobuf.js
did. It doesn't cover all case: users who were previously changing the
variant of a type will need to update their code, as assigning to a new
variant no longer automatically removes the old one, which will cause an
error when we try to convert back from JSON. But I suspect the large majority
of users are adjusting the current variant rather than creating a new one,
and this saves us having to write proxy wrappers, so it seems like a
reasonable compromise.
One other change I made at the same time was to rename value->kind for
the oneofs in our custom study protos, as 'value' was easily confused
with the 'case/value' output that protobuf-es has.
With protobuf.js codegen removed, touching a proto file and invoking
./run drops from about 8s to 6s.
This closes#2043.
* Allow tree-shaking on protobuf types
* Display backend error messages in our ts alert()
* Make sourcemap generation opt-in for ts-run
Considerably slows down build, and not used most of the time.
* Skip linting target folder
Contains build files not passing the copyright header check.
* Implicitly clear duplicate keys when serializing
Fixes `originalStockKind` not being cleared from `other`, as it had
mistakenly been added to the field list for `NoteFieldSchema11`.
* Migrate check_copyright to Rust
* Add a new lint to check accidental usages of /// in ts/svelte comments
* Fix a bunch of incorrect jdoc comments
* Move contributor check into minilints
Will allow users to detect the issue locally with './ninja check'
before pushing to CI.
* Make Cargo.toml consistent with other crates
