Commit graph

124 commits

Author SHA1 Message Date
Aleksa Sarai
da040372da rslib: disable bzip2 feature
We can't link against libzip2.so.1 when building manylinux wheels, but
the only dependency which required this was "zip" and we don't appear to
make use of bzip2 in Anki.

This fixes the following "make build" error on Linux:

  x maturin failed
    Caused by: Failed to ensure manylinux compliance
    Caused by: Your library is not manylinux compliant because it links the following forbidden libraries: ["libbz2.so.1"]

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
2020-09-05 19:39:11 +10:00
Aleksa Sarai
96da137314 rs: remove most &mut references in BackendService
The previous implementation had some slightly questionable memory safety
properties (older versions of PyO3 didn't uphold the Rust aliasing rules
and would thus create multiple &mut references to #[pyclass] objects).
This explains why Backend has internal Mutex<T>s even though all of its
methods took &mut self.

The solution is to simply make all methods take &self, which luckily
doesn't pose too make issues -- most of the code inside Backend already
has sufficient locking. The only two things which needed to be
explicitly handled where:

1. "self.runtime" which was fairly easy to handle. All usages of
   the Runtime only require an immutable reference to create a new
   Handle, so we could switch to OnceCell which provides
   lazy-initialisation semantics without needing a more heavy-handed
   Mutex<tokio::runtime::Handle>.

2. "self.sync_abort" was simply wrapped in a Mutex<>, though some of the
   odd semantics of sync_abort (not being able to handle multiple
   processes synchronising at the same time) become pretty obvious with
   this change (for now we just log a warning in that case). In
   addition, switch to an RAII-style guard to make sure we don't forget
   to clear the abort_handle.

As a result, we now no longer break Rust's aliasing rules and we can
build with newer versions of PyO3 which have runtime checks for these
things (and build on stable Rust).

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
2020-09-05 19:37:59 +10:00
Damien Elmes
f3c6d565b0 bump version 2020-08-31 11:07:54 +10:00
Damien Elmes
ed1af61fa6 use commit hash in reqwests dependency 2020-08-26 18:50:25 +10:00
Damien Elmes
2e923b0990 vendor reqwest for now, to support read/write timeouts
https://github.com/seanmonstar/reqwest/issues/870

adding back now that fix has been applied to a fork of tokio-io-timeout
2020-08-26 13:24:51 +10:00
Damien Elmes
2f6d816cdc bump version 2020-08-26 08:05:45 +10:00
Damien Elmes
a6f5d48cca Revert "vendor reqwest for now, to support read/write timeouts"
This reverts commit b3edeaac86.

The separate read and write timeouts were causing syncs to fail - a
long full upload would time out due to no read activity for example.
Reverting until a better fix can be put in place.
2020-08-25 13:28:54 +10:00
Damien Elmes
58c8609332 bump version 2020-08-24 11:28:22 +10:00
Damien Elmes
b3edeaac86 vendor reqwest for now, to support read/write timeouts
https://github.com/seanmonstar/reqwest/issues/870
2020-08-21 18:50:00 +10:00
Damien Elmes
4629dd38fb bump version 2020-08-09 13:44:59 +10:00
Damien Elmes
d3653ab6d5 bump version 2020-07-28 12:59:56 +10:00
Damien Elmes
7e78d12b99 bump version 2020-07-21 15:54:14 +10:00
Damien Elmes
56833f0ac7 panic fix was merged into hyper 0.13.7 2020-07-16 09:09:13 +10:00
Damien Elmes
22a87a4f3b vendor coarsetime so we can get the use after free fix 2020-07-13 13:34:37 +10:00
Damien Elmes
bbc75cdf5d update rslib/rspy deps 2020-07-12 11:38:25 +10:00
Damien Elmes
f5694a7e4a migrate card stats to backend
Currently this renders the HTML directly like the previous Python
implementation - doing it in JS would probably make more sense in the
future.
2020-06-15 17:22:16 +10:00
Damien Elmes
4092da9dbe use vendored hyper for panic fix 2020-06-11 13:30:35 +10:00
Damien Elmes
6207b1012f Revert "use trust-dns in reqwest"
This reverts commit f15abb5c58.

trust-dns has introduced its own issues, such as when resolv.conf is
inaccessible, or contains hostnames with certain characters
2020-06-08 22:03:24 +10:00
Damien Elmes
f15abb5c58 use trust-dns in reqwest
The default DNS resolution, if interrupted by an Abortable, will
cause a panic:

57ef271500/src/client/connect/dns.rs (L144)
2020-06-03 15:47:54 +10:00
Damien Elmes
42c34b60c0 use a shared async runtime instead of recreating each time 2020-06-02 09:35:27 +10:00
Damien Elmes
4d1e8d678d full upload/download HTTP code 2020-06-01 13:57:10 +10:00
Damien Elmes
f17dec9f2a start on HTTP client for syncing 2020-06-01 13:57:10 +10:00
Damien Elmes
3e156fdb49 bump version 2020-05-18 12:50:20 +10:00
Damien Elmes
17efa85d75 update rusqlite 2020-05-16 19:49:08 +10:00
Damien Elmes
d810ba5db9 template changes and card generation
Cloze cards are not yet supported, missing decks are not handled,
and more testing is still required.
2020-05-12 21:13:33 +10:00
Damien Elmes
4a2256aecb the start of note adding, and note type changes 2020-05-12 21:13:33 +10:00
Damien Elmes
f2b1796e22 update crate version
make check was missing this
2020-05-11 20:41:16 +10:00
Damien Elmes
2c102f87c4 update rslib version 2020-05-02 10:46:58 +10:00
Damien Elmes
91248aa8cf bump version 2020-04-28 08:57:56 +10:00
Damien Elmes
9b66ecd12d add socks support 2020-04-09 14:28:36 +10:00
Damien Elmes
76ecc9739e update rusqlite 2020-04-09 12:45:51 +10:00
Damien Elmes
68142ed32e Cow<str> workaround no longer required 2020-04-07 10:08:03 +10:00
Damien Elmes
b0ea1c24d0 update rusqlite for param count assert fix 2020-04-07 08:54:57 +10:00
Damien Elmes
ac54f85840 update tag handling
- tag list stored in a separate DB table
- non-wildcard searches now do full unicode case folding
(eg tag:masse matches 'Maße')
- wildcard matches do simple unicode case folding
- some functions haven't been updated yet, so ascii folding will
continue to be used in some operations
2020-04-03 19:34:46 +10:00
Damien Elmes
83c50488fd use support URL and point to source repo 2020-04-03 08:44:59 +10:00
evandrocoan
30107ab09d Created the GitHub Actions step Upload to PyPi to run only on
tagged releases of anki:
https://anki.tenderapp.com/discussions/ankidesktop/40025-anki-in-python-package-index
2020-04-02 02:57:34 -03:00
Damien Elmes
284ee0860b allow aborting a media sync while http req in flight
The progress handling code needs a rethink, as we now have two separate
ways to flag that the media sync should abort. In the future, it may
make sense to switch to polling the backend for progress, instead of
passing a callback in.
2020-04-01 09:49:25 +10:00
Damien Elmes
c4a78ffd82 roll back the vec cache changes
prepare_cached() is sufficiently fast, and allowing the vec cache
to persist across calls is complicated due to lifetime restrictions
2020-03-29 09:26:09 +10:00
Damien Elmes
cb03c87f8b Revert "add benchmark for vec cache"
This reverts commit 0247d65ccd.
2020-03-29 09:16:11 +10:00
Damien Elmes
0247d65ccd add benchmark for vec cache
test storage::sqlite::bench::bench_hash_cache ... bench:         399 ns/iter (+/- 27)
test storage::sqlite::bench::bench_no_cache   ... bench:       4,854 ns/iter (+/- 499)
test storage::sqlite::bench::bench_vec_cache  ... bench:           0 ns/iter (+/- 0)
2020-03-27 09:59:48 +10:00
Damien Elmes
d1c44ec06d store cached statements in a vec instead of separate optionals 2020-03-27 09:51:19 +10:00
Damien Elmes
6aced5f125 handle fields with embedded nuls 2020-03-24 14:53:33 +10:00
Damien Elmes
982e1a811d add unicase collation
sqlite's like is hard-coded to use ASCII comparisons, so we can't
take advantage of this yet
2020-03-21 12:40:20 +10:00
Damien Elmes
eef158b20c add field_at_index() sql func 2020-03-20 21:15:23 +10:00
Damien Elmes
3b6fef5613 add card queue/type enums 2020-03-20 21:15:23 +10:00
Damien Elmes
0d9ea33389 bump version 2020-03-19 10:46:01 +10:00
Damien Elmes
f6d5d4ea26 rspy version 2020-03-06 19:15:02 +10:00
Damien Elmes
0aa0c42d20 update to failure 0.1.7 2020-03-06 15:08:58 +10:00
Damien Elmes
0b4a7c1d10 work around bug in failure crate 2020-03-05 11:06:44 +10:00
Damien Elmes
c97b37b7ae make logging explicit, and support logging to a file 2020-02-29 20:44:01 +10:00
Alan Du
a6751c893b Update blake3 to 0.2 2020-02-26 22:50:48 -05:00
Alan Du
a6441dd4e0 Update prost-build version to 0.6
This is the same version as Prost, and lets us reduce the number of deps
we have to compile
2020-02-26 22:48:16 -05:00
Damien Elmes
1f6e7805ec add workaround for panic on 32 bit systems
https://anki.tenderapp.com/discussions/beta-testing/1817-failed-to-generate-operands-out-of-fluentnumber-could-not-convert-string-to-integer
2020-02-25 13:22:41 +10:00
Damien Elmes
0d1a30c6d8 rework Fluent handling
- all .ftl files for a language are concatenated into a single file
at build time
- all languages are included in the binary
- external ftl files placed in the ftl folder can override the
built-in definitions
- constants are automatically generated for each string key
- dropped the separate StringsGroup enum
2020-02-23 13:22:50 +10:00
Damien Elmes
ecc055d2f4 fluent-rs git no longer required 2020-02-21 21:30:11 +10:00
Damien Elmes
b851577063 fix revision pin 2020-02-21 15:29:38 +10:00
Damien Elmes
cf1824dac5 update to latest fluent-rs and add basic locale-aware decimals
- git version pinned at the moment until the concurrency fix
lands in 0.10.2
- currently float values are hard-coded at 2 decimal places;
we should switch to using NUMBER() in the future
2020-02-21 15:29:38 +10:00
Damien Elmes
0cd1ee92ee pin coarsetime for now, as .12 requires Sierra 2020-02-17 17:55:39 +10:00
Damien Elmes
43005f00a5 bump version 2020-02-17 08:40:17 +10:00
Damien Elmes
36d2536262 add i18n support to Rust code using Fluent 2020-02-17 08:40:17 +10:00
Damien Elmes
92b76a5ba7 gather field references in Rust; media check now mostly complete 2020-02-17 08:40:17 +10:00
Damien Elmes
a0680553ea use our own trash folder instead of using the system trash
the trash crate was invoking external commands on Macs and Linux
which is slow and likely to fall over if a large number of files need
to be deleted at once.
2020-02-17 08:40:17 +10:00
Damien Elmes
9c60f2c77e log media DB change registration progress, and allow aborting it 2020-02-17 08:40:17 +10:00
Damien Elmes
a769fb6172 update rslib version automatically 2020-02-17 08:40:17 +10:00
Damien Elmes
2c27c11584 comply with manylinux wheel requirements, and vendor sqlite on Windows 2020-02-17 08:40:17 +10:00
Damien Elmes
e26ab9073b send removed files to the trash
The way the trash crate implements deletion on a Mac is ugly, and
we may need to look into alternatives.
2020-02-17 08:40:16 +10:00
Damien Elmes
78bfbef863 media sync working, but unpolished 2020-02-17 08:40:16 +10:00
Damien Elmes
6bf4b9e83e port change tracking 2020-02-17 08:40:16 +10:00
Damien Elmes
9bc38647e6 initial rs implementation of media database 2020-02-17 08:40:16 +10:00
Damien Elmes
9ec6d26db7 add another implementation of media.addFile() and cleanFilename() et al
Instead of adding an incrementing number in the conflict case,
the file hash is appended.
2020-02-17 08:40:16 +10:00
Damien Elmes
548ed31ef7 add a simple TTS field filter
This simply wraps the field in extra text that the frontend will
deal with. Also added some helpers for extracting and stripping
audio and TTS tags from the rendered text.
2020-01-20 19:16:24 +10:00
Damien Elmes
d671f091ff type: and hint: support
We may need to keep handling hints in the Python code for now until
i18n is sorted out.
2020-01-11 09:35:41 +10:00
Damien Elmes
8f3df55da2 flatten->render; pass field content in
This is paving the way to move the standard filters into Rust.

Non-empty fields are now determined in Rust, using a single regex
instead of the overkill stripHTMLMedia(). The old implementation
has been moved into the Pystache code for now.
2020-01-10 14:59:29 +10:00
Damien Elmes
2a00e0a6b0 tweaking the folder names again
hopefully that's the last of it
2020-01-03 07:48:38 +10:00
Renamed from lib-rust/Cargo.toml (Browse further)