Anki/rslib/src
Damien Elmes 778e02415b Move more of the graph processing into the backend
The existing architecture serializes all cards and revlog entries in
the search range into a protobuf message, which the web frontend needs
to decode and then process. The thinking at the time was that this would
make it easier for add-ons to add extra graphs, but in the ~2.5 years
since the new graphs were introduced, no add-ons appear to have taken
advantage of it.

The cards and revlog entries can grow quite large on large collections -
on a collection I tested with approximately 2.5M reviews, the serialized
data is about 110MB, which is a lot to have to deserialize in JavaScript.

This commit shifts the preliminary processing of the data to the Rust end,
which means the data is able to be processed faster, and less needs to
be sent to the frontend. On the test collection above, this reduces the
serialized data from about 110MB to about 160KB, resulting in a more
than 2x performance improvement, and reducing frontend memory usage from
about 400MB to about 40MB.

This also makes #2043 more feasible - while it is still about 50-100%
slower than protobufjs, with the much smaller message size, the difference
is only about 10ms.
2022-12-16 21:42:17 +10:00
..
backend Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
card Move away from Bazel (#2202) 2022-11-27 15:24:20 +10:00
card_rendering Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
collection Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
config Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
deckconfig Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
decks Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
error Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
import_export Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
media Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
notes Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
notetype Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
revlog Update to latest rules_rust and Rust 1.64 2022-09-24 11:12:58 +10:00
scheduler Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
search Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
stats Move more of the graph processing into the backend 2022-12-16 21:42:17 +10:00
storage Move more of the graph processing into the backend 2022-12-16 21:42:17 +10:00
sync Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
tags Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
undo Refactor error handling (#2136) 2022-10-21 18:02:12 +10:00
adding.rs Refactor error handling (#2136) 2022-10-21 18:02:12 +10:00
browser_table.rs Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
cloze.rs Expose cloze text as HTML attribute on question side (#1968) 2022-07-19 04:22:57 +10:00
dbcheck.rs Refactor error handling (#2136) 2022-10-21 18:02:12 +10:00
findreplace.rs Remove redundant error mapping (#1593) 2022-01-16 13:46:27 +10:00
i18n.rs drop the legacy enum from rslib, and pass separate module/message idx 2021-03-27 11:56:31 +10:00
io.rs Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
latex.rs Update to latest rules_rust and Rust 1.64 2022-09-24 11:12:58 +10:00
lib.rs Partially migrate type answer to backend 2022-07-22 19:28:34 +10:00
links.rs Expose backend_proto publicly for AnkiDroid, and rename to pb 2022-06-27 15:27:53 +10:00
log.rs Backups (#1685) 2022-03-07 15:11:31 +10:00
markdown.rs render deck description with markdown; strip images 2021-02-06 15:02:40 +10:00
ops.rs Update to latest rules_rust and Rust 1.64 2022-09-24 11:12:58 +10:00
pb.rs Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
preferences.rs Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
prelude.rs Refactor error handling (#2136) 2022-10-21 18:02:12 +10:00
serde.rs Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
template.rs Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
template_filters.rs Expose cloze text as HTML attribute on question side (#1968) 2022-07-19 04:22:57 +10:00
tests.rs Minor Rust cleanups (#2272) 2022-12-16 21:40:27 +10:00
text.rs Add a small unit test 2022-12-12 14:54:11 +10:00
timestamp.rs Update Chrono Crate (#2242) 2022-12-07 17:00:14 +10:00
typeanswer.rs Update to latest rules_rust and Rust 1.64 2022-09-24 11:12:58 +10:00
types.rs update scheduling ops 2021-04-06 16:38:42 +10:00
version.rs Move away from Bazel (#2202) 2022-11-27 15:24:20 +10:00