Commit graph

359 commits

Author SHA1 Message Date
Damien Elmes
258a8282d4 move .reopen() to mw; fix exporting 2020-03-20 21:15:23 +10:00
Damien Elmes
a02c4ed994 drop the DB progress handler code
This code was an awful hack to provide some semblance of UI
responsiveness while executing DB statements on the main thread.
Instead, we can just run DB statements in a background thread now,
keeping the UI responsive.
2020-03-20 21:15:23 +10:00
Damien Elmes
8cb3ed7435 drop lock() and setAutocommit()
We no longer need to worry about pysqlite implicitly beginning
transactions, and can be more explicit about beginning/ending
transactions

save() now also has a trx argument controlling whether a
transaction should be started / left open
2020-03-20 21:15:23 +10:00
Damien Elmes
3256c74409 pass weakref in from storage to fix type checking/code completion 2020-03-20 21:15:23 +10:00
Damien Elmes
116f7fa3f1 support first_row_only 2020-03-20 21:15:23 +10:00
Damien Elmes
7e5f713f05 proper implementation of executemany(); drop executescript() 2020-03-20 21:15:23 +10:00
Damien Elmes
3f3a45e2ba add Collection struct, and get media check working again
- media check no longer needs collection to be closed
- use savepoints for operations initiated by Rust, so they are
atomic without forcing a commit
2020-03-20 21:15:23 +10:00
Damien Elmes
700d713f80 add begin/commit/rollback, and support creating collections
all but one unit test is now passing
2020-03-20 21:15:23 +10:00
Damien Elmes
0147d9e470 drop log= argument from Collection 2020-03-20 21:15:23 +10:00
Damien Elmes
70c7cde2cd remove remaining db kwargs 2020-03-20 21:15:23 +10:00
Damien Elmes
d42df6e0f1 add a temporary executemany() 2020-03-20 21:15:23 +10:00
Damien Elmes
bd7ded18c3 drop the protobuf prototype 2020-03-20 21:15:23 +10:00
Damien Elmes
38d4ea5311 use (or)json for DB bridge
Some initial testing with orjson indicates performance varies from
slightly better than pysqlite to about 2x slower depending on the type
of query.

Performance could be improved by building the Python list in rspy
instead of sending back json that needs to be decoded, but it may make
more sense to rewrite the hotspots in Rust instead. More testing is
required in any case.
2020-03-20 21:15:23 +10:00
Damien Elmes
a129b8f8cf initial work on DB handling in Rust
committing the Protobuf implementation for posterity, but will replace
it with json, as Protobuf measures about 6x slower for some workloads
like 'select * from notes'
2020-03-20 21:15:23 +10:00
Damien Elmes
1db876ba1c tweak db type hints 2020-03-20 21:15:23 +10:00
Damien Elmes
7fbc229dcd drop usage of pysqlite Cursor 2020-03-20 21:15:23 +10:00
Damien Elmes
e5a465cccb tweak layout of db methods 2020-03-20 21:15:23 +10:00
Damien Elmes
7bf27febe6 drop .cursor() 2020-03-20 21:15:23 +10:00
Damien Elmes
fb9db72dde use total_changes() sqlite func 2020-03-20 21:15:23 +10:00
Damien Elmes
45bc1cd73e drop the schema <11 upgrade code 2020-03-20 21:15:23 +10:00
Damien Elmes
fcfe1bd66c drop context manager 2020-03-20 21:15:23 +10:00
Damien Elmes
9e67ea736a drop named sql arguments 2020-03-20 21:15:23 +10:00
Damien Elmes
6d0c84a1e9 drop progress handler and timeout arg 2020-03-20 21:15:23 +10:00
Damien Elmes
45399df30d drop echo and text factory 2020-03-20 21:15:23 +10:00
Damien Elmes
923d85f66b clone db.py into dbproxy.py 2020-03-20 21:15:23 +10:00
evandrocoan
5d75a0c9f1 Fixed moved comment line on qt/aqt/main.py, long variables names on
pylib/Makefile and qt/Makefile, and rg --version not going to /dev/null
on .github/scripts/trailing-newlines.sh
2020-03-18 11:36:50 -03:00
evandrocoan
51477a4033 Fixed mypy and pylint not building on Windows
1. error: unused 'type: ignore' comment
2. Module 'pywintypes' has no 'error' member
3. Module 'win32pipe' has no 'SetNamedPipeHandleState' member,
   but source is unavailable. Consider adding this module to
   extension-pkg-whitelist if you want to perform analysis based on
   run-time introspection of living objects.
2020-03-17 19:48:16 -03:00
evandrocoan
105e2a2502 Fixed make check for pylint not working on Windows due
F:\Python\python.exe: can't open file '/cygdrive/f/anki/pyenv/Scripts/pylint': [Errno 2] No such file or directory
2020-03-17 19:48:16 -03:00
evandrocoan
7d976249db Fixed test_importing.py unit tests failing on windows with
PermissionError: [Errno 13] Permission denied
2020-03-17 12:49:36 -03:00
evandrocoan
f5b02423c1 Fixed UnicodeEncodeError: 'charmap' codec can't encode character
'\u2068' in position 2724: character maps to <undefined> for
pylib/tests/test_stats.py
2020-03-17 12:49:25 -03:00
evandrocoan
d6fecea4ff Fixed BeautifulSoup breaking string paths
https://anki.tenderapp.com/discussions/ankidesktop/39543-anki-is-replacing-the-character-by-when-i-exit-the-html-edit-mode-ctrlshiftx
2020-03-12 21:19:09 -03:00
Damien Elmes
a5021ee92e Merge pull request #501 from evandroforks/fix_beautiful_soup
Renamed BeautifulSoup import name for consistency with other usages of it
2020-03-11 15:55:15 +10:00
Damien Elmes
9a9265f113 don't error when an add-on passes None to CardStats
https://anki.tenderapp.com/discussions/beta-testing/1842-syncing-message
2020-03-11 15:18:28 +10:00
Damien Elmes
a216fbedbc Spanish is 'es' on disk 2020-03-11 14:53:19 +10:00
Damien Elmes
ea094a60da Revert "es-ES -> es"
This reverts commit 7409deae34.

This change broke Anki:

https://anki.tenderapp.com/discussions/ankidesktop/39589-lang_to_disk_lang-startup-error
2020-03-11 14:52:35 +10:00
evandrocoan
52599f364e Renamed _decode_htmlescapes s variable 2020-03-10 13:57:09 -03:00
Damien Elmes
404a8ca3ec add switch in prefs for new timezone handling code; bump sync version
AD devs: before updating the sync version to 10, please make sure
the new timezone code has been ported over. The core change to the
scheduler is in _updateCutoff():

b198ba8073/pylib/anki/schedv2.py (L1357)

with the following supporting Python functions:

b198ba8073/pylib/anki/schedv2.py (L1410)

 _timing_today() is calculated in Rust (lines up to 92, the rest are
tests):

b198ba8073/rslib/src/sched/cutoff.rs (L20)

The change went through a few iterations before stabilising, so it's
probably easier to refer to the above code than the patches that got
us to that point.
2020-03-10 17:50:18 +10:00
Damien Elmes
e512199ace add restore media action 2020-03-10 13:35:09 +10:00
Damien Elmes
cac9ccfa0d add empty trash action in aqt 2020-03-10 12:49:40 +10:00
Damien Elmes
0dc4886558 add col.weakref() and fix more reference cycles in pylib 2020-03-05 09:47:47 +10:00
Damien Elmes
0589573dfa fix retain cycle caused by col.tr
We can't use a weakref.proxy with a method, so manually proxy the
call instead.
2020-03-05 09:45:51 +10:00
Arthur Milchior
d9eb170962 hooks for limit new/rev for a single deck 2020-03-04 16:11:13 +01:00
Arthur Milchior
bf0e5f0d6e single return in sched2 _deckRevLimitSingle 2020-03-04 16:07:15 +01:00
Damien Elmes
2343ddcb02 add diffing script 2020-03-04 21:34:18 +10:00
Damien Elmes
950d39571e remove a few more common methods 2020-03-04 21:33:44 +10:00
Damien Elmes
27438d8625 use _deckLimit() in a few more places 2020-03-04 21:24:09 +10:00
Damien Elmes
db00452ded reuse _leftToday() 2020-03-04 12:26:57 +10:00
Damien Elmes
7a32a0eb6f import tidyup 2020-03-04 12:25:02 +10:00
Damien Elmes
d7c3c5d3b1 reuse _deckNewLimitSingle() 2020-03-04 12:23:56 +10:00
Damien Elmes
7bd1c59abd reuse v2's _logRev() 2020-03-04 12:23:42 +10:00
Damien Elmes
7880227b43 remove duplicate methods from sched.py 2020-03-04 12:14:26 +10:00
Damien Elmes
966cfaec84 add some type hints that were only in the v1 scheduler 2020-03-04 12:10:38 +10:00
Damien Elmes
7ab3c4ce67 make v1 scheduler inherit from v2 so we can delete duplicate code 2020-03-04 11:41:23 +10:00
Damien Elmes
28e1e01068 use weakref for backrefs so collection doesn't need to be garbage collected 2020-03-03 21:25:32 +10:00
Damien Elmes
475f8e7f04 fix backend not deallocating due to circular reference 2020-03-03 21:12:38 +10:00
Damien Elmes
e592ac06ea Merge pull request #478 from Arthur-Milchior/missing_constant
countsIdx return a queue type.
2020-03-02 15:43:31 +10:00
Alan Du
63a0b36d91 Monkeytype qt/aqt/reviewer.py 2020-03-01 10:16:08 -05:00
Arthur Milchior
80e5a97e39 countsIdx return a queue type.
I missed this constant before
2020-03-01 13:00:36 +01:00
Damien Elmes
ba97457e8a add nfd tag fix to db check
we already normalize in the editor and importing, so perhaps these
tags were from an old version

https://anki.tenderapp.com/discussions/ankidesktop/39120-bug-filter-do-not-work-with-tags-which-contain-german-letters-like
2020-03-01 11:21:25 +10:00
Damien Elmes
19ef578556 Merge pull request #473 from evandroforks/fix_docs_urllink
Fixed anki website docs link on pylib/anki/consts.py
2020-02-29 21:39:11 +10:00
Damien Elmes
c97b37b7ae make logging explicit, and support logging to a file 2020-02-29 20:44:01 +10:00
evandrocoan
bc5a9da396 Fixed anki website docs link on pylib/anki/consts.py 2020-02-28 22:18:19 -03:00
Alan Du
13a258184e Add some more types 2020-02-27 21:27:59 -05:00
Alan Du
ab96ac324e Monkeytype pylib/anki/importing/supermemo_xml.py 2020-02-27 19:27:32 -05:00
Alan Du
e132ef35ee Type pylib/anki/schedv2.py 2020-02-27 19:27:32 -05:00
Alan Du
793877ce77 Monkeytype anki/rsbackend.py 2020-02-27 19:27:32 -05:00
Alan Du
3c2822ba4c Monkeytype pylib/anki/sched.py 2020-02-27 19:27:32 -05:00
Alan Du
397afaf29b Monkeytype pylib/anki/find.py 2020-02-27 19:27:32 -05:00
Damien Elmes
2de865d7f0 migrate strings with trailing newlines to fluent 2020-02-27 20:36:40 +10:00
Alan Du
9a5eb6c704 Fix type warning 2020-02-27 00:07:19 -05:00
Alan Du
1523412a37 Monkeytype pylib/anki/latex.py 2020-02-27 00:07:19 -05:00
Alan Du
6076e0b21a Monkeytype pylib/anki/stdmodels.py 2020-02-27 00:07:19 -05:00
Alan Du
c74ccbe245 Monkeytype pylib/anki/stats.py 2020-02-27 00:07:19 -05:00
Alan Du
fe2d8cdea0 Monkeytype pylib/anki/template.py 2020-02-27 00:07:19 -05:00
Alan Du
4606447e4c Monkeytype anki/tags.py 2020-02-27 00:07:19 -05:00
Alan Du
7ce81f80db Monkeytype pylib/anki/errors.py 2020-02-27 00:07:19 -05:00
Alan Du
acf1b00cd2 Monkeytype pylib/anki/db.py 2020-02-27 00:07:19 -05:00
Alan Du
a96f1abb33 Monkeytype pylib/anki/decks.py 2020-02-27 00:07:19 -05:00
evandrocoan
59d45f309a Merge remote-tracking branch 'danielelmes/master' into fix_windows_build
# Conflicts:
#	CONTRIBUTORS
2020-02-27 00:54:16 -03:00
Alan Du
a095c44f40 Improve mypy config
- Make mypy stricter
- Warn for unnecessary workarounds
2020-02-26 22:22:40 -05:00
evandrocoan
4a2456f62f Merge remote-tracking branch 'danielelmes/master' into fix_windows_build
# Conflicts:
#	Makefile
2020-02-27 00:20:34 -03:00
Damien Elmes
5d09981667 add self.col.tr() shortcut 2020-02-27 12:33:15 +10:00
Damien Elmes
93c2e04267 FString -> TR 2020-02-27 12:25:19 +10:00
Damien Elmes
83d894ac87 return a localized error for all error kinds
some errors are not yet localized, but now the Python code
doesn't need to think about which property to use
2020-02-27 10:46:24 +10:00
Damien Elmes
59328d9ade tweak rounding
- avoid rounding minutes
- round the seconds taken in the card info screen
- provide different precise and imprecise modes, since we need
to display seconds to multiple decimals in some areas
2020-02-25 17:29:06 +10:00
evandrocoan
2d316748a7 Merge remote-tracking branch 'danielelmes/master' into fix_windows_build
# Conflicts:
#	Makefile
2020-02-25 03:16:57 -03:00
evandrocoan
9e690f80e7 Renamed FIND_EXEC to FIND, replaced IS_WINDOWS by specific commands
as PYTHON_BIN, ACTIVE_SCRIPT and INSTALL_PYAUDIO.
Fixed echo statements not using @ to suppress double message output.
Deprecated the usage of ECHOCMD := /bin/echo -e because it has no
effect: https://stackoverflow.com/questions/60387684/how-to-make-the-makefile-echos-to-use-bin-echo-e

# Conflicts:
#	Makefile
2020-02-25 03:15:11 -03:00
Damien Elmes
64851afa5e cap answer buttons to 1 decimal place
we can switch to NUMBER() instead in the future, but will need
to update all the translations at the same time
2020-02-25 13:24:29 +10:00
evandrocoan
6d16298254 Fixed development compilation with Windows 10
1. Documented on README.development how to setup the environment
   for Windows.
2. Fixed qt/ts/package.json not working due usage of ; instead
   of &&
3. Fixed copy-qt-files rsync using Windows paths instead of Unix
   ones
4. Fixed Makefile's using Windows Linux Subsystem bash instead of
    the Cygwin one.
5. Ensured running the correct pip module by using python -m pip
    instead of just pip.
6. Fixed Makefiles using Windows `find` command, instead of the
    Cygwin's one (POSIX find).
7. Fixed pyenv sourcing/activate using /pyevn/bin/ instead of
    /python/Scripts/ on Windows.
8. Fixed pyaudio not installing/linking with portaudio on Windows
    by installing for a patched fork at evandroforks/pyaudio
9. Forked and fixed portaudio not building with Visual Studio 2017
    or superior and added the reference for the patched fork on
    README.development at evandroforks/portaudio.
2020-02-24 14:59:11 -03:00
Damien Elmes
43dae5c90e Merge pull request #456 from Arthur-Milchior/noDefaultDeckInBrowser
No default deck in browser
2020-02-24 19:35:49 +10:00
Damien Elmes
db486dd25c enable v2 apkg export 2020-02-24 18:54:55 +10:00
Damien Elmes
ea22296d4c expand clozes before rendering LaTeX
https://anki.tenderapp.com/discussions/ankidesktop/39071-latex-media-windows-version-2121-0cd1ee92-fail-rendering-latex
2020-02-24 18:34:08 +10:00
Damien Elmes
70ae99778f remove old fmtTimeSpan() and associated strings 2020-02-23 18:47:16 +10:00
Damien Elmes
024126b3be support aqt-specific FTL files 2020-02-23 18:00:13 +10:00
Damien Elmes
c98247fa0f translations no longer require an open collection 2020-02-23 14:57:02 +10:00
Damien Elmes
0d1a30c6d8 rework Fluent handling
- all .ftl files for a language are concatenated into a single file
at build time
- all languages are included in the binary
- external ftl files placed in the ftl folder can override the
built-in definitions
- constants are automatically generated for each string key
- dropped the separate StringsGroup enum
2020-02-23 13:22:50 +10:00
Arthur Milchior
adfbd0d70f should(default)deckbedisplayed in snake case 2020-02-22 04:08:01 -08:00
Arthur Milchior
be98614d41 defaultDeck to default_deck 2020-02-22 04:03:42 -08:00
Arthur Milchior
6a50730f0d Remove options for all/allNames
Instead, we always see default deck if it has a child
As indicated in https://github.com/ankitects/anki/pull/452
2020-02-22 04:03:42 -08:00
Arthur Milchior
89b79d9800 forceDefault to force_default 2020-02-22 04:01:42 -08:00