peter/searxng
1
0
Fork 0
mirror of https://github.com/searxng/searxng.git synced 2026-07-21 22:13:01 +02:00
Code Issues Packages Projects Releases Wiki Activity
9,624 Commits 7 Branches 0 Tags
master
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexandre FlamentandGitHub 9ed9a9aa53 [fix] searx.network: don't trigger DeprecationWarning (#4845) 
Avoid a confusing warning:

    DeprecationWarning: Setting per-request cookies=<...> is being deprecated

Code based on httpx unit test [1]

[1] https://github.com/encode/httpx/blob/6a99f6f2b3a638719f70200de9983f80d618ee1c/tests/client/test_cookies.py#L123-L137

Closes: https://github.com/searxng/searxng/issues/4833
 2025-05-24 12:40:05 +02:00
Alexandre FlamentandGitHub 7a3742ae56 [mod] upgrade to httpx 0.28.1 (#4674) 2025-05-20 18:18:07 +02:00
Alexandre FlamentandMarkus Heiser 82b9e533b2 [fix] static.build: upgrade min. node version to 18.17 
For sharp [1] dependency / this commit was cherry picked from PR #3157

[1] https://www.npmjs.com/package/sharp
 2025-03-18 06:19:44 +01:00
Alexandre FlamentandMarkus Heiser 96c32549be [fix] requirements-dev.txt: remove autodoc_pydantic 
Related to #3727
 2025-01-05 09:31:03 +01:00
Alexandre FlamentandMarkus Heiser 37d0438f25 Dockerfile: use Alpine 3.20 & Python 3.12 2024-07-14 16:38:30 +02:00
Alexandre FlamentandMarkus Heiser 75e4b65127 [fix] ResultContainer: use self._lock and self._closed for all properties 
Close #3474
 2024-05-24 15:38:03 +02:00
Alexandre FlamentGitHubMarkus Heiser
 		38fdd2288a Drop typing-extensions dependency (#3265) 
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
 2024-03-02 13:10:31 +01:00
Alexandre FlamentandMarkus Heiser ed66ed758d [mod] reduce memory footprint by not calling babel.Locale.parse at runtime 
babel.Locale.parse loads more than 60MB in RAM.  The only purpose is to get:

    LOCALE_NAMES   - searx.data.LOCALES["LOCALE_NAMES"]
    RTL_LOCALES    - searx.data.LOCALES["RTL_LOCALES"]

This commit calls babel.Locale.parse when the translations are update from
weblate and stored in::

    searx/data/locales.json

This file can be build by::

    ./manage data.locales

By store these variables in searx.data when the translations are updated we save
round about 65MB (usually 4 worker = 260MB of RAM saved.

Suggested-by: https://github.com/searxng/searxng/discussions/2633#discussioncomment-8490494
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
 2024-02-20 10:43:20 +01:00
Alexandre FlamentandMarkus Heiser 60bc5baea3 [mod] ./utils/get_setting.py tiny YAML parser for settings.yml 
This allow to read settings on the fly even without virtualenv.  The ultimate
goal of the commit is to remove utils/brand.env from the git repository.

The code includes a tiny yaml parser that **should** be good enough.  The code
read searx/settings.yml directly (and ignore the environment variables).

yq [1] is a more reliable alternative but this require to download a binary from
github which is not great.

[1] https://github.com/mikefarah/yq/#install
 2024-01-09 16:31:19 +01:00
Alexandre FlamentGitHubMarkus Heiser
 		bd3f526859 Docker: add UWSGI_WORKERS and UWSGI_THREAD environment variables (#2992) 
* Docker: add UWSGI_WORKERS and UWSGI_THREAD.

UWSGI_WORKERS specifies the number of process.
UWSGI_THREADS specifies the number of threads.

The Docker convention is to specify the whole configuration
through environment variables. While not done in SearXNG, these two
additional variables allows admins to skip uwsgi.ini

In additional, https://github.com/searxng/preview-environments starts Docker
without additional files through searxng-helm-chat.
Each instance consumes 1Go of RAM which is a lot especially when there are a
lot of instances / pull requests.

* [scripts] add environments UWSGI_WORKERS and UWSGI_THREADS

- UWSGI_WORKERS specifies the number of process.
- UWSGI_THREADS specifies the number of threads.

Templates for uwsgi scripts can be tested by::

    UWSGI_WORKERS=8 UWSGI_THREADS=9 \
      ./utils/searxng.sh --cmd\
      eval "echo \"$(cat utils/templates/etc/uwsgi/*/searxng.ini*)\""\
      | grep "workers\|threads"

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

---------

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-11-12 16:46:34 +00:00
Alexandre Flament 9aeae2142b Add support for Python 3.12 2023-10-29 11:25:58 +01:00
Alexandre Flament fb88721532 Upgrade dependencies 
* geckodriver: 0.33.0
* node 16.20.2
* pyright 1.1.329
* eslint 8.50.0
 2023-10-01 08:25:00 +02:00
Alexandre Flament 19383921d7 /config: expose limiter configuration 2023-09-24 11:01:11 +02:00
Alexandre Flament d07c006aed Replace chompjs with pure Python code 
The new implementation is good enough for the current usage (brave)
 2023-09-09 13:02:36 +02:00
Alexandre FlamentandMarkus Heiser faa4280e1a [mod] bing: resolve redirect without additional requests 
Remove the usage of searx.network.multi_requests
The results from Bing contains the target URL encoded in base64
See the u parameter, remove the first two character "a1", and done.

Also add a comment the check of the result_len / pageno
( from https://github.com/searx/searx/pull/1387 )
 2023-08-29 07:39:06 +02:00
Alexandre Flament 97b1df1629 [mod] searx.network: memory optimization 
Avoid to create a SSLContext in AsyncHTTPTransportNoHttp

See:
* https://github.com/encode/httpx/blob/0f61aa58d66680c239ce43c8cdd453e7dc532bfc/httpx/_transports/default.py#L271
* https://github.com/encode/httpx/issues/2298
 2023-08-27 11:49:40 +02:00
Alexandre Flament e16c007c22 [fix] openstreetmap engine 
It seems there is an API change:
extratags can be either a dictionnary or None.

This commit avoid crash when extratags is None

Test query "!osm gare du nord"
 2023-08-27 11:49:16 +02:00
Alexandre Flamentandmrpaulblack b4e4cfc026 Bump httpx 0.21.2 from to 0.24.1 2023-08-21 22:05:12 +02:00
Alexandre FlamentandMarkus Heiser 4573417b6c uwsgi.ini: remove unused cache2 
cache2 was used before PR #1856
 2023-08-13 07:55:05 +02:00
Alexandre FlamentandGitHub b07e44e552 Update AUTHORS.rst 2023-04-12 14:01:54 +02:00
Alexandre FlamentandGitHub 71bd091f6b Merge pull request #2327 from return42/ui-pref-add-bang 
[mod] in the preference page, show !bang of subgrouping categories
 2023-04-08 11:20:03 +02:00
Alexandre FlamentandGitHub 3841435587 Merge pull request #2274 from searxng/codespaces 
Add GitHub Codespaces configuration
 2023-03-22 08:17:34 +01:00
Alexandre Flament 7b711cb15d Add GitHub Codespaces configuration 2023-03-21 20:58:13 +00:00
Alexandre FlamentandGitHub 3e9cddc606 rollback test 2023-03-15 19:55:20 +01:00
Alexandre FlamentandGitHub 41ed0ef0c7 test 2023-03-15 19:53:53 +01:00
Alexandre FlamentandGitHub abd2d8605e Merge pull request #2231 from return42/fix-docs_prebuild 
[fix] make docs: NameError: name 'DOC' is not defined
 2023-03-02 18:37:25 +01:00
Alexandre FlamentandGitHub 714e83d5ea Merge pull request #2220 from Solirs/gentoo_engine_timeout 
Increase timeout for gentoo wiki engine
 2023-03-01 17:57:21 +01:00
Alexandre FlamentandGitHub 1632f18631 Merge pull request #2227 from searxng/update_data_update_engine_descriptions.py 
Update searx.data - update_engine_descriptions.py
 2023-03-01 17:52:57 +01:00
Alexandre FlamentandGitHub 5bbbb14b62 Merge pull request #2226 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2023-03-01 17:52:02 +01:00
Alexandre FlamentandGitHub 507cdf3fd3 Merge pull request #2223 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2023-03-01 17:51:40 +01:00
Alexandre FlamentandGitHub d669da81fb Merge pull request #2027 from dalf/fix_2018 
Add "auto" as a language.
 2023-02-20 12:17:38 +01:00
Alexandre Flament 6748e8e2d5 Add "Auto-detected" as a language. 
When the user choose "Auto-detected", the choice remains on the following queries.
The detected language is displayed.

For example "Auto-detected (en)":
* the next query language is going to be auto detected
* for the current query, the detected language is English.

This replace the autodetect_search_language plugin.
 2023-02-17 15:17:36 +00:00
Alexandre FlamentandGitHub 675dc04917 Merge pull request #2183 from return42/node.env.dev 
[mod] install developer tools by: make node.env.dev
 2023-02-11 18:30:29 +01:00
Alexandre FlamentandGitHub 9d102fb08f Merge pull request #2132 from dalf/update_pr_1967 
search.suspended_time settings: bug fixes
 2023-01-29 20:48:43 +01:00
Alexandre Flament bfca63c536 wikipedia engine: update _fetch_supported_languages 
the layout https://meta.wikimedia.org/wiki/List_of_Wikipedias has changed
 2023-01-29 10:01:58 +00:00
Alexandre Flament 8256de2fe8 peertube engine: update _fetch_supported_languages 
There is now an API to get the list of supported languages
https://docs.joinpeertube.org/api-rest-reference.html#tag/Video/operation/getLanguages
 2023-01-29 10:01:54 +00:00
Alexandre Flament 37addec69e search.suspended_time settings: bug fixes 
* fix type in settings.yml: replace suspend_times by suspended_times
* always use delay defined in settings.yml:
  * HTTP status 402 and 403: read the value from settings.yml instead of using the hardcoded value of 1 day.
  * startpage engine: CAPTCHA suspend the engine for one day instead of one week
 2023-01-28 10:24:14 +00:00
Alexandre FlamentandGitHub 6b71721ce8 Merge pull request #2106 from searxng/dependabot/pip/master/mock-5.0.1 
Bump mock from 5.0.0 to 5.0.1
 2023-01-27 11:51:42 +01:00
Alexandre FlamentandGitHub 7d676880e2 Merge pull request #2104 from searxng/dependabot/pip/master/yamllint-1.29.0 
Bump yamllint from 1.28.0 to 1.29.0
 2023-01-27 10:44:44 +01:00
Alexandre FlamentandGitHub f6c6de2f9e Merge pull request #2103 from searxng/dependabot/pip/master/pylint-2.15.10 
Bump pylint from 2.15.9 to 2.15.10
 2023-01-27 10:44:14 +01:00
Alexandre FlamentandGitHub 28356f4284 Merge pull request #2115 from searxng/dependabot/pip/master/flask-babel-3.0.0 
Bump flask-babel from 2.0.0 to 3.0.0
 2023-01-27 10:43:17 +01:00
Alexandre FlamentandGitHub a8eb9d15e2 Merge pull request #2122 from dalf/fix_version_again 
SearXNG version: YYYY.MM.DD without leading zero
 2023-01-22 14:31:00 +01:00
Alexandre Flament 06b5068a2e SearXNG version: YYYY.MM.DD without leading zero 2023-01-22 10:45:00 +00:00
Alexandre FlamentandGitHub 674dae946a Merge pull request #2119 from dalf/weblate_codeberg 
weblate: migration to https://translate.codeberg.org/
 2023-01-21 17:01:16 +01:00
Alexandre Flament a9d6f7532a weblate: migration to https://translate.codeberg.org/ 2023-01-21 15:45:12 +00:00
Alexandre FlamentandGitHub 316fcd5e4b Merge pull request #2116 from searxng/translations_update 
Update translations
 2023-01-20 13:11:50 +01:00
Alexandre FlamentandGitHub 934d97b472 Merge pull request #2118 from dalf/fix_version_continuation 
SearXNG version: fix make docker
 2023-01-20 12:31:24 +01:00
Alexandre Flament f759a84af4 SearXNG version: fix make docker 
continuation of #2117
related to #2111

This commit:
* fixes the Docker tag using an additional variable DOCKER_TAG, see searx/version.py
* fixes the Docker labels org.label-schema.vcs-ref and org.opencontainers.image.revision
* adds searx/version_frozen to .gitignore
 2023-01-20 10:46:00 +00:00
Alexandre FlamentandGitHub eed9b09d98 Merge pull request #2117 from dalf/fix_version_number 
Version number follows PEP440 : replace - by +
 2023-01-20 10:17:08 +01:00
Alexandre Flament 0515f2974b Version number follows PEP440 : replace - by + 
Close #2111
 2023-01-20 09:05:44 +00:00
Alexandre FlamentandGitHub 6d72ef3cbe Merge pull request #2109 from ahmad-alkadri/fix/highlight-full-word 
Standalone words highlighting for query result in non-CJK characters
 2023-01-17 23:24:04 +01:00
Alexandre FlamentandGitHub 13b0c251c4 Merge pull request #2100 from nexryai/master 
Add goo engine
 2023-01-15 23:08:28 +01:00
Alexandre FlamentandGitHub 52d4155997 Merge pull request #1967 from tiekoetter/suspended_time-settings 
settings.yml: add search.suspended_times
 2023-01-15 10:07:44 +01:00
Alexandre FlamentandGitHub b720a495f0 Merge pull request #2105 from searxng/dependabot/pip/master/redis-4.4.2 
Bump redis from 4.4.0 to 4.4.2
 2023-01-14 18:28:17 +01:00
Alexandre FlamentandGitHub 449aebaea6 Merge pull request #2107 from searxng/translations_update 
Update translations
 2023-01-14 18:27:54 +01:00
Alexandre FlamentandGitHub 18d895ffed Merge pull request #2097 from immanuelfodor/patch-1 
Adjust uWSGI workers and threads from env
 2023-01-14 16:36:57 +01:00
Alexandre FlamentandGitHub 0d4896170f Merge pull request #2047 from dalf/fasttext-predict 
Use fasttext-predict instead of fasttext(-wheel)
 2022-12-26 20:50:09 +01:00
Alexandre Flament f3515041af Lazy load fasttext-predict 2022-12-26 18:44:23 +00:00
Alexandre Flament ec4237586d Use fasttext-predict instead of fasttext or fasttext-wheel 
See https://github.com/searxng/fasttext-predict
 2022-12-26 18:44:21 +00:00
Alexandre FlamentandGitHub b927482195 Merge pull request #2019 from ArtikusHG/fasttext 
Replace langdetect with fasttext (followup of #1969)
 2022-12-16 21:54:07 +01:00
Alexandre FlamentandGitHub 2a51c85672 Merge pull request #1974 from fehho/query-string-deserial 
[fix] Deserialize preferences which are explicitly blank strings
 2022-12-16 18:56:30 +01:00
Alexandre FlamentandGitHub 9e9f57e48b Merge pull request #1954 from dalf/fix.redis.init.2 
[fix] follow up of PR-1856
 2022-12-14 07:08:19 +01:00
Alexandre Flament 3050e2b6e8 [fix] documentation about update-searxng.rst 2022-12-10 10:06:54 +01:00
Alexandre Flament b971167ced move searx.shared.redisdb to searx.redisdb 2022-12-10 09:26:38 +01:00
Alexandre FlamentandMarkus Heiser 34e260f88f [fix] follow up of PR-1856 
- the environment variable SEARXNG_REDIS_URL overrides the setting value redis.url
- ./manage sets SEARXNG_REDIS_URL to unix:///usr/local/searxng-redis/run/redis.sock if:
   - the socket exists
   - SEARXNG_REDIS_URL is not already defined

Update of PR #1856

Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-12-10 09:26:38 +01:00
Alexandre FlamentandGitHub abb33bd7dd Merge pull request #2016 from searxng/translations_update 
Update translations
 2022-12-09 10:17:35 +01:00
Alexandre FlamentandGitHub 0b1f09faa4 Merge pull request #2013 from searxng/dependabot/pip/master/redis-4.4.0 
Bump redis from 4.3.5 to 4.4.0
 2022-12-09 10:16:47 +01:00
Alexandre FlamentandGitHub 6479b67cd2 Merge pull request #2014 from searxng/dependabot/pip/master/selenium-4.7.2 
Bump selenium from 4.7.0 to 4.7.2
 2022-12-09 10:13:55 +01:00
Alexandre FlamentandGitHub a6d870d5cf Merge pull request #2015 from searxng/dependabot/pip/master/pylint-2.15.8 
Bump pylint from 2.15.7 to 2.15.8
 2022-12-09 10:08:12 +01:00
Alexandre FlamentandGitHub 3df1a9839a Merge pull request #2017 from searxng/dependabot/pip/certifi-2022.12.7 
Bump certifi from 2022.9.24 to 2022.12.7
 2022-12-09 10:07:18 +01:00
Alexandre FlamentandGitHub 67eea86b35 Merge pull request #2006 from dalf/fix_1951 
Fix: don't crash when engine or name is missing in settings.yml
 2022-12-05 13:43:38 +01:00
Alexandre Flament 269326063a Fix: don't crash when engine or name is missing in settings.yml 
SearXNG crashes if the engine or name fields are missing.
With this commit, the app displays an error in the log and keeps loading.

Close #1951
 2022-12-04 23:43:59 +01:00
Alexandre FlamentandGitHub 768659f2bd Merge pull request #1988 from ryankask/fix-redis-conn-logging 
Copy client connection kwargs before mutating
 2022-11-29 19:38:54 +01:00
Alexandre FlamentandGitHub b5371b7a85 Merge pull request #1963 from searxng/translations_update 
Update translations
 2022-11-19 17:26:05 +01:00
Alexandre FlamentandGitHub 1cdadf4bfd Merge pull request #1948 from searxng/dependabot/pip/master/selenium-4.6.0 
Bump selenium from 4.5.0 to 4.6.0
 2022-11-18 14:20:11 +01:00
Alexandre FlamentandGitHub fe8b0472dd Merge pull request #1949 from searxng/translations_update 
Update translations
 2022-11-18 14:19:51 +01:00
Alexandre FlamentandGitHub d3949269a9 Merge pull request #1942 from dalf/fix_redis_initialization 
[FIX] Redis initialization
 2022-11-07 23:16:04 +01:00
Alexandre FlamentandGitHub 8f19bdaf17 Merge pull request #1882 from fehho/metacpan 
Add MetaCPAN engine
 2022-11-07 21:54:11 +01:00
Alexandre FlamentandGitHub ae54c7d525 Merge pull request #1938 from Evhorizon/master 
Update network.py
 2022-11-06 21:20:38 +01:00
Alexandre FlamentandGitHub 2dc5c0e1d6 Merge pull request #1928 from searxng/translations_update 
Update translations
 2022-11-06 11:29:28 +01:00
Alexandre FlamentandGitHub 4fe5463671 Merge pull request #1869 from vgerak/1836-ddg-weather 
Fix floating point format in DDG weather humidity
 2022-11-05 22:46:50 +01:00
Alexandre Flament e473addaff User agent: don't include the patch number in the Firefox version 
The Firefox version in the user agent doesn't include the patch version: 106.0 not 106.0.2

Close #1914
 2022-11-05 22:04:37 +01:00
Alexandre FlamentandGitHub e9f42e1c22 Merge pull request #1918 from LinuxMeow/patch-1 
Remove unused volume
 2022-11-05 19:45:13 +01:00
Alexandre FlamentandGitHub d37afb8ab9 Merge pull request #1856 from dalf/checker_requires_redis 
The checker requires Redis
 2022-11-05 17:50:40 +01:00
Alexandre Flament fe419e355b The checker requires Redis 
Remove the abstraction in searx.shared.SharedDict.
Implement a basic and dedicated scheduler for the checker using a Redis script.
 2022-11-05 12:04:50 +01:00
Alexandre FlamentandGitHub d764d94a70 Merge pull request #1927 from searxng/dependabot/pip/master/babel-2.11.0 
Bump babel from 2.10.3 to 2.11.0
 2022-11-05 11:38:16 +01:00
Alexandre FlamentandGitHub d3a7399ed7 Merge pull request #1929 from dalf/github_workflow_update 
GH workflow: various improvements
 2022-11-05 11:14:09 +01:00
Alexandre FlamentandGitHub 710a3a001f Merge pull request #1873 from searxng/translations_update 
Update translations
 2022-10-21 10:06:32 +02:00
Alexandre FlamentandGitHub 1a5b096578 Merge pull request #1860 from dalf/fix-startpage 
[fix] startpage engine
 2022-10-14 20:40:51 +02:00
Alexandre FlamentandGitHub 096d9defa6 Merge pull request #1847 from searxng/dependabot/pip/master/typing-extensions-4.4.0 
Bump typing-extensions from 4.3.0 to 4.4.0
 2022-10-14 16:26:19 +02:00
Alexandre FlamentandGitHub e2dd5a80bb Merge pull request #1831 from searxng/dependabot/pip/master/sphinx-5.2.3 
Bump sphinx from 5.1.1 to 5.2.3
 2022-10-14 16:25:45 +02:00
Alexandre FlamentandGitHub 4d4dfc58d4 Merge pull request #1858 from searxng/dependabot/pip/master/pylint-2.15.4 
Bump pylint from 2.15.3 to 2.15.4
 2022-10-14 16:24:40 +02:00
Alexandre FlamentandGitHub 2eb817011c Merge pull request #1529 from dalf/network_trust_env 
searx.network: use SSL_CERT_FILE and SSL_CERT_DIR when they are defined
 2022-10-14 16:06:29 +02:00
Alexandre FlamentandAlexandre FLAMENT 32e8c2cf09 searx.network: add "verify" option to the networks 
Each network can define a verify option:
* false to disable certificate verification
* a path to existing certificate.

SearXNG uses SSL_CERT_FILE and SSL_CERT_DIR when they are defined
see https://www.python-httpx.org/environment_variables/#ssl_cert_file
 2022-10-14 13:59:22 +00:00
Alexandre FlamentandGitHub 72f6367e23 Merge pull request #1859 from searxng/translations_update 
Update translations
 2022-10-14 12:18:39 +02:00
Alexandre FlamentandGitHub 666cd1f635 Merge pull request #1846 from searxng/dependabot/pip/master/black-22.10.0 
Bump black from 22.8.0 to 22.10.0
 2022-10-07 10:07:24 +02:00
Alexandre FlamentandGitHub 84f61af839 Merge pull request #1848 from searxng/translations_update 
Update translations
 2022-10-07 10:05:26 +02:00
Alexandre FlamentandGitHub 901143f056 Merge pull request #1830 from MohamedElashri/master 
Use official sci-hub mirrors
 2022-10-01 09:59:07 +02:00
Alexandre FlamentandGitHub 14d0fb2cd4 Merge pull request #1823 from searxng/dependabot/pip/master/selenium-4.5.0 
Bump selenium from 4.4.3 to 4.5.0
 2022-10-01 09:42:11 +02:00
Alexandre FlamentandGitHub e9af772b65 Merge pull request #1826 from searxng/dependabot/pip/master/myst-parser-0.18.1 
Bump myst-parser from 0.18.0 to 0.18.1
 2022-10-01 09:34:36 +02:00