Commit Graph

30 Commits

Author SHA1 Message Date
Mark Felder 14b4bd69a8 Add additional flags to the Pleroma.Search.Indexer Mix task 2024-05-29 10:44:40 -04:00
Lain Soykaf a9be4907c0 SearchBackend: Add drop_index 2024-05-16 10:47:24 +04:00
Lain Soykaf 1261c43a7a SearchBackend: Add create_index 2024-05-14 17:19:36 +04:00
Lain Soykaf cd7e2138d1 Search: Basic Qdrant/Ollama search 2024-05-14 14:13:37 +04:00
Ekaterina Vaartis 5a39866388 Specifically strip mentions for search indexing 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 5ac6763238 Make add_to_index and remove_from_index report errors 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 84608be87e Change updateId to uid because apparently that's the new name 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 1e23f527e3 Change the meilisearch key auth to conform to 0.25.0 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 571533ae26 Don't support meilisearch < 0.24.0, since it breaks things 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 3179ed0921 Make chunk size configurable 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis a12f63bc81 Implement suggestions from the Meilisearch MR
- Index unlisted posts
- Move version check outside of the streaming and only do it once
- Use a PUT request instead of checking manually if there is need to insert
- Add error handling, sort of
2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 39e596a5b5 Style fixes 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 9c1a930707 Support reindexing meilisearch >=0.24.0
It has has a different error code key
2022-10-10 20:19:09 +03:00
Ekaterina Vaartis e928e307f3 Add a reindex option
Signed-off-by: Ekaterina Vaartis <vaartis@kotobank.ch>
2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 40280cc273 Reorder ranking rules for (maybe) better results 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 09a1ae1b6e Add the meilisearch.stats command 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis a67f9da5cc Add a message with a count of posts to index 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 2c7d973af7 Implement meilisearch auth 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 410c8cb765 Make indexing logs rewrite themselves 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 35e9192ced Rework task indexing to share code with the main module
The code in the main module now scrubs new posts too
2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 3dedadf192 Adjust content indexing to skip more unneeded stuff 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 9f16ca80e0 Mark only content as searchable for meilisearch 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis e35d87ea54 Make the chunk size smaller 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 00c48a33ac Use content instead of source and scrub it 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis 9beaebd97e Tweak search ordering to hopefully return newer results 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis 38996f551a Make meilisearch sort on publish date converted to unix time 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis ea6a6a1287 Make the indexing batch differently and more, show number indexed 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis 365024abec Ensure only indexing public posts and implement clearing and delete 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis 0318e9a599 Add logging to milisiearch index and make it use desc(id) 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis e154ebbf79 Initial meilisearch implementation, doesn't delete posts yet 2022-10-10 20:19:08 +03:00