Commit Graph

592 Commits

Author SHA1 Message Date
marcin mikołajczak 017e35fbf1 Fix some more typos
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-12-28 00:17:04 +01:00
Lain Soykaf 0c5cc51983 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into pleroma-meilisearch 2023-11-12 13:53:18 +04:00
Mark Felder e23672d82f Ensure benchee doesn't run unless we are executing benchmarks 2023-11-08 12:44:57 -05:00
Haelwenn (lanodan) Monnier 69caedc591 instance gen: Reduce permissions of pleroma directories and config files 2023-08-04 09:50:28 +02:00
tusooa 3b4b84b74c
Force spec for every operation to have a listed tag 2023-01-15 18:31:37 -05:00
lain e853cfe7c3 Revert "Merge branch 'copyright-bump' into 'develop'"
This reverts merge request !3825
2023-01-02 20:38:50 +00:00
marcin mikołajczak 10886eeaa2 Bump copyright year
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-01-01 12:13:06 +01:00
Ekaterina Vaartis 5a39866388 Specifically strip mentions for search indexing 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 5ac6763238 Make add_to_index and remove_from_index report errors 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 84608be87e Change updateId to uid because apparently that's the new name 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 1e23f527e3 Change the meilisearch key auth to conform to 0.25.0 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 571533ae26 Don't support meilisearch < 0.24.0, since it breaks things 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 3179ed0921 Make chunk size configurable 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis a12f63bc81 Implement suggestions from the Meilisearch MR
- Index unlisted posts
- Move version check outside of the streaming and only do it once
- Use a PUT request instead of checking manually if there is need to insert
- Add error handling, sort of
2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 39e596a5b5 Style fixes 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 9c1a930707 Support reindexing meilisearch >=0.24.0
It has has a different error code key
2022-10-10 20:19:09 +03:00
Ekaterina Vaartis e928e307f3 Add a reindex option
Signed-off-by: Ekaterina Vaartis <vaartis@kotobank.ch>
2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 40280cc273 Reorder ranking rules for (maybe) better results 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 09a1ae1b6e Add the meilisearch.stats command 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis a67f9da5cc Add a message with a count of posts to index 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 2c7d973af7 Implement meilisearch auth 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 410c8cb765 Make indexing logs rewrite themselves 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 35e9192ced Rework task indexing to share code with the main module
The code in the main module now scrubs new posts too
2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 3dedadf192 Adjust content indexing to skip more unneeded stuff 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 9f16ca80e0 Mark only content as searchable for meilisearch 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis e35d87ea54 Make the chunk size smaller 2022-10-10 20:19:09 +03:00
Ekaterina Vaartis 00c48a33ac Use content instead of source and scrub it 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis 9beaebd97e Tweak search ordering to hopefully return newer results 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis 38996f551a Make meilisearch sort on publish date converted to unix time 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis ea6a6a1287 Make the indexing batch differently and more, show number indexed 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis 365024abec Ensure only indexing public posts and implement clearing and delete 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis 0318e9a599 Add logging to milisiearch index and make it use desc(id) 2022-10-10 20:19:08 +03:00
Ekaterina Vaartis e154ebbf79 Initial meilisearch implementation, doesn't delete posts yet 2022-10-10 20:19:08 +03:00
Haelwenn 5d7d623390 Merge branch 'bugfix/elixir-1.14' into 'develop'
Bugfix: Elixir 1.14

See merge request pleroma/pleroma!3740
2022-09-27 21:46:03 +00:00
tusooa f7c2073103 Merge branch 'bump/min-elixir-1.10' into 'develop'
Bump minimum Elixir version to 1.10

See merge request pleroma/pleroma!3741
2022-09-05 15:19:34 +00:00
Haelwenn (lanodan) Monnier e124776d14 Elixir 1.14 formatting 2022-09-03 03:58:22 +02:00
Haelwenn (lanodan) Monnier 21ab7369ca Bump minimum Elixir version to 1.10
With the release of Elixir 1.14, Elixir 1.9 is now end-of-life.

Elixir 1.10 Release Notes:
https://github.com/elixir-lang/elixir/releases/tag/v1.10.0
2022-09-02 22:53:54 +02:00
Tusooa Zhu 3885ee182a
Switch to associated_object_id index 2022-08-20 20:43:46 -04:00
floatingghost f2a9285ff0
bugfix/follow-state (#104)
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/104
2022-08-03 01:07:53 -04:00
Ilja 8303af84ce Rename the Exiftool module
No migrations or checks yet
2022-07-01 12:15:02 +02:00
Ilja 551721e41a Rename the new module 2022-07-01 12:13:46 +02:00
Ilja cd316d7269 Use EXIF data of image to prefill image description
During attachment upload Pleroma returns a "description" field. Pleroma-fe has an MR to use that to pre-fill the image description field, <https://git.pleroma.social/pleroma/pleroma-fe/-/merge_requests/1399>

* This MR allows Pleroma to read the EXIF data during upload and return the description to the FE
    * If a description is already present (e.g. because a previous module added it), it will use that
    * Otherwise it will read from the EXIF data. First it will check -ImageDescription, if that's empty, it will check -iptc:Caption-Abstract
    * If no description is found, it will simply return nil, just like before
* When people set up a new instance, they will be asked if they want to read metadata and this module will be activated if so

This was taken from an MR i did on Pleroma and isn't finished yet.
2022-07-01 12:13:46 +02:00
Sean King 17aa3644be
Copyright bump for 2022 2022-02-25 23:11:42 -07:00
Alex Gleason e1b89fe3aa
Merge remote-tracking branch 'origin/develop' into live-dashboard 2021-12-15 19:05:36 -05:00
Alex Gleason 29d80b39f2
Add Phoenix LiveDashboard
Co-authored-by: Egor Kislitsyn <egor@kislitsyn.com>
2021-12-15 19:05:27 -05:00
Haelwenn (lanodan) Monnier a17910a6c6
CI: Bump lint stage to elixir-1.12
Elixir 1.12 changed formatting rules, this allows to avoid having to rollback to run `mix format`
2021-10-06 08:11:05 +02:00
someone cc4f20b130
mix pleroma.database set_text_search_config now runs concurrently and infinitely
Backport of: https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3514
2021-08-28 18:29:43 +02:00
someone 4b940e441a mix pleroma.database set_text_search_config now runs concurrently and infinitely 2021-08-15 13:49:12 -04:00
feld bc51dea425 Update lib/mix/tasks/pleroma/database.ex 2021-06-07 20:02:28 +00:00
faried nawaz 5be9d13981
a better query to delete from hashtags
old query:

Delete on hashtags  (cost=5089.81..5521.63 rows=6160 width=18)
   ->  Hash Semi Join  (cost=5089.81..5521.63 rows=6160 width=18)
         Hash Cond: (hashtags.id = ht.id)
         ->  Seq Scan on hashtags  (cost=0.00..317.28 rows=17528 width=14)
         ->  Hash  (cost=5012.81..5012.81 rows=6160 width=20)
               ->  Merge Anti Join  (cost=0.70..5012.81 rows=6160 width=20)
                     Merge Cond: (ht.id = hto.hashtag_id)
                     ->  Index Scan using hashtags_pkey on hashtags ht  (cost=0.29..610.53 rows=17528 width=14)
                     ->  Index Scan using hashtags_objects_pkey on hashtags_objects hto  (cost=0.42..3506.48 rows=68158 width=14)

new query:

Delete on hashtags ht  (cost=0.70..5012.81 rows=6160 width=12)
   ->  Merge Anti Join  (cost=0.70..5012.81 rows=6160 width=12)
         Merge Cond: (ht.id = hto.hashtag_id)
         ->  Index Scan using hashtags_pkey on hashtags ht  (cost=0.29..610.53 rows=17528 width=14)
         ->  Index Scan using hashtags_objects_pkey on hashtags_objects hto  (cost=0.42..3506.48 rows=68158 width=14)
2021-05-08 02:00:43 +05:00