Commit Graph

593 Commits

Author SHA1 Message Date
faried nawaz 5be9d13981
a better query to delete from hashtags
old query:

Delete on hashtags  (cost=5089.81..5521.63 rows=6160 width=18)
   ->  Hash Semi Join  (cost=5089.81..5521.63 rows=6160 width=18)
         Hash Cond: (hashtags.id = ht.id)
         ->  Seq Scan on hashtags  (cost=0.00..317.28 rows=17528 width=14)
         ->  Hash  (cost=5012.81..5012.81 rows=6160 width=20)
               ->  Merge Anti Join  (cost=0.70..5012.81 rows=6160 width=20)
                     Merge Cond: (ht.id = hto.hashtag_id)
                     ->  Index Scan using hashtags_pkey on hashtags ht  (cost=0.29..610.53 rows=17528 width=14)
                     ->  Index Scan using hashtags_objects_pkey on hashtags_objects hto  (cost=0.42..3506.48 rows=68158 width=14)

new query:

Delete on hashtags ht  (cost=0.70..5012.81 rows=6160 width=12)
   ->  Merge Anti Join  (cost=0.70..5012.81 rows=6160 width=12)
         Merge Cond: (ht.id = hto.hashtag_id)
         ->  Index Scan using hashtags_pkey on hashtags ht  (cost=0.29..610.53 rows=17528 width=14)
         ->  Index Scan using hashtags_objects_pkey on hashtags_objects hto  (cost=0.42..3506.48 rows=68158 width=14)
2021-05-08 02:00:43 +05:00
faried nawaz a0c9a2b4cc
mix prune_objects: remove unused hashtags after pruning remote objects 2021-05-08 02:00:42 +05:00
Alexander Strizhakov 4cd34d0197
suggestion 2021-03-23 17:27:02 +03:00
Alexander Strizhakov 03843a5386
migrating config to tmp folder 2021-03-23 14:29:02 +03:00
Ivan Tashkinov 882dd46843 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-03-02 08:26:30 +03:00
feld 2217b35bd4 Merge branch 'ecto-rollback-in-test-env' into 'develop'
Possibility to run rollback in test env

See merge request pleroma/pleroma!3212
2021-02-26 16:47:53 +00:00
Ivan Tashkinov 40d4362261 [#3213] `mix pleroma.database rollback` tweaks. 2021-02-23 18:11:25 +03:00
Ivan Tashkinov 5992382cf8 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework
# Conflicts:
#	CHANGELOG.md
#	lib/mix/tasks/pleroma/database.ex
#	lib/pleroma/web/templates/feed/feed/_activity.rss.eex
2021-02-11 19:31:57 +03:00
rinpatch f13f5d9303 OpenAPI spec task: Load pleroma application to get version info
For whatever reason it seems to pick up the version without loading the
app on my machine, but not on the CI.
2021-02-09 22:10:09 +03:00
feld 593c0851d9 Merge branch 'develop' into 'new-user-emails' 2021-02-08 16:01:50 -06:00
Ivan Tashkinov d1c6dd97aa [#3213] Partially addressed code review points.
migration rollback task changes, hashtags-related config handling tweaks, `hashtags.data` deletion (unused).
2021-02-07 22:24:12 +03:00
rinpatch 6e68058b63 Merge branch 'feat/allow_alt_text_search_config' into 'develop'
allow user defined text search config in database

See merge request pleroma/pleroma!3275
2021-02-06 09:42:20 +00:00
hyperion 8d4e0342e1 Update priv/repo/migrations/20190501125843_add_fts_index_to_objects.exs, priv/repo/optional_migrations/rum_indexing/20190510135645_add_fts_index_to_objects_two.exs files 2021-02-06 09:42:17 +00:00
Mark Felder af37a5c51a Also make this maybe_ for consistency 2021-02-04 14:33:49 -06:00
rinpatch e2d7c4fd57 Merge branch 'feat/openapi-spec-export' into 'develop'
Improve OpenAPI spec and deploy it to api.pleroma.social

See merge request pleroma/pleroma!3296
2021-02-03 13:21:19 +00:00
rinpatch 9fcff7851f Improve OpenAPI spec and deploy it to api.pleroma.social
Too many changes in OpenAPI spec to describe each one, but
basically it is tag fixes, bringing consitency to operation summaries
and fixing some incorrect information.
2021-02-03 16:07:44 +03:00
Ivan Tashkinov 4e14945670 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework
# Conflicts:
#	CHANGELOG.md
#	lib/pleroma/web/activity_pub/activity_pub.ex
2021-02-03 09:31:38 +03:00
Ivan Tashkinov 108e90b18e [#3213] Explicitly defined PKs in hashtags_objects and data_migration_failed_ids. Added "pleroma.database rollback" task to revert a single migration. 2021-01-31 22:03:59 +03:00
Alibek Omarov aaceb042c5 fix format 2021-01-28 10:20:25 +00:00
Alibek Omarov 1547a2fda4 mix: instance: ensure all needed folders are created before generating config 2021-01-28 09:39:53 +00:00
Ivan Tashkinov 694d98be55 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-01-25 21:24:07 +03:00
Mark Felder f868dcf3ac Ensure consistent behavior between pleroma.user activate/deactivate mix tasks 2021-01-25 10:48:28 -06:00
Mark Felder 5e8da27e14 Provide pleroma.user mix task for both activate and deactivate 2021-01-25 10:36:51 -06:00
Mark Felder 537ba1c5e0 Merge branch 'develop' into refactor/deactivated_user_field 2021-01-25 09:45:06 -06:00
Ivan Tashkinov 2634a16b4c Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework
# Conflicts:
#	CHANGELOG.md
2021-01-21 20:20:35 +03:00
feld 2905df841b Merge branch 'update/oban' into 'develop'
Update Oban to 2.3.4

See merge request pleroma/pleroma!3256
2021-01-20 20:32:23 +00:00
Mark Felder 28581e03ad Merge branch 'develop' into refactor/deactivated_user_field 2021-01-18 14:58:21 -06:00
Alexander Strizhakov 0e48c80d7f
start oban app in migrations and mix tasks 2021-01-17 09:58:41 +03:00
Mark Felder 2c0fe2ea9e Remove toggle_confirmation; require explicit state change
Also cosmetic changes to make the code clearer
2021-01-15 13:11:51 -06:00
Mark Felder d36182c088 Change user.confirmation_pending field to user.is_confirmed 2021-01-15 12:44:41 -06:00
Mark Felder ebd7d1365b Make setting user activation status explicit 2021-01-15 11:37:17 -06:00
Mark Felder 7516660753 Remove toggle_activation 2021-01-15 11:25:32 -06:00
Mark Felder 860b5c7804 Change user.deactivated field to user.is_active 2021-01-15 11:24:46 -06:00
Ivan Tashkinov e350898828 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-01-13 22:11:16 +03:00
Ivan Tashkinov 3e4d84729a [#3213] Prototype of data migrations functionality / HashtagsTableMigrator. 2021-01-13 22:07:38 +03:00
Haelwenn (lanodan) Monnier c4439c630f
Bump Copyright to 2021
grep -rl '# Copyright © .* Pleroma' * | xargs sed -i 's;Copyright © .* Pleroma .*;Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>;'
2021-01-13 07:49:50 +01:00
Ivan Tashkinov 8c972de045 [#3213] transfer_hashtags mix task refactoring. 2021-01-10 11:44:39 +03:00
Ivan Tashkinov 0d521022fe [#3213] Removed PK from hashtags_objects table. Improved hashtags_transfer mix task (logging of failed ids). 2021-01-07 12:20:29 +03:00
Ivan Tashkinov 367f0c31c3 [#3213] Added query options support for Repo.chunk_stream/4.
Used infinite timeout in transfer_hashtags select query.
2020-12-31 09:36:26 +03:00
Ivan Tashkinov a25c1e8ec0 [#3213] Improved `database.transfer_hashtags` mix task: proper rollback, speedup. 2020-12-30 14:35:19 +03:00
Haelwenn 3966add048 Revert "Merge branch 'features/hashtag-column' into 'develop'"
This reverts merge request !2824
2020-12-28 12:02:16 +00:00
Haelwenn (lanodan) Monnier d0c2479710
pleroma.database fill_old_hashtags: Add month_limit argument 2020-12-28 11:05:25 +01:00
Ivan Tashkinov cbb19d0e18 [#3213] Hashtag-filtering functions in ActivityPub. Mix task for migrating hashtags to `hashtags` table. 2020-12-26 22:20:55 +03:00
Alexander Strizhakov 538af14d52
possibility to run rollback in test env 2020-12-22 17:55:30 +03:00
Haelwenn (lanodan) Monnier acb03d591b
Insert text representation of hashtags into object["hashtags"]
Includes a new mix task: pleroma.database fill_old_hashtags
2020-12-22 05:15:34 +01:00
Alex Gleason 80891e83d8
Merge remote-tracking branch 'upstream/develop' into registration-workflow 2020-12-17 09:05:36 -06:00
lain 8b8c317c0f Merge branch 'fix/2336-mastoapi-ws-posts-dropping' into 'develop'
Fix mastoapi ws posts dropping

Closes #2336

See merge request pleroma/pleroma!3198
2020-12-16 11:35:45 +00:00
Alexander Strizhakov cebe3c7def Fix for dropping posts/notifs in WS when mix task is executed
- start oban in mix tasks with empty queues, plugins and crontab
- fix for update_users_following_followers_counts
- fix for removed logo.png
- typo in resend confirmation emails mix task docs
- fix for uploads mix task (start Majic.Pool)
- fix for creating user mix task (start :fast_html app)
2020-12-14 11:02:32 -06:00
rinpatch 7133c0c5ea instance.gen: Warn that stripping exif requires exiftool
And default to no if it is not installed

Closes #2343
2020-12-12 20:38:51 +03:00
rinpatch 6aece536eb instance.gen task: Only show files which will be actually overwritten 2020-12-12 20:35:38 +03:00
lain ed76323776 Merge branch 'feature/configdb-mix-tasks' into 'develop'
Add mix tasks to give additional ConfigDB recovery and debugging options

See merge request pleroma/pleroma!3174
2020-12-07 18:06:06 +00:00
Mark Felder 93428d7c11 Print out settings that will be removed when specifying the group and key for consistency
Fix error message when specified key doesn't exist
2020-12-07 11:45:56 -06:00
Mark Felder 61494b5245 Formatting 2020-12-07 11:22:07 -06:00
feld d817bae802 Apply 1 suggestion(s) to 1 file(s) 2020-12-07 17:13:29 +00:00
Ivan Tashkinov e00c667145 [#3174] Refactoring: ConfigDB fetching functions, ConfigDB tests.
Minor fixes.
2020-12-06 18:02:30 +03:00
Mark Felder 50aadc3d5c shell_yes?/1 was not showing the correct message and always defaults to yes which is dangerous 2020-12-04 11:49:56 -06:00
Mark Felder 9dfda37821 More compact representation 2020-12-04 11:37:49 -06:00
Mark Felder 3bf5c5b015 Ensure deleting entire group prints out settings that will be removed before actually removing them 2020-12-04 11:30:48 -06:00
Mark Felder 696d39c3dc Fix deleting an entire group. Also utilize Pleroma.ConfigDB.delete/1 2020-12-04 11:19:58 -06:00
Mark Felder 685e5c8509 Use Pleroma.ConfigDB.delete/1 instead of rolling our own 2020-12-04 11:09:13 -06:00
Mark Felder 4aad066091 Use Enum.any? to ensure we return true if there are results 2020-12-04 11:04:53 -06:00
Alexander Strizhakov a02eb88396
config_db search methods 2020-12-03 19:34:23 +03:00
Mark Felder 7fd4f4908b dump_key/2 no longer used 2020-12-03 10:03:44 -06:00
feld 60c4ac0f70 Apply 6 suggestion(s) to 1 file(s) 2020-12-03 16:03:14 +00:00
Mark Felder 95e908e4e2 Credo 2020-12-03 09:58:24 -06:00
Mark Felder 16bdc2bcd0 Make the --force flag for reset command consistent with the others and deduplicate db truncation 2020-12-02 16:34:23 -06:00
Mark Felder e379ab8277 Add --force flag for delete and reset commands
Bunch of reorganization and consolidation
2020-12-02 16:24:32 -06:00
Mark Felder 20a911f9f7 Add comment for this mysterious behavior 2020-12-02 14:25:07 -06:00
Mark Felder 25fab7da69 No need for a separate functions here 2020-12-02 14:20:11 -06:00
Mark Felder 13947999ad Use a callback strategy to short circuit the functions and print a nice error 2020-12-02 12:33:34 -06:00
Mark Felder a7b5280b5b Centralize check that configdb is enabled which now raises an exception 2020-12-02 10:43:22 -06:00
Mark Felder 53a5ec1952 Left public during debugging 2020-12-02 10:43:22 -06:00
Mark Felder 5135a8189f Use inspect instead of faking the output 2020-12-02 10:43:22 -06:00
Mark Felder 6334ba9ad3 Remove unnecessary keys command 2020-12-02 10:43:22 -06:00
Mark Felder d0cb73527f Ensure scary warning starts on a new line 2020-12-02 10:43:22 -06:00
Mark Felder 0847e3e496 Print whole config when resetting and include a scary looking message. 2020-12-02 10:43:22 -06:00
Mark Felder d4320e0daf Both are really atoms 2020-12-02 10:43:22 -06:00
Mark Felder 4bdfcf1682 Transform strings to atoms for all cases, including when the atom is a module like Pleroma.Emails.Mailer 2020-12-02 10:43:22 -06:00
Mark Felder 3df115b2b0 Support atoms and strings as args to the mix task
Improve output. Show the user what will be deleted before the prompt.
2020-12-02 10:43:22 -06:00
Mark Felder ae7d37de06 Fix deletion regression due to strings instead of atoms
Improve message after successful deletion
2020-12-02 10:43:22 -06:00
Mark Felder c6a0ca2213 Improve dumping groups and specific keys; add prompts for delete and reset 2020-12-02 10:43:22 -06:00
Mark Felder 67437feafc Support listing groups, listing keys in a group, and dumping the config based on group or specific key in that group 2020-12-02 10:43:22 -06:00
Mark Felder a51da3c1d8 Sort output by group
Not the best sorting, but better than nothing.
2020-12-02 10:43:22 -06:00
Mark Felder ada073f251 Rename keys to groups 2020-12-02 10:43:22 -06:00
Mark Felder e8a4062d9d Document how to delete individual configuration groups and completely reset the config without SQL 2020-12-02 10:43:22 -06:00
Mark Felder a82ba66662 Better deletion message 2020-12-02 10:43:22 -06:00
Mark Felder 6dcc36baa9 Add mix tasks to give additional recovery and debugging options
- pleroma.config dump: prints the entire config as it would be exported to the filesystem
- pleroma.config dump KEY: prints the configuration under a specific ConfigDB key in the database
- pleroma.config keylist: lists the available keys in ConfigDB
- pleroma.config keydel KEY: deletes ConfigDB entry stored under the key

This should prevent the need for users to manually execute SQL queries.
2020-12-02 10:43:22 -06:00
Mark Felder 751712d970 Prevent mix tasks from spewing any internal logging unless DEBUG is in the env
e.g., DEBUG=1 mix pleroma.config migrate_from_db
2020-11-25 12:52:39 -06:00
Alex Gleason 30ed7b502f
Merge remote-tracking branch 'upstream/develop' into registration-workflow 2020-11-21 10:23:53 -06:00
Alexander Strizhakov 0a5b22bc3b
start limiters in mix tasks 2020-11-20 11:37:01 +03:00
feld a1056aace1 Merge branch 'fix/mix-task-user-delete_activities' into 'develop'
Fix mix pleroma.user delete_activities task

See merge request pleroma/pleroma!3156
2020-11-18 18:57:47 +00:00
Mark Felder 97201f7e37 Need to start web_resp cache or mix task fails 2020-11-17 15:15:53 +00:00
Egor Kislitsyn f69fe36ebf
Merge branch 'develop' into frontend-admin-api 2020-11-17 18:34:04 +04:00
Alex Gleason 9546c1444c
Merge remote-tracking branch 'upstream/develop' into registration-workflow 2020-11-14 19:48:47 -06:00
Haelwenn (lanodan) Monnier e2f573d68b
pleroma.instance: Fix Exiftool module name 2020-11-14 22:31:01 +01:00
Egor Kislitsyn 08cbd655d1
Merge branch 'develop' into frontend-admin-api 2020-11-11 19:53:40 +04:00
rinpatch cc45c69fff Remove release_env
While taking a final look at instance.gen before releasing I noticed
that the release_env task outputs messages in broken english. Upon
further inspection it seems to have even more severe issues which, in
my opinion, warrant it's at least temporary removal:
- We do not explain what it actually does, anywhere. Neither the task
 docs nor instance.gen, nor installation instructions.
- It does not respect FHS on OTP releases (uses /opt/pleroma/config even
 though we store the config in /etc/pleroma/config.exs).
- It doesn't work on OTP releases, which is the main reason it exists.
Neither systemd nor openrc service files for OTP include it.
- It is not mentioned in install guides other than the ones for Debian
and OTP releases.
2020-11-08 11:56:09 +03:00
Egor Kislitsyn 03e306785b
Add an API endpoint to install a new frontend 2020-10-27 19:20:21 +04:00
Maksim Pechnikov 2030ffd490 fix test 2020-10-15 22:31:52 +03:00
Maksim Pechnikov cf53e300f8 added generate the release env to `pleroma.instance gen` 2020-10-15 22:31:00 +03:00