Merge remote-tracking branch 'upstream/develop' into registration-workflow

.gitattributes

@@ -1,2 +1,8 @@
 *.ex diff=elixir
 *.exs diff=elixir
+# At the time of writing all js/css files included
+# in the repo are minified bundles, and we don't want
+# to search/diff those as text files.
+*.js binary
+*.js.map binary
+*.css binary

.gitlab-ci.yml

@@ -229,7 +229,7 @@ arm:
   only: *release-only
   tags:
     - arm32
-  image: elixir:1.10.3
+  image: arm32v7/elixir:1.10.3
   cache: *release-cache
   variables: *release-variables
   before_script: *before-release
@@ -241,7 +241,7 @@ arm-musl:
   only: *release-only
   tags:
     - arm32
-  image: elixir:1.10.3-alpine
+  image: arm32v7/elixir:1.10.3-alpine
   cache: *release-cache
   variables: *release-variables
   before_script: *before-release-musl
@@ -253,7 +253,7 @@ arm64:
   only: *release-only
   tags:
     - arm
-  image: elixir:1.10.3
+  image: arm64v8/elixir:1.10.3
   cache: *release-cache
   variables: *release-variables
   before_script: *before-release
@@ -265,8 +265,7 @@ arm64-musl:
   only: *release-only
   tags:
     - arm
-  # TODO: Replace with upstream image when 1.9.0 comes out
+  image: arm64v8/elixir:1.10.3-alpine
-  image: elixir:1.10.3-alpine
   cache: *release-cache
   variables: *release-variables
   before_script: *before-release-musl

CHANGELOG.md

@@ -1,54 +1,91 @@
 # Changelog
+
 All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ## Unreleased
 
+### Changed
+
+- Polls now always return a `voters_count`, even if they are single-choice.
+- Admin Emails: The ap id is used as the user link in emails now.
+- **Breaking:** Changed `mix pleroma.user toggle_confirmed` to `mix pleroma.user confirm`
+
 ### Added
-- Mix tasks for controlling user account confirmation status in bulk (`mix pleroma.user confirm_all` and `mix pleroma.user unconfirm_all`)
+
-- Mix task for sending confirmation emails to all unconfirmed users (`mix pleroma.email send_confirmation_mails`)
-- Mix task option for force-unfollowing relays
-- Media preview proxy (requires `ffmpeg` and `ImageMagick` to be installed and media proxy to be enabled; see `:media_preview_proxy` config for more details).
 - Reports now generate notifications for admins and mods.
-- Pleroma API: Importing the mutes users from CSV files.
 - Experimental websocket-based federation between Pleroma instances.
-- Support pagination of blocks and mutes
+- Support for local-only statuses
-- App metrics: ability to restrict access to specified IP whitelist.
+- Support pagination of blocks and mutes.
-- Account backup
+- Account backup.
 - Configuration: Add `:instance, autofollowing_nicknames` setting to provide a way to make accounts automatically follow new users that register on the local Pleroma instance.
-- Ability to view remote timelines, with ex. `/api/v1/timelines/public?instance=lain.com` and streams `public:remote` and `public:remote:media`
+- Ability to view remote timelines, with ex. `/api/v1/timelines/public?instance=lain.com` and streams `public:remote` and `public:remote:media`.
 - The site title is now injected as a `title` tag like preloads or metadata.
+- Password reset tokens now are not accepted after a certain age.
+
+<details>
+  <summary>API Changes</summary>
+- Admin API: (`GET /api/pleroma/admin/users`) filter users by `unconfirmed` status and `actor_type`.
+- Pleroma API: Add `idempotency_key` to the chat message entity that can be used for optimistic message sending.
+- Pleroma API: (`GET /api/v1/pleroma/federation_status`) Add a way to get a list of unreachable instances.
+- Mastodon API: User and conversation mutes can now auto-expire if `expires_in` parameter was given while adding the mute.
+- Admin API: An endpoint to manage frontends
+
+</details>
+
+### Fixed
+
+- Users with `is_discoverable` field set to false (default value) will appear in in-service search results but be hidden from external services (search bots etc.).
+
+<details>
+  <summary>API Changes</summary>
+  - Mastodon API: Current user is now included in conversation if it's the only participant.
+  - Mastodon API: Fixed last_status.account being not filled with account data.
+</details>
+
+## Unreleased (Patch)
+
+### Changed
+
+### Fixed
+
+- Config generation: rename `Pleroma.Upload.Filter.ExifTool` to `Pleroma.Upload.Filter.Exiftool`.
+- Search: RUM index search speed has been fixed.
+- S3 Uploads with Elixir 1.11.
+- Emoji Reaction activity filtering from blocked and muted accounts.
+- Mix task pleroma.user delete_activities for source installations.
+- Fix ability to update Pleroma Chat push notifications with PUT /api/v1/push/subscription and alert type pleroma:chat_mention
+- Forwarded reports duplication from Pleroma instances.
+
+<details>
+  <summary>API</summary>
+- Statuses were not displayed for Mastodon forwarded reports.
+
+</details>
+
+## [2.2.0] - 2020-11-12
+
+### Security
+
+- Fixed the possibility of using file uploads to spoof posts.
 
 ### Changed
 
 - **Breaking** Requires `libmagic` (or `file`) to guess file types.
+- **Breaking:** App metrics endpoint (`/api/pleroma/app_metrics`) is disabled by default, check `docs/API/prometheus.md` on enabling and configuring.
 - **Breaking:** Pleroma Admin API: emoji packs and files routes changed.
 - **Breaking:** Sensitive/NSFW statuses no longer disable link previews.
-- **Breaking:** App metrics endpoint (`/api/pleroma/app_metrics`) is disabled by default, check `docs/API/prometheus.md` on enabling and configuring.
 - Search: Users are now findable by their urls.
 - Renamed `:await_up_timeout` in `:connections_pool` namespace to `:connect_timeout`, old name is deprecated.
 - Renamed `:timeout` in `pools` namespace to `:recv_timeout`, old name is deprecated.
 - The `discoverable` field in the `User` struct will now add a NOINDEX metatag to profile pages when false.
-- Users with the `discoverable` field set to false will not show up in searches.
+- Users with the `is_discoverable` field set to false will not show up in searches ([bug](https://git.pleroma.social/pleroma/pleroma/-/issues/2301)).
 - Minimum lifetime for ephmeral activities changed to 10 minutes and made configurable (`:min_lifetime` option).
 - Introduced optional dependencies on `ffmpeg`, `ImageMagick`, `exiftool` software packages. Please refer to `docs/installation/optional/media_graphics_packages.md`.
-- Changed `mix pleroma.user toggle_confirmed` to `mix pleroma.user confirm`
+- <details>
-- Polls now always return a `voters_count`, even if they are single-choice
-- Admin Emails: The ap id is used as the user link in emails now.
-
-<details>
   <summary>API Changes</summary>
-
+- API: Empty parameter values for integer parameters are now ignored in non-strict validaton mode.
-- Pleroma API: Importing the mutes users from CSV files.
-- Admin API: Importing emoji from a zip file
-- Pleroma API: Pagination for remote/local packs and emoji.
-- Admin API: (`GET /api/pleroma/admin/users`) added filters user by `unconfirmed` status
-- Admin API: (`GET /api/pleroma/admin/users`) added filters user by `actor_type`
-- Pleroma API: Add `idempotency_key` to the chat message entity that can be used for optimistic message sending.
-- Pleroma API: (`GET /api/v1/pleroma/federation_status`) Add a way to get a list of unreachable instances.
-- Mastodon API: User and conversation mutes can now auto-expire if `expires_in` parameter was given while adding the mute.
-
 </details>
 
 ### Removed
@@ -59,23 +96,38 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Removed `:managed_config` option. In practice, it was accidentally removed with 2.0.0 release when frontends were
 switched to a new configuration mechanism, however it was not officially removed until now.
 
+### Added
+
+- Media preview proxy (requires `ffmpeg` and `ImageMagick` to be installed and media proxy to be enabled; see `:media_preview_proxy` config for more details).
+- Mix tasks for controlling user account confirmation status in bulk (`mix pleroma.user confirm_all` and `mix pleroma.user unconfirm_all`)
+- Mix task for sending confirmation emails to all unconfirmed users (`mix pleroma.email send_confirmation_mails`)
+- Mix task option for force-unfollowing relays
+- App metrics: ability to restrict access to specified IP whitelist.
+
+<details>
+  <summary>API Changes</summary>
+
+- Admin API: Importing emoji from a zip file
+- Pleroma API: Importing the mutes users from CSV files.
+- Pleroma API: Pagination for remote/local packs and emoji.
+
+</details>
+
 ### Fixed
 
 - Add documented-but-missing chat pagination.
 - Allow sending out emails again.
-- Allow sending chat messages to yourself.
+- Allow sending chat messages to yourself
-- Fix remote users with a whitespace name.
 - OStatus / static FE endpoints: fixed inaccessibility for anonymous users on non-federating instances, switched to handling per `:restrict_unauthenticated` setting.
-- Mastodon API: Current user is now included in conversation if it's the only participant
+- Fix remote users with a whitespace name.
-- Mastodon API: Fixed last_status.account being not filled with account data
 
-## Unreleased (Patch)
+### Upgrade notes
 
-### Changed
+1. Install libmagic and development headers (`libmagic-dev` on Ubuntu/Debian, `file-dev` on Alpine Linux)
-- API: Empty parameter values for integer parameters are now ignored in non-strict validaton mode.
+2. Run database migrations (inside Pleroma directory):
-
+  - OTP: `./bin/pleroma_ctl migrate`
-### Fixes
+  - From Source: `mix ecto.migrate`
-- Config generation: rename `Pleroma.Upload.Filter.ExifTool` to `Pleroma.Upload.Filter.Exiftool`
+3. Restart Pleroma
 
 ## [2.1.2] - 2020-09-17
 

Dockerfile

@@ -31,7 +31,7 @@ LABEL maintainer="ops@pleroma.social" \
 ARG HOME=/opt/pleroma
 ARG DATA=/var/lib/pleroma
 
-RUN echo "https://nl.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories &&\
+RUN echo "http://nl.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories &&\
 	apk update &&\
 	apk add exiftool imagemagick ncurses postgresql-client &&\
 	adduser --system --shell /bin/false --home ${HOME} pleroma &&\

SECURITY.md

@@ -6,7 +6,7 @@ Currently, Pleroma offers bugfixes and security patches only for the latest mino
 
 | Version | Support 
 |---------| --------
-| 2.1     | Bugfixes and security patches
+| 2.2     | Bugfixes and security patches
 
 ## Reporting a vulnerability
 

config/config.exs

@@ -129,7 +129,6 @@
     dispatch: [
       {:_,
        [
-         {"/api/fedsocket/v1", Pleroma.Web.FedSockets.IncomingHandler, []},
          {"/api/v1/streaming", Pleroma.Web.MastodonAPI.WebsocketHandler, []},
          {"/websocket", Phoenix.Endpoint.CowboyWebSocket,
           {Phoenix.Transports.WebSocket,
@@ -264,7 +263,8 @@
       length: 16
     ]
   ],
-  show_reactions: true
+  show_reactions: true,
+  password_reset_token_validity: 60 * 60 * 24
 
 config :pleroma, :welcome,
   direct_message: [

config/description.exs

@@ -272,19 +272,6 @@
       }
     ]
   },
-  %{
-    group: :pleroma,
-    key: :fed_sockets,
-    type: :group,
-    description: "Websocket based federation",
-    children: [
-      %{
-        key: :enabled,
-        type: :boolean,
-        description: "Enable FedSockets"
-      }
-    ]
-  },
   %{
     group: :pleroma,
     key: Pleroma.Emails.Mailer,

docs/API/admin_api.md

@@ -554,7 +554,7 @@ Response:
   * `show_role`
   * `skip_thread_containment`
   * `fields`
-  * `discoverable`
+  * `is_discoverable`
   * `actor_type`
 
 * Responses:
@@ -1499,3 +1499,66 @@ Returns the content of the document
   "url": "https://example.com/instance/panel.html"
 }
 ```
+
+## `GET /api/pleroma/admin/frontends
+
+### List available frontends
+
+- Response:
+
+```json
+[
+   {
+    "build_url": "https://git.pleroma.social/pleroma/fedi-fe/-/jobs/artifacts/${ref}/download?job=build",
+    "git": "https://git.pleroma.social/pleroma/fedi-fe",
+    "installed": true,
+    "name": "fedi-fe",
+    "ref": "master"
+  },
+  {
+    "build_url": "https://git.pleroma.social/lambadalambda/kenoma/-/jobs/artifacts/${ref}/download?job=build",
+    "git": "https://git.pleroma.social/lambadalambda/kenoma",
+    "installed": false,
+    "name": "kenoma",
+    "ref": "master"
+  }
+]
+```
+
+## `POST /api/pleroma/admin/frontends/install`
+
+### Install a frontend
+
+- Params:
+  - `name`: frontend name, required
+  - `ref`: frontend ref
+  - `file`: path to a frontend zip file
+  - `build_url`: build URL
+  - `build_dir`: build directory
+
+- Response:
+
+```json
+[
+   {
+    "build_url": "https://git.pleroma.social/pleroma/fedi-fe/-/jobs/artifacts/${ref}/download?job=build",
+    "git": "https://git.pleroma.social/pleroma/fedi-fe",
+    "installed": true,
+    "name": "fedi-fe",
+    "ref": "master"
+  },
+  {
+    "build_url": "https://git.pleroma.social/lambadalambda/kenoma/-/jobs/artifacts/${ref}/download?job=build",
+    "git": "https://git.pleroma.social/lambadalambda/kenoma",
+    "installed": false,
+    "name": "kenoma",
+    "ref": "master"
+  }
+]
+```
+
+```json
+{
+  "error": "Could not install frontend"
+}
+```

docs/API/differences_in_mastoapi_responses.md

@@ -18,7 +18,7 @@ Adding the parameter `instance=lain.com` to the public timeline will show only s
 
 ## Statuses
 
-- `visibility`: has an additional possible value `list`
+- `visibility`: has additional possible values `list` and `local` (for local-only statuses)
 
 Has these additional fields under the `pleroma` object:
 
@@ -84,7 +84,7 @@ Has these additional fields under the `pleroma` object:
 
 - `show_role`: boolean, nullable, true when the user wants his role (e.g admin, moderator) to be shown
 - `no_rich_text` - boolean, nullable, true when html tags are stripped from all statuses requested from the API
-- `discoverable`: boolean, true when the user allows discovery of the account in search results and other services.
+- `discoverable`: boolean, true when the user allows external services (search bots) etc. to index / list the account (regardless of this setting, user will still appear in regular search results)
 - `actor_type`: string, the type of this account.
 
 ## Conversations
@@ -173,7 +173,7 @@ Additional parameters can be added to the JSON body/Form data:
 - `preview`: boolean, if set to `true` the post won't be actually posted, but the status entitiy would still be rendered back. This could be useful for previewing rich text/custom emoji, for example.
 - `content_type`: string, contain the MIME type of the status, it is transformed into HTML by the backend. You can get the list of the supported MIME types with the nodeinfo endpoint.
 - `to`: A list of nicknames (like `lain@soykaf.club` or `lain` on the local server) that will be used to determine who is going to be addressed by this post. Using this will disable the implicit addressing by mentioned names in the `status` body, only the people in the `to` list will be addressed. The normal rules for for post visibility are not affected by this and will still apply.
-- `visibility`: string, besides standard MastoAPI values (`direct`, `private`, `unlisted` or `public`) it can be used to address a List by setting it to `list:LIST_ID`.
+- `visibility`: string, besides standard MastoAPI values (`direct`, `private`, `unlisted`, `local` or `public`) it can be used to address a List by setting it to `list:LIST_ID`.
 - `expires_in`: The number of seconds the posted activity should expire in. When a posted activity expires it will be deleted from the server, and a delete request for it will be federated. This needs to be longer than an hour.
 - `in_reply_to_conversation_id`: Will reply to a given conversation, addressing only the people who are part of the recipient set of that conversation. Sets the visibility to `direct`.
 
@@ -207,7 +207,7 @@ Additional parameters can be added to the JSON body/Form data:
 - `skip_thread_containment` - if true, skip filtering out broken threads
 - `allow_following_move` - if true, allows automatically follow moved following accounts
 - `pleroma_background_image` - sets the background image of the user. Can be set to "" (an empty string) to reset.
-- `discoverable` - if true, discovery of this account in search results and other services is allowed.
+- `discoverable` - if true, external services (search bots) etc. are allowed to index / list the account (regardless of this setting, user will still appear in regular search results).
 - `actor_type` - the type of this account.
 - `accepts_chat_messages` - if false, this account will reject all chat messages.
 
@@ -233,7 +233,7 @@ Post here request with `grant_type=refresh_token` to obtain new access token. Re
 
 `POST /api/v1/accounts`
 
-Has theses additional parameters (which are the same as in Pleroma-API):
+Has these additional parameters (which are the same as in Pleroma-API):
 
 - `fullname`: optional
 - `bio`: optional
@@ -261,6 +261,16 @@ Has theses additional parameters (which are the same as in Pleroma-API):
 - `pleroma.metadata.post_formats`: A list of the allowed post format types
 - `vapid_public_key`: The public key needed for push messages
 
+## Push Subscription
+
+`POST /api/v1/push/subscription`
+`PUT /api/v1/push/subscription`
+
+Permits these additional alert types:
+
+- pleroma:chat_mention
+- pleroma:emoji_reaction
+
 ## Markers
 
 Has these additional fields under the `pleroma` object:

docs/configuration/cheatsheet.md

@@ -63,6 +63,7 @@ To add configuration to your config file, you can copy it from the base config.
 * `external_user_synchronization`: Enabling following/followers counters synchronization for external users.
 * `cleanup_attachments`: Remove attachments along with statuses. Does not affect duplicate files and attachments without status. Enabling this will increase load to database when deleting statuses on larger instances.
 * `show_reactions`: Let favourites and emoji reactions be viewed through the API (default: `true`).
+* `password_reset_token_validity`: The time after which reset tokens aren't accepted anymore, in seconds (default: one day).
 
 ## Welcome
 * `direct_message`: - welcome message sent as a direct message.
@@ -220,18 +221,6 @@ config :pleroma, :mrf_user_allowlist, %{
 * `total_user_limit`: the number of scheduled activities a user is allowed to create in total (Default: `300`)
 * `enabled`: whether scheduled activities are sent to the job queue to be executed
 
-## FedSockets
-FedSockets is an experimental feature allowing for Pleroma backends to federate using a persistant websocket connection as opposed to making each federation a seperate http connection. This feature is currently off by default. It is configurable throught he following options.
-
-### :fedsockets
-* `enabled`: Enables FedSockets for this instance. `false` by default.
-* `connection_duration`: Time an idle websocket is kept open.
-* `rejection_duration`: Failures to connect via FedSockets will not be retried for this period of time.
-* `fed_socket_fetches` and `fed_socket_rejections`: Settings passed to `cachex` for the fetch registry, and rejection stacks. See `Pleroma.Web.FedSockets` for more details.
-
-
-## Frontends
-
 ### :frontend_configurations
 
 This can be used to configure a keyword list that keeps the configuration data for any kind of frontend. By default, settings for `pleroma_fe` and `masto_fe` are configured. You can find the documentation for `pleroma_fe` configuration into [Pleroma-FE configuration and customization for instance administrators](/frontend/CONFIGURATION/#options).

docs/installation/debian_based_en.md

@@ -35,7 +35,7 @@ sudo apt full-upgrade
 * Install some of the above mentioned programs:
 
 ```shell
-sudo apt install git build-essential postgresql postgresql-contrib cmake libmagic-devel
+sudo apt install git build-essential postgresql postgresql-contrib cmake libmagic-dev
 ```
 
 ### Install Elixir and Erlang

installation/pleroma.nginx

@@ -93,9 +93,4 @@ server {
         chunked_transfer_encoding on;
         proxy_pass         http://phoenix;
     }
-
-    location /api/fedsocket/v1 {
-        proxy_request_buffering off;
-        proxy_pass http://phoenix/api/fedsocket/v1;
-    }
 }

lib/mix/pleroma.ex

@@ -14,11 +14,12 @@ defmodule Mix.Pleroma do
     :swoosh,
     :timex
   ]
-  @cachex_children ["object", "user", "scrubber"]
+  @cachex_children ["object", "user", "scrubber", "web_resp"]
   @doc "Common functions to be reused in mix tasks"
   def start_pleroma do
     Pleroma.Config.Holder.save_default()
     Pleroma.Config.Oban.warn()
+    Pleroma.Application.limiters_setup()
     Application.put_env(:phoenix, :serve_endpoints, false, persistent: true)
 
     if Pleroma.Config.get(:env) != :test do

lib/mix/tasks/pleroma/frontend.ex

@@ -17,8 +17,6 @@ def run(["install", "none" | _args]) do
   end
 
   def run(["install", frontend | args]) do
-    log_level = Logger.level()
-    Logger.configure(level: :warn)
     start_pleroma()
 
     {options, [], []} =
@@ -33,109 +31,6 @@ def run(["install", frontend | args]) do
         ]
       )
 
-    instance_static_dir =
+    Pleroma.Frontend.install(frontend, options)
-      with nil <- options[:static_dir] do
-        Pleroma.Config.get!([:instance, :static_dir])
-      end
-
-    cmd_frontend_info = %{
-      "name" => frontend,
-      "ref" => options[:ref],
-      "build_url" => options[:build_url],
-      "build_dir" => options[:build_dir]
-    }
-
-    config_frontend_info = Pleroma.Config.get([:frontends, :available, frontend], %{})
-
-    frontend_info =
-      Map.merge(config_frontend_info, cmd_frontend_info, fn _key, config, cmd ->
-        # This only overrides things that are actually set
-        cmd || config
-      end)
-
-    ref = frontend_info["ref"]
-
-    unless ref do
-      raise "No ref given or configured"
-    end
-
-    dest =
-      Path.join([
-        instance_static_dir,
-        "frontends",
-        frontend,
-        ref
-      ])
-
-    fe_label = "#{frontend} (#{ref})"
-
-    tmp_dir = Path.join([instance_static_dir, "frontends", "tmp"])
-
-    with {_, :ok} <-
-           {:download_or_unzip, download_or_unzip(frontend_info, tmp_dir, options[:file])},
-         shell_info("Installing #{fe_label} to #{dest}"),
-         :ok <- install_frontend(frontend_info, tmp_dir, dest) do
-      File.rm_rf!(tmp_dir)
-      shell_info("Frontend #{fe_label} installed to #{dest}")
-
-      Logger.configure(level: log_level)
-    else
-      {:download_or_unzip, _} ->
-        shell_info("Could not download or unzip the frontend")
-
-      _e ->
-        shell_info("Could not install the frontend")
-    end
-  end
-
-  defp download_or_unzip(frontend_info, temp_dir, file) do
-    if file do
-      with {:ok, zip} <- File.read(Path.expand(file)) do
-        unzip(zip, temp_dir)
-      end
-    else
-      download_build(frontend_info, temp_dir)
-    end
-  end
-
-  def unzip(zip, dest) do
-    with {:ok, unzipped} <- :zip.unzip(zip, [:memory]) do
-      File.rm_rf!(dest)
-      File.mkdir_p!(dest)
-
-      Enum.each(unzipped, fn {filename, data} ->
-        path = filename
-
-        new_file_path = Path.join(dest, path)
-
-        new_file_path
-        |> Path.dirname()
-        |> File.mkdir_p!()
-
-        File.write!(new_file_path, data)
-      end)
-
-      :ok
-    end
-  end
-
-  defp download_build(frontend_info, dest) do
-    shell_info("Downloading pre-built bundle for #{frontend_info["name"]}")
-    url = String.replace(frontend_info["build_url"], "${ref}", frontend_info["ref"])
-
-    with {:ok, %{status: 200, body: zip_body}} <-
-           Pleroma.HTTP.get(url, [], pool: :media, recv_timeout: 120_000) do
-      unzip(zip_body, dest)
-    else
-      e -> {:error, e}
-    end
-  end
-
-  defp install_frontend(frontend_info, source, dest) do
-    from = frontend_info["build_dir"] || "dist"
-    File.rm_rf!(dest)
-    File.mkdir_p!(dest)
-    File.cp_r!(Path.join([source, from]), dest)
-    :ok
   end
 end

lib/pleroma/activity.ex

@@ -356,4 +356,15 @@ def pinned_by_actor?(%Activity{} = activity) do
     actor = user_actor(activity)
     activity.id in actor.pinned_activities
   end
+
+  @spec get_by_object_ap_id_with_object(String.t()) :: t() | nil
+  def get_by_object_ap_id_with_object(ap_id) when is_binary(ap_id) do
+    ap_id
+    |> Queries.by_object_id()
+    |> with_preloaded_object()
+    |> first()
+    |> Repo.one()
+  end
+
+  def get_by_object_ap_id_with_object(_), do: nil
 end

lib/pleroma/activity/search.ex

@@ -27,7 +27,10 @@ def search(user, search_query, options \\ []) do
     |> maybe_restrict_local(user)
     |> maybe_restrict_author(author)
     |> maybe_restrict_blocked(user)
-    |> Pagination.fetch_paginated(%{"offset" => offset, "limit" => limit}, :offset)
+    |> Pagination.fetch_paginated(
+      %{"offset" => offset, "limit" => limit, "skip_order" => index_type == :rum},
+      :offset
+    )
     |> maybe_fetch(user, search_query)
   end
 

lib/pleroma/application.ex

@@ -57,6 +57,7 @@ def start(_type, _args) do
     setup_instrumenters()
     load_custom_modules()
     Pleroma.Docs.JSON.compile()
+    limiters_setup()
 
     adapter = Application.get_env(:tesla, :adapter)
 
@@ -207,8 +208,7 @@ defp dont_run_in_test(_) do
          name: Pleroma.Web.Streamer.registry(),
          keys: :duplicate,
          partitions: System.schedulers_online()
-       ]},
+       ]}
-      Pleroma.Web.FedSockets.Supervisor
     ]
   end
 
@@ -273,4 +273,10 @@ defp http_children(Tesla.Adapter.Gun, _) do
   end
 
   defp http_children(_, _), do: []
+
+  @spec limiters_setup() :: :ok
+  def limiters_setup do
+    [Pleroma.Web.RichMedia.Helpers, Pleroma.Web.MediaProxy]
+    |> Enum.each(&ConcurrentLimiter.new(&1, 1, 0))
+  end
 end

lib/pleroma/constants.ex

@@ -26,4 +26,6 @@ defmodule Pleroma.Constants do
     do:
       ~w(index.html robots.txt static static-fe finmoji emoji packs sounds images instance sw.js sw-pleroma.js favicon.png schemas doc embed.js embed.css)
   )
+
+  def as_local_public, do: Pleroma.Web.base_url() <> "/#Public"
 end

lib/pleroma/emails/admin_email.ex

@@ -48,6 +48,9 @@ def report(to, reporter, account, statuses, comment) do
               status_url = Helpers.o_status_url(Pleroma.Web.Endpoint, :notice, id)
               "<li><a href=\"#{status_url}\">#{status_url}</li>"
 
+            %{"id" => id} when is_binary(id) ->
+              "<li><a href=\"#{id}\">#{id}</li>"
+
             id when is_binary(id) ->
               "<li><a href=\"#{id}\">#{id}</li>"
           end)

lib/pleroma/frontend.ex

@@ -0,0 +1,110 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Frontend do
+  alias Pleroma.Config
+
+  require Logger
+
+  def install(name, opts \\ []) do
+    frontend_info = %{
+      "ref" => opts[:ref],
+      "build_url" => opts[:build_url],
+      "build_dir" => opts[:build_dir]
+    }
+
+    frontend_info =
+      [:frontends, :available, name]
+      |> Config.get(%{})
+      |> Map.merge(frontend_info, fn _key, config, cmd ->
+        # This only overrides things that are actually set
+        cmd || config
+      end)
+
+    ref = frontend_info["ref"]
+
+    unless ref do
+      raise "No ref given or configured"
+    end
+
+    dest = Path.join([dir(), name, ref])
+
+    label = "#{name} (#{ref})"
+    tmp_dir = Path.join(dir(), "tmp")
+
+    with {_, :ok} <-
+           {:download_or_unzip, download_or_unzip(frontend_info, tmp_dir, opts[:file])},
+         Logger.info("Installing #{label} to #{dest}"),
+         :ok <- install_frontend(frontend_info, tmp_dir, dest) do
+      File.rm_rf!(tmp_dir)
+      Logger.info("Frontend #{label} installed to #{dest}")
+    else
+      {:download_or_unzip, _} ->
+        Logger.info("Could not download or unzip the frontend")
+        {:error, "Could not download or unzip the frontend"}
+
+      _e ->
+        Logger.info("Could not install the frontend")
+        {:error, "Could not install the frontend"}
+    end
+  end
+
+  def dir(opts \\ []) do
+    if is_nil(opts[:static_dir]) do
+      Pleroma.Config.get!([:instance, :static_dir])
+    else
+      opts[:static_dir]
+    end
+    |> Path.join("frontends")
+  end
+
+  defp download_or_unzip(frontend_info, temp_dir, nil),
+    do: download_build(frontend_info, temp_dir)
+
+  defp download_or_unzip(_frontend_info, temp_dir, file) do
+    with {:ok, zip} <- File.read(Path.expand(file)) do
+      unzip(zip, temp_dir)
+    end
+  end
+
+  def unzip(zip, dest) do
+    with {:ok, unzipped} <- :zip.unzip(zip, [:memory]) do
+      File.rm_rf!(dest)
+      File.mkdir_p!(dest)
+
+      Enum.each(unzipped, fn {filename, data} ->
+        path = filename
+
+        new_file_path = Path.join(dest, path)
+
+        new_file_path
+        |> Path.dirname()
+        |> File.mkdir_p!()
+
+        File.write!(new_file_path, data)
+      end)
+    end
+  end
+
+  defp download_build(frontend_info, dest) do
+    Logger.info("Downloading pre-built bundle for #{frontend_info["name"]}")
+    url = String.replace(frontend_info["build_url"], "${ref}", frontend_info["ref"])
+
+    with {:ok, %{status: 200, body: zip_body}} <-
+           Pleroma.HTTP.get(url, [], pool: :media, recv_timeout: 120_000) do
+      unzip(zip_body, dest)
+    else
+      {:error, e} -> {:error, e}
+      e -> {:error, e}
+    end
+  end
+
+  defp install_frontend(frontend_info, source, dest) do
+    from = frontend_info["build_dir"] || "dist"
+    File.rm_rf!(dest)
+    File.mkdir_p!(dest)
+    File.cp_r!(Path.join([source, from]), dest)
+    :ok
+  end
+end

lib/pleroma/object/fetcher.ex

@@ -12,7 +12,6 @@ defmodule Pleroma.Object.Fetcher do
   alias Pleroma.Web.ActivityPub.ObjectValidator
   alias Pleroma.Web.ActivityPub.Transmogrifier
   alias Pleroma.Web.Federator
-  alias Pleroma.Web.FedSockets
 
   require Logger
   require Pleroma.Constants
@@ -183,16 +182,16 @@ defp maybe_date_fetch(headers, date) do
     end
   end
 
-  def fetch_and_contain_remote_object_from_id(prm, opts \\ [])
+  def fetch_and_contain_remote_object_from_id(id)
 
-  def fetch_and_contain_remote_object_from_id(%{"id" => id}, opts),
+  def fetch_and_contain_remote_object_from_id(%{"id" => id}),
-    do: fetch_and_contain_remote_object_from_id(id, opts)
+    do: fetch_and_contain_remote_object_from_id(id)
 
-  def fetch_and_contain_remote_object_from_id(id, opts) when is_binary(id) do
+  def fetch_and_contain_remote_object_from_id(id) when is_binary(id) do
     Logger.debug("Fetching object #{id} via AP")
 
     with {:scheme, true} <- {:scheme, String.starts_with?(id, "http")},
-         {:ok, body} <- get_object(id, opts),
+         {:ok, body} <- get_object(id),
          {:ok, data} <- safe_json_decode(body),
          :ok <- Containment.contain_origin_from_id(id, data) do
       {:ok, data}
@@ -208,22 +207,10 @@ def fetch_and_contain_remote_object_from_id(id, opts) when is_binary(id) do
     end
   end
 
-  def fetch_and_contain_remote_object_from_id(_id, _opts),
+  def fetch_and_contain_remote_object_from_id(_id),
     do: {:error, "id must be a string"}
 
-  defp get_object(id, opts) do
+  defp get_object(id) do
-    with false <- Keyword.get(opts, :force_http, false),
-         {:ok, fedsocket} <- FedSockets.get_or_create_fed_socket(id) do
-      Logger.debug("fetching via fedsocket - #{inspect(id)}")
-      FedSockets.fetch(fedsocket, id)
-    else
-      _other ->
-        Logger.debug("fetching via http - #{inspect(id)}")
-        get_object_http(id)
-    end
-  end
-
-  defp get_object_http(id) do
     date = Pleroma.Signature.signed_date()
 
     headers =

lib/pleroma/password_reset_token.ex

@@ -40,6 +40,7 @@ def used_changeset(struct) do
   @spec reset_password(binary(), map()) :: {:ok, User.t()} | {:error, binary()}
   def reset_password(token, data) do
     with %{used: false} = token <- Repo.get_by(PasswordResetToken, %{token: token}),
+         false <- expired?(token),
          %User{} = user <- User.get_cached_by_id(token.user_id),
          {:ok, _user} <- User.reset_password(user, data),
          {:ok, token} <- Repo.update(used_changeset(token)) do
@@ -48,4 +49,14 @@ def reset_password(token, data) do
       _e -> {:error, token}
     end
   end
+
+  def expired?(%__MODULE__{inserted_at: inserted_at}) do
+    validity = Pleroma.Config.get([:instance, :password_reset_token_validity], 0)
+
+    now = NaiveDateTime.utc_now()
+
+    difference = NaiveDateTime.diff(now, inserted_at)
+
+    difference > validity
+  end
 end

lib/pleroma/signature.ex

@@ -39,7 +39,7 @@ def key_id_to_actor_id(key_id) do
   def fetch_public_key(conn) do
     with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn),
          {:ok, actor_id} <- key_id_to_actor_id(kid),
-         {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id, force_http: true) do
+         {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
       {:ok, public_key}
     else
       e ->
@@ -50,8 +50,8 @@ def fetch_public_key(conn) do
   def refetch_public_key(conn) do
     with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn),
          {:ok, actor_id} <- key_id_to_actor_id(kid),
-         {:ok, _user} <- ActivityPub.make_user_from_ap_id(actor_id, force_http: true),
+         {:ok, _user} <- ActivityPub.make_user_from_ap_id(actor_id),
-         {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id, force_http: true) do
+         {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
       {:ok, public_key}
     else
       e ->

lib/pleroma/user.ex

@@ -245,6 +245,18 @@ def unquote(:"#{outgoing_relation_target}_ap_ids")(user, restrict_deactivated? \
     end
   end
 
+  def cached_blocked_users_ap_ids(user) do
+    Cachex.fetch!(:user_cache, "blocked_users_ap_ids:#{user.ap_id}", fn _ ->
+      blocked_users_ap_ids(user)
+    end)
+  end
+
+  def cached_muted_users_ap_ids(user) do
+    Cachex.fetch!(:user_cache, "muted_users_ap_ids:#{user.ap_id}", fn _ ->
+      muted_users_ap_ids(user)
+    end)
+  end
+
   defdelegate following_count(user), to: FollowingRelationship
   defdelegate following(user), to: FollowingRelationship
   defdelegate following?(follower, followed), to: FollowingRelationship
@@ -1068,6 +1080,8 @@ def invalidate_cache(user) do
     Cachex.del(:user_cache, "ap_id:#{user.ap_id}")
     Cachex.del(:user_cache, "nickname:#{user.nickname}")
     Cachex.del(:user_cache, "friends_ap_ids:#{user.ap_id}")
+    Cachex.del(:user_cache, "blocked_users_ap_ids:#{user.ap_id}")
+    Cachex.del(:user_cache, "muted_users_ap_ids:#{user.ap_id}")
   end
 
   @spec get_cached_by_ap_id(String.t()) :: User.t() | nil
@@ -1374,6 +1388,8 @@ def mute(%User{} = muter, %User{} = mutee, params \\ %{}) do
         )
       end
 
+      Cachex.del(:user_cache, "muted_users_ap_ids:#{muter.ap_id}")
+
       {:ok, Enum.filter([user_mute, user_notification_mute], & &1)}
     end
   end
@@ -1382,6 +1398,7 @@ def unmute(%User{} = muter, %User{} = mutee) do
     with {:ok, user_mute} <- UserRelationship.delete_mute(muter, mutee),
          {:ok, user_notification_mute} <-
            UserRelationship.delete_notification_mute(muter, mutee) do
+      Cachex.del(:user_cache, "muted_users_ap_ids:#{muter.ap_id}")
       {:ok, [user_mute, user_notification_mute]}
     end
   end
@@ -1827,12 +1844,12 @@ def html_filter_policy(%User{no_rich_text: true}) do
 
   def html_filter_policy(_), do: Config.get([:markup, :scrub_policy])
 
-  def fetch_by_ap_id(ap_id, opts \\ []), do: ActivityPub.make_user_from_ap_id(ap_id, opts)
+  def fetch_by_ap_id(ap_id), do: ActivityPub.make_user_from_ap_id(ap_id)
 
-  def get_or_fetch_by_ap_id(ap_id, opts \\ []) do
+  def get_or_fetch_by_ap_id(ap_id) do
     cached_user = get_cached_by_ap_id(ap_id)
 
-    maybe_fetched_user = needs_update?(cached_user) && fetch_by_ap_id(ap_id, opts)
+    maybe_fetched_user = needs_update?(cached_user) && fetch_by_ap_id(ap_id)
 
     case {cached_user, maybe_fetched_user} do
       {_, {:ok, %User{} = user}} ->
@@ -1905,8 +1922,8 @@ def public_key(%{public_key: public_key_pem}) when is_binary(public_key_pem) do
 
   def public_key(_), do: {:error, "key not found"}
 
-  def get_public_key_for_ap_id(ap_id, opts \\ []) do
+  def get_public_key_for_ap_id(ap_id) do
-    with {:ok, %User{} = user} <- get_or_fetch_by_ap_id(ap_id, opts),
+    with {:ok, %User{} = user} <- get_or_fetch_by_ap_id(ap_id),
          {:ok, public_key} <- public_key(user) do
       {:ok, public_key}
     else
@@ -2388,13 +2405,19 @@ def unblock_domain(user, domain_blocked) do
   @spec add_to_block(User.t(), User.t()) ::
           {:ok, UserRelationship.t()} | {:error, Ecto.Changeset.t()}
   defp add_to_block(%User{} = user, %User{} = blocked) do
-    UserRelationship.create_block(user, blocked)
+    with {:ok, relationship} <- UserRelationship.create_block(user, blocked) do
+      Cachex.del(:user_cache, "blocked_users_ap_ids:#{user.ap_id}")
+      {:ok, relationship}
+    end
   end
 
   @spec add_to_block(User.t(), User.t()) ::
           {:ok, UserRelationship.t()} | {:ok, nil} | {:error, Ecto.Changeset.t()}
   defp remove_from_block(%User{} = user, %User{} = blocked) do
-    UserRelationship.delete_block(user, blocked)
+    with {:ok, relationship} <- UserRelationship.delete_block(user, blocked) do
+      Cachex.del(:user_cache, "blocked_users_ap_ids:#{user.ap_id}")
+      {:ok, relationship}
+    end
   end
 
   def set_invisible(user, invisible) do

lib/pleroma/user/search.ex

@@ -85,7 +85,6 @@ defp search_query(query_string, for_user, following, top_user_ids) do
     |> base_query(following)
     |> filter_blocked_user(for_user)
     |> filter_invisible_users()
-    |> filter_discoverable_users()
     |> filter_internal_users()
     |> filter_blocked_domains(for_user)
     |> fts_search(query_string)
@@ -163,10 +162,6 @@ defp filter_invisible_users(query) do
     from(q in query, where: q.invisible == false)
   end
 
-  defp filter_discoverable_users(query) do
-    from(q in query, where: q.is_discoverable == true)
-  end
-
   defp filter_internal_users(query) do
     from(q in query, where: q.actor_type != "Application")
   end

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -123,7 +123,9 @@ def insert(map, local \\ true, fake \\ false, bypass_actor_check \\ false) when
       # Splice in the child object if we have one.
       activity = Maps.put_if_present(activity, :object, object)
 
-      BackgroundWorker.enqueue("fetch_data_for_activity", %{"activity_id" => activity.id})
+      ConcurrentLimiter.limit(Pleroma.Web.RichMedia.Helpers, fn ->
+        Task.start(fn -> Pleroma.Web.RichMedia.Helpers.fetch_data_for_activity(activity) end)
+      end)
 
       {:ok, activity}
     else
@@ -332,15 +334,21 @@ defp do_unfollow(follower, followed, activity_id, local) do
   end
 
   @spec flag(map()) :: {:ok, Activity.t()} | {:error, any()}
-  def flag(
+  def flag(params) do
-        %{
+    with {:ok, result} <- Repo.transaction(fn -> do_flag(params) end) do
-          actor: actor,
+      result
-          context: _context,
+    end
-          account: account,
+  end
-          statuses: statuses,
+
-          content: content
+  defp do_flag(
-        } = params
+         %{
-      ) do
+           actor: actor,
+           context: _context,
+           account: account,
+           statuses: statuses,
+           content: content
+         } = params
+       ) do
     # only accept false as false value
     local = !(params[:local] == false)
     forward = !(params[:forward] == false)
@@ -358,7 +366,8 @@ def flag(
          {:ok, activity} <- insert(flag_data, local),
          {:ok, stripped_activity} <- strip_report_status_data(activity),
          _ <- notify_and_stream(activity),
-         :ok <- maybe_federate(stripped_activity) do
+         :ok <-
+           maybe_federate(stripped_activity) do
       User.all_superusers()
       |> Enum.filter(fn user -> not is_nil(user.email) end)
       |> Enum.each(fn superuser ->
@@ -368,6 +377,8 @@ def flag(
       end)
 
       {:ok, activity}
+    else
+      {:error, error} -> Repo.rollback(error)
     end
   end
 
@@ -1289,12 +1300,10 @@ defp object_to_user_data(data) do
 
   def fetch_follow_information_for_user(user) do
     with {:ok, following_data} <-
-           Fetcher.fetch_and_contain_remote_object_from_id(user.following_address,
+           Fetcher.fetch_and_contain_remote_object_from_id(user.following_address),
-             force_http: true
-           ),
          {:ok, hide_follows} <- collection_private(following_data),
          {:ok, followers_data} <-
-           Fetcher.fetch_and_contain_remote_object_from_id(user.follower_address, force_http: true),
+           Fetcher.fetch_and_contain_remote_object_from_id(user.follower_address),
          {:ok, hide_followers} <- collection_private(followers_data) do
       {:ok,
        %{
@@ -1368,8 +1377,8 @@ def user_data_from_user_object(data) do
     end
   end
 
-  def fetch_and_prepare_user_from_ap_id(ap_id, opts \\ []) do
+  def fetch_and_prepare_user_from_ap_id(ap_id) do
-    with {:ok, data} <- Fetcher.fetch_and_contain_remote_object_from_id(ap_id, opts),
+    with {:ok, data} <- Fetcher.fetch_and_contain_remote_object_from_id(ap_id),
          {:ok, data} <- user_data_from_user_object(data) do
       {:ok, maybe_update_follow_information(data)}
     else
@@ -1412,13 +1421,13 @@ def maybe_handle_clashing_nickname(data) do
     end
   end
 
-  def make_user_from_ap_id(ap_id, opts \\ []) do
+  def make_user_from_ap_id(ap_id) do
     user = User.get_cached_by_ap_id(ap_id)
 
     if user && !User.ap_enabled?(user) do
       Transmogrifier.upgrade_user_from_ap_id(ap_id)
     else
-      with {:ok, data} <- fetch_and_prepare_user_from_ap_id(ap_id, opts) do
+      with {:ok, data} <- fetch_and_prepare_user_from_ap_id(ap_id) do
         if user do
           user
           |> User.remote_user_changeset(data)

lib/pleroma/web/activity_pub/activity_pub_controller.ex

@@ -82,7 +82,8 @@ def user(conn, %{"nickname" => nickname}) do
   def object(conn, _) do
     with ap_id <- Endpoint.url() <> conn.request_path,
          %Object{} = object <- Object.get_cached_by_ap_id(ap_id),
-         {_, true} <- {:public?, Visibility.is_public?(object)} do
+         {_, true} <- {:public?, Visibility.is_public?(object)},
+         {_, false} <- {:local?, Visibility.is_local_public?(object)} do
       conn
       |> assign(:tracking_fun_data, object.id)
       |> set_cache_ttl_for(object)
@@ -92,6 +93,9 @@ def object(conn, _) do
     else
       {:public?, false} ->
         {:error, :not_found}
+
+      {:local?, true} ->
+        {:error, :not_found}
     end
   end
 
@@ -108,7 +112,8 @@ def track_object_fetch(conn, object_id) do
   def activity(conn, _params) do
     with ap_id <- Endpoint.url() <> conn.request_path,
          %Activity{} = activity <- Activity.normalize(ap_id),
-         {_, true} <- {:public?, Visibility.is_public?(activity)} do
+         {_, true} <- {:public?, Visibility.is_public?(activity)},
+         {_, false} <- {:local?, Visibility.is_local_public?(activity)} do
       conn
       |> maybe_set_tracking_data(activity)
       |> set_cache_ttl_for(activity)
@@ -117,6 +122,7 @@ def activity(conn, _params) do
       |> render("object.json", object: activity)
     else
       {:public?, false} -> {:error, :not_found}
+      {:local?, true} -> {:error, :not_found}
       nil -> {:error, :not_found}
     end
   end

lib/pleroma/web/activity_pub/builder.ex

@@ -222,6 +222,9 @@ def announce(actor, object, options \\ []) do
         actor.ap_id == Relay.ap_id() ->
           [actor.follower_address]
 
+        public? and Visibility.is_local_public?(object) ->
+          [actor.follower_address, object.data["actor"], Pleroma.Constants.as_local_public()]
+
         public? ->
           [actor.follower_address, object.data["actor"], Pleroma.Constants.as_public()]
 

lib/pleroma/web/activity_pub/mrf/media_proxy_warming_policy.ex

@@ -8,7 +8,6 @@ defmodule Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy do
 
   alias Pleroma.HTTP
   alias Pleroma.Web.MediaProxy
-  alias Pleroma.Workers.BackgroundWorker
 
   require Logger
 
@@ -17,7 +16,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy do
     recv_timeout: 10_000
   ]
 
-  def perform(:prefetch, url) do
+  defp prefetch(url) do
     # Fetching only proxiable resources
     if MediaProxy.enabled?() and MediaProxy.url_proxiable?(url) do
       # If preview proxy is enabled, it'll also hit media proxy (so we're caching both requests)
@@ -25,17 +24,25 @@ def perform(:prefetch, url) do
 
       Logger.debug("Prefetching #{inspect(url)} as #{inspect(prefetch_url)}")
 
-      HTTP.get(prefetch_url, [], @adapter_options)
+      if Pleroma.Config.get(:env) == :test do
+        fetch(prefetch_url)
+      else
+        ConcurrentLimiter.limit(MediaProxy, fn ->
+          Task.start(fn -> fetch(prefetch_url) end)
+        end)
+      end
     end
   end
 
-  def perform(:preload, %{"object" => %{"attachment" => attachments}} = _message) do
+  defp fetch(url), do: HTTP.get(url, [], @adapter_options)
+
+  defp preload(%{"object" => %{"attachment" => attachments}} = _message) do
     Enum.each(attachments, fn
       %{"url" => url} when is_list(url) ->
         url
         |> Enum.each(fn
           %{"href" => href} ->
-            BackgroundWorker.enqueue("media_proxy_prefetch", %{"url" => href})
+            prefetch(href)
 
           x ->
             Logger.debug("Unhandled attachment URL object #{inspect(x)}")
@@ -51,7 +58,7 @@ def filter(
         %{"type" => "Create", "object" => %{"attachment" => attachments} = _object} = message
       )
       when is_list(attachments) and length(attachments) > 0 do
-    BackgroundWorker.enqueue("media_proxy_preload", %{"message" => message})
+    preload(message)
 
     {:ok, message}
   end

lib/pleroma/web/activity_pub/object_validators/announce_validator.ex

@@ -67,7 +67,12 @@ def validate_announcable(cng) do
          %Object{} = object <- Object.get_cached_by_ap_id(object),
          false <- Visibility.is_public?(object) do
       same_actor = object.data["actor"] == actor.ap_id
-      is_public = Pleroma.Constants.as_public() in (get_field(cng, :to) ++ get_field(cng, :cc))
+      recipients = get_field(cng, :to) ++ get_field(cng, :cc)
+      local_public = Pleroma.Constants.as_local_public()
+
+      is_public =
+        Enum.member?(recipients, Pleroma.Constants.as_public()) or
+          Enum.member?(recipients, local_public)
 
       cond do
         same_actor && is_public ->

lib/pleroma/web/activity_pub/pipeline.ex

@@ -11,6 +11,7 @@ defmodule Pleroma.Web.ActivityPub.Pipeline do
   alias Pleroma.Web.ActivityPub.MRF
   alias Pleroma.Web.ActivityPub.ObjectValidator
   alias Pleroma.Web.ActivityPub.SideEffects
+  alias Pleroma.Web.ActivityPub.Visibility
   alias Pleroma.Web.Federator
 
   @spec common_pipeline(map(), keyword()) ::
@@ -55,7 +56,7 @@ defp maybe_federate(%Activity{} = activity, meta) do
     with {:ok, local} <- Keyword.fetch(meta, :local) do
       do_not_federate = meta[:do_not_federate] || !Config.get([:instance, :federating])
 
-      if !do_not_federate && local do
+      if !do_not_federate and local and not Visibility.is_local_public?(activity) do
         activity =
           if object = Keyword.get(meta, :object_data) do
             %{activity | data: Map.put(activity.data, "object", object)}

lib/pleroma/web/activity_pub/publisher.ex

@@ -13,7 +13,6 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.Relay
   alias Pleroma.Web.ActivityPub.Transmogrifier
-  alias Pleroma.Web.FedSockets
 
   require Pleroma.Constants
 
@@ -50,28 +49,6 @@ def is_representable?(%Activity{} = activity) do
   """
   def publish_one(%{inbox: inbox, json: json, actor: %User{} = actor, id: id} = params) do
     Logger.debug("Federating #{id} to #{inbox}")
-
-    case FedSockets.get_or_create_fed_socket(inbox) do
-      {:ok, fedsocket} ->
-        Logger.debug("publishing via fedsockets - #{inspect(inbox)}")
-        FedSockets.publish(fedsocket, json)
-
-      _ ->
-        Logger.debug("publishing via http - #{inspect(inbox)}")
-        http_publish(inbox, actor, json, params)
-    end
-  end
-
-  def publish_one(%{actor_id: actor_id} = params) do
-    actor = User.get_cached_by_id(actor_id)
-
-    params
-    |> Map.delete(:actor_id)
-    |> Map.put(:actor, actor)
-    |> publish_one()
-  end
-
-  defp http_publish(inbox, actor, json, params) do
     uri = %{path: path} = URI.parse(inbox)
     digest = "SHA-256=" <> (:crypto.hash(:sha256, json) |> Base.encode64())
 
@@ -110,6 +87,15 @@ defp http_publish(inbox, actor, json, params) do
     end
   end
 
+  def publish_one(%{actor_id: actor_id} = params) do
+    actor = User.get_cached_by_id(actor_id)
+
+    params
+    |> Map.delete(:actor_id)
+    |> Map.put(:actor, actor)
+    |> publish_one()
+  end
+
   defp signature_host(%URI{port: port, scheme: scheme, host: host}) do
     if port == URI.default_port(scheme) do
       host

lib/pleroma/web/activity_pub/side_effects.ex

@@ -24,7 +24,6 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.Push
   alias Pleroma.Web.Streamer
-  alias Pleroma.Workers.BackgroundWorker
 
   require Logger
 
@@ -191,7 +190,9 @@ def handle(%{data: %{"type" => "Create"}} = activity, meta) do
         Object.increase_replies_count(in_reply_to)
       end
 
-      BackgroundWorker.enqueue("fetch_data_for_activity", %{"activity_id" => activity.id})
+      ConcurrentLimiter.limit(Pleroma.Web.RichMedia.Helpers, fn ->
+        Task.start(fn -> Pleroma.Web.RichMedia.Helpers.fetch_data_for_activity(activity) end)
+      end)
 
       meta =
         meta

lib/pleroma/web/activity_pub/transmogrifier.ex

@@ -1008,7 +1008,7 @@ def perform(:user_upgrade, user) do
 
   def upgrade_user_from_ap_id(ap_id) do
     with %User{local: false} = user <- User.get_cached_by_ap_id(ap_id),
-         {:ok, data} <- ActivityPub.fetch_and_prepare_user_from_ap_id(ap_id, force_http: true),
+         {:ok, data} <- ActivityPub.fetch_and_prepare_user_from_ap_id(ap_id),
          {:ok, user} <- update_user(user, data) do
       TransmogrifierWorker.enqueue("user_upgrade", %{"user_id" => user.id})
       {:ok, user}

lib/pleroma/web/activity_pub/utils.ex

@@ -175,7 +175,8 @@ def maybe_federate(%Activity{local: true, data: %{"type" => type}} = activity) d
     outgoing_blocks = Config.get([:activitypub, :outgoing_blocks])
 
     with true <- Config.get!([:instance, :federating]),
-         true <- type != "Block" || outgoing_blocks do
+         true <- type != "Block" || outgoing_blocks,
+         false <- Visibility.is_local_public?(activity) do
       Pleroma.Web.Federator.publish(activity)
     end
 
@@ -701,14 +702,30 @@ def make_flag_data(%{actor: actor, context: context, content: content} = params,
 
   def make_flag_data(_, _), do: %{}
 
-  defp build_flag_object(%{account: account, statuses: statuses} = _) do
+  defp build_flag_object(%{account: account, statuses: statuses}) do
-    [account.ap_id] ++ build_flag_object(%{statuses: statuses})
+    [account.ap_id | build_flag_object(%{statuses: statuses})]
   end
 
   defp build_flag_object(%{statuses: statuses}) do
     Enum.map(statuses || [], &build_flag_object/1)
   end
 
+  defp build_flag_object(%Activity{data: %{"id" => id}, object: %{data: data}}) do
+    activity_actor = User.get_by_ap_id(data["actor"])
+
+    %{
+      "type" => "Note",
+      "id" => id,
+      "content" => data["content"],
+      "published" => data["published"],
+      "actor" =>
+        AccountView.render(
+          "show.json",
+          %{user: activity_actor, skip_visibility_check: true}
+        )
+    }
+  end
+
   defp build_flag_object(act) when is_map(act) or is_binary(act) do
     id =
       case act do
@@ -719,22 +736,14 @@ defp build_flag_object(act) when is_map(act) or is_binary(act) do
 
     case Activity.get_by_ap_id_with_object(id) do
       %Activity{} = activity ->
-        activity_actor = User.get_by_ap_id(activity.object.data["actor"])
+        build_flag_object(activity)
 
-        %{
+      nil ->
-          "type" => "Note",
+        if activity = Activity.get_by_object_ap_id_with_object(id) do
-          "id" => activity.data["id"],
+          build_flag_object(activity)
-          "content" => activity.object.data["content"],
+        else
-          "published" => activity.object.data["published"],
+          %{"id" => id, "deleted" => true}
-          "actor" =>
+        end
-            AccountView.render(
-              "show.json",
-              %{user: activity_actor, skip_visibility_check: true}
-            )
-        }
-
-      _ ->
-        %{"id" => id, "deleted" => true}
     end
   end
 

lib/pleroma/web/activity_pub/views/user_view.ex

@@ -110,6 +110,7 @@ def render("user.json", %{user: user}) do
       "endpoints" => endpoints,
       "attachment" => fields,
       "tag" => emoji_tags,
+      # Note: key name is indeed "discoverable" (not an error)
       "discoverable" => user.is_discoverable,
       "capabilities" => capabilities
     }

lib/pleroma/web/activity_pub/visibility.ex

@@ -17,7 +17,19 @@ def is_public?(%Object{data: data}), do: is_public?(data)
   def is_public?(%Activity{data: %{"type" => "Move"}}), do: true
   def is_public?(%Activity{data: data}), do: is_public?(data)
   def is_public?(%{"directMessage" => true}), do: false
-  def is_public?(data), do: Utils.label_in_message?(Pleroma.Constants.as_public(), data)
+
+  def is_public?(data) do
+    Utils.label_in_message?(Pleroma.Constants.as_public(), data) or
+      Utils.label_in_message?(Pleroma.Constants.as_local_public(), data)
+  end
+
+  def is_local_public?(%Object{data: data}), do: is_local_public?(data)
+  def is_local_public?(%Activity{data: data}), do: is_local_public?(data)
+
+  def is_local_public?(data) do
+    Utils.label_in_message?(Pleroma.Constants.as_local_public(), data) and
+      not Utils.label_in_message?(Pleroma.Constants.as_public(), data)
+  end
 
   def is_private?(activity) do
     with false <- is_public?(activity),
@@ -114,6 +126,9 @@ def get_visibility(object) do
       Pleroma.Constants.as_public() in cc ->
         "unlisted"
 
+      Pleroma.Constants.as_local_public() in to ->
+        "local"
+
       # this should use the sql for the object's activity
       Enum.any?(to, &String.contains?(&1, "/followers")) ->
         "private"

lib/pleroma/web/admin_api/controllers/frontend_controller.ex

@@ -0,0 +1,40 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.FrontendController do
+  use Pleroma.Web, :controller
+
+  alias Pleroma.Config
+  alias Pleroma.Web.Plugs.OAuthScopesPlug
+
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
+  plug(OAuthScopesPlug, %{scopes: ["write"], admin: true} when action == :install)
+  plug(OAuthScopesPlug, %{scopes: ["read"], admin: true} when action == :index)
+  action_fallback(Pleroma.Web.AdminAPI.FallbackController)
+
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.Admin.FrontendOperation
+
+  def index(conn, _params) do
+    installed = installed()
+
+    frontends =
+      [:frontends, :available]
+      |> Config.get([])
+      |> Enum.map(fn {name, desc} ->
+        Map.put(desc, "installed", name in installed)
+      end)
+
+    render(conn, "index.json", frontends: frontends)
+  end
+
+  def install(%{body_params: params} = conn, _params) do
+    with :ok <- Pleroma.Frontend.install(params.name, Map.delete(params, :name)) do
+      index(conn, %{})
+    end
+  end
+
+  defp installed do
+    File.ls!(Pleroma.Frontend.dir())
+  end
+end

lib/pleroma/web/admin_api/views/frontend_view.ex

@@ -0,0 +1,21 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.FrontendView do
+  use Pleroma.Web, :view
+
+  def render("index.json", %{frontends: frontends}) do
+    render_many(frontends, __MODULE__, "show.json")
+  end
+
+  def render("show.json", %{frontend: frontend}) do
+    %{
+      name: frontend["name"],
+      git: frontend["git"],
+      build_url: frontend["build_url"],
+      ref: frontend["ref"],
+      installed: frontend["installed"]
+    }
+  end
+end

lib/pleroma/web/api_spec/operations/account_operation.ex

@@ -139,6 +139,12 @@ def statuses_operation do
             :query,
             %Schema{type: :array, items: VisibilityScope},
             "Exclude visibilities"
+          ),
+          Operation.parameter(
+            :with_muted,
+            :query,
+            BooleanLike,
+            "Include reactions from muted acccounts."
           )
         ] ++ pagination_params(),
       responses: %{
@@ -618,7 +624,7 @@ defp update_credentials_request do
           allOf: [BooleanLike],
           nullable: true,
           description:
-            "Discovery of this account in search results and other services is allowed."
+            "Discovery (listing, indexing) of this account by external services (search bots etc.) is allowed."
         },
         actor_type: ActorType
       },

lib/pleroma/web/api_spec/operations/admin/frontend_operation.ex

@@ -0,0 +1,85 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Admin.FrontendOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Admin", "Reports"],
+      summary: "Get a list of available frontends",
+      operationId: "AdminAPI.FrontendController.index",
+      security: [%{"oAuth" => ["read"]}],
+      responses: %{
+        200 => Operation.response("Response", "application/json", list_of_frontends()),
+        403 => Operation.response("Forbidden", "application/json", ApiError)
+      }
+    }
+  end
+
+  def install_operation do
+    %Operation{
+      tags: ["Admin", "Reports"],
+      summary: "Install a frontend",
+      operationId: "AdminAPI.FrontendController.install",
+      security: [%{"oAuth" => ["read"]}],
+      requestBody: request_body("Parameters", install_request(), required: true),
+      responses: %{
+        200 => Operation.response("Response", "application/json", list_of_frontends()),
+        403 => Operation.response("Forbidden", "application/json", ApiError),
+        400 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  defp list_of_frontends do
+    %Schema{
+      type: :array,
+      items: %Schema{
+        type: :object,
+        properties: %{
+          name: %Schema{type: :string},
+          git: %Schema{type: :string, format: :uri, nullable: true},
+          build_url: %Schema{type: :string, format: :uri, nullable: true},
+          ref: %Schema{type: :string},
+          installed: %Schema{type: :boolean}
+        }
+      }
+    }
+  end
+
+  defp install_request do
+    %Schema{
+      title: "FrontendInstallRequest",
+      type: :object,
+      required: [:name],
+      properties: %{
+        name: %Schema{
+          type: :string
+        },
+        ref: %Schema{
+          type: :string
+        },
+        file: %Schema{
+          type: :string
+        },
+        build_url: %Schema{
+          type: :string
+        },
+        build_dir: %Schema{
+          type: :string
+        }
+      }
+    }
+  end
+end

lib/pleroma/web/api_spec/operations/emoji_reaction_operation.ex

@@ -24,6 +24,12 @@ def index_operation do
         Operation.parameter(:id, :path, FlakeID, "Status ID", required: true),
         Operation.parameter(:emoji, :path, :string, "Filter by a single unicode emoji",
           required: nil
+        ),
+        Operation.parameter(
+          :with_muted,
+          :query,
+          :boolean,
+          "Include reactions from muted acccounts."
         )
       ],
       security: [%{"oAuth" => ["read:statuses"]}],

lib/pleroma/web/api_spec/operations/status_operation.ex

@@ -31,6 +31,12 @@ def index_operation do
           :query,
           %Schema{type: :array, items: FlakeID},
           "Array of status IDs"
+        ),
+        Operation.parameter(
+          :with_muted,
+          :query,
+          BooleanLike,
+          "Include reactions from muted acccounts."
         )
       ],
       operationId: "StatusController.index",
@@ -67,7 +73,15 @@ def show_operation do
       description: "View information about a status",
       operationId: "StatusController.show",
       security: [%{"oAuth" => ["read:statuses"]}],
-      parameters: [id_param()],
+      parameters: [
+        id_param(),
+        Operation.parameter(
+          :with_muted,
+          :query,
+          BooleanLike,
+          "Include reactions from muted acccounts."
+        )
+      ],
       responses: %{
         200 => status_response(),
         404 => Operation.response("Not Found", "application/json", ApiError)

lib/pleroma/web/api_spec/operations/subscription_operation.ex

@@ -146,6 +146,11 @@ defp create_request do
                   allOf: [BooleanLike],
                   nullable: true,
                   description: "Receive chat notifications?"
+                },
+                "pleroma:emoji_reaction": %Schema{
+                  allOf: [BooleanLike],
+                  nullable: true,
+                  description: "Receive emoji reaction notifications?"
                 }
               }
             }
@@ -210,6 +215,16 @@ defp update_request do
                   allOf: [BooleanLike],
                   nullable: true,
                   description: "Receive poll notifications?"
+                },
+                "pleroma:chat_mention": %Schema{
+                  allOf: [BooleanLike],
+                  nullable: true,
+                  description: "Receive chat notifications?"
+                },
+                "pleroma:emoji_reaction": %Schema{
+                  allOf: [BooleanLike],
+                  nullable: true,
+                  description: "Receive emoji reaction notifications?"
                 }
               }
             }

lib/pleroma/web/api_spec/schemas/account.ex

@@ -127,7 +127,7 @@ defmodule Pleroma.Web.ApiSpec.Schemas.Account do
               discoverable: %Schema{
                 type: :boolean,
                 description:
-                  "whether the user allows discovery of the account in search results and other services."
+                  "whether the user allows indexing / listing of the account by external services (search engines etc.)."
               },
               no_rich_text: %Schema{
                 type: :boolean,

lib/pleroma/web/api_spec/schemas/visibility_scope.ex

@@ -9,6 +9,6 @@ defmodule Pleroma.Web.ApiSpec.Schemas.VisibilityScope do
     title: "VisibilityScope",
     description: "Status visibility",
     type: :string,
-    enum: ["public", "unlisted", "private", "direct", "list"]
+    enum: ["public", "unlisted", "local", "private", "direct", "list"]
   })
 end

lib/pleroma/web/common_api.ex

@@ -15,6 +15,7 @@ defmodule Pleroma.Web.CommonAPI do
   alias Pleroma.Web.ActivityPub.Pipeline
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.ActivityPub.Visibility
+  alias Pleroma.Web.CommonAPI.ActivityDraft
 
   import Pleroma.Web.Gettext
   import Pleroma.Web.CommonAPI.Utils
@@ -358,7 +359,7 @@ def public_announce?(object, _) do
   def get_visibility(_, _, %Participation{}), do: {"direct", "direct"}
 
   def get_visibility(%{visibility: visibility}, in_reply_to, _)
-      when visibility in ~w{public unlisted private direct},
+      when visibility in ~w{public local unlisted private direct},
       do: {visibility, get_replied_to_visibility(in_reply_to)}
 
   def get_visibility(%{visibility: "list:" <> list_id}, in_reply_to, _) do
@@ -399,31 +400,13 @@ def check_expiry_date(expiry_str) do
   end
 
   def listen(user, data) do
-    visibility = Map.get(data, :visibility, "public")
+    with {:ok, draft} <- ActivityDraft.listen(user, data) do
-
+      ActivityPub.listen(draft.changes)
-    with {to, cc} <- get_to_and_cc(user, [], nil, visibility, nil),
-         listen_data <-
-           data
-           |> Map.take([:album, :artist, :title, :length])
-           |> Map.new(fn {key, value} -> {to_string(key), value} end)
-           |> Map.put("type", "Audio")
-           |> Map.put("to", to)
-           |> Map.put("cc", cc)
-           |> Map.put("actor", user.ap_id),
-         {:ok, activity} <-
-           ActivityPub.listen(%{
-             actor: user,
-             to: to,
-             object: listen_data,
-             context: Utils.generate_context_id(),
-             additional: %{"cc" => cc}
-           }) do
-      {:ok, activity}
     end
   end
 
   def post(user, %{status: _} = data) do
-    with {:ok, draft} <- Pleroma.Web.CommonAPI.ActivityDraft.create(user, data) do
+    with {:ok, draft} <- ActivityDraft.create(user, data) do
       ActivityPub.create(draft.changes, draft.preview?)
     end
   end

lib/pleroma/web/common_api/activity_draft.ex

@@ -22,7 +22,7 @@ defmodule Pleroma.Web.CommonAPI.ActivityDraft do
             in_reply_to_conversation: nil,
             visibility: nil,
             expires_at: nil,
-            poll: nil,
+            extra: nil,
             emoji: %{},
             content_html: nil,
             mentions: [],
@@ -35,9 +35,14 @@ defmodule Pleroma.Web.CommonAPI.ActivityDraft do
             preview?: false,
             changes: %{}
 
-  def create(user, params) do
+  def new(user, params) do
     %__MODULE__{user: user}
     |> put_params(params)
+  end
+
+  def create(user, params) do
+    user
+    |> new(params)
     |> status()
     |> summary()
     |> with_valid(&attachments/1)
@@ -57,6 +62,30 @@ def create(user, params) do
     |> validate()
   end
 
+  def listen(user, params) do
+    user
+    |> new(params)
+    |> visibility()
+    |> to_and_cc()
+    |> context()
+    |> listen_object()
+    |> with_valid(&changes/1)
+    |> validate()
+  end
+
+  defp listen_object(draft) do
+    object =
+      draft.params
+      |> Map.take([:album, :artist, :title, :length])
+      |> Map.new(fn {key, value} -> {to_string(key), value} end)
+      |> Map.put("type", "Audio")
+      |> Map.put("to", draft.to)
+      |> Map.put("cc", draft.cc)
+      |> Map.put("actor", draft.user.ap_id)
+
+    %__MODULE__{draft | object: object}
+  end
+
   defp put_params(draft, params) do
     params = Map.put_new(params, :in_reply_to_status_id, params[:in_reply_to_id])
     %__MODULE__{draft | params: params}
@@ -121,7 +150,7 @@ defp expires_at(draft) do
   defp poll(draft) do
     case Utils.make_poll_data(draft.params) do
       {:ok, {poll, poll_emoji}} ->
-        %__MODULE__{draft | poll: poll, emoji: Map.merge(draft.emoji, poll_emoji)}
+        %__MODULE__{draft | extra: poll, emoji: Map.merge(draft.emoji, poll_emoji)}
 
       {:error, message} ->
         add_error(draft, message)
@@ -129,32 +158,18 @@ defp poll(draft) do
   end
 
   defp content(draft) do
-    {content_html, mentions, tags} =
+    {content_html, mentioned_users, tags} = Utils.make_content_html(draft)
-      Utils.make_content_html(
+
-        draft.status,
+    mentions =
-        draft.attachments,
+      mentioned_users
-        draft.params,
+      |> Enum.map(fn {_, mentioned_user} -> mentioned_user.ap_id end)
-        draft.visibility
+      |> Utils.get_addressed_users(draft.params[:to])
-      )
 
     %__MODULE__{draft | content_html: content_html, mentions: mentions, tags: tags}
   end
 
   defp to_and_cc(draft) do
-    addressed_users =
+    {to, cc} = Utils.get_to_and_cc(draft)
-      draft.mentions
-      |> Enum.map(fn {_, mentioned_user} -> mentioned_user.ap_id end)
-      |> Utils.get_addressed_users(draft.params[:to])
-
-    {to, cc} =
-      Utils.get_to_and_cc(
-        draft.user,
-        addressed_users,
-        draft.in_reply_to,
-        draft.visibility,
-        draft.in_reply_to_conversation
-      )
-
     %__MODULE__{draft | to: to, cc: cc}
   end
 
@@ -172,19 +187,7 @@ defp object(draft) do
     emoji = Map.merge(Pleroma.Emoji.Formatter.get_emoji_map(draft.full_payload), draft.emoji)
 
     object =
-      Utils.make_note_data(
+      Utils.make_note_data(draft)
-        draft.user.ap_id,
-        draft.to,
-        draft.context,
-        draft.content_html,
-        draft.attachments,
-        draft.in_reply_to,
-        draft.tags,
-        draft.summary,
-        draft.cc,
-        draft.sensitive,
-        draft.poll
-      )
       |> Map.put("emoji", emoji)
       |> Map.put("source", draft.status)
 

lib/pleroma/web/common_api/utils.ex

@@ -16,6 +16,7 @@ defmodule Pleroma.Web.CommonAPI.Utils do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.ActivityPub.Visibility
+  alias Pleroma.Web.CommonAPI.ActivityDraft
   alias Pleroma.Web.MediaProxy
   alias Pleroma.Web.Plugs.AuthenticationPlug
 
@@ -50,67 +51,62 @@ def attachments_from_ids_descs(ids, descs_str) do
     {_, descs} = Jason.decode(descs_str)
 
     Enum.map(ids, fn media_id ->
-      case Repo.get(Object, media_id) do
+      with %Object{data: data} <- Repo.get(Object, media_id) do
-        %Object{data: data} ->
+        Map.put(data, "name", descs[media_id])
-          Map.put(data, "name", descs[media_id])
-
-        _ ->
-          nil
       end
     end)
     |> Enum.reject(&is_nil/1)
   end
 
-  @spec get_to_and_cc(
+  @spec get_to_and_cc(ActivityDraft.t()) :: {list(String.t()), list(String.t())}
-          User.t(),
-          list(String.t()),
-          Activity.t() | nil,
-          String.t(),
-          Participation.t() | nil
-        ) :: {list(String.t()), list(String.t())}
 
-  def get_to_and_cc(_, _, _, _, %Participation{} = participation) do
+  def get_to_and_cc(%{in_reply_to_conversation: %Participation{} = participation}) do
     participation = Repo.preload(participation, :recipients)
     {Enum.map(participation.recipients, & &1.ap_id), []}
   end
 
-  def get_to_and_cc(user, mentioned_users, inReplyTo, "public", _) do
+  def get_to_and_cc(%{visibility: visibility} = draft) when visibility in ["public", "local"] do
-    to = [Pleroma.Constants.as_public() | mentioned_users]
+    to =
-    cc = [user.follower_address]
+      case visibility do
+        "public" -> [Pleroma.Constants.as_public() | draft.mentions]
+        "local" -> [Pleroma.Constants.as_local_public() | draft.mentions]
+      end
 
-    if inReplyTo do
+    cc = [draft.user.follower_address]
-      {Enum.uniq([inReplyTo.data["actor"] | to]), cc}
+
+    if draft.in_reply_to do
+      {Enum.uniq([draft.in_reply_to.data["actor"] | to]), cc}
     else
       {to, cc}
     end
   end
 
-  def get_to_and_cc(user, mentioned_users, inReplyTo, "unlisted", _) do
+  def get_to_and_cc(%{visibility: "unlisted"} = draft) do
-    to = [user.follower_address | mentioned_users]
+    to = [draft.user.follower_address | draft.mentions]
     cc = [Pleroma.Constants.as_public()]
 
-    if inReplyTo do
+    if draft.in_reply_to do
-      {Enum.uniq([inReplyTo.data["actor"] | to]), cc}
+      {Enum.uniq([draft.in_reply_to.data["actor"] | to]), cc}
     else
       {to, cc}
     end
   end
 
-  def get_to_and_cc(user, mentioned_users, inReplyTo, "private", _) do
+  def get_to_and_cc(%{visibility: "private"} = draft) do
-    {to, cc} = get_to_and_cc(user, mentioned_users, inReplyTo, "direct", nil)
+    {to, cc} = get_to_and_cc(struct(draft, visibility: "direct"))
-    {[user.follower_address | to], cc}
+    {[draft.user.follower_address | to], cc}
   end
 
-  def get_to_and_cc(_user, mentioned_users, inReplyTo, "direct", _) do
+  def get_to_and_cc(%{visibility: "direct"} = draft) do
     # If the OP is a DM already, add the implicit actor.
-    if inReplyTo && Visibility.is_direct?(inReplyTo) do
+    if draft.in_reply_to && Visibility.is_direct?(draft.in_reply_to) do
-      {Enum.uniq([inReplyTo.data["actor"] | mentioned_users]), []}
+      {Enum.uniq([draft.in_reply_to.data["actor"] | draft.mentions]), []}
     else
-      {mentioned_users, []}
+      {draft.mentions, []}
     end
   end
 
-  def get_to_and_cc(_user, mentions, _inReplyTo, {:list, _}, _), do: {mentions, []}
+  def get_to_and_cc(%{visibility: {:list, _}, mentions: mentions}), do: {mentions, []}
 
   def get_addressed_users(_, to) when is_list(to) do
     User.get_ap_ids_by_nicknames(to)
@@ -203,30 +199,25 @@ defp validate_poll_expiration(expires_in, %{min_expiration: min, max_expiration:
     end
   end
 
-  def make_content_html(
+  def make_content_html(%ActivityDraft{} = draft) do
-        status,
-        attachments,
-        data,
-        visibility
-      ) do
     attachment_links =
-      data
+      draft.params
       |> Map.get("attachment_links", Config.get([:instance, :attachment_links]))
       |> truthy_param?()
 
-    content_type = get_content_type(data[:content_type])
+    content_type = get_content_type(draft.params[:content_type])
 
     options =
-      if visibility == "direct" && Config.get([:instance, :safe_dm_mentions]) do
+      if draft.visibility == "direct" && Config.get([:instance, :safe_dm_mentions]) do
         [safe_mention: true]
       else
         []
       end
 
-    status
+    draft.status
     |> format_input(content_type, options)
-    |> maybe_add_attachments(attachments, attachment_links)
+    |> maybe_add_attachments(draft.attachments, attachment_links)
-    |> maybe_add_nsfw_tag(data)
+    |> maybe_add_nsfw_tag(draft.params)
   end
 
   defp get_content_type(content_type) do
@@ -308,33 +299,21 @@ def format_input(text, "text/markdown", options) do
     |> Formatter.html_escape("text/html")
   end
 
-  def make_note_data(
+  def make_note_data(%ActivityDraft{} = draft) do
-        actor,
-        to,
-        context,
-        content_html,
-        attachments,
-        in_reply_to,
-        tags,
-        summary \\ nil,
-        cc \\ [],
-        sensitive \\ false,
-        extra_params \\ %{}
-      ) do
     %{
       "type" => "Note",
-      "to" => to,
+      "to" => draft.to,
-      "cc" => cc,
+      "cc" => draft.cc,
-      "content" => content_html,
+      "content" => draft.content_html,
-      "summary" => summary,
+      "summary" => draft.summary,
-      "sensitive" => truthy_param?(sensitive),
+      "sensitive" => draft.sensitive,
-      "context" => context,
+      "context" => draft.context,
-      "attachment" => attachments,
+      "attachment" => draft.attachments,
-      "actor" => actor,
+      "actor" => draft.user.ap_id,
-      "tag" => Keyword.values(tags) |> Enum.uniq()
+      "tag" => Keyword.values(draft.tags) |> Enum.uniq()
     }
-    |> add_in_reply_to(in_reply_to)
+    |> add_in_reply_to(draft.in_reply_to)
-    |> Map.merge(extra_params)
+    |> Map.merge(draft.extra)
   end
 
   defp add_in_reply_to(object, nil), do: object

lib/pleroma/web/fed_sockets.ex

@@ -1,185 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.FedSockets do
-  @moduledoc """
-  This documents the FedSockets framework. A framework for federating
-  ActivityPub objects between servers via persistant WebSocket connections.
-
-  FedSockets allow servers to authenticate on first contact and maintain that
-  connection, eliminating the need to authenticate every time data needs to be shared.
-
-  ## Protocol
-  FedSockets currently support 2 types of data transfer:
-    * `publish` method which doesn't require a response
-    * `fetch` method requires a response be sent
-
-    ### Publish
-    The publish operation sends a json encoded map of the shape:
-      %{action: :publish, data: json}
-    and accepts (but does not require) a reply of form:
-      %{"action" => "publish_reply"}
-
-    The outgoing params represent
-      * data: ActivityPub object encoded into json
-
-
-    ### Fetch
-    The fetch operation sends a json encoded map of the shape:
-      %{action: :fetch, data: id, uuid: fetch_uuid}
-    and requires a reply of form:
-      %{"action" => "fetch_reply", "uuid" => uuid, "data" => data}
-
-    The outgoing params represent
-      * id: an ActivityPub object URI
-      * uuid: a unique uuid generated by the sender
-
-    The reply params represent
-      * data: an ActivityPub object encoded into json
-      * uuid: the uuid sent along with the fetch request
-
-  ## Examples
-  Clients of FedSocket transfers shouldn't need to use any of the functions outside of this module.
-
-  A typical publish operation can be performed through the following code, and a fetch operation in a similar manner.
-
-    case FedSockets.get_or_create_fed_socket(inbox) do
-      {:ok, fedsocket} ->
-        FedSockets.publish(fedsocket, json)
-
-      _ ->
-        alternative_publish(inbox, actor, json, params)
-    end
-
-  ## Configuration
-  FedSockets have the following config settings
-
-  config :pleroma, :fed_sockets,
-  enabled: true,
-  ping_interval: :timer.seconds(15),
-  connection_duration: :timer.hours(1),
-  rejection_duration: :timer.hours(1),
-  fed_socket_fetches: [
-    default: 12_000,
-    interval: 3_000,
-    lazy: false
-  ]
-    * enabled - turn FedSockets on or off with this flag. Can be toggled at runtime.
-    * connection_duration - How long a FedSocket can sit idle before it's culled.
-    * rejection_duration - After failing to make a FedSocket connection a host will be excluded
-    from further connections for this amount of time
-    * fed_socket_fetches - Use these parameters to pass options to the Cachex queue backing the FetchRegistry
-    * fed_socket_rejections - Use these parameters to pass options to the Cachex queue backing the FedRegistry
-
-    Cachex options are
-      * default: the minimum amount of time a fetch can wait before it times out.
-      * interval: the interval between checks for timed out entries. This plus the default represent the maximum time allowed
-      * lazy: leave at false for consistant and fast lookups, set to true for stricter timeout enforcement
-
-  """
-  require Logger
-
-  alias Pleroma.Web.FedSockets.FedRegistry
-  alias Pleroma.Web.FedSockets.FedSocket
-  alias Pleroma.Web.FedSockets.SocketInfo
-
-  @doc """
-  returns a FedSocket for the given origin. Will reuse an existing one or create a new one.
-
-  address is expected to be a fully formed URL such as:
-  "http://www.example.com" or "http://www.example.com:8080"
-
-  It can and usually does include additional path parameters,
-  but these are ignored as the FedSockets are organized by host and port info alone.
-  """
-  def get_or_create_fed_socket(address) do
-    with {:cache, {:error, :missing}} <- {:cache, get_fed_socket(address)},
-         {:connect, {:ok, _pid}} <- {:connect, FedSocket.connect_to_host(address)},
-         {:cache, {:ok, fed_socket}} <- {:cache, get_fed_socket(address)} do
-      Logger.debug("fedsocket created for - #{inspect(address)}")
-      {:ok, fed_socket}
-    else
-      {:cache, {:ok, socket}} ->
-        Logger.debug("fedsocket found in cache - #{inspect(address)}")
-        {:ok, socket}
-
-      {:cache, {:error, :rejected} = e} ->
-        e
-
-      {:connect, {:error, _host}} ->
-        Logger.debug("set host rejected for - #{inspect(address)}")
-        FedRegistry.set_host_rejected(address)
-        {:error, :rejected}
-
-      {_, {:error, :disabled}} ->
-        {:error, :disabled}
-
-      {_, {:error, reason}} ->
-        Logger.warn("get_or_create_fed_socket error - #{inspect(reason)}")
-        {:error, reason}
-    end
-  end
-
-  @doc """
-  returns a FedSocket for the given origin. Will not create a new FedSocket if one does not exist.
-
-  address is expected to be a fully formed URL such as:
-    "http://www.example.com" or "http://www.example.com:8080"
-  """
-  def get_fed_socket(address) do
-    origin = SocketInfo.origin(address)
-
-    with {:config, true} <- {:config, Pleroma.Config.get([:fed_sockets, :enabled], false)},
-         {:ok, socket} <- FedRegistry.get_fed_socket(origin) do
-      {:ok, socket}
-    else
-      {:config, _} ->
-        {:error, :disabled}
-
-      {:error, :rejected} ->
-        Logger.debug("FedSocket previously rejected - #{inspect(origin)}")
-        {:error, :rejected}
-
-      {:error, reason} ->
-        {:error, reason}
-    end
-  end
-
-  @doc """
-  Sends the supplied data via the publish protocol.
-  It will not block waiting for a reply.
-  Returns :ok but this is not an indication of a successful transfer.
-
-  the data is expected to be JSON encoded binary data.
-  """
-  def publish(%SocketInfo{} = fed_socket, json) do
-    FedSocket.publish(fed_socket, json)
-  end
-
-  @doc """
-  Sends the supplied data via the fetch protocol.
-  It will block waiting for a reply or timeout.
-
-  Returns {:ok, object} where object is the requested object (or nil)
-          {:error, :timeout} in the event the message was not responded to
-
-  the id is expected to be the URI of an ActivityPub object.
-  """
-  def fetch(%SocketInfo{} = fed_socket, id) do
-    FedSocket.fetch(fed_socket, id)
-  end
-
-  @doc """
-  Disconnect all and restart FedSockets.
-  This is mainly used in development and testing but could be useful in production.
-  """
-  def reset do
-    FedRegistry
-    |> Process.whereis()
-    |> Process.exit(:testing)
-  end
-
-  def uri_for_origin(origin),
-    do: "ws://#{origin}/api/fedsocket/v1"
-end

lib/pleroma/web/fed_sockets/fed_registry.ex

@@ -1,185 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.FedSockets.FedRegistry do
-  @moduledoc """
-  The FedRegistry stores the active FedSockets for quick retrieval.
-
-  The storage and retrieval portion of the FedRegistry is done in process through
-  elixir's `Registry` module for speed and its ability to monitor for terminated processes.
-
-  Dropped connections will be caught by `Registry` and deleted. Since the next
-  message will initiate a new connection there is no reason to try and reconnect at that point.
-
-  Normally outside modules should have no need to call or use the FedRegistry themselves.
-  """
-
-  alias Pleroma.Web.FedSockets.FedSocket
-  alias Pleroma.Web.FedSockets.SocketInfo
-
-  require Logger
-
-  @default_rejection_duration 15 * 60 * 1000
-  @rejections :fed_socket_rejections
-
-  @doc """
-  Retrieves a FedSocket from the Registry given it's origin.
-
-  The origin is expected to be a string identifying the endpoint "example.com" or "example2.com:8080"
-
-  Will return:
-    * {:ok, fed_socket} for working FedSockets
-    * {:error, :rejected} for origins that have been tried and refused within the rejection duration interval
-    * {:error, some_reason} usually :missing for unknown origins
-  """
-  def get_fed_socket(origin) do
-    case get_registry_data(origin) do
-      {:error, reason} ->
-        {:error, reason}
-
-      {:ok, %{state: :connected} = socket_info} ->
-        {:ok, socket_info}
-    end
-  end
-
-  @doc """
-  Adds a connected FedSocket to the Registry.
-
-  Always returns {:ok, fed_socket}
-  """
-  def add_fed_socket(origin, pid \\ nil) do
-    origin
-    |> SocketInfo.build(pid)
-    |> SocketInfo.connect()
-    |> add_socket_info
-  end
-
-  defp add_socket_info(%{origin: origin, state: :connected} = socket_info) do
-    case Registry.register(FedSockets.Registry, origin, socket_info) do
-      {:ok, _owner} ->
-        clear_prior_rejection(origin)
-        Logger.debug("fedsocket added: #{inspect(origin)}")
-
-        {:ok, socket_info}
-
-      {:error, {:already_registered, _pid}} ->
-        FedSocket.close(socket_info)
-        existing_socket_info = Registry.lookup(FedSockets.Registry, origin)
-
-        {:ok, existing_socket_info}
-
-      _ ->
-        {:error, :error_adding_socket}
-    end
-  end
-
-  @doc """
-  Mark this origin as having rejected a connection attempt.
-  This will keep it from getting additional connection attempts
-  for a period of time specified in the config.
-
-  Always returns {:ok, new_reg_data}
-  """
-  def set_host_rejected(uri) do
-    new_reg_data =
-      uri
-      |> SocketInfo.origin()
-      |> get_or_create_registry_data()
-      |> set_to_rejected()
-      |> save_registry_data()
-
-    {:ok, new_reg_data}
-  end
-
-  @doc """
-  Retrieves the FedRegistryData from the Registry given it's origin.
-
-  The origin is expected to be a string identifying the endpoint "example.com" or "example2.com:8080"
-
-  Will return:
-    * {:ok, fed_registry_data} for known origins
-    * {:error, :missing} for uniknown origins
-    * {:error, :cache_error} indicating some low level runtime issues
-  """
-  def get_registry_data(origin) do
-    case Registry.lookup(FedSockets.Registry, origin) do
-      [] ->
-        if is_rejected?(origin) do
-          Logger.debug("previously rejected fedsocket requested")
-          {:error, :rejected}
-        else
-          {:error, :missing}
-        end
-
-      [{_pid, %{state: :connected} = socket_info}] ->
-        {:ok, socket_info}
-
-      _ ->
-        {:error, :cache_error}
-    end
-  end
-
-  @doc """
-  Retrieves a map of all sockets from the Registry. The keys are the origins and the values are the corresponding SocketInfo
-  """
-  def list_all do
-    (list_all_connected() ++ list_all_rejected())
-    |> Enum.into(%{})
-  end
-
-  defp list_all_connected do
-    FedSockets.Registry
-    |> Registry.select([{{:"$1", :_, :"$3"}, [], [{{:"$1", :"$3"}}]}])
-  end
-
-  defp list_all_rejected do
-    {:ok, keys} = Cachex.keys(@rejections)
-
-    {:ok, registry_data} =
-      Cachex.execute(@rejections, fn worker ->
-        Enum.map(keys, fn k -> {k, Cachex.get!(worker, k)} end)
-      end)
-
-    registry_data
-  end
-
-  defp clear_prior_rejection(origin),
-    do: Cachex.del(@rejections, origin)
-
-  defp is_rejected?(origin) do
-    case Cachex.get(@rejections, origin) do
-      {:ok, nil} ->
-        false
-
-      {:ok, _} ->
-        true
-    end
-  end
-
-  defp get_or_create_registry_data(origin) do
-    case get_registry_data(origin) do
-      {:error, :missing} ->
-        %SocketInfo{origin: origin}
-
-      {:ok, socket_info} ->
-        socket_info
-    end
-  end
-
-  defp save_registry_data(%SocketInfo{origin: origin, state: :connected} = socket_info) do
-    {:ok, true} = Registry.update_value(FedSockets.Registry, origin, fn _ -> socket_info end)
-    socket_info
-  end
-
-  defp save_registry_data(%SocketInfo{origin: origin, state: :rejected} = socket_info) do
-    rejection_expiration =
-      Pleroma.Config.get([:fed_sockets, :rejection_duration], @default_rejection_duration)
-
-    {:ok, true} = Cachex.put(@rejections, origin, socket_info, ttl: rejection_expiration)
-    socket_info
-  end
-
-  defp set_to_rejected(%SocketInfo{} = socket_info),
-    do: %SocketInfo{socket_info | state: :rejected}
-end

lib/pleroma/web/fed_sockets/fed_socket.ex

@@ -1,137 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.FedSockets.FedSocket do
-  @moduledoc """
-  The FedSocket module abstracts the actions to be taken taken on connections regardless of
-  whether the connection started as inbound or outbound.
-
-
-  Normally outside modules will have no need to call the FedSocket module directly.
-  """
-
-  alias Pleroma.Object
-  alias Pleroma.Object.Containment
-  alias Pleroma.User
-  alias Pleroma.Web.ActivityPub.ObjectView
-  alias Pleroma.Web.ActivityPub.UserView
-  alias Pleroma.Web.ActivityPub.Visibility
-  alias Pleroma.Web.FedSockets.FetchRegistry
-  alias Pleroma.Web.FedSockets.IngesterWorker
-  alias Pleroma.Web.FedSockets.OutgoingHandler
-  alias Pleroma.Web.FedSockets.SocketInfo
-
-  require Logger
-
-  @shake "61dd18f7-f1e6-49a4-939a-a749fcdc1103"
-
-  def connect_to_host(uri) do
-    case OutgoingHandler.start_link(uri) do
-      {:ok, pid} ->
-        {:ok, pid}
-
-      error ->
-        {:error, error}
-    end
-  end
-
-  def close(%SocketInfo{pid: socket_pid}),
-    do: Process.send(socket_pid, :close, [])
-
-  def publish(%SocketInfo{pid: socket_pid}, json) do
-    %{action: :publish, data: json}
-    |> Jason.encode!()
-    |> send_packet(socket_pid)
-  end
-
-  def fetch(%SocketInfo{pid: socket_pid}, id) do
-    fetch_uuid = FetchRegistry.register_fetch(id)
-
-    %{action: :fetch, data: id, uuid: fetch_uuid}
-    |> Jason.encode!()
-    |> send_packet(socket_pid)
-
-    wait_for_fetch_to_return(fetch_uuid, 0)
-  end
-
-  def receive_package(%SocketInfo{} = fed_socket, json) do
-    json
-    |> Jason.decode!()
-    |> process_package(fed_socket)
-  end
-
-  defp wait_for_fetch_to_return(uuid, cntr) do
-    case FetchRegistry.check_fetch(uuid) do
-      {:error, :waiting} ->
-        Process.sleep(:math.pow(cntr, 3) |> Kernel.trunc())
-        wait_for_fetch_to_return(uuid, cntr + 1)
-
-      {:error, :missing} ->
-        Logger.error("FedSocket fetch timed out - #{inspect(uuid)}")
-        {:error, :timeout}
-
-      {:ok, _fr} ->
-        FetchRegistry.pop_fetch(uuid)
-    end
-  end
-
-  defp process_package(%{"action" => "publish", "data" => data}, %{origin: origin} = _fed_socket) do
-    if Containment.contain_origin(origin, data) do
-      IngesterWorker.enqueue("ingest", %{"object" => data})
-    end
-
-    {:reply, %{"action" => "publish_reply", "status" => "processed"}}
-  end
-
-  defp process_package(%{"action" => "fetch_reply", "uuid" => uuid, "data" => data}, _fed_socket) do
-    FetchRegistry.register_fetch_received(uuid, data)
-    {:noreply, nil}
-  end
-
-  defp process_package(%{"action" => "fetch", "uuid" => uuid, "data" => ap_id}, _fed_socket) do
-    {:ok, data} = render_fetched_data(ap_id, uuid)
-    {:reply, data}
-  end
-
-  defp process_package(%{"action" => "publish_reply"}, _fed_socket) do
-    {:noreply, nil}
-  end
-
-  defp process_package(other, _fed_socket) do
-    Logger.warn("unknown json packages received #{inspect(other)}")
-    {:noreply, nil}
-  end
-
-  defp render_fetched_data(ap_id, uuid) do
-    {:ok,
-     %{
-       "action" => "fetch_reply",
-       "status" => "processed",
-       "uuid" => uuid,
-       "data" => represent_item(ap_id)
-     }}
-  end
-
-  defp represent_item(ap_id) do
-    case User.get_by_ap_id(ap_id) do
-      nil ->
-        object = Object.get_cached_by_ap_id(ap_id)
-
-        if Visibility.is_public?(object) do
-          Phoenix.View.render_to_string(ObjectView, "object.json", object: object)
-        else
-          nil
-        end
-
-      user ->
-        Phoenix.View.render_to_string(UserView, "user.json", user: user)
-    end
-  end
-
-  defp send_packet(data, socket_pid) do
-    Process.send(socket_pid, {:send, data}, [])
-  end
-
-  def shake, do: @shake
-end

lib/pleroma/web/fed_sockets/fetch_registry.ex

@@ -1,151 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.FedSockets.FetchRegistry do
-  @moduledoc """
-  The FetchRegistry acts as a broker for fetch requests and return values.
-  This allows calling processes to block while waiting for a reply.
-  It doesn't impose it's own process instead using `Cachex` to handle fetches in process, allowing
-  multi threaded processes to avoid bottlenecking.
-
-  Normally outside modules will have no need to call or use the FetchRegistry themselves.
-
-  The `Cachex` parameters can be controlled from the config. Since exact timeout intervals
-  aren't necessary the following settings are used by default:
-
-  config :pleroma, :fed_sockets,
-    fed_socket_fetches: [
-      default: 12_000,
-      interval: 3_000,
-      lazy: false
-    ]
-
-  """
-
-  defmodule FetchRegistryData do
-    defstruct uuid: nil,
-              sent_json: nil,
-              received_json: nil,
-              sent_at: nil,
-              received_at: nil
-  end
-
-  alias Ecto.UUID
-
-  require Logger
-
-  @fetches :fed_socket_fetches
-
-  @doc """
-  Registers a json request wth the FetchRegistry and returns the identifying UUID.
-  """
-  def register_fetch(json) do
-    %FetchRegistryData{uuid: uuid} =
-      json
-      |> new_registry_data
-      |> save_registry_data
-
-    uuid
-  end
-
-  @doc """
-  Reports on the status of a Fetch given the identifying UUID.
-
-  Will return
-    * {:ok, fetched_object} if a fetch has completed
-    * {:error, :waiting} if a fetch is still pending
-    * {:error, other_error} usually :missing to indicate a fetch that has timed out
-  """
-  def check_fetch(uuid) do
-    case get_registry_data(uuid) do
-      {:ok, %FetchRegistryData{received_at: nil}} ->
-        {:error, :waiting}
-
-      {:ok, %FetchRegistryData{} = reg_data} ->
-        {:ok, reg_data}
-
-      e ->
-        e
-    end
-  end
-
-  @doc """
-  Retrieves the response to a fetch given the identifying UUID.
-  The completed fetch will be deleted from the FetchRegistry
-
-  Will return
-    * {:ok, fetched_object} if a fetch has completed
-    * {:error, :waiting} if a fetch is still pending
-    * {:error, other_error} usually :missing to indicate a fetch that has timed out
-  """
-  def pop_fetch(uuid) do
-    case check_fetch(uuid) do
-      {:ok, %FetchRegistryData{received_json: received_json}} ->
-        delete_registry_data(uuid)
-        {:ok, received_json}
-
-      e ->
-        e
-    end
-  end
-
-  @doc """
-  This is called to register a fetch has returned.
-  It expects the result data along with the UUID that was sent in the request
-
-  Will return the fetched object or :error
-  """
-  def register_fetch_received(uuid, data) do
-    case get_registry_data(uuid) do
-      {:ok, %FetchRegistryData{received_at: nil} = reg_data} ->
-        reg_data
-        |> set_fetch_received(data)
-        |> save_registry_data()
-
-      {:ok, %FetchRegistryData{} = reg_data} ->
-        Logger.warn("tried to add fetched data twice - #{uuid}")
-        reg_data
-
-      {:error, _} ->
-        Logger.warn("Error adding fetch to registry - #{uuid}")
-        :error
-    end
-  end
-
-  defp new_registry_data(json) do
-    %FetchRegistryData{
-      uuid: UUID.generate(),
-      sent_json: json,
-      sent_at: :erlang.monotonic_time(:millisecond)
-    }
-  end
-
-  defp get_registry_data(origin) do
-    case Cachex.get(@fetches, origin) do
-      {:ok, nil} ->
-        {:error, :missing}
-
-      {:ok, reg_data} ->
-        {:ok, reg_data}
-
-      _ ->
-        {:error, :cache_error}
-    end
-  end
-
-  defp set_fetch_received(%FetchRegistryData{} = reg_data, data),
-    do: %FetchRegistryData{
-      reg_data
-      | received_at: :erlang.monotonic_time(:millisecond),
-        received_json: data
-    }
-
-  defp save_registry_data(%FetchRegistryData{uuid: uuid} = reg_data) do
-    {:ok, true} = Cachex.put(@fetches, uuid, reg_data)
-    reg_data
-  end
-
-  defp delete_registry_data(origin),
-    do: {:ok, true} = Cachex.del(@fetches, origin)
-end

lib/pleroma/web/fed_sockets/incoming_handler.ex

@@ -1,88 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.FedSockets.IncomingHandler do
-  require Logger
-
-  alias Pleroma.Web.FedSockets.FedRegistry
-  alias Pleroma.Web.FedSockets.FedSocket
-  alias Pleroma.Web.FedSockets.SocketInfo
-
-  import HTTPSignatures, only: [validate_conn: 1, split_signature: 1]
-
-  @behaviour :cowboy_websocket
-
-  def init(req, state) do
-    shake = FedSocket.shake()
-
-    with true <- Pleroma.Config.get([:fed_sockets, :enabled]),
-         sec_protocol <- :cowboy_req.header("sec-websocket-protocol", req, nil),
-         headers = %{"(request-target)" => ^shake} <- :cowboy_req.headers(req),
-         true <- validate_conn(%{req_headers: headers}),
-         %{"keyId" => origin} <- split_signature(headers["signature"]) do
-      req =
-        if is_nil(sec_protocol) do
-          req
-        else
-          :cowboy_req.set_resp_header("sec-websocket-protocol", sec_protocol, req)
-        end
-
-      {:cowboy_websocket, req, %{origin: origin}, %{}}
-    else
-      _ ->
-        {:ok, req, state}
-    end
-  end
-
-  def websocket_init(%{origin: origin}) do
-    case FedRegistry.add_fed_socket(origin) do
-      {:ok, socket_info} ->
-        {:ok, socket_info}
-
-      e ->
-        Logger.error("FedSocket websocket_init failed - #{inspect(e)}")
-        {:error, inspect(e)}
-    end
-  end
-
-  # Use the ping to  check if the connection should be expired
-  def websocket_handle(:ping, socket_info) do
-    if SocketInfo.expired?(socket_info) do
-      {:stop, socket_info}
-    else
-      {:ok, socket_info, :hibernate}
-    end
-  end
-
-  def websocket_handle({:text, data}, socket_info) do
-    socket_info = SocketInfo.touch(socket_info)
-
-    case FedSocket.receive_package(socket_info, data) do
-      {:noreply, _} ->
-        {:ok, socket_info}
-
-      {:reply, reply} ->
-        {:reply, {:text, Jason.encode!(reply)}, socket_info}
-
-      {:error, reason} ->
-        Logger.error("incoming error - receive_package: #{inspect(reason)}")
-        {:ok, socket_info}
-    end
-  end
-
-  def websocket_info({:send, message}, socket_info) do
-    socket_info = SocketInfo.touch(socket_info)
-
-    {:reply, {:text, message}, socket_info}
-  end
-
-  def websocket_info(:close, state) do
-    {:stop, state}
-  end
-
-  def websocket_info(message, state) do
-    Logger.debug("#{__MODULE__} unknown message #{inspect(message)}")
-    {:ok, state}
-  end
-end

lib/pleroma/web/fed_sockets/ingester_worker.ex

@@ -1,33 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.FedSockets.IngesterWorker do
-  use Pleroma.Workers.WorkerHelper, queue: "ingestion_queue"
-  require Logger
-
-  alias Pleroma.Web.Federator
-
-  @impl Oban.Worker
-  def perform(%Job{args: %{"op" => "ingest", "object" => ingestee}}) do
-    try do
-      ingestee
-      |> Jason.decode!()
-      |> do_ingestion()
-    rescue
-      e ->
-        Logger.error("IngesterWorker error - #{inspect(e)}")
-        e
-    end
-  end
-
-  defp do_ingestion(params) do
-    case Federator.incoming_ap_doc(params) do
-      {:error, reason} ->
-        {:error, reason}
-
-      {:ok, object} ->
-        {:ok, object}
-    end
-  end
-end

lib/pleroma/web/fed_sockets/outgoing_handler.ex

@@ -1,151 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.FedSockets.OutgoingHandler do
-  use GenServer
-
-  require Logger
-
-  alias Pleroma.Application
-  alias Pleroma.Web.ActivityPub.InternalFetchActor
-  alias Pleroma.Web.FedSockets
-  alias Pleroma.Web.FedSockets.FedRegistry
-  alias Pleroma.Web.FedSockets.FedSocket
-  alias Pleroma.Web.FedSockets.SocketInfo
-
-  def start_link(uri) do
-    GenServer.start_link(__MODULE__, %{uri: uri})
-  end
-
-  def init(%{uri: uri}) do
-    case initiate_connection(uri) do
-      {:ok, ws_origin, conn_pid} ->
-        FedRegistry.add_fed_socket(ws_origin, conn_pid)
-
-      {:error, reason} ->
-        Logger.debug("Outgoing connection failed - #{inspect(reason)}")
-        :ignore
-    end
-  end
-
-  def handle_info({:gun_ws, conn_pid, _ref, {:text, data}}, socket_info) do
-    socket_info = SocketInfo.touch(socket_info)
-
-    case FedSocket.receive_package(socket_info, data) do
-      {:noreply, _} ->
-        {:noreply, socket_info}
-
-      {:reply, reply} ->
-        :gun.ws_send(conn_pid, {:text, Jason.encode!(reply)})
-        {:noreply, socket_info}
-
-      {:error, reason} ->
-        Logger.error("incoming error - receive_package: #{inspect(reason)}")
-        {:noreply, socket_info}
-    end
-  end
-
-  def handle_info(:close, state) do
-    Logger.debug("Sending close frame !!!!!!!")
-    {:close, state}
-  end
-
-  def handle_info({:gun_down, _pid, _prot, :closed, _}, state) do
-    {:stop, :normal, state}
-  end
-
-  def handle_info({:send, data}, %{conn_pid: conn_pid} = socket_info) do
-    socket_info = SocketInfo.touch(socket_info)
-    :gun.ws_send(conn_pid, {:text, data})
-    {:noreply, socket_info}
-  end
-
-  def handle_info({:gun_ws, _, _, :pong}, state) do
-    {:noreply, state, :hibernate}
-  end
-
-  def handle_info(msg, state) do
-    Logger.debug("#{__MODULE__} unhandled event #{inspect(msg)}")
-    {:noreply, state}
-  end
-
-  def terminate(reason, state) do
-    Logger.debug(
-      "#{__MODULE__} terminating outgoing connection for #{inspect(state)} for #{inspect(reason)}"
-    )
-
-    {:ok, state}
-  end
-
-  def initiate_connection(uri) do
-    ws_uri =
-      uri
-      |> SocketInfo.origin()
-      |> FedSockets.uri_for_origin()
-
-    %{host: host, port: port, path: path} = URI.parse(ws_uri)
-
-    with {:ok, conn_pid} <- :gun.open(to_charlist(host), port, %{protocols: [:http]}),
-         {:ok, _} <- :gun.await_up(conn_pid),
-         reference <-
-           :gun.get(conn_pid, to_charlist(path), [
-             {'user-agent', to_charlist(Application.user_agent())}
-           ]),
-         {:response, :fin, 204, _} <- :gun.await(conn_pid, reference),
-         headers <- build_headers(uri),
-         ref <- :gun.ws_upgrade(conn_pid, to_charlist(path), headers, %{silence_pings: false}) do
-      receive do
-        {:gun_upgrade, ^conn_pid, ^ref, [<<"websocket">>], _} ->
-          {:ok, ws_uri, conn_pid}
-      after
-        15_000 ->
-          Logger.debug("Fedsocket timeout connecting to #{inspect(uri)}")
-          {:error, :timeout}
-      end
-    else
-      {:response, :nofin, 404, _} ->
-        {:error, :fedsockets_not_supported}
-
-      e ->
-        Logger.debug("Fedsocket error connecting to #{inspect(uri)}")
-        {:error, e}
-    end
-  end
-
-  defp build_headers(uri) do
-    host_for_sig = uri |> URI.parse() |> host_signature()
-
-    shake = FedSocket.shake()
-    digest = "SHA-256=" <> (:crypto.hash(:sha256, shake) |> Base.encode64())
-    date = Pleroma.Signature.signed_date()
-    shake_size = byte_size(shake)
-
-    signature_opts = %{
-      "(request-target)": shake,
-      "content-length": to_charlist("#{shake_size}"),
-      date: date,
-      digest: digest,
-      host: host_for_sig
-    }
-
-    signature = Pleroma.Signature.sign(InternalFetchActor.get_actor(), signature_opts)
-
-    [
-      {'signature', to_charlist(signature)},
-      {'date', date},
-      {'digest', to_charlist(digest)},
-      {'content-length', to_charlist("#{shake_size}")},
-      {to_charlist("(request-target)"), to_charlist(shake)},
-      {'user-agent', to_charlist(Application.user_agent())}
-    ]
-  end
-
-  defp host_signature(%{host: host, scheme: scheme, port: port}) do
-    if port == URI.default_port(scheme) do
-      host
-    else
-      "#{host}:#{port}"
-    end
-  end
-end

lib/pleroma/web/fed_sockets/socket_info.ex

@@ -1,52 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.FedSockets.SocketInfo do
-  defstruct origin: nil,
-            pid: nil,
-            conn_pid: nil,
-            state: :default,
-            connected_until: nil
-
-  alias Pleroma.Web.FedSockets.SocketInfo
-  @default_connection_duration 15 * 60 * 1000
-
-  def build(uri, conn_pid \\ nil) do
-    uri
-    |> build_origin()
-    |> build_pids(conn_pid)
-    |> touch()
-  end
-
-  def touch(%SocketInfo{} = socket_info),
-    do: %{socket_info | connected_until: new_ttl()}
-
-  def connect(%SocketInfo{} = socket_info),
-    do: %{socket_info | state: :connected}
-
-  def expired?(%{connected_until: connected_until}),
-    do: connected_until < :erlang.monotonic_time(:millisecond)
-
-  def origin(uri),
-    do: build_origin(uri).origin
-
-  defp build_pids(socket_info, conn_pid),
-    do: struct(socket_info, pid: self(), conn_pid: conn_pid)
-
-  defp build_origin(uri) when is_binary(uri),
-    do: uri |> URI.parse() |> build_origin
-
-  defp build_origin(%{host: host, port: nil, scheme: scheme}),
-    do: build_origin(%{host: host, port: URI.default_port(scheme)})
-
-  defp build_origin(%{host: host, port: port}),
-    do: %SocketInfo{origin: "#{host}:#{port}"}
-
-  defp new_ttl do
-    connection_duration =
-      Pleroma.Config.get([:fed_sockets, :connection_duration], @default_connection_duration)
-
-    :erlang.monotonic_time(:millisecond) + connection_duration
-  end
-end

lib/pleroma/web/fed_sockets/supervisor.ex

@@ -1,59 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.FedSockets.Supervisor do
-  use Supervisor
-  import Cachex.Spec
-
-  def start_link(opts) do
-    Supervisor.start_link(__MODULE__, opts, name: __MODULE__)
-  end
-
-  def init(args) do
-    children = [
-      build_cache(:fed_socket_fetches, args),
-      build_cache(:fed_socket_rejections, args),
-      {Registry, keys: :unique, name: FedSockets.Registry, meta: [rejected: %{}]}
-    ]
-
-    opts = [strategy: :one_for_all, name: Pleroma.Web.Streamer.Supervisor]
-    Supervisor.init(children, opts)
-  end
-
-  defp build_cache(name, args) do
-    opts = get_opts(name, args)
-
-    %{
-      id: String.to_atom("#{name}_cache"),
-      start: {Cachex, :start_link, [name, opts]},
-      type: :worker
-    }
-  end
-
-  defp get_opts(cache_name, args)
-       when cache_name in [:fed_socket_fetches, :fed_socket_rejections] do
-    default = get_opts_or_config(args, cache_name, :default, 15_000)
-    interval = get_opts_or_config(args, cache_name, :interval, 3_000)
-    lazy = get_opts_or_config(args, cache_name, :lazy, false)
-
-    [expiration: expiration(default: default, interval: interval, lazy: lazy)]
-  end
-
-  defp get_opts(name, args) do
-    Keyword.get(args, name, [])
-  end
-
-  defp get_opts_or_config(args, name, key, default) do
-    args
-    |> Keyword.get(name, [])
-    |> Keyword.get(key)
-    |> case do
-      nil ->
-        Pleroma.Config.get([:fed_sockets, name, key], default)
-
-      value ->
-        value
-    end
-  end
-end

lib/pleroma/web/mastodon_api/controllers/account_controller.ex

@@ -208,7 +208,9 @@ def update_credentials(%{assigns: %{user: user}, body_params: params} = conn, _p
         if bot, do: {:ok, "Service"}, else: {:ok, "Person"}
       end)
       |> Maps.put_if_present(:actor_type, params[:actor_type])
+      # Note: param name is indeed :locked (not an error)
       |> Maps.put_if_present(:is_locked, params[:locked])
+      # Note: param name is indeed :discoverable (not an error)
       |> Maps.put_if_present(:is_discoverable, params[:discoverable])
 
     # What happens here:
@@ -292,7 +294,8 @@ def statuses(%{assigns: %{user: reading_user}} = conn, params) do
       |> render("index.json",
         activities: activities,
         for: reading_user,
-        as: :activity
+        as: :activity,
+        with_muted: Map.get(params, :with_muted, false)
       )
     else
       error -> user_visibility_error(conn, error)

lib/pleroma/web/mastodon_api/controllers/status_controller.ex

@@ -109,7 +109,7 @@ defmodule Pleroma.Web.MastodonAPI.StatusController do
 
   `ids` query param is required
   """
-  def index(%{assigns: %{user: user}} = conn, %{ids: ids} = _params) do
+  def index(%{assigns: %{user: user}} = conn, %{ids: ids} = params) do
     limit = 100
 
     activities =
@@ -121,7 +121,8 @@ def index(%{assigns: %{user: user}} = conn, %{ids: ids} = _params) do
     render(conn, "index.json",
       activities: activities,
       for: user,
-      as: :activity
+      as: :activity,
+      with_muted: Map.get(params, :with_muted, false)
     )
   end
 
@@ -189,13 +190,14 @@ def create(%{assigns: %{user: _user}, body_params: %{media_ids: _} = params} = c
   end
 
   @doc "GET /api/v1/statuses/:id"
-  def show(%{assigns: %{user: user}} = conn, %{id: id}) do
+  def show(%{assigns: %{user: user}} = conn, %{id: id} = params) do
     with %Activity{} = activity <- Activity.get_by_id_with_object(id),
          true <- Visibility.visible_for_user?(activity, user) do
       try_render(conn, "show.json",
         activity: activity,
         for: user,
-        with_direct_conversation_id: true
+        with_direct_conversation_id: true,
+        with_muted: Map.get(params, :with_muted, false)
       )
     else
       _ -> {:error, :not_found}

lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex

@@ -62,7 +62,8 @@ def home(%{assigns: %{user: user}} = conn, params) do
     |> render("index.json",
       activities: activities,
       for: user,
-      as: :activity
+      as: :activity,
+      with_muted: Map.get(params, :with_muted, false)
     )
   end
 
@@ -119,7 +120,8 @@ def public(%{assigns: %{user: user}} = conn, params) do
       |> render("index.json",
         activities: activities,
         for: user,
-        as: :activity
+        as: :activity,
+        with_muted: Map.get(params, :with_muted, false)
       )
     end
   end
@@ -173,7 +175,8 @@ def hashtag(%{assigns: %{user: user}} = conn, params) do
       |> render("index.json",
         activities: activities,
         for: user,
-        as: :activity
+        as: :activity,
+        with_muted: Map.get(params, :with_muted, false)
       )
     end
   end
@@ -202,7 +205,8 @@ def list(%{assigns: %{user: user}} = conn, %{list_id: id} = params) do
       render(conn, "index.json",
         activities: activities,
         for: user,
-        as: :activity
+        as: :activity,
+        with_muted: Map.get(params, :with_muted, false)
       )
     else
       _e -> render_error(conn, :forbidden, "Error.")

lib/pleroma/web/mastodon_api/views/instance_view.ex

@@ -23,7 +23,7 @@ def render("show.json", _) do
         streaming_api: Pleroma.Web.Endpoint.websocket_url()
       },
       stats: Pleroma.Stats.get_stats(),
-      thumbnail: Keyword.get(instance, :instance_thumbnail),
+      thumbnail: Pleroma.Web.base_url() <> Keyword.get(instance, :instance_thumbnail),
       languages: ["en"],
       registrations: Keyword.get(instance, :registrations_open),
       approval_required: Keyword.get(instance, :account_approval_required),
@@ -34,7 +34,7 @@ def render("show.json", _) do
       avatar_upload_limit: Keyword.get(instance, :avatar_upload_limit),
       background_upload_limit: Keyword.get(instance, :background_upload_limit),
       banner_upload_limit: Keyword.get(instance, :banner_upload_limit),
-      background_image: Keyword.get(instance, :background_image),
+      background_image: Pleroma.Web.base_url() <> Keyword.get(instance, :background_image),
       chat_limit: Keyword.get(instance, :chat_limit),
       description_limit: Keyword.get(instance, :description_limit),
       pleroma: %{

lib/pleroma/web/mastodon_api/views/status_view.ex

@@ -19,6 +19,7 @@ defmodule Pleroma.Web.MastodonAPI.StatusView do
   alias Pleroma.Web.MastodonAPI.PollView
   alias Pleroma.Web.MastodonAPI.StatusView
   alias Pleroma.Web.MediaProxy
+  alias Pleroma.Web.PleromaAPI.EmojiReactionController
 
   import Pleroma.Web.ActivityPub.Visibility, only: [get_visibility: 1, visible_for_user?: 2]
 
@@ -294,21 +295,16 @@ def render("show.json", %{activity: %{data: %{"object" => _object}} = activity}
       end
 
     emoji_reactions =
-      with %{data: %{"reactions" => emoji_reactions}} <- object do
+      object.data
-        Enum.map(emoji_reactions, fn
+      |> Map.get("reactions", [])
-          [emoji, users] when is_list(users) ->
+      |> EmojiReactionController.filter_allowed_users(
-            build_emoji_map(emoji, users, opts[:for])
+        opts[:for],
-
+        Map.get(opts, :with_muted, false)
-          {emoji, users} when is_list(users) ->
+      )
-            build_emoji_map(emoji, users, opts[:for])
+      |> Stream.map(fn {emoji, users} ->
-
+        build_emoji_map(emoji, users, opts[:for])
-          _ ->
+      end)
-            nil
+      |> Enum.to_list()
-        end)
-        |> Enum.reject(&is_nil/1)
-      else
-        _ -> []
-      end
 
     # Status muted state (would do 1 request per status unless user mutes are preloaded)
     muted =

lib/pleroma/web/metadata/providers/restrict_indexing.ex

@@ -6,7 +6,7 @@ defmodule Pleroma.Web.Metadata.Providers.RestrictIndexing do
   @behaviour Pleroma.Web.Metadata.Providers.Provider
 
   @moduledoc """
-  Restricts indexing of remote users.
+  Restricts indexing of remote and/or non-discoverable users.
   """
 
   @impl true

lib/pleroma/web/pleroma_api/controllers/chat_controller.ex

@@ -140,8 +140,8 @@ def messages(%{assigns: %{user: user}} = conn, %{id: id} = params) do
 
   def index(%{assigns: %{user: %{id: user_id} = user}} = conn, params) do
     exclude_users =
-      User.blocked_users_ap_ids(user) ++
+      User.cached_blocked_users_ap_ids(user) ++
-        if params[:with_muted], do: [], else: User.muted_users_ap_ids(user)
+        if params[:with_muted], do: [], else: User.cached_muted_users_ap_ids(user)
 
     chats =
       user_id

lib/pleroma/web/pleroma_api/controllers/emoji_reaction_controller.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.PleromaAPI.EmojiReactionController do
 
   alias Pleroma.Activity
   alias Pleroma.Object
+  alias Pleroma.User
   alias Pleroma.Web.CommonAPI
   alias Pleroma.Web.MastodonAPI.StatusView
   alias Pleroma.Web.Plugs.OAuthScopesPlug
@@ -29,13 +30,42 @@ def index(%{assigns: %{user: user}} = conn, %{id: activity_id} = params) do
          %Activity{} = activity <- Activity.get_by_id_with_object(activity_id),
          %Object{data: %{"reactions" => reactions}} when is_list(reactions) <-
            Object.normalize(activity) do
-      reactions = filter(reactions, params)
+      reactions =
+        reactions
+        |> filter(params)
+        |> filter_allowed_users(user, Map.get(params, :with_muted, false))
+
       render(conn, "index.json", emoji_reactions: reactions, user: user)
     else
       _e -> json(conn, [])
     end
   end
 
+  def filter_allowed_users(reactions, user, with_muted) do
+    exclude_ap_ids =
+      if is_nil(user) do
+        []
+      else
+        User.cached_blocked_users_ap_ids(user) ++
+          if not with_muted, do: User.cached_muted_users_ap_ids(user), else: []
+      end
+
+    filter_emoji = fn emoji, users ->
+      case Enum.reject(users, &(&1 in exclude_ap_ids)) do
+        [] -> nil
+        users -> {emoji, users}
+      end
+    end
+
+    reactions
+    |> Stream.map(fn
+      [emoji, users] when is_list(users) -> filter_emoji.(emoji, users)
+      {emoji, users} when is_list(users) -> filter_emoji.(emoji, users)
+      _ -> nil
+    end)
+    |> Stream.reject(&is_nil/1)
+  end
+
   defp filter(reactions, %{emoji: emoji}) when is_binary(emoji) do
     Enum.filter(reactions, fn [e, _] -> e == emoji end)
   end

lib/pleroma/web/pleroma_api/views/emoji_reaction_view.ex

@@ -11,7 +11,7 @@ def render("index.json", %{emoji_reactions: emoji_reactions} = opts) do
     render_many(emoji_reactions, __MODULE__, "show.json", opts)
   end
 
-  def render("show.json", %{emoji_reaction: [emoji, user_ap_ids], user: user}) do
+  def render("show.json", %{emoji_reaction: {emoji, user_ap_ids}, user: user}) do
     users = fetch_users(user_ap_ids)
 
     %{

lib/pleroma/web/push/impl.ex

@@ -16,7 +16,7 @@ defmodule Pleroma.Web.Push.Impl do
   require Logger
   import Ecto.Query
 
-  @types ["Create", "Follow", "Announce", "Like", "Move"]
+  @types ["Create", "Follow", "Announce", "Like", "Move", "EmojiReact"]
 
   @doc "Performs sending notifications for user subscriptions"
   @spec perform(Notification.t()) :: list(any) | :error | {:error, :unknown_type}
@@ -149,6 +149,15 @@ def format_body(
     "@#{actor.nickname} repeated: #{Utils.scrub_html_and_truncate(content, 80)}"
   end
 
+  def format_body(
+        %{activity: %{data: %{"type" => "EmojiReact", "content" => content}}},
+        actor,
+        _object,
+        _mastodon_type
+      ) do
+    "@#{actor.nickname} reacted with #{content}"
+  end
+
   def format_body(
         %{activity: %{data: %{"type" => type}}} = notification,
         actor,
@@ -179,6 +188,7 @@ def format_title(%{type: type}, mastodon_type) do
       "reblog" -> "New Repeat"
       "favourite" -> "New Favorite"
       "pleroma:chat_mention" -> "New Chat Message"
+      "pleroma:emoji_reaction" -> "New Reaction"
       type -> "New #{String.capitalize(type || "event")}"
     end
   end

lib/pleroma/web/push/subscription.ex

@@ -25,7 +25,8 @@ defmodule Pleroma.Web.Push.Subscription do
     timestamps()
   end
 
-  @supported_alert_types ~w[follow favourite mention reblog pleroma:chat_mention]a
+  # credo:disable-for-next-line Credo.Check.Readability.MaxLineLength
+  @supported_alert_types ~w[follow favourite mention reblog pleroma:chat_mention pleroma:emoji_reaction]a
 
   defp alerts(%{data: %{alerts: alerts}}) do
     alerts = Map.take(alerts, @supported_alert_types)

lib/pleroma/web/rich_media/helpers.ex

@@ -78,11 +78,6 @@ def fetch_data_for_activity(%Activity{data: %{"type" => "Create"}} = activity) d
 
   def fetch_data_for_activity(_), do: %{}
 
-  def perform(:fetch, %Activity{} = activity) do
-    fetch_data_for_activity(activity)
-    :ok
-  end
-
   def rich_media_get(url) do
     headers = [{"user-agent", Pleroma.Application.user_agent() <> "; Bot"}]
 

lib/pleroma/web/router.ex

@@ -244,6 +244,9 @@ defmodule Pleroma.Web.Router do
     get("/chats/:id/messages", ChatController, :messages)
     delete("/chats/:id/messages/:message_id", ChatController, :delete_message)
 
+    get("/frontends", FrontendController, :index)
+    post("/frontends/install", FrontendController, :install)
+
     post("/backups", AdminAPIController, :create_backup)
   end
 

lib/pleroma/web/twitter_api/controllers/password_controller.ex

@@ -17,6 +17,7 @@ defmodule Pleroma.Web.TwitterAPI.PasswordController do
 
   def reset(conn, %{"token" => token}) do
     with %{used: false} = token <- Repo.get_by(PasswordResetToken, %{token: token}),
+         false <- PasswordResetToken.expired?(token),
          %User{} = user <- User.get_cached_by_id(token.user_id) do
       render(conn, "reset.html", %{
         token: token,

lib/pleroma/workers/background_worker.ex

@@ -3,9 +3,7 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Workers.BackgroundWorker do
-  alias Pleroma.Activity
   alias Pleroma.User
-  alias Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy
 
   use Pleroma.Workers.WorkerHelper, queue: "background"
 
@@ -32,19 +30,6 @@ def perform(%Job{args: %{"op" => op, "user_id" => user_id, "identifiers" => iden
     {:ok, User.Import.perform(String.to_atom(op), user, identifiers)}
   end
 
-  def perform(%Job{args: %{"op" => "media_proxy_preload", "message" => message}}) do
-    MediaProxyWarmingPolicy.perform(:preload, message)
-  end
-
-  def perform(%Job{args: %{"op" => "media_proxy_prefetch", "url" => url}}) do
-    MediaProxyWarmingPolicy.perform(:prefetch, url)
-  end
-
-  def perform(%Job{args: %{"op" => "fetch_data_for_activity", "activity_id" => activity_id}}) do
-    activity = Activity.get_by_id(activity_id)
-    Pleroma.Web.RichMedia.Helpers.perform(:fetch, activity)
-  end
-
   def perform(%Job{
         args: %{"op" => "move_following", "origin_id" => origin_id, "target_id" => target_id}
       }) do

mix.exs

@@ -4,7 +4,7 @@ defmodule Pleroma.Mixfile do
   def project do
     [
       app: :pleroma,
-      version: version("2.1.50"),
+      version: version("2.2.50"),
       elixir: "~> 1.9",
       elixirc_paths: elixirc_paths(Mix.env()),
       compilers: [:phoenix, :gettext] ++ Mix.compilers(),
@@ -133,17 +133,14 @@ defp deps do
       {:calendar, "~> 1.0"},
       {:cachex, "~> 3.2"},
       {:poison, "~> 3.0", override: true},
-      {:tesla,
+      {:tesla, "~> 1.4.0", override: true},
-       git: "https://github.com/teamon/tesla.git",
-       ref: "9f7261ca49f9f901ceb73b60219ad6f8a9f6aa30",
-       override: true},
       {:castore, "~> 0.1"},
       {:cowlib, "~> 2.9", override: true},
       {:gun,
        github: "ninenines/gun", ref: "921c47146b2d9567eac7e9a4d2ccc60fffd4f327", override: true},
       {:jason, "~> 1.2"},
       {:mogrify, "~> 0.7.4"},
-      {:ex_aws, "~> 2.1"},
+      {:ex_aws, "~> 2.1.6"},
       {:ex_aws_s3, "~> 2.0"},
       {:sweet_xml, "~> 0.6.6"},
       {:earmark, "1.4.3"},

mix.lock

@@ -37,7 +37,7 @@
   "esshd": {:hex, :esshd, "0.1.1", "d4dd4c46698093a40a56afecce8a46e246eb35463c457c246dacba2e056f31b5", [:mix], [], "hexpm", "d73e341e3009d390aa36387dc8862860bf9f874c94d9fd92ade2926376f49981"},
   "eternal": {:hex, :eternal, "1.2.1", "d5b6b2499ba876c57be2581b5b999ee9bdf861c647401066d3eeed111d096bc4", [:mix], [], "hexpm", "b14f1dc204321429479c569cfbe8fb287541184ed040956c8862cb7a677b8406"},
   "ex2ms": {:hex, :ex2ms, "1.5.0", "19e27f9212be9a96093fed8cdfbef0a2b56c21237196d26760f11dfcfae58e97", [:mix], [], "hexpm"},
-  "ex_aws": {:hex, :ex_aws, "2.1.3", "26b6f036f0127548706aade4a509978fc7c26bd5334b004fba9bfe2687a525df", [:mix], [{:configparser_ex, "~> 4.0", [hex: :configparser_ex, repo: "hexpm", optional: true]}, {:hackney, "~> 1.9", [hex: :hackney, repo: "hexpm", optional: true]}, {:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: true]}, {:jsx, "~> 2.8", [hex: :jsx, repo: "hexpm", optional: true]}, {:sweet_xml, "~> 0.6", [hex: :sweet_xml, repo: "hexpm", optional: true]}], "hexpm", "0bdbe2aed9f326922fc5a6a80417e32f0c895f4b3b2b0b9676ebf23dd16c5da4"},
+  "ex_aws": {:hex, :ex_aws, "2.1.6", "41ab8b4caa48035c96d07faa035d2d9de6df480e7e084c054e662ac888dcd4d4", [:mix], [{:configparser_ex, "~> 4.0", [hex: :configparser_ex, repo: "hexpm", optional: true]}, {:hackney, "~> 1.9", [hex: :hackney, repo: "hexpm", optional: true]}, {:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: true]}, {:jsx, "~> 2.8", [hex: :jsx, repo: "hexpm", optional: true]}, {:sweet_xml, "~> 0.6", [hex: :sweet_xml, repo: "hexpm", optional: true]}], "hexpm", "a541bd042c1ee26412bb1e749ddf2a1c327e4fb7e382b1cd227e1b00eed3d469"},
   "ex_aws_s3": {:hex, :ex_aws_s3, "2.0.2", "c0258bbdfea55de4f98f0b2f0ca61fe402cc696f573815134beb1866e778f47b", [:mix], [{:ex_aws, "~> 2.0", [hex: :ex_aws, repo: "hexpm", optional: false]}, {:sweet_xml, ">= 0.0.0", [hex: :sweet_xml, repo: "hexpm", optional: true]}], "hexpm", "0569f5b211b1a3b12b705fe2a9d0e237eb1360b9d76298028df2346cad13097a"},
   "ex_const": {:hex, :ex_const, "0.2.4", "d06e540c9d834865b012a17407761455efa71d0ce91e5831e86881b9c9d82448", [:mix], [], "hexpm", "96fd346610cc992b8f896ed26a98be82ac4efb065a0578f334a32d60a3ba9767"},
   "ex_doc": {:hex, :ex_doc, "0.22.2", "03a2a58bdd2ba0d83d004507c4ee113b9c521956938298eba16e55cc4aba4a6c", [:mix], [{:earmark_parser, "~> 1.4.0", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.14", [hex: :makeup_elixir, repo: "hexpm", optional: false]}], "hexpm", "cf60e1b3e2efe317095b6bb79651f83a2c1b3edcb4d319c421d7fcda8b3aff26"},
@@ -115,7 +115,7 @@
   "swoosh": {:hex, :swoosh, "1.0.6", "6765e334c67dacabe721f0d701c7e5a6f06e4595c90df6f91e73ebd54d555833", [:mix], [{:cowboy, "~> 1.1 or ~> 2.4", [hex: :cowboy, repo: "hexpm", optional: true]}, {:gen_smtp, "~> 0.13 or ~> 1.0", [hex: :gen_smtp, repo: "hexpm", optional: true]}, {:hackney, "~> 1.9", [hex: :hackney, repo: "hexpm", optional: true]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:mail, "~> 0.2", [hex: :mail, repo: "hexpm", optional: true]}, {:mime, "~> 1.1", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_cowboy, ">= 1.0.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}], "hexpm", "7c50ef78e4acfd1cbd4907dc1fa87b5540675a6be9dc979d04890f49d7ec1830"},
   "syslog": {:hex, :syslog, "1.1.0", "6419a232bea84f07b56dc575225007ffe34d9fdc91abe6f1b2f254fd71d8efc2", [:rebar3], [], "hexpm", "4c6a41373c7e20587be33ef841d3de6f3beba08519809329ecc4d27b15b659e1"},
   "telemetry": {:hex, :telemetry, "0.4.2", "2808c992455e08d6177322f14d3bdb6b625fbcfd233a73505870d8738a2f4599", [:rebar3], [], "hexpm", "2d1419bd9dda6a206d7b5852179511722e2b18812310d304620c7bd92a13fcef"},
-  "tesla": {:git, "https://github.com/teamon/tesla.git", "9f7261ca49f9f901ceb73b60219ad6f8a9f6aa30", [ref: "9f7261ca49f9f901ceb73b60219ad6f8a9f6aa30"]},
+  "tesla": {:hex, :tesla, "1.4.0", "1081bef0124b8bdec1c3d330bbe91956648fb008cf0d3950a369cda466a31a87", [:mix], [{:castore, "~> 0.1", [hex: :castore, repo: "hexpm", optional: true]}, {:exjsx, ">= 3.0.0", [hex: :exjsx, repo: "hexpm", optional: true]}, {:finch, "~> 0.3", [hex: :finch, repo: "hexpm", optional: true]}, {:fuse, "~> 2.4", [hex: :fuse, repo: "hexpm", optional: true]}, {:gun, "~> 1.3", [hex: :gun, repo: "hexpm", optional: true]}, {:hackney, "~> 1.6", [hex: :hackney, repo: "hexpm", optional: true]}, {:ibrowse, "~> 4.4.0", [hex: :ibrowse, repo: "hexpm", optional: true]}, {:jason, ">= 1.0.0", [hex: :jason, repo: "hexpm", optional: true]}, {:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:mint, "~> 1.0", [hex: :mint, repo: "hexpm", optional: true]}, {:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: true]}], "hexpm", "bf1374a5569f5fca8e641363b63f7347d680d91388880979a33bc12a6eb3e0aa"},
   "timex": {:hex, :timex, "3.6.2", "845cdeb6119e2fef10751c0b247b6c59d86d78554c83f78db612e3290f819bc2", [:mix], [{:combine, "~> 0.10", [hex: :combine, repo: "hexpm", optional: false]}, {:gettext, "~> 0.10", [hex: :gettext, repo: "hexpm", optional: false]}, {:tzdata, "~> 0.1.8 or ~> 0.5 or ~> 1.0.0", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm", "26030b46199d02a590be61c2394b37ea25a3664c02fafbeca0b24c972025d47a"},
   "trailing_format_plug": {:hex, :trailing_format_plug, "0.0.7", "64b877f912cf7273bed03379936df39894149e35137ac9509117e59866e10e45", [:mix], [{:plug, "> 0.12.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm", "bd4fde4c15f3e993a999e019d64347489b91b7a9096af68b2bdadd192afa693f"},
   "tzdata": {:hex, :tzdata, "1.0.4", "a3baa4709ea8dba552dca165af6ae97c624a2d6ac14bd265165eaa8e8af94af6", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm", "b02637db3df1fd66dd2d3c4f194a81633d0e4b44308d36c1b2fdfd1e4e6f169b"},

priv/gettext/he/LC_MESSAGES/errors.po

@@ -0,0 +1,596 @@
+msgid ""
+msgstr ""
+"Project-Id-Version: PACKAGE VERSION\n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2020-11-10 13:39+0000\n"
+"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+"Last-Translator: Automatically generated\n"
+"Language-Team: none\n"
+"Language: he\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=UTF-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"X-Generator: Translate Toolkit 2.5.1\n"
+
+## This file is a PO Template file.
+##
+## `msgid`s here are often extracted from source code.
+## Add new translations manually only if they're dynamic
+## translations that can't be statically extracted.
+##
+## Run `mix gettext.extract` to bring this file up to
+## date. Leave `msgstr`s empty as changing them here as no
+## effect: edit them in PO (`.po`) files instead.
+## From Ecto.Changeset.cast/4
+msgid "can't be blank"
+msgstr ""
+
+## From Ecto.Changeset.unique_constraint/3
+msgid "has already been taken"
+msgstr ""
+
+## From Ecto.Changeset.put_change/3
+msgid "is invalid"
+msgstr ""
+
+## From Ecto.Changeset.validate_format/3
+msgid "has invalid format"
+msgstr ""
+
+## From Ecto.Changeset.validate_subset/3
+msgid "has an invalid entry"
+msgstr ""
+
+## From Ecto.Changeset.validate_exclusion/3
+msgid "is reserved"
+msgstr ""
+
+## From Ecto.Changeset.validate_confirmation/3
+msgid "does not match confirmation"
+msgstr ""
+
+## From Ecto.Changeset.no_assoc_constraint/3
+msgid "is still associated with this entry"
+msgstr ""
+
+msgid "are still associated with this entry"
+msgstr ""
+
+## From Ecto.Changeset.validate_length/3
+msgid "should be %{count} character(s)"
+msgid_plural "should be %{count} character(s)"
+msgstr[0] ""
+msgstr[1] ""
+msgstr[2] ""
+msgstr[3] ""
+
+msgid "should have %{count} item(s)"
+msgid_plural "should have %{count} item(s)"
+msgstr[0] ""
+msgstr[1] ""
+msgstr[2] ""
+msgstr[3] ""
+
+msgid "should be at least %{count} character(s)"
+msgid_plural "should be at least %{count} character(s)"
+msgstr[0] ""
+msgstr[1] ""
+msgstr[2] ""
+msgstr[3] ""
+
+msgid "should have at least %{count} item(s)"
+msgid_plural "should have at least %{count} item(s)"
+msgstr[0] ""
+msgstr[1] ""
+msgstr[2] ""
+msgstr[3] ""
+
+msgid "should be at most %{count} character(s)"
+msgid_plural "should be at most %{count} character(s)"
+msgstr[0] ""
+msgstr[1] ""
+msgstr[2] ""
+msgstr[3] ""
+
+msgid "should have at most %{count} item(s)"
+msgid_plural "should have at most %{count} item(s)"
+msgstr[0] ""
+msgstr[1] ""
+msgstr[2] ""
+msgstr[3] ""
+
+## From Ecto.Changeset.validate_number/3
+msgid "must be less than %{number}"
+msgstr ""
+
+msgid "must be greater than %{number}"
+msgstr ""
+
+msgid "must be less than or equal to %{number}"
+msgstr ""
+
+msgid "must be greater than or equal to %{number}"
+msgstr ""
+
+msgid "must be equal to %{number}"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:505
+#, elixir-format
+msgid "Account not found"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:339
+#, elixir-format
+msgid "Already voted"
+msgstr ""
+
+#: lib/pleroma/web/oauth/oauth_controller.ex:359
+#, elixir-format
+msgid "Bad request"
+msgstr ""
+
+#: lib/pleroma/web/activity_pub/activity_pub_controller.ex:426
+#, elixir-format
+msgid "Can't delete object"
+msgstr ""
+
+#: lib/pleroma/web/controller_helper.ex:105
+#: lib/pleroma/web/controller_helper.ex:111
+#, elixir-format
+msgid "Can't display this activity"
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/account_controller.ex:285
+#, elixir-format
+msgid "Can't find user"
+msgstr ""
+
+#: lib/pleroma/web/pleroma_api/controllers/account_controller.ex:61
+#, elixir-format
+msgid "Can't get favorites"
+msgstr ""
+
+#: lib/pleroma/web/activity_pub/activity_pub_controller.ex:438
+#, elixir-format
+msgid "Can't like object"
+msgstr ""
+
+#: lib/pleroma/web/common_api/utils.ex:563
+#, elixir-format
+msgid "Cannot post an empty status without attachments"
+msgstr ""
+
+#: lib/pleroma/web/common_api/utils.ex:511
+#, elixir-format
+msgid "Comment must be up to %{max_size} characters"
+msgstr ""
+
+#: lib/pleroma/config/config_db.ex:191
+#, elixir-format
+msgid "Config with params %{params} not found"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:181
+#: lib/pleroma/web/common_api/common_api.ex:185
+#, elixir-format
+msgid "Could not delete"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:231
+#, elixir-format
+msgid "Could not favorite"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:453
+#, elixir-format
+msgid "Could not pin"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:278
+#, elixir-format
+msgid "Could not unfavorite"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:463
+#, elixir-format
+msgid "Could not unpin"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:216
+#, elixir-format
+msgid "Could not unrepeat"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:512
+#: lib/pleroma/web/common_api/common_api.ex:521
+#, elixir-format
+msgid "Could not update state"
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex:207
+#, elixir-format
+msgid "Error."
+msgstr ""
+
+#: lib/pleroma/web/twitter_api/twitter_api.ex:106
+#, elixir-format
+msgid "Invalid CAPTCHA"
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/account_controller.ex:116
+#: lib/pleroma/web/oauth/oauth_controller.ex:568
+#, elixir-format
+msgid "Invalid credentials"
+msgstr ""
+
+#: lib/pleroma/plugs/ensure_authenticated_plug.ex:38
+#, elixir-format
+msgid "Invalid credentials."
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:355
+#, elixir-format
+msgid "Invalid indices"
+msgstr ""
+
+#: lib/pleroma/web/admin_api/controllers/fallback_controller.ex:29
+#, elixir-format
+msgid "Invalid parameters"
+msgstr ""
+
+#: lib/pleroma/web/common_api/utils.ex:414
+#, elixir-format
+msgid "Invalid password."
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/account_controller.ex:220
+#, elixir-format
+msgid "Invalid request"
+msgstr ""
+
+#: lib/pleroma/web/twitter_api/twitter_api.ex:109
+#, elixir-format
+msgid "Kocaptcha service unavailable"
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/account_controller.ex:112
+#, elixir-format
+msgid "Missing parameters"
+msgstr ""
+
+#: lib/pleroma/web/common_api/utils.ex:547
+#, elixir-format
+msgid "No such conversation"
+msgstr ""
+
+#: lib/pleroma/web/admin_api/controllers/admin_api_controller.ex:388
+#: lib/pleroma/web/admin_api/controllers/admin_api_controller.ex:414 lib/pleroma/web/admin_api/controllers/admin_api_controller.ex:456
+#, elixir-format
+msgid "No such permission_group"
+msgstr ""
+
+#: lib/pleroma/plugs/uploaded_media.ex:84
+#: lib/pleroma/web/activity_pub/activity_pub_controller.ex:486 lib/pleroma/web/admin_api/controllers/fallback_controller.ex:11
+#: lib/pleroma/web/feed/user_controller.ex:71 lib/pleroma/web/ostatus/ostatus_controller.ex:143
+#, elixir-format
+msgid "Not found"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:331
+#, elixir-format
+msgid "Poll's author can't vote"
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/fallback_controller.ex:20
+#: lib/pleroma/web/mastodon_api/controllers/poll_controller.ex:37 lib/pleroma/web/mastodon_api/controllers/poll_controller.ex:49
+#: lib/pleroma/web/mastodon_api/controllers/poll_controller.ex:50 lib/pleroma/web/mastodon_api/controllers/status_controller.ex:306
+#: lib/pleroma/web/mastodon_api/controllers/subscription_controller.ex:71
+#, elixir-format
+msgid "Record not found"
+msgstr ""
+
+#: lib/pleroma/web/admin_api/controllers/fallback_controller.ex:35
+#: lib/pleroma/web/feed/user_controller.ex:77 lib/pleroma/web/mastodon_api/controllers/fallback_controller.ex:36
+#: lib/pleroma/web/ostatus/ostatus_controller.ex:149
+#, elixir-format
+msgid "Something went wrong"
+msgstr ""
+
+#: lib/pleroma/web/common_api/activity_draft.ex:107
+#, elixir-format
+msgid "The message visibility must be direct"
+msgstr ""
+
+#: lib/pleroma/web/common_api/utils.ex:573
+#, elixir-format
+msgid "The status is over the character limit"
+msgstr ""
+
+#: lib/pleroma/plugs/ensure_public_or_authenticated_plug.ex:31
+#, elixir-format
+msgid "This resource requires authentication."
+msgstr ""
+
+#: lib/pleroma/plugs/rate_limiter/rate_limiter.ex:206
+#, elixir-format
+msgid "Throttled"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:356
+#, elixir-format
+msgid "Too many choices"
+msgstr ""
+
+#: lib/pleroma/web/activity_pub/activity_pub_controller.ex:443
+#, elixir-format
+msgid "Unhandled activity type"
+msgstr ""
+
+#: lib/pleroma/web/admin_api/controllers/admin_api_controller.ex:485
+#, elixir-format
+msgid "You can't revoke your own admin status."
+msgstr ""
+
+#: lib/pleroma/web/oauth/oauth_controller.ex:221
+#: lib/pleroma/web/oauth/oauth_controller.ex:308
+#, elixir-format
+msgid "Your account is currently disabled"
+msgstr ""
+
+#: lib/pleroma/web/oauth/oauth_controller.ex:183
+#: lib/pleroma/web/oauth/oauth_controller.ex:331
+#, elixir-format
+msgid "Your login is missing a confirmed e-mail address"
+msgstr ""
+
+#: lib/pleroma/web/activity_pub/activity_pub_controller.ex:390
+#, elixir-format
+msgid "can't read inbox of %{nickname} as %{as_nickname}"
+msgstr ""
+
+#: lib/pleroma/web/activity_pub/activity_pub_controller.ex:473
+#, elixir-format
+msgid "can't update outbox of %{nickname} as %{as_nickname}"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:471
+#, elixir-format
+msgid "conversation is already muted"
+msgstr ""
+
+#: lib/pleroma/web/activity_pub/activity_pub_controller.ex:314
+#: lib/pleroma/web/activity_pub/activity_pub_controller.ex:492
+#, elixir-format
+msgid "error"
+msgstr ""
+
+#: lib/pleroma/web/pleroma_api/controllers/mascot_controller.ex:32
+#, elixir-format
+msgid "mascots can only be images"
+msgstr ""
+
+#: lib/pleroma/web/activity_pub/activity_pub_controller.ex:62
+#, elixir-format
+msgid "not found"
+msgstr ""
+
+#: lib/pleroma/web/oauth/oauth_controller.ex:394
+#, elixir-format
+msgid "Bad OAuth request."
+msgstr ""
+
+#: lib/pleroma/web/twitter_api/twitter_api.ex:115
+#, elixir-format
+msgid "CAPTCHA already used"
+msgstr ""
+
+#: lib/pleroma/web/twitter_api/twitter_api.ex:112
+#, elixir-format
+msgid "CAPTCHA expired"
+msgstr ""
+
+#: lib/pleroma/plugs/uploaded_media.ex:57
+#, elixir-format
+msgid "Failed"
+msgstr ""
+
+#: lib/pleroma/web/oauth/oauth_controller.ex:410
+#, elixir-format
+msgid "Failed to authenticate: %{message}."
+msgstr ""
+
+#: lib/pleroma/web/oauth/oauth_controller.ex:441
+#, elixir-format
+msgid "Failed to set up user account."
+msgstr ""
+
+#: lib/pleroma/plugs/oauth_scopes_plug.ex:38
+#, elixir-format
+msgid "Insufficient permissions: %{permissions}."
+msgstr ""
+
+#: lib/pleroma/plugs/uploaded_media.ex:104
+#, elixir-format
+msgid "Internal Error"
+msgstr ""
+
+#: lib/pleroma/web/oauth/fallback_controller.ex:22
+#: lib/pleroma/web/oauth/fallback_controller.ex:29
+#, elixir-format
+msgid "Invalid Username/Password"
+msgstr ""
+
+#: lib/pleroma/web/twitter_api/twitter_api.ex:118
+#, elixir-format
+msgid "Invalid answer data"
+msgstr ""
+
+#: lib/pleroma/web/nodeinfo/nodeinfo_controller.ex:33
+#, elixir-format
+msgid "Nodeinfo schema version not handled"
+msgstr ""
+
+#: lib/pleroma/web/oauth/oauth_controller.ex:172
+#, elixir-format
+msgid "This action is outside the authorized scopes"
+msgstr ""
+
+#: lib/pleroma/web/oauth/fallback_controller.ex:14
+#, elixir-format
+msgid "Unknown error, please check the details and try again."
+msgstr ""
+
+#: lib/pleroma/web/oauth/oauth_controller.ex:119
+#: lib/pleroma/web/oauth/oauth_controller.ex:158
+#, elixir-format
+msgid "Unlisted redirect_uri."
+msgstr ""
+
+#: lib/pleroma/web/oauth/oauth_controller.ex:390
+#, elixir-format
+msgid "Unsupported OAuth provider: %{provider}."
+msgstr ""
+
+#: lib/pleroma/uploaders/uploader.ex:72
+#, elixir-format
+msgid "Uploader callback timeout"
+msgstr ""
+
+#: lib/pleroma/web/uploader_controller.ex:23
+#, elixir-format
+msgid "bad request"
+msgstr ""
+
+#: lib/pleroma/web/twitter_api/twitter_api.ex:103
+#, elixir-format
+msgid "CAPTCHA Error"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:290
+#, elixir-format
+msgid "Could not add reaction emoji"
+msgstr ""
+
+#: lib/pleroma/web/common_api/common_api.ex:301
+#, elixir-format
+msgid "Could not remove reaction emoji"
+msgstr ""
+
+#: lib/pleroma/web/twitter_api/twitter_api.ex:129
+#, elixir-format
+msgid "Invalid CAPTCHA (Missing parameter: %{name})"
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/list_controller.ex:92
+#, elixir-format
+msgid "List not found"
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/account_controller.ex:123
+#, elixir-format
+msgid "Missing parameter: %{name}"
+msgstr ""
+
+#: lib/pleroma/web/oauth/oauth_controller.ex:210
+#: lib/pleroma/web/oauth/oauth_controller.ex:321
+#, elixir-format
+msgid "Password reset is required"
+msgstr ""
+
+#: lib/pleroma/tests/auth_test_controller.ex:9
+#: lib/pleroma/web/activity_pub/activity_pub_controller.ex:6 lib/pleroma/web/admin_api/controllers/admin_api_controller.ex:6
+#: lib/pleroma/web/admin_api/controllers/config_controller.ex:6 lib/pleroma/web/admin_api/controllers/fallback_controller.ex:6
+#: lib/pleroma/web/admin_api/controllers/invite_controller.ex:6 lib/pleroma/web/admin_api/controllers/media_proxy_cache_controller.ex:6
+#: lib/pleroma/web/admin_api/controllers/oauth_app_controller.ex:6 lib/pleroma/web/admin_api/controllers/relay_controller.ex:6
+#: lib/pleroma/web/admin_api/controllers/report_controller.ex:6 lib/pleroma/web/admin_api/controllers/status_controller.ex:6
+#: lib/pleroma/web/controller_helper.ex:6 lib/pleroma/web/embed_controller.ex:6
+#: lib/pleroma/web/fallback_redirect_controller.ex:6 lib/pleroma/web/feed/tag_controller.ex:6
+#: lib/pleroma/web/feed/user_controller.ex:6 lib/pleroma/web/mailer/subscription_controller.ex:2
+#: lib/pleroma/web/masto_fe_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/account_controller.ex:6
+#: lib/pleroma/web/mastodon_api/controllers/app_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/auth_controller.ex:6
+#: lib/pleroma/web/mastodon_api/controllers/conversation_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/custom_emoji_controller.ex:6
+#: lib/pleroma/web/mastodon_api/controllers/domain_block_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/fallback_controller.ex:6
+#: lib/pleroma/web/mastodon_api/controllers/filter_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/follow_request_controller.ex:6
+#: lib/pleroma/web/mastodon_api/controllers/instance_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/list_controller.ex:6
+#: lib/pleroma/web/mastodon_api/controllers/marker_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/mastodon_api_controller.ex:14
+#: lib/pleroma/web/mastodon_api/controllers/media_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/notification_controller.ex:6
+#: lib/pleroma/web/mastodon_api/controllers/poll_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/report_controller.ex:8
+#: lib/pleroma/web/mastodon_api/controllers/scheduled_activity_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/search_controller.ex:6
+#: lib/pleroma/web/mastodon_api/controllers/status_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/subscription_controller.ex:7
+#: lib/pleroma/web/mastodon_api/controllers/suggestion_controller.ex:6 lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex:6
+#: lib/pleroma/web/media_proxy/media_proxy_controller.ex:6 lib/pleroma/web/mongooseim/mongoose_im_controller.ex:6
+#: lib/pleroma/web/nodeinfo/nodeinfo_controller.ex:6 lib/pleroma/web/oauth/fallback_controller.ex:6
+#: lib/pleroma/web/oauth/mfa_controller.ex:10 lib/pleroma/web/oauth/oauth_controller.ex:6
+#: lib/pleroma/web/ostatus/ostatus_controller.ex:6 lib/pleroma/web/pleroma_api/controllers/account_controller.ex:6
+#: lib/pleroma/web/pleroma_api/controllers/chat_controller.ex:5 lib/pleroma/web/pleroma_api/controllers/conversation_controller.ex:6
+#: lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex:2 lib/pleroma/web/pleroma_api/controllers/emoji_reaction_controller.ex:6
+#: lib/pleroma/web/pleroma_api/controllers/mascot_controller.ex:6 lib/pleroma/web/pleroma_api/controllers/notification_controller.ex:6
+#: lib/pleroma/web/pleroma_api/controllers/scrobble_controller.ex:6
+#: lib/pleroma/web/pleroma_api/controllers/two_factor_authentication_controller.ex:7 lib/pleroma/web/static_fe/static_fe_controller.ex:6
+#: lib/pleroma/web/twitter_api/controllers/password_controller.ex:10 lib/pleroma/web/twitter_api/controllers/remote_follow_controller.ex:6
+#: lib/pleroma/web/twitter_api/controllers/util_controller.ex:6 lib/pleroma/web/twitter_api/twitter_api_controller.ex:6
+#: lib/pleroma/web/uploader_controller.ex:6 lib/pleroma/web/web_finger/web_finger_controller.ex:6
+#, elixir-format
+msgid "Security violation: OAuth scopes check was neither handled nor explicitly skipped."
+msgstr ""
+
+#: lib/pleroma/plugs/ensure_authenticated_plug.ex:28
+#, elixir-format
+msgid "Two-factor authentication enabled, you must use a access token."
+msgstr ""
+
+#: lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex:210
+#, elixir-format
+msgid "Unexpected error occurred while adding file to pack."
+msgstr ""
+
+#: lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex:138
+#, elixir-format
+msgid "Unexpected error occurred while creating pack."
+msgstr ""
+
+#: lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex:278
+#, elixir-format
+msgid "Unexpected error occurred while removing file from pack."
+msgstr ""
+
+#: lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex:250
+#, elixir-format
+msgid "Unexpected error occurred while updating file in pack."
+msgstr ""
+
+#: lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex:179
+#, elixir-format
+msgid "Unexpected error occurred while updating pack metadata."
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/subscription_controller.ex:61
+#, elixir-format
+msgid "Web push subscription is disabled on this Pleroma instance"
+msgstr ""
+
+#: lib/pleroma/web/admin_api/controllers/admin_api_controller.ex:451
+#, elixir-format
+msgid "You can't revoke your own admin/moderator status."
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex:126
+#, elixir-format
+msgid "authorization required for timeline view"
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/fallback_controller.ex:24
+#, elixir-format
+msgid "Access denied"
+msgstr ""
+
+#: lib/pleroma/web/mastodon_api/controllers/account_controller.ex:282
+#, elixir-format
+msgid "This API requires an authenticated user"
+msgstr ""
+
+#: lib/pleroma/plugs/user_is_admin_plug.ex:21
+#, elixir-format
+msgid "User is not an admin."
+msgstr ""

priv/repo/migrations/20200915095704_remove_background_jobs.exs

@@ -0,0 +1,22 @@
+defmodule Pleroma.Repo.Migrations.RemoveBackgroundJobs do
+  use Ecto.Migration
+
+  import Ecto.Query, only: [from: 2]
+
+  def up do
+    from(j in "oban_jobs",
+      where:
+        j.queue == ^"background" and
+          fragment("?->>'op'", j.args) in ^[
+            "fetch_data_for_activity",
+            "media_proxy_prefetch",
+            "media_proxy_preload"
+          ] and
+          j.worker == ^"Pleroma.Workers.BackgroundWorker",
+      select: [:id]
+    )
+    |> Pleroma.Repo.delete_all()
+  end
+
+  def down, do: :ok
+end

priv/static/adminfe/index.html

@@ -1 +1 @@
-<!DOCTYPE html><html><head><meta charset=utf-8><meta http-equiv=X-UA-Compatible content="IE=edge,chrome=1"><meta name=renderer content=webkit><meta name=viewport content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no"><title>Admin FE</title><link rel="shortcut icon" href=favicon.ico><link href=chunk-elementUI.f77689d7.css rel=stylesheet><link href=chunk-libs.5cf7f50a.css rel=stylesheet><link href=app.6fb984d1.css rel=stylesheet></head><body><div id=app></div><script type=text/javascript src=static/js/runtime.8f631d12.js></script><script type=text/javascript src=static/js/chunk-elementUI.21957ec8.js></script><script type=text/javascript src=static/js/chunk-libs.32ea9181.js></script><script type=text/javascript src=static/js/app.69891fda.js></script></body></html>
+<!DOCTYPE html><html><head><meta charset=utf-8><meta http-equiv=X-UA-Compatible content="IE=edge,chrome=1"><meta name=renderer content=webkit><meta name=viewport content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no"><title>Admin FE</title><link rel="shortcut icon" href=favicon.ico><link href=chunk-elementUI.f77689d7.css rel=stylesheet><link href=chunk-libs.5cf7f50a.css rel=stylesheet><link href=app.6fb984d1.css rel=stylesheet></head><body><div id=app></div><script type=text/javascript src=static/js/runtime.ba96836e.js></script><script type=text/javascript src=static/js/chunk-elementUI.21957ec8.js></script><script type=text/javascript src=static/js/chunk-libs.32ea9181.js></script><script type=text/javascript src=static/js/app.c67f9a2f.js></script></body></html>