Merge remote-tracking branch 'origin/develop' into conversations-import

.gitignore

@@ -38,3 +38,7 @@ erl_crash.dump
 
 # Prevent committing docs files
 /priv/static/doc/*
+
+# Code test coverage
+/cover
+/Elixir.*.coverdata

.gitlab-ci.yml

@@ -52,6 +52,7 @@ unit-testing:
     - mix ecto.create
     - mix ecto.migrate
     - mix test --trace --preload-modules
+    - mix coveralls
 
 lint:
   stage: test

CHANGELOG.md

@@ -21,12 +21,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Pleroma API: Healthcheck endpoint
 - Admin API: Endpoints for listing/revoking invite tokens
 - Admin API: Endpoints for making users follow/unfollow each other
+- Admin API: added filters (role, tags, email, name) for users endpoint
+- AdminFE: initial release with basic user management accessible at /pleroma/admin/
 - Mastodon API: [Scheduled statuses](https://docs.joinmastodon.org/api/rest/scheduled-statuses/)
 - Mastodon API: `/api/v1/notifications/destroy_multiple` (glitch-soc extension)
 - Mastodon API: `/api/v1/pleroma/accounts/:id/favourites` (API extension)
 - Mastodon API: [Reports](https://docs.joinmastodon.org/api/rest/reports/)
+- Mastodon API: `POST /api/v1/accounts` (account creation API)
 - ActivityPub C2S: OAuth endpoints
-- Metadata RelMe provider
+- Metadata: RelMe provider
 - OAuth: added support for refresh tokens
 - Emoji packs and emoji pack manager
 
@@ -41,8 +44,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Federation: Removed `inReplyToStatusId` from objects
 - Configuration: Dedupe enabled by default
 - Configuration: Added `extra_cookie_attrs` for setting non-standard cookie attributes. Defaults to ["SameSite=Lax"] so that remote follows work.
-- Pleroma API: Support for emoji tags in `/api/pleroma/emoji` resulting in a breaking API change
 - Timelines: Messages involving people you have blocked will be excluded from the timeline in all cases instead of just repeats.
+- Admin API: Move the user related API to `api/pleroma/admin/users`
+- Pleroma API: Support for emoji tags in `/api/pleroma/emoji` resulting in a breaking API change
 - Mastodon API: Support for `exclude_types`, `limit` and `min_id` in `/api/v1/notifications`
 - Mastodon API: Add `languages` and `registrations` to `/api/v1/instance`
 - Mastodon API: Provide plaintext versions of cw/content in the Status entity
@@ -56,17 +60,18 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: Add `with_muted` parameter to timeline endpoints
 - Mastodon API: Actual reblog hiding instead of a dummy
 - Mastodon API: Remove attachment limit in the Status entity
+- Mastodon API: Added support max_id & since_id for bookmark timeline endpoints.
 - Deps: Updated Cowboy to 2.6
 - Deps: Updated Ecto to 3.0.7
 - Don't ship finmoji by default, they can be installed as an emoji pack
-- Mastodon API: Added support max_id & since_id for bookmark timeline endpoints.
+- Hide deactivated users and their statuses
 
 ### Fixed
 - Added an FTS index on objects. Running `vacuum analyze` and setting a larger `work_mem` is recommended.
 - Followers counter not being updated when a follower is blocked
 - Deactivated users being able to request an access token
 - Limit on request body in rich media/relme parsers being ignored resulting in a possible memory leak
-- proper Twitter Card generation instead of a dummy
+- Proper Twitter Card generation instead of a dummy
 - Deletions failing for users with a large number of posts
 - NodeInfo: Include admins in `staffAccounts`
 - ActivityPub: Crashing when requesting empty local user's outbox
@@ -90,6 +95,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: Handling of `reblogs` in `/api/v1/accounts/:id/follow`
 - Mastodon API: Correct `reblogged`, `favourited`, and `bookmarked` values in the reblog status JSON
 - Mastodon API: Exposing default scope of the user to anyone
+- Mastodon API: Make `irreversible` field default to `false` [`POST /api/v1/filters`]
+
+## Removed
+- Configuration: `config :pleroma, :fe` in favor of the more flexible `config :pleroma, :frontend_configurations` 
 
 ## [0.9.9999] - 2019-04-05
 ### Security

COPYING

@@ -15,6 +15,14 @@ priv/static/images/pleroma-tan.png
 
 ---
 
+The following files are copyright © 2019 shitposter.club, and are distributed
+under the Creative Commons Attribution 4.0 International license, you should
+have received a copy of the license file as CC-BY-4.0.
+
+priv/static/images/pleroma-fox-tan-shy.png
+
+---
+
 The following files are copyright © 2017-2019 Pleroma Authors
 <https://pleroma.social/>, and are distributed under the Creative Commons
 Attribution-ShareAlike 4.0 International license, you should have received

README.md

@@ -12,7 +12,7 @@ For clients it supports both the [GNU Social API with Qvitter extensions](https:
 
 - [Client Applications for Pleroma](https://docs-develop.pleroma.social/clients.html)
 
-No release has been made yet, but several servers have been online for months already. If you want to run your own server, feel free to contact us at @lain@pleroma.soykaf.com or in our dev chat at #pleroma on freenode or via matrix at <https://matrix.heldscal.la/#/room/#freenode_#pleroma:matrix.org>.
+If you want to run your own server, feel free to contact us at @lain@pleroma.soykaf.com or in our dev chat at #pleroma on freenode or via matrix at <https://matrix.heldscal.la/#/room/#freenode_#pleroma:matrix.org>.
 
 ## Installation
 

config/config.exs

@@ -212,6 +212,11 @@
   registrations_open: true,
   federating: true,
   federation_reachability_timeout_days: 7,
+  federation_publisher_modules: [
+    Pleroma.Web.ActivityPub.Publisher,
+    Pleroma.Web.Websub,
+    Pleroma.Web.Salmon
+  ],
   allow_relay: true,
   rewrite_policy: Pleroma.Web.ActivityPub.MRF.NoOpPolicy,
   public: true,
@@ -234,6 +239,8 @@
   safe_dm_mentions: false,
   healthcheck: false
 
+config :pleroma, :app_account_creation, enabled: true, max_requests: 25, interval: 1800
+
 config :pleroma, :markup,
   # XXX - unfortunately, inline images must be enabled by default right now, because
   # of custom emoji.  Issue #275 discusses defanging that somehow.
@@ -246,25 +253,6 @@
     Pleroma.HTML.Scrubber.Default
   ]
 
-# Deprecated, will be gone in 1.0
-config :pleroma, :fe,
-  theme: "pleroma-dark",
-  logo: "/static/logo.png",
-  logo_mask: true,
-  logo_margin: "0.1em",
-  background: "/static/aurora_borealis.jpg",
-  redirect_root_no_login: "/main/all",
-  redirect_root_login: "/main/friends",
-  show_instance_panel: true,
-  scope_options_enabled: false,
-  formatting_options_enabled: false,
-  collapse_message_with_subject: false,
-  hide_post_stats: false,
-  hide_user_stats: false,
-  scope_copy: true,
-  subject_line_behavior: "email",
-  always_show_subject_input: true
-
 config :pleroma, :frontend_configurations,
   pleroma_fe: %{
     theme: "pleroma-dark",
@@ -476,6 +464,9 @@
   token_expires_in: 600,
   issue_new_refresh_token: true
 
+config :http_signatures,
+  adapter: Pleroma.Signature
+
 # Import environment specific config. This must remain at the bottom
 # of this file so it overrides the configuration defined above.
 import_config "#{Mix.env()}.exs"

config/test.exs

@@ -59,6 +59,8 @@
   total_user_limit: 3,
   enabled: false
 
+config :pleroma, :app_account_creation, max_requests: 5
+
 try do
   import_config "test.secret.exs"
 rescue

docs/api/admin_api.md

@@ -8,15 +8,20 @@ Authentication is required and the user must be an admin.
 
 - Method `GET`
 - Query Params:
-  - *optional* `query`: **string** search term
+  - *optional* `query`: **string** search term (e.g. nickname, domain, nickname@domain)
   - *optional* `filters`: **string** comma-separated string of filters:
     - `local`: only local users
     - `external`: only external users
     - `active`: only active users
     - `deactivated`: only deactivated users
+    - `is_admin`: users with admin role
+    - `is_moderator`: users with moderator role
   - *optional* `page`: **integer** page number
   - *optional* `page_size`: **integer** number of users per page (default is `50`)
-- Example: `https://mypleroma.org/api/pleroma/admin/users?query=john&filters=local,active&page=1&page_size=10`
+  - *optional* `tags`: **[string]** tags list
+  - *optional* `name`: **string** user display name
+  - *optional* `email`: **string** user email
+- Example: `https://mypleroma.org/api/pleroma/admin/users?query=john&filters=local,active&page=1&page_size=10&tags[]=some_tag&tags[]=another_tag&name=display_name&email=email@example.com`
 - Response:
 
 ```JSON
@@ -40,7 +45,7 @@ Authentication is required and the user must be an admin.
 }
 ```
 
-## `/api/pleroma/admin/user`
+## `/api/pleroma/admin/users`
 
 ### Remove a user
 
@@ -58,7 +63,7 @@ Authentication is required and the user must be an admin.
   - `password`
 - Response: User’s nickname
 
-## `/api/pleroma/admin/user/follow`
+## `/api/pleroma/admin/users/follow`
 ### Make a user follow another user
 
 - Methods: `POST`
@@ -68,7 +73,7 @@ Authentication is required and the user must be an admin.
 - Response:
  - "ok"
 
-## `/api/pleroma/admin/user/unfollow`
+## `/api/pleroma/admin/users/unfollow`
 ### Make a user unfollow another user
 
 - Methods: `POST`
@@ -111,7 +116,7 @@ Authentication is required and the user must be an admin.
   - `nickname`
   - `tags`
 
-## `/api/pleroma/admin/permission_group/:nickname`
+## `/api/pleroma/admin/users/:nickname/permission_group`
 
 ### Get user user permission groups membership
 
@@ -126,7 +131,7 @@ Authentication is required and the user must be an admin.
 }
 ```
 
-## `/api/pleroma/admin/permission_group/:nickname/:permission_group`
+## `/api/pleroma/admin/users/:nickname/permission_group/:permission_group`
 
 Note: Available `:permission_group` is currently moderator and admin. 404 is returned when the permission group doesn’t exist.
 
@@ -160,7 +165,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
   - On success: JSON of the `user.info`
 - Note: An admin cannot revoke their own admin status.
 
-## `/api/pleroma/admin/activation_status/:nickname`
+## `/api/pleroma/admin/users/:nickname/activation_status`
 
 ### Active or deactivate a user
 
@@ -198,7 +203,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
 - Response:
   - On success: URL of the unfollowed relay
 
-## `/api/pleroma/admin/invite_token`
+## `/api/pleroma/admin/users/invite_token`
 
 ### Get an account registration invite token
 
@@ -210,7 +215,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
   ]
 - Response: invite token (base64 string)
 
-## `/api/pleroma/admin/invites`
+## `/api/pleroma/admin/users/invites`
 
 ### Get a list of generated invites
 
@@ -236,7 +241,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
 }
 ```
 
-## `/api/pleroma/admin/revoke_invite`
+## `/api/pleroma/admin/users/revoke_invite`
 
 ### Revoke invite by token
 
@@ -259,7 +264,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
 ```
 
 
-## `/api/pleroma/admin/email_invite`
+## `/api/pleroma/admin/users/email_invite`
 
 ### Sends registration invite via email
 
@@ -268,7 +273,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
   - `email`
   - `name`, optional
 
-## `/api/pleroma/admin/password_reset`
+## `/api/pleroma/admin/users/:nickname/password_reset`
 
 ### Get a password reset token for a given nickname
 

docs/api/differences_in_mastoapi_responses.md

@@ -87,3 +87,13 @@ Additional parameters can be added to the JSON body/Form data:
 
 `POST /oauth/token`
 Post here request with grant_type=refresh_token to obtain new access token. Returns an access token.
+
+## Account Registration
+`POST /api/v1/accounts`
+
+Has theses additionnal parameters (which are the same as in Pleroma-API):
+    * `fullname`: optional
+    * `bio`: optional
+    * `captcha_solution`: optional, contains provider-specific captcha solution,
+    * `captcha_token`: optional, contains provider-specific captcha token
+    * `token`: invite token required when the registerations aren't public.

docs/api/pleroma_api.md

@@ -61,6 +61,15 @@ Request parameters can be passed via [query strings](https://en.wikipedia.org/wi
 * Response: JSON. Returns `{"status": "success"}` if the deletion was successful, `{"error": "[error message]"}` otherwise
 * Example response: `{"error": "Invalid password."}`
 
+## `/api/pleroma/disable_account`
+### Disable an account
+* Method `POST`
+* Authentication: required
+* Params:
+    * `password`: user's password
+* Response: JSON. Returns `{"status": "success"}` if the account was successfully disabled, `{"error": "[error message]"}` otherwise
+* Example response: `{"error": "Invalid password."}`
+
 ## `/api/account/register`
 ### Register a new user
 * Method `POST`

docs/config.md

@@ -105,6 +105,12 @@ config :pleroma, Pleroma.Emails.Mailer,
 * `safe_dm_mentions`: If set to true, only mentions at the beginning of a post will be used to address people in direct messages. This is to prevent accidental mentioning of people when talking about them (e.g. "@friend hey i really don't like @enemy"). (Default: `false`)
 * `healthcheck`: if set to true, system data will be shown on ``/api/pleroma/healthcheck``.
 
+## :app_account_creation
+REST API for creating an account settings
+* `enabled`: Enable/disable registration
+* `max_requests`: Number of requests allowed for creating accounts
+* `interval`: Interval for restricting requests for one ip (seconds)
+
 ## :logger
 * `backends`: `:console` is used to send logs to stdout, `{ExSyslogger, :ex_syslogger}` to log to syslog, and `Quack.Logger` to log to Slack
 

lib/mix/tasks/pleroma/emoji.ex

@@ -137,7 +137,7 @@ def run(["get-packs" | args]) do
           ])
         )
 
-        files = Tesla.get!(client(), files_url).body |> Poison.decode!()
+        files = Tesla.get!(client(), files_url).body |> Jason.decode!()
 
         IO.puts(IO.ANSI.format(["Unpacking ", :bright, pack_name]))
 
@@ -239,7 +239,7 @@ def run(["gen-pack", src]) do
 
     emoji_map = Pleroma.Emoji.make_shortcode_to_file_map(tmp_pack_dir, exts)
 
-    File.write!(files_name, Poison.encode!(emoji_map, pretty: true))
+    File.write!(files_name, Jason.encode!(emoji_map, pretty: true))
 
     IO.puts("""
 
@@ -248,11 +248,11 @@ def run(["gen-pack", src]) do
     """)
 
     if File.exists?("index.json") do
-      existing_data = File.read!("index.json") |> Poison.decode!()
+      existing_data = File.read!("index.json") |> Jason.decode!()
 
       File.write!(
         "index.json",
-        Poison.encode!(
+        Jason.encode!(
           Map.merge(
             existing_data,
             pack_json
@@ -263,14 +263,14 @@ def run(["gen-pack", src]) do
 
       IO.puts("index.json file has been update with the #{name} pack")
     else
-      File.write!("index.json", Poison.encode!(pack_json, pretty: true))
+      File.write!("index.json", Jason.encode!(pack_json, pretty: true))
 
       IO.puts("index.json has been created with the #{name} pack")
     end
   end
 
   defp fetch_manifest(from) do
-    Poison.decode!(
+    Jason.decode!(
       if String.starts_with?(from, "http") do
         Tesla.get!(client(), from).body
       else

lib/mix/tasks/pleroma/user.ex

@@ -138,7 +138,7 @@ def run(["new", nickname, email | rest]) do
         bio: bio
       }
 
-      changeset = User.register_changeset(%User{}, params, confirmed: true)
+      changeset = User.register_changeset(%User{}, params, need_confirmation: false)
       {:ok, _user} = User.register(changeset)
 
       Mix.shell().info("User #{nickname} created")

lib/pleroma/activity.ex

@@ -132,7 +132,10 @@ def get_by_ap_id_with_object(ap_id) do
   end
 
   def get_by_id(id) do
-    Repo.get(Activity, id)
+    Activity
+    |> where([a], a.id == ^id)
+    |> restrict_deactivated_users()
+    |> Repo.one()
   end
 
   def get_by_id_with_object(id) do
@@ -200,6 +203,7 @@ def get_all_create_by_object_ap_id(ap_id) do
 
   def get_create_by_object_ap_id(ap_id) when is_binary(ap_id) do
     create_by_object_ap_id(ap_id)
+    |> restrict_deactivated_users()
     |> Repo.one()
   end
 
@@ -287,8 +291,41 @@ def all_by_actor_and_id(actor, status_ids) do
     |> Repo.all()
   end
 
+  def follow_requests_for_actor(%Pleroma.User{ap_id: ap_id}) do
+    from(
+      a in Activity,
+      where:
+        fragment(
+          "? ->> 'type' = 'Follow'",
+          a.data
+        ),
+      where:
+        fragment(
+          "? ->> 'state' = 'pending'",
+          a.data
+        ),
+      where:
+        fragment(
+          "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
+          a.data,
+          a.data,
+          ^ap_id
+        )
+    )
+  end
+
   @spec query_by_actor(actor()) :: Ecto.Query.t()
   def query_by_actor(actor) do
     from(a in Activity, where: a.actor == ^actor)
   end
+
+  def restrict_deactivated_users(query) do
+    from(activity in query,
+      where:
+        fragment(
+          "? not in (SELECT ap_id FROM users WHERE info->'deactivated' @> 'true')",
+          activity.actor
+        )
+    )
+  end
 end

lib/pleroma/captcha/kocaptcha.ex

@@ -15,7 +15,7 @@ def new do
         %{error: "Kocaptcha service unavailable"}
 
       {:ok, res} ->
-        json_resp = Poison.decode!(res.body)
+        json_resp = Jason.decode!(res.body)
 
         %{
           type: :kocaptcha,

lib/pleroma/config.ex

@@ -12,8 +12,12 @@ def get(key), do: get(key, nil)
   def get([key], default), do: get(key, default)
 
   def get([parent_key | keys], default) do
-    Application.get_env(:pleroma, parent_key)
+    case :pleroma
-    |> get_in(keys) || default
+         |> Application.get_env(parent_key)
+         |> get_in(keys) do
+      nil -> default
+      any -> any
+    end
   end
 
   def get(key, default) do

lib/pleroma/config/deprecation_warnings.ex

@@ -5,15 +5,6 @@
 defmodule Pleroma.Config.DeprecationWarnings do
   require Logger
 
-  def check_frontend_config_mechanism do
-    if Pleroma.Config.get(:fe) do
-      Logger.warn("""
-      !!!DEPRECATION WARNING!!!
-      You are using the old configuration mechanism for the frontend. Please check config.md.
-      """)
-    end
-  end
-
   def check_hellthread_threshold do
     if Pleroma.Config.get([:mrf_hellthread, :threshold]) do
       Logger.warn("""
@@ -24,7 +15,6 @@ def check_hellthread_threshold do
   end
 
   def warn do
-    check_frontend_config_mechanism()
     check_hellthread_threshold()
   end
 end

lib/pleroma/conversation.ex

@@ -47,8 +47,8 @@ def get_for_ap_id(ap_id) do
   """
   def create_or_bump_for(activity, opts \\ []) do
     with true <- Pleroma.Web.ActivityPub.Visibility.is_direct?(activity),
-         object <- Pleroma.Object.normalize(activity),
          "Create" <- activity.data["type"],
+         object <- Pleroma.Object.normalize(activity),
          "Note" <- object.data["type"],
          ap_id when is_binary(ap_id) and byte_size(ap_id) > 0 <- object.data["context"] do
       {:ok, conversation} = create_for_ap_id(ap_id)

lib/pleroma/gopher/server.ex

@@ -77,13 +77,13 @@ def render_activities(activities) do
       user = User.get_cached_by_ap_id(activity.data["actor"])
 
       object = Object.normalize(activity)
-      like_count = object["like_count"] || 0
+      like_count = object.data["like_count"] || 0
-      announcement_count = object["announcement_count"] || 0
+      announcement_count = object.data["announcement_count"] || 0
 
       link("Post ##{activity.id} by #{user.nickname}", "/notices/#{activity.id}") <>
         info("#{like_count} likes, #{announcement_count} repeats") <>
         "i\tfake\t(NULL)\t0\r\n" <>
-        info(HTML.strip_tags(String.replace(object["content"], "<br>", "\r")))
+        info(HTML.strip_tags(String.replace(object.data["content"], "<br>", "\r")))
     end)
     |> Enum.join("i\tfake\t(NULL)\t0\r\n")
   end

lib/pleroma/notification.ex

@@ -33,6 +33,13 @@ def changeset(%Notification{} = notification, attrs) do
   def for_user_query(user) do
     Notification
     |> where(user_id: ^user.id)
+    |> where(
+      [n, a],
+      fragment(
+        "? not in (SELECT ap_id FROM users WHERE info->'deactivated' @> 'true')",
+        a.actor
+      )
+    )
     |> join(:inner, [n], activity in assoc(n, :activity))
     |> join(:left, [n, a], object in Object,
       on:

lib/pleroma/plugs/ensure_public_or_authenticated_plug.ex

@@ -0,0 +1,31 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug do
+  import Plug.Conn
+  alias Pleroma.Config
+  alias Pleroma.User
+
+  def init(options) do
+    options
+  end
+
+  def call(conn, _) do
+    public? = Config.get!([:instance, :public])
+
+    case {public?, conn} do
+      {true, _} ->
+        conn
+
+      {false, %{assigns: %{user: %User{}}}} ->
+        conn
+
+      {false, _} ->
+        conn
+        |> put_resp_content_type("application/json")
+        |> send_resp(403, Jason.encode!(%{error: "This resource requires authentication."}))
+        |> halt
+    end
+  end
+end

lib/pleroma/plugs/http_signature.ex

@@ -4,7 +4,6 @@
 
 defmodule Pleroma.Web.Plugs.HTTPSignaturePlug do
   alias Pleroma.Web.ActivityPub.Utils
-  alias Pleroma.Web.HTTPSignatures
   import Plug.Conn
   require Logger
 

lib/pleroma/plugs/oauth_plug.ex

@@ -8,6 +8,7 @@ defmodule Pleroma.Plugs.OAuthPlug do
 
   alias Pleroma.Repo
   alias Pleroma.User
+  alias Pleroma.Web.OAuth.App
   alias Pleroma.Web.OAuth.Token
 
   @realm_reg Regex.compile!("Bearer\:?\s+(.*)$", "i")
@@ -21,22 +22,43 @@ def call(%{params: %{"access_token" => access_token}} = conn, _) do
       conn
       |> assign(:token, token_record)
       |> assign(:user, user)
+    else
+      _ ->
+        # token found, but maybe only with app
+        with {:ok, app, token_record} <- fetch_app_and_token(access_token) do
+          conn
+          |> assign(:token, token_record)
+          |> assign(:app, app)
         else
           _ -> conn
         end
     end
+  end
 
   def call(conn, _) do
-    with {:ok, token_str} <- fetch_token_str(conn),
+    case fetch_token_str(conn) do
-         {:ok, user, token_record} <- fetch_user_and_token(token_str) do
+      {:ok, token} ->
+        with {:ok, user, token_record} <- fetch_user_and_token(token) do
           conn
           |> assign(:token, token_record)
           |> assign(:user, user)
+        else
+          _ ->
+            # token found, but maybe only with app
+            with {:ok, app, token_record} <- fetch_app_and_token(token) do
+              conn
+              |> assign(:token, token_record)
+              |> assign(:app, app)
             else
               _ -> conn
             end
         end
 
+      _ ->
+        conn
+    end
+  end
+
   # Gets user by token
   #
   @spec fetch_user_and_token(String.t()) :: {:ok, User.t(), Token.t()} | nil
@@ -54,6 +76,16 @@ defp fetch_user_and_token(token) do
     end
   end
 
+  @spec fetch_app_and_token(String.t()) :: {:ok, App.t(), Token.t()} | nil
+  defp fetch_app_and_token(token) do
+    query =
+      from(t in Token, where: t.token == ^token, join: app in assoc(t, :app), preload: [app: app])
+
+    with %Token{app: app} = token_record <- Repo.one(query) do
+      {:ok, app, token_record}
+    end
+  end
+
   # Gets token from session by :oauth_token key
   #
   @spec fetch_token_from_session(Plug.Conn.t()) :: :no_token_found | {:ok, String.t()}

lib/pleroma/plugs/rate_limit_plug.ex

@@ -0,0 +1,36 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.RateLimitPlug do
+  import Phoenix.Controller, only: [json: 2]
+  import Plug.Conn
+
+  def init(opts), do: opts
+
+  def call(conn, opts) do
+    enabled? = Pleroma.Config.get([:app_account_creation, :enabled])
+
+    case check_rate(conn, Map.put(opts, :enabled, enabled?)) do
+      {:ok, _count} -> conn
+      {:error, _count} -> render_error(conn)
+      %Plug.Conn{} = conn -> conn
+    end
+  end
+
+  defp check_rate(conn, %{enabled: true} = opts) do
+    max_requests = opts[:max_requests]
+    bucket_name = conn.remote_ip |> Tuple.to_list() |> Enum.join(".")
+
+    ExRated.check_rate(bucket_name, opts[:interval] * 1000, max_requests)
+  end
+
+  defp check_rate(conn, _), do: conn
+
+  defp render_error(conn) do
+    conn
+    |> put_status(:forbidden)
+    |> json(%{error: "Rate limit exceeded."})
+    |> halt()
+  end
+end

lib/pleroma/signature.ex

@@ -0,0 +1,41 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Signature do
+  @behaviour HTTPSignatures.Adapter
+
+  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.ActivityPub
+  alias Pleroma.Web.ActivityPub.Utils
+  alias Pleroma.Web.Salmon
+  alias Pleroma.Web.WebFinger
+
+  def fetch_public_key(conn) do
+    with actor_id <- Utils.get_ap_id(conn.params["actor"]),
+         {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
+      {:ok, public_key}
+    else
+      e ->
+        {:error, e}
+    end
+  end
+
+  def refetch_public_key(conn) do
+    with actor_id <- Utils.get_ap_id(conn.params["actor"]),
+         {:ok, _user} <- ActivityPub.make_user_from_ap_id(actor_id),
+         {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
+      {:ok, public_key}
+    else
+      e ->
+        {:error, e}
+    end
+  end
+
+  def sign(%User{} = user, headers) do
+    with {:ok, %{info: %{keys: keys}}} <- WebFinger.ensure_keys_present(user),
+         {:ok, private_key, _} <- Salmon.keys_from_pem(keys) do
+      HTTPSignatures.sign(private_key, user.ap_id <> "#main-key", headers)
+    end
+  end
+end

lib/pleroma/stats.ex

@@ -34,7 +34,7 @@ def schedule_update do
   def update_stats do
     peers =
       from(
-        u in Pleroma.User,
+        u in User,
         select: fragment("distinct split_part(?, '@', 2)", u.nickname),
         where: u.local != ^true
       )
@@ -44,10 +44,13 @@ def update_stats do
     domain_count = Enum.count(peers)
 
     status_query =
-      from(u in User.local_user_query(), select: fragment("sum((?->>'note_count')::int)", u.info))
+      from(u in User.Query.build(%{local: true}),
+        select: fragment("sum((?->>'note_count')::int)", u.info)
+      )
 
     status_count = Repo.one(status_query)
-    user_count = Repo.aggregate(User.active_local_user_query(), :count, :id)
+
+    user_count = Repo.aggregate(User.Query.build(%{local: true, active: true}), :count, :id)
 
     Agent.update(__MODULE__, fn _ ->
       {peers, %{domain_count: domain_count, status_count: status_count, user_count: user_count}}

lib/pleroma/uploaders/swift/keystone.ex

@@ -14,7 +14,7 @@ def process_url(url) do
 
   def process_response_body(body) do
     body
-    |> Poison.decode!()
+    |> Jason.decode!()
   end
 
   def get_token do
@@ -38,7 +38,7 @@ def get_token do
   end
 
   def make_auth_body(username, password, tenant) do
-    Poison.encode!(%{
+    Jason.encode!(%{
       :auth => %{
         :passwordCredentials => %{
           :username => username,

lib/pleroma/user.ex

@@ -105,10 +105,8 @@ def ap_followers(%User{follower_address: fa}) when is_binary(fa), do: fa
   def ap_followers(%User{} = user), do: "#{ap_id(user)}/followers"
 
   def user_info(%User{} = user) do
-    oneself = if user.local, do: 1, else: 0
-
     %{
-      following_count: length(user.following) - oneself,
+      following_count: following_count(user),
       note_count: user.info.note_count,
       follower_count: user.info.follower_count,
       locked: user.info.locked,
@@ -117,6 +115,20 @@ def user_info(%User{} = user) do
     }
   end
 
+  def restrict_deactivated(query) do
+    from(u in query,
+      where: not fragment("? \\? 'deactivated' AND ?->'deactivated' @> 'true'", u.info, u.info)
+    )
+  end
+
+  def following_count(%User{following: []}), do: 0
+
+  def following_count(%User{} = user) do
+    user
+    |> get_friends_query()
+    |> Repo.aggregate(:count, :id)
+  end
+
   def remote_user_creation(params) do
     params =
       params
@@ -204,14 +216,15 @@ def reset_password(user, data) do
   end
 
   def register_changeset(struct, params \\ %{}, opts \\ []) do
-    confirmation_status =
+    need_confirmation? =
-      if opts[:confirmed] || !Pleroma.Config.get([:instance, :account_activation_required]) do
+      if is_nil(opts[:need_confirmation]) do
-        :confirmed
+        Pleroma.Config.get([:instance, :account_activation_required])
       else
-        :unconfirmed
+        opts[:need_confirmation]
       end
 
-    info_change = User.Info.confirmation_changeset(%User.Info{}, confirmation_status)
+    info_change =
+      User.Info.confirmation_changeset(%User.Info{}, need_confirmation: need_confirmation?)
 
     changeset =
       struct
@@ -254,10 +267,7 @@ defp autofollow_users(user) do
     candidates = Pleroma.Config.get([:instance, :autofollowed_nicknames])
 
     autofollowed_users =
-      from(u in User,
+      User.Query.build(%{nickname: candidates, local: true, deactivated: false})
-        where: u.local == true,
-        where: u.nickname in ^candidates
-      )
       |> Repo.all()
 
     follow_all(user, autofollowed_users)
@@ -415,24 +425,6 @@ def following?(%User{} = follower, %User{} = followed) do
     Enum.member?(follower.following, followed.follower_address)
   end
 
-  def follow_import(%User{} = follower, followed_identifiers)
-      when is_list(followed_identifiers) do
-    Enum.map(
-      followed_identifiers,
-      fn followed_identifier ->
-        with {:ok, %User{} = followed} <- get_or_fetch(followed_identifier),
-             {:ok, follower} <- maybe_direct_follow(follower, followed),
-             {:ok, _} <- ActivityPub.follow(follower, followed) do
-          followed
-        else
-          err ->
-            Logger.debug("follow_import failed for #{followed_identifier} with: #{inspect(err)}")
-            err
-        end
-      end
-    )
-  end
-
   def locked?(%User{} = user) do
     user.info.locked || false
   end
@@ -554,8 +546,7 @@ def get_or_fetch_by_nickname(nickname) do
         with [_nick, _domain] <- String.split(nickname, "@"),
              {:ok, user} <- fetch_by_nickname(nickname) do
           if Pleroma.Config.get([:fetch_initial_posts, :enabled]) do
-            # TODO turn into job
+            fetch_initial_posts(user)
-            {:ok, _} = Task.start(__MODULE__, :fetch_initial_posts, [user])
           end
 
           {:ok, user}
@@ -566,29 +557,20 @@ def get_or_fetch_by_nickname(nickname) do
   end
 
   @doc "Fetch some posts when the user has just been federated with"
-  def fetch_initial_posts(user) do
+  def fetch_initial_posts(user),
-    pages = Pleroma.Config.get!([:fetch_initial_posts, :pages])
+    do: PleromaJobQueue.enqueue(:background, __MODULE__, [:fetch_initial_posts, user])
 
-    Enum.each(
+  @spec get_followers_query(User.t(), pos_integer() | nil) :: Ecto.Query.t()
-      # Insert all the posts in reverse order, so they're in the right order on the timeline
+  def get_followers_query(%User{} = user, nil) do
-      Enum.reverse(Utils.fetch_ordered_collection(user.info.source_data["outbox"], pages)),
+    User.Query.build(%{followers: user, deactivated: false})
-      &Pleroma.Web.Federator.incoming_ap_doc/1
-    )
-  end
-
-  def get_followers_query(%User{id: id, follower_address: follower_address}, nil) do
-    from(
-      u in User,
-      where: fragment("? <@ ?", ^[follower_address], u.following),
-      where: u.id != ^id
-    )
   end
 
   def get_followers_query(user, page) do
     from(u in get_followers_query(user, nil))
-    |> paginate(page, 20)
+    |> User.Query.paginate(page, 20)
   end
 
+  @spec get_followers_query(User.t()) :: Ecto.Query.t()
   def get_followers_query(user), do: get_followers_query(user, nil)
 
   def get_followers(user, page \\ nil) do
@@ -603,19 +585,17 @@ def get_followers_ids(user, page \\ nil) do
     Repo.all(from(u in q, select: u.id))
   end
 
-  def get_friends_query(%User{id: id, following: following}, nil) do
+  @spec get_friends_query(User.t(), pos_integer() | nil) :: Ecto.Query.t()
-    from(
+  def get_friends_query(%User{} = user, nil) do
-      u in User,
+    User.Query.build(%{friends: user, deactivated: false})
-      where: u.follower_address in ^following,
-      where: u.id != ^id
-    )
   end
 
   def get_friends_query(user, page) do
     from(u in get_friends_query(user, nil))
-    |> paginate(page, 20)
+    |> User.Query.paginate(page, 20)
   end
 
+  @spec get_friends_query(User.t()) :: Ecto.Query.t()
   def get_friends_query(user), do: get_friends_query(user, nil)
 
   def get_friends(user, page \\ nil) do
@@ -630,33 +610,10 @@ def get_friends_ids(user, page \\ nil) do
     Repo.all(from(u in q, select: u.id))
   end
 
-  def get_follow_requests_query(%User{} = user) do
+  @spec get_follow_requests(User.t()) :: {:ok, [User.t()]}
-    from(
-      a in Activity,
-      where:
-        fragment(
-          "? ->> 'type' = 'Follow'",
-          a.data
-        ),
-      where:
-        fragment(
-          "? ->> 'state' = 'pending'",
-          a.data
-        ),
-      where:
-        fragment(
-          "coalesce((?)->'object'->>'id', (?)->>'object') = ?",
-          a.data,
-          a.data,
-          ^user.ap_id
-        )
-    )
-  end
-
   def get_follow_requests(%User{} = user) do
     users =
-      user
+      Activity.follow_requests_for_actor(user)
-      |> User.get_follow_requests_query()
       |> join(:inner, [a], u in User, on: a.actor == u.ap_id)
       |> where([a, u], not fragment("? @> ?", u.following, ^[user.follower_address]))
       |> group_by([a, u], u.id)
@@ -720,18 +677,15 @@ def update_note_count(%User{} = user) do
 
     info_cng = User.Info.set_note_count(user.info, note_count)
 
-    cng =
+    user
-      change(user)
+    |> change()
     |> put_embed(:info, info_cng)
-
+    |> update_and_set_cache()
-    update_and_set_cache(cng)
   end
 
   def update_follower_count(%User{} = user) do
     follower_count_query =
-      User
+      User.Query.build(%{followers: user, deactivated: false})
-      |> where([u], ^user.follower_address in u.following)
-      |> where([u], u.id != ^user.id)
       |> select([u], %{count: count(u.id)})
 
     User
@@ -755,38 +709,19 @@ def update_follower_count(%User{} = user) do
     end
   end
 
-  def get_users_from_set_query(ap_ids, false) do
+  @spec get_users_from_set([String.t()], boolean()) :: [User.t()]
-    from(
-      u in User,
-      where: u.ap_id in ^ap_ids
-    )
-  end
-
-  def get_users_from_set_query(ap_ids, true) do
-    query = get_users_from_set_query(ap_ids, false)
-
-    from(
-      u in query,
-      where: u.local == true
-    )
-  end
-
   def get_users_from_set(ap_ids, local_only \\ true) do
-    get_users_from_set_query(ap_ids, local_only)
+    criteria = %{ap_id: ap_ids, deactivated: false}
+    criteria = if local_only, do: Map.put(criteria, :local, true), else: criteria
+
+    User.Query.build(criteria)
     |> Repo.all()
   end
 
+  @spec get_recipients_from_activity(Activity.t()) :: [User.t()]
   def get_recipients_from_activity(%Activity{recipients: to}) do
-    query =
+    User.Query.build(%{recipients_from_activity: to, local: true, deactivated: false})
-      from(
+    |> Repo.all()
-        u in User,
-        where: u.ap_id in ^to,
-        or_where: fragment("? && ?", u.following, ^to)
-      )
-
-    query = from(u in query, where: u.local == true)
-
-    Repo.all(query)
   end
 
   def search(query, resolve \\ false, for_user \\ nil) do
@@ -883,6 +818,7 @@ defp fts_search_subquery(term, query \\ User) do
           ^processed_query
         )
     )
+    |> restrict_deactivated()
   end
 
   defp trigram_search_subquery(term) do
@@ -901,23 +837,7 @@ defp trigram_search_subquery(term) do
       },
       where: fragment("trim(? || ' ' || coalesce(?, '')) % ?", u.nickname, u.name, ^term)
     )
-  end
+    |> restrict_deactivated()
-
-  def blocks_import(%User{} = blocker, blocked_identifiers) when is_list(blocked_identifiers) do
-    Enum.map(
-      blocked_identifiers,
-      fn blocked_identifier ->
-        with {:ok, %User{} = blocked} <- get_or_fetch(blocked_identifier),
-             {:ok, blocker} <- block(blocker, blocked),
-             {:ok, _} <- ActivityPub.block(blocker, blocked) do
-          blocked
-        else
-          err ->
-            Logger.debug("blocks_import failed for #{blocked_identifier} with: #{inspect(err)}")
-            err
-        end
-      end
-    )
   end
 
   def mute(muter, %User{ap_id: ap_id}) do
@@ -1048,14 +968,23 @@ def subscribed_to?(user, %{ap_id: ap_id}) do
     end
   end
 
-  def muted_users(user),
+  @spec muted_users(User.t()) :: [User.t()]
-    do: Repo.all(from(u in User, where: u.ap_id in ^user.info.mutes))
+  def muted_users(user) do
+    User.Query.build(%{ap_id: user.info.mutes, deactivated: false})
+    |> Repo.all()
+  end
 
-  def blocked_users(user),
+  @spec blocked_users(User.t()) :: [User.t()]
-    do: Repo.all(from(u in User, where: u.ap_id in ^user.info.blocks))
+  def blocked_users(user) do
+    User.Query.build(%{ap_id: user.info.blocks, deactivated: false})
+    |> Repo.all()
+  end
 
-  def subscribers(user),
+  @spec subscribers(User.t()) :: [User.t()]
-    do: Repo.all(from(u in User, where: u.ap_id in ^user.info.subscribers))
+  def subscribers(user) do
+    User.Query.build(%{ap_id: user.info.subscribers, deactivated: false})
+    |> Repo.all()
+  end
 
   def block_domain(user, domain) do
     info_cng =
@@ -1081,77 +1010,25 @@ def unblock_domain(user, domain) do
     update_and_set_cache(cng)
   end
 
-  def maybe_local_user_query(query, local) do
+  def deactivate_async(user, status \\ true) do
-    if local, do: local_user_query(query), else: query
+    PleromaJobQueue.enqueue(:background, __MODULE__, [:deactivate_async, user, status])
-  end
-
-  def local_user_query(query \\ User) do
-    from(
-      u in query,
-      where: u.local == true,
-      where: not is_nil(u.nickname)
-    )
-  end
-
-  def maybe_external_user_query(query, external) do
-    if external, do: external_user_query(query), else: query
-  end
-
-  def external_user_query(query \\ User) do
-    from(
-      u in query,
-      where: u.local == false,
-      where: not is_nil(u.nickname)
-    )
-  end
-
-  def maybe_active_user_query(query, active) do
-    if active, do: active_user_query(query), else: query
-  end
-
-  def active_user_query(query \\ User) do
-    from(
-      u in query,
-      where: fragment("not (?->'deactivated' @> 'true')", u.info),
-      where: not is_nil(u.nickname)
-    )
-  end
-
-  def maybe_deactivated_user_query(query, deactivated) do
-    if deactivated, do: deactivated_user_query(query), else: query
-  end
-
-  def deactivated_user_query(query \\ User) do
-    from(
-      u in query,
-      where: fragment("(?->'deactivated' @> 'true')", u.info),
-      where: not is_nil(u.nickname)
-    )
-  end
-
-  def active_local_user_query do
-    from(
-      u in local_user_query(),
-      where: fragment("not (?->'deactivated' @> 'true')", u.info)
-    )
-  end
-
-  def moderator_user_query do
-    from(
-      u in User,
-      where: u.local == true,
-      where: fragment("?->'is_moderator' @> 'true'", u.info)
-    )
   end
 
   def deactivate(%User{} = user, status \\ true) do
     info_cng = User.Info.set_activation_status(user.info, status)
 
-    cng =
+    with {:ok, friends} <- User.get_friends(user),
-      change(user)
+         {:ok, followers} <- User.get_followers(user),
+         {:ok, user} <-
+           user
+           |> change()
            |> put_embed(:info, info_cng)
+           |> update_and_set_cache() do
+      Enum.each(followers, &invalidate_cache(&1))
+      Enum.each(friends, &update_follower_count(&1))
 
-    update_and_set_cache(cng)
+      {:ok, user}
+    end
   end
 
   def update_notification_settings(%User{} = user, settings \\ %{}) do
@@ -1182,6 +1059,75 @@ def perform(:delete, %User{} = user) do
     delete_user_activities(user)
   end
 
+  @spec perform(atom(), User.t()) :: {:ok, User.t()}
+  def perform(:fetch_initial_posts, %User{} = user) do
+    pages = Pleroma.Config.get!([:fetch_initial_posts, :pages])
+
+    Enum.each(
+      # Insert all the posts in reverse order, so they're in the right order on the timeline
+      Enum.reverse(Utils.fetch_ordered_collection(user.info.source_data["outbox"], pages)),
+      &Pleroma.Web.Federator.incoming_ap_doc/1
+    )
+
+    {:ok, user}
+  end
+
+  def perform(:deactivate_async, user, status), do: deactivate(user, status)
+
+  @spec perform(atom(), User.t(), list()) :: list() | {:error, any()}
+  def perform(:blocks_import, %User{} = blocker, blocked_identifiers)
+      when is_list(blocked_identifiers) do
+    Enum.map(
+      blocked_identifiers,
+      fn blocked_identifier ->
+        with {:ok, %User{} = blocked} <- get_or_fetch(blocked_identifier),
+             {:ok, blocker} <- block(blocker, blocked),
+             {:ok, _} <- ActivityPub.block(blocker, blocked) do
+          blocked
+        else
+          err ->
+            Logger.debug("blocks_import failed for #{blocked_identifier} with: #{inspect(err)}")
+            err
+        end
+      end
+    )
+  end
+
+  @spec perform(atom(), User.t(), list()) :: list() | {:error, any()}
+  def perform(:follow_import, %User{} = follower, followed_identifiers)
+      when is_list(followed_identifiers) do
+    Enum.map(
+      followed_identifiers,
+      fn followed_identifier ->
+        with {:ok, %User{} = followed} <- get_or_fetch(followed_identifier),
+             {:ok, follower} <- maybe_direct_follow(follower, followed),
+             {:ok, _} <- ActivityPub.follow(follower, followed) do
+          followed
+        else
+          err ->
+            Logger.debug("follow_import failed for #{followed_identifier} with: #{inspect(err)}")
+            err
+        end
+      end
+    )
+  end
+
+  def blocks_import(%User{} = blocker, blocked_identifiers) when is_list(blocked_identifiers),
+    do:
+      PleromaJobQueue.enqueue(:background, __MODULE__, [
+        :blocks_import,
+        blocker,
+        blocked_identifiers
+      ])
+
+  def follow_import(%User{} = follower, followed_identifiers) when is_list(followed_identifiers),
+    do:
+      PleromaJobQueue.enqueue(:background, __MODULE__, [
+        :follow_import,
+        follower,
+        followed_identifiers
+      ])
+
   def delete_user_activities(%User{ap_id: ap_id} = user) do
     stream =
       ap_id
@@ -1235,8 +1181,8 @@ def get_or_fetch_by_ap_id(ap_id) do
       resp = fetch_by_ap_id(ap_id)
 
       if should_fetch_initial do
-        with {:ok, %User{} = user} = resp do
+        with {:ok, %User{} = user} <- resp do
-          {:ok, _} = Task.start(__MODULE__, :fetch_initial_posts, [user])
+          fetch_initial_posts(user)
         end
       end
 
@@ -1306,7 +1252,7 @@ def ap_enabled?(%User{info: info}), do: info.ap_enabled
   def ap_enabled?(_), do: false
 
   @doc "Gets or fetch a user by uri or nickname."
-  @spec get_or_fetch(String.t()) :: User.t()
+  @spec get_or_fetch(String.t()) :: {:ok, User.t()} | {:error, String.t()}
   def get_or_fetch("http" <> _host = uri), do: get_or_fetch_by_ap_id(uri)
   def get_or_fetch(nickname), do: get_or_fetch_by_nickname(nickname)
 
@@ -1423,22 +1369,12 @@ def error_user(ap_id) do
     }
   end
 
+  @spec all_superusers() :: [User.t()]
   def all_superusers do
-    from(
+    User.Query.build(%{super_users: true, local: true, deactivated: false})
-      u in User,
-      where: u.local == true,
-      where: fragment("?->'is_admin' @> 'true' OR ?->'is_moderator' @> 'true'", u.info, u.info)
-    )
     |> Repo.all()
   end
 
-  defp paginate(query, page, page_size) do
-    from(u in query,
-      limit: ^page_size,
-      offset: ^((page - 1) * page_size)
-    )
-  end
-
   def showing_reblogs?(%User{} = user, %User{} = target) do
     target.ap_id not in user.info.muted_reblogs
   end

lib/pleroma/user/info.ex

@@ -8,6 +8,8 @@ defmodule Pleroma.User.Info do
 
   alias Pleroma.User.Info
 
+  @type t :: %__MODULE__{}
+
   embedded_schema do
     field(:banner, :map, default: %{})
     field(:background, :map, default: %{})
@@ -210,21 +212,23 @@ def profile_update(info, params) do
     ])
   end
 
-  def confirmation_changeset(info, :confirmed) do
+  @spec confirmation_changeset(Info.t(), keyword()) :: Ecto.Changerset.t()
-    confirmation_changeset(info, %{
+  def confirmation_changeset(info, opts) do
-      confirmation_pending: false,
+    need_confirmation? = Keyword.get(opts, :need_confirmation)
-      confirmation_token: nil
-    })
-  end
 
-  def confirmation_changeset(info, :unconfirmed) do
+    params =
-    confirmation_changeset(info, %{
+      if need_confirmation? do
+        %{
           confirmation_pending: true,
           confirmation_token: :crypto.strong_rand_bytes(32) |> Base.url_encode64()
-    })
+        }
+      else
+        %{
+          confirmation_pending: false,
+          confirmation_token: nil
+        }
       end
 
-  def confirmation_changeset(info, params) do
     cast(info, params, [:confirmation_pending, :confirmation_token])
   end
 

lib/pleroma/user/query.ex

@@ -0,0 +1,154 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.User.Query do
+  @moduledoc """
+  User query builder module. Builds query from new query or another user query.
+
+    ## Example:
+        query = Pleroma.User.Query(%{nickname: "nickname"})
+        another_query = Pleroma.User.Query.build(query, %{email: "email@example.com"})
+        Pleroma.Repo.all(query)
+        Pleroma.Repo.all(another_query)
+
+  Adding new rules:
+    - *ilike criteria*
+      - add field to @ilike_criteria list
+      - pass non empty string
+      - e.g. Pleroma.User.Query.build(%{nickname: "nickname"})
+    - *equal criteria*
+      - add field to @equal_criteria list
+      - pass non empty string
+      - e.g. Pleroma.User.Query.build(%{email: "email@example.com"})
+    - *contains criteria*
+      - add field to @containns_criteria list
+      - pass values list
+      - e.g. Pleroma.User.Query.build(%{ap_id: ["http://ap_id1", "http://ap_id2"]})
+  """
+  import Ecto.Query
+  import Pleroma.Web.AdminAPI.Search, only: [not_empty_string: 1]
+  alias Pleroma.User
+
+  @type criteria ::
+          %{
+            query: String.t(),
+            tags: [String.t()],
+            name: String.t(),
+            email: String.t(),
+            local: boolean(),
+            external: boolean(),
+            active: boolean(),
+            deactivated: boolean(),
+            is_admin: boolean(),
+            is_moderator: boolean(),
+            super_users: boolean(),
+            followers: User.t(),
+            friends: User.t(),
+            recipients_from_activity: [String.t()],
+            nickname: [String.t()],
+            ap_id: [String.t()]
+          }
+          | %{}
+
+  @ilike_criteria [:nickname, :name, :query]
+  @equal_criteria [:email]
+  @role_criteria [:is_admin, :is_moderator]
+  @contains_criteria [:ap_id, :nickname]
+
+  @spec build(criteria()) :: Query.t()
+  def build(query \\ base_query(), criteria) do
+    prepare_query(query, criteria)
+  end
+
+  @spec paginate(Ecto.Query.t(), pos_integer(), pos_integer()) :: Ecto.Query.t()
+  def paginate(query, page, page_size) do
+    from(u in query,
+      limit: ^page_size,
+      offset: ^((page - 1) * page_size)
+    )
+  end
+
+  defp base_query do
+    from(u in User)
+  end
+
+  defp prepare_query(query, criteria) do
+    Enum.reduce(criteria, query, &compose_query/2)
+  end
+
+  defp compose_query({key, value}, query)
+       when key in @ilike_criteria and not_empty_string(value) do
+    # hack for :query key
+    key = if key == :query, do: :nickname, else: key
+    where(query, [u], ilike(field(u, ^key), ^"%#{value}%"))
+  end
+
+  defp compose_query({key, value}, query)
+       when key in @equal_criteria and not_empty_string(value) do
+    where(query, [u], ^[{key, value}])
+  end
+
+  defp compose_query({key, values}, query) when key in @contains_criteria and is_list(values) do
+    where(query, [u], field(u, ^key) in ^values)
+  end
+
+  defp compose_query({:tags, tags}, query) when is_list(tags) and length(tags) > 0 do
+    Enum.reduce(tags, query, &prepare_tag_criteria/2)
+  end
+
+  defp compose_query({key, _}, query) when key in @role_criteria do
+    where(query, [u], fragment("(?->? @> 'true')", u.info, ^to_string(key)))
+  end
+
+  defp compose_query({:super_users, _}, query) do
+    where(
+      query,
+      [u],
+      fragment("?->'is_admin' @> 'true' OR ?->'is_moderator' @> 'true'", u.info, u.info)
+    )
+  end
+
+  defp compose_query({:local, _}, query), do: location_query(query, true)
+
+  defp compose_query({:external, _}, query), do: location_query(query, false)
+
+  defp compose_query({:active, _}, query) do
+    where(query, [u], fragment("not (?->'deactivated' @> 'true')", u.info))
+    |> where([u], not is_nil(u.nickname))
+  end
+
+  defp compose_query({:deactivated, false}, query) do
+    User.restrict_deactivated(query)
+  end
+
+  defp compose_query({:deactivated, true}, query) do
+    where(query, [u], fragment("?->'deactivated' @> 'true'", u.info))
+    |> where([u], not is_nil(u.nickname))
+  end
+
+  defp compose_query({:followers, %User{id: id, follower_address: follower_address}}, query) do
+    where(query, [u], fragment("? <@ ?", ^[follower_address], u.following))
+    |> where([u], u.id != ^id)
+  end
+
+  defp compose_query({:friends, %User{id: id, following: following}}, query) do
+    where(query, [u], u.follower_address in ^following)
+    |> where([u], u.id != ^id)
+  end
+
+  defp compose_query({:recipients_from_activity, to}, query) do
+    where(query, [u], u.ap_id in ^to or fragment("? && ?", u.following, ^to))
+  end
+
+  defp compose_query(_unsupported_param, query), do: query
+
+  defp prepare_tag_criteria(tag, query) do
+    or_where(query, [u], fragment("? = any(?)", ^tag, u.tags))
+  end
+
+  defp location_query(query, local) do
+    where(query, [u], u.local == ^local)
+    |> where([u], not is_nil(u.nickname))
+  end
+end

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -5,7 +5,6 @@
 defmodule Pleroma.Web.ActivityPub.ActivityPub do
   alias Pleroma.Activity
   alias Pleroma.Conversation
-  alias Pleroma.Instances
   alias Pleroma.Notification
   alias Pleroma.Object
   alias Pleroma.Object.Fetcher
@@ -15,7 +14,6 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.MRF
   alias Pleroma.Web.ActivityPub.Transmogrifier
-  alias Pleroma.Web.Federator
   alias Pleroma.Web.WebFinger
 
   import Ecto.Query
@@ -24,8 +22,6 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
 
   require Logger
 
-  @httpoison Application.get_env(:pleroma, :httpoison)
-
   # For Announce activities, we filter the recipients based on following status for any actors
   # that match actual users.  See issue #164 for more information about why this is necessary.
   defp get_recipients(%{"type" => "Announce"} = data) do
@@ -137,9 +133,7 @@ def insert(map, local \\ true, fake \\ false) when is_map(map) do
           activity
         end
 
-      Task.start(fn ->
+      PleromaJobQueue.enqueue(:background, Pleroma.Web.RichMedia.Helpers, [:fetch, activity])
-        Pleroma.Web.RichMedia.Helpers.fetch_data_for_activity(activity)
-      end)
 
       Notification.create_notifications(activity)
 
@@ -848,6 +842,7 @@ def fetch_activities_query(recipients, opts \\ %{}) do
     |> restrict_reblogs(opts)
     |> restrict_pinned(opts)
     |> restrict_muted_reblogs(opts)
+    |> Activity.restrict_deactivated_users()
   end
 
   def fetch_activities(recipients, opts \\ %{}) do
@@ -951,89 +946,6 @@ def make_user_from_nickname(nickname) do
     end
   end
 
-  def should_federate?(inbox, public) do
-    if public do
-      true
-    else
-      inbox_info = URI.parse(inbox)
-      !Enum.member?(Pleroma.Config.get([:instance, :quarantined_instances], []), inbox_info.host)
-    end
-  end
-
-  def publish(actor, activity) do
-    remote_followers =
-      if actor.follower_address in activity.recipients do
-        {:ok, followers} = User.get_followers(actor)
-        followers |> Enum.filter(&(!&1.local))
-      else
-        []
-      end
-
-    public = is_public?(activity)
-
-    {:ok, data} = Transmogrifier.prepare_outgoing(activity.data)
-    json = Jason.encode!(data)
-
-    (Pleroma.Web.Salmon.remote_users(activity) ++ remote_followers)
-    |> Enum.filter(fn user -> User.ap_enabled?(user) end)
-    |> Enum.map(fn %{info: %{source_data: data}} ->
-      (is_map(data["endpoints"]) && Map.get(data["endpoints"], "sharedInbox")) || data["inbox"]
-    end)
-    |> Enum.uniq()
-    |> Enum.filter(fn inbox -> should_federate?(inbox, public) end)
-    |> Instances.filter_reachable()
-    |> Enum.each(fn {inbox, unreachable_since} ->
-      Federator.publish_single_ap(%{
-        inbox: inbox,
-        json: json,
-        actor: actor,
-        id: activity.data["id"],
-        unreachable_since: unreachable_since
-      })
-    end)
-  end
-
-  def publish_one(%{inbox: inbox, json: json, actor: actor, id: id} = params) do
-    Logger.info("Federating #{id} to #{inbox}")
-    host = URI.parse(inbox).host
-
-    digest = "SHA-256=" <> (:crypto.hash(:sha256, json) |> Base.encode64())
-
-    date =
-      NaiveDateTime.utc_now()
-      |> Timex.format!("{WDshort}, {0D} {Mshort} {YYYY} {h24}:{m}:{s} GMT")
-
-    signature =
-      Pleroma.Web.HTTPSignatures.sign(actor, %{
-        host: host,
-        "content-length": byte_size(json),
-        digest: digest,
-        date: date
-      })
-
-    with {:ok, %{status: code}} when code in 200..299 <-
-           result =
-             @httpoison.post(
-               inbox,
-               json,
-               [
-                 {"Content-Type", "application/activity+json"},
-                 {"Date", date},
-                 {"signature", signature},
-                 {"digest", digest}
-               ]
-             ) do
-      if !Map.has_key?(params, :unreachable_since) || params[:unreachable_since],
-        do: Instances.set_reachable(inbox)
-
-      result
-    else
-      {_post_result, response} ->
-        unless params[:unreachable_since], do: Instances.set_unreachable(inbox)
-        {:error, response}
-    end
-  end
-
   # filter out broken threads
   def contain_broken_threads(%Activity{} = activity, %User{} = user) do
     entire_thread_visible_for_user?(activity, user)

lib/pleroma/web/activity_pub/publisher.ex

@@ -0,0 +1,152 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.Publisher do
+  alias Pleroma.Activity
+  alias Pleroma.Config
+  alias Pleroma.Instances
+  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.Relay
+  alias Pleroma.Web.ActivityPub.Transmogrifier
+
+  import Pleroma.Web.ActivityPub.Visibility
+
+  @behaviour Pleroma.Web.Federator.Publisher
+
+  require Logger
+
+  @httpoison Application.get_env(:pleroma, :httpoison)
+
+  @moduledoc """
+  ActivityPub outgoing federation module.
+  """
+
+  @doc """
+  Determine if an activity can be represented by running it through Transmogrifier.
+  """
+  def is_representable?(%Activity{} = activity) do
+    with {:ok, _data} <- Transmogrifier.prepare_outgoing(activity.data) do
+      true
+    else
+      _e ->
+        false
+    end
+  end
+
+  @doc """
+  Publish a single message to a peer.  Takes a struct with the following
+  parameters set:
+
+  * `inbox`: the inbox to publish to
+  * `json`: the JSON message body representing the ActivityPub message
+  * `actor`: the actor which is signing the message
+  * `id`: the ActivityStreams URI of the message
+  """
+  def publish_one(%{inbox: inbox, json: json, actor: %User{} = actor, id: id} = params) do
+    Logger.info("Federating #{id} to #{inbox}")
+    host = URI.parse(inbox).host
+
+    digest = "SHA-256=" <> (:crypto.hash(:sha256, json) |> Base.encode64())
+
+    date =
+      NaiveDateTime.utc_now()
+      |> Timex.format!("{WDshort}, {0D} {Mshort} {YYYY} {h24}:{m}:{s} GMT")
+
+    signature =
+      Pleroma.Signature.sign(actor, %{
+        host: host,
+        "content-length": byte_size(json),
+        digest: digest,
+        date: date
+      })
+
+    with {:ok, %{status: code}} when code in 200..299 <-
+           result =
+             @httpoison.post(
+               inbox,
+               json,
+               [
+                 {"Content-Type", "application/activity+json"},
+                 {"Date", date},
+                 {"signature", signature},
+                 {"digest", digest}
+               ]
+             ) do
+      if !Map.has_key?(params, :unreachable_since) || params[:unreachable_since],
+        do: Instances.set_reachable(inbox)
+
+      result
+    else
+      {_post_result, response} ->
+        unless params[:unreachable_since], do: Instances.set_unreachable(inbox)
+        {:error, response}
+    end
+  end
+
+  defp should_federate?(inbox, public) do
+    if public do
+      true
+    else
+      inbox_info = URI.parse(inbox)
+      !Enum.member?(Pleroma.Config.get([:instance, :quarantined_instances], []), inbox_info.host)
+    end
+  end
+
+  @doc """
+  Publishes an activity to all relevant peers.
+  """
+  def publish(%User{} = actor, %Activity{} = activity) do
+    remote_followers =
+      if actor.follower_address in activity.recipients do
+        {:ok, followers} = User.get_followers(actor)
+        followers |> Enum.filter(&(!&1.local))
+      else
+        []
+      end
+
+    public = is_public?(activity)
+
+    if public && Config.get([:instance, :allow_relay]) do
+      Logger.info(fn -> "Relaying #{activity.data["id"]} out" end)
+      Relay.publish(activity)
+    end
+
+    {:ok, data} = Transmogrifier.prepare_outgoing(activity.data)
+    json = Jason.encode!(data)
+
+    (Pleroma.Web.Salmon.remote_users(activity) ++ remote_followers)
+    |> Enum.filter(fn user -> User.ap_enabled?(user) end)
+    |> Enum.map(fn %{info: %{source_data: data}} ->
+      (is_map(data["endpoints"]) && Map.get(data["endpoints"], "sharedInbox")) || data["inbox"]
+    end)
+    |> Enum.uniq()
+    |> Enum.filter(fn inbox -> should_federate?(inbox, public) end)
+    |> Instances.filter_reachable()
+    |> Enum.each(fn {inbox, unreachable_since} ->
+      Pleroma.Web.Federator.Publisher.enqueue_one(
+        __MODULE__,
+        %{
+          inbox: inbox,
+          json: json,
+          actor: actor,
+          id: activity.data["id"],
+          unreachable_since: unreachable_since
+        }
+      )
+    end)
+  end
+
+  def gather_webfinger_links(%User{} = user) do
+    [
+      %{"rel" => "self", "type" => "application/activity+json", "href" => user.ap_id},
+      %{
+        "rel" => "self",
+        "type" => "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"",
+        "href" => user.ap_id
+      }
+    ]
+  end
+
+  def gather_nodeinfo_protocol_names, do: ["activitypub"]
+end

lib/pleroma/web/activity_pub/utils.ex

@@ -682,7 +682,7 @@ def make_flag_data(params, additional) do
   """
   def fetch_ordered_collection(from, pages_left, acc \\ []) do
     with {:ok, response} <- Tesla.get(from),
-         {:ok, collection} <- Poison.decode(response.body) do
+         {:ok, collection} <- Jason.decode(response.body) do
       case collection["type"] do
         "OrderedCollection" ->
           # If we've encountered the OrderedCollection and not the page,

lib/pleroma/web/activity_pub/visibility.ex

@@ -59,4 +59,28 @@ def entire_thread_visible_for_user?(
         visible_for_user?(tail, user)
     end
   end
+
+  def get_visibility(object) do
+    public = "https://www.w3.org/ns/activitystreams#Public"
+    to = object.data["to"] || []
+    cc = object.data["cc"] || []
+
+    cond do
+      public in to ->
+        "public"
+
+      public in cc ->
+        "unlisted"
+
+      # this should use the sql for the object's activity
+      Enum.any?(to, &String.contains?(&1, "/followers")) ->
+        "private"
+
+      length(cc) > 0 ->
+        "private"
+
+      true ->
+        "direct"
+    end
+  end
 end

lib/pleroma/web/admin_api/admin_api_controller.ex

@@ -59,7 +59,7 @@ def user_create(
       bio: "."
     }
 
-    changeset = User.register_changeset(%User{}, user_data, confirmed: true)
+    changeset = User.register_changeset(%User{}, user_data, need_confirmation: false)
     {:ok, user} = User.register(changeset)
 
     conn
@@ -101,7 +101,10 @@ def list_users(conn, params) do
     search_params = %{
       query: params["query"],
       page: page,
-      page_size: page_size
+      page_size: page_size,
+      tags: params["tags"],
+      name: params["name"],
+      email: params["email"]
     }
 
     with {:ok, users, count} <- Search.user(Map.merge(search_params, filters)),
@@ -116,11 +119,11 @@ def list_users(conn, params) do
            )
   end
 
-  @filters ~w(local external active deactivated)
+  @filters ~w(local external active deactivated is_admin is_moderator)
-
-  defp maybe_parse_filters(filters) when is_nil(filters) or filters == "", do: %{}
 
   @spec maybe_parse_filters(String.t()) :: %{required(String.t()) => true} | %{}
+  defp maybe_parse_filters(filters) when is_nil(filters) or filters == "", do: %{}
+
   defp maybe_parse_filters(filters) do
     filters
     |> String.split(",")

lib/pleroma/web/admin_api/search.ex

@@ -10,45 +10,23 @@ defmodule Pleroma.Web.AdminAPI.Search do
 
   @page_size 50
 
-  def user(%{query: term} = params) when is_nil(term) or term == "" do
+  defmacro not_empty_string(string) do
-    query = maybe_filtered_query(params)
+    quote do
+      is_binary(unquote(string)) and unquote(string) != ""
+    end
+  end
+
+  @spec user(map()) :: {:ok, [User.t()], pos_integer()}
+  def user(params \\ %{}) do
+    query = User.Query.build(params) |> order_by([u], u.nickname)
 
     paginated_query =
-      maybe_filtered_query(params)
+      User.Query.paginate(query, params[:page] || 1, params[:page_size] || @page_size)
-      |> paginate(params[:page] || 1, params[:page_size] || @page_size)
 
-    count = query |> Repo.aggregate(:count, :id)
+    count = Repo.aggregate(query, :count, :id)
 
     results = Repo.all(paginated_query)
 
     {:ok, results, count}
   end
-
-  def user(%{query: term} = params) when is_binary(term) do
-    search_query = from(u in maybe_filtered_query(params), where: ilike(u.nickname, ^"%#{term}%"))
-
-    count = search_query |> Repo.aggregate(:count, :id)
-
-    results =
-      search_query
-      |> paginate(params[:page] || 1, params[:page_size] || @page_size)
-      |> Repo.all()
-
-    {:ok, results, count}
-  end
-
-  defp maybe_filtered_query(params) do
-    from(u in User, order_by: u.nickname)
-    |> User.maybe_local_user_query(params[:local])
-    |> User.maybe_external_user_query(params[:external])
-    |> User.maybe_active_user_query(params[:active])
-    |> User.maybe_deactivated_user_query(params[:deactivated])
-  end
-
-  defp paginate(query, page, page_size) do
-    from(u in query,
-      limit: ^page_size,
-      offset: ^((page - 1) * page_size)
-    )
-  end
 end

lib/pleroma/web/auth/pleroma_authenticator.ex

@@ -74,7 +74,7 @@ def create_from_registration(
                password_confirmation: random_password
              },
              external: true,
-             confirmed: true
+             need_confirmation: false
            )
            |> Repo.insert(),
          {:ok, _} <-

lib/pleroma/web/common_api/common_api.ex

@@ -116,32 +116,34 @@ def unfavorite(id_or_ap_id, user) do
     end
   end
 
-  def get_visibility(%{"visibility" => visibility})
+  def get_visibility(%{"visibility" => visibility}, in_reply_to)
       when visibility in ~w{public unlisted private direct},
-      do: visibility
+      do: {visibility, get_replied_to_visibility(in_reply_to)}
 
-  def get_visibility(%{"in_reply_to_status_id" => status_id}) when not is_nil(status_id) do
+  def get_visibility(_, in_reply_to) when not is_nil(in_reply_to) do
-    case get_replied_to_activity(status_id) do
+    visibility = get_replied_to_visibility(in_reply_to)
-      nil ->
+    {visibility, visibility}
-        "public"
-
-      in_reply_to ->
-        # XXX: these heuristics should be moved out of MastodonAPI.
-        with %Object{} = object <- Object.normalize(in_reply_to) do
-          Pleroma.Web.MastodonAPI.StatusView.get_visibility(object)
-        end
-    end
   end
 
-  def get_visibility(_), do: "public"
+  def get_visibility(_, in_reply_to), do: {"public", get_replied_to_visibility(in_reply_to)}
+
+  def get_replied_to_visibility(nil), do: nil
+
+  def get_replied_to_visibility(activity) do
+    with %Object{} = object <- Object.normalize(activity) do
+      Pleroma.Web.ActivityPub.Visibility.get_visibility(object)
+    end
+  end
 
   def post(user, %{"status" => status} = data) do
-    visibility = get_visibility(data)
     limit = Pleroma.Config.get([:instance, :limit])
 
     with status <- String.trim(status),
          attachments <- attachments_from_ids(data),
          in_reply_to <- get_replied_to_activity(data["in_reply_to_status_id"]),
+         {visibility, in_reply_to_visibility} <- get_visibility(data, in_reply_to),
+         {_, false} <-
+           {:private_to_public, in_reply_to_visibility == "direct" && visibility != "direct"},
          {content_html, mentions, tags} <-
            make_content_html(
              status,
@@ -185,6 +187,8 @@ def post(user, %{"status" => status} = data) do
         )
 
       res
+    else
+      e -> {:error, e}
     end
   end
 

lib/pleroma/web/endpoint.ex

@@ -29,6 +29,13 @@ defmodule Pleroma.Web.Endpoint do
     # credo:disable-for-previous-line Credo.Check.Readability.MaxLineLength
   )
 
+  plug(Plug.Static.IndexHtml, at: "/pleroma/admin/")
+
+  plug(Plug.Static,
+    at: "/pleroma/admin/",
+    from: {:pleroma, "priv/static/adminfe/"}
+  )
+
   # Code reloading can be explicitly enabled under the
   # :code_reloader configuration of your endpoint.
   if code_reloading? do

lib/pleroma/web/federator/federator.ex

@@ -7,13 +7,10 @@ defmodule Pleroma.Web.Federator do
   alias Pleroma.Object.Containment
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
-  alias Pleroma.Web.ActivityPub.Relay
   alias Pleroma.Web.ActivityPub.Transmogrifier
   alias Pleroma.Web.ActivityPub.Utils
-  alias Pleroma.Web.ActivityPub.Visibility
+  alias Pleroma.Web.Federator.Publisher
   alias Pleroma.Web.Federator.RetryQueue
-  alias Pleroma.Web.OStatus
-  alias Pleroma.Web.Salmon
   alias Pleroma.Web.WebFinger
   alias Pleroma.Web.Websub
 
@@ -42,14 +39,6 @@ def publish(activity, priority \\ 1) do
     PleromaJobQueue.enqueue(:federator_outgoing, __MODULE__, [:publish, activity], priority)
   end
 
-  def publish_single_ap(params) do
-    PleromaJobQueue.enqueue(:federator_outgoing, __MODULE__, [:publish_single_ap, params])
-  end
-
-  def publish_single_websub(websub) do
-    PleromaJobQueue.enqueue(:federator_outgoing, __MODULE__, [:publish_single_websub, websub])
-  end
-
   def verify_websub(websub) do
     PleromaJobQueue.enqueue(:federator_outgoing, __MODULE__, [:verify_websub, websub])
   end
@@ -62,10 +51,6 @@ def refresh_subscriptions do
     PleromaJobQueue.enqueue(:federator_outgoing, __MODULE__, [:refresh_subscriptions])
   end
 
-  def publish_single_salmon(params) do
-    PleromaJobQueue.enqueue(:federator_outgoing, __MODULE__, [:publish_single_salmon, params])
-  end
-
   # Job Worker Callbacks
 
   def perform(:refresh_subscriptions) do
@@ -95,23 +80,7 @@ def perform(:publish, activity) do
     with actor when not is_nil(actor) <- User.get_cached_by_ap_id(activity.data["actor"]) do
       {:ok, actor} = WebFinger.ensure_keys_present(actor)
 
-      if Visibility.is_public?(activity) do
+      Publisher.publish(actor, activity)
-        if OStatus.is_representable?(activity) do
-          Logger.info(fn -> "Sending #{activity.data["id"]} out via WebSub" end)
-          Websub.publish(Pleroma.Web.OStatus.feed_path(actor), actor, activity)
-
-          Logger.info(fn -> "Sending #{activity.data["id"]} out via Salmon" end)
-          Pleroma.Web.Salmon.publish(actor, activity)
-        end
-
-        if Keyword.get(Application.get_env(:pleroma, :instance), :allow_relay) do
-          Logger.info(fn -> "Relaying #{activity.data["id"]} out" end)
-          Relay.publish(activity)
-        end
-      end
-
-      Logger.info(fn -> "Sending #{activity.data["id"]} out via AP" end)
-      Pleroma.Web.ActivityPub.ActivityPub.publish(actor, activity)
     end
   end
 
@@ -148,25 +117,11 @@ def perform(:incoming_ap_doc, params) do
       _e ->
         # Just drop those for now
         Logger.info("Unhandled activity")
-        Logger.info(Poison.encode!(params, pretty: 2))
+        Logger.info(Jason.encode!(params, pretty: true))
         :error
     end
   end
 
-  def perform(:publish_single_salmon, params) do
-    Salmon.send_to_user(params)
-  end
-
-  def perform(:publish_single_ap, params) do
-    case ActivityPub.publish_one(params) do
-      {:ok, _} ->
-        :ok
-
-      {:error, _} ->
-        RetryQueue.enqueue(params, ActivityPub)
-    end
-  end
-
   def perform(
         :publish_single_websub,
         %{xml: _xml, topic: _topic, callback: _callback, secret: _secret} = params

lib/pleroma/web/federator/publisher.ex

@@ -0,0 +1,95 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Federator.Publisher do
+  alias Pleroma.Activity
+  alias Pleroma.Config
+  alias Pleroma.User
+  alias Pleroma.Web.Federator.RetryQueue
+
+  require Logger
+
+  @moduledoc """
+  Defines the contract used by federation implementations to publish messages to
+  their peers.
+  """
+
+  @doc """
+  Determine whether an activity can be relayed using the federation module.
+  """
+  @callback is_representable?(Pleroma.Activity.t()) :: boolean()
+
+  @doc """
+  Relays an activity to a specified peer, determined by the parameters.  The
+  parameters used are controlled by the federation module.
+  """
+  @callback publish_one(Map.t()) :: {:ok, Map.t()} | {:error, any()}
+
+  @doc """
+  Enqueue publishing a single activity.
+  """
+  @spec enqueue_one(module(), Map.t()) :: :ok
+  def enqueue_one(module, %{} = params),
+    do: PleromaJobQueue.enqueue(:federation_outgoing, __MODULE__, [:publish_one, module, params])
+
+  @spec perform(atom(), module(), any()) :: {:ok, any()} | {:error, any()}
+  def perform(:publish_one, module, params) do
+    case apply(module, :publish_one, [params]) do
+      {:ok, _} ->
+        :ok
+
+      {:error, _e} ->
+        RetryQueue.enqueue(params, module)
+    end
+  end
+
+  def perform(type, _, _) do
+    Logger.debug("Unknown task: #{type}")
+    {:error, "Don't know what to do with this"}
+  end
+
+  @doc """
+  Relays an activity to all specified peers.
+  """
+  @callback publish(Pleroma.User.t(), Pleroma.Activity.t()) :: :ok | {:error, any()}
+
+  @spec publish(Pleroma.User.t(), Pleroma.Activity.t()) :: :ok
+  def publish(%User{} = user, %Activity{} = activity) do
+    Config.get([:instance, :federation_publisher_modules])
+    |> Enum.each(fn module ->
+      if module.is_representable?(activity) do
+        Logger.info("Publishing #{activity.data["id"]} using #{inspect(module)}")
+        module.publish(user, activity)
+      end
+    end)
+
+    :ok
+  end
+
+  @doc """
+  Gathers links used by an outgoing federation module for WebFinger output.
+  """
+  @callback gather_webfinger_links(Pleroma.User.t()) :: list()
+
+  @spec gather_webfinger_links(Pleroma.User.t()) :: list()
+  def gather_webfinger_links(%User{} = user) do
+    Config.get([:instance, :federation_publisher_modules])
+    |> Enum.reduce([], fn module, links ->
+      links ++ module.gather_webfinger_links(user)
+    end)
+  end
+
+  @doc """
+  Gathers nodeinfo protocol names supported by the federation module.
+  """
+  @callback gather_nodeinfo_protocol_names() :: list()
+
+  @spec gather_nodeinfo_protocol_names() :: list()
+  def gather_nodeinfo_protocol_names do
+    Config.get([:instance, :federation_publisher_modules])
+    |> Enum.reduce([], fn module, links ->
+      links ++ module.gather_nodeinfo_protocol_names()
+    end)
+  end
+end

lib/pleroma/web/http_signatures/http_signatures.ex

@@ -1,91 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-# https://tools.ietf.org/html/draft-cavage-http-signatures-08
-defmodule Pleroma.Web.HTTPSignatures do
-  alias Pleroma.User
-  alias Pleroma.Web.ActivityPub.ActivityPub
-  alias Pleroma.Web.ActivityPub.Utils
-
-  require Logger
-
-  def split_signature(sig) do
-    default = %{"headers" => "date"}
-
-    sig =
-      sig
-      |> String.trim()
-      |> String.split(",")
-      |> Enum.reduce(default, fn part, acc ->
-        [key | rest] = String.split(part, "=")
-        value = Enum.join(rest, "=")
-        Map.put(acc, key, String.trim(value, "\""))
-      end)
-
-    Map.put(sig, "headers", String.split(sig["headers"], ~r/\s/))
-  end
-
-  def validate(headers, signature, public_key) do
-    sigstring = build_signing_string(headers, signature["headers"])
-    Logger.debug("Signature: #{signature["signature"]}")
-    Logger.debug("Sigstring: #{sigstring}")
-    {:ok, sig} = Base.decode64(signature["signature"])
-    :public_key.verify(sigstring, :sha256, sig, public_key)
-  end
-
-  def validate_conn(conn) do
-    # TODO: How to get the right key and see if it is actually valid for that request.
-    # For now, fetch the key for the actor.
-    with actor_id <- Utils.get_ap_id(conn.params["actor"]),
-         {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
-      if validate_conn(conn, public_key) do
-        true
-      else
-        Logger.debug("Could not validate, re-fetching user and trying one more time")
-        # Fetch user anew and try one more time
-        with actor_id <- Utils.get_ap_id(conn.params["actor"]),
-             {:ok, _user} <- ActivityPub.make_user_from_ap_id(actor_id),
-             {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
-          validate_conn(conn, public_key)
-        end
-      end
-    else
-      _e ->
-        Logger.debug("Could not public key!")
-        false
-    end
-  end
-
-  def validate_conn(conn, public_key) do
-    headers = Enum.into(conn.req_headers, %{})
-    signature = split_signature(headers["signature"])
-    validate(headers, signature, public_key)
-  end
-
-  def build_signing_string(headers, used_headers) do
-    used_headers
-    |> Enum.map(fn header -> "#{header}: #{headers[header]}" end)
-    |> Enum.join("\n")
-  end
-
-  def sign(user, headers) do
-    with {:ok, %{info: %{keys: keys}}} <- Pleroma.Web.WebFinger.ensure_keys_present(user),
-         {:ok, private_key, _} = Pleroma.Web.Salmon.keys_from_pem(keys) do
-      sigstring = build_signing_string(headers, Map.keys(headers))
-
-      signature =
-        :public_key.sign(sigstring, :sha256, private_key)
-        |> Base.encode64()
-
-      [
-        keyId: user.ap_id <> "#main-key",
-        algorithm: "rsa-sha256",
-        headers: Map.keys(headers) |> Enum.join(" "),
-        signature: signature
-      ]
-      |> Enum.map(fn {k, v} -> "#{k}=\"#{v}\"" end)
-      |> Enum.join(",")
-    end
-  end
-end

lib/pleroma/web/mastodon_api/mastodon_api_controller.ex

@@ -39,12 +39,22 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
   alias Pleroma.Web.OAuth.Authorization
   alias Pleroma.Web.OAuth.Scopes
   alias Pleroma.Web.OAuth.Token
+  alias Pleroma.Web.TwitterAPI.TwitterAPI
 
   alias Pleroma.Web.ControllerHelper
   import Ecto.Query
 
   require Logger
 
+  plug(
+    Pleroma.Plugs.RateLimitPlug,
+    %{
+      max_requests: Config.get([:app_account_creation, :max_requests]),
+      interval: Config.get([:app_account_creation, :interval])
+    }
+    when action in [:account_register]
+  )
+
   @httpoison Application.get_env(:pleroma, :httpoison)
   @local_mastodon_name "Mastodon-Local"
 
@@ -168,7 +178,7 @@ def user(%{assigns: %{user: for_user}} = conn, %{"id" => nickname_or_id}) do
     end
   end
 
-  @mastodon_api_level "2.6.5"
+  @mastodon_api_level "2.7.2"
 
   def masto_instance(conn, _params) do
     instance = Config.get(:instance)
@@ -1536,7 +1546,7 @@ def create_filter(
       user_id: user.id,
       phrase: phrase,
       context: context,
-      hide: Map.get(params, "irreversible", nil),
+      hide: Map.get(params, "irreversible", false),
       whole_word: Map.get(params, "boolean", true)
       # expires_at
     }
@@ -1693,6 +1703,53 @@ def reports(%{assigns: %{user: user}} = conn, params) do
     end
   end
 
+  def account_register(
+        %{assigns: %{app: app}} = conn,
+        %{"username" => nickname, "email" => _, "password" => _, "agreement" => true} = params
+      ) do
+    params =
+      params
+      |> Map.take([
+        "email",
+        "captcha_solution",
+        "captcha_token",
+        "captcha_answer_data",
+        "token",
+        "password"
+      ])
+      |> Map.put("nickname", nickname)
+      |> Map.put("fullname", params["fullname"] || nickname)
+      |> Map.put("bio", params["bio"] || "")
+      |> Map.put("confirm", params["password"])
+
+    with {:ok, user} <- TwitterAPI.register_user(params, need_confirmation: true),
+         {:ok, token} <- Token.create_token(app, user, %{scopes: app.scopes}) do
+      json(conn, %{
+        token_type: "Bearer",
+        access_token: token.token,
+        scope: app.scopes,
+        created_at: Token.Utils.format_created_at(token)
+      })
+    else
+      {:error, errors} ->
+        conn
+        |> put_status(400)
+        |> json(Jason.encode!(errors))
+    end
+  end
+
+  def account_register(%{assigns: %{app: _app}} = conn, _params) do
+    conn
+    |> put_status(400)
+    |> json(%{error: "Missing parameters"})
+  end
+
+  def account_register(conn, _) do
+    conn
+    |> put_status(403)
+    |> json(%{error: "Invalid credentials"})
+  end
+
   def conversations(%{assigns: %{user: user}} = conn, params) do
     participations = Participation.for_user_with_last_activity_id(user, params)
 

lib/pleroma/web/mastodon_api/views/status_view.ex

@@ -16,6 +16,8 @@ defmodule Pleroma.Web.MastodonAPI.StatusView do
   alias Pleroma.Web.MastodonAPI.StatusView
   alias Pleroma.Web.MediaProxy
 
+  import Pleroma.Web.ActivityPub.Visibility, only: [get_visibility: 1]
+
   # TODO: Add cached version.
   defp get_replied_to_activities(activities) do
     activities
@@ -340,30 +342,6 @@ def get_reply_to(%{data: %{"object" => _object}} = activity, _) do
     end
   end
 
-  def get_visibility(object) do
-    public = "https://www.w3.org/ns/activitystreams#Public"
-    to = object.data["to"] || []
-    cc = object.data["cc"] || []
-
-    cond do
-      public in to ->
-        "public"
-
-      public in cc ->
-        "unlisted"
-
-      # this should use the sql for the object's activity
-      Enum.any?(to, &String.contains?(&1, "/followers")) ->
-        "private"
-
-      length(cc) > 0 ->
-        "private"
-
-      true ->
-        "direct"
-    end
-  end
-
   def render_content(%{data: %{"type" => "Video"}} = object) do
     with name when not is_nil(name) and name != "" <- object.data["name"] do
       "<p><a href=\"#{object.data["id"]}\">#{name}</a></p>#{object.data["content"]}"

lib/pleroma/web/nodeinfo/nodeinfo_controller.ex

@@ -10,6 +10,7 @@ defmodule Pleroma.Web.Nodeinfo.NodeinfoController do
   alias Pleroma.User
   alias Pleroma.Web
   alias Pleroma.Web.ActivityPub.MRF
+  alias Pleroma.Web.Federator.Publisher
 
   plug(Pleroma.Web.FederatingPlug)
 
@@ -137,7 +138,7 @@ def raw_nodeinfo do
         name: Pleroma.Application.name() |> String.downcase(),
         version: Pleroma.Application.version()
       },
-      protocols: ["ostatus", "activitypub"],
+      protocols: Publisher.gather_nodeinfo_protocol_names(),
       services: %{
         inbound: [],
         outbound: []

lib/pleroma/web/oauth/app.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.OAuth.App do
   import Ecto.Changeset
 
   @type t :: %__MODULE__{}
+
   schema "apps" do
     field(:client_name, :string)
     field(:redirect_uris, :string)

lib/pleroma/web/oauth/authorization.ex

@@ -14,6 +14,7 @@ defmodule Pleroma.Web.OAuth.Authorization do
   import Ecto.Query
 
   @type t :: %__MODULE__{}
+
   schema "oauth_authorizations" do
     field(:token, :string)
     field(:scopes, {:array, :string}, default: [])
@@ -25,28 +26,45 @@ defmodule Pleroma.Web.OAuth.Authorization do
     timestamps()
   end
 
+  @spec create_authorization(App.t(), User.t() | %{}, [String.t()] | nil) ::
+          {:ok, Authorization.t()} | {:error, Changeset.t()}
   def create_authorization(%App{} = app, %User{} = user, scopes \\ nil) do
-    scopes = scopes || app.scopes
+    %{
-    token = :crypto.strong_rand_bytes(32) |> Base.url_encode64(padding: false)
+      scopes: scopes || app.scopes,
-
-    authorization = %Authorization{
-      token: token,
-      used: false,
       user_id: user.id,
-      app_id: app.id,
+      app_id: app.id
-      scopes: scopes,
-      valid_until: NaiveDateTime.add(NaiveDateTime.utc_now(), 60 * 10)
     }
-
+    |> create_changeset()
-    Repo.insert(authorization)
+    |> Repo.insert()
   end
 
+  @spec create_changeset(map()) :: Changeset.t()
+  def create_changeset(attrs \\ %{}) do
+    %Authorization{}
+    |> cast(attrs, [:user_id, :app_id, :scopes, :valid_until])
+    |> validate_required([:app_id, :scopes])
+    |> add_token()
+    |> add_lifetime()
+  end
+
+  defp add_token(changeset) do
+    token = :crypto.strong_rand_bytes(32) |> Base.url_encode64(padding: false)
+    put_change(changeset, :token, token)
+  end
+
+  defp add_lifetime(changeset) do
+    put_change(changeset, :valid_until, NaiveDateTime.add(NaiveDateTime.utc_now(), 60 * 10))
+  end
+
+  @spec use_changeset(Authtorizatiton.t(), map()) :: Changeset.t()
   def use_changeset(%Authorization{} = auth, params) do
     auth
     |> cast(params, [:used])
     |> validate_required([:used])
   end
 
+  @spec use_token(Authorization.t()) ::
+          {:ok, Authorization.t()} | {:error, Changeset.t()} | {:error, String.t()}
   def use_token(%Authorization{used: false, valid_until: valid_until} = auth) do
     if NaiveDateTime.diff(NaiveDateTime.utc_now(), valid_until) < 0 do
       Repo.update(use_changeset(auth, %{used: true}))
@@ -57,6 +75,7 @@ def use_token(%Authorization{used: false, valid_until: valid_until} = auth) do
 
   def use_token(%Authorization{used: true}), do: {:error, "already used"}
 
+  @spec delete_user_authorizations(User.t()) :: {integer(), any()}
   def delete_user_authorizations(%User{id: user_id}) do
     from(
       a in Pleroma.Web.OAuth.Authorization,

lib/pleroma/web/oauth/oauth_controller.ex

@@ -19,8 +19,6 @@ defmodule Pleroma.Web.OAuth.OAuthController do
 
   if Pleroma.Config.oauth_consumer_enabled?(), do: plug(Ueberauth)
 
-  @expires_in Pleroma.Config.get([:oauth2, :token_expires_in], 600)
-
   plug(:fetch_session)
   plug(:fetch_flash)
 
@@ -144,14 +142,14 @@ defp handle_create_authorization_error(conn, error, %{"authorization" => _}) do
   @doc "Renew access_token with refresh_token"
   def token_exchange(
         conn,
-        %{"grant_type" => "refresh_token", "refresh_token" => token} = params
+        %{"grant_type" => "refresh_token", "refresh_token" => token} = _params
       ) do
-    with %App{} = app <- get_app_from_request(conn, params),
+    with {:ok, app} <- Token.Utils.fetch_app(conn),
          {:ok, %{user: user} = token} <- Token.get_by_refresh_token(app, token),
          {:ok, token} <- RefreshToken.grant(token) do
       response_attrs = %{created_at: Token.Utils.format_created_at(token)}
 
-      json(conn, response_token(user, token, response_attrs))
+      json(conn, Token.Response.build(user, token, response_attrs))
     else
       _error ->
         put_status(conn, 400)
@@ -160,14 +158,14 @@ def token_exchange(
   end
 
   def token_exchange(conn, %{"grant_type" => "authorization_code"} = params) do
-    with %App{} = app <- get_app_from_request(conn, params),
+    with {:ok, app} <- Token.Utils.fetch_app(conn),
          fixed_token = Token.Utils.fix_padding(params["code"]),
          {:ok, auth} <- Authorization.get_by_token(app, fixed_token),
          %User{} = user <- User.get_cached_by_id(auth.user_id),
          {:ok, token} <- Token.exchange_token(app, auth) do
       response_attrs = %{created_at: Token.Utils.format_created_at(token)}
 
-      json(conn, response_token(user, token, response_attrs))
+      json(conn, Token.Response.build(user, token, response_attrs))
     else
       _error ->
         put_status(conn, 400)
@@ -179,14 +177,14 @@ def token_exchange(
         conn,
         %{"grant_type" => "password"} = params
       ) do
-    with {_, {:ok, %User{} = user}} <- {:get_user, Authenticator.get_user(conn)},
+    with {:ok, %User{} = user} <- Authenticator.get_user(conn),
-         %App{} = app <- get_app_from_request(conn, params),
+         {:ok, app} <- Token.Utils.fetch_app(conn),
          {:auth_active, true} <- {:auth_active, User.auth_active?(user)},
          {:user_active, true} <- {:user_active, !user.info.deactivated},
          {:ok, scopes} <- validate_scopes(app, params),
          {:ok, auth} <- Authorization.create_authorization(app, user, scopes),
          {:ok, token} <- Token.exchange_token(app, auth) do
-      json(conn, response_token(user, token))
+      json(conn, Token.Response.build(user, token))
     else
       {:auth_active, false} ->
         # Per https://github.com/tootsuite/mastodon/blob/
@@ -218,11 +216,23 @@ def token_exchange(
     token_exchange(conn, params)
   end
 
+  def token_exchange(conn, %{"grant_type" => "client_credentials"} = _params) do
+    with {:ok, app} <- Token.Utils.fetch_app(conn),
+         {:ok, auth} <- Authorization.create_authorization(app, %User{}),
+         {:ok, token} <- Token.exchange_token(app, auth) do
+      json(conn, Token.Response.build_for_client_credentials(token))
+    else
+      _error ->
+        put_status(conn, 400)
+        |> json(%{error: "Invalid credentials"})
+    end
+  end
+
   # Bad request
   def token_exchange(conn, params), do: bad_request(conn, params)
 
   def token_revoke(conn, %{"token" => _token} = params) do
-    with %App{} = app <- get_app_from_request(conn, params),
+    with {:ok, app} <- Token.Utils.fetch_app(conn),
          {:ok, _token} <- RevokeToken.revoke(app, params) do
       json(conn, %{})
     else
@@ -252,7 +262,7 @@ def prepare_request(conn, %{"provider" => provider, "authorization" => auth_attr
       auth_attrs
       |> Map.delete("scopes")
       |> Map.put("scope", scope)
-      |> Poison.encode!()
+      |> Jason.encode!()
 
     params =
       auth_attrs
@@ -316,7 +326,7 @@ def callback(conn, params) do
   end
 
   defp callback_params(%{"state" => state} = params) do
-    Map.merge(params, Poison.decode!(state))
+    Map.merge(params, Jason.decode!(state))
   end
 
   def registration_details(conn, %{"authorization" => auth_attrs}) do
@@ -405,33 +415,6 @@ defp do_create_authorization(
     end
   end
 
-  defp get_app_from_request(conn, params) do
-    conn
-    |> fetch_client_credentials(params)
-    |> fetch_client
-  end
-
-  defp fetch_client({id, secret}) when is_binary(id) and is_binary(secret) do
-    Repo.get_by(App, client_id: id, client_secret: secret)
-  end
-
-  defp fetch_client({_id, _secret}), do: nil
-
-  defp fetch_client_credentials(conn, params) do
-    # Per RFC 6749, HTTP Basic is preferred to body params
-    with ["Basic " <> encoded] <- get_req_header(conn, "authorization"),
-         {:ok, decoded} <- Base.decode64(encoded),
-         [id, secret] <-
-           Enum.map(
-             String.split(decoded, ":"),
-             fn s -> URI.decode_www_form(s) end
-           ) do
-      {id, secret}
-    else
-      _ -> {params["client_id"], params["client_secret"]}
-    end
-  end
-
   # Special case: Local MastodonFE
   defp redirect_uri(conn, "."), do: mastodon_api_url(conn, :login)
 
@@ -442,18 +425,6 @@ defp get_session_registration_id(conn), do: get_session(conn, :registration_id)
   defp put_session_registration_id(conn, registration_id),
     do: put_session(conn, :registration_id, registration_id)
 
-  defp response_token(%User{} = user, token, opts \\ %{}) do
-    %{
-      token_type: "Bearer",
-      access_token: token.token,
-      refresh_token: token.refresh_token,
-      expires_in: @expires_in,
-      scope: Enum.join(token.scopes, " "),
-      me: user.ap_id
-    }
-    |> Map.merge(opts)
-  end
-
   @spec validate_scopes(App.t(), map()) ::
           {:ok, list()} | {:error, :missing_scopes | :unsupported_scopes}
   defp validate_scopes(app, params) do

lib/pleroma/web/oauth/token.ex

@@ -45,12 +45,16 @@ def get_by_refresh_token(%App{id: app_id} = _app, token) do
     |> Repo.find_resource()
   end
 
+  @spec exchange_token(App.t(), Authorization.t()) ::
+          {:ok, Token.t()} | {:error, Changeset.t()}
   def exchange_token(app, auth) do
     with {:ok, auth} <- Authorization.use_token(auth),
          true <- auth.app_id == app.id do
+      user = if auth.user_id, do: User.get_cached_by_id(auth.user_id), else: %User{}
+
       create_token(
         app,
-        User.get_cached_by_id(auth.user_id),
+        user,
         %{scopes: auth.scopes}
       )
     end
@@ -81,12 +85,13 @@ defp put_valid_until(changeset, attrs) do
     |> validate_required([:valid_until])
   end
 
+  @spec create_token(App.t(), User.t(), map()) :: {:ok, Token} | {:error, Changeset.t()}
   def create_token(%App{} = app, %User{} = user, attrs \\ %{}) do
     %__MODULE__{user_id: user.id, app_id: app.id}
     |> cast(%{scopes: attrs[:scopes] || app.scopes}, [:scopes])
-    |> validate_required([:scopes, :user_id, :app_id])
+    |> validate_required([:scopes, :app_id])
     |> put_valid_until(attrs)
-    |> put_token
+    |> put_token()
     |> put_refresh_token(attrs)
     |> Repo.insert()
   end

lib/pleroma/web/oauth/token/response.ex

@@ -0,0 +1,32 @@
+defmodule Pleroma.Web.OAuth.Token.Response do
+  @moduledoc false
+
+  alias Pleroma.User
+  alias Pleroma.Web.OAuth.Token.Utils
+
+  @expires_in Pleroma.Config.get([:oauth2, :token_expires_in], 600)
+
+  @doc false
+  def build(%User{} = user, token, opts \\ %{}) do
+    %{
+      token_type: "Bearer",
+      access_token: token.token,
+      refresh_token: token.refresh_token,
+      expires_in: @expires_in,
+      scope: Enum.join(token.scopes, " "),
+      me: user.ap_id
+    }
+    |> Map.merge(opts)
+  end
+
+  def build_for_client_credentials(token) do
+    %{
+      token_type: "Bearer",
+      access_token: token.token,
+      refresh_token: token.refresh_token,
+      created_at: Utils.format_created_at(token),
+      expires_in: @expires_in,
+      scope: Enum.join(token.scopes, " ")
+    }
+  end
+end

lib/pleroma/web/oauth/token/utils.ex

@@ -3,6 +3,44 @@ defmodule Pleroma.Web.OAuth.Token.Utils do
   Auxiliary functions for dealing with tokens.
   """
 
+  alias Pleroma.Repo
+  alias Pleroma.Web.OAuth.App
+
+  @doc "Fetch app by client credentials from request"
+  @spec fetch_app(Plug.Conn.t()) :: {:ok, App.t()} | {:error, :not_found}
+  def fetch_app(conn) do
+    res =
+      conn
+      |> fetch_client_credentials()
+      |> fetch_client
+
+    case res do
+      %App{} = app -> {:ok, app}
+      _ -> {:error, :not_found}
+    end
+  end
+
+  defp fetch_client({id, secret}) when is_binary(id) and is_binary(secret) do
+    Repo.get_by(App, client_id: id, client_secret: secret)
+  end
+
+  defp fetch_client({_id, _secret}), do: nil
+
+  defp fetch_client_credentials(conn) do
+    # Per RFC 6749, HTTP Basic is preferred to body params
+    with ["Basic " <> encoded] <- Plug.Conn.get_req_header(conn, "authorization"),
+         {:ok, decoded} <- Base.decode64(encoded),
+         [id, secret] <-
+           Enum.map(
+             String.split(decoded, ":"),
+             fn s -> URI.decode_www_form(s) end
+           ) do
+      {id, secret}
+    else
+      _ -> {conn.params["client_id"], conn.params["client_secret"]}
+    end
+  end
+
   @doc "convert token inserted_at to unix timestamp"
   def format_created_at(%{inserted_at: inserted_at} = _token) do
     inserted_at

lib/pleroma/web/ostatus/activity_representer.ex

@@ -18,14 +18,17 @@ defp get_href(id) do
     end
   end
 
-  defp get_in_reply_to(%{"object" => %{"inReplyTo" => in_reply_to}}) do
+  defp get_in_reply_to(activity) do
+    with %Object{data: %{"inReplyTo" => in_reply_to}} <- Object.normalize(activity) do
       [
         {:"thr:in-reply-to",
          [ref: to_charlist(in_reply_to), href: to_charlist(get_href(in_reply_to))], []}
       ]
+    else
+      _ ->
+        []
+    end
   end
-
-  defp get_in_reply_to(_), do: []
 
   defp get_mentions(to) do
     Enum.map(to, fn id ->
@@ -98,7 +101,7 @@ def to_simple_form(%{data: %{"type" => "Create"}} = activity, user, with_author)
          []}
       end)
 
-    in_reply_to = get_in_reply_to(activity.data)
+    in_reply_to = get_in_reply_to(activity)
     author = if with_author, do: [{:author, UserRepresenter.to_simple_form(user)}], else: []
     mentions = activity.recipients |> get_mentions
 
@@ -146,7 +149,6 @@ def to_simple_form(%{data: %{"type" => "Like"}} = activity, user, with_author) d
     updated_at = activity.data["published"]
     inserted_at = activity.data["published"]
 
-    _in_reply_to = get_in_reply_to(activity.data)
     author = if with_author, do: [{:author, UserRepresenter.to_simple_form(user)}], else: []
     mentions = activity.recipients |> get_mentions
 
@@ -177,7 +179,6 @@ def to_simple_form(%{data: %{"type" => "Announce"}} = activity, user, with_autho
     updated_at = activity.data["published"]
     inserted_at = activity.data["published"]
 
-    _in_reply_to = get_in_reply_to(activity.data)
     author = if with_author, do: [{:author, UserRepresenter.to_simple_form(user)}], else: []
 
     retweeted_activity = Activity.get_create_by_object_ap_id(activity.data["object"])

lib/pleroma/web/ostatus/ostatus.ex

@@ -16,6 +16,7 @@ defmodule Pleroma.Web.OStatus do
   alias Pleroma.Web
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.Transmogrifier
+  alias Pleroma.Web.ActivityPub.Visibility
   alias Pleroma.Web.OStatus.DeleteHandler
   alias Pleroma.Web.OStatus.FollowHandler
   alias Pleroma.Web.OStatus.NoteHandler
@@ -30,7 +31,7 @@ def is_representable?(%Activity{} = activity) do
       is_nil(object) ->
         false
 
-      object.data["type"] == "Note" ->
+      Visibility.is_public?(activity) && object.data["type"] == "Note" ->
         true
 
       true ->

lib/pleroma/web/rich_media/helpers.ex

@@ -34,4 +34,6 @@ def fetch_data_for_activity(%Activity{data: %{"type" => "Create"}} = activity) d
   end
 
   def fetch_data_for_activity(_), do: %{}
+
+  def perform(:fetch, %Activity{} = activity), do: fetch_data_for_activity(activity)
 end

lib/pleroma/web/router.ex

@@ -84,11 +84,13 @@ defmodule Pleroma.Web.Router do
     plug(Pleroma.Plugs.EnsureUserKeyPlug)
   end
 
-  pipeline :oauth_read_or_unauthenticated do
+  pipeline :oauth_read_or_public do
     plug(Pleroma.Plugs.OAuthScopesPlug, %{
       scopes: ["read"],
       fallback: :proceed_unauthenticated
     })
+
+    plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
   end
 
   pipeline :oauth_read do
@@ -146,34 +148,52 @@ defmodule Pleroma.Web.Router do
   scope "/api/pleroma/admin", Pleroma.Web.AdminAPI do
     pipe_through([:admin_api, :oauth_write])
 
-    post("/user/follow", AdminAPIController, :user_follow)
+    post("/users/follow", AdminAPIController, :user_follow)
-    post("/user/unfollow", AdminAPIController, :user_unfollow)
+    post("/users/unfollow", AdminAPIController, :user_unfollow)
-
-    get("/users", AdminAPIController, :list_users)
-    get("/users/:nickname", AdminAPIController, :user_show)
 
+    # TODO: to be removed at version 1.0
     delete("/user", AdminAPIController, :user_delete)
-    patch("/users/:nickname/toggle_activation", AdminAPIController, :user_toggle_activation)
     post("/user", AdminAPIController, :user_create)
+
+    delete("/users", AdminAPIController, :user_delete)
+    post("/users", AdminAPIController, :user_create)
+    patch("/users/:nickname/toggle_activation", AdminAPIController, :user_toggle_activation)
     put("/users/tag", AdminAPIController, :tag_users)
     delete("/users/tag", AdminAPIController, :untag_users)
 
+    # TODO: to be removed at version 1.0
     get("/permission_group/:nickname", AdminAPIController, :right_get)
     get("/permission_group/:nickname/:permission_group", AdminAPIController, :right_get)
     post("/permission_group/:nickname/:permission_group", AdminAPIController, :right_add)
     delete("/permission_group/:nickname/:permission_group", AdminAPIController, :right_delete)
 
-    put("/activation_status/:nickname", AdminAPIController, :set_activation_status)
+    get("/users/:nickname/permission_group", AdminAPIController, :right_get)
+    get("/users/:nickname/permission_group/:permission_group", AdminAPIController, :right_get)
+    post("/users/:nickname/permission_group/:permission_group", AdminAPIController, :right_add)
+
+    delete(
+      "/users/:nickname/permission_group/:permission_group",
+      AdminAPIController,
+      :right_delete
+    )
+
+    put("/users/:nickname/activation_status", AdminAPIController, :set_activation_status)
 
     post("/relay", AdminAPIController, :relay_follow)
     delete("/relay", AdminAPIController, :relay_unfollow)
 
-    get("/invite_token", AdminAPIController, :get_invite_token)
+    get("/users/invite_token", AdminAPIController, :get_invite_token)
-    get("/invites", AdminAPIController, :invites)
+    get("/users/invites", AdminAPIController, :invites)
-    post("/revoke_invite", AdminAPIController, :revoke_invite)
+    post("/users/revoke_invite", AdminAPIController, :revoke_invite)
-    post("/email_invite", AdminAPIController, :email_invite)
+    post("/users/email_invite", AdminAPIController, :email_invite)
 
+    # TODO: to be removed at version 1.0
     get("/password_reset", AdminAPIController, :get_password_reset)
+
+    get("/users/:nickname/password_reset", AdminAPIController, :get_password_reset)
+
+    get("/users", AdminAPIController, :list_users)
+    get("/users/:nickname", AdminAPIController, :user_show)
   end
 
   scope "/", Pleroma.Web.TwitterAPI do
@@ -197,6 +217,7 @@ defmodule Pleroma.Web.Router do
       post("/change_password", UtilController, :change_password)
       post("/delete_account", UtilController, :delete_account)
       put("/notification_settings", UtilController, :update_notificaton_settings)
+      post("/disable_account", UtilController, :disable_account)
     end
 
     scope [] do
@@ -367,6 +388,8 @@ defmodule Pleroma.Web.Router do
   scope "/api/v1", Pleroma.Web.MastodonAPI do
     pipe_through(:api)
 
+    post("/accounts", MastodonAPIController, :account_register)
+
     get("/instance", MastodonAPIController, :masto_instance)
     get("/instance/peers", MastodonAPIController, :peers)
     post("/apps", MastodonAPIController, :create_app)
@@ -383,7 +406,7 @@ defmodule Pleroma.Web.Router do
     get("/accounts/search", MastodonAPIController, :account_search)
 
     scope [] do
-      pipe_through(:oauth_read_or_unauthenticated)
+      pipe_through(:oauth_read_or_public)
 
       get("/timelines/public", MastodonAPIController, :public_timeline)
       get("/timelines/tag/:tag", MastodonAPIController, :hashtag_timeline)
@@ -404,7 +427,7 @@ defmodule Pleroma.Web.Router do
   end
 
   scope "/api/v2", Pleroma.Web.MastodonAPI do
-    pipe_through([:api, :oauth_read_or_unauthenticated])
+    pipe_through([:api, :oauth_read_or_public])
     get("/search", MastodonAPIController, :search2)
   end
 
@@ -434,7 +457,7 @@ defmodule Pleroma.Web.Router do
     )
 
     scope [] do
-      pipe_through(:oauth_read_or_unauthenticated)
+      pipe_through(:oauth_read_or_public)
 
       get("/statuses/user_timeline", TwitterAPI.Controller, :user_timeline)
       get("/qvitter/statuses/user_timeline", TwitterAPI.Controller, :user_timeline)
@@ -452,7 +475,7 @@ defmodule Pleroma.Web.Router do
   end
 
   scope "/api", Pleroma.Web do
-    pipe_through([:api, :oauth_read_or_unauthenticated])
+    pipe_through([:api, :oauth_read_or_public])
 
     get("/statuses/public_timeline", TwitterAPI.Controller, :public_timeline)
 
@@ -466,7 +489,7 @@ defmodule Pleroma.Web.Router do
   end
 
   scope "/api", Pleroma.Web, as: :twitter_api_search do
-    pipe_through([:api, :oauth_read_or_unauthenticated])
+    pipe_through([:api, :oauth_read_or_public])
     get("/pleroma/search_user", TwitterAPI.Controller, :search_user)
   end
 
@@ -650,7 +673,7 @@ defmodule Pleroma.Web.Router do
     delete("/auth/sign_out", MastodonAPIController, :logout)
 
     scope [] do
-      pipe_through(:oauth_read_or_unauthenticated)
+      pipe_through(:oauth_read_or_public)
       get("/web/*path", MastodonAPIController, :index)
     end
   end

lib/pleroma/web/salmon/salmon.ex

@@ -3,12 +3,18 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Web.Salmon do
+  @behaviour Pleroma.Web.Federator.Publisher
+
   @httpoison Application.get_env(:pleroma, :httpoison)
 
   use Bitwise
 
+  alias Pleroma.Activity
   alias Pleroma.Instances
   alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.Visibility
+  alias Pleroma.Web.Federator.Publisher
+  alias Pleroma.Web.OStatus
   alias Pleroma.Web.OStatus.ActivityRepresenter
   alias Pleroma.Web.XML
 
@@ -165,12 +171,12 @@ def remote_users(%{data: %{"to" => to} = data}) do
   end
 
   @doc "Pushes an activity to remote account."
-  def send_to_user(%{recipient: %{info: %{salmon: salmon}}} = params),
+  def publish_one(%{recipient: %{info: %{salmon: salmon}}} = params),
-    do: send_to_user(Map.put(params, :recipient, salmon))
+    do: publish_one(Map.put(params, :recipient, salmon))
 
-  def send_to_user(%{recipient: url, feed: feed, poster: poster} = params) when is_binary(url) do
+  def publish_one(%{recipient: url, feed: feed} = params) when is_binary(url) do
     with {:ok, %{status: code}} when code in 200..299 <-
-           poster.(
+           @httpoison.post(
              url,
              feed,
              [{"Content-Type", "application/magic-envelope+xml"}]
@@ -184,11 +190,11 @@ def send_to_user(%{recipient: url, feed: feed, poster: poster} = params) when is
       e ->
         unless params[:unreachable_since], do: Instances.set_reachable(url)
         Logger.debug(fn -> "Pushing Salmon to #{url} failed, #{inspect(e)}" end)
-        :error
+        {:error, "Unreachable instance"}
     end
   end
 
-  def send_to_user(_), do: :noop
+  def publish_one(_), do: :noop
 
   @supported_activities [
     "Create",
@@ -199,13 +205,19 @@ def send_to_user(_), do: :noop
     "Delete"
   ]
 
+  def is_representable?(%Activity{data: %{"type" => type}} = activity)
+      when type in @supported_activities,
+      do: Visibility.is_public?(activity)
+
+  def is_representable?(_), do: false
+
   @doc """
   Publishes an activity to remote accounts
   """
-  @spec publish(User.t(), Pleroma.Activity.t(), Pleroma.HTTP.t()) :: none
+  @spec publish(User.t(), Pleroma.Activity.t()) :: none
-  def publish(user, activity, poster \\ &@httpoison.post/3)
+  def publish(user, activity)
 
-  def publish(%{info: %{keys: keys}} = user, %{data: %{"type" => type}} = activity, poster)
+  def publish(%{info: %{keys: keys}} = user, %{data: %{"type" => type}} = activity)
       when type in @supported_activities do
     feed = ActivityRepresenter.to_simple_form(activity, user, true)
 
@@ -229,15 +241,29 @@ def publish(%{info: %{keys: keys}} = user, %{data: %{"type" => type}} = activity
       |> Enum.each(fn remote_user ->
         Logger.debug(fn -> "Sending Salmon to #{remote_user.ap_id}" end)
 
-        Pleroma.Web.Federator.publish_single_salmon(%{
+        Publisher.enqueue_one(__MODULE__, %{
           recipient: remote_user,
           feed: feed,
-          poster: poster,
           unreachable_since: reachable_urls_metadata[remote_user.info.salmon]
         })
       end)
     end
   end
 
-  def publish(%{id: id}, _, _), do: Logger.debug(fn -> "Keys missing for user #{id}" end)
+  def publish(%{id: id}, _), do: Logger.debug(fn -> "Keys missing for user #{id}" end)
+
+  def gather_webfinger_links(%User{} = user) do
+    {:ok, _private, public} = keys_from_pem(user.info.keys)
+    magic_key = encode_key(public)
+
+    [
+      %{"rel" => "salmon", "href" => OStatus.salmon_path(user)},
+      %{
+        "rel" => "magic-public-key",
+        "href" => "data:application/magic-public-key,#{magic_key}"
+      }
+    ]
+  end
+
+  def gather_nodeinfo_protocol_names, do: []
 end

lib/pleroma/web/twitter_api/controllers/util_controller.ex

@@ -173,8 +173,6 @@ def notifications_read(%{assigns: %{user: user}} = conn, %{"id" => notification_
 
   def config(conn, _params) do
     instance = Pleroma.Config.get(:instance)
-    instance_fe = Pleroma.Config.get(:fe)
-    instance_chat = Pleroma.Config.get(:chat)
 
     case get_format(conn) do
       "xml" ->
@@ -219,31 +217,7 @@ def config(conn, _params) do
             if(Pleroma.Config.get([:instance, :safe_dm_mentions]), do: "1", else: "0")
         }
 
-        pleroma_fe =
+        pleroma_fe = Pleroma.Config.get([:frontend_configurations, :pleroma_fe])
-          if instance_fe do
-            %{
-              theme: Keyword.get(instance_fe, :theme),
-              background: Keyword.get(instance_fe, :background),
-              logo: Keyword.get(instance_fe, :logo),
-              logoMask: Keyword.get(instance_fe, :logo_mask),
-              logoMargin: Keyword.get(instance_fe, :logo_margin),
-              redirectRootNoLogin: Keyword.get(instance_fe, :redirect_root_no_login),
-              redirectRootLogin: Keyword.get(instance_fe, :redirect_root_login),
-              chatDisabled: !Keyword.get(instance_chat, :enabled),
-              showInstanceSpecificPanel: Keyword.get(instance_fe, :show_instance_panel),
-              scopeOptionsEnabled: Keyword.get(instance_fe, :scope_options_enabled),
-              formattingOptionsEnabled: Keyword.get(instance_fe, :formatting_options_enabled),
-              collapseMessageWithSubject:
-                Keyword.get(instance_fe, :collapse_message_with_subject),
-              hidePostStats: Keyword.get(instance_fe, :hide_post_stats),
-              hideUserStats: Keyword.get(instance_fe, :hide_user_stats),
-              scopeCopy: Keyword.get(instance_fe, :scope_copy),
-              subjectLineBehavior: Keyword.get(instance_fe, :subject_line_behavior),
-              alwaysShowSubjectInput: Keyword.get(instance_fe, :always_show_subject_input)
-            }
-          else
-            Pleroma.Config.get([:frontend_configurations, :pleroma_fe])
-          end
 
         managed_config = Keyword.get(instance, :managed_config)
 
@@ -309,8 +283,13 @@ def follow_import(%{assigns: %{user: follower}} = conn, %{"list" => list}) do
            Enum.map(lines, fn line ->
              String.split(line, ",") |> List.first()
            end)
-           |> List.delete("Account address"),
+           |> List.delete("Account address") do
-         {:ok, _} = Task.start(fn -> User.follow_import(follower, followed_identifiers) end) do
+      PleromaJobQueue.enqueue(:background, User, [
+        :follow_import,
+        follower,
+        followed_identifiers
+      ])
+
       json(conn, "job started")
     end
   end
@@ -320,8 +299,13 @@ def blocks_import(conn, %{"list" => %Plug.Upload{} = listfile}) do
   end
 
   def blocks_import(%{assigns: %{user: blocker}} = conn, %{"list" => list}) do
-    with blocked_identifiers <- String.split(list),
+    with blocked_identifiers <- String.split(list) do
-         {:ok, _} = Task.start(fn -> User.blocks_import(blocker, blocked_identifiers) end) do
+      PleromaJobQueue.enqueue(:background, User, [
+        :blocks_import,
+        blocker,
+        blocked_identifiers
+      ])
+
       json(conn, "job started")
     end
   end
@@ -360,6 +344,17 @@ def delete_account(%{assigns: %{user: user}} = conn, params) do
     end
   end
 
+  def disable_account(%{assigns: %{user: user}} = conn, params) do
+    case CommonAPI.Utils.confirm_current_password(user, params["password"]) do
+      {:ok, user} ->
+        User.deactivate_async(user)
+        json(conn, %{status: "success"})
+
+      {:error, msg} ->
+        json(conn, %{error: msg})
+    end
+  end
+
   def captcha(conn, _params) do
     json(conn, Pleroma.Captcha.new())
   end

lib/pleroma/web/twitter_api/twitter_api.ex

@@ -128,7 +128,7 @@ def upload(%Plug.Upload{} = file, %User{} = user, format \\ "xml") do
     end
   end
 
-  def register_user(params) do
+  def register_user(params, opts \\ []) do
     token = params["token"]
 
     params = %{
@@ -162,13 +162,22 @@ def register_user(params) do
       # I have no idea how this error handling works
       {:error, %{error: Jason.encode!(%{captcha: [error]})}}
     else
-      registrations_open = Pleroma.Config.get([:instance, :registrations_open])
+      registration_process(
-      registration_process(registrations_open, params, token)
+        params,
+        %{
+          registrations_open: Pleroma.Config.get([:instance, :registrations_open]),
+          token: token
+        },
+        opts
+      )
     end
   end
 
-  defp registration_process(registration_open, params, token)
+  defp registration_process(params, %{registrations_open: true}, opts) do
-       when registration_open == false or is_nil(registration_open) do
+    create_user(params, opts)
+  end
+
+  defp registration_process(params, %{token: token}, opts) do
     invite =
       unless is_nil(token) do
         Repo.get_by(UserInviteToken, %{token: token})
@@ -182,19 +191,15 @@ defp registration_process(registration_open, params, token)
 
       invite when valid_invite? ->
         UserInviteToken.update_usage!(invite)
-        create_user(params)
+        create_user(params, opts)
 
       _ ->
         {:error, "Expired token"}
     end
   end
 
-  defp registration_process(true, params, _token) do
+  defp create_user(params, opts) do
-    create_user(params)
+    changeset = User.register_changeset(%User{}, params, opts)
-  end
-
-  defp create_user(params) do
-    changeset = User.register_changeset(%User{}, params)
 
     case User.register(changeset) do
       {:ok, user} ->
@@ -231,12 +236,15 @@ def password_reset(nickname_or_email) do
   def get_user(user \\ nil, params) do
     case params do
       %{"user_id" => user_id} ->
-        case target = User.get_cached_by_nickname_or_id(user_id) do
+        case User.get_cached_by_nickname_or_id(user_id) do
           nil ->
             {:error, "No user with such user_id"}
 
-          _ ->
+          %User{info: %{deactivated: true}} ->
-            {:ok, target}
+            {:error, "User has been disabled"}
+
+          user ->
+            {:ok, user}
         end
 
       %{"screen_name" => nickname} ->

lib/pleroma/web/twitter_api/twitter_api_controller.ex

@@ -440,7 +440,7 @@ def confirm_email(conn, %{"user_id" => uid, "token" => token}) do
          true <- user.local,
          true <- user.info.confirmation_pending,
          true <- user.info.confirmation_token == token,
-         info_change <- User.Info.confirmation_changeset(user.info, :confirmed),
+         info_change <- User.Info.confirmation_changeset(user.info, need_confirmation: false),
          changeset <- Changeset.change(user) |> Changeset.put_embed(:info, info_change),
          {:ok, _} <- User.update_and_set_cache(changeset) do
       conn

lib/pleroma/web/twitter_api/views/activity_view.ex

@@ -170,7 +170,7 @@ def render("activity.json", %{activity: %{data: %{"type" => "Announce"}} = activ
     created_at = activity.data["published"] |> Utils.date_to_asctime()
     announced_activity = Activity.get_create_by_object_ap_id(activity.data["object"])
 
-    text = "#{user.nickname} retweeted a status."
+    text = "#{user.nickname} repeated a status."
 
     retweeted_status = render("activity.json", Map.merge(opts, %{activity: announced_activity}))
 
@@ -310,7 +310,7 @@ def render(
       "tags" => tags,
       "activity_type" => "post",
       "possibly_sensitive" => possibly_sensitive,
-      "visibility" => StatusView.get_visibility(object),
+      "visibility" => Pleroma.Web.ActivityPub.Visibility.get_visibility(object),
       "summary" => summary,
       "summary_html" => summary |> Formatter.emojify(object.data["emoji"]),
       "card" => card,

lib/pleroma/web/web_finger/web_finger.ex

@@ -7,7 +7,7 @@ defmodule Pleroma.Web.WebFinger do
 
   alias Pleroma.User
   alias Pleroma.Web
-  alias Pleroma.Web.OStatus
+  alias Pleroma.Web.Federator.Publisher
   alias Pleroma.Web.Salmon
   alias Pleroma.Web.XML
   alias Pleroma.XmlBuilder
@@ -50,70 +50,40 @@ def webfinger(resource, fmt) when fmt in ["XML", "JSON"] do
     end
   end
 
-  def represent_user(user, "JSON") do
+  defp gather_links(%User{} = user) do
-    {:ok, user} = ensure_keys_present(user)
+    [
-    {:ok, _private, public} = Salmon.keys_from_pem(user.info.keys)
-    magic_key = Salmon.encode_key(public)
-
-    %{
-      "subject" => "acct:#{user.nickname}@#{Pleroma.Web.Endpoint.host()}",
-      "aliases" => [user.ap_id],
-      "links" => [
-        %{
-          "rel" => "http://schemas.google.com/g/2010#updates-from",
-          "type" => "application/atom+xml",
-          "href" => OStatus.feed_path(user)
-        },
       %{
         "rel" => "http://webfinger.net/rel/profile-page",
         "type" => "text/html",
         "href" => user.ap_id
-        },
-        %{"rel" => "salmon", "href" => OStatus.salmon_path(user)},
-        %{
-          "rel" => "magic-public-key",
-          "href" => "data:application/magic-public-key,#{magic_key}"
-        },
-        %{"rel" => "self", "type" => "application/activity+json", "href" => user.ap_id},
-        %{
-          "rel" => "self",
-          "type" => "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"",
-          "href" => user.ap_id
-        },
-        %{
-          "rel" => "http://ostatus.org/schema/1.0/subscribe",
-          "template" => OStatus.remote_follow_path()
       }
-      ]
+    ] ++ Publisher.gather_webfinger_links(user)
+  end
+
+  def represent_user(user, "JSON") do
+    {:ok, user} = ensure_keys_present(user)
+
+    %{
+      "subject" => "acct:#{user.nickname}@#{Pleroma.Web.Endpoint.host()}",
+      "aliases" => [user.ap_id],
+      "links" => gather_links(user)
     }
   end
 
   def represent_user(user, "XML") do
     {:ok, user} = ensure_keys_present(user)
-    {:ok, _private, public} = Salmon.keys_from_pem(user.info.keys)
+
-    magic_key = Salmon.encode_key(public)
+    links =
+      gather_links(user)
+      |> Enum.map(fn link -> {:Link, link} end)
 
     {
       :XRD,
       %{xmlns: "http://docs.oasis-open.org/ns/xri/xrd-1.0"},
       [
         {:Subject, "acct:#{user.nickname}@#{Pleroma.Web.Endpoint.host()}"},
-        {:Alias, user.ap_id},
+        {:Alias, user.ap_id}
-        {:Link,
+      ] ++ links
-         %{
-           rel: "http://schemas.google.com/g/2010#updates-from",
-           type: "application/atom+xml",
-           href: OStatus.feed_path(user)
-         }},
-        {:Link,
-         %{rel: "http://webfinger.net/rel/profile-page", type: "text/html", href: user.ap_id}},
-        {:Link, %{rel: "salmon", href: OStatus.salmon_path(user)}},
-        {:Link,
-         %{rel: "magic-public-key", href: "data:application/magic-public-key,#{magic_key}"}},
-        {:Link, %{rel: "self", type: "application/activity+json", href: user.ap_id}},
-        {:Link,
-         %{rel: "http://ostatus.org/schema/1.0/subscribe", template: OStatus.remote_follow_path()}}
-      ]
     }
     |> XmlBuilder.to_doc()
   end

lib/pleroma/web/websub/websub.ex

@@ -4,10 +4,14 @@
 
 defmodule Pleroma.Web.Websub do
   alias Ecto.Changeset
+  alias Pleroma.Activity
   alias Pleroma.Instances
   alias Pleroma.Repo
+  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.Visibility
   alias Pleroma.Web.Endpoint
   alias Pleroma.Web.Federator
+  alias Pleroma.Web.Federator.Publisher
   alias Pleroma.Web.OStatus
   alias Pleroma.Web.OStatus.FeedRepresenter
   alias Pleroma.Web.Router.Helpers
@@ -18,6 +22,8 @@ defmodule Pleroma.Web.Websub do
 
   import Ecto.Query
 
+  @behaviour Pleroma.Web.Federator.Publisher
+
   @httpoison Application.get_env(:pleroma, :httpoison)
 
   def verify(subscription, getter \\ &@httpoison.get/3) do
@@ -56,6 +62,13 @@ def verify(subscription, getter \\ &@httpoison.get/3) do
     "Undo",
     "Delete"
   ]
+
+  def is_representable?(%Activity{data: %{"type" => type}} = activity)
+      when type in @supported_activities,
+      do: Visibility.is_public?(activity)
+
+  def is_representable?(_), do: false
+
   def publish(topic, user, %{data: %{"type" => type}} = activity)
       when type in @supported_activities do
     response =
@@ -88,12 +101,14 @@ def publish(topic, user, %{data: %{"type" => type}} = activity)
         unreachable_since: reachable_callbacks_metadata[sub.callback]
       }
 
-      Federator.publish_single_websub(data)
+      Publisher.enqueue_one(__MODULE__, data)
     end)
   end
 
   def publish(_, _, _), do: ""
 
+  def publish(actor, activity), do: publish(Pleroma.Web.OStatus.feed_path(actor), actor, activity)
+
   def sign(secret, doc) do
     :crypto.hmac(:sha, secret, to_string(doc)) |> Base.encode16() |> String.downcase()
   end
@@ -299,4 +314,20 @@ def publish_one(%{xml: xml, topic: topic, callback: callback, secret: secret} =
         {:error, response}
     end
   end
+
+  def gather_webfinger_links(%User{} = user) do
+    [
+      %{
+        "rel" => "http://schemas.google.com/g/2010#updates-from",
+        "type" => "application/atom+xml",
+        "href" => OStatus.feed_path(user)
+      },
+      %{
+        "rel" => "http://ostatus.org/schema/1.0/subscribe",
+        "template" => OStatus.remote_follow_path()
+      }
+    ]
+  end
+
+  def gather_nodeinfo_protocol_names, do: ["ostatus"]
 end

lib/xml_builder.ex

@@ -35,6 +35,7 @@ def to_doc(content), do: ~s(<?xml version="1.0" encoding="UTF-8"?>) <> to_xml(co
   defp make_open_tag(tag, attributes) do
     attributes_string =
       for {attribute, value} <- attributes do
+        value = String.replace(value, "\"", "&quot;")
         "#{attribute}=\"#{value}\""
       end
       |> Enum.join(" ")

mix.exs

@@ -13,6 +13,7 @@ def project do
       start_permanent: Mix.env() == :prod,
       aliases: aliases(),
       deps: deps(),
+      test_coverage: [tool: ExCoveralls],
 
       # Docs
       name: "Pleroma",
@@ -103,6 +104,9 @@ defp deps do
       {:auto_linker,
        git: "https://git.pleroma.social/pleroma/auto_linker.git",
        ref: "c00c4e75b35367fa42c95ffd9b8c455bf9995829"},
+      {:http_signatures,
+       git: "https://git.pleroma.social/pleroma/http_signatures.git",
+       ref: "9789401987096ead65646b52b5a2ca6bf52fc531"},
       {:pleroma_job_queue, "~> 0.2.0"},
       {:telemetry, "~> 0.3"},
       {:prometheus_ex, "~> 3.0"},
@@ -113,7 +117,10 @@ defp deps do
       {:recon, github: "ferd/recon", tag: "2.4.0"},
       {:quack, "~> 0.1.1"},
       {:benchee, "~> 1.0"},
-      {:esshd, "~> 0.1.0"}
+      {:esshd, "~> 0.1.0"},
+      {:ex_rated, "~> 1.2"},
+      {:plug_static_index_html, "~> 1.0.0"},
+      {:excoveralls, "~> 0.11.1", only: :test}
     ] ++ oauth_deps
   end
 

mix.lock

@@ -24,17 +24,21 @@
   "ecto_sql": {:hex, :ecto_sql, "3.0.5", "7e44172b4f7aca4469f38d7f6a3da394dbf43a1bcf0ca975e958cb957becd74e", [:mix], [{:db_connection, "~> 2.0", [hex: :db_connection, repo: "hexpm", optional: false]}, {:ecto, "~> 3.0.6", [hex: :ecto, repo: "hexpm", optional: false]}, {:mariaex, "~> 0.9.1", [hex: :mariaex, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.14.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.3.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm"},
   "esshd": {:hex, :esshd, "0.1.0", "6f93a2062adb43637edad0ea7357db2702a4b80dd9683482fe00f5134e97f4c1", [:mix], [], "hexpm"},
   "eternal": {:hex, :eternal, "1.2.0", "e2a6b6ce3b8c248f7dc31451aefca57e3bdf0e48d73ae5043229380a67614c41", [:mix], [], "hexpm"},
+  "ex2ms": {:hex, :ex2ms, "1.5.0", "19e27f9212be9a96093fed8cdfbef0a2b56c21237196d26760f11dfcfae58e97", [:mix], [], "hexpm"},
   "ex_aws": {:hex, :ex_aws, "2.1.0", "b92651527d6c09c479f9013caa9c7331f19cba38a650590d82ebf2c6c16a1d8a", [:mix], [{:configparser_ex, "~> 2.0", [hex: :configparser_ex, repo: "hexpm", optional: true]}, {:hackney, "1.6.3 or 1.6.5 or 1.7.1 or 1.8.6 or ~> 1.9", [hex: :hackney, repo: "hexpm", optional: true]}, {:jsx, "~> 2.8", [hex: :jsx, repo: "hexpm", optional: true]}, {:poison, ">= 1.2.0", [hex: :poison, repo: "hexpm", optional: true]}, {:sweet_xml, "~> 0.6", [hex: :sweet_xml, repo: "hexpm", optional: true]}, {:xml_builder, "~> 0.1.0", [hex: :xml_builder, repo: "hexpm", optional: true]}], "hexpm"},
   "ex_aws_s3": {:hex, :ex_aws_s3, "2.0.1", "9e09366e77f25d3d88c5393824e613344631be8db0d1839faca49686e99b6704", [:mix], [{:ex_aws, "~> 2.0", [hex: :ex_aws, repo: "hexpm", optional: false]}, {:sweet_xml, ">= 0.0.0", [hex: :sweet_xml, repo: "hexpm", optional: true]}], "hexpm"},
   "ex_doc": {:hex, :ex_doc, "0.20.2", "1bd0dfb0304bade58beb77f20f21ee3558cc3c753743ae0ddbb0fd7ba2912331", [:mix], [{:earmark, "~> 1.3", [hex: :earmark, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.10", [hex: :makeup_elixir, repo: "hexpm", optional: false]}], "hexpm"},
   "ex_machina": {:hex, :ex_machina, "2.3.0", "92a5ad0a8b10ea6314b876a99c8c9e3f25f4dde71a2a835845b136b9adaf199a", [:mix], [{:ecto, "~> 2.2 or ~> 3.0", [hex: :ecto, repo: "hexpm", optional: true]}, {:ecto_sql, "~> 3.0", [hex: :ecto_sql, repo: "hexpm", optional: true]}], "hexpm"},
+  "ex_rated": {:hex, :ex_rated, "1.3.2", "6aeb32abb46ea6076f417a9ce8cb1cf08abf35fb2d42375beaad4dd72b550bf1", [:mix], [{:ex2ms, "~> 1.5", [hex: :ex2ms, repo: "hexpm", optional: false]}], "hexpm"},
   "ex_syslogger": {:git, "https://github.com/slashmili/ex_syslogger.git", "f3963399047af17e038897c69e20d552e6899e1d", [tag: "1.4.0"]},
+  "excoveralls": {:hex, :excoveralls, "0.11.1", "dd677fbdd49114fdbdbf445540ec735808250d56b011077798316505064edb2c", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm"},
   "floki": {:hex, :floki, "0.20.4", "be42ac911fece24b4c72f3b5846774b6e61b83fe685c2fc9d62093277fb3bc86", [:mix], [{:html_entities, "~> 0.4.0", [hex: :html_entities, repo: "hexpm", optional: false]}, {:mochiweb, "~> 2.15", [hex: :mochiweb, repo: "hexpm", optional: false]}], "hexpm"},
   "gen_smtp": {:hex, :gen_smtp, "0.13.0", "11f08504c4bdd831dc520b8f84a1dce5ce624474a797394e7aafd3c29f5dcd25", [:rebar3], [], "hexpm"},
   "gettext": {:hex, :gettext, "0.15.0", "40a2b8ce33a80ced7727e36768499fc9286881c43ebafccae6bab731e2b2b8ce", [:mix], [], "hexpm"},
   "hackney": {:hex, :hackney, "1.15.1", "9f8f471c844b8ce395f7b6d8398139e26ddca9ebc171a8b91342ee15a19963f4", [:rebar3], [{:certifi, "2.5.1", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "6.0.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "1.0.1", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~>1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "1.1.4", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}], "hexpm"},
   "html_entities": {:hex, :html_entities, "0.4.0", "f2fee876858cf6aaa9db608820a3209e45a087c5177332799592142b50e89a6b", [:mix], [], "hexpm"},
   "html_sanitize_ex": {:hex, :html_sanitize_ex, "1.3.0", "f005ad692b717691203f940c686208aa3d8ffd9dd4bb3699240096a51fa9564e", [:mix], [{:mochiweb, "~> 2.15", [hex: :mochiweb, repo: "hexpm", optional: false]}], "hexpm"},
+  "http_signatures": {:git, "https://git.pleroma.social/pleroma/http_signatures.git", "9789401987096ead65646b52b5a2ca6bf52fc531", [ref: "9789401987096ead65646b52b5a2ca6bf52fc531"]},
   "httpoison": {:hex, :httpoison, "1.2.0", "2702ed3da5fd7a8130fc34b11965c8cfa21ade2f232c00b42d96d4967c39a3a3", [:mix], [{:hackney, "~> 1.8", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
   "idna": {:hex, :idna, "6.0.0", "689c46cbcdf3524c44d5f3dde8001f364cd7608a99556d8fbd8239a5798d4c10", [:rebar3], [{:unicode_util_compat, "0.4.1", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm"},
   "jason": {:hex, :jason, "1.1.2", "b03dedea67a99223a2eaf9f1264ce37154564de899fd3d8b9a21b1a6fd64afe7", [:mix], [{:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm"},
@@ -59,6 +63,7 @@
   "plug": {:hex, :plug, "1.7.2", "d7b7db7fbd755e8283b6c0a50be71ec0a3d67d9213d74422d9372effc8e87fd1", [:mix], [{:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}], "hexpm"},
   "plug_cowboy": {:hex, :plug_cowboy, "2.0.1", "d798f8ee5acc86b7d42dbe4450b8b0dadf665ce588236eb0a751a132417a980e", [:mix], [{:cowboy, "~> 2.5", [hex: :cowboy, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "plug_crypto": {:hex, :plug_crypto, "1.0.0", "18e49317d3fa343f24620ed22795ec29d4a5e602d52d1513ccea0b07d8ea7d4d", [:mix], [], "hexpm"},
+  "plug_static_index_html": {:hex, :plug_static_index_html, "1.0.0", "840123d4d3975585133485ea86af73cb2600afd7f2a976f9f5fd8b3808e636a0", [:mix], [{:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "poison": {:hex, :poison, "3.1.0", "d9eb636610e096f86f25d9a46f35a9facac35609a7591b3be3326e99a0484665", [:mix], [], "hexpm"},
   "poolboy": {:hex, :poolboy, "1.5.2", "392b007a1693a64540cead79830443abf5762f5d30cf50bc95cb2c1aaafa006b", [:rebar3], [], "hexpm"},
   "postgrex": {:hex, :postgrex, "0.14.1", "63247d4a5ad6b9de57a0bac5d807e1c32d41e39c04b8a4156a26c63bcd8a2e49", [:mix], [{:connection, "~> 1.0", [hex: :connection, repo: "hexpm", optional: false]}, {:db_connection, "~> 2.0", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.5", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}], "hexpm"},

priv/repo/migrations/20190411094120_add_index_on_user_info_deactivated.exs

@@ -0,0 +1,7 @@
+defmodule Pleroma.Repo.Migrations.AddIndexOnUserInfoDeactivated do
+  use Ecto.Migration
+
+  def change do
+    create(index(:users, ["(info->'deactivated')"], name: :users_deactivated_index, using: :gin))
+  end
+end

priv/repo/migrations/20190513175809_change_hide_column_in_filter_table.exs

@@ -0,0 +1,9 @@
+defmodule Pleroma.Repo.Migrations.ChangeHideColumnInFilterTable do
+  use Ecto.Migration
+
+  def change do
+    alter table(:filters) do
+      modify :hide, :boolean, default: false
+    end
+  end
+end

priv/static/adminfe/index.html

@@ -0,0 +1 @@
+<!DOCTYPE html><html><head><meta charset=utf-8><meta http-equiv=X-UA-Compatible content="IE=edge,chrome=1"><meta name=renderer content=webkit><meta name=viewport content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no"><title>Admin FE</title><link rel="shortcut icon" href=favicon.ico><link href=static/css/chunk-elementUI.4296cedf.css rel=stylesheet><link href=static/css/chunk-libs.bd17d456.css rel=stylesheet><link href=static/css/app.cea15678.css rel=stylesheet></head><body><script src=/pleroma/admin/static/tinymce4.7.5/tinymce.min.js></script><div id=app></div><script type=text/javascript src=static/js/runtime.7144b2cf.js></script><script type=text/javascript src=static/js/chunk-elementUI.d388c21d.js></script><script type=text/javascript src=static/js/chunk-libs.48e79a9e.js></script><script type=text/javascript src=static/js/app.25699e3d.js></script></body></html>