Commit Graph

52 Commits

Author SHA1 Message Date
Mark Felder f4e48bc53e Rename variable to make the worker retry logic easier to read
The boolean value matches the intent of the "last_attempt" variable name now
2024-02-23 11:12:10 -05:00
Mark Felder ac55764599 Gun Connection Pool was not attempting to free a connection and retry once if the pool was full. 2024-02-22 14:07:46 -05:00
Mark Felder 0eca3e38eb Fix Gun connection supervisor logic error
This was recently changed to solve a Dialyzer error, but the replacement logic was faulty as "retry" would only be compared to :error and not have its truthiness evaluated.

The original logic was also faulty as it returned {:error, :pool_full} even retry was true. It never retried when the pool was full.
2024-02-09 10:48:40 -05:00
Mark Felder 5c08153fc5 Pleroma.Gun.ConnectionPool.Reclaimer: dialyzer error
lib/pleroma/gun/connection_pool/reclaimer.ex:12:call
The function call will not succeed.

:gen_server.start(Pleroma.Gun.ConnectionPool.Reclaimer, [], [
  {:name, {:via, Registry, {Pleroma.Gun.ConnectionPool, <<_::72>>}}},
  ...
])

will never return since the success typing is:
(atom(), any(), [
  {:debug, [:log | :statistics | :trace | {_, _}]}
  | {:hibernate_after, timeout()}
  | {:spawn_opt, [:link | :monitor | {_, _}]}
  | {:timeout, timeout()}
]) :: :ignore | {:error, _} | {:ok, pid() | {pid(), reference()}}

and the contract is
(Module :: module(), Args :: term(), Options :: [start_opt()]) :: start_ret()
2024-01-27 14:25:16 -05:00
Mark Felder 6ce7011a2e Pleroma.Gun.ConnectionPool.WorkerSupervisor: fix dialyzer error
lib/pleroma/gun/connection_pool/worker_supervisor.ex:24:guard_fail
The guard clause:

when _ :: true === nil

can never succeed.
2024-01-22 18:37:13 -05:00
Mark Felder 9896b64f54 Elixir 1.15: Chase the Logger.warn deprecation 2023-12-20 20:16:26 +00:00
lain e853cfe7c3 Revert "Merge branch 'copyright-bump' into 'develop'"
This reverts merge request !3825
2023-01-02 20:38:50 +00:00
marcin mikołajczak 10886eeaa2 Bump copyright year
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
2023-01-01 12:13:06 +01:00
Sean King 17aa3644be
Copyright bump for 2022 2022-02-25 23:11:42 -07:00
Haelwenn (lanodan) Monnier a17910a6c6
CI: Bump lint stage to elixir-1.12
Elixir 1.12 changed formatting rules, this allows to avoid having to rollback to run `mix format`
2021-10-06 08:11:05 +02:00
Alex Gleason 0ada3fe823
Gun: use runtime deps in ConnectionPool
Speed up recompilation time by breaking compile-time cycles
2021-05-29 11:35:53 -05:00
Haelwenn (lanodan) Monnier c4439c630f
Bump Copyright to 2021
grep -rl '# Copyright © .* Pleroma' * | xargs sed -i 's;Copyright © .* Pleroma .*;Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>;'
2021-01-13 07:49:50 +01:00
Alexander Strizhakov 0374df1d12
other files consistency 2020-10-13 16:38:19 +03:00
Mark Felder 8539e386c3 Add missing Copyright headers 2020-10-12 12:00:50 -05:00
rinpatch adb1b0282d ConnectionPool Worker: use monitor flush instead of checking ref
`:flush` removes the DOWN message if one had arrived, so this check
should no longer be necessary.
2020-09-16 17:29:07 +03:00
rinpatch 89a7efab69 ConnectionPool: Log possible HTTP1 blocks 2020-09-11 15:37:39 +03:00
Alexander Strizhakov a83916fdac
adapter options unification
not needed options deletion
2020-09-07 19:59:17 +03:00
Alexander Strizhakov c17d83cd73
improvements and fixes for http requests
- fix for gun worker termination in some circumstances
- pool for http clients (ex_aws, tzdata)
- default pool timeouts for gun
- gun retries on gun_down messages
- s3 upload timeout if streaming enabled
2020-09-02 09:04:23 +03:00
rinpatch 45be1fe00e ConnectionPool: fix gun open errors being returned without an error tuple
When gun shuts down due to the host being unreachable, the worker
process shuts down with the same shutdown reason since they are linked.
Gun doesn't have error tuples in it's shutdown reason though, so we need
to handle it in get_conn.

Closes #2008
2020-08-01 18:02:26 +03:00
rinpatch dab1d8c98e gun ConnectionPool: Re-add a missing cast for remove_client 2020-07-28 23:48:41 +03:00
rinpatch 4ce6179dc7 gun ConnectionPool: replace casts with calls
The slowdown from this is most likely immesurable, however it eliminates
possible false positives when tracking dead clients.
2020-07-28 20:49:48 +03:00
rinpatch 7115c5f82e ConnectionPool.Worker: do not stop with an error when there is a timeout
This produced error log messages about GenServer termination
every time the connection was not open due to a timeout.

Instead we stop with `{:shutdown, <gun_error>}` since shutting down
when the connection can't be established is normal behavior.
2020-07-15 15:58:08 +03:00
href afd378f84c host is now useless 2020-07-15 15:26:35 +03:00
href ce1a42bd04 Simplify TLS opts
- `verify_fun` is not useful now
- use `customize_check_hostname` (OTP 20+ so OK)
- `partial_chain` is useless as of OTP 21.1 (wasn't there, but hackney/..
uses it)
2020-07-15 15:26:35 +03:00
href 53ba6815b1 parentheses... 2020-07-15 15:26:35 +03:00
href 23d714ed30 Fix race in enforcer/reclaimer start 2020-07-15 15:26:35 +03:00
href 6a0f2bdf8c Ensure connections error get known by the caller 2020-07-15 15:26:35 +03:00
href 46dd276d68 ConnectionPool.Worker: Open gun conn in continue instead of init 2020-07-15 15:26:35 +03:00
rinpatch a705637dcf Connection Pool: fix LRFU implementation to not actually be LRU
The numbers of the native time unit were so small the CRF was always 1,
making it an LRU. This commit switches the time to miliseconds and changes
the time delta multiplier to the one yielding mostly highest hit rates according
to the paper
2020-07-15 15:26:35 +03:00
rinpatch 007843b75e Add documentation for new connection pool settings and remove some
`:retry_timeout` and `:retry` got removed because reconnecting on failure is
something the new pool intentionally doesn't do.

`:max_overflow` had to go in favor of `:max_waiting`, I didn't reuse the key because
the settings are very different in their behaviour.

`:checkin_timeout` got removed in favor of `:connection_acquisition_wait`,
I didn't reuse the key because the settings are somewhat different.

I didn't do any migrations/deprecation warnings/changelog entries because
these settings were never in stable.
2020-07-15 15:26:35 +03:00
rinpatch 7882f28569 Use erlang monotonic time for CRF calculation 2020-07-15 15:26:35 +03:00
rinpatch 281ddd5e37 Connection pool: fix connections being supervised by gun_sup 2020-07-15 15:26:35 +03:00
rinpatch 1b15cb066c Connection pool: Add client death tracking
While running this in production I noticed a number of ghost
processes with all their clients dead before they released the connection,
so let's track them to log it and remove them from clients
2020-07-15 15:26:35 +03:00
rinpatch e94ba05e52 Connection pool: Fix a possible infinite recursion if the pool is exhausted 2020-07-15 15:26:35 +03:00
rinpatch 7738fbbaf5 Connection pool: implement logging and telemetry events 2020-07-15 15:26:25 +03:00
rinpatch 0ffde499b8 Connection Pool: register workers using :via 2020-07-15 15:24:47 +03:00
rinpatch ec9d0d146b Connection pool: Fix race conditions in limit enforcement
Fixes race conditions in limit enforcement by putting worker processes
in a DynamicSupervisor
2020-07-15 15:17:27 +03:00
rinpatch d08b157699 Connection pool: check that there actually is a result
Sometimes connections died before being released to the pool, resulting
in MatchErrors
2020-07-15 15:17:27 +03:00
rinpatch fffbcffb8c Connection Pool: don't enforce pool limits if no new connection needs to be opened 2020-07-15 15:17:27 +03:00
rinpatch 58a4f350a8 Refactor gun pooling and simplify adapter option insertion
This patch refactors gun pooling to use Elixir process registry and
simplifies adapter option insertion.

Having the pool use process registry instead of a GenServer has a number of advantages:
- Simpler code: the initial implementation adds about half the lines of code it deletes
- Concurrency: unlike a GenServer, ETS-based registry can handle multiple checkout/checkin
requests at the same time
- Precise and easy idle connection clousure: current proposal for closing idle connections in
the GenServer-based pool needs to filter through all connections once a minute and compare their
last active time with closing time. With Elixir process registry this can be done
by just using `Process.send_after`/`Process.cancel_timer` in the worker process.
- Lower memory footprint: In my tests `gun-memory-leak` branch uses about 290mb on peak load (250 connections)
and 235mb on idle (5-10 connections). Registry-based pool uses 210mb on idle and 240mb on peak load
2020-07-15 15:17:27 +03:00
Alexander Strizhakov 5739c498c0
fix for gun connections pool 2020-04-08 18:46:01 +03:00
Alexander Strizhakov 1306b92997
clean up 2020-03-12 18:29:07 +03:00
Alexander Strizhakov f39e1b9eff
add verify tls_opts only when we open connection
for other requests tesla will add tls_opts
2020-03-10 15:54:11 +03:00
Alexander Strizhakov c93c3096d5
little refactor 2020-03-06 21:04:18 +03:00
Mark Felder d9c5ae7c09 Update Copyrights for gun related files 2020-03-03 17:16:24 -06:00
Alexander Strizhakov 884d9710b2
refactoring for gun api modules 2020-03-03 19:24:14 +03:00
Alexander Strizhakov 8854770fc4
retry and retry_timeout settings default change 2020-03-03 18:01:35 +03:00
Alexander Strizhakov 8fc00b7cbf
return error if connection failed to open 2020-03-03 16:42:24 +03:00
Alexander Strizhakov 7d68924e4f
naming 2020-03-03 16:42:24 +03:00
Alexander Strizhakov e605e79df9
simplification of formatting host method
case for format_proxy method
2020-03-03 16:42:03 +03:00