Commit Graph

52 Commits

Author SHA1 Message Date
Mark Felder e8bf060e6e Move construction of S3 base URL with optional namespace and bucket to Upload.base_url/0
Now we should have a correct base URL for S3 hosted objects throughout the codebase.
2021-01-08 17:32:42 -06:00
Mark Felder 3c936061d5 Apply Upload.base_url for S3 2021-01-08 17:32:42 -06:00
Alexander Strizhakov a5987155f7
UploadedMedia module name 2020-10-13 16:42:51 +03:00
Alexander Strizhakov 84fbf16161
timeout option moved to gun adapter helper 2020-09-02 10:50:51 +03:00
Alexander Strizhakov c17d83cd73
improvements and fixes for http requests
- fix for gun worker termination in some circumstances
- pool for http clients (ex_aws, tzdata)
- default pool timeouts for gun
- gun retries on gun_down messages
- s3 upload timeout if streaming enabled
2020-09-02 09:04:23 +03:00
Haelwenn (lanodan) Monnier 6da6540036
Bump copyright years of files changed after 2020-01-07
Done via the following command:
git diff fcd5dd259a --stat --name-only | xargs sed -i '/Pleroma Authors/c# Copyright © 2017-2020 Pleroma Authors <https:\/\/pleroma.social\/>'
2020-03-02 06:08:45 +01:00
Roman Chvanikov 88f0eed0f2 Delete attachments when status is deleted 2020-01-12 18:48:58 +00:00
Hakaba Hitoyo 0c9c62509d Remove MDII uploader 2020-01-11 17:19:54 +00:00
Rachel Fae Fox 63af6951fa add tunable for stream uploads, as needed for jortage to work. 2019-09-23 20:38:53 +00:00
Maksim 11d08c2de0 tests for Pleroma.Uploaders 2019-08-10 18:46:26 +00:00
Maksim bb9c539580 Uploader.S3 added support stream uploads 2019-08-10 11:27:59 +00:00
Maksim b6b748d3e7 tests for Uploader with webhook 2019-07-18 12:30:18 +00:00
Egor Kislitsyn 5104f65b69 Wrap error messages into gettext helpers 2019-07-10 18:10:09 +07:00
Egor Kislitsyn 4f42093220 Remove Uploaders.Swift 2019-07-03 14:11:04 +07:00
William Pitcock 9bec891eb4 kill @httpoison 2019-05-25 04:24:21 +00:00
Mark Felder 498bfdf403 Switch to Jason over Poison 2019-05-13 15:37:38 -05:00
Austin Lanari 6c96d68f77 add overriding truncated_namespace condition for truncating paths for digital ocean 2019-03-24 11:32:09 -07:00
Haelwenn (lanodan) Monnier 4b73ca638e
[Credo][CI] Add readability as it’s fixed 2019-03-13 04:26:56 +01:00
Haelwenn (lanodan) Monnier c42d34b2ec
[Credo] fix Credo.Check.Readability.MaxLineLength 2019-03-13 04:26:56 +01:00
Haelwenn (lanodan) Monnier fb82f6fc7c
[Credo] Remove parentesis on argument-less functions 2019-03-13 04:26:56 +01:00
hakabahitoyo b7bc666200 bugfix mdii uploader 2019-02-13 15:46:42 +09:00
Haelwenn (lanodan) Monnier 106f4e7a0f
Credo fixes: parameter consistency 2019-02-09 14:59:20 +01:00
href 4aff4efa8d
Use multiple hackney pools
* federation (ap, salmon)
* media (rich media, media proxy)
* upload (uploader proxy)

Each "part" will stop fighting others ones -- a huge federation outbound
could before make the media proxy fail to checkout a connection in time.

splitted media and uploaded media for the good reason than an upload
pool will have all connections to the same host (the uploader upstream).
it also has a longer default retention period for connections.
2019-01-30 15:06:46 +01:00
kaniini 4a278cd80a Merge branch 's3-namespace' into 'develop'
S3 uploader: support for namespaced bucket

Closes #532

See merge request pleroma/pleroma!699
2019-01-23 13:25:04 +00:00
href f9a3269099
Uploader callback controller 2019-01-21 22:44:14 +01:00
href 97412d9f94
S3 Namespace 2019-01-21 14:42:16 +01:00
William Pitcock 980b5288ed update copyright years to 2019 2018-12-31 15:41:47 +00:00
William Pitcock 2791ce9a1f add license boilerplate to pleroma core 2018-12-23 20:56:42 +00:00
Maksim Pechnikov 074fa790ba fix compile warnings 2018-12-09 20:50:08 +03:00
Maksim Pechnikov 87109482f3 status_code -> status 2018-12-04 14:04:06 +03:00
href 02d3dc6869
Uploads fun, part. 2 2018-11-30 18:02:37 +01:00
href b19597f602
reverse proxy / uploads 2018-11-30 18:00:47 +01:00
hakabahitoyo 59e079f641 fallbacking into local uploader 2018-11-17 20:16:25 +09:00
hakabahitoyo 8fd0556c78 better config reading 2018-11-17 18:14:42 +09:00
hakabahitoyo 55abd8482e better config 2018-11-16 20:41:12 +09:00
hakabahitoyo 52224de39f better extension detection 2018-11-16 20:22:36 +09:00
hakabahitoyo 4fbfacf5e1 debug 2018-11-15 16:08:55 +09:00
hakabahitoyo 8e707aba29 format 2018-11-15 15:11:59 +09:00
Hakaba Hitoyo ebe658c169 debuf 2018-11-15 14:46:43 +09:00
Hakaba Hitoyo 698cb3587c omplement mdii uploader 2018-11-15 14:38:45 +09:00
Hakaba Hitoyo 58af0787be add mdii uploader 2018-11-15 14:19:10 +09:00
href 5bb88fd174
Runtime configuration
Related to #85

Everything should now be configured at runtime, with the exception of
the `Pleroma.HTML` scrubbers (the scrubbers used can be
changed at runtime, but their configuration is compile-time) because
it's building a module with a macro.
2018-11-06 19:41:15 +01:00
William Pitcock 36825932eb s3 uploader: add new feature to force public attachment URIs to go through media proxy 2018-10-29 18:00:59 +00:00
Martin Kühl f77ec96707 Uploaders.S3: Replace unsafe characters in object key
According to [the S3 docs][s3], the characters safe for use in object keys are:

* 0-9
* a-z
* A-Z
* !
* -
* _
* .
* *
* '
* (
* )

(The / character is not listed but mentioned being safe outside of the list.)

Several characters that are valid in filenames can cause problems, for example
spaces are not valid in URLs and need to be escaped,
sequences of spaces can become squeezed by S3,
some characters like \ are documented to require “significant special handling”.

To avoid these problems, this change encodes the filename
before using it as part of the S3 object name
by replacing all characters except those documented as “safe” with dashes.

[s3]: https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html
2018-09-24 15:38:32 +02:00
Haelwenn (lanodan) Monnier 754deb26dd
[Pleroma.Uploaders.Local]: Add configuration for custom url path
One use-case being an external caching proxy
2018-09-02 19:00:16 +02:00
Thurloat adffad5502 increase uploader behaviour documentation accuracy. 2018-08-30 09:20:29 -03:00
Thurloat af01f0196a Add backend failure handling with :ok | :error so the uploader can handle it.
defaulting to :ok, since that's the currently level of error handling.
2018-08-29 22:07:28 -03:00
Thurloat 2ff25ac0ce A hobbldey-working swift client.
apparently, all elixir openstack libraries are trash
luckily, the APIs are stupid easy.
2018-08-28 22:32:24 -03:00
Thurloat dad39b24a1 add the behaviour, work on actually making it work. 2018-08-28 19:48:03 -03:00
Thurloat 8d2d7a8859 Implement uploader behaviour
run formatter <#
2018-08-28 09:57:41 -03:00