f66a15c4a5
RichMedia parser: do not set a cache TTL for unchanging errors
2020-09-14 14:44:25 +03:00
170599c390
RichMedia: do not log webpages missing metadata as errors
...
Also fixes the return value of Parser.parse on errors, previously
was just `:ok` due to the logger call in the end
2020-09-05 22:05:35 +03:00
19691389b9
Rich media: Add failure tracking
2020-09-02 14:59:52 +03:00
03d06062ab
don't fail on url fetch
2020-09-01 19:39:07 +03:00
016d8d6c56
Consolidate construction of Rich Media Parser HTTP requests
2020-08-03 12:37:31 -05:00
18438a9bf0
Add "Bot" to User Agent to coerce Twitter into serving OGP <meta> tags.
2020-07-07 11:21:05 -05:00
58e4e3db8b
Merge remote-tracking branch 'origin/develop' into merge-ogp-twitter-parsers
2020-06-15 16:03:40 +04:00
520367d6fd
Fix atom leak in Rich Media Parser
2020-06-13 12:08:46 +03:00
1f35acce54
Merge OGP parser with TwitterCard
2020-06-11 17:57:31 +04:00
509c81e4b1
Merge branch 'develop' into gun
2020-03-03 10:08:07 +03:00
6da6540036
Bump copyright years of files changed after 2020-01-07
...
Done via the following command:
git diff fcd5dd259a
2020-03-02 06:08:45 +01:00
cf94349287
Merge branch 'develop' into gun
2020-02-18 09:06:27 -06:00
514c899275
adding gun adapter
2020-02-18 08:19:01 +03:00
472132215e
Use floki's new APIs for parsing fragments
2020-02-16 01:55:26 +03:00
237b2068f9
Revert "Merge branch 'feat/floki-fasthtml' into 'develop'"
...
This reverts merge request !2194
2020-02-11 16:55:18 +00:00
ea1631d7e6
Make Floki use fast_html
2020-02-11 16:17:21 +03:00
b4cf74c106
added prepare html for RichMedia.Parser
2019-09-15 14:53:58 +03:00
d3bdb8e704
rich media: parser: splice the given URL into the result
2019-07-23 23:51:29 +00:00
3368174785
Fix rich media parser failing when no TTL can be found by image TTL
...
setters
2019-07-21 18:22:22 +03:00
de9906ad56
change the structure of image ttl parsar
2019-07-19 11:43:42 +05:45
18234cc44e
add the rich media ttl based on image exp time
2019-07-17 00:20:34 +05:45
f4447d82b8
parsers configurable
2019-07-14 09:21:56 +03:00
92213fb87c
Replace Mix.env with Pleroma.Config.get(:env)
...
Mix.env/0 is not availible in release environments such as distillery or
elixir's built-in releases.
2019-06-06 23:59:51 +03:00
1690be991e
Replace missing non-nullable Card attributes with empty strings
2019-05-30 21:03:31 +00:00
4615e56219
Add `with_body: true` to requests relying on `max_body: val`
2019-04-12 00:16:33 +07:00
19afd9f81f
http: rework connection timeouts to match hackney docs, enforce 1 second max TCP connection timeout
2019-03-08 22:56:16 +00:00
45e57dd187
rich media: tighten fetching timeouts and size limits
2019-02-10 21:54:08 +00:00
d83dbd9070
rich media: parser: reject any data which cannot be explicitly encoded into JSON
2019-02-05 20:50:57 +00:00
46dba03098
rich media: parser: only try to validate strings, not numbers (OEmbed)
2019-01-31 16:19:31 +00:00
dafb6f0b5e
rich media: parser: reject OGP fields we cannot safely process
2019-01-31 16:03:56 +00:00
5ea0397e2d
Fix 4aff4efa typos
2019-01-30 21:08:41 +01:00
4aff4efa8d
Use multiple hackney pools
...
* federation (ap, salmon)
* media (rich media, media proxy)
* upload (uploader proxy)
Each "part" will stop fighting others ones -- a huge federation outbound
could before make the media proxy fail to checkout a connection in time.
splitted media and uploaded media for the good reason than an upload
pool will have all connections to the same host (the uploader upstream).
it also has a longer default retention period for connections.
2019-01-30 15:06:46 +01:00
0f11254a06
rich media: parser: add some basic sanity checks on the returned data with pattern matching
2019-01-28 20:43:21 +00:00
83b7062634
rich media: parser: cache negatives
2019-01-28 20:19:07 +00:00
8fb16e9f0f
rich media: parser: add copyright header
2019-01-28 20:00:01 +00:00
de42646634
rich media: add try/rescue to ensure we catch parsing and fetching failures
2019-01-28 05:53:17 +00:00
8f2f471e94
rich media: gracefully handle fetching nil URIs
2019-01-26 16:36:17 +00:00
b8a77c5d70
Add OEmbed parser
2019-01-13 02:06:50 +02:00
1f851a0723
Add Twitter Card parser
2019-01-10 18:09:56 +00:00
a2d7f0e0e9
Remove :commit since a tuple is already returned
2019-01-09 21:35:01 +03:00
487c00d36d
rich media: disable cachex in test mode
2019-01-04 23:53:26 +00:00
0964c207eb
rich media: use cachex to avoid flooding remote servers
2019-01-04 23:32:01 +00:00
48e81d3d40
Add RichMediaController and tests
2019-01-02 17:02:50 +03:00
2aab4e03c3
Add OGP parser
2019-01-01 23:26:40 +03:00
rinpatch