moon
/
sync
mirror of https://github.com/calzoneman/sync.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,434 Commits 12 Branches 9 Tags 14 MiB
Commit Graph

398 Commits

Author SHA1 Message Date
Calvin Montgomery 49661a95ab Upgrade dependencies 2018-02-15 19:58:33 -08:00
Calvin Montgomery 966da1ac58 Revert "Replace quadratic emote list impl with Map" 
This reverts commit 0f9bc44925.

The original commit was not backwards compatible with use cases that
users were relying on, such as emotes being sorted in insertion order by
default.

I will develop a new patch which fixes the performance issue in a
backwards compatible way.
 2018-02-01 17:39:45 -08:00
Calvin Montgomery aeab31825e Fix a raw file error caused by facebook CDN violating RFC 2616 2018-01-21 18:53:16 -08:00
Calvin Montgomery e7781b5c09 Remove accidentally committed script 2018-01-18 19:48:32 -08:00
Calvin Montgomery 8821de0e7d Try to reduce the extra crap logged when a db query fails 2018-01-18 19:47:55 -08:00
Calvin Montgomery 0f9bc44925 Replace quadratic emote list impl with Map 2018-01-18 19:34:57 -08:00
Calvin Montgomery 8399eab33f Fix error on invalid regex for /clean 2018-01-17 21:54:43 -08:00
Calvin Montgomery 1797e11b43 Sanitize google drive IDs to remove URL hash etc. 2018-01-14 15:15:59 -08:00
Calvin Montgomery 46a738b7f4 Minor tweak to playlist dirty check 2018-01-14 15:08:55 -08:00
Calvin Montgomery d706bf63b1 Fix ustream 2018-01-14 15:02:15 -08:00
Calvin Montgomery e350eb731b Fix #728 2018-01-07 15:14:20 -08:00
Calvin Montgomery cf9b95a265 Add experimental support for dash streaming 2018-01-06 11:00:59 -08:00
Calvin Montgomery 67fbc8e267 Add more information to the voteskip passed log message 2018-01-06 10:31:59 -08:00
Calvin Montgomery 4b48966e1d Add ffprobe errors for ECONNRESET and CERT_HAS_EXPIRED 2018-01-06 10:13:07 -08:00
Calvin Montgomery 6d0498987a Add sanity check for one instance of error unload 
Unfortunately I think this is just one of a whole class of race
conditions caused by errored channels being unloaded immediately without
waiting for the refcounter to reach 0.

However, this one is the only one that appears commonly in the logs so
adding this check should buy time to rethink the overall problem.
 2018-01-06 10:09:22 -08:00
Calvin Montgomery 78bffad888 Fix errored channels getting stuck during unload 2018-01-06 09:59:18 -08:00
Zynjec 7b328b10c3 Removed Vidme provider from getInfo 
Loading a channel fails due to it being removed.

`Uncaught exception: Error: Cannot find module 'cytube-mediaquery/lib/provider/vidme'`
 2017-12-27 18:15:21 -08:00
Calvin Montgomery 95e147b5a0 Use socket.handshake instead of socket.client.request 
Fixes a bug where sockets would be rejected if they connected directly
with the 'websocket' transport instead of doing an AJAX connection with
websocket upgrade (e.g. if `transports: ['websocket']` is passed to the
socket.io-client constructor).

See https://github.com/socketio/socket.io/blob/master/docs/API.md#sockethandshake
 2017-12-27 14:24:33 -08:00
Calvin Montgomery 24322d3b52 Remove config option that is no longer used 2017-12-26 11:00:18 -08:00
Calvin Montgomery b7bc93f194 Disable vid.me (RIP) 2017-12-24 11:19:30 -08:00
Calvin Montgomery 0c330a82ce Add dirty check to playlist for efficiency of channel saving 2017-12-16 10:34:04 -08:00
Calvin Montgomery 7fbd62142e Minor tweaks/fixes 2017-12-15 19:10:32 -08:00
Calvin Montgomery 29be9233e9 Add check for weird setAFK edge case 2017-12-11 22:46:41 -08:00
Calvin Montgomery 1e969117c4 Fix #722 2017-12-10 19:28:05 -08:00
Calvin Montgomery fbee6d2ab7 Fix a few common causes of error logs (incl. better ffprobe error messages) 2017-12-10 16:39:06 -08:00
Calvin Montgomery c4cc22dd05 Add experimental feature to reduce database writes for channel data 2017-12-10 10:36:28 -08:00
Calvin Montgomery a9062159ed Fix partial saving for flatfile channel data 2017-12-10 09:48:40 -08:00
Calvin Montgomery 4e8c97bfb5 Fix deprecation warning about no callback to fs.unlink 2017-12-06 22:16:25 -08:00
Calvin Montgomery 39587a8448 Add DB query error count metric 2017-12-06 22:13:07 -08:00
Calvin Montgomery 9886f648f2 Workaround for #724 2017-12-06 22:10:06 -08:00
Calvin Montgomery 875337d9a6 web/account: add referrer check 2017-11-05 16:17:37 -08:00
Calvin Montgomery b876c8907a ffmpeg: preserve cookies when following redirects in pre-flight req 2017-11-05 16:01:39 -08:00
Calvin Montgomery 3cd8bfa8c7 Remove /sioconfig for real 2017-09-30 15:26:47 -07:00
Calvin Montgomery a2be65aead Reset prometheus summaries for more accurate percentiles per 5 minutes 2017-09-27 21:55:42 -07:00
Calvin Montgomery 014f3f008e Remove config key that is no longer used 2017-09-27 21:50:51 -07:00
Calvin Montgomery f975f7ef85 Update password reset to use new nodemailer impl 2017-09-26 21:22:15 -07:00
Calvin Montgomery 9cfe71d4c4 Start working on nodemailer upgrade 2017-09-25 22:31:45 -07:00
Calvin Montgomery bfc7cfc193 Remove old /useragreement 2017-09-19 22:07:00 -07:00
Calvin Montgomery 9868a97dbd Remove a couple config keys that are no longer used 2017-09-19 22:03:34 -07:00
Calvin Montgomery c159fa8060 Remove old HTTPS redirect kludges 2017-09-19 20:49:33 -07:00
Calvin Montgomery de5cc3352a Fix another bug with prometheus socket.io emtrics 2017-09-19 19:03:43 -07:00
Calvin Montgomery 9a1d50dcd3 Add support for v8-profiler (optional dep) 2017-09-18 21:54:36 -07:00
Calvin Montgomery 4db78deda3 Support updating profile via /account/data 2017-09-06 22:53:34 -07:00
Calvin Montgomery 9e3426633d Support updating email via /account/data 2017-09-05 23:11:28 -07:00
Calvin Montgomery 5b6f86668a Refactoring 2017-09-05 22:47:29 -07:00
Calvin Montgomery 3eb97bab6a Fix bug in cytube_sockets_num_connected metric 2017-09-04 10:04:33 -07:00
Calvin Montgomery 45d0e0b4c3 Guard unfinished web route with env variable 2017-09-03 17:22:57 -07:00
Calvin Montgomery b76869e2d2 Add some basic tests for implemented /account/data handlers 2017-09-01 21:20:07 -07:00
Calvin Montgomery 8b1b501bbd Start working on /account/data controller 2017-08-30 22:45:48 -07:00
Calvin Montgomery 33b2bc2d30 Add basic knex methods for channel data needed for /account/* 2017-08-29 21:23:04 -07:00
Calvin Montgomery 269aa6bfe6 Add basic knex methods to be used for /account/* pages 2017-08-28 23:37:32 -07:00
Calvin Montgomery 3d50b8f52e Fix getSafeReferrer when referrer is null 2017-08-24 20:55:18 -07:00
Calvin Montgomery cc69b3c225 Revert "Remove legacy /sioconfig and user agreement link" 
ACP has a dependency on `/sioconfig`.  Reverting until that can be
fixed.

This reverts commit a48cab81b9.
 2017-08-23 23:15:30 -07:00
Calvin Montgomery cacde7f72d Fix unhandled rejections in webserver 2017-08-23 23:02:08 -07:00
Calvin Montgomery 7e6312f9d1 Remove ?dest= redirect logic for /login and use referrer instead 2017-08-22 17:25:18 -07:00
Calvin Montgomery a48cab81b9 Remove legacy /sioconfig and user agreement link 
- `/sioconfig` has been deprecated for ages in favor of
    `/socketconfig/${channel}.json`
  - Each website administrator should be responsible for determining the
    appropriate terms of service for their website instead of CyTube
    providing a default one.
 2017-08-21 23:19:19 -07:00
Calvin Montgomery ae7098085c Work on knexifying password resets 2017-08-16 23:28:29 -07:00
Calvin Montgomery 791a712a68 Move channel register/delete reload logic to message bus 2017-08-15 18:55:36 -07:00
Calvin Montgomery d16cfb7328 Add message bus for #677 2017-08-15 18:23:03 -07:00
Calvin Montgomery 9ee650461f Change unhandledRejection from fatal log level to error 2017-08-14 20:35:30 -07:00
Calvin Montgomery 2990d83c02 ffmpeg: add ETIMEDOUT error message 2017-08-14 20:33:09 -07:00
Calvin Montgomery 99076412b6 Fix unhandled rejection 2017-08-14 20:31:45 -07:00
Calvin Montgomery c6c3bafca2 database: include legacy query() in prometheus metrics 2017-08-14 18:24:53 -07:00
Calvin Montgomery 82004aab73 ioserver: change on disconnect to once to avoid double-counting 2017-08-14 18:23:07 -07:00
Calvin Montgomery 70b875c0e9 Remove some ancient db upgrade stuff 
If anyone is still running a database from 2014 they want to upgrade,
sorry.
 2017-08-13 22:19:47 -07:00
Calvin Montgomery 4102d6eaf2 Refactor index.js logic into src/main 2017-08-13 22:16:42 -07:00
Calvin Montgomery a90d88ad65 Fix race condition that might be causing errors 2017-08-12 13:30:24 -07:00
Calvin Montgomery 8a8ed0a932 ffmpeg: better error messages for ECONNREFUSED and ENOTFOUND 2017-08-12 13:20:44 -07:00
Calvin Montgomery d0c1e8cbd9 Change metric names to follow prometheus naming guide 2017-08-12 13:12:58 -07:00
Calvin Montgomery 92f0a956b9 custom-media: import spec and fix a minor missed validation 2017-08-08 20:46:10 -07:00
Calvin Montgomery 04c9d48779 custom-media: implement queueing and playback changes 2017-08-08 20:35:17 -07:00
Calvin Montgomery a6de8731b3 custom-media: add metadata downloader 2017-08-07 22:37:56 -07:00
Calvin Montgomery f4ce2fe69d custom-media: add converter to CyTube Media object 2017-08-07 21:44:55 -07:00
Calvin Montgomery c7f7dcfed3 custom-media: use url.parse, not whatwg URL (node v6 compat) 2017-08-06 21:59:14 -07:00
Calvin Montgomery ea6e3f921f custom-media: add validator 
Initial work for #655
 2017-08-06 21:50:27 -07:00
Calvin Montgomery 331a4626a0 Fix borrow-rank 2017-08-06 20:42:33 -07:00
Calvin Montgomery 0b560f15a9 Add prometheus counter for changeMedia 2017-08-05 18:50:27 -07:00
Calvin Montgomery cb6cfc8455 Instrument some more metrics with prometheus 2017-08-02 21:24:44 -07:00
Calvin Montgomery 6043647cb7 Skip full user auth for most page renders 
Previously, the user's session cookie was being checked against the
database for all non-static requests.  However, this is not really
needed and wastes resources (and is slow).

For most page views (e.g. index, channel page), just parsing the value
of the cookie is sufficient:

  * The cookies are already HMAC signed, so tampering with them ought to
    be for all reasonable purposes, impossible.
  * Assuming the worst case, all a nefarious user could manage to do is
    change the text of the "Welcome, {user}" and cause a (non-functional)
    ACP link to appear clientside, both of which are already possible by
    using the Inspect Element tool.

For authenticated pages (currently, the ACP, and anything under
/account/), the full database check is still performed (for now).
 2017-08-01 21:40:26 -07:00
Calvin Montgomery 0118a6fb15 Refactor socket.io controller 2017-08-01 19:29:11 -07:00
Calvin Montgomery 107155a661 Stop knex from thrashing idle connections 2017-07-27 18:01:40 -07:00
Calvin Montgomery f593f7283c Replace alert() with modal for ACP password reset 
Some browsers (e.g. Chrome) don't allow copying text out of alert()
dialogs.
 2017-07-24 22:35:15 -07:00
Calvin Montgomery 5a78056c91 Some small refactoring 2017-07-24 22:08:26 -07:00
Calvin Montgomery 9dd0ee4446 Fix logger misreference in copied-over lualoader 2017-07-22 11:44:33 -07:00
Calvin Montgomery 08a42f6739 ffmpeg: add specific error for invalid SSL cert 2017-07-22 11:32:43 -07:00
Calvin Montgomery 282ad986b6 Deprecate legacy vimeo-oauth lookup 2017-07-22 11:14:29 -07:00
Calvin Montgomery 52030506b5 deps: remove status-message-polyfill 
This hasn't been necessary since node v0.10, and CyTube only supports
node v6.x+ by this point.
 2017-07-22 10:45:36 -07:00
Calvin Montgomery a8f1e48157 ffmpeg: remove bitrate and codec warning 
Browsers which don't support CyTube's limited subset of
generally-supported codecs probably aren't worth warning about.

1Mbps is way too low of a threshold to warn about bandwidth, but even if
the threshold for warning were raised, it's probably still not that
useful.
 2017-07-22 10:43:18 -07:00
Calvin Montgomery ffde151ebd Make redis announcement channel configurable 
Finally fix the bug where announcements bleed across beta & live due to
sharing a redis pubsub channel.
 2017-07-22 10:41:22 -07:00
Calvin Montgomery 964feb7243 Add id field to announcements and hide previously closed announcements 2017-07-22 10:35:45 -07:00
Calvin Montgomery ff3ececc36 Copy utils from cytube-common and remove dep 
The `cytube-common` module was created as part of a now-defunct
experiment and since then has just remained a crufty container for a few
utils.  Moved the utils to the main repo and removed the dependency.
 2017-07-19 20:47:02 -07:00
Calvin Montgomery e780e7dadb Deprecate stats table in favor of prometheus integration 2017-07-17 21:58:58 -07:00
Calvin Montgomery c7bec6251e Begin prometheus integration 
Add a dependency on `prom-client` and emit a basic latency metric for
testing purposes.  Add a new configuration file for enabling/disabling
prometheus exporter and configuring the listen address.
 2017-07-16 22:35:33 -07:00
Calvin Montgomery c152a19624 Ignore library cached metadata when queueing 
The use of the channel library as a cache for metadata to avoid
re-requesting metadata for known media is an optimization that dates
back to 1.0.  However, it doesn't have any TTL, is prone to bugs, and is
of dubious value.

This commit ignores the results of the library check when queueing a new
video, opting to always re-request the metadata.  This fixes a few bugs:

  * Google Drive metadata being lost when storing in library
  * Streamable metadata being lost when storing in library
  * Videos in the channel library that are now unavailable on their
    source website being queueable and then failing to play (e.g. deleted
    YouTube videos).

In its place, a small fail-open check is left behind to emit metric
counters on how many queues would have been cache-hits, to provide
insight into whether a proper caching solution (i.e. one not tacked on
top of the library) would be worth pursuing or not.  This will be
removed eventually.
 2017-07-15 14:41:37 -07:00
Calvin Montgomery b7ceee8ef4 Fix video sources being lost when playlist is saved 2017-07-15 14:12:32 -07:00
Calvin Montgomery fc66e758ac Minor fix 2017-07-09 22:40:09 -07:00
Calvin Montgomery 637bcad816 camo: include subdomains of whitelisted domains in whitelist 2017-07-08 20:46:42 -07:00
Calvin Montgomery 07179d6c83 Upgrade to jsli 2.0 2017-07-08 20:11:54 -07:00
Calvin Montgomery 486ce04a3e camo: support URL encoding option 2017-07-08 19:21:14 -07:00
Calvin Montgomery 54045766f2 Replace instances of cytube-common logger with jsli 2017-07-02 22:38:54 -07:00