moon
/
sync
mirror of https://github.com/calzoneman/sync.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,532 Commits 12 Branches 9 Tags 14 MiB
Commit Graph

440 Commits

Author SHA1 Message Date
Calvin Montgomery 46a738b7f4 Minor tweak to playlist dirty check 2018-01-14 15:08:55 -08:00
Calvin Montgomery d706bf63b1 Fix ustream 2018-01-14 15:02:15 -08:00
Calvin Montgomery e350eb731b Fix #728 2018-01-07 15:14:20 -08:00
Calvin Montgomery cf9b95a265 Add experimental support for dash streaming 2018-01-06 11:00:59 -08:00
Calvin Montgomery 67fbc8e267 Add more information to the voteskip passed log message 2018-01-06 10:31:59 -08:00
Calvin Montgomery 4b48966e1d Add ffprobe errors for ECONNRESET and CERT_HAS_EXPIRED 2018-01-06 10:13:07 -08:00
Calvin Montgomery 6d0498987a Add sanity check for one instance of error unload 
Unfortunately I think this is just one of a whole class of race
conditions caused by errored channels being unloaded immediately without
waiting for the refcounter to reach 0.

However, this one is the only one that appears commonly in the logs so
adding this check should buy time to rethink the overall problem.
 2018-01-06 10:09:22 -08:00
Calvin Montgomery 78bffad888 Fix errored channels getting stuck during unload 2018-01-06 09:59:18 -08:00
Zynjec 7b328b10c3 Removed Vidme provider from getInfo 
Loading a channel fails due to it being removed.

`Uncaught exception: Error: Cannot find module 'cytube-mediaquery/lib/provider/vidme'`
 2017-12-27 18:15:21 -08:00
Calvin Montgomery 95e147b5a0 Use socket.handshake instead of socket.client.request 
Fixes a bug where sockets would be rejected if they connected directly
with the 'websocket' transport instead of doing an AJAX connection with
websocket upgrade (e.g. if `transports: ['websocket']` is passed to the
socket.io-client constructor).

See https://github.com/socketio/socket.io/blob/master/docs/API.md#sockethandshake
 2017-12-27 14:24:33 -08:00
Calvin Montgomery 24322d3b52 Remove config option that is no longer used 2017-12-26 11:00:18 -08:00
Calvin Montgomery b7bc93f194 Disable vid.me (RIP) 2017-12-24 11:19:30 -08:00
Calvin Montgomery 0c330a82ce Add dirty check to playlist for efficiency of channel saving 2017-12-16 10:34:04 -08:00
Calvin Montgomery 7fbd62142e Minor tweaks/fixes 2017-12-15 19:10:32 -08:00
Calvin Montgomery 29be9233e9 Add check for weird setAFK edge case 2017-12-11 22:46:41 -08:00
Calvin Montgomery 1e969117c4 Fix #722 2017-12-10 19:28:05 -08:00
Calvin Montgomery fbee6d2ab7 Fix a few common causes of error logs (incl. better ffprobe error messages) 2017-12-10 16:39:06 -08:00
Calvin Montgomery c4cc22dd05 Add experimental feature to reduce database writes for channel data 2017-12-10 10:36:28 -08:00
Calvin Montgomery a9062159ed Fix partial saving for flatfile channel data 2017-12-10 09:48:40 -08:00
Calvin Montgomery 4e8c97bfb5 Fix deprecation warning about no callback to fs.unlink 2017-12-06 22:16:25 -08:00
Calvin Montgomery 39587a8448 Add DB query error count metric 2017-12-06 22:13:07 -08:00
Calvin Montgomery 9886f648f2 Workaround for #724 2017-12-06 22:10:06 -08:00
Calvin Montgomery 875337d9a6 web/account: add referrer check 2017-11-05 16:17:37 -08:00
Calvin Montgomery b876c8907a ffmpeg: preserve cookies when following redirects in pre-flight req 2017-11-05 16:01:39 -08:00
Calvin Montgomery 3cd8bfa8c7 Remove /sioconfig for real 2017-09-30 15:26:47 -07:00
Calvin Montgomery a2be65aead Reset prometheus summaries for more accurate percentiles per 5 minutes 2017-09-27 21:55:42 -07:00
Calvin Montgomery 014f3f008e Remove config key that is no longer used 2017-09-27 21:50:51 -07:00
Calvin Montgomery f975f7ef85 Update password reset to use new nodemailer impl 2017-09-26 21:22:15 -07:00
Calvin Montgomery 9cfe71d4c4 Start working on nodemailer upgrade 2017-09-25 22:31:45 -07:00
Calvin Montgomery bfc7cfc193 Remove old /useragreement 2017-09-19 22:07:00 -07:00
Calvin Montgomery 9868a97dbd Remove a couple config keys that are no longer used 2017-09-19 22:03:34 -07:00
Calvin Montgomery c159fa8060 Remove old HTTPS redirect kludges 2017-09-19 20:49:33 -07:00
Calvin Montgomery de5cc3352a Fix another bug with prometheus socket.io emtrics 2017-09-19 19:03:43 -07:00
Calvin Montgomery 9a1d50dcd3 Add support for v8-profiler (optional dep) 2017-09-18 21:54:36 -07:00
Calvin Montgomery 4db78deda3 Support updating profile via /account/data 2017-09-06 22:53:34 -07:00
Calvin Montgomery 9e3426633d Support updating email via /account/data 2017-09-05 23:11:28 -07:00
Calvin Montgomery 5b6f86668a Refactoring 2017-09-05 22:47:29 -07:00
Calvin Montgomery 3eb97bab6a Fix bug in cytube_sockets_num_connected metric 2017-09-04 10:04:33 -07:00
Calvin Montgomery 45d0e0b4c3 Guard unfinished web route with env variable 2017-09-03 17:22:57 -07:00
Calvin Montgomery b76869e2d2 Add some basic tests for implemented /account/data handlers 2017-09-01 21:20:07 -07:00
Calvin Montgomery 8b1b501bbd Start working on /account/data controller 2017-08-30 22:45:48 -07:00
Calvin Montgomery 33b2bc2d30 Add basic knex methods for channel data needed for /account/* 2017-08-29 21:23:04 -07:00
Calvin Montgomery 269aa6bfe6 Add basic knex methods to be used for /account/* pages 2017-08-28 23:37:32 -07:00
Calvin Montgomery 3d50b8f52e Fix getSafeReferrer when referrer is null 2017-08-24 20:55:18 -07:00
Calvin Montgomery cc69b3c225 Revert "Remove legacy /sioconfig and user agreement link" 
ACP has a dependency on `/sioconfig`.  Reverting until that can be
fixed.

This reverts commit a48cab81b9.
 2017-08-23 23:15:30 -07:00
Calvin Montgomery cacde7f72d Fix unhandled rejections in webserver 2017-08-23 23:02:08 -07:00
Calvin Montgomery 7e6312f9d1 Remove ?dest= redirect logic for /login and use referrer instead 2017-08-22 17:25:18 -07:00
Calvin Montgomery a48cab81b9 Remove legacy /sioconfig and user agreement link 
- `/sioconfig` has been deprecated for ages in favor of
    `/socketconfig/${channel}.json`
  - Each website administrator should be responsible for determining the
    appropriate terms of service for their website instead of CyTube
    providing a default one.
 2017-08-21 23:19:19 -07:00
Calvin Montgomery ae7098085c Work on knexifying password resets 2017-08-16 23:28:29 -07:00
Calvin Montgomery 791a712a68 Move channel register/delete reload logic to message bus 2017-08-15 18:55:36 -07:00
Calvin Montgomery d16cfb7328 Add message bus for #677 2017-08-15 18:23:03 -07:00
Calvin Montgomery 9ee650461f Change unhandledRejection from fatal log level to error 2017-08-14 20:35:30 -07:00
Calvin Montgomery 2990d83c02 ffmpeg: add ETIMEDOUT error message 2017-08-14 20:33:09 -07:00
Calvin Montgomery 99076412b6 Fix unhandled rejection 2017-08-14 20:31:45 -07:00
Calvin Montgomery c6c3bafca2 database: include legacy query() in prometheus metrics 2017-08-14 18:24:53 -07:00
Calvin Montgomery 82004aab73 ioserver: change on disconnect to once to avoid double-counting 2017-08-14 18:23:07 -07:00
Calvin Montgomery 70b875c0e9 Remove some ancient db upgrade stuff 
If anyone is still running a database from 2014 they want to upgrade,
sorry.
 2017-08-13 22:19:47 -07:00
Calvin Montgomery 4102d6eaf2 Refactor index.js logic into src/main 2017-08-13 22:16:42 -07:00
Calvin Montgomery a90d88ad65 Fix race condition that might be causing errors 2017-08-12 13:30:24 -07:00
Calvin Montgomery 8a8ed0a932 ffmpeg: better error messages for ECONNREFUSED and ENOTFOUND 2017-08-12 13:20:44 -07:00
Calvin Montgomery d0c1e8cbd9 Change metric names to follow prometheus naming guide 2017-08-12 13:12:58 -07:00
Calvin Montgomery 92f0a956b9 custom-media: import spec and fix a minor missed validation 2017-08-08 20:46:10 -07:00
Calvin Montgomery 04c9d48779 custom-media: implement queueing and playback changes 2017-08-08 20:35:17 -07:00
Calvin Montgomery a6de8731b3 custom-media: add metadata downloader 2017-08-07 22:37:56 -07:00
Calvin Montgomery f4ce2fe69d custom-media: add converter to CyTube Media object 2017-08-07 21:44:55 -07:00
Calvin Montgomery c7f7dcfed3 custom-media: use url.parse, not whatwg URL (node v6 compat) 2017-08-06 21:59:14 -07:00
Calvin Montgomery ea6e3f921f custom-media: add validator 
Initial work for #655
 2017-08-06 21:50:27 -07:00
Calvin Montgomery 331a4626a0 Fix borrow-rank 2017-08-06 20:42:33 -07:00
Calvin Montgomery 0b560f15a9 Add prometheus counter for changeMedia 2017-08-05 18:50:27 -07:00
Calvin Montgomery cb6cfc8455 Instrument some more metrics with prometheus 2017-08-02 21:24:44 -07:00
Calvin Montgomery 6043647cb7 Skip full user auth for most page renders 
Previously, the user's session cookie was being checked against the
database for all non-static requests.  However, this is not really
needed and wastes resources (and is slow).

For most page views (e.g. index, channel page), just parsing the value
of the cookie is sufficient:

  * The cookies are already HMAC signed, so tampering with them ought to
    be for all reasonable purposes, impossible.
  * Assuming the worst case, all a nefarious user could manage to do is
    change the text of the "Welcome, {user}" and cause a (non-functional)
    ACP link to appear clientside, both of which are already possible by
    using the Inspect Element tool.

For authenticated pages (currently, the ACP, and anything under
/account/), the full database check is still performed (for now).
 2017-08-01 21:40:26 -07:00
Calvin Montgomery 0118a6fb15 Refactor socket.io controller 2017-08-01 19:29:11 -07:00
Calvin Montgomery 107155a661 Stop knex from thrashing idle connections 2017-07-27 18:01:40 -07:00
Calvin Montgomery f593f7283c Replace alert() with modal for ACP password reset 
Some browsers (e.g. Chrome) don't allow copying text out of alert()
dialogs.
 2017-07-24 22:35:15 -07:00
Calvin Montgomery 5a78056c91 Some small refactoring 2017-07-24 22:08:26 -07:00
Calvin Montgomery 9dd0ee4446 Fix logger misreference in copied-over lualoader 2017-07-22 11:44:33 -07:00
Calvin Montgomery 08a42f6739 ffmpeg: add specific error for invalid SSL cert 2017-07-22 11:32:43 -07:00
Calvin Montgomery 282ad986b6 Deprecate legacy vimeo-oauth lookup 2017-07-22 11:14:29 -07:00
Calvin Montgomery 52030506b5 deps: remove status-message-polyfill 
This hasn't been necessary since node v0.10, and CyTube only supports
node v6.x+ by this point.
 2017-07-22 10:45:36 -07:00
Calvin Montgomery a8f1e48157 ffmpeg: remove bitrate and codec warning 
Browsers which don't support CyTube's limited subset of
generally-supported codecs probably aren't worth warning about.

1Mbps is way too low of a threshold to warn about bandwidth, but even if
the threshold for warning were raised, it's probably still not that
useful.
 2017-07-22 10:43:18 -07:00
Calvin Montgomery ffde151ebd Make redis announcement channel configurable 
Finally fix the bug where announcements bleed across beta & live due to
sharing a redis pubsub channel.
 2017-07-22 10:41:22 -07:00
Calvin Montgomery 964feb7243 Add id field to announcements and hide previously closed announcements 2017-07-22 10:35:45 -07:00
Calvin Montgomery ff3ececc36 Copy utils from cytube-common and remove dep 
The `cytube-common` module was created as part of a now-defunct
experiment and since then has just remained a crufty container for a few
utils.  Moved the utils to the main repo and removed the dependency.
 2017-07-19 20:47:02 -07:00
Calvin Montgomery e780e7dadb Deprecate stats table in favor of prometheus integration 2017-07-17 21:58:58 -07:00
Calvin Montgomery c7bec6251e Begin prometheus integration 
Add a dependency on `prom-client` and emit a basic latency metric for
testing purposes.  Add a new configuration file for enabling/disabling
prometheus exporter and configuring the listen address.
 2017-07-16 22:35:33 -07:00
Calvin Montgomery c152a19624 Ignore library cached metadata when queueing 
The use of the channel library as a cache for metadata to avoid
re-requesting metadata for known media is an optimization that dates
back to 1.0.  However, it doesn't have any TTL, is prone to bugs, and is
of dubious value.

This commit ignores the results of the library check when queueing a new
video, opting to always re-request the metadata.  This fixes a few bugs:

  * Google Drive metadata being lost when storing in library
  * Streamable metadata being lost when storing in library
  * Videos in the channel library that are now unavailable on their
    source website being queueable and then failing to play (e.g. deleted
    YouTube videos).

In its place, a small fail-open check is left behind to emit metric
counters on how many queues would have been cache-hits, to provide
insight into whether a proper caching solution (i.e. one not tacked on
top of the library) would be worth pursuing or not.  This will be
removed eventually.
 2017-07-15 14:41:37 -07:00
Calvin Montgomery b7ceee8ef4 Fix video sources being lost when playlist is saved 2017-07-15 14:12:32 -07:00
Calvin Montgomery fc66e758ac Minor fix 2017-07-09 22:40:09 -07:00
Calvin Montgomery 637bcad816 camo: include subdomains of whitelisted domains in whitelist 2017-07-08 20:46:42 -07:00
Calvin Montgomery 07179d6c83 Upgrade to jsli 2.0 2017-07-08 20:11:54 -07:00
Calvin Montgomery 486ce04a3e camo: support URL encoding option 2017-07-08 19:21:14 -07:00
Calvin Montgomery 54045766f2 Replace instances of cytube-common logger with jsli 2017-07-02 22:38:54 -07:00
Calvin Montgomery 00901f9cdb Remove junk from an old abandoned project 2017-07-02 22:35:12 -07:00
Calvin Montgomery 860775a90b Remove html5hack (legacy google drive setting) 2017-07-02 22:30:19 -07:00
Xaekai 18bf1b946b Minor fixes to afk stuff. 2017-06-29 19:04:49 -07:00
Calvin Montgomery 7ebf3c18ab Add knex AliasesDB 2017-06-28 22:58:40 -07:00
Calvin Montgomery 76e0d1b7ec Use `proxy-addr` for parsing x-forwarded-for 
Closes #683 by providing functionality to trust proxies other than
localhost.
 2017-06-27 23:37:18 -07:00
Xaekai 5f71c4d368 Send shadowmuted messages to anons 
Resolves #689
 2017-06-20 22:29:27 -07:00
Calvin Montgomery a96f7976d8 Change Tor exit list 
Use the endpoint suggested in #688 to avoid unnecessarilly banning
relays.
 2017-06-17 10:12:15 -07:00
Calvin Montgomery 6161f4ad44 Add ffmpeg error log for request failure case 2017-06-17 09:47:22 -07:00
Calvin Montgomery 53cee986c6 Resend userlist if rank changes meta visibility 
Fixes #681.  Technically, resending the entire userlist is not
necessary; it would be sufficient to resent setUserMeta, but there's not
currently a bulk frame for that so sending the userlist is probably more
efficient.
 2017-06-17 09:47:22 -07:00
Xaekai be8318f014 Fix a typo. 2017-06-16 21:50:17 -07:00
Calvin Montgomery 33f632036e Merge pull request #684 from Xaekai/afk.meta 
Single source of truth for AFK
 2017-06-16 21:42:23 -07:00
Xaekai 6d4558c978 Allow channel path to be customizable 
We now allow server operators to customize the /r/ part of the channel links
The new config option in the template is commented and the config module validates and will terminate with status 78 if an improper value is used.
We've also dropped some old cruft and uses a more elegant method to assign CHANNEL.name

Resolves #668
 2017-06-16 20:09:36 -07:00
Xaekai f89832a6d1 Gracefully allow script authors time to update their code 2017-06-15 22:09:09 -07:00
Xaekai df0fc769d9 Single source of truth for AFK 
Resolves #678
 2017-06-15 21:48:17 -07:00
Calvin Montgomery 00a65a1584 Deprecate legacy global ban junk 2017-06-05 23:18:20 -07:00
Calvin Montgomery b23a858a8c Integrate socket.io ban check with GlobalBanDB 2017-06-05 23:14:45 -07:00
Calvin Montgomery ed811db6ec Integrate ACP with GlobalBanDB class 2017-06-05 22:53:35 -07:00
Calvin Montgomery b80a87ba01 Add integration test for global bans 2017-06-05 22:45:14 -07:00
Calvin Montgomery 8ad9b4e543 Remove redundant template local 2017-06-05 18:57:04 -07:00
Calvin Montgomery 58c65a5bac Use host header instead of req.host which is actually req.hostname 2017-06-05 18:53:36 -07:00
Calvin Montgomery 830486bc4f Fix channel registration error due to extra knex query param 2017-06-05 18:46:41 -07:00
Xaekai 699aa2abe1 Finish validation touchup 2017-06-05 00:06:15 -07:00
Xaekai d42de93d74 Round 2. 2017-06-04 22:16:40 -07:00
Calvin Montgomery d0712d007e Work on refactoring global IP ban database calls 2017-05-31 22:46:15 -07:00
Calvin Montgomery 7fcf31dec6 Merge pull request #671 from calzoneman/knex 
The knexening: part 1
 2017-05-29 13:16:35 -07:00
Calvin Montgomery 2a694e73af The knexening: part 1 2017-05-28 22:39:27 -07:00
Calvin Montgomery 22a9acfc90 Support proxying chat images via camo 
Camo: https://github.com/atmos/camo.  This has a couple advantages over
just allowing images to be dumped as-is:

  - Prevents mixed-content warnings by allowing the server to proxy HTTP
    images to an HTTPS camo instance
  - Protects users' privacy by not exposing their browser directly to
    the image host
  - Allows the camo proxy to intercept and reject bad image sources
    (URLs that are not actually images, gigapixel-sized images likely to
    DoS users' browsers, etc.)

Whitelisting specific domains is supported for cases where the source is
known to be trustworthy.
 2017-05-28 19:38:43 -07:00
Calvin Montgomery f968521936 Remove google drive refresh logic 
No longer relevant since the video links are retrieved by the
userscript.
 2017-05-28 18:35:13 -07:00
Calvin Montgomery e9c519c6e2 Add twitch clip support (#659) 2017-05-27 11:49:43 -07:00
Xaekai 1d65eb036e Use a more salient variable name. 
Use a style thats readable with brevity.
 2017-05-24 19:18:47 -07:00
Xaekai 93ef067b8c [ACP] Allow searching users by email. 2017-05-24 04:44:55 -07:00
Calvin Montgomery d23b5278b1 Rename Hitbox -> Smashcast 2017-05-20 16:50:00 -07:00
Calvin Montgomery 55b03d51d7 Fix setOptions for playlist_max_duration_per_user 2017-05-20 16:31:52 -07:00
Xaekai 9cfd97088e Some validation 2017-05-18 20:12:00 -07:00
Xaekai 8434d20826 Fix minor issues with emote rename 2017-05-17 09:50:47 -07:00
Xaekai 8e3ce4e1c3 Emote renaming 
This allow emotes to be renamed in the same fashion the image URLs can be changed.
 2017-05-16 10:08:53 -07:00
Xaekai 8b95b9fc41 Fix UStream URL Generation 2017-05-14 04:02:32 -07:00
Calvin Montgomery de309d675e Remove redundant signing logic from IP session cookie 2017-05-01 21:51:11 -07:00
Calvin Montgomery 6bfbbc0c01 Support hot-swapping HTTPS certificates 2017-04-30 17:20:19 -07:00
Calvin Montgomery e92afcb203 Fix error logger to only log if error is non-null 2017-04-30 16:49:23 -07:00
Calvin Montgomery a0af0ccab5 Remove dead/commented-out code 2017-04-29 17:08:43 -07:00
Calvin Montgomery 089ac75e9a Fix DB purge of expired password reset reqs 
3 year old bug introduced when refactoring 2.x -> 3.0.
Never worked in the first place.
 2017-04-29 17:05:45 -07:00
Calvin Montgomery fac94d46a6 Bugfix: stringify first parameter to Logger.xxx() 2017-04-27 21:06:16 -07:00
Calvin Montgomery 5b58c30011 Fix TypeError 2017-04-24 22:31:51 -07:00
Calvin Montgomery ef9c744003 Use HTTPS in formatURL 2017-04-11 21:57:05 -07:00
Calvin Montgomery 8d40c87dda Deprecate jwplayer and googleplus videos 2017-04-11 21:55:31 -07:00
Calvin Montgomery 25c663c110 Fix node deprecation warning about calling fs.writeFile without cb 2017-04-04 23:12:02 -07:00
Calvin Montgomery 8306d2d1b6 Refactor logging 2017-04-04 23:02:31 -07:00
Calvin Montgomery b1a328d2e0 Implement max total video time per user 2017-04-03 21:18:40 -07:00
Calvin Montgomery f42e3bf2b7 Fix #656 2017-04-03 20:31:21 -07:00
Calvin Montgomery 5bdf8b4aaf Fix #657 2017-03-26 11:13:10 -07:00
Calvin Montgomery 7595faf11d Fix voteskip issue when there are no videos left 2017-03-21 20:04:06 -07:00
Calvin Montgomery 309cd40da2 Compare owner name case-insensitively when deleting channel 2017-03-21 19:47:31 -07:00
Calvin Montgomery 0613083eb0 Handle the case where no socket.io ack exists 2017-03-20 22:09:16 -07:00
Calvin Montgomery 9dc82ad444 Enforce stricter validation on polls 2017-03-20 21:37:32 -07:00
Calvin Montgomery 41a538c655 Fix playlist visibility: wait for U_HAS_CHANNEL_RANK instead of just login 2017-03-18 18:53:49 -07:00
Calvin Montgomery d8b9e3dab6 Merge pull request #648 from zeratul0/patch-1 
Change fallback voteskip ratio to .5 from 0
 2017-03-16 21:48:24 -07:00
zeratul0 88044e11d5 Error response -> template literal as requested 2017-03-16 23:34:38 -04:00