sync/NEWS.md

2022-09-21

Upgrade intervention required

This release adds a feature to ban channels, replacing the earlier (hastily added) configuration-based channel-blacklist. If you have any entries in channel-blacklist in your config.yaml, you will need to migrate them to the new bans table by using a command after upgrading (the ACP web interface hasn't been updated for this feature):

./bin/admin.js ban-channel <channel-name> <external-reason> <internal-reason>

The external reason will be displayed when users attempt to join the banned channel, while the internal reason is only displayed when using the show-channel-ban command.

You can later use unban-channel to remove a ban. The owner of the banned channel can still delete it, but the banned state will persist, so the channel cannot be re-registered later.

2022-08-28

This release integrates Xaekai's added support for Bandcamp, BitChute, Odysee, and Nicovideo playback support into the main repository. The updated support for custom fonts and audio tracks in custom media manifests is also included, but does not work out of the box -- it requires a separate channel script; this may be addressed in the future.

2021-08-14

CyTube has been upgraded to socket.io v4 (from v2).

Breaking change: Newer versions of socket.io require CORS to validate the origin initiating the socket connection. CyTube allows the origins specified in the io.domain and https.domain configuration keys by default, which should work for many use cases, however, if you host your website on a different domain than the socket connection, you will need to configure the allowed origins (see config.template.yaml under io.cors).

CyTube enables the allowEIO3 configuration in socket.io by default, which means that existing clients and bots using socket.io-client v2 should continue to work.

2021-08-12

The legacy metrics recorder (counters.log file) has been removed. For over 4 years now, CyTube has integrated with Prometheus, which provides a superior way to monitor the application. Copy conf/example/prometheus.toml to conf/prometheus.toml and edit it to configure CyTube's Prometheus support.

2021-08-12

Due to changes in Soundcloud's authorization scheme, support has been dropped from core due to requiring each server owner to register an API key (which is currently impossible as they have not accepted new API key registrations for years).

If you happen to already have an API key registered, or if Soundcloud reopens registration at some point in the future, feel free to reach out to me for patches to reintroduce support for it.

2020-08-21

Some of CyTube's dependencies depends on features in newer versions of node.js. Accordingly, node 10 is no longer supported. Administrators are recommended to use node 12 (the active LTS), or node 14 (the current version).

2020-06-22

Twitch has updated their embed player, which adds new requirements for embedding Twitch:

1. The origin website must be served over HTTPS
2. The origin website must be served over the default port (i.e., the hostname cannot include a port; https://example.com:8443 won't work)

Additionally, third-party cookies must be enabled for whatever internal subdomains Twitch is using.

CyTube now sets the parameters expected by Twitch, and displays an error message if it detects (1) or (2) above are not met.

2020-02-15

Old versions of CyTube defaulted to storing channel state in flatfiles located in the chandump directory. The default was changed a while ago, and the flatfile storage mechanism has now been removed.

Admins who have not already migrated their installation to the "database" channel storage type can do so by following these instructions:

1. Run git checkout e3a9915b454b32e49d3871c94c839899f809520a to temporarily switch to temporarily revert to the previous version of the code that supports the "file" channel storage type
2. Run npm run build-server to build the old version
3. Run node lib/channel-storage/migrator.js |& tee migration.log to migrate channel state from files to the database
4. Inspect the output of the migration tool for errors
5. Set channel-storage/type to "database" in config.yaml and start the server. Load a channel to verify the migration worked as expected
6. Upgrade back to the latest version with git checkout 3.0 and npm run build-server
7. Remove the channel-storage block from config.yaml and remove the chandump directory since it is no longer needed (you may wish to archive it somewhere in case you later discover the migration didn't work as expected).

If you encounter any errors during the process, please file an issue on GitHub and attach the output of the migration tool (which if you use the above commands will be written to migration.log).

2019-12-01

In accordance with node v8 LTS becoming end-of-life on 2019-12-31, CyTube no longer supports v8.

Please upgrade to v10 or v12 (active LTS); refer to https://nodejs.org/en/about/releases/ for the node.js support timelines.

2018-12-07

Users can now self-service request their account to be deleted, and it will be automatically purged after 7 days. In order to send a notification email to the user about the request, copy the email configuration to conf/email.toml (the same file used for password reset emails).

2018-10-21

The sanitize-html dependency has made a change that results in " no longer being replaced by " when not inside an HTML attribute value. This potentially breaks any chat filters matching quotes as " (on my particular instance, this seems to be quite rare). These filters will need to be updated in order to continue matching quotes.

2018-08-27

Support for node.js 6.x has been dropped, in order to bump the babel preset to generate more efficient code (8.x supports async-await and other ES6+ features natively and is the current node.js LTS).

If you are unable to upgrade to node.js 8.x, you can revert the changes to package.json in this commit, however, be warned that I no longer test on 6.x.

2018-06-03

Dependency upgrades

In order to support node.js 10, the bcrypt dependency has been upgraded to version 2. bcrypt version 2 defaults to the $2b$ algorithm, whereas version 1 defaults to the $2a$ algorithm. Existing password hashes will continue to be readable, however hashes created with version 2 will not be readable by version 1. See https://github.com/kelektiv/node.bcrypt.js for details.

In addition, the optional dependency on v8-profiler has been removed, since this is not compatible with newer versions of v8.

Supported node.js versions

In accordance with the node.js release schedule, node.js 4.x, 5.x, 7.x, and 9.x are end-of-life and are no longer maintained upstream. Accordingly, these versions are no longer supported by CyTube.

Please upgrade to 8.x (LTS) or 10.x (current). 6.x is still supported, but is in the "maintenance" phase upstream, and should be phased out.

2018-01-07

Build changes: When the babel dependency was first added to transpile ES6 code to ES5, an interactive prompt was added to the postinstall script before transpilation, in case the user had made local modifications to the files in lib which previously would have been detected as a git conflict when pulling.

It has now been sufficiently long that this is no longer needed, so I've removed it. As always, users wishing to make local modifications (or forks) should edit the code in src/ and run npm run build-server to regenerate lib/.

This commit also removes the bundled www/js/player.js file in favor of having postinstall generate it from the sources in player/.

2017-12-24

As of December 2017, Vid.me is no longer in service. Accordingly, Vid.me support in CyTube has been deprecated.

2017-11-27

The Google Drive userscript has been updated once again. Violentmonkey is now explicitly supported. Google login redirects are caught and handled. See directly below on how to regenerate the user script again.

2017-11-15

The Google Drive userscript has been updated due to breaking changes in Greasemonkey 4.0. Remember to generate the script by running:

$ npm run generate-userscript "Your Site Name" http://your-site.example.com/r/*

2017-11-05

The latest commit introduces a referrer check in the account page handlers. This is added as a short-term mitigation for a recent report that account management functions (such as deleting channels) can be executed without the user's consent if placed in channel JS.

Longer term options are being considered, such as moving account management to a separate subdomain to take advantage of cross-origin checks in browsers, and requiring the user to re-enter their password to demonstrate intent. As always, I recommend admins take extreme caution when accepting channel JS.

2017-09-26

Breaking change: the nodemailer dependency has been upgraded to version 4.x. I also took this opportunity to make some modifications to the email configuration and move it out of config.yaml to conf/email.toml.

To upgrade:

• Run npm upgrade (or rm -rf node_modules; npm install)
• Copy conf/example/email.toml to conf/email.toml
• Edit conf/email.toml to your liking
• Remove the mail: block from config.yaml

This feature only supports sending via SMTP for now. If there is demand for other transports, feel free to open an issue or submit a pull request.

2017-09-19

The /useragreement default page has been removed. Server administrators can substitute their own terms of service page by editing templates/footer.pug

2017-09-19

This commit removes an old kludge that redirected users to HTTPS (when enabled) specifically for the account authorization pages (e.g., /login). The code for doing this was to work around limitations that no longer exist, and does not represent current security best practices.

The recommended solution to ensure that users are logged in securely (assuming you've configured support for HTTPS) is to use Strict-Transport-Security to direct browsers to access the HTTPS version of the website at all times. You can enable this by configuring a reverse proxy (e.g. nginx) in front of CyTube to intercept HTTP traffic and redirect it to HTTPS, and add the Strict-Transport-Security header when returning the response from CyTube.

2017-07-22

Support for the old version of Vimeo's OAuth API (the vimeo-oauth configuration block) has been dropped. It's unlikely anyone was using this, since you haven't been able to register new API keys for it in years (it was superseded by a newer OAuth API, which CyTube does not support), and in fact I lost my credentials for this API and no longer have a way to test it.

Vimeo videos can still be added -- the metadata will be queried from the anonymous API which has been the default since the beginning.

2017-07-17

The stats database table and associated ACP subpage have been removed in favor of integration with Prometheus. You can enable Prometheus reporting by copying conf/example/prometheus.toml to conf/prometheus.toml and editing it to your liking. I recommend integrating Prometheus with Grafana for dashboarding needs.

The particular metrics that were saved in the stats table are reported by the following Prometheus metrics:

• Channel count: cytube_channels_num_active gauge.
• User count: cytube_sockets_num_connected gauge (labeled by socket.io transport).
• CPU/Memory: default metrics emitted by the prom-client module.

More Prometheus metrics will be added in the future to make CyTube easier to monitor :)

2017-07-15

The latest commit upgrades socket.io to version 2.0, a major version change from 1.4. This release improves performance by switching to uws for the websocket transport, and fixes several bugs; you can read about it here.

For browser clients, the upgrade should basically just work with no intervention. For node.js clients, all that is needed is to upgrade socket.io-client to 2.0. For other clients, work required may vary depending on whether the implementation has compatibility problems with 2.0.

2017-06-20

The latest commit drops support for node.js versions below 6 (the current LTS). This is to allow the babel preset to avoid generating inefficient code to polyfill ES2015+ features that are now implemented in the node.js core.

New versions of node.js can be downloaded from the node.js website, if they are not already available in your distribution's package manager.

2017-03-20

Polls are now more strictly validated, including the number of options. The default limit is 50 options, which you can configure via poll.max-options.

2017-03-11

Commit f8183bea1b modifes the schema of the users table to include a new column (name_dedupe) which has a UNIQUE constraint. This column is populated with a modified version of the user's name to prevent the registration of usernames which are bitwise distinct but visually similar. 'l', 'L', and '1' are all mapped to '1'; 'o', 'O', and '0' are all mapped to '0'; '_' and '-' are mapped to '_'. On first startup after upgrading, the new column will be added and populated.

This replaces the earlier solution which was put in place to mitigate PR#489 but was overly-restrictive since it wildcarded these characters against any character, not just characters in the same group.

2017-03-03

The dependency on sanitize-html, which previously pointed to a fork, has now been switched back to the upstream module. XSS filtering has been turned off for the chat filter replacement itself (since this provides no additional security), and is now only run on the final chat message after filtering. Certain chat filters and MOTDs which relied on syntactically incorrect HTML, such as unclosed tags, may have different behavior now, since sanitize-html fixes these.

2016-11-02

After upgrading the dependency on yamljs, you may see this error if you didn't notice and correct a typo in the config.yaml template:

Error loading config file config.yaml:
{ [Error: Unexpected characters near ",".]
  message: 'Unexpected characters near ",".',
  parsedLine: 88,
  snippet: 'title: \'CyTube\',' }

The fix is to edit config.yaml and remove the trailing comma for the title: property under html-template. If there are other syntax errors that the old version didn't detect, you will need to correct those as well.

Longer term, I am looking to move away from using yamljs to parse configuration because it's a little buggy and the current configuration system is confusing.

2016-10-20

Google Drive changed the URL schema for retrieving video metadata, which broke CyTube's Google Drive support, even with the userscript. I have updated the userscript source with the new URL, so server administrators will have to regenerate the userscript for their site and users will be prompted to install the newer version.

Additionally, fixing Drive lookups required an update to the mediaquery module, so you will have to do an npm install to pull that fix in.

2016-08-23

A few weeks ago, the previous Google Drive player stopped working. This is nothing new; Google Drive has consistently broken a few times a year ever since support for it was added. However, it's becoming increasingly difficult and complicated to provide good support for Google Drive, so I've made the decision to phase out the native player and require a userscript for it, in order to bypass CORS and allow each browser to request the video stream itself.

See the updated documentation for details on how to enable this for your users.

2016-04-27

A new dependency has been added on cytube-common, a module that will hold common code shared between the current version of CyTube and the upcoming work around splitting it into multiple services. You will need to be sure to run npm install after pulling in this change to pull in the new dependency.

2016-01-06

This release updates socket.io to version 1.4.0. The updates to socket.io include a few security-related fixes, so please be sure to run npm install to ensure the updated version is installed before restarting your CyTube server.

• https://nodesecurity.io/advisories/67
• 391ce0dc8b

2015-10-25

In order to support future clustering support, the legacy /sioconfig endpoint is being deprecated. Instead, you should make a request to /socketconfig/<channel name>.json. See the documentation for more information.

2015-10-04

• The channel data storage system has been refactored a bit. For compatibility, the default remains to store JSON objects for each channel in the chandump folder, however there is now also the option of storing channel data in the database. You can take advantage of this by setting channel-storage: type: 'database' in your config.yaml.

  • In order to migrate existing channel data from the chandump files to the database, run node lib/channel-storage/migrate.js.

• The database storage method uses foreign keys to associate the channel data with the corresponding row in the channels table. This requires that the tables be stored using the InnoDB engine rather than MyISAM. If your CyTube tables defaulted to MyISAM, you can fix them by running

  ALTER TABLE `channels` ENGINE = InnoDB;
  

2015-09-21

• CyTube is now transpiled with babel to allow the use of ES6/ES2015 features. All source files have been moved from lib to src.
• Running npm install or npm run postinstall will prompt you to build from src to lib.
• Running npm run build-server will run the build script without any prompts.
• After updating with git pull, you should run npm install or npm run build-server in order to rebuild after the changes.

2015-07-25

• CyTube now supports subtitles for Google Drive videos. In order to take advantage of this, you must upgrade mediaquery by running npm install cytube/mediaquery. Subtitles are cached in the google-drive-subtitles folder.

2015-07-07

• CyTube and CyTube/mediaquery have both been updated to use calzoneman/status-message-polyfill to polyfill res.statusMessage on older versions of node (e.g., v0.10). After pulling, run npm install to update this dependency. This fixes an issue where HTTP status messages from mediaquery were reported as undefined, and removes the need for manually looking up status messages in lib/ffmpeg.js.

2015-07-06

• As part of the video player rewrite, Google Drive and Google+ metadata lookups are now offloaded to CyTube/mediaquery. After pulling the new changes, run npm install or npm update to update the mediaquery dependency.

• www/js/player.js is now built from the CoffeeScript source files in the player/ directory. Instead of modifying it directly, modify the relevant player implementations in player/ and run npm run build-player (or node build-player.js) to generate www/js/player.js.

• Also as part of the video player rewrite, the schema for custom embeds changed so any custom embeds stored in the channel_libraries table need to be updated. The automatic upgrade script will convert any custom embeds that are parseable (i.e., not truncated by the width of the id field using the old format) and will delete the rest (you may see a lot of WARNING: unable to convert xxx messages-- this is normal). Custom embeds in channel playlists in the chandumps will be converted when the channel is loaded.