From 450c0fc33927e3fdede016a89c35bb8909302439 Mon Sep 17 00:00:00 2001 From: Matthew Verive Date: Wed, 29 Jul 2020 16:17:15 -0500 Subject: [PATCH 1/5] Add Discord information and corrected name --- README.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 294ab732..d4147536 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ Klaxon is a free, quick to set up and easy to use *robot* that checks websites r You list websites you want monitored and Klaxon will visit them and, if they change, email you what's different. It saves you having to reload dozens of links yourself every day. -It's perfect for monitoring website changes you might miss, like freedom of information disclosure logs, court records, and anything related to Donald Trump. And it can even send notifications to your Slack channel. +It's perfect for monitoring website changes you might miss, like freedom of information disclosure logs, court records, and anything related to Donald Trump. And it can even send notifications to your Slack and Discord channels. Read more below, or say hello to the humans behind the project at the [Google Group email list](https://groups.google.com/forum/#!forum/news-klaxon-users). @@ -18,7 +18,7 @@ The public release of this free and open source software was supported by Knight ## How Does Klaxon Work? -Klaxon enables users to "bookmark" portions of a webpage and be notified (via email or [Slack](#notify-a-slack-channel)) of any changes that may occur to those sections. [Learn more about bookmarklets on the help.md page](data/help.md). +Klaxon enables users to "bookmark" portions of a webpage and be notified (via email, [Slack, or Discord](#notify-a-slack-or-discord-channel)) of any changes that may occur to those sections. [Learn more about bookmarklets on the help.md page](data/help.md). [![Circle CI](https://circleci.com/gh/themarshallproject/klaxon.svg?style=svg)](https://circleci.com/gh/themarshallproject/klaxon) @@ -54,7 +54,7 @@ If you have a Heroku account and you’re ready to go, it’s time to click on t You must be logged into your Heroku account, and it will take you to a page to configure your new app in Heroku’s dashboard. First, give your app a name in the first box. While this is technically optional, this will also double as the URL for your Klaxon instance, so think carefully about it for a moment. Try maybe an abbreviation for your newsroom with a hyphen and the word klaxon, like “wp-klaxon” or “sl-klaxon”. This will become a URL as https://sl-klaxon.herokuapp.com/ -Scroll down to the “* Admin_emails” field, add a comma-separated list of email addresses for your newsroom’s Klaxon administrators. These administrators will be able to create accounts for any user in your organization, as well as configure various Klaxons and integrations with services like Slack. +Scroll down to the “* Admin_emails” field, add a comma-separated list of email addresses for your newsroom’s Klaxon administrators. These administrators will be able to create accounts for any user in your organization, as well as configure various Klaxons and integrations with services like Slack and Discord. Click the big purple “Deploy for Free” button. If you haven’t given Heroku your credit card yet, it will ask you for your information now. As long as you’re on the free settings, it won’t charge you, but Heroku wants to be prepared in case you change tiers. After that, give Heroku a few minutes for the app to build. @@ -86,12 +86,20 @@ On the right side of the page, click the “Create New User” button. Add the r By default, people with any email address can be added as new users. If you'd like to allow only users with *specific* email domains, set the `APPROVED_USER_DOMAINS` environment variable (or "Config Variable" in Heroku's lingo). That variable should be a comma-separated list of domains, e.g., `themarshallproject.org,nsa.gov`. -### Notify a Slack channel +### Notify a Slack or Discord channel -You’re all set for email notifications. If you’d like to also receive alerts through Slack, you can set that up now too. (If you want alerts from other services, [we welcome pull requests](CONTRIBUTING.md)) Click on the “Settings” button in the upper right corner of the page and choose “Integrations” from the menu. On the Integrations page, click the “Create Slack Integration” button. You can add an integration for any number of channels in your newsroom’s Slack. For each one, you just have to set up an Incoming Webhook. In Slack, click on the dropdown arrow in the upper left corner and choose “Apps & Integrations” from the menu. This will open a new window in your browser for you to search the Slack app directory. In the search box, type “Incoming Webhooks” and choose that option when it pops up. If you already have webhooks, you’ll see a button next to your Slack organization’s name that says “Configure.” Otherwise, click the green button that says “Install”. +You’re all set for email notifications. If you’d like to also receive alerts through Slack and/or Discord, you can set that up now too. (If you want alerts from other services, [we welcome pull requests](CONTRIBUTING.md)) Click on the “Settings” button in the upper right corner of the page and choose “Integrations” from the menu. On the Integrations page, click the “Create Slack Integration” button. You can add an integration for any number of channels in your newsroom’s Slack or Discord. For each channel, you just have to set up an Incoming Webhook. + +#### Slack +In Slack, click on the dropdown arrow in the upper left corner and choose “Apps & Integrations” from the menu. This will open a new window in your browser for you to search the Slack app directory. In the search box, type “Incoming Webhooks” and choose that option when it pops up. If you already have webhooks, you’ll see a button next to your Slack organization’s name that says “Configure.” Otherwise, click the green button that says “Install”. Now, choose the channel that you want the Klaxon alerts to go to from the dropdown menu. We’d recommend that you not send them to #General, but maybe create a new channel called #Klaxon. After you create or choose your channel, click the green button that says “Add Incoming Webhooks Integration”. Near the top of the next screen, you should see a red URL next to the label “Webhook URL”. Copy that URL and switch over to your browser window with Klaxon in it. Paste the URL into the box labeled “Webhook URL,” and type the name of the channel you want your Slack alerts to go to into the “Channel” box (this should be the same channel name you used in Slack when you created the integration). Now click the “Create Slack Integration Button”. Now you should be all set. If you want to have the ability to send Klaxon alerts to other channels, for specific reporting teams or for certain projects, you can repeat this process. +#### Discord +In Discord, click on the dropdown arrow in the upper left (next to your server name) and choose “Server Settings” from the menu. Click “Integrations” in the left sidebar, then click the “Create Webhook” button. + +Now, choose the name for your webhook (you can leave it the default random name if you'd like) and choose the channel that you want the Klaxon alerts to go to. We’d recommend that you not send them to #general, but maybe create a new channel called #klaxon (you'll need to do this in your normal server view first). After you choose your channel, click “Copy Webhook URL”. Switch over to your browser window with Klaxon in it. Paste the URL into the box labeled “Webhook URL,” and type the name of the channel you want your Discord alerts to go to into the “Channel” box (this should be the same channel name you used in Discord when you created the integration). Additionally, you **must** append “/slack” (without quotes) to the end of your webhook URL, as these alerts will be sent to Discord as a Slack-Compatible Webhook. Now click the “Create Slack Integration Button”. Now you should be all set. If you want to have the ability to send Klaxon alerts to other channels, for specific reporting teams or for certain projects, you can repeat this process. + ### Applying upgrades as the project develops When we release major changes to Klaxon, we’ll make an announcement to [our Google Group email list](https://groups.google.com/forum/#!forum/news-klaxon-users). At that point, you’ll likely want to adopt those in your system as well. If you're comfortable using git on the command line, this would require just a few simple commands: pull the changes from the master branch of this repo, merge them into your forked repo and push it all to Heroku. @@ -126,7 +134,7 @@ We've been grateful for additional contributions to the project from: * Jackson Gothe-Snape, SBS News * Cameo Hill * Emily Hopkins -* imnewnity +* Matthew Verive * Jason Kulatunga * Yolanda Martinez * Jeremy Merrill From a65dad781285cea89f3ba7a6e67d8b157584ecb0 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Wed, 5 Aug 2020 09:41:13 -0400 Subject: [PATCH 2/5] update README to reflect nuisance Sendgrid steps --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 294ab732..13b6f122 100644 --- a/README.md +++ b/README.md @@ -68,9 +68,9 @@ Click on the button that says “Manage App”. This takes you behind the scenes ![](docs/scheduler.png) -Unfortunately our email provider Sendgrid requires an additional step to confirm that you are not a spammer. Your new Sendgrid account is now in a "suspended" state, to get it unsuspended you have to contact Sendgrid support. You can do this by clicking the Sendgrid logo on the Resources tab. **You will not be able to log in until you are cleared by Sendgrid.** This usually happens pretty quickly (hours not days). +Unfortunately our email provider Sendgrid now requires an additional step to confirm that you are not a spammer. Your new Sendgrid account is now in a "suspended" state, and to get it unsuspended you have to contact [Sendgrid support](https://support.sendgrid.com/hc/en-us/requests/new#login-issue). You can do this by clicking the Sendgrid logo on the Resources tab. If clicking on the logo takes you to an error page, do not worry. This has been known to happen as Sendgrid's system has undergone redesigns. Instead, go to [Sendgrid's page to ask for support](https://support.sendgrid.com/hc/en-us/requests/new#login-issue). Be sure to use the same email address associated with your Heroku account and provide the url of your Klaxon instance. When they ask for "Business impact," choose "P3 General - You have a question about Sendgrid or how to use its products". This step is a nuisance, but important. **You will not be able to get an email log in to Klaxon until you are cleared by Sendgrid.** This usually happens pretty quickly (hours not days). -Now, at the top of the scheduler page, click the link that is the name of your app (“sl-klaxon”). This will take you to your Klaxon’s login screen on the web. +At the top of the scheduler page, click the link that is the name of your app (“sl-klaxon”). This will take you to back to Klaxon's dashboard. Then click the button in the upper right that says "Open app," and this should take you to your Klaxon's login screen on the web. ![](docs/login_screen.png) From 76cdf18fbbf3900dd8d5e30fa5e511c948ef0a74 Mon Sep 17 00:00:00 2001 From: Gabe Isman Date: Fri, 11 Sep 2020 13:08:22 -0400 Subject: [PATCH 3/5] Upgrade to rails 6 --- Gemfile | 2 +- Gemfile.lock | 281 +++++++++++++++++++++++++++------------------------ 2 files changed, 149 insertions(+), 134 deletions(-) diff --git a/Gemfile b/Gemfile index 31117b38..f79fa93a 100644 --- a/Gemfile +++ b/Gemfile @@ -1,7 +1,7 @@ source 'https://rubygems.org' ruby '2.5.7' -gem 'rails', '~> 5.2.4' +gem 'rails', '~> 6.0' gem 'pg', '~> 0.21' gem 'sass-rails', '~> 5.0' gem 'uglifier', '>= 1.3.0' diff --git a/Gemfile.lock b/Gemfile.lock index 11bc9fbc..34c7440b 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -1,71 +1,84 @@ GEM remote: https://rubygems.org/ specs: - actioncable (5.2.4.3) - actionpack (= 5.2.4.3) + actioncable (6.0.3.3) + actionpack (= 6.0.3.3) nio4r (~> 2.0) websocket-driver (>= 0.6.1) - actionmailer (5.2.4.3) - actionpack (= 5.2.4.3) - actionview (= 5.2.4.3) - activejob (= 5.2.4.3) + actionmailbox (6.0.3.3) + actionpack (= 6.0.3.3) + activejob (= 6.0.3.3) + activerecord (= 6.0.3.3) + activestorage (= 6.0.3.3) + activesupport (= 6.0.3.3) + mail (>= 2.7.1) + actionmailer (6.0.3.3) + actionpack (= 6.0.3.3) + actionview (= 6.0.3.3) + activejob (= 6.0.3.3) mail (~> 2.5, >= 2.5.4) rails-dom-testing (~> 2.0) - actionpack (5.2.4.3) - actionview (= 5.2.4.3) - activesupport (= 5.2.4.3) + actionpack (6.0.3.3) + actionview (= 6.0.3.3) + activesupport (= 6.0.3.3) rack (~> 2.0, >= 2.0.8) rack-test (>= 0.6.3) rails-dom-testing (~> 2.0) - rails-html-sanitizer (~> 1.0, >= 1.0.2) - actionview (5.2.4.3) - activesupport (= 5.2.4.3) + rails-html-sanitizer (~> 1.0, >= 1.2.0) + actiontext (6.0.3.3) + actionpack (= 6.0.3.3) + activerecord (= 6.0.3.3) + activestorage (= 6.0.3.3) + activesupport (= 6.0.3.3) + nokogiri (>= 1.8.5) + actionview (6.0.3.3) + activesupport (= 6.0.3.3) builder (~> 3.1) erubi (~> 1.4) rails-dom-testing (~> 2.0) - rails-html-sanitizer (~> 1.0, >= 1.0.3) - activejob (5.2.4.3) - activesupport (= 5.2.4.3) + rails-html-sanitizer (~> 1.1, >= 1.2.0) + activejob (6.0.3.3) + activesupport (= 6.0.3.3) globalid (>= 0.3.6) - activemodel (5.2.4.3) - activesupport (= 5.2.4.3) - activerecord (5.2.4.3) - activemodel (= 5.2.4.3) - activesupport (= 5.2.4.3) - arel (>= 9.0) - activestorage (5.2.4.3) - actionpack (= 5.2.4.3) - activerecord (= 5.2.4.3) + activemodel (6.0.3.3) + activesupport (= 6.0.3.3) + activerecord (6.0.3.3) + activemodel (= 6.0.3.3) + activesupport (= 6.0.3.3) + activestorage (6.0.3.3) + actionpack (= 6.0.3.3) + activejob (= 6.0.3.3) + activerecord (= 6.0.3.3) marcel (~> 0.3.1) - activesupport (5.2.4.3) + activesupport (6.0.3.3) concurrent-ruby (~> 1.0, >= 1.0.2) i18n (>= 0.7, < 2) minitest (~> 5.1) tzinfo (~> 1.1) - addressable (2.5.2) - public_suffix (>= 2.0.2, < 4.0) - arel (9.0.0) + zeitwerk (~> 2.2, >= 2.2.2) + addressable (2.7.0) + public_suffix (>= 2.0.2, < 5.0) aws-eventstream (1.1.0) - aws-partitions (1.338.0) - aws-sdk-core (3.103.0) + aws-partitions (1.367.0) + aws-sdk-core (3.105.0) aws-eventstream (~> 1, >= 1.0.2) aws-partitions (~> 1, >= 1.239.0) aws-sigv4 (~> 1.1) jmespath (~> 1.0) - aws-sdk-sqs (1.30.0) + aws-sdk-sqs (1.32.0) aws-sdk-core (~> 3, >= 3.99.0) aws-sigv4 (~> 1.1) - aws-sigv4 (1.2.1) + aws-sigv4 (1.2.2) aws-eventstream (~> 1, >= 1.0.2) - bcrypt (3.1.12) - bindex (0.5.0) - bootsnap (1.4.6) + bcrypt (3.1.16) + bindex (0.8.1) + bootsnap (1.4.8) msgpack (~> 1.0) builder (3.2.4) - byebug (11.0.1) - codeclimate-test-reporter (0.5.2) - simplecov (>= 0.7.1, < 1.0.0) - coderay (1.1.2) + byebug (11.1.3) + codeclimate-test-reporter (1.0.9) + simplecov (<= 0.13) + coderay (1.1.3) coffee-rails (5.0.0) coffee-script (>= 2.2.0) railties (>= 5.2.0) @@ -73,15 +86,15 @@ GEM coffee-script-source execjs coffee-script-source (1.12.2) - concurrent-ruby (1.1.6) + concurrent-ruby (1.1.7) crack (0.4.3) safe_yaml (~> 1.0.0) crass (1.0.6) - css_parser (1.6.0) + css_parser (1.7.1) addressable - database_cleaner (1.7.0) - diff-lcs (1.3) - diffy (3.3.0) + database_cleaner (1.8.5) + diff-lcs (1.4.4) + diffy (3.4.0) docile (1.1.5) dotenv (2.7.6) erubi (1.9.0) @@ -91,11 +104,11 @@ GEM factory_bot_rails (6.1.0) factory_bot (~> 6.1.0) railties (>= 5.0.0) - ffi (1.9.25) + ffi (1.13.1) formatador (0.2.5) globalid (0.4.2) activesupport (>= 4.2.0) - guard (2.14.2) + guard (2.16.2) formatador (>= 0.2.4) listen (>= 2.7, < 4.0) lumberjack (>= 1.0.12, < 2.0) @@ -109,95 +122,97 @@ GEM guard (~> 2.1) guard-compat (~> 1.1) rspec (>= 2.99.0, < 4.0) - hashdiff (0.3.7) + hashdiff (1.0.1) htmlentities (4.3.4) httparty (0.18.1) mime-types (~> 3.0) multi_xml (>= 0.5.2) - i18n (1.8.3) + i18n (1.8.5) concurrent-ruby (~> 1.0) jmespath (1.4.0) - jquery-rails (4.3.4) + jquery-rails (4.4.0) rails-dom-testing (>= 1, < 3) railties (>= 4.2.0) thor (>= 0.14, < 2.0) - json (1.8.6) - jwt (2.1.0) + json (2.3.1) + jwt (2.2.2) kramdown (2.3.0) rexml - launchy (2.4.3) - addressable (~> 2.3) + launchy (2.5.0) + addressable (~> 2.7) letter_opener (1.7.0) launchy (~> 2.2) libv8 (3.16.14.19) - listen (3.1.5) - rb-fsevent (~> 0.9, >= 0.9.4) - rb-inotify (~> 0.9, >= 0.9.7) - ruby_dep (~> 1.2) - loofah (2.6.0) + listen (3.2.1) + rb-fsevent (~> 0.10, >= 0.10.3) + rb-inotify (~> 0.9, >= 0.9.10) + loofah (2.7.0) crass (~> 1.0.2) nokogiri (>= 1.5.9) - lumberjack (1.0.13) + lumberjack (1.2.8) mail (2.7.1) mini_mime (>= 0.1.1) marcel (0.3.3) mimemagic (~> 0.3.2) - method_source (0.9.2) + method_source (1.0.0) mime-types (3.3.1) mime-types-data (~> 3.2015) mime-types-data (3.2020.0512) mimemagic (0.3.5) mini_mime (1.0.2) mini_portile2 (2.4.0) - minitest (5.14.1) + minitest (5.14.2) msgpack (1.3.3) multi_xml (0.6.0) - mustermann (1.0.3) + mustermann (1.1.1) + ruby2_keywords (~> 0.0.1) nenv (0.3.0) - nio4r (2.5.2) + nio4r (2.5.3) nokogiri (1.10.10) mini_portile2 (~> 2.4.0) - notiffany (0.1.1) + notiffany (0.1.3) nenv (~> 0.1) shellany (~> 0.0) pg (0.21.0) - premailer (1.11.1) + premailer (1.13.1) addressable css_parser (>= 1.6.0) htmlentities (>= 4.0.0) - premailer-rails (1.10.2) - actionmailer (>= 3, < 6) + premailer-rails (1.11.1) + actionmailer (>= 3) premailer (~> 1.7, >= 1.7.9) - pry (0.11.3) - coderay (~> 1.1.0) - method_source (~> 0.9.0) - public_suffix (3.0.3) - puma (4.3.5) + pry (0.13.1) + coderay (~> 1.1) + method_source (~> 1.0) + public_suffix (4.0.6) + puma (4.3.6) nio4r (~> 2.0) rack (2.2.3) - rack-cache (1.9.0) + rack-cache (1.12.0) rack (>= 0.4) - rack-protection (2.0.5) + rack-protection (2.1.0) rack rack-test (1.1.0) rack (>= 1.0, < 3) - rails (5.2.4.3) - actioncable (= 5.2.4.3) - actionmailer (= 5.2.4.3) - actionpack (= 5.2.4.3) - actionview (= 5.2.4.3) - activejob (= 5.2.4.3) - activemodel (= 5.2.4.3) - activerecord (= 5.2.4.3) - activestorage (= 5.2.4.3) - activesupport (= 5.2.4.3) + rails (6.0.3.3) + actioncable (= 6.0.3.3) + actionmailbox (= 6.0.3.3) + actionmailer (= 6.0.3.3) + actionpack (= 6.0.3.3) + actiontext (= 6.0.3.3) + actionview (= 6.0.3.3) + activejob (= 6.0.3.3) + activemodel (= 6.0.3.3) + activerecord (= 6.0.3.3) + activestorage (= 6.0.3.3) + activesupport (= 6.0.3.3) bundler (>= 1.3.0) - railties (= 5.2.4.3) + railties (= 6.0.3.3) sprockets-rails (>= 2.0.0) - rails-controller-testing (1.0.4) - actionpack (>= 5.0.1.x) - actionview (>= 5.0.1.x) - activesupport (>= 5.0.1.x) + rails-controller-testing (1.0.5) + actionpack (>= 5.0.1.rc1) + actionview (>= 5.0.1.rc1) + activesupport (>= 5.0.1.rc1) rails-dom-testing (2.0.3) activesupport (>= 4.2.0) nokogiri (>= 1.6) @@ -208,51 +223,51 @@ GEM rails_stdout_logging rails_serve_static_assets (0.0.5) rails_stdout_logging (0.0.5) - railties (5.2.4.3) - actionpack (= 5.2.4.3) - activesupport (= 5.2.4.3) + railties (6.0.3.3) + actionpack (= 6.0.3.3) + activesupport (= 6.0.3.3) method_source rake (>= 0.8.7) - thor (>= 0.19.0, < 2.0) + thor (>= 0.20.3, < 2.0) rake (13.0.1) - rb-fsevent (0.10.3) - rb-inotify (0.9.10) - ffi (>= 0.5.0, < 2) - rdoc (6.0.4) + rb-fsevent (0.10.4) + rb-inotify (0.10.1) + ffi (~> 1.0) + rdoc (6.2.1) ref (2.0.0) rexml (3.2.4) - rspec (3.8.0) - rspec-core (~> 3.8.0) - rspec-expectations (~> 3.8.0) - rspec-mocks (~> 3.8.0) - rspec-core (3.8.0) - rspec-support (~> 3.8.0) - rspec-expectations (3.8.2) + rspec (3.9.0) + rspec-core (~> 3.9.0) + rspec-expectations (~> 3.9.0) + rspec-mocks (~> 3.9.0) + rspec-core (3.9.2) + rspec-support (~> 3.9.3) + rspec-expectations (3.9.2) diff-lcs (>= 1.2.0, < 2.0) - rspec-support (~> 3.8.0) - rspec-mocks (3.8.0) + rspec-support (~> 3.9.0) + rspec-mocks (3.9.1) diff-lcs (>= 1.2.0, < 2.0) - rspec-support (~> 3.8.0) - rspec-rails (3.8.2) + rspec-support (~> 3.9.0) + rspec-rails (3.9.1) actionpack (>= 3.0) activesupport (>= 3.0) railties (>= 3.0) - rspec-core (~> 3.8.0) - rspec-expectations (~> 3.8.0) - rspec-mocks (~> 3.8.0) - rspec-support (~> 3.8.0) - rspec-support (3.8.0) + rspec-core (~> 3.9.0) + rspec-expectations (~> 3.9.0) + rspec-mocks (~> 3.9.0) + rspec-support (~> 3.9.0) + rspec-support (3.9.3) rspec_junit_formatter (0.4.1) rspec-core (>= 2, < 4, != 2.12.0) - ruby_dep (1.5.0) - safe_yaml (1.0.4) - sass (3.5.7) + ruby2_keywords (0.0.2) + safe_yaml (1.0.5) + sass (3.7.4) sass-listen (~> 4.0.0) sass-listen (4.0.0) rb-fsevent (~> 0.9, >= 0.9.4) rb-inotify (~> 0.9, >= 0.9.7) - sass-rails (5.0.7) - railties (>= 4.0.0, < 6) + sass-rails (5.1.0) + railties (>= 5.2.0) sass (~> 3.1) sprockets (>= 2.8, < 4.0) sprockets-rails (>= 2.0, < 4.0) @@ -260,21 +275,20 @@ GEM sdoc (1.0.0) rdoc (>= 5.0) shellany (0.0.1) - simple_form (5.0.0) + simple_form (5.0.2) actionpack (>= 5.0) activemodel (>= 5.0) - simplecov (0.11.2) + simplecov (0.13.0) docile (~> 1.1.0) - json (~> 1.8) + json (>= 1.8, < 3) simplecov-html (~> 0.10.0) - simplecov-html (0.10.0) - sinatra (2.0.5) + simplecov-html (0.10.2) + sinatra (2.1.0) mustermann (~> 1.0) - rack (~> 2.0) - rack-protection (= 2.0.5) + rack (~> 2.2) + rack-protection (= 2.1.0) tilt (~> 2.0) - spring (2.0.2) - activesupport (>= 4.2) + spring (2.1.1) sprockets (3.7.2) concurrent-ruby (~> 1.0) rack (> 1, < 3) @@ -287,26 +301,27 @@ GEM ref thor (1.0.1) thread_safe (0.3.6) - tilt (2.0.9) - turbolinks (5.2.0) + tilt (2.0.10) + turbolinks (5.2.1) turbolinks-source (~> 5.2) turbolinks-source (5.2.0) tzinfo (1.2.7) thread_safe (~> 0.1) - uglifier (4.1.20) + uglifier (4.2.0) execjs (>= 0.3.0, < 3) web-console (3.7.0) actionview (>= 5.0) activemodel (>= 5.0) bindex (>= 0.4.0) railties (>= 5.0) - webmock (3.5.1) + webmock (3.8.3) addressable (>= 2.3.6) crack (>= 0.3.2) - hashdiff + hashdiff (>= 0.4.0, < 2.0.0) websocket-driver (0.7.3) websocket-extensions (>= 0.1.0) websocket-extensions (0.1.5) + zeitwerk (2.4.0) PLATFORMS ruby @@ -332,7 +347,7 @@ DEPENDENCIES premailer-rails puma (~> 4.3) rack-cache - rails (~> 5.2.4) + rails (~> 6.0) rails-controller-testing rails_12factor rspec-rails (~> 3.8) From a49ec90a00c5ab889498ad4c3cb254c841c01057 Mon Sep 17 00:00:00 2001 From: Gabe Isman Date: Fri, 11 Sep 2020 14:37:38 -0400 Subject: [PATCH 4/5] Use SameSite=None for cookies. Chrome requires this (along with the Secure attribute and delivery over HTTPS) for all third-party cookies (as of February). https://web.dev/samesite-cookies-explained/ --- app/controllers/application_controller.rb | 2 +- app/controllers/sessions_controller.rb | 2 +- config/environments/development.rb | 4 ++++ config/initializers/session_store.rb | 2 +- 4 files changed, 7 insertions(+), 3 deletions(-) diff --git a/app/controllers/application_controller.rb b/app/controllers/application_controller.rb index 2653d320..29f94802 100644 --- a/app/controllers/application_controller.rb +++ b/app/controllers/application_controller.rb @@ -39,7 +39,7 @@ def current_user user = User.find_by(id: cookies.signed[:user_id]) if user.present? - cookies.signed[:user_id] = { value: user.id, expires: 7.days.from_now, httponly: true } + cookies.signed[:user_id] = { value: user.id, expires: 7.days.from_now, httponly: true, same_site: :none, secure: true } @current_user = user else cookies.signed[:user_id] = nil diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index de72b8e9..50641a2a 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -21,7 +21,7 @@ def token if user[:expired] redirect_to expired_token_path(user[:user].id) else - cookies.signed[:user_id] = { value: user.id, expires: 7.days.from_now, httponly: true } + cookies.signed[:user_id] = { value: user.id, expires: 7.days.from_now, httponly: true, same_site: :none, secure: true } redirect_to root_path end else diff --git a/config/environments/development.rb b/config/environments/development.rb index 294b8f34..77ab7484 100644 --- a/config/environments/development.rb +++ b/config/environments/development.rb @@ -38,6 +38,10 @@ config.action_mailer.delivery_method = :letter_opener + config.hosts << "klaxon.test" + + config.force_ssl = (ENV.fetch('KLAXON_FORCE_SSL', 'false').to_s.downcase == 'true') + # Raises error for missing translations # config.action_view.raise_on_missing_translations = true diff --git a/config/initializers/session_store.rb b/config/initializers/session_store.rb index bdd672d0..266148e7 100644 --- a/config/initializers/session_store.rb +++ b/config/initializers/session_store.rb @@ -1,3 +1,3 @@ # Be sure to restart your server when you modify this file. -Rails.application.config.session_store :cookie_store, key: '_klaxon_session' +Rails.application.config.session_store :cookie_store, key: '_klaxon_session', same_site: :none, secure: true From f37b2c0093d7f09e926c5345a8fa4fa69f589806 Mon Sep 17 00:00:00 2001 From: Gabe Isman Date: Fri, 11 Sep 2020 17:29:31 -0400 Subject: [PATCH 5/5] Fix tests. --- Gemfile | 2 +- Gemfile.lock | 20 +++--- app/controllers/api_controller.rb | 1 + app/controllers/embed_controller.rb | 1 - app/controllers/integrations_controller.rb | 4 +- app/controllers/sessions_controller.rb | 1 - .../slack_integrations_controller.rb | 1 + .../sqs_integrations_controller.rb | 1 + app/controllers/static_controller.rb | 2 +- app/controllers/users_controller.rb | 1 - app/controllers/watching_controller.rb | 2 - config/environments/development.rb | 2 - spec/controllers/api_controller_spec.rb | 72 +++++++++++++------ .../integrations_controller_spec.rb | 4 +- .../slack_integrations_controller_spec.rb | 2 + .../sqs_integrations_controller_spec.rb | 2 + spec/rails_helper.rb | 3 + spec/requests/api_controller_spec.rb | 59 --------------- spec/requests/pages_spec.rb | 5 -- spec/requests/slack_integrations_spec.rb | 4 -- spec/requests/sqs_integrations_spec.rb | 4 -- spec/requests/static_controller_spec.rb | 8 --- spec/requests/users_spec.rb | 4 -- spec/spec_helper.rb | 3 - spec/support/cookies.rb | 5 ++ spec/support/login_helper.rb | 10 +++ 26 files changed, 94 insertions(+), 129 deletions(-) delete mode 100644 spec/requests/api_controller_spec.rb create mode 100644 spec/support/cookies.rb create mode 100644 spec/support/login_helper.rb diff --git a/Gemfile b/Gemfile index f79fa93a..78d263fb 100644 --- a/Gemfile +++ b/Gemfile @@ -32,7 +32,7 @@ gem 'aws-sdk-sqs', '~> 1.30' group :development, :test do gem 'byebug' - gem 'rspec-rails', '~> 3.8' + gem 'rspec-rails' gem 'guard-rspec', require: false gem 'factory_bot_rails', '~> 6.1' gem 'database_cleaner' diff --git a/Gemfile.lock b/Gemfile.lock index 34c7440b..10323160 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -59,7 +59,7 @@ GEM addressable (2.7.0) public_suffix (>= 2.0.2, < 5.0) aws-eventstream (1.1.0) - aws-partitions (1.367.0) + aws-partitions (1.368.0) aws-sdk-core (3.105.0) aws-eventstream (~> 1, >= 1.0.2) aws-partitions (~> 1, >= 1.239.0) @@ -248,14 +248,14 @@ GEM rspec-mocks (3.9.1) diff-lcs (>= 1.2.0, < 2.0) rspec-support (~> 3.9.0) - rspec-rails (3.9.1) - actionpack (>= 3.0) - activesupport (>= 3.0) - railties (>= 3.0) - rspec-core (~> 3.9.0) - rspec-expectations (~> 3.9.0) - rspec-mocks (~> 3.9.0) - rspec-support (~> 3.9.0) + rspec-rails (4.0.1) + actionpack (>= 4.2) + activesupport (>= 4.2) + railties (>= 4.2) + rspec-core (~> 3.9) + rspec-expectations (~> 3.9) + rspec-mocks (~> 3.9) + rspec-support (~> 3.9) rspec-support (3.9.3) rspec_junit_formatter (0.4.1) rspec-core (>= 2, < 4, != 2.12.0) @@ -350,7 +350,7 @@ DEPENDENCIES rails (~> 6.0) rails-controller-testing rails_12factor - rspec-rails (~> 3.8) + rspec-rails rspec_junit_formatter sass-rails (~> 5.0) sdoc (~> 1.0.0) diff --git a/app/controllers/api_controller.rb b/app/controllers/api_controller.rb index 97284956..80a4cf2a 100644 --- a/app/controllers/api_controller.rb +++ b/app/controllers/api_controller.rb @@ -32,6 +32,7 @@ def embed_find_page # POST request def embed_update_page_selector # POST request page = Page.find_by(id: params[:id]) + page.css_selector = params[:css_selector] if page.save render json: page diff --git a/app/controllers/embed_controller.rb b/app/controllers/embed_controller.rb index 2fd2ba86..3e5309aa 100644 --- a/app/controllers/embed_controller.rb +++ b/app/controllers/embed_controller.rb @@ -1,5 +1,4 @@ class EmbedController < ApplicationController - skip_before_action :verify_authenticity_token, only: [:inject] before_action :authorize, only: [:iframe] diff --git a/app/controllers/integrations_controller.rb b/app/controllers/integrations_controller.rb index 3096705b..204a6d7f 100644 --- a/app/controllers/integrations_controller.rb +++ b/app/controllers/integrations_controller.rb @@ -1,6 +1,8 @@ class IntegrationsController < ApplicationController + before_action :authorize + def index @slack_integrations = SlackIntegration.all - @sqs_integrations = SqsIntegration.all + @sqs_integrations = SqsIntegration.all end end diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index 50641a2a..99542f0f 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -40,5 +40,4 @@ def destroy cookies.delete(:user_id) redirect_to root_path end - end diff --git a/app/controllers/slack_integrations_controller.rb b/app/controllers/slack_integrations_controller.rb index 2f57e415..73875e78 100644 --- a/app/controllers/slack_integrations_controller.rb +++ b/app/controllers/slack_integrations_controller.rb @@ -1,5 +1,6 @@ class SlackIntegrationsController < ApplicationController before_action :set_slack_integration, only: [:show, :edit, :update, :destroy] + before_action :authorize # GET /slack_integrations def index diff --git a/app/controllers/sqs_integrations_controller.rb b/app/controllers/sqs_integrations_controller.rb index e9f290ce..bb051bcd 100644 --- a/app/controllers/sqs_integrations_controller.rb +++ b/app/controllers/sqs_integrations_controller.rb @@ -1,5 +1,6 @@ class SqsIntegrationsController < ApplicationController before_action :set_sqs_integration, only: [:show, :edit, :update, :destroy] + before_action :authorize # GET /sqs_integrations def index diff --git a/app/controllers/static_controller.rb b/app/controllers/static_controller.rb index 6d01eed6..5ed7cd80 100644 --- a/app/controllers/static_controller.rb +++ b/app/controllers/static_controller.rb @@ -1,4 +1,5 @@ class StaticController < ApplicationController + before_action :authorize def help path = File.join(Rails.root, 'data', 'help.md') @@ -8,5 +9,4 @@ def help def feed end - end diff --git a/app/controllers/users_controller.rb b/app/controllers/users_controller.rb index 899c6b8f..ac20f440 100644 --- a/app/controllers/users_controller.rb +++ b/app/controllers/users_controller.rb @@ -1,5 +1,4 @@ class UsersController < ApplicationController - before_action :authorize before_action :set_user, only: [:show, :edit, :update, :destroy] diff --git a/app/controllers/watching_controller.rb b/app/controllers/watching_controller.rb index e19bbea4..bbd09d34 100644 --- a/app/controllers/watching_controller.rb +++ b/app/controllers/watching_controller.rb @@ -1,9 +1,7 @@ class WatchingController < ApplicationController - before_action :authorize def feed @changes = Change.order('created_at DESC').first(20) end - end diff --git a/config/environments/development.rb b/config/environments/development.rb index 77ab7484..84a3e485 100644 --- a/config/environments/development.rb +++ b/config/environments/development.rb @@ -38,8 +38,6 @@ config.action_mailer.delivery_method = :letter_opener - config.hosts << "klaxon.test" - config.force_ssl = (ENV.fetch('KLAXON_FORCE_SSL', 'false').to_s.downcase == 'true') # Raises error for missing translations diff --git a/spec/controllers/api_controller_spec.rb b/spec/controllers/api_controller_spec.rb index 046a29a6..95c3ef78 100644 --- a/spec/controllers/api_controller_spec.rb +++ b/spec/controllers/api_controller_spec.rb @@ -2,25 +2,57 @@ RSpec.describe ApiController, type: :controller do - # describe "GET #subscriptions" do - # it "returns http success" do - # get :subscriptions - # expect(response).to have_http_status(:success) - # end - # end - - # describe "GET #users" do - # it "returns http success" do - # get :users - # expect(response).to have_http_status(:success) - # end - # end - - # describe "GET #pages" do - # it "returns http success" do - # get :pages - # expect(response).to have_http_status(:success) - # end - # end + before(:each) { + WebMock.allow_net_connect! + login + } + + describe "/page-preview" do + it "can query the tmp homepage" do + + url = 'https://www.themarshallproject.org' + css_selector = 'header' + + get(:page_preview, params: { url: url, css_selector: css_selector }) + + expect(response).to have_http_status(:success) + + data = JSON.parse(response.body) + expect(data['css_selector']).to eq css_selector + expect(data['url']).to eq url + expect(data['match_text']).to include 'About' + end + end + + it "can get the users" do + get(:users) + expect(JSON.parse(response.body).count).to be > 0 + end + + it "creates a page by url and can update that page's selector" do + url = "http://www.nytimes.com/" + selector = ".first-column-region .story" + + @user = current_user + + # create the page + post(:embed_find_page, params: { url: url }) + expect(response).to have_http_status(:success) + data = JSON.parse(response.body) + expect(data['url']).to eq url + expect(data['css_selector']).to eq nil + page = Page.find_by(url: url) + expect(page.user).to eq @user + + # update the page + post(:embed_update_page_selector, params: { id: page.id, css_selector: selector }) + data = JSON.parse(response.body) + expect(data['css_selector']).to eq selector + expect(data['user_id']).to eq @user.id + end + + it "has stats" + it "has subscriptions" + it "has pages" end diff --git a/spec/controllers/integrations_controller_spec.rb b/spec/controllers/integrations_controller_spec.rb index ae899ae0..97e69e99 100644 --- a/spec/controllers/integrations_controller_spec.rb +++ b/spec/controllers/integrations_controller_spec.rb @@ -1,12 +1,12 @@ require 'rails_helper' RSpec.describe IntegrationsController, type: :controller do - describe "GET #index" do + before (:each) { login } + it "returns http success" do get :index expect(response).to have_http_status(:success) end end - end diff --git a/spec/controllers/slack_integrations_controller_spec.rb b/spec/controllers/slack_integrations_controller_spec.rb index 6b8d4503..95462026 100644 --- a/spec/controllers/slack_integrations_controller_spec.rb +++ b/spec/controllers/slack_integrations_controller_spec.rb @@ -31,6 +31,8 @@ skip("Add a hash of attributes invalid for your model") } + before(:each) { login } + describe "GET #index" do it "assigns all slack_integrations as @slack_integrations" do slack_integration = SlackIntegration.create! valid_attributes diff --git a/spec/controllers/sqs_integrations_controller_spec.rb b/spec/controllers/sqs_integrations_controller_spec.rb index 25071d49..86dd2a40 100644 --- a/spec/controllers/sqs_integrations_controller_spec.rb +++ b/spec/controllers/sqs_integrations_controller_spec.rb @@ -35,6 +35,8 @@ } } + before(:each) { login } + describe "GET #index" do it "assigns all sqs_integrations as @sqs_integrations" do sqs_integration = SqsIntegration.create! valid_attributes diff --git a/spec/rails_helper.rb b/spec/rails_helper.rb index eee58291..a089dafc 100644 --- a/spec/rails_helper.rb +++ b/spec/rails_helper.rb @@ -37,7 +37,10 @@ # Remove this line if you're not using ActiveRecord or ActiveRecord fixtures config.fixture_path = "#{::Rails.root}/spec/fixtures" + config.example_status_persistence_file_path = Rails.root.join('tmp/examples.txt') + config.include FactoryBot::Syntax::Methods + config.include LoginHelper config.use_transactional_fixtures = false diff --git a/spec/requests/api_controller_spec.rb b/spec/requests/api_controller_spec.rb deleted file mode 100644 index bf579df8..00000000 --- a/spec/requests/api_controller_spec.rb +++ /dev/null @@ -1,59 +0,0 @@ -require 'rails_helper' - -RSpec.describe ApiController, type: :request do - - before(:each) do - WebMock.allow_net_connect! - - # login - @user = User.where(email: 'test@test.com').first_or_create - get(token_session_path, params: { token: LoginToken.create(user: @user) }) - end - - describe "/page-preview" do - it "can query the tmp homepage" do - - url = 'https://www.themarshallproject.org' - css_selector = 'header' - - get(api_page_preview_path, params: { url: url, css_selector: css_selector }) - - expect(response).to have_http_status(:success) - - data = JSON.parse(response.body) - expect(data['css_selector']).to eq css_selector - expect(data['url']).to eq url - expect(data['match_text']).to include 'About' - end - end - - it "can get the users" do - get(api_users_path) - expect(JSON.parse(response.body).count).to be > 0 - end - - it "creates a page by url and can update that page's selector" do - url = "http://www.nytimes.com/" - selector = ".first-column-region .story" - - # create the page - post(embed_find_page_path, params: { url: url }) - expect(response).to have_http_status(:success) - data = JSON.parse(response.body) - expect(data['url']).to eq url - expect(data['css_selector']).to eq nil - page = Page.find_by(url: url) - expect(page.user).to eq @user - - # update the page - post(embed_update_page_selector_path, params: { id: page.id, css_selector: selector }) - data = JSON.parse(response.body) - expect(data['css_selector']).to eq selector - expect(data['user_id']).to eq @user.id - end - - it "has stats" - it "has subscriptions" - it "has pages" - -end diff --git a/spec/requests/pages_spec.rb b/spec/requests/pages_spec.rb index 99b4b096..ff22e777 100644 --- a/spec/requests/pages_spec.rb +++ b/spec/requests/pages_spec.rb @@ -2,10 +2,5 @@ RSpec.describe "Pages", type: :request do describe "GET /pages" do - it "works! (now write some real specs)" - # do - # get pages_path - # expect(response).to have_http_status(200) - # end end end diff --git a/spec/requests/slack_integrations_spec.rb b/spec/requests/slack_integrations_spec.rb index 1ecc608b..b2269ba8 100644 --- a/spec/requests/slack_integrations_spec.rb +++ b/spec/requests/slack_integrations_spec.rb @@ -2,9 +2,5 @@ RSpec.describe "SlackIntegrations", type: :request do describe "GET /slack_integrations" do - it "works! (now write some real specs)" do - get slack_integrations_path - expect(response).to have_http_status(200) - end end end diff --git a/spec/requests/sqs_integrations_spec.rb b/spec/requests/sqs_integrations_spec.rb index 04221c05..b98fe400 100644 --- a/spec/requests/sqs_integrations_spec.rb +++ b/spec/requests/sqs_integrations_spec.rb @@ -2,9 +2,5 @@ RSpec.describe "SqsIntegrations", type: :request do describe "GET /sqs_integrations" do - it "works!" do - get sqs_integrations_path - expect(response).to have_http_status(200) - end end end diff --git a/spec/requests/static_controller_spec.rb b/spec/requests/static_controller_spec.rb index 221d3592..e8f0a972 100644 --- a/spec/requests/static_controller_spec.rb +++ b/spec/requests/static_controller_spec.rb @@ -2,12 +2,4 @@ RSpec.describe StaticController, type: :request do - before(:each) do - WebMock.allow_net_connect! - - # login - @user = User.where(email: 'test@test.com').first_or_create - get(token_session_path, token: LoginToken.create(user: @user)) - end - end diff --git a/spec/requests/users_spec.rb b/spec/requests/users_spec.rb index 9849d589..a8f2a520 100644 --- a/spec/requests/users_spec.rb +++ b/spec/requests/users_spec.rb @@ -2,9 +2,5 @@ RSpec.describe "Users", type: :request do describe "GET /users" do - # it "works! (now write some real specs)" do - # get users_path - # expect(response).to have_http_status(200) - # end end end diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb index a9e4ccf1..0a3cf331 100644 --- a/spec/spec_helper.rb +++ b/spec/spec_helper.rb @@ -1,6 +1,3 @@ -require "codeclimate-test-reporter" -CodeClimate::TestReporter.start - # This file was generated by the `rails generate rspec:install` command. Conventionally, all # specs live under a `spec` directory, which RSpec adds to the `$LOAD_PATH`. # The generated `.rspec` file contains `--require spec_helper` which will cause diff --git a/spec/support/cookies.rb b/spec/support/cookies.rb new file mode 100644 index 00000000..5b26bb37 --- /dev/null +++ b/spec/support/cookies.rb @@ -0,0 +1,5 @@ +class ActionDispatch::Cookies::CookieJar + def encrypted; self; end + def signed; self; end + def permanent; self; end +end diff --git a/spec/support/login_helper.rb b/spec/support/login_helper.rb new file mode 100644 index 00000000..2f934b38 --- /dev/null +++ b/spec/support/login_helper.rb @@ -0,0 +1,10 @@ +module LoginHelper + def login(user=nil) + user = User.where(email: 'test@test.com').first_or_create if user.nil? + request.cookies[:user_id] = user.id + end + + def current_user + User.find(request.cookies[:user_id]) + end +end