Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace broken links on import + add a script to download links #64

Merged
merged 4 commits into from
Nov 22, 2024
Merged

Replace broken links on import + add a script to download links #64

merged 4 commits into from
Nov 22, 2024

Commits on Nov 22, 2024

  1. Replace no-longer-operational grants.gov url with new one 

    We noticed that some links 404 but still return a 200 level status code. This is very annoying. It is also not something we can do much about, as we only control one website.

The upshot is that our external link checker can't do anything with these links.
If we check for that URL, it will say 200 all good, even if the user is actually
seeing the Page not found message.

However, we can find/replace stuff that we import, so if we know what URLs have this behaviour, we can resolve the problem when the file is imported. Right now there are 3 links we have identified:

| bad link | good link  |
|--------|--------|
|    [www.grants.gov/web/grants/search-grants.html](https://www.grants.gov/web/grants/search-grants.html)    | [grants.gov/search-grants](https://grants.gov/search-grants)    |
| [www.grants.gov/web/grants/forms/sf-424-family.html](https://www.grants.gov/web/grants/forms/sf-424-family.html) | [grants.gov/forms/forms-repository/sf-424-family](https://grants.gov/forms/forms-repository/sf-424-family) |
| [www.cdc.gov/grants/dictionary/index.html](https://www.cdc.gov/grants/dictionary/index.html) | [www.cdc.gov/grants/dictionary-of-terms/](https://www.cdc.gov/grants/dictionary-of-terms/) |
    @pcraig3
    pcraig3 committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    1a891b4 View commit details
    Browse the repository at this point in the history

  2. Let's export those links yo 

    Pretty simple: added a script that loops through all non-archived NOFOs and will pull all external links. This helps us do analysis on them if we need to for some reason. (In this case, our reason was to manually replace bad links)
    @pcraig3
    pcraig3 committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    cd7a0e9 View commit details
    Browse the repository at this point in the history

  3. Only show non-archived NOFOs

    @pcraig3
    pcraig3 committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    04a0327 View commit details
    Browse the repository at this point in the history

  4. Remove the view to export links from an individual NOFO 

    We were not using this, it was a one-off.

Plus, the new script handles this same functionality.

The best code is no code at all.™️
    @pcraig3
    pcraig3 committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    cef5f3b View commit details
    Browse the repository at this point in the history