Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EPIC] Simplify Search #3311

Open
pgwillia opened this issue Dec 14, 2023 · 3 comments
Open

[EPIC] Simplify Search #3311

pgwillia opened this issue Dec 14, 2023 · 3 comments
Assignees
Labels

Comments

@pgwillia
Copy link
Member

pgwillia commented Dec 14, 2023

Epic / Very High Level User Story

Step 1: Describe The Problem We're Here to Solve

We have 9 servers (SolrCloud nodes, Zookeeper, etc) supporting a end of life version of Solr (6.6) for just ERA now. It would be a big win to be able to simplify this. Coming up with a plan and estimates for that work would be a good first step.

New information enforcing this as a priority is that yesterday we had a power supply failure for one of the Solr servers.
Henry had the foresight to order extras the last time they were replaced so Neil could quickly resolve the issue. Because Solr is redundant there was no impact to ERA. We may have more difficulty finding replacement power supplies in the future. Wendy will see if she can find another spare.

Obviously, we're good in the short term, we can borrow parts from staging servers or let nodes go dark. Longer term we should simplify this and retire the ageing hardware.

Additional information:

#2406

  • Simplifying the stack will make the decommission of ECHA datacenter easier, because we have several bare metal servers to support this.
  • Simplifying the stack will make the OS migration project easier, because we have many systems to migrate as part of this supporting SolrCloud
  • The Metadata Strategies Unit uses Solr directly to get reports/data from ERA. Will need to consider their needs.

Step 2: The Elevator Pitch & Refinement

Elevator Pitch

(more information: https://agilewarrior.wordpress.com/2010/11/06/the-agile-inception-deck/)

A feature-oriented proposal for a solution in user-understandable language (and related artifacts like mockups, if those might be helpful), generated by Developers & Metadata specialists based on the problem description, for
approval by Product Owner & relevant stakeholders. Should address the following:

  • What are we doing?

  • What are we NOT doing?

  • What resources will be needed to implement this?

Refinement

(as much or as little as makes sense for the pitch)

Metadata

What metadata is involved? What is changing in terms of metadata tracked for specific entities? Links to documentation etc.

Backend

Things like whiteboard sketches of architectural changes, Rails model component interactions, pictures of the outcome of event storm sessions, etc

Frontend

Things like wireframes, sketches of potential UI approaches, etc

Step 3: Tickets

Links to specific, achievable, estimable tickets with clear "Done" criteria, describing the work needed to implement the solution outlined above. By the time these are added to this document, they should be in a state ready for estimation.

@pgwillia pgwillia added the epic label Dec 14, 2023
@pgwillia pgwillia changed the title [EPIC] [EPIC] Simplify Search Dec 14, 2023
@murny
Copy link
Contributor

murny commented Mar 9, 2024

The plan here for a short term win is to simply upgrade Solr to a version that is supported and not EOL.

So this will require slowly upgrading Solr until version 8 or 9. We will go to 7 first, then up to 8/9 next.

I expect lots of issues with our configuration. For example simply bumping our docker container to 7 gets us:

Can't load schema /opt/solr/server/solr/mycores/development/schema.xml: Setting default operator in schema (solrQueryParser/@defaultOperator) not supported

@murny
Copy link
Contributor

murny commented Mar 9, 2024

Hyrax is on 8.11 now, so hopefully we can see how they upgraded theirs and follow suit

@pgwillia
Copy link
Member Author

Right sizing the project will be important because it's sounding like we'll be moving to DSpace sometime in 2025 (I've heard December, July and Q1 in the last few weeks). I think the driving requirements of the project are

  • Solr Cloud is supported by several bare metal servers that are end of life and in the ECHA datacenter that will be decommissioned in the next year
  • Solr Cloud is supported by several servers with an OS that is end of life in June 2024
  • Solr 6.6 is end of life many years ago
  • Metadata Strategies uses Solr in their workflow for generating reports and auditing
    So another option might be to upgrade Solr and/or simplify the back-end architecture supporting it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants