Refactor inconsistent and confusing search/listing endpoints and views.

[deiferni]

Currently we provide quite a bunch of different search/listing API endpoints and views for searching. Some of them transparently use SOLR, if it is available, some of them always use the catalog.

We need to discuss which of them are needed, and what could/should be refactored. Figuring out use-cases is key. Potential Goals/Discussion Items:

• Unify request API
• Unify result/response structure
• Make clear when SOLR is used and when we use the catalog

This is a list of endpoints/views:

• GET endpoint implemented in SearchGet in plone.restapi in https://github.com/plone/plone.restapi/blob/30a04ba342bf9c101677c3081398e2c111400b3d/src/plone/restapi/services/search/get.py#L7.
  • Always queries Plone's portal_catalog. Uses serializers to serialize content to JSON. We implement custom serializers in GEVER to extend it with type-specific custom attributes https://github.com/4teamwork/opengever.core/blob/b2f71596b4ef1c34d3e6277636a65ffcb6a9d9cd/opengever/api/serializer.py
  • Documented in https://plonerestapi.readthedocs.io/en/latest/content.html#reading-a-resource-with-get and https://docs.onegovgever.ch/dev-manual/api/summaries/#summarische-auflistungen
  • Scoped by context (Provides summary of children)
• @listing implemented in Listing in

  opengever.core/opengever/api/listing.py

  Line 232 in b2f7159

  class Listing(Service):

  • Transparently uses SOLR, when available, some functionality it provides like facets are only available with SOLR
  • Documented in https://docs.onegovgever.ch/dev-manual/api/listings/#auflistungen
  • Scoped by context
  • Returns an object with query metadata as following:

{
  "@id": "http://localhost:8080/fd/ordnungssystem/fuehrung/dossier-23/@listing?name=documents&columns%3Alist=title&columns%3Alist=modified&columns%3Alist=filesize",
  "b_size": 25,
  "b_start": 0,
  "items": [
    {
      "@id": "http://localhost:8080/fd//ordnungssystem/fuehrung/dossier-23/document-59",
      "filesize": 12303,
      "modified": "2019-03-11T13:50:14+00:00",
      "title": "Ein Brief"
    }
  ],
  "items_total": 1
}

• @solrsearch implemented in the SolrSearchGet class in

  opengever.core/opengever/api/solrsearch.py

  Line 9 in b2f7159

  class SolrSearchGet(Service):

  • Only searches in SOLR. Needs to be changed to avoid trimming SOLR metadata as described in @solrsearch trims SOLR result/query metadata and http status #5816
  • Documented in https://docs.onegovgever.ch/dev-manual/api/searching/#solr-suche
  • Not scoped by context
  • Returns a list of queried objects without query metadata as following:

[
    {
        "Description": "",
        "Title": "Lorem ipsum dolor",
        "UID": "97d068586e2447479704a7835b547437",
        "path": "/fd/sitzungen/committee-1/submitted-proposal-151/document-29"
    }
]

• @search implemented in plone.restapi in the SearchGet class in https://github.com/plone/plone.restapi/blob/30a04ba342bf9c101677c3081398e2c111400b3d/src/plone/restapi/services/search/get.py#L7
  • Always Queries Plone's portal_catalog (Not overriden in GEVER)
  • Documented in https://docs.onegovgever.ch/dev-manual/api/searching/#suche
  • Scoped by context
  • Returns an object with query metadata as following:

{
    "@id": "http://localhost:8080/fd/ordnungssystem/fuehrung/kommunikation/allgemeines/dossier-1/@search?SearchableText=lorem",
    "items": [
        {
            "@id": "http://localhost:8080/fd/ordnungssystem/fuehrung/kommunikation/allgemeines/dossier-1",
            "@type": "opengever.dossier.businesscasedossier",
            "description": "",
            "review_state": "dossier-state-active",
            "title": "Jahresdossier 2014"
        }
    ],
    "items_total": 1
}

• @livesearch implemented in the GeverLiveSearchGet class in https://github.com/4teamwork/opengever.core/blob/master/opengever/api/livesearch.py#L8
  • Transparently uses SOLR when available, portal_catalog otherwise. In case of SOLR re-uses opengever.base.browserlivesearch.LiveSearchReplyView. In case of catalog reuses plone.restapi.services.SearchGet.
  • Couldn't find documentation
  • Not scoped by context
  • Returns a list of queried objects without query metadata as following:

[
    {
        "@id": "http://localhost:8080/fd/ordnungssystem/gesellschaftliches/dossier-28/document-48",
        "@type": "opengever.document.document",
        "filename": "Kopie von lorem.docx",
        "title": "Kopie von lorem"
    }
]

• @@livesearch_reply browser view implemented in the class LiveSearchReplyView in

  opengever.core/opengever/base/browser/livesearch.py

  Line 38 in b2f7159

  class LiveSearchReplyView(BrowserView):

  • Transparently uses SOLR, when available
  • Not scoped by context
  • Renders HTML
• @@search browser view implemented in the class OpengeverSearch in

  opengever.core/opengever/base/browser/search.py

  Line 30 in b2f7159

  class OpengeverSearch(Search):

  • Transparently uses SOLR, when available
  • Not scoped by context
  • Renders HTML