Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add OpenSearch implementation #107

Draft
wants to merge 96 commits into
base: master
Choose a base branch
from
Draft

Add OpenSearch implementation #107

wants to merge 96 commits into from

Commits on Jun 13, 2024

  1. Create initial GitHub Actions workflow

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    a382a58 View commit details
    Browse the repository at this point in the history

  2. Add OpenSearch dependency

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    7e4662c View commit details
    Browse the repository at this point in the history

  3. Create initial OpenSearch interface

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    7d35230 View commit details
    Browse the repository at this point in the history

  4. Add OpenSearch insertion and deletion functions

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    86cf6fa View commit details
    Browse the repository at this point in the history

  5. Add OpenSearch property functions

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    0f59b23 View commit details
    Browse the repository at this point in the history

  6. Add openmock dependency

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    21bbae2 View commit details
    Browse the repository at this point in the history

  7. Add OpenSearch example notebooks

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    375b1fe View commit details
    Browse the repository at this point in the history

  8. Add functions required for OpenSearch CLI

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    1ffd18d View commit details
    Browse the repository at this point in the history

  9. Add function to add properties

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    4b6103c View commit details
    Browse the repository at this point in the history

  10. Add function to rename properties

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    d52a2bf View commit details
    Browse the repository at this point in the history

  11. Add function to delete properties

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    e10bee1 View commit details
    Browse the repository at this point in the history

  12. Add luqum query parsing for OpenSearch

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    4a47df3 View commit details
    Browse the repository at this point in the history

  13. Fix unit tests for OpenSearch 

    Made schema analysis optional for query building, primarily since get_mapping is not supported by openmock, although it may be useful in itself.
    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    3426d54 View commit details
    Browse the repository at this point in the history

  14. Refactor default OpenSearch query

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    131e41f View commit details
    Browse the repository at this point in the history

  15. Add function to return data as dictionary

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    13fcde8 View commit details
    Browse the repository at this point in the history

  16. Update OpenSearch useage tutorial

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    1d082d0 View commit details
    Browse the repository at this point in the history

  17. Add function to delete documents by ID

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    b94ca47 View commit details
    Browse the repository at this point in the history

  18. Add function to update documents by ID

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    fae3062 View commit details
    Browse the repository at this point in the history

  19. Update extra info parsing for Lucene-like inputs 

    Add option for colons when parsing key-value pairs for extra
information, to allow formatting closer to Lucene queries
    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    e2e8f92 View commit details
    Browse the repository at this point in the history

  20. Add docstrings and type hints

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    65fcfeb View commit details
    Browse the repository at this point in the history

  21. Tidy code 

    Removes Pylance type errors due to incorrect type hints from lark parser
    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    4c42cd9 View commit details
    Browse the repository at this point in the history

  22. Fix formatting for non-string value error

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    bc09c6d View commit details
    Browse the repository at this point in the history

  23. Add class to read csv data

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    8b1294e View commit details
    Browse the repository at this point in the history

  24. Enable spacegroups to be stored from Atoms

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    bf0938d View commit details
    Browse the repository at this point in the history

  25. Move structure files to list and make units optional

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    82ed451 View commit details
    Browse the repository at this point in the history

  26. Update README for OpenSearch

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    3f509f0 View commit details
    Browse the repository at this point in the history

  27. Update assertions for database tests

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    6b74e34 View commit details
    Browse the repository at this point in the history

  28. Add tests for csv reader

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    5899529 View commit details
    Browse the repository at this point in the history

  29. Add flake8 and black

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    f50be40 View commit details
    Browse the repository at this point in the history

  30. Apply black formatting

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    61727d1 View commit details
    Browse the repository at this point in the history

  31. Conform to flake8 style

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    014ef31 View commit details
    Browse the repository at this point in the history

  32. Add notebook for extra information examples

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    fa6faef View commit details
    Browse the repository at this point in the history

  33. Enable kwargs for OpenSearch client settings

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    7177392 View commit details
    Browse the repository at this point in the history

  34. Fix property function if property not present

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    bbf2c07 View commit details
    Browse the repository at this point in the history

  35. Run OpenSearch unit tests with GitHub Actions

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    92c5818 View commit details
    Browse the repository at this point in the history

  36. Add refresh function for OpenSearch

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    5a54d1a View commit details
    Browse the repository at this point in the history

  37. Update unit tests

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    b1b81d9 View commit details
    Browse the repository at this point in the history

  38. Fix delete property function

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    684aa18 View commit details
    Browse the repository at this point in the history

  39. Add tolerance for failed connections in testing

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    d8bef29 View commit details
    Browse the repository at this point in the history

  40. Update README

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    6e45d18 View commit details
    Browse the repository at this point in the history

  41. Update OpenSearch notebook

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    c9006a5 View commit details
    Browse the repository at this point in the history

  42. Apply black formatting

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    99bfd0f View commit details
    Browse the repository at this point in the history

  43. Improve property file reader 

    Throw errors for incorrect encoding, rather than replacing characters
silently, and allow Excel spreadsheets, in addition to csv files.
    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    e6476ad View commit details
    Browse the repository at this point in the history

  44. Add benchmarking examples

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    e2e9664 View commit details
    Browse the repository at this point in the history

  45. Enable uploading list of extra info

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    97e3898 View commit details
    Browse the repository at this point in the history

  46. Extend benchmarking

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    4cf65ee View commit details
    Browse the repository at this point in the history

  47. Fix OpenSearch CLI queries 

    Also changes implementation for MongoDB to enable lists of queries in
for non-CLI use.
    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    da36f9a View commit details
    Browse the repository at this point in the history

  48. Tidy unit tests

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    cd0333d View commit details
    Browse the repository at this point in the history

  49. Add option to disabled SSL

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    40d5ba3 View commit details
    Browse the repository at this point in the history

  50. Add initial CLI integration tests

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    93d86f5 View commit details
    Browse the repository at this point in the history

  51. Catch process errors in CLI tests

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    622297f View commit details
    Browse the repository at this point in the history

  52. Add chunk and timeout kwargs for bulk push

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    4d1baea View commit details
    Browse the repository at this point in the history

  53. Refactor backend code

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    34dd0b8 View commit details
    Browse the repository at this point in the history

  54. Update minimum openserch for security

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    9edd356 View commit details
    Browse the repository at this point in the history

  55. Update admin default password in CI

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    bc2c25d View commit details
    Browse the repository at this point in the history

  56. Add query test

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    67b9a67 View commit details
    Browse the repository at this point in the history

  57. Add OpenSearch refresh CLI command

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    a0f1fc3 View commit details
    Browse the repository at this point in the history

  58. Replace luqum with native query_string

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    1fd03f3 View commit details
    Browse the repository at this point in the history

  59. Speed up property function

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    dcba2d2 View commit details
    Browse the repository at this point in the history

  60. Apply black formatter

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    05dae68 View commit details
    Browse the repository at this point in the history

  61. Add timeout to count

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    5c7edf0 View commit details
    Browse the repository at this point in the history

  62. Add comment for property function

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    085a581 View commit details
    Browse the repository at this point in the history

  63. Fix deleting keys via CLI

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    a018e04 View commit details
    Browse the repository at this point in the history

  64. Fix adding keys via CLI

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    623a5b8 View commit details
    Browse the repository at this point in the history

  65. Fix renaming properties

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    a56761b View commit details
    Browse the repository at this point in the history

  66. Add notebook with example queries

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    0dc63fd View commit details
    Browse the repository at this point in the history

  67. Allow multiple properties to be returned

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    881137b View commit details
    Browse the repository at this point in the history

  68. Fix CLI code formatting

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    f0314d3 View commit details
    Browse the repository at this point in the history

  69. Apply suggestions from code review 

    Co-authored-by: Jacob Wilkins <[email protected]>
    @ElliottKasoar @oerc0122
    ElliottKasoar and oerc0122 committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    ef08360 View commit details
    Browse the repository at this point in the history

  70. Tidy for flake8

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    0f47586 View commit details
    Browse the repository at this point in the history

  71. Tidy README formatting

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    8702649 View commit details
    Browse the repository at this point in the history

  72. Tidy optional type hints

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    562d1d4 View commit details
    Browse the repository at this point in the history

  73. Add return type hint

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    7781612 View commit details
    Browse the repository at this point in the history

  74. Apply suggestions from code review 

    Co-authored-by: Jacob Wilkins <[email protected]>
    @ElliottKasoar @oerc0122
    ElliottKasoar and oerc0122 committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    932b4e2 View commit details
    Browse the repository at this point in the history

  75. Update abcd/backends/atoms_opensearch.py 

    Co-authored-by: Jacob Wilkins <[email protected]>
    @ElliottKasoar @oerc0122
    ElliottKasoar and oerc0122 committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    7a6affe View commit details
    Browse the repository at this point in the history

  76. Fix type Optional Union type hints

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    5abd153 View commit details
    Browse the repository at this point in the history

  77. Fix connection type

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    5356d39 View commit details
    Browse the repository at this point in the history

  78. Apply suggestions from code review 

    Co-authored-by: Jacob Wilkins <[email protected]>
    @ElliottKasoar @oerc0122
    ElliottKasoar and oerc0122 committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    9b05307 View commit details
    Browse the repository at this point in the history

  79. Fix renaming keys

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    c546d6d View commit details
    Browse the repository at this point in the history

  80. Tidy setting db

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    3dddc38 View commit details
    Browse the repository at this point in the history

  81. Tidy code

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    c77316c View commit details
    Browse the repository at this point in the history

  82. Tidy logs

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    3164fbc View commit details
    Browse the repository at this point in the history

  83. Tidy code

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    70449ee View commit details
    Browse the repository at this point in the history

  84. Fix extra info

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    2a1ef18 View commit details
    Browse the repository at this point in the history

  85. Tidy code

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    3276277 View commit details
    Browse the repository at this point in the history

  86. Update mongomock tests for pytest

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    8da3c11 View commit details
    Browse the repository at this point in the history

  87. Update mock opensearch tests for pytest

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    02afdc9 View commit details
    Browse the repository at this point in the history

  88. Update CLI tests for pytest

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    cd95560 View commit details
    Browse the repository at this point in the history

  89. Update property tests for pytest

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    cf70beb View commit details
    Browse the repository at this point in the history

  90. Update opensearch tests for pytest

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    b95b0b8 View commit details
    Browse the repository at this point in the history

  91. Fix opensearch mock tests

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    84637db View commit details
    Browse the repository at this point in the history

  92. Fix opensearch test

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    be2d64b View commit details
    Browse the repository at this point in the history

  93. Tidy code

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    db3c36a View commit details
    Browse the repository at this point in the history

  94. Fix opensearch test

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    74d24c7 View commit details
    Browse the repository at this point in the history

  95. Fix mock opensearch tests

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    3ba579d View commit details
    Browse the repository at this point in the history

  96. Fix histogram query

    @ElliottKasoar
    ElliottKasoar committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    39086be View commit details
    Browse the repository at this point in the history