WG-13: update python-client and add example used in load testing (#125)

* Add client configuration

Removed (inadvertantly?) in 6b919b3

* Bump package version

* Update readme for client

* Add example used to test prod

* Add link to main readme

* Remove blank lines

* Bump poetry versions

* Update versions of request and urllib3

To work around https://stackoverflow.com/questions/76172523/error-httpresponse-object-has-no-attribute-strict-when-verifying-id-token-wi

* Bump github actions versions

* Update CI poetry version

.github/workflows/main.yml

@@ -9,9 +9,9 @@ jobs:
   Linting:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       - name: Setup Python
-        uses: actions/setup-python@v1
+        uses: actions/setup-python@v4
         with:
           python-version: 3.9
       - name: Install flake8
@@ -38,17 +38,17 @@ jobs:
           --health-timeout 5s
           --health-retries 5
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
     - name: Setup Python
-      uses: actions/setup-python@v1
+      uses: actions/setup-python@v4
       with:
         python-version: 3.9
     - name: Install ffmpeg
       run: sudo apt-get install -y ffmpeg
     - name: Setup Poetry
       uses: abatilo/[email protected]
       with:
-        poetry-version: 1.1.13
+        poetry-version: 1.4.2
     - uses: actions/cache@v1
       with:
         path: ~/.cache/pypoetry/virtualenvs
@@ -76,7 +76,7 @@ jobs:
           --health-timeout 5s
           --health-retries 5
     steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v3
     - uses: docker/login-action@v1
       with:
         password: ${{ secrets.DOCKER_PASSWORD }}

Dockerfile

@@ -6,7 +6,7 @@ RUN apt-get update -q && apt-get install -q -y \
   curl \
   git
 
-ENV POETRY_VERSION=1.1.13
+ENV POETRY_VERSION=1.4.2
 ENV POETRY_HOME=/opt/poetry
 ENV PATH="$POETRY_HOME/bin:$PATH"
 RUN curl -sSL https://install.python-poetry.org | python3 -

Dockerfile.potree

@@ -27,7 +27,7 @@ RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 2
 
 RUN pip3 install --upgrade pip
 
-ENV POETRY_VERSION=1.1.13
+ENV POETRY_VERSION=1.4.2
 ENV POETRY_HOME=/opt/poetry
 ENV PATH="$POETRY_HOME/bin:$PATH"
 RUN curl -sSL https://install.python-poetry.org | python3 -

client/README.md

@@ -0,0 +1,3 @@
+This is the configuration files used to create a python client.
+
+See [main README.md](../README.md) for info on generating the client.

client/README.mustache

@@ -0,0 +1,110 @@
+# {{{projectName}}}
+{{#appDescription}}
+{{{appDescription}}}
+{{/appDescription}}
+
+
+- API version: {{appVersion}}
+- Package version: {{packageVersion}}
+{{^hideGenerationTimestamp}}
+- Build date: {{generatedDate}}
+{{/hideGenerationTimestamp}}
+- Build package: {{generatorClass}}
+{{#infoUrl}}
+For more information, please visit [{{{infoUrl}}}]({{{infoUrl}}})
+{{/infoUrl}}
+
+For more information about the [GeoAPI](https://github.com/TACC-Cloud/geoap) and how this client is generated using [Swagger Codegen](https://github.com/swagger-api/swagger-codegen), visit https://github.com/TACC-Cloud/geoap .
+
+## Requirements.
+
+Python 2.7 and 3.4+
+
+## Installation & Usage
+### pip install
+
+The python package can be found at [PyPi](https://pypi.org/project/geoapi-client/)
+
+```sh
+pip install {{{projectName}}} --user
+```
+
+Then import the package:
+```python
+import {{{packageName}}} 
+```
+
+## Getting Started
+
+Please follow the [installation procedure](#installation--usage) and then run the following:
+
+```python
+import geoapi_client
+from geoapi_client.rest import ApiException
+
+configuration = geoapi_client.Configuration()
+configuration.host = MY_HOST # e.g. https://agave.designsafe-ci.org/geo/v2
+configuration.api_key_prefix['Authorization'] = 'Bearer'
+configuration.api_key['Authorization'] = TOKEN
+
+api_client = geoapi_client.ApiClient(configuration)
+api_instance = geoapi_client.ProjectsApi(api_client=api_client)
+```
+
+To get api/help:
+
+
+```
+help(api_instance)
+```
+To list projects and create a project:
+```
+try:
+    projects = api_instance.get_projects()
+    print(projects)
+    project = api_instance.create_project(payload={"project": {"name": "My project"}})
+    print(project)
+except ApiException as e:
+    print("Exception: %s\n" % e)
+```
+
+## API Endpoints
+
+All URIs are relative to *{{basePath}}*
+
+Class | Method | HTTP request | Description
+------------ | ------------- | ------------- | -------------
+{{#apiInfo}}{{#apis}}{{#operations}}{{#operation}}*{{classname}}* | **{{operationId}}** | **{{httpMethod}}** {{path}} | {{#summary}}{{summary}}{{/summary}}
+{{/operation}}{{/operations}}{{/apis}}{{/apiInfo}}
+
+## Models
+
+{{#models}}{{#model}} - {{{classname}}}
+{{/model}}{{/models}}
+
+## Documentation For Authorization
+
+{{^authMethods}} All endpoints do not require authorization.
+{{/authMethods}}{{#authMethods}}{{#last}} Authentication schemes defined for the API:{{/last}}{{/authMethods}}
+{{#authMethods}}## {{{name}}}
+
+{{#isApiKey}}- **Type**: API key
+- **API key parameter name**: {{{keyParamName}}}
+- **Location**: {{#isKeyInQuery}}URL query string{{/isKeyInQuery}}{{#isKeyInHeader}}HTTP header{{/isKeyInHeader}}
+{{/isApiKey}}
+{{#isBasic}}- **Type**: HTTP basic authentication
+{{/isBasic}}
+{{#isOAuth}}- **Type**: OAuth
+- **Flow**: {{{flow}}}
+- **Authorization URL**: {{{authorizationUrl}}}
+- **Scopes**: {{^scopes}}N/A{{/scopes}}
+{{#scopes}} - **{{{scope}}}**: {{{description}}}
+{{/scopes}}
+{{/isOAuth}}
+
+{{/authMethods}}
+
+## Author
+
+Texas Advanced Computing Center
+[email protected]

client/config.json

@@ -0,0 +1,6 @@
+{
+  "packageName":"geoapi_client",
+  "projectName":"geoapi-client",
+  "packageVersion":"2.10",
+  "packageUrl":"https://github.com/TACC-Cloud/geoapi"
+}

client/setup.mustache

@@ -0,0 +1,56 @@
+# coding: utf-8
+
+{{>partial_header}}
+
+from setuptools import setup, find_packages  # noqa: H301
+
+NAME = "{{{projectName}}}"
+VERSION = "{{packageVersion}}"
+{{#apiInfo}}
+{{#apis}}
+{{^hasMore}}
+# To install the library, run the following
+#
+# python setup.py install
+#
+# prerequisite: setuptools
+# http://pypi.python.org/pypi/setuptools
+
+REQUIRES = [
+    "certifi>=2017.4.17",
+    "python-dateutil>=2.1",
+    "six>=1.10",
+    "urllib3>=1.23"
+]
+
+{{#asyncio}}
+REQUIRES.append("aiohttp")
+{{/asyncio}}
+{{#tornado}}
+REQUIRES.append("tornado")
+{{/tornado}}
+
+# read the contents of your README file
+from os import path
+import io
+this_directory = path.abspath(path.dirname(__file__))
+with io.open(path.join(this_directory, 'README.md'), encoding='utf-8') as f:
+    long_description = f.read()
+
+setup(
+    name=NAME,
+    version=VERSION,
+    description="{{appName}}: {{appDescription}} ",
+    author="Texas Advanced Computing Center",
+    author_email="[email protected]",
+    url="{{packageUrl}}",
+    keywords=["TACC", "{{appName}}"],
+    install_requires=REQUIRES,
+    packages=find_packages(),
+    include_package_data=True,
+    long_description=long_description,
+    long_description_content_type='text/markdown'
+)
+{{/hasMore}}
+{{/apis}}
+{{/apiInfo}}

client_use_examples/load_test_point_cloud_import_prod.py

@@ -0,0 +1,37 @@
+import geoapi_client
+from geoapi_client.rest import ApiException
+import concurrent.futures
+import os
+
+TOKEN = os.environ['TOKEN']
+EXAMPLE_FILE = {"files": [{"path": "/3_Deliverables/Safeway Supermarket/Pointclouds/TLS/LAZ_combined/Safeway_combined_3cm_20190329.laz", "system": "project-159846449346309655-242ac119-0001-012"}]}
+configuration = geoapi_client.Configuration()
+configuration.host = "https://agave.designsafe-ci.org/geo/v2"
+configuration.api_key_prefix['Authorization'] = 'Bearer'
+configuration.api_key['Authorization'] = TOKEN
+
+api_client = geoapi_client.ApiClient(configuration)
+api_instance = geoapi_client.ProjectsApi(api_client=api_client)
+
+def create_point_cloud_and_import_file(project_id, index):
+    point_cloud = api_instance.add_point_cloud(project_id=project_id, payload={"description": f"point_cloud{index}", "files_info": "dummy" })
+    point_cloud_importing = api_instance.import_point_cloud_file_from_tapis(project_id=project.id, point_cloud_id=point_cloud.id, payload=EXAMPLE_FILE)
+    print(f"submitted: {index} {point_cloud_importing}")
+    return index, point_cloud_importing
+
+try:
+    test_id = "Testing multiple point cloud load"
+    project = api_instance.create_project(payload={"project": {"name": f"My project {test_id}"}})
+    project_id = project.id
+    workers = 20
+    number_of_point_clouds=100
+    with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as executor:
+        importing_point_cloud_futures = [executor.submit(create_point_cloud_and_import_file, project_id, index) for index in list(range(1, number_of_point_clouds))]
+
+        results = []
+        for future in concurrent.futures.as_completed(importing_point_cloud_futures):
+            index, result = future.result()
+        results.append(result)
+        print(results)
+except ApiException as e:
+    print("Exception: %s\n" % e)