v0.3.1

Key Features

• Introduces the aqueduct.llm_op API and the aqueduct-llm package. Aqueduct
  now has support for invoking LLMs with a single API call and comes with
  pre-built Docker images optimized for executing LLMs on Kubernetes. The
  llm_op API supports both ad hoc execution, as pictured below, as well as
  batch execution over a list of inputs or a Pandas Series. See our
  documentation for more details.

  from aqueduct import Client, llm_op
  
  client = Client() # initialize Aqueduct client so we can check if the engine name below is valid
  vicuna = llm_op('vicuna_7b', engine='my_k8s_engine')
  vicuna('What is the best LLM?')

• Reorganizes integrations around the concept of resources. Resources are any
  external tool, system, or API that Aqueduct can connect to; existing data
  and compute integrations are automatically converted into resources. A
  container registry resource is added in this release, and future releases
  will introduce new resource types. The recommended SDK API for accessing
  resources is now client.resource, with client.integration slated to
  deprecated in a future release.
• Allows users to specify a custom Docker image when running an Aqueduct
  operator on Kubernetes. The Docker image is required to have the Aqueduct
  executor scaffolding installed; for more details, please see our
  documentation here.

Enhancements

• Improves logging and error handling when an operator fails because it's able
  to successfully generate a result, typically in the setup phase.
• Enables connecting a Databricks cluster to Aqueduct via the Python SDK.

Bugfixes

• Fixes bug where installing pre-requisites for using Aqueduct-managed
  Kubernetes clusters would fail on an M1 Mac with certain configurations.

All Changes

v0.2.12...v0.3.1

All commits

• Fix spark dkr build by @likawind in #1250
• Eng 2142 add ability to run the conda integration test by @likawind in #1216
• Fix bug where on-demand k8s integration registration fails on Mac by @cw75 in #1242
• [1/n] Allow pypi pkg version injection by splitting version into a separate file by @likawind in #1259
• Updates README.md by @vsreekanti in #1263
• Improve logging when operator failed before generating artifact by @likawind in #1252
• Adds DatabricksConfig to connect_config by @hsubbaraj-spiral in #1256
• Adds support for running integration tests against Airflow by @saurav-c in #1204
• Adds status checks for integration test Redshift cluster by @saurav-c in #1255
• Allow users to specify custom Docker image when running operators on K8s by @cw75 in #1264
• Increase server setup timeouts in integration tests by @likawind in #1254
• Implement use argument for LLM for decorators by @cw75 in #1234
• Add Docker images that includes the LLM library and dependencies by @cw75 in #1235
• Do not schedule execution of save operator if other computer operator… by @jpurusho65 in #1253
• Small improvement to ECR error surfacing by @cw75 in #1272
• Improves error message returned to user when Airflow creds are incorrect by @saurav-c in #1275
• Add MANIFEST file to LLM python package by @cw75 in #1278
• This PR contains everything that's part of resource reframing milestone 1 by @cw75 in #1277
• Rename integrations to resources on the SDK and example notebooks by @kenxu95 in #1276
• Add more white space to the bottom of the integrations page by @cw75 in #1279
• ENG-2895 Updates link for GCS storage documentation in GCS dialog. by @agiron123 in #1281
• Remove additional log line by @jpurusho65 in #1282