Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update via SDK Studio #3

Merged
merged 1 commit into from
Feb 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,12 @@ cd groq-node
# With yarn
yarn link
cd ../my-package
yarn link groq
yarn link groq-sdk

# With pnpm
pnpm link --global
cd ../my-package
pnpm link -—global groq
pnpm link -—global groq-sdk
```

## Running tests
Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Groq Node API Library

[![NPM version](https://img.shields.io/npm/v/groq.svg)](https://npmjs.org/package/groq)
[![NPM version](https://img.shields.io/npm/v/groq-sdk.svg)](https://npmjs.org/package/groq-sdk)

This library provides convenient access to the Groq REST API from server-side TypeScript or JavaScript.

Expand All @@ -9,9 +9,9 @@ The REST API documentation can be found [on console.groq.com](https://console.gr
## Installation

```sh
npm install --save groq
npm install --save groq-sdk
# or
yarn add groq
yarn add groq-sdk
```

## Usage
Expand All @@ -20,7 +20,7 @@ The full API of this library can be found in [api.md](api.md).

<!-- prettier-ignore -->
```js
import Groq from 'groq';
import Groq from 'groq-sdk';

const groq = new Groq();

Expand All @@ -42,7 +42,7 @@ This library includes TypeScript definitions for all request params and response

<!-- prettier-ignore -->
```ts
import Groq from 'groq';
import Groq from 'groq-sdk';

const groq = new Groq();

Expand Down Expand Up @@ -196,19 +196,19 @@ add the following import before your first import `from "Groq"`:
```ts
// Tell TypeScript and the package to use the global web fetch instead of node-fetch.
// Note, despite the name, this does not add any polyfills, but expects them to be provided if needed.
import 'groq/shims/web';
import Groq from 'groq';
import 'groq-sdk/shims/web';
import Groq from 'groq-sdk';
```

To do the inverse, add `import "groq/shims/node"` (which does import polyfills).
To do the inverse, add `import "groq-sdk/shims/node"` (which does import polyfills).
This can also be useful if you are getting the wrong TypeScript types for `Response` - more details [here](https://github.com/groq/groq-node/tree/main/src/_shims#readme).

You may also provide a custom `fetch` function when instantiating the client,
which can be used to inspect or alter the `Request` or `Response` before/after each request:

```ts
import { fetch } from 'undici'; // as one example
import Groq from 'groq';
import Groq from 'groq-sdk';

const client = new Groq({
fetch: async (url: RequestInfo, init?: RequestInfo): Promise<Response> => {
Expand Down Expand Up @@ -265,7 +265,7 @@ TypeScript >= 4.5 is supported.
The following runtimes are supported:

- Node.js 18 LTS or later ([non-EOL](https://endoflife.date/nodejs)) versions.
- Deno v1.28.0 or higher, using `import Groq from "npm:groq"`.
- Deno v1.28.0 or higher, using `import Groq from "npm:groq-sdk"`.
- Bun 1.0 or later.
- Cloudflare Workers.
- Vercel Edge Runtime.
Expand Down
6 changes: 3 additions & 3 deletions build
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ node scripts/check-version.cjs

# Build into dist and will publish the package from there,
# so that src/resources/foo.ts becomes <package root>/resources/foo.js
# This way importing from `"groq/resources/foo"` works
# This way importing from `"groq-sdk/resources/foo"` works
# even with `"moduleResolution": "node"`

rm -rf dist; mkdir dist
Expand Down Expand Up @@ -44,8 +44,8 @@ node scripts/postprocess-files.cjs

# make sure that nothing crashes when we require the output CJS or
# import the output ESM
(cd dist && node -e 'require("groq")')
(cd dist && node -e 'import("groq")' --input-type=module)
(cd dist && node -e 'require("groq-sdk")')
(cd dist && node -e 'import("groq-sdk")' --input-type=module)

if command -v deno &> /dev/null && [ -e ./build-deno ]
then
Expand Down
102 changes: 52 additions & 50 deletions examples/chat_completion.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,56 +3,58 @@ const Groq = require('groq');
const groq = new Groq();

async function main() {
groq.chat.completions.create({
//
// Required parameters
//
messages: [
// Set an optional system message. This sets the behavior of the
// assistant and can be used to provide specific instructions for
// how it should behave throughout the conversation.
{
"role": "system",
"content": "you are a helpful assistant."
},
// Set a user message for the assistant to respond to.
{
"role": "user",
"content": "Explain the importance of low latency LLMs",
}
],

// The language model which will generate the completion.
model: "mixtral-8x7b-32768",

//
// Optional parameters
//

// Controls randomness: lowering results in less random completions.
// As the temperature approaches zero, the model will become deterministic
// and repetitive.
temperature: 0.5,

// The maximum number of tokens to generate. Requests can use up to
// 2048 tokens shared between prompt and completion.
max_tokens: 1024,

// Controls diversity via nucleus sampling: 0.5 means half of all
// likelihood-weighted options are considered.
top_p: 1,

// A stop sequence is a predefined or user-specified text string that
// signals an AI to stop generating content, ensuring its responses
// remain focused and concise. Examples include punctuation marks and
// markers like "[end]".
stop: null,

// If set, partial message deltas will be sent.
stream: false,
}).then((chatCompletion) => {
process.stdout.write(chatCompletion.choices[0]?.message?.content || '');
})
groq.chat.completions
.create({
//
// Required parameters
//
messages: [
// Set an optional system message. This sets the behavior of the
// assistant and can be used to provide specific instructions for
// how it should behave throughout the conversation.
{
role: 'system',
content: 'you are a helpful assistant.',
},
// Set a user message for the assistant to respond to.
{
role: 'user',
content: 'Explain the importance of low latency LLMs',
},
],

// The language model which will generate the completion.
model: 'mixtral-8x7b-32768',

//
// Optional parameters
//

// Controls randomness: lowering results in less random completions.
// As the temperature approaches zero, the model will become deterministic
// and repetitive.
temperature: 0.5,

// The maximum number of tokens to generate. Requests can use up to
// 2048 tokens shared between prompt and completion.
max_tokens: 1024,

// Controls diversity via nucleus sampling: 0.5 means half of all
// likelihood-weighted options are considered.
top_p: 1,

// A stop sequence is a predefined or user-specified text string that
// signals an AI to stop generating content, ensuring its responses
// remain focused and concise. Examples include punctuation marks and
// markers like "[end]".
stop: null,

// If set, partial message deltas will be sent.
stream: false,
})
.then((chatCompletion) => {
process.stdout.write(chatCompletion.choices[0]?.message?.content || '');
});
}

main();
14 changes: 7 additions & 7 deletions examples/chat_completion_stop.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,18 @@ async function main() {
// assistant and can be used to provide specific instructions for
// how it should behave throughout the conversation.
{
"role": "system",
"content": "you are a helpful assistant."
role: 'system',
content: 'you are a helpful assistant.',
},
// Set a user message for the assistant to respond to.
{
"role": "user",
"content": "Start at 1 and count to 10. Separate each number with a comma and a space"
}
role: 'user',
content: 'Start at 1 and count to 10. Separate each number with a comma and a space',
},
],

// The language model which will generate the completion.
model: "mixtral-8x7b-32768",
model: 'mixtral-8x7b-32768',

//
// Optional parameters
Expand All @@ -50,7 +50,7 @@ async function main() {
// For this example, we will use ", 6" so that the llm stops counting at 5.
// If multiple stop values are needed, an array of string may be passed,
// stop: [", 6", ", six", ", Six"]
stop: ", 6",
stop: ', 6',

// If set, partial message deltas will be sent.
stream: true,
Expand Down
12 changes: 6 additions & 6 deletions examples/chat_completion_streaming.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,18 @@ async function main() {
// assistant and can be used to provide specific instructions for
// how it should behave throughout the conversation.
{
"role": "system",
"content": "you are a helpful assistant."
role: 'system',
content: 'you are a helpful assistant.',
},
// Set a user message for the assistant to respond to.
{
"role": "user",
"content": "Explain the importance of low latency LLMs",
}
role: 'user',
content: 'Explain the importance of low latency LLMs',
},
],

// The language model which will generate the completion.
model: "mixtral-8x7b-32768",
model: 'mixtral-8x7b-32768',

//
// Optional parameters
Expand Down
6 changes: 3 additions & 3 deletions jest.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ const config: JestConfigWithTsJest = {
preset: 'ts-jest/presets/default-esm',
testEnvironment: 'node',
moduleNameMapper: {
'^groq$': '<rootDir>/src/index.ts',
'^groq/_shims/auto/(.*)$': '<rootDir>/src/_shims/auto/$1-node',
'^groq/(.*)$': '<rootDir>/src/$1',
'^groq-sdk$': '<rootDir>/src/index.ts',
'^groq-sdk/_shims/auto/(.*)$': '<rootDir>/src/_shims/auto/$1-node',
'^groq-sdk/(.*)$': '<rootDir>/src/$1',
},
modulePathIgnorePatterns: [
'<rootDir>/ecosystem-tests/',
Expand Down
6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"name": "groq",
"name": "groq-sdk",
"version": "0.1.0",
"description": "The official TypeScript library for the Groq API",
"author": "Groq <[email protected]>",
Expand Down Expand Up @@ -60,8 +60,8 @@
"./shims/web.mjs"
],
"imports": {
"groq": ".",
"groq/*": "./src/*"
"groq-sdk": ".",
"groq-sdk/*": "./src/*"
},
"exports": {
"./_shims/auto/*": {
Expand Down
4 changes: 2 additions & 2 deletions scripts/postprocess-files.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ const fs = require('fs');
const path = require('path');
const { parse } = require('@typescript-eslint/parser');

const pkgImportPath = process.env['PKG_IMPORT_PATH'] ?? 'groq/'
const pkgImportPath = process.env['PKG_IMPORT_PATH'] ?? 'groq-sdk/'

const distDir =
process.env['DIST_PATH'] ?
Expand Down Expand Up @@ -142,7 +142,7 @@ async function postprocess() {

if (file.endsWith('.d.ts')) {
// work around bad tsc behavior
// if we have `import { type Readable } from 'groq/_shims/index'`,
// if we have `import { type Readable } from 'groq-sdk/_shims/index'`,
// tsc sometimes replaces `Readable` with `import("stream").Readable` inline
// in the output .d.ts
transformed = transformed.replace(/import\("stream"\).Readable/g, 'Readable');
Expand Down
32 changes: 16 additions & 16 deletions src/_shims/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# 👋 Wondering what everything in here does?

`groq` supports a wide variety of runtime environments like Node.js, Deno, Bun, browsers, and various
`groq-sdk` supports a wide variety of runtime environments like Node.js, Deno, Bun, browsers, and various
edge runtimes, as well as both CommonJS (CJS) and EcmaScript Modules (ESM).

To do this, `groq` provides shims for either using `node-fetch` when in Node (because `fetch` is still experimental there) or the global `fetch` API built into the environment when not in Node.
To do this, `groq-sdk` provides shims for either using `node-fetch` when in Node (because `fetch` is still experimental there) or the global `fetch` API built into the environment when not in Node.

It uses [conditional exports](https://nodejs.org/api/packages.html#conditional-exports) to
automatically select the correct shims for each environment. However, conditional exports are a fairly new
Expand All @@ -15,32 +15,32 @@ getting the wrong raw `Response` type from `.asResponse()`, for example.

The user can work around these issues by manually importing one of:

- `import 'groq/shims/node'`
- `import 'groq/shims/web'`
- `import 'groq-sdk/shims/node'`
- `import 'groq-sdk/shims/web'`

All of the code here in `_shims` handles selecting the automatic default shims or manual overrides.

### How it works - Runtime

Runtime shims get installed by calling `setShims` exported by `groq/_shims/registry`.
Runtime shims get installed by calling `setShims` exported by `groq-sdk/_shims/registry`.

Manually importing `groq/shims/node` or `groq/shims/web`, calls `setShims` with the respective runtime shims.
Manually importing `groq-sdk/shims/node` or `groq-sdk/shims/web`, calls `setShims` with the respective runtime shims.

All client code imports shims from `groq/_shims/index`, which:
All client code imports shims from `groq-sdk/_shims/index`, which:

- checks if shims have been set manually
- if not, calls `setShims` with the shims from `groq/_shims/auto/runtime`
- re-exports the installed shims from `groq/_shims/registry`.
- if not, calls `setShims` with the shims from `groq-sdk/_shims/auto/runtime`
- re-exports the installed shims from `groq-sdk/_shims/registry`.

`groq/_shims/auto/runtime` exports web runtime shims.
If the `node` export condition is set, the export map replaces it with `groq/_shims/auto/runtime-node`.
`groq-sdk/_shims/auto/runtime` exports web runtime shims.
If the `node` export condition is set, the export map replaces it with `groq-sdk/_shims/auto/runtime-node`.

### How it works - Type time

All client code imports shim types from `groq/_shims/index`, which selects the manual types from `groq/_shims/manual-types` if they have been declared, otherwise it exports the auto types from `groq/_shims/auto/types`.
All client code imports shim types from `groq-sdk/_shims/index`, which selects the manual types from `groq-sdk/_shims/manual-types` if they have been declared, otherwise it exports the auto types from `groq-sdk/_shims/auto/types`.

`groq/_shims/manual-types` exports an empty namespace.
Manually importing `groq/shims/node` or `groq/shims/web` merges declarations into this empty namespace, so they get picked up by `groq/_shims/index`.
`groq-sdk/_shims/manual-types` exports an empty namespace.
Manually importing `groq-sdk/shims/node` or `groq-sdk/shims/web` merges declarations into this empty namespace, so they get picked up by `groq-sdk/_shims/index`.

`groq/_shims/auto/types` exports web type definitions.
If the `node` export condition is set, the export map replaces it with `groq/_shims/auto/types-node`, though TS only picks this up if `"moduleResolution": "nodenext"` or `"moduleResolution": "bundler"`.
`groq-sdk/_shims/auto/types` exports web type definitions.
If the `node` export condition is set, the export map replaces it with `groq-sdk/_shims/auto/types-node`, though TS only picks this up if `"moduleResolution": "nodenext"` or `"moduleResolution": "bundler"`.
2 changes: 1 addition & 1 deletion src/_shims/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
* Disclaimer: modules in _shims aren't intended to be imported by SDK users.
*/
import { manual } from './manual-types';
import * as auto from 'groq/_shims/auto/types';
import * as auto from 'groq-sdk/_shims/auto/types';
import { type RequestOptions } from '../core';

type SelectType<Manual, Auto> = unknown extends Manual ? Auto : Manual;
Expand Down
2 changes: 1 addition & 1 deletion src/_shims/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
* Disclaimer: modules in _shims aren't intended to be imported by SDK users.
*/
const shims = require('./registry');
const auto = require('groq/_shims/auto/runtime');
const auto = require('groq-sdk/_shims/auto/runtime');
if (!shims.kind) shims.setShims(auto.getRuntime(), { auto: true });
for (const property of Object.keys(shims)) {
Object.defineProperty(exports, property, {
Expand Down
Loading