Using boards as kits with Build API

[TinaNikou]

We'd like some guidance on how to use boards as kits with the Build API. This came up when migrating the "Gemini Simple" example board (packages/example-boards/src/boards/playground/gemini-simple.ts) and using the Gemini Kit, but receiving an error when trying to pass the text property to the output. It looks like the build package doesn't know how to get ports from an invoked board, so it's failing.

Here is the code for the attempted migration:

import { board, input, output } from "@breadboard-ai/build";
import { gemini } from "@google-labs/gemini-kit";

const prompt = input({
  title: "Prompt",
  type: "string",
  examples: ["Write a rhyming poem about breadboards"]
});

const llm = gemini.text({
  $id: "llm",
  text: prompt
});

const response = output(llm.text)

export default board({
  title: "Gemini Simple",
  description: "The simplest possible example of using Gemini Kit.",
  version: "0.1.0",
  inputs: { prompt },
  outputs: { response },
});

And we get the error Argument of type 'AbstractValue<unknown> & ((...args: unknown[]) => unknown)' is not assignable to parameter of type 'OutputPortReference<JsonSerializable> | Input<JsonSerializable> | InputWithDefault<JsonSerializable>'. Type 'AbstractValue<unknown> & ((...args: unknown[]) => unknown)' is not assignable to type 'Input<JsonSerializable>'.

[aomarks]

Try importing the board you want directly, instead of via a kit:

import geminiText from '@google-labs/gemini-kit/boards/gemini-generator.js`

[TinaNikou]

Hmmm, I can't seem to get this working on my end so far, as the module can't be found. I've done a clean, reinstalled packages, and rebuilt, but no luck yet.

[aomarks]

The board wasn't exported from the gemini package. Fixed in #2559. You should be able to do this now:

import { geminiText } from "@google-labs/gemini-kit";

[dglazkov]

is the core problem that the Gemini Kit isn't exported for use with build API? In other words, users of build API can't use Gemini Kit?

[aomarks]

Gemini Text and Nano are already converted to the Build API, so we're good there since this is for Text. I filed #2570 for myself to convert the final board (Vision), and do some final cleanup.

[TinaNikou]

Can get it working another way, using logic from The Calculator Board (packages/example-boards/src/boards/playground/math.ts) example, by invoking the board:

import { annotate, board, input, object, output } from "@breadboard-ai/build";
import { invoke } from "@google-labs/core-kit";

const prompt = input({
  title: "Prompt",
  type: "string",
  examples: ["Write a rhyming poem about breadboards"]
});

const generator = input({
  type: annotate(object({}), {
    behavior: ["board"],
  }),
  default: { kind: "board", path: "gemini-generator.json" },
});

const llmResponse = invoke({
  $board: generator,
  text: prompt,
}).unsafeOutput("text");


const response = output(llmResponse)

export default board({
  title: "Gemini Simple",
  description: "The simplest possible example of using Gemini Kit.",
  version: "0.1.0",
  inputs: { prompt, generator },
  outputs: { response },
});

However, will hopefully manage to get a cleaner solution working.

[aomarks]

I think this should work for now! However I have filed #2571 for myself which I think is what you'll want for the cleaner solution. This will allow you to do this:

import {board} from '@breadboard-ai/build';
import {geminiText} from '@google-labs/gemini-kit';

const prompt = input();
const completion = geminiText({prompt});
export default board({inputs: {prompt}, outputs: {completion}, ...});

Under the hood this will automatically create an invoke node configured in the correct way to call the gemini text board. I'll bump this up in priority since I think it's important that boards act exactly like a discrete component (just import it and call it, that's it!).

[dglazkov]

We can just remove the vision board. It's for the model that's been deprecated and removed from Gemini API.

[TinaNikou]

That'd be perfect! I'll continue using invoke for any board migrations that need it for now. Thank you for the help! ❤️

[aomarks]

@TinaNikou I have landed #2613 which adds board composition. So the snippet in https://github.com/orgs/breadboard-ai/discussions/2542#discussioncomment-10090352 should now work; you can just instantiate a nested board now, and it will automatically generate the invoke node for you and wire everything up correctly.

[TinaNikou]

@aomarks I can now directly import and instantiate the board, but seems like the board input property types aren't quite right. All properties, except model, are never:

So when passing in my prompt input to the board's text property, like geminiText({ model: modelInput, text: prompt }), I get the following error:
Type 'Input<string>' is not assignable to type 'ValueOrOutputPort<never>'. Type 'Input<string>' is not assignable to type 'Input<never>'. Type 'string' is not assignable to type 'never'.ts(2322) and gemini-generator.d.ts(14, 5): The expected type comes from property 'text' which is declared here on type 'ValuesOrOutputPorts<ExtractPortTypes<{ systemInstruction: never; text: never; model: Input<"gemini-1.5-flash-latest" | "gemini-1.5-pro-latest">; responseMimeType: never; tools: never; context: never; useStreaming: never; retry: never; safetySettings: never; stopSequences: never; }>>''

Here's the snippet for the example:

import { board, enumeration, input, output } from "@breadboard-ai/build";
import { geminiText } from "@google-labs/gemini-kit";

const prompt = input({
  title: "Prompt",
  type: "string",
  examples: ["Write a rhyming poem about breadboards"]
});

const modelInput = input({
  type: enumeration("gemini-1.5-flash-latest", "gemini-1.5-pro-latest")
});

const llmResponse = geminiText({ model: modelInput, text: prompt });

const text = output(llmResponse.outputs.text)

export default board({
  title: "Gemini Simple",
  description: "The simplest possible example of using Gemini Kit.",
  version: "0.1.0",
  inputs: { prompt },
  outputs: { text },
});

Not looked into it properly yet, but reckon it's worth raising a new issue for it?