Add Google Lens Integration to Langchain JS/TS

[e-than-c]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

We’d like to propose the addition of Google Lens API support to the LangChain JavaScript/TypeScript library, matching similar functionality available in the LangChain Python version. This integration would create a tool for visual search and image analysis capabilities directly within LangChain for JS/TS users.

Motivation

Google Lens data provides access to powerful image recognition and search functionalities, which are used for extracting and identifying image content such as text, objects, landmarks, and more. Although the Python-based LangChain library already includes Google Lens integration, there is no equivalent in the JavaScript/TypeScript version. This gap limits the flexibility of JS/TS developers who wish to use Google Lens data in their applications.

Proposal (If applicable)

We propose creating a Google Lens tool using SerpApi for LangChain’s JavaScript/TypeScript version. This tool would allow users to perform reverse image searches, retrieve metadata, and access links to related resources in a manner consistent with the Python version’s structure.

The feature will include:

Image Subject Recognition:
Identify primary subjects (like notable people or objects) within images
Related Content Retrieval:
Fetch relevant images, articles, and links based on Google Lens reverse image searches.
Error Handling and Logging:
Handle errors with SerpApi calls and log them for debugging.

We aim to submit a pull request for this feature by mid-November.

Example Usage:

const tool = new GoogleLensTool(process.env.SERPAPI_API_KEY);
tool._call({ imageUrl: "https://i.imgur.com/image.png" })
  .then((results) => {
    console.log(`Results:`, results);
  });

[e-than-c]

Hi @jacoblee93, we were wondering if this is something we can create an issue for? We've been working on the coding for it

[e-than-c]

Hi @jacoblee93 , just wanted to hear your thoughts on this proposal whenever you had a moment. Thank you!