agent_app.md

Section 9: Applications and Frameworks

• 900 most popular open source AI tools:🏆What I learned from looking at 900 most popular open source AI tools list [Mar 2024]
• Open100: Top 100 Open Source achievements.
• Generative AI Design Patterns: 9 architecture patterns for working with LLMs. [Feb 2024]
• Awesome LLM Apps: A curated collection of awesome LLM apps built with RAG and AI agents. [Apr 2024]
• GenAI Agents:🏆Tutorials and implementations for various Generative AI Agent techniques, from basic to advanced. [Sep 2024]
• The Last Mile Problem: Why Your AI Models Stumble Before the Finish Line: According to Gartner, by 2025, at least 30 percent of GenAI projects will be abandoned after the POC stage. [25 Oct 2024]

Applications, Frameworks, and User Interface (UI/UX)

LLM Training/Build

1. fastText: A library for efficient learning of word representations and sentence classification [Aug 2016]
2. Pytorch: PyTorch is the most favorite library among researchers. Papers with code Trends [Sep 2016]
3. fairseq: a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling [Sep 2017]
4. huggingface/transformers: 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX. (github.com) [Oct 2018]
5. jax: JAX is Autograd (automatically differentiate native Python & Numpy) and XLA (compile and run NumPy) [Oct 2018]
6. Sentence Transformers: Python framework for state-of-the-art sentence, text and image embeddings. Useful for semantic textual similar, semantic search, or paraphrase mining. git [27 Aug 2019]
7. Weights & Biases: Visualizing and tracking your machine learning experiments wandb.ai doc: deeplearning.ai/wandb [Jan 2020]
8. mosaicml/llm-foundry: LLM training code for MosaicML foundation models [Jun 2022]
9. vLLM: Easy-to-use library for LLM inference and serving. [Feb 2023]
10. string2string: an open-source tool that offers a comprehensive suite of efficient algorithms for a broad range of string-to-string problems. [Mar 2023]
11. GPT4All: Open-source large language models that run locally on your CPU [Mar 2023]
12. Visual Blocks: Google visual programming framework that lets you create ML pipelines in a no-code graph editor. [Mar 2023]
13. LLaMA-Factory: Unify Efficient Fine-Tuning of 100+ LLMs [May 2023]
14. ollama: Running with Large language models locally [Jun 2023]
15. unsloth: Finetune Mistral, Gemma, Llama 2-5x faster with less memory! [Nov 2023]
16. LM Studio: UI for Discover, download, and run local LLMs [May 2024]
17. YaFSDP: Yet another Fully Sharded Data Parallel (FSDP): enhanced for distributed training. YaFSDP vs DeepSpeed. [May 2024]
18. exo: Run your own AI cluster at home with everyday devices [Jun 2024]
19. BitNet: Official inference framework for 1-bit LLMs [Aug 2024]
20. Meta Lingua: a minimal and fast LLM training and inference library designed for research. [Oct 2024]

LLM Application Development

1. mindsdb: The open-source virtual database for building AI from enterprise data. It supports SQL syntax for development and deployment, with over 70 technology and data integrations. [Aug 2018]
2. superduper: Build end-to-end AI-data workflows and applications with your favourite tools. [Aug 2022]
3. langflow: LangFlow is a UI for LangChain, designed with react-flow. [Feb 2023]
4. MiniChain: A tiny library for coding with llm [Feb 2023]
5. marvin: a lightweight AI toolkit for building natural language interfaces. [Mar 2023]
6. microsoft/Tokenizer: Tiktoken in C#: .NET and TypeScript implementation of BPE tokenizer for OpenAI LLMs. [Mar 2023]
7. Azure OpenAI Proxy: OpenAI API requests converting into Azure OpenAI API requests [Mar 2023]
8. ChainForge: An open-source visual programming environment for battle-testing prompts to LLMs. [Mar 2023]
9. E2B: an open-source infrastructure that allows you run to AI-generated code in secure isolated sandboxes in the cloud. [Mar 2023]
10. Flowise Drag & drop UI to build your customized LLM flow [Apr 2023]
11. Dify: an open-source platform for building applications with LLMs, featuring an intuitive interface for AI workflows and model management. [Apr 2023]
12. ThinkGPT: Chain of Thoughts library [Apr 2023]
13. langfuse: Traces, evals, prompt management and metrics to debug and improve your LLM application. [May 2023]
14. Superagent: AI Assistant Framework & API [May 2023]
15. Mem0:💡A self-improving memory layer for personalized AI experiences. [Jun 2023]
16. DemoGPT: Automatic generation of LangChain code [Jun 2023]
17. Spring AI: Developing AI applications for Java. [Jul 2023]
18. litellm: Python SDK to call 100+ LLM APIs in OpenAI format [Jul 2023]
19. Opencopilot: Build and embed open-source AI Copilots. [Aug 2023]
20. BISHENG: an open LLM application devops platform, focusing on enterprise scenarios. [Aug 2023]
21. langfun: leverages PyGlove to integrate LLMs and programming. [Aug 2023]
22. Letta (previously MemGPT): Virtual context management to extend the limited context window of LLM. A tiered memory system and a set of functions that allow it to manage its own memory. ref / git:old [12 Oct 2023]
23. mirascope: a library that simplifies working with LLMs via a unified interface for multiple providers. [Dec 2023]
24. Llama Stack: building blocks for Large Language Model (LLM) development [Jun 2024]

LLM Application

1. OpenBB: The first financial Platform that is free and fully open source. AI-powered workspace [Dec 2020]
2. knowledge: Tool for saving, searching, accessing, and exploring websites and files. Electron based app, built-in Chromium browser, knowledge graph [Jul 2021]
3. guardrails: Adding guardrails to large language models. [Jan 2023]
4. aider: AI pair programming in your terminal [Jan 2023]
5. BookGPT: Generate books based on your specification [Jan 2023]
6. KnowledgeGPT: Upload your documents and get answers to your questions, with citations [Jan 2023]
7. DocsGPT: Chatbot for document with your data [Feb 2023]
8. BIG-AGI FKA nextjs-chatgpt-app [Mar 2023]
9. Next.js AI Chatbot:💡An Open-Source AI Chatbot Template Built With Next.js and the AI SDK by Vercel. [May 2023]
10. dataline: Chat with your data - AI data analysis and visualization [Apr 2023]
11. pyspark-ai: English instructions and compile them into PySpark objects like DataFrames. [Apr 2023]
12. vanna: Chat with your SQL database [May 2023]
13. Continue: open-source AI code assistant inside of VS Code and JetBrains. [May 2023]
14. localGPT: Chat with your documents on your local device [May 2023]
15. anything-llm: All-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more. [Jun 2023]
16. Dialoqbase: Create custom chatbots with your own knowledge base using PostgreSQL [Jun 2023]
17. GPT Researcher: Autonomous agent designed for comprehensive online research [Jul 2023] / GPT Newspaper: Autonomous agent designed to create personalized newspapers [Jan 2024]
18. Postiz: AI social media scheduling tool. An alternative to: Buffer.com, Hypefury, Twitter Hunter. [Jul 2023]
19. SolidGPT: AI searching assistant for developers (VSCode Extension) [Aug 2023]
20. notesGPT: Record voice notes & transcribe, summarize, and get tasks [Nov 2023]
21. screenshot-to-code: Drop in a screenshot and convert it to clean code (HTML/Tailwind/React/Vue) [Nov 2023]
22. Geppeto: Advanced Slack bot using multiple AI models [Jan 2024]
23. code2prompt: a command-line tool (CLI) that converts your codebase into a single LLM prompt with a source tree [Mar 2024]
24. OpenHands: OpenHands (formerly OpenDevin), a platform for software development agents [Mar 2024]
25. Auto_Jobs_Applier_AIHawk: automates the jobs application [Aug 2024]
26. LlamaFS: Automatically renames and organizes your files based on their contents [May 2024]
27. Nyro: AI-Powered Desktop Productivity Tool [Aug 2024]
28. PDF2Audio: an open-source alternative to NotebookLM for podcast creation [Sep 2024]
29. o1-engineer: a command-line tool designed to assist developers [Sep 2024]
30. Zed: AI code editor from the creators of Atom and Tree-sitter [Sep 2024]
31. Cofounder: full stack generative web apps ; backend + db + stateful web apps [Sep 2024]

Code editor incl. Proprietary Software

• AI Code Editor: Replit Agent [09 Sep 2024] / Cursor [Mar 2023]
• Vercel AI Vercel AI Toolkit for TypeScript
• Cline: CLI aNd Editor. Autonomous coding agent. VSCode Extension. [Jul 2024]
• void OSS Cursor alternative. a fork of vscode [Oct 2024]
• Github Spark: an AI-powered tool for creating and sharing micro apps (“sparks”) [29 Oct 2024]
• bolt.new: Dev Sanbox with AI from stackblitz [Sep 2024]
• Windsurf editor: Flows = Agents + Copilots. Cascades (a specific implementation of AI Flows. Advanced chat interface). [13 Nov 2024]

UI/UX

1. Gradio: Build Machine Learning Web Apps - in Python [Mar 2023]
2. Text generation web UI: Text generation web UI [Mar 2023]
3. Open AI Chat Mockup: An open source ChatGPT UI. mckaywrigley/chatbot-ui [Mar 2023]
4. chainlit: Build production-ready Conversational AI applications in minutes. [Mar 2023]
5. CopilotKit: Built-in React UI components [Jun 2023]
6. Open-source GPT Wrappers 1. ChatGPT-Next-Web [Mar 2023] 2. FastGPT [Feb 2023] 3. Lobe Chat [Jan 2024]
7. anse: UI for multiple models such as ChatGPT, DALL-E and Stable Diffusion. [Apr 2023]

Data Processing and Management

1. Camelot a Python library that can help you extract tables from PDFs! ref: Comparison with other PDF Table Extraction libraries [Jul 2016]
2. Trafilatura: Gather text from the web and convert raw HTML into structured, meaningful data. [Apr 2019]
3. Math formula OCR: MathPix, OSS LaTeX-OCR [Jan 2021]
4. activeloopai/deeplake: AI Vector Database for LLMs/LangChain. Doubles as a Data Lake for Deep Learning. Store, query, version, & visualize any data. Stream data in real-time to PyTorch/TensorFlow. ref [Jun 2021]
5. PostgresML: The GPU-powered AI application database. [Apr 2022]
6. unstructured: Open-Source Pre-Processing Tools for Unstructured Data [Sep 2022]
7. outlines: Structured Text Generation [Mar 2023]
8. Instructor: Structured outputs for LLMs, easily map LLM outputs to structured data. [Jun 2023]
9. Nougat: Neural Optical Understanding for Academic Documents: The academic document PDF parser that understands LaTeX math and tables. git [25 Aug 2023]
10. Marker: converts PDF to markdown [Oct 2023]
11. Maxun: Open-Source No-Code Web Data Extraction Platform [Oct 2023]
12. Crawl4AI: Open-source LLM Friendly Web Crawler & Scrapper [May 2024]
13. firecrawl: Scrap entire websites into LLM-ready markdown or structured data. [Apr 2024]
14. Zerox OCR: Zero shot pdf OCR with gpt-4o-mini [Jul 2024]
15. Azure AI Document Intelligence (FKA. Azure Form Recognizer): ref: Table and Meta data Extraction in the Document
16. Table to Markdown: LLM can recognize Markdown-formatted tables more effectively than raw table formats.

Sample code

• Streaming with Azure OpenAI SSE [May 2023]
• TaxyAI/browser-extension: Browser Automation by Chrome debugger API and Prompt > src/helpers/determineNextAction.ts [Mar 2023]
• Embedding does not use Open AI. Can be executed locally: pdfGPT [Mar 2023]
• Langchain Ask PDF (Tutorial): git [Apr 2023]

Cross-reference

• RAG: x-ref
• Agent Applications and Libraries: x-ref
• OSS Alternatives for OpenAI Code Interpreter: x-ref
• LLMOps: Large Language Model Operations: x-ref

Agent

Agentic Design Frameworks

• Agentic Design Frameworks focus on managing autonomous or semi-autonomous AI agents for complex tasks.
• e.g., Autogen, and crewAI

Agent Design Patterns

• The Rise and Potential of Large Language Model Based Agents: A Survey: The papers list for LLM-based agents [cnt] / git [14 Sep 2023]
• AgentBench Evaluating LLMs as Agents: Assess LLM-as Agent’s reasoning and decision-making abilities. [7 Aug 2023]
• Agentic Design Patterns ref [Mar 2024]
  • Reflection: LLM self-evaluates to improve.
    • Self-Refine [30 Mar 2023]
    • Reflexion [20 Mar 2023 ]
    • CRITIC [19 May 2023]
  • Tool use: LLM uses tools for information gathering, action, or data processing.
    • Gorilla [24 May 2023]
    • MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action [20 Mar 2023]
    • Efficient Tool Use with Chain-of-Abstraction Reasoning [30 Jan 2024]
  • Planning: LLM devises and executes multistep plans to reach goals.
    • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models [28 Jan 2022]
    • HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face [30 Mar 2023]
    • Understanding the planning of LLM agents: A survey [5 Feb 2024]
  • Multi-agent collaboration: Multiple AI agents collaborate for better solutions.
    • Communicative Agents for Software Development [16 Jul 2023]
    • AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation [16 Aug 2023]
    • MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework [1 Aug 2023]
    • Framework: Autogen / LangGraph / crewAI
• Generate the code ref [Jun 2024]
  • AgentCoder: Multiagent-Code Generation with Iterative Testing and Optimisation [20 Dec 2023]
  • LDB: A Large Language Model Debugger via Verifying Runtime Execution Step by Step [25 Feb 2024]
  • SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering [6 May 2024]

---

• Generative AI Design Patterns for Agentic AI Systems: Design Patterns for Agentic solutions in Azure [May 2023]
• Exploring Generative AI (martinfowler.com): Memos on how LLMs are being used to enhance software delivery practices, including Toochain, Test-Driven Development (TDD) with GitHub Copilot, pair programming, and multi-file editing. [26 Jul 2023 ~ ]
• Cognitive Architectures for Language Agents: Cognitive Architectures for Language Agents (CoALA). Procedural (how to perform tasks), Semantic (long-term store of knowledge), Episodic Memory (recall specific past events) ref [5 Sep 2023]
• Language Agent Tree Search Method (LATS): LATS leverages an external environment and an MCTS (Monte Carlo Tree Search)-based search [6 Oct 2023]
• AI Agents That Matter: AI agent evaluations for optimizing both accuracy and cost. Focusing solely on accuracy can lead to overfitting and high costs. retry, warming, escalation [1 Jul 2024]
• Automated Design of Agentic Systems: Automated Design of Agentic Systems (ADAS), which aims to automatically create powerful agentic system designs, including inventing novel building blocks and/or combining them. [15 Aug 2024]
• The Different Ochestration Frameworks:💡Orchestration frameworks for LLM applications: Micro-orchestration / Macro-orchestration / Agentic Design Frameworks / Optimizer frameworks [11 Oct 2024]

Tool use: LLM to Master APIs

• Berkeley Function-Calling Leaderboard V2 [Aug 2024]

• Gorilla: An API store for LLMs: [cnt]: Gorilla: Large Language Model Connected with Massive APIs git [24 May 2023]

  1. Used GPT-4 to generate a dataset of instruction-api pairs for fine-tuning Gorilla.
  2. Used the abstract syntax tree (AST) of the generated code to match with APIs in the database and test set for evaluation purposes.

  Another user asked how Gorilla compared to LangChain; Patil replied: LangChain is a terrific project that tries to teach agents how to use tools using prompting. Our take on this is that prompting is not scalable if you want to pick between 1000s of APIs. So Gorilla is a LLM that can pick and write the semantically and syntactically correct API for you to call! A drop in replacement into LangChain! cite [04 Jul 2023]

• Meta: Toolformer: [cnt]: Language Models That Can Use Tools, by MetaAI git [9 Feb 2023]

• ToolLLM: [cnt]: : Facilitating Large Language Models to Master 16000+ Real-world APIs git [31 Jul 2023]

• APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets [26 Jun 2024]

• ToolShed: Toolshed Knowledge Bases & Advanced RAG-Tool Fusion, optimized for storing and retrieving tools in a vector database for large-scale agents. To address the limitations of primary methods, two approaches are: 1. tuning-based tool calling via LLM fine-tuning, and 2. retriever-based tool selection and planning. [18 Oct 2024]

• Anthropic Claude's computer use APIs:

  1. OpenInterpreter starts to support Computer Use API
  2. Agent.exe: Electron app to use computer use APIs. [Oct 2024]
  3. The Dawn of GUI Agent: An out-of-the-box version of Anthropic Claude Computer Use for Windows and macOS [Oct 2024]

Agent Framework

1. Huginn: A hackable version of IFTTT or Zapier on your own server for building agents that perform automated tasks. [Mar 2013]
2. Botpress Cloud: The open-source hub to build & deploy GPT/LLM Agents. [Nov 2016]
3. n8n: A workflow automation tool for integrating various tools. [Jan 2019]
4. phidata: Build AI Assistants with memory, knowledge, and tools [May 2022]
5. Cheshire-Cat (Stregatto): Framework to build custom AIs with memory and plugins [Feb 2023]
6. MetaGPT: Multi-Agent Framework. Assign different roles to GPTs to form a collaborative entity for complex tasks. e.g., Data Interpreter [Jun 2023]
7. SuperAGI: Autonomous AI Agents framework [May 2023]
8. AgentVerse: Primarily providing: task-solving and simulation. [May 2023]
9. OpenAgents: Three distinct agents: Data Agent for data analysis, Plugins Agent for plugin integration, and Web Agent for autonomous web browsing. [Aug 2023]
10. AgentOps:Python SDK for AI agent monitoring, LLM cost tracking, benchmarking. [Aug 2023]
11. Lagent: Inspired by the design philosophy of PyTorch. A lightweight framework for building LLM-based agents. [Aug 2023]
12. Autogen:💡Customizable and conversable agents framework [Aug 2023]
13. LangGraph: Built on top of LangChain [Aug 2023]
14. crewAI: Framework for orchestrating role-playing, autonomous AI agents. [Oct 2023]
15. XAgent: Autonomous LLM Agent for complex task solving like data analysis, recommendation, and model training [Oct 2023]
16. Qwen-Agent: Agent framework built upon Qwen1.5, featuring Function Calling, Code Interpreter, RAG, and Chrome extension. [Sep 2023]
17. Open AI Assistant API [6 Nov 2023]
18. Burr: Create an application as a state machine (graph/flowchart) for managing state, decisions, human feedback, and workflows. [Jan 2024]
19. TaskingAI: A BaaS (Backend as a Service) platform for LLM-based Agent Development and Deployment. [Jan 2024]
20. AgentScope: To build LLM-empowered multi-agent applications. [Jan 2024]
21. composio: Integration of Agents with 100+ Tools [Feb 2024]
22. maestro: A Framework for Claude Opus, GPT, and local LLMs to Orchestrate Subagents [Mar 2024]
23. AWS: Multi-Agent Orchestrator: a framework for managing multiple AI agents and handling complex conversations. [Jul 2024]
24. SwarmZero: SwarmZero's SDK for building AI agents, swarms of agents. [Aug 2024]
25. Bee Agent Framework: The TypeScript framework for building scalable agentic applications. [Oct 2024]
26. Agent-S: To build intelligent GUI agents that autonomously learn and perform complex tasks on your computer. [Oct 2024]
27. OpenAI Swarm: An experimental and educational framework for lightweight multi-agent orchestration. [11 Oct 2024]
28. Dynamiq: An orchestration framework for RAG, agentic AI, and LLM applications [Sep 2024]
29. Microsoft Agent Frameworks x-ref
30. Agent Framework used in MLE-bench: GPT-4o (AIDE) earned the highest score x-ref
    • AIDE: The state-of-the-art machine learning engineer agent [Apr 2024]
    • OpenHands: OpenHands (formerly OpenDevin), a platform for software development agents [Mar 2024]
    • MLAB ResearchAgent: Evaluating Language Agents on Machine Learning Experimentation [Aug 2023]

Agent Application

1. KHOJ: Open-source, personal AI agents. Cloud or Self-Host, Multiple Interfaces. Python Django based [Aug 2021]
2. PR-Agent: Efficient code review and handle pull requests, by providing AI feedbacks and suggestions [Jan 2023]
3. Auto-GPT: Most popular [Mar 2023]
4. CAMEL: CAMEL: Communicative Agents for “Mind” Exploration of Large Scale Language Model Society [Mar 2023]
5. babyagi: Simplest implementation - Coworking of 4 agents [Apr 2023]
6. SuperAGI: GUI for agent settings [May 2023]
7. AgentGPT: Assemble, configure, and deploy autonomous AI agents in your browser [Apr 2023]
8. GPT Pilot: The first real AI developer. Dev tool that writes scalable apps from scratch while the developer oversees the implementation [Jul 2023]
9. OpenDAN : Your Personal AIOS: OpenDAN, an open-source Personal AI OS consolidating various AI modules in one place [May 2023]
10. RasaGPT: Built with Rasa, FastAPI, Langchain, and LlamaIndex [Apr 2023]
11. ChatDev: Virtual software company. Create Customized Software using LLM-powered Multi-Agent Collaboration [Sep 2023]
12. skyagi: Simulating believable human behaviors. Role playing [Apr 2023]
13. AIOS: LLM Agent Operating System [Jan 2024]
14. SeeAct: GPT-4V(ision) is a Generalist Web Agent, if Grounded git [Jan 2024]
15. WrenAI: Open-source SQL AI Agent for Text-to-SQL [Mar 2024]
16. Devon: An open-source pair programmer. [Mar 2024]
17. Project Astra: Google DeepMind, A universal AI agent that is helpful in everyday life [14 May 2024]
18. SakanaAI AI-Scientist: Towards Fully Automated Open-Ended Scientific Discovery [Aug 2024]
19. Integuru: An AI agent that generates integration code by reverse-engineering platforms' internal APIs. [Oct 2024]
20. skyvern: Automate browser-based workflows with LLMs and Computer Vision [Feb 2024]
21. LaVague: Automate automation with Large Action Model framework. Generate Selenium code. [Feb 2024]
22. Mobile-Agent: The Powerful Mobile Device Operation Assistant Family. [Jan 2024]
23. AppAgent-TencentQQGYLab: Multimodal Agents as Smartphone Users. [Dec 2023]
24. TEN Agent: The world’s first real-time multimodal agent integrated with the OpenAI Realtime API. [Jun 2024]
25. Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks [Nov 2024]

OSS Alternatives for OpenAI Code Interpreter (aka. Advanced Data Analytics)

1. OpenAI Code Interpreter Integration with Sandboxed python execution environment [23 Mar 2023]
   • We provide our models with a working Python interpreter in a sandboxed, firewalled execution environment, along with some ephemeral disk space.
2. SlashGPT The tool integrated with "jupyter" agent [Apr 2023]
3. gpt-code-ui An open source implementation of OpenAI's ChatGPT Code interpreter. [May 2023]
4. OSS Code Interpreter A LangChain implementation of the ChatGPT Code Interpreter. [Jul 2023]
5. Open Interpreter: Let language models run code on your computer. [Jul 2023]

Caching

• Caching: A technique to store data that has been previously retrieved or computed, so that future requests for the same data can be served faster.
• To reduce latency, cost, and LLM requests by serving pre-computed or previously served responses.
• Strategies for caching: Caching can be based on item IDs, pairs of item IDs, constrained input, or pre-computation. Caching can also leverage embedding-based retrieval, approximate nearest neighbor search, and LLM-based evaluation. ref
• GPTCache: Semantic cache for LLMs. Fully integrated with LangChain and llama_index. git [Mar 2023]
• Prompt Cache: Modular Attention Reuse for Low-Latency Inference: LLM inference by reusing precomputed attention states from overlapping prompts. [7 Nov 2023]
• Prompt caching with Claude: Reducing costs by up to 90% and latency by up to 85% for long prompts. [15 Aug 2024]

Defensive UX

• Defensive UX: A design strategy that aims to prevent and handle errors in user interactions with machine learning or LLM-based products.
• Why defensive UX?: Machine learning and LLMs can produce inaccurate or inconsistent output, which can affect user trust and satisfaction. Defensive UX can help by increasing accessibility, trust, and UX quality.
• Guidelines for Human-AI Interaction: Microsoft: Based on a survey of 168 potential guidelines from various sources, they narrowed it down to 18 action rules organized by user interaction stages.
• People + AI Guidebook: Google: Google’s product teams and academic research, they provide 23 patterns grouped by common questions during the product development process3.
• Human Interface Guidelines for Machine Learning: Apple: Based on practitioner knowledge and experience, emphasizing aspects of UI rather than model functionality4.

LLM for Robotics: Bridging AI and Robotics

• PromptCraft-Robotics: Robotics and a robot simulator with ChatGPT integration git [Feb 2023]
• ChatGPT-Robot-Manipulation-Prompts: A set of prompts for Communication between humans and robots for executing tasks. git [Apr 2023]
• Siemens Industrial Copilot ref [31 Oct 2023]
• LeRobot: Hugging Face. LeRobot aims to provide models, datasets, and tools for real-world robotics in PyTorch. git [Jan 2024]
• Mobile ALOHA: Stanford’s mobile ALOHA robot learns from humans to cook, clean, do laundry. Mobile ALOHA extends the original ALOHA system by mounting it on a wheeled base ref [4 Jan 2024] / ALOHA: A Low-cost Open-source Hardware System for Bimanual Teleoperation.
• Figure 01 + OpenAI: Humanoid Robots Powered by OpenAI ChatGPT 📺 [Mar 2024]

Awesome demo

• FRVR Official Teaser: Prompt to Game: AI-powered end-to-end game creation [16 Jun 2023]
• rewind.ai: Rewind captures everything you’ve seen on your Mac and iPhone [Nov 2023]
• Vercel announced V0.dev: Make a snake game with chat [Oct 2023]
• Mobile ALOHA: A day of Mobile ALOHA [4 Jan 2024]
• groq: An LPU Inference Engine, the LPU is reported to be 10 times faster than NVIDIA’s GPU performance ref [Jan 2024]
• Sora: Introducing Sora — OpenAI’s text-to-video model [Feb 2024]
• Oasis: Minecraft clone. Generated by AI in Real-Time. The first playable AI model that generates open-world games. ref git [31 Oct 2024]