AI MicroMind Docs

Drag & Drop UI to build your customized LLM flow

🙌 Contributing

We love contributions! Feel free to submit Pull Request and we will review. Reach out to us at Discord if you have any questions or issues.

📄 License

Source code in this repository is made available under the Apache License Version 2.0.

Get Started

Cloud

Self-hosting requires more technical skill to setup instance, backing up database and maintaning updates. If you aren't experienced at managing servers and just want to use the webapp, we recommend using aimicromindCloud.

Quick Start

{% hint style="info" %} Pre-requisite: ensure NodeJS is installed on machine. Node v18.15.0 or v20 and above is supported. {% endhint %}

Install aimicromind locally using NPM.

1. Install AiMicromind:

npm install -g aimicromind

You can also install a specific version. Refer to available versions.

npm install -g aimicromind@x.x.x

2. Start AiMicromind:

npx aimicromind start

3. Open: http://localhost:3000

Docker

There are two ways to deploy aimicromind with Docker:

Docker Compose

1. Go to docker folder at the root of the project
2. Copy the .env.example file and paste it as another file named .env
3. Run:

docker compose up -d

4. Open: http://localhost:3000
5. You can bring the containers down by running:

docker compose stop

Docker Image

1. Build the image:

docker build --no-cache -t aimicromind.

2. Run image:

docker run -d --name aimicromind-p 3000:3000 aimicromind

3. Stop image:

docker stop aimicromind

For Developers

AiMicromind has 3 different modules in a single mono repository:

• Server: Node backend to serve API logics
• UI: React frontend
• Components: Integration components

Prerequisite

Install PNPM.

npm i -g pnpm

Setup 1

Simple setup using PNPM:

1. Clone the repository

git clone https://github.com/operativestech/AiMicroMind_Platform_2025.git

2. Go into repository folder

cd AiMicromind

3. Install all dependencies of all modules:

pnpm install

4. Build the code:

pnpm build

Start the app at http://localhost:3000

pnpm start

Setup 2

Step-by-step setup for project contributors:

1. Fork the official AiMicromind Github Repository
2. Clone your forked repository
3. Create a new branch, see guide. Naming conventions:
   • For feature branch: feature/<Your New Feature>
   • For bug fix branch: bugfix/<Your New Bugfix>.
4. Switch to the branch you just created
5. Go into repository folder:

cd AiMicromind

6. Install all dependencies of all modules:

pnpm install

7. Build the code:

pnpm build

8. Start the app at http://localhost:3000

pnpm start

9. For development build:

• Create .env file and specify the PORT (refer to .env.example) in packages/ui
• Create .env file and specify the PORT (refer to .env.example) in packages/server

pnpm dev

• Any changes made in packages/ui or packages/server will be reflected at http://localhost:8080

• For changes made in packages/components, you will need to build again to pickup the changes

• After making all the changes, run:

  pnpm build
  

  and

  pnpm start
  

  to make sure everything works fine in production.

For Enterprise

Enterprise plans have separate repository and docker image.

Once granted access to both, the setup is the same as #setup-1. Before starting the app, enterprise users are required to fill in the values for Enterprise Parameters in the .env file. Refer to .env.example for the required changes.

Reach out to support@aimicromind.com for the value of following env variables:

LICENSE_URL
AIMICROMIND_EE_LICENSE_KEY

For Docker Installation:

cd docker
cd enterprise
docker compose up -d

Learn More

In this video tutorial (coming soon)

Community Guide

• Introduction to [Practical] Building LLM Applications with AiMicromind/ LangChain
• AiMicromind/ LangChainによるLLMアプリケーション構築[実践]入門

description: Learn how to contribute to this project

Contribution Guide

We appreciate all contributions! No matter your skill level or technical background, you can help this project grow. Here are a few ways to contribute:

⭐ Star

Star and share the Github Repo.

🙌 Share Chatflow

Yes! Sharing how you use aimicromind is a way of contribution. Export your chatflow as JSON, attach a screenshot and share it in Show and Tell section.

💡 Ideas

We welcome ideas for new features, apps integrations. Submit your suggestions to the Ideas section.

🙋 Q&A

Want to learn more? Search for answers to any questions in the Q&A section. If you can't find one, don't hesitate to create a new question. It might help others who have similar questions.

🐞 Report Bugs

Found an issue? Report it.

📖 Contribute to Docs

1. Fork the official AiMicromind Docs Repo

2. Clone your forked repository

3. Create a new branch

4. Switch to the branch you just created

5. Go into repository folder

   cd AiMicromindDocs
   

6. Make changes

7. Commit changes and submit Pull Request from forked branch pointing to AiMicromind Docs main

👨‍💻 Contribute to Code

To learn how to contribute code, go to the For Developers section and follow the instructions.

If you are contributing to a new node integration, read the Building Node guide.

🏷️ Pull Request process

A member of the AiMicromind team will automatically be notified/assigned when you open a pull request. You can also reach out to us on Discord.

📜 Code of Conduct

This project and everyone participating in it are governed by the Code of Conduct which can be found in the file. By participating, you are expected to uphold this code.

Please report unacceptable behavior to hello@aimicromind.com.

Building Node

Install Git

First, install Git and clone aimicromind repository. You can follow the steps from the Get Started guide.

Structure

AiMicromind separate every node integration under the folder packages/components/nodes. Let's try to create a simple Tool!

Create Calculator Tool

Create a new folder named Calculator under the packages/components/nodes/tools folder. Then create a new file named Calculator.ts. Inside the file, we will first write the base class.

import { INode } from '../../../src/Interface'
import { getBaseClasses } from '../../../src/utils'

class Calculator_Tools implements INode {
    label: string
    name: string
    version: number
    description: string
    type: string
    icon: string
    category: string
    author: string
    baseClasses: string[]

    constructor() {
        this.label = 'Calculator'
        this.name = 'calculator'
        this.version = 1.0
        this.type = 'Calculator'
        this.icon = 'calculator.svg'
        this.category = 'Tools'
        this.author = 'Your Name'
        this.description = 'Perform calculations on response'
        this.baseClasses = [this.type, ...getBaseClasses(Calculator)]
    }
}

module.exports = { nodeClass: Calculator_Tools }

Every node will implements the INode base class. Breakdown of what each property means:

Define Class

Now the component class is partially finished, we can go ahead to define the actual Tool class, in this case - Calculator.

Create a new file under the same Calculator folder, and named as core.ts

import { Parser } from "expr-eval"
import { Tool } from "@langchain/core/tools"

export class Calculator extends Tool {
    name = "calculator"
    description = `Useful for getting the result of a math expression. The input to this tool should be a valid mathematical expression that could be executed by a simple calculator.`
 
    async _call(input: string) {
        try {
            return Parser.evaluate(input).toString()
        } catch (error) {
            return "I don't know how to do that."
        }
    }
}

Finishing

Head back to the Calculator.ts file, we can finish this up by having the async init function. In this function, we will initialize the Calculator class we created above. When the flow is being executed, the init function in each node will be called, and the _call function will be executed when LLM decides to call this tool.

import { INode } from '../../../src/Interface'
import { getBaseClasses } from '../../../src/utils'
import { Calculator } from './core'

class Calculator_Tools implements INode {
    label: string
    name: string
    version: number
    description: string
    type: string
    icon: string
    category: string
    author: string
    baseClasses: string[]

    constructor() {
        this.label = 'Calculator'
        this.name = 'calculator'
        this.version = 1.0
        this.type = 'Calculator'
        this.icon = 'calculator.svg'
        this.category = 'Tools'
        this.author = 'Your Name'
        this.description = 'Perform calculations on response'
        this.baseClasses = [this.type, ...getBaseClasses(Calculator)]
    }
    
 
    async init() {
        return new Calculator()
    }
}

module.exports = { nodeClass: Calculator_Tools }

Build and Run

In the .env file inside packages/server, create a new env variable:

SHOW_COMMUNITY_NODES=true

Now we can use pnpm build and pnpm start to bring the component alive!

API Reference

Using aimicromind public API, you can programmatically execute many of the same tasks as you can in the GUI. This section introduces aimicromindREST API.

• Assistants
• Chat Message
• Chatflows
• Document Store
• Feedback
• Leads
• Ping
• Prediction
• Tools
• Upsert History
• Variables

Assistants

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/assistants" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1).yml" path="/assistants" method="get" %} swagger (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1).yml" path="/assistants/{id}" method="get" %} swagger (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1).yml" path="/assistants/{id}" method="put" %} swagger (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1).yml" path="/assistants/{id}" method="delete" %} swagger (1) (1).yml {% endswagger %}

Attachments

{% swagger src="../.gitbook/assets/swagger (1).yml" path="/attachments/{chatflowId}/{chatId}" method="post" %} swagger (1).yml {% endswagger %}

Chat Message

{% swagger src="../.gitbook/assets/swagger (3).yml" path="/chatmessage/{id}" method="get" %} swagger (3).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (4).yml" path="/chatmessage/{id}" method="delete" %} swagger (4).yml {% endswagger %}

Chatflows

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/chatflows" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/chatflows" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/chatflows/{id}" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/chatflows/{id}" method="put" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/chatflows/{id}" method="delete" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/chatflows/apikey/{apikey}" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

Document Store

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/document-store/store" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/document-store/store" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/document-store/store/{id}" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/document-store/store/{id}" method="put" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/document-store/store/{id}" method="delete" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (5).yml" path="/document-store/upsert/{id}" method="post" %} swagger (5).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (3).yml" path="/document-store/refresh/{id}" method="post" %} swagger (3).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/document-store/vectorstore/query" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (2).yml" path="/document-store/loader/{storeId}/{loaderId}" method="delete" %} swagger (2).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/document-store/vectorstore/{id}" method="delete" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (2).yml" path="/document-store/chunks/{storeId}/{loaderId}/{pageNo}" method="get" %} swagger (2).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (2).yml" path="/document-store/chunks/{storeId}/{loaderId}/{chunkId}" method="put" %} swagger (2).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (2).yml" path="/document-store/chunks/{storeId}/{loaderId}/{chunkId}" method="delete" %} swagger (2).yml {% endswagger %}

Feedback

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/feedback/{id}" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/feedback" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/feedback/{id}" method="put" %} swagger (1) (1) (1).yml {% endswagger %}

Leads

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/leads/{id}" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/leads" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

Ping

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/ping" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

Prediction

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/prediction/{id}" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

Tools

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/tools" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/tools" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/tools/{id}" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/tools/{id}" method="put" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/tools/{id}" method="delete" %} swagger (1) (1) (1).yml {% endswagger %}

Upsert History

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/upsert-history/{id}" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/upsert-history/{id}" method="patch" %} swagger (1) (1) (1).yml {% endswagger %}

Variables

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/variables" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/variables" method="get" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/variables/{id}" method="put" %} swagger (1) (1) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/variables/{id}" method="delete" %} swagger (1) (1) (1).yml {% endswagger %}

Vector Upsert

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/vector/upsert/{id}" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

description: Learn about some core functionalities built into AiMicromind

Using AiMicromind

This section provides in-depth guides on core aimicromind functionalities, including API usage, variables, and telemetry collection practices.

Guides

• Agentflows
  • Multi-Agent
  • Sequential Agents
    • Video Tutorials
• API
• Analytic
• Document Stores
• Embed
• Streaming
• Telemetry
• Variables

description: Learn about how to build agentic systems in Aimicromind

Agentflows

Introducing Agentic Systems in AiMicromind

AiMicromind's Agentflows section provides a platform for building agent-based systems that can interact with external tools and data sources.

Currently, AiMicromind offers two approaches for designing these systems: Multi-Agents¹ and Sequential Agents². These approaches provide different levels of control and complexity, allowing you to choose the best fit for your needs.

{% hint style="success" %} This documentation will explore both the Sequential Agent and Multi-Agent approaches, explaining their features and how they can be used to build different types of conversational workflows. {% endhint %}

1. Multi-Agents, built on top of the Sequential Agent architecture, simplify the process of building and managing teams of agents by pre-configuring core elements and providing a higher-level abstraction. ↩

2. Sequential Agents provide developers with direct access to the underlying workflow structure, enabling granular control over every step of the conversation flow and offering maximum flexibility for building highly customized conversational applications. ↩

description: Learn how to use Multi-Agents in Aimicromind, written by @toi500

Multi-Agents

This guide intends to provide an introduction of the multi-agent AI system architecture within Aimicromind, detailing its components, operational constraints, and workflow.

Concept

Analogous to a team of domain experts collaborating on a complex project, a multi-agent system uses the principle of specialization within artificial intelligence.

This multi-agent system utilizes a hierarchical, sequential workflow, maximizing efficiency and specialization.

1. System Architecture

We can define the multi-agent AI architecture as a scalable AI system capable of handling complex projects by breaking them down into manageable sub-tasks.

In Aimicromind, a multi-agent system comprises two primary nodes or agent types and a user, interacting in a hierarchical graph to process requests and deliver a targeted outcome:

1. User: The user acts as the system's starting point, providing the initial input or request. While a multi-agent system can be designed to handle a wide range of requests, it's important that these user requests align with the system's intended purpose. Any request falling outside this scope can lead to inaccurate results, unexpected loops, or even system errors. Therefore, user interactions, while flexible, should always align with the system's core functionalities for optimal performance.
2. Supervisor AI: The Supervisor acts as the system's orchestrator, overseeing the entire workflow. It analyzes user requests, decomposes them into a sequence of sub-tasks, assigns these sub-tasks to the specialized worker agents, aggregates the results, and ultimately presents the processed output back to the user.
3. Worker AI Team: This team consists of specialized AI agents, or Workers, each instructed - via prompt messages - to handle a specific task within the workflow. These Workers operate independently, receiving instructions and data from the Supervisor, executing their specialized functions, using tools as needed, and returning the results to the Supervisor.

2. Operational Constraints

To maintain order and simplicity, this multi-agent system operates under two important constraints:

• One task at a time: The Supervisor is intentionally designed to focus on a single task at a time. It waits for the active Worker to complete its task and return the results before it analyzes the next step and delegates the subsequent task. This ensures each step is completed successfully before moving on, preventing overcomplexity.
• One Supervisor per flow: While it's theoretically possible to implement a set of nested multi-agent systems to form a more sophisticated hierarchical structure for highly complex workflows, what LangChain defines as "Hierarchical Agent Teams", with a top-level supervisor and mid-level supervisors managing teams of workers, Aimicromind's multi-agent systems currently operate with a single Supervisor.

{% hint style="info" %} These two constraints are important when planning your application's workflow. If you try to design a workflow where the Supervisor needs to delegate multiple tasks simultaneously, in parallel, the system won't be able to handle it and you'll encounter an error. {% endhint %}

The Supervisor

The Supervisor, as the agent governing the overall workflow and responsible for delegating tasks to the appropriate Worker, requires a set of components to function correctly:

• Chat Model capable of function calling to manage the complexities of task decomposition, delegation, and result aggregation.
• Agent Memory (optional): While the Supervisor can function without Agent Memory, this node can significantly enhance workflows that require access to past Supervisor states. This state preservation could allow the Supervisor to resume the job from a specific point or leverage past data for improved decision-making.

Supervisor Prompt

By default, the Supervisor Prompt is worded in a way that instructs the Supervisor to analyze user requests, decompose them into a sequence of sub-tasks, and assign these sub-tasks to the specialized worker agents.

While the Supervisor Prompt is customizable to fit specific application needs, it always requires the following two key elements:

• The {team_members} Variable: This variable is crucial for the Supervisor's understanding of the available workforce since it provides the Supervisor with list of Worker names. This allows the Supervisor to diligently delegate tasks to the most appropriate Worker based on their expertise.
• The "FINISH" Keyword: This keyword serves as a signal within the Supervisor Prompt. It indicates when the Supervisor should consider the task complete and present the final output to the user. Without a clear "FINISH" directive, the Supervisor might continue delegating tasks unnecessarily or fail to deliver a coherent and finalized result to the user. It signals that all necessary sub-tasks have been executed and the user's request has been fulfilled.

{% hint style="info" %} It's important to understand that the Supervisor plays a very distinct role from Workers. Unlike Workers, which can be tailored with highly specific instructions, the Supervisor operates most effectively with general directives, which allow it to plan and delegate tasks as it deems appropriate. If you're new to multi-agent systems, we recommend sticking with the default Supervisor prompt {% endhint %}

Understanding Recursion Limit in Supervisor node:

This parameter restricts the maximum depth of nested function calls within our application. In our current context, it limits how many times the Supervisor can trigger itself within a single workflow execution. This is important for preventing unbounded recursion and ensuring resources are used efficiently.

How the Supervisor works

Upon receiving a user query, the Supervisor initiates the workflow by analyzing the request and discerning the user's intended outcome.

Then, leveraging the {team_members} variable in the Supervisor Prompt, which only provides a list of available Worker AI names, the Supervisor infers each Worker's specialty and strategically selects the most suitable Worker for each task within the workflow.

{% hint style="info" %} Since the Supervisor only has the Workers' names to infer their functionality inside the workflow, it is very important that those names are set accordingly. Clear, concise, and descriptive names that accurately reflect the Worker's role or area of expertise are crucial for the Supervisor to make informed decisions when delegating tasks. This ensures that the right Worker is selected for the right job, maximizing the system's accuracy in fulfilling the user's request. {% endhint %}

The Worker

The Worker, as a specialized agent instructed to handle a specific task within the system, requires two essential components to function correctly:

• A Supervisor: Each Worker must be connected to the Supervisor so it can be called upon when a task needs to be delegated. This connection establishes the essential hierarchical relationship within the multi-agent system, ensuring that the Supervisor can efficiently distribute work to the appropriate specialized Workers.
• A Chat Model node capable of function calling: By default, Workers inherit the Supervisor's Chat Model node unless assigned one directly. This function-calling capability enables the Worker to interact with tools designed for its specialized task.

{% hint style="info" %} The ability to assign different Chat Models to each Worker provides significant flexibility and optimization opportunities for our application. By selecting Chat Models tailored to specific tasks, we can leverage more cost-effective solutions for simpler tasks and reserve specialized, potentially more expensive, models when truly necessary. {% endhint %}

Understanding Max Iteration parameter in Workers

LangChain refers to Max Iterations Cap as a important control mechanism for preventing haywire within an agentic system. In our current this context, it serves us as a guardrail against excessive, potentially infinite, interactions between the Supervisor and Worker.

Unlike the Supervisor node's Recursion Limit, which restricts how many times the Supervisor can call itself, the Worker node's Max Iteration parameter limits how many times a Supervisor can iterated or query a specific Worker.

By capping or limiting the Max Iteration, we ensure that costs remain under control, even in cases of unexpected system behavior.

Example: A practical user case

Now that we've established a foundational understanding of how Multi-Agent systems work within Aimicromind, let's explore a practical application.

Imagine a Lead Outreach multi-agent system (available in the Marketplace) designed to automate the process of identifying, qualifying, and engaging with potential leads. This system would utilize a Supervisor to orchestrate the following two Workers:

• Lead Researcher: This Worker, using the Google Search Tool, will be responsible for gathering potential leads based on user-defined criteria.
• Lead Sales Generator: This Worker will utilize the information gathered by the Lead Researcher to create personalized email drafts for the sales team.

Background: A user working at Solterra Renewables wants to gather available information about Evergreen Energy Group, a reputable renewable energy company located in the UK, and target its CEO, Amelia Croft, as a potential lead.

User Request: The Solterra Renewables employee provides the following query to the multi-agent system: "I need information about Evergreen Energy Group and Amelia Croft as a potential new customer for our business."

1. Supervisor:
   • The Supervisor receives the user request and delegates the "Lead Research" task to the Lead Researcher Worker.
2. Lead Researcher Worker:
   • The Lead Researcher Worker, using the Google Search Tool, gathers information about Evergreen Energy Group, focusing on:
     • Company background, industry, size, and location.
     • Recent news and developments.
     • Key executives, including confirming Amelia Croft's role as CEO.
   • The Lead Researcher sends the gathered information back to the Supervisor.
3. Supervisor:
   • The Supervisor receives the research data from the Lead Researcher Worker and confirms that Amelia Croft is a relevant lead.
   • The Supervisor delegates the "Generate Sales Email" task to the Lead Sales Generator Worker, providing:
     • The research information on Evergreen Energy Group.
     • Amelia Croft's email.
     • Context about Solterra Renewables.
4. Lead Sales Generator Worker:
   • The Lead Sales Generator Worker crafts a personalized email draft tailored to Amelia Croft, taking into account:
     • Her role as CEO and the relevance of Solterra Renewables' services to her company.
     • Information from the research about Evergreen Energy Group's current focus or projects.
   • The Lead Sales Generator Worker sends the completed email draft back to the Supervisor.
5. Supervisor:
   • The Supervisor receives the generated email draft and issues the "FINISH" directive.
   • The Supervisor outputs the email draft back to the user, the Solterra Renewables employee.
6. User Receives Output: The Solterra Renewables employee receives a personalized email draft ready to be reviewed and sent to Amelia Croft.

Video Tutorials (Coming soon)

description: Learn the Fundamentals of Sequential Agents in Aimicromind, written by @toi500

Sequential Agents

This guide offers a complete overview of the Sequential Agent AI system architecture within Aimicromind, exploring its core components and workflow design principles.

{% hint style="warning" %} Disclaimer: This documentation is intended to help aimicromind users understand and build conversational workflows using the Sequential Agent system architecture. It is not intended to be a comprehensive technical reference for the LangGraph framework and should not be interpreted as defining industry standards or core LangGraph concepts. {% endhint %}

Concept

Built on top of LangGraph, Aimicromind's Sequential Agents architecture facilitates the development of conversational agentic systems by structuring the workflow as a directed cyclic graph (DCG), allowing controlled loops and iterative processes.

This graph, composed of interconnected nodes, defines the sequential flow of information and actions, enabling the agents to process inputs, execute tasks, and generate responses in a structured manner.

Understanding Sequential Agents' DCG Architecture

This architecture simplifies the management of complex conversational workflows by defining a clear and understandable sequence of operations through its DCG structure.

Let's explore some key elements of this approach:

{% tabs %} {% tab title="Core Principles" %}

• Node-based processing: Each node in the graph represents a discrete processing unit, encapsulating its own functionality like language processing, tool execution, or conditional logic.
• Data flow as connections: Edges in the graph represent the flow of data between nodes, where the output of one node becomes the input for the subsequent node, enabling a chain of processing steps.
• State management: State is managed as a shared object, persisting throughout the conversation. This allows nodes to access relevant information as the workflow progresses. {% endtab %}

{% tab title="Terminology" %}

• Flow: The movement or direction of data within the workflow. It describes how information passes between nodes during a conversation.
• Workflow: The overall design and structure of the system. It's the blueprint that defines the sequence of nodes, their connections, and the logic that orchestrates the conversation flow.
• State: A shared data structure that represents the current snapshot of the conversation. It includes the conversation history state.messages and any custom State variables defined by the user.
• Custom State: User-defined key-value pairs added to the state object to store additional information relevant to the workflow.
• Tool: An external system, API, or service that can be accessed and executed by the workflow to perform specific tasks, such as retrieving information, processing data, or interacting with other applications.
• Human-in-the-Loop (HITL): A feature that allows human intervention in the workflow, primarily during tool execution. It enables a human reviewer to approve or reject a tool call before it's executed.
• Parallel node execution: It refers to the ability to execute multiple nodes concurrently within a workflow by using a branching mechanism. This means that different branches of the workflow can process information or interact with tools simultaneously, even though the overall flow of execution remains sequential. {% endtab %} {% endtabs %}

Sequential Agents vs Multi-Agents

While both Multi-Agent and Sequential Agent systems in aimicromind are built upon the LangGraph framework and share the same fundamental principles, the Sequential Agent architecture provides a lower level of abstraction¹, offering more granular control over every step of the workflow.

Multi-Agent systems, which are characterized by a hierarchical structure with a central supervisor agent delegating tasks to specialized worker agents, excel at handling complex workflows by breaking them down into manageable sub-tasks. This decomposition into sub-tasks is made possible by pre-configuring core system elements under the hood, such as condition nodes, which would require manual setup in a Sequential Agent system. As a result, users can more easily build and manage teams of agents.

In contrast, Sequential Agent systems operate like a streamlined assembly line, where data flows sequentially through a chain of nodes, making them ideal for tasks demanding a precise order of operations and incremental data refinement. Compared to the Multi-Agent system, its lower-level access to the underlying workflow structure makes it fundamentally more flexible and customizable, offering parallel node execution and full control over the system logic, incorporating conditions, state, and loop nodes into the workflow, allowing for the creation of new dynamic branching capabilities.

Introducing State, Loop and Condition Nodes

Aimicromind's Sequential Agents offer new capabilities for creating conversational systems that can adapt to user input, make decisions based on context, and perform iterative tasks.

These capabilities are made possible by the introduction of four new core nodes; the State Node, the Loop Node, and two Condition Nodes.

• State Node: We define State as a shared data structure that represents the current snapshot of our application or workflow. The State Node allows us to add a custom State to our workflow from the start of the conversation. This custom State is accessible and modifiable by other nodes in the workflow, enabling dynamic behavior and data sharing.
• Loop Node: This node introduces controlled cycles within the Sequential Agent workflow, enabling iterative processes where a sequence of nodes can be repeated based on specific conditions. This allows agents to refine outputs, gather additional information from the user, or perform tasks multiple times.
• Condition Nodes: The Condition and Condition Agent Node provide the necessary control to create complex conversational flows with branching paths. The Condition Node evaluates conditions directly, while the Condition Agent Node uses an agent's reasoning to determine the branching logic. This allows us to dynamically guide the flow's behavior based on user input, the custom State, or results of actions taken by other nodes.

Choosing the right system

Selecting the ideal system for your application depends on understanding your specific workflow needs. Factors like task complexity, the need for parallel processing, and your desired level of control over data flow are all key considerations.

• For simplicity: If your workflow is relatively straightforward, where tasks can be completed one after the other and therefore does not require parallel node execution or Human-in-the-Loop (HITL), the Multi-Agent approach offers ease of use and quick setup.
• For flexibility: If your workflow needs parallel execution, dynamic conversations, custom State management, and the ability to incorporate HITL, the Sequential Agent approach provides the necessary flexibility and control.

Here's a table comparing Multi-Agent and Sequential Agent implementations in Aimicromind, highlighting key differences and design considerations:

{% hint style="info" %} Note: Even though Multi-Agent systems are technically a higher-level layer built upon the Sequential Agent architecture, they offer a distinct user experience and approach to workflow design. The comparison above treats them as separate systems to help you select the best option for your specific needs. {% endhint %}

Sequential Agents Nodes

Sequential Agents bring a whole new dimension to Aimicromind, introducing 10 specialized nodes, each serving a specific purpose, offering more control over how our conversational agents interact with users, process information, make decisions, and execute actions.

The following sections aim to provide a comprehensive understanding of each node's functionality, inputs, outputs, and best practices, ultimately enabling you to craft sophisticated conversational workflows for a variety of applications.

1. Start Node

As its name implies, the Start Node is the entry point for all workflows in the Sequential Agent architecture. It receives the initial user query, initializes the conversation State, and sets the flow in motion.

Understanding the Start Node

The Start Node ensures that our conversational workflows have the necessary setup and context to function correctly. It's responsible for setting up key functionalities that will be used throughout the rest of the workflow:

• Defining the default LLM: The Start Node requires us to specify a Chat Model (LLM) compatible with function calling, enabling agents in the workflow to interact with tools and external systems. It will be the default LLM used under the hood in the workflow.
• Initializing Memory: We can optionally connect an Agent Memory Node to store and retrieve conversation history, enabling more context-aware responses.
• Setting a custom State: By default, the State contains an immutable state.messages array, which acts as the transcript or history of the conversation between the user and the agents. The Start Node allows you to connect a custom State to the workflow adding a State Node, enabling the storage of additional information relevant to your workflow
• Enabling moderation: Optionally, we can connect Input Moderation to analyze the user's input and prevent potentially harmful content from being sent to the LLM.

Inputs

Outputs

The Start Node can connect to the following nodes as outputs:

• Agent Node: Routes the conversation flow to an Agent Node, which can then execute actions or access tools based on the conversation's context.
• LLM Node: Routes the conversation flow to an LLM Node for processing and response generation.
• Condition Agent Node: Connects to a Condition Agent Node to implement branching logic based on the agent's evaluation of the conversation.
• Condition Node: Connects to a Condition Node to implement branching logic based on predefined conditions.

Best Practices

{% tabs %} {% tab title="Pro Tips" %} Choose the right Chat Model

Ensure your selected LLM supports function calling, a key feature for enabling agent-tool interactions. Additionally, choose an LLM that aligns with the complexity and requirements of your application. You can override the default LLM by setting it at the Agent/LLM/Condition Agent node level when necessary.

Consider context and persistence

If your use case demands it, utilize Agent Memory Node to maintain context and personalize interactions. {% endtab %}

{% tab title="Potential Pitfalls" %} Incorrect Chat Model (LLM) selection

• Problem: The Chat Model selected in the Start Node is not suitable for the intended tasks or capabilities of the workflow, resulting in poor performance or inaccurate responses.
• Example: A workflow requires a Chat Model with strong summarization capabilities, but the Start Node selects a model optimized for code generation, leading to inadequate summaries.
• Solution: Choose a Chat Model that aligns with the specific requirements of your workflow. Consider the model's strengths, weaknesses, and the types of tasks it excels at. Refer to the documentation and experiment with different models to find the best fit.

Overlooking Agent Memory Node configuration

• Problem: The Agent Memory Node is not properly connected or configured, resulting in the loss of conversation history data between sessions.
• Example: You intend to use persistent memory to store user preferences, but the Agent Memory Node is not connected to the Start Node, causing preferences to be reset on each new conversation.
• Solution: Ensure that the Agent Memory Node is connected to the Start Node and configured with the appropriate database (SQLite). For most use cases, the default SQLite database will be sufficient.

Inadequate Input Moderation

• Problem: The "Input Moderation" is not enabled or configured correctly, allowing potentially harmful or inappropriate user input to reach the LLM and generate undesirable responses.
• Example: A user submits offensive language, but the input moderation fails to detect it or is not set up at all, allowing the query to reach the LLM.
• Solution: Add and configure an input moderation node in the Start Node to filter out potentially harmful or inappropriate language. Customize the moderation settings to align with your specific requirements and use cases. {% endtab %} {% endtabs %}

2. Agent Memory Node

The Agent Memory Node provides a mechanism for persistent memory storage, allowing the Sequential Agent workflow to retain the conversation history state.messages and any custom State previously defined across multiple interactions

This long-term memory is essential for agents to learn from previous interactions, maintain context over extended conversations, and provide more relevant responses.

Where the data is recorded

By default, aimicromindutilizes its built-in SQLite database to store conversation history and custom state data, creating a "checkpoints" table to manage this persistent information.

Understanding the "checkpoints" table structure and data format

This table stores snapshots of the system's State at various points during a conversation, enabling the persistence and retrieval of conversation history. Each row represents a specific point or "checkpoint" in the workflow's execution.

Table structure

• thread_id: A unique identifier representing a specific conversation session, our session ID. It groups together all checkpoints related to a single workflow execution.
• checkpoint_id: A unique identifier for each execution step (node execution) within the workflow. It helps track the order of operations and identify the State at each step.
• parent_id: Indicates the checkpoint_id of the preceding execution step that led to the current checkpoint. This establishes a hierarchical relationship between checkpoints, allowing for the reconstruction of the workflow's execution flow.
• checkpoint: Contains a JSON string representing the current State of the workflow at that specific checkpoint. This includes the values of variables, the messages exchanged, and any other relevant data captured at that point in the execution.
• metadata: Provides additional context about the checkpoint, specifically related to node operations.

How it works

As a Sequential Agent workflow executes, the system records a checkpoint in this table for each significant step. This mechanism provides several benefits:

• Execution tracking: Checkpoints enable the system to understand the execution path and the order of operations within the workflow.
• State management: Checkpoints store the State of the workflow at each step, including variable values, conversation history, and any other relevant data. This allows the system to maintain contextual awareness and make informed decisions based on the current State.
• Workflow resumption: If the workflow is paused or interrupted (e.g., due to a system error or user request), the system can use the stored checkpoints to resume execution from the last recorded State. This ensures that the conversation or task continues from where it left off, preserving the user's progress and preventing data loss.

Inputs

The Agent Memory Node has no specific input connections.

Node Setup

Additional Parameters

Outputs

The Agent Memory Node interacts solely with the Start Node, making the conversation history available from the very beginning of the workflow.

Best Practices

{% tabs %} {% tab title="Pro Tips" %} Strategic use

Employ Agent Memory only when necessary. For simple, stateless interactions, it might be overkill. Reserve it for scenarios where retaining information across turns or sessions is essential. {% endtab %}

{% tab title="Potential Pitfalls" %} Unnecessary overhead

• The Problem: Using Agent Memory for every interaction, even when not needed, introduces unnecessary storage and processing overhead. This can slow down response times and increase resource consumption.
• Example: A simple weather chatbot that provides information based on a single user request doesn't need to store conversation history.
• Solution: Analyze the requirements of your system and only utilize Agent Memory when persistent data storage is essential for functionality or user experience. {% endtab %} {% endtabs %}

3. State Node

The State Node, which can only be connected to the Start Node, provides a mechanism to set a user-defined or custom State into our workflow from the start of the conversation. This custom State is a JSON object that is shared and can be updated by nodes in the graph, passing from one node to another as the flow progresses.

Understanding the State Node

By default, the State includes a state.messages array, which acts as our conversation history. This array stores all messages exchanged between the user and the agents, or any other actors in the workflow, preserving it throughout the workflow execution.

Since by definition this state.messages array is immutable and cannot be modified, the purpose of the State Node is to allow us to define custom key-value pairs, expanding the state object to hold any additional information relevant to our workflow.

{% hint style="info" %} When no Agent Memory Node is used, the State operates in-memory and is not persisted for future use. {% endhint %}

Inputs

The State Node has no specific input connections.

Outputs

The State Node can only connect to the Start Node, allowing the setup of a custom State from the beginning of the workflow and allowing other nodes to access and potentially modify this shared custom State.

Additional Parameters

How to set a custom State

Specify the key, operation type, and default value for the state object. The operation type can be either "Replace" or "Append".

• Replace
  1. Replace the existing value with the new value.
  2. If the new value is null, the existing value will be retained.
• Append
  1. Append the new value to the existing value.
  2. Default values can be empty or an array. Ex: ["a", "b"]
  3. Final value is an array.

Example using JS

{% code overflow="wrap" %}

{
    aggregate: {
        value: (x, y) => x.concat(y), // here we append the new message to the existing messages
        default: () => []
    }
}

{% endcode %}

Example using Table

To define a custom State using the table interface in the State Node, follow these steps:

1. Add item: Click the "+ Add Item" button to add rows to the table. Each row represents a key-value pair in your custom State.
2. Specify keys: In the "Key" column, enter the name of each key you want to define in your state object. For example, you might have keys like "userName", "userLocation", etc.
3. Choose operations: In the "Operation" column, select the desired operation for each key. You have two options:
   • Replace: This will replace the existing value of the key with the new value provided by a node. If the new value is null, the existing value will be retained.
   • Append: This will append the new value to the existing value of the key. The final value will be an array.
4. Set default values: In the "Default Value" column, enter the initial value for each key. This value will be used if no other node provides a value for the key. The default value can be empty or an array.

Example Table

1. This table defines one key in the custom State: userName.
2. The userName key will use the "Replace" operation, meaning its value will be updated whenever a node provides a new value.
3. The userName key has a default value of null, indicating that it has no initial value.

{% hint style="info" %} Remember that this table-based approach is an alternative to defining the custom State using JavaScript. Both methods achieve the same result. {% endhint %}

Example using API

{
    "question": "hello",
    "overrideConfig": {
        "stateMemory": [
            {
                "Key": "userName",
                "Operation": "Replace",
                "Default Value": "somevalue"
            }
        ]
    }
}

Best Practices

{% tabs %} {% tab title="Pro-Tips" %} Plan your custom State structure

Before building your workflow, design the structure of your custom State. A well-organized custom State will make your workflow easier to understand, manage, and debug.

Use meaningful key names

Choose descriptive and consistent key names that clearly indicate the purpose of the data they hold. This will improve the readability of your code and make it easier for others (or you in the future) to understand how the custom State is being used.

Keep custom State minimal

Only store information in the custom State that is essential for the workflow's logic and decision-making.

Consider State persistence

If you need to preserve State across multiple conversation sessions (e.g., for user preferences, order history, etc.), use the Agent Memory Node to store the State in a persistent database. {% endtab %}

{% tab title="Potential Pitfalls" %} Inconsistent State Updates

• Problem: Updating the custom State in multiple nodes without a clear strategy can lead to inconsistencies and unexpected behavior.
• Example
  1. Agent 1 updates orderStatus to "Payment Confirmed".
  2. Agent 2, in a different branch, updates orderStatus to "Order Complete" without checking the previous status.
• Solution: Use Conditions Nodes to control the flow of the custom State updates and ensure that custom State transitions happen in a logical and consistent manner. {% endtab %} {% endtabs %}

4. Agent Node

The Agent Node is a core component of the Sequential Agent architecture. It acts as a decision-maker and orchestrator within our workflow.

Understanding the Agent Node

Upon receiving input from preceding nodes, which always includes the full conversation history state.messages and any custom State at that point in the execution, the Agent Node uses its defined "persona", established by the System Prompt, to determine if external tools are necessary to fulfill the user's request.

• If tools are required, the Agent Node autonomously selects and executes the appropriate tool. This execution can be automatic or, for sensitive tasks, require human approval (HITL) before proceeding. Once the tool completes its operation, the Agent Node receives the results, processes them using the designated Chat Model (LLM), and generates a comprehensive response.
• In cases where no tools are needed, the Agent Node directly leverages the Chat Model (LLM) to formulate a response based on the current conversation context.

Inputs

{% hint style="info" %} The Agent Node requires at least one connection from the following nodes: Start Node, Agent Node, Condition Node, Condition Agent Node, LLM Node, or Tool Node. {% endhint %}

Outputs

The Agent Node can connect to the following nodes as outputs:

• Agent Node: Passes control to a subsequent Agent Node, enabling the chaining of multiple agent actions within a workflow. This allows for more complex conversational flows and task orchestration.
• LLM Node: Passes the agent's output to an LLM Node, enabling further language processing, response generation, or decision-making based on the agent's actions and insights.
• Condition Agent Node: Directs the flow to a Condition Agent Node. This node evaluates the Agent Node's output and its predefined conditions to determine the appropriate next step in the workflow.
• Condition Node: Similar to the Condition Agent Node, the Condition Node uses predefined conditions to assess the Agent Node's output, directing the flow along different branches based on the outcome.
• End Node: Concludes the conversation flow.
• Loop Node: Redirects the flow back to a previous node, enabling iterative or cyclical processes within the workflow. This is useful for tasks that require multiple steps or involve refining results based on previous interactions. For example, you might loop back to an earlier Agent Node or LLM Node to gather additional information or refine the conversation flow based on the current Agent Node's output.

Node Setup

Additional Parameters

Best Practices

{% tabs %} {% tab title="Pro Tips" %} Clear system prompt

Craft a concise and unambiguous System Prompt that accurately reflects the agent's role and capabilities. This guides the agent's decision-making and ensures it acts within its defined scope.

Strategic tool selection

Choose and configure the tools available to the Agent Node, ensuring they align with the agent's purpose and the overall goals of the workflow.

HITL for sensitive tasks

Utilize the 'Require Approval' option for tasks involving sensitive data, requiring human judgment, or carrying a risk of unintended consequences.

Leverage custom State updates

Update the custom State object strategically to store gathered information or influence the behavior of downstream nodes. {% endtab %}

{% tab title="Potential Pitfalls" %} Agent inaction due to tool overload

• Problem: When an Agent Node has access to a large number of tools within a single workflow execution, it might struggle to decide which tool is the most appropriate to use, even when a tool is clearly necessary. This can lead to the agent failing to call any tool at all, resulting in incomplete or inaccurate responses.
• Example: Imagine a customer support agent designed to handle a wide range of inquiries. You've equipped it with tools for order tracking, billing information, product returns, technical support, and more. A user asks, "What's the status of my order?" but the agent, overwhelmed by the number of potential tools, responds with a generic answer like, "I can help you with that. What's your order number?" without actually using the order tracking tool.
• Solution
  1. Refine system prompts: Provide clearer instructions and examples within the Agent Node's System Prompt to guide it towards the correct tool selection. If needed, emphasize the specific capabilities of each tool and the situations in which they should be used.
  2. Limit tool choices per node: If possible, break down complex workflows into smaller, more manageable segments, each with a more focused set of tools. This can help reduce the cognitive load on the agent and improve its tool-selection accuracy.

Overlooking HITL for sensitive tasks

• Problem: Failing to utilize the Agent Node's "Require Approval" (HITL) feature for tasks involving sensitive information, critical decisions, or actions with potential real-world consequences can lead to unintended outcomes or damage to user trust.
• Example: Your travel booking agent has access to a user's payment information and can automatically book flights and hotels. Without HITL, a misinterpretation of user intent or an error in the agent's understanding could result in an incorrect booking or unauthorized use of the user's payment details.
• Solution
  1. Identify sensitive actions: Analyze your workflow and identify any actions that involve accessing or processing sensitive data (e.g., payment info, personal details).
  2. Implement "Require Approval": For these sensitive actions, enable the "Require Approval" option within the Agent Node. This ensures that a human reviews the agent's proposed action and the relevant context before any sensitive data is accessed or any irreversible action is taken.
  3. Design clear approval prompts: Provide clear and concise prompts for human reviewers, summarizing the agent's intent, the proposed action, and the relevant information needed for the reviewer to make an informed decision.

Unclear or incomplete system prompt

• Problem: The System Prompt provided to the Agent Node lacks the necessary specificity and context to guide the agent effectively in carrying out its intended tasks. A vague or overly general prompt can lead to irrelevant responses, difficulty in understanding user intent, and an inability to leverage tools or data appropriately.
• Example: You're building a travel booking agent, and your System Prompt simply states "You are a helpful AI assistant." This lacks the specific instructions and context needed for the agent to effectively guide users through flight searches, hotel bookings, and itinerary planning.
• Solution: Craft a detailed and context-aware System Prompt:

{% code overflow="wrap" %}

You are a travel booking agent. Your primary goal is to assist users in planning and booking their trips. 
- Guide them through searching for flights, finding accommodations, and exploring destinations.
- Be polite, patient, and offer travel recommendations based on their preferences.
- Utilize available tools to access flight data, hotel availability, and destination information.

{% endcode %} {% endtab %} {% endtabs %}

5. LLM Node

Like the Agent Node, the LLM Node is a core component of the Sequential Agent architecture. Both nodes utilize the same Chat Models (LLMs) by default, providing the same basic language processing capabilities, but the LLM Node distinguishes itself in these key areas.

Key advantages of the LLM Node

While a detailed comparison between the LLM Node and the Agent Node is available in this section, here's a brief overview of the LLM Node's key advantages:

• Structured data: The LLM Node provides a dedicated feature to define a JSON schema for its output. This makes it exceptionally easy to extract structured information from the LLM's responses and pass that data to consequent nodes in the workflow. The Agent Node does not have this built-in JSON schema feature
• HITL: While both nodes support HITL for tool execution, the LLM Node defers this control to the Tool Node itself, providing more flexibility in workflow design.

Inputs

{% hint style="info" %} The LLM Node requires at least one connection from the following nodes: Start Node, Agent Node, Condition Node, Condition Agent Node, LLM Node, or Tool Node. {% endhint %}

Node Setup

Outputs

The LLM Node can connect to the following nodes as outputs:

• Agent Node: Passes the LLM's output to an Agent Node, which can then use the information to decide on actions, execute tools, or guide the conversation flow.
• LLM Node: Passes the output to a subsequent LLM Node, enabling chaining of multiple LLM operations. This is useful for tasks like refining text generation, performing multiple analyses, or breaking down complex language processing into stages.
• Tool Node: Passes the output to a Tool Node, enabling the execution of a specific tool based on the LLM Node's instructions.
• Condition Agent Node: Directs the flow to a Condition Agent Node. This node evaluates the LLM Node's output and its predefined conditions to determine the appropriate next step in the workflow.
• Condition Node: Similar to the Condition Agent Node, the Condition Node uses predefined conditions to assess the LLM Node's output, directing the flow along different branches based on the outcome.
• End Node: Concludes the conversation flow.
• Loop Node: Redirects the flow back to a previous node, enabling iterative or cyclical processes within the workflow. This could be used to refine the LLM's output over multiple iterations.

Additional Parameters

Best Practices

{% tabs %} {% tab title="Pro Tips" %} Clear system prompt

Craft a concise and unambiguous System Prompt that accurately reflects the LLM Node's role and capabilities. This guides the LLM Node's decision-making and ensures it acts within its defined scope.

Optimize for structured output

Keep your JSON schemas as straightforward as possible, focusing on the essential data elements. Only enable JSON Structured Output when you need to extract specific data points from the LLM's response or when downstream nodes require JSON input.

Strategic tool selection

Choose and configure the tools available to the LLM Node (via the Tool Node), ensuring they align with the application purpose and the overall goals of the workflow.

HITL for sensitive tasks

Utilize the 'Require Approval' option for tasks involving sensitive data, requiring human judgment, or carrying a risk of unintended consequences.

Leverage State updates

Update the custom State object strategically to store gathered information or influence the behavior of downstream nodes. {% endtab %}

{% tab title="Potential Pitfalls" %} Unintentional tool execution due to Incorrect HITL setup

• Problem: While the LLM Node can trigger Tool Nodes, it relies on the Tool Node's configuration for Human-in-the-Loop (HITL) approval. Failing to properly configure HITL for sensitive actions can lead to tools being executed without human review, potentially causing unintended consequences.
• Example: Your LLM Node is designed to interact with a tool that makes changes to user data. You intend to have a human review these changes before execution, but the connected Tool Node's "Require Approval" option is not enabled. This could result in the tool automatically modifying user data based solely on the LLM's output, without any human oversight.
• Solution
  1. Double-Check tool node settings: Always ensure that the "Require Approval" option is enabled within the settings of any Tool Node that handles sensitive actions.
  2. Test HITL thoroughly: Before deploying your workflow, test the HITL process to ensure that human review steps are triggered as expected and that the approval/rejection mechanism functions correctly.

Overuse or misunderstanding of JSON structured output

• Problem: While the LLM Node's JSON Structured Output feature is powerful, misusing it or not fully understanding its implications can lead to data errors.
• Example: You define a complex JSON schema for the LLM Node's output, even though the downstream tasks only require a simple text response. This adds unnecessary complexity and makes your workflow harder to understand and maintain. Additionally, if the LLM's output doesn't conform to the defined schema, it can cause errors in subsequent nodes.
• Solution
  1. Use JSON output strategically: Only enable JSON Structured Output when you have a clear need to extract specific data points from the LLM's response or when the downstream Tool Nodes require JSON input.
  2. Keep schemas simple: Design your JSON schemas to be as simple and concise as possible, focusing only on the data elements that are absolutely necessary for the task. {% endtab %} {% endtabs %}

6. Tool Node

The Tool Node is a valuable component of Aimicromind's Sequential Agent system, enabling the integration and execution of external tools within conversational workflows. It acts as a bridge between the language-based processing of LLM Nodes and the specialized functionalities of external tools, APIs, or services.

Understanding the Tool Node

The Tool Node's primary function is to execute external tools based on instructions received from an LLM Node and to provide flexibility for Human-in-the-Loop (HITL) intervention in the tool execution process.

Here's a step-by-step explanation of how it works

1. Tool Call Reception: The Tool Node receives input from an LLM Node. If the LLM's output contains the tool_calls property, the Tool Node will proceed with tool execution.
2. Execution: The Tool Node directly passes the LLM's tool_calls (which include the tool name and any required parameters) to the specified external tool. Otherwise, the Tool Node does not execute any tools in that particular workflow execution. It does not process or interpret the LLM's output in any way.
3. Human-in-the-Loop (HITL): The Tool Node allows for optional HITL, enabling human review and approval or rejection of tool execution before it occurs.
4. Output passing: After the tool execution (either automatic or after HITL approval), the Tool Node receives the tool's output and passes it to the next node in the workflow. If the Tool Node's output is not connected to a subsequent node, the tool's output is returned to the original LLM Node for further processing.

Inputs

Node Setup

Outputs

The Tool Node can connect to the following nodes as outputs:

• Agent Node: Passes the Tool Node's output (the result of the executed tool) to an Agent Node. The Agent Node can then use this information to decide on actions, execute further tools, or guide the conversation flow.
• LLM Node: Passes the output to a subsequent LLM Node. This enables the integration of tool results into the LLM's processing, allowing for further analysis or refinement of the conversation flow based on the tool's output.
• Condition Agent Node: Directs the flow to a Condition tool Node. This node evaluates the Tool Node's output and its predefined conditions to determine the appropriate next step in the workflow.
• Condition Node: Similar to the Condition Agent Node, the Condition Node uses predefined conditions to assess the Tool Node's output, directing the flow along different branches based on the outcome.
• End Node: Concludes the conversation flow.
• Loop Node: Redirects the flow back to a previous node, enabling iterative or cyclical processes within the workflow. This could be used for tasks that require multiple tool executions or involve refining the conversation based on tool results.

Additional Parameters

Best Practices

{% tabs %} {% tab title="Pro Tips" %} Strategic HITL placement

Consider which tools require human oversight (HITL) and enable the "Require Approval" option accordingly.

Informative Approval Prompts

When using HITL, design clear and informative prompts for human reviewers. Provide sufficient context from the conversation and summarize the tool's intended action. {% endtab %}

{% tab title="Potential Pitfalls" %} Unhandled tool output formats

• Problem: The Tool Node outputs data in a format that is not expected or handled by subsequent nodes in the workflow, leading to errors or incorrect processing.
• Example: A Tool Node retrieves data from an API in JSON format, but the following LLM Node expects text input, causing a parsing error.
• Solution: Ensure that the output format of the external tool is compatible with the input requirements of the nodes connected to the Tool Node's output. {% endtab %} {% endtabs %}

7. Condition Node

The Condition Node acts as a decision-making point in Sequential Agent workflows, evaluating a set of predefined conditions to determine the flow's next path.

Understanding the Condition Node

The Condition Node is essential for building workflows that adapt to different situations and user inputs. It examines the current State of the conversation, which includes all messages exchanged and any custom State variables previously defined. Then, based on the evaluation of the conditions specified in the node setup, the Condition Node directs the flow to one of its outputs.

For instance, after an Agent or LLM Node provides a response, a Condition Node could check if the response contains a specific keyword or if a certain condition is met in the custom State. If it does, the flow might be directed to an Agent Node for further action. If not, it could lead to a different path, perhaps ending the conversation or prompting the user with additional questions.

This enables us to create branches in our workflow, where the path taken depends on the data flowing through the system.

Here's a step-by-step explanation of how it works

1. The Condition Node receives input from any preceding node: Start Node, Agent Node, LLM Node, or Tool Node.
2. It has access to the full conversation history and the custom State (if any), giving it plenty of context to work with.
3. We define a condition that the node will evaluate. This could be checking for keywords, comparing values in the state, or any other logic we could implement via JavaScript.
4. Based on whether the condition evaluates to true or false, the Condition Node sends the flow down one of its predefined output paths. This creates a "fork in the road" or branch for our workflow.

How to set up conditions

The Condition Node allows us to define dynamic branching logic in our workflow by choosing either a table-based interface or a JavaScript code editor to define the conditions that will control the conversation flow.

const lastMessage = $flow.state.messages[$flow.state.messages.length - 1].content; 
return lastMessage.includes("yes") ? "Output 1" : "Output 2";

return $flow.state.orderStatus === "confirmed" ? "Output 1" : "Output 2";

Defining conditions using the table interface

This visual approach allows you to easily set up rules that determine the path of your conversational flow, based on factors like user input, the current state of the conversation, or the results of actions taken by other nodes.

Inputs

{% hint style="info" %} The Condition Node requires at least one connection from the following nodes: Start Node, Agent Node, LLM Node, or Tool Node. {% endhint %}

Outputs

The Condition Node dynamically determines its output path based on the predefined conditions, using either the table-based interface or JavaScript. This provides flexibility in directing the workflow based on condition evaluations.

Condition evaluation logic

• Table-Based conditions: The conditions in the table are evaluated sequentially, from top to bottom. The first condition that evaluates to true triggers its corresponding output. If none of the predefined conditions are met, the workflow is directed to the default "End" output.
• Code-Based conditions: When using JavaScript, we must explicitly return the name of the desired output path, including a name for the default "End" output.
• Single output path: Only one output path is activated at a time. Even if multiple conditions could be true, only the first matching condition determines the flow.

Connecting outputs

Each predefined output, including the default "End" output, can be connected to any of the following nodes:

• Agent Node: To continue the conversation with an agent, potentially taking actions based on the condition's outcome.
• LLM Node: To process the current State and conversation history with an LLM, generating responses or making further decisions.
• End Node: To terminate the conversation flow. If any output, including the default "End" output, is connected to an End Node, the Condition Node will output the last response from the preceding node and end the workflow.
• Loop Node: To redirect the flow back to a previous sequential node, enabling iterative processes based on the condition's outcome.

Node Setup

Best Practices

{% tabs %} {% tab title="Pro Tips" %} Clear condition naming

Use descriptive names for your conditions (e.g., "If user is under 18, then Policy Advisor Agent", "If order is confirmed, then End Node") to make your workflow easier to understand and debug.

Prioritize simple conditions

Start with simple conditions and gradually add complexity as needed. This makes your workflow more manageable and reduces the risk of errors. {% endtab %}

{% tab title="Potential Pitfalls" %} Mismatched condition logic and workflow design

• Problem: The conditions you define in the Condition Node do not accurately reflect the intended logic of your workflow, leading to unexpected branching or incorrect execution paths.
• Example: You set up a condition to check if the user's age is greater than 18, but the output path for that condition leads to a section designed for users under 18.
• Solution: Review your conditions and ensure that the output paths associated with each condition match the intended workflow logic. Use clear and descriptive names for your outputs to avoid confusion.

Insufficient State management

• Problem: The Condition Node relies on a custom state variable that is not updated correctly, leading to inaccurate condition evaluations and incorrect branching.
• Example: You're tracking a "userLocation" variable in the custom State, but the variable is not updated when the user provides their location. The Condition Node evaluates the condition based on the outdated value, leading to an incorrect path.
• Solution: Ensure that any custom state variables used in your conditions are updated correctly throughout the workflow. {% endtab %} {% endtabs %}

8. Condition Agent Node

The Condition Agent Node provides dynamic and intelligent routing within Sequential Agent flows. It combines the capabilities of the LLM Node (LLM and JSON Structured Output) and the Condition Node (user-defined conditions), allowing us to leverage agent-based reasoning and conditional logic within a single node.

Key functionalities

• Unified agent-based routing: Combines agent reasoning, structured output, and conditional logic in a single node, simplifying workflow design.
• Contextual awareness: The agent considers the entire conversation history and any custom State when evaluating conditions.
• Flexibility: Provides both table-based and code-based options for defining conditions, when catering to different user preferences and skill levels.

Setting up the Condition Agent Node

The Condition Agent Node acts as a specialized agent that can both process information and make routing decisions. Here's how to configure it:

1. Define the agent's persona
   • In the "System Prompt" field, provide a clear and concise description of the agent's role and the task it needs to perform for conditional routing. This prompt will guide the agent's understanding of the conversation and its decision-making process.
2. Structure the Agent's Output (Optional)
   • If you want the agent to produce structured output, use the "JSON Structured Output" feature. Define the desired schema for the output, specifying the keys, data types, and any enum values. This structured output will be used by the agent when evaluating conditions.
3. Define conditions
   • Choose either the table-based interface or the JavaScript code editor to define the conditions that will determine the routing behavior.
     • Table-Based interface: Add rows to the table, specifying the variable to check, the comparison operation, the value to compare against, and the output name to follow if the condition is met.
     • JavaScript code: Write custom JavaScript snippets to evaluate conditions. Use the return statement to specify the name of the output path to follow based on the condition's result.
4. Connect outputs
   • Connect each predefined output, including the default "End" output, to the appropriate subsequent node in the workflow. This could be an Agent Node, LLM Node, Loop Node, or an End Node.

How to set up conditions

The Condition Agent Node allows us to define dynamic branching logic in our workflow by choose either a table-based interface or a JavaScript code editor to define the conditions that will control the conversation flow.

const lastMessage = $flow.state.messages[$flow.state.messages.length - 1].content; 
return lastMessage.includes("yes") ? "Output 1" : "Output 2";

return $flow.state.orderStatus === "confirmed" ? "Output 1" : "Output 2";

Defining conditions using the table interface

This visual approach allows you to easily set up rules that determine the path of your conversational flow, based on factors like user input, the current state of the conversation, or the results of actions taken by other nodes.

Inputs

{% hint style="info" %} The Condition Agent Node requires at least one connection from the following nodes: Start Node, Agent Node, LLM Node, or Tool Node. {% endhint %}

Node Setup

Outputs

The Condition Agent Node, like the Condition Node, dynamically determines its output path based on the conditions defined, using either the table-based interface or JavaScript. This provides flexibility in directing the workflow based on condition evaluations.

Condition evaluation logic

• Table-Based conditions: The conditions in the table are evaluated sequentially, from top to bottom. The first condition that evaluates to true triggers its corresponding output. If none of the predefined conditions are met, the workflow is directed to the default "End" output.
• Code-Based conditions: When using JavaScript, we must explicitly return the name of the desired output path, including a name for the default "End" output.
• Single output path: Only one output path is activated at a time. Even if multiple conditions could be true, only the first matching condition determines the flow.

Connecting outputs

Each predefined output, including the default "End" output, can be connected to any of the following nodes:

• Agent Node: To continue the conversation with an agent, potentially taking actions based on the condition's outcome.
• LLM Node: To process the current State and conversation history with an LLM, generating responses or making further decisions.
• End Node: To terminate the conversation flow. If the default "End" output is connected to an End Node, the Condition Node will output the last response from the preceding node and end the conversation.
• Loop Node: To redirect the flow back to a previous sequential node, enabling iterative processes based on the condition's outcome.

Key differences from the Condition Node

• The Condition Agent Node incorporates an agent's reasoning and structured output into the condition evaluation process.
• It provides a more integrated approach to agent-based condition routing.

Additional Parameters

Best Practices

{% tabs %} {% tab title="Pro Tips" %} Craft a clear and focused system prompt

Provide a well-defined persona and clear instructions to the agent in the System Prompt. This will guide its reasoning and help it generate relevant output for the conditional logic.

Structure output for reliable conditions

Use the JSON Structured Output feature to define a schema for the Condition Agent's output. This will ensure that the output is consistent and easily parsable, making it more reliable for use in conditional evaluations. {% endtab %}

{% tab title="Potential Pitfalls" %} Unreliable routing due to unstructured output

• Problem: The Condition Agent Node is not configured to output structured JSON data, leading to unpredictable output formats that can make it difficult to define reliable conditions.
• Example: The Condition Agent Node is asked to determine user sentiment (positive, negative, neutral) but outputs its assessment as a free-form text string. The variability in the agent's language makes it challenging to create accurate conditions in the conditional table or code.
• Solution: Use the JSON Structured Output feature to define a schema for the agent's output. For example, specify a "sentiment" key with an enum of "positive," "negative," and "neutral." This will ensure that the agent's output is consistently structured, making it much easier to create reliable conditions. {% endtab %} {% endtabs %}

9. Loop Node

The Loop Node allows us to create loops within our conversational flow, redirecting the conversation back to a specific point. This is useful for scenarios where we need to repeat a certain sequence of actions or questions based on user input or specific conditions.

Understanding the Loop Node

The Loop Node acts as a connector, redirecting the flow back to a specific point in the graph, allowing us to create loops within our conversational flow. It passes the current State, which includes the output of the node preceding the Loop Node to our target node. This data transfer allows our target node to process information from the previous iteration of the loop and adjust its behavior accordingly.

For instance, let's say we're building a chatbot that helps users book flights. We might use a loop to iteratively refine the search criteria based on user feedback.

Here's how the Loop Node could be used

1. LLM Node (Initial Search): The LLM Node receives the user's initial flight request (e.g., "Find flights from Madrid to New York in July"). It queries a flight search API and returns a list of possible flights.
2. Agent Node (Present Options): The Agent Node presents the flight options to the user and asks if they would like to refine their search (e.g., "Would you like to filter by price, airline, or departure time?").
3. Condition Agent Node: The Condition Agent Node checks the user's response and has two outputs:
   • If the user wants to refine: The flow goes to the "Refine Search" LLM Node.
   • If the user is happy with the results: The flow proceeds to the booking process.
4. LLM Node (Refine Search): This LLM Node gathers the user's refinement criteria (e.g., "Show me only flights under $500") and updates the State with the new search parameters.
5. Loop Node: The Loop Node redirects the flow back to the initial LLM Node ("Initial Search"). It passes the updated State, which now includes the refined search criteria.
6. Iteration: The initial LLM Node performs a new search using the refined criteria, and the process repeats from step 2.

In this example, the Loop Node enables an iterative search refinement process. The system can continue to loop back and refine the search results until the user is satisfied with the options presented.

Inputs

{% hint style="info" %} The Loop Node requires at least one connection from the following nodes: Agent Node, LLM Node, Tool Node, Condition Node, or Condition Agent Node. {% endhint %}

Node Setup

Outputs

The Loop Node does not have any direct output connections. It redirects the flow back to the specific sequential node in the graph.

Best Practices

{% tabs %} {% tab title="Pro Tips" %} Clear loop purpose

Define a clear purpose for each loop in your workflow. If possible, document with a sticky note what you're trying to achieve with the loop. {% endtab %}

{% tab title="Potencial Pitfalls" %} Confusing workflow structure

• Problem: Excessive or poorly designed loops make the workflow difficult to understand and maintain.
• Example: You use multiple nested loops without clear purpose or labels, making it hard to follow the flow of the conversation.
• Solution: Use loops sparingly and only when necessary. Clearly document your Loop Nodes and the nodes they connect to.

Infinite loops due to missing or incorrect exit conditions

• Problem: The loop never terminates because the condition that should trigger the loop's exit is either missing or incorrectly defined.
• Example: A Loop Node is used to iteratively gather user information. However, the workflow lacks a Conditional Agent Node to check if all required information has been collected. As a result, the loop continues indefinitely, repeatedly asking the user for the same information.
• Solution: Always define clear and accurate exit conditions for loops. Use Condition Nodes to check state variables, user input, or other factors that indicate when the loop should terminate. {% endtab %} {% endtabs %}

10. End Node

The End Node marks the definitive termination point of the conversation in a Sequential Agent workflow. It signifies that no further processing, actions, or interactions are required.

Understanding the End Node

The End Node serves as a signal within Aimicromind's Sequential Agent architecture, indicating that the conversation has reached its intended conclusion. Upon reaching the End Node, the system "understands" that the conversational objective has been met, and no further actions or interactions are required within the flow.

Inputs

{% hint style="info" %} The End Node requires at least one connection from the following nodes: Agent Node, LLM Node, or Tool Node. {% endhint %}

Outputs

The End Node does not have any output connections as it signifies the termination of the information flow.

Best Practices

{% tabs %} {% tab title="Pro Tips" %} Provide a final response

If appropriate, connect the End Node to an dedicated LLM or Agent Node to generate a final message or summary for the user, providing closure to the conversation. {% endtab %}

{% tab title="Potencial Pitfalls" %} Premature conversation termination

• Problem: The End Node is placed too early in the workflow, causing the conversation to end before all necessary steps are completed or the user's request is fully addressed.
• Example: A chatbot designed to collect user feedback ends the conversation after the user provides their first comment, without giving them an opportunity to provide additional feedback or ask questions.
• Solution: Review your workflow logic and ensure that the End Node is placed only after all essential steps have been completed or the user has explicitly indicated their intent to end the conversation.

Lack of closure for the user

• Problem: The conversation ends abruptly without a clear signal to the user or a final message that provides a sense of closure.
• Example: A customer support chatbot ends the conversation immediately after resolving an issue, without confirming the resolution with the user or offering further assistance.
• Solution: Connect the End Node to a dedicate LLM or Agent Node to generate a final response that summarizes the conversation, confirms any actions taken, and provides a sense of closure for the user. {% endtab %} {% endtabs %}

Condition Node vs. Condition Agent Node

The Condition and Condition Agent Nodes are essential in Aimicromind's Sequential Agent architecture for creating dynamic conversational experiences.

These nodes enable adaptive workflows, responding to user input, context, and complex decisions, but differ in their approach to condition evaluation and sophistication.

Summarizing

Choosing the right node

• Condition Node: Use the Condition Node when your routing logic involves straightforward decisions based on easily definable conditions. For instance, it's perfect for checking for specific keywords, comparing values in the State, or evaluating other simple logical expressions.
• Condition Agent Node: However, when your routing demands a deeper understanding of the conversation's nuances, the Condition Agent Node is the better choice. This node acts as your intelligent routing assistant, leveraging an LLM to analyze the conversation, make judgments based on context, and provide structured output that drives more sophisticated and dynamic routing.

Agent Node vs. LLM Node

It's important to understand that both the LLM Node and the Agent Node can be considered agentic entities within our system, as they both leverage the capabilities of a large language model (LLM) or Chat Model.

However, while both nodes can process language and interact with tools, they are designed for different purposes within a workflow.

Summarizing

Choosing the right node

• Choose the Agent Node: Use the Agent Node when you need to create a conversational system that can manage the execution of multiple tools, all of which share the same HITL setting (enabled or disabled for the entire Agent Node). The Agent Node is also well-suited for handling complex multi-step conversations where consistent agent-like behavior is desired.
• Choose the LLM Node: On the other hand, use the LLM Node when you need to extract structured data from the LLM's output using the JSON schema feature, a capability not available in the Agent Node. The LLM Node also excels at orchestrating tool execution with fine-grained control over HITL at the individual tool level, allowing you to mix automated and human-reviewed tool executions by using multiple Tool Nodes connected to the LLM Node.

1. In our current context, a lower level of abstraction refers to a system that exposes a greater degree of implementation detail to the developer. ↩

description: Learn Sequential Agents from the Community

Video Tutorials (Coming Soon)

description: >- Learn more about the details of some of the most used APIs: prediction, vector-upsert

API

Refer to API Reference for full list of public APIs

Prediction

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/prediction/{id}" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

Using Python/TS Library

aimicromindprovides 2 libraries:

• Python: pip install aimicromind
• Typescript: npm install aimicromind-sdk

{% tabs %} {% tab title="Python SDK" %}

from aimicromindimport AiMicromind, PredictionData

def test_non_streaming():
    client = AiMicromind()

    # Test non-streaming prediction
    completion = client.create_prediction(
        PredictionData(
            chatflowId="<chatflow-id>",
            question="What is the capital of France?",
            streaming=False
        )
    )

    # Process and print the response
    for response in completion:
        print("Non-streaming response:", response)

def test_streaming():
    client = AiMicromind()

    # Test streaming prediction
    completion = client.create_prediction(
        PredictionData(
            chatflowId="<chatflow-id>",
            question="Tell me a joke!",
            streaming=True
        )
    )

    # Process and print each streamed chunk
    print("Streaming response:")
    for chunk in completion:
        print(chunk)


if __name__ == "__main__":
    # Run non-streaming test
    test_non_streaming()

    # Run streaming test
    test_streaming()

{% endtab %}

{% tab title="Typescript SDK" %}

import { AiMicromindClient } from 'aimicromind-sdk'

async function test_streaming() {
  const client = new AiMicromindClient({ baseUrl: 'http://localhost:3000' });

  try {
    // For streaming prediction
    const prediction = await client.createPrediction({
      chatflowId: 'fe1145fa-1b2b-45b7-b2ba-bcc5aaeb5ffd',
      question: 'What is the revenue of Apple?',
      streaming: true,
    });

    for await (const chunk of prediction) {
        console.log(chunk);
    }
    
  } catch (error) {
    console.error('Error:', error);
  }
}

async function test_non_streaming() {
    const client = new AiMicromindClient({ baseUrl: 'http://localhost:3000' });
  
    try {
      // For streaming prediction
      const prediction = await client.createPrediction({
        chatflowId: 'fe1145fa-1b2b-45b7-b2ba-bcc5aaeb5ffd',
        question: 'What is the revenue of Apple?',
      });
  
      console.log(prediction);
      
    } catch (error) {
      console.error('Error:', error);
    }
}

// Run non-streaming test
test_non_streaming()

// Run streaming test
test_streaming()

{% endtab %} {% endtabs %}

Override Config

Override existing input configuration of the chatflow with overrideConfig property.

Due to security reason, override config is disabled by default. User has to enable this by going into Chatflow Configuration -> Security tab. Then select the property that can be overriden.

{% tabs %} {% tab title="Python API" %}

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatflowId>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "question": "Hey, how are you?",
    "overrideConfig": {
        "sessionId": "123",
        "returnSourceDocuments": true
    }
})

{% endtab %}

{% tab title="Javascript API" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatflowId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "question": "Hey, how are you?",
    "overrideConfig": {
        "sessionId": "123",
        "returnSourceDocuments": true
    }
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

History

You can prepend history messages to give some context to LLM. For example, if you want the LLM to remember user's name:

{% tabs %} {% tab title="Python API" %}

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatflowId>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "question": "Hey, how are you?",
    "history": [
        {
            "role": "apiMessage",
            "content": "Hello how can I help?"
        },
        {
            "role": "userMessage",
            "content": "Hi my name is Brian"
        },
        {
            "role": "apiMessage",
            "content": "Hi Brian, how can I help?"
        },
    ]
})

{% endtab %}

{% tab title="Javascript API" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatflowId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "question": "Hey, how are you?",
    "history": [
        {
            "role": "apiMessage",
            "content": "Hello how can I help?"
        },
        {
            "role": "userMessage",
            "content": "Hi my name is Brian"
        },
        {
            "role": "apiMessage",
            "content": "Hi Brian, how can I help?"
        },
    ]
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Persists Memory

You can pass a sessionId to persists the state of the conversation, so the every subsequent API calls will have context about previous conversation. Otherwise, a new session will be generated each time.

{% tabs %} {% tab title="Python API" %}

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatflowId>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "question": "Hey, how are you?",
    "overrideConfig": {
        "sessionId": "123"
    } 
})

{% endtab %}

{% tab title="Javascript API" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatflowId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "question": "Hey, how are you?",
    "overrideConfig": {
        "sessionId": "123"
    }
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Variables

Pass variables in the API to be used by the nodes in the flow. See more: Variables

{% tabs %} {% tab title="Python API" %}

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatflowId>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "question": "Hey, how are you?",
    "overrideConfig": {
        "vars": {
            "foo": "bar"
        }
    }
})

{% endtab %}

{% tab title="Javascript API" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatflowId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "question": "Hey, how are you?",
    "overrideConfig": {
        "vars": {
            "foo": "bar"
        }
    }
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Image Uploads

When Allow Image Upload is enabled, images can be uploaded from chat interface.

{% tabs %} {% tab title="Python API" %}

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatflowId>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "question": "Can you describe the image?",
    "uploads": [
        {
            "data": 'data:image/png;base64,iVBORw0KGgdM2uN0', # base64 string or url
            "type": 'file', # file | url
            "name": 'AiMicroMind.png',
            "mime": 'image/png'
        }
    ]
})

{% endtab %}

{% tab title="Javascript API" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatflowId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "question": "Can you describe the image?",
    "uploads": [
        {
            "data": 'data:image/png;base64,iVBORw0KGgdM2uN0', //base64 string or url
            "type": 'file', //file | url
            "name": 'AiMicromind.png',
            "mime": 'image/png'
        }
    ]
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Speech to Text

When Speech to Text is enabled, users can speak directly into microphone and speech will be transcribed into text.

{% tabs %} {% tab title="Python API" %}

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatflowId>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "uploads": [
        {
            "data": 'data:audio/webm;codecs=opus;base64,GkXf', #base64 string
            "type": 'audio',
            "name": 'audio.wav',
            "mime": 'audio/webm'
        }
    ]
})

{% endtab %}

{% tab title="Javascript API" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatflowId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "uploads": [
        {
            "data": 'data:audio/webm;codecs=opus;base64,GkXf', //base64 string
            "type": 'audio',
            "name": 'audio.wav',
            "mime": 'audio/webm'
        }
    ]
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Vector Upsert API

{% swagger src="../.gitbook/assets/swagger (1) (1) (1).yml" path="/vector/upsert/{id}" method="post" %} swagger (1) (1) (1).yml {% endswagger %}

Document Loaders with File Upload

Some document loaders in aimicromind allow user to upload files:

• CSV File
• Docx File
• Json File
• Json Lines File
• PDF File
• Text File
• Unstructured File

If the flow contains Document Loaders with Upload File functionality, the API looks slightly different. Instead of passing body as JSON, form data is being used. This allows you to send files to the API.

{% hint style="info" %} Make sure the sent file type is compatible with the expected file type from document loader. For example, if a PDF File Loader is being used, you should only send .pdf files.

To avoid having separate loaders for different file types, we recommend to use File Loader {% endhint %}

{% tabs %} {% tab title="Python API" %}

import requests

API_URL = "http://localhost:3000/api/v1/vector/upsert/<chatflowId>"

# use form data to upload files
form_data = {
    "files": ('state_of_the_union.txt', open('state_of_the_union.txt', 'rb'))
}

body_data = {
    "returnSourceDocuments": True
}

def query(form_data):
    response = requests.post(API_URL, files=form_data, data=body_data)
    print(response)
    return response.json()

output = query(form_data)
print(output)

{% endtab %}

{% tab title="Javascript API" %}

// use FormData to upload files
let formData = new FormData();
formData.append("files", input.files[0]);
formData.append("returnSourceDocuments", true);

async function query(formData) {
    const response = await fetch(
        "http://localhost:3000/api/v1/vector/upsert/<chatflowId>",
        {
            method: "POST",
            body: formData
        }
    );
    const result = await response.json();
    return result;
}

query(formData).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Document Loaders without Upload

For other Document Loaders nodes without Upload File functionality, the API body is in JSON format similar to Prediction API.

{% tabs %} {% tab title="Python API" %}

import requests

API_URL = "http://localhost:3000/api/v1/vector/upsert/<chatflowId>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    print(response)
    return response.json()

output = query({
    "overrideConfig": { # optional
        "returnSourceDocuments": true
    }
})
print(output)

{% endtab %}

{% tab title="Javascript API" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/vector/upsert/<chatflowId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "overrideConfig": { // optional
        "returnSourceDocuments": true
    }
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Document Upsert/Refresh API

Refer to Document Stores section for more information about how to use the API.

{% swagger src="../.gitbook/assets/swagger (2) (1).yml" path="/document-store/upsert/{id}" method="post" %} swagger (2) (1).yml {% endswagger %}

{% swagger src="../.gitbook/assets/swagger (2) (1).yml" path="/document-store/refresh/{id}" method="post" %} swagger (2) (1).yml {% endswagger %}

Video Tutorials (Coming soon)

Those video tutorials (coming soon) cover the main use cases for implementing the aimicromindAPI.

description: Learn how to analyze and troubleshoot your chatflows and agentflows

Analytic

There are several analytic providers aimicromind integrates with:

• LunaryAI
• Langsmith
• Langfuse
• LangWatch
• Arize
• Phoenix
• Opik

Setup

1. At the top right corner of your Chatflow or Agentflow, click Settings > Configuration

2. Then go to the Analyse Chatflow section

3. You will see a list of providers, along with their configuration fields

4. Fill in the credentials and other configuration details, then turn the provider ON. Click Save.

API

Once the analytic has been turned ON from the UI, you can override or provide additional configuration in the body of the Prediction API:

{
  "question": "hi there",
  "overrideConfig": {
    "analytics": {
      "langFuse": {
        // langSmith, langFuse, lunary, langWatch, opik
        "userId": "user1"
      }
    }
  }
}

description: Learn how to setup Arize to analyze and troubleshoot your chatflows and agentflows

Arize

Arize AI is a production-grade observability platform for monitoring, debugging, and improving LLM applications and AI Agents at scale. For a free, open-source alternative, explore Phoenix.

Setup

1. At the top right corner of your Chatflow or Agentflow, click Settings > Configuration

2. Then go to the Analyse Chatflow section

3. You will see a list of providers, along with their configuration fields. Click on Arize.

4. Create credentials for Arize. Refer to the official guide on how to get the Arize API key.

5. Fill in other configuration details, then turn the provider ON

Langfuse

Langfuse is an open source LLM engineering platform that helps teams trace API calls, monitor performance, and debug issues in their AI applications.

With the native integration, you can use aimicromind to quickly create complex LLM applications in no-code and then use Langfuse to monitor and improve them.

The integration supports all use cases of AiMicromind, including: interactively in the UI, API, and embeds.(coming soon)

You can optionally add release to tag the current version of the flow. You usually don't need to change the other options.

Lunary

Lunary is a monitoring and analytics platform for LLM chatbots.

AiMicromind has partnered with Lunary to provide a complete integration supporting user tracing, feedback tracking, conversation replays and detailed LLM analytics.

AiMicromind users can get a 30% discount on the Teams Plan using code MICROMINDERSFRIENDS during checkout.

Read more on how to setup Lunary with aimicromindhere.

description: Learn how to setup Opik to analyze and troubleshoot your chatflows and agentflows

Opik

Setup

1. At the top right corner of your Chatflow or Agentflow, click Settings > Configuration

2. Then go to the Analyse Chatflow section

3. You will see a list of providers, along with their configuration fields. Click on Opik.

4. Create credentials for Opik. Refer to the official guide on how to get the Opik API key.

5. Fill in other configuration details, then turn the provider ON

Now you can analyze your chatflows and agentflows using Opik UI:

description: Learn how to setup Phoenix to analyze and troubleshoot your chatflows and agentflows

Phoenix

Phoenix is an open-source observability tool designed for experimentation, evaluation, and troubleshooting of AI and LLM applications. It can be access in its Cloud form online, or self-hosted and run on your own machine or server.

Setup

1. At the top right corner of your Chatflow or Agentflow, click Settings > Configuration

2. Then go to the Analyse Chatflow section

3. You will see a list of providers, along with their configuration fields. Click on Phoenix.

4. Create credentials for Phoenix. Refer to the official guide on how to get the Phoenix API key.

5. Fill in other configuration details, then turn the provider ON. Click Save.

description: Learn how to use the aimicromind Document Stores, written by @toi500

Document Stores

AiMicromind's Document Stores offer a versatile approach to data management, enabling you to upload, split, and prepare your dataset and upsert it in a single location.

This centralized approach simplifies data handling and allows for efficient management of various data formats, making it easier to organize and access your data within the aimicromind app.

Setup

In this tutorial, we will set up a Retrieval Augmented Generation (RAG) system to retrieve information about the LibertyGuard Deluxe Homeowners Policy, a topic that LLMs are not extensively trained on.

Using the aimicromind Document Stores, we'll prepare and upsert data about LibertyGuard and its set of home insurance policies. This will enable our RAG system to accurately answer user queries about LibertyGuard's home insurance offerings.

1. Add a Document Store

• Start by adding a Document Store and naming it. In our case, "LibertyGuard Deluxe Homeowners Policy".

2. Select a Document Loader

• Enter the Document Store that you just created and select the Document Loader you want to use. In our case, since our dataset is in PDF format, we'll use the PDF Loader.

3. Prepare Your Data

• First, we start by uploading our PDF file.
• Then, we add a unique metadata key. This is optional, but a good practice as it allows us to target and filter down this same dataset later on if we need to.

• Finally, select the Text Splitter you want to use to chunk your data. In our particular case, we will use the Recursive Character Text Splitter.

{% hint style="info" %} In this guide, we've added a generous Chunk Overlap size to ensure no relevant data gets missed between chunks. However, the optimal overlap size is dependent on the complexity of your data. You may need to adjust this value based on your specific dataset and the nature of the information you want to extract. More about this topic in this guide. {% endhint %}

4. Preview Your Data

• We can now preview how our data will be chunked using our current Text Splitter configuration; chunk_size=1500and chunk_overlap=750.

• It's important to experiment with different Text Splitters, Chunk Sizes, and Overlap values to find the optimal configuration for your specific dataset. This preview allows you to refine the chunking process and ensure that the resulting chunks are suitable for your RAG system.

{% hint style="info" %} Note that our custom metadata company: "liberty" has been inserted into each chunk. This metadata allows us to easily filter and retrieve information from this specific dataset later on, even if we use the same vector store index for other datasets. {% endhint %}

5. Process Your Data

• Once you are satisfied with the chunking process, it's time to process your data.

After processing your data, you retain the ability to refine individual chunks by deleting or adding content. This granular control offers several advantages:

• Enhanced Accuracy: Identify and rectify inaccuracies or inconsistencies present in the original data, ensuring the information used in your application is reliable.
• Improved Relevance: Refine chunk content to emphasize key information and remove irrelevant sections, thereby increasing the precision and effectiveness of your retrieval process.
• Query Optimization: Tailor chunks to better align with anticipated user queries, making them more targeted and improving the overall user experience.

6. Configure the Upsert Process

• With our data properly processed - loaded via a Document Loader and appropriately chunked -, we can now proceed to configure the upsert process.

The upsert process comprises three fundamental steps:

• Embedding Selection: We begin by choosing the appropriate embedding model to encode our dataset. This model will transform our data into a numerical vector representation.
• Data Store Selection: Next, we determine the Vector Store where our dataset will reside.
• Record Manager Selection (Optional): Finally, we have the option to implement a Record Manager. This component provides the functionalities for managing our dataset once it's stored within the Vector Store.

1. Select Embeddings

• Click on the "Select Embeddings" card and choose your preferred embedding model. In our case, we will select OpenAI as the embedding provider and use the "text-embedding-ada-002" model with 1536 dimensions.

2. Select Vector Store

• Click on the "Select Vector Store" card and choose your preferred Vector Store. In our case, as we need a production-ready option, we will select Upstash.

3. Select Record Manager

• For advanced dataset management within the Vector Store, you can optionally select and configure a Record Manager. Detailed instructions on how to set up and utilize this feature can be found in the dedicated guide.

7. Upsert Your Data to a Vector Store

• To begin the upsert process and transfer your data to the Vector Store, click the "Upsert" button.

• As illustrated in the image below, our data has been successfully upserted into the Upstash vector database. The data was divided into 85 chunks to optimize the upsertion process and ensure efficient storage and retrieval.

8. Test Your Dataset

• To quickly test the functionality of your dataset without navigating away from the Document Store, simply utilize the "Retrieval Query" button. This initiates a test query, allowing you to verify the accuracy and effectiveness of your data retrieval process.

• In our case, we see that when querying for information about kitchen flooring coverage in our insurance policy, we retrieve 4 relevant chunks from Upstash, our designated Vector Store. This retrieval is limited to 4 chunks as per the defined "top k" parameter, ensuring we receive the most pertinent information without unnecessary redundancy.

9. Test Your RAG

• Finally, our Retrieval-Augmented Generation (RAG) system is operational. It's noteworthy how the LLM effectively interprets the query and successfully leverages relevant information from the chunked data to construct a comprehensive response.

You can use the vector store that was configured earlier:

Or, use the Document Store (Vector):

10. API

There are also APIs support for creating, updating and deleting document store. Refer to Document Store API for more details. In this section, we are going to highlight the 2 of the most used APIs: upsert and refresh.

Upsert API

There are a few different scenarios for upserting process, and each have different outcomes.

Scenario 1: In the same document store, use an existing document loader configuration, upsert as new document loader.

{% hint style="success" %} docId represents the existing document loader ID. It is required in the request body for this scenario. {% endhint %}

{% tabs %} {% tab title="Python" %}

import requests
import json

DOC_STORE_ID = "your_doc_store_id"
DOC_LOADER_ID = "your_doc_loader_id"
API_URL = f"http://localhost:3000/api/v1/document-store/upsert/{DOC_STORE_ID}"
API_KEY = "your_api_key_here"

form_data = {
    "files": ('my-another-file.pdf', open('my-another-file.pdf', 'rb'))
}

body_data = {
    "docId": DOC_LOADER_ID
}

headers = {
    "Authorization": f"Bearer {BEARER_TOKEN}"
}

def query(form_data):
    response = requests.post(API_URL, files=form_data, data=body_data, headers=headers)
    print(response)
    return response.json()

output = query(form_data)
print(output)

{% endtab %}

{% tab title="Javascript" %}

const DOC_STORE_ID = "your_doc_store_id"
const DOC_LOADER_ID = "your_doc_loader_id"

let formData = new FormData();
formData.append("files", input.files[0]);
formData.append("docId", DOC_LOADER_ID)

async function query(formData) {
    const response = await fetch(
        `http://localhost:3000/api/v1/document-store/upsert/${DOC_STORE_ID}`,
        {
            method: "POST",
            headers: {
                "Authorization": "Bearer <your_api_key_here>"
            },
            body: formData
        }
    );
    const result = await response.json();
    return result;
}

query(formData).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Scenario 2: In the same document store, replace an existing document loader with new files.

{% hint style="success" %} docId and replaceExisting are both required in the request body for this scenario. {% endhint %}

{% tabs %} {% tab title="Python" %}

import requests
import json

DOC_STORE_ID = "your_doc_store_id"
DOC_LOADER_ID = "your_doc_loader_id"
API_URL = f"http://localhost:3000/api/v1/document-store/upsert/{DOC_STORE_ID}"
API_KEY = "your_api_key_here"

form_data = {
    "files": ('my-another-file.pdf', open('my-another-file.pdf', 'rb'))
}

body_data = {
    "docId": DOC_LOADER_ID,
    "replaceExisting": True
}

headers = {
    "Authorization": f"Bearer {BEARER_TOKEN}"
}

def query(form_data):
    response = requests.post(API_URL, files=form_data, data=body_data, headers=headers)
    print(response)
    return response.json()

output = query(form_data)
print(output)

{% endtab %}

{% tab title="Javascript" %}

const DOC_STORE_ID = "your_doc_store_id";
const DOC_LOADER_ID = "your_doc_loader_id";

let formData = new FormData();
formData.append("files", input.files[0]);
formData.append("docId", DOC_LOADER_ID);
formData.append("replaceExisting", true);

async function query(formData) {
    const response = await fetch(
        `http://localhost:3000/api/v1/document-store/upsert/${DOC_STORE_ID}`,
        {
            method: "POST",
            headers: {
                "Authorization": "Bearer <your_api_key_here>"
            },
            body: formData
        }
    );
    const result = await response.json();
    return result;
}

query(formData).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Scenario 3: In the same document store, upsert as new document loader from scratch.

{% hint style="success" %} loader, splitter, embedding, vectorStore are all required in the request body for this scenario. recordManager is optional. {% endhint %}

{% tabs %} {% tab title="Python" %}

import requests
import json

DOC_STORE_ID = "your_doc_store_id"
API_URL = f"http://localhost:3000/api/v1/document-store/upsert/{DOC_STORE_ID}"
API_KEY = "your_api_key_here"

form_data = {
    "files": ('my-another-file.pdf', open('my-another-file.pdf', 'rb'))
}

loader = {
    "name": "pdfFile",
    "config": {} # you can leave empty to use default config
}

splitter = {
    "name": "recursiveCharacterTextSplitter",
    "config": {
        "chunkSize": 1400,
        "chunkOverlap": 100
    }
}

embedding = {
    "name": "openAIEmbeddings",
    "config": {
        "modelName": "text-embedding-ada-002",
        "credential": <your_credential_id>
    }
}

vectorStore = {
    "name": "pinecone",
    "config": {
        "pineconeIndex": "exampleindex",
        "pineconeNamespace": "examplenamespace",
        "credential":  <your_credential_i
    }
}

body_data = {
    "docId": DOC_LOADER_ID,
    "loader": json.dumps(loader),
    "splitter": json.dumps(splitter),
    "embedding": json.dumps(embedding),
    "vectorStore": json.dumps(vectorStore)
}

headers = {
    "Authorization": f"Bearer {BEARER_TOKEN}"
}

def query(form_data):
    response = requests.post(API_URL, files=form_data, data=body_data, headers=headers)
    print(response)
    return response.json()

output = query(form_data)
print(output)

{% endtab %}

{% tab title="Javascript" %}

const DOC_STORE_ID = "your_doc_store_id";
const API_URL = `http://localhost:3000/api/v1/document-store/upsert/${DOC_STORE_ID}`;
const API_KEY = "your_api_key_here";

const formData = new FormData();
formData.append("files", new Blob([await (await fetch('my-another-file.pdf')).blob()]), "my-another-file.pdf");

const loader = {
    name: "pdfFile",
    config: {} // You can leave empty to use the default config
};

const splitter = {
    name: "recursiveCharacterTextSplitter",
    config: {
        chunkSize: 1400,
        chunkOverlap: 100
    }
};

const embedding = {
    name: "openAIEmbeddings",
    config: {
        modelName: "text-embedding-ada-002",
        credential: "your_credential_id"
    }
};

const vectorStore = {
    name: "pinecone",
    config: {
        pineconeIndex: "exampleindex",
        pineconeNamespace: "examplenamespace",
        credential: "your_credential_id"
    }
};

const bodyData = {
    docId: "DOC_LOADER_ID",
    loader: JSON.stringify(loader),
    splitter: JSON.stringify(splitter),
    embedding: JSON.stringify(embedding),
    vectorStore: JSON.stringify(vectorStore)
};

const headers = {
    "Authorization": `Bearer BEARER_TOKEN`
};

async function query() {
    try {
        const response = await fetch(API_URL, {
            method: "POST",
            headers: headers,
            body: formData
        });

        const result = await response.json();
        console.log(result);
        return result;
    } catch (error) {
        console.error("Error:", error);
    }
}

query();

{% endtab %} {% endtabs %}

{% hint style="danger" %} Creating from scratch is not recommended as it exposes your credential ID. The recommended way is to create a placeholder document store and configure the parameters on the UI. Then use the placeholder as the base for adding new document loader or creating new document store. {% endhint %}

Scenario 4: Create new document store for every upsert

{% hint style="success" %} createNewDocStore and docStore are both required in the request body for this scenario. {% endhint %}

{% tabs %} {% tab title="Python" %}

import requests
import json

DOC_STORE_ID = "your_doc_store_id"
DOC_LOADER_ID = "your_doc_loader_id"
API_URL = f"http://localhost:3000/api/v1/document-store/upsert/{DOC_STORE_ID}"
API_KEY = "your_api_key_here"

form_data = {
    "files": ('my-another-file.pdf', open('my-another-file.pdf', 'rb'))
}

body_data = {
    "docId": DOC_LOADER_ID,
    "createNewDocStore": True,
    "docStore": json.dumps({"name":"My NEW Doc Store"})
}

headers = {
    "Authorization": f"Bearer {BEARER_TOKEN}"
}

def query(form_data):
    response = requests.post(API_URL, files=form_data, data=body_data, headers=headers)
    print(response)
    return response.json()

output = query(form_data)
print(output)

{% endtab %}

{% tab title="Javascript" %}

const DOC_STORE_ID = "your_doc_store_id";
const DOC_LOADER_ID = "your_doc_loader_id";

let formData = new FormData();
formData.append("files", input.files[0]);
formData.append("docId", DOC_LOADER_ID);
formData.append("createNewDocStore", true);
formData.append("docStore", JSON.stringify({ "name": "My NEW Doc Store" }));

async function query(formData) {
    const response = await fetch(
        `http://localhost:3000/api/v1/document-store/upsert/${DOC_STORE_ID}`,
        {
            method: "POST",
            headers: {
                "Authorization": "Bearer <your_api_key_here>"
            },
            body: formData
        }
    );
    const result = await response.json();
    return result;
}

query(formData).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Q: Where to find Document Store ID and Document Loader ID?

A: You can find the respective IDs from the URL.

Q: Where can I find the available configs to override?

A: You can find the available configs from the View API button on each document loader:

For each upsert, there are 5 elements involved:

• loader
• splitter
• embedding
• vectorStore
• recordManager

You can override existing configuration with the config body of the element. For example, using the screenshot above, you can create a new document loader with a new url:

{% tabs %} {% tab title="Python" %}

import requests

API_URL = "http://localhost:3000/api/v1/document-store/upsert/<storeId>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()

output = query({
    "docId": <docLoaderId>,
    # override existing configuration
    "loader": {
        "config": {
            "url": "https://new-url.com"
        }
    }
})
print(output)

{% endtab %}

{% tab title="Javascript" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/document-store/upsert/<storeId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "docId": <docLoaderId>,
    // override existing configuration
    "loader": {
        "config": {
            "url": "https://new-url.com"
        }
    }
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

What if the loader has file upload? Yes, you guessed it right, we have to use form data as body!

Using the image below as an example, we can override the usage parameter of the PDF File Loader like so:

{% tabs %} {% tab title="Python" %}

import requests
import json

API_URL = "http://localhost:3000/api/v1/document-store/upsert/<storeId>"
API_KEY = "your_api_key_here"

form_data = {
    "files": ('my-another-file.pdf', open('my-another-file.pdf', 'rb'))
}

override_loader_config = {
    "config": {
        "usage": "perPage"
    }
}

body_data = {
    "docId": <docLoaderId>,
    "loader": json.dumps(override_loader_config) # Override existing configuration
}

headers = {
    "Authorization": f"Bearer {BEARER_TOKEN}"
}

def query(form_data):
    response = requests.post(API_URL, files=form_data, data=body_data, headers=headers)
    print(response)
    return response.json()

output = query(form_data)
print(output)

{% endtab %}

{% tab title="Javascript" %}

const DOC_STORE_ID = "your_doc_store_id";
const DOC_LOADER_ID = "your_doc_loader_id";

const overrideLoaderConfig = {
    "config": {
        "usage": "perPage"
    }
}

let formData = new FormData();
formData.append("files", input.files[0]);
formData.append("docId", DOC_LOADER_ID);
formData.append("loader", JSON.stringify(overrideLoaderConfig));

async function query(formData) {
    const response = await fetch(
        `http://localhost:3000/api/v1/document-store/upsert/${DOC_STORE_ID}`,
        {
            method: "POST",
            headers: {
                "Authorization": "Bearer <your_api_key_here>"
            },
            body: formData
        }
    )
    const result = await response.json();
    return result;
}

query(formData).then((response) => {
    console.log(response);
});e

{% endtab %} {% endtabs %}

Q: When to use Form Data vs JSON as the body of API request?

A: For Document Loaders that have File Upload functionality, such as PDF, DOCX, TXT, etc, body must be sent as Form Data.

{% hint style="warning" %} Make sure the sent file type is compatible with the expected file type from document loader.

For example, if a PDF File Loader is being used, you should only send .pdf files.

To avoid having separate loaders for different file types, we recommend to use File Loader {% endhint %}

{% tabs %} {% tab title="Python API" %}

import requests
import json

API_URL = "http://localhost:3000/api/v1/document-store/upsert/<storeId>"

# use form data to upload files
form_data = {
    "files": ('my-another-file.pdf', open('my-another-file.pdf', 'rb'))
}

body_data = {
    "docId": <docId>
}

def query(form_data):
    response = requests.post(API_URL, files=form_data, data=body_data)
    print(response)
    return response.json()

output = query(form_data)
print(output)

{% endtab %}

{% tab title="Javascript API" %}

// use FormData to upload files
let formData = new FormData();
formData.append("files", input.files[0]);
formData.append("docId", <docId>);

async function query(formData) {
    const response = await fetch(
        "http://localhost:3000/api/v1/document-store/upsert/<storeId>",
        {
            method: "POST",
            body: formData
        }
    );
    const result = await response.json();
    return result;
}

query(formData).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

For other Document Loaders nodes without Upload File functionality, the API body is in JSON format:

{% tabs %} {% tab title="Python API" %}

import requests

API_URL = "http://localhost:3000/api/v1/document-store/upsert/<storeId>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()

output = query({
    "docId": <docId>
})
print(output)

{% endtab %}

{% tab title="Javascript API" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/document-store/upsert/<storeId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "docId": <docId>
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Q: Can I add new metadata?

A: You can provide new metadata by passing the metadata inside the body request:

{
    "docId": <doc-id>,
    "metadata": {
        "source: "abc"
    }
}

Refresh API

Often times you might want to re-process every documents loaders within document store to fetch the latest data, and upsert to vector store, to keep everything in sync. This can be done via Refresh API:

{% tabs %} {% tab title="Python API" %}

import requests

API_URL = "http://localhost:3000/api/v1/document-store/refresh/<storeId>"

def query():
    response = requests.post(API_URL)
    return response.json()

output = query()
print(output)

{% endtab %}

{% tab title="Javascript API" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/document-store/refresh/<storeId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            }
        }
    );
    const result = await response.json();
    return result;
}

query().then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

You can also override existing configuration of specific document loader:

{% tabs %} {% tab title="Python API" %}

import requests

API_URL = "http://localhost:3000/api/v1/document-store/refresh/<storeId>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()

output = query(
{
    "items": [
        {
            "docId": <docId>,
            "splitter": {
                "name": "recursiveCharacterTextSplitter",
                "config": {
                    "chunkSize": 2000,
                    "chunkOverlap": 100
                }
            }
        }
    ]
}
)
print(output)

{% endtab %}

{% tab title="Javascript API" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/document-store/refresh/<storeId>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "items": [
        {
            "docId": <docId>,
            "splitter": {
                "name": "recursiveCharacterTextSplitter",
                "config": {
                    "chunkSize": 2000,
                    "chunkOverlap": 100
                }
            }
        }
    ]
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

11. Summary

We started by creating a Document Store to organize the LibertyGuard Deluxe Homeowners Policy data. This data was then prepared by uploading, chunking, processing, and upserting it, making it ready for our RAG system.

Advantages of the Document Store:

Document Stores offer several benefits for managing and preparing data for Retrieval Augmented Generation (RAG) systems:

• Organization and Management: They provide a central location for storing, managing, and preparing your data.
• Data Quality: The chunking process helps structure data for accurate retrieval and analysis.
• Flexibility: Document Stores allow for refining and adjusting data as needed, improving the accuracy and relevance of your RAG system.

12. Video Tutorials

RAG Like a Boss - AiMicromind Document Store Tutorial (coming soon)

description: Learn how to customize and embed our chat widget

Embed

You can easily add the chat widget to your website. Just copy the provided widget script and paste it anywhere between the <body> and </body> tags of your HTML file.

Widget Setup

The following video shows how to inject the widget script into any webpage.

{% embed url="https://github.com/operativestech/AiMicroMind_Platform_2025/assets/26460777/c128829a-2d08-4d60-b821-1e41a9e677d0" %}

Using Specific Version

You can specify the version of aimicromind -embed's web.js to use. For full list of versions: [https://www.npmjs.com/package/aimicromind -embed](https://www.npmjs.com/package/aimicromind -embed)

<script type="module">
  import Chatbot from 'https://cdn.jsdelivr.net/npm/aimicromind-embed@<some-version>/dist/web.js';
  Chatbot.init({
    chatflowid: 'your-chatflowid-here',
    apiHost: 'your-apihost-here',
  })
</script>

{% hint style="warning" %} In aimicromindv2.1.0, we have modified the way streaming works. If your aimicromindversion is lower than that, you might find your embedded chatbot not able to receive messages.

You can either update aimicromind to v2.1.0 and above

Or, if for some reason you prefer not to update AiMicromind , you can specify the latest v1.x.x version of AiMicromind -Embed. Last maintained web.js version is v1.3.14.

For instance:

https://cdn.jsdelivr.net/npm/aimicromind-embed@1.3.14/dist/web.js {% endhint %}

Chatflow Config

You can pass chatflowConfig JSON object to override existing configuration. This is the same as #override-config in API.

<script type="module">
  import Chatbot from 'https://cdn.jsdelivr.net/npm/aimicromind-embed/dist/web.js';
  Chatbot.init({
    chatflowid: 'your-chatflowid-here',
    apiHost: 'your-apihost-here',
    chatflowConfig: {
      "sessionId": "123",
      "returnSourceDocuments": true
    }
  })
</script>

Observer Config

This allows you to execute code in parent based upon signal observations within the chatbot.

<script type="module">
  import Chatbot from 'https://cdn.jsdelivr.net/npm/aimicromind-embed/dist/web.js';
  Chatbot.init({
    chatflowid: 'your-chatflowid-here',
    apiHost: 'your-apihost-here',
    observersConfig: {
      // User input has changed
      observeUserInput: (userInput) => {
        console.log({ userInput });
      },
      // The bot message stack has changed
      observeMessages: (messages) => {
        console.log({ messages });
      },
      // The bot loading signal changed
      observeLoading: (loading) => {
        console.log({ loading });
      },
    },
  })
</script>

Theme

You can change the full appearance of the embedded chatbot and enable functionalities like tooltips, disclaimers, custom welcome messages, and more using the theme property. This allows you to deeply customize the look and feel of the widget, including:

• Button: Position, size, color, icon, drag-and-drop behavior, and automatic opening.
• Tooltip: Visibility, message text, background color, text color, and font size.
• Disclaimer: Title, message, colors for text, buttons, and background, including a blurred overlay option.
• Chat Window: Title, agent/user message display, welcome/error messages, background color/image, dimensions, font size, starter prompts, HTML rendering, message styling (colors, avatars), text input behavior (placeholder, colors, character limits, sounds), feedback options, date/time display, and footer customization.
• Custom CSS: Directly inject CSS code for even finer control over the appearance, overriding default styles as needed (see the instructions guide below)

<script type="module">
  import Chatbot from 'https://cdn.jsdelivr.net/npm/aimicromind-embed/dist/web.js';
  Chatbot.init({
    chatflowid: 'your-chatflowid-here',
    apiHost: 'your-apihost-here',
    theme: {
      button: {
        backgroundColor: '#3B81F6',
        right: 20,
        bottom: 20,
        size: 48, // small | medium | large | number
        dragAndDrop: true,
        iconColor: 'white',
        customIconSrc: 'https://raw.githubusercontent.com/walkxcode/dashboard-icons/main/svg/google-messages.svg',
        autoWindowOpen: {
          autoOpen: true, //parameter to control automatic window opening
          openDelay: 2, // Optional parameter for delay time in seconds
          autoOpenOnMobile: false, //parameter to control automatic window opening in mobile
        },
      },
      tooltip: {
        showTooltip: true,
        tooltipMessage: 'Hi There 👋!',
        tooltipBackgroundColor: 'black',
        tooltipTextColor: 'white',
        tooltipFontSize: 16,
      },
      disclaimer: {
        title: 'Disclaimer',
        message: 'By using this chatbot, you agree to the <a target="_blank" href="https://chat.aimicromind.com/terms">Terms & Condition</a>',
        textColor: 'black',
        buttonColor: '#3b82f6',
        buttonText: 'Start Chatting',
        buttonTextColor: 'white',
        blurredBackgroundColor: 'rgba(0, 0, 0, 0.4)', //The color of the blurred background that overlays the chat interface
        backgroundColor: 'white',
      },
      customCSS: ``, // Add custom CSS styles. Use !important to override default styles
      chatWindow: {
        showTitle: true,
        showAgentMessages: true,
        title: 'aimicromindBot',
        titleAvatarSrc: 'https://raw.githubusercontent.com/walkxcode/dashboard-icons/main/svg/google-messages.svg',
        welcomeMessage: 'Hello! This is custom welcome message',
        errorMessage: 'This is a custom error message',
        backgroundColor: '#ffffff',
        backgroundImage: 'enter image path or link', // If set, this will overlap the background color of the chat window.
        height: 700,
        width: 400,
        fontSize: 16,
        starterPrompts: ['What is a bot?', 'Who are you?'], // It overrides the starter prompts set by the chat flow passed
        starterPromptFontSize: 15,
        clearChatOnReload: false, // If set to true, the chat will be cleared when the page reloads
        sourceDocsTitle: 'Sources:',
        renderHTML: true,
        botMessage: {
          backgroundColor: '#f7f8ff',
          textColor: '#303235',
          showAvatar: true,
          avatarSrc: 'https://raw.githubusercontent.com/zahidkhawaja/langchain-chat-nextjs/main/public/parroticon.png',
        },
        userMessage: {
          backgroundColor: '#3B81F6',
          textColor: '#ffffff',
          showAvatar: true,
          avatarSrc: 'https://raw.githubusercontent.com/zahidkhawaja/langchain-chat-nextjs/main/public/usericon.png',
        },
        textInput: {
          placeholder: 'Type your question',
          backgroundColor: '#ffffff',
          textColor: '#303235',
          sendButtonColor: '#3B81F6',
          maxChars: 50,
          maxCharsWarningMessage: 'You exceeded the characters limit. Please input less than 50 characters.',
          autoFocus: true, // If not used, autofocus is disabled on mobile and enabled on desktop. true enables it on both, false disables it on both.
          sendMessageSound: true,
          // sendSoundLocation: "send_message.mp3", // If this is not used, the default sound effect will be played if sendSoundMessage is true.
          receiveMessageSound: true,
          // receiveSoundLocation: "receive_message.mp3", // If this is not used, the default sound effect will be played if receiveSoundMessage is true.
        },
        feedback: {
          color: '#303235',
        },
        dateTimeToggle: {
          date: true,
          time: true,
        },
        footer: {
          textColor: '#303235',
          text: 'Powered by',
          company: 'AiMicromind ',
          companyLink: 'https://chat.aimicromind.com/',
        },
      },
    },
  });
</script>

Note: See full configuration list

Custom Code Modification

To modify the full source code of embedded chat widget, follow these steps:

1. Fork the AiMicromindChat Embed repository
2. Run yarn install to install the necessary dependencies
3. Then you can make any code changes
4. Run yarn build to pick up the changes
5. Push changes to the forked repository
6. You can then use your custom web.js as embedded chat like so:

Replace username to your Github username, and forked-repo to your forked repo.

<script type="module">
      import Chatbot from "https://cdn.jsdelivr.net/gh/username/forked-repo/dist/web.js"
      Chatbot.init({
          chatflowid: "your-chatflowid-here",
          apiHost: "your-apihost-here",
      })
</script>

<script type="module">
      import Chatbot from "https://cdn.jsdelivr.net/gh/HenryHengZJ/AiMicromind ChatEmbed-Test/dist/web.js"
      Chatbot.init({
          chatflowid: "your-chatflowid-here",
          apiHost: "your-apihost-here",
      })
</script>

{% hint style="info" %} An alternative to jsdelivr is unpkg. Here is an example:

https://unpkg.com/aimicromind-embed/dist/web.js

{% endhint %}

Custom CSS Modification

You can now directly add custom CSS to style your embedded chat widget, eliminating the need for custom web.js files (requires v2.0.8 or later). This allows you to:

• Give each embedded chatbot a unique look and feel
• Use the official web.js—no more custom builds or hosting are needed for styling
• Update styles instantly

Here's how to use it:

<script src="https://cdn.jsdelivr.net/gh/AiMicromind/AiMicromindChatEmbed@main/dist/web.js"></script>
<script>
  Chatbot.init({
    chatflowid: "your-chatflowid-here",
    apiHost: "your-apihost-here",
    theme: {
      // ... other theme settings
      customCSS: `
        /* Your custom CSS here */
        /* Use !important to override default styles */
      `,
    }
  });
</script>

CORS

When using embedded chat widget, there's chance that you might face CORS issue like:

{% hint style="danger" %} Access to fetch at 'https://<your-aimicromind.com>/api/v1/prediction/' from origin 'https://<your-aimicromind.com>' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. {% endhint %}

To fix it, specify the following environment variables:

CORS_ORIGINS=*
IFRAME_ORIGINS=*

For example, if you are using npx aimicromind start

npx aimicromind start --CORS_ORIGINS=* --IFRAME_ORIGINS=*

If using Docker, place the env variables inside AiMicromind /docker/.env

If using local Git clone, place the env variables inside AiMicromind /packages/server/.env

Video Tutorials (Coming soon)

Monitoring

AiMicromind has native support for Prometheus with Grafana and OpenTelemetry. However, only high-level metrics such as API requests, counts of flows/predictions are tracked. Refer here for the lists of counter metrics. For details node by node observability, we recommend using Analytic.

Prometheus

Prometheus is an open-source monitoring and alerting solution.

Before setting up Prometheus, configure the following env variables in AiMicromind :

ENABLE_METRICS=true
METRICS_PROVIDER=prometheus
METRICS_INCLUDE_NODE_METRICS=true

After Prometheus is installed, run it using a configuration file. AiMicromind provides a default configuration file that can be found here.

Remember to have aimicromind instance also running. You can open browser and navigate to port 9090. From the dashboard, you should be able to see the metric endpoint - /api/v1/metrics is now live.

By default, /api/v1/metrics is available for Prometheus to pull the metrics from.

Grafana

Prometheus collects rich metrics and provides a powerful querying language; Grafana transforms metrics into meaningful visualizations.

Grafana can be installed in various ways. Refer to the guide.

Grafana by default will expose port 9091:

On the left side bar, click Add new connection, and select Prometheus:

Since our Prometheus is serving at port 9090:

Scroll to the bottom and test the connection:

Take note of the data source ID shown in the toolbar, we'll need this for creating dashboards:

Now that connection is added successfully, we can start adding dashboard. From the left side bar, click Dashboards, and Create Dashboard.

AiMicromind provides 2 template dashboards:

• grafana.dashboard.app.json.txt: API metrics such as number of chatflows/agentflows, predictions count, tools, assistant, upserted vectors, etc.
• grafana.dashboard.server.json.txt: metrics of the aimicromindnode.js instance such as heap, CPU, RAM usage

If you are using templates above, find and replace all occurence of cds4j1ybfuhogb with the data source ID you created and saved earlier.

You can also choose to import first then edit the JSON later:

Now, try to perform some actions on the AiMicromind , you should be able to see the metrics displayed:

OpenTelemetry

OpenTelemetry is an open source framework for creating and managing telemetry data. To enable OTel, configure the following env variables in AiMicromind :

ENABLE_METRICS=true
METRICS_PROVIDER=open_telemetry
METRICS_INCLUDE_NODE_METRICS=true
METRICS_OPEN_TELEMETRY_METRIC_ENDPOINT=http://localhost:4318/v1/metrics
METRICS_OPEN_TELEMETRY_PROTOCOL=http # http | grpc | proto (default is http)
METRICS_OPEN_TELEMETRY_DEBUG=true

Next, we need OpenTelemetry Collector to receive, process and export telemetry data. AiMicromind provides a [docker compose file](https://github.com/AiMicromind AI/AiMicromind /blob/main/metrics/otel/compose.yaml) which can be used to start the collector container.

cd AiMicromind  
cd metrics && cd otel
docker compose up -d

The collector will be using the otel.config.yml file under the same directory for configurations. Currently only Datadog and Prometheus are supported, refer to the Open Telemetry documentation to configure different APM tools such as Zipkin, Jeager, New Relic, Splunk and others.

Make sure to replace with the necessary API key for the exporters within the yml file.

description: Learn how aimicromind streaming works

Streaming

If streaming is set when making prediction, tokens will be sent as data-only server-sent events as they become available.

Using Python/TS Library

aimicromindprovides 2 libraries:

• Python: pip install aimicromind
• Typescript: npm install aimicromind -sdk

{% tabs %} {% tab title="Python" %}

from aimicromind import AiMicromind   , PredictionData

def test_streaming():
    client = AiMicromind()

    # Test streaming prediction
    completion = client.create_prediction(
        PredictionData(
            chatflowId="<chatflow-id>",
            question="Tell me a joke!",
            streaming=True
        )
    )

    # Process and print each streamed chunk
    print("Streaming response:")
    for chunk in completion:
        # {event: "token", data: "hello"}
        print(chunk)


if __name__ == "__main__":
    test_streaming()

{% endtab %}

{% tab title="Typescript" %}

import { AiMicromind   Client } from 'aimicromind   -sdk'

async function test_streaming() {
  const client = new AiMicromindClient({ baseUrl: 'http://localhost:3000' });

  try {
    // For streaming prediction
    const prediction = await client.createPrediction({
      chatflowId: '<chatflow-id>',
      question: 'What is the capital of France?',
      streaming: true,
    });

    for await (const chunk of prediction) {
        // {event: "token", data: "hello"}
        console.log(chunk);
    }
    
  } catch (error) {
    console.error('Error:', error);
  }
}

// Run streaming test
test_streaming()

{% endtab %}

{% tab title="cURL" %}

curl https://localhost:3000/api/v1/predictions/{chatflow-id} \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Hello world!",
    "streaming": true
  }'

{% endtab %} {% endtabs %}

event: token
data: Once upon a time...

A prediction's event stream consists of the following event types:

Streamlit App

https://github.com/HenryHengZJ/aimicromind-streamlit

description: Learn how aimicromind collects anonymous app usage information

Telemetry

AiMicromind open source repository has a built-in telemetry that collects anonymous usage information. This helps us to better understand usage of AiMicromind , enabling us to prioritize our efforts towards developing new features and resolving issues, and enhancing the performance and stability of AiMicromind .

{% hint style="warning" %} Important - We never collect any confidential information about the node input/output, messages, or any sort of credentials and variables. Only events are being sent. {% endhint %}

You can verify these claims by finding all locations telemetry.sendTelemetry is called from the source code.

{
    "version": <app-version>,
    "chatlowId": <chatflow-id>,
    "flowGraph": {
        "nodes": [<nodeid-1>, <nodeid-2>],
        "edges": [
            {
                "source": <nodeid-1>,
                "target": <nodeid-2>
            }
        ]
    }
}

{
    "version": <app-version>,
    "toolId": <tool-id>,
    "toolName": <tool-name>
}

{
    "version": <app-version>,
    "assistantId": <assistant-id>
}

{
    "version": <app-version>,
    "chatlowId": <chatflow-id>,
    "type": "INTERNAL", // EXTERNAL
    "flowGraph": {
        "nodes": [<nodeid-1>, <nodeid-2>],
        "edges": [
            {
                "source": <nodeid-1>,
                "target": <nodeid-2>
            }
        ]
    },
    "stopNodeId": <nodeid-1>
}

{
    "version": <app-version>,
    "chatlowId": <chatflow-id>,
    "chatId": <chat-id>,
    "type": "INTERNAL", // EXTERNAL
    "flowGraph": {
        "nodes": [<nodeid-1>, <nodeid-2>],
        "edges": [
            {
                "source": <nodeid-1>,
                "target": <nodeid-2>
            }
        ]
    }
}

Disable Telemetry

Users can disable telemetry by setting DISABLE_AiMicromind_TELEMETRY to true in .env file.

description: Learn how to use upload images, audio, and other files

Uploads

AiMicromind lets you upload images, audio, and other files from the chat. In this section, you'll learn how to enable and use these features.

Image

Certain chat models allow you to input images. Always refer to the official documentation of the LLM to confirm if the model supports image input.

• ChatOpenAI
• AzureChatOpenAI
• ChatAnthropic
• AWSChatBedrock
• ChatGoogleGenerativeAI
• ChatOllama
• Google Vertex AI

{% hint style="warning" %} Image processing only works with certain chains/agents in Chatflow.

LLMChain, Conversation Chain, ReAct Agent, Conversational Agent, Tool Agent {% endhint %}

If you enable Allow Image Upload, you can upload images from the chat interface.

To upload images with the API:

{% tabs %} {% tab title="Python" %}

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatflowid>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "question": "Can you describe the image?",
    "uploads": [
        {
            "data": "data:image/png;base64,iVBORw0KGgdM2uN0", # base64 string or url
            "type": "file", # file | url
            "name": "AiMicromind     .png",
            "mime": "image/png"
        }
    ]
})

{% endtab %}

{% tab title="Javascript" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatflowid>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "question": "Can you describe the image?",
    "uploads": [
        {
            "data": "data:image/png;base64,iVBORw0KGgdM2uN0", //base64 string or url
            "type": "file", // file | url
            "name": "AiMicromind     .png",
            "mime": "image/png"
        }
    ]
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Audio

In the Chatflow Configuration, you can select a speech-to-text module. Supported integrations include:

• OpenAI
• AssemblyAI
• LocalAI

When this is enabled, users can speak directly into the microphone. Their speech is be transcribed into text.

To upload audio with the API:

{% tabs %} {% tab title="Python" %}

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatflowid>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "uploads": [
        {
            "data": "data:audio/webm;codecs=opus;base64,GkXf", # base64 string
            "type": "audio",
            "name": "audio.wav",
            "mime": "audio/webm"
        }
    ]
})

{% endtab %}

{% tab title="Javascript" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatflowid>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "uploads": [
        {
            "data": "data:audio/webm;codecs=opus;base64,GkXf", // base64 string
            "type": "audio",
            "name": "audio.wav",
            "mime": "audio/webm"
        }
    ]
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Files

You can upload files in two ways:

• Retrieval augmented generation (RAG) file uploads
• Full file uploads

When both options are on, full file uploads take precedence.

RAG File Uploads

You can upsert uploaded files on the fly to the vector store. To enable file uploads, make sure you meet these prerequisites:

• You must include a vector store that supports file uploads in the chatflow.
  • Pinecone
  • Milvus
  • Postgres
  • Qdrant
  • Upstash
• If you have multiple vector stores in a chatflow, you can only turn on file upload for one vector store at a time.
• You must connect at least one document loader node to the vector store's document input.
• Supported document loaders:
  • CSV File
  • Docx File
  • Json File
  • Json Lines File
  • PDF File
  • Text File
  • Unstructured File

You can upload one or more files in the chat:

Here's how it works:

1. The metadata for uploaded files is updated with the chatId.
2. This associates the file with the chatId.
3. When querying, an OR filter applies:

• Metadata contains aimicromind_chatId, and the value is the current chat session ID
• Metadata does not contain aimicromind_chatId

An example of a vector embedding upserted on Pinecone:

To do this with the API, follow these two steps:

1. Use the Vector Upsert API with formData and chatId:

{% tabs %} {% tab title="Python" %}

import requests

API_URL = "http://localhost:3000/api/v1/vector/upsert/<chatflowid>"

# Use form data to upload files
form_data = {
    "files": ("state_of_the_union.txt", open("state_of_the_union.txt", "rb"))
}

body_data = {
    "chatId": "some-session-id"
}

def query(form_data):
    response = requests.post(API_URL, files=form_data, data=body_data)
    print(response)
    return response.json()

output = query(form_data)
print(output)

{% endtab %}

{% tab title="Javascript" %}

// Use FormData to upload files
let formData = new FormData();
formData.append("files", input.files[0]);
formData.append("chatId", "some-session-id");

async function query(formData) {
    const response = await fetch(
        "http://localhost:3000/api/v1/vector/upsert/<chatflowid>",
        {
            method: "POST",
            body: formData
        }
    );
    const result = await response.json();
    return result;
}

query(formData).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

2. Use the Prediction API with uploads and the chatId from step 1:

{% tabs %} {% tab title="Python" %}

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatflowid>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "question": "What is the speech about?",
    "chatId": "same-session-id-from-step-1",
    "uploads": [
        {
            "data": "data:text/plain;base64,TWFkYWwcy4=",
            "type": "file:rag",
            "name": "state_of_the_union.txt",
            "mime": "text/plain"
        }
    ]
})

{% endtab %}

{% tab title="Javascript" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatflowid>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "question": "What is the speech about?",
    "chatId": "same-session-id-from-step-1",
    "uploads": [
        {
            "data": "data:text/plain;base64,TWFkYWwcy4=",
            "type": "file:rag",
            "name": "state_of_the_union.txt",
            "mime": "text/plain"
        }
    ]
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

Full File Uploads

With RAG file uploads, you can't work with structured data like spreadsheets or tables, and you can't perform full summarization due to lack of full context. In some cases, you might want to include all the file content directly in the prompt for an LLM, especially with models like Gemini and Claude that have longer context windows. This research paper is one of many that compare RAG with longer context windows.

To enable full file uploads, go to Chatflow Configuration, open the File Upload tab, and click the switch:

You can see the File Attachment button in the chat, where you can upload one or more files. Under the hood, the File Loader processes each file and converts it into text.

Note that if your chatflow uses a Chat Prompt Template node, an input must be created from Format Prompt Values to pass the file data. The specified input name (e.g. {file}) should be included in the Human Message field.

To upload files with the API:

{% tabs %} {% tab title="Python" %}

import requests
API_URL = "http://localhost:3000/api/v1/prediction/<chatflowid>"

def query(payload):
    response = requests.post(API_URL, json=payload)
    return response.json()
    
output = query({
    "question": "What is the data about?",
    "chatId": "some-session-id",
    "uploads": [
        {
            "data": "data:text/plain;base64,TWFkYWwcy4=",
            "type": "file:full",
            "name": "state_of_the_union.txt",
            "mime": "text/plain"
        }
    ]
})

{% endtab %}

{% tab title="Javascript" %}

async function query(data) {
    const response = await fetch(
        "http://localhost:3000/api/v1/prediction/<chatflowid>",
        {
            method: "POST",
            headers: {
                "Content-Type": "application/json"
            },
            body: JSON.stringify(data)
        }
    );
    const result = await response.json();
    return result;
}

query({
    "question": "What is the data about?",
    "chatId": "some-session-id",
    "uploads": [
        {
            "data": "data:text/plain;base64,TWFkYWwcy4=",
            "type": "file:full",
            "name": "state_of_the_union.txt",
            "mime": "text/plain"
        }
    ]
}).then((response) => {
    console.log(response);
});

{% endtab %} {% endtabs %}

As you can see in the examples, uploads require a base64 string. To get a base64 string for a file, use the Create Attachments API.

Difference between Full & RAG Uploads

Both Full and RAG (Retrieval-Augmented Generation) file uploads serve different purposes.

• Full File Upload: This method parses the entire file into a string and sends it to the LLM (Large Language Model). It's beneficial for summarizing the document or extracting key information. However, with very large files, the model might produce inaccurate results or "hallucinations" due to token limitations.
• RAG File Upload: Recommended if you aim to reduce token costs by not sending the entire text to the LLM. This approach is suitable for Q&A tasks on the documents but isn't ideal for summarization since it lacks the full document context. This approach might takes longer time because of the upsert process.

description: Learn how to use variables in AiMicromind

Variables

AiMicromind allow users to create variables that can be used in the nodes. Variables can be Static or Runtime.

Static

Static variable will be saved with the value specified, and retrieved as it is.

Runtime

Value of the variable will be fetched from .env file using process.env

Override or setting variable through API

In order to override variable value, user must explicitly enable it from Chatflow Configuration -> Security tab:

If there is an existing variable created, variable value provided in the API will override the existing value.

{
    "question": "hello",
    "overrideConfig": {
        "vars": {
            "var": "some-override-value"
        }
    }
}

Using Variables

Variables can be used by the nodes in AiMicromind. For instance, a variable named character is created:

We can then use this variable as $vars.<variable-name> in the Function of the following nodes:

• Custom Tool
• Custom Function
• Custom Loader
• If Else

Besides, user can also use the variable in text input of any node with the following format:

{{$vars.<variable-name>}}

For example, in Agent System Message:

In Prompt Template:

Resources

• Pass Variables to Function

Workspaces

{% hint style="info" %} Workspaces is only available for Enterprise for now. Coming soon to Cloud Pro plan {% endhint %}

Upon your initial login, a default workspace will be automatically generated for you. Workspaces serve to partition resources among various teams or business units. Inside each workspace, Role-Based Access Control (RBAC) is used to manage permissions and access, ensuring users have access only to the resources and settings required for their role.

Setting up Admin Account

JWT_AUTH_TOKEN_SECRET
JWT_REFRESH_TOKEN_SECRET
JWT_ISSUER
JWT_AUDIENCE
JWT_TOKEN_EXPIRY_IN_MINUTES
JWT_REFRESH_TOKEN_EXPIRY_IN_MINUTES
PASSWORD_RESET_TOKEN_EXPIRY_IN_MINS
PASSWORD_SALT_HASH_ROUNDS
TOKEN_HASH_SECRET

By default, new installation of aimicromind will require an admin setup, similar to how you have to setup a root user for your database initially.

After setting up, user will be brought to aimicromind dashboard. From the left side bar, you will see User & Workspace Management section. A default workspace was automatically created.

Creating Workspace

To create a new Workspace, click Add New:

You will see yourself added as the Organization Admin in the workspace you created.

To invite new users to the workspace, you need to create a Role first.

Creating Role

Navigate to Roles in the left side bar, and click Add Role:

User can specify granular control of permissions for each resources. The only exceptions are the resources in User & Workspace Management (Roles, Users, Workspaces, Login Activity). These are only available for Account Admin for now.

Here, we create an editor role which has access to everything. And another role with view-only permissions.

Invite User

INVITE_TOKEN_EXPIRY_IN_HOURS
SMTP_HOST
SMTP_PORT
SMTP_USER
SMTP_PASSWORD

Navigate to Users in left side bar, you will see yourself as the account admin. This is indicated by the person icon with a star:

Click Invite User, and enter email to be invited, the workspace to be assigned, and the role as well.

Click Send Invite. The invited email will receive an invitation:

Upon clicking the invitation link, invited user will be brought to a Sign Up page.

After signed up and logged in as invited user, you will be in the workspace assigned, and there will be no User & Workspace Management section:

If you are invited into multiple workspaces, you can switch to different workspaces from the top right dropdown button. Here we are assigned to Workspace 2 with view only permission. You can notice the Add New button for Chatflow is no longer visible. This ensure user can only view, not create, update nor delete. The same RBAC rules apply for API as well.

Now, back to Account Admin, you will be able to see the users invited, their status, roles, and active workspace:

Account admin can also modify the settings for other users:

Login Activity

Admin will be able to see every login and logout from all users:

Creating item in Workspace

Every items created in a workspace, are isolated from another workspace. Workspaces are a way to logically group users and resources within an organization, ensuring separate trust boundaries for resource management and access control. It is recommended to create distinct workspaces for each team.

Here, we create a Chatflow named Chatflow1 in Workspace1:

When we switch to Workspace2, Chatflow1 will not be visible. This applies to every resources such as Agentflows, Tools, Assistants, etc.

The diagram below illustrates the relationship between organizations, workspaces, and the various resources associated with and contained within a workspace.

Sharing Credential

You can share credential to other workspaces. This allow users to reuse same set of credentials in different workspaces.

After creating a credential, Account Admin or user with Share Credential permission from the RBAC will be able to click Share:

User can select the workspaces to share the credential with:

Now, switch to the workspace where the credential was shared, you will see the Shared Credential. User is not able to edit shared credential.

Deleting a Workspace

Currently only Account Admin can delete workspaces. By default, you are not able to delete a workspace if there are still users within that workspace.

You will need to unlink all of the invited users first. This allow flexibility in case you just want to remove certain users from a workspace. Note that Organization Owner who created the workspace is not able to be unlinked from a workspace.

After unlinking invited users, and the only user left within the workspace is the Organization Owner, delete button is now clickable:

Deleting a workspace is an irreversible action and will cascade delete all items within that workspace. You will see a warning box:

After deleting a workspace, user will fallback to the Default workspace. Default workspace that was automatically created at the start is not able to be deleted.

Evaluations

{% hint style="info" %} Evaluations are only available for Cloud and Enterprise plan {% endhint %}

Evaluations help you monitor and understand the performance of your Chatflow/Agentflow application. On the high level, an evaluation is a process that takes a set of inputs and corresponding outputs from your Chatflow/Agentflow, and generates scores. These scores can be derived by comparing outputs to reference results, such as through string matching, numeric comparison, or even leveraging an LLM as a judge. These evaluations are conducted using Datasets and Evaluators.

Datasets

Datasets are the inputs that will be used to run your Chatflow/Agentflow, along with the corresponding outputs for comparison. User can add the input and anticipated output manually, or upload a CSV file with 2 columns: Input and Output.

Evaluators

Evaluators are like unit tests. During an evaluation, the inputs from Datasets are ran on the selected flows and the outputs are evaluated using selected evaluators. There are 3 types of evaluators:

• Text Based: string based checking:
  • Contains Any
  • Contains All
  • Does Not Contains Any
  • Does Not Contains All
  • Starts With
  • Does Not Starts With

• Numeric Based: numbers type checking:
  • Total Tokens
  • Prompt Tokens
  • Completion Tokens
  • API Latency
  • LLM Latency
  • Chatflow Latency
  • Agentflow Latency (coming)
  • Output Characters Length

• LLM Based: using another LLM to grade the output
  • Hallucination
  • Correctness

Evaluations

Now that we have Datasets and Evaluators prepared, we can start running an evaluation.

1.) Select dataset and chatflow to evaluate. You can select multiple datasets and chatflows. Using the example below, every inputs from Dataset1 will be ran against 2 chatflows. Since Dataset1 has 2 inputs, a total of 4 outputs will be produced and evaluated.

2.) Select the evaluators. Only string based and numeric based evaluators are available to be selected at this stage.

3.) (Optional) Select LLM Based evaluator. Start Evaluation:

4.) Wait for evaluation to be completed:

5.) After evaluation is completed, click the graph icon at the right side to view the details:

The 3 charts above show the summary of the evaluation:

• Pass/fail rate
• Average prompt and completion tokens used
• Average latency of the request

Table below the charts shows the details of each execution.

Re-run evaluation

When the flows used on evaluation have been updated/modified, a warning message will be shown:

You can re-run the same evaluation using the Re-Run Evaluation button at the top right corner. You will be able to see the different versions:

You can also view and compare the results from different versions:

Video Tutorial (Coming soon)

description: Learn how to set up and run aimicromind instances

Configuration

This section will guide you through various configuration options to customize your aimicromind instances for development, testing, and production environments.

We'll also provide in-depth guides for deploying aimicromind on different Platform as a Service (PaaS) options, ensuring a smooth and successful deployment.

Guides

• Auth
• Databases
• Deployment
• Environment Variables
• Rate Limit

description: Learn how to secure your aimicromindInstances

Auth

This section guides you through configuring security with AiMicromind, focusing on authentication mechanisms at the application and chatflow levels.

By implementing robust authentication, you can protect your aimicromind instances and ensure only authorized users can access and interact with your chatflows.

Supported Methods

• App level
• Chatflow level

description: Learn how to set up app-level access control for your aimicromindinstances

App Level

App level authorization protects your aimicromindinstance by username and password. This protects your apps from being accessible by anyone when deployed online.

How to Set Username & Password

Npm

1. Install AiMicromind

npm install -g aimicromind     

2. Start aimicromindwith username & password

npx aimicromind start --AIMICROMIND_USERNAME=user --AIMICROMIND_PASSWORD=1234

3. Open http://localhost:3000

Docker

1. Navigate to docker folder

cd docker

2. Create .env file and specify the PORT, AIMICROMIND _USERNAME, and AIMICROMIND _PASSWORD

PORT=3000
AIMICROMIND_USERNAME=user
AIMICROMIND_PASSWORD=1234

3. Pass AIMICROMIND _USERNAME and AIMICROMIND _PASSWORD to the docker-compose.yml file:

environment:
    - PORT=${PORT}
    - AIMICROMIND_USERNAME=${AIMICROMIND_USERNAME}
    - AIMICROMIND_PASSWORD=${AIMICROMIND_PASSWORD}

4. docker compose up -d
5. Open http://localhost:3000
6. You can bring the containers down by docker compose stop

Git clone

To enable app level authentication, add AIMICROMIND _USERNAME and AIMICROMIND _PASSWORD to the .env file in packages/server:

AIMICROMIND_USERNAME=user
AIMICROMIND_PASSWORD=1234

description: Learn how to set up chatflow-level access control for your aimicromindinstances

Chatflow Level

After you have a chatflow / agentflow constructed, by default, your flow is available to public. Anyone that has access to the Chatflow ID is able to run prediction through Embed or API.

In cases where you might want to allow certain people to be able to access and interact with it, you can do so by assigning an API key for that specific chatflow.

API Key

In dashboard, navigate to API Keys section, and you should be able to see a DefaultKey created. You can also add or delete any keys.

Chatflow

Navigate to the chatflow, and now you can select the API Key you want to use to protect the chatflow.

After assigning an API key, one can only access the chatflow API when the Authorization header is provided with the correct API key specified during a HTTP call.

"Authorization": "Bearer <your-api-key>"

An example of calling the API using POSTMAN

You can specify the location where the api keys are stored by specifying APIKEY_PATH env variables. Read more environment-variables.md

description: Learn how to connect your aimicromind instance to a database

Databases

Setup

aimicromind supports 4 database types:

• SQLite
• MySQL
• PostgreSQL
• MariaDB

SQLite (Default)

SQLite will be the default database. These databases can be configured with following env variables:

DATABASE_TYPE=sqlite
DATABASE_PATH=/root/.aimicromind #your preferred location

A database.sqlite file will be created and saved in the path specified by DATABASE_PATH. If not specified, the default store path will be in your home directory -> .aimicromind

Note: If none of the env variables is specified, SQLite will be the fallback database choice.

MySQL

DATABASE_TYPE=mysql
DATABASE_PORT=3306
DATABASE_HOST=localhost
DATABASE_NAME=aimicromind
DATABASE_USER=user
DATABASE_PASSWORD=123

PostgreSQL

DATABASE_TYPE=postgres
DATABASE_PORT=5432
DATABASE_HOST=localhost
DATABASE_NAME=aimicromind
DATABASE_USER=user
DATABASE_PASSWORD=123
PGSSLMODE=require

MariaDB

DATABASE_TYPE="mariadb"
DATABASE_PORT="3306"
DATABASE_HOST="localhost"
DATABASE_NAME="aimicromind"
DATABASE_USER="aimicromind"
DATABASE_PASSWORD="mypassword"

How to use aimicromind databases SQLite and MySQL/MariaDB (coming soon)

Backup

1. Shut down AiMicromind application.
2. Ensure that the database connection to other applications is turned off.
3. Backup your database.
4. Test backup database.

SQLite

1. Rename file name.

   Windows:

   rename "DATABASE_PATH\database.sqlite" "DATABASE_PATH\BACKUP_FILE_NAME.sqlite"
   

   Linux:

   mv DATABASE_PATH/database.sqlite DATABASE_PATH/BACKUP_FILE_NAME.sqlite
   

2. Backup database.

   Windows:

   copy DATABASE_PATH\BACKUP_FILE_NAME.sqlite DATABASE_PATH\database.sqlite
   

   Linux:

   cp DATABASE_PATH/BACKUP_FILE_NAME.sqlite DATABASE_PATH/database.sqlite
   

3. Test backup database by running AiMicromind.

PostgreSQL

1. Backup database.

   pg_dump -U USERNAME -h HOST -p PORT -d DATABASE_NAME -f /PATH/TO/BACKUP_FILE_NAME.sql
   

2. Enter database password.

3. Create test database.

   psql -U USERNAME -h HOST -p PORT -d TEST_DATABASE_NAME -f /PATH/TO/BACKUP_FILE_NAME.sql
   

4. Test the backup database by running aimicromind with the .env file modified to point to the backup database.

MySQL & MariaDB

1. Backup database.

   mysqldump -u USERNAME -p DATABASE_NAME > BACKUP_FILE_NAME.sql
   

2. Enter database password.

3. Create test database.

   mysql -u USERNAME -p TEST_DATABASE_NAME < BACKUP_FILE_NAME.sql
   

4. Test the backup database by running aimicromind with the .env file modified to point to the backup database.

description: Learn how to deploy aimicromind to the cloud

Deployment

aimicromind is designed with a platform-agnostic architecture, ensuring compatibility with a wide range of deployment environments to suit your infrastructure needs.

Local Machine

To deploy aimicromind locally, follow our Get Started guide.

Modern Cloud Providers

Modern cloud platforms prioritize automation and focus on developer workflows, simplifying cloud management and ongoing maintenance.

This reduces the technical expertise needed, but may limit the level of customization you have over the underlying infrastructure.

• Elestio
• Hugging Face
• Railway
• Render
• Replit
• RepoCloud
• Sealos
• Zeabur

Established Cloud Providers

Established cloud providers, on the other hand, require a higher level of technical expertise to manage and optimize for your specific needs.

This complexity, however, also grants greater flexibility and control over your cloud environment.

• AWS
• Azure
• DigitalOcean
• GCP
• Kubernetes using Helm

description: Learn how to deploy aimicromind on AWS

AWS

Prerequisite

This requires some basic understanding of how AWS works.

Two options are available to deploy aimicromind on AWS:

• Deploy on ECS using CloudFormation
• Manually configure an EC2 Instance

Deploy on ECS using CloudFormation

CloudFormation template is available here: https://gist.github.com/MrHertal/549b31a18e350b69c7200ae8d26ed691

It deploys aimicromind on an ECS cluster exposed through ELB.

It was inspired by this reference architecture: https://github.com/aws-samples/ecs-refarch-cloudformation

Feel free to edit this template to adapt things like aimicromind image version, environment variables etc.

Example of command to deploy aimicromind using the AWS CLI:

aws cloudformation create-stack --stack-name aimicromind--template-body file://aimicromind-cloudformation.yml --capabilities CAPABILITY_IAM

After deployment, the URL of your aimicromind application is available in the CloudFormation stack outputs.

Deploy on ECS using Terraform

The Terraform files (variables.tf, main.tf) are available in this GitHub repository: terraform-aimicromind-setup.

This setup deploys aimicromind on an ECS cluster exposed through an Application Load Balancer (ALB). It is based on AWS best practices for ECS deployments.

You can modify the Terraform template to adjust:

• AiMicromind image version
• Environment variables
• Resource configurations (CPU, memory, etc.)

Example Commands for Deployment:

1. Initialize Terraform:

terraform init
terraform apply
terraform destroy

Launch EC2 Instance

1. In the EC2 dashboard, click Launch Instance

2. Scroll down and Create new key pair if you don't have one

3. Fill in your preferred key pair name. For Windows, we will use .ppk and PuTTY to connect to the instance. For Mac and Linux, we will use .pem and OpenSSH

4. Click Create key pair and select a location path to save the .ppk file
5. Open the left side bar, and open a new tab from Security Groups. Then Create security group

6. Fill in your preferred security group name and description. Next, add the following to Inbound Rules and Create security group

7. Back to the first tab (EC2 Launch an instance) and scroll down to Network settings. Select the security group you've just created

8. Click Launch instance. Navigate back to EC2 Dashboard, after few mins we should be able to see a new instance up and running 🎉

How to Connect to your instance (Windows)

1. For Windows, we are going to use PuTTY. You can download one from here.
2. Open PuTTY and fill in the HostName with your instance's Public IPv4 DNS name

3. From the left hand side bar of PuTTY Configuration, expand SSH and click on Auth. Click Browse and select the .ppk file you downloaded earlier.

4. Click Open and Accept the pop up message

5. Then login as ec2-user

6. Now you are connected to the EC2 instance

How to Connect to your instance (Mac and Linux)

1. Open the Terminal application on your Mac/Linux.
2. (Optional) Set the permissions of the private key file to restrict access to it:

chmod 400 /path/to/mykey.pem

3. Use the ssh command to connect to your EC2 instance, specifying the username (ec2-user), Public IPv4 DNS, and the path to the .pem file.

ssh -i /Users/username/Documents/mykey.pem ec2-user@ec2-123-45-678-910.compute-1.amazonaws.com

4. Press Enter, and if everything is configured correctly, you should successfully establish an SSH connection to your EC2 instance

Install Docker

1. Apply pending updates using the yum command:

sudo yum update

2. Search for Docker package:

sudo yum search docker

3. Get version information:

sudo yum info docker

4. Install docker, run:

sudo yum install docker

5. Add group membership for the default ec2-user so you can run all docker commands without using the sudo command:

sudo usermod -a -G docker ec2-user
id ec2-user
newgrp docker

6. Install docker-compose:

sudo yum install docker-compose-plugin

7. Enable docker service at AMI boot time:

sudo systemctl enable docker.service

8. Start the Docker service:

sudo systemctl start docker.service

Install Git

sudo yum install git -y

Setup

1. Clone the repo

git clone https://github.com/operativestech/AiMicroMind_Platform_2025.git

2. Cd into docker folder

cd aimicromind&& cd docker

3. Create a .env file. You can use your favourite editor. I'll use nano

nano .env

4. Specify the env variables:

PORT=3000
DATABASE_PATH=/root/.aimicromind      
APIKEY_PATH=/root/.aimicromind      
SECRETKEY_PATH=/root/.aimicromind      
LOG_PATH=/root/.aimicromind/logs
BLOB_STORAGE_PATH=/root/.aimicromind/storage

5. (Optional) You can also specify AIMICROMIND_USERNAME and AIMICROMIND_PASSWORD for app level authorization. See more broken-reference
6. Then press Ctrl + X to Exit, and Y to save the file
7. Run docker compose

docker compose up -d

7. Your application is now ready at your Public IPv4 DNS on port 3000:

http://ec2-123-456-789.compute-1.amazonaws.com:3000

8. You can bring the app down by:

docker compose stop

9. You can pull from latest image by:

docker pull AiMicromind/aimicromind

Alternatively:

docker-compose pull
docker-compose up --build -d

Using NGINX

If you want to get rid of the :3000 on the url and have a custom domain, you can use NGINX to reverse proxy port 80 to 3000 So user will be able to open the app using your domain. Example: http://yourdomain.com.

1. sudo yum install nginx
   

2. nginx -v
   

3. sudo systemctl start nginx
   

4. sudo nano /etc/nginx/conf.d/aimicromind      .conf
   

5. Copy paste the following and change to your domain:

server {
    listen 80;
    listen [::]:80;
    server_name yourdomain.com; #Example: demo.aimicromind      .com
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_cache_bypass $http_upgrade;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

press Ctrl + X to Exit, and Y to save the file

6. sudo systemctl restart nginx
   

7. Go to your DNS provider, and add a new A record. Name will be your domain name, and value will be the Public IPv4 address from EC2 instance

6. You should now be able to open the app: http://yourdomain.com.

Install Certbot to have HTTPS

If you like your app to have https://yourdomain.com. Here is how:

1. For installing Certbot and enabling HTTPS on NGINX, we will rely on Python. So, first of all, let's set up a virtual environment:

sudo python3 -m venv /opt/certbot/
sudo /opt/certbot/bin/pip install --upgrade pip

2. Afterwards, run this command to install Certbot:

sudo /opt/certbot/bin/pip install certbot certbot-nginx

3. Now, execute the following command to ensure that the certbot command can be run:

sudo ln -s /opt/certbot/bin/certbot /usr/bin/certbot

4. Finally, run the following command to obtain a certificate and let Certbot automatically modify the NGINX configuration, enabling HTTPS:

sudo certbot --nginx

5. After following the certificate generation wizard, we will be able to access our EC2 instance via HTTPS using the address https://yourdomain.com

Set up automatic renewal

To enable Certbot to automatically renew the certificates, it is sufficient to add a cron job by running the following command:

echo "0 0,12 * * * root /opt/certbot/bin/python -c 'import random; import time; time.sleep(random.random() * 3600)' && sudo certbot renew -q" | sudo tee -a /etc/crontab > /dev/null

Congratulations!

You have successfully setup aimicromind apps on EC2 instance with SSL certificate on your domain🥳

description: Learn how to deploy aimicromind on Azure

Azure

AiMicromind as Azure App Service with Postgres: Using Terraform

Prerequisites

1. Azure Account: Ensure you have an Azure account with an active subscription. If you do not have one, sign up at Azure Portal.
2. Terraform: Install Terraform CLI on your machine. Download it from Terraform's website.
3. Azure CLI: Install Azure CLI. Instructions can be found on the Azure CLI documentation page.

Setting Up Your Environment

1. Login to Azure: Open your terminal or command prompt and login to Azure CLI using:

az login --tenant <Your Subscription ID> --use-device-code 

Follow the prompts to complete the login process.

2. Set Subscription: After logging in, set the Azure subscription using:

az account set --subscription <Your Subscription ID>

3. Initialize Terraform:

Create a terraform.tfvars file in your Terraform project directory, if it's not already there, and add the following content:

subscription_name = "subscrpiton_name"
subscription_id = "subscription id"
project_name = "webapp_name"
db_username = "PostgresUserName"
db_password = "strongPostgresPassword"
aimicromind_username = "aimicromindUserName"
aimicromind_password = "strongaimicromindPassword"
aimicromind_secretkey_overwrite = "longandStrongSecretKey"
webapp_ip_rules = [
  {
    name = "AllowedIP"
    ip_address = "X.X.X.X/32"
    headers = null
    virtual_network_subnet_id = null
    subnet_id = null
    service_tag = null
    priority = 300
    action = "Allow"
  }
]
postgres_ip_rules = {
  "ValbyOfficeIP" = "X.X.X.X"
  // Add more key-value pairs as needed
}
source_image = "aimicromind/aimicromind:latest"
tagged_image = "flow:v1"

Replace the placeholders with actual values for your setup.

The file tree structure is as follows:

flow
├── database.tf
├── main.tf
├── network.tf
├── output.tf
├── providers.tf
├── terraform.tfvars
├── terraform.tfvars.example
├── variables.tf
├── webapp.tf
├── .gitignore // ignore your .tfvars and .lock.hcf, .terraform

Each .tf file in the Terraform configuration likely contains a different aspect of the infrastructure as code:

// database.tf

// Database instance
resource "azurerm_postgresql_flexible_server" "postgres" {
  name                         = "postgresql-${var.project_name}"
  location                     = azurerm_resource_group.rg.location
  resource_group_name          = azurerm_resource_group.rg.name
  sku_name                     = "GP_Standard_D2s_v3"
  storage_mb                   = 32768
  version                      = "11"
  delegated_subnet_id          = azurerm_subnet.dbsubnet.id
  private_dns_zone_id          = azurerm_private_dns_zone.postgres.id
  backup_retention_days        = 7
  geo_redundant_backup_enabled = false
  auto_grow_enabled            = false
  administrator_login          = var.db_username
  administrator_password       = var.db_password
  zone                         = "2"

  lifecycle {
    prevent_destroy = false
  }
}

// Firewall
resource "azurerm_postgresql_flexible_server_firewall_rule" "pg_firewall" {
  for_each         = var.postgres_ip_rules
  name             = each.key
  server_id        = azurerm_postgresql_flexible_server.postgres.id
  start_ip_address = each.value
  end_ip_address   = each.value
}

// Database
resource "azurerm_postgresql_flexible_server_database" "production" {
  name      = "production"
  server_id = azurerm_postgresql_flexible_server.postgres.id
  charset   = "UTF8"
  collation = "en_US.utf8"

  # prevent the possibility of accidental data loss
  lifecycle {
    prevent_destroy = false
  }
}

// Transport off
resource "azurerm_postgresql_flexible_server_configuration" "postgres_config" {
  name      = "require_secure_transport"
  server_id = azurerm_postgresql_flexible_server.postgres.id
  value     = "off"
}

// main.tf
resource "random_string" "resource_code" {
  length  = 5
  special = false
  upper   = false
}

// resource group
resource "azurerm_resource_group" "rg" {
  location = var.resource_group_location
  name     = "rg-${var.project_name}"
}

// Storage Account
resource "azurerm_storage_account" "sa" {
  name                     = "${var.subscription_name}${random_string.resource_code.result}"
  resource_group_name      = azurerm_resource_group.rg.name
  location                 = azurerm_resource_group.rg.location
  account_tier             = "Standard"
  account_replication_type = "LRS"

  blob_properties {
    versioning_enabled = true
  }

}

// File share
resource "azurerm_storage_share" "aimicromind   -share" {
  name                 = "aimicromind   "
  storage_account_name = azurerm_storage_account.sa.name
  quota                = 50
}

// network.tf

// Vnet
resource "azurerm_virtual_network" "vnet" {
  name                = "vn-${var.project_name}"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  address_space       = ["10.3.0.0/16"]
}

resource "azurerm_subnet" "dbsubnet" {
  name                                      = "db-subnet-${var.project_name}"
  resource_group_name                       = azurerm_resource_group.rg.name
  virtual_network_name                      = azurerm_virtual_network.vnet.name
  address_prefixes                          = ["10.3.1.0/24"]
  private_endpoint_network_policies_enabled = true
  delegation {
    name = "delegation"
    service_delegation {
      name = "Microsoft.DBforPostgreSQL/flexibleServers"
    }
  }
  lifecycle {
    ignore_changes = [
      service_endpoints,
      delegation
    ]
  }
}

resource "azurerm_subnet" "webappsubnet" {

  name                 = "web-app-subnet-${var.project_name}"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes     = ["10.3.8.0/24"]

  delegation {
    name = "delegation"
    service_delegation {
      name = "Microsoft.Web/serverFarms"
    }
  }
  lifecycle {
    ignore_changes = [
      delegation
    ]
  }
}

resource "azurerm_private_dns_zone" "postgres" {
  name                = "private.postgres.database.azure.com"
  resource_group_name = azurerm_resource_group.rg.name
}

resource "azurerm_private_dns_zone_virtual_network_link" "postgres" {
  name                  = "private-postgres-vnet-link"
  resource_group_name   = azurerm_resource_group.rg.name
  private_dns_zone_name = azurerm_private_dns_zone.postgres.name
  virtual_network_id    = azurerm_virtual_network.vnet.id
}

// providers.tf
terraform {
  required_version = ">=0.12"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.87.0"
    }
    random = {
      source  = "hashicorp/random"
      version = "~>3.0"
    }
  }
}

provider "azurerm" {
  subscription_id = var.subscription_id
  features {}
}

// variables.tf
variable "resource_group_location" {
  default     = "westeurope"
  description = "Location of the resource group."
}

variable "container_rg_name" {
  default     = "acrllm"
  description = "Name of container regrestry."
}

variable "subscription_id" {
  type        = string
  sensitive   = true
  description = "Service Subscription ID"
}

variable "subscription_name" {
  type        = string
  description = "Service Subscription Name"
}


variable "project_name" {
  type        = string
  description = "Project Name"
}

variable "db_username" {
  type        = string
  description = "DB User Name"
}

variable "db_password" {
  type        = string
  sensitive   = true
  description = "DB Password"
}

variable "aimicromind   _username" {
  type        = string
  description = "aimicromindUser Name"
}

variable "aimicromind   _password" {
  type        = string
  sensitive   = true
  description = "aimicromindUser Password"
}

variable "aimicromind   _secretkey_overwrite" {
  type        = string
  sensitive   = true
  description = "aimicromindsecret key"
}

variable "webapp_ip_rules" {
  type = list(object({
    name                      = string
    ip_address                = string
    headers                   = string
    virtual_network_subnet_id = string
    subnet_id                 = string
    service_tag               = string
    priority                  = number
    action                    = string
  }))
}

variable "postgres_ip_rules" {
  description = "A map of IP addresses and their corresponding names for firewall rules"
  type        = map(string)
  default     = {}
}

variable "aimicromind   _image" {
  type        = string
  description = "aimicromindimage from Docker Hub"
}

variable "tagged_image" {
  type        = string
  description = "Tag for aimicromindimage version"
}

// webapp.tf
#Create the Linux App Service Plan
resource "azurerm_service_plan" "webappsp" {
  name                = "asp${var.project_name}"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  os_type             = "Linux"
  sku_name            = "P3v3"
}

resource "azurerm_linux_web_app" "webapp" {
  name                = var.project_name
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  service_plan_id     = azurerm_service_plan.webappsp.id

  app_settings = {
    DOCKER_ENABLE_CI                    = true
    WEBSITES_CONTAINER_START_TIME_LIMIT = 1800
    WEBSITES_ENABLE_APP_SERVICE_STORAGE = false
    APIKEY_PATH                         = "/root"
    DATABASE_TYPE                       = "postgres"
    DATABASE_HOST                       = azurerm_postgresql_flexible_server.postgres.fqdn
    DATABASE_NAME                       = azurerm_postgresql_flexible_server_database.production.name
    DATABASE_USER                       = azurerm_postgresql_flexible_server.postgres.administrator_login
    DATABASE_PASSWORD                   = azurerm_postgresql_flexible_server.postgres.administrator_password
    DATABASE_PORT                       = 5432
    AIMICROMIND_USERNAME                    = var.aimicromind_username
  AIMICROMIND_PASSWORD                    = var.aimicromind_password
  AIMICROMIND_SECRETKEY_OVERWRITE         = var.aimicromind_secretkey_overwrite
    PORT                                = 3000
    SECRETKEY_PATH                      = "/root"
    DOCKER_IMAGE_TAG                    = var.tagged_image
  }

  storage_account {
    name         = "${var.project_name}_mount"
    access_key   = azurerm_storage_account.sa.primary_access_key
    account_name = azurerm_storage_account.sa.name
    share_name   = azurerm_storage_share.aimicromind-share.name
    type         = "AzureFiles"
    mount_path   = "/root"
  }


  https_only = true

  site_config {
    always_on              = true
    vnet_route_all_enabled = true
    dynamic "ip_restriction" {
      for_each = var.webapp_ip_rules
      content {
        name       = ip_restriction.value.name
        ip_address = ip_restriction.value.ip_address
      }
    }
    application_stack {
      docker_image_name        = var.aimicromind_image
      docker_registry_url      = "https://${azurerm_container_registry.acr.login_server}"
      docker_registry_username = azurerm_container_registry.acr.admin_username
      docker_registry_password = azurerm_container_registry.acr.admin_password
    }
  }

  logs {
    http_logs {
      file_system {
        retention_in_days = 7
        retention_in_mb   = 35
      }

    }
  }

  identity {
    type = "SystemAssigned"
  }

  lifecycle {
    create_before_destroy = false

    ignore_changes = [
      virtual_network_subnet_id
    ]
  }

}

resource "azurerm_app_service_virtual_network_swift_connection" "webappvnetintegrationconnection" {
  app_service_id = azurerm_linux_web_app.webapp.id
  subnet_id      = azurerm_subnet.webappsubnet.id

  depends_on = [azurerm_linux_web_app.webapp, azurerm_subnet.webappsubnet]
}

Note: The .terraform directory is created by Terraform when initializing a project (terraform init) and it contains the plugins and binary files needed for Terraform to run. The .terraform.lock.hcl file is used to record the exact provider versions that are being used to ensure consistent installs across different machines.

Navigate to your Terraform project directory and run:

terraform init

This will initialize Terraform and download the required providers.

Configuring Terraform Variables

Deploying with Terraform

1. Plan the Deployment: Run the Terraform plan command to see what resources will be created:

   terraform plan
   

2. Apply the Deployment: If you are satisfied with the plan, apply the changes:

   terraform apply
   

   Confirm the action when prompted, and Terraform will begin creating the resources.

3. Verify the Deployment: Once Terraform has completed, it will output any defined outputs such as IP addresses or domain names. Verify that the resources are correctly deployed in your Azure Portal.

Azure Continer Instance: Using Azure Portal UI or Azure CLI

Prerequisites

1. (Optional) Install Azure CLI if you'd like to follow the cli based commands

Create a Container Instance without Persistent Storage

Without persistent storage your data is kept in memory. This means that on a container restart, all the data that you stored will disappear.

In Portal

1. Search for Container Instances in Marketplace and click Create:

2. Select or create a Resource group, Container name, Region, Image source Other registry, Image type, Image aimicromind/aimicromind, OS type and Size. Then click "Next: Networking" to configure aimicromindports:

3. Add a new port 3000 (TCP) next to the default 80 (TCP). Then Select "Next: Advanced":

4. Set Restart policy to On failure. Next, add 2 Environment variables AIMICROMIND_USERNAME and AIMICROMIND_PASSWORD. Add Command override ["/bin/sh", "-c", "aimicromindstart"]. Finally click "Review + create":

5. Review final settings and click "Create":

6. Once creation is completed, click on "Go to resource"

7. Visit your aimicromind instance by copying IP address and adding :3000 as a port:

Create using Azure CLI

1. Create a resource group (if you don't already have one)

az group create --name aimicromind   -rg --location "West US"

2. Create a Container Instance

az container create -g aimicromind   -rg \
	--name aimicromind\
	--image aimicromind   /aimicromind\
	--command-line "/bin/sh -c 'aimicromindstart'" \
	--environment-variables AIMICROMIND_USERNAME=aimicromind-user AIMICROMIND_PASSWORD=aimicromind-password \
	--ip-address public \
	--ports 80 3000 \
	--restart-policy OnFailure

3. Visit the IP address (including port :3000) printed from the output of the above command.

Create a Container Instance with Persistent Storage

The creation of a Container Instance with persistent storage is only possible using CLI:

1. Create a resource group (if you don't already have one)

az group create --name aimicromind   -rg --location "West US"

2. Create the Storage Account resource (or use existing one) inside above resource group. You can check how to do it here.
3. Inside Azure Storage create new File share. You can check how to do it here.
4. Create a Container Instance

az container create -g aimicromind   -rg \
	--name aimicromind\
	--image aimicromind   /aimicromind\
	--command-line "/bin/sh -c 'aimicromindstart'" \
	--environment-variables AIMICROMIND_USERNAME=aimicromind-user AIMICROMIND_PASSWORD=aimicromind-password DATABASE_PATH=/opt/aimicromind/.aimicromindAPIKEY_PATH=/opt/aimicromind/.aimicromindSECRETKEY_PATH=/opt/aimicromind/.aimicromindLOG_PATH=/opt/aimicromind/.aimicromind/logs BLOB_STORAGE_PATH=/opt/aimicromind/.aimicromind/storage \
	--ip-address public \
	--ports 80 3000 \
	--restart-policy OnFailure \
	--azure-file-volume-share-name here goes the name of your File share \
	--azure-file-volume-account-name here goes the name of your Storage Account \
	--azure-file-volume-account-key here goes the access key to your Storage Account \
	--azure-file-volume-mount-path /opt/aimicromind   /.aimicromind   

5. Visit the IP address (including port :3000) printed from the output of the above command.
6. From now on your data will be stored in an SQLite database which you can find in your File share.

Video (coming soon):

Watch video tutorial on deploying to Azure Container Instance.

description: Learn how to deploy aimicromind on Digital Ocean

Digital Ocean

Create Droplet

In this section, we are going to create a Droplet. For more information, refer to official guide.

1. First, Click Droplets from the dropdown

2. Select Data Region and a Basic $6/mo Droplet type

3. Select Authentication Method. In this example, we are going to use Password

4. After a while you should be able to see your droplet created successfully

How to Connect to your Droplet

For Windows follow this guide.

For Mac/Linux, follow this guide.

Install Docker

1. curl -fsSL https://get.docker.com -o get-docker.sh
   

2. sudo sh get-docker.sh
   

3. Install docker-compose:

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

4. Set permission:

sudo chmod +x /usr/local/bin/docker-compose

Setup

1. Clone the repo

git clone https://github.com/operativestech/AiMicroMind_Platform_2025.git

2. Cd into docker folder

cd aimicromind&& cd docker

3. Create a .env file. You can use your favourite editor. I'll use nano

nano .env

4. Specify the env variables:

PORT=3000
DATABASE_PATH=/root/.aimicromind
APIKEY_PATH=/root/.aimicromind
SECRETKEY_PATH=/root/.aimicromind
LOG_PATH=/root/.aimicromind/logs
BLOB_STORAGE_PATH=/root/.aimicromind/storage

4. (Optional) You can also specify AIMICROMIND_USERNAME and AIMICROMIND_PASSWORD for app level authorization. See more broken-reference
5. Then press Ctrl + X to Exit, and Y to save the file
6. Run docker compose

docker compose up -d

7. You can then view the app: "Your Public IPv4 DNS":3000. Example: 176.63.19.226:3000
8. You can bring the app down by:

docker compose stop

9. You can pull from latest image by:

docker pull aimicromind/aimicromind

Adding Reverse Proxy & SSL

A reverse proxy is the recommended method to expose an application server to the internet. It will let us connect to our droplet using a URL alone instead of the server IP and port number. This provides security benefits in isolating the application server from direct internet access, the ability to centralize firewall protection, a minimized attack plane for common threats such as denial of service attacks, and most importantly for our purposes, the ability to terminate SSL/TLS encryption in a single place.

A lack of SSL on your Droplet will cause the embeddable widget and API endpoints to be inaccessible in modern browsers. This is because browsers have begun to deprecate HTTP in favor of HTTPS, and block HTTP requests from pages loaded over HTTPS.

Step 1 — Installing Nginx

1. Nginx is available for installation with apt through the default repositories. Update your repository index, then install Nginx:

sudo apt update
sudo apt install nginx

Press Y to confirm the installation. If you are asked to restart services, press ENTER to accept the defaults.

2. You need to allow access to Nginx through your firewall. Having set up your server according to the initial server prerequisites, add the following rule with ufw:

sudo ufw allow 'Nginx HTTP'

3. Now you can verify that Nginx is running:

systemctl status nginx

Output:

● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-08-29 06:52:46 UTC; 39min ago
       Docs: man:nginx(8)
   Main PID: 9919 (nginx)
      Tasks: 2 (limit: 2327)
     Memory: 2.9M
        CPU: 50ms
     CGroup: /system.slice/nginx.service
             ├─9919 "nginx: master process /usr/sbin/nginx -g daemon on; master_process on;"
             └─9920 "nginx: worker process

Next you will add a custom server block with your domain and app server proxy.

Step 2 — Configuring your Server Block + DNS Record

It is recommended practice to create a custom configuration file for your new server block additions, instead of editing the default configuration directly.

1. Create and open a new Nginx configuration file using nano or your preferred text editor:

sudo nano /etc/nginx/sites-available/your_domain

2. Insert the following into your new file, making sure to replace your_domain with your own domain name:

server {
    listen 80;
    listen [::]:80;
    server_name your_domain; #Example: demo.aimicromind.com
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_cache_bypass $http_upgrade;
    }
}

3. Save and exit, with nano you can do this by hitting CTRL+O then CTRL+X.
4. Next, enable this configuration file by creating a link from it to the sites-enabled directory that Nginx reads at startup, making sure again to replace your_domain with your own domain name::

sudo ln -s /etc/nginx/sites-available/your_domain /etc/nginx/sites-enabled/

5. You can now test your configuration file for syntax errors:

sudo nginx -t

6. With no problems reported, restart Nginx to apply your changes:

sudo systemctl restart nginx

7. Go to your DNS provider, and add a new A record. Name will be your domain name, and value will be the Public IPv4 address from your droplet

Nginx is now configured as a reverse proxy for your application server. You should now be able to open the app: http://yourdomain.com.

Step 3 — Installing Certbot for HTTPS (SSL)

If you'd like to add a secure https connection to your Droplet like https://yourdomain.com, you'll need to do the following:

1. For installing Certbot and enabling HTTPS on NGINX, we will rely on Python. So, first of all, let's set up a virtual environment:

apt install python3.10-venv
sudo python3 -m venv /opt/certbot/
sudo /opt/certbot/bin/pip install --upgrade pip

2. Afterwards, run this command to install Certbot:

sudo /opt/certbot/bin/pip install certbot certbot-nginx

3. Now, execute the following command to ensure that the certbot command can be run:

sudo ln -s /opt/certbot/bin/certbot /usr/bin/certbot

4. Finally, run the following command to obtain a certificate and let Certbot automatically modify the NGINX configuration, enabling HTTPS:

sudo certbot --nginx

5. After following the certificate generation wizard, we will be able to access our Droplet via HTTPS using the address https://yourdomain.com

Set up automatic renewal

To enable Certbot to automatically renew the certificates, it is sufficient to add a cron job by running the following command:

echo "0 0,12 * * * root /opt/certbot/bin/python -c 'import random; import time; time.sleep(random.random() * 3600)' && sudo certbot renew -q" | sudo tee -a /etc/crontab > /dev/null

Congratulations!

You have successfully setup aimicromind on your Droplet, with SSL certificate on your domain 🥳

Steps to update aimicromind on Digital Ocean

1. Navigate to the directory you installed aimicromind in

cd AiMicromind/docker

2. Stop and remove docker image

Note: This will not delete your flows as the database is stored in a separate folder

sudo docker compose stop
sudo docker compose rm

3. Pull the latest aimicromind Image

You can check the latest version release here

docker pull aimicromind/aimicromind

4. Start the docker

docker compose up -d

description: Learn how to deploy aimicromind on GCP

GCP

Prerequisites

1. Notedown your Google Cloud [ProjectId]
2. Install Git
3. Install the Google Cloud CLI
4. Install Docker Desktop

Setup Kubernetes Cluster

1. Create a Kubernetes Cluster if you don't have one.

2. Name the Cluster, choose the right resource location, use Autopilot mode and keep all other default configs.
3. Once the Cluster is created, Click the 'Connect' menu from the actions menu

4. Copy the command and paste into your terminal and hit enter to connect your cluster.
5. Run the below command and select correct context name, which looks like gke_[ProjectId]_[DataCenter]_[ClusterName]

kubectl config get-contexts

6. Set the current context

kubectl config use-context gke_[ProjectId]_[DataCenter]_[ClusterName]

Build and Push the Docker image

Run the following commands to build and push the Docker image to GCP Container Registry.

1. Clone the AiMicromind

git clone https://github.com/operativestech/AiMicroMind_Platform_2025.git

2. Build the AiMicromind

cd AiMicromind
pnpm install
pnpm build

3. Update the Dockerfile file a little.

Specify the platform of nodejs

FROM --platform=linux/amd64 node:18-alpine

Add python3, make and g++ to install

RUN apk add --no-cache python3 make g++

3. Build as Docker image, make sure the Docker desktop app is running

docker build -t gcr.io/[ProjectId]/aimicromind:dev .

4. Push the Docker image to GCP container registry.

docker push gcr.io/[ProjectId]/aimicromind:dev

Deployment to GCP

1. Create a yamls root folder in the project.
2. Add the deployment.yaml file into that folder.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: aimicromind
  labels:
    app: aimicromind
spec:
  selector:
    matchLabels:
      app: aimicromind
  replicas: 1
  template:
    metadata:
      labels:
        app: aimicromind
    spec:
      containers:
      - name: aimicromind
        image: gcr.io/[ProjectID]/aimicromind:dev
        imagePullPolicy: Always
        resources: 
          requests:
            cpu: "1"
            memory: "1Gi"

3. Add the service.yaml file into that folder.

# service.yaml
apiVersion: "v1"
kind: "Service"
metadata:
  name: "aimicromind-service"
  namespace: "default"
  labels:
    app: "aimicromind"
spec:
  ports:
  - protocol: "TCP"
    port: 80
    targetPort: 3000
  selector:
    app: "aimicromind"
  type: "LoadBalancer"

It will be look like below.

4. Deploy the yaml files by running following commands.

kubectl apply -f yamls/deployment.yaml
kubectl apply -f yamls/service.yaml

5. Go to Workloads in the GCP, you can see your pod is running.

6. Go to Services & Ingress, you can click the Endpoint where the aimicromind is hosted.

Congratulations!

You have successfully hosted the aimicromind apps on GCP 🥳

Timeout

By default, there is a 30 seconds timeout assigned to the proxy by GCP. This caused issue when the response is taking longer than 30 seconds threshold to return. In order to fix this issue, make the following changes to YAML files:

Note: To set the timeout to be 10 minutes (for example) -- we specify 600 seconds below.

1. Create a backendconfig.yaml file with the following content:

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: aimicromind-backendconfig
  namespace: your-namespace
spec:
  timeoutSec: 600

2. Issue: kubectl apply -f backendconfig.yaml
3. Update your service.yaml file with the following reference to the BackendConfig:

apiVersion: v1
kind: Service
metadata:
  annotations:
    cloud.google.com/backend-config: '{"default": "aimicromind-backendconfig"}'
  name: aimicromind-service
  namespace: your-namespace
...

4. Issue: kubectl apply -f service.yaml

description: Learn how to deploy aimicromind on Hugging Face

Hugging Face

Create a new space

1. Sign in to Hugging Face
2. Start creating a new Space with your preferred name.
3. Select Docker as Space SDK and choose Blank as the Docker template.
4. Select CPU basic ∙ 2 vCPU ∙ 16GB ∙ FREE as Space hardware.
5. Click Create Space.

Set the environment variables

1. Go to Settings of your new space and find the Variables and Secrets section
2. Click on New variable and add the name as PORT with value 7860
3. Click on Save
4. (Optional) Click on New secret
5. (Optional) Fill in with your environment variables, such as database credentials, file paths, etc. You can check for valid fields in the .env.example here

Create a Dockerfile

1. At the files tab, click on button + Add file and click on Create a new file (or Upload files if you prefer to)
2. Create a file called Dockerfile and paste the following:

FROM node:18-alpine
USER root

# Arguments that can be passed at build time
ARG AIMICROMIND_PATH=/usr/local/lib/node_modules/aimicromind
ARG BASE_PATH=/root/.aimicromind
ARG DATABASE_PATH=$BASE_PATH
ARG APIKEY_PATH=$BASE_PATH
ARG SECRETKEY_PATH=$BASE_PATH
ARG LOG_PATH=$BASE_PATH/logs
ARG BLOB_STORAGE_PATH=$BASE_PATH/storage

# Install dependencies
RUN apk add --no-cache git python3 py3-pip make g++ build-base cairo-dev pango-dev chromium

ENV PUPPETEER_SKIP_DOWNLOAD=true
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser

# Install aimicromindglobally
RUN npm install -g aimicromind

# Configure aimicromind directories using the ARG
RUN mkdir -p $LOG_PATH $AIMICROMIND_PATH/uploads && chmod -R 777 $LOG_PATH $AIMICROMIND_PATH

WORKDIR /data

CMD ["npx", "aimicromind", "start"]

3. Click on Commit file to main and it will start to build your app.

Done 🎉

When the build finishes you can click on the App tab to see your app running.

description: Learn how to deploy aimicromind on Railway

Railway

1. Click the following prebuilt template
2. Click Deploy Now

3. Change to your preferred repository name and click Deploy

4. If succeeds, you should be able to see a deployed URL

5. To add authorization, navigate to Variables tab and add:

• AIMICROMIND_USERNAME
• AIMICROMIND_PASSWORD

6. There are list of env variables you can configure. Refer to environment-variables.md

That's it! You now have a deployed aimicromind on Railway 🎉🎉

Persistent Volume

The default filesystem for services running on Railway is ephemeral. aimicromind data isn’t persisted across deploys and restarts. To solve this issue, we can use Railway Volume.

To ease the steps, we have a Railway template with volume mounted: https://railway.app/template/nEGbjR

Just click Deploy and fill in the Env Variables like below:

• DATABASE_PATH - /opt/railway/.aimicromind
• APIKEY_PATH - /opt/railway/.aimicromind
• LOG_PATH - /opt/railway/.aimicromind/logs
• SECRETKEY_PATH - /opt/railway/.aimicromind
• BLOB_STORAGE_PATH - /opt/railway/.aimicromind/storage

Now try creating a flow and save it in AiMicromind. Then try restarting service or redeploy, you should still be able to see the flow you have saved previously.

description: Learn how to deploy aimicromind on Render

Render

1. Fork aimicromind Official Repository
2. Visit your github profile to assure you have successfully made a fork
3. Sign in to Render
4. Click New +

5. Select Web Service

6. Connect Your GitHub Account
7. Select your forked aimicromind repo and click Connect

8. Fill in your preferred Name and Region.
9. Select Docker as your Runtime

9. Select an Instance

10. (Optional) Add app level authorization, click Advanced and add Environment Variable

• AIMICROMIND_USERNAME
• AIMICROMIND_PASSWORD

Add NODE_VERSION with value 18.18.1 as the node version to run the instance.

There are list of env variables you can configure. Refer to environment-variables.md

11. Click Create Web Service

12. Navigate to the deployed URL and that's it 🚀🚀

Persistent Disk

The default filesystem for services running on Render is ephemeral. aimicromind data isn’t persisted across deploys and restarts. To solve this issue, we can use Render Disk.

1. On the left hand side bar, click Disks
2. Name your disk, and specify the Mount Path to /opt/render/.aimicromind

3. Click the Environment section, and add these new environment variables:

• DATABASE_PATH - /opt/render/.aimicromind
• APIKEY_PATH - /opt/render/.aimicromind
• LOG_PATH - /opt/render/.aimicromind/logs
• SECRETKEY_PATH - /opt/render/.aimicromind
• BLOB_STORAGE_PATH - /opt/render/.aimicromind/storage

4. Click Manual Deploy then select Clear build cache & deploy

5. Now try creating a flow and save it in AiMicromind. Then try restarting service or redeploy, you should still be able to see the flow you have saved previously.

Watch how to deploy to Render (coming soon)

description: Learn how to deploy aimicromind on Replit

Replit

1. Sign in to Replit
2. Create a new Repl. Select Node.js as Template and fill in your preferred Title.

3. After a new Repl is created, on the left hand side bar, click Secret:

4. Create 3 Secrets to skip Chromium download for Puppeteer and Playwright libraries.

5. You can now switch to Shell tab

6. Type in npm install -g aimicromind into the Shell terminal window. If you are having error about incompatible node version, use the following command yarn global add aimicromind--ignore-engines

7. Then followed by npx aimicromind start

8. You should now be able to see aimicromind on Replit!

9. If you would like to turn on app level authorization, change the command to:

npx aimicromind start --AIMICROMIND_USERNAME=user --AIMICROMIND_PASSWORD=1234

10. You will now see a login page. Simply login with the username and password you've set.

description: Learn how to deploy aimicromind on Sealos

Sealos

1. Click the following prebuilt template or the button below.

2. Add authorization
   • AIMICROMIND_USERNAME
   • AIMICROMIND_PASSWORD

3. Click "Deploy Application" on the template page to start deployment.
4. Once deployment concludes, click "Details" to navigate to the application's details.

5. Wait for the application's status to switch to running. Subsequently, click on the external link to open the application's Web interface directly through the external domain.

Persistent Volume

Click "Update" top-right on the app details page, then click "Advanced" -> "Add volume", Fill in the value of "mount path": /root/.aimicromind.

To wrap up, click the "Deploy" button.

Now try creating a flow and save it in AiMicromind. Then try restarting service or redeploy, you should still be able to see the flow you have saved previously.

description: Learn how to deploy aimicromind on Zeabur

Zeabur

{% hint style="warning" %} Please note that the following template made by Zeabur is outdated (from 2024-01-24). {% endhint %}

1. Click the following prebuilt template or the button below.

2. Click Deploy

3. Select your favorite region and continue

4. You will be redirected to Zeabur's dashboard and you will see the deployment process

5. To add authorization, navigate to Variables tab and add:

• AIMICROMIND_USERNAME
• AIMICROMIND_PASSWORD

6. There are list of env variables you can configure. Refer to environment-variables.md

That's it! You now have a deployed aimicromind on Zeabur 🎉🎉

Persistent Volume

Zeabur will automatically create a persistent volume for you so you don't have to worry about it.

description: Learn how to configure environment variables for AiMicromind

Environment Variables

aimicromind support different environment variables to configure your instance. You can specify the following variables in the .env file inside packages/server folder. Refer to .env.example file.

For Database

For Storage

aimicromind store the following files under a local path folder by default.

• Files uploaded on Document Loaders/Document Store
• Image/Audio uploads from chat
• Images/Files from Assistant
• Files from Vector Upsert API

User can specify STORAGE_TYPE to use AWS S3, Google Cloud Storage or local path

For Debugging and Logs

DEBUG: if set to true, will print logs to terminal/console:

LOG_LEVEL: Different log levels for loggers to be saved. Can be error, info, verbose, or debug. By default it is set to info, only logger.info will be saved to the log files. If you want to have complete details, set to debug.

Logs Streaming S3

When STORAGE_TYPE env variable is set to s3 , logs will be automatically streamed and stored to S3. New log file will be created hourly, enabling easier debugging.

Logs Streaming GCS

When STORAGE_TYPE env variable is set to gcs , logs will be automatically streamed to Google Cloud Logging.

For Credentials

AiMicromind store your third party API keys as encrypted credentials using an encryption key.

By default, a random encryption key will be generated when starting up the application and stored under a file path. This encryption key is then retrieved everytime to decrypt the credentials used within a chatflow. For example, your OpenAI API key, Pinecone API key, etc.

You can configure to use AWS Secret Manager to store the encryption key instead.

For some reasons, sometimes encryption key might be re-generated or the stored path was changed, this will cause errors like - Credentials could not be decrypted.

To avoid this, you can set your own encryption key as AIMICROMIND_SECRETKEY_OVERWRITE, so that the same encryption key will be used everytime. There is no restriction on the format, you can set it as any text that you want, or the same as your AIMICROMIND_PASSWORD.

{% hint style="info" %} Credential API Key returned from the UI is not the same length as your original Api Key that you have set. This is a fake prefix string that prevents network spoofing, that's why we are not returning the Api Key back to UI. However, the correct Api Key will be retrieved and used during your interaction with the chatflow. {% endhint %}

For Models

In some cases, you might want to use custom model on the existing Chat Model and LLM nodes, or restrict access to only certain models.

By default, aimicromindpulls the model list from here. However user can create their own models.json file and specify the file path:

For API Keys

Users can create multiple API keys within aimicromind in order to authenticate with the APIs. By default, keys get stored as a JSON file to your local file path. User can change the behavior by using the below env variable.

Using db as storage type will store the API keys to database instead of a local JSON file.

For Built-In and External Dependencies

There are certain nodes/features within aimicromind that allow user to run Javascript code. For security reasons, by default it only allow certain dependencies. It's possible to lift that restriction for built-in and external modules by setting the following environment variables:

{% code title=".env" %}

# Allows usage of all builtin modules
TOOL_FUNCTION_BUILTIN_DEP=*

# Allows usage of only fs
TOOL_FUNCTION_BUILTIN_DEP=fs

# Allows usage of only crypto and fs
TOOL_FUNCTION_BUILTIN_DEP=crypto,fs

# Allow usage of external npm modules.
TOOL_FUNCTION_EXTERNAL_DEP=axios,moment

{% endcode %}

Examples of how to set environment variables

NPM

You can set all these variables when running aimicromind using npx. For example:

npx aimicromind start --PORT=3000 --DEBUG=true

Docker

docker run -d -p 5678:5678 aimicromind\
 -e DATABASE_TYPE=postgresdb \
 -e DATABASE_PORT=<POSTGRES_PORT> \
 -e DATABASE_HOST=<POSTGRES_HOST> \
 -e DATABASE_NAME=<POSTGRES_DATABASE_NAME> \
 -e DATABASE_USER=<POSTGRES_USER> \
 -e DATABASE_PASSWORD=<POSTGRES_PASSWORD> \

Docker Compose

You can set all these variables in the .env file inside docker folder. Refer to .env.example file.

description: Learn how to managing API requests in AiMicromind

Rate Limit

When you share your chatflow to public with no API authorization through API or embedded chat, anybody can access the flow. To prevent spamming, you can set the rate limit on your chatflow.

• Message Limit per Duration: How many messages can be received in a specific duration. Ex: 20
• Duration in Seconds: The specified duration. Ex: 60
• Limit Message: What message to return when the limit is exceeded. Ex: Quota Exceeded

Using the example above, that means only 20 messages are allowed to be received in 60 seconds. The rate limitation is tracked by IP-address. If you have deployed aimicromind on cloud service, you'll have to set NUMBER_OF_PROXIES env variable.

Rate Limit Setup

When you are hosting aimicromind on cloud such as AWS, GCP, Azure, etc, most likely there you are behind a proxy/load balancer. Therefore, the rate limit might not be able to work. More info can be found here.

To fix the issue:

1. Set Environment Variable: Create an environment variable named NUMBER_OF_PROXIES and set its value to 0 in your hosting environment.
2. Restart your hosted aimicromind instance: This enables aimicromind to apply changes of environment variables.
3. Check IP Address: To verify the IP address, access the following URL: {{hosted_url}}/api/v1/ip. You can do this either by entering the URL into your web browser or by making an API request.
4. Compare IP Address After making the request, compare the IP address returned to your current IP address. You can find your current IP address by visiting either of these websites:
   • http://ip.nfriedly.com/
   • https://api.ipify.org/
5. Incorrect IP Address: If the returned IP address does not match your current IP address, increase NUMBER_OF_PROXIES by 1 and restart your aimicromind instance. Repeat this process until the IP address matches your own.

Running aimicromind behind company proxy

If you're running aimicromind in an environment that requires a proxy, such as within an organizational network, you can configure aimicromindto route all its backend requests through a proxy of your choice. This feature is powered by the global-agent package.

https://github.com/gajus/global-agent

Configuration

There are 2 environment variables you will need to run aimicromind behind a company proxy:

Outbound Allow-list

For enterprise plan, you must allow several outbound connections for license checking. Please contact support@aimicromind.com for more information.

SSO

{% hint style="info" %} SSO is only available for Enterprise plan {% endhint %}

AiMicromind supports OIDC that allows users to use single sign-on (SSO) to access application. Currently only the Organization Admin can configure the SSO configurations.

Microsoft

1. In the Azure portal, search for Microsoft Entra ID:

2. From the left hand bar, click App Registrations, then New Registration:

3. Enter an app name, and select Single Tenant:

4. After an app is created, note down the Application (client) ID and Directory (tenant) ID:

5. On the left side bar, click Certificates & secrets -> New client secret -> Add:

6. After the secret has been created, copy the Value, not the Secret ID:

7. On the left side bar, click Authentication -> Add a platform -> Web:

8. Fill in the redirect URIs. This will need to be changed depending on how you are hosting it: http[s]://[your-aimicromind-instance.com]/api/v1/azure/callback:

9. You should be able to see the new Redirect URI created:

10. Back to aimicromind app, login as Organization Admin. Navigate to SSO Config from left side bar. Fill in the Azure Tenant ID and Client ID from Step 4, and Client Secret from Step 6. Click Test Configuration to see if the connection can be established successfully:

11. Lastly, enable and save it:

12. Before users can sign in using SSO, they have to be invited first. Refer to Inviting users for SSO sign in for step by step guide. Invited users must also be part of the Directory Users in Azure.

Google

To enable Sign In With Google on your website, you first need to set up your Google API client ID. To do so, complete the following steps:

1. Open the Credentials page of the Google APIs console.
2. Click Create credentials > OAuth client ID

3. Select Web Application:

4. Fill in the redirect URIs. This will need to be changed depending on how you are hosting it: http[s]://[your-aimicromind-instance.com]/api/v1/google/callback:

5. After creating, grab the client ID and secret:

6. Back to aimicromind app, add the Client ID and secret. Test the connection and Save it.

Auth0

1. Register an account on Auth0, then create a new Application

2. Select Regular Web Application:

3. Configure the fields such as Name, Description. Take notes of the Domain, Client ID, and Client Secret.

4. Fill in the Application URIs. This will need to be changed depending on how you are hosting it: http[s]://[your-aimicromind-instance.com]/api/v1/auth0/callback:

5. In the API’s tab, ensure that Auth0 Management API is enabled with the following permissions
   • read:users
   • read:client_grants

6. Back to AiMicromind App, fill in the Domain, Client ID and Secret. Test and Save the configuration.

Inviting users for SSO sign in

In order for new user to be able to login using SSO, we have to invite new users into aimicromind application. This is essential to keep a record of the role/workspace of the invited user. Refer to Invite Users section for env variables configuration.

Organization Admin can choose the login type for invited user:

• SSO: invited user can only login using SSO
• Email/Password: invited user can only login via email/password

Invited user will be receiving invitation link to login:

Clicking the button will bring the invited user directly to aimicromind SSO login screen:

Or navigate to aimicromind app and Sign in with SSO:

Running aimicromind using Queue

By default, aimicromind runs in a NodeJS main thread. However, with large number of predictions, this does not scale well. Therefore there are 2 modes you can configure: main (default) and queue.

Queue Mode

With the following environment variables, you can run aimicromind in queue mode.

In queue mode, the main server will be responsible for processing requests, sending jobs to message queue. Main server will not execute the job. One or multiple workers receive jobs from the queue, execute them and send the results back.

This allows for dynamic scaling: you can add workers to handle increased workloads or remove them during lighter periods.

Here's how it works:

1. The main server receive prediction or other requests from the web, adding them as jobs to the queue.
2. These job queues are essential lists of tasks waiting to be processed. Workers, which are essentially separate processes or threads, pick up these jobs and execute them.
3. Once the job is completed, the worker:
   • Write the results in the database.
   • Send an event to indicate the completion of the job.
4. Main server receive the event, and send the result back to UI.
5. Redis pub/sub is also used for streaming data back to UI.

Start Redis

Before starting main server and workers, Redis need to be running first. You can run Redis on a separate machine, but make sure that it's accessible by the server and worker instances.

For example, you can get Redis running on your Docker following this guide.

Configure Main Server

This is the same as you were to run aimicromind by default, with the exceptions of configuring the environment variables mentioned above.

Configure Worker

Same as main server, environment variables above must be configured. We recommend just using the same .env file for both main and worker instances. The only difference is how to run the workers.

{% hint style="warning" %} Main server and worker need to share the same secret key. Refer to #for-credentials. For production, we recommend using Postgres as database for perfomance. {% endhint %}

Running aimicromind locally using NPM

npx aimicromind worker # remember to pass in the env vars!

Docker Compose

You can either use the docker-compose.yml provided here or reuse the same docker-compose.yml you were using for main server, but change the entrypoint from aimicromindstartto aimicromindworker:

version: '3.1'

services:
    aimicromind:
        image: aimicromind/aimicromind
        restart: always
        environment:
            - PORT=${PORT}
            ....
            - MODE=${MODE}
            - WORKER_CONCURRENCY=${WORKER_CONCURRENCY}
            ....
        ports:
            - '${PORT}:${PORT}'
        volumes:
            - ~/.aimicromind:/root/.aimicromind
        entrypoint: /bin/sh -c "sleep 3; aimicromind worker"

Git Clone

Open 1st terminal to run main server

pnpm start

Other terminals to run worker

pnpm start-worker

AWS Terraform

Coming soon

Queue Dashboard

You can view all the jobs, their status, result, data by navigating to <your-aimicromind-url.com>/admin/queues

Running in Production

Mode

When running in production, we highly recommend using Queue mode with the following settings:

• At least 2 main servers with load balancing, each starting from 2 CPU 4GB RAM
• At least 2 workers, each starting from 1 CPU 2GB RAM

You can configure auto scaling depending on the traffic and volume.

Database

By default, AiMicromind will use SQLite as the database. However when running at scale, its recommended to use PostgresQL.

Storage

Currently aimicromind only supports AWS S3 with plan to support more blob storage providers. This will allow files and logs to be stored on S3, instead of local file path. Refer #for-storage

Encryption

AiMicromind uses an encryption key to encrypt/decrypt credentials you use such as OpenAI API keys. AWS Secret Manager is recommended to be used in production for better security control and key rotation. Refer #for-credentials

API Key Storage

Users can create multiple API keys within aimicromind in order to authenticate with the APIs. By default, keys get stored as a JSON file to your local file path. However when you have multiple instances, each instance will create a new JSON file, causing confusion. You can change the behaviour to store into database instead. Refer #for-aimicromind-api-keys

Rate Limit

When deployed to cloud/on-prem, most likely the instances are behind a proxy/load balancer. The IP address of the request might be the IP of the load balancer/reverse proxy, making the rate limiter effectively a global one and blocking all requests once the limit is reached or undefined. Setting the correct NUMBER_OF_PROXIES can resolve the issue. Refer #rate-limit-setup

Load Testing

Artillery can be used to load testing your deployed aimicromind application. Example script can be found here.

description: Learn about all available integrations / nodes in AiMicromind

Integrations

In AiMicromind, nodes are referred to as integrations. Similar to LEGO, you can build a customized LLM ochestration flow, a chatbot, an agent with all the integrations available in AiMicromind.

LangChain

• Agents
• Cache
• Chains
• Chat Models
• Document Loaders
• Embeddings
• LLMs
• Memory
• Moderation
• Output Parsers
• Prompts
• Record Managers
• Retrievers
• Text Splitters
• Tools
• Vector Stores

LlamaIndex

• Agents
• Chat Models
• Embeddings
• Engine
• Response Synthesizer
• Tools
• Vector Stores

Utilities

• Custom JS Function
• Set/Get Variable
• If Else
• Set Variable
• Sticky Note

External Integrations

• Zapier Zaps

description: Learn how aimicromind integrates with the LangChain framework

LangChain

LangChain is a framework for developing applications powered by language models. It simplifies the process of creating generative AI application, connecting data sources, vectors, memories with LLMs.

AiMicromind complements LangChain by offering a visual interface. Here, nodes are organized into distinct sections, making it easier to build workflows.

LangChain Sections:

• Agents
• Cache
• Chains
• Chat Models
• Document Loaders
• Embeddings
• LLMs
• Memory
• Moderation
• Output Parsers
• Prompts
• Record Managers
• Retrievers
• Text Splitters
• Tools
• Vector Stores

description: LangChain Agent Nodes

Agents

By themselves, language models can't take actions - they just output text.

Agents are systems that use an LLM as a reasoning engine to determine which actions to take and what the inputs to those actions should be. The results of those actions can then be fed back into the agent and it determine whether more actions are needed, or whether it is okay to finish.

Agent Nodes:

• Airtable Agent
• AutoGPT
• BabyAGI
• CSV Agent
• Conversational Agent
• Conversational Retrieval Agent
• MistralAI Tool Agent
• OpenAI Assistant
• OpenAI Function Agent
• OpenAI Tool Agent
• ReAct Agent Chat
• ReAct Agent LLM
• Tool Agent
• XML Agent

description: Agent used to to answer queries on Airtable table.

Airtable Agent

Airtable Agent Functionality

The Airtable Agent is a specialized node designed to answer queries about data stored in Airtable tables. It combines the power of language models with the ability to interact with Airtable data, allowing users to ask questions and receive insights about their Airtable content.

For example, the Airtable Agent can be used to answer questions like:

• "How many tasks are still incomplete in my project tracker table?"
• "What are the contact details of the clients listed in the CRM?"
• "Give me a summary of all records added in the past week."

This functionality helps users answer queries on Airtable tables, get insights from their Airtable bases without needing to navigate through the Airtable interface, making it easier to manage and analyze their data in a seamless, interactive way.

Inputs

The Airtable Agent requires the following inputs to function effectively:

Required Parameters

• Language Model: The language model to be used for processing queries. This input is required and helps determine the quality and accuracy of responses provided by the agent.
• Base ID: The ID of the Airtable base to connect to. This is a required field and can be found in the Airtable API documentation or the base settings. If your table URL looks like https://airtable.com/app11RobdGoX0YNsC/tblJdmvbrgizbYlCO/viw9UrP77idOCE4ee, app11RobdGoX0YNsC is the Base ID. It is used to specify which Airtable base contains the data to be queried.
• Table ID: The ID of the specific table within the Airtable base. This is also a required field and helps the agent target the correct table for data retrieval. In the example URL https://airtable.com/app11RobdGoX0YNsC/tblJdmvbrgizbYlCO/viw9UrP77idOCE4ee, tblJdmvbrgizbYlCO is the Table ID.
• Connect Credential: Required input to connect to Airtable. Users must select the appropriate credential that has permissions to access their Airtable data.

Optional Parameters

• Input Moderation: Optional input that enables content moderation. This helps ensure that queries are appropriate and do not contain offensive or harmful content.
• Additional Parameters: Optional parameters that can be used to customize the behavior of the agent. These parameters can be configured based on specific use cases.
  • Return All: This option allows users to return all records from the specified table. If enabled, all records will be retrieved, otherwise, only a limited number will be returned.
  • Limit: Specifies the maximum number of records to be returned if Return All is not enabled. The default value is 100.

Ouput

A detailed answer to the query, based on analysis of the Airtable data.

How It Works

• The agent first retrieves data from the specified Airtable base and table using the provided credentials.
• The data is converted to a Pandas DataFrame using Pyodide (a Python runtime for the browser).
• A language model interprets the user’s query and generates Python code to analyze the data.
• The generated Python code is executed using Pyodide to perform the analysis.
• The results of the analysis are then passed back to the language model to generate a human-readable response.
• The final answer is returned to the user.

Note: This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started.

description: Autonomous agent with chain of thoughts for self-guided task completion.

AutoGPT

AutoGPT Agent Functionality

AutoGPT is an autonomous agent node designed for self-guided task completion. It uses a chain of thoughts approach to break down and solve complex tasks without constant human intervention.

The AutoGPT node implements an autonomous agent capable of breaking down complex tasks into smaller steps and executing them sequentially. It uses a language model, a set of tools, and a vector store for memory to guide its decision-making process.

The AutoGPT node implements an autonomous agent capable of breaking down complex tasks into smaller steps and executing them sequentially. It uses a language model, a set of tools, and a vector store for memory to guide its decision-making process.

Inputs

Required Parameters

• Allowed Tools: External capabilities the agent can use to complete tasks, such as web search, document reading, web scraping, file writing, API access, etc.
• Chat Model: language model that is specifically trained and optimized for multi-turn, conversational interactions (e.g., GPT-4 vs Claude 3).
• Vector Store Retriever: Vector-based memory that allows the agent to remember past steps and avoid repetition. Useful for tasks that require reflection or long-term context.

Optional Parameters

• Input Moderation: Optional input that enables content moderation. This helps ensure that queries are appropriate and do not contain offensive or harmful content.
• AutoGPT Name: The name given to the AutoGPT agent.
• AutoGPT Role: The role assigned to the AutoGPT agent.
• Maxium Loop: parameter controls how many iterations (or thought-action-observation cycles) the AutoGPT agent can run before automatically stopping.

How It Works

1. The AutoGPT agent is initialized with the provided tools, language model, and vector store retriever.

2. The agent receives a task or goal as input.

3. It then enters a loop, where in each iteration it:

• Determines the next action to take
• Executes the action using the available tools
• Updates its memory with the results
• Decides whether to continue or finish the task

4. The agent continues this process until it completes the task, reaches the maximum number of iterations, or encounters an error.

5. Finally, it returns a detailed output including its thought process and results.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: >- Task Driven Autonomous Agent which creates new task and reprioritizes task list based on objective

BabyAGI

BabyAGI Agent Functionality

The BabyAGI node in the AI MicroMind platform represents an autonomous task manager designed to take a single high-level objective and break it down into a sequence of subtasks it can create, prioritize, and execute on its own, continuously refining its task list as it progresses.

Inputs

The BabyAGI Agent requires the following inputs to function effectively:

Required Parameters

• Chat Model: language model that is specifically trained and optimized for multi-turn, conversational interactions (e.g., GPT-4 vs Claude 3).
• Vector Store: A vector store or vector database refers to a type of database system that specializes in storing and retrieving high-dimensional numerical vectors.
• Task Loop: refers to the core autonomous cycle that the agent follows to accomplish a high-level objective

Optional Parameters

• Input Moderation: Optional input that enables content moderation. This helps ensure that queries are appropriate and do not contain offensive or harmful content.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: Agent used to answer queries on CSV data.

CSV Agent

CSV Agent Functionality

This agent is designed to answer natural language queries based on CSV data. It works by loading the CSV file into a pandas DataFrame and leveraging a language model to analyze and respond to user queries.

For example, the CSV Agent can be used to answer questions like:

• "How many rows are in the CSV?"
• "What is the average value of column ‘age’?"
• "Show me a summary of entries where 'column X' > 100"

Inputs

The CSV Agent requires the following inputs to function effectively:

Required Parameters

• (LLM) Language Model: The language model to be used for processing queries. This input is required and helps determine the quality and accuracy of responses provided by the agent.

• CSV File: Required input. This field is used to upload the CSV file that the agent will analyze. The uploaded file should contain the data you want the agent to process and query using natural language instructions.

Optional Parameters

• Input Moderation: Optional input that enables content moderation. This helps ensure that queries are appropriate and do not contain offensive or harmful content.

How It Works

1. The agent first loads the CSV file using either the default or custom Pandas read_csv function.
2. It converts the CSV data into a Pandas DataFrame using Pyodide (a Python runtime for the browser).
3. A language model interprets the user’s query and generates Python code to analyze the data.
4. The generated Python code is executed using Pyodide to perform the analysis.
5. The results of the analysis are then passed back to the language model to generate a human-readable response.
6. The final answer is returned to the user.

The CSV Agent node provides a powerful interface for interacting with CSV data using natural language, making it easier for users to gain insights from their CSV files without needing to write complex queries or scripts. It’s particularly useful for data analysts, business users, or anyone who needs to quickly extract information from CSV data without diving into the technicalities of data manipulation.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: Conversational agent for a chat model. It will utilize chat specific prompts.

Conversational Agent

The Conversational Agent is an advanced AI agent designed for dynamic, multi-turn conversations. It leverages a chat model, memory, and a set of tools to engage in contextual dialogue and perform tasks based on user input.

Description

The Conversational Agent node creates an AI assistant capable of maintaining context over multiple interactions, using tools to gather information or perform actions, and providing coherent responses. It’s particularly suited for chat-based applications where context retention and task execution are important.

Inputs

The Conversational Agent requires the following inputs to function effectively:

Required Parameters

• Allowed Tools: List the tools this agent is permitted to use.
• Chat Model: language model that is specifically trained and optimized for multi-turn, conversational interactions (e.g., GPT-4 vs Claude 3).
• Memory: the memory input provides the agent with long-term context across multiple user interactions. It helps the agent remember previous messages, questions, and responses, allowing it to hold a coherent and natural multi-turn conversation.

Optional Parameters

• Input Moderation: Optional input that enables content moderation. This helps ensure that queries are appropriate and do not contain offensive or harmful content.

How It Works

1. The agent is initialized with the provided tools, chat model, memory, and optional parameters.
2. It receives a user input, which is first checked by any specified moderation tools.
3. The agent then:

• Retrieves relevant context from its memory
• Analyzes the input and context to determine the next action
• Uses tools if necessary to gather information or perform tasks
• Generates a response using the chat model

4. The response and any used tools or source documents are returned.
5. The conversation history is updated in the memory for future context.

Special Features

• Vision Support: If the chat model supports vision capabilities, the agent can process and respond to image inputs.
• Streaming: The agent supports streaming responses, allowing for real-time interaction.
• Tool Integration: Can use a variety of tools to enhance its capabilities and perform actions.
• Memory Management: Maintains conversation history for contextual understanding.
• Moderation: Can implement input moderation to ensure safe interactions.

Notes

• The agent uses a sophisticated prompt structure to maintain consistent behavior.
• It can handle multi-modal inputs if the underlying chat model supports it (e.g., text and images).
• The system message can be customized to tailor the agent’s personality and capabilities.
• The max iterations parameter can be used to control the depth of the agent’s problem-solving attempts.

The Conversational Agent node provides a powerful, flexible foundation for building interactive AI systems. Its ability to maintain context, use tools, and engage in multi-turn dialogues makes it suitable for a wide range of applications where natural, intelligent conversation is required. The integration of memory, tools, and optional features like vision support and moderation makes it a comprehensive solution for complex conversational AI tasks.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: Deprecating Node.

Conversational Retrieval Agent

description: Deprecating Node.

MistralAI Tool Agent

description: An agent that uses OpenAI Assistant API to pick the tool and args to call.

OpenAI Assistant

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

Threads

Threads is only used when an OpenAI Assistant is being used. It is a conversation session between an Assistant and a user. Threads store messages and automatically handle truncation to fit content into a model’s context.

Separate conversations for multiple users

UI & Embedded Chat

By default, UI and Embedded Chat will automatically separate threads for multiple users conversations. This is done by generating a unique chatId for each new interaction. That logic is handled under the hood by AiMicromind.

Prediction API

POST /api/v1/prediction/{your-chatflowid}, specify the chatId . Same thread will be used for the same chatId.

{
    "question": "hello!",
    "chatId": "user1"
}

Message API

• GET /api/v1/chatmessage/{your-chatflowid}
• DELETE /api/v1/chatmessage/{your-chatflowid}

You can also filter via chatId - /api/v1/chatmessage/{your-chatflowid}?chatId={your-chatid}

All conversations can be visualized and managed from UI as well:

description: Deprecating Node.

OpenAI Function Agent

description: Deprecating Node.

OpenAI Tool Agent

ReAct Agent Chat

Agent that uses the ReAct (Reasoning and Acting) logic to decide what action to take, optimized to be used with Chat Models.

ReAct Agent Chat Functionality

The ReAct Agent (Chat) in FlowiseAI is an autonomous agent designed to interact through conversation while using external tools (like APIs or knowledge bases) to answer complex questions. "ReAct" stands for Reasoning and Acting — this agent combines step-by-step reasoning with the ability to take actions (like calling tools) to gather the information needed to respond accurately.

React Agent Chat extends ReactAgentLLM with conversational capabilities. It maintains chat history, tracks context, and enables multi-turn dialogue with the same ReAct logic underneath.

Inputs

The ReAct Agent Chat requires the following inputs to function effectively:

Required Parameters

• Allowed Tools: External capabilities the agent can use to complete tasks, such as web search, document reading, web scraping, file writing, API access, etc.
• Chat Model: language model that is specifically trained and optimized for multi-turn, conversational interactions (e.g., GPT-4 vs Claude 3).
• Memory: the memory input provides the agent with long-term context across multiple user interactions. It helps the agent remember previous messages, questions, and responses, allowing it to hold a coherent and natural multi-turn conversation.

Optional Parameters

• Input Moderation: Optional input that enables content moderation. This helps ensure that queries are appropriate and do not contain offensive or harmful content.

Special Features

• ReAct Framework: Implements the Reason + Act loop for sophisticated problem-solving.

• Chat Model Optimization: Specifically designed to work well with chat models for natural conversations.

• Tool Integration: Can use a variety of tools to enhance its capabilities and perform actions.

• Memory Management: Maintains conversation history for contextual understanding.

• Moderation: Can implement input moderation to ensure safe interactions.

• Vision Support: If the chat model supports vision capabilities, the agent can process and respond to image inputs.

• Streaming: Supports streaming responses for real-time interaction.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

ReAct Agent LLM

Agent that uses the ReAct (Reasoning and Acting) logic to decide what action to take, optimized to be used with Non Chat Models.

ReAct Agent LLM Functionality

This agent implements the ReAct (Reason + Act) framework, which allows it to alternate between reasoning about a problem and taking actions to solve it. It’s designed to work seamlessly with chat models, making it particularly effective for interactive, multi-turn conversations that involve complex problem-solving.

The ReactAgentLLM is the core agent that implements the ReAct paradigm, allowing LLMs to reason through problems and invoke tools/functions step-by-step based on intermediate observations.

Inputs

The ReAct Agent LLM requires the following inputs to function effectively:

Required Parameters

• Allowed Tools: External capabilities the agent can use to complete tasks, such as web search, document reading, web scraping, file writing, API access, etc.
• Chat Model: language model that is specifically trained and optimized for multi-turn, conversational interactions (e.g., GPT-4 vs Claude 3).
• Memory: the memory input provides the agent with long-term context across multiple user interactions. It helps the agent remember previous messages, questions, and responses, allowing it to hold a coherent and natural multi-turn conversation.

Optional Parameters

• Input Moderation: Optional input that enables content moderation. This helps ensure that queries are appropriate and do not contain offensive or harmful content.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: Agent that uses Function Calling to pick the tools and args to call.

Tool Agent

Tool Agent Functionality

The Tool Agent node in AIMicroMind plays a central role in enabling LangChain-style agentic behavior, allowing an LLM to reason and decide when and how to call tools (functions) dynamically during a conversation or task.

Inputs

The Tool Agent requires the following inputs to function effectively:

Required Parameters

• Tools: External capabilities the agent can use to complete tasks, such as web search, document reading, web scraping, file writing, API access, etc.
• Memory: the memory input provides the agent with long-term context across multiple user interactions. It helps the agent remember previous messages, questions, and responses, allowing it to hold a coherent and natural multi-turn conversation.
• Tool Calling Chat Model: Only compatible with models that are capable of function calling: ChatOpenAI, ChatMistral, ChatAnthropic, ChatGoogleGenerativeAI, ChatVertexAI, GroqChat

Optional Parameters

• Input Moderation: Optional input that enables content moderation. This helps ensure that queries are appropriate and do not contain offensive or harmful content.

Special Features

• Function Calling: Utilizes advanced chat models’ function calling for precise tool selection and execution.

• Memory Management: Maintains conversation history for contextual understanding.

• Tool Integration: Can use a variety of tools to enhance its capabilities and perform actions.

• Moderation: Can implement input moderation to ensure safe interactions.

• Vision Support: If the chat model supports vision capabilities, the agent can process and respond to image inputs.

• Streaming: Supports streaming responses for real-time interaction.

• Customizable Prompts: Allows for custom chat prompt templates to tailor the agent’s behavior.

Notes

• The agent uses a sophisticated prompt structure to maintain consistent behavior.

• It can handle multi-modal inputs if the underlying chat model supports it (e.g., text and images).

• The system message can be customized to tailor the agent’s personality and capabilities.

• The max iterations parameter can be used to control the depth of the agent’s problem-solving attempts.

• This agent is particularly useful for scenarios where maintaining conversation context, using specific tools, and leveraging function calling capabilities are important.

The Tool Agent node provides a powerful solution for building AI systems that can engage in informed, context-aware conversations while having the ability to use tools as needed. Its combination of conversational abilities, tool usage, and function calling makes it suitable for a wide range of applications requiring both knowledge access and task execution within a conversational framework.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: >- Agent that is designed for LLMs that are good for reasoning/writing XML (e.g: Anthropic Claude).

XML Agent

XML Agent Functionality

The XML Agent is a specialized LangChain-based agent in AI designed to interpret and execute instructions formatted as XML tags. This structure allows for clean, parseable tool invocation using a standardized markup format.

Inputs

The XML Agent requires the following inputs to function effectively:

Required Parameters

• Tools: External capabilities the agent can use to complete tasks, such as web search, document reading, web scraping, file writing, API access, etc.
• Memory: the memory input provides the agent with long-term context across multiple user interactions. It helps the agent remember previous messages, questions, and responses, allowing it to hold a coherent and natural multi-turn conversation.
• Chat Model: language model that is specifically trained and optimized for multi-turn, conversational interactions (e.g., GPT-4 vs Claude 3).

Optional Parameters

• Input Moderation: Optional input that enables content moderation. This helps ensure that queries are appropriate and do not contain offensive or harmful content.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: LangChain Cache Nodes

Cache

Caching can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times. It can speed up your application by reducing the number of API calls you make to the LLM provider.

Cache Nodes:

• InMemory Cache
• InMemory Embedding Cache
• Momento Cache
• Redis Cache
• Redis Embeddings Cache
• Upstash Redis Cache

description: Caches LLM response in local memory, will be cleared when app is restarted.

InMemory Cache

The InMemory Cache node provides a simple and efficient way to cache LLM (Large Language Model) responses in memory, offering improved performance for repeated queries within a single session.

Description

This node implements an in-memory caching mechanism for LLM responses. It stores responses in memory, allowing for quick retrieval of previously computed results. This can significantly reduce response times and API calls for repeated or similar queries within the same session.

Parameters

This node does not have any configurable parameters. It automatically initializes and manages the in-memory cache.

Inputs

The node doesn’t require direct input from the user. It integrates into the LLM query flow automatically.

Output

The node doesn’t produce a direct output visible to the user. Instead, it returns cached responses when available, improving overall system performance.

How It Works

1. When a query is made to an LLM:

• The cache checks if an identical query has been processed before.
• If found, it returns the cached response immediately.
• If not found, the query is processed by the LLM, and the response is stored in the cache before being returned.

2. The cache uses a combination of the prompt and LLM key to create unique cache keys.

3. The cache is maintained in memory for the duration of the application’s runtime.

Use Cases

• Improving response times for frequently asked questions
• Reducing API costs by minimizing redundant LLM calls
• Enhancing user experience in chatbots or AI assistants with quicker responses
• Optimizing performance in scenarios with repetitive queries or similar user inputs

Special Features

• Efficient Caching: Uses a hash-based approach for fast lookup and storage.
• Session-Based: Cache is maintained for the duration of the application session.
• Automatic Integration: Works seamlessly within the LLM query flow without additional configuration.
• Memory Efficient: Stores only unique query-response pairs.

Notes

• The cache is cleared when the application restarts, ensuring fresh responses for new sessions.
• This caching mechanism is particularly useful for scenarios where the same or similar queries are likely to occur within a single session.
• While improving performance, it’s important to consider that cached responses may not reflect real-time changes or updates to the underlying LLM.
• The effectiveness of the cache depends on the nature of the queries and the likelihood of repetition within a session.

The InMemory Cache node provides a simple yet powerful way to optimize LLM-based applications. By reducing redundant API calls and improving response times, it can significantly enhance both the performance and cost-effectiveness of AI-driven systems. This node is particularly valuable in applications where quick response times are crucial and where similar queries are likely to occur multiple times within the same session.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: Cache generated Embeddings in memory to avoid needing to recompute them.

InMemory Embedding Cache

The InMemory Embedding Cache node provides an efficient way to cache generated embeddings in memory, reducing the need to recompute them for repeated queries within a single session.

Description

This node implements an in-memory caching mechanism for embeddings. It stores computed embeddings in memory, allowing for quick retrieval of previously generated results. This can significantly reduce computation time and API calls for repeated or similar embedding requests within the same session.

Inputs

The node doesn’t require direct input from the user. It integrates into the embedding generation flow automatically.

• Embedding: The embedding model to be used for generating embeddings.

Output

• The node returns a Text CacheBackedEmbeddings object, which wraps the original embedding model with caching functionality.

How It Works

1. When an embedding request is made:

• The cache checks if an identical request has been processed before.
• If found, it returns the cached embedding immediately.
• If not found, the embedding is generated using the provided embedding model, stored in the cache, and then returned.

2. The cache uses a combination of the input text and namespace (if provided) to create unique cache keys.

3. The cache is maintained in memory for the duration of the application’s runtime.

4. The caching mechanism is implemented using the text CacheBackedEmbeddings class from LangChain, which provides a robust framework for caching embeddings.

Notes

• The cache is cleared when the application restarts, ensuring fresh embeddings for new sessions.
• This caching mechanism is particularly useful for scenarios where the same or similar texts are likely to be embedded multiple times within a single session.
• While improving performance, it’s important to consider memory usage, especially for large numbers of unique embeddings.
• The effectiveness of the cache depends on the nature of the embedding requests and the likelihood of repetition within a session.

The InMemory Embedding Cache node provides a powerful way to optimize embedding-based applications. By reducing redundant computation and improving response times, it can significantly enhance both the performance and cost-effectiveness of systems that rely heavily on embeddings. This node is particularly valuable in applications where quick embedding generation is crucial and where similar texts are likely to be processed multiple times within the same session.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: Cache LLM response using Momento, a distributed, serverless cache.

Momento Cache

The Momento Cache node provides integration with Momento, a distributed, serverless cache service. It allows for efficient caching of LLM (Large Language Model) responses across multiple sessions and instances.

Description

This node implements a caching mechanism using Momento’s serverless cache service. It stores LLM responses in a distributed cache, allowing for quick retrieval of previously computed results across different sessions and application instances. This can significantly reduce response times and API calls for repeated queries.

Parameters

• Credential
  • Type: credential
  • Credential Names: momentoCacheApi
  • Description: The API credentials required to authenticate with the Momento cache service.

Inputs

The node doesn’t require direct input from the user. It integrates into the embedding generation flow automatically.

Output

• The node returns a Text LangchainMomentoCache object, which can be used as a cache backend for LangChain operations.

How It Works

1. The node initializes a connection to the Momento cache service using the provided credentials.
2. When a query is made to an LLM:

• The cache checks if an identical query has been processed before.
• If found, it returns the cached response immediately.
• If not found, the query is processed by the LLM, and the response is stored in the Momento cache before being returned.

3. The cache uses a combination of the prompt and LLM key to create unique cache keys.
4. The cached data is stored in Momento’s distributed cache, making it accessible across multiple sessions and instances.

Use Cases

• Improving response times for frequently asked questions across multiple application instances
• Reducing API costs by minimizing redundant LLM calls in distributed systems
• Enhancing user experience in scalable chatbots or AI assistants with quicker responses
• Optimizing performance in scenarios with repetitive queries across different user sessions

Special Features

• Distributed Caching: Utilizes Momento’s serverless cache for cross-instance caching.
• Scalability: Easily scales with application growth without managing cache infrastructure.
• Persistence: Cache persists beyond individual application sessions.
• Automatic Integration: Works seamlessly within the LLM query flow.
• Configurable TTL: Supports setting Time-To-Live for cached items.

Notes

• Requires a Momento account and API credentials to function.
• The cache persists across application restarts, ensuring continuity of cached responses.
• This caching mechanism is particularly useful for scenarios where the same or similar queries are likely to occur across different sessions or application instances.
• While improving performance, it’s important to consider that cached responses may not reflect real-time changes or updates to the underlying LLM.
• The effectiveness of the cache depends on the nature of the queries and the likelihood of repetition across different users or sessions.

The Momento Cache node provides a powerful solution for optimizing LLM-based applications in distributed or serverless environments. By leveraging Momento’s serverless cache, it offers efficient caching capabilities that can significantly enhance both the performance and cost-effectiveness of AI-driven systems at scale. This node is particularly valuable in applications where quick response times are crucial across multiple instances or where similar queries are likely to occur from different users or sessions.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: >- Cache LLM response in Redis, useful for sharing cache across multiple processes or servers.

Redis Cache

The Redis Cache node provides integration with Redis, a popular in-memory data structure store, for caching LLM (Large Language Model) responses. It offers efficient caching capabilities across multiple sessions and instances.

Parameters

• Credential
  • Type: credential
  • Credential Names: momentoCacheApi
  • Description: The API credentials required to authenticate with the Momento cache service.

Inputs

The node doesn’t require direct input from the user. It integrates into the embedding generation flow automatically.

Output

• The node returns a Text LangchainRedisCache object, which can be used as a cache backend for LangChain operations.

How It Works

1. The node initializes a connection to the Redis server using the provided credentials.
2. When a query is made to an LLM:

• The cache checks if an identical query has been processed before.
• If found, it returns the cached response immediately.
• If not found, the query is processed by the LLM, and the response is stored in the Redis cache before being returned.

3. The cache uses a combination of the prompt and LLM key to create unique cache keys.
4. If a TTL is specified, cached items will expire after the set duration.

Use Cases

• Improving response times for frequently asked questions across multiple application instances
• Reducing API costs by minimizing redundant LLM calls in distributed systems
• Enhancing user experience in scalable chatbots or AI assistants with quicker responses
• Optimizing performance in scenarios with repetitive queries across different user sessions
• Sharing cache across multiple processes or servers

Special Features

• Distributed Caching: Utilizes Redis for cross-instance caching.
• Persistence: Cache can persist beyond individual application sessions.
• Configurable TTL: Supports setting Time-To-Live for cached items.
• Flexible Connection: Supports both direct Redis configuration and URL-based connection.
• SSL Support: Offers secure connections to Redis servers.

Notes

• Requires access to a Redis server, either self-hosted or cloud-based.
• The cache persists across application restarts, ensuring continuity of cached responses.
• This caching mechanism is particularly useful for scenarios where the same or similar queries are likely to occur across different sessions or application instances.
• While improving performance, it’s important to consider that cached responses may not reflect real-time changes or updates to the underlying LLM.
• The effectiveness of the cache depends on the nature of the queries and the likelihood of repetition across different users or sessions.
• The node supports both standalone Redis setups and clustered environments.

The Redis Cache node provides a robust solution for optimizing LLM-based applications in distributed environments. By leveraging Redis’s fast in-memory data store, it offers efficient caching capabilities that can significantly enhance both the performance and cost-effectiveness of AI-driven systems at scale. This node is particularly valuable in applications where quick response times are crucial across multiple instances or where similar queries are likely to occur from different users or sessions, and where sharing cache across multiple processes or servers is beneficial.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: >- Cache LLM response in Redis, useful for sharing cache across multiple processes or servers.

Redis Embeddings Cache

The Redis Embeddings Cache node provides a mechanism to cache generated embeddings using Redis, a high-performance in-memory data store. This node is designed to improve efficiency and reduce computational costs when working with embedding models.

Parameters

• Embeddings (Required)

  • Type: Embeddings
  • Description: The embedding model to be used for generating embeddings.

• Credential

  • Type: credential
  • Credential Names: momentoCacheApi
  • Description: The API credentials required to authenticate with the Momento cache service.

Inputs

• The node doesn’t require direct input from the user. It integrates into the embedding generation flow automatically.

Output

• The node returns a Text CacheBackedEmbeddings object, which wraps the original embedding model with Redis-based caching functionality.

How It Works

1. The node initializes a connection to the Redis server using the provided credentials.
2. When a query is made to an LLM:

• The cache checks if an identical query has been processed before.
• If found, it returns the cached response immediately.
• If not found, the query is processed by the LLM, and the response is stored in the Redis cache before being returned.

3. The cache uses a combination of the prompt and LLM key to create unique cache keys.
4. If a TTL is specified, cached items will expire after the set duration.

Use Cases

• Improving response times for applications that frequently generate embeddings
• Reducing API costs by minimizing redundant embedding generation calls
• Enhancing performance in scenarios with repetitive text inputs or similar queries
• Optimizing vector search operations by caching frequently used embeddings
• Sharing cached embeddings across multiple processes or servers

Special Features

• Distributed Caching: Utilizes Redis for cross-instance caching of embeddings.
• Configurable TTL: Supports setting Time-To-Live for cached embeddings.
• Namespace Support: Allows for organization of multiple caches within the same Redis instance.
• Flexible Connection: Supports both direct Redis configuration and URL-based connection.
• SSL Support: Offers secure connections to Redis servers.
• LangChain Integration: Built on top of LangChain’s CacheBackedEmbeddings for reliability and consistency.

Notes

• Requires access to a Redis server, either self-hosted or cloud-based.
• The cache persists across application restarts, ensuring continuity of cached embeddings.
• This caching mechanism is particularly useful for scenarios where the same or similar texts are likely to be embedded multiple times across different sessions or application instances.
• While improving performance, it’s important to consider Redis memory usage, especially for large numbers of unique embeddings.
• The effectiveness of the cache depends on the nature of the embedding requests and the likelihood of repetition across different users or sessions.
• Supports both standalone Redis setups and clustered environments.

The Redis Embeddings Cache node provides a powerful solution for optimizing embedding-based applications in distributed environments. By leveraging Redis’s fast in-memory data store, it offers efficient caching capabilities that can significantly enhance both the performance and cost-effectiveness of systems that rely heavily on embeddings. This node is particularly valuable in applications where quick embedding generation is crucial across multiple instances or where similar texts are likely to be processed multiple times from different users or sessions.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: Cache LLM response in Upstash Redis, serverless data for Redis and Kafka.

Upstash Redis Cache

The Upstash Redis Cache node provides integration with Upstash Redis, a serverless Redis service, for caching LLM (Large Language Model) responses. It offers efficient, scalable caching capabilities suitable for serverless and edge computing environments.

Parameters

• Credential
  • Type: credential
  • Credential Names: upstashRedisApi
  • Description: The credentials required to connect to the Upstash Redis service.

Input

• The node doesn’t require direct input from the user. It integrates into the LLM query flow automatically.

Output

• The node initializes and returns an Upstash Redis Cache instance that can be used as a caching backend for LLM operations.

How It Works

1. The node establishes a connection to the Upstash Redis service using the provided credentials.
2. When a query is made to an LLM:

• The cache checks if an identical query has been processed before.
• If found, it returns the cached response immediately.
• If not found, the query is processed by the LLM, and the response is stored in the Upstash Redis cache before being returned.

3. The cache uses a combination of the prompt and LLM key to create unique cache keys.
4. Cached data is stored in Upstash’s serverless Redis, making it accessible across multiple serverless function invocations or edge locations.

Use Cases

• Improving response times for LLM queries in serverless architectures
• Reducing API costs by minimizing redundant LLM calls in edge computing scenarios
• Enhancing user experience in globally distributed applications with quicker responses
• Optimizing performance in scenarios with repetitive queries across different serverless function invocations
• Implementing efficient caching for LLM-based chatbots or AI assistants deployed on edge networks

Special Features

• Serverless Caching: Utilizes Upstash’s serverless Redis for scalable, managed caching.
• Global Distribution: Supports caching across multiple regions for low-latency access.
• Automatic Scaling: Scales automatically with application demand without managing infrastructure.
• Persistence: Cache persists beyond individual function invocations or application restarts.
• Compatibility: Works seamlessly with serverless platforms and edge computing environments.

Notes

• Requires an Upstash account and API credentials to function.
• Ideal for serverless and edge computing scenarios where traditional Redis setups might be challenging.
• The cache persists across function invocations, ensuring continuity of cached responses in serverless environments.
• This caching mechanism is particularly useful for scenarios where the same or similar queries are likely to occur across different regions or function invocations.
• While improving performance, it’s important to consider that cached responses may not reflect real-time changes or updates to the underlying LLM.
• The effectiveness of the cache depends on the nature of the queries and the likelihood of repetition across different users or sessions.

The Upstash Redis Cache node provides a powerful solution for optimizing LLM-based applications in serverless and edge computing environments. By leveraging Upstash’s serverless Redis service, it offers efficient caching capabilities that can significantly enhance both the performance and cost-effectiveness of AI-driven systems at global scale. This node is particularly valuable in applications where quick response times are crucial across multiple regions or where similar queries are likely to occur from different serverless function invocations or edge locations.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: LangChain Chain Nodes

Chains

In the context of chatbots and large language models, "chains" typically refer to sequences of text or conversation turns. These chains are used to store and manage the conversation history and context for the chatbot or language model. Chains help the model understand the ongoing conversation and provide coherent and contextually relevant responses.

Here's how chains work:

1. Conversation History: When a user interacts with a chatbot or language model, the conversation is often represented as a series of text messages or conversation turns. Each message from the user and the model is stored in chronological order to maintain the context of the conversation.
2. Input and Output: Each chain consists of both user input and model output. The user's input is usually referred to as the "input chain," while the model's responses are stored in the "output chain." This allows the model to refer back to previous messages in the conversation.
3. Contextual Understanding: By preserving the entire conversation history in these chains, the model can understand the context and refer to earlier messages to provide coherent and contextually relevant responses. This is crucial for maintaining a natural and meaningful conversation with users.
4. Maximum Length: Chains have a maximum length to manage memory usage and computational resources. When a chain becomes too long, older messages may be removed or truncated to make room for new messages. This can potentially lead to loss of context if important conversation details are removed.
5. Continuation of Conversation: In a real-time chatbot or language model interaction, the input chain is continually updated with the user's new messages, and the output chain is updated with the model's responses. This allows the model to keep track of the ongoing conversation and respond appropriately.

Chains are a fundamental concept in building and maintaining chatbot and language model conversations. They ensure that the model has access to the context it needs to generate meaningful and context-aware responses, making the interaction more engaging and useful for users.

Chain Nodes:

• GET API Chain
• OpenAPI Chain
• POST API Chain
• Conversation Chain
• Conversational Retrieval QA Chain
• LLM Chain
• Multi Prompt Chain
• Multi Retrieval QA Chain
• Retrieval QA Chain
• Sql Database Chain
• Vectara QA Chain
• VectorDB QA Chain

description: Chain to run queries against GET API.

GET API Chain

The GET API Chain node is designed to run queries against GET APIs. It constructs API URLs based on given documentation and user questions, then processes the API responses to answer queries.

Parameters

• Language Model (Required)

  • Type: BaseLanguageModel
  • Description: The language model used to generate API URLs and process responses.

• API Documentation (Required)

  • Type: string
  • Description: Description of how the API works. Should include details about endpoints, parameters, and response formats.
  • Rows: 4

• Headers (Optional)

  • Type: json
  • Description: Headers to be included in the API request.
  • Additional Params: true

• URL Prompt (Optional)

  • Type: string
  • Description: Prompt used to tell LLMs how to construct the URL.
  • Default: A predefined prompt template for URL construction.
  • Rows: 4
  • Additional Params: true

• Answer Prompt (Optional)

  • Type: string
  • Description: Prompt used to tell LLMs how to process the API response.
  • Default: A predefined prompt template for answer generation.
  • Rows: 4
  • Additional Params: true

Input

A string containing the user’s question or query about the API.

Output

A string containing the answer to the user’s question, based on the API response.

How It Works

1. The chain receives a user question about the API.
2. It uses the language model and the URL prompt to generate an appropriate API URL based on the API documentation and the question.
3. The chain makes a GET request to the constructed URL, including any specified headers.
4. Upon receiving the API response, it uses the language model and the answer prompt to generate a human-readable answer to the original question.
5. The final answer is returned as output.

Use Cases

• Querying external APIs to answer user questions
• Automating API interactions in chatbots or virtual assistants
• Simplifying complex API documentation for end-users
• Creating natural language interfaces for data retrieval from APIs
• Integrating multiple API sources to answer complex queries

Special Features

• Dynamic URL Generation: Constructs API URLs based on natural language questions.
• Flexible API Documentation: Can work with various APIs by providing appropriate documentation.
• Customizable Prompts: Allows fine-tuning of URL construction and answer generation processes.
• Header Support: Enables authentication and other custom headers for API requests.
• Language Model Integration: Leverages advanced language models for intelligent API interaction. ​

Notes

• The quality of results depends significantly on the completeness and accuracy of the provided API documentation.
• Custom URL and answer prompts can be used to optimize the chain for specific APIs or use cases.
• The chain is designed for GET requests only and may not be suitable for APIs requiring other HTTP methods.
• Proper error handling should be implemented when using this chain in production environments.
• The effectiveness of the chain can vary depending on the complexity of the API and the nature of the user queries.

The GET API Chain node provides a powerful tool for creating natural language interfaces to GET APIs. By combining API documentation, language models, and customizable prompts, it enables developers to build sophisticated systems that can interact with external APIs based on user queries. This node is particularly valuable in scenarios where you want to provide a user-friendly interface to complex API systems or integrate API-based data retrieval into conversational AI applications.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: Chain that automatically select and call APIs based only on an OpenAPI spec.

OpenAPI Chain

The OpenAI Chain node enables seamless integration with OpenAI's language models, allowing you to generate, process, and analyze text using advanced AI within your workflows. This node is designed for flexible, intelligent interactions with OpenAI's API, supporting a wide range of natural language tasks.

Parameters

1. Chat Model (Required)

• Type: BaseChatModel
• Description: The chat model used for interpreting queries and generating responses.

2. YAML Link (Optional)

• Type: string
• Description: URL link to the OpenAPI specification in YAML format.
• Placeholder: https://api.speak.com/openapi.yaml
• Note: If YAML link is provided, uploaded YAML File will be ignored.

3. YAML File (Optional)

• Type: file
• File Type: .yaml
• Description: Uploaded OpenAPI specification file in YAML format.
• Note: Ignored if YAML link is provided.

4. Headers (Optional)

• Type: json
• Description: Additional headers to be included in API requests.
• Additional Params: true

5. Input Moderation (Optional)

• Type: Moderation[]
• Description: Moderation tools to detect and prevent harmful input.
• Additional Params: true

How It Works

1. The chain loads the OpenAPI specification from either the provided YAML link or uploaded file.
2. It receives a user query or instruction.
3. The chat model interprets the query to determine which API endpoint and method to use.
4. The chain constructs the appropriate API request, including any necessary parameters or headers.
5. It executes the API call and receives the response.
6. The chat model then interprets the API response and generates a human-readable answer to the original query.
7. The final response is returned, potentially including both the raw API data and the interpreted answer.

Notes

• The effectiveness of the chain depends on the quality and completeness of the OpenAPI specification.
• It’s important to ensure that the provided chat model is capable of understanding and working with API concepts.
• The chain supports both YAML and JSON formats for OpenAPI specifications, but YAML is preferred.
• When using file upload, ensure that the YAML file is properly formatted and complete.
• The chain can handle complex API structures, including nested objects and arrays in requests and responses.
• For optimal performance, it’s recommended to use a chat model that has been fine-tuned or trained on API-related tasks.

The OpenAPI Chain node provides a powerful solution for creating intelligent, natural language interfaces to API systems defined by OpenAPI specifications. By leveraging advanced language models and standardized API descriptions, it enables developers to quickly build sophisticated applications that can interact with complex APIs based on simple user queries. This node is particularly valuable in scenarios where you want to provide easy access to API functionality without requiring users to understand the technical details of API operation.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: Chain to run queries against POST API.

POST API Chain

The POST API Chain node is designed to interact with APIs that require POST requests. It enables users to send data to external APIs by constructing request bodies and URLs based on provided documentation and user input, then processes the API responses to generate answers.

Parameters

• Language Model (Required)

  • Type: BaseLanguageModel
  • Description: The language model used to generate API URLs, request bodies, and process responses.

• API Documentation (Required)

  • Type: string
  • Description: Description of how the API works. Should include details about endpoints, parameters, request/response formats, and body schema.
  • Rows: 4

• Headers (Optional)

  • Type: json
  • Description: Headers to be included in the API request (e.g., authentication, content-type).
  • Additional Params: true

• URL Prompt (Optional)

  • Type: string
  • Description: Prompt used to tell LLMs how to construct the URL.
  • Default: A predefined prompt template for URL construction.
  • Rows: 4
  • Additional Params: true

• Answer Prompt (Optional)

  • Type: string
  • Description: Prompt used to tell LLMs how to process the API response.
  • Default: A predefined prompt template for answer generation.
  • Rows: 4
  • Additional Params: true

How It Works

1. The chain receives a user question about the API.
2. It uses the language model and the URL prompt to generate an appropriate API URL based on the API documentation and the question.
3. The chain uses the body prompt to generate the required request body (payload) for the POST request.
4. It makes a POST request to the constructed URL, including any specified headers and the generated request body.
5. Upon receiving the API response, it uses the language model and the answer prompt to generate a human-readable answer to the original question.
6. The final answer is returned as output.

Use Cases

• Creating or updating resources via external APIs
• Automating data submission tasks in chatbots or virtual assistants
• Simplifying complex API documentation for end-users
• Enabling natural language interfaces for data creation or modification
• Integrating multiple API sources to automate workflows requiring POST requests

it empowers developers to build systems that can interact with external APIs for data creation, updates, or other POST operations based on user instructions.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

description: Chat models specific conversational chain with memory.

Conversation Chain

The Conversation Chain is a specialized chain designed for maintaining coherent, context-aware conversations using chat-based language models. It integrates memory components to retain conversation history, allowing for more natural and contextually relevant interactions.

Parameters

• Chat Model (Required)

  • Type: BaseChatModel
  • Description: The chat-based language model used for generating responses.

• Memory (Required)

  • Type: BaseMemory
  • Description: The memory component used to store and retrieve conversation history.

• Chat Prompt Template (Optional)

  • Type: ChatPromptTemplate
  • Description: Custom prompt template for the conversation. Must include variable in the human message.
  • Additional Params: true

• Input Moderation (Optional)

  • Type: Moderation[]
  • Description: Moderation tools to detect and prevent harmful input.
  • Additional Params: true

• System Message (Optional)

  • Type: string
  • Description: Custom system message to set the behavior of the AI assistant.
  • Default: A predefined system message template.
  • Rows: 4
  • Additional Params: true

How It Works

1. The chain receives a user input.
2. If input moderation is enabled, it checks the input for potential harmful content.
3. It retrieves the conversation history from the memory component.
4. The chat prompt template (or default if not provided) is populated with the conversation history and current input.
5. The populated prompt is sent to the chat model for processing.
6. The model generates a response based on the prompt and conversation context.
7. The response is returned as output.
8. The conversation history in the memory component is updated with the new interaction.

Use Cases

• Building conversational AI assistants or chatbots
• Creating interactive storytelling or role-playing experiences
• Developing personalized tutoring or coaching systems
• Implementing customer support chatbots with context retention
• Designing conversational interfaces for complex applications

Notes

• The quality and coherence of conversations heavily depend on the capabilities of the chosen chat model.
• Custom chat prompt templates can significantly influence the conversation style and flow.
• The system message can be used to set specific personality traits or knowledge domains for the AI.
• For multi-turn conversations, ensure that the memory component is properly configured to retain necessary context.
• The chain supports both text-only and multi-modal (text + image) inputs, depending on the chat model’s capabilities.
• Proper error handling should be implemented, especially for potential API failures or moderation issues.
• The effectiveness of the chain can vary based on the complexity of the conversation and the specific use case.

The Conversation Chain node provides a powerful foundation for building sophisticated, context-aware conversational AI systems. By combining advanced chat models with flexible memory components and customizable prompts, it enables the creation of natural, engaging, and persistent conversational experiences. This node is particularly valuable in scenarios where maintaining conversation context, personality consistency, and interactive responsiveness are crucial, such as in virtual assistants, interactive storytelling, or personalized customer engagement systems.

{% hint style="info" %} This section is a work in progress. We appreciate any help you can provide in completing this section. Please check our Contribution Guide to get started. {% endhint %}

Conversational Retrieval QA Chain

A chain for performing question-answering tasks with a retrieval component.

Definitions

A retrieval-based question-answering chain, which integrates with a retrieval component and allows you to configure input parameters and perform question-answering tasks.
Retrieval-Based Chatbots: Retrieval-based chatbots are chatbots that generate responses by selecting pre-defined responses from a database or a set of possible responses. They "retrieve" the most appropriate response based on the input from the user.
QA (Question Answering): QA systems are designed to answer questions posed in natural language. They typically involve understanding the question and searching for or generating an appropriate answer.