https://avatars.githubusercontent.com/u/51735404?s=200&v=4

If you need your database to support FAISS like indexing, Milvus is the way to go.

Installing using docker

curl -sfL <https://raw.githubusercontent.com/milvus-io/milvus/master/scripts/standalone_embed.sh> -o standalone_embed.sh

bash standalone_embed.sh start

Now you can access Milvus at http://localhost:19530

Basic Operations

1. Create record

For creating records I recommend using Langchain as it greatly simplifies the process

pip install -U -q langchain langchain_milvus pymilvus

from langchain_milvus.vectorstores import Milvus
from utils.llm import embeddings

Make sure you have an embeddings model defined

This can be OpenAI Embeddings or your chosen embedding model

Next, prepare your records for insertion

from langchain.docstore.document import Document

doc = Document(page_content="Record text", metadata={"unqiue_id": 1234})

URI = "<http://localhost:19530>" # This is your Milvus URI

vector_db = Milvus.from_documents(
    [doc ,
    embeddings,
    connection_args={"uri": URI},
    collection_name="CollectionName"
)

2. Fetch Record

from pymilvus import connections, Collection, utility

connections.connect(
    alias="default",
    host="localhost",
    port="19530"
)

if not utility.has_collection(collection_name):
    print(f"Collection {collection_name} does not exist.")
else:
    collection = Collection(name=collection_name)

expr = "uniqueID == 1"
# expr = f"uniqueID in [1,2,3]"

query_result = collection.query(
    expr=expr,
    output_fields=["uniqueID ", "vector", "text"],  # Specify the fields you want to retrieve
    limit=10  # Limit the number of items to fetch
)