viking DB

viking DB is a database that stores, indexes, and manages massive embedding vectors generated by deep neural networks and other machine learning (ML) models.

This notebook shows how to use functionality related to the VikingDB vector database.

You'll need to install langchain-community with pip install -qU langchain-community to use this integration

To run, you should have a viking DB instance up and running.

!pip install --upgrade volcengine

We want to use VikingDBEmbeddings so we have to get the VikingDB API Key.

import getpass
import os

os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")

from langchain_community.document_loaders import TextLoader
from langchain_community.vectorstores.vikingdb import VikingDB, VikingDBConfig
from langchain_openai import OpenAIEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter

API Reference:TextLoader | VikingDB | VikingDBConfig | OpenAIEmbeddings | RecursiveCharacterTextSplitter

loader = TextLoader("./test.txt")
documents = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=10, chunk_overlap=0)
docs = text_splitter.split_documents(documents)

embeddings = OpenAIEmbeddings()

db = VikingDB.from_documents(
    docs,
    embeddings,
    connection_args=VikingDBConfig(
        host="host", region="region", ak="ak", sk="sk", scheme="http"
    ),
    drop_old=True,
)

query = "What did the president say about Ketanji Brown Jackson"
docs = db.similarity_search(query)

docs[0].page_content

Compartmentalize the data with viking DB Collections

You can store different unrelated documents in different collections within same viking DB instance to maintain the context

Here's how you can create a new collection

db = VikingDB.from_documents(
    docs,
    embeddings,
    connection_args=VikingDBConfig(
        host="host", region="region", ak="ak", sk="sk", scheme="http"
    ),
    collection_name="collection_1",
    drop_old=True,
)

And here is how you retrieve that stored collection

db = VikingDB.from_documents(
    embeddings,
    connection_args=VikingDBConfig(
        host="host", region="region", ak="ak", sk="sk", scheme="http"
    ),
    collection_name="collection_1",
)

After retrieval you can go on querying it as usual.