Aeca Blog

We explain the process of data collection and processing, search, and service development for product search using Aeca. Learn how to index when structured and unstructured data are mixed, and how to transform queries for search using LLM.

You can easily create RAG (Retrieval Augmented Generation) with just one AI database without complex infrastructure setup.

Methods to overcome the limitations of Large Language Models (LLMs) by utilizing Vector Databases (VectorDBs) are gaining attention. To provide accurate answers on specialized information such as law firm case precedents or company communication records—domain data that is not included in the training data—we can use a Vector Database that can convert, store, and search all kinds of data into vector embeddings, serving as a long-term memory storage for LLMs. To illustrate this, we examine a concrete case of how a vector database can complement an LLM through processes like data preprocessing, vectorization, storage, and search, using a Q&A system based on Wikipedia.

Many services are introducing recommendation systems to increase user retention time in modern applications, and this is an important factor directly related to sales, especially in content and e-commerce sectors. Recommendation systems analyze user behavior to understand their interests and provide related items, thereby increasing retention time and inducing purchases. How can vector databases be utilized in this context?

The mobile applications and web services we use have search functions. Most are developed using basic text search provided by databases or full-text search provided by search engines like Elasticsearch. Full-Text Search is one of the traditional methods mainly used for searching text data, focusing on finding specific keywords, words, phrases, etc., in documents, web pages, databases, and more. It typically involves inputting keywords or short sentences to search text data and finding documents that match the keywords, but it does not consider context or semantic similarity.

Vector embedding is a concept that converts various forms of data (documents, images, audio, video, etc.) into arrays of numbers to measure similarity. For example, colors can be represented as three-dimensional vector data in RGB format. By calculating the distance between these vector embeddings, we can determine the similarity between data. This plays an important role in natural language processing, recommendation algorithms, and more. Various data can be converted into vectors through Transformer models, allowing us to measure the similarity between different types of data. For instance, it is possible to measure the similarity between the text "cat" and a picture of a cat in vector space.