​
RAG API

This document outlines the API endpoints for managing Retrieval-Augmented Generation (RAG) components in PySpur.

​
Document Collections

​
Create Document Collection

Description: Creates a new document collection from uploaded files and metadata. The files are processed asynchronously in the background.

URL: /rag/collections/

Method: POST

Form Data:

files: List[UploadFile]  # List of files to upload (optional)
metadata: str  # JSON string containing collection configuration

Where metadata is a JSON string representing:

class DocumentCollectionCreateSchema:
    name: str  # Name of the collection
    description: str  # Description of the collection
    text_processing: ChunkingConfigSchema  # Configuration for text processing

Response Schema:

class DocumentCollectionResponseSchema:
    id: str  # ID of the document collection
    name: str  # Name of the collection
    description: str  # Description of the collection
    status: str  # Status of the collection (processing, ready, failed)
    created_at: str  # When the collection was created (ISO format)
    updated_at: str  # When the collection was last updated (ISO format)
    document_count: int  # Number of documents in the collection
    chunk_count: int  # Number of chunks in the collection
    error_message: Optional[str]  # Error message if processing failed

​
List Document Collections

Description: Lists all document collections.

URL: /rag/collections/

Method: GET

Response Schema:

List[DocumentCollectionResponseSchema]

​
Get Document Collection

Description: Gets details of a specific document collection.

URL: /rag/collections/{collection_id}/

Method: GET

Parameters:

collection_id: str  # ID of the document collection

Response Schema:

class DocumentCollectionResponseSchema:
    id: str  # ID of the document collection
    name: str  # Name of the collection
    description: str  # Description of the collection
    status: str  # Status of the collection (processing, ready, failed)
    created_at: str  # When the collection was created (ISO format)
    updated_at: str  # When the collection was last updated (ISO format)
    document_count: int  # Number of documents in the collection
    chunk_count: int  # Number of chunks in the collection
    error_message: Optional[str]  # Error message if processing failed

​
Delete Document Collection

Description: Deletes a document collection and its associated data.

URL: /rag/collections/{collection_id}/

Method: DELETE

Parameters:

collection_id: str  # ID of the document collection

Response: 200 OK with message

​
Get Collection Progress

Description: Gets the processing progress of a document collection.

URL: /rag/collections/{collection_id}/progress/

Method: GET

Parameters:

collection_id: str  # ID of the document collection

Response Schema:

class ProcessingProgressSchema:
    id: str  # ID of the collection
    status: str  # Status of processing
    progress: float  # Progress percentage (0-100)
    current_step: Optional[str]  # Current processing step
    total_files: Optional[int]  # Total number of files
    processed_files: Optional[int]  # Number of processed files
    total_chunks: Optional[int]  # Total number of chunks
    processed_chunks: Optional[int]  # Number of processed chunks
    error_message: Optional[str]  # Error message if processing failed
    created_at: str  # When processing started (ISO format)
    updated_at: str  # When processing was last updated (ISO format)

​
Add Documents to Collection

Description: Adds documents to an existing collection. The documents are processed asynchronously in the background.

URL: /rag/collections/{collection_id}/documents/

Method: POST

Parameters:

collection_id: str  # ID of the document collection

Form Data:

files: List[UploadFile]  # List of files to upload

Response Schema:

class DocumentCollectionResponseSchema:
    # Same as Get Document Collection

​
Get Collection Documents

Description: Gets all documents and their chunks for a collection.

URL: /rag/collections/{collection_id}/documents/

Method: GET

Parameters:

collection_id: str  # ID of the document collection

Response Schema:

List[DocumentWithChunksSchema]

Where DocumentWithChunksSchema contains:

class DocumentWithChunksSchema:
    id: str  # ID of the document
    title: str  # Title of the document
    metadata: Dict[str, Any]  # Metadata about the document
    chunks: List[DocumentChunkSchema]  # List of chunks in the document

​
Delete Document from Collection

Description: Deletes a document from a collection.

URL: /rag/collections/{collection_id}/documents/{document_id}/

Method: DELETE

Parameters:

collection_id: str  # ID of the document collection
document_id: str  # ID of the document to delete

Response: 200 OK with message

​
Preview Chunk

Description: Previews how a document would be chunked with a given configuration.

URL: /rag/collections/preview_chunk/

Method: POST

Form Data:

file: UploadFile  # File to preview
chunking_config: str  # JSON string containing chunking configuration

Response Schema:

{
    "chunks": List[Dict[str, Any]],  # Preview of chunks
    "total_chunks": int  # Total number of chunks
}

​
Vector Indices

​
Create Vector Index

Description: Creates a new vector index from a document collection. The index is created asynchronously in the background.

URL: /rag/indices/

Method: POST

Request Payload:

class VectorIndexCreateSchema:
    name: str  # Name of the index
    description: str  # Description of the index
    collection_id: str  # ID of the document collection
    embedding: EmbeddingConfigSchema  # Configuration for embedding

Response Schema:

class VectorIndexResponseSchema:
    id: str  # ID of the vector index
    name: str  # Name of the index
    description: str  # Description of the index
    collection_id: str  # ID of the document collection
    status: str  # Status of the index (processing, ready, failed)
    created_at: str  # When the index was created (ISO format)
    updated_at: str  # When the index was last updated (ISO format)
    document_count: int  # Number of documents in the index
    chunk_count: int  # Number of chunks in the index
    embedding_model: str  # Name of the embedding model
    vector_db: str  # Name of the vector database
    error_message: Optional[str]  # Error message if processing failed

​
List Vector Indices

Description: Lists all vector indices.

URL: /rag/indices/

Method: GET

Response Schema:

List[VectorIndexResponseSchema]

​
Get Vector Index

Description: Gets details of a specific vector index.

URL: /rag/indices/{index_id}/

Method: GET

Parameters:

index_id: str  # ID of the vector index

Response Schema:

class VectorIndexResponseSchema:
    # Same as Create Vector Index response

​
Delete Vector Index

Description: Deletes a vector index and its associated data.

URL: /rag/indices/{index_id}/

Method: DELETE

Parameters:

index_id: str  # ID of the vector index

Response: 200 OK with message

​
Get Index Progress

Description: Gets the processing progress of a vector index.

URL: /rag/indices/{index_id}/progress/

Method: GET

Parameters:

index_id: str  # ID of the vector index

Response Schema:

class ProcessingProgressSchema:
    # Same as Get Collection Progress response

​
Retrieve from Index

Description: Retrieves relevant chunks from a vector index based on a query.

URL: /rag/indices/{index_id}/retrieve/

Method: POST

Parameters:

index_id: str  # ID of the vector index

Request Payload:

class RetrievalRequestSchema:
    query: str  # Query to search for
    top_k: Optional[int] = 5  # Number of results to return
    score_threshold: Optional[float] = None  # Minimum score threshold
    semantic_weight: Optional[float] = 1.0  # Weight for semantic search
    keyword_weight: Optional[float] = 0.0  # Weight for keyword search

Response Schema:

class RetrievalResponseSchema:
    results: List[RetrievalResultSchema]  # List of retrieval results
    total_results: int  # Total number of results

Where RetrievalResultSchema contains:

class RetrievalResultSchema:
    text: str  # Text of the chunk
    score: float  # Relevance score
    metadata: ChunkMetadataSchema  # Metadata about the chunk