calute.memory.storage

calute.memory.storage#

Storage backends for Calute memory system.

class calute.memory.storage.FileStorage(storage_dir: str = '.calute_memory')[source]#

Bases: MemoryStorage

File-based persistent storage using pickle serialisation.

Each key-value pair is stored as a separate .pkl file named by the MD5 hash of its key. An _index.json file tracks the key-to-filename mapping. Suitable for moderate-sized datasets that need persistence across process restarts.

storage_dir#: Path to the directory holding pickle files and the index.

_index#: Dictionary mapping string keys to their pickle filenames.

Example

>>> from calute.memory.storage import FileStorage
>>> fs = FileStorage("/tmp/my_memory")
>>> fs.save("key1", {"data": 42})
True
>>> fs.load("key1")
{'data': 42}

clear() → int[source]#

Clear all pickle files and reset index.

Returns: Number of items cleared

delete(key: str) → bool[source]#

Delete a pickle file and its index entry.

Parameters: key – Unique identifier to delete
Returns: True if key was found and deleted, False otherwise

exists(key: str) → bool[source]#

Check if key exists in the index.

Parameters: key – Unique identifier to check
Returns: True if key exists, False otherwise

list_keys(pattern: str | None = None) → list[str][source]#

List all stored keys, optionally filtered by pattern.

Parameters: pattern – Optional substring pattern to filter keys
Returns: List of matching key strings

load(key: str) → Any | None[source]#

Load data from a pickle file.

Parameters: key – Unique identifier to retrieve
Returns: Stored data if found, None otherwise

save(key: str, data: Any) → bool[source]#

Save data to a pickle file.

Parameters

key – Unique identifier for the data
data – Data to store (must be pickle-serializable)

Returns

True if save was successful, False on error

class calute.memory.storage.MemoryStorage[source]#

Bases: ABC

Abstract base class for memory storage backends.

Defines the common key-value interface that all Calute storage implementations must provide. Concrete subclasses include SimpleStorage (in-memory), FileStorage (pickle files), SQLiteStorage (database), and RAGStorage (vector-enhanced wrapper).

Subclasses must implement save(), load(), delete(), exists(), list_keys(), and clear().

abstract clear() → int[source]#

Clear all stored data.

Returns: Number of items cleared

abstract delete(key: str) → bool[source]#

Delete data by key.

Parameters: key – Unique identifier to delete
Returns: True if deletion was successful, False if key not found

abstract exists(key: str) → bool[source]#

Check if key exists in storage.

Parameters: key – Unique identifier to check
Returns: True if key exists, False otherwise

abstract list_keys(pattern: str | None = None) → list[str][source]#

List all stored keys, optionally filtered by pattern.

Parameters: pattern – Optional substring pattern to filter keys
Returns: List of matching key strings

abstract load(key: str) → Any | None[source]#

Load data by key.

Parameters: key – Unique identifier to retrieve
Returns: Stored data if found, None otherwise

abstract save(key: str, data: Any) → bool[source]#

Save data with a key.

Parameters

key – Unique identifier for the data
data – Data to store

Returns

True if save was successful, False otherwise

class calute.memory.storage.RAGStorage(backend: calute.memory.storage.MemoryStorage | None = None, embedding_model: str | None = None, embedding_api_key: str | None = None)[source]#

Bases: MemoryStorage

RAG-enhanced storage with vector similarity search capabilities.

Wraps another MemoryStorage backend and augments it with dense vector embeddings for semantic similarity search via search_similar(). Supports multiple embedding backends:

TF-IDF (default) – no external dependencies, hash-based 256-dimensional vectors.
sentence-transformers – dense embeddings from a local model (requires the sentence-transformers package).
OpenAI – remote embeddings via the OpenAI API (requires the openai package and a valid API key).

backend#: The underlying MemoryStorage that handles actual data persistence.

embeddings#: Dictionary mapping storage keys to their computed embedding vectors.

Example

>>> from calute.memory.storage import RAGStorage, SimpleStorage
>>> rag = RAGStorage(SimpleStorage(), embedding_model="tfidf")
>>> rag.save("doc1", "The cat sat on the mat")
True
>>> results = rag.search_similar("feline sitting", limit=5)

clear() → int[source]#

Clear all data and embeddings.

Returns: Number of items cleared

delete(key: str) → bool[source]#

Delete data and its embedding.

Parameters: key – Unique identifier to delete
Returns: True if key was found and deleted, False otherwise

exists(key: str) → bool[source]#

Check if key exists in backend storage.

Parameters: key – Unique identifier to check
Returns: True if key exists, False otherwise

list_keys(pattern: str | None = None) → list[str][source]#

List keys from backend storage.

Parameters: pattern – Optional substring pattern to filter keys
Returns: List of matching key strings

load(key: str) → Any | None[source]#

Load data from backend storage.

Parameters: key – Unique identifier to retrieve
Returns: Stored data if found, None otherwise

save(key: str, data: Any) → bool[source]#

Save data with computed embedding.

Parameters

key – Unique identifier for the data
data – Data to store (embeddings computed for str/dict types)

Returns

True if save was successful, False otherwise

search_similar(query: str, limit: int = 10, threshold: float = 0.0) → list[tuple[str, float, Any]][source]#

Search for items similar to the query.

Parameters

query – Search query text
limit – Maximum number of results to return
threshold – Minimum similarity score (0.0 to 1.0)

Returns

List of tuples (key, similarity_score, data) sorted by similarity

class calute.memory.storage.SQLiteStorage(db_path: str = '.calute_memory/memory.db')[source]#

Bases: MemoryStorage

SQLite-based persistent storage with ACID guarantees.

Uses a local SQLite database for reliable persistent key-value storage. Data is serialised with pickle and stored as BLOBs. When the WRITE_MEMORY environment variable is not set to "1", the backend transparently falls back to a plain in-memory dictionary so that read-only or ephemeral sessions incur no disk I/O.

write_enabled#: True when the WRITE_MEMORY environment variable is "1", enabling actual SQLite persistence.

db_path#: Path to the SQLite database file.

_memory_storage#: In-memory fallback dictionary used when write_enabled is False.

Example

>>> import os
>>> os.environ["WRITE_MEMORY"] = "1"
>>> from calute.memory.storage import SQLiteStorage
>>> store = SQLiteStorage("/tmp/mem.db")
>>> store.save("k", {"v": 1})
True

clear() → int[source]#

Clear all data from storage.

Returns: Number of items cleared

delete(key: str) → bool[source]#

Delete from database or in-memory storage.

Parameters: key – Unique identifier to delete
Returns: True if key was found and deleted, False otherwise

exists(key: str) → bool[source]#

Check if key exists in storage.

Parameters: key – Unique identifier to check
Returns: True if key exists, False otherwise

list_keys(pattern: str | None = None) → list[str][source]#

List all stored keys, optionally filtered by pattern.

Parameters: pattern – Optional substring pattern to filter keys
Returns: List of matching key strings, ordered by creation date (newest first)

load(key: str) → Any | None[source]#

Load data from database or in-memory storage.

Parameters: key – Unique identifier to retrieve
Returns: Stored data if found, None otherwise

save(key: str, data: Any) → bool[source]#

Save data to database or in-memory storage.

Parameters

key – Unique identifier for the data
data – Data to store (must be pickle-serializable)

Returns

True if save was successful, False on error

class calute.memory.storage.SimpleStorage[source]#

Bases: MemoryStorage

Simple in-memory storage (non-persistent).

Provides fast dictionary-backed key-value storage that exists only in memory. All data is lost when the process terminates. Suitable for testing, development, and short-lived applications.

_data#: Internal dictionary holding all stored key-value pairs.

clear() → int[source]#

Clear all data from memory.

Returns: Number of items cleared

delete(key: str) → bool[source]#

Delete data from memory.

Parameters: key – Unique identifier to delete
Returns: True if key was found and deleted, False otherwise

exists(key: str) → bool[source]#

Check if key exists in memory.

Parameters: key – Unique identifier to check
Returns: True if key exists, False otherwise

list_keys(pattern: str | None = None) → list[str][source]#

List all keys, optionally filtered by pattern.

Parameters: pattern – Optional substring pattern to filter keys
Returns: List of matching key strings

load(key: str) → Any | None[source]#

Load data from memory.

Parameters: key – Unique identifier to retrieve
Returns: Stored data if found, None otherwise

save(key: str, data: Any) → bool[source]#

Save data in memory.

Parameters

key – Unique identifier for the data
data – Data to store

Returns

Always returns True as in-memory saves don’t fail

calute.memory.storage

Contents

calute.memory.storage#