Agent Memory at Monster Scale with Mem0 and ScyllaDB Cloud

Combine Mem0’s memory management with ScyllaDB’s persistence features to deploy large-scale AI agents

Scaling AI agents to handle persistent memory for a large number of users (e.g. 500k+ DAU) introduces engineering bottlenecks. These complications generally compound across two areas: how context is filtered for the model and how that data is stored globally.

Mem0 addresses context efficiency by processing raw interactions into concise, entity-based facts. Instead of storing entire chat transcripts, it extracts and updates specific user preferences and historical context across sessions. This extraction method reduces the volume of data sent to the LLM, keeping prompt size manageable while retaining long-term user context.

To operate at a global scale, this memory layer requires a database that handles high-throughput, low-latency vector search across multiple regions. ScyllaDB Cloud provides this distributed database infrastructure with high availability and strict P99 guarantees. Combining Mem0’s memory management with ScyllaDB’s persistence features allows teams to deploy large-scale AI agents.

This post shows you how Mem0 and ScyllaDB Cloud can be used together.

Memory: A Core Building Block of Agentic Applications

An agent might run for minutes, make dozens of tool calls, coordinate with other agents, and serve the same user across days or weeks. At each step, it needs context: what did this user say before? What facts are relevant right now? What happened earlier in this run?

Agent memory (semantic memory) covers facts, preferences, and learned context that persist across sessions and agents. “Tom is vegetarian.” “Nicole prefers concise answers.” This layer needs vector search so the agent can retrieve relevant memories by meaning, not just by key lookup.

An agent memory framework solves the application side of this problem. It extracts facts from conversations, stores them as vector embeddings, and retrieves the most relevant ones at query time. You also need a database that the framework will use to store the embeddings and memories. In the case of Mem0, you have plenty to choose from.

Global, large-scale agentic use cases require the memory layer to be highly available, geographically distributed, and scale with the application. This is why choosing the right database backing your memory infrastructure matters as much, or even more, as the memory framework itself.

Another important context layer element is agent state. Agent state covers task progress, intermediate results, checkpoints, and tool outputs. A great stack for agent state is ScyllaDB + LangGraph. You can learn more about that in this post.

What Is Mem0?

Mem0 is an open-source memory layer for AI agents. It sits between your agent code and your storage backend. Mem0 handles the hard parts of agent memory:

• Extraction: given a conversation or a piece of text, Mem0 calls an LLM to extract discrete facts.
• Deduplication: before storing a new fact, Mem0 checks whether a semantically similar memory already exists and updates it instead of creating a duplicate.
• Retrieval: search() embeds a query and runs an ANN lookup to return the most relevant memories for a given user or agent.
• Lifecycle management: add(), get(), get_all(), search(), update(), delete(). A clean API that abstracts the underlying vector store.

Mem0 supports multiple LLM providers (OpenAI, Groq, Anthropic, etc.) and multiple vector store backends. One of those backends is ScyllaDB.

What Is ScyllaDB Cloud?

ScyllaDB is a distributed high-performance database with vector support built for high-throughput, multi-region workloads. ScyllaDB Cloud is a fully managed database-as-a-service. Once connected, the ScyllaDB team handles infrastructure, monitoring, autoscaling, and backups automatically.

The properties that make ScyllaDB Cloud a good fit for an agent memory backend:

• Vector Search: vector<float, N> is a first-class column type. HNSW indexes are created with a single CREATE CUSTOM INDEX statement and support multiple similarity functions.
• Millions of ops/sec: ScyllaDB is designed to handle millions of operations per second with low and predictable latency.
• Horizontal scalability: add new nodes or even new regions to your cluster without complexity.
• Autoscaling capabilities: automatically scales up and down based on real-time load, ensuring consistent performance for unpredictable agentic workloads.
• High availability: data is replicated across nodes automatically. The cluster continues to serve reads and writes even if a node goes down. There is no single point of failure.
• Battle-tested at scale: major enterprises such as Discord, Zillow, and Freshworks depend on ScyllaDB for their data-intensive workloads.
• Trusted for AI: industry leaders such as Tripadvisor, Tubi, and ShareChat rely on ScyllaDB to power their AI-driven workloads.

Serving Global AI Apps with Mem0 and ScyllaDB Cloud

Let’s say you are expecting thousands, hundreds of thousands, and millions of agent loops, messages, and threads daily with hundreds of thousands of concurrent sessions across multiple continents. Your database that powers the memory layer is very likely to become the bottleneck. It’s a good idea to decouple your components — application, LLM, and database — so you can scale each part individually to meet demand. Mem0 and ScyllaDB Cloud work well together to allow this kind of decoupling.

ScyllaDB Cloud offers multi-region support out of the box so you can serve memories from clusters that are closest to your application instance and users.

ScyllaDB Cloud also helps you optimize and scale two distinct kinds of workloads independently:

• Standard key-based lookups (get thread by id)
• Vector search queries (memory semantic search)

In ScyllaDB Cloud, non-vector search queries are served by data nodes. Vector search queries are served by vector search nodes. This design allows you to optimize resources and reduce infrastructure costs.

Decoupling your database from your memory layer lets you scale it on your own terms: add nodes (horizontal scaling) or CPUs (vertical scaling) to absorb load without touching application code or accepting latency regressions.

How ScyllaDB and Mem0 Work Together

Mem0 handles fact extraction, deduplication, embedding, and retrieval. ScyllaDB provides the storage: high-availability, multi-region, low-latency storage with vector search.

On add(), Mem0 sends the input to an LLM to pull out structured facts, embeds each fact, and writes the resulting vectors to ScyllaDB. On search(), Mem0 embeds the query and runs an ANN query against ScyllaDB’s vector index, returning the most semantically relevant memories.

The agent does not manage embeddings or indexes directly; instead, it calls add() and search().

Schema

The schema below creates a memories table with a vector<float, 384> column (matching all-MiniLM-L6-v2) and a vector index for ANN search. Secondary indexes on user_id support get_all() queries.

CREATE KEYSPACE mem0;

CREATE TABLE mem0.memories (
    id         UUID,
    user_id    TEXT,
    agent_id   TEXT,
    run_id     TEXT,
    hash       TEXT,
    data       TEXT,
    metadata   TEXT,
    vector     vector<float, 384>,
    created_at TIMESTAMP,
    updated_at TIMESTAMP,
    PRIMARY KEY (id)
);

CREATE CUSTOM INDEX IF NOT EXISTS memories_vector_idx
    ON mem0.memories (vector)
    USING 'vector_index'
    WITH OPTIONS = {
        'similarity_function': 'COSINE'
    };

CREATE INDEX IF NOT EXISTS memories_user_id_idx
    ON mem0.memories (user_id);

ScyllaDB and Mem0 Configuration

ScyllaDB is CQL (Cassandra Query Language) compatible. You can integrate ScyllaDB with Mem0 using the following configuration:

import os
from dotenv import load_dotenv
from cassandra.policies import DCAwareRoundRobinPolicy
from mem0 import Memory

load_dotenv()

config = {
    "vector_store": {
        "provider": "cassandra",
        "config": {
            "contact_points": [
                os.environ["SCYLLADB_ADDRESS"]
            ],
            "port": 9042,
            "username": os.environ["SCYLLADB_USERNAME"],
            "password": os.environ["SCYLLADB_PASSWORD"],
            "keyspace": "mem0",
            "collection_name": "memories",
            # DC-aware routing
            "load_balancing_policy": DCAwareRoundRobinPolicy(
                local_dc=os.environ.get(
                    "SCYLLADB_DATACENTER",
                    "AWS_US_EAST_1"
                )
            ),
            # matches all-MiniLM-L6-v2
            "embedding_model_dims": 384,
        },
    },
    "llm": {
        "provider": "groq",
        "config": {
            "model": "mllama-3.1-70b-versatile",
            "api_key": os.environ["GROQ_API_KEY"],
        },
    },
    "embedder": {
        "provider": "huggingface",
        "config": {
            "model": "sentence-transformers/all-MiniLM-L6-v2"
        },
    },
}

m = Memory.from_config(config)

After configuration, you can start using the Mem0 functions: add(), search(), update(), delete().

Adding Memories

USER_ID = "alice"

# Add a single fact
m.add(
    "I love hiking in the mountains, especially in autumn.",
    user_id=USER_ID,
)

# Add a conversation — Mem0 extracts facts automatically
m.add(
    [
        {
            "role": "user",
            "content": "What's a good pasta recipe?",
        },
        {
            "role": "assistant",
            "content": (
                "Try cacio e pepe — "
                "it only needs three ingredients."
            ),
        },
        {
            "role": "user",
            "content": "I'm vegetarian, by the way.",
        },
    ],
    user_id=USER_ID,
)

Mem0 calls the LLM to extract facts from the conversation (“user is vegetarian”, “user was recommended cacio e pepe”), embeds each fact using sentence-transformers as defined in the embedder configuration, and writes them to ScyllaDB. If a semantically equivalent memory already exists for this user, Mem0 updates it instead.

Searching and Retrieving Memories

# Semantic search
results = m.search(
    "outdoor activities",
    filters={"user_id": USER_ID},
    limit=3,
)
for entry in results["results"]:
    print(f"[{entry['score']:.3f}]  {entry['memory']}")
# [0.91]  User loves hiking in the mountains, especially in autumn.

# List all memories for a user
all_memories = m.get_all(filters={"user_id": USER_ID})

# Update or delete a specific memory
m.update(
    all_memories["results"][0]["id"],
    "I love hiking and trail running in the mountains.",
)
m.delete(all_memories["results"][-1]["id"])

In an agentic loop, call m.search(user_query, filters={"user_id": user_id}) at the start of each turn to inject relevant context into the prompt, and m.add(conversation_turn, user_id=user_id) at the end to persist what was learned.

Try ScyllaDB + Mem0

The full example code is available in the repository. To get started, provision a ScyllaDB Cloud cluster at cloud.scylladb.com.

Resources:

• ScyllaDB Cloud free trial
• Mem0 documentation
• ScyllaDB Vector Search docs
• scylla-driver on PyPI