End of Day Reports - TheBrain - Yahya T. SuratwalaPreview
Open in TheBrain

Clarification on Data Structure and Openness for RAG Workflows #100006

https://thebrain.zendesk.com/agent/tickets/100006

Hello Kim,

Thank you for the detailed explanation, this helps clarify what you’re aiming to do.

You’re absolutely right that for RAG-style workflows, having clean, predictable access to content for chunking, embedding, and retrieval is critical.

To clarify how TheBrain is structured today:

Your notes and attachments are stored as Markdown and files on disk, so the content itself is accessible in a file-based way, similar in principle to tools like Obsidian.

Where TheBrain differs is in how relationships and structure are handled. Instead of relying purely on inline Markdown links, those relationships are maintained as a structured layer, which enables the dynamic graph behavior inside the application.

So in practice:

If your workflow depends on external systems, TheBrain also provides a REST API that allows programmatic access to Brain data, including thoughts, links, and metadata. This can be used to extract structured information for external processing or AI pipelines.

At the same time, TheBrain’s current direction has also been to enable AI to work directly within this structure. In TheBrain 15, the agentic AI (Cerebro) operates on the graph itself, allowing it to understand relationships and context beyond what a flat file system typically provides.

On your questions:

At the moment, we don’t have any specific plans to announce around changing the underlying architecture or shifting toward a different “native format” over the next few years. The current direction is to continue building on this model, where content remains file-based and accessible, while the relational layer powers both the graph experience and AI capabilities within TheBrain.

That said, your use case is completely valid, and your comparison highlights the trade-off between a pure file-based vault and a structured graph system with built-in context awareness.

To better understand what would make this workable for you, could you share what an ideal structure would look like from your side? For example, what specific limitations are you running into when trying to extract or process TheBrain data for your RAG pipeline?

We can use that to give you a more concrete answer based on your exact requirements.

Best regards,

John