COMPUTER SCIENCE AND ENGINEERING

NeuroRAG: Polychronous Multimodal Episodic Memory

Sujal Srivastava Published August 15, 2026 CC-BY

Modern enterprise information systems contain heterogeneous streams of information that differ fundamentally in representation, temporal structure, and semantics. Relational databases encode structured attributes and transactions; knowledge graphs encode entities and relations; textual systems encode sequential symbolic information; telemetry and event streams additionally encode temporal dynamics. Conventional memory systems typically transform these modalities into a common vector space and subsequently perform similarity search, graph traversal, or learned re- trieval. Such approaches provide effective retrieval but do not naturally model one-shot associative learning, temporal binding, pattern completion, continual online learning, or biologically inspired memory consolidation. This work proposes a Polychronous Multimodal Episodic Memory (PMEM) architecture based on a spiking neural substrate inspired by the entorhinal cortex (EC), dentate gyrus (DG), hip- pocampal CA3, and CA1. Heterogeneous modalities are independently transformed into tempo- rally structured spike trains and converged into a common entorhinal representation. The DG performs sparse pattern separation, while recurrent CA3 dynamics form autoassociative attractor states capable of completing memories from partial or cross-modal cues. Spike-timing-dependent plasticity (STDP), neuromodulatory gating, synaptic delays, and polychronous convergence pro- vide mechanisms for rapid associative binding. CA1 provides a learned readout from the latent hippocampal state back to modality-specific representations. Subsequent extensions introduce episodic temporal sequences, replay-based consolidation, continual learning, homeostatic plastic- ity, modular scaling, and cross-modal retrieval. The resulting system is not intended to replace relational databases, graph databases, or lan- guage models. Instead, it defines a neural associative memory substrate operating over these systems, in which structured records, graph relations, and textual episodes become temporally en- coded neural events. The central research hypothesis is that multimodal information can be bound through spike timing and recurrent dynamics without requiring a single shared semantic embed- ding or an explicit retrieval index. The current prototype demonstrates encoder reconstruction, CA3 pattern completion, multimodal binding, engram stability, novelty-sensitive plasticity, and continuous two-stage encoding; however, several components remain prototype-level, particularly pure neural retrieval without symbolic assistance, large-scale capacity, semantic text encoding, continual consolidation, and rigorous comparison against modern memory baselines

0 comments

Sign in to join the discussion