🚀 OpenMythos: Unlocking the Secrets of Recurrent-Depth Transformers | 揭秘遞歸深度 Transformer 的推理之鑰

[CN] 在當前 LLM 的開發中，開發者面臨著一個巨大的痛點：如何在不增加參數總量的情況下，提升模型處理複雜推理的能力？傳統的 Transformer 依賴於堆疊數百層不同的權重，這導致了巨大的顯存壓力與計算冗餘。OpenMythos 旨在擊碎這一困境，它基於對 Claude Mythos 架构的理論重建，探索『遞歸深度 Transformer (RDT)』的可能性——讓模型在單次前向傳播中通過權重循環來『深思』，而非單純依靠增加層數。這正是當前 AI 領域追求『計算自適應推理』的核心熱點。

[EN] In current LLM development, developers face a massive pain point: how to enhance a model's complex reasoning capabilities without exponentially increasing the total parameter count? Traditional Transformers rely on stacking hundreds of unique layers, leading to immense memory pressure and computational redundancy. OpenMythos shatters this limitation by theoretically reconstructing the Claude Mythos architecture, exploring the potential of Recurrent-Depth Transformers (RDT). It enables the model to 'think deeper' via weight loops within a single forward pass rather than simply adding more layers, hitting the current AI hotspot of 'compute-adaptive reasoning'.

🛠️ 核心功能 | Key Features

• [CN] 遞歸深度架構 (RDT)： 解決了傳統模型『層數固定』的僵化問題，通過 Prelude $\rightarrow$ Recurrent Block $\rightarrow$ Coda 的設計，實現動態推理深度。
• [EN] Recurrent-Depth Architecture (RDT): Solves the rigidity of 'fixed layer counts' in traditional models, implementing a Prelude $\rightarrow$ Recurrent Block $\rightarrow$ Coda design for dynamic reasoning depth.
• [CN] 可切換的注意力機制： 內建 MLA 與 GQA，讓開發者能快速對比不同注意力機制對記憶體與性能的影響，無需手動重寫底層邏輯。
• [EN] Switchable Attention Mechanisms: Built-in MLA and GQA, allowing developers to quickly compare the impact of different attention mechanisms on memory and performance without rewriting low-level logic.
• [CN] 稀疏 MoE 專家系統： 結合路由專家與共享專家，解決了模型在提升能力時計算成本激增的痛點。
• [EN] Sparse MoE Expert System: Combines routed and shared experts, solving the pain point of skyrocketing computational costs when scaling model capabilities.

💡 技術亮點 | Why It Matters

• [CN] 隱藏式潛在空間推理： 不同於 Chain-of-Thought (CoT) 需要輸出中間 Token，OpenMythos 在潛在空間內靜默循環，極大地提升了推理效率並減少了輸出冗餘。
• [EN] Hidden Latent Space Reasoning: Unlike Chain-of-Thought (CoT) which requires outputting intermediate tokens, OpenMythos loops silently within the latent space, drastically improving reasoning efficiency and reducing output redundancy.
• [CN] 第一原理重建： 它不僅僅是一個庫，而是一個基於公開研究文獻的理論實作，為開發者提供了一個低門檻的實驗環境來研究『深度可變』模型。
• [EN] First-Principles Reconstruction: More than just a library, it is a theoretical implementation based on public research, providing developers with a low-barrier experimental environment to study 'depth-variable' models.

📦 快速上手 | Quick Start

[CN] 快速啟動步驟：
1. 安裝庫：pip install open-mythos
2. 配置參數：定義 MythosConfig（可選擇 mla 或 gqa）。
3. 初始化模型：model = OpenMythos(cfg)
4. 執行推理：調用 model.generate(ids, n_loops=8) 即可體驗遞歸深度推理。

[EN] Quick Start Steps:
1. Install the library: pip install open-mythos
2. Configure parameters: Define MythosConfig (choose between mla or gqa).
3. Initialize model: model = OpenMythos(cfg)
4. Execute inference: Call model.generate(ids, n_loops=8) to experience recurrent-depth reasoning.

[CN] OpenMythos 將複雜的理論論文轉化為可執行的代碼，將開發者從繁瑣的底層架構實現中解放出來，讓研究焦點回歸到『如何優化推理深度』。如果你想探索下一代 LLM 的架構演進，這是一個絕佳的起點。立即訪問 GitHub 加入社群。

[EN] OpenMythos transforms complex theoretical papers into executable code, liberating developers from the tedious implementation of low-level architectures and shifting the focus back to 'optimizing reasoning depth'. If you want to explore the architectural evolution of next-gen LLMs, this is the perfect starting point. Visit GitHub to join the community now.

🔗 View on GitHub