Skip to main content

Reasoning

Core Reasoning Breakthroughs​

  • Language Models Are Few-Shot Learners — Brown et al., 2020. arXiv
  • Language Models Are Zero-Shot Reasoners — Kojima et al., 2022. arXiv
  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — Wei et al., 2022. arXiv
  • Automatic Chain-of-Thought Prompting in Large Language Models — Zhang et al., 2022. arXiv
  • Self-Consistency Improves Chain-of-Thought Reasoning in Language Models — Wang et al., 2022. arXiv
  • Finetuned Language Models Are Zero-Shot Learners — Wei et al., 2021. arXiv
  • STaR: Self-Taught Reasoner — Zelikman et al., 2022. arXiv
  • Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking — Zelikman et al., 2024. arXiv
  • Large Language Models Are Contrastive Reasoners — Yao, 2024. arXiv

Prompt Engineering & Optimization​

  • Language Models Are Human-Level Prompt Engineers — Zhou et al., 2022. arXiv
  • Meta Prompting for AGI Systems — Zhang, 2023. arXiv
  • Large Language Models as Optimizers — Yang et al., 2023. arXiv
  • AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts — Shin et al., 2020. arXiv
  • Prefix-Tuning: Optimizing Continuous Prompts for Generation — Li & Liang, 2021. arXiv
  • Active Prompting with Chain-of-Thought for Large Language Models — Diao et al., 2023. arXiv
  • Guiding Large Language Models via Directional Stimulus Prompting — Li et al., 2024. arXiv
  • Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL — Sun, Hüyük & van der Schaar, ICLR 2024. OpenReview
  • Reinforcement Learning in the Era of LLMs: What Is Essential? — Sun, 2023. arXiv
  • Deep Reinforcement Learning from Human Preferences — Christiano et al., 2017. arXiv

Multi-Step & Tool-Augmented Reasoning​

  • ReAct: Synergizing Reasoning and Acting in Language Models — Yao et al., 2022. arXiv
  • ART: Automatic Multi-Step Reasoning and Tool-Use for Large Language Models — Paranjape et al., 2023. arXiv
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models — Yao et al., 2024. arXiv
  • Large Language Model Guided Tree-of-Thought — Long, 2023. arXiv
  • Graph of Thoughts: Solving Elaborate Problems with Large Language Models — Besta et al., 2023. arXiv
  • MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in LLMs — Liu et al., 2023. arXiv
  • GraphReader: Building Graph-Based Agents to Enhance Long-Context Abilities of LLMs — Li et al., 2024. arXiv
  • GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks — Liu et al., WWW 2023. ACM DL
  • PAL: Program-Aided Language Models — Gao et al., ICML 2023. PMLR

Evaluation & Cognitive Inspiration​

  • Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models — Mondorf & Plank, 2024. arXiv
  • Accessible Summary of GPT-3 Experiments — OpenAI, 2020. PDF
  • Neural Mechanisms of Human Decision-Making — Busemeyer et al., 2020. Springer
  • Neuroscience-Inspired Artificial Intelligence — Hassabis et al., 2017. PubMed
  • Matthew Botvinick’s Neuro-Inspired AI Research — Google Scholar Profile