Active Topological Learning (ATL): A Geometric Framework for Targeted Knowledge Injection in Small Language Models
Tania Swanepoel
Zenodo (CERN European Organization for Nuclear Research)
Problems Identified (5)
Targeted knowledge injection: The paper addresses the problem of inserting targeted conceptual knowledge into pre-trained small language models.
Fine-tuning resource cost: Current fine-tuning paradigms require large datasets and significant compute.
Fine-tuning knowledge degradation: Existing fine-tuning approaches risk degrading knowledge already present in the model.
Resource-constrained specialization: The abstract raises the challenge of enabling on-demand model specialization in resource-constrained deployments.
Targeted knowledge injection: The paper addresses the problem of inserting targeted conceptual knowledge into pre-trained small language models.
Proposed Solutions (5)
Active Topological Learning: ATL is proposed as a geometric methodology for targeted conceptual knowledge insertion into SLM embedding spaces.
Geometric micro-corpus training: The approach designs a geometrically derived micro-corpus and trains along identified load-bearing dimensions using anisotropic gradient masking.
Conceptual void detection: The method detects conceptual voids and measures existing manifold structure before knowledge injection.
Active Topological Learning: ATL is proposed as a geometric methodology for targeted conceptual knowledge insertion into SLM embedding spaces.
Geometric micro-corpus training: The approach designs a geometrically derived micro-corpus and trains along identified load-bearing dimensions using anisotropic gradient masking.
Results (3)
Fast concept localization:
No catastrophic forgetting:
Naive-query discoverability:
Research Domain
Small language models; embedding-space knowledge injection