 |
International Journal for Research Trends and Innovation
International Peer Reviewed & Refereed Journals, Open Access Journal
ISSN Approved Journal No: 2456-3315 | Impact factor: 8.14 | ESTD Year: 2016
Scholarly open access journals, Peer-reviewed, and Refereed Journals, Impact factor 8.14 (Calculate by google scholar and Semantic Scholar | AI-Powered Research Tool) , Multidisciplinary, Monthly, Indexing in all major database & Metadata, Citation Generator, Digital Object Identifier(DOI)
|
Issue: June 2026
Volume 11 | Issue 6
Review Result and Publication of Paper within : 2-3 days
Click Here For more DetailsFor Authors
Forms / Download
Published Issue Details
Editorial Board
Other IMP Links
Facts & Figure
Impact Factor : 8.14
Issue per Year : 12
Volume Published : 11
Issue Published : 121
Article Submitted : 24719
Article Published : 9358
Total Authors : 24865
Total Reviewer : 861
Total Countries : 169
Indexing Partner
Licence
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
|
Published Paper Details
|
|
| Paper Title: | Role-Specific Gradient-Weighted Merging for Drafter–Verifier Pipelines on Small Language Models |
| Authors Name: | Taiyab Mohammad Nabil Ul Haque , Takshil Aggarwal , Nischay Narain Sharma |
| Download E-Certificate: | Download |
| Author Reg. ID: |
IJRTI_212869
|
| Published Paper Id: | IJRTI2605140 |
| Published In: | Volume 11 Issue 5, May-2026 |
| DOI: | |
| Abstract: | We present Role-Specific Gradient-Weighted Merging (RS-GWM), a one-pass method that creates both a drafter and a verifier adapter from a shared pool of task-specific LoRA fine-tunes. We split parameters into two groups: those critical for generating answers and those critical for verifying them. This classification is based on the gradient EMA magnitudes recorded during fine-tuning. The two adapters work together in a simple inference pipeline (GWM-DV-V2). Here, the drafter suggests an answer while the verifier assesses it. If the verifier finds an unsupported draft, it triggers a retry loop. We tested this approach on TinyLlama-1.1B and the GSM8K math benchmark, using 100 test items. The drafter alone achieves 0.72 accuracy with a hallucination rate of 0.18 and an average response time of 2104.87 ms. Using the same drafter in the GWM-DV-V2 pipeline increases accuracy to 0.84 (a 12-point rise), nearly cuts the hallucination rate in half to 0.10, and lowers the false-accept rate from 0.0244 to 0.0115 (a 53% drop). This increase in accuracy costs 1.56 times more in wall-clock latency, with an average of 3284.51 ms compared to 2104.87 ms. We believe this trade-off is reasonable for tasks where incorrect answers are costly. Tests on verifier acceptance thresholds and training data balance show that the verifier is crucial. It keeps the false-accept rate at 0.16 or lower, even in the most lenient conditions, and it remains the key factor for adjusting the precision and recall of the pipeline. |
| Keywords: | Small language models • LoRA fine-tuning • Model merging • Drafter–verifier pipelines • GSM8K • TinyLlama |
| Cite Article: | "Role-Specific Gradient-Weighted Merging for Drafter–Verifier Pipelines on Small Language Models", International Journal for Research Trends and Innovation (www.ijrti.org), ISSN:2456-3315, Vol.11, Issue 5, page no.b325-b330, May-2026, Available :http://www.ijrti.org/papers/IJRTI2605140.pdf |
| Downloads: | 00092 |
| ISSN: |
2456-3315 | IMPACT FACTOR: 8.14 Calculated By Google Scholar| ESTD YEAR: 2016 An International Scholarly Open Access Journal, Peer-Reviewed, Refereed Journal Impact Factor 8.14 Calculate by Google Scholar and Semantic Scholar | AI-Powered Research Tool, Multidisciplinary, Monthly, Multilanguage Journal Indexing in All Major Database & Metadata, Citation Generator |
| Publication Details: |
Published Paper ID: IJRTI2605140
Registration ID:212869
Published In: Volume 11 Issue 5, May-2026
DOI (Digital Object Identifier):
Page No: b325-b330 Country: rohini, delhi, India Research Area: Computer Science & Technology Publisher : IJ Publication Published Paper URL : https://www.ijrti.org/viewpaperforall?paper=IJRTI2605140 Published Paper PDF: https://www.ijrti.org/papers/IJRTI2605140 |
| Share Article: | |
|
Click Here to Download This Article |
|
| Article Preview | |
|
|
|
Major Indexing from www.ijrti.org
| Google Scholar | ResearcherID Thomson Reuters | Mendeley : reference manager | Academia.edu |
| arXiv.org : cornell university library | Research Gate | CiteSeerX | DOAJ : Directory of Open Access Journals |
| DRJI | Index Copernicus International | Scribd | DocStoc |
ISSN Details
 |
 |
|
ISSN: 2456-3315
Impact Factor: 8.14 and ISSN APPROVED,
Journal Starting Year (ESTD) : 2016
DOI (A digital object identifier)
| Providing A digital object identifier by DOI.ONE How to Get DOI? |
Conference
Open Access License Policy
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Important Details
Join RMS/Earn 300
WhatsApp
Click Here
Click Here
Indexing Partner |
|||
| Copyright © 2026 - All Rights Reserved - IJRTI | |||