ISSN2456-3315
Impact Factor10.57
UGC CAREApproved
Low Cost₹599 / $45
CrossRefDOI
ESTD2016
Peer ReviewTransparent
AccessOpen
Submission: Always Open
Paper Title
ReBERT- An Enhanced BERT
Authors
Priyanka Shee , Santayo Kundu , Anirban Bhar , Moumita Ghosh
Keywords
BERT, NLP, User-Generated Content (UGC), Syntactic Parsing, Name-Entity Recognition
Abstract
BERT (Bidirectional Encoder Representations from Transformers) is a transformer-based language model that comprehends the context of words by considering surrounding words in both directions. It revolutionized natural language processing by capturing rich contextual information, enhancing performance in various language understanding tasks like sentiment analysis, text classification. In this article, focusing on User Generated Content (UGC) in a resource-scarce scenario, we study the ability of BERT (Devlinet al., 2018) to perform lexical normalization. by enhancing its architecture and by carefully finetuning it, we show that BERT can be a competitive lexical normalization model without the need of any UGC resources aside from 3,000 training sentences. The enhanced BERT model features a hierarchical contextualization module for improved long-range dependency understanding, a domain-specific adaptation layer for specialized language contexts, and efficiency optimization through dynamic attention head pruning and weight sharing. Fine-tuned pre-training broadens language comprehension, while task-specific heads enable fine-tuning. Rigorous evaluation and iterative refinement ensure performance enhancement across tasks, addressing limitations and advancing language understanding. It will be our first work done in adapting and analyzing the ability of this model to handle noisy UGC data.
Full Paper, Downloads & Preview
How To Cite
Choose the style your journal or department asks for, then copy it. Every version below is generated from this paper's own record.
IJRTI — journal style
"ReBERT- An Enhanced BERT", IJRTI - International Journal for Research Trends and Innovation (www.ijrti.org), ISSN:2456-3315, Vol.8, Issue 10, page no.484 - 494, October-2023, Available :https://ijrti.org/papers/IJRTI2310067.pdf
APA — 7th edition
Shee, P., Kundu, S., Bhar, A., & Ghosh, M. (2023). ReBERT- An Enhanced BERT. International Journal for Research Trends and Innovation, 8(10), 484 - 494. https://ijrti.org/viewpaperforall.php?paper=IJRTI2310067
MLA — 9th edition
Shee, Priyanka, et al. "ReBERT- An Enhanced BERT." International Journal for Research Trends and Innovation, vol. 8, no. 10, 2023, pp. 484 - 494, https://ijrti.org/viewpaperforall.php?paper=IJRTI2310067.
Chicago — 17th, bibliography
Shee, Priyanka, et al. "ReBERT- An Enhanced BERT." International Journal for Research Trends and Innovation 8, no. 10 (2023): 484 - 494. https://ijrti.org/viewpaperforall.php?paper=IJRTI2310067.
Harvard — author–date
Shee, P. et al. (2023) 'ReBERT- An Enhanced BERT', International Journal for Research Trends and Innovation, 8(10), pp. 484 - 494. Available at: https://ijrti.org/viewpaperforall.php?paper=IJRTI2310067
IEEE — numbered reference
P. Shee, S. Kundu, A. Bhar and M. Ghosh, "ReBERT- An Enhanced BERT," IJRTI, vol. 8, no. 10, pp. 484 - 494, Oct. 2023.
Vancouver — biomedical
Shee P, Kundu S, Bhar A, Ghosh M. ReBERT- An Enhanced BERT. IJRTI. 2023 Oct;8(10):484 - 494.
AMA — 11th edition
Shee P, Kundu S, Bhar A, Ghosh M. ReBERT- An Enhanced BERT. IJRTI. 2023;8(10):484 - 494. Accessed at https://ijrti.org/viewpaperforall.php?paper=IJRTI2310067
BibTeX — Zotero, Mendeley, LaTeX
@article{IJRTI2310067,
author = {Priyanka Shee and Santayo Kundu and Anirban Bhar and Moumita Ghosh},
title = {ReBERT- An Enhanced BERT},
journal = {International Journal for Research Trends and Innovation},
volume = {8},
number = {10},
pages = {484 - 494},
year = {2023},
month = {October},
issn = {2456-3315},
url = {https://ijrti.org/viewpaperforall.php?paper=IJRTI2310067}
}
RIS — EndNote, RefWorks
TY - JOUR
AU - Shee, Priyanka
AU - Kundu, Santayo
AU - Bhar, Anirban
AU - Ghosh, Moumita
TI - ReBERT- An Enhanced BERT
T2 - International Journal for Research Trends and Innovation
JA - IJRTI
VL - 8
IS - 10
PY - 2023
SN - 2456-3315
UR - https://ijrti.org/viewpaperforall.php?paper=IJRTI2310067
SP - 484
EP - 494
ER -
Issue
Volume 8 Issue 10,
October-2023
Pages : 484 - 494
Other Publication Details
Paper Reg. ID
IJRTI_188286
Published Paper ID
IJRTI2310067
Downloads
205,627
Research Area
Engineering
Country
-, -, India
Published
October 2023
Published Paper PDF
https://ijrti.org/papers/IJRTI2310067.pdf
Published Paper URL
https://ijrti.org/viewpaperforall.php?paper=IJRTI2310067
About Publisher
International Journal for Research Trends and Innovation
Published by IJRTI (JW Publication)
2456-3315
ISSN
10.57
Impact Factor
2016
ESTD Year
Open
Access
Impact Factor 10.57 calculated by Google Scholar and Semantic Scholar.
An International Scholarly Open Access Journal, Peer-Reviewed, Refereed Journal Impact Factor 10.57 Calculate by Google Scholar and Semantic Scholar | AI-Powered Research Tool, Multidisciplinary, Monthly, Multilanguage Journal Indexing in All Major Database & Metadata, Citation Generator
UGC CARE Approved Journal — Transparent Peer-reviewed journal aligned with UGC’s 2025 suggestive parameters, with CrossRef DOI registration and Scopus Standard metadata on every published paper.
Licence
© 2023 — Authors hold the copyright of this article.
This work is licensed under a
Creative Commons Attribution 4.0 International License
and The Open Definition.
This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0),
which permits unrestricted use, distribution and reproduction in any medium, provided the original author and source are credited.
Disclaimer:
The content, data and findings in this article are based on the authors’ research and have been
peer-reviewed for academic purposes only. Readers are advised to verify all information before practical
or commercial use. The journal and its editorial board are not liable for any errors, losses or
consequences arising from its use.
Declarations
Funding
No external funding was received for this study.
Conflict of Interest
The authors declare that they have no conflict of interest.
Acknowledgements
The authors would like to thank the reviewers and the editorial board of International Journal for Research Trends and Innovation for their careful reading and constructive comments, and all colleagues who supported the preparation of this manuscript.
