UGC CARE norms ugc approved journal norms IJRTI Research Journal

WhatsApp
Click Here

International Journal for Research Trends and Innovation
International Peer Reviewed & Refereed Journals, Open Access Journal
ISSN Approved Journal No: 2456-3315 | Impact factor: 8.14 | ESTD Year: 2016
Scholarly open access journals, Peer-reviewed, and Refereed Journals, Impact factor 8.14 (Calculate by google scholar and Semantic Scholar | AI-Powered Research Tool) , Multidisciplinary, Monthly, Indexing in all major database & Metadata, Citation Generator, Digital Object Identifier(DOI)

Call For Paper

For Authors

Forms / Download

Published Issue Details

Editorial Board

Other IMP Links

Facts & Figure

Impact Factor : 8.14

Issue per Year : 12

Volume Published : 7

Issue Published : 74

Article Submitted : 3609

Article Published : 2079

Total Authors : 5515

Total Reviewer : 528

Total Countries : 39

Indexing Partner

Licence

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Published Paper Details
Paper Title: Performance analysis of a Non-Concurrent Crawler, a Concurrent Crawler (Python and Go), and a script using asyncio and aiohttp
Authors Name: P. Manohar Venkat , R. Joshitha , Kannadasan
Download E-Certificate: Download
Author Reg. ID:
IJRTI_182281
Published Paper Id: IJRTI2206093
Published In: Volume 7 Issue 6, June-2022
DOI:
Abstract: Developing a concurrent crawler using Python and Go and comparing its performance with a single-threaded crawler Due to the issues that Python has with concurrency due to GIL (Global Interpreter Lock), the default Python scrapers, and crawlers are relatively slow. The first version will have no concurrency and will just request each website one at a time. The second version makes use of concurrent futures’ thread pool executor, allowing me to send concurrent requests by making use of threads. This paper will solve these issues by using asyncio and aiohttp, which will allow concurrent requests to be made via an event loop. Then see if altering the implementation language has any effect on the speed of retrieval, internal processing, or memory needs for conducting the crawl. Google introduced GO to allow for increased concurrency.
Keywords: Web Crawler, GIL (Global Interpreter Lock), asyncio, aiohttp, Python scrapers.
Cite Article: "Performance analysis of a Non-Concurrent Crawler, a Concurrent Crawler (Python and Go), and a script using asyncio and aiohttp", International Journal of Science & Engineering Development Research (www.ijrti.org), ISSN:2455-2631, Vol.7, Issue 6, page no.559 - 566, June-2022, Available :http://www.ijrti.org/papers/IJRTI2206093.pdf
Downloads: 00084154
ISSN: 2456-3315 | IMPACT FACTOR: 8.14 Calculated By Google Scholar| ESTD YEAR: 2016
An International Scholarly Open Access Journal, Peer-Reviewed, Refereed Journal Impact Factor 8.14 Calculate by Google Scholar and Semantic Scholar | AI-Powered Research Tool, Multidisciplinary, Monthly, Multilanguage Journal Indexing in All Major Database & Metadata, Citation Generator
Publication Details: Published Paper ID: IJRTI2206093
Registration ID:182281
Published In: Volume 7 Issue 6, June-2022
DOI (Digital Object Identifier):
Page No: 559 - 566
Country: Vellore, Tamil Nadu, India
Research Area: Computer Science & Technology 
Publisher : IJ Publication
Published Paper URL : https://www.ijrti.org/viewpaperforall?paper=IJRTI2206093
Published Paper PDF: https://www.ijrti.org/papers/IJRTI2206093
Share Article:

Share Facebook Twitter Google+ Pinterest LinkedIn Email Tumblr WhatsApp Google Gmail

Click Here to Download This Article

Article Preview
Click Here to Download This Article
Major Indexing from www.ijrti.org
Google Scholar ResearcherID Thomson Reuters Mendeley : reference manager Academia.edu
arXiv.org : cornell university library Research Gate CiteSeerX DOAJ : Directory of Open Access Journals
DRJI Index Copernicus International Scribd DocStoc

ISSN Details

ISSN: 2456-3315
Impact Factor: 8.14 and ISSN APPROVED
Journal Starting Year (ESTD) : 2016

DOI (A digital object identifier)


Providing A digital object identifier by DOI.ONE
How to Get DOI?

Conference

Open Access License Policy

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

Creative Commons License This material is Open Knowledge This material is Open Data This material is Open Content

Important Details

Social Media

Join RMS/Earn 300
IJRTI

Indexing Partner
Copyright © 2022 - All Rights Reserved - IJRTI