Ảnh bìa

A High Performance Computing Web Search Engine Based on Big Data and Parallel Distributed Models

Đã lưu trong:
Xuất bản năm:Informatica vol. 48, no. 20 (Dec 2024), p. 27
Tác giả chính: Ma, Jun
Được phát hành:
Slovenian Society Informatika / Slovensko drustvo Informatika
Những chủ đề:
Big Data
Machine learning
Search engines
Computers
User needs
Performance evaluation
Artificial intelligence
Data structures
Cloud computing
Computational grids
Data search
System effectiveness
Data storage
High performance computing
Distributed processing
Efficiency
Information retrieval
Vector spaces
Truy cập trực tuyến:Citation/Abstract
Full Text
Full Text - PDF
Các nhãn: Thêm thẻ
Không có thẻ, Là người đầu tiên thẻ bản ghi này!
Bài tóm tắt:This paper presents a high-performance web search system leveraging big data technology. Utilizing a heterogeneous architecture and a parallel distributed computing model based on the MapReduce framework, the system significantly enhances efficiency, scalability, and reliability. The design includes a storage management scheme that integrates cloud storage and grid computing technologies, facilitating efficient storage and rapid access to large-scale data. Key components such as an inverted index structure, vector space model, and semantic analysis models are employed to implement functionalities across the data, logic, and display layers. An experimental environment was set up on the Microsoft Azure cloud platform using the Common Crawl dataset for testing. Performance evaluation, based on metrics including response time, accuracy, and stability, demonstrates the system's superior performance compared to two existing systems, thereby validating its effectiveness.
số ISSN:0350-5596
1854-3871
DOI:10.31449/inf.v48i20.6776
Nguồn:Advanced Technologies & Aerospace Database