Apache Hama
Developer(s)Apache Software Foundation
Preview release
0.7.1 / March 14, 2016 (2016-03-14)
RepositoryHama Repository
Written inJava
Operating systemCross-platform
TypeDistributed computing, bulk synchronous parallel
LicenseApache License 2.0
Websitehama.apache.org

Apache Hama is a distributed computing framework based on bulk synchronous parallel computing techniques for massive scientific computations e.g., matrix, graph and network algorithms.[1] Originally a sub-project of Hadoop,[2] it became an Apache Software Foundation top level project in 2012. It was created by Edward J. Yoon,[3] who named it (short for "Hadoop Matrix Algebra"),[4] and Hama also means hippopotamus in Yoon's native Korean language (하마),[5] following the trend of naming Apache projects after animals and zoology (such as Apache Pig). Hama was inspired by Google's Pregel large-scale graph computing framework described in 2010.[6] When executing graph algorithms, Hama showed a fifty-fold performance increase relative to Hadoop.[7]

Retired in April 2020,[8] project resources are made available as part of the Apache Attic. Yoon cited issues of installation, scalability, and a difficult programming model[9] for its lack of adoption.

Architecture

Hama consists of three major components: BSPMaster, GroomServers and Zookeeper.[10]

BSPMaster

BSPMaster is responsible for:

  • Maintaining groom server status
  • Controlling super steps in a cluster
  • Maintaining job progress information
  • Scheduling jobs and assigning tasks to groom servers
  • Disseminating execution class across groom servers
  • Controlling fault
  • Providing users with the cluster control interface.

A BSP Master and multiple grooms are started by the script. Then, the bsp master starts up with a RPC server for groom servers. Groom servers starts up with a BSPPeer instance and a RPC proxy to contact the bsp master. After started, each groom periodically sends a heartbeat message that encloses its groom server status, including maximum task capacity, unused memory, and so on.

Each time the BSP master receives a heartbeat message, it brings up-to-date groom server status - the bsp master makes use of groom servers' status in order to effectively assign tasks to idle groom servers - and returns a heartbeat response that contains assigned tasks and others actions that a groom server has to do. For now, we have a FIFO job scheduler and very simple task assignment algorithms.

GroomServer

A groom server (shortly referred to as groom) is a process that performs BSP tasks assigned by BSPMaster. Each groom contacts the BSPMaster, and it takes assigned tasks and reports its status by means of periodical piggybacks with BSPMaster. Each groom is designed to run with HDFS or other distributed storages. Basically, a groom server and a data node should be run on one physical node.

Zookeeper

A Zookeeper is used to manage the efficient barrier synchronisation of the BSPPeers.

See also

References

  1. ↑ "Apache Hama". Project web site. Retrieved September 20, 2013.
  2. ↑ Seo, Sangwon; Yoon, Edward (2010). Hama: An efficient matrix computation with the mapreduce framework (PDF). 2010 IEEE Second International Conference on Cloud Computing Technology and Science. IEEE.
  3. ↑ Apache Hama Proposal
  4. ↑ Di, Liping (2023-07-24). Remote Sensing Big Data. Springer Nature. p. 180. ISBN 9783031339325.
  5. ↑ "하마 - 위키낱말사전". ko.wiktionary.org (in Korean). Retrieved 2019-02-21.
  6. ↑ Grzegorz Malewicz; Matthew H. Austern; Aart J.C Bik; James C. Dehnert; Ilan Horn; Naty Leiser; Grzegorz Czajkowski (2010). "Pregel: a system for large-scale graph processing". Proceedings of the 2010 ACM SIGMOD International Conference on Management of data. pp. 135–146. doi:10.1145/1807167.1807184. ISBN 9781450300322.
  7. ↑ Elser, Benedikt; Montresor, Alberto (2013). An Evaluation Study of BigData Frameworks for Graph Processing (PDF). 2013 IEEE International Conference on Big Data. IEEE.
  8. ↑ Apache Hama - Apache Attic
  9. ↑ Jungblut, Thomas; Yoon, Edward; Lin, Chia-Hung Lin (2020-11-14). "Apache Hama - why it didn't become successful". thomasjungblut.com. Retrieved 2023-12-14.
  10. ↑ Apache Hama Architecture
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.