程序師世界是廣大編程愛好者互助、分享、學習的平台,程序師世界有你更精彩!
首頁
編程語言
C語言|JAVA編程
Python編程
網頁編程
ASP編程|PHP編程
JSP編程
數據庫知識
MYSQL數據庫|SqlServer數據庫
Oracle數據庫|DB2數據庫
 程式師世界 >> 編程語言 >> 更多編程語言 >> 更多關於編程 >> Giraph基礎介紹

Giraph基礎介紹

編輯:更多關於編程

       Giraph介紹:

      Apache Giraph is an iterative graph processing system built for high scalability. For example, it is currently used at Facebook to analyze the social graph formed by users and their connections. Giraph originated as the open-source counterpart to Pregel, the graph processing architecture developed at Google and described in a 2010 paper. Both systems are inspired by the Bulk Synchronous Parallel model of distributed computation introduced by Leslie Valiant. Giraph adds several features beyond the basic Pregel model, including master computation, sharded aggregators, edge-oriented input, out-of-core computation, and more. With a steady development cycle and a growing community of users worldwide, Giraph is a natural choice for unleashing the potential of structured datasets at a massive scale.

      原理:Giraph基於Hadoop而建,將MapReduce中Mapper進行封裝,未使用reducer。在Mapper中進行多次迭代,每次迭代等價於BSP模型中的SuperStep。一個Hadoop Job等價於一次BSP作業。基礎結構如下圖所示。

    Giraph基礎介紹    三聯

      Giraph 的基礎架構

      每部分的功能如下:

      (1)ZooKeeper: responsible for computation state

      –partition/worker mapping

      –global state: #superstep

      –checkpoint paths, aggregator values, statistics

      (2) Master: responsible for coordination

      –assigns partitions to workers

      –coordinates synchronization

      –requests checkpoints

      –aggregates aggregator values

      –collects health statuses

      (3) Worker: responsible for vertices

      –invokes active vertices compute() function

      –sends, receives and assigns messages

      –computes local aggregation values

    n塊i*讦…�

    源碼。 

    1. 上一頁:
    2. 下一頁:
    Copyright © 程式師世界 All Rights Reserved