Package | Description |
---|---|
org.apache.reef.driver |
Driver APIs for REEF in terms of Rx.
|
org.apache.reef.driver.client |
Representations of the Client (i.e.
|
org.apache.reef.driver.context |
Driver Contexts and their configurations.
|
org.apache.reef.driver.evaluator |
Evaluator-related utilities.
|
org.apache.reef.driver.restart |
This package provides restart event service handlers.
|
org.apache.reef.driver.task |
Tasks and their configurations.
|
org.apache.reef.examples.data.loading |
A line counter example using data loading service.
|
org.apache.reef.examples.group.bgd |
Runs BGD on the given runtime.
|
org.apache.reef.examples.group.broadcast |
Broadcast Group Communication example.
|
org.apache.reef.exception |
Exceptions thrown to Job Drivers.
|
org.apache.reef.io.checkpoint.fs |
FileSystem based checkpoints.
|
org.apache.reef.io.data.loading.api |
Data Loading Service API.
|
org.apache.reef.io.data.loading.impl |
Implementations of Data Loading Service.
|
org.apache.reef.io.network.group.api.config |
The operator configurations used by users.
|
org.apache.reef.io.network.group.api.driver |
Elastic Group Communications Driver-side API Interfaces.
|
org.apache.reef.io.network.group.impl.driver |
This package contains the implementation of the driver side of the
Group Communication Service using the tree/flat topology.
|
org.apache.reef.javabridge |
The Java-side of the CLR/Java bridge.
|
org.apache.reef.javabridge.generic |
Generic java bridge driver/client.
|
org.apache.reef.runtime.common.driver |
Implementation of the Driver-side REEF APIs.
|
org.apache.reef.runtime.common.driver.api |
Driver-Side Event Handlers to be implemented by a specific resource manager.
|
org.apache.reef.runtime.common.driver.client |
Objects representing the Client on the Driver.
|
org.apache.reef.runtime.common.driver.context |
Implementations of Driver-Side representations of Contexts running on an Evaluator.
|
org.apache.reef.runtime.common.driver.evaluator |
Driver-Side representations of Evaluators.
|
org.apache.reef.runtime.common.driver.evaluator.pojos |
POJOs in this package encapsulate the content of
an EvaluatorHeartbeatProto on the DriverSide.
|
org.apache.reef.runtime.common.driver.resourcemanager |
Classes that interface with the resourcemanager (Local, YARN, ...) in the Driver.
|
org.apache.reef.runtime.common.driver.task |
Driver-Side representations of tasks.
|
org.apache.reef.runtime.common.launch |
Common launch code between Driver and Evaluator.
|
org.apache.reef.runtime.common.utils |
Runtime utilities used by various REEF modules.
|
org.apache.reef.runtime.local.driver |
The resource manager for the local resourcemanager.
|
org.apache.reef.runtime.yarn.driver |
The Driver-Side implementation of the YARN adapter for REEF.
|
org.apache.reef.runtime.yarn.driver.restart |
The package contains classes that are used specifically on restart.
|
org.apache.reef.tests.group |
Tests for group communication.
|
org.apache.reef.vortex.common |
Vortex Code used both in Vortex Driver and Vortex Evaluator.
|
org.apache.reef.vortex.driver |
Vortex Code that runs as part of REEF Driver.
|
Package | Description |
---|---|
org.apache.reef.runtime.common.driver.task |
Driver-Side representations of tasks.
|
org.apache.reef.runtime.common.driver.context |
Implementations of Driver-Side representations of Contexts running on an Evaluator.
|
org.apache.reef.runtime.common.driver.evaluator.pojos |
POJOs in this package encapsulate the content of
an EvaluatorHeartbeatProto on the DriverSide.
|
org.apache.reef.runtime.common.driver.evaluator |
Driver-Side representations of Evaluators.
|
org.apache.reef.runtime.common.driver.client |
Objects representing the Client on the Driver.
|
org.apache.reef.driver.client |
Representations of the Client (i.e.
|
Modifier and Type | Interface and Description |
---|---|
interface |
ContextAndTaskSubmittable
Base interface for classes that support the simultaneous submission of both Context and Task configurations.
|
interface |
ContextSubmittable
Base interface for classes that support Context submission.
|
class |
DefaultProgressProvider
The default implementation of
ProgressProvider . |
interface |
PreemptionEvent
Represents Preemption requests issued by the underlying resource manager.
|
interface |
PreemptionHandler
This EventHandler will receive preemption events from the underlying resourcemanager.
|
interface |
ProgressProvider
Used in the to report the progress of the REEF application.
|
interface |
StrictPreemptionEvent
Represents a strict preemption event: It contains the set of Evaluators that the underlying resource manager will
take away from the Driver.
|
interface |
TaskSubmittable
Base interface for classes that support Task submission.
|
Modifier and Type | Interface and Description |
---|---|
interface |
JobMessageObserver
The driver uses this interface to communicate with the job client.
|
Modifier and Type | Interface and Description |
---|---|
interface |
ActiveContext
Represents an active context on an Evaluator.
|
interface |
ClosedContext
Represents a Context that has been closed successfully.
|
interface |
ContextBase
A common base interface for all Driver-side representations of Contexts.
|
class |
ContextConfiguration
A ConfigurationModule for Context Configuration.
|
interface |
ContextMessage
Driver-side representation of a message sent by a Context to the Driver.
|
interface |
FailedContext
Represents Context that failed.
|
Modifier and Type | Interface and Description |
---|---|
interface |
AllocatedEvaluator
Represents an Evaluator that is allocated, but is not running yet.
|
class |
CLRProcessFactory
Factory to setup new CLR processes.
|
interface |
CompletedEvaluator
Represents a successfully closed Evaluator.
|
interface |
EvaluatorProcessFactory<T>
Factory to create new evaluator process setups.
|
class |
EvaluatorRequest
A request for one ore more Evaluators.
|
interface |
EvaluatorRequestor
Interface through which Evaluators can be requested.
|
interface |
FailedEvaluator
Represents an Evaluator that became unavailable.
|
class |
JVMProcessFactory
Factory to setup new JVM processes.
|
Modifier and Type | Class and Description |
---|---|
class |
DriverRestartedImpl |
class |
DriverRestartManager
The manager that handles aspects of driver restart such as determining whether the driver is in
restart mode, what to do on restart, whether restart is completed, and others.
|
class |
DriverRestartState
Represents the current driver restart progress.
|
interface |
DriverRuntimeRestartManager
Classes implementing this interface are in charge of recording evaluator
changes as they are allocated as well as recovering Evaluators and
discovering which evaluators are lost on the event of a driver restart.
|
class |
EvaluatorRestartInfo
An object that encapsulates the information needed to construct an
EvaluatorManager for a recovered evaluator
on restart. |
class |
EvaluatorRestartState
The state that the evaluator is in in the driver restart process.
|
class |
RestartEvaluators
Represents holds the set of Evaluator information needed to recover EvaluatorManagers
on the restarted Driver.
|
Modifier and Type | Interface and Description |
---|---|
interface |
CompletedTask
Represents a completed Task.
|
class |
FailedTask
An error message that REEF Driver gets from a failed Task.
|
interface |
RunningTask
Represents a running Task.
|
interface |
SuspendedTask
Represents a suspended Task.
|
class |
TaskConfigurationOptions
Configuration parameters for the TaskConfiguration class.
|
interface |
TaskMessage
A message from a running task to the driver.
|
Modifier and Type | Class and Description |
---|---|
class |
LineCounter
Driver side for the line counting demo that uses the data loading service.
|
Modifier and Type | Class and Description |
---|---|
class |
BGDDriver
Driver for BGD example.
|
Modifier and Type | Class and Description |
---|---|
class |
BroadcastDriver
Driver for broadcast example.
|
Modifier and Type | Class and Description |
---|---|
class |
EvaluatorException
Exception thrown to the Driver when an Evaluator becomes unusable.
|
Modifier and Type | Class and Description |
---|---|
class |
FSCheckPointServiceConfiguration
ConfigurationModule for the FSCheckPointService.
|
Modifier and Type | Class and Description |
---|---|
class |
DataLoader
The driver component for the DataLoadingService
Also acts as the central point for resource requests
All the allocated evaluators pass through this and
the ones that need data loading have a context stacked
that enables a task to get access to Data via the
DataSet . |
interface |
DataLoadingService
All data loading services should implement this interface.
|
interface |
EvaluatorToPartitionStrategy<V extends org.apache.hadoop.mapred.InputSplit>
Interface that tracks the mapping between evaluators and the data partitions
assigned to those evaluators.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractEvaluatorToPartitionStrategy
This is an abstract class useful for
EvaluatorToPartitionStrategy
implementations. |
class |
InputFormatExternalConstructor
A Tang External Constructor to inject the required.
|
class |
InputFormatLoadingService<K,V>
An implementation of
DataLoadingService
that uses the Hadoop InputFormat to find
partitions of data and request resources. |
class |
MultiDataCenterEvaluatorToPartitionStrategy
This is an online version which satisfies requests based on the locations the
users ask the data to be loaded, for multiple data center network topologies.
|
class |
SingleDataCenterEvaluatorToPartitionStrategy
This is an online version which satisfies
requests in a greedy way, for single data center network topologies.
|
Modifier and Type | Interface and Description |
---|---|
interface |
OperatorSpec
The specification of an operator submitted by the user
while configuring the communication group.
|
Modifier and Type | Interface and Description |
---|---|
interface |
CommunicationGroupDriver
The driver side interface of a Communication Group
Lets one add operators and tasks.
|
interface |
GroupCommDriver
The driver side interface of Group Communication.
|
Modifier and Type | Class and Description |
---|---|
class |
CommunicationGroupDriverImpl |
Modifier and Type | Class and Description |
---|---|
class |
ActiveContextBridgeFactory
Factory for ActiveContextBridge instances.
|
class |
AllocatedEvaluatorBridgeFactory
The Factory object used to create AllocatedEvaluatorBridge objects.
|
class |
DriverRestartedBridge
The Java-CLR bridge object for
DriverRestarted events. |
Modifier and Type | Class and Description |
---|---|
class |
DriverRestartCompletedBridge
The Java-CLR bridge object for
DriverRestartCompleted events. |
Modifier and Type | Class and Description |
---|---|
class |
DriverExceptionHandler
Exception handler for exceptions thrown by client code in the Driver.
|
interface |
EvaluatorPreserver
A interface to preserve evaluators across driver restarts.
|
Modifier and Type | Interface and Description |
---|---|
interface |
ResourceLaunchEvent
Event from Driver Process to Driver Runtime.
|
interface |
ResourceReleaseEvent
Event from Driver Process to Driver Runtime.
|
interface |
ResourceRequestEvent
Event from Driver Process to Driver Runtime.
|
Modifier and Type | Class and Description |
---|---|
class |
ClientConnection
Represents the communication channel to the client.
|
class |
ClientManager
Represents the Client in the Driver.
|
class |
JobMessageObserverImpl
An implementation of JobMessageObserver.
|
Modifier and Type | Class and Description |
---|---|
class |
ClosedContextImpl
Driver side representation of a closed context.
|
class |
ContextControlHandler
Handles context control messages.
|
class |
ContextMessageImpl
Driver-side representation of a context message.
|
class |
ContextRepresenters
Driver-Side representation of all contexts on an Evaluator.
|
class |
EvaluatorContext
Driver-side representation of a Context on an Evaluator.
|
class |
FailedContextImpl
Driver-Side representation of a failed context.
|
Modifier and Type | Class and Description |
---|---|
class |
AllocatedEvaluatorImpl
Driver-Side representation of an allocated evaluator.
|
class |
EvaluatorControlHandler
This class handles the sending of Evaluator control messages to the Evaluator.
|
class |
EvaluatorHeartbeatHandler
Receives heartbeats from all Evaluators and dispatches them to the right EvaluatorManager instance.
|
class |
EvaluatorManager
Manages a single Evaluator instance including all lifecycle instances:
(AllocatedEvaluator, CompletedEvaluator, FailedEvaluator).
|
class |
EvaluatorManagerFactory
Helper class that creates new EvaluatorManager instances from allocations.
|
class |
Evaluators
Manages all Evaluators.
|
Modifier and Type | Class and Description |
---|---|
class |
ContextMessagePOJO
DriverSide representation of ContextMessageProto.
|
class |
ContextState
DriverSide representation of ContextStatusProto.State.
|
class |
ContextStatusPOJO
DriverSide representation of ContextStatusProto.
|
class |
EvaluatorStatusPOJO
DriverSide representation of EvaluatorStatusProto.
|
class |
State
DriverSide representation of ReefServiceProtos.State.
|
class |
TaskMessagePOJO
DriverSide representation of TaskMessageProto.
|
class |
TaskStatusPOJO
DriverSide representation of TaskStatusProto.
|
Modifier and Type | Interface and Description |
---|---|
interface |
NodeDescriptorEvent
Event from Driver Runtime to Driver Process.
|
class |
NodeDescriptorHandler
Updates the ResourceCatalog with a new Node.
|
interface |
ResourceAllocationEvent
Event from Driver Runtime to Driver Process
A Resource allocated by the Driver Runtime.
|
class |
ResourceAllocationHandler
Handles new resource allocations by adding a new EvaluatorManager.
|
interface |
ResourceEvent
An interface capturing the characteristics of a resource event.
|
class |
ResourceManagerStatus
Manages the status of the Resource Manager.
|
interface |
ResourceRecoverEvent
A Resource recovered by the
DriverRestartManager . |
interface |
ResourceStatusEvent
Event from Driver Runtime to Driver Process.
|
interface |
RuntimeStatusEvent
Event from Driver Runtime to Driver Process.
|
Modifier and Type | Class and Description |
---|---|
class |
CompletedTaskImpl
Implementation of CompletedTask.
|
class |
RunningTaskImpl
Implements the RunningTask client interface.
|
class |
SuspendedTaskImpl
Implementation of SuspendedTask.
|
class |
TaskMessageImpl
Implementation of TaskMessage.
|
class |
TaskRepresenter
Represents a Task on the Driver.
|
Modifier and Type | Class and Description |
---|---|
class |
REEFMessageCodec
Codec for REEF's control flow messages.
|
Modifier and Type | Class and Description |
---|---|
class |
DispatchingEStage
Delayed event router that dispatches messages to the proper event handler by type.
|
Modifier and Type | Class and Description |
---|---|
class |
LocalResourceReleaseHandler
Takes Resource Release requests and patches them through to the resource
manager.
|
class |
ResourceManager
A resource manager that uses threads to execute containers.
|
Modifier and Type | Class and Description |
---|---|
class |
DefaultRackNameFormatter
Default implementation of rack names formatter.
|
interface |
RackNameFormatter
Provides a method to retrieve the rack name from a container, which
may be dependent on the Hadoop distribution.
|
class |
YarnDriverRuntimeRestartManager
The implementation of restart manager for YARN.
|
class |
YarnResourceRequestHandler
Accepts resource requests from the REEF layer, translates them into requests for YARN and hands them to the
appropriate handler for those.
|
Modifier and Type | Class and Description |
---|---|
class |
DFSEvaluatorPreserver
An Evaluator Preserver that uses the DFS on YARN.
|
Modifier and Type | Class and Description |
---|---|
class |
MultipleCommGroupsDriver
Driver used for testing multiple communication groups.
|
Modifier and Type | Class and Description |
---|---|
class |
TaskletAggregateExecutionRequest<TInput>
A request from the Vortex Driver to run an aggregate-able function.
|
class |
TaskletAggregationRequest<TInput,TOutput>
A request from the Vortex Driver for the
VortexWorker to
record aggregate functions for later execution. |
class |
TaskletAggregationResultReport
Report of a Tasklet aggregation execution result.
|
class |
TaskletFailureReport
Report of a Tasklet exception.
|
interface |
TaskletReport
The interface for a status report from the
VortexWorker . |
class |
TaskletResultReport
Report of a tasklet execution result.
|
class |
VortexAvroUtils
Serialize and deserialize Vortex message to/from byte array.
|
interface |
VortexFutureDelegate
Exposes functions to be called by the
VortexMaster
to note that a list of Tasklets associated with a Future has completed. |
class |
WorkerReport
Worker-to-Master protocol.
|
Modifier and Type | Interface and Description |
---|---|
interface |
VortexMaster
The heart of Vortex.
|
class |
VortexMasterConf
Vortex Master configuration.
|
class |
VortexWorkerConf
Vortex Worker configuration.
|
Copyright © 2016 The Apache Software Foundation. All rights reserved.