Package | Description |
---|---|
org.apache.reef.driver |
Driver APIs for REEF in terms of Rx.
|
org.apache.reef.driver.evaluator |
Evaluator-related utilities.
|
org.apache.reef.driver.restart |
This package provides restart event service handlers.
|
org.apache.reef.evaluator.context.parameters |
Named parameters used by the Evaluator.
|
org.apache.reef.io.data.loading.impl |
Implementations of Data Loading Service.
|
org.apache.reef.io.network.group.api.config |
The operator configurations used by users.
|
org.apache.reef.io.network.group.api.driver |
Elastic Group Communications Driver-side API Interfaces.
|
org.apache.reef.io.network.group.api.task |
Elastic Group Communications Task-side API Interfaces.
|
org.apache.reef.io.network.group.impl.driver |
This package contains the implementation of the driver side of the
Group Communication Service using the tree/flat topology.
|
org.apache.reef.io.watcher.param |
Parameters for Watcher.
|
org.apache.reef.io.watcher.util |
Utility classes for Watcher.
|
org.apache.reef.javabridge |
The Java-side of the CLR/Java bridge.
|
org.apache.reef.javabridge.generic |
Generic java bridge driver/client.
|
org.apache.reef.runtime.common.client |
Implementation of the client-side REEF API.
|
org.apache.reef.runtime.common.driver |
Implementation of the Driver-side REEF APIs.
|
org.apache.reef.runtime.common.driver.catalog |
Implementations of descriptors for resources available to a REEF instance (physical nodes, racks etc).
|
org.apache.reef.runtime.common.driver.client |
Objects representing the Client on the Driver.
|
org.apache.reef.runtime.common.driver.context |
Implementations of Driver-Side representations of Contexts running on an Evaluator.
|
org.apache.reef.runtime.common.driver.evaluator |
Driver-Side representations of Evaluators.
|
org.apache.reef.runtime.common.driver.evaluator.pojos |
POJOs in this package encapsulate the content of
an EvaluatorHeartbeatProto on the DriverSide.
|
org.apache.reef.runtime.common.driver.resourcemanager |
Classes that interface with the resourcemanager (Local, YARN, ...) in the Driver.
|
org.apache.reef.runtime.common.driver.task |
Driver-Side representations of tasks.
|
org.apache.reef.runtime.common.evaluator.context |
Context implementation of the Evaluator resourcemanager.
|
org.apache.reef.runtime.common.evaluator.task |
Task-related implementation of the Evaluator resourcemanager.
|
org.apache.reef.runtime.common.evaluator.task.defaults |
Default implementations for the optional task interfaces.
|
org.apache.reef.runtime.common.evaluator.task.exceptions |
Exceptions thrown by Tasks and Task Handlers.
|
org.apache.reef.runtime.common.files |
This package will contain the implementation of the REEF file system standard.
|
org.apache.reef.runtime.common.launch |
Common launch code between Driver and Evaluator.
|
org.apache.reef.runtime.common.utils |
Runtime utilities used by various REEF modules.
|
org.apache.reef.runtime.hdinsight.client |
The client to submit jobs to HDInsight.
|
org.apache.reef.runtime.local.driver |
The resource manager for the local resourcemanager.
|
org.apache.reef.runtime.mesos.driver |
Driver-side event handlers for Mesos resourcemanager.
|
org.apache.reef.runtime.yarn.driver |
The Driver-Side implementation of the YARN adapter for REEF.
|
org.apache.reef.runtime.yarn.driver.restart |
The package contains classes that are used specifically on restart.
|
org.apache.reef.runtime.yarn.util |
Utility for YARN runtime.
|
org.apache.reef.util |
Utilities for using REEF on Hadoop.
|
org.apache.reef.vortex.api |
Vortex user api for running jobs on Vortex.
|
org.apache.reef.vortex.common |
Vortex Code used both in Vortex Driver and Vortex Evaluator.
|
org.apache.reef.vortex.driver |
Vortex Code that runs as part of REEF Driver.
|
Package | Description |
---|---|
org.apache.reef.runtime.common.driver.task |
Driver-Side representations of tasks.
|
org.apache.reef.runtime.common.driver.context |
Implementations of Driver-Side representations of Contexts running on an Evaluator.
|
org.apache.reef.runtime.common.driver.evaluator.pojos |
POJOs in this package encapsulate the content of
an EvaluatorHeartbeatProto on the DriverSide.
|
org.apache.reef.runtime.common.driver.evaluator |
Driver-Side representations of Evaluators.
|
Modifier and Type | Interface and Description |
---|---|
interface |
FlexiblePreemptionEvent
Represents a flexible preemption request: It contains:
|
Modifier and Type | Class and Description |
---|---|
class |
CLRProcessFactory
Factory to setup new CLR processes.
|
Modifier and Type | Class and Description |
---|---|
class |
DriverRestartedImpl |
class |
DriverRestartManager
The manager that handles aspects of driver restart such as determining whether the driver is in
restart mode, what to do on restart, whether restart is completed, and others.
|
class |
DriverRestartState
Represents the current driver restart progress.
|
interface |
DriverRuntimeRestartManager
Classes implementing this interface are in charge of recording evaluator
changes as they are allocated as well as recovering Evaluators and
discovering which evaluators are lost on the event of a driver restart.
|
class |
EvaluatorRestartInfo
An object that encapsulates the information needed to construct an
EvaluatorManager for a recovered evaluator
on restart. |
class |
EvaluatorRestartState
The state that the evaluator is in in the driver restart process.
|
class |
RestartEvaluators
Represents holds the set of Evaluator information needed to recover EvaluatorManagers
on the restarted Driver.
|
Modifier and Type | Class and Description |
---|---|
class |
Services
A set of classes to be instantiated and shared as singletons within this context and all child context.
|
Modifier and Type | Class and Description |
---|---|
class |
AvroEvaluatorRequestSerializer
Serialize and deserialize EvaluatorRequest objects using Avro.
|
Modifier and Type | Interface and Description |
---|---|
interface |
OperatorSpec
The specification of an operator submitted by the user
while configuring the communication group.
|
Modifier and Type | Interface and Description |
---|---|
interface |
GroupCommServiceDriver |
Modifier and Type | Interface and Description |
---|---|
interface |
CommunicationGroupServiceClient |
Modifier and Type | Class and Description |
---|---|
class |
CommunicationGroupDriverImpl |
Modifier and Type | Class and Description |
---|---|
class |
EventStreams
Set of EventStreams.
|
Modifier and Type | Class and Description |
---|---|
class |
WatcherAvroUtil |
Modifier and Type | Class and Description |
---|---|
class |
ActiveContextBridge
The Java-CLR bridge object for
ActiveContext . |
class |
ActiveContextBridgeFactory
Factory for ActiveContextBridge instances.
|
class |
AllocatedEvaluatorBridge
The AllocatedEvaluatorBridge object to bridge operations between REEF .NET and Java allocated evaluator operations.
|
class |
AllocatedEvaluatorBridgeFactory
The Factory object used to create AllocatedEvaluatorBridge objects.
|
class |
BridgeHandlerManager
A class that holds all handles to the .NET side.
|
class |
ClosedContextBridge
The Java-CLR bridge object for
ClosedContext . |
class |
CompletedEvaluatorBridge
The Java-CLR bridge object for
CompletedEvaluator . |
class |
CompletedTaskBridge
The Java-CLR bridge object for
CompletedTask . |
class |
ContextMessageBridge
The Java-CLR bridge object for
ContextMessage . |
class |
DriverRestartedBridge
The Java-CLR bridge object for
DriverRestarted events. |
class |
EvaluatorRequestorBridge
The Java-CLR bridge object for
EvaluatorRequestor . |
class |
FailedContextBridge
The Java-CLR bridge object for
FailedContext . |
class |
FailedEvaluatorBridge
The Java-CLR bridge object for
FailedEvaluator . |
class |
FailedTaskBridge
The Java-CLR bridge object for
FailedTask . |
class |
HttpServerEventBridge
The Java-CLR bridge object for http server events.
|
class |
InteropLogger
Logger called from CLR code.
|
class |
InteropReturnInfo
Information about return codes/exceptions coming from bridge.
|
class |
JavaBridge
TODO[JIRA REEF-383] Document/Refactor JavaBridge.
|
class |
NativeBridge
Abstract bridge class.
|
class |
NativeInterop
Java interfaces of CLR/Java bridge.
|
class |
RunningTaskBridge
The Java-CLR bridge object for
RunningTask . |
class |
SuspendedTaskBridge
The Java-CLR bridge object for
SuspendedTask . |
class |
TaskMessageBridge
The Java-CLR bridge object for
TaskMessage . |
class |
Utilities
CLR/Java bridge utilities.
|
Modifier and Type | Class and Description |
---|---|
class |
DriverRestartCompletedBridge
The Java-CLR bridge object for
DriverRestartCompleted events. |
Modifier and Type | Class and Description |
---|---|
class |
REEFImplementation
Default REEF implementation.
|
class |
RunningJobImpl
Implementation of RunningJob.
|
Modifier and Type | Class and Description |
---|---|
class |
DriverExceptionHandler
Exception handler for exceptions thrown by client code in the Driver.
|
class |
DriverRuntimeConfiguration
ConfigurationModule for driver runtime.
|
class |
DriverRuntimeRestartConfiguration
The base configuration module for driver restart configurations of all runtimes.
|
interface |
EvaluatorPreserver
A interface to preserve evaluators across driver restarts.
|
Modifier and Type | Class and Description |
---|---|
class |
NodeDescriptorImpl
Descriptor of the physical setup of an Evaluator.
|
class |
ResourceCatalogImpl
A catalog of the resources available to a REEF instance.
|
Modifier and Type | Class and Description |
---|---|
class |
ClientManager
Represents the Client in the Driver.
|
Modifier and Type | Class and Description |
---|---|
class |
ClosedContextImpl
Driver side representation of a closed context.
|
class |
ContextControlHandler
Handles context control messages.
|
class |
ContextMessageImpl
Driver-side representation of a context message.
|
class |
ContextRepresenters
Driver-Side representation of all contexts on an Evaluator.
|
class |
EvaluatorContext
Driver-side representation of a Context on an Evaluator.
|
class |
FailedContextImpl
Driver-Side representation of a failed context.
|
Modifier and Type | Class and Description |
---|---|
class |
AllocatedEvaluatorImpl
Driver-Side representation of an allocated evaluator.
|
class |
EvaluatorControlHandler
This class handles the sending of Evaluator control messages to the Evaluator.
|
class |
EvaluatorHeartbeatHandler
Receives heartbeats from all Evaluators and dispatches them to the right EvaluatorManager instance.
|
class |
EvaluatorManager
Manages a single Evaluator instance including all lifecycle instances:
(AllocatedEvaluator, CompletedEvaluator, FailedEvaluator).
|
class |
EvaluatorManagerFactory
Helper class that creates new EvaluatorManager instances from allocations.
|
class |
EvaluatorResourceManagerErrorHandler
The error handler receives all resourcemanager errors from all evaluators in the system.
|
class |
Evaluators
Manages all Evaluators.
|
Modifier and Type | Class and Description |
---|---|
class |
ContextMessagePOJO
DriverSide representation of ContextMessageProto.
|
class |
ContextState
DriverSide representation of ContextStatusProto.State.
|
class |
ContextStatusPOJO
DriverSide representation of ContextStatusProto.
|
class |
EvaluatorStatusPOJO
DriverSide representation of EvaluatorStatusProto.
|
class |
State
DriverSide representation of ReefServiceProtos.State.
|
class |
TaskMessagePOJO
DriverSide representation of TaskMessageProto.
|
class |
TaskStatusPOJO
DriverSide representation of TaskStatusProto.
|
Modifier and Type | Class and Description |
---|---|
class |
NodeDescriptorHandler
Updates the ResourceCatalog with a new Node.
|
class |
ResourceAllocationHandler
Handles new resource allocations by adding a new EvaluatorManager.
|
interface |
ResourceEvent
An interface capturing the characteristics of a resource event.
|
class |
ResourceManagerStatus
Manages the status of the Resource Manager.
|
class |
ResourceStatusHandler
A ResourceStatusProto message comes from the ResourceManager layer to indicate what it thinks
about the current state of a given resource.
|
Modifier and Type | Class and Description |
---|---|
class |
CompletedTaskImpl
Implementation of CompletedTask.
|
class |
RunningTaskImpl
Implements the RunningTask client interface.
|
class |
SuspendedTaskImpl
Implementation of SuspendedTask.
|
class |
TaskMessageImpl
Implementation of TaskMessage.
|
class |
TaskRepresenter
Represents a Task on the Driver.
|
Modifier and Type | Class and Description |
---|---|
class |
ContextManager
Manages the stack of context in the Evaluator.
|
class |
ContextRuntime
The evaluator side resourcemanager for Contexts.
|
Modifier and Type | Class and Description |
---|---|
class |
TaskRuntime
The execution environment for a Task.
|
Modifier and Type | Class and Description |
---|---|
class |
DefaultCloseHandler
Default implementation for
EventHandler<CloseEvent> . |
class |
DefaultDriverMessageHandler
A default implementation of
EventHandler<DriverMessage> . |
class |
DefaultSuspendHandler
Default handler for SuspendEvent.
|
Modifier and Type | Class and Description |
---|---|
class |
TaskCallFailure
Thrown when a Task.call() throws an exception.
|
class |
TaskCloseHandlerFailure
Thrown when a Task Close Handler throws an exception.
|
class |
TaskMessageHandlerFailure
Thrown when a Task Message Handler throws an exception.
|
class |
TaskStartHandlerFailure
Thrown when a TaskStart handler throws an exception.
|
class |
TaskStopHandlerFailure
Thrown when a TaskStop handler throws an exception.
|
class |
TaskSuspendHandlerFailure
Thrown when a Task Suspend Handler throws an exception.
|
Modifier and Type | Class and Description |
---|---|
class |
JobJarMaker
Utility that takes a JobSubmissionProto and turns it into a Job Submission Jar.
|
Modifier and Type | Class and Description |
---|---|
class |
REEFMessageCodec
Codec for REEF's control flow messages.
|
Modifier and Type | Class and Description |
---|---|
class |
DispatchingEStage
Delayed event router that dispatches messages to the proper event handler by type.
|
Modifier and Type | Class and Description |
---|---|
class |
HDInsightDriverConfiguration
ConfigurationModule to create a Driver configuration.
|
class |
HDInsightJobSubmissionHandler
Handles job submission to a HDInsight instance.
|
Modifier and Type | Class and Description |
---|---|
class |
LocalResourceReleaseHandler
Takes Resource Release requests and patches them through to the resource
manager.
|
class |
ResourceManager
A resource manager that uses threads to execute containers.
|
class |
RuntimeIdentifier
Runtime Identifier Implementation.
|
Modifier and Type | Class and Description |
---|---|
class |
RuntimeIdentifier
Runtime Identifier Implementation.
|
Modifier and Type | Class and Description |
---|---|
class |
DefaultRackNameFormatter
Default implementation of rack names formatter.
|
class |
RuntimeIdentifier
Runtime Identifier Implementation.
|
class |
YarnDriverRuntimeRestartManager
The implementation of restart manager for YARN.
|
class |
YarnResourceRequestHandler
Accepts resource requests from the REEF layer, translates them into requests for YARN and hands them to the
appropriate handler for those.
|
Modifier and Type | Class and Description |
---|---|
class |
DFSEvaluatorLogAppendWriter
The DFS evaluator logger that performs regular append.
|
class |
DFSEvaluatorLogOverwriteWriter
The DFS evaluator logger that does not support append and does append by overwrite.
|
interface |
DFSEvaluatorLogWriter
The Evaluator log writer that writes to DFS.
|
Modifier and Type | Class and Description |
---|---|
class |
YarnTypes
Helper class that creates the various records in the YARN API.
|
class |
YarnUtilities
A helper class for YARN applications.
|
Modifier and Type | Class and Description |
---|---|
class |
ExceptionHandlingEventHandler<T>
An ExceptionHandler that wraps another one, but catches all exceptions thrown by that one and forwards them
to an ExceptionHandler.
|
Modifier and Type | Method and Description |
---|---|
void |
VortexFuture.aggregationCompleted(List<Integer> taskletIds,
byte[] serializedResult)
VortexMaster should never call this.
|
void |
VortexAggregateFuture.aggregationCompleted(List<Integer> taskletIds,
byte[] serializedResult)
Aggregation has completed for a list of Tasklets, with an aggregated result.
|
void |
VortexFuture.aggregationThrewException(List<Integer> taskletIds,
Exception exception)
VortexMaster should never call this.
|
void |
VortexAggregateFuture.aggregationThrewException(List<Integer> taskletIds,
Exception exception)
A list of Tasklets has failed during aggregation phase.
|
void |
VortexFuture.cancelled(int pTaskletId)
Called by VortexMaster to let the user know that the Tasklet was cancelled.
|
void |
VortexAggregateFuture.cancelled(int taskletId)
Not implemented for local aggregation.
|
void |
VortexFuture.completed(int pTaskletId,
byte[] serializedResult)
Called by VortexMaster to let the user know that the Tasklet completed.
|
void |
VortexAggregateFuture.completed(int taskletId,
byte[] serializedResult)
A Tasklet associated with the aggregation has completed.
|
void |
VortexFuture.threwException(int pTaskletId,
Exception exception)
Called by VortexMaster to let the user know that the Tasklet threw an exception.
|
void |
VortexAggregateFuture.threwException(int taskletId,
Exception exception)
A Tasklet associated with the aggregation has failed.
|
Constructor and Description |
---|
VortexAggregateException(String message,
Throwable cause,
List<?> inputList) |
VortexAggregateException(Throwable cause,
List<?> inputList) |
VortexAggregateFuture(Executor executor,
Map<Integer,TInput> taskletIdInputMap,
Codec<TOutput> aggOutputCodec,
FutureCallback<AggregateResult<TInput,TOutput>> callbackHandler) |
VortexFuture(Executor executor,
VortexMaster vortexMaster,
int taskletId,
Codec<TOutput> outputCodec)
Creates a
VortexFuture . |
VortexFuture(Executor executor,
VortexMaster vortexMaster,
int taskletId,
Codec<TOutput> outputCodec,
FutureCallback<TOutput> callbackHandler)
Creates a
VortexFuture with a callback. |
Modifier and Type | Class and Description |
---|---|
class |
AggregateFunctionRepository
A repository for
VortexAggregateFunction and its associated VortexFunction ,
used to pass functions between VortexMaster and RunningWorkers, as well as used to cache functions
for VortexWorkers on AggregateRequests and AggregateExecutionRequests. |
class |
TaskletAggregateExecutionRequest<TInput>
A request from the Vortex Driver to run an aggregate-able function.
|
class |
TaskletAggregationRequest<TInput,TOutput>
A request from the Vortex Driver for the
VortexWorker to
record aggregate functions for later execution. |
class |
TaskletAggregationResultReport
Report of a Tasklet aggregation execution result.
|
class |
TaskletExecutionRequest<TInput,TOutput>
Request to execute a tasklet.
|
class |
TaskletFailureReport
Report of a Tasklet exception.
|
interface |
TaskletReport
The interface for a status report from the
VortexWorker . |
class |
TaskletResultReport
Report of a tasklet execution result.
|
class |
VortexAvroUtils
Serialize and deserialize Vortex message to/from byte array.
|
interface |
VortexFutureDelegate
Exposes functions to be called by the
VortexMaster
to note that a list of Tasklets associated with a Future has completed. |
class |
WorkerReport
Worker-to-Master protocol.
|
Modifier and Type | Method and Description |
---|---|
Configuration |
VortexJobConf.getConfiguration()
Convert to the Tang Configuration.
|
Copyright © 2016 The Apache Software Foundation. All rights reserved.