Package | Description |
---|---|
org.apache.reef.driver |
Driver APIs for REEF in terms of Rx.
|
org.apache.reef.driver.evaluator |
Evaluator-related utilities.
|
org.apache.reef.driver.restart |
This package provides restart event service handlers.
|
org.apache.reef.evaluator.context.parameters |
Named parameters used by the Evaluator.
|
org.apache.reef.io.data.loading.impl |
Implementations of Data Loading Service.
|
org.apache.reef.io.network.group.api.config |
The operator configurations used by users.
|
org.apache.reef.io.network.group.api.driver |
Elastic Group Communications Driver-side API Interfaces.
|
org.apache.reef.io.network.group.api.task |
Elastic Group Communications Task-side API Interfaces.
|
org.apache.reef.io.network.group.impl.driver |
This package contains the implementation of the driver side of the
Group Communication Service using the tree/flat topology.
|
org.apache.reef.io.watcher.param |
Parameters for Watcher.
|
org.apache.reef.io.watcher.util |
Utility classes for Watcher.
|
org.apache.reef.javabridge |
The Java-side of the CLR/Java bridge.
|
org.apache.reef.javabridge.generic |
Generic java bridge driver/client.
|
org.apache.reef.runtime.common.client |
Implementation of the client-side REEF API.
|
org.apache.reef.runtime.common.client.defaults |
Default implementations for the optional client-side event handlers.
|
org.apache.reef.runtime.common.driver |
Implementation of the Driver-side REEF APIs.
|
org.apache.reef.runtime.common.driver.catalog |
Implementations of descriptors for resources available to a REEF instance (physical nodes, racks etc).
|
org.apache.reef.runtime.common.driver.client |
Objects representing the Client on the Driver.
|
org.apache.reef.runtime.common.driver.context |
Implementations of Driver-Side representations of Contexts running on an Evaluator.
|
org.apache.reef.runtime.common.driver.evaluator |
Driver-Side representations of Evaluators.
|
org.apache.reef.runtime.common.driver.evaluator.pojos |
POJOs in this package encapsulate the content of
an EvaluatorHeartbeatProto on the DriverSide.
|
org.apache.reef.runtime.common.driver.resourcemanager |
Classes that interface with the resourcemanager (Local, YARN, ...) in the Driver.
|
org.apache.reef.runtime.common.driver.task |
Driver-Side representations of tasks.
|
org.apache.reef.runtime.common.evaluator.context |
Context implementation of the Evaluator resourcemanager.
|
org.apache.reef.runtime.common.evaluator.task |
Task-related implementation of the Evaluator resourcemanager.
|
org.apache.reef.runtime.common.evaluator.task.defaults |
Default implementations for the optional task interfaces.
|
org.apache.reef.runtime.common.evaluator.task.exceptions |
Exceptions thrown by Tasks and Task Handlers.
|
org.apache.reef.runtime.common.files |
This package will contain the implementation of the REEF file system standard.
|
org.apache.reef.runtime.common.launch |
Common launch code between Driver and Evaluator.
|
org.apache.reef.runtime.common.utils |
Runtime utilities used by various REEF modules.
|
org.apache.reef.runtime.hdinsight.client |
The client to submit jobs to HDInsight.
|
org.apache.reef.runtime.hdinsight.driver |
HDInsight support for REEF.
|
org.apache.reef.runtime.local.client |
Client-side event handlers for the local resourcemanager.
|
org.apache.reef.runtime.local.driver |
The resource manager for the local resourcemanager.
|
org.apache.reef.runtime.mesos.driver |
Driver-side event handlers for Mesos resourcemanager.
|
org.apache.reef.runtime.multi.driver |
A multi runtime implementation of REEF that uses several runtime implementationsi for execution.
|
org.apache.reef.runtime.standalone.driver |
The resource manager for the standalone resourcemanager.
|
org.apache.reef.runtime.yarn.client |
Client-side event handlers for YARN resourcemanager.
|
org.apache.reef.runtime.yarn.client.unmanaged |
Client-side event handlers for YARN resourcemanager and AM running in unmanaged mode.
|
org.apache.reef.runtime.yarn.driver |
The Driver-Side implementation of the YARN adapter for REEF.
|
org.apache.reef.runtime.yarn.driver.restart |
The package contains classes that are used specifically on restart.
|
org.apache.reef.runtime.yarn.util |
Utility for YARN runtime.
|
org.apache.reef.tests.fail.driver |
Tests for Driver-side failures.
|
org.apache.reef.util |
Utilities for using REEF on Hadoop.
|
org.apache.reef.vortex.api |
Vortex user api for running jobs on Vortex.
|
org.apache.reef.vortex.common |
Code commonly used in VortexMaster and VortexWorker.
|
org.apache.reef.vortex.driver |
Vortex Code that runs as part of REEF Driver.
|
org.apache.reef.vortex.protocol.mastertoworker |
Protocol from VortexMaster to VortexWorker.
|
org.apache.reef.vortex.protocol.workertomaster |
Protocol from VortexWorker to VortexMaster.
|
Package | Description |
---|---|
org.apache.reef.runtime.common.driver.evaluator |
Driver-Side representations of Evaluators.
|
org.apache.reef.runtime.common.driver.context |
Implementations of Driver-Side representations of Contexts running on an Evaluator.
|
org.apache.reef.runtime.common.driver.task |
Driver-Side representations of tasks.
|
org.apache.reef.runtime.common.driver.evaluator.pojos |
POJOs in this package encapsulate the content of
an EvaluatorHeartbeatProto on the DriverSide.
|
Modifier and Type | Interface and Description |
---|---|
interface |
FlexiblePreemptionEvent
Represents a flexible preemption request: It contains:
|
Modifier and Type | Class and Description |
---|---|
class |
CLRProcessFactory
Factory to setup new CLR processes.
|
Constructor and Description |
---|
EvaluatorRequest.Builder() |
Modifier and Type | Class and Description |
---|---|
class |
DriverRestartedImpl |
class |
DriverRestartManager
The manager that handles aspects of driver restart such as determining whether the driver is in
restart mode, what to do on restart, whether restart is completed, and others.
|
class |
DriverRestartState
Represents the current driver restart progress.
|
interface |
DriverRuntimeRestartManager
Classes implementing this interface are in charge of recording evaluator
changes as they are allocated as well as recovering Evaluators and
discovering which evaluators are lost on the event of a driver restart.
|
class |
EvaluatorRestartInfo
An object that encapsulates the information needed to construct an
EvaluatorManager for a recovered evaluator
on restart. |
class |
EvaluatorRestartState
The state that the evaluator is in in the driver restart process.
|
class |
RestartEvaluators
Represents holds the set of Evaluator information needed to recover EvaluatorManagers
on the restarted Driver.
|
Modifier and Type | Class and Description |
---|---|
class |
Services
A set of classes to be instantiated and shared as singletons within this context and all child context.
|
Modifier and Type | Class and Description |
---|---|
class |
AvroEvaluatorRequestSerializer
Serialize and deserialize EvaluatorRequest objects using Avro.
|
Modifier and Type | Interface and Description |
---|---|
interface |
OperatorSpec
The specification of an operator submitted by the user
while configuring the communication group.
|
Modifier and Type | Interface and Description |
---|---|
interface |
GroupCommServiceDriver |
Modifier and Type | Interface and Description |
---|---|
interface |
CommunicationGroupServiceClient |
Modifier and Type | Class and Description |
---|---|
class |
CommunicationGroupDriverImpl |
Modifier and Type | Class and Description |
---|---|
class |
EventStreams
Set of EventStreams.
|
Modifier and Type | Class and Description |
---|---|
class |
WatcherAvroUtil |
Modifier and Type | Class and Description |
---|---|
class |
ActiveContextBridge
The Java-CLR bridge object for
ActiveContext . |
class |
ActiveContextBridgeFactory
Factory for ActiveContextBridge instances.
|
class |
AllocatedEvaluatorBridge
The AllocatedEvaluatorBridge object to bridge operations between REEF .NET and Java allocated evaluator operations.
|
class |
AllocatedEvaluatorBridgeFactory
The Factory object used to create AllocatedEvaluatorBridge objects.
|
class |
BridgeHandlerManager
A class that holds all handles to the .NET side.
|
class |
ClosedContextBridge
The Java-CLR bridge object for
ClosedContext . |
class |
CompletedEvaluatorBridge
The Java-CLR bridge object for
CompletedEvaluator . |
class |
CompletedTaskBridge
The Java-CLR bridge object for
CompletedTask . |
class |
ContextMessageBridge
The Java-CLR bridge object for
ContextMessage . |
class |
DriverRestartedBridge
The Java-CLR bridge object for
DriverRestarted events. |
class |
EvaluatorRequestorBridge
The Java-CLR bridge object for
EvaluatorRequestor . |
class |
FailedContextBridge
The Java-CLR bridge object for
FailedContext . |
class |
FailedEvaluatorBridge
The Java-CLR bridge object for
FailedEvaluator . |
class |
FailedTaskBridge
The Java-CLR bridge object for
FailedTask . |
class |
HttpServerEventBridge
The Java-CLR bridge object for http server events.
|
class |
InteropLogger
Logger called from CLR code.
|
class |
NativeBridge
Abstract bridge class.
|
class |
NativeInterop
Java interfaces of CLR/Java bridge.
|
class |
RunningTaskBridge
The Java-CLR bridge object for
RunningTask . |
class |
SuspendedTaskBridge
The Java-CLR bridge object for
SuspendedTask . |
class |
TaskMessageBridge
The Java-CLR bridge object for
TaskMessage . |
class |
Utilities
CLR/Java bridge utilities.
|
Modifier and Type | Class and Description |
---|---|
class |
DriverRestartCompletedBridge
The Java-CLR bridge object for
DriverRestartCompleted events. |
Modifier and Type | Class and Description |
---|---|
class |
REEFImplementation
Default REEF implementation.
|
class |
RunningJobImpl
Implementation of RunningJob.
|
Modifier and Type | Class and Description |
---|---|
class |
DefaultUserCredentials
A holder object for REEF user credentials.
|
Modifier and Type | Class and Description |
---|---|
class |
DriverExceptionHandler
Exception handler for exceptions thrown by client code in the Driver.
|
class |
DriverRuntimeConfiguration
ConfigurationModule for driver runtime.
|
class |
DriverRuntimeRestartConfiguration
The base configuration module for driver restart configurations of all runtimes.
|
interface |
EvaluatorPreserver
A interface to preserve evaluators across driver restarts.
|
Modifier and Type | Class and Description |
---|---|
class |
NodeDescriptorImpl
Descriptor of the physical setup of an Evaluator.
|
class |
ResourceCatalogImpl
A catalog of the resources available to a REEF instance.
|
Modifier and Type | Class and Description |
---|---|
class |
ClientManager
Represents the Client in the Driver.
|
Modifier and Type | Class and Description |
---|---|
class |
ClosedContextImpl
Driver side representation of a closed context.
|
class |
ContextControlHandler
Handles context control messages.
|
class |
ContextMessageImpl
Driver-side representation of a context message.
|
class |
ContextRepresenters
Driver-Side representation of all contexts on an Evaluator.
|
class |
EvaluatorContext
Driver-side representation of a Context on an Evaluator.
|
class |
FailedContextImpl
Driver-Side representation of a failed context.
|
Modifier and Type | Class and Description |
---|---|
class |
AllocatedEvaluatorImpl
Driver-Side representation of an allocated evaluator.
|
class |
EvaluatorControlHandler
This class handles the sending of Evaluator control messages to the Evaluator.
|
class |
EvaluatorHeartbeatHandler
Receives heartbeats from all Evaluators and dispatches them to the right EvaluatorManager instance.
|
class |
EvaluatorIdlenessThreadPool
Runs threads in a thread pool to check the completion of Evaluators on the closing
of an
EvaluatorManager in order to trigger Evaluator idleness checks. |
class |
EvaluatorManager
Manages a single Evaluator instance including all lifecycle instances:
(AllocatedEvaluator, CompletedEvaluator, FailedEvaluator).
|
class |
EvaluatorManagerFactory
Helper class that creates new EvaluatorManager instances from allocations.
|
class |
EvaluatorResourceManagerErrorHandler
The error handler receives all resourcemanager errors from all evaluators in the system.
|
class |
Evaluators
Manages all Evaluators.
|
Modifier and Type | Class and Description |
---|---|
class |
ContextMessagePOJO
DriverSide representation of ContextMessageProto.
|
class |
ContextState
DriverSide representation of ContextStatusProto.State.
|
class |
ContextStatusPOJO
DriverSide representation of ContextStatusProto.
|
class |
EvaluatorStatusPOJO
DriverSide representation of EvaluatorStatusProto.
|
class |
State
DriverSide representation of ReefServiceProtos.State.
|
class |
TaskMessagePOJO
DriverSide representation of TaskMessageProto.
|
class |
TaskStatusPOJO
DriverSide representation of TaskStatusProto.
|
Modifier and Type | Class and Description |
---|---|
class |
NodeDescriptorHandler
Updates the ResourceCatalog with a new Node.
|
class |
ResourceAllocationHandler
Handles new resource allocations by adding a new EvaluatorManager.
|
interface |
ResourceEvent
An interface capturing the characteristics of a resource event.
|
class |
ResourceManagerStatus
Manages the status of the Resource Manager and tracks whether it is idle.
|
class |
ResourceStatusHandler
A ResourceStatusProto message comes from the ResourceManager layer to indicate what it thinks
about the current state of a given resource.
|
Modifier and Type | Class and Description |
---|---|
class |
CompletedTaskImpl
Implementation of CompletedTask.
|
class |
RunningTaskImpl
Implements the RunningTask client interface.
|
class |
SuspendedTaskImpl
Implementation of SuspendedTask.
|
class |
TaskMessageImpl
Implementation of TaskMessage.
|
class |
TaskRepresenter
Represents a Task on the Driver.
|
Modifier and Type | Class and Description |
---|---|
class |
ContextManager
Manages the stack of context in the Evaluator.
|
class |
ContextRuntime
The evaluator side resourcemanager for Contexts.
|
Modifier and Type | Class and Description |
---|---|
class |
TaskRuntime
The execution environment for a Task.
|
Modifier and Type | Class and Description |
---|---|
class |
DefaultCloseHandler
Default implementation for
EventHandler<CloseEvent> . |
class |
DefaultDriverMessageHandler
A default implementation of
EventHandler<DriverMessage> . |
class |
DefaultSuspendHandler
Default handler for SuspendEvent.
|
Modifier and Type | Class and Description |
---|---|
class |
TaskCallFailure
Thrown when a Task.call() throws an exception.
|
class |
TaskCloseHandlerFailure
Thrown when a Task Close Handler throws an exception.
|
class |
TaskMessageHandlerFailure
Thrown when a Task Message Handler throws an exception.
|
class |
TaskStartHandlerFailure
Thrown when a TaskStart handler throws an exception.
|
class |
TaskStopHandlerFailure
Thrown when a TaskStop handler throws an exception.
|
class |
TaskSuspendHandlerFailure
Thrown when a Task Suspend Handler throws an exception.
|
Modifier and Type | Class and Description |
---|---|
class |
JobJarMaker
Utility that takes a JobSubmissionProto and turns it into a Job Submission Jar.
|
Modifier and Type | Class and Description |
---|---|
class |
REEFMessageCodec
Codec for REEF's control flow messages.
|
Modifier and Type | Class and Description |
---|---|
class |
DispatchingEStage
Delayed event router that dispatches messages to the proper event handler by type.
|
Modifier and Type | Class and Description |
---|---|
class |
HDInsightDriverConfiguration
ConfigurationModule to create a Driver configuration.
|
class |
HDInsightJobSubmissionHandler
Handles job submission to a HDInsight instance.
|
Modifier and Type | Class and Description |
---|---|
class |
RuntimeIdentifier
Runtime Identifier Implementation.
|
Modifier and Type | Class and Description |
---|---|
class |
LocalJobSubmissionHandler
Handles Job Submissions for the Local and the Standalone Runtime.
|
Modifier and Type | Interface and Description |
---|---|
interface |
Container
Represents a Container: A slice of a machine.
|
class |
LocalResourceReleaseHandler
Takes Resource Release requests and patches them through to the resource
manager.
|
class |
ProcessContainer
A Container that runs an Evaluator in a Process.
|
class |
ResourceManager
A resource manager that uses threads to execute containers.
|
class |
RuntimeIdentifier
Runtime Identifier Implementation.
|
Modifier and Type | Class and Description |
---|---|
class |
RuntimeIdentifier
Runtime Identifier Implementation.
|
Modifier and Type | Class and Description |
---|---|
class |
MultiRuntimeDriverConfiguration
ConfigurationModule for the multi-runtime Driver.
|
Modifier and Type | Class and Description |
---|---|
class |
StandaloneResourceReleaseHandler
Takes Resource Release requests and closes process managers.
|
Modifier and Type | Class and Description |
---|---|
class |
YarnDriverConfigurationProviderImpl
Default driver configuration provider for yarn runtime.
|
Modifier and Type | Class and Description |
---|---|
class |
YarnProxyUser
A holder class for the proxy UserGroupInformation object
required for Unmanaged YARN Application Master in REEF-on-REEF or REEF-on-Spark mode.
|
Modifier and Type | Class and Description |
---|---|
class |
DefaultRackNameFormatter
Default implementation of rack names formatter.
|
class |
REEFEventHandlers
Helper that represents the REEF layer to the YARN runtime.
|
class |
RuntimeIdentifier
Runtime Identifier Implementation.
|
class |
YarnDriverRuntimeRestartManager
The implementation of restart manager for YARN.
|
class |
YarnResourceRequestHandler
Accepts resource requests from the REEF layer, translates them into requests for YARN and hands them to the
appropriate handler for those.
|
Modifier and Type | Method and Description |
---|---|
void |
REEFEventHandlers.onResourceAllocation(ResourceAllocationEvent resourceAllocationEvent)
Inform REEF of a fresh resource allocation.
|
void |
REEFEventHandlers.onRuntimeStatus(RuntimeStatusEvent runtimeStatusEvent)
Update REEF's view on the runtime status.
|
Modifier and Type | Class and Description |
---|---|
class |
DFSEvaluatorLogAppendReaderWriter
The DFS evaluator logger that performs regular append.
|
class |
DFSEvaluatorLogOverwriteReaderWriter
The DFS evaluator logger that does not support append and does append by overwrite.
|
interface |
DFSEvaluatorLogReaderWriter
The Evaluator log reader/writer that reads from and writes to the Hadoop DFS.
|
Modifier and Type | Class and Description |
---|---|
class |
YarnTypes
Helper class that creates the various records in the YARN API.
|
class |
YarnUtilities
A helper class for YARN applications.
|
Modifier and Type | Class and Description |
---|---|
class |
FailClient
Client for the test REEF job that fails on different stages of execution.
|
Modifier and Type | Class and Description |
---|---|
class |
ExceptionHandlingEventHandler<T>
An ExceptionHandler that wraps another one, but catches all exceptions thrown by that one and forwards them
to an ExceptionHandler.
|
Modifier and Type | Method and Description |
---|---|
void |
VortexFuture.aggregationCompleted(List<Integer> taskletIds,
TOutput result)
VortexMaster should never call this.
|
void |
VortexAggregateFuture.aggregationCompleted(List<Integer> taskletIds,
TOutput result)
Aggregation has completed for a list of Tasklets, with an aggregated result.
|
void |
VortexFuture.aggregationThrewException(List<Integer> taskletIds,
Exception exception)
VortexMaster should never call this.
|
void |
VortexAggregateFuture.aggregationThrewException(List<Integer> taskletIds,
Exception exception)
A list of Tasklets has failed during aggregation phase.
|
void |
VortexFuture.cancelled(int pTaskletId)
Called by VortexMaster to let the user know that the Tasklet was cancelled.
|
void |
VortexAggregateFuture.cancelled(int taskletId)
Not implemented for local aggregation.
|
void |
VortexFuture.completed(int pTaskletId,
TOutput result)
Called by VortexMaster to let the user know that the Tasklet completed.
|
void |
VortexAggregateFuture.completed(int taskletId,
TOutput result)
A Tasklet associated with the aggregation has completed.
|
void |
VortexFuture.threwException(int pTaskletId,
Exception exception)
Called by VortexMaster to let the user know that the Tasklet threw an exception.
|
void |
VortexAggregateFuture.threwException(int taskletId,
Exception exception)
A Tasklet associated with the aggregation has failed.
|
Constructor and Description |
---|
VortexAggregateException(String message,
Throwable cause,
List<?> inputList) |
VortexAggregateException(Throwable cause,
List<?> inputList) |
VortexAggregateFuture(Executor executor,
Map<Integer,TInput> taskletIdInputMap,
FutureCallback<AggregateResult<TInput,TOutput>> callbackHandler) |
VortexFuture(Executor executor,
VortexMaster vortexMaster,
int taskletId)
Creates a
VortexFuture . |
VortexFuture(Executor executor,
VortexMaster vortexMaster,
int taskletId,
FutureCallback<TOutput> callbackHandler)
Creates a
VortexFuture with a callback. |
Modifier and Type | Class and Description |
---|---|
class |
KryoUtils
The one and only serializer for the Vortex protocol.
|
Modifier and Type | Interface and Description |
---|---|
interface |
VortexFutureDelegate<TOutput>
Exposes functions to be called by the
VortexMaster
to note that a list of Tasklets associated with a Future has completed. |
Modifier and Type | Method and Description |
---|---|
Configuration |
VortexJobConf.getConfiguration()
Convert to the Tang Configuration.
|
Modifier and Type | Class and Description |
---|---|
class |
TaskletAggregateExecutionRequest<TInput>
A request from the Vortex Driver to run an aggregate-able function.
|
class |
TaskletAggregationRequest<TInput,TOutput>
A request from the Vortex Driver for the
VortexWorker to
record aggregate functions for later execution. |
class |
TaskletExecutionRequest<TInput,TOutput>
Request to execute a tasklet.
|
Modifier and Type | Class and Description |
---|---|
class |
TaskletAggregationResultReport
Report of a Tasklet aggregation execution result.
|
class |
TaskletFailureReport
Report of a Tasklet exception.
|
class |
TaskletResultReport
Report of a tasklet execution result.
|
interface |
WorkerToMasterReport
The interface for a status report from the
VortexWorker . |
class |
WorkerToMasterReports
Container for multiple WorkerToMasterReports.
|
Copyright © 2017 The Apache Software Foundation. All rights reserved.