Package | Description |
---|---|
org.apache.reef.client |
Client APIs for REEF.
|
org.apache.reef.driver |
Driver APIs for REEF in terms of Rx.
|
org.apache.reef.driver.catalog |
Interfaces describing resources available to a REEF instance (physical nodes, racks etc).
|
org.apache.reef.driver.parameters |
Named parameters used by the Driver.
|
org.apache.reef.driver.restart |
This package provides restart event service handlers.
|
org.apache.reef.io |
APIs for I/O in REEF:
Codec s and
Serializer s. |
org.apache.reef.io.data.loading.api |
Data Loading Service API.
|
org.apache.reef.io.data.loading.impl |
Implementations of Data Loading Service.
|
org.apache.reef.io.network |
TODO: Document.
|
org.apache.reef.io.watcher |
Watcher subscribes events and writes them to the destination using a certain EventStream.
|
org.apache.reef.io.watcher.param |
Parameters for Watcher.
|
org.apache.reef.io.watcher.util |
Utility classes for Watcher.
|
org.apache.reef.javabridge |
The Java-side of the CLR/Java bridge.
|
org.apache.reef.javabridge.generic |
Generic java bridge driver/client.
|
org.apache.reef.runtime.common.driver |
Implementation of the Driver-side REEF APIs.
|
org.apache.reef.runtime.standalone.client |
Client-side event handlers for the standalone resourcemanager.
|
org.apache.reef.runtime.standalone.client.parameters |
Parameters of the standalone runtime.
|
org.apache.reef.runtime.yarn.driver |
The Driver-Side implementation of the YARN adapter for REEF.
|
org.apache.reef.runtime.yarn.driver.restart |
The package contains classes that are used specifically on restart.
|
org.apache.reef.task |
Tasks.
|
org.apache.reef.vortex.api |
Vortex user api for running jobs on Vortex.
|
org.apache.reef.vortex.common |
Vortex Code used both in Vortex Driver and Vortex Evaluator.
|
org.apache.reef.vortex.driver |
Vortex Code that runs as part of REEF Driver.
|
org.apache.reef.vortex.evaluator |
Vortex Code that runs as part of REEF Evaluator.
|
Package | Description |
---|---|
org.apache.reef.io |
APIs for I/O in REEF:
Codec s and
Serializer s. |
Modifier and Type | Class and Description |
---|---|
class |
DriverRestartConfiguration
EventHandlers specific to Driver Restart.
|
Modifier and Type | Interface and Description |
---|---|
interface |
FlexiblePreemptionEvent
Represents a flexible preemption request: It contains:
|
interface |
PreemptionEvent
Represents Preemption requests issued by the underlying resource manager.
|
interface |
PreemptionHandler
This EventHandler will receive preemption events from the underlying resourcemanager.
|
interface |
StrictPreemptionEvent
Represents a strict preemption event: It contains the set of Evaluators that the underlying resource manager will
take away from the Driver.
|
Modifier and Type | Interface and Description |
---|---|
interface |
NodeDescriptor
Descriptor of the physical setup of an Evaluator.
|
interface |
RackDescriptor
A rack in the cluster.
|
interface |
ResourceCatalog
A catalog of the resources available to a REEF instance.
|
Modifier and Type | Class and Description |
---|---|
class |
DriverRestartEvaluatorRecoverySeconds
Represents the amount of time in seconds that the driver restart waits for evaluators to report back.
|
Modifier and Type | Interface and Description |
---|---|
interface |
DriverRestartCompleted
The object used to notify the user that the Driver Restart process has been completed.
|
interface |
DriverRestarted
Am event encapsulating the time of Driver restart as well as
the set of Evaluator IDs of Evaluators that are expected to
report back to the Driver after restart.
|
class |
DriverRestartedImpl |
class |
DriverRestartManager
The manager that handles aspects of driver restart such as determining whether the driver is in
restart mode, what to do on restart, whether restart is completed, and others.
|
class |
DriverRestartState
Represents the current driver restart progress.
|
interface |
DriverRuntimeRestartManager
Classes implementing this interface are in charge of recording evaluator
changes as they are allocated as well as recovering Evaluators and
discovering which evaluators are lost on the event of a driver restart.
|
class |
EvaluatorRestartInfo
An object that encapsulates the information needed to construct an
EvaluatorManager for a recovered evaluator
on restart. |
class |
EvaluatorRestartState
The state that the evaluator is in in the driver restart process.
|
class |
RestartEvaluators
Represents holds the set of Evaluator information needed to recover EvaluatorManagers
on the restarted Driver.
|
Modifier and Type | Interface and Description |
---|---|
interface |
Accumulable<T>
Similar to
Iterable , this is an interface for objects that can return
Accumulator s. |
interface |
Accumulator<T>
A basic Accumulator interface.
|
interface |
ExternalMap<T>
A Map interface for data that can be swapped out to other processes, managed
by native code, paged to disk, stored remotely, etc...
|
class |
PartitionSpec
This was another more type safe alternative to integer partitions.
|
interface |
Spool<T>
Spool files can be appended to, and can be scanned starting at the beginning.
|
class |
Tuple<K,V>
A key-value pair of elements.
|
Modifier and Type | Class and Description |
---|---|
class |
DistributedDataSet
Represents a distributed data set that is split across data centers.
|
interface |
EvaluatorToPartitionStrategy<V extends org.apache.hadoop.mapred.InputSplit>
Interface that tracks the mapping between evaluators and the data partitions
assigned to those evaluators.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractEvaluatorToPartitionStrategy
This is an abstract class useful for
EvaluatorToPartitionStrategy
implementations. |
class |
DistributedDataSetPartition
POJO that represents a distributed data set partition.
|
class |
MultiDataCenterEvaluatorToPartitionStrategy
This is an online version which satisfies requests based on the locations the
users ask the data to be loaded, for multiple data center network topologies.
|
Modifier and Type | Interface and Description |
---|---|
interface |
NetworkConnectionService
NetworkConnectionService.
|
Modifier and Type | Interface and Description |
---|---|
interface |
EventStream
Write events to a certain destination.
|
class |
EventType
Event types that Watcher subscribes.
|
class |
FileEventStream
Write events to a file in the root directory of the driver.
|
class |
LogEventStream
Write events to driver.err using Logger.
|
class |
Watcher
Subscribe events and transfer them as wrapping with corresponding avro classes.
|
class |
WatcherConfiguration
ConfigurationModule for Watcher.
|
Modifier and Type | Class and Description |
---|---|
class |
EventStreams
Set of EventStreams.
|
Modifier and Type | Class and Description |
---|---|
class |
RunnableExecutingHandler
Handler that executes Runnable commands.
|
class |
WatcherAvroUtil |
Modifier and Type | Class and Description |
---|---|
class |
DriverRestartedBridge
The Java-CLR bridge object for
DriverRestarted events. |
Modifier and Type | Class and Description |
---|---|
class |
DriverRestartCompletedBridge
The Java-CLR bridge object for
DriverRestartCompleted events. |
Modifier and Type | Interface and Description |
---|---|
interface |
EvaluatorPreserver
A interface to preserve evaluators across driver restarts.
|
Modifier and Type | Class and Description |
---|---|
class |
StandaloneRuntimeConfiguration
A ConfigurationModule to configure the standalone resourcemanager.
|
Modifier and Type | Class and Description |
---|---|
class |
NodeListFilePath
The file which will contain information of remote nodes.
|
class |
RootFolder
The folder where logs etc.
|
Modifier and Type | Class and Description |
---|---|
class |
YarnDriverRestartConfiguration
Use this ConfigurationModule to configure YARN-specific Restart options for the driver.
|
class |
YarnDriverRuntimeRestartManager
The implementation of restart manager for YARN.
|
Modifier and Type | Class and Description |
---|---|
class |
DFSEvaluatorPreserver
An Evaluator Preserver that uses the DFS on YARN.
|
Modifier and Type | Class and Description |
---|---|
class |
HeartBeatTriggerManager
Helper class for immediately sending heartbeat messages.
|
Modifier and Type | Class and Description |
---|---|
class |
AggregateResult<TInput,TOutput>
The result of an aggregate.
|
class |
AggregateResultSynchronous<TInput,TOutput>
The synchronous result of an aggregate, returned by
VortexAggregateFuture.get() . |
class |
VortexAggregateException
Exception thrown when an aggregate function fails.
|
interface |
VortexAggregateFunction<TOutput>
Typed user function for Local Aggregation.
|
class |
VortexAggregateFuture<TInput,TOutput>
The interface between user code and aggregation Tasklets.
|
class |
VortexAggregatePolicy
The policy for local aggregation on the
VortexWorker s. |
interface |
VortexFunction<TInput,TOutput>
Typed user function.
|
class |
VortexFuture<TOutput>
The interface between user code and submitted task.
|
interface |
VortexStart
The starting point of Vortex user thread.
|
class |
VortexThreadPool
Distributed thread pool.
|
Modifier and Type | Class and Description |
---|---|
class |
AggregateFunctionRepository
A repository for
VortexAggregateFunction and its associated VortexFunction ,
used to pass functions between VortexMaster and RunningWorkers, as well as used to cache functions
for VortexWorkers on AggregateRequests and AggregateExecutionRequests. |
class |
TaskletAggregateExecutionRequest<TInput>
A request from the Vortex Driver to run an aggregate-able function.
|
class |
TaskletAggregationFailureReport
Report of a tasklet exception on aggregation.
|
class |
TaskletAggregationRequest<TInput,TOutput>
A request from the Vortex Driver for the
VortexWorker to
record aggregate functions for later execution. |
class |
TaskletAggregationResultReport
Report of a Tasklet aggregation execution result.
|
class |
TaskletCancellationRequest
A
VortexRequest to cancel tasklets. |
class |
TaskletCancelledReport
The report of a cancelled Tasklet.
|
class |
TaskletExecutionRequest<TInput,TOutput>
Request to execute a tasklet.
|
class |
TaskletFailureReport
Report of a Tasklet exception.
|
interface |
TaskletReport
The interface for a status report from the
VortexWorker . |
class |
TaskletResultReport
Report of a tasklet execution result.
|
class |
VortexAvroUtils
Serialize and deserialize Vortex message to/from byte array.
|
interface |
VortexFutureDelegate
Exposes functions to be called by the
VortexMaster
to note that a list of Tasklets associated with a Future has completed. |
interface |
VortexRequest
Master-to-Worker protocol.
|
class |
WorkerReport
Worker-to-Master protocol.
|
Modifier and Type | Class and Description |
---|---|
class |
VortexJobConf
Helper class for building a configuration for Vortex.
|
class |
VortexLauncher
Launches a Vortex Job.
|
interface |
VortexMaster
The heart of Vortex.
|
class |
VortexMasterConf
Vortex Master configuration.
|
class |
VortexWorkerConf
Vortex Worker configuration.
|
Modifier and Type | Class and Description |
---|---|
class |
VortexWorker
Receives commands from VortexMaster, executes them, and returns the results.
|
Copyright © 2016 The Apache Software Foundation. All rights reserved.