public class InputFormatLoadingService<K,V> extends Object implements DataLoadingService
DataLoadingService
that uses the Hadoop InputFormat
to find
partitions of data and request resources.
The InputFormat is taken from the job configurations
The EvaluatorToPartitionStrategy
is injected via Tang,
in order to support different ways to map evaluators to data
Constructor and Description |
---|
InputFormatLoadingService(EvaluatorToPartitionStrategy<org.apache.hadoop.mapred.InputSplit> evaluatorToPartitionStrategy,
boolean inMemory,
String inputFormatClass) |
Modifier and Type | Method and Description |
---|---|
String |
getComputeContextIdPrefix() |
Configuration |
getContextConfiguration(AllocatedEvaluator allocatedEvaluator) |
int |
getNumberOfPartitions()
This method actually returns the number of splits in all partition of the data.
|
Configuration |
getServiceConfiguration(AllocatedEvaluator allocatedEvaluator) |
boolean |
isComputeContext(ActiveContext context) |
boolean |
isDataLoadedContext(ActiveContext context)
Distinguishes data loaded contexts from compute contexts.
|
@Inject public InputFormatLoadingService(EvaluatorToPartitionStrategy<org.apache.hadoop.mapred.InputSplit> evaluatorToPartitionStrategy, boolean inMemory, String inputFormatClass)
public int getNumberOfPartitions()
getNumberOfPartitions
in interface DataLoadingService
public Configuration getContextConfiguration(AllocatedEvaluator allocatedEvaluator)
getContextConfiguration
in interface DataLoadingService
public Configuration getServiceConfiguration(AllocatedEvaluator allocatedEvaluator)
getServiceConfiguration
in interface DataLoadingService
public String getComputeContextIdPrefix()
getComputeContextIdPrefix
in interface DataLoadingService
public boolean isComputeContext(ActiveContext context)
isComputeContext
in interface DataLoadingService
public boolean isDataLoadedContext(ActiveContext context)
DataLoadingService
isDataLoadedContext
in interface DataLoadingService
Copyright © 2017 The Apache Software Foundation. All rights reserved.