org.apache.reef.runtime.yarn.driver.restart

Class DFSEvaluatorLogOverwriteReaderWriter

java.lang.Object

org.apache.reef.runtime.yarn.driver.restart.DFSEvaluatorLogOverwriteReaderWriter

All Implemented Interfaces:

AutoCloseable, DFSEvaluatorLogReaderWriter

public final class DFSEvaluatorLogOverwriteReaderWriter
extends Object
implements DFSEvaluatorLogReaderWriter

The DFS evaluator logger that does not support append and does append by overwrite. dfs.support.append should be false.

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Closes the FileSystem.
CloseableIterable<String>	readFromEvaluatorLog() Since the FileSystem does not support appends, this DFSEvaluatorLogReaderWriter uses a two-file approach, where when we read, we always read from the newer file.
void	writeToEvaluatorLog(String formattedEntry) Writes a formatted entry (addition or removal) for an Evaluator ID into the DFS evaluator log.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

writeToEvaluatorLog

public void writeToEvaluatorLog(String formattedEntry)
                         throws IOException

Writes a formatted entry (addition or removal) for an Evaluator ID into the DFS evaluator log. The log is appended to by reading first, adding on the information, and then overwriting the entire log. Since the FileSystem does not support appends, this DFSEvaluatorLogReaderWriter uses a two-file approach, where when we write, we always overwrite the older file.

Specified by:

writeToEvaluatorLog in interface DFSEvaluatorLogReaderWriter

Parameters:

formattedEntry - The formatted entry (entry with evaluator ID and addition/removal information).

Throws:

IOException - when file cannot be written.

readFromEvaluatorLog

public CloseableIterable<String> readFromEvaluatorLog()
                                               throws IOException

Since the FileSystem does not support appends, this DFSEvaluatorLogReaderWriter uses a two-file approach, where when we read, we always read from the newer file.

Specified by:

readFromEvaluatorLog in interface DFSEvaluatorLogReaderWriter

Returns:

the formatted entry.

Throws:

IOException

close

public void close()
           throws Exception

Closes the FileSystem.

Specified by:

close in interface AutoCloseable

Throws:

Exception