public class DefaultContainerExecutor extends ContainerExecutor
DefaultContainerExecuter class offers generic container
 execution services. Process execution is handled in a platform-independent
 way via ProcessBuilder.| Modifier and Type | Class and Description | 
|---|---|
| protected class  | DefaultContainerExecutor.LocalWrapperScriptBuilderThis class is a utility to create a wrapper script that is platform
 appropriate. | 
ContainerExecutor.DelayedProcessKiller, ContainerExecutor.ExitCode, ContainerExecutor.Signal| Modifier and Type | Field and Description | 
|---|---|
| protected org.apache.hadoop.fs.FileContext | lfsA  FileContextfor the local file system. | 
DIRECTORY_CONTENTS, TASK_LAUNCH_SCRIPT_PERMISSION, TOKEN_FILE_NAME_FMT, WILDCARD| Constructor and Description | 
|---|
| DefaultContainerExecutor()Default constructor for use in testing. | 
| Modifier and Type | Method and Description | 
|---|---|
| protected org.apache.hadoop.util.Shell.CommandExecutor | buildCommandExecutor(String wrapperScriptPath,
                    String containerIdStr,
                    String user,
                    org.apache.hadoop.fs.Path pidFile,
                    org.apache.hadoop.yarn.api.records.Resource resource,
                    File workDir,
                    Map<String,String> environment)Create a new  Shell.ShellCommandExecutorusing the parameters. | 
| void | clearLogDirPermissions()Clear the internal variable for repeatable testing. | 
| static boolean | containerIsAlive(String pid)Returns true if the process with the specified pid is alive. | 
| protected void | copyFile(org.apache.hadoop.fs.Path src,
        org.apache.hadoop.fs.Path dst,
        String owner)Copy a file using the  lfsFileContext. | 
| protected ContainerLocalizer | createContainerLocalizer(String user,
                        String appId,
                        String locId,
                        String tokenFileName,
                        List<String> localDirs,
                        org.apache.hadoop.fs.FileContext localizerFc)Create a new  ContainerLocalizerinstance. | 
| protected void | createDir(org.apache.hadoop.fs.Path dirPath,
         org.apache.hadoop.fs.permission.FsPermission perms,
         boolean createParent,
         String user)Use the  lfsFileContextto create the target directory. | 
| void | deleteAsUser(DeletionAsUserContext ctx)Delete specified directories as a given user. | 
| org.apache.hadoop.hdfs.protocol.datatransfer.IOStreamPair | execContainer(ContainerExecContext ctx)Perform interactive docker command into running container. | 
| protected DefaultContainerExecutor.LocalWrapperScriptBuilder | getLocalWrapperScriptBuilder(String containerIdStr,
                            org.apache.hadoop.fs.Path containerWorkDir)Create a  DefaultContainerExecutor.LocalWrapperScriptBuilderfor the given container ID
 and path that is appropriate to the current platform. | 
| String | getLogDirPermissions()Return the default container log directory permissions. | 
| protected org.apache.hadoop.fs.Path | getWorkingDir(List<String> localDirs,
             String user,
             String appId)Return a randomly chosen application directory from a list of local storage
 directories. | 
| void | init(Context nmContext)Run the executor initialization steps. | 
| boolean | isContainerAlive(ContainerLivenessContext ctx)Check if a container is alive. | 
| protected void | killContainer(String pid,
             ContainerExecutor.Signal signal)Send a specified signal to the specified pid | 
| int | launchContainer(ContainerStartContext ctx)Launch the container on the node. | 
| boolean | reapContainer(ContainerReapContext ctx)No-op for reaping containers within the DefaultContainerExecutor. | 
| int | relaunchContainer(ContainerStartContext ctx)Relaunch the container on the node. | 
| protected void | setScriptExecutable(org.apache.hadoop.fs.Path script,
                   String owner)Make a file executable using the  lfsFileContext. | 
| boolean | signalContainer(ContainerSignalContext ctx)Signal container with the specified signal. | 
| void | startLocalizer(LocalizerStartContext ctx)Prepare the environment for containers in this application to execute. | 
| void | symLink(String target,
       String symlink)Create a symlink file which points to the target. | 
| void | updateYarnSysFS(Context ctx,
               String user,
               String appId,
               String spec)Update cluster information inside container. | 
activateContainer, cleanupBeforeRelaunch, deactivateContainer, getConf, getExposedPorts, getIpAndHost, getLocalIpAndHost, getLocalResources, getNMEnvVar, getPidFilePath, getProcessId, getRunCommand, getRunCommand, getRunCommandForOther, getRunCommandForWindows, isContainerActive, localizeClasspathJar, logOutput, pauseContainer, prepareContainer, reacquireContainer, readDirAsUser, resumeContainer, setConf, start, stop, writeLaunchEnv, writeLaunchEnvprotected final org.apache.hadoop.fs.FileContext lfs
FileContext for the local file system.public DefaultContainerExecutor()
protected void copyFile(org.apache.hadoop.fs.Path src,
                        org.apache.hadoop.fs.Path dst,
                        String owner)
                 throws IOException
lfs FileContext.src - the file to copydst - where to copy the fileowner - the owner of the new copy. Used only in secure Windows
 clustersIOException - when the copy failsWindowsSecureContainerExecutorprotected void setScriptExecutable(org.apache.hadoop.fs.Path script,
                                   String owner)
                            throws IOException
lfs FileContext.script - the path to make executableowner - the new owner for the file. Used only in secure Windows
 clustersIOException - when the change mode operation failsWindowsSecureContainerExecutorpublic void init(Context nmContext) throws IOException
ContainerExecutorinit in class ContainerExecutornmContext - Context of NMIOException - if initialization failspublic void startLocalizer(LocalizerStartContext ctx) throws IOException, InterruptedException
ContainerExecutorFor $x in local.dirs create $x/$user/$appId Copy $nmLocal/appTokens -> $N/$user/$appId For $rsrc in private resources Copy $rsrc -> $N/$user/filecache/[idef] For $rsrc in job resources Copy $rsrc -> $N/$user/$appId/filecache/idef
startLocalizer in class ContainerExecutorctx - LocalizerStartContext that encapsulates necessary information
            for starting a localizer.IOException - for most application init failuresInterruptedException - if application init thread is halted by NM@InterfaceAudience.Private protected ContainerLocalizer createContainerLocalizer(String user, String appId, String locId, String tokenFileName, List<String> localDirs, org.apache.hadoop.fs.FileContext localizerFc) throws IOException
ContainerLocalizer instance.user - the user who owns the job for which the localization is being
 runappId - the ID of the application for which the localization is being
 runlocId - the ID of the container for which the localization is being
 runlocalDirs - a list of directories to use as destinations for the
 localizationlocalizerFc - the FileContext to use when localizing filesContainerLocalizer instanceIOException - if user or locId is null or if
 the container localizer has an initialization failurepublic int launchContainer(ContainerStartContext ctx) throws IOException, org.apache.hadoop.yarn.exceptions.ConfigurationException
ContainerExecutorlaunchContainer in class ContainerExecutorctx - Encapsulates information necessary for launching containers.IOException - if the container launch failsorg.apache.hadoop.yarn.exceptions.ConfigurationException - if config error was foundpublic int relaunchContainer(ContainerStartContext ctx) throws IOException, org.apache.hadoop.yarn.exceptions.ConfigurationException
ContainerExecutorrelaunchContainer in class ContainerExecutorctx - Encapsulates information necessary for relaunching containers.IOException - if the container relaunch failsorg.apache.hadoop.yarn.exceptions.ConfigurationException - if config error was foundprotected org.apache.hadoop.util.Shell.CommandExecutor buildCommandExecutor(String wrapperScriptPath, String containerIdStr, String user, org.apache.hadoop.fs.Path pidFile, org.apache.hadoop.yarn.api.records.Resource resource, File workDir, Map<String,String> environment)
Shell.ShellCommandExecutor using the parameters.wrapperScriptPath - the path to the script to executecontainerIdStr - the container IDuser - the application owner's usernamepidFile - the path to the container's PID fileresource - this parameter controls memory and CPU limits.workDir - If not-null, specifies the directory which should be set
 as the current working directory for the command. If null,
 the current working directory is not modified.environment - the container environmentShell.ShellCommandExecutorShell.ShellCommandExecutorprotected DefaultContainerExecutor.LocalWrapperScriptBuilder getLocalWrapperScriptBuilder(String containerIdStr, org.apache.hadoop.fs.Path containerWorkDir)
DefaultContainerExecutor.LocalWrapperScriptBuilder for the given container ID
 and path that is appropriate to the current platform.containerIdStr - the container IDcontainerWorkDir - the container's working directoryDefaultContainerExecutor.LocalWrapperScriptBuilderpublic boolean signalContainer(ContainerSignalContext ctx) throws IOException
ContainerExecutorsignalContainer in class ContainerExecutorctx - Encapsulates information necessary for signaling containers.IOException - if signaling the container failspublic boolean reapContainer(ContainerReapContext ctx)
reapContainer in class ContainerExecutorctx - Encapsulates information necessary for reaping containers.public boolean isContainerAlive(ContainerLivenessContext ctx) throws IOException
ContainerExecutorisContainerAlive in class ContainerExecutorctx - Encapsulates information necessary for container liveness check.IOException - if there is a failure while checking the container
 statuspublic static boolean containerIsAlive(String pid) throws IOException
pid - String pidIOException - if the command to test process liveliness failsprotected void killContainer(String pid, ContainerExecutor.Signal signal) throws IOException
pid - the pid of the process [group] to signal.signal - signal to sendIOException - if the command to kill the process failspublic void deleteAsUser(DeletionAsUserContext ctx) throws IOException, InterruptedException
ContainerExecutordeleteAsUser in class ContainerExecutorctx - Encapsulates information necessary for deletion.IOException - if delete failsInterruptedException - if interrupted while waiting for the deletion
 operation to completepublic void symLink(String target, String symlink) throws IOException
ContainerExecutorsymLink in class ContainerExecutortarget - The target for symlinksymlink - the symlink fileIOException - Error when creating symlinksprotected org.apache.hadoop.fs.Path getWorkingDir(List<String> localDirs, String user, String appId) throws IOException
localDirs - the target directories from which to selectuser - the user who owns the applicationappId - the application IDIOException - if no application directories for the user can be
 foundprotected void createDir(org.apache.hadoop.fs.Path dirPath,
                         org.apache.hadoop.fs.permission.FsPermission perms,
                         boolean createParent,
                         String user)
                  throws IOException
lfs FileContext to create the target directory.dirPath - the target directoryperms - the target permissions for the target directorycreateParent - whether the parent directories should also be createduser - the user as whom the target directory should be created.
 Used only on secure Windows hosts.IOException - if there's a failure performing a file operationWindowsSecureContainerExecutorpublic String getLogDirPermissions()
public void clearLogDirPermissions()
public org.apache.hadoop.hdfs.protocol.datatransfer.IOStreamPair execContainer(ContainerExecContext ctx) throws ContainerExecutionException
ContainerExecutorexecContainer in class ContainerExecutorctx - Encapsulates information necessary for exec containers.ContainerExecutionExceptionpublic void updateYarnSysFS(Context ctx, String user, String appId, String spec) throws IOException
ContainerExecutorupdateYarnSysFS in class ContainerExecutorctx - ContainerRuntimeContextuser - Owner of applicationappId - YARN application IDspec - Service SpecificationIOException - if there is a failure while writing spec to diskCopyright © 2008–2023 Apache Software Foundation. All rights reserved.