@InterfaceAudience.Private @InterfaceStability.Unstable public class DominantResourceCalculator extends ResourceCalculator
ResourceCalculator which uses the concept of
 dominant resource to compare multi-dimensional resources.
 Essentially the idea is that the in a multi-resource environment,
 the resource allocation should be determined by the dominant share
 of an entity (user or queue), which is the maximum share that the
 entity has been allocated of any resource.
 In a nutshell, it seeks to maximize the minimum dominant share across
 all entities.
 For example, if user A runs CPU-heavy tasks and user B runs
 memory-heavy tasks, it attempts to equalize CPU share of user A
 with Memory-share of user B.
 In the single resource case, it reduces to max-min fairness for that resource.
 See the Dominant Resource Fairness paper for more details:
 www.cs.berkeley.edu/~matei/papers/2011/nsdi_drf.pdf| Constructor and Description | 
|---|
| DominantResourceCalculator() | 
| Modifier and Type | Method and Description | 
|---|---|
| int | compare(org.apache.hadoop.yarn.api.records.Resource clusterResource,
       org.apache.hadoop.yarn.api.records.Resource lhs,
       org.apache.hadoop.yarn.api.records.Resource rhs,
       boolean singleType)On a cluster with capacity  clusterResource, comparelhsandrhs. | 
| long | computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available,
                          org.apache.hadoop.yarn.api.records.Resource required)Compute the number of containers which can be allocated given
  availableandrequiredresources. | 
| float | divide(org.apache.hadoop.yarn.api.records.Resource clusterResource,
      org.apache.hadoop.yarn.api.records.Resource numerator,
      org.apache.hadoop.yarn.api.records.Resource denominator)Divide resource  numeratorby resourcedenominatorusing specified policy (domination, average, fairness etc.); hence overallclusterResourceis provided for context. | 
| org.apache.hadoop.yarn.api.records.Resource | divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
             float denominator)Divide-and-ceil  numeratorbydenominator. | 
| org.apache.hadoop.yarn.api.records.Resource | divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
             int denominator)Divide-and-ceil  numeratorbydenominator. | 
| org.apache.hadoop.yarn.api.records.Resource | divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
             long denominator) | 
| boolean | fitsIn(org.apache.hadoop.yarn.api.records.Resource smaller,
      org.apache.hadoop.yarn.api.records.Resource bigger)Check if a smaller resource can be contained by bigger resource. | 
| Set<String> | getInsufficientResourceNames(org.apache.hadoop.yarn.api.records.Resource required,
                            org.apache.hadoop.yarn.api.records.Resource available)Get insufficient resource names via comparing required resource and
 capacity resource. | 
| boolean | isAllInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)Determine if all resources are zero. | 
| boolean | isAnyMajorResourceAboveZero(org.apache.hadoop.yarn.api.records.Resource resource)Check if resource has any major resource types (which are all NodeManagers
 included) has a > 0 value. | 
| boolean | isAnyMajorResourceZeroOrNegative(org.apache.hadoop.yarn.api.records.Resource resource)Check if resource has any major resource types (which are all NodeManagers
 included) a zero value or negative value. | 
| boolean | isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)Determine if a resource is not suitable for use as a divisor
 (will result in divide by 0, etc) | 
| org.apache.hadoop.yarn.api.records.Resource | multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
                        double by,
                        org.apache.hadoop.yarn.api.records.Resource stepFactor)Multiply resource  rby factorbyand normalize down using step-factorstepFactor. | 
| org.apache.hadoop.yarn.api.records.Resource | multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
                      double[] by,
                      org.apache.hadoop.yarn.api.records.Resource stepFactor)Multiply resource  rby factorbyand normalize up using step-factorstepFactor. | 
| org.apache.hadoop.yarn.api.records.Resource | multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
                      double by,
                      org.apache.hadoop.yarn.api.records.Resource stepFactor)Multiply resource  rby factorbyand normalize up using step-factorstepFactor. | 
| org.apache.hadoop.yarn.api.records.Resource | normalize(org.apache.hadoop.yarn.api.records.Resource r,
         org.apache.hadoop.yarn.api.records.Resource minimumResource,
         org.apache.hadoop.yarn.api.records.Resource maximumResource,
         org.apache.hadoop.yarn.api.records.Resource stepFactor)Normalize resource  rgiven the baseminimumResourceand verify against max allowedmaximumResourceusing a step factor for the normalization. | 
| org.apache.hadoop.yarn.api.records.Resource | normalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
             org.apache.hadoop.yarn.api.records.Resource stepFactor)Get resource  rand normalize down using step-factorstepFactor. | 
| float | ratio(org.apache.hadoop.yarn.api.records.Resource a,
     org.apache.hadoop.yarn.api.records.Resource b)Ratio of resource  ato resourceb. | 
| org.apache.hadoop.yarn.api.records.Resource | roundDown(org.apache.hadoop.yarn.api.records.Resource r,
         org.apache.hadoop.yarn.api.records.Resource stepFactor)Round-down resource  rgiven factorstepFactor. | 
| org.apache.hadoop.yarn.api.records.Resource | roundUp(org.apache.hadoop.yarn.api.records.Resource r,
       org.apache.hadoop.yarn.api.records.Resource stepFactor)Round-up resource  rgiven factorstepFactor. | 
compare, divideAndCeil, divideAndCeil, divideAndCeil, divideAndCeil, divideSafelyAsFloat, roundDown, roundDown, roundUp, roundUppublic int compare(org.apache.hadoop.yarn.api.records.Resource clusterResource,
                   org.apache.hadoop.yarn.api.records.Resource lhs,
                   org.apache.hadoop.yarn.api.records.Resource rhs,
                   boolean singleType)
ResourceCalculatorclusterResource, compare lhs
 and rhs. Consider all resources unless singleType is set
 to true. When singleType is set to true, consider only one
 resource as per the ResourceCalculator implementation; the
 DefaultResourceCalculator considers memory and
 DominantResourceCalculator considers the dominant resource.compare in class ResourceCalculatorclusterResource - cluster capacitylhs - First Resource to comparerhs - Second Resource to comparesingleType - Whether to consider a single resource type or all
                   resource typeslhs is smaller, 0 if equal and 1 if it is largerpublic long computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available,
                                       org.apache.hadoop.yarn.api.records.Resource required)
ResourceCalculatoravailable and required resources.computeAvailableContainers in class ResourceCalculatoravailable - available resourcesrequired - required resourcespublic float divide(org.apache.hadoop.yarn.api.records.Resource clusterResource,
                    org.apache.hadoop.yarn.api.records.Resource numerator,
                    org.apache.hadoop.yarn.api.records.Resource denominator)
ResourceCalculatornumerator by resource denominator
 using specified policy (domination, average, fairness etc.); hence overall
 clusterResource is provided for context.divide in class ResourceCalculatorclusterResource - cluster resourcesnumerator - numeratordenominator - denominatornumerator/denominator 
         using specific policypublic boolean isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)
ResourceCalculatorisInvalidDivisor in class ResourceCalculatorr - resourcepublic boolean isAllInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)
ResourceCalculatorisAllInvalidDivisor in class ResourceCalculatorr - resourcepublic float ratio(org.apache.hadoop.yarn.api.records.Resource a,
                   org.apache.hadoop.yarn.api.records.Resource b)
ResourceCalculatora to resource b.ratio in class ResourceCalculatora - resourceb - resourcea to resource bpublic org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
                                                                 int denominator)
ResourceCalculatornumerator by denominator.divideAndCeil in class ResourceCalculatornumerator - numerator resourcedenominator - denominatorpublic org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
                                                                 long denominator)
public org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
                                                                 float denominator)
ResourceCalculatornumerator by denominator.divideAndCeil in class ResourceCalculatornumerator - numerator resourcedenominator - denominatorpublic org.apache.hadoop.yarn.api.records.Resource normalize(org.apache.hadoop.yarn.api.records.Resource r,
                                                             org.apache.hadoop.yarn.api.records.Resource minimumResource,
                                                             org.apache.hadoop.yarn.api.records.Resource maximumResource,
                                                             org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr given the base 
 minimumResource and verify against max allowed
 maximumResource using a step factor for the normalization.normalize in class ResourceCalculatorr - resourceminimumResource - minimum valuemaximumResource - the upper bound of the resource to be allocatedstepFactor - the increment for resources to be allocatedpublic org.apache.hadoop.yarn.api.records.Resource roundUp(org.apache.hadoop.yarn.api.records.Resource r,
                                                           org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr given factor stepFactor.roundUp in class ResourceCalculatorr - resourcestepFactor - step-factorpublic org.apache.hadoop.yarn.api.records.Resource roundDown(org.apache.hadoop.yarn.api.records.Resource r,
                                                             org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr given factor stepFactor.roundDown in class ResourceCalculatorr - resourcestepFactor - step-factorpublic org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
                                                                          double[] by,
                                                                          org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr by factor by
 and normalize up using step-factor stepFactor.multiplyAndNormalizeUp in class ResourceCalculatorr - resource to be multipliedby - multiplier array for all resource typesstepFactor - factor by which to normalize uppublic org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
                                                                          double by,
                                                                          org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr by factor by 
 and normalize up using step-factor stepFactor.multiplyAndNormalizeUp in class ResourceCalculatorr - resource to be multipliedby - multiplierstepFactor - factor by which to normalize uppublic org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
                                                                            double by,
                                                                            org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr by factor by 
 and normalize down using step-factor stepFactor.multiplyAndNormalizeDown in class ResourceCalculatorr - resource to be multipliedby - multiplierstepFactor - factor by which to normalize downpublic boolean fitsIn(org.apache.hadoop.yarn.api.records.Resource smaller,
                      org.apache.hadoop.yarn.api.records.Resource bigger)
ResourceCalculatorfitsIn in class ResourceCalculatorpublic org.apache.hadoop.yarn.api.records.Resource normalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
                                                                 org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorrand normalize down using step-factor
 stepFactor.normalizeDown in class ResourceCalculatorr - resource to be multipliedstepFactor - factor by which to normalize downpublic boolean isAnyMajorResourceZeroOrNegative(org.apache.hadoop.yarn.api.records.Resource resource)
ResourceCalculatorisAnyMajorResourceZeroOrNegative in class ResourceCalculatorresource - resourcepublic boolean isAnyMajorResourceAboveZero(org.apache.hadoop.yarn.api.records.Resource resource)
ResourceCalculatorisAnyMajorResourceAboveZero in class ResourceCalculatorresource - resourcepublic Set<String> getInsufficientResourceNames(org.apache.hadoop.yarn.api.records.Resource required, org.apache.hadoop.yarn.api.records.Resource available)
ResourceCalculatorgetInsufficientResourceNames in class ResourceCalculatorrequired - - required resourceavailable - - available resourceCopyright © 2008–2023 Apache Software Foundation. All rights reserved.