public class ProportionalCapacityPreemptionPolicy extends Object implements SchedulingEditPolicy, CapacitySchedulerPreemptionContext
SchedulingEditPolicy
that is designed to be
paired with the CapacityScheduler
. At every invocation of editSchedule()
it computes the ideal amount of resources assigned to each
queue (for each queue in the hierarchy), and determines whether preemption
is needed. Overcapacity is distributed among queues in a weighted fair manner,
where the weight is the amount of guaranteed capacity for the queue.
Based on this ideal assignment it determines whether preemption is required
and select a set of containers from each application that would be killed if
the corresponding amount of resources is not freed up by the application.
If not in observeOnly
mode, it triggers preemption requests via a
ContainerPreemptEvent
that the ResourceManager
will ensure
to deliver to the application (or to execute).
If the deficit of resources is persistent over a long enough period of time
this policy will trigger forced termination of containers (again by generating
ContainerPreemptEvent
).Modifier and Type | Class and Description |
---|---|
static class |
ProportionalCapacityPreemptionPolicy.IntraQueuePreemptionOrderPolicy
IntraQueuePreemptionOrder will be used to define various priority orders
which could be configured by admin.
|
Constructor and Description |
---|
ProportionalCapacityPreemptionPolicy() |
ProportionalCapacityPreemptionPolicy(RMContext context,
CapacityScheduler scheduler,
org.apache.hadoop.yarn.util.Clock clock) |
public ProportionalCapacityPreemptionPolicy()
public ProportionalCapacityPreemptionPolicy(RMContext context, CapacityScheduler scheduler, org.apache.hadoop.yarn.util.Clock clock)
public void init(org.apache.hadoop.conf.Configuration config, RMContext context, ResourceScheduler sched)
init
in interface SchedulingEditPolicy
public org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()
getResourceCalculator
in interface CapacitySchedulerPreemptionContext
public void editSchedule()
SchedulingEditPolicy
editSchedule
in interface SchedulingEditPolicy
public long getMonitoringInterval()
getMonitoringInterval
in interface SchedulingEditPolicy
public String getPolicyName()
getPolicyName
in interface SchedulingEditPolicy
public Map<RMContainer,Long> getToPreemptContainers()
public TempQueuePerPartition getQueueByPartition(String queueName, String partition)
getQueueByPartition
in interface CapacitySchedulerPreemptionContext
public Collection<TempQueuePerPartition> getQueuePartitions(String queueName)
getQueuePartitions
in interface CapacitySchedulerPreemptionContext
public CapacityScheduler getScheduler()
getScheduler
in interface CapacitySchedulerPreemptionContext
public RMContext getRMContext()
getRMContext
in interface CapacitySchedulerPreemptionContext
public boolean isObserveOnly()
isObserveOnly
in interface CapacitySchedulerPreemptionContext
public Set<org.apache.hadoop.yarn.api.records.ContainerId> getKillableContainers()
getKillableContainers
in interface CapacitySchedulerPreemptionContext
public double getMaxIgnoreOverCapacity()
getMaxIgnoreOverCapacity
in interface CapacitySchedulerPreemptionContext
public double getNaturalTerminationFactor()
getNaturalTerminationFactor
in interface CapacitySchedulerPreemptionContext
public Set<String> getLeafQueueNames()
getLeafQueueNames
in interface CapacitySchedulerPreemptionContext
public Set<String> getAllPartitions()
getAllPartitions
in interface CapacitySchedulerPreemptionContext
public int getClusterMaxApplicationPriority()
getClusterMaxApplicationPriority
in interface CapacitySchedulerPreemptionContext
public float getMaxAllowableLimitForIntraQueuePreemption()
getMaxAllowableLimitForIntraQueuePreemption
in interface CapacitySchedulerPreemptionContext
public float getMinimumThresholdForIntraQueuePreemption()
getMinimumThresholdForIntraQueuePreemption
in interface CapacitySchedulerPreemptionContext
public org.apache.hadoop.yarn.api.records.Resource getPartitionResource(String partition)
getPartitionResource
in interface CapacitySchedulerPreemptionContext
public LinkedHashSet<String> getUnderServedQueuesPerPartition(String partition)
getUnderServedQueuesPerPartition
in interface CapacitySchedulerPreemptionContext
public void addPartitionToUnderServedQueues(String queueName, String partition)
addPartitionToUnderServedQueues
in interface CapacitySchedulerPreemptionContext
public ProportionalCapacityPreemptionPolicy.IntraQueuePreemptionOrderPolicy getIntraQueuePreemptionOrderPolicy()
getIntraQueuePreemptionOrderPolicy
in interface CapacitySchedulerPreemptionContext
public boolean getCrossQueuePreemptionConservativeDRF()
getCrossQueuePreemptionConservativeDRF
in interface CapacitySchedulerPreemptionContext
public boolean getInQueuePreemptionConservativeDRF()
getInQueuePreemptionConservativeDRF
in interface CapacitySchedulerPreemptionContext
public long getDefaultMaximumKillWaitTimeout()
getDefaultMaximumKillWaitTimeout
in interface CapacitySchedulerPreemptionContext
Copyright © 2008–2023 Apache Software Foundation. All rights reserved.