Previous research has shown that loading information on working memory affects selective attention. However, whether the load effect on selective attention is domain-general or domain-specific remains unresolved. The domain-general effect refers to the findings that load in one content (e.g. phonological) domain in working memory influences processing in another content (e.g., visuospatial) domain. Attentional control supervises selection regardless of information domain. The domain-specific effect refers to the constraint of influence only when maintenance and processing operate in the same domain. Selective attention operates in a specific content domain. This study is designed to resolve this controversy. Across three experiments, we manipulated the type of representation maintained in working memory and the type of representation upon which the participants must exert control to resolve conflict and select a target into the focus of attention. In Experiments 1a and 1b, participants maintained digits and nonverbalized objects, respectively, in working memory while selecting a target in a letter array. In Experiment 2, we presented auditory digits with a letter flanker task to exclude the involvement of resource competition within the same input modality. In Experiments 3a and 3b, we replaced the letter flanker task with an object flanker task while manipulating the memory load on object and digit representation, respectively. The results consistently showed that memory load modulated distractibility only when the stimuli of the two tasks were represented in the same domain. The magnitude of distractor interference was larger under high load than under low load, reflecting a lower efficacy of information prioritization. When the stimuli of the two tasks were represented in different domains, memory load did not modulate distractibility. Control of processing priority in selective attention demands domain-specific resources.