Layer Masking: Free Photoshop Tutorial

Topics Covered in This Photoshop Tutorial:

Master the fundamentals of non-destructive editing: creating layer masks and refining them through precision painting techniques

Exercise Preview

Photo by Lucas Sankey on Unsplash

Exercise Overview

Layer masking represents one of the most powerful non-destructive editing techniques in Photoshop. Rather than permanently deleting pixels—a destructive approach that eliminates creative flexibility—masks allow you to selectively hide and reveal image content while preserving the original data intact. This foundational skill underpins virtually every advanced Photoshop workflow, from complex compositing to subtle retouching work. Mastering masks early in your Photoshop journey will dramatically expand your creative possibilities and professional efficiency.

Creating a Mask

We'll begin by establishing the basic mask workflow, demonstrating how to create precise selections and convert them into editable masks.

1. From the Photoshop Class folder, open the 3C Hat guy.jpg file.
2. In the Tools panel, click and hold on the Rectangular Marquee tool and choose the Elliptical Marquee tool .
3. In the image, hold Shift (to constrain the selection to a perfect circle) and drag a selection marquee around the subject's head. Don't worry about precise positioning at this stage—you'll have complete control over placement in the following steps.
4. In the Layers panel, click the Add a mask button at the bottom of the panel. Notice how Photoshop immediately converts your selection into a layer mask, with the selected area remaining visible and everything outside the selection hidden.
5. Choose the Move tool .
6. Drag within the circular area to observe how both the mask and the image content move together as a linked unit.

7. To gain independent control over the mask and image positioning—a crucial technique for precise compositing—click the link icon between the layer thumbnail and mask thumbnail to unlink them.

   

8. Select the layer thumbnail (the left thumbnail showing the actual image content).

   

9. Drag within the image to reposition the photograph within the circular window. Notice how the circular boundary remains stationary while the image content shifts independently.

10. Now select the mask thumbnail (the right thumbnail displaying the black and white mask).

    

11. Drag to reposition the circular mask while the image content remains fixed. This independent control is essential for precise compositional adjustments in professional workflows.

Adjusting a Mask by Painting on It

Now we'll explore the true power of masks: the ability to paint directly onto them for pixel-perfect control over visibility. This technique forms the backbone of advanced retouching and compositing work.

1. In the Layers panel, ensure the mask thumbnail remains selected—you'll see a white border around it when active.
2. Select the Brush tool from the Tools panel.

3. Configure your brush settings in the Options bar by clicking the Brush Preset picker and setting:

   • Size: 500px • Hardness: 0% (soft edges provide more natural blending)

4. Near the bottom of the Tools panel, click the Default Colors icon to set white as the foreground color and black as the background color.

   TIP: The keyboard shortcut D provides instant access to default colors—a workflow accelerator you'll use constantly in professional work.

5. Starting from within the circular area, paint outward beyond the circle's edge. Watch as you reveal previously hidden portions of the photograph, demonstrating how white paint on a mask reveals image content.
6. Press X to swap foreground and background colors, making black your active painting color.

7. Paint over any visible areas to hide them, illustrating how black paint conceals image content.

   Brush Settings for Mask Editing

   Size Configuration

   Set brush size to 500px for broad coverage. Adjust based on the detail level of your edits and image resolution.

   Hardness Setting

   Use 0% hardness for smooth, natural transitions. Hard brushes create sharp edges that can look artificial in masks.

Making the Hat Pop Out of the Circle

Let's elevate this exercise by creating a sophisticated layered effect where elements appear to break free from their boundaries—a popular technique in modern digital design and advertising.

1. Navigate to File > Revert to restore the original image state.
2. Select the Elliptical Marquee tool .
3. Hold Shift and create a circular selection around the subject's head, ensuring some of the hat extends beyond the circle's boundary.

4. To achieve precise positioning and sizing:

   • Choose Select > Transform Selection to enter selection modification mode.

   • Adjust the size using the corner handles (avoid holding Shift here to allow aspect ratio flexibility), and drag from within the selection to position it so the hat protrudes above the circle as shown:

     

   • Press Return (Mac) or Enter (Windows) to commit the selection transformation.

5. Click the Add a mask button in the Layers panel.
6. Double-click the layer name and rename it to guy for clear project organization.
7. Create a professional backdrop using a fill layer—a more flexible approach than painting on a regular layer. Click the Create new fill or adjustment layer button at the bottom of the Layers panel, and select Solid Color.
8. Choose black and click OK.

9. In the Layers panel:

   • Drag the fill layer beneath the guy layer to establish proper layer hierarchy.

   • Select the guy layer's mask thumbnail (displaying the circular black and white pattern).

10. If the Properties panel isn't visible above the Layers panel, access it via Window > Properties.

11. In the Properties panel, adjust the Feather slider to soften the circle's edge, creating a more organic, less geometric appearance.

    NOTE: This non-destructive adjustment remains fully editable throughout your project, allowing for real-time refinement based on client feedback or creative evolution.

12. To create the "pop-out" effect, we need a duplicate layer containing only the protruding hat portion. Select the guy layer in the Layers panel.

13. Press Cmd–J (Mac) or Ctrl–J (Windows) to duplicate the layer efficiently.

    NOTE: This shortcut combines Layer > New > Layer Via Copy into a single keystroke—essential for maintaining workflow momentum in time-sensitive projects.

14. Rename the duplicated layer from guy copy to hat for clarity.
15. Select the hat layer's mask thumbnail.
16. In the Properties panel, reset Feather to 0 to maintain sharp edges on the protruding hat elements.
17. Choose Select > Select and Mask to access Photoshop's advanced selection refinement workspace.

18. In the Properties panel on the right:

    • Click the View thumbnail and choose On Black for optimal contrast.
    • Set Opacity to 100% for maximum preview clarity.

19. In the Options bar, click Select Subject to leverage Photoshop's AI-powered selection technology.

    • If prompted about discarding your current selection, click OK—we're upgrading to a more sophisticated selection method.

20. Switch the view to On Layers to preview the effect in context with your existing design.

    Focus on refining the hat area that extends beyond the circle. To soften the hard edges that often result from automated selections, adjust these settings:

    • Feather: 3px (adds subtle edge softness)
    • Shift Edge: +20% (expands the selection slightly for better coverage)

    Click OK to apply the refined selection.

21. Switch back to the Rectangular Marquee tool by clicking and holding the Elliptical Marquee tool .

22. To maintain the illusion that only the hat breaks the circular boundary, select the lower portion of the subject as shown:

    

23. Click the Default Colors icon or press D to ensure black is your background color.
24. Confirm the hat layer's mask thumbnail is selected.
25. Press Delete (Mac) or Backspace (Windows) to fill the selected mask area with black, effectively hiding the lower portion of the subject.

26. Press Cmd–D (Mac) or Ctrl–D (Windows) to deselect and evaluate your work.

    • The layered effect should now be clearly visible, with the hat appearing to emerge from the circular frame.
    • To demonstrate the continued flexibility of your non-destructive workflow, let's experiment with the circular edge treatment.
27. Select the guy layer's mask thumbnail.

28. In the Properties panel, experiment with different Feather values to see how dramatically you can alter the design aesthetic without starting over—a key advantage of professional non-destructive workflows.

Optional Enhancement: Optimizing Canvas Size

Complete your composition with strategic cropping that enhances the visual impact while maintaining perfect proportions.

1. Select the Crop tool .
2. Hold Shift while dragging to create a perfectly square crop area. Initial positioning doesn't need to be exact.

3. Refine your crop area using these techniques:

   • Drag from within the crop boundary to adjust positioning
   • Hold Shift when dragging corner handles to maintain the square aspect ratio
   • Extend beyond the original document boundaries to add breathing room—the transparent areas will automatically fill with your background color upon crop completion

4. When satisfied with your composition, press Return (Mac) or Enter (Windows) to finalize the crop and complete your sophisticated layered design.