Manipulating Images and Mouse/Keyboard GUI
Module 7 · Manipulating Images and Mouse/Keyboard GUI

Manipulating Images and Mouse/Keyboard GUI

Learn to process and transform images with Pillow, and automate mouse and keyboard interactions across any desktop application with PyAutoGUI.

Image Manipulation with Pillow

Pillow (PIL fork) is Python’s standard image processing library. It handles opening, transforming, filtering, and saving images in dozens of formats.

pip install Pillow

Opening and Inspecting Images

from PIL import Image

img = Image.open("photo.jpg")
print(img.size)     # (width, height) in pixels
print(img.mode)     # "RGB", "RGBA", "L" (grayscale), etc.
print(img.format)   # "JPEG", "PNG", etc.

img.show()          # open in system viewer

Transformations

# Resize
resized = img.resize((800, 600))
thumbnail = img.copy()
thumbnail.thumbnail((256, 256))   # preserves aspect ratio

# Rotate and flip
rotated = img.rotate(90, expand=True)
flipped = img.transpose(Image.FLIP_LEFT_RIGHT)

# Crop — (left, upper, right, lower)
cropped = img.crop((100, 100, 400, 400))

# Convert color mode
grayscale = img.convert("L")
rgba = img.convert("RGBA")

# Save
resized.save("resized.jpg", quality=85)
grayscale.save("gray.png")

Filters and Enhancements

from PIL import ImageFilter, ImageEnhance

# Blur and sharpen
blurred = img.filter(ImageFilter.GaussianBlur(radius=5))
sharpened = img.filter(ImageFilter.SHARPEN)

# Adjust brightness, contrast, saturation
enhancer = ImageEnhance.Brightness(img)
brighter = enhancer.enhance(1.5)    # 1.0 = original, >1 = brighter

contrast_enh = ImageEnhance.Contrast(img)
high_contrast = contrast_enh.enhance(2.0)

Batch Processing

from pathlib import Path

input_dir = Path("photos/")
output_dir = Path("thumbnails/")
output_dir.mkdir(exist_ok=True)

for img_path in input_dir.glob("*.jpg"):
    with Image.open(img_path) as img:
        img.thumbnail((512, 512))
        img.save(output_dir / img_path.name)

Drawing on Images

from PIL import ImageDraw, ImageFont

img = Image.new("RGB", (400, 200), color="white")
draw = ImageDraw.Draw(img)

# Shapes
draw.rectangle([10, 10, 200, 100], outline="black", width=3, fill="lightblue")
draw.ellipse([220, 10, 390, 190], outline="red", width=2)
draw.line([0, 100, 400, 100], fill="gray", width=1)

# Text
draw.text((20, 110), "Hello, Pillow!", fill="black")

img.save("drawing.png")

GUI Automation with PyAutoGUI

PyAutoGUI controls the mouse and keyboard programmatically — useful for automating repetitive desktop tasks.

pip install pyautogui
import pyautogui
import time

# Safety: move mouse to corner to abort
pyautogui.FAILSAFE = True

# Mouse
pyautogui.moveTo(500, 300, duration=0.5)    # move to (x, y)
pyautogui.click()                            # left click
pyautogui.doubleClick()
pyautogui.rightClick()
pyautogui.dragTo(700, 300, duration=1)

# Keyboard
pyautogui.typewrite("Hello, World!", interval=0.05)
pyautogui.press("enter")
pyautogui.hotkey("ctrl", "c")               # copy

# Screenshot and locate
screenshot = pyautogui.screenshot()
location = pyautogui.locateOnScreen("button.png", confidence=0.9)
if location:
    pyautogui.click(location)

Practical Automation Pattern

import pyautogui, time

def fill_form(data_rows):
    for row in data_rows:
        pyautogui.click(100, 200)           # click Name field
        pyautogui.typewrite(row["name"])
        pyautogui.press("tab")
        pyautogui.typewrite(row["email"])
        pyautogui.hotkey("ctrl", "enter")   # submit
        time.sleep(1)                        # wait for page
⚠️ Common Pitfall: Running PyAutoGUI Automation Unsupervised

PyAutoGUI takes full control of your mouse and keyboard — a bug in your script can click anything on screen. Always: (1) keep FAILSAFE = True so moving the mouse to the top-left corner aborts the script; (2) test with small datasets first; (3) add time.sleep() delays for UI transitions; (4) never run automation scripts unattended on a machine with important unsaved work open.


🧠 Knowledge Check

You need to process 5,000 product photos: resize each to 800×800 (padding with white to preserve aspect ratio), add a watermark text in the bottom-right corner, and save as PNG. Outline your Pillow implementation.

Reveal Answer & Explanation
from PIL import Image, ImageDraw, ImageFont
from pathlib import Path

def process_product_image(src_path, dst_path, size=800, watermark="© MyStore"):
    with Image.open(src_path) as img:
        img = img.convert("RGB")

        # Resize maintaining aspect ratio
        img.thumbnail((size, size))

        # Pad to exact square with white background
        canvas = Image.new("RGB", (size, size), "white")
        offset = ((size - img.width) // 2, (size - img.height) // 2)
        canvas.paste(img, offset)

        # Add watermark
        draw = ImageDraw.Draw(canvas)
        draw.text((size - 10, size - 10), watermark,
                  fill=(180, 180, 180), anchor="rb")

        canvas.save(dst_path, "PNG")

# Batch
src_dir, dst_dir = Path("raw/"), Path("processed/")
dst_dir.mkdir(exist_ok=True)
for p in src_dir.glob("*.jpg"):
    process_product_image(p, dst_dir / p.with_suffix(".png").name)

Key decisions: thumbnail() preserves aspect ratio; pasting onto a white canvas creates the square without stretching; anchor="rb" positions watermark text at the bottom-right corner.