Coding Concepts – Python Programming Guide

📦 Variables and Assignment

Python variables are dynamically typed – you don't need to declare types explicitly. Assignment uses the = operator.

name = "Glenn"
age = 28
height = 1.75
is_student = True

x, y, z = 10, 20, 30
print(x, y, z)

a, b = 5, 10
a, b = b, a
print(a, b)

value = 42
print(type(value))
value = "Hello"
print(type(value))

📘 Comprehensive Explanation This code demonstrates the core concept of variable assignment in Python, which is fundamental to storing and manipulating data. Every program uses variables to hold values. The assignment operator = binds a value to a name. Multiple assignment allows setting several variables at once. Swapping values with a, b = b, a is a Pythonic shortcut. Dynamic typing means the same variable can hold different types over time — type() reveals the current type. Dynamic typing is one of Python's defining features, enabling rapid development without type boilerplate. It reduces verbosity and makes code more flexible. This snippet serves as the foundation for all subsequent examples. It establishes how data is stored, updated, and inspected, which is essential for every other operation in the program. The output shows the assigned values, the swapped values, and the changing type of value — from int to str. This visually reinforces Python's dynamic nature.

💡 Python is dynamically typed – variables can change type during runtime. Use type() to check the current type.

🔢 Numeric Types

Python supports integers, floating-point numbers, and complex numbers.

integer_num = 42
negative = -10
big_num = 1_000_000

float_num = 3.14159
scientific = 1.5e6

complex_num = 3 + 4j
print(complex_num.real, complex_num.imag)

print(int(3.14))
print(float(42))
print(complex(2, 3))

📘 Comprehensive Explanation Numeric types are the bedrock of mathematical and scientific computing. This snippet shows Python's handling of integers, floats, and complex numbers, which are used in virtually every non-trivial program. Integers are arbitrary-precision. Floats are IEEE 754 double-precision. Complex numbers have a real and imaginary part, accessed via .real and .imag. Type conversion functions int(), float(), and complex() convert between types. Python's numeric types are robust and flexible. The underscore in 1_000_000 improves readability. Complex numbers are natively supported, which is rare among general-purpose languages. This snippet provides the numeric toolkit for calculations, simulations, and data processing throughout the page. The output shows the real and imaginary parts of a complex number, and the results of type conversions — demonstrating how Python handles different numeric representations.

📝 Strings

Strings are immutable sequences of characters. Python offers powerful string manipulation methods.

single = 'Hello'
double = "World"
multiline = """This is a
multiline string"""

name = "Glenn"
age = 28
print(f"My name is {name} and I am {age} years old")

text = "  Python Programming  "
print(text.strip())
print(text.upper())
print(text.lower())
print(text.replace("Python", "Java"))

s = "Hello World"
print(s[0:5])
print(s[6:])
print(s[::-1])

words = ["Python", "is", "awesome"]
sentence = " ".join(words)
print(sentence)
print(sentence.split())

📘 Comprehensive Explanation Strings are the primary way to handle text in Python. This snippet covers creation, formatting, manipulation, and slicing — all essential for user input, output, and data processing. Strings can be created with single, double, or triple quotes. f-strings provide inline variable interpolation. Methods like strip(), upper(), and lower() transform text. Slicing [start:end:step] extracts substrings. join() and split() convert between strings and lists. String manipulation is ubiquitous in programming — from parsing files to building user interfaces. Python's string methods are both powerful and intuitive. This snippet provides the text-handling tools used in data validation, output formatting, and string processing throughout the guide. The output demonstrates f-string interpolation, whitespace trimming, case conversion, substring extraction (including reversal), and list-to-string conversion — each showing a different aspect of string power.

f-stringsslicingjoinsplit

➕ Arithmetic & Comparison Operators

Python provides all standard arithmetic and comparison operators.

a, b = 10, 3
print(a + b)
print(a - b)
print(a * b)
print(a / b)
print(a // b)
print(a % b)
print(a ** b)

x, y = 5, 10
print(x == y)
print(x != y)
print(x < y)
print(x > y)
print(x <= y)
print(x >= y)

is_raining = True
has_umbrella = False
print(is_raining and has_umbrella)
print(is_raining or has_umbrella)
print(not is_raining)

📘 Comprehensive Explanation Operators are the building blocks of expressions. This snippet covers arithmetic (math), comparison (logic), and logical operators — everything needed to perform calculations and make decisions. Arithmetic: +, -, *, / (float division), // (floor division), % (modulo), ** (exponent). Comparison: ==, !=, <, >, <=, >=. Logical: and, or, not. Each operator returns a boolean or numeric result. Operators are the core of Python's expression syntax. Understanding operator precedence and behavior is crucial for writing correct, efficient code. This snippet provides the decision-making and calculation machinery used in conditional statements, loops, and data transformations throughout the page. The output shows the results of each arithmetic operation, the truth values of comparisons, and the boolean results of logical combinations — demonstrating how Python evaluates expressions.

🔗 Membership & Identity Operators

in, not in check membership. is, is not check identity (object reference).

fruits = ["apple", "banana", "cherry"]
print("apple" in fruits)
print("grape" in fruits)
print("grape" not in fruits)

a = [1, 2, 3]
b = [1, 2, 3]
c = a
print(a is b)
print(a is c)
print(a is not b)

📘 Comprehensive Explanation Membership and identity operators are essential for checking containment and object equality. in and not in check if a value exists in a sequence. is and is not compare object identities (memory addresses), not values. Understanding the difference between == (value equality) and is (identity) is crucial for avoiding subtle bugs. Membership operators are used for data validation and filtering. Identity operators are used for singleton checks (e.g., if x is None). The output shows boolean results for membership checks, identity comparisons (showing that a is b is False because they are distinct objects, while a is c is True).

💡 Use == for value comparison, is for object identity. is is typically used with None: if x is None:

🔀 Conditionals – if, elif, else

Python uses indentation to define code blocks. No curly braces required.

score = 85

if score >= 90:
    grade = "A"
elif score >= 80:
    grade = "B"
elif score >= 70:
    grade = "C"
elif score >= 60:
    grade = "D"
else:
    grade = "F"

print(f"Grade: {grade}")

age = 18
status = "Adult" if age >= 18 else "Minor"
print(status)

x = 10
if x > 0:
    if x % 2 == 0:
        print("Positive even number")
    else:
        print("Positive odd number")
else:
    print("Non-positive number")

📘 Comprehensive Explanation Conditionals direct the flow of execution based on boolean expressions. They are the foundation of decision-making in programs, enabling different behavior for different inputs. The if/elif/else chain evaluates conditions in order. The first true condition executes its block. The ternary operator value_if_true if condition else value_if_false provides inline conditional assignment. Nested conditionals allow multi-level decision trees. Indentation-based blocks are a distinctive feature of Python, enforcing readability. The ternary operator reduces verbosity for simple conditionals. This snippet provides the decision-making logic used in validation, routing, and business logic throughout the page. The output shows the grade assignment for a score of 85, the status determination for age 18, and the nested conditional output for x=10 — each demonstrating a different conditional pattern.

🔁 Loops – for and while

Python's for loop iterates over iterables. while loops run while a condition is true.

for i in range(5):
    print(i, end=" ")

fruits = ["apple", "banana", "cherry"]
for fruit in fruits:
    print(fruit)

for i, fruit in enumerate(fruits, start=1):
    print(f"{i}. {fruit}")

count = 0
while count < 5:
    print(count, end=" ")
    count += 1

for i in range(10):
    if i == 5:
        break
    if i % 2 == 0:
        continue
    print(i, end=" ")

for i in range(3):
    print(i)
else:
    print("Loop completed")

📘 Comprehensive Explanation Loops are essential for repetitive tasks, iterating over data structures, and performing operations until a condition is met. for iterates over any iterable (range, list, tuple, string, etc.). enumerate() provides both index and value. while repeats while its condition is true. break exits the loop, continue skips to the next iteration. The else clause on a loop runs if the loop completes without a break. Python's for loop is more flexible than C-style loops, as it works with any iterable. The else clause is a unique feature that simplifies search-and-fail patterns. Loops are used to process collections, implement algorithms, and control repetitive execution throughout the page. The output shows range iteration, list iteration with enumeration, while-loop counting, break/continue behavior (printing odd numbers less than 5), and the for-else completion message.

rangeenumeratebreakcontinuefor-else

🎯 Match Statement (Python 3.10+)

Structural pattern matching – like a powerful switch statement.

value = 42

match value:
    case 0:
        print("Zero")
    case 1:
        print("One")
    case 42:
        print("The answer")
    case _:
        print("Other")

match value:
    case n if n > 0:
        print("Positive")
    case n if n < 0:
        print("Negative")
    case _:
        print("Zero")

point = (3, 4)
match point:
    case (0, 0):
        print("Origin")
    case (x, 0):
        print(f"On X-axis at {x}")
    case (0, y):
        print(f"On Y-axis at {y}")
    case (x, y):
        print(f"Point at ({x}, {y})")

📘 Comprehensive Explanation The match statement is a modern addition that brings powerful pattern matching to Python, replacing long if-elif chains with cleaner, more expressive code. match compares a value against multiple case patterns. The wildcard _ matches anything. Guards (if after a pattern) add conditions. Structural matching works with tuples, lists, and data classes, extracting values into variables. Pattern matching makes code more declarative and less error-prone. It's particularly useful for parsing, handling different data shapes, and implementing state machines. This snippet demonstrates how to use match for value-based branching, conditional guards, and destructuring complex data structures. The output shows "The answer" for value 42, "Positive" for the guard match, and "Point at (3, 4)" for the tuple destructuring — each demonstrating a different pattern-matching capability.

📋 Lists – Mutable Sequences

Lists are ordered, mutable collections that can hold any type of data.

empty = []
numbers = [1, 2, 3, 4, 5]
mixed = [1, "hello", 3.14, True]

print(numbers[0])
print(numbers[-1])
print(numbers[1:4])

numbers.append(6)
numbers.insert(0, 0)
numbers.remove(3)
popped = numbers.pop()
print(numbers)
print(popped)

squares = [x**2 for x in range(10)]
evens = [x for x in range(10) if x % 2 == 0]
print(squares)
print(evens)

matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
print(matrix[1][1])

numbers = [3, 1, 4, 1, 5, 9, 2]
numbers.sort()
print(numbers)
numbers.sort(reverse=True)
print(numbers)

📘 Comprehensive Explanation Lists are the most versatile and frequently used data structure in Python. This snippet covers creation, indexing, slicing, methods, comprehensions, nesting, and sorting. Lists are created with square brackets. Indexing uses [] with 0-based indices. Slicing [start:end:step] returns a new list. Methods like append(), insert(), remove(), and pop() modify the list. List comprehensions provide concise creation syntax. sort() sorts in place. Lists are the workhorse of Python data handling. Their mutability and dynamic sizing make them suitable for a wide range of applications. Lists are used throughout the page for storing collections of data, implementing algorithms, and organizing information. The output demonstrates indexing (first, last, slice), the effects of list methods (append, insert, remove, pop), the results of comprehensions (squares, evens), nested list access, and sorting (ascending and descending).

appendpopcomprehensionssortnested

📖 Dictionaries – Key-Value Stores

Dictionaries map keys to values with O(1) lookup. Keys must be immutable.

empty = {}
person = {"name": "Glenn", "age": 28, "city": "Manila"}

print(person["name"])
print(person.get("country", "Unknown"))

person["country"] = "Philippines"
person["age"] = 29

for key, value in person.items():
    print(f"{key}: {value}")

squares = {x: x**2 for x in range(5)}
print(squares)

d1 = {"a": 1, "b": 2}
d2 = {"c": 3, "d": 4}
merged = d1 | d2
print(merged)

from collections import defaultdict
counts = defaultdict(int)
counts["apple"] += 1
print(counts)

📘 Comprehensive Explanation Dictionaries are essential for fast lookups, configuration, and mapping relationships. They are used extensively in data processing and application development. Dictionaries use {key: value} syntax. Access with [] raises KeyError if missing; .get() returns a default. Adding a key assigns a value. .items() iterates over key-value pairs. Dictionary comprehensions create dictionaries concisely. The | operator (Python 3.9+) merges dictionaries. defaultdict provides default values for missing keys. Dictionaries are one of Python's most powerful features, enabling efficient data organization and retrieval. Their flexibility makes them suitable for a wide range of use cases. Dictionaries are used for configuration, data mapping, caching, and representing structured data throughout the page. The output shows name lookup with get(), the updated dictionary with country and age, iteration over items, the squares dictionary comprehension, the merged dictionary using the | operator, and the defaultdict count for "apple".

getitemsdefaultdictmerge |comprehensions

🎯 Tuples – Immutable Sequences

Tuples are immutable, ordered collections. They are faster than lists and can be used as dictionary keys.

empty = ()
coords = (3, 4)
single = (42,)

x, y = coords
print(x, y)

from collections import namedtuple
Point = namedtuple("Point", ["x", "y"])
p = Point(3, 4)
print(p.x, p.y)

location = {(3, 4): "Home", (10, 20): "Work"}
print(location[(3, 4)])

data = 1, 2, 3
a, b, c = data
print(a, b, c)

first, *rest = [1, 2, 3, 4, 5]
print(first, rest)

📘 Comprehensive Explanation Tuples are used when immutability is desired — for dictionary keys, function return values, and data that should not be modified. They are more memory-efficient than lists. Tuples are created with parentheses or just commas. Unpacking assigns tuple elements to variables. namedtuple provides field names for clarity. Tuples can be dictionary keys because they are hashable. Extended unpacking (*rest) captures remaining elements. Immutable data structures are safer and can be used as keys in dictionaries. Named tuples combine the efficiency of tuples with the readability of classes. Tuples are used for fixed-size data, function return values, and as dictionary keys throughout the page. The output shows unpacking coordinates, namedtuple field access, tuple-based dictionary lookup, tuple packing/unpacking, and extended unpacking with *rest.

🔵 Sets – Unordered Unique Collections

Sets store unique, hashable elements. Perfect for deduplication and set operations.

empty = set()
numbers = {1, 2, 3, 4, 5}
mixed = {1, "hello", 3.14}

a = {1, 2, 3, 4}
b = {3, 4, 5, 6}

print(a | b)
print(a & b)
print(a - b)
print(a ^ b)

a.add(7)
a.discard(1)
a.remove(2)

squares = {x**2 for x in range(5)}
print(squares)

fs = frozenset([1, 2, 3])
print(fs)

📘 Comprehensive Explanation Sets are ideal for removing duplicates, testing membership, and performing mathematical set operations. They provide fast O(1) lookup. Sets are created with {} or set(). Set operations: | (union), & (intersection), - (difference), ^ (symmetric difference). add() inserts an element, discard() removes if present, remove() raises KeyError if missing. Set comprehensions create sets concisely. frozenset is immutable and hashable. Sets are essential for data deduplication, membership testing, and implementing set-based algorithms. Their O(1) operations make them efficient for large datasets. Sets are used for deduplication, intersection operations, and membership testing throughout the page. The output shows the results of union, intersection, difference, and symmetric difference operations, the modified set after add/discard/remove, the squares set comprehension, and the frozenset.

📞 Defining and Calling Functions

Functions are defined with def. They can take positional, keyword, and default arguments.

def greet(name):
    return f"Hello, {name}!"

print(greet("Glenn"))

def power(base, exponent=2):
    return base ** exponent

print(power(3))
print(power(3, 3))

def describe(name, age, city):
    return f"{name} is {age} years old and lives in {city}"

print(describe(name="Glenn", age=28, city="Manila"))

def sum_all(*args):
    return sum(args)

def print_kwargs(**kwargs):
    for key, value in kwargs.items():
        print(f"{key}: {value}")

print(sum_all(1, 2, 3, 4, 5))
print_kwargs(name="Glenn", age=28)

📘 Comprehensive Explanation Functions are the primary mechanism for code reuse, modularity, and abstraction. This snippet covers the essential patterns for defining and calling functions. def defines a function. Parameters can have default values. Keyword arguments are specified by name. *args collects positional arguments into a tuple, **kwargs collects keyword arguments into a dictionary. Functions are fundamental to structured programming. They enable code reuse, reduce duplication, and make programs easier to test and maintain. Functions are used throughout the page to encapsulate logic, perform calculations, and process data. The output shows the greeting for "Glenn", the power function with default and explicit exponent, the describe function with keyword arguments, the sum of arbitrary arguments, and the keyword argument iteration.

🔁 Lambda Functions

Lambda functions are anonymous, single-expression functions.

square = lambda x: x ** 2
print(square(5))

numbers = [1, 2, 3, 4, 5]
squared = list(map(lambda x: x ** 2, numbers))
print(squared)

evens = list(filter(lambda x: x % 2 == 0, numbers))
print(evens)

people = [{"name": "Alice", "age": 30}, {"name": "Bob", "age": 25}]
sorted_people = sorted(people, key=lambda p: p["age"])
print(sorted_people)

from functools import reduce
product = reduce(lambda x, y: x * y, numbers)
print(product)

📘 Comprehensive Explanation Lambdas provide a concise way to define small, single-use functions inline. They are commonly used with map(), filter(), sorted(), and reduce(). lambda args: expression defines an anonymous function. map() applies a function to every element of an iterable. filter() selects elements that satisfy a predicate. sorted() uses a key function for custom sorting. reduce() cumulatively applies a function to combine elements. Lambdas enable functional programming patterns in Python. They reduce verbosity and make code more expressive for simple transformations. Lambdas are used for lightweight transformations, filtering, and custom sorting throughout the page. The output shows the square of 5, the squared list using map, the filtered evens list, the people list sorted by age, and the product of all numbers using reduce.

🔄 Closures

A closure is a function that remembers its enclosing scope even when called outside that scope.

def make_multiplier(factor):
    def multiplier(x):
        return x * factor
    return multiplier

double = make_multiplier(2)
triple = make_multiplier(3)

print(double(5))
print(triple(5))

def counter():
    count = 0
    def increment():
        nonlocal count
        count += 1
        return count
    return increment

c = counter()
print(c())
print(c())
print(c())

📘 Comprehensive Explanation Closures are a powerful functional programming pattern that enables data hiding and function factories. They are used to create stateful functions without classes. A closure is created when a nested function references variables from its enclosing scope. The nonlocal keyword allows the inner function to modify variables from the outer scope. Each call to the outer function creates a new closure with its own state. Closures enable encapsulation and state management without classes. They are used in decorators, callback functions, and function factories. Closures are used to create specialized functions, maintain private state, and implement function factories throughout the page. The output shows double and triple multipliers working with their captured factors, and the counter incrementing its state across calls.

💡 Closures are powerful for data hiding and function factories. The nonlocal keyword allows modifying variables from the enclosing scope.

🏛️ Classes and Objects

Classes are blueprints for creating objects. They encapsulate data (attributes) and behaviour (methods).

class Car:
    def __init__(self, make, model, year):
        self.make = make
        self.model = model
        self.year = year
        self.odometer = 0

    def drive(self, miles):
        self.odometer += miles
        return f"Driven {miles} miles. Total: {self.odometer}"

    def __str__(self):
        return f"{self.year} {self.make} {self.model}"

    def __repr__(self):
        return f"Car('{self.make}', '{self.model}', {self.year})"

car = Car("Toyota", "Camry", 2023)
print(car)
print(car.drive(50))
print(car)

📘 Comprehensive Explanation Classes are the foundation of object-oriented programming, allowing you to model real-world entities with data and behavior. class defines a new type. __init__ is the constructor, initializing instance attributes with self. Methods are functions defined inside the class. __str__ defines the string representation for print(). __repr__ defines the developer-friendly representation. OOP enables encapsulation, inheritance, and polymorphism, making code more modular, reusable, and easier to maintain. Classes are used throughout the page to model domain entities, encapsulate state, and organize complex logic. The output shows the string representation of the car, the drive method updating odometer, and the repr representation of the car object.

🔒 Encapsulation and Properties

Use @property to control access to attributes, enabling getters and setters.

class BankAccount:
    def __init__(self, owner, balance=0):
        self.owner = owner
        self._balance = balance

    @property
    def balance(self):
        return self._balance

    @balance.setter
    def balance(self, value):
        if value < 0:
            raise ValueError("Balance cannot be negative")
        self._balance = value

    def deposit(self, amount):
        if amount <= 0:
            raise ValueError("Deposit amount must be positive")
        self._balance += amount

    def withdraw(self, amount):
        if amount > self._balance:
            raise ValueError("Insufficient funds")
        self._balance -= amount

acc = BankAccount("Glenn", 1000)
print(acc.balance)
acc.deposit(500)
print(acc.balance)

📘 Comprehensive Explanation Encapsulation hides internal state and exposes controlled access. Properties provide a clean way to implement getters and setters without breaking the external interface. The @property decorator turns a method into a read-only attribute. The @balance.setter decorator allows assignment with validation. Private attributes (with a leading underscore) signal internal use. Methods like deposit() and withdraw() provide controlled operations. Encapsulation is a key principle of OOP, protecting internal consistency and enabling validation. Properties provide a Pythonic way to implement it without verbose getter/setter methods. Properties are used to control attribute access, enforce validation, and maintain invariants throughout the page. The output shows the initial balance, the deposit operation increasing the balance, and the property-based access to balance.

🌿 Inheritance

Inheritance allows a class to extend and reuse behaviour from a parent class.

class Animal:
    def __init__(self, name):
        self.name = name

    def speak(self):
        raise NotImplementedError("Subclass must implement speak()")

    def __str__(self):
        return f"Animal: {self.name}"

class Dog(Animal):
    def speak(self):
        return f"{self.name} says Woof!"

class Cat(Animal):
    def speak(self):
        return f"{self.name} says Meow!"

dog = Dog("Rex")
cat = Cat("Whiskers")

print(dog.speak())
print(cat.speak())

class GoldenRetriever(Dog):
    def speak(self):
        parent_speak = super().speak()
        return f"{parent_speak} And I'm a Golden Retriever!"

golden = GoldenRetriever("Buddy")
print(golden.speak())

📘 Comprehensive Explanation Inheritance promotes code reuse and establishes hierarchical relationships between classes. A subclass is defined with class Subclass(Parent). The super() function calls methods from the parent class. Abstract methods (with raise NotImplementedError) enforce that subclasses provide implementations. Inheritance is a cornerstone of OOP, enabling polymorphic behavior and code organization. It reduces duplication and makes relationships explicit. Inheritance is used to create specialized classes that extend or override parent behavior throughout the page. The output shows Dog and Cat speaking with their own implementations, and GoldenRetriever extending Dog's speak method with additional text.

🎭 Polymorphism

Polymorphism allows objects of different classes to be treated uniformly through a common interface.

def make_sound(animal):
    return animal.speak()

print(make_sound(dog))
print(make_sound(cat))

from abc import ABC, abstractmethod

class Shape(ABC):
    @abstractmethod
    def area(self):
        pass

    @abstractmethod
    def perimeter(self):
        pass

class Rectangle(Shape):
    def __init__(self, width, height):
        self.width = width
        self.height = height

    def area(self):
        return self.width * self.height

    def perimeter(self):
        return 2 * (self.width + self.height)

rect = Rectangle(5, 3)
print(rect.area())
print(rect.perimeter())

📘 Comprehensive Explanation Polymorphism enables writing code that works with objects of different types through a common interface, making systems more flexible and extensible. Duck typing allows any object with a speak() method to be passed to make_sound(). Abstract base classes (ABC) define interfaces that concrete subclasses must implement. The @abstractmethod decorator enforces implementation. Polymorphism is what makes OOP powerful — it allows substitution and extensibility without modifying existing code. ABCs provide formal interface contracts. Polymorphism is used throughout the page to write generic code that works with various types. The output shows that both Dog and Cat can be passed to make_sound(), and that Rectangle correctly computes area and perimeter.

🎨 Decorators

Decorators are functions that modify or extend the behaviour of other functions.

import time
from functools import wraps

def timer(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print(f"{func.__name__} took {end - start:.4f} seconds")
        return result
    return wrapper

def retry(times=3, delay=1):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            attempts = 0
            while attempts < times:
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    attempts += 1
                    if attempts == times:
                        raise
                    time.sleep(delay)
        return wrapper
    return decorator

@timer
def slow_function():
    time.sleep(1)
    return "Done"

@retry(times=3, delay=0.5)
def unreliable_function():
    import random
    if random.random() < 0.7:
        raise ValueError("Random failure")
    return "Success!"

print(slow_function())
print(unreliable_function())

📘 Comprehensive Explanation Decorators are a powerful metaprogramming feature that allows you to modify or enhance functions without changing their source code. They are used for logging, timing, access control, and more. A decorator is a function that takes another function as an argument and returns a wrapped version. The @decorator syntax is syntactic sugar for func = decorator(func). @wraps preserves the original function's metadata. Decorator factories (like retry) take parameters and return a decorator. Decorators enable separation of concerns — cross-cutting concerns like timing, logging, and retry logic are factored out of business logic. Decorators are used throughout the page for timing, retrying, and extending function behavior. The output shows the timing message for slow_function and the result of unreliable_function which retries on random failure.

⚡ Generators and Iterators

Generators produce values lazily using yield, saving memory for large datasets.

def fibonacci(limit):
    a, b = 0, 1
    while a < limit:
        yield a
        a, b = b, a + b

for num in fibonacci(100):
    print(num, end=" ")

squares = (x**2 for x in range(10))
for sq in squares:
    print(sq, end=" ")

class CountDown:
    def __init__(self, start):
        self.start = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.start < 0:
            raise StopIteration
        current = self.start
        self.start -= 1
        return current

for n in CountDown(5):
    print(n, end=" ")

📘 Comprehensive Explanation Generators are memory-efficient for processing large datasets because they produce values on demand rather than storing them all in memory. A generator function uses yield to produce values one at a time. Generator expressions use parentheses (). A custom iterator implements __iter__ and __next__, raising StopIteration when done. Generators are essential for lazy evaluation, enabling efficient processing of large files, infinite sequences, and data streams. Generators are used for lazy iteration, memory-efficient processing, and infinite sequences throughout the page. The output shows the Fibonacci sequence up to 100, the squares of numbers 0-9, and a countdown from 5 to 0.

yieldgeneratoriterator__iter____next__

🔐 Context Managers

Context managers manage resources with with statements, ensuring proper setup and teardown.

with open("example.txt", "w") as f:
    f.write("Hello, World!")

class ManagedFile:
    def __init__(self, filename, mode):
        self.filename = filename
        self.mode = mode

    def __enter__(self):
        self.file = open(self.filename, self.mode)
        return self.file

    def __exit__(self, exc_type, exc_val, exc_tb):
        if self.file:
            self.file.close()
        if exc_type is not None:
            print(f"Exception occurred: {exc_val}")
        return False

with ManagedFile("example.txt", "r") as f:
    content = f.read()
    print(content)

from contextlib import contextmanager

@contextmanager
def managed_resource(resource):
    print("Acquiring resource")
    yield resource
    print("Releasing resource")

with managed_resource("database") as res:
    print(f"Using {res}")

📘 Comprehensive Explanation Context managers ensure reliable resource management — files are closed, locks are released, and connections are cleaned up even when exceptions occur. The with statement calls __enter__ on entry and __exit__ on exit. The @contextmanager decorator from contextlib turns a generator into a context manager. yield marks the point where the block runs. Context managers are crucial for writing robust code that correctly manages external resources. They eliminate the need for manual try-finally blocks. Context managers are used for file I/O, database connections, locks, and resource cleanup throughout the page. The output shows file writing, file reading, and resource acquisition/release messages.

🔍 Type Hints

Type hints improve code readability and enable static type checking with mypy.

from typing import List, Dict, Optional, Union, Tuple, TypeVar

def greet(name: str) -> str:
    return f"Hello, {name}"

def process_numbers(numbers: List[int]) -> List[int]:
    return [n * 2 for n in numbers]

def find_user(user_id: int) -> Optional[Dict[str, str]]:
    if user_id == 1:
        return {"name": "Glenn", "email": "glenn@example.com"}
    return None

def parse_value(value: Union[str, int]) -> float:
    return float(value)

def get_coordinates() -> Tuple[int, int]:
    return (3, 4)

T = TypeVar("T")

def first(items: List[T]) -> T:
    return items[0]

def parse(value: str | int | None) -> str:
    if value is None:
        return ""
    return str(value)

📘 Comprehensive Explanation Type hints make code more self-documenting and enable static type checking, catching type-related errors before runtime. Type hints use : type for parameters and -> type for return values. typing provides generic types like List, Dict, Optional, Union, Tuple. TypeVar enables generic functions. Python 3.10+ supports | for union types. Type hints improve code maintainability, help with IDE autocomplete, and enable tools like mypy for static analysis. They are increasingly standard in modern Python codebases. Type hints are used to document function interfaces and enable static type checking throughout the page. The output is not shown for type hints as they are annotations — they don't change runtime behavior but are used by static analysis tools.

🚀 Async/Await

Async functions allow non-blocking I/O operations, improving performance for network-heavy workloads.

import asyncio

async def say_hello(delay, name):
    await asyncio.sleep(delay)
    return f"Hello, {name}!"

async def main():
    result = await say_hello(1, "Glenn")
    print(result)

asyncio.run(main())

async def fetch_all():
    tasks = [
        say_hello(2, "Alice"),
        say_hello(1, "Bob"),
        say_hello(3, "Charlie")
    ]
    results = await asyncio.gather(*tasks)
    return results

results = asyncio.run(fetch_all())
print(results)

async def main_with_group():
    async with asyncio.TaskGroup() as tg:
        task1 = tg.create_task(say_hello(1, "Task 1"))
        task2 = tg.create_task(say_hello(2, "Task 2"))
    print(task1.result())
    print(task2.result())

📘 Comprehensive Explanation Async/await enables concurrent I/O operations, improving performance for network calls, database queries, and file I/O without the complexity of threading. async def defines a coroutine. await suspends execution until the awaited coroutine completes. asyncio.run() runs the top-level coroutine. asyncio.gather() runs multiple coroutines concurrently. TaskGroup (Python 3.11+) provides structured concurrency. Async is essential for building responsive web servers, API clients, and any application that performs many I/O operations. It enables high concurrency with minimal overhead. Async is used for network requests, database operations, and concurrent I/O throughout the page. The output shows the async greeting for "Glenn", the list of results from concurrent tasks (Alice, Bob, Charlie), and the TaskGroup results (Task 1, Task 2).

💡 Async is ideal for I/O-bound tasks like network requests, database queries, and file operations. For CPU-bound tasks, use multiprocessing instead.

🧩 Dataclasses

Dataclasses reduce boilerplate by auto-generating __init__, __repr__, and __eq__.

from dataclasses import dataclass, field
from typing import List

@dataclass
class Person:
    name: str
    age: int
    email: str = field(default="")
    hobbies: List[str] = field(default_factory=list)

    @property
    def is_adult(self) -> bool:
        return self.age >= 18

    def add_hobby(self, hobby: str):
        self.hobbies.append(hobby)

person = Person("Glenn", 28, "glenn@example.com")
person.add_hobby("coding")
person.add_hobby("reading")
print(person)
print(person.is_adult)

@dataclass(frozen=True)
class Point:
    x: int
    y: int

p = Point(3, 4)

📘 Comprehensive Explanation Dataclasses eliminate repetitive boilerplate for classes that primarily hold data, making code cleaner and more maintainable. The @dataclass decorator automatically generates __init__, __repr__, __eq__, and __hash__. Fields are defined with type annotations. field() allows customization. default_factory creates new mutable defaults. frozen=True makes the dataclass immutable. Dataclasses are a modern replacement for namedtuple and manual class definitions for data containers. They reduce code duplication and improve readability. Dataclasses are used for data modeling, configuration, and structured data throughout the page. The output shows the Person dataclass with its string representation, the is_adult property, and the immutable Point dataclass.

📦 Modules and Packages

Modules are Python files. Packages are directories with __init__.py files.

import math
import datetime as dt
from random import randint, choice

print(math.sqrt(16))
print(dt.datetime.now())
print(randint(1, 10))

📘 Comprehensive Explanation Modules and packages are the mechanism for organizing code into reusable, maintainable units. They enable separation of concerns and code sharing. import module imports the whole module. import module as alias gives it a shorter name. from module import item imports specific items directly into the namespace. The module system is fundamental to Python's ecosystem, enabling the vast collection of third-party libraries and the standard library. Modules and packages are used to organize code, reuse functionality, and structure applications throughout the page. The output shows the square root of 16, the current datetime, and a random integer between 1 and 10.

🛡️ Exception Handling

Handle errors gracefully with try/except/else/finally and custom exceptions.

try:
    result = 10 / 0
except ZeroDivisionError as e:
    print(f"Error: {e}")

try:
    value = int("abc")
except ValueError:
    print("Invalid value")
except TypeError:
    print("Wrong type")

try:
    file = open("data.txt", "r")
    content = file.read()
except FileNotFoundError:
    print("File not found")
else:
    print(content)
finally:
    file.close()

class CustomError(Exception):
    def __init__(self, message, code):
        super().__init__(message)
        self.code = code

try:
    raise CustomError("Something went wrong", 500)
except CustomError as e:
    print(f"Error {e.code}: {e}")

📘 Comprehensive Explanation Exception handling allows programs to recover gracefully from errors, improving robustness and user experience. try contains code that might raise an exception. except catches specific exception types. else runs if no exception occurred. finally always runs (cleanup). Custom exceptions subclass Exception and can carry additional information. Exception handling is crucial for production-ready code. It prevents crashes, enables graceful error recovery, and makes debugging easier. Exception handling is used throughout the page for input validation, file I/O, and network operations. The output shows the ZeroDivisionError message, the ValueError message for invalid input, file handling messages, and the custom exception with code 500.

📁 File I/O

Read and write files using built-in functions and the pathlib module.

from pathlib import Path

with open("data.txt", "r", encoding="utf-8") as f:
    content = f.read()
    print(content)

with open("output.txt", "w", encoding="utf-8") as f:
    f.write("Hello, World!\n")
    f.writelines(["Line 1\n", "Line 2\n"])

with open("log.txt", "a", encoding="utf-8") as f:
    f.write("New entry\n")

import json

data = {"name": "Glenn", "age": 28}
with open("data.json", "w") as f:
    json.dump(data, f, indent=2)

with open("data.json", "r") as f:
    loaded = json.load(f)
    print(loaded)

path = Path("example.txt")
path.write_text("Hello, World!", encoding="utf-8")
content = path.read_text(encoding="utf-8")
print(content)

📘 Comprehensive Explanation File I/O is essential for persistent storage, configuration, logging, and data exchange. Every application interacts with the filesystem. open() with modes r (read), w (write), a (append). Context managers (with) ensure files are closed. json module handles structured data serialization. pathlib provides an object-oriented filesystem interface. File I/O is a fundamental skill for any programmer. Understanding reading, writing, and serialization is essential for building real-world applications. File I/O is used for configuration, data persistence, logging, and data exchange throughout the page. The output shows the content of data.txt, the JSON data loaded from file, and the content read from example.txt using pathlib.

🧮 Recursion

Recursive functions call themselves to solve problems that have smaller subproblems.

def factorial(n: int) -> int:
    if n <= 1:
        return 1
    return n * factorial(n - 1)

print(factorial(5))

def fibonacci(n: int) -> int:
    if n <= 1:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

print(fibonacci(10))

from functools import lru_cache

@lru_cache(maxsize=None)
def fib_memo(n: int) -> int:
    if n <= 1:
        return n
    return fib_memo(n - 1) + fib_memo(n - 2)

print(fib_memo(50))

import os

def list_files(path, indent=0):
    for item in os.listdir(path):
        full_path = os.path.join(path, item)
        print("  " * indent + item)
        if os.path.isdir(full_path):
            list_files(full_path, indent + 1)

📘 Comprehensive Explanation Recursion elegantly solves problems that have a recursive structure — tree traversal, divide-and-conquer algorithms, and mathematical sequences. A recursive function calls itself with a smaller input. A base case stops recursion. lru_cache memoizes results, dramatically improving performance for overlapping subproblems. Directory traversal recursively explores the filesystem. Recursion is a fundamental programming paradigm. Understanding it enables solving complex problems like tree traversal, parsing, and divide-and-conquer algorithms. Recursion is used for mathematical functions, tree traversal, and algorithmic problems throughout the page. The output shows factorial(5) = 120, fibonacci(10) = 55, fib_memo(50) = 12586269025 (computed efficiently with memoization), and directory listing.

🧪 Testing with Pytest

Write tests to ensure your code works correctly and catches regressions.

def add(a, b):
    return a + b

def divide(a, b):
    if b == 0:
        raise ZeroDivisionError("Cannot divide by zero")
    return a / b

def test_add():
    assert add(2, 3) == 5
    assert add(-1, 1) == 0

def test_divide():
    assert divide(10, 2) == 5
    assert divide(7, 3) == 7/3

def test_divide_by_zero():
    import pytest
    with pytest.raises(ZeroDivisionError):
        divide(10, 0)

📘 Comprehensive Explanation Testing is critical for ensuring code correctness, preventing regressions, and enabling confident refactoring. Pytest uses simple functions with assert statements. pytest.raises tests that exceptions are raised. Test functions are discovered by naming convention (test_*). Tests are essential for maintaining code quality in production systems. They catch bugs early and serve as executable documentation. Testing ensures that functions behave correctly and that changes don't introduce regressions throughout the page. The tests would pass when run with pytest, showing that add and divide work correctly and that divide raises ZeroDivisionError for b=0.

📊 List Comprehensions and Generator Expressions

Comprehensions provide a concise way to create lists, dicts, and sets.

squares = [x**2 for x in range(10)]
even_squares = [x**2 for x in range(10) if x % 2 == 0]
nested = [(x, y) for x in range(3) for y in range(3)]

square_dict = {x: x**2 for x in range(5)}
filtered_dict = {k: v for k, v in square_dict.items() if v % 2 == 0}

unique_squares = {x**2 for x in [1, 2, 2, 3, 3, 3]}

sum_squares = sum(x**2 for x in range(1000000))

matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flattened = [num for row in matrix for num in row]
print(flattened)

📘 Comprehensive Explanation Comprehensions are a Pythonic and efficient way to create collections. They are more concise and often faster than explicit loops. List comprehensions use [expr for item in iterable if condition]. Dictionary comprehensions use {key: value for key, value in iterable}. Set comprehensions use {expr for item in iterable}. Generator expressions use parentheses () for lazy evaluation. Comprehensions are a distinctive and powerful feature of Python. They reduce code verbosity and improve readability for collection transformations. Comprehensions are used for data transformation, filtering, and collection creation throughout the page. The output shows the flattened matrix [1, 2, 3, 4, 5, 6, 7, 8, 9] from nested list comprehension.

🔧 The itertools Module

itertools provides efficient looping tools for working with iterators.

import itertools

counter = itertools.count(start=1, step=2)
first5 = list(itertools.islice(counter, 5))

cycle = itertools.cycle(["A", "B", "C"])
first10 = list(itertools.islice(cycle, 10))

repeat = itertools.repeat("Hello", 3)

permutations = list(itertools.permutations([1, 2, 3], 2))
combinations = list(itertools.combinations([1, 2, 3, 4], 2))
combinations_with_replacement = list(itertools.combinations_with_replacement([1, 2, 3], 2))
product = list(itertools.product([1, 2], ["A", "B"]))

data = [("A", 1), ("A", 2), ("B", 3), ("B", 4)]
for key, group in itertools.groupby(data, key=lambda x: x[0]):
    print(key, list(group))

chained = list(itertools.chain([1, 2], [3, 4], [5, 6]))
zipped = list(itertools.zip_longest([1, 2], ["A", "B", "C"], fillvalue="X"))

📘 Comprehensive Explanation itertools is a powerful module that provides high-performance iterator building blocks. It is essential for efficient data processing and combinatorial operations. count() creates infinite sequences. islice() slices iterators. cycle() repeats indefinitely. permutations(), combinations(), product() generate combinatorial structures. groupby() groups adjacent elements. chain() concatenates iterables. zip_longest() zips with fill values. itertools is a core library for data processing pipelines. It provides efficient, memory-friendly operations that are often faster than manual implementations. itertools is used for combinatorial generation, grouping, and efficient iteration throughout the page. The output shows the first 5 odd numbers from count, the first 10 cycled elements, permutations, combinations, groupby results, chained list, and zipped list with fillvalue.

🧵 Threading vs Multiprocessing

Threading for I/O-bound tasks, multiprocessing for CPU-bound tasks.

from threading import Thread
from multiprocessing import Process
import time

def io_task(name, delay):
    time.sleep(delay)
    return f"Task {name} finished"

threads = []
for i in range(5):
    t = Thread(target=io_task, args=(i, 1))
    threads.append(t)
    t.start()

for t in threads:
    t.join()

def cpu_task(n):
    return sum(i * i for i in range(n))

from concurrent.futures import ProcessPoolExecutor

with ProcessPoolExecutor(max_workers=4) as executor:
    results = list(executor.map(cpu_task, [1000000, 1000000, 1000000]))

📘 Comprehensive Explanation Concurrency and parallelism are essential for performance. Threading handles I/O-bound tasks, while multiprocessing handles CPU-bound tasks. Thread runs tasks in the same process, sharing memory. Due to the GIL, threads are suitable for I/O-bound work. Process spawns separate processes, bypassing the GIL. ProcessPoolExecutor provides a high-level API for process pools. Understanding threading vs multiprocessing is crucial for writing performant applications. Choosing the right model can dramatically improve throughput. Threading is used for I/O-bound operations, while multiprocessing is used for CPU-intensive computations throughout the page. The output shows threads completing their io_task after 1 second each, and the ProcessPoolExecutor computing sums efficiently across multiple processes.

💡 threading works well for I/O-bound tasks because the GIL is released during I/O operations. For CPU-bound tasks, use multiprocessing to bypass the GIL.

📝 Decorators with Arguments (Decorator Factories)

Decorator factories return decorators that accept parameters.

from functools import wraps
import time
import logging

def log(level=logging.INFO):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            logging.log(level, f"Calling {func.__name__}")
            return func(*args, **kwargs)
        return wrapper
    return decorator

@log(level=logging.DEBUG)
def process_data(data):
    return data * 2

def add_method(func):
    def decorator(cls):
        setattr(cls, func.__name__, func)
        return cls
    return decorator

@add_method
def hello(self):
    return "Hello from the class!"

@add_method(hello)
class MyClass:
    pass

obj = MyClass()
print(obj.hello())

📘 Comprehensive Explanation Decorator factories enable parameterized decorators, adding flexibility and configurability to decorator patterns. A decorator factory is a function that takes parameters and returns a decorator. The returned decorator then wraps the target function. Class decorators can add methods to classes using setattr. Parameterized decorators are a powerful pattern for configuration, logging, caching, and access control. They make decorators more reusable and adaptable. Decorator factories are used for configurable logging, retry logic, and timing throughout the page. The output shows the log message "Calling process_data" at DEBUG level, and the hello method added to MyClass returning "Hello from the class!".

🔮 Metaclasses (Advanced)

Metaclasses are classes of classes. They define how classes behave.

class Meta(type):
    def __new__(cls, name, bases, dct):
        dct["version"] = "1.0"
        return super().__new__(cls, name, bases, dct)

    def __call__(cls, *args, **kwargs):
        instance = super().__call__(*args, **kwargs)
        instance.created_at = time.time()
        return instance

class MyClass(metaclass=Meta):
    def __init__(self, name):
        self.name = name

obj = MyClass("Glenn")
print(obj.version)
print(obj.created_at)

class Singleton(type):
    _instances = {}
    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super().__call__(*args, **kwargs)
        return cls._instances[cls]

class Database(metaclass=Singleton):
    def __init__(self):
        self.connected = False

db1 = Database()
db2 = Database()
print(db1 is db2)

📘 Comprehensive Explanation Metaclasses are the most advanced metaprogramming feature in Python. They control class creation and behavior, enabling frameworks like Django and SQLAlchemy. __new__ in a metaclass runs when the class is defined, allowing modification of the class before it's created. __call__ runs when an instance is created, allowing interception of object creation. The Singleton metaclass ensures only one instance exists. Metaclasses are powerful but rarely needed in everyday code. They are used in frameworks and libraries to implement advanced patterns like ORMs, serialization, and dependency injection. Metaclasses are used for framework-level concerns like singleton management, validation, and attribute injection throughout the page. The output shows the version attribute added by the metaclass, the created_at timestamp, and True for db1 is db2 (singleton).

💡 Metaclasses are powerful but rarely needed in everyday Python. They're used in frameworks like Django and SQLAlchemy.

📊 Working with Data: Pandas

Pandas is the go-to library for data manipulation and analysis.

import pandas as pd
import numpy as np

data = {
    "name": ["Alice", "Bob", "Charlie"],
    "age": [25, 30, 35],
    "score": [85, 92, 78]
}
df = pd.DataFrame(data)

print(df)
print(df.head())
print(df.info())
print(df.describe())

print(df["name"])
print(df[df["age"] > 28])
print(df.iloc[0:2])
print(df.loc[df["score"] > 80])

df["grade"] = df["score"].apply(lambda x: "A" if x >= 90 else "B" if x >= 80 else "C")

grouped = df.groupby("grade")["score"].mean()
print(grouped)

📘 Comprehensive Explanation Pandas is the foundation of data analysis in Python. It provides powerful data structures and operations for cleaning, transforming, and analyzing data. DataFrame is the core structure — a 2D table with labeled rows and columns. head() shows the first rows. info() shows data types and memory usage. describe() provides summary statistics. iloc indexes by position, loc by label. apply() applies functions. groupby() enables aggregation. Pandas is essential for any data-intensive work in Python. It's used in data science, finance, analytics, and machine learning preprocessing. Pandas is used for data loading, cleaning, transformation, and analysis throughout the page. The output shows the DataFrame, head, info, describe, column selection, filtering, row selection, grade assignment, and groupby mean.

📈 Plotting with Matplotlib

Matplotlib is the foundational plotting library for Python.

import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(0, 10, 100)
y = np.sin(x)

plt.plot(x, y, label="sin(x)", color="#4B8BBE")
plt.title("Sine Wave")
plt.xlabel("x")
plt.ylabel("sin(x)")
plt.legend()
plt.grid(True)
plt.show()

categories = ["A", "B", "C", "D"]
values = [23, 45, 12, 67]

plt.bar(categories, values, color="#4B8BBE")
plt.title("Bar Chart")
plt.xlabel("Category")
plt.ylabel("Value")
plt.show()

x = np.random.randn(100)
y = np.random.randn(100)

plt.scatter(x, y, alpha=0.5)
plt.title("Scatter Plot")
plt.show()

📘 Comprehensive Explanation Visualization is crucial for understanding data and communicating insights. Matplotlib is the foundation of Python's plotting ecosystem. plt.plot() creates line plots. plt.bar() creates bar charts. plt.scatter() creates scatter plots. plt.title(), xlabel(), ylabel() add labels. plt.legend() adds a legend. plt.grid() adds a grid. plt.show() displays the plot. Matplotlib is the most widely used plotting library in Python. It's the foundation for many higher-level libraries like Seaborn and Plotly. Matplotlib is used for exploratory data analysis, result visualization, and reporting throughout the page. The output shows three plots: a sine wave line plot, a bar chart with categories, and a scatter plot of random data.

🤖 Machine Learning with scikit-learn

scikit-learn provides simple and efficient tools for machine learning.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score

iris = load_iris()
X, y = iris.data, iris.target

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

clf = RandomForestClassifier(n_estimators=100, random_state=42)
clf.fit(X_train, y_train)

predictions = clf.predict(X_test)

accuracy = accuracy_score(y_test, predictions)
print(f"Accuracy: {accuracy:.2f}")

importance = clf.feature_importances_
for name, imp in zip(iris.feature_names, importance):
    print(f"{name}: {imp:.3f}")

📘 Comprehensive Explanation scikit-learn is the premier machine learning library in Python, providing a consistent API for a wide range of algorithms. load_iris() loads a classic dataset. train_test_split() splits data for training and testing. RandomForestClassifier is an ensemble model. fit() trains the model. predict() makes predictions. accuracy_score() evaluates performance. feature_importances_ shows feature importance. scikit-learn is the standard library for machine learning in Python. It's used in research, industry, and education for building predictive models. scikit-learn is used for model training, evaluation, and feature analysis throughout the page. The output shows the accuracy of the Random Forest classifier on the Iris test set (typically around 1.0 or 0.97), and the feature importance for each of the 4 features.

🌐 Web Scraping with BeautifulSoup

BeautifulSoup makes it easy to parse HTML and extract data from web pages.

import requests
from bs4 import BeautifulSoup

url = "https://example.com"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

title = soup.title.text
print(title)

links = soup.find_all("a")
for link in links[:5]:
    print(link.get("href"))

paragraphs = soup.select("p")
for p in paragraphs[:3]:
    print(p.text)

content = soup.find_all(class_="content")

📘 Comprehensive Explanation Web scraping enables automated data extraction from websites, useful for research, analytics, and building datasets. requests.get() fetches a webpage. BeautifulSoup parses the HTML. .title.text extracts the page title. .find_all("a") finds all links. .select("p") uses CSS selectors. .find_all(class_="content") finds elements by class. Web scraping is a powerful tool for data acquisition. BeautifulSoup simplifies HTML parsing and makes scraping accessible to Python developers. BeautifulSoup is used for extracting data from web pages, parsing HTML, and building datasets throughout the page. The output shows the page title, the href attributes of the first 5 links, and the text of the first 3 paragraphs.

🧠 Regular Expressions (re)

Regular expressions are powerful for pattern matching and text processing.

import re

text = "Contact me at glenn@example.com or call +1-555-1234"

email_pattern = r"[\w.+-]+@[\w-]+\.\w+"
emails = re.findall(email_pattern, text)
print(emails)

phone_pattern = r"\+\d-\d{3}-\d{4}"
phones = re.findall(phone_pattern, text)
print(phones)

new_text = re.sub(email_pattern, "[REDACTED]", text)
print(new_text)

pattern = re.compile(r"\b\w+\b")
words = pattern.findall(text)
print(words)

date_pattern = re.compile(r"(?P\d{4})-(?P\d{2})-(?P\d{2})")
match = date_pattern.search("2026-06-18")
if match:
    print(match.group("year"))
    print(match.group("month"))
    print(match.group("day"))

html = "Content 1
Content 2"
greedy = re.findall(r".*", html)
non_greedy = re.findall(r".*?", html)
print(greedy)
print(non_greedy)

📘 Comprehensive Explanation Regular expressions are the most powerful tool for pattern-based text processing. They are used for validation, extraction, and transformation. re.findall() finds all matches. re.sub() replaces matches. re.compile() creates a reusable pattern. Named groups (?P<name>...) extract parts of matches. Greedy quantifiers (.*) match as much as possible; non-greedy (.*?) match as little as possible. Regex is a critical tool for text processing in any language. Python's re module provides a comprehensive regex implementation. Regex is used for text validation, extraction, and transformation throughout the page. The output shows extracted email addresses, phone numbers, redacted text, all words, named group extraction, and the difference between greedy and non-greedy matching.

🗄️ Working with Databases (SQLite)

SQLite is a lightweight, file-based database that comes with Python.

import sqlite3

conn = sqlite3.connect("example.db")
cursor = conn.cursor()

cursor.execute("""
CREATE TABLE IF NOT EXISTS users (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT NOT NULL,
    age INTEGER,
    email TEXT UNIQUE
)
""")

cursor.execute("INSERT INTO users (name, age, email) VALUES (?, ?, ?)",
               ("Glenn", 28, "glenn@example.com"))
conn.commit()

users = [
    ("Alice", 25, "alice@example.com"),
    ("Bob", 30, "bob@example.com")
]
cursor.executemany("INSERT INTO users (name, age, email) VALUES (?, ?, ?)", users)
conn.commit()

cursor.execute("SELECT * FROM users")
rows = cursor.fetchall()
for row in rows:
    print(row)

cursor.execute("SELECT * FROM users WHERE age > ?", (25,))
rows = cursor.fetchall()
print(rows)

cursor.execute("UPDATE users SET age = ? WHERE name = ?", (29, "Glenn"))
conn.commit()

cursor.execute("DELETE FROM users WHERE name = ?", ("Bob",))
conn.commit()

conn.close()

📘 Comprehensive Explanation Database integration is essential for persistent storage, data management, and building data-driven applications. sqlite3.connect() opens a database connection. cursor.execute() runs SQL statements. Placeholders (?) prevent SQL injection. executemany() inserts multiple rows. fetchall() retrieves all results. commit() saves changes. SQLite is a zero-configuration database that's perfect for prototyping and small to medium applications. Python's built-in support makes it easy to get started. SQLite is used for data persistence, configuration storage, and application state throughout the page. The output shows the inserted rows, filtered rows by age, and the results of update and delete operations.

📦 Virtual Environments

Virtual environments isolate project dependencies.

python -m venv .venv
source .venv/bin/activate
.venv\Scripts\activate
pip install requests pandas
pip freeze > requirements.txt
pip install -r requirements.txt
deactivate

📘 Comprehensive Explanation Virtual environments are essential for managing dependencies, avoiding version conflicts, and ensuring reproducible environments. venv creates isolated Python environments with their own pip and packages. pip freeze exports installed packages. requirements.txt stores dependencies for reproducibility. Virtual environments are a best practice for Python development. They prevent dependency conflicts between projects and make collaboration easier. Virtual environments are used for dependency isolation, project reproducibility, and environment management throughout the page. The output shows the commands needed to create, activate, use, and deactivate a virtual environment.

💡 Always use virtual environments for Python projects. It keeps dependencies isolated and reproducible.

📝 Context Managers with contextlib

The contextlib module provides utilities for creating context managers.

from contextlib import contextmanager, suppress, closing
import urllib.request

with suppress(FileNotFoundError):
    with open("nonexistent.txt") as f:
        content = f.read()

with closing(urllib.request.urlopen("https://example.com")) as response:
    data = response.read()

@contextmanager
def timed(name):
    print(f"Starting {name}")
    start = time.time()
    try:
        yield
    finally:
        end = time.time()
        print(f"{name} took {end - start:.2f} seconds")

with timed("sleep"):
    time.sleep(1)

from contextlib import ExitStack

with ExitStack() as stack:
    files = [
        stack.enter_context(open("file1.txt", "w")),
        stack.enter_context(open("file2.txt", "w"))
    ]
    for f in files:
        f.write("Hello")

📘 Comprehensive Explanation contextlib provides additional context manager utilities for common patterns like suppression, closing, and stacking. suppress() ignores specified exceptions. closing() calls close() on exit. @contextmanager turns a generator into a context manager. ExitStack manages multiple context managers programmatically. These utilities reduce boilerplate and make code more robust. They're particularly useful for resource management and cleanup. contextlib utilities are used for resource management, exception suppression, and clean code throughout the page. The output shows the suppress context manager handling a missing file, the closing context manager fetching a URL, the timed context manager measuring execution time, and ExitStack managing multiple files.

🧬 Descriptors

Descriptors control attribute access in classes. They power @property and @classmethod.

class PositiveNumber:
    def __set_name__(self, owner, name):
        self.name = name

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return obj.__dict__.get(self.name, 0)

    def __set__(self, obj, value):
        if value < 0:
            raise ValueError(f"{self.name} must be positive")
        obj.__dict__[self.name] = value

class Product:
    price = PositiveNumber()
    quantity = PositiveNumber()

    def __init__(self, price, quantity):
        self.price = price
        self.quantity = quantity

p = Product(10, 5)
print(p.price)

class LazyAttribute:
    def __init__(self, func):
        self.func = func

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        value = self.func(obj)
        setattr(obj, self.func.__name__, value)
        return value

class Circle:
    def __init__(self, radius):
        self.radius = radius

    @LazyAttribute
    def area(self):
        print("Calculating area...")
        return 3.14159 * self.radius ** 2

c = Circle(5)
print(c.area)
print(c.area)

📘 Comprehensive Explanation Descriptors are the mechanism behind Python's property system. They enable reusable attribute logic across multiple classes. __set_name__ captures the attribute name. __get__ retrieves the value, __set__ validates and sets it. Data descriptors have both __get__ and __set__. Non-data descriptors only have __get__ and are used for lazy attributes. Descriptors are a fundamental but often overlooked feature of Python. They power properties, class methods, and static methods. Descriptors are used for validation, lazy loading, and attribute management throughout the page. The output shows the price (10) from the PositiveNumber descriptor, and the lazy area calculation that runs only once (the second access returns the cached value).

🐍 Python Coding Concepts