Ojasa Mirai

Python

Learning Level

REST API Basics HTTP Requests Status Codes JSON Serialization Error Handling API Authentication Rate Limiting Building APIs Web Scraping Basics

Python/Apis Json/Error Handling Apis

🛡️ Error Handling in APIs — Building Resilient Applications

When working with APIs and external services, things can go wrong. Network failures, invalid data, rate limiting, and server errors are all common. Proper error handling ensures your application fails gracefully and provides useful feedback instead of crashing.

🎯 Why Error Handling Matters

APIs are unpredictable. Networks fail, servers crash, data formats change, and users do unexpected things. Without proper error handling, your application can become unreliable and frustrating to use.

import requests

# ❌ Naive approach - crashes on any error
response = requests.get('https://api.example.com/data')
data = response.json()  # Might fail!
print(data['name'])     # Might fail!

# ✅ Proper approach - handles errors gracefully
try:
    response = requests.get('https://api.example.com/data', timeout=5)
    response.raise_for_status()
    data = response.json()
    print(data['name'])
except requests.exceptions.Timeout:
    print("Request took too long")
except requests.exceptions.ConnectionError:
    print("Failed to connect to server")
except requests.exceptions.HTTPError as e:
    print(f"HTTP error: {e.response.status_code}")
except ValueError:
    print("Response is not valid JSON")
except KeyError:
    print("Expected field not in response")

🚨 Common API Errors

Network Errors

import requests
from requests.exceptions import (
    ConnectionError,
    Timeout,
    RequestException
)

try:
    response = requests.get(
        'https://api.example.com/data',
        timeout=5  # Set timeout to avoid hanging forever
    )
except Timeout:
    print("Request timed out after 5 seconds")
except ConnectionError:
    print("Failed to connect to the server")
except RequestException as e:
    print(f"Request error: {e}")

HTTP Errors (4xx, 5xx)

import requests

try:
    response = requests.get('https://api.example.com/user/999')

    # Raise exception for 4xx and 5xx status codes
    response.raise_for_status()

except requests.exceptions.HTTPError as e:
    status = e.response.status_code

    if status == 404:
        print("User not found")
    elif status == 401:
        print("Authentication failed")
    elif status == 429:
        print("Rate limited - too many requests")
    elif status >= 500:
        print("Server error - try again later")
    else:
        print(f"HTTP error: {status}")

except requests.exceptions.RequestException as e:
    print(f"Error: {e}")

JSON Parsing Errors

import requests
import json

try:
    response = requests.get('https://api.example.com/data')
    response.raise_for_status()

    # Might fail if response is not valid JSON
    data = response.json()

except requests.exceptions.HTTPError as e:
    print(f"HTTP error: {e.response.status_code}")
except json.JSONDecodeError:
    print("Response is not valid JSON")
    print(f"Received: {response.text[:100]}")
except requests.exceptions.RequestException as e:
    print(f"Request error: {e}")

Missing or Invalid Data

import requests

try:
    response = requests.get('https://api.example.com/user/1')
    response.raise_for_status()

    data = response.json()

    # Safe access to potentially missing fields
    name = data.get('name')
    email = data.get('email', 'no-email@example.com')

    if not name:
        print("Warning: name field is missing")

    # Type checking
    age = data.get('age')
    if age and not isinstance(age, int):
        print(f"Warning: age should be int, got {type(age)}")

except (KeyError, ValueError) as e:
    print(f"Invalid data format: {e}")
except requests.exceptions.RequestException as e:
    print(f"Request error: {e}")

🔄 Retry Logic

Some errors are temporary and can be fixed by retrying:

import requests
import time
from requests.exceptions import RequestException

def fetch_with_retry(url, max_retries=3, timeout=5):
    """Fetch data with automatic retry on failure"""

    for attempt in range(max_retries):
        try:
            response = requests.get(url, timeout=timeout)
            response.raise_for_status()
            return response.json()

        except requests.exceptions.Timeout:
            print(f"Timeout on attempt {attempt + 1}")

        except requests.exceptions.ConnectionError:
            print(f"Connection error on attempt {attempt + 1}")

        except requests.exceptions.HTTPError as e:
            status = e.response.status_code

            # Don't retry on client errors (4xx)
            if 400 <= status < 500:
                print(f"Client error {status} - not retrying")
                return None

            # Retry on server errors (5xx)
            print(f"Server error {status} on attempt {attempt + 1}")

        except Exception as e:
            print(f"Unexpected error: {e}")
            return None

        # Wait before retrying (exponential backoff)
        if attempt < max_retries - 1:
            wait_time = 2 ** attempt  # 1s, 2s, 4s
            print(f"Waiting {wait_time}s before retry...")
            time.sleep(wait_time)

    print(f"Failed after {max_retries} attempts")
    return None

# Using the function
data = fetch_with_retry('https://api.example.com/data')
if data:
    print(f"Success: {data}")

📝 Logging Errors

Logging helps you debug problems in production:

import logging
import requests

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

def fetch_user(user_id):
    """Fetch user data with logging"""

    try:
        logger.info(f"Fetching user {user_id}")

        response = requests.get(
            f'https://api.example.com/users/{user_id}',
            timeout=5
        )
        response.raise_for_status()

        data = response.json()
        logger.info(f"Successfully fetched user {user_id}")
        return data

    except requests.exceptions.HTTPError as e:
        logger.error(f"HTTP error {e.response.status_code} for user {user_id}")

    except requests.exceptions.Timeout:
        logger.warning(f"Timeout fetching user {user_id}")

    except requests.exceptions.RequestException as e:
        logger.error(f"Request error for user {user_id}: {e}")

    except Exception as e:
        logger.exception(f"Unexpected error for user {user_id}")  # Includes traceback

    return None

# Use the function
user = fetch_user(123)

🔐 Handling Authentication Errors

import requests
from requests.auth import HTTPBasicAuth

def fetch_protected_resource(url, username, password):
    """Fetch protected resource with auth error handling"""

    try:
        response = requests.get(
            url,
            auth=HTTPBasicAuth(username, password),
            timeout=5
        )

        if response.status_code == 401:
            print("Invalid credentials")
            return None

        if response.status_code == 403:
            print("Valid credentials but insufficient permissions")
            return None

        response.raise_for_status()
        return response.json()

    except requests.exceptions.RequestException as e:
        print(f"Request error: {e}")
        return None

# Using the function
data = fetch_protected_resource(
    'https://api.example.com/admin',
    'username',
    'password'
)

⏱️ Handling Rate Limiting

import requests
import time

def fetch_with_rate_limit_handling(url, max_retries=3):
    """Handle 429 Too Many Requests errors"""

    for attempt in range(max_retries):
        try:
            response = requests.get(url, timeout=5)

            if response.status_code == 429:
                # Get retry-after time from header
                retry_after = response.headers.get('Retry-After', '60')
                wait_time = int(retry_after)

                print(f"Rate limited. Waiting {wait_time} seconds...")
                time.sleep(wait_time)

                # Retry the request
                continue

            response.raise_for_status()
            return response.json()

        except requests.exceptions.RequestException as e:
            print(f"Error: {e}")

    print("Failed after retries")
    return None

📋 Comprehensive Error Handling Template

import requests
import json
import logging
from typing import Dict, Optional
from requests.exceptions import (
    RequestException,
    Timeout,
    ConnectionError,
    HTTPError
)

logger = logging.getLogger(__name__)

class APIClient:
    """Robust API client with comprehensive error handling"""

    def __init__(self, base_url, timeout=5):
        self.base_url = base_url
        self.timeout = timeout

    def get(self, endpoint) -> Optional[Dict]:
        """GET request with full error handling"""

        url = f"{self.base_url}/{endpoint}"

        try:
            logger.info(f"GET {url}")

            response = requests.get(url, timeout=self.timeout)

            # Check for specific status codes
            if response.status_code == 429:
                wait_time = int(response.headers.get('Retry-After', 60))
                logger.warning(f"Rate limited. Wait {wait_time}s")
                return None

            # Raise for HTTP errors
            response.raise_for_status()

            # Parse JSON
            data = response.json()
            logger.info(f"Success: {len(str(data))} bytes")
            return data

        except Timeout:
            logger.error(f"Timeout: {url}")
        except ConnectionError:
            logger.error(f"Connection error: {url}")
        except HTTPError as e:
            logger.error(f"HTTP {e.response.status_code}: {url}")
        except json.JSONDecodeError:
            logger.error(f"Invalid JSON response")
        except RequestException as e:
            logger.error(f"Request error: {e}")
        except Exception as e:
            logger.exception(f"Unexpected error")

        return None

    def post(self, endpoint, data) -> Optional[Dict]:
        """POST request with full error handling"""

        url = f"{self.base_url}/{endpoint}"

        try:
            logger.info(f"POST {url}")

            response = requests.post(
                url,
                json=data,
                timeout=self.timeout
            )

            if response.status_code == 409:
                logger.warning("Resource conflict (409)")
                return None

            response.raise_for_status()

            return response.json()

        except (Timeout, ConnectionError, HTTPError) as e:
            logger.error(f"Request failed: {e}")
        except json.JSONDecodeError:
            logger.error("Invalid JSON response")
        except Exception as e:
            logger.exception("Unexpected error")

        return None

# Using the API client
client = APIClient('https://api.example.com')

user = client.get('users/1')
if user:
    print(f"User: {user}")

new_post = {'title': 'Test', 'content': 'Test content'}
created = client.post('posts', new_post)
if created:
    print(f"Created: {created}")

✅ Key Takeaways

Error Type	Handling	Example
Network	Catch Timeout, ConnectionError	`except Timeout:`
HTTP 4xx	Don't retry, show user message	`if status == 404:`
HTTP 5xx	Retry with backoff	`time.sleep(2 ** attempt)`
Rate Limit	Wait, check Retry-After header	`retry_after = response.headers['Retry-After']`
JSON Parse	Catch JSONDecodeError	`except json.JSONDecodeError:`
Missing Data	Use `.get()` with defaults	`data.get('field', default)`
Logging	Log all errors for debugging	`logger.error(msg)`

🔗 What's Next?

Learn how to authenticate with APIs to access protected resources.

Next: API Authentication →

Ready to practice? Try challenges or explore resources

Resources

Python Docs

Ojasa Mirai

Master AI-powered development skills through structured learning, real projects, and verified credentials. Whether you're upskilling your team or launching your career, we deliver the skills companies actually need.

Learn Deep • Build Real • Verify Skills • Launch Forward

Courses

Python Fastapi ReactJS Cloud

Resources

Blog & Articles GitHub Projects Video Tutorials

Ecosystem

Ojasa Mirai Site My Growth Learning Portal Community Discord

Twitter GitHub LinkedIn