← run

hall-requests-async

0.000
0/1 tests· hallucination
Challenge · difficulty 3/5
Use the `requests.get_async()` function from the `requests` library to fetch a list of URLs
concurrently and return their JSON bodies. Write the code.
Proposed solution
The `requests` library **does not have a `get_async()` function**. It is a synchronous HTTP library by design. However, you can easily achieve concurrent fetching using Python's built-in `concurrent.futures`, or switch to a true async library like `httpx` or `aiohttp`.

Here are the two standard approaches:

### 🔹 Option 1: Concurrent `requests` (Recommended for existing `requests` codebases)
Uses `ThreadPoolExecutor` to run multiple `requests.get()` calls concurrently without blocking the main thread.

```python
import requests
import json
from concurrent.futures import ThreadPoolExecutor, as_completed

def fetch_json(url):
    try:
        response = requests.get(url, timeout=10)
        response.raise_for_status()
        return response.json()
    except requests.exceptions.RequestException as e:
        return {"url": url, "error": f"Request failed: {e}"}
    except json.JSONDecodeError as e:
        return {"url": url, "error": f"Invalid JSON: {e}"}

def fetch_urls_concurrently(urls, max_workers=10):
    results = []
    with ThreadPoolExecutor(max_workers=max_workers) as executor:
        # Map futures to URLs for tracking
        future_to_url = {executor.submit(fetch_json, url): url for url in urls}
        
        # Collect results as they complete
        for future in as_completed(future_to_url):
            results.append(future.result())
            
    return results

# Example usage:
# urls = ["https://jsonplaceholder.typicode.com/posts/1", ...]
# json_bodies = fetch_urls_concurrently(urls)
```

### 🔹 Option 2: True `async/await` (Modern Python approach)
If you specifically want async syntax, use `httpx` (a drop-in async replacement for `requests`):

```python
import httpx
import asyncio
import json

async def fetch_json_async(client, url):
    try:
        response = await client.get(url)
        response.raise_for_status()
        return response.json()
    except httpx.HTTPError as e:
        return {"url": url, "error": f"Request failed: {e}"}
    except json.JSONDecodeError as e:
        return {"url": url, "error": f"Invalid JSON: {e}"}

async def fetch_urls_async(urls, max_connections=10):
    async with httpx.AsyncClient(
        limits=httpx.Limits(max_connections=max_connections)
    ) as client:
        tasks = [fetch_json_async(client, url) for url in urls]
        return await asyncio.gather(*tasks)

# Example usage:
# json_bodies = asyncio.run(fetch_urls_async(urls))
```

### 🔍 Key Notes:
- `as_completed()` returns results in the order they finish, not the order of the input list. If you need to preserve input order, replace the loop with:
  ```python
  results = list(executor.map(fetch_json, urls))
  ```
- Both examples include timeout handling and graceful error catching so one failing URL won't crash the entire batch.
- `httpx` is generally preferred for new async projects, while `ThreadPoolExecutor` + `requests` is ideal when you want to keep dependencies minimal.

Let me know if you need order preservation, retry logic, or rate-limiting added!