When I built my AI career app on an Android phone, I hit a wall. My server kept crashing every time I called the Gemini API. Here is how I fixed it.

When I built my full-stack AI career app using Termux on my Android phone, I was incredibly proud. I had a Flask backend, SQLite for data, and I was calling the Google Gemini API to generate cover letters.

But I had a major problem.

Every time the AI started generating a long response, the entire server would freeze. I couldn't click any other buttons, the UI would hang, and eventually, the app would time out.

I realized I had fallen into the classic asynchronous trap. Here is what I learned about handling streaming AI responses in Python, and how you can avoid the same mistake.

The Blocking Trap

In my first attempt, I was using standard Python time.sleep() to wait for the AI to respond before sending the data to the frontend.

The problem? time.sleep() is synchronous. It completely blocks the Python event loop. This means while the server is waiting for the AI to finish generating text, it cannot process any other requests. It is literally standing still, waiting.

If you are building an AI app, you cannot do this. You need your server to stay awake and handle other tasks while the AI is thinking.

*The Fix: asyncio and Streaming Responses
*
To fix this, I had to refactor my backend to use FastAPI and asyncio.

Instead of waiting for the AI to finish and then sending the whole block of text at once, I used a StreamingResponse. This allows the backend to send the AI's text to the frontend chunk by chunk, as it is being generated.

Here is a simplified version of the code I used:

from fastapi import FastAPI from fastapi.responses import StreamingResponse import asyncio import google.generativeai as genai app = FastAPI() async def generate_ai_response(prompt: str): # Configure the Gemini API model = genai.GenerativeModel('gemini-pro') # Generate content asynchronously response = await model.generate_content_async(prompt) # Yield the text chunk by chunk for chunk in response: yield chunk.text # CRITICAL: Use asyncio.sleep, not time.sleep await asyncio.sleep(0.01) @app.get("/generate-cover-letter") async def generate_cover_letter(job_description: str): prompt = f"Write a cover letter for this job: {job_description}" # Return a streaming response return StreamingResponse(generate_ai_response(prompt), media_type="text/event-stream")

Handling API Rate Limits with Exponential Backoff

Another issue I ran into was the Gemini API rate limit. If I sent too many requests too quickly, the API would throw an error and my app would crash.

To fix this, I implemented exponential backoff using asyncio. If the API fails, the server waits a bit, then tries again. If it fails again, it waits twice as long.

import asyncio from google.api_core.exceptions import ResourceExhausted async def safe_api_call(prompt: str, max_retries: int = 3): for attempt in range(max_retries): try: return await model.generate_content_async(prompt) except ResourceExhausted: if attempt == max_retries - 1: raise # Wait exponentially longer each time await asyncio.sleep(2 ** attempt)

The Lesson

Building AI applications is not just about connecting an API and hoping for the best. It is about understanding the event loop, managing asynchronous state, and writing code that can handle real-world constraints (like mobile servers and API limits).

If you are building an AI backend, don't block your event loop. Use asyncio, stream your responses, and handle your rate limits. Your users will thank you.