Why the 9‑second limit matters for small‑business sites

When I added a conversational assistant to my own Dream Futures Tech site, the first thing I hit was the 9‑second execution ceiling of AWS Lambda (the same limit you get on most serverless platforms). A timeout means the visitor sees a blank chat window, the request is logged as a failure, and the whole experience feels broken. For a local service business that lives on trust, that’s a deal‑breaker.

My three‑step approach

I solved the problem with a combination of warm EC2 hosting, a Lambda timeout guard, and a static FAQ fallback. The pieces are simple, cheap, and keep the user experience under the limit.

1. Warm EC2 for the LLM inference

Free, lightweight LLMs such as GPT‑4‑All‑J or Mistral‑7B‑Instruct run fast enough on a t3.medium when the instance stays warm. I set up a single EC2 that:

  • Starts on boot and loads the model into RAM.
  • Exposes a tiny HTTP endpoint (/infer) that accepts a JSON payload.
  • Keeps the process alive with a health‑check loop.

EC2 startup script (user‑data) 

#!/bin/bash # Install dependencies apt-get update && apt-get install -y python3-pip git pip3 install torch transformers fastapi uvicorn # Clone the model repo (example with GPT‑4‑All‑J) git clone https://github.com/nomic-ai/gpt4all.git /opt/gpt4all cd /opt/gpt4all # Download the model binary (replace with your own URL) wget -O model.bin https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin # Create a simple FastAPI server cat > app.py <<'EOF' from fastapi import FastAPI, Request from transformers import AutoModelForCausalLM, AutoTokenizer import torch, json app = FastAPI() model = AutoModelForCausalLM.from_pretrained('/opt/gpt4all', torch_dtype=torch.float16, low_cpu_mem_usage=True) tokenizer = AutoTokenizer.from_pretrained('/opt/gpt4all') @app.post('/infer') async def infer(req: Request): data = await req.json() prompt = data.get('prompt', '') inputs = tokenizer(prompt, return_tensors='pt') output = model.generate(**inputs, max_new_tokens=150) text = tokenizer.decode(output[0], skip_special_tokens=True) return {'response': text} EOF # Run the server in the background nohup uvicorn app:app --host 0.0.0.0 --port 8000 &

The EC2 stays warm because the process never exits. I also enable Instance Auto‑Recovery in the console so a hardware glitch won’t kill the model.

2. Lambda wrapper with a timeout guard

The public site still calls a Lambda function – it’s cheap, scales automatically, and lets me keep the same API contract for the front‑end. The Lambda does three things:

  1. Start a timer (using signal.alarm).
  2. Forward the request to the EC2 inference endpoint.
  3. If the timer fires, return a static fallback instead of letting Lambda crash.

Lambda handler (Python 3.9) 

import json, os, signal, urllib.request TIMEOUT_SECONDS = 8 # leave a buffer before Lambda's 9‑second limit EC2_ENDPOINT = os.getenv('EC2_ENDPOINT', 'http://<ec2‑public‑ip>:8000/infer') class TimeoutException(Exception): pass def _handle_timeout(signum, frame): raise TimeoutException() def lambda_handler(event, context): # Install the alarm  signal.signal(signal.SIGALRM, _handle_timeout) signal.alarm(TIMEOUT_SECONDS) try: payload = json.dumps({"prompt": event.get('prompt', '')}).encode() req = urllib.request.Request(EC2_ENDPOINT, data=payload, headers={'Content-Type': 'application/json'}) with urllib.request.urlopen(req, timeout=TIMEOUT_SECONDS) as resp: body = json.load(resp) signal.alarm(0) # cancel alarm  return {"statusCode": 200, "body": json.dumps(body)} except TimeoutException: # Fallback: static FAQ answer  fallback = { "response": "I’m still learning, but you can check our FAQ at /faq for common questions about our services." } return {"statusCode": 200, "body": json.dumps(fallback)} except Exception as e: # Unexpected error – log and return generic message  print('Error:', e) return {"statusCode": 500, "body": json.dumps({"response": "Sorry, something went wrong. Please try again later."})}

The guard ensures the Lambda never exceeds the platform limit. If the EC2 call takes too long, the user still gets a helpful message instead of a timeout error.

3. Static FAQ fallback (S3 + CloudFront)

Even with the guard, it’s good to have a zero‑latency answer for the most common queries. I store a small JSON file in an S3 bucket (/faq.json) and serve it through CloudFront. The Lambda’s fallback pulls the same content, so the wording is consistent across the site.

Sample faq.json 

{ "What services do you offer?": "We provide web design, SEO, and automation for small businesses in South Florida.", "How much does a website cost?": "Pricing starts at $1,500 for a basic site. Contact us for a custom quote.", "Do you offer ongoing support?": "Yes, we have monthly maintenance packages starting at $99 per month." }

The front‑end chat widget checks the response type: if it receives the fallback flag, it renders the FAQ list instantly, keeping the interaction under a second.

Putting it together

  1. Deploy the EC2 with the user‑data script above. Tag it Chatbot-Model and note the public IP.
  2. Create the Lambda (Node or Python) and set the EC2_ENDPOINT environment variable.
  3. Add the S3 FAQ and configure CloudFront to cache it for 24 hours.
  4. Update the chat widget to call the Lambda endpoint (/chat) and handle three possible payloads:
    • Normal LLM response.
    • Fallback with static FAQ.
    • Generic error.

All of this runs for under $15/month on my AWS bill (t3.medium EC2 + Lambda invocations + S3/CF). The biggest cost is the EC2 instance staying warm, but the price is predictable and the latency stays comfortably under the 9‑second ceiling.

Lessons learned

  • Warm‑up matters – a cold start on EC2 adds 2‑3 seconds, which eats into the Lambda budget.
  • Never trust a single path – the timeout guard and static fallback keep the user experience graceful.
  • Monitoring is cheap – a CloudWatch metric on Duration quickly shows if you’re approaching the limit, letting you adjust the buffer.

By keeping the heavy LLM work on a dedicated warm instance and using Lambda only as a thin wrapper, I got a responsive chatbot that never exceeds the serverless timeout, even on a free‑tier model.

I build sites and automations for small businesses at dreamfuturestech.com.