Your code works perfectly on your machine. Then you deploy it, and everything starts breaking. Sound familiar?
There's a moment almost every Python developer experiences.
You've spent hours building a script. You've tested it several times. Everything works exactly as expected.
Then you move it to a server.
And suddenly:
- The API stops responding.
- A file can't be found.
- A library behaves differently.
- The application crashes without a useful error message.
- A process that previously took 30 seconds now runs indefinitely.
The frustrating part?
You haven't changed the code.
So what went wrong?
The answer is usually simple: your local environment and your production environment are not the same.
And writing code that works is only the first step toward building software that works reliably.
Let's explore seven common problems that turn working Python scripts into production failures — and how to prevent them.
1. The Hidden Danger of Missing Timeouts
Let's start with something that looks completely harmless.
import requests response = requests.get( "https://api.example.com/products" ) print(response.json()) This code works perfectly when the API is available and responds quickly.
But what happens when the server stops responding?
By default, the requests library doesn't set a timeout.
That means your application could wait indefinitely for a response.
Now imagine this script is running as part of a scheduled job.
One unavailable API could block the entire workflow.
A better approach
import requests def fetch_products(): try: response = requests.get( "https://api.example.com/products", timeout=(3, 10) ) response.raise_for_status() return response.json() except requests.Timeout: print("Request timed out") except requests.RequestException as error: print(f"Request failed: {error}") except ValueError: print("Invalid JSON response") return None Here, the timeout configuration specifies:
- 3 seconds for establishing the connection.
- 10 seconds for waiting between socket read operations.
One important detail: these timeouts are not a strict 13-second limit on the total request duration.
A server that continuously sends small amounts of data may keep the connection active much longer.
For strict execution deadlines, you need additional controls at the application or infrastructure level.
Lesson: External services are not guaranteed to respond, even when your code is correct.
2. Hardcoded Paths Work Until They Don't
Consider a script that saves data to a CSV file.
import csv with open( "C:/Users/Developer/Desktop/products.csv", "w" ) as file: writer = csv.writer(file) writer.writerow(["id", "name", "price"]) This might work perfectly on your Windows laptop.
But what happens when you deploy it to a Linux server?
The directory doesn't exist.
The application fails.
This problem becomes even more common when working with Docker containers, virtual machines, or cloud environments.
Use pathlib instead
Python provides a much better way to handle filesystem paths.
from pathlib import Path import csv BASE_DIR = Path(__file__).resolve().parent DATA_DIR = BASE_DIR / "data" DATA_DIR.mkdir( parents=True, exist_ok=True ) OUTPUT_FILE = DATA_DIR / "products.csv" with OUTPUT_FILE.open( "w", newline="", encoding="utf-8" ) as file: writer = csv.writer(file) writer.writerow(["id", "name", "price"]) This approach is more portable and avoids operating-system-specific path separators.
However, there's another important production consideration.
The application directory might be read-only.
In containerized environments, you may also lose local files when a container is replaced.
For persistent data, configure an appropriate storage location or use an external storage service.
Lesson: Never assume that the production filesystem looks like your development machine.
3. Your Dependencies Are Part of Your Application
Here's a situation that can be surprisingly difficult to debug.
You develop a Python application using several libraries.
Everything works.
A colleague clones the repository and runs the same code.
It crashes.
Why?
Because your environments contain different dependency versions.
For example, your application might rely on behavior that changed between major versions of a library.
The wrong approach
pip install requests beautifulsoup4 pandas This installs available versions that satisfy the requirements at installation time.
But those versions may differ across machines or deployments.
A more reliable approach
For a simple project, you can maintain a dependency file with pinned versions.
requests==2.32.3 beautifulsoup4==4.12.3 pandas==2.2.3 These are example versions, not necessarily the versions your project should use.
Then install dependencies with:
python -m pip install -r requirements.txt For larger applications, consider tools such as uv, Poetry, or pip-tools to generate and maintain lockfiles.
Also remember that dependency versions aren't the entire environment.
Python version, operating system, native libraries, and system packages can affect application behavior.
For example, a package with native extensions might work on one platform but fail on another.
Lesson: Reproducibility is a requirement, not a convenience.
4. Print Statements Are Not a Logging Strategy
During development, many of us debug code like this:
print("Starting parser") print("Opening page") print("Extracting products") print("Done") There's nothing inherently wrong with print().
For small scripts, it's often sufficient.
But imagine your application runs every 15 minutes on a remote server.
One execution fails at 3:00 AM.
You need to know:
When did the failure occur?
Which operation failed?
What exception was raised?
How frequently has this happened?
Was the failure temporary or persistent?
Random print statements quickly become difficult to manage.
Use Python's logging module
import logging
logging.basicConfig(
level=logging.INFO,
format=(
"%(asctime)s | "
"%(levelname)s | "
"%(name)s | "
"%(message)s"
)
)
logger = logging.getLogger(name)
def process_products():
logger.info("Starting product processing")
try: result = 10 / 0 return result except ZeroDivisionError: logger.exception( "Product processing failed" ) raise The logger.exception() method automatically includes traceback information when called inside an exception handler.
That makes debugging significantly easier.
For production systems, structured JSON logs can make filtering and analyzing events even more effective.
One important rule: never log passwords, access tokens, or other sensitive information.
Lesson: If your application fails and you can't explain why, you have an observability problem.
- Environment Variables Are Better Than Hardcoded Secrets
Imagine finding this in a repository:
DATABASE_HOST = "192.168.1.100"
DATABASE_USER = "admin"
DATABASE_PASSWORD = "super_secret_password"
There are two problems.
First, these settings might only work in one environment.
Second, credentials embedded in source code create unnecessary security risks.
Separate configuration from code
import os
DATABASE_HOST = os.environ.get(
"DATABASE_HOST",
"localhost"
)
DATABASE_USER = os.environ["DATABASE_USER"]
DATABASE_PASSWORD = os.environ[
"DATABASE_PASSWORD"
]
Now the application can use different settings in development, testing, and production without modifying its source code.
For example:
export DATABASE_HOST="db.internal"
export DATABASE_USER="app_user"
export DATABASE_PASSWORD="your-secret"
Environment variables are useful, but they aren't automatically a complete secrets-management solution.
For sensitive production credentials, consider using a dedicated secrets manager, restricting access, and rotating credentials regularly.
Lesson: Configuration should change between environments without requiring code changes.
- Successful Execution Doesn't Mean Correct Results
This is one of the most dangerous problems in data-processing applications.
Imagine a web scraper that collects product information.
The script runs successfully.
No exceptions occur.
The output file is created.
Everything looks fine.
Except the file contains only 8 products instead of the expected 200.
Technically, the program completed.
But from a business perspective, it failed.
This is especially common in web scraping, ETL pipelines, API integrations, and scheduled data-processing jobs.
Validate the output
def validate_products(products):
if not isinstance(products, list):
raise TypeError(
"Expected a list of products"
)
if not products: raise ValueError( "No products were collected" ) for index, product in enumerate(products): if not isinstance(product, dict): raise TypeError( f"Invalid product at index {index}" ) required_fields = { "id", "name", "price" } missing = required_fields - product.keys() if missing: raise ValueError( f"Product {index} is missing: {missing}" ) return True This example checks the basic structure of the collected data.
In a real application, you might also validate:
Field types and allowed values.
Duplicate records.
Unexpectedly low record counts.
Missing identifiers.
Data freshness.
Sudden changes in price distributions.
A nonempty list is not proof that a scraping job succeeded.
And a zero exit code is not proof that the business requirements were satisfied.
Lesson: Validate outcomes, not just execution.
- Your Script Needs a Recovery Strategy
Let's imagine a Python application processing 10,000 products.
After processing 8,500 products, the server restarts.
What happens next?
If the application starts from the beginning, it might repeat thousands of operations.
If it doesn't save progress, some data could be lost.
This is where a simple script starts becoming a reliable system.
Introduce checkpoints
A basic approach is to track completed product identifiers.
import json
from pathlib import Path
CHECKPOINT_FILE = Path(
"completed_products.json"
)
def load_completed():
if not CHECKPOINT_FILE.exists():
return set()
with CHECKPOINT_FILE.open( "r", encoding="utf-8" ) as file: return set(json.load(file)) def save_completed(completed):
temporary = CHECKPOINT_FILE.with_suffix(
".tmp"
)
with temporary.open( "w", encoding="utf-8" ) as file: json.dump( sorted(completed), file ) temporary.replace(CHECKPOINT_FILE) Now a processing loop can skip previously completed records.
completed = load_completed()
for product_id in product_ids:
if product_id in completed:
continue
process_product(product_id) completed.add(product_id) save_completed(completed) This is a useful starting point for a single-process script.
However, it has limitations.
It assumes process_product() completes its work before the checkpoint is saved. If a crash occurs between an external side effect and the checkpoint update, that operation may be repeated.
For this reason, production workflows should consider idempotency.
An idempotent operation can be safely repeated without producing unwanted additional effects.
For larger systems, database-backed job states, durable queues, and transactional processing are often more appropriate than local JSON files.
Lesson: A reliable application should be designed for interruptions, not just successful runs.
- A Practical Production Readiness Checklist
Before deploying a Python application, I find it useful to ask a few questions.
Reliability
Do external requests have appropriate timeouts?
Are expected failures handled explicitly?
Can interrupted operations be safely retried?
Does the application validate its output?
Configuration
Are configuration values separated from source code?
Are secrets excluded from the repository?
Are dependency versions reproducible?
Does the application work in the target environment?
Observability
Are important operations logged?
Can failures be diagnosed from logs?
Are sensitive values excluded from logs?
Is there a way to detect repeated failures?
Data integrity
Can the application recover after a restart?
Are duplicate operations handled safely?
Is persistent data stored appropriately?
Can incomplete results be detected?
You don't need a complicated microservices architecture to address these problems.
Even a small Python application benefits from these principles.
- The Difference Between a Script and a System
One of the most valuable lessons in software development is understanding that a working implementation is not necessarily a reliable implementation.
A script answers the question:
"Can this task be automated?"
A production-ready system must answer additional questions:
"What happens when something goes wrong?"
"Can we detect the problem?"
"Can we recover without losing data?"
"Can another developer maintain this six months later?"
These questions fundamentally change how we design software.
The code itself might represent only a small portion of the engineering effort.
The rest involves understanding failure scenarios, infrastructure, dependencies, observability, and operational requirements.
And that's why moving from a local Python script to a production environment is such an important learning experience.
Final Thoughts
Python makes it incredibly easy to build useful applications quickly.
That's one of its greatest strengths.
But the simplicity of writing Python code can sometimes hide the complexity of operating software in real environments.
A script that works on your laptop is a good beginning.
A system that continues working when networks fail, servers restart, dependencies change, and unexpected data appears is a much more significant achievement.
The difference between a working script and reliable software isn't the number of lines of code. It's how well you've prepared for everything that can go wrong.
And that mindset is what turns programming into software engineering.
What's the most unexpected problem you've encountered when deploying a Python application?
Was it an environment issue, an API failure, a dependency conflict, or something completely different?
Share your experience in the comments. I'd love to hear about it.
