Tripadvisor Scraper GitHub: How to Monitor Hotel and Restaurant Reviews with Python
You can monitor a set of hotels and restaurants on Tripadvisor with a small Python workflow: capture public review records for each property into JSON, normalize ratings and dates into a consistent schema, and compute weekly deltas so you can see whether a property's rating trend is improving, flat, or degrading. The hard part is not the arithmetic — it is keeping the dataset clean across repeated runs: review counts that drift between captures, dates in mixed formats, properties that change names, and the temptation to compare a snapshot from March with a snapshot from November as if they were the same population. This guide is for hospitality analysts, agency teams managing reputation for multiple properties, and developers who want a repeatable monitoring loop built on the data-scrape/tripadvisor-scraper repository instead of a spreadsheet that someone updates by hand.
TL;DR
- Use
data-scrape/tripadvisor-scraperto capture public review and rating records for a bounded list of properties, one capture per run. - Normalize every record into a fixed schema:
property_id,capture_date,rating,review_count,review_date,review_text_snippet. Fixed schema first, analysis second. - Store raw captures immutably and derive everything else. When a number looks wrong, you diff two raw captures instead of guessing.
- Compute weekly deltas per property: rating change, review-count change, and the share of recent reviews versus the historical baseline.
- Deduplicate reviews on a stable key (property plus review date plus normalized text prefix), because repeated captures overlap.
- Track a coverage metric for every run, so "the rating dropped" is distinguishable from "the page failed to parse".
- Ratings are a lagging signal: pair review monitoring with the public pricing pipeline in
data-scrape/booking-scraperif you also need rate context.
Why Review Monitoring Breaks Past Ten Properties
Monitoring one hotel is a bookmark. Monitoring thirty — across cities, with different review volumes and cadences — is a data problem, and it fails in predictable ways:
Ratings arrive in mixed formats. A property page shows a bubble score on a five-point scale, but exports and copies frequently carry strings like "4.5 of 5 bubbles" or "4,5". Every numeric field needs an explicit parser with a documented fallback, or your averages silently skew.
Review counts drift between captures. Old reviews get removed, moderation changes counts, and a property can lose reviews it gained. If your trend line is built on latest-value snapshots rather than dated captures, you cannot reconstruct what you actually observed on a given day.
Repeated captures overlap. If you capture weekly, most reviews in week N also appeared in week N-1. Without deduplication on a stable key, your "new reviews this week" count is inflated and your sentiment sample double-counts.
Dates come in several shapes. Relative dates ("a week ago"), locale-formatted dates, and full timestamps can coexist. Normalize to ISO 8601 at ingestion, or your "recent reviews" filter silently selects the wrong window.
A snapshot is not a trend. The whole point of monitoring is change over time. That requires immutable, dated raw captures and a derivation layer — not a single CSV that gets overwritten every Monday.
What the Verified Repositories Provide
The data-scrape profile hosts Python repositories that share one pattern: a scraper.py entry point, a requirements.txt, an examples/ folder with representative output, and a README documenting the supported inputs and flags. They are reference implementations you run yourself, not hosted services, and each README states the code is intended for public-data research and educational use, with the user responsible for complying with the target site's terms of service, robots directives, and applicable law. Read the README before you run anything, and verify the current flags against the repository rather than assuming them from this article.
data-scrape/tripadvisor-scraper is the primary collection layer for this workflow: it targets public Tripadvisor property and review pages and emits structured JSON records. Run it yourself, inspect the examples/ output, and map its actual field names into the schema below — the exact field names are the repository's to define, and the normalization layer is where you absorb any differences.
data-scrape/booking-scraper covers public hotel and pricing data on Booking.com. It is optional here, but reputation and rate are two halves of the same revenue question, and keeping both pipelines in the same JSONL pattern means one analysis layer can consume either.
The Workflow
The loop has four stages, and the ordering matters:
- Capture. Run the scraper for a bounded list of property URLs — one geography or one client portfolio per run. Write raw JSONL to a dated path like
raw/2026-11-21/property_id.jsonland never modify it afterward. - Normalize. Map each raw record to the fixed schema: parse ratings to floats, parse dates to ISO 8601, trim review text to a bounded snippet, and reject rows that fail validation into a quarantine file rather than dropping them silently.
- Deduplicate and diff. Within a capture, deduplicate on the stable review key. Across captures, compute per-property deltas against the previous capture of the same property.
- Report. Emit a weekly summary: current rating, rating delta, review-count delta, new-review count, and the coverage rate of each stage.
Configuration lives in environment variables — the property list path, the output directory, and any credentials or endpoint values the repository's README documents — so the same code moves from a laptop to a scheduler without edits.
Runnable Python: Normalize, Deduplicate, and Compute Deltas
This script is the analysis layer. It consumes the JSONL records the repository's examples produce (adjust the field mapping to the repository's actual output), keeps raw captures immutable, and produces a weekly delta report. It uses only the standard library, so it runs anywhere Python 3.9+ runs.
""" tripadvisor_review_monitor.py Normalize public Tripadvisor review captures, deduplicate them, and compute per-property weekly deltas. Usage: export TA_CAPTURES_DIR=/path/to/captures # contains dated subfolders export TA_PROPERTIES=/path/to/properties.json python tripadvisor_review_monitor.py """ import json import os import re import sys from datetime import date, datetime from pathlib import Path CAPTURES_DIR = Path(os.environ["TA_CAPTURES_DIR"]) PROPERTIES_FILE = Path(os.environ["TA_PROPERTIES"]) RATING_MAX = 5.0 SNIPPET_LIMIT = 280 def parse_rating(raw) -> float | None: """Accept floats, '4.5', '4,5', or '4.5 of 5'; reject everything else.""" if raw is None: return None if isinstance(raw, (int, float)) and 0 <= raw <= RATING_MAX: return round(float(raw), 2) text = str(raw).strip().replace(",", ".") match = re.match(r"^(\d(?:\.\d)?)", text) if not match: return None value = float(match.group(1)) return round(value, 2) if 0 <= value <= RATING_MAX else None def parse_date(raw) -> str | None: """Normalize common date shapes to ISO 8601; return None when unparseable.""" if not raw: return None text = str(raw).strip() for fmt in ("%Y-%m-%d", "%B %Y", "%b %d, %Y", "%d %B %Y", "%Y-%m-%dT%H:%M:%S"): try: return datetime.strptime(text, fmt).date().isoformat() except ValueError: continue return None def load_properties() -> dict: """properties.json: [{"property_id": "h001", "name": "...", "url": "..."}]""" props = json.loads(PROPERTIES_FILE.read_text(encoding="utf-8")) return {p["property_id"]: p for p in props} def normalize_record(raw: dict, property_id: str, capture_date: str) -> dict | None: rating = parse_rating(raw.get("rating")) review_date = parse_date(raw.get("review_date")) text = (raw.get("review_text") or "").strip() if rating is None or not text: return None # quarantine candidates: count them, don't drop silently return { "property_id": property_id, "capture_date": capture_date, "rating": rating, "review_date": review_date, "snippet": text[:SNIPPET_LIMIT], # stable dedup key: property + review date + normalized text prefix "dedup_key": f"{property_id}|{review_date}|{text[:80].lower().strip()}", } def load_latest_captures() -> dict[str, list[dict]]: """Return {property_id: [normalized records]} from the two most recent captures.""" capture_dates = sorted( (d.name for d in CAPTURES_DIR.iterdir() if d.is_dir()), reverse=True )[:2] if not capture_dates: sys.exit(f"No capture folders found under {CAPTURES_DIR}") by_property: dict[str, list[dict]] = {} quarantine = 0 for capture_date in capture_dates: for path in (CAPTURES_DIR / capture_date).glob("*.jsonl"): property_id = path.stem for line in path.read_text(encoding="utf-8").splitlines(): if not line.strip(): continue record = normalize_record(json.loads(line), property_id, capture_date) if record is None: quarantine += 1 continue by_property.setdefault(property_id, []).append(record) if quarantine: print(f"WARNING: {quarantine} rows quarantined (missing rating or text)") return by_property def deduplicate(records: list[dict]) -> list[dict]: seen, unique = set(), [] for record in records: if record["dedup_key"] not in seen: seen.add(record["dedup_key"]) unique.append(record) return unique def property_delta(records: list[dict], previous_date: str, current_date: str) -> dict: prev = [r for r in records if r["capture_date"] == previous_date] curr = [r for r in records if r["capture_date"] == current_date] prev_unique, curr_unique = deduplicate(prev), deduplicate(curr) prev_ratings = [r["rating"] for r in prev_unique] curr_ratings = [r["rating"] for r in curr_unique] prev_avg = round(sum(prev_ratings) / len(prev_ratings), 2) if prev_ratings else None curr_avg = round(sum(curr_ratings) / len(curr_ratings), 2) if curr_ratings else None prev_keys = {r["dedup_key"] for r in prev_unique} new_reviews = [r for r in curr_unique if r["dedup_key"] not in prev_keys] return { "previous_capture": previous_date, "current_capture": current_date, "avg_rating_previous": prev_avg, "avg_rating_current": curr_avg, "rating_delta": ( round(curr_avg - prev_avg, 2) if prev_avg is not None and curr_avg is not None else None ), "review_count_previous": len(prev_unique), "review_count_current": len(curr_unique), "new_reviews_this_period": len(new_reviews), "coverage_rate": round(len(curr_unique) / len(curr), 2) if curr else 0.0, } def main() -> None: properties = load_properties() captures = load_latest_captures() capture_dates = sorted( (d.name for d in CAPTURES_DIR.iterdir() if d.is_dir()), reverse=True )[:2] if len(capture_dates) < 2: print("Only one capture found; deltas appear after the next run.") return previous_date, current_date = sorted(capture_dates) report = [] for property_id in properties: records = captures.get(property_id, []) if not records: report.append({"property_id": property_id, "status": "no_data"}) continue report.append({"property_id": property_id, **property_delta(records, previous_date, current_date)}) out_path = CAPTURES_DIR / f"delta_report_{date.today().isoformat()}.json" out_path.write_text(json.dumps(report, indent=2), encoding="utf-8") print(f"Wrote {out_path} ({len(report)} properties)") if __name__ == "__main__": main() Representative Output
The exact values are examples of the report shape, not measurements of any real property:
[ { "property_id": "h001", "previous_capture": "2026-11-14", "current_capture": "2026-11-21", "avg_rating_previous": 4.42, "avg_rating_current": 4.38, "rating_delta": -0.04, "review_count_previous": 96, "review_count_current": 103, "new_reviews_this_period": 7, "coverage_rate": 0.98 }, { "property_id": "h002", "status": "no_data" } ] Three things to read in that output. rating_delta is computed only where both captures have parseable ratings — a property with a parsing failure in one week shows None, not a fake zero. new_reviews_this_period is deduplicated, so it counts genuinely new reviews rather than re-captured ones. h002 reporting no_data is the coverage metric doing its job: you know the property needs attention, rather than misreading absence as stability.
Comparison: Build This Layer vs. Alternatives
| Dimension | Repository-based workflow | Manual spreadsheet | Hosted review API |
|---|---|---|---|
| Best for | Developers and analysts who own the loop | One property, ad-hoc checks | Teams that want data without code |
| Setup model | Clone repo, configure env vars | None | Account and key setup |
| Output format | JSONL you define, dated captures | Spreadsheet cells | Provider's fixed schema |
| Trend history | Full: raw captures are immutable | Usually overwritten | Depends on provider retention |
| Maintenance burden | You own parser updates when pages change | Human effort each cycle | Provider-managed, verify freshness terms |
| Cost model | Your runtime; verify any repo/service pricing on its official page | Staff time | Verify current pricing on the provider's official pricing page |
The honest tradeoff: the repository approach gives you full control and a complete observation history, at the cost of owning the maintenance. A hosted provider removes parser work but you depend on its schema and retention choices — verify both on the provider's official documentation before committing.
Pre-Run Checklist
- [ ] Property list is bounded: one portfolio or geography per run, with stable
property_idvalues. - [ ] Captures write to dated, immutable folders — nothing ever overwrites a raw capture.
- [ ] Rating parser handles the formats the repository's
examples/output actually shows. - [ ] Deduplication key is stable across runs (property + review date + normalized text prefix).
- [ ] Quarantine count is logged every run, not silently dropped.
- [ ] Deltas compare captures of the same property only — never across properties.
- [ ] The repository README has been read, including its terms-of-service and compliance statements.
Use Cases
Multi-property reputation management. Agencies managing a portfolio watch weekly deltas and flag any property whose rating drops by more than a threshold — a task that is genuinely infeasible by hand past a dozen properties.
Competitive benchmarking for hotel groups. Track your properties alongside nearby competitors' public ratings to see whether a rating move is property-specific or market-wide.
Restaurant group quality ops. For a chain, per-location review deltas often surface operational issues (a new menu, a staffing change) days before they show up in internal metrics.
Content and travel research. Writers and analysts building destination guides can ground claims in structured public review data — the same data foundation described in the Chinese-language public-web-data guide on market research with public web data, which covers how public web data supports competitive and market analysis workflows.
Limits, Compliance, and Maintenance
Only public data. This workflow targets publicly displayed property pages and reviews. It does not cover private profiles, gated content, or anything behind authentication you do not legitimately control — and no workflow should.
Respect the platform. You are responsible for complying with Tripadvisor's terms of service, robots directives where applicable, and applicable law in your jurisdiction, including privacy rules around any personal data that appears in review text. The repository READMEs state the same. If a use case is commercial and continuous, consider whether the platform's official partner or API channels fit better.
Rate and pace. Capture on a schedule, not in a loop. Weekly is sufficient for review trends; more frequent captures mostly re-collect the same reviews.
Ratings lag reality. A rating is an average over accumulated history; the recent-review window in your delta is a better operational signal than the headline number.
Pages change. When a capture's parse rate drops, the page layout or field names changed. Fix the mapping in one place — the normalization layer — and your historical data stays usable because raw captures were never rewritten.
Verify before production. Field names, supported inputs, and repository behavior can change. Confirm them against the repository's current README and examples before relying on any run.
FAQ
Is there an official Tripadvisor API?
Tripadvisor offers official content and partner channels for eligible businesses. Availability and scope depend on the program, so verify current terms directly with Tripadvisor. The repository workflow in this article is an independent, public-data approach for teams without partner access.
What data fields does the workflow need?
At minimum: a rating, a review date, and review text per record, plus the property identifier and capture date you attach at ingestion. Match the repository's actual output field names in the normalization layer rather than assuming them.
How often should the workflow run?
Weekly is a sensible default for review monitoring. The deduplication and delta logic assume repeated captures, so a single run gives you a normalized dataset but no trend.
What happens when the page layout changes?
Parse failures show up as a rising quarantine count and falling coverage rate in your report — that is your signal to update the field mapping. Because raw captures are immutable, fixing the parser does not corrupt history.
Can this connect to Python schedules, dashboards, or an AI agent?
Yes. The output is plain JSONL and JSON, so it feeds cron or n8n schedules, a dashboard, or an agent that consumes a weekly delta summary as context.
Does this cover pricing data too?
Not in this article. For public hotel rate data, the same JSONL pattern applies to data-scrape/booking-scraper; keep the two datasets keyed by the same property IDs so they can be joined.
What should I verify before production use?
Three things: the repository's current README and examples (field names and flags can change), the platform's current terms of service for your use case, and your own jurisdiction's privacy obligations for handling review text.
Start Monitoring This Week
Clone data-scrape/tripadvisor-scraper, run it on a bounded list of five to ten properties, and wire the output into the normalization script above. After two captures you have your first real delta report; after a quarter you have a rating history you can actually query. Browse the rest of the data-scrape repositories for the other platforms in your monitoring scope, and read the repository READMEs before running anything.