Cinema WatchThe social network for film
HomeAboutBlogStudioAPI
HomeBlogAboutStudioAPI
Cinema WatchThe social network for film

Track what you watch, share what you love, and find your next favorite.

Explore

  • Movies
  • TV shows
  • Actors
  • Swipe
  • Collections

Company

  • About
  • Blog
  • What's new
  • API
  • Help center
  • Contact

Legal

  • Privacy
  • Terms
  • Cookies
  • Spider

© 2023 - 2026 Cinema Watch. All rights reserved.

Behind the catalog

Meet the spider that helps build our catalog.

Cinema Watch has its own indexing spider. It crawls the web for movies and TV shows, finds titles we're missing, and fills in the details: release dates, runtimes, posters, genres and trailers. Then a person checks every change before it reaches the catalog.

How it worksFor site owners
CatalogThe world wide web

We crawl politely

The spider is a guest on every site it visits. It takes its time on purpose, and we have no intention of ever slowing down or bringing down anyone's website.

  • Requests to each site are spaced out, never fired in bursts.
  • If a site asks us to slow down, we pause and keep the slower pace for good.
  • If a site blocks us, the spider backs off and stops.
  • Site owners can opt out with robots.txt or by contacting us.

How it works

Six steps from a page on the web to a title you can log, rate and add to a list.

  1. 01

    Discover

    The spider crawls the world wide web, looking for titles we don't have yet and gaps in the ones we do, like a missing poster or runtime.

  2. 02

    Fetch, politely

    Every site gets its own speed limit, shared by all our crawlers. If a site asks us to slow down, the spider waits and keeps the slower pace from then on.

  3. 03

    Match

    Each find is matched to a title by a standard film identifier where there is one, otherwise by title and year. Editions, "4K" labels and punctuation are ignored, and every match gets a confidence score.

  4. 04

    Propose

    The spider never writes to the catalog. It turns what it found into proposed changes, each showing the current value, the new one, and the page it came from.

  5. 05

    Review

    A person on our team checks every proposal and approves or rejects it. Low-confidence matches and anything that would replace an existing value are flagged.

  6. 06

    Publish

    Approved changes go live straight away. Images are resized and optimised first, and we record where each value came from.

What it collects

Facts about titles, never anything about people who use the sites it visits.

  • New movies & shows
  • Release dates by country
  • Runtimes
  • Posters & artwork
  • Genres
  • Trailers
AI-assisted

Where AI helps

Raw data from the web is messy. We may use AI prompts to tidy up and improve what the spider brings back, but only using what it actually found, and always with a person checking the result.

  • Rewrite

    Turn scraped synopses into short, original descriptions in our own words, so we don't copy anyone's text.

  • Clean up

    Normalise messy values: dates in odd formats, runtimes like "2h 28m", stray symbols and leftover markup.

  • Match labels

    Map the many ways sites describe genres and ratings onto our own list, and flag anything that doesn't fit.

  • Sanity-check

    Spot likely mistakes before review, like a release year that doesn't fit the description or two titles that look like the same film.

AI suggestions go through the same review as everything else: nothing an AI writes reaches the catalog until a person approves it, and it never replaces details our members have set.

The rules it follows

  • Nothing goes live on its own

    Every change waits for a person to approve it. A broken page or a bad match can't reach the catalog.

  • People's edits win

    If a member or moderator set a value, the spider leaves it alone. Replacing it takes a deliberate choice in review.

  • Every value has a source

    We keep a record of who or what set each field and when, with a link back to the page it came from.

  • Polite by default

    A named User-Agent, per-site rate limits, backing off when asked, and robots.txt checks before fetching pages.

For site owners

Our spider identifies itself with cinema-watch-crawler in its User-Agent. To keep it off your site, or part of it, add a rule for that name to your robots.txt. It's picked up within an hour.

Crawling too fast, or something else we should know? Get in touch and we'll adjust or stop.

robots.txt
# Block the spider everywhere
User-agent: cinema-watch-crawler
Disallow: /

# …or just one section
User-agent: cinema-watch-crawler
Disallow: /private/

Spot something the spider missed?

Suggest a title, a release date or a better poster. Moderators check members' fixes too, and the spider won't overwrite them.

Start contributing