To hire someone to scrape website data, write the brief first. A vague ask gets a vague quote, and a scraper that fails the day the site changes.
The brief: 8 lines
| line | example |
|---|---|
| 1. goal | what the data is for: price tracking, a list of competitors, training data |
| 2. sources | the sites, or the kind of site, and whether the pages are public |
| 3. items | what one row is: a product, a listing, a company, a post |
| 4. fields | the columns, with one example row filled in by hand |
| 5. scope | countries, languages, how many items |
| 6. refresh | once, daily, weekly |
| 7. format | CSV, JSON file or API |
| 8. quality | what counts as wrong: duplicates, empty fields, old rows |
Four ways to buy
| route | you get | you keep doing |
|---|---|---|
| freelancer | code, or a one-off file | running it, fixing it when the site changes |
| scraping tool or API | software | setting it up, the cleaning, the blocks |
| agency or managed service | the scraper run for you | the spec, the checking |
| data feed | the data on a schedule | reading the file |
The first two sell the scraper. The last two sell the result. Custom data feeds is the last kind.
Ask before you pay
- A free sample: 100 rows in your schema.
- Source address and fetch date on every row.
- How duplicates are merged.
- What happens when the site changes: who fixes it, how fast, at what cost.
- Who owns the code and the data.
- Where the data comes from: loaded from public pages, or bought, and from whom.
- Whether any personal data is in it.
Test the sample
- Open 10 random rows on the source site and compare by eye.
- Count empty fields per column.
- Look for the same item twice.
- Check the dates.
A note on rules
Robots.txt is guidance, not a lock: RFC 9309 says “these rules are not a form of access authorization” (RFC 9309). Terms of service, personal data and licences still matter. Some sources need a licence for commercial use: for Reddit, see GummySearch alternative.
What we do
You send the brief, even a rough one. We reply with what is possible, a sample and a price. We load public pages and public registers without logging in. We do not do lead lists. Details: data sourcing, AI and data teams.