AI Lead Generator

View Code
Tech Stack:
Rust
Mistralrs

Web-scraper/lead-generator

Feed it a list of websites, crawl the site for their personnel pages, and extract contact information with regex's and LLMs. Written in Rust because I wanted to learn something new.

Motivation

I needed more leads for Patense.ai. Last time I spent an afternoon saving pages and extracting contacts by tweaking regexes for each page. It wasn't great but it wasn't terrible. Most pages were similar so I just had to make minor tweaks. But I ran out of leads and needed more. So this time, instead of spending a few mindnumbing hours doing it manually, I spent a week making a AI do it.

The Solution

  • Load companies from json
  • Crawl company website
  • Regex all the 'a' tags and add them to a queue
  • Track visited pages with hashmap
  • Regex to any 'emailto:'
  • Recursively jump up parent nodes to collect context
  • Send context to an LLM and have it identify name and title
  • Regex the output (this was before I learned about structured generation)