Web Scraping

For sources behind JavaScript SPAs or competitor product pages.

Firecrawl Scraping

uv sync --extra scrape
# Add to .env: FIRECRAWL_API_KEY=fc-your-key

bioingest scrape competitor quanterix    # crawl product pages
bioingest scrape competitor somalogic    # crawl SomaLogic
bioingest scrape resolve corum           # resolve SPA download URLs

Competitor Assay Specs

bioingest scrape competitors msd         # MSD assay list PDF → TSV
bioingest scrape competitors quanterix   # Quanterix kit pages → TSV
bioingest scrape competitors alamar      # Alamar NULISAseq panels

Local Browser Scraping

For authenticated sites (uses your browser cookies):

bioingest scrape local https://olink.rmtplus.se --source-id olink_rmt