Davriellelouna@lemmy.world to Technology@lemmy.worldEnglish · edit-22 months agoThe AI company Perplexity is complaining their bots can't bypass Cloudflare's firewallwww.searchenginejournal.comexternal-linkmessage-square248linkfedilinkarrow-up1864arrow-down16
arrow-up1858arrow-down1external-linkThe AI company Perplexity is complaining their bots can't bypass Cloudflare's firewallwww.searchenginejournal.comDavriellelouna@lemmy.world to Technology@lemmy.worldEnglish · edit-22 months agomessage-square248linkfedilink
minus-squarethreeganzi@sh.itjust.workslinkfedilinkEnglisharrow-up2·2 months agoDoes it not need to be scraped to be indexed, assuming it’s semi-typical RAG stuff?
minus-squareElectricdlinkfedilinkEnglisharrow-up1·2 months agoI assume their script does some search engine stuff like query google or bing and then “scrap” the links they go on Some selenium stuff
Does it not need to be scraped to be indexed, assuming it’s semi-typical RAG stuff?
I assume their script does some search engine stuff like query google or bing and then “scrap” the links they go on
Some selenium stuff