The internet architecture provider will also let some publishers make known AI scrapers pay to crawl their sites.