crawler

Example Sentences

Examples are provided to illustrate real-world usage of words in context. Any opinions expressed do not reflect the views of Dictionary.com.

“Walmart is clearly ahead of the curve here, while others have been slow to adapt or even made efforts to block AI web crawlers,” he wrote.

From Barron's

Cloudflare's tech targets AI firm bots - also known as crawlers - programmes that explore the web, indexing and collecting data as they go.

From BBC

It instructs bots and web crawlers to not access certain pages and material, where present.

From BBC

When the crew of the colonization mission discover that their new home, an ice planet called Niflheim, is inhabited by armadillo-like crawler creatures, they wonder what to do about this unanticipated roadblock in their efforts.

From Salon

When Adam Cooper came across an old junkyard filled with RVs and rock crawlers in South Central, he immediately saw its potential.

From Los Angeles Times