I usually use it if I don’t want any of my pages to be crawled by the search engine. Many I know, if there as other use of robots file too.