I have this line:

Crawl-delay: 50

in the robots.txt file at the root of my website.

But it seems that it is not working at all. I get 400 visits from the Yandex user-agent in a few minutes.

I don't want to Disable the User Agent but to make it to respect the Delay value (50 in my situation)

How to make "Crawl-delay" works for all "User-Agent" in robots.txt? I also read somewhere that this can be done in .htaccess, is this correct?

I have this line:

Crawl-delay: 50

in the robots.txt file at the root of my website.

But it seems that it is not working at all. I get 400 visits from the Yandex user agent in a few minutes.

I don't want to disable the user agent but to make it to respect the delay value (50 in my situation).

How to make "crawl-delay" works for all "user-agent" in robots.txt? I also read somewhere that this can be done in .htaccess, is this correct?

I have this line:

Crawl-delay: 50

in the robots.txt file at the root of my website.

But it seems that it is not working at all. I get 400 visits from the Yandex user-agent in a few minutes.

How to make "Crawl-delay" works for all "User-Agent" in robots.txt? I also read somewhere that this can be done in .htaccess, is this correct?

I have this line:

Crawl-delay: 50

in the robots.txt file at the root of my website.

But it seems that it is not working at all. I get 400 visits from the Yandex user-agent in a few minutes.

I don't want to Disable the User Agent but to make it to respect the Delay value (50 in my situation)

How to make "Crawl-delay" works for all "User-Agent" in robots.txt? I also read somewhere that this can be done in .htaccess, is this correct?