I have dynamically created pages, which can have any name.
www.example.com/pages/
www.example.com/pages/dynamicpage1/
www.example.com/pages/dynamicpage2/
I want the first url to pages to be crawled, but not any of the dynamic pages
User-agent: *
Disallow: /pages/*
The above will block all three pages listed. How do I just block all the dynamic pages, but not the first url.
I figured it out..
User-agent: *
Disallow: /pages/*/
Happens all the time, solve the problem seconds after you post it.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With