the empty quarter

Changing the port number of SSH daemon is a quick way of reducing the number of SSH brute force attacks your server might face (check the file /var/log/auth.log to see if there are many failed SSH login attempts). Just to be on the safe side, create a backup copy of the SSH daemon config file. […]

This article on The Register has a link to a very good page on CentOS website with tips on securing OpenSSH. The tips are applicable for all distros, not just CentOS, and are definitely worth going through and implementing if you have SSH enabled on your server. Good luck and stay secure.