If you serve only specific file types (eg. images) it can’t hurt to secure your Wordpress uploads directory a .htaccess file. Add to the file extensions as necessary.
Order Allow,Deny
Deny from all
<Files ~ "\.(jpeg|jpg|png|gif|pdf|gz|zip)$">
Allow from all
</Files>
