How can I make my link not have this ugly border?
Links normally don’t have a border, as such, but they do have a default underline. Post a code sample if you can, or a link, and we’ll show you what to do. 
Try this in your stylesheet.css
a { outline: 0; border:0 }
If youi hide the outline then those without a mouse will not know which link has the focus.
Indeed. If it’s focus outline the OP is asking about, the outline should be replaced by something more pleasing on focus. E.g.
a:focus {outline: none; color: white; background: black;}
OK, not very pretty styling, but hopefully gets the idea across.
If you are talking about images in IE that get a blue border when they are surrounded by an anchor then do this.
a img{border:none}