unless I'm mistaken, that doesn't make much sense, or does it work? Ids should be unique in a document, so you should only have one #slideshow-caption. Now, if instead you marked them as class="slideshow-caption" then yes, you could do like this:
$(".slideshow-caption").each(function() {
$(this).css("display", $(this).attr('alt').length == 0 ? "none" : "block")
})
this would hide all elements of class 'slideshow-caption' with empty 'alt', and display those with something in the 'alt', but I don't think that's what you mean to do.
If you want to display the caption on the image if the image has a caption, you would have to create a span or div or something with the text, and show this, but I'd need to see the code to see how it'd work.