remove previous element only if it is of type <hr>?

remove previous element only if it is of type <hr>?

Starting with

  1. <hr>
  2. <div class="aaa">...</div>
  3. ...
  4. <p>
  5. <div class="aaa">...</div>

I want to remove the previous element of  <div class="aaa"> elements but ONLY if it is a <hr> element

$("div.aaa").prev().remove();

removes them all.

How can specify only <hr>-removals?