Sortable with Toggle div not working together?

I have divs that have a header with an image that toggles the content div under it. I also have the parent div set as sortable so all the child divs are sortable. Sort works great but now I can toggle the divs closed.

Please help.