Draggable div

I have a div in which is made draggable. There is a child inside that div that has the possibility of having both a vertical and/or horizontal scroll bar. When the scroll bars are present, you cannot actually scroll them as when you click on them they activate the drag. Is there a way to tell it to ignore scroll bars?