What $.Widget.prototype.destroy.apply( this, arguments ) destroys depends on the ui widget.
If you look at the destroy function of ui.tabs you'll see it unbinds events associated with the tabs widget, it removes classes that were set by ui-tabs and it also removes data associated with ui.tabs. It even restores some links to it's original state and clears cookie's
In other words, everything a widget adds and isn't needed anymore should be removed or restored. Therefore it's important to call $.Widget.prototype.destroy.apply( this, arguments ) if you derive a plug-in from an existing widget.
In your own destroy function remove therefore everything you've added, that can be removed. It's also good practice to namespace your events and data. Instead of lets say simply bind 'click' , bind it with 'click.myWidget'. Same goes for data. That way you can remove your events and data them without interfering with other widgets,
In your specific application, what you need to remove it depends if you load pages with with AJAX, or load each page completely from the server.
If you use AJAX, consider that some widgets can be reused, In ui-tabs for example can you remove a tab, and add new ones. Destroying and rebuilding the entrie widget is most often not the the most efficient way to go about.
Also some data will be retained over several pages if you use AJAX, you want to keep some data with the data() function. Only if you don't need the data anymore remove it.