I found a temporary solution to this issue:
In the A3 uncompressed source, I changed the line that reads (should be line 1700):
- dialogHashKey = "&ui-state=dialog";
to:
- dialogHashKey = "#";
This appears to correct the back button history when navigating to a new page from a jqm dialog. The empty hash still shows up in the urlHistory, but back button functionality is now working as expected.
I'm still doing some testing, but I haven't noticed any side effects.
I'm attempting to find a more permanent solution that keeps the dialogHashKey, but removes the entry from the urlHistory stack.
If anybody has any suggestions, please post them.