Search jQuery

jQuery Forum

How to realize google crawlable links without executing the unnecessary 301 request?

in jQuery Mobile • 12 years ago

Since Google crawls hashed links (by adding an exclamation mark after the hash: #!, for details go to http://code.google.com/intl/en/web/ajaxcrawling/docs/specification.html) I tryed to figure out how it could be possible to realize this with the jquery-mobile API and especially without the unnecessary 301 AJAX request I am faced with. I couldn't find a clean solution. For me it looks like this isn't considered by the current implementation yet.

Here the details:

It's possible to add an exclamation mark at the beginning of a link, e.g:

<a href="!a/b/c.html>link</a>

After clicking this link, followoing URL will be displayed withinin the address bar, which is exactly what I want:
http://mywebsite.com/#!a/b/c.html

BUT, the AJAX-call which is now executed tries to resolve following (wrong) URL, which returns a 301:

http://mywebsite.com/!a/b/c.html

Afterwards another request (the correct one) is automaticall executed:

http://mywebsite.com/a/b/c.html

To solve this for the moment, I hacked the source code by removing the questionmark out of the fileUrl. So now the correct request ist executed immediately. Sure it's not nice to modify the source but it was the only way I was able to avoid the unnecessary request:

$.ajax({
url: fileUrl.replace("!",""),
type: type,
data: data,
dataType: "html",
success: function( html ) {
...
}
});

Back again to my question: is there an elegant way to realize google crawlable hashed urls without executing the unnecessary request by using the default API of jquery-mobile or at least without hacking the source code?

Replies(7)

edelabar

Re: How to realize google crawlable links without executing the unnecessary 301 request?

12 years ago

You may be over-thinking the problem...

Disable JavaScript in your browser and load up the jQuery Mobile docs page. This is how Google sees a jQuery Mobile site. It's perfectly crawlable and all content is reachable so the #! is not necessary.

It's very important to remember that Google does not crawl with JavaScript, so all JavaScript functionality is ignored, including jQuery Mobile's hash management.

In your solution, by prefixing your links with "!" you actually force Google's spider into a 301 redirect that it wouldn't have needed because it could have just followed the link without the "!".

-Eric

aykutarik

Re: Re: How to realize google crawlable links without executing the unnecessary 301 request?

12 years ago

Hey Eric,

I understand what you're saying. From that point you are absolutely right. The exclamation mark is obviously a big fail in the beginning of a href value. Thank you for that hint :)

But one important aspect is missing:

I want that google is also able to index 'my' hashed-links at external websites like bulletin boards, news groups or other social sites. Therefore I need a way to configure not only a hash is added to the url bar but also an exclamation mark after a link was clicked. I couldn't find a solution to achieve this with jquery mobile 'built in' options.

-Aykut

cleanelec..

Re: Re: How to realize google crawlable links without executing the unnecessary 301 request?

11 years ago

Thanks edelabar for bringing me back down to earth. Like Aykut, I was over thinking the problem. You're right, a quick use of the "Fetch as Google" tool, shows what you're saying is true. All the links on the page, although via the browser show the hash, Google reads them as regular URLs. This combined with best practices like sitemaps, should be sufficient.

Thanks!!

edelabar

Re: How to realize google crawlable links without executing the unnecessary 301 request?

12 years ago

Ok, you got me on that one, copy-and-paste will not work. :)

There is a branch being worked on that uses history.pushState instead of the hash, which works in a lot of mobile devices and changes the entire URL instead of just the hash:

https://github.com/jquery/jquery-mobile/tree/pushstate

It may be worth a look if your launch timeline is long enough.

I would imagine, due to the necessity of a server-side component, we won't see #! support in the jQuery Mobile core, maybe as a plugin in the future though...

-Eric

aykutarik

Re: Re: How to realize google crawlable links without executing the unnecessary 301 request?

12 years ago

I see. The implementation of history.pushState into the framework would be too nice. For now I'll do it with a sourcecode hack. Thank you for your contribution Eric.

- Aykut

aykutarik

Re: How to realize google crawlable links without executing the unnecessary 301 request?

12 years ago

This is the way I'm going to implement until a later framwork supports either the hashed or the history.pushState version. Maybe this is helpfull for someone who has same requirements.

// set location hash to path
set: function( path ){
// original code line: location.hash = path;
// modification:
location.hash = path.replace(/^\/*(.*)/,'!$1');
}

This modification was done in jquery.mobile-1.0a4.1.js

Before updating the 'hash value' within the URL bar, after a link was clicked, the string after the hash will be prefixed with an exclamation mark. (And also the leading slash will be removed, if there is one). Thats all.

-Aykut

yuripave

Re: How to realize google crawlable links without executing the unnecessary 301 request?

11 years ago

Instead of hacking the jquery mobile js source code, I think the solution below is better by overriding its function on mobileinit, tested on 1.0b2.

To make this google crawlable you have to implement #! to _escaped_fragment_ on server side specified on google ajaxcrawling documentation as well:

$(document).bind("mobileinit", function(){
$.extend($.mobile.path, {
set: function(path) {
location.hash = '!'+path;
},
stripHash: function(url) {
return url.replace(/^#!/,'');
},
makeUrlAbsolute_ori: $.mobile.path.makeUrlAbsolute,
makeUrlAbsolute: function(relUrl, absUrl) {
return $.mobile.path.makeUrlAbsolute_ori(relUrl.replace("/#!", ""), absUrl);
}
});
});

Now you can have anchor like this and jquery mobile will make a correct ajax request while having a hash-bang link:

<a href="/#!/a/b/c.htm">link</a>

So far it works in my case but I have not fully tested it so it may have introduced bugs as well as it may break on future jquery mobile version.

Top Reply

Response title

Attachments

jQuery Forum

How to realize google crawlable links without executing the unnecessary 301 request?

Replies(7)

Re: How to realize google crawlable links without executing the unnecessary 301 request?

Re: Re: How to realize google crawlable links without executing the unnecessary 301 request?

Re: Re: How to realize google crawlable links without executing the unnecessary 301 request?

Re: How to realize google crawlable links without executing the unnecessary 301 request?

Re: Re: How to realize google crawlable links without executing the unnecessary 301 request?

Re: How to realize google crawlable links without executing the unnecessary 301 request?

Re: How to realize google crawlable links without executing the unnecessary 301 request?

Statistics

Tags

Actions