Search jQuery

jQuery Forum

How to fix slowdown problems when parsing large xml files with ajax?

in Using jQuery • 11 years ago

Hi,
I'm trying to load info from an xml file into a table. The xml structure is roughly as follows (the actual file is a bit more complex but simplified here for illustrative purposes):

<xml>
<record>
<id></id>
<title></title>
<year></year>
<author></author>
<journal></journal>
<abstract></abstract>
</record>
</xml>

It contains about 300 <record> entries, with each tag containing text content (the <abstract> tags can each contain up to a couple paragraphs of text). All up quite a bit of data.

I'm trying to:
a) parse each <record> into it's own row in a table, currently as follows:

$.ajax({
type: "GET",
url: "ref.xml",
dataType: "xml",
success: function(xml) {
$(xml).find('record').each(function(){
var id = $(this).find('id').text();
var year = $(this).find('year').text();
var title = $(this).find('title').text();
var auth = $(this).find('author').text();
var journal = $(this).find('journal').text();
$('<tr class="items visible searchable" id="link_'+id+'"></tr>').html('<td class="year">'+year+'</div>').appendTo('table tbody')
$('<td class="author">'+auth+'</td>').appendTo('#link_'+id);
$('<td class="title">'+title+'</td>').appendTo('#link_'+id);
$('<td class="journal">'+journal+'</td>').appendTo('#link_'+id);
}); //end each function
} //end success function
}); //end ajax call

and b) and filter table rows on-the-fly through a text input as follows:

//Send search term to 'filterAll' function
$('#filter').keyup(function(event){
if (event.keyCode == 27 || $(this).val() == '') {
//if esc is pressed we want to clear the value of search box
$(this).val('').blur();
//Make each row visible again
$('tbody tr').removeClass('visible odd even').show().addClass('visible').css({backgroundColor: ""});
$('.visible:even').addClass('odd');
$('.visible:odd').addClass('even');
} else {
var searchTerm = $(this).val();
if (t) clearTimeout(t);
t = setTimeout(function(){filterAll(searchTerm)}, 200);
}
}); //end keyup

function filterAll(phrase){
var words = phrase.toLowerCase().split(" ");
var table = $('#itemList tbody');
var rows = table.find('tr.searchable');
var ele;
for (var r = 0; r < table.find('tr').length; r++){
ele = rows.eq(r).html().replace(/<[^>]+>/g,"");
for (var i = 0; i < words.length; i++) {
if (ele.toLowerCase().indexOf(words[i])>=0){
rows.eq(r).show().removeClass('visible').addClass('visible');
} else {
rows.eq(r).hide().removeClass('visible');
break;
}
};
//After filtering, restore zebra stripes to visible rows
rows.removeClass('even odd').css({backgroundColor: ""});
$('.visible:even').addClass('odd');
$('.visible:odd').addClass('even');
};
}; // end filterAll function

Both a) and b) are essentially working as I want them to, but my problem is the browser slows down to a crawl on page load (usually throwing the 'a script is causing this page to load slowly etc' error), and it takes forever when filtering data. I'm not even touching the <abstract> dataset yet either.

I'm sure the efficiency of my code could be improved a hundred-fold, but I'm not sure whether that would make enough of a difference to make the page useable, particularly as I want to include <abstract> data later too. I'm wondering if I should be approaching the issue from a whole different angle (i.e. converting the xml data into a database, or using JSON?). This is my first foray into ajax, and I've been given the data in XML format. If anyone knows how to approach parsing and filtering through large amounts of XML data quickly, any assistance would be much appreciated.

PS - I've considered paginating the table to speed up the initial page load, but have been avoiding this as I still want to be able to quickly filter all rows on the fly and paginating wouldn't fix that particular issue.

Cheers

Replies(4)

Yomodo Mo..

Re: How to fix slowdown problems when parsing large xml files with ajax?

11 years ago

Maybe you should try experimenting a bit with .xsl style sheets.

It will allow you to do things like:

...
<td><xsl:value-of select="//record/author"/></td>
<td><xsl:value-of select="//record/journal"/></td>
...

From a personal point of view I always try to prevent doing things like:

var id = $(this).find('id').text();
etc...

Although it's a common jQuery practice I think, I still find it doesn't make sense since all the data is already in the .xml file, why would one also want to parse it in memory again as JS vars.

Using .xsl you can directly access the data from the .xml, ofcourse the .xsl solution might not always apply in all situations; sometimes you need to read an .xml file, add/change nodes and then apply a stylesheet.
Most of the times jQuery gives me a hard time doing this, since I rarely have a real XML-document object to work with.

jaminday

Re: How to fix slowdown problems when parsing large xml files with ajax?

11 years ago

@Yomodo

Thanks for the response. I did toy with the idea of using an xsl stylesheet, but I don't have write access to the xml file to add a .xsl reference in the header (there could be another way to reference a .xsl stylesheet but I don't know it).

I ended up moving all the xml parsing (initial page load and filtering) to the server side using php and ajax instead. It's much faster and doesn't hang the browser, so my problems (at least for now) are solved.

Cheers

jules

Re: Re: How to fix slowdown problems when parsing large xml files with ajax?

11 years ago

I've done this in the past with only 36 to 60 rows. The data source is in xml, translated to json and generate table using template. Not sure if this approach is fast enough for your situation. Here is the snippet.

<script src="../Scripts/jquery-1.4.4.js" type="text/javascript"></script>
<script src="../Scripts/jquery.tmpl.js" type="text/javascript"></script>
<script src="../Scripts/jquery.xml2json.js" type="text/javascript"></script>
var opt = {
url: "../server/GetBilling.ashx",
success: function(data) {
var $data = $.xml2json(data);
var $bill = $data.parametersReturn.billSchedule.bill;
var template = "<tr><td>${number}</td><td>${date}</td><td>${Billcol1}</td><td>${Billcol2}</td><td>${Billcol4}</td><td>${Billcol8}</td></tr>";
$.tmpl(template, $bill).appendTo("#tbl");
$("#tbl tbody tr").each(function(i) {
$(this).addClass(i % 2 == 0 ? "row1" : "row2");
});
$("#tbl").show();
}
};
$.ajax(opt);

Yomodo Mo..

Re: How to fix slowdown problems when parsing large xml files with ajax?

11 years ago

@Jules,
Yeah, xml is not always the best options, JSON has a lot of advantages.
But still, in your code additional steps are taken to finally get to the needed result.

@jaminday
Yeah, let the Server serve! at least as much as possible.

Still, perhaps this is the other way you're referring to.

There is really no need to alter the original xml files, just:

      - Load the XML document, I use $.get() for that.

            - In the success callback, I load the .xsl file, also with $.get().

                  - In the success callback I perform a transform, actually the browser does the hard part:

                        $("#el").html( <transform_method> )

The .xml and .xsl are now nicely be merged, no additional parsing needed!
Through the use of XPath, xml data is reachable from within the xsl file.

I mostly use jQuery for .HTA, thus IE only; the method used for IE is .transFormNode()
For other browsers it would be someting with: XSLTProcessor

Top Reply

Response title

Attachments

jQuery Forum

How to fix slowdown problems when parsing large xml files with ajax?

Replies(4)

Re: How to fix slowdown problems when parsing large xml files with ajax?

Re: How to fix slowdown problems when parsing large xml files with ajax?

Re: Re: How to fix slowdown problems when parsing large xml files with ajax?

Re: How to fix slowdown problems when parsing large xml files with ajax?

Statistics

Tags

Actions