Response title
This is preview!
Move this topic
Rerun tests in qUnit
Need more info
- Need more info
- Working on it
- Answered
in QUnit and Testing
•
12 years ago
I'm trying to make an extension to qunit that enables it to power selenium to run other browsers. For this to work, I need qunit to be able to rerun the same tests over again. I was able to get this to work by using my own version of qunit.js with a custom synchronize function that stores all callbacks. But I'd like to avoid that.
Is there a way to do this? Would you consider making synchronize a public method in future releases? Thanks,
Justin
1
Replies(7)
Re: Rerun tests in qUnit
12 years ago
I want to see an actual usecase to consider any API changes, in order to be able to properly test and document that API based on that usecase. You mentioned pinkook/funcunit on Twitter, but http://github.com/pinkook/funcunit is 404.
Having said that, the QUnit API is not frozen as we still haven't released a 1.0, therefore we're open to adding or exposing more methods.
Also I'm personally very interested in integration of QUnit and Selenium - show some code, dude!
Having said that, the QUnit API is not frozen as we still haven't released a 1.0, therefore we're open to adding or exposing more methods.
Also I'm personally very interested in integration of QUnit and Selenium - show some code, dude!
Leave a comment on joern.zaefferer's reply
Re: Rerun tests in qUnit
12 years ago
Darn it. It's http://github.com/pinhook/funcunit (not pinkook).
By tomorrow, we should have a release that you can play with. We had one yesterday, but I decided that it would be best to not have a custom qunit.
Here's how it's shaping up to work / look.
We'd like to provide 3 things:
- A qunit.envjs.js script that enables qunit to report and run in envjs.
- A envjs shell script that loads rhino, selenium, envjs, a settings and opens a page.
- A funcunit.js script that enables qunit to drive selenium.
The first 2 are nothing new and have been done by other people. This would let people take an existing qunit test and do something like:
./envjs mytest.html
The settings file is so that EnvJS can make Ajax requests as if it was not running from the filesystem.
But the funcunit.js part is very special. It allows your tests to open another page (via the browser or via selenium) and perform the clicks and keypresses a user would do. It's got a sweet syntax. Here's what an auto-suggest looks like:
- S('#auto_suggest').click().type("qUnit")
- S('.result').exists().size(function(size){
- equal(size, 5,"there are 5 results")
- })
This clicks and types in an auto suggest button, waits for a result to exists, and then checks the number of results.
You might be asking yourself, why the callback for size? The reason is that functional tests do lots of asynchronous behavior and very little checking. Our previous version was practically unusable b/c we were always having to callback multiple functions. This new syntax allows you to call $.wait and still keep your test code very linear (no nested functions).
To illustrate this, here's an example of an old test: http://f-it.googlecode.com/svn/trunk/public/test/functional/entries_controller_test.js
This was for an ftp app. I'd often have to create a file, do something to it, and test the result. This is why you'll see the createFile function. It's a helper. Notice all the timeoutCallbacks that move to another function and have to pass the original callback. Gross! This could be done with something like:
- Helpers = {
//takes a filename and returns a selector - convert : function(fileName){ .... },
- createFile : function(fileName){
- S(this.convert(fileName.replace(/\/[^\/]+$/,"") )).leftClick()
- S("#contextmenu .new_file").visible().click()
- S("#input.newEntry").exists().type(fileName+"\n")
- S(this.convert(fileName)).exists()
- },
- ....
And in my qunit test I could make a few folders as easily as:
- test("create a file and move it to another folder", function(){
- Helpers.createFile("foo/bar.txt")
- var selector = Helpers.convert("foo/bar.txt")
- S( selector ).dragTo( Helpers.convert("adifferentFolder") )
- S(Helpers.convert("adifferentFolder/bar.txt")).exists(function(){
ok(true,"Drag a file to another works")
}) - Helpers.removeFile("adifferentFolder/bar.txt")
- })
Testing solved?
I think I solved a bunch of problems by combining all approaches to testing and adding an 'asynchronous' aware syntax.
qUnit - Can't be automated. Can't do functional tests.
EnvJS - Not the 'real' browser.
Selenium - Very difficult to write and debug tests. With FuncUnit, all these tests run in the browser. You can debug them w/ firebug. Also, the API is nicer in my opinion.
How it works
funcunit.js has 2 modes - rhino and browser. In the browser, funcunit basically wraps:
Synthetic uses mostly feature detection to enable clicks and kepresses on dom objects (including their side effects).
If the page is running in envjs/rhino, it issues these commands to selenium. We have a custom version of selenium that uses synthetic (which fixes numerous bugs).
What we need
FuncUnit (in rhino) needs to run the same tests multiple times, one for each browser. I need a qUnit restart. I have that in http://github.com/pinhook/funcunit/blob/master/qunit/qunit.js#L330 (excuse the tabs/spaces formatting). I hacked this in by adding another parameter to synchronize (true if it should cache the object for later) http://github.com/pinhook/funcunit/blob/master/qunit/qunit.js#L580
Leave a comment on justinbmeyer's reply
Re: Rerun tests in qUnit
12 years ago
I forgot to mention. The ability to forget about callbacks enables cucumber style testing which I want ooooh so badly. I've got a little experimental code for it. It lets you map statements like:
- Given I have created a new file foo/bar.txt
- When I move it to someFolder
- Then someFolder/bar.txt exists
To FuncUnit helpers. So you can easily document your functional tests with human language. It's the only hope TDD has in JavaScript.
Leave a comment on justinbmeyer's reply
Re: Rerun tests in qUnit
12 years ago
Sorry for not getting back to this. At one point I had a response almost ready, then the forum software managed to waste it all.
Anyway, there just was another thread asking for an option to rerun tests. I'm still hesitant to add that, but I'm considering it. I'll try to discuss the change with John Resig, to come up with a generic solution that is useful for other cases, too.
Other then that, QUnit is supposed to run within CommonJS, so that you wouldn't need env.js. Looking at the usage of document and window within QUnit, that probably won't work currently, but could be interesting for your server-side based work.
Anyway, there just was another thread asking for an option to rerun tests. I'm still hesitant to add that, but I'm considering it. I'll try to discuss the change with John Resig, to come up with a generic solution that is useful for other cases, too.
Other then that, QUnit is supposed to run within CommonJS, so that you wouldn't need env.js. Looking at the usage of document and window within QUnit, that probably won't work currently, but could be interesting for your server-side based work.
Leave a comment on joern.zaefferer's reply
Re: Rerun tests in qUnit
12 years ago
I don't need to be able to rerun tests, just enough exposed so I might be able to.
Why are you hesitant to add it if people need it an it adds no performance cost and can be done with very little code?
I am not doing any server-side based work. I must have not been very clear.
Everything I am doing is purely for testing client side code. In my opinion, qUnit, although great and very easy to work with, is impractical for large teams because it can't be automated.
By using it with Env, you can automate qUnit tests written for Client-Side JavaScript. The results aren't as perfect as running in an actual browser (which is what Selenium can do), but it provides an important rapid sanity check before checkin.
The 'Restart' ability is needed for Selenium tests which need to run the same tests in every browser.
Leave a comment on justinbmeyer's reply
Re: Rerun tests in qUnit
12 years ago
Being hesistant to add features is a general approahc for API design. If I add anything people ask for... And I want to understand why something is useful, so that I can properly document it.
If you run QUnit to control Selenium, how about calling QUnit.init(), then loading the test files again?
If you run QUnit to control Selenium, how about calling QUnit.init(), then loading the test files again?
Leave a comment on joern.zaefferer's reply
Re: Rerun tests in qUnit
12 years ago
I'd prefer to make using this functionality as easy as possible. It should work with people just adding a single script to their page - that people didn't also have to identify which tests/files need to be reloaded.
Leave a comment on justinbmeyer's reply
Change topic type
Link this topic
Provide the permalink of a topic that is related to this topic
Reply to justinbmeyer's question