Surprise - ISAS uses Linux

A few days ago I took the International Survey of Adult Skills after my home address was randomly selected to participate.  They apparently want to determine relative levels of education and computer-use skills across households in 25 countries by using a random statistical sample.

The survey taker had a government issued laptop to administer the test.  It was a large, no-brand name laptop running Windows Vista Basic.  After booting into Windows, they run some app that tracks the with the test takers' home addresses, then launches the test.  To my surprise, the test itself is within a VMWare Player virtual machine.  It booted what looked like some stripped down variant of Debian, then directly into X and the test interface.

The test itself was fairly well designed... obviously some real thought was put into it.  It tests your ability to use a basic e-mail client, spreadsheet and understand search engine results.  Some tests were reading data from a spreadsheet or chart and calculating answers with a calculator.  Some were reading e-mail and inputting values into a spreadsheet.

The test did have some problems.

• Software bugs ... often clicking would fail to work.  You would need to figure out workarounds, often involving clicking away then re-entering a part of the UI, to make it work as intended.  These bugs often made it frustrating to participate, and probably made me take 25% longer to complete the test.  I can see this screwing up their end results as some test takers may just give up due to bugs rather than demonstrate skills that they know.
• I noticed several instances where the on-screen instructions had typos, ambiguously written instructions, or ambiguous text to read and interpret in order to understand the goal of a particular test.  This made me wonder if the ambiguity combined with software bugs were INCLUDED INTENTIONALLY as a means to test real-world problem solving skills.  If so, this might be a clever design.
• But then I noticed another problem that surely cannot be intended.  The survey taker said it is supposed to be administered identically and in English in the 25 countries.  I saw several things in the test that were obviously American-centric, and I highly doubt they will be able to successfully test in 25 countries in English.
• The survey administrator complained that the laptops were EXTREMELY SLOW and often the VMWare Player would get stuck during shutdown, preventing her from collecting the results and moving on.

It was surprising that this government study used Linux for the test interface.  It seems clear that they used Linux as a method of deploying many identical copies of the survey interface across the world at the lowest cost possible.  It is strange that they would use Linux within Windows, when Windows seemed to be the cause of the extreme slowness.  Between Windows bootup and lots of time wasted fighting the test UI, it seems we wasted ~45 minutes.  That kind of time wasted can really add up to big money when multiplied across many thousands of test takers.  As a whole it seems well thought out, but they could have made the overall program far more effective had they fixed a few of these annoying and time consuming bugs.