Metro Boulot Dodo

Parallels screws up Windows

2007-08-13T12:29:00.000+01:00

Since Parallels 3 has come out there have been a lot of complaints that it can screw up your Windows XP installation, resulting in an error like:

/system32/ntoskrnl.exe file is missing

Parallels has finally posted a fix:
http://kb.parallels.com/entry/30/516/

However, it doesn't tell you how to get your Parallels VM to boot from the Windows CD in the first place. The trick is to start the VM, then go to edit/Virtual Machine, and in there you'll find an option to change the Boot order to CD first.

In my case, this only partially fixed the problem, perhaps because I'd previously botched things by following other advice on the internet. The solution for me was to completely deinstall parallels (using the _installer_, weirdly), then reinstalling it. Be warned that this might then need you to reactivate your windows installation.

J1 visas at the London US embassy

2007-08-13T10:22:00.000+01:00

The US embassy website at http://london.usembassy.gov makes applying for a J1 visa appear to be an intimidating and unpleasant process. It's big on the "don'ts" - Don't bring a mobile phone, don't expect to be allowed inside with a bag, don't arrive too early or too late. Then there's the forms: half a dead tree's worth, requiring you to list your experience in nuclear, chemical and biological weapons, every foreign country you've been to in the last ten years (the form was designed by an American...right? They only give you a box big enough to write "Mexico and Canada".)
With a start like that, it's hardly surprising that the actual experience is, in fact, a whole lot better, at least if you're white, middle class and British.

It's still expensive though: for my family of 3 the bill was:
$100 X 3 visa fees
$100 SEVIS fee
£15 mail courier
£170 transport to, from and around london
£15 phone call to make the appointments
------
£400

So, it's worth getting the paperwork right first time.

Ingredients for a J1 (and associated J2) visas:

DS2019 forms for everyone - these should be provided by your sponsor

An appointment at the Embassy in London (the Consulate in Edinburgh doesn't do visas. I wonder what they DO do.)

The appointments are booked via a phone line at £1.20 per minute

After you make the appointment, you will receive via email an appointment letter and a payment receipt for each member of your family
Visa application forms for everybody - this is form DS156, which you complete online, and then print out.
Contact forms DS158 for everybody. This is a PDF file that you print out, and then complete (unless you're lucky enough to have a PDF editor).
Form DS157 for males of military age (and assorted other categories such as scientists). This is where you have fun listing your chemical warfare experience. It's also a PDF that you print out.
A SEVIS fee receipt - this is a receipt that you've coughed up the $100 to take part in the program. You can print this off from the web after you've filled in the online form. Note that they don't tell you that you can use this as official proof until you've already agreed to pay an additional $30 to have the certificate couriered over from the US.
passports for all the family
One passport photo for each family member.

What happens at the embassy
You show up at the embassy shortly before the time on your appointment letter and join a queue. There's one queue (red) for US citizens doing US-citizen things and one queue (yellow) for the rest of us (though when I was there, there was noone in the red queue). At some point a friendly english girl comes down the line and checks your appointment letter, crosses you off a list of expected people, and ushers you on to the next queue at the security checkpoint. At around your allotted time you are waved forward into the checkpoint, which is a portacabin containing an airport style xray machine and a cheery Australian chap. Your documents are looked at again, and you have to put your stuff through the xray machine. There's nothing here that's any more scary than going into an airport departure lounge. Off you trot round the outside of the embassy following the "visas" sign, up some steps and to reception, where your appointment letters are checked again, and you're given a sticker with a number on it. You go into a waiting room and, well, wait. Eventually your number is called and you are told to go to a counter window, where all your documents are taken and examined. These booths are also manned by surprisingly friendly (since it must be a tedious job) Brits and Australians. At this point you get to play with the finger print machine. You're given a blue form with your sticker attached and told to go and sit down and wait again. Tum-de-tum. After an hour or so you get called again, this time to the booths down the corridor for your "interview". If you were expecting an interview behind closed doors with a guy with a rubber hose and a desk light then you'll be pleasantly surprised. Again, the interview takes place at a counter window, in my case with a charming and friendly girl who ought to have been working for the tourist board, not the embassy. This job only seems to be trusted to Americans though, since they presumably have a bigger incentive to keep ne'er-do-wells out of their country. You will be asked about your research project, but we're talking about delivering your elevator pitch here...it's not a viva exam. Click click, stamp stamp. Your visa has been granted. The whole interview took about 5 minutes. Including waiting time, the process takes about 2 hours (though I was told it was a particularly light day). From the order the numbers were being called it was clear that some people had been waiting considerably longer, presumably for being too muslim, too dark skinned or having too interesting a police record. Finally, you take your blue form to a counter at the back of the waiting room where you hand over £15 pounds to pay for the safe return of your passports. I guess you could skip this bit, but it would render the trip a little pointless.

Frequently Asked Questions
Well, I was was asking them, anyway.

How long does the appointment phone call last?

At £1.20 a minute, you'd expect them to milk it for all the cash they could get, but mine proceeded at such a rapid pace you'd think it was them paying, not me. Everything was done in about 10 minutes.

Do children need to attend?

Apparently, children under 14 don't need to go.

Can children attend?

Yes. They advise that the embassy isn't the most fun place in the world for a young child, but you're not prohibited from taking child + pram into the embassy.
The pram will probably need to go through the xray though.

What about my mobile phone, key fob etc....?

The website says these aren't allowed. It's lying. When you go through security they'll be taken off you and stored till you return.

The website says that the visa photos need to be 2 inches square.

Normal UK passport sized photos seem to be acceptable.

Are food and drink allowed in?

They say that you can't take liquids in, and I've no reason to doubt this. However, there's a cafe inside that serves tea, coffee and pop. Beware that it closes at 3ish though.

Can I take a bag in?

The website says that you can't take a large rucksack in....I think they mean one of those 70 litre jobs. I took in a reasonably large briefcase. From the setup I expect that they'll tolerate anything you could take as hand luggage on a plane, though their patience might be more limited on busy days.

Faster pussycat

2007-08-02T14:12:00.000+01:00

"...but Java's so slow...."

1) Java's bytecodes are interpreted by a virtual machine, therefore Java can never be as fast as a statically compiled language such as C
2) Java's VM can perform dynamic optimization in ways not available to a statically optimizing C compiler, therefore Java can outperform C.

The web is full of opinions on which is the fastest language and why. There are plenty of benchmarks proving once and for all one view or the other. However, you never really believe it unless you've tried it yourself. So here, for the first time ever is conclusive proof that really there ain't much in it one way or another.

My completely unscientific test case: an algorithm to sum the first N Fibonacci numbers. The numbers themselves are calculated by a deliberately naive recursive algorithm - don't try this at home. The running time grows exponentially with N. However, it is easy to get a good range of running times. The algorithm can be coded almost identically, line for line, in my 3 target languages of Java, C, Python.

Here are the results:

N	Java (client vm)	Java (server vm)	C (gcc standard options)	C (gcc -O3)	Python
0	0.2	0.2	0	0	0
10	0.2	0.2	0	0	0
20	0.2	0.2	0	0	0
30	0.3	0.2	0	0	1.4
35	0.5		0.4	0.2	15
40	2.5		4.8	1.8	171
45*	24.5	20.6	52	19.8
50	272	232	600	227

*For N>=45 the result overflows when using 32 bit integers. Here's some results using Java longs and C long longs (64 bits):

N	Java (server vm)	C (gcc -O3)
45	21	23.5
50	221	265

No doubt you can find algorithms (and better C compilers) that change the results. But change them enough for you to care?

Details:
The tests were run on Ubuntu Linux running in a Parallels VM on a 2007 model Macbook Pro with a couple of gig of RAM.
The Java code was compiled and run with Java 6, the C code with gcc, and the Python code run with Python 2.4.3
Times are the sum of user and system time in secs, recorded to the nearest 0.1 of a sec, and the tests were run a handful of times until I reckoned the time looked about right. Times include any VM startup times.

Boxplots for beginners

2007-06-07T17:14:00.000+01:00

How to create Boxplots in R
Refs:
http://en.wikipedia.org/wiki/Boxplot
http://www.r-project.org/

Suppose you have data such as:
number_of_threads thread time
1 1 10
2 1 20.5
2 2 19.5
3 1 30
3 2 25
3 3 35

This table represents made up data describing some system testing, where the system is exercised with 1,2,3... concurrent users. You want to produce a boxplot showing the timings for each number of concurrent users. Here's the magic incantation in R:
Assuming the above file is saved to data.txt
1) Use R's Misc menu to change to the correct working directory
2) Load the data into a dataframe with
myresults = read.table("data.txt", header=TRUE)
3) Generate the boxplot with
boxplot( time ~ number_of_threads, myresults)

To jazz it up with some axis labels:

boxplot( time ~ number_of_threads, myresults, xlab="Concurrent users", ylab="time")

Adding several boxplots on the same axis:
boxplot( time ~ number_of_threads, myresults, add=TRUE)
*Warning* - if you don't see your new boxplot, then it's because for some nutty reason the plot doesn't automatically rescale, and you need to set the scale explicitly with ylim=c(0,40) or similar.
Making the boxes a particular colour:
boxplot( time ~ number_of_threads, myresults, col="yellow")
and adding a legend:
legend(2,9m c("plot1","plot2"), fill=c("red","yellow"))

2007-06-06T19:17:00.001+01:00

http://www.flickr.com/photos/josiefraser/532394229/in/set-72157600317386945/

Our Emerge slide.

The Web 2.0 Expo, San Francisco

2007-04-24T16:36:00.000+01:00

Audio files from some of the talks:

Check out my Public Files on myDataBus!
ArchitectureforHumanityWeb2.0Expo.mp3
ArchitectureforHumanityWeb2.0Expo.mp3
BadgesandWidgetsWeb2.0.mp3
BuildingSMSWeb2.0Expo.mp3
BuilttoLast.mp3
GoogleWeb2.0.mp3
JeffBezosWeb2.0Expo.mp3
JeffBezosWeb2.0Expo.mp3
LaunchPad1Web2.0Expo.mp3
LaunchPad2Web2.0Expo.mp3
LocationtoanappWeb2.0.mp3
MashupsincompleteWeb2.0.mp3
Mobile2.0Web2.0Expo.mp3
StateoftheWeb2.0.mp3
Syphilis.mp3
Provided to you by

Converting jpgs to ps or eps

2007-02-06T16:42:00.001Z

A bit of googling finds a number of commercial and shareware packages that will do this task, but you can also do:

jpegtopnm MyDog.jpg | pnmtops > MyDog.ps

on a unix machine.

http://en.wikipedia.org/wiki/Portable_Pixmap_file_format

Moving mail to gmail

2007-01-31T13:03:00.000Z

I've decided to move completely over to gmail so that I don't have to configure my mail clients to deal with umpteen different accounts. I also want to store all my legacy mail so it's all searchable in the one place. How do I upload my Thunderbird mail files to gmail?
This: http://www.marklyon.org/gmail/download.htm seems to do the job. It's a small python app that goes through your Thunderbird mbox files and forwards them all. It's slow, and it's going to mess up your emails' timestamps, but it does the job.

Weekly summary

2007-01-19T10:45:00.001Z

* Helping Isa debug PLASTIC functionality in VOSpec

http://esavo02.esac.esa.int:8080/vospec/
* Prep for MSc
* Installed latest STILTS
* Looking into an example for VisIVO paper
* New Plastic group in Apps WG
* Uni of Queensland Connection

--
------------------------------------------------------------------------
AstroGrid/VOTech
&
WFAU, Institute for Astronomy, Edinburgh
Skype:johndavidtaylor <skype:johndavidtaylor?chat>

------------------------------------------------------------------------
*Gratuitous advertising:*
Plastic - http://plastic.sourceforge.net | AstroRuntime -

http://www2.astrogrid.org/desktop
AstroGrid - http://www.astrogrid.org | WFAU -

http://www.roe.ac.uk/ifa/wfau/

How can a bash or shell script find its own directory?

2007-01-11T15:13:00.001Z

It's often desirable to write a small bash script to launch an
application. However, you want the application to be portable, so you
don't want to hard-wire absolute file locations into the script.
Unfortunately, this means that if you execute the script from somewhere
other than the directory which holds the script, then it won't locate
the files. A bit of googling suggests that the general problem of
making a script aware of the directory in which it resides is quite
hard. However, the following works for me:
cwd=`dirname $0`
old=$pwd
cd $cwd
#launch your application here, for example
nohup python main.py &
#where main.py is in the same directory as the script
cd $old

Subclipse, Eclipse plugin does not work on Ubuntu

2007-01-11T13:30:00.001Z

You've installed Eclipse, the subclipse subversion plugin, set up your
repository locations and are ready to go. Left click on your favourite
subversion repository to see the contents. Uh-oh:
'
Folder " does not exist remotely.
'

This is because the JVM that ships with Ubuntu (GNU gij) doesn't even
remotely work.
OK that's very unfair. It does seem to screw up Eclipse and Subclipse
though.
To fix this problem, you need to download the latest Sun JDK install it
somewhere convenient, and (importantly) make sure that this JDK's bin
folder is on the front of your path. If you've got it set correctly
then executing java -version at the command line should give you the
reassuring
"
java version "1.6.0"
....
java HotSpot(TM) Client VM.....

"Multiproject" site generation in Maven 2.0.4

2007-01-04T00:41:00.001Z

Yet again Maven has shat upon me from on high by randomly updating a
plugin and changing its behaviour. Previously, if you had a project
configured as a "multiproject" build of modules, the way to include a
link in your parent project's docs to the child documentation was to add
${modules} into the parent's site.xml. No longer! Try this now, and
your mvn will spit out some garbage about your site.xml being malformed
xml, when it plainly isn't. No, now the secret incantation is to put a
menu item in thus: <menu ref="modules"/>. The same goes for ${reports}
and ${parent} presumably. The new syntax is certainly nicer, if you
don't mind the extra wear and tear on your fingers, but I wish they
had just deprecated the old syntax, rather than secretly removing it.

Problems with Image loading in Java

2007-01-04T00:37:00.001Z

I'm trying to load images in Java for use in icons with the following:
new ImageIconr(url)

Mostly, this works fine. But just once in a while it fails with:

Uncaught error fetching image:
java.lang.SecurityException
at java.lang.SecurityManager.checkPermission(SecurityManager.java:562)
at java.lang.SecurityManager.checkConnect(SecurityManager.java:1078)
at sun.awt.image.URLImageSource.checkSecurity(URLImageSource.java:81)
at sun.awt.image.ImageRepresentation.imageComplete(ImageRepresentation.j
ava:597)
at sun.awt.image.InputStreamImageSource.errorConsumer(InputStreamImageSo
urce.java:131)
at sun.awt.image.InputStreamImageSource.setDecoder(InputStreamImageSourc
e.java:331)
at sun.awt.image.InputStreamImageSource.doFetch(InputStreamImageSource.j
ava:252)
at sun.awt.image.ImageFetcher.fetchloop(ImageFetcher.java:172)
at sun.awt.image.ImageFetcher.run(ImageFetcher.java:136)

What's worse, surrounding the code with a try-catch block doesn't save you - the application simply locks up. Note that this
only happens when running under a SecurityManager such as under WebStart. It seems there's a bug in the JDK
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4881229
which means that if the URL gets redirected with an http 302, then an internal thread in the JDK will fail. This
is pretty hopeless - can't fix it, can't trap it.

The workaround is to use a different part of the API. Instead of constructing an Icon from a URL, you first get an Image using
Image im = ImageIO.read(url);
then construct an icon from that:
new Icon(im);

Subversion on sourceforge refuses to make copies

2006-12-13T23:22:00.000Z

If you find that SourceForge's subversion system is buggered (again) and
refuses to let you make copies, it could be because you hadn't noticed
their not-very-well-advertised switch of access URL to:

"On November 31, 2006 the access method for Subversion changed. This
document reflects those changes. The old method had numerous problems,
including spurious 50x error messages and other issues that kept it from
functioning fully. This newly documented access method solves many, if
not all of the issues with the old mechanism.

Users of the old method
(https://svn.sourceforge.net/svnroot/PROJECTNAME) should switch to the
new access method
(https://PROJECTNAME.svn.sourceforge.net/svnroot/PROJECTNAME) using
these steps:

1. Make a copy of your local working copy.
2. Run 'svn info' at the root of the repository content, it should
display a line that appears similar to: URL:
https://svn.sourceforge.net/svnroot/PROJECTNAME/trunk
3. Run the following command at the root of the working copy: svn
switch --relocate
https://svn.sourceforge.net/svnroot/PROJECTNAME/trunk
https://PROJECTNAME.svn.sourceforge.net/svnroot/PROJECTNAME/trunk

SVN is a complex tool."

Right, and it doesn't get any easier when SourceForge keep arseing about
with it.

http://sourceforge.net/docman/display_doc.php?docid=31070&group_id=1#notice

AstroGrid registry bug with namespaces

2006-12-11T13:36:00.001Z

We've recently upgrade our WFAU publishing registry to version Version
2006.3.03r. This has introduced a bug where previously registered
entries cannot be edited. When you submit the edits you get a:

env:ServerServer Error: No namespace defined for prefix vororg.exist.xmldb.LocalXPathQueryService.compile(LocalXPathQueryService.java:213) org.exist.xmldb.LocalXPathQueryService.doQuery(LocalXPathQueryService.java:280) org.exist.xmldb.LocalXPathQueryService.query(LocalXPathQueryService.java:112) org.exist.xmldb.LocalXPathQueryService.query(LocalXPathQueryService.java:103) org.astrogrid.xmldb.client.XMLDBServiceImpl.queryXQuery(XMLDBServiceImpl.java:235) org.astrogrid.registry.server.xmldb.XMLDBRegistry.query(XMLDBRegistry.java:71) org.astrogrid.registry.server.admin.AuthorityListManager.getManagedAuthorities(AuthorityListManager.java:118) org.astrogrid.registry.server.admin.AuthorityListManager.populateManagedMaps(AuthorityListManager.java:106) org.astrogrid.registry.server.admin.RegistryAdminService.updateResource(RegistryAdminService.java:282) org.apache.jsp.admin.addResourceEntry_jsp._jspService(addResourceEntry_jsp.java:488) org.apache.jasper.runtime.HttpJspBase.service(H
ttpJspBase.java:94) javax.servlet.http.HttpServlet.service(HttpServlet.java:802) org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:324) org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:292) org.apache.jasper.servlet.JspServlet.service(JspServlet.java:236) javax.servlet.http.HttpServlet.service(HttpServlet.java:802) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapp
Valve.java:214) org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152) org.apache.catal
ina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540) org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102) org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137) org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118) org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102) org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) org.apache.c
atalina.core.ContainerBase.invoke(ContainerBase.java:929) org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160) org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799) org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705) org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577) org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683) java.lang.Thread.run(Thread.java:595)

The fix is to copy the
xmlns:vr="http://www.ivoa.net/xml/VOResource/v0.10" from the
vor:Resource into the root vor:VOResources element.

VOMon - the VO service monitor has been released:

2006-12-11T13:36:00.000Z

VOMon - the VO service monitor has been released:

http://vomon.sourceforge.net

VOMon is a mashup of the Astro Runtime
<http://www2.astrogrid.org/desktop/astro-runtime> and MARS
<http://leapfrog-mars.sourceforge.net/>, glued together with Python.
VOMon uses the Astro Runtime to query a Registry to obtain all the
services that match an admin-specified query. By default it queries the
central AstroGrid registry, but can be configured to query any
IVOA-compliant registry. This list of services is updated every hour or
so, so that VOMon will always have a reasonably up to date list of
services to monitor. The services are passed to MARS which does the
actual monitoring. Currently this monitoring consists of simply
pinging the accessURL and checking the response. Some services such as
Cone and SIAP, have default parameters appended to the accessURL to
ensure they do not fail. Once the VOSI interface is widely supported,
VOMon will be upgraded to use that. VOMon can either output a report to
be hosted on a web server, or present the user with a GUI interface.

An instance has been deployed at:
http://thor.roe.ac.uk/vomon/status.xml

Weekly Summary

2006-12-05T15:48:00.000Z

* Prepared proposal for JISC CoP participation
* Plastic Website maintenance http://plastic.sourceforge.net and
general email business
* Finally, published IVOA Plastic Note:
http://ivoa.net/Documents/latest/PlasticDesktopInterop.html

Weekly update 15/11/06

2006-11-15T12:23:00.000Z

* Submitted ADASS paper
* Currently moving the fastem algorithm off hempriggs. In the process
I'm upgrading to a new version supplied by CMU that doesn't require user
intervention when things go wrong, thus making the web service more
reliable. Unfortunately, this new version has a slightly different
interface, so it's not a straight drop in replacement.
* Given access to all CMU's algorithms
* Tony has asked me to lead a bid to join the U&I CofP. Email
attached. Deadline is next week, shouldn't take too much work.
Co-conspirators are Elizabeth A and Norman.
* Upgraded the Workbench Plastic library and made some fixes

Cygwin/X - logging into remote linux machines from Windows

2006-11-09T11:34:00.000Z

Logging into linux machines from Windows can be done in various ways,
such as using ssh via PuTTY. In alternative is to use Cygwin, which not
only gives you a fairly complete Linux emulation on your windows box,
but also makes it easy to use X-forwarding, and thus use remote
GUI-based applications.

Step 1) Install Cygwin. The installer will give you a choice of
packages to install. Make sure you install the x11 packages
Step 2) Start the x server by running [cygwin
installation]\usr\X11R6\bin\startxwin.bat
Step 3) From the resulting xterm window, log into the remote machine
with ssh -X (remote machine name)
Any remotely started GUI apps should now magically appear on your
Windows box.

If you get an error of the form
warning, error event received:
X Error of failed request: BadAtom (invalid Atom parameter)
Major opcode of failed request: 18 (X_ChangeProperty)
Atom id in failed request: 0x1b3
Serial number of failed request: 944
Current serial number in output stream: 946

when you try to start the GUI app, try using -Y instead of -X.

Increasing the memory available to Eclipse.

2006-11-08T16:06:00.000Z

To increase the memory allocated to the JVM for Eclipse, edit the
eclipse.ini file in your Eclipse directory to be something like:
-vmargs -Xms40m -Xmx512m

Weekly Summary

2006-11-08T14:07:00.000Z

AG@WFAU Operations
=================
* Helped test ukidss
* Suggested compression as a solution to the large metadoc file problem.
* Moved test-registry
* China-AGVO operational
Datamining
=======
* Swotting up on datamining
* Been in touch with Andrew Moore about fastem - applied for access to
software

Plastic
====
* ADASS followup
* Been in touch with Ruby people
* TKH unavailable
* Contacted Jason about site spruce up but heard nothing yet

Other
=====
* Preparing Eclipse lecture
* Setting up my Eclipse to do C++

AOB
deadline for adass manuscripts is Monday

Setting up C++ in Eclipse on Windows

2006-11-07T13:42:00.000Z

Although Eclipse works most naturally as a Java IDE, it doesn't do too
badly for other languages, and has the advantage that you don't need to
learn multiple IDEs and can keep all your code together in a single
environment.
Steps in setting up C++ in Eclipse, on Windows:
* If it's not already installed, get the CDT plugin - this is available
as part of the standard Callisto bundle from the Eclipse website.
(help/software updates/find and install/add new remote site
http://download.eclipse.org/tools/cdt/releases/callisto)
* The CDT doesn't come with its own compilers etc - you'll need to
download them separately. One option is to install Cygwin, making sure
that you're ticked all the boxes to get make and gcc. Important: don't
forget to add cygwin/bin to your windows path. The problem is, that any
executables you then make will need the cygwin1.dll to be bundled with
them, which is hardly ideal.
* An alternative, mentioned in
http://wiki.fredlackey.com/pmwiki.php?n=HelpfulInfo.DevelopmentEclipseCppCofnig
and others is to install MinGW from
http://prdownloads.sourceforge.net/mingw/MinGW-5.0.3.exe?download which
will produce exes with no dependencies. Again, make sure it's on your path.
* Managed vs Standard projects. If you create a managed project then
Eclipse will generate a simple makefile for you, otherwise you are
expected to create a makefile yourself, and will spend many happy hours
hitting the "build" button and wondering why nothing happens.

xml-rpc from C++ on Windows pt I

2006-11-06T13:47:00.000Z

As part of the Plastic http://plastic.sourceforge.net project I'm
looking at making xml-rpc calls from different languages. Current focus
is on C/C++ since it opens up the possibility of using SWIG to get
Plastic access from a large range of other languages "for free".

My setup:
OS: Windows XP
IDE: Eclipse, with the CDT plugin
Tools: g++ and make, as part of the MinGW and MSYS systems

The most promising xml-rpc library is this one:
http://xmlrpcpp.sourceforge.net/links.html. Promising because a) I've
got it working before b) It has no dependencies c) It does client and
server (necessary for a fully-fledged plastic application).

Steps
1) Download and install the xml-rpc library files into a suitable folder.
2) Add this folder to the list of source folders in project
properties/c++ build/Tool settings/compiler/directories
3) Add the _WINDOWS flag to the compiler/preprocessor
If you hit build now, you'll get linker errors as listed below. The
problem is that although you get the winsock header files included on
the path automatically, you need to explicitly specify the corresponding
library to the linker. The library in question is probably called
something like: libws2_32.a and can be found in the MinGW/lib folder.
4) project properties/c++ build/Tool settings/linker/libraries/ add
library ..... and here's the real doozy....you add ws2_32 (!!!not!!!
libws2_32.a .... the linker prepends the lib and adds the .a for you).

Hit build, and you're away. All the above assumes that you have already
written your xml-rpc client of course....

=================================================================
The linker errors (for google's benefit):
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:45:
undefined reference to `WSAStartup@8'
./xmlrpc/XmlRpcSocket.o(.text+0x5d): In function
`ZN6XmlRpc12XmlRpcSocket6socketEv':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:80:
undefined reference to `socket@12'
./xmlrpc/XmlRpcSocket.o(.text+0x8e): In function
`ZN6XmlRpc12XmlRpcSocket5closeEi':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:89:
undefined reference to `closesocket@4'
./xmlrpc/XmlRpcSocket.o(.text+0xbc): In function
`ZN6XmlRpc12XmlRpcSocket14setNonBlockingEi':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:103:
undefined reference to `ioctlsocket@12'
./xmlrpc/XmlRpcSocket.o(.text+0x10b): In function
`ZN6XmlRpc12XmlRpcSocket12setReuseAddrEi':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:115:
undefined reference to `setsockopt@20'
./xmlrpc/XmlRpcSocket.o(.text+0x157): In function
`ZN6XmlRpc12XmlRpcSocket4bindEii':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:126:
undefined reference to `htonl@4'
./xmlrpc/XmlRpcSocket.o(.text+0x16b):E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:127:
undefined reference to `htons@4'
./xmlrpc/XmlRpcSocket.o(.text+0x18c):E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:128:
undefined reference to `bind@12'
./xmlrpc/XmlRpcSocket.o(.text+0x1bc): In function
`ZN6XmlRpc12XmlRpcSocket6listenEii':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:136:
undefined reference to `listen@8'
./xmlrpc/XmlRpcSocket.o(.text+0x1fa): In function
`ZN6XmlRpc12XmlRpcSocket6acceptEi':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:151:
undefined reference to `accept@12'
./xmlrpc/XmlRpcSocket.o(.text+0x23a): In function
`ZN6XmlRpc12XmlRpcSocket7connectEiRSsi':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:164:
undefined reference to `gethostbyname@4'
./xmlrpc/XmlRpcSocket.o(.text+0x290):E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:169:
undefined reference to `htons@4'
./xmlrpc/XmlRpcSocket.o(.text+0x2b1):E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:173:
undefined reference to `connect@12'
./xmlrpc/XmlRpcSocket.o(.text+0x383): In function
`ZN6XmlRpc12XmlRpcSocket6nbReadEiRSsPb':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:191:
undefined reference to `recv@16'
./xmlrpc/XmlRpcSocket.o(.text+0x49e): In function
`ZN6XmlRpc12XmlRpcSocket7nbWriteEiRSsPi':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:223:
undefined reference to `send@16'
./xmlrpc/XmlRpcSocket.o(.text+0x517): In function
`ZN6XmlRpc12XmlRpcSocket8getErrorEv':
E:/jdt/data/experiments/PlasticListener2/Debug/../xmlrpc/XmlRpcSocket.cpp:248:
undefined reference to `WSAGetLastError@0'

--
------------------------------------------------------------------------
AstroGrid/VOTech
&
WFAU, Institute for Astronomy, Edinburgh
Skype:johndavidtaylor <skype:johndavidtaylor?chat>

------------------------------------------------------------------------
*Gratuitous advertising:*
Plastic - http://plastic.sourceforge.net | AstroRuntime -
http://www2.astrogrid.org/desktop
AstroGrid - http://www.astrogrid.org | WFAU -
http://www.roe.ac.uk/ifa/wfau/

Weekly Summary 1/11/06

2006-11-01T13:52:00.000Z

Debate about coverage in registry and possible solutions
Visit to Cambridge to cover Taverna

Link to Guy's summary

Side debate on merits of SourceForge over managing our own code

Discovered that with the exception of 2MASS, our cone searches aren't showing in AstroScope - should do soon following redeploy
Discovered many missing services from AstroGrid's registry...Kevin fixing....
Swotting up on Data Mining
Downloaded VOTechBroker
Following up ADASS

Weasl - Wfau Edinburgh Astro Service Locator

2006-10-31T11:00:00.000Z

At the last IVOA meeting in Russia, David Schade argued that "position
is the least important attribute of an object". He was talking about
astronomical objects of course, but it's probably true of VO services
too. Nevertheless, I wrote this:

http://thor.roe.ac.uk/weasl/index.html

It's built from the Astro Server Runtime, a google map, a GeoEncoder
service and some sticky tape.

It's not particularly slick at the moment - the server just writes out a webpage with all the javascript required to place the points on the map. It would be far better to make it more AJAXy and have the webpage request the points. Could make it a lot more dynamic then.

Gotchas
=======
None really. The ASR made it very easy, and Google is, of course, wonderful.
Registry Access
---------------
All done through the ASR. The trick was to check the type of all the returned Resources to see if it's a Service. Only then do you get access to the attributes you need.
Goole Map
---------
The map javascript is pretty good, though it gets a bit unresponsive above a hundred or so points. One gotcha is that the <div> that contains the
map worked fine as
<div id="map" height="" width=""></div>
but if you compressed the tag into
<div id="map" height="" width=""/>
the browser nested any following divs inside it, with hilarious consequences.

Getting Java Server Pages up and running in Tomcat

2006-10-27T10:03:00.000+01:00

Trying to get JSF up and running to use as a front end to WEASL (coming
soon). Huge problems with missing classes when running the latest JSF
(1.2) in Tomcat 5.0.28, so fallen back to 1.1. Apparently 1.2 requires
a version of JSP that Tomcat doesn't yet support. Trying out the
examples in Core JSF (http://www.horstmann.com/corejsf/)
but got an NPE;
27-Oct-2006 00:30:41 org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Allocate exception for servlet Faces Servlet
java.lang.NullPointerException
at javax.faces.webapp.FacesServlet.init(FacesServlet.java:165)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1105)
at
org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:757)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:130)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)

The solution to this is simply to remove the working directory from
under Tomcat and restart the webapp. All will then be well.

Two weekly summary

2006-10-25T12:56:00.000+01:00

* ADASS
* 6DF database problem (see bugzilla
http://www.astrogrid.org/bugzilla/show_bug.cgi?id=1913)
* Upgraded Plastic library in AstroWeka, got the code into SVN and fixed
the Ant build script
--
------------------------------------------------------------------------
AstroGrid/VOTech
&
WFAU, Institute for Astronomy, Edinburgh
Skype:johndavidtaylor <skype:johndavidtaylor?chat>

(Two) Weekly Summary 11/10/06

2006-10-11T12:30:00.000+01:00

* Started trying to collate all the apps we do in DS6 here:
http://wiki.eurovotech.org/twiki/bin/view/VOTech/DS6Software and
chivvy people along into having a proper release process.
* Looked at Maven 2.1-SNAPSHOT in the hope that it has solved the many
bugs in Maven 2.0.4 that are stopping me getting _my_ software into shape
* Maven is still a disaster for site generation. Refactored the Plastic
site to remove as many subprojects as possible in the hope that this
will help.
* In depth look at Mirage
* Installed STILTS-1.3 and got authorityId changed to Starlink, which we
manage here at ROE.
* Checked all our dbs except usnob, conesearch and adql through workbench.
* Created bugzilla entries for all our components to keep on top of
issues. See
http://www.astrogrid.org/bugzilla/editproducts.cgi?action=edit&product=WFAU
* Released PAM - the Plastic Application Monitor plastic.sourceforge.net/tupperware/plastic-application-monitor/
* Continued drafting Plastic Note
* ADASS preparation.

Maven site generation still buggered

2006-10-05T23:15:00.000+01:00

I'd hoped that some of the issues with maven site generation
http://jontayler.wordpress.com/2006/07/26/porting-the-plastic-site-over-to-maven-2/
might be fixed by now. There seems to be no hurry to release the next
version of Maven
(see http://www.mail-archive.com/users@maven.apache.org/msg51783.html)
so I ventured into bleeding-edge territory and downloaded the latest
SNAPSHOT from
http://maven.zones.apache.org/~maven/builds/branches/maven-2.0.x/

Neither the site:staging bug nor the
windows-maven-site-ignores-settings.xml bug appear fixed. Admittedly,
these are bugs in plugins, rather than the core, so perhaps I just need
to force it to use the latest plugins. How hard can that be?

It's reassuring that I'm not the only one losing patience with the whole Maven endeavour:
http://www.ctoforaday.com/archives/000050.html

Recursive IFRAME IIb

2006-10-03T20:48:00.001+01:00

scrolling="auto" frameborder="1">
[Your user agent does not support frames or is currently configured
not to display frames. However, you may visit
the related document.]

Even trying to use two pages to do the recursion doesn't fool Firefox.

Recursive IFRAME II

2006-10-03T20:48:00.000+01:00

scrolling="auto" frameborder="1">
[Your user agent does not support frames or is currently configured
not to display frames. However, you may visit
the related document.]

IFrames

2006-10-03T20:35:00.000+01:00

Inspired by this:
http://communitygrids.blogspot.com/2006_03_01_communitygrids_archive.html
which is an interesting blog in its own right. I was tempted to see if you can do a recursive IFrame.

scrolling="auto" frameborder="1">
[Your user agent does not support frames or is currently configured
not to display frames. However, you may visit
the related document.]

Quite sensibly - Firefox spots the recursion and just presents a white box.

Animated GIFS for WFAU

2006-10-03T20:33:00.000+01:00

Mark has been working on some rather neat animated GIFS for WFAU. Hope to embed these in our registry entries - that should blow CASU off the page in AstroScope.
http://eurovotech.org/wfau/logos/wfau_logos.htm

Will get similar done for Plastic....

Evaluating Skype's security

2006-10-01T18:06:00.000+01:00

http://www.skype.com/security/files/2005-031%20security%20evaluation.pdf

System Tests for AG@ROE

2006-09-29T16:48:00.000+01:00

One of the biggest things we have to address if we are to persuade astros that the VO is a serious project is reliability. Currently too many services fall over or don't meet interoperability standards fully.
At Edinburgh we are hosting the full range of VO Services from datasets such as the WFCAM Science Archive to applications such as the Anomaly Detector and STILTS to infrastructure such as MySpace. We have to make sure that we know that systems have fallen over before our customers do.
We are going to need a multilayered solution, from simple-minded pinging of webservices such as that provided by MARS (currently down, awaiting a java upgrade) to the heartBeat service in the VOSupportInterfaces currently being developed by the IVOA GWS group. Going to greater depth, I think we need automated system tests of all of our user-facing components, testing them in the same way as a user would. These system tests would be run (say) once a day, rather than providing the real-time feedback offered by the other techniques. For example, MySpace should be tested by creating folders and files, storing and retrieving data. A dataset should be tested by running a standard conesearch etc.

A convenient way to run these tests is via the Astro Server Runtime developed by Noel Winstanley. This offers an easy API to all the VO services we deploy here at Edinburgh and allows me to write the tests in my language-of-choice. From my previous experience of running the Integration&Release process, I've decided to go with JUnit tests wrapped in a Maven project run by a cron job. This is because JUnit tests are easy to write, and Maven provides easy facilities to publish reports to the web on the success rate of the tests.

Status of the System Tests project:

Tests for Community and MySpace written
Tests for conesearches fleshed out
Code has been put into cvs under: http://www.astrogrid.org/viewcvs/test/AgAtRoe/
Bugs in the Workbench and Astro Server Runtime have been identified that block further development
Need to restructure build to use the full, un-Proguarded ASR.

(Two) Weekly Summary 27/9/06

2006-09-26T12:25:00.000+01:00

One Week at IVOA Moscow (Notes)
Released a draft of the Plastic spec for the IVOA note, which received quite a thorough going over from MT.
Started work on System tests for AG@ROE

Anomaly Detector problem

2006-09-26T12:21:00.000+01:00

There seems to be a problem running the Anomaly Detector algorithm at
http://eurovotech.org/twiki/bin/view/VOTech/AnomalyDetection

This algorithm is currently in the ROE Test registry, not in the main AstroGrid registry. Unfortunately the new workbench seems to be disregarding registry settings and using Leicester, regardless. This might be linked to the change over to use the streaming registry client.

Useful page:
Workbench configuration options.

(Problem now fixed in HEAD)

(Two) Weekly Summary 13/9/06

2006-09-13T12:17:00.000+01:00

In Strasbourg at VOTech meeting.

Progress on messages for clustering and spectra
approval from MT and TB for IVOA note
Interesting tool from Bruno...might help with SIAP service for SSA
Information from Guy on secure SIAP - seems that TLS is the way forward, provided we can find a container that can deal with certificates.

I've asked Pierre Fernique (deputising for Mark Allen) for a slot at the IVOA meeting - it seems they're not inundated with speakers at the apps IG.
Marked MSc dissertation.

Actions Items:
--------------

(68.2) JDT to give an ROE coffee talk with a demo of AstroGrid
- maybe also including access to UKIDSS data.
[open: decided to postpone this until after registration of users]

(69.2) JDT to pass this up to Noel Winstanley and/or Kevin Benson.
[closed: see AG@ROE below]
Follow up: JDT to fix registry entries

(70.4) JDT to contact Guy Rixon to check that their plans for wrapping
STIL(TS) as CEA services are consistent.
[closed: JDT decided that the best thing to do would be to deploy here
a clone of the Cambridge set-up. STILTS can auto-generate the required
application description, so that it is possible to have the description
for the two installations being identical, which would enable JES to
pick on or other of them.]

Followup: JDT has now cloned the STILTS installation at the ROE. This is using STILTS' auto-generated app-description. There are a couple of minor issues: some of the description metadata is html-formatted, and doesn't play nicely with the Workbench, and one of the parameter types has been incorrectly specified. [the constant specifying the STILTS command]. One of the disadvantages of cloning the Cambridge installation is that we in Edinburgh don't get any credit - it's got IoA logos all over it.

(70.5) JDT to contact Phil Nicholson to ask about his plans for
a standard SIAP component.
[closed: see AG@ROE below]

Follow up: see JDT comments above.

AOB
---

IBM hardware
JDT mentioned that there had been an email circulated about the possibility of
some surplus IBM hardware becoming available on site soon.
Any news?

Thunderbird mail forwarding from a filter broken

2006-08-29T13:59:00.000+01:00

I want to use a spare gmail account to backup all my incoming and outgoing mail in Thunderbird. Outgoing is easy enough - for each account under tools/accounts/Copies & Folders you add a line to the Bcc section. To set up backup of incoming mail, I created a filter to act on the inbox and find "read" messages, and forward them. This ought to work but doesn't....I just get an alert popup box with "sending of messages failed." It seems I'm not the only one.

Java signed applet security insanity

2006-08-29T09:41:00.000+01:00

I've been looking into using signed applets as a javascript-plastic bridge. The applets have to be signed so that they can make RMI connections to the Plastic Hub - this works well. Then through the wonders of LiveConnect you can call methods on the applet from JavaScript, and thanks to a magical object called JSObject you can make callbacks back to the JavaScript from the Java. This also, mostly, works well. However, try deploying the applet anywhere but localhost, and some of the JavaScript->Java calls fail with security exceptions (see below). Not all, just some. The reason? Apparently, if the JavaScript itself isn't signed, then any method calls originating in the JavaScript are treated as if the applet itself isn't signed - you lose all your signed applet privs. What kind of deranged madman thought that up? It's useless as a security benefit - all you do is put the controversial stuff into a separate thread, so that the JavaScript to Java calls merely set and get properties...so it's trivially subverted. Instead it just makes your code more complicated than it need be, and confounds the poor programmer with mysterious errors.

Implementation note: it's not good enough just to spawn an new thread in the method called by JavaScript, as this will inherit the security of the calling thread. Instead you need to instantiate a worker thread in (say) the start() method of the applet, and have the JavaScript-called method hand tasks to this worker. You can see a (rather naive) implementation here.

-----------
An alternative solution:
Put the "untrusted code" in:

  somemethod() {
       ...normal code here...
       String user = (String) AccessController.doPrivileged(
         new PrivilegedAction() {
           public Object run() {
               return System.getProperty("user.name");
           }
         }
       );

Stacktrace for google's benefit:
java.security.PrivilegedActionException: java.lang.reflect.InvocationTargetException
at java.security.AccessController.doPrivileged(Native Method)
at sun.plugin.liveconnect.SecureInvocation$2.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.plugin.liveconnect.SecureInvocation.CallMethod(Unknown Source)
at sun.plugin.liveconnect.SecureInvocation.access$300(Unknown Source)
at sun.plugin.liveconnect.SecureInvocation$CallMethodThread.run(Unknown Source)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.plugin.javascript.invoke.JSInvoke.invoke(Unknown Source)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.plugin.javascript.JSClassLoader.invoke(Unknown Source)
at sun.plugin.liveconnect.PrivilegedCallMethodAction.run(Unknown Source)
... 6 more
Caused by: java.security.AccessControlException: access denied (java.net.SocketPermission 10.0.0.8:2234 connect,resolve)
at java.security.AccessControlContext.checkPermission(Unknown Source)
at java.security.AccessController.checkPermission(Unknown Source)
at java.lang.SecurityManager.checkPermission(Unknown Source)
at java.lang.SecurityManager.checkConnect(Unknown Source)
at sun.rmi.transport.tcp.TCPChannel.checkConnectPermission(Unknown Source)
at sun.rmi.transport.tcp.TCPChannel.newConnection(Unknown Source)
at sun.rmi.server.UnicastRef.invoke(Unknown Source)
at net.ladypleaser.rmilite.impl.RemoteInvocationHandlerImpl_Stub.invoke(Unknown Source)
at net.ladypleaser.rmilite.impl.LocalInvocationHandlerImpl.invokeRemote(LocalInvocationHandlerImpl.java:53)
at net.ladypleaser.rmilite.impl.LocalInvocationHandlerImpl.invoke(LocalInvocationHandlerImpl.java:41)
at $Proxy0.requestToSubset(Unknown Source)
at org.votech.plastic.managers.PlasticApplication.sendMessage(PlasticApplication.java:166)
at org.votech.plastic.managers.PlasticApplication.sendMessage(PlasticApplication.java:140)
at uk.ac.roe.PlasticHubApplet.sendLoadVOTable(Unknown Source)
... 16 more

How to get an applet to repaint

2006-08-23T21:33:00.000+01:00

You have an applet. In response to some event you want to change all the components it contains. However, your changes don't appear...why?
If I was using a JFrame, I'd liberally throw in a few pack() statements, which causes the JFrame to resize itself based on the new components, and redraw, but you don't have this option with an applet. After some javadocing, it seems that a combination of validate(), invalidate() and repaint() ought to do it. Some experiments later, it seems that

invalidate does nothing
validate shows up the new components, but over the old ones
repaint erases everything

invalidate followed by validate is the same as validate
invalidate followed by repaint is the same as repaint

validate followed by repaint does the trick

Copying a list of files in Ant

2006-08-23T20:28:00.000+01:00

One might think that such an everyday task as copying a list of files would be trivial in Ant. Not so - it's a bloody nightmare.

First attempt: creating a FileList would seem to be what you want. Nope - the Copy task doesn't support FileLists. Why?????
Second attempt: create a FileSet (with the usual substitution of angle brackets....would it be too much to ask to have a blog site that doesn't stuff up xml?):


[copy todir="foo"]
  [fileset dir="bar"]
     [filename name="pathfrombarto/file"/]
     [filename name="differentpathfrombarto/file2"/]
  [/fileset]
[/copy]

In any case, the fact that you're forced to use a FileSet and its pattern matching means that if you have a large set of files below "dir" (e.g. a Maven repository) then the above takes an age.

The solution, incidentally, is:

[copy todir="foo"]
  [fileset dir="bar"]
    [or]
     [filename name="pathfrombarto/file"/]
     [filename name="differentpathfrombarto/file2"/]
    [/or]
  [/fileset]
[/copy]

Though it still takes an age to process.

Weekly Summary 30/8/06

2006-08-21T15:09:00.000+01:00

Guy has created a new community with password change facility - passed I&R and ready to install.

Tried to install, but it won't self-register: see bug 1789
Guy has put a fix on the merge queue for this...in the meantime we can hack the reg entry....done...appears to work well. Except.....you can change your password with any old password.
Mark to reinstall when it passes I&R? How did this error slip through I&R in the first place?

Submitted abstract to ADASS 2006 http://www.adass.org:8080/Conferences/2006/Venue/people/participants/abstract?abstract_id=82
Planning: using Open Workbench http://www.openworkbench.org/ to create GANTT charts and managed dependencies between tasks.
Downloaded and installed Eirik http://wiki.eurovotech.org/twiki/bin/view/VOTech/EirikDemo on a Debian machine. Plastic and AR functionality appear fine. Couldn’t test further due to Window size issues. Installation straightforward - install debian version of R first, then hand edit the Eirik startup script to find the location or libR.so.
- Tried to install Eirik on Shuna (Bob’s machine). Since I’m not root this involved building R from source using ./config and make. This is fairly easy to do, following the instructions, but care must be taken in the config stage to request a shared library libR.so to be built, and also to set –with-readline=no on Shuna. Unfortunately getting Eirik to run on Shuna wasn’t do-able due to assorted missing shared libs on this machine.
- However, following several iterations, did get a windows version running.
Hardened the hub to concurrency problems and clients passing shit values.
Jeff and others are experiencing some AR problems - AR is refusing RMI connections. I also have the same problem from time to time and it’s usually cured with a reboot. Mark’s Plaskit hub doesn’t suffer from this. Suspect it’s something to do with the way the AR deals with IP addresses - a network outtage is enough to screw it up.
Finished Anomaly Detection pipeline

Looking at deploying stilts at the same time. There are problems with the existings STILTS deployments discussed here. (Fixed by Kona). Get Kona/Mark to install?
Standard and Advanced interfaces now done, and registered. All works smoothly, but not very robust to errors in data since fastem locks up waiting for a user response....not a great idea for a CEA app.
Created an applet to allow users to simply change their workbench registry settings. This was also a learning exercise for creating a plastic-applet.
Instructions for use are here.

Finished a prototype Plastic-compatible web page...this is aimed at our own WSA, but also to get DataScope on board.
Received request from Noel to smarten up our metadata - we need to discuss logos.
Checked Son Of GAIA for Nigel, and found that I can’t run the webstart version either due to missing classes. Probably due to the 1.5 java libraries having renamed some classes. Reported to ussc@star.rl.ac.uk...address is defunct...emailed Mark Taylor...apparently you need to install the JAI package first (not documented!).
Assisted Brian getting his weka cea apps deployed to our test registry. We encountered a lot of out-of-memory errors in the registry, and also a particularly annoying bug necessitating the reconfiguration of the registry and changing its authority Id.
Assisted Brian changing the registry his workbench uses.
Realised that if we have a test registry, we need a test JES, so installed one here. The official release of JES is lacking an astrogrid-security jar, which you have to add by hand to WEB-INF/lib. Catherine has now released a new build, which Mark should test.
Thanks to Brian, reproduced the mysterious "Found Resource Document, but had no AccessURL" that has been plaguing systems that harvest from our publishing registry and try to access our date sets. Bugzilla report. It appears to be due to our use of Java 1.5. I've found that most of our old datasets suffer from the problem, but that it doesn't occur in the latest snapshot build of the workbench. Hence we could ignore it - it will go away when we reregister our new services anyway. Left it in Mark and Kevin's capable hands.
Republished Plastic site with google analytics code throughout

mvn 2.0.5 not yet released, and the bugs in 2.0.4 make site publication a painful process as

site:deploy doesn't work in windows
site:stage doesn't copy the contents of directories, effectively making multiproject site deployment useless

Actions
(68.2) JDT to give an ROE coffee talk with a demo of AstroGrid
- maybe also including access to UKIDSS data.
pending...

(69.1) JDT and KEA to produce a plan for providing password-changing
functionality for ROE account holders registering for AG.
DONE - see above

(69.2) JDT to pass this up to Noel Winstanley and/or Kevin Benson. DONE - see above

ACTION (70.1) MSH to install the new version of Community.
done by JDT

ACTION: (70.4) JDT to contact Guy Rixon to check that their plans for wrapping
STIL(TS) as CEA services are consistent.
DONE - see above

Thoughts:
Need better cover for MSH's absence

Creating the Anomaly Detector CEC

2006-08-21T14:39:00.000+01:00

I'm installing Andy Connolly's Expectation Maximization/Anomaly Detection algorithm into an AstroGrid CEC (Common Execution Connector) to make it available to the VO. Along the way I've been using the splendid STILTS utility to do all the VOTable conversions.
Things I've learned along the way

STILTS is sufficient to take a table in assorted different formats, and extract a user-defined set of columns (specified by position or name), and export the results as an EM-friendly space-separated text file. It is also smart enough to take the generated list of row numbers and p_values and do an exact cross match on the original table...thus generating a clone of the original table, sorted by p_value and with the p_values attached as an extra column. In short, it does all the table manipulation needed to transform a VO-standard table into a proprietary format and back again, with only a little help from awk.
Any processes started by the CEC must NOT ask the user for input. The current EM-algorithm prompts the user for input if it fails for some reason...this seriously screws up your CEC. The solution is to add a test on the return value from the STILTS preprocessing step, before the EM-algorith is called:

#echo $?
#Try to protect fastem from bad data
if [ "$?" -ne "0" ]; then
echo "STILTS failed to prepare data. Aborting"
exit 1
fi

You can only specify a registry template for a single CEA application. This means that if you're CEC supports several applications, you need to edit the registry description by hand after it's been generated.
You can specify an optional argument in the CEC-CL config with (insert)
The CEC-CL only supports the following kinds of parameters:

-key value
positional
key=value

Unfortunately the em algorithm takes parameters as

key value

You can get round this by pattern matching on the params with something like ${@//=/ } which takes all the parameters in one go and replaces all = with ' '

New week, new blog

2006-08-18T13:42:00.000+01:00

Still looking for the perfect home for my blog. All I want is something that's quick and responsive that I can use to jot down notes from work to give my ageing memory a break. Plus...when you spend all day wrestling maven into submission, it's possible that what you've learned just might save someone else some grey hairs.

So, I'm defecting from wordpress: http://jontayler.wordpress.com/ to here.