Portal efforts (Plone saga)

We are in the process of setting up a portal site for AFIS (Advanced Fire Information System). Basically, three groups within the CSIR are involved in this project, they are ICT4EO, Remote Sensing, and the Satellite Applications Center (SAC).  There is an operational system up and running, providing fire alerts reliably to interested parties, and there is a whole lot of research around the application going on. ICT4EO, my group, has been trying to build an open-source, open-standards-compliant version of the application – our research is mainly focussed around Senosor Web Enablement middleware. Remote Sensing works on information products coming out of the remotely sensed data  - they are working on, for example, fire hazard predictions. SAC run the operational system.  So there are all these “products” within the CSIR that are called AFIS, and no real clarity on what it is, where it’s going.  To solve this, we are putting up an AFIS portal, to aggregate all the info about the system, including current and future research directions. It will be cool.

We decided to use plone as our basis for building the site. We need some sort of content management because different stakeholders must update different areas of the site, and we have worked with plone before. We needed plone 2.5.+ because we want to integrate Open Layers into the site. Ultimately, this site will be deployed on a windows machine, but while we’re developing it, it is sitting on the gentoo server run by ICT4EO.

I emerged zope and plone separately (possibly a mistake?), using zope 2.9.3, and plone 2.5.2. I used zope-config to install plone in my zope instance, and happily started it:
/etc/init.d/zope-afis start

All looked fine - but nothing was actually running. Checking the status produced:

* status: started
* Status of Zope in /var/lib/zope/zope-afis …
daemon manager not running

I tried all sorts of things, and then ran from the bin directory of my zope instance (/var/lib/zope/zope-afis/bin), using

./runzope

Which gave me a happy little error message to the effect that there was already something running on port 8080*. So I editted  the zope instance config file (/var/lib/zope/zope-afis/bin) to change the http-server port and voila! A working instance.

Except not quite - major site errors. Turns out plone 2.5.2 doesn’t like zope to be less than 2.9.6. It works beautifully now.

Summary:

1) If it doesn’t start - check that you don’t have anything else running on port 8080 before trawling the net for hours :).

2) Use at least zope 2.9.6. with plone 2.5.2.  (In fact, I came upon a newsgroup post suggesting that using zope 2.10.x wasn’t working with this plone version either – but I didn’t check that.

*Which I should have known, as it’s Tomcat, and I installed it! Doh!

All I wanted for christmas….

I decided to give my Dell laptop (D610) the gcc4 compiler for christmas – but it decided to fight me all the way.  I did “emerge -ufND world” and had to remove some packages (including some KDE ones) because of conflicts. Then I took the laptop home to do the actual emerge – but, when I turned it on, it informed me that /var/lib/init.d was not accessible, and when I tried to “ls” it, the permissions were simply a  line of question marks.   Arrgh! Anyway, apparently one needs to then have a liveCD and fsck the offending partition. Not having access to a liveCD, I managed to rename the whole of the /var/lib directory, and then make a new one, and copy all the uncorrupted files to the new directory. Then my machine booted, but I had no windows manager.

The upshot of a lot of swearing and heartache was that xdm was not in my “world” file, and one of the packages that I had unemerged had removed it.  Finally I got a graphic display up and running again, only to discover that with the ati-drivers version 8.32.5, I had disgusting cursor corruption on the second screen of my dual-head display. The cursor basically displayed as a large square of garbage, and it annoyed my boss. Switching to a software cursor was even worse, as every time I moved it, it left artefacts all over the screen, and was simply impossible to work with. Eventually, I reverted to ati-drivers 8.30. 3, and everything is working happily again.

The rest of my research unit runs Ubuntu, but I am running Gentoo because the servers run gentoo, and I’d rather stuff something up on my machine than on the servers. Also, tomcat installs slightly differently on gentoo and ubuntu, and as we are largely currently deploying web services in the pursuit of technology demonstrators, knowing the gentoo idiosyncrasies without compromising the server is worth while.

Still, I think if the servers were not running gentoo, I would definately prefer Ubuntu – it seems to install more seamlessly and run with less manual configuration (good and bad – but then I would at least match the rest of the group).

Another day half gone…

I work half-days. Somehow this seems a lot less than the 5 hours it actually is. I seem to have far less than 5/8ths of the time available that the other team members do.

Yesterday and today were completely swallowed by yet another Gentoo issue. Basically, I corrupted a portage database, and it took me two days to get it back. At some point, I’m probably going to move to Ubuntu, since this seems to break far less often. Anyway, summary of problem and solution follow:

Problem: pc wouldn’t shut down on Friday. Now emerge gives the following error:

Traceback (most recent call last):
File “/usr/bin/emerge”, line 12, in ?
import portage
File “/usr/lib/portage/pym/portage.py”, line 7235, in ?
init_legacy_globals()
File “/usr/lib/portage/pym/portage.py”, line 7201, in init_legacy_globals
mtimedb = load_mtimedb(f)
File “/usr/lib/portage/pym/portage.py”, line 7126, in load_mtimedb
d = mypickle.load()
cPickle.UnpicklingError: invalid load key, ‘[‘.
————–

Aaargh!
So I searched around on line, and then tried manually fixing my portage installation, as per http://www.gentoo.org/proj/en/portage/doc/manually-fixing-portage.xml

Current portage installation (as at 11/09/2006) was 2.1.1, so I did:
wget -P /usr/portage/distfiles http://distfiles.gentoo.org/distfiles/portage-2.1.1.tar.bz2

Then I try to emerge again, and get another error!!!!

Traceback (most recent call last):
File “/usr/bin/emerge”, line 4049, in ?
emerge_main()
File “/usr/bin/emerge”, line 3768, in emerge_main
settings, trees, mtimedb = load_emerge_config()
File “/usr/bin/emerge”, line 3689, in load_emerge_config
mtimedb = portage.MtimeDB(mtimedbfile)
File “/usr/lib/portage/pym/portage.py”, line 6717, in __init__
self._load(filename)
File “/usr/lib/portage/pym/portage.py”, line 6724, in _load
d = mypickle.load()
cPickle.UnpicklingError: invalid load key, ‘[‘.

————————-

Now I call the system administrator.

He suggests:

http://forums.gentoo.org/viewtopic.php?t=143863&highlight=bestmatchvisible
Tried: rm -rf /var/cache/edb/dep/*
emerge –metadata

Still nothing. Then omniscient-type system admin person did some debugging…..

Added print statement to line 6717 of File “/usr/lib/portage/pym/portage.py”

This showed that the file that was loading before the unpickling error was /var/cache/edb/mtimedb

Then moved /var/cache/edb/mtimedb to /var/cache/edb/old

Did:
emerge –metadata

Hooray!!!!!

Then, just to make sure, did emerge sys-apps/portage. So now, 2 days later, my system sort of works again.

————————————————————————————————————–