Sean Cull

852 XPages runtime engine faced an OutOfMemoryError on EC2

Sean Cull  August 27 2010 00:24:14

Update : Back on 851, just can't get 852 to run reliably on my ec2 setup ( note Ubuntu is unsupported )

Ever since I upgraded my EC2 Ubuntu 8.04 instance to 852 I have been having daily crashes due to Outofmemory Errors.

08.26.2010 23:19:48   HTTP JVM: >>>>
08.26.2010 23:19:48   HTTP JVM: The XPages runtime engine faced an OutOfMemoryError
08.26.2010 23:19:48   HTTP JVM: You can fix this by increasing the value of the HTTPJVMMaxHeapSize variable in notes.ini
08.26.2010 23:19:48   HTTP JVM: >>>>

A small EC2 instance only has 1.7Gb of memory so it is going to struggle in any case. I have tried HTTPJVMMaxHeapSize = 64M, 128M and 256M.
851 was set to 256Mb by default whereas 852 is set to 64Mb as default.

This technote http://www-01.ibm.com/support/docview.wss?uid=swg21377202 explains the background but it the only part I fully understand is the bit that reads

"XPages is an exciting new way to work with Domino applications. It brings Domino fully into Web 2.0 technologies as a world class leader. "



It basically says go to 64 Bit - is this windows,  Linux or both. If Linux then I assume the 32bit installation file as there is no 64bit version ?
Will 64 Bit make any difference if it is memory limited in any case ?
If I am memory limited do I make the value higher or lower ?

I appreciate that this is not an issue to people running on proper production hardware but it is an issue for development servers and appliances.

It could also be affected by using Ubuntu but it has been very good in all other respects up until now.

If you can't read this its because my server has crashed !

Sean

 Amazon EC2  Lotus 


Domino 852 - make sure that you test your Xpage applications first

Sean Cull  August 25 2010 11:27:12

Just a quick tip for any other "traditional notes client developers". With traditional Notes applications it has been very very rare that an upgrade has caused problems with functionality, I have only ever seen it a couple of times over thousands of applications.

We upgraded to 8.5.2 yesterday and both of our Xpages applications currently under development stopped working. In both cases 8.5.2 was fussier than 8.5.1 about tolerating some dubious SSJS code.

as an example this code worked on 8.5.1

var Statusnumber : integer = Statustext

whereas in 8.5.2 it required the more correct

var Statusnumber : integer = parseInt(Statustext)

While this may indeed be the correct thing to happen the combination of steep learning curve, significant server changes in a point release and a complacency built up over years as a Notes client developer may also affect a few other people so heads up.

Sean


 Admin Tips  Dev Tips  Lotus  Notes 8.5 


This blog is now running on 8.5.2 on Amazon EC2, S3 Firefox Organiser was useful

Sean Cull  August 24 2010 11:25:19

Now running on 852, the whole upgrade took 5 or 6 minutes, in fact the hardest part was getting the install files to the server.

I used Amazon S3 storage and the wget command from Linux.

One useful tip is that although the S3 control panel has an upload limit of 600Mb the Firefox S3 Firefox Organiser does not and doesn't require flash either.

Well done IBM for getting the files up promptly. Stuart McIntyre has a list of part codes.

For some reason the Linux Traveler server is listed under Windows but it is there.

Sean

 Admin Tips  Amazon EC2  Lotus  Notes 8.5 


Xpages and why my tendency to Gold Plate has got harder to manage.

Sean Cull  July 26 2010 22:06:03

When we are working on a new application we often talk about bronze, silver and gold plating.

I have a tendency to want to make things too good when in fact the customer is no better off for my efforts, we refer to this as Sean's Silver or Gold plating. Its a particular issue as we almost always work to a fixed price and experience has also shown me that every extra bell and whistle increases testing and support throughout the life of the application.

Xpages allows you to take this to whole new levels because the traditional constraints imposed by designing applications within the Notes client have been removed. Now you can have two equally functional Xpage applications  that are wildly different in terms of the interface bells and whistles.

The new functionality is obviously much welcomed and as we develop more design patterns it gets easier but it does make me wonder if it is also a risk to Notes ?

I have heard the opinion expressed several times over the years that one Notes real strengths was that you could ( only ) produce cheap applications and it was therefore very competitive. With Xpages even a simple departmental app can now be very feature rich ( or Gold Plated ). Pricing ourselves out of jobs or failing to deliver value for money to our customers is now something that we have to work harder at because our toolbox is so much bigger. A good problem to have but one that needs thought.

Sean

 Opinion  Xpages  Lotus 


Notes basic ( non eclipse ) is very useful now but will it hold Notes back in the future ?

Sean Cull  July 26 2010 21:42:31

I had my "road to Damascus moment" recently when I saw the same app run in Xpages on the client and on the web with no extra coding, or more accurately not much extra coding.

I realised that Notes had passed a huge milestone and some of my long held beliefs were unceremoniously dumped. One of these was that customers were best with the Notes basic ( non-eclipse ) client. It is considerably faster and despite working on notes apps full time neither I nor my colleagues had yet had cause to use any of the eclipse functionality.

Now when we are looking at new apps our first inclination is to want to do it in Xpages if it is likely to be a repeatable solution. This is not just because we are geeks and it is fun but because with our business heads on we believe that we have a much better chance of selling that application to potential customers who do not, and unlikely to ever, run Notes clients.

Notes basic was good for getting people with older hardware ( often < 3 years old ) onto newer versions of Notes. That was a very good thing in the battle for hearts and minds. We needed to get them off R5 / 6 and onto something which really showed what Notes could do.

But hang on, is there not a danger that we will be repeating history with Xpages and Notes Basic. If developers mostly want to code in Xpages and as a consequence people with Notes basic will progressively get left behind and, even though they are on Notes 8.x or maybe 9.x. They  will have that dangerously substandard experience that is analogous to users being on R5. I am stretching the point a little but I do think that it is a very valid concern.

So do I think Notes basic is a good thing ? Yes I do for now but I think it could quite possibly be a bad thing if it were continued on into Notes 9.x.


Sean

 Lotus  Opinion  XPages 


Receiving POP3 mail into a Linux Domino server using getmail and sendmail

Sean Cull  July 26 2010 07:42:55

I have a need to receive mail from a POP3 account into a domino server running on Centos 5.5 as an appliance.  

Some time ago on windows I used the Chimeras Email Forwarding System which worked very well but this was a Linux server.

The scheme I came up with uses a combination of  Getmail version 4 and Sendmail.

Getmail is a Linux utility which will poll different types of mail accounts and then pass those messages on. It has an option to pipe the messages to an external mail delivery agent such as Sendmail. Sendmail can then send them on to the Domino server as SMTP on the localhost address.

These notes are for a Centos 5.5 installation with the Webmin web based management console

Install and configure Getmail



Create a new user called mailer or similar and a password. This is because Getmail will not call Sendmail if it is run as root.

As Root download Getmail

see http://pyropus.ca/software/getmail/documentation.html#installing

$ cd /tmp/
$ wget http://pyropus.ca/software/getmail/old-versions/getmail-4.20.0.tar.gz
$ tar xzf getmail-version.tar.gz
$ rpm -ihv getmail-version-release.noarch.rpm
$ cd getmail-version
$ python setup.py build


I then used Winscp to create the configuration file /home/mailer/.getmail/getmailrc


In this case the receiving account on the Domino server will be the Administrator account. Getmail has lots of other options

[retriever]
type = SimplePOP3Retriever
server = acme.com
username = peter pan
password =

[destination]
type = MDA_external
path = /usr/sbin/sendmail
arguments = ("-i","-bm", "administrator@localhost")

[options]
verbose = 0
read_all = false
delete = false
message_log = ~/.getmail/log

[default]

Configure Sendmail



Sendmail and Domino cannot normally run on the same box because they both need to bind to port 25. In this scenario Sendmail does not need to listen to port 25 so it can be configured to listen on port 26 ( or any other unused port ). I used the Webmin client to make this change under Servers > Sendmail > Network Ports


Image:Receiving POP3 mail into a Linux Domino server using getmail and sendmail


Testing the System



Log in as mailer

enter getmail -v
The -v gives a verbose output so that you can see what is happening.


Scheduling the getmail task



I used webmin to set up the cron job running as the mailer user


System > Scheduled Cron Jobs


Image:Receiving POP3 mail into a Linux Domino server using getmail and sendmail

Other things to remember



You may need to set Sendmail up to start on bootup

You may need to allow pop3 mail through your firewall


As usual all feedback on better schemes welcomed !

Sean


 Admin Tips  Appliance  Linux  Lotus  Show-n-Tell Thursday 


Please read Jakes post and give your views in the comments

Sean Cull  July 13 2010 11:20:29

Jake has apparently been asked to leave Planet Lotus. This is just daft.

I don't know Jake from Adam but he is certainly part of this community and his insights into both Domino and Sharepoint ( and PHP and RUBY etc ) are very much valued. Please express your opinions on Jakes blog.

http://www.codestore.net/store.nsf/unid/BLOG-20100713-0453

Sean

 Lotus 
Comments Disabled


EC2, too clever for me, snapshots, multiple servers, instant servers, but "release" your IP address and it all falls over ( for days ).

Sean Cull  July 2 2010 09:34:47

Dear reader, you will probably be reading this in a few days time because I have just pressed the wrong button in EC2.

It's been an interesting lesson. Part of moving to the cloud is all the extra redundancy that you get. You feel that the incredibly powerful controls that you are given mean that in some respects you haven't really given away control but rather gained it.

You might feel uncomfortable going for a small or new host organisation but you feel safer with the big people like Amazon EC2.

The problem is that humans are still involved and some of the services that you have been given are only on loan. In my case the fixed IP address from Amazon was only on loan and when I accidentally gave it back I couldn't get it back again. In the on premise paradigm people had redundant power supplies, redundant drives, redundant switches but did any one worry as much about loosing an IP address whilst still with the same connectivity provider ?

Its good that its Friday ( shame it wasn't Friday evening though ) because when you see this on Monday on Planet Lotus the link might actually work depending on your local ISP's DNS caching. Was it a good weekend for you ?

What happened



EC2 has a great feature called an "Elastic IP Address" that allows you to have a static IP address that you can switch between EC2 instances ( servers ).

If you would like to avoid some chaos then you SHOULD NOT EVER choose to "release" your Elastic IP address for once you do it disappears and is lost forever.

This means that you need to get a new one and re-do all of your DNS A records again. This means that your site will be down for days.

When moving the IP address between instances you should use the Disassociate option below. You will notice that the Release option is greyed out.

Image:EC2, too clever for me, snapshots, multiple servers, instant servers, but "release" your IP address and it all falls over ( for days ).

However, if you have just terminated the instance that had been associated with that IP address the console ( quite logically ) looks like this

Image:EC2, too clever for me, snapshots, multiple servers, instant servers, but "release" your IP address and it all falls over ( for days ).

Now I know that the logical thing to do ( in hind sight ) would be to click the Associate button but my brain has not twigged that the Dissociate action happened automatically when the instance was terminated so I clicked the "Release Address" button and got this.

Image:EC2, too clever for me, snapshots, multiple servers, instant servers, but "release" your IP address and it all falls over ( for days ).

I really want them to change it to say this

Image:EC2, too clever for me, snapshots, multiple servers, instant servers, but "release" your IP address and it all falls over ( for days ).











 Admin Tips  Amazon EC2  Lotus 


Amazon EC2 - good and bad

Sean Cull  July 1 2010 15:49:44

Some of you may have noticed that the site has just been down for about 25 minutes.

I decided to make a new AMI ( basically a template ) of the running server now that it has FP3 and for some reason doing this took the server down - its probably in the small print somewhere.

I had to terminate the instance to get the data volume back under my control but I was then able to bring up a new server and switch the IP address within another 10 minutes. While I was at it I brought a bigger server up.

I had been updating the AMI so that I could create a temporary cluster to move from a small server to a larger server seamlessly - something I'll need to try another time !

Previous article on setting up a domino server on EC2

 Admin Tips  Amazon EC2  Lotus 


We didn’t sell Lotus Notes but we used Notes to make a customer 30% more efficient over £12M of work since 2002

Sean Cull  July 1 2010 07:56:02

Following the suggestion that we should all have a theme on the value that users get from Notes I thought that I would mention this one.



It is a Notes application that we have been developing with ABB since 2002 and which Ed Brill has mentioned before. Our customer has now been good enough to quantify the benefits, hence the quote on the first page.

“Working with FoCul has delivered quality and efficiency improvement to our consultancy processes and services, whilst retaining the ability to customise work processes at short notice to meet specific customer needs

The benefit has been a 30% efficiency saving, allowing ABB to deliver over £12M of studies more competitively since 2002”  

David Stanier, Principal Consultant, ABB

I should point out that the application has not made ABB £3M better off by increasing their margins because the efficiency savings have helped them to be more competitive and win more work - try putting an ROI on that.

No individual part of the application is rocket science ( although we are very proud of the whole piece ), it is just a good Notes Application, a business tool. The benefits come from enabling very highly skilled engineering consultants to collaborate with each other and with their clients, to use a shared best practice approach and to significantly reduce the time they spend on administration - its not hard to kill the competitiveness of a project by having experts compiling huge word documents.

The development costs have been 300K in total over the 8 years with the costs being split pretty evenly across that period as more functionality has been added. ABB have probably spent a similar amount of time internally in terms of specification workshops, testing and peer training / support.

It is worth noting that FoCul hardly ever sells Lotus Notes ( hoping to sell XPages though ) and really does very little in terms of renewals. IBM probably sells ABB 100 - 200k licences a year.

 Download  Lotus  Case Study 


Am I happy with where Notes is from a technical standpoint ? Yes more than ever

Sean Cull  June 18 2010 14:13:03

I have just recorded a podcast with Stuart McIntyre, Darren Duke, Sharon Bellamy, Matt White and Bruce Elgort and it got me thinking.

Although I have commented on some blogs over the last week I have not posted anything myself so here goes.

Am I happy with where Notes is from a technical standpoint - yes more than ever.

My company focuses on a "double niche" - organisations that are in the manufacturing sector and have a pre-existing Notes infrastructure. With XPages we can now think of it as less of a double niche because we can design once for the web and the Notes client at the same time. This is a huge advance and as someone pointed out on the podcast pretty unusual in terms of platforms. Notes has always been much more capable than the vast majority of its users know and now this is even more the case.

We are developing our first application which will run as an appliance at a customer site that is non notes. But it also runs as a local replica in the Notes client - how cool is that ? Think Google Gears but better ( and still supported ). I might regret this but here are the daily and weekly builds ( sorry Andrew ! ). Trust me it looks the same in the Notes Client.

I deliberately used the phrase "where Notes is technically" because I am not as happy around the non technical side. Its not that it has suddenly got worse rather that the gap between "what Notes can do" and what people "think Notes can do" has got much wider and so my frustration has got greater.

I will openly admit to having looked to see if we can offer solution on the MS platforms as well as Domino and I have always come away confident that for what we do Notes has very limited competition. So what are the issues ?

IBM has got to value retaining clients

Reps need to be incentivised for renewals - it will stop companies on < 8.x thinking the product is dated.

Its a virtuous circle that will ensure that IBM customers get better value from Notes through applications, DAOS, servers running faster on existing hardware etc.  and IBM spends less time fighting a rear guard battle against a mail platform and a file server.

People often talk about life cycles - what is IBM's view on the lifecycle of a customer ?

IBM has got to be wholistic about Notes being a development platform

If I sell a 10K Xpage app to a company with <1000 employees the whole project is 12K ( 2k for the utility express server ). If I sell the same App to a company with 1001 employees the whole project costs maybe 25k. This means that I can't be competitive in large companies - that just seems daft.

At the same time pricing by individual users is not the answer either - for example if I write an app that manages brochures or safety data sheets the client will absolutely not pay £X per month for each customer just because they are listed in a NAB somewhere ( in reality it drives you to LAMP or to use a daft email based system with response parsing to avoid the restrictions ).

I honestly don't know the answer but the current scheme is not helpful. There may be ways and means around this but if so this should be open and transparent or people will just not bother.

IBM needs to lower the threshold for existing and "net new" developers to start using XPages

While I applaud the work by individuals in IBM the whole documentation piece is very very poor and the learning curve is much too steep.

The Wikis are a nice idea but they are too diverse - new users want focused examples and tutorials. For example follow this link to look at the wiki articles on  XPages in the client - a strategically important area for Lotus ?. While again I applaud the work of the individuals who made the entries the formatting alone makes it unpalatable. If IBM is going to depend on Wikis they need to employ some editors.

IBM needs to bundle 2 or 3 really good Apps with Domino

so that businesses get more value and so that they can see how good Notes can be in terms of Business Applications.

I would caution that this is not an opportunity to deploy sexy Composite Apps because businesses need to be able to dissect the bundled apps and learn how to put new ones together - in most uses cases Composite apps are too expensive. If there was a good Holidays Management App wouldn't most customers use it ?

Some people will argue that OpenNtf does this. While I have huge respect for the IBM employed OpenNtf developers the Wiki as an example is not fit for production ( test being would you encourage the CXO to use it ). It is a marvellous thing for developers to use and dissect but it is too buggy and incomplete to give to a customer. I say this as someone who uses them a lot. I would say the same of the Blog template as well ( although Steve Castledine did recently seek feedback on this one ).

Just a couple more observations :



IBMs current methods to promote Notes do not work -  I think that is a statement many people would respond "A strongly agree".  Einstein said

"Insanity: doing the same thing over and over again and expecting different results
"

IBM also wonder why LAMP, Ruby etc.. get such uptake when they don't have marketing either. To me the answer is in the barriers to adoption. If, for example, the on premises suite from IBM was available free for up to ten users would the Lotus community grow much ? Would the public perception of Notes improve ? I am not wedded to this idea, I don't think that it would directly create work for me, but it would require a shift in mindset on the part of IBM and I think that is what I am really looking for. It would also have negligible cost ( this licence would be unsupported) . Yes I know that there is a downside to giving away fee stuff, been there and had my fingers burnt, but it should be considered - think about Einstein's quote above.

The action list above is all "IBM to do". People will say two things - "what about you Mr Business Partner, ISV etc.. ?" and "look, just use another platform ?"

To the first I would say yes we all need to do more but our passion needs to be matched by IBM, not just Ed, IBM. There also needs to be a clearer roadmap about Notes as a design platform. People have rightly cautioned against building careers and businesses around a single vendor. I agree and it is a concern for me but to be good at what you do it is sometimes necessary. The thing that scares people away is not seeing a shared roadmap for the product. I don't understand the roadmap for Notes as a design platform and the lack of a sensible licensing structure sometimes makes me wonder if there is a roadmap.

To the second point I would respond that the platform is very very good and from a technical perspective is the best choice for me and my work. The hardest part is getting over the general lack of confidence in the platform that exists in the wider IT community.

Lastly ( I'm impressed your still here ! ) I think that some rebranding should be considered. I love Lotus Notes but "Lotus Notes" has negative connotations for too many people.

I personally would like to see "IBM Xpages" being pushed as a brand for non mail web delivered applications. It is easier to say to potential customers that the solution is based on IBM XPages rather than on Notes ( I think hardly anyone outside of this audience has heard of Domino ). Xpages could be positioned as an "Add-in" to Notes. I also like the idea of a better XPages specific development environment in addition to DDE. I feel like I've just publically disowned a child !

Anyhow, in conclusion I am hugely pleased with Notes, it is so much better than Exchange + sharepoint, I think the IBM developers are doing a fantastic job with the resources that they have but this misguided public impression of Notes needs to be addressed and the old ways are not working.

p.s.I know nothing about marketing so all of the above could be rubbish !

 8.5.1  Lotus  templates  winge  Wiki Template  XPages 


Fixing an Overheating Toshiba M700

Sean Cull  June 17 2010 21:42:44

I have been quite frustrated with my Toshiba M700 for the last couple of months. It never seemed to run as well as it should with the CPU maxing out at 100%, the battery going very quickly and the casing getting hot.

I had deliberately bought the M700 as a desktop replacement but I was at the point where I was just about to but a desktop anyway because I thought it was beyond the M700 to run two external monitors. I even thought about a Mac Book Pro until I saw the price.

I noticed that while the CPU was maxed out the Windows 7 Resource monitor was showing that the CPU frequency was only around 40% of the maximum attainable. I downloaded coretemp which showed that one of the cores was running at 103 C and I noticed that there was a correlation between the temperature and the CPU frequency - the system was throttling the frequency at these extreme temperatures.

Image:Fixing an Overheating Toshiba M700
Continue Reading "Fixing an Overheating Toshiba M700 " »

 My hardware  Tablet PC 


Sorted : Help ! Anyone know what has happened to the A Deep-dive on Building Lotus Domino Applications with XPages webcast ?

Sean Cull  June 17 2010 16:03:04

Sorry folks, I seem to be attracting a lot of readers which was not my intention.

I was desperate to see the " Deep-dive on Building Lotus Domino Applications with XPages webcast " but the link in my confirmation email just went to a page that said "thank you".

Image:Sorted : Help ! Anyone know what has happened to the A Deep-dive on Building Lotus Domino Applications with XPages webcast ?

When I followed the other link in the confirmation email the web cast was not listed either - the link was for the Rational site for some reason.

I got there in the end and it was an excellent webcast - the link below should show a replay by tomorrow - the start is a bit deep but it then goes on to show some good stuff and wets the appetite for 8.5.2

https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?lang=en_US&source=dw-c-wcsdp&S_PKG=062310A&S_CMP=dW&ca=ddc-

 Lotus 


AppendRTItem - only appends the first 32 k - I never knew that

Sean Cull  June 16 2010 09:32:32

A comment buried in the official example

Note The changes are not visible until the document (after the script has completed) is closed and reopened. Also note that the AppendRichTextItem method appends only the first 32K of a RichTextItem; the remainder is truncated.

Thanks to Jake's code store error list - I didn't even know the list existed.

update :
I have fed this back to IBM.

 Dev Tips  Lotus 


SNTT : Installing Traveler on an old 32 bit server is so much easier with linux than windows server 2008

Sean Cull  June 10 2010 08:00:00

Image:SNTT : Installing Traveler on an old 32 bit server is so much easier with linux than windows server 2008

Note 1 : Notes 8.5.2  is beta and there are no guarantees that the features described here will be in the final product that IBM ships.

Note 2 : The description below is for a development server and may or may not be suitable for production depending on your circumstances.

Having moved to Amazon EC2 I I recently decommissioned an old Dell Poweredge 650 server and decided to put it to good use to test 8.5.2 CD5. This server has no DVD drive, is 32 bit and cannot boot from USB.

My original plan had been to get some more experience of Server 2008 R2 but I ended up going down the Linux route as it was much simpler. Continue Reading "SNTT : Installing Traveler on an old 32 bit server is so much easier with linux than windows server 2008" »

 Admin Tips  Lotus  Show-n-Tell Thursday  traveler