How to improve the load time for data in the viewer

Post Reply
opakman84
Posts: 3
Joined: 28 Jul 2011, 20:20

Hello,
I recently deployed a large data set to the essential viewer. The "reportXML.xml" file is close to 78MB. I am having problems will viewing some reports such as the, application module catalogue reports. The report loads very slowly and a few times I got a java heap size error. Is there a way for the essential viewer to perform better?

I tried increasing the heap size memory for Tomcat but that only improved things a little bit. Are there any more suggestions you can provide? Thanks.
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

The time taken to produce any of the catalogues will depend on how many elements there are in there. Can I ask how many Application Provider instances you have in the model? And also, how long are we talking about here? Seconds/Minutes? Thanks.

Assuming that there is sufficient memory available to Tomcat, the main resource is CPU power. What platform, (OS, CPUs, hardware) are you running on?

We're working on some updates to the out-of-the-box views that should be more efficient and those will be available soon.

On the memory side, how large have you set the Tomcat heap? We normally run with 2048MB on the maximum heap but it may be worth trying larger than that if your operating system will allow. Also make sure that the machine that you are running Tomcat on has enough physical memory, e.g. 4GB or so.

I'll take this away to the team, though, and we will look into options for improving performance with such large repositories.

Jonathan
Essential Project Team
opakman84
Posts: 3
Joined: 28 Jul 2011, 20:20

Thanks Johnathan for the reply.

I think I am lagging resources on my server, for the amount of processing that is required for these reports. My server is a VM, with a single CPU and 2 GB of RAM and is running Windows 2003 Server x64. I set the tomcat heap to about 1200Mb, but it takes about 10-15 minutes for the reports to load. I have about 4446 instances of Application provider.

Is there any way to create query based reports or some way of creating subsets of the data using certain parameters. Right now, all of the instances of Application provider load at once, which is not neccessary and causes performance issues.
User avatar
jonathan.carter
Posts: 1087
Joined: 04 Feb 2009, 15:44

Thanks very much for the details - it's really useful to get some idea of what you are working with.

You definitely need some more power on that server - you're now working with a pretty large dataset and the performance issues are showing that some more platform power is now required.

You will see a big improvement just by asking for another CPU and 4GB of RAM on the Virtual Server. Then you can allocated 2048MB to Tomcat's heap. There are some limitations on the amount of heap you can give to a JVM on Windows 32-bit platforms but as you're on a 64-bit you should be able to take the Java heap to 3GB or more.

Even with plenty of memory, a lot of the processing requires CPU - and when you have nearly 4500 Application Providers in there, things like the Application Module Catalogue are going to take a while to render. That's a mighty big HTML page you'll get back after 10+ minutes and perhaps an alphabetical list of over 4000 things isn't actually going to be that helpful. That's got a major contributory reason for the poor performance.

Having said all this, we are looking at the Essential Viewer Engine and improving the way it uses memory and handles very large XML documents.

It is most certainly possible to create Views that are keyed off subsets of the data - and as I suggested above, it would be useful even for users of the catalogue to have another way of slicing up the Application Catalogue. Perhaps it would help to define variants of the catalogue that divide the catalogue by the Application Services, Business Domains, etc.

This is a core part of Essential Viewer - it is a toolkit not a finished set of Views and Reports. Hopefully, the OOTB Views are useful as well as providing some basic starting points for putting your own Views together.
We'd be happy to help you define any Views that you might need.

Jonathan
Essential Project Team
Post Reply