When trying to estimate an appropriate heap size for a java process there are many variables to consider.
In any modern software system, allowing a single process to dominate the available RAM (or any other limited resource) is likely to incur a significant overhead on the host operating system as it will almost certainly cause resource starvation for every other process.
In Java, heap memory size is generally defined when the process starts, (-Xms, -Xmx etc.), the java heap is memory that is used internally by Java objects but it is not the only memory that the process will consume; the interpreter must also have non-heap native memory assigned to run the virtual machine itself and manage class loading and other side effects of a managed runtime.
Furthermore, since java applications must also interact with the host operating system for activities such as networking and disk I/O, direct memory buffers will be continuously allocated and de-allocated throughout the lifespan of the process.
Externally to the actual java process, in addition to the fixed overhead of actually running an operating system, modern operating systems will consume memory that is largely driven by workload. One such source of memory consumption that is of significant importance to Jazz applications is file caching that will provide accelerated I/O performance for files. (In Jazz applications on-disk files are frequently read and written to provide rapid search capabilities through indexes that are not stored in the database but are too large to hold directly in RAM.)
Considering these factors you must consider the normal memory conditions of the server and then attempt to determine what is appropriate.
From personal experience and what I’ve learned in the past two years diagnosing and monitoring CLM deployments, if I were planning new CLM deployments I would be even more agressive than IBM advises for java web services in general and suggest the following:
25% of RAM for java heap.
25% for native memory
25% for disk caching
25% for the OS
TL;DR Trying to use more than half of your RAM for java heaps is daft unless you want bad performance, because that’s how you get bad performance.