jUnit tests work with large amounts of data, in particular, this error occurs when we read data from the database to the object.

RAM - 16 GB, 4 virtual processors

when an error occurred

java.lang.OutOfMemoryError: GC overhead limit exceeded at sun.reflect.GeneratedConstructorAccessor47.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at java.lang.Class.newInstance(Class.java:442)

then I added the -XX:-UseGCOverheadLimit parameter -XX:-UseGCOverheadLimit but got the following error:

 java.lang.OutOfMemoryError: GC overhead limit exceeded at java.lang.reflect.Field.copy(Field.java:150) at java.lang.reflect.ReflectAccess.copyField(ReflectAccess.java:144) at sun.reflect.ReflectionFactory.copyField(ReflectionFactory.java:309) at java.lang.Class.copyFields(Class.java:3115) at java.lang.Class.getFields(Class.java:1557) at main.java.DataFormat.DbTable.loadObjectFromResultSet(DbTable.java:101) at main.java.DataFormat.DbTable.getEntities(DbTable.java:57)

all parameters of JVM -Xmx8192M -XX:-UseGCOverheadLimit

at the time of error, the following system parameters were: RAM is stable at 46% (7400MB), CPU 77% -82% (82% at the time the error occurred)

tell me how to fix this problem?

Sparvka

  1. java.lang.OutOfMemoryError: GC overhead limit limit exceeded

This error may occur as if the first and second areas overflow. It is connected with the fact that the memory is low and the GC is constantly working, trying to free up some space. This error can be disabled using the -XX parameter: -UseGCOverheadLimit, but, of course, it should not be disabled, and either solve the problem of memory leakage, or allocate more volume, or change the GC settings.

Answer
String object weighs too much. and I have a lot of strings there. For example, they counted a file that weighs 250 MB, has 1600 characters per line and 220,000 lines. I fill in ArrayList<String> lines , where each line is written as an object of 250 fields that take a value of 1600 characters. Such an object weighed a little more than 6Гб .

Solution: increased memory to 14Гб . Now the problem is that the 6GB object doesn't fit on the heap, you need to think something else and sacrifice the test execution time.

  • And where is this parameter set? -XX: -UseGCOverheadLimit - user219650

2 answers 2

In this case, the error java.lang.OutOfMemoryError: GC overhead limit exceeded indicates that you are trying to process such a volume of data that does not fit in memory. In the help you cited, it is mentioned that an error is thrown when the memory area occupied only by the objects created is full. Plus, in the stack trace, you can see that your classes (

 at main.java.DataFormat.DbTable.loadObjectFromResultSet(DbTable.java:101) at main.java.DataFormat.DbTable.getEntities(DbTable.java:57)

) at the time of receiving the error just trying to unload data from the database.

Therefore, a simple and disappointing conclusion from the above, test on a smaller amount of data or allocate a larger volume for unloading data (-Xmx12000M) if they fit there, the error will disappear. Adjusting garbage collection in this case is useless. GC will not be able to remove the loaded objects because they are still in use.

  • we'll see. We'll see the result tomorrow - Senior Pomidor
  • if I am not mistaken, it is not recommended to allocate more than 8GB for Xmx - Senior Pomidor
  • @Senior Automator, but if you really want it you can. - Mikhailov Valentine
  • @Senior Automator, but seriously it depends on the task, and if your data does not fit in memory, then you will not have a choice to either process them in parts, process out of memory (but for example in the database) or increase the heap size. - Mikhailov Valentine
  • one
    String object weighs too much. and I have a lot of strings there. For example, they counted a file that weighs 250 MB, has 1600 characters per line and 220,000 lines. I fill in ArrayList <String> lines, where each line is written as an object of 250 fields that take a value of 1600 characters. Such an object weighed a little more than 6GB. Solution: increased memory to 14GB. Now the problem is that a 6GB object is placed on a heap or thinking of something else. - Senior Pomidor

Nachtu with the basics for a better understanding of the error.

The JVM has two memory areas: Heap Memory and Non-Heap Memory .

  • Heap Memory - stores objects;
  • Non-Heap Memory - stores method parameters, primitive types, etc.

JVM memory structure

In your case, the Heap Memory overflows because A lot of objects are created that do not fit into Heap Memory . You can solve the problem by increasing Heap Memory (-Xmx key), which does not always help, because the amount of data may be more than the available RAM, therefore it is better to implement data processing from the database in batches, unload a part of the data so that the objects fit in memory, then kill them (assign null to unload the GC ), notice the next part, etc.

PS It should be understood that Heap is not equal to RAM . Heap is only reserved memory for JVM, it cannot use more than this reserve even if there is an excess of RAM.