Written task to java. It uses postgresql, spring boot, rest. Everything is simple, you need to read the data from the database, an object with two fields of ID and a name, and then pass it in response to json. Here is checking it one of the errors was the inefficient use of RAM.

One of the conditions is not to use SQL capabilities. Also in the database by the condition of more than a million records.

What to do to fix? I will not throw off the controller yet, because it seems that the problem is on the lower layers, but if I have to lay it out right away.

My code is:

@Repository public class ContactRepositoryImpl implements ContactRepository {     private static final BeanPropertyRowMapper<Contact> ROW_MAPPER = BeanPropertyRowMapper.newInstance(Contact.class);     JdbcTemplate jdbcTemplate;     @Autowired     public ContactRepositoryImpl(@Qualifier("dataSource") DataSource dataSource) {         this.jdbcTemplate = new JdbcTemplate(dataSource);     }     public List<Contact> getAll() {         System.out.println(jdbcTemplate.toString());         return jdbcTemplate.query("SELECT * FROM contacts", ROW_MAPPER);     } } @Service public class ContactServiceImpl implements ContactService {     @Autowired     private ContactRepository repository;     @Override     public List<Contact> getFilteredContacts(String nameFilter) {         Pattern pattern = Pattern.compile(nameFilter);         List<Contact> list = repository.getAll();         List<Contact> list2 = list.stream()                 .filter(Objects::nonNull)                 .filter(it -> it.getName() != null)                 .filter(it -> !pattern.matcher(it.getName()).matches())                 .collect(Collectors.toList());         return list2;     } }

    2 answers 2

    I understand the limitations are to use the features of Spring Data?
    Look towards the PagingAndSortingRepository .
    In this interface there is a method:

     Page<T> findAll(Pageable pageable)

    which allows you to pull the record pages.
    You can read records page by page and download results.

    If you don’t want to break the brevity of streams, you can wrap all the page-ination logic into your stream by replacing list.stream () with your own implementation. How to implement a Java stream? .


    If you are too lazy to do handles, you can try using fetch size hint

     @QueryHints(@javax.persistence.QueryHint(name="org.hibernate.fetchSize", value="50")) List<Foo> findAll();

    Set the fetch size with Spring Data.

      filtering in getFilteredContacts can be done in the query: SELECT * FROM contacts WHERE name IS NOT NULL AND name LIKE :name . You pull out all the results from the base and drive here and there. It makes no sense.

      • This condition. Do not use SQL features. - alex safsafsd
      • Try to put LIMIT 5 in the request and see if there will be a memory leak. If yes, then the problem is this. It depends on how many entries you have in contacts. About the conditions - it is strange. For the basic things inherent in any relational database (if the reason suddenly migrate to another). and just load the base - just criminal :) - tcpack4
      • Well, I understand. That this is not logical (by the way, according to the opinion in the database there are more than a million records. - alex safsafsd
      • A: name is exactly what you need for kidney matches? I'm just not looking through the standard value, but with the help of regex. - alex safsafsd
      • in the final view, it should be like WHERE name LIKE '% Mik' (the percentage indicates the match of the first letters). or LIKE '% Mik%' on both sides. Well, you can use REGEXP (for mysql, others have their own analogs). - tcpack4