You will find the part 1 & 2 of this series on Performance Tuning in Liferay here:
Performance tuning in Liferay – Part 1 | LinkedIn
Performance tuning in Liferay – Part 2 | LinkedIn
Further to above, find below some tips on Performance Tuning as part 3 of the series.
- Explore Java 11 instead of Java 8 with Liferay. Java 11 has performance and concurrency improvements. Check your compatibility matrix for compatibility.
- Explore various types of Garbage collection options – In my experience for certain scenarios G1 is especially helpful: Types of Garbage Collector in Java – Javatpoint
- Explore fragment caching button and option if you are using fragments
- Fine tune your Heap Size, analyse your heap dump, queries in Glowroot, MBeans in Glowroot to change EHCache values in Liferay – Make this an ongoing exercise
- Explore fast load css, fast load javascript, minifier, no pragma and cache.browser.disabled, users.update.last.login configurations
- Explore configuration in control panel for documents and media for private and public cache
- Upgrade and map to latest quarterly release, generally they have improvements
- Explore row lock error, Ora-1795, etc. settings in help center which cause known issues while scaling and workarounds for which are added in Help Center
- While doing performance tuning, do a warmup with low users to fill up cache of Liferay before actual testing – clear all 4 types of cache in CONTROL PANEL -> SERVER ADMINISTRATION when you make web content or other changes for them to reflect properly before load test
- Be cognizant of load times on JMeter and browser both – different configurations affect both differently
Email me: Neil@HarwaniSystems.in
Here is a general checklist for performance tuning & go-live in Java – Part 1:
- Servers: CPU, Memory, Threads, Network configurations, page size, hardening and more
- JVM: MBeans, Security, Garbage collection type and settings, Arguments, Heap, Version, Core allotment, Generations and more
- App server: Thread pool, JDBC/JNDI, Type of connection pooling, Session replication, caching, log rotation, log levels and more
- Database: Indexing, Thread pool, Privileges, Connections, Top queries/plans and more
- Network: Latency, DNS, Security, Firewalls/WAF/CDN
- Other components like search/filestores/integrations/etc.: Caching, thread pools and more
- VAPT/Security: SSL, OWASP, SAST/DAST, CSP, code quality and more
- Documentation on architecture, deployment, requirements, design and more
- APM/Monitoring tools – Instrumentation, gauges and more
- Thread profiling and heap dumps
- Service profile for APIs
- Code optimization
- Optimum auditing, reporting & SSO/IAM
- Reusable patterns / architecture styles and designs for performance, usability & security
- DevSecOps strategy and implementation
- DR strategy and implementation
- Regular load & performance tests
- Horizontal scaling / clustering & vertical scaling
- Rule of thumb: Work layer by layer and tackle the biggest problems first.
Email me: Neil@HarwaniSystems.in
Following up on the Performance tuning in Liferay – Part 1 post – here are some additional points for performance tuning:
- The blue circle in Glowroot slow traces indicates that the transaction is still ongoing whereas yellow indicates it’s completed. Red indicates there is an error.
- You can change the JVM gauges as needed to see lot of different types of details of JVM
- You can use instrumentation from configuration using type ahead drop down (AJAX style)
- You can allocate specific cores to a JVM if need be using an argument on core allotment
- You can see thread profiles and take heap dumps from Glowroot itself
- Glowroot has sections on errors, queries and service calls
- You can get Glowroot to sustain it’s data through restarts using arguments
- For frequent garbage collection, check if some queries, etc. are continuously filling up memory and any fine tuning on caching can help
- It helps to follow these sections in no particular order and check them regularly: JVM – guages, MBeans (You can search your cache related things here – it’s like seeing things in VisualVM), threads and more, transactions, slow traces (all three areas throughput/etc.), thread profiles (search “blocked” here), queries, service calls, errors, sorting options in transactions, instrumentation, etc.
- Latest Glowroot works with Java 11 that is compatible with Liferay for customizations and product usage but not source code compile. This latest Glowroot gives lot many more options especially in transactions.
- If you are using Java 8, check if your Glowroot version is compatible with it. Typically 0.13.6 and below are compatible with Java 8 as far as I know.
Email me: Neil@HarwaniSystems.in
Expanding on my post here on performance tuning: Post | Feed | LinkedIn
Below are the main points to work on for a performance tuning engagement in Liferay – Part 1.
- Firstly, we need to find out what is slow: Database, service calls, elastic search, memory is an issue, threads are blocked / waiting, how much memory is a module taking, logs are printing what, etc.
- Check your configurations as per this post: How to debug Liferay? – Some pointers – Part 1 | LinkedIn
- Install Glowroot if possible, in central pattern and check following sections in it: slow traces, errors, service calls, threads, heap, instrumentation, configurations and so on for the problem timeframe.
- You can enable tracing in logs & Glowroot instrumentation on targets. You can also use plugins by Fabian Bouché like for fragment analysis or follow his blogs on www.liferay.dev/blogs for using Glowroot in upgrades.
- The above will give you hints on what is slow. Especially open the FLAME graphs and threads along with heap dumps to analyze which threads are blocked or waiting, how much memory is allocated to what in slow traces of modules and so on.
- Then run a load test in simulated environment after checking compatibility matrix to get latest statistics for various scenarios like web content on portlet or in fragment, API calls, integrations, heavy load on Elasticsearch and so on with experimentation on themes.
- After getting the slow threads and details in flame graphs plus slow traces, if it’s custom code or configuration or DB call or ES which is slow, optimize it like Hikari pool connections or if it’s source code of Liferay, open the GITHUB repo for Liferay portal, check the source code and reach out of Customer Success / Global Services / Support with inferences depending on your engagement in account. Your Customer Success Manager or Sales can guide you on this.
- GS / CS will work internally in Liferay to get you the best options and / or patches in case they already exist. Many a times this could also have been fixed in a Hot fix or Fix Pack already. Alternatively, configurations could also solve such problems many a times. To check these go to Liferay customer portal and check the changelog for fix packs. You can also refer to Liferay Learn and Help Center for help articles and tutorials.
- Various areas of performance tuning: Database, HTTP calls, App server, ElasticSearch, Threads, Heap optimization, Caching and more. We will follow up this post with more pointers on performance tuning in Part – 2. A good list of areas is to check in the deployment guide for your version.
- Thanks to Fabian Bouché David Nebinger and many more at Liferay Global Services / Support / Customer Success and Customer Support / Engineering due to which I am able to compile the above. Above is a compilation of work from many sources internally in Liferay via work with customers & externally which hopefully should help many of Liferay customers and partners. This also serves as a case study on performance tuning.
- Email me: Neil@HarwaniSystems.in
Ideas on Innovation around Software. We Thrive On Ideas. We are Learner Centered, Open Source & Digital Focused.