Slow load times and intermittent timeout

Incident Report for Portfolio Manager

Postmortem

Our team investigated the incident and determined that some of the recurring automated tuning operations performed by the PostgreSQL database platform created a bottleneck in the LiquidPlanner operating environment. Requests backed up and some users experienced slow operations and errors.

We've since added additional database I/O capacity and adjusted the automated tuning operations to prevent this issue in the future.

Thanks for your continuing support and patience.

Posted Mar 08, 2019 - 14:57 PST

Resolved

We determined that the database came under heavy load and while it was catching up on requests, the site performance degraded for some users.
There was no impact to customer data during this period. The application is accessible and operating as expected at this time.

Posted Mar 01, 2019 - 16:29 PST

Update

We've made temporary mitigations to reduce the database load, and the site's performance is getting closer to normal levels. We are still investigating the cause. Thank you for your patience while we are continuing to work on the resolution.

Posted Mar 01, 2019 - 14:17 PST

Investigating

Some users may be experiencing trouble logging into LiquidPlanner and navigating around LiquidPlanner.

Our team is currently investigating issues related to slow page loads and intermittent timeouts.

We'll update you again in 30 minutes. If we have we have a better idea about the resolution time before then, we'll add it here immediately.

Posted Mar 01, 2019 - 13:43 PST

This incident affected: Portfolio Manager CLASSIC (https://app.liquidplanner.com) (Your workspace at https://app.liquidplanner.com/).