Skip to content

Conversation

mrnicegyu11
Copy link
Member

@mrnicegyu11 mrnicegyu11 commented Sep 10, 2025

What do these changes do?

Fixes #1112

If postgres goes offline / is restarted, the postgres query exporter will disconnect and stay disconnected from postgres, logging an error approx. every minute when trying to execute the postgres query. This PR introduces a healthcheck based on listening to the low-level linux TCP socket file of master_postgres:5432. If postgres is offline, either the DNS name resolve or the TCP connection will fail. The healthcheck will then cause a new postgres query exporter container to be created, hopefully addressing the issue. Note that postgres communicates using raw TCP, and not HTTP(s).

Bonus: Updates docker image of the postgres query exporter.

Related issue/s

#1112

Related PR/s

Checklist

  • I tested and it works

mrnicegyu11 and others added 30 commits October 15, 2024 16:18
Merge remote-tracking branch 'upstream/main'
…oundation#979)

* Introduce longhorn chart

* Further longhorn configuration

* Longhorn: further settings configuration

* Fix longhorn configuration bugs

Extra: introduce longhorn pv vales for portainer

* Add comment for deletion longhorn

* Further longhorn configuration

* Add README.md for Longhorn wit FAQ

* Update Longhorn readme

* Update readme

* Futher LH configuration

* Update LH's Readme

* Update Longhorn Readme

* Improve LH's Readme

* LH: Reduce reserved default disk space to 5%

Since we use a dedicated disk for LH, we can go ahead with 5%

* Use values to set Longhorn storage class

* Update LH's Readme

* LH Readme: add requirements reference

* PR Review: bring back portainer s3 pv

* LH: decrease portinaer volume size
Merge remote-tracking branch 'upstream/main'
@mrnicegyu11 mrnicegyu11 added this to the Cheops milestone Sep 10, 2025
@mrnicegyu11 mrnicegyu11 self-assigned this Sep 10, 2025
@mrnicegyu11 mrnicegyu11 added t:bug Something isn't working observability alerting/monitoring FAST labels Sep 10, 2025
@mrnicegyu11 mrnicegyu11 marked this pull request as ready for review September 10, 2025 17:17
@mrnicegyu11
Copy link
Member Author

tested on osparc.local

@mrnicegyu11 mrnicegyu11 changed the title Fix: postgres query exporter staying disconnected when postgres is (briefly) offline 🐛 Fix: postgres query exporter staying disconnected when postgres is (briefly) offline Sep 10, 2025
@mrnicegyu11 mrnicegyu11 merged commit 8913dc7 into ITISFoundation:main Sep 12, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FAST observability alerting/monitoring t:bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

monitoring_pgsql-query-exporter constantly report error in logs
2 participants