TrueNas SCALE - Rebuild NetData Metrics

 I came across an issue after having some problems with one of my arrays. The metrics for things like network/cpu/etc were no longer loading. Regular console log kept saying "loading defaults" and not much information. After doing some digging I finally found that the DB files used with the metrics became corrupt. The fix is simple enough, rename the old DB files and restart the netdata daemon.

First off saw the netdata daemon kept stopping even after starting it.

systemctl status netdata


What gave this away was finally finding in the NetData log that it had a SQLite error.

Log Directory: /var/log/netdata
Error Log: error.log
Error Entry: SQLite error during database setup, rc = 10 (disk I/O error)

I decided to take the hammer approach and rather than attempt to fix the DB files, I just renamed them and started the netdata daemon.

DB Directory: /var/cache/netdata
DB Files: netdata-meta.db netdata-meta.db-shm netdata-meta.db-wal

Here is what I ended up running and everything started working


mv /var/cache/netdata/netdata-meta.db /var/cache/netdata/netdata-meta.db.old

mv /var/cache/netdata/netdata-meta.db-shm /var/cache/netdata/netdata-meta.db-shm.old

mv /var/cache/netdata/netdata-meta.db-wal /var/cache/netdata/netdata-meta.db-shm.wal

systemctl start netdata

Comments

Post a Comment

Popular posts from this blog

Intune Hybrid - NDES Cert Issue

As many of you who might be running a SCCM/Intune hybrid scenario for MDM will have learned. There are a few to many certificates in use between NDES, Intune, Wifi Profiles, VPN profiles, and well anything else that may have a cert buried in it. We had an instance were the NDES wasn't handing out the certs for WiFi and VPN profiles. However it was handing out the base communication certificate so phones could sync correctly. This was one of those head scratchers on our park, but if you know where to look it makes sense. Problem is (at least in my opinion) there is a lack of useful documentation and the logging leaves a little to be desired. Therefore, I'm going to give a bit about our journey and hopefully help some of you folks out if you run into something similar. When we were notified of this issue, we started to look at the usual logs, you know CRP.log NDESPlugin.log those types of things. First thing you will notice is that the CRP.log will show the same message ove...
Read more

Stuck @ "Waiting for user logon"

This may or may not be something that you would commonly see in your environment. We only see it occasionally but it used to be a pretty common problem. One of my teammates found a technet forum post that addressed this specifically. Pretty interesting read if you are fighting this issue on a particular client. https://social.technet.microsoft.com/Forums/en-US/1569e1e2-91bc-435d-8998-beb817d5b453/waiting-for-user-logon?forum=configmanagergeneral Here is the meat and potatoes of the forum discussion. $CITask = get-wmiobject -query "select * from CCM_CITask where TaskState != ' PendingSoftReboot' AND TaskState != 'PendingHardReboot' AND TaskState != 'InProgress'" -namespace root\ccm\CITasks if ($CITask -ne $NULL) { $CITask | remove-wmiobject -Whatif $CITask | remove-wmiobject } ELSE { "CCM_CITasks is empty. Nothing to do" }
Read more

Triggering a software update install via Powershell

Howdy Folks This post is a holiday slice of pie. Today we focus on triggering update(s) that are deployed to a machine. Now for the pie filling. Triggering an update scan on a client: ([wmiclass]'ROOT\ccm:SMS_Client').TriggerSchedule('{00000000-0000-0000-0000-000000000113}') Trigger install of all updates: ([wmiclass]'ROOT\ccm\ClientSDK:CCM_SoftwareUpdatesManager').InstallUpdates() Here is a nice addition if you only want to install specific update(s) you just have to modify the select statement: ([wmiclass]'ROOT\ccm\ClientSDK:CCM_SoftwareUpdatesManager').InstallUpdates([System.Management.ManagementObject[]] (get-wmiobject -query 'SELECT * FROM CCM_SoftwareUpdate' -namespace 'ROOT\ccm\ClientSDK')) If you would like to see if there are updates applying, if true they are running: $CCMUpdate = get-wmiobject -query "SELECT * FROM CCM_SoftwareUpdate" -namespace "ROOT\ccm\ClientSDK" if(@($CCMUpdate | where { ...
Read more