Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ossec-dbd not reconnecting after database failure #447

Open
ChristianBeer opened this issue Nov 22, 2014 · 5 comments
Open

ossec-dbd not reconnecting after database failure #447

ChristianBeer opened this issue Nov 22, 2014 · 5 comments

Comments

@ChristianBeer
Copy link
Contributor

I was babysitting a new server and ran into memory trouble twice. During this I noticed an odd behaviour of ossec-dbd. The mysql server threads where killed at 03:57:53 (out of memory) but in the same second restarted. Ossec looses connectivity for only this query but can't reconnect after this. I have to manually restart ossec in order to reconnect ossec-dbd back to the database.

Can someone else please confirm this behaviour? Do I have to change something on the mysql server side?

2014/11/22 03:57:58 ossec-dbd(5203): ERROR: Error executing query 'SELECT id FROM location WHERE name = 'hopper->/var/log/messages' AND server_id = '1' LIMIT 1'. Error: 'MySQL server has gone away'.
2014/11/22 03:57:58 ossec-dbd(5209): INFO: Closing connection to database.
2014/11/22 03:57:58 ossec-dbd(5210): INFO: Attempting to reconnect to database.
2014/11/22 03:57:58 ossec-dbd(5202): ERROR: Error connecting to database 'localhost'(ossec): ERROR: Unknown MySQL server host 'localhost' (2).
2014/11/22 03:58:00 ossec-dbd(5210): INFO: Attempting to reconnect to database.
2014/11/22 03:58:00 ossec-dbd(5202): ERROR: Error connecting to database 'localhost'(ossec): ERROR: Unknown MySQL server host 'localhost' (2).
2014/11/22 03:58:04 ossec-dbd(5210): INFO: Attempting to reconnect to database.
2014/11/22 03:58:04 ossec-dbd(5202): ERROR: Error connecting to database 'localhost'(ossec): ERROR: Unknown MySQL server host 'localhost' (2).
2014/11/22 03:58:12 ossec-dbd(5210): INFO: Attempting to reconnect to database.
2014/11/22 03:58:12 ossec-dbd(5202): ERROR: Error connecting to database 'localhost'(ossec): ERROR: Unknown MySQL server host 'localhost' (2).
2014/11/22 03:58:28 ossec-dbd(5210): INFO: Attempting to reconnect to database.
2014/11/22 03:58:28 ossec-dbd(5202): ERROR: Error connecting to database 'localhost'(ossec): ERROR: Unknown MySQL server host 'localhost' (2).
2014/11/22 03:59:00 ossec-dbd(5210): INFO: Attempting to reconnect to database.
2014/11/22 03:59:00 ossec-dbd(5202): ERROR: Error connecting to database 'localhost'(ossec): ERROR: Unknown MySQL server host 'localhost' (2).
2014/11/22 04:00:04 ossec-dbd(5210): INFO: Attempting to reconnect to database.
2014/11/22 04:00:04 ossec-dbd(5202): ERROR: Error connecting to database 'localhost'(ossec): ERROR: Unknown MySQL server host 'localhost' (2).
@jrossi
Copy link
Member

jrossi commented Jan 5, 2015

@atomicturtle Have you seen anything like this you do a fair amount of DB and OSSEC :)

@atomicturtle
Copy link
Member

Yes, there is an internal_options setting to increase the attempts:

dbd.reconnect_attempts=10

The error itself is correct, its attempting a TCP connection (it does not use the socket because of the chroot) over 3306 and mysql is not responding. It could happen if mysqld wasnt running, exceeded max connections, firewall rule was blocking the port, etc

@drknite72
Copy link

I Just had the same issue and in my case the data table was corrupt
logging in to MySQL and running
OPTIMIZE TABLE alert;
OPTIMIZE TABLE data;

show my data table to be corrupt
used

REPAIR TABLE data;
to repair it and restarted ossec and everything worked again.

hope this helps you, or points you in the right direction.

@ChristianBeer
Copy link
Contributor Author

As far as I can tell I had no data corruption. I wondered why ossec couldn't reconnect to the DB even after it was back up. I always had to restart ossec manually to get it reconnected.

@atomicturtle
Copy link
Member

It probably exceeded dbd.reconnect_attempts=10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jrossi @ChristianBeer @atomicturtle @drknite72