Hello @denizciftci-sec

Could you tell me which of the two deployments you are using (single_node or multi_node) and what steps did you take for it?

I recommend that to deploy an environment with docker you follow the steps in our documentation: https://documentation.wazuh.com/current/deployment-options/docker/wazuh-container.html

Hi @vcerenu ,

many thanks for the reply. We went for the single mode deployment and followed the guide except the certification generation part. The generate-indexer-certs.yml is clearly not working so I downloaded the bash script(wazuh-certs-tool.sh) and created/edited config.yml then I generated all certificates manually/successfully and moved them in wazuh_indexer_ssl_certs directory.

Certificate generation error:
[root@t-ifs-wazuh-srv01 single-node]# docker-compose -f generate-indexer-certs.yml run --rm generator
WARN[0000] Found orphan containers ([single-node-wazuh.dashboard-1 single-node-wazuh.manager-1 single-node-wazuh.indexer-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Cert tool does not exist in any bucket
ERROR: certificates were not created

[root@t-ifs-wazuh-srv01 single-node]# docker --version
Docker version 20.10.21, build baeda1f
[root@t-ifs-wazuh-srv01 single-node]# docker-compose --version
Docker Compose version v2.12.2

docker ps output:

root@wazuh:/# [root@t-ifs-wazuh-srv01 single-node]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8ca7bcee5754 wazuh/wazuh-dashboard:4.3.10 "/entrypoint.sh" 11 minutes ago Up 6 minutes 443/tcp, 0.0.0.0:443->5601/tcp single-node-wazuh.dashboard-1
53d99bb9a1b9 wazuh/wazuh-manager:4.3.10 "/init" 11 minutes ago Up 8 seconds 0.0.0.0:1514-1515->1514-1515/tcp, 0.0.0.0:514->514/udp, 0.0.0.0:55000->55000/tcp, 1516/tcp single-node-wazuh.manager-1
e8a14083200f wazuh/wazuh-indexer:4.3.10 "/entrypoint.sh open…" 11 minutes ago Up 6 minutes 0.0.0.0:9200->9200/tcp single-node-wazuh.indexer-1

I am not sure is this relevant but I can paste some indications that we have seen so far:

root@wazuh:/# service wazuh-manager status
wazuh-clusterd not running...
wazuh-modulesd not running...
wazuh-monitord not running...
wazuh-logcollector not running...
wazuh-remoted not running...
wazuh-syscheckd not running...
wazuh-analysisd not running...
wazuh-maild not running...
wazuh-execd not running...
wazuh-db not running...
wazuh-authd not running...
wazuh-agentlessd not running...
wazuh-integratord not running...
wazuh-dbd not running...
wazuh-csyslogd not running...
wazuh-apid not running...

root@wazuh:/# /var/ossec/bin/wazuh-apid -f
wazuh-apid: Orphan child process 404 was terminated.
wazuh-apid: Orphan child process 407 was terminated.
wazuh-apid: Orphan child process 333 was terminated.
Starting API in foreground
wazuh-apid: Orphan child process 407 was terminated.
wazuh-apid: Orphan child process 410 was terminated.
wazuh-apid: Orphan child process 332 was terminated.
Starting API in foreground

Got this error in wazuh-manager;
root@wazuh:/# cat /var/ossec/logs/ossec.log | grep -iE "ERROR|CRITICAL"
2022/11/22 13:26:46 wazuh-db: ERROR: at run_worker(): at recv(): Connection reset by peer (104)

The part of docker logs output of wazuh-manager (tried to capture events related to apid)

The path /etc/filebeat is already mounted
Identified Wazuh configuration files to mount...
'/wazuh-config-mount/etc/ossec.conf' -> '/var/ossec/etc/ossec.conf'
[cont-init.d] 0-wazuh-init: exited 0.
[cont-init.d] 1-config-filebeat: executing...
[cont-init.d] 1-config-filebeat: exited 0.
[cont-init.d] 2-manager: executing...
Starting Wazuh v4.3.10...
wazuh-apid: Process 404 not used by Wazuh, removing...
wazuh-apid: Non existent process 475, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 478, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 475, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 478, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 475, removing from /var/ossec/var/run...
wazuh-apid: Non existent process 478, removing from /var/ossec/var/run...
Started wazuh-apid...
Started wazuh-csyslogd...
Started wazuh-dbd...
Started wazuh-integratord...
Started wazuh-agentlessd...
wazuh-authd: Process 444 not used by Wazuh, removing...
Started wazuh-authd...
Started wazuh-db...
Started wazuh-execd...
Started wazuh-analysisd...
Started wazuh-syscheckd...
Started wazuh-remoted...
Started wazuh-logcollector...
Started wazuh-monitord...
wazuh-modulesd: Process 762 not used by Wazuh, removing...
Started wazuh-modulesd...
Completed.

Hi @denizciftci-sec

I see that when you tried to create you had containers up, that means that you should already have the certificates or you have created directories with the names of the certificates.
I recommend that you delete all the files and directories inside the wazuh_indexer_ssl_certs directory, delete all the wazuh stack containers that are running and generate the certificates again with the command docker-compose -f generate-indexer-certs.yml run --rm generator.

It checks if the internet connection is open for the container that is created for the creation of certificates, which checks that the wazuh-certs-tool.sh file is in our repository.

Also, I ask you, are you running on linux? because certificate creation doesn't work on MacOS.

Hi @vcerenu,

many thanks for the reply. There is an internet connection for sure - where we were able to pull the images successfully from the docker repository. Specifically, we are using proxy for docker process via > /etc/systemd/system/docker.service.d/http-proxy.conf

I got the following error when I execute it;

[root@t-ifs-wazuh-srv01 single-node]# docker-compose -f generate-indexer-certs.yml run --rm generator
WARN[0000] Found orphan containers ([single-node-wazuh.dashboard-1 single-node-wazuh.manager-1 single-node-wazuh.indexer-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Cert tool does not exist in any bucket
ERROR: certificates were not created

About the containers that I told you, it is referred to this warning:

WARN[0000] Found orphan containers ([single-node-wazuh.dashboard-1 single-node-wazuh.manager-1 single-node-

You must download all the containers that are running, so that it is not taking the certificate files.
Also, the last error does not seem to reach the repository, I recommend that you try if from that PC you reach the following paths:

https://packages.wazuh.com/4.3/
https://packages-dev.wazuh.com/4.3/

Within those two buckets you access the file that the container has to use to create the certificates, otherwise in the wazuh-docker repository you can check what the container does to create the certificates in the file indexer-certs-creator/config/ entrypoint.sh.

Hi @vcerenu,

I deleted all the containers, volumes and files/folders inside wazuh_indexer_ssl_certs as you recommended.
Initially, I defined the proxy in generate-indexer-certs.yml, but still having the same error (We should use proxy in our infrastructure). There is no firewalld or iptables service is running.

[root@t-ifs-wazuh-srv01 single-node]# vi generate-indexer-certs.yml

Wazuh App Copyright (C) 2021 Wazuh Inc. (License GPLv2)
version: '3'

services:
generator:
image: wazuh/wazuh-certs-generator:0.0.1
hostname: wazuh-certs-generator
volumes:
- ./config/wazuh_indexer_ssl_certs/:/certificates/
- ./config/certs.yml:/config/certs.yml
environment:
- HTTP_PROXY=http://192.168.38.10:3128 (alternatively tried https & with/without quotes did not worked)

[root@t-ifs-wazuh-srv01 single-node]# docker-compose -f generate-indexer-certs.yml run --rm generator
Cert tool does not exist in any bucket

I guess, it connects the relevant paths with 200/OK.

[root@t-ifs-wazuh-srv01 single-node]# curl -X HEAD -i https://packages.wazuh.com/4.3/
HTTP/1.1 200 Connection established

HTTP/1.1 403 Forbidden
Content-Type: application/xml
Connection: keep-alive
Date: Wed, 23 Nov 2022 08:37:58 GMT
Server: AmazonS3
X-Cache: Error from cloudfront
Via: 1.1 2afacc6ad96dbba3f0b477cd95f16458.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: FRA2-C2
X-Amz-Cf-Id: jxKn2k0R8R6bF3lqYUui24BEs4uM28X-W-lCe4x6eJxGPYmPSApzPw==

[root@t-ifs-wazuh-srv01 single-node]# curl -X HEAD -i https://packages-dev.wazuh.com/4.3/
HTTP/1.1 200 Connection established

HTTP/1.1 404 Not Found
Content-Type: application/xml
Connection: keep-alive
Date: Wed, 23 Nov 2022 08:38:31 GMT
Server: AmazonS3
X-Cache: Error from cloudfront
Via: 1.1 dc0aad619823d3400ef947433d0af8fa.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: FRA60-P3
X-Amz-Cf-Id: u3LuXpcwJ5cPTnd8zcCtkm--x5L543hOgwo1FeRaBA8NdLRYL5H0_w==

Hello @denizciftci-sec

You can check directly with the commands with which we check in the image if the tool exists

curl --silent -I https://packages.wazuh.com/4.3/wazuh-certs-tool.sh | grep -E "^HTTP" | awk '{print $2}'
curl --silent -I https://packages-dev.wazuh.com/4.3/wazuh-certs-tool.sh | grep -E "^HTTP" | awk '{print $2}'

Either of these two commands should return 200, which indicates that you are reaching the tool. It may be that you get another code through the proxy and this does not allow it to complete, so you can perform the test that I tell you, see what response code the first command gives you on the packages.wazuh.com address and with that response modify the entrypoint indexer-certs-creator/config/entrypoint.sh, then on the path indexer-certs-creator/ you can generate the image with the modified code with the following command:

docker build -t wazuh/wazuh-certs-generator:0.0.1 .

With this you generate the modified image so that it takes the code that you changed and you can launch the certificate generation command again.

Let me know how it went when you finished these tasks.

hi @vcerenu,
many thanks for the rapid reply. I was able to solve it via > following the official procedure on a test-PC (has no problems with proxy) and able to generate the certificates - and these are generated on the IP address of the main server. When I moved all certificates into wazuh_indexer_ssl_certs > these API errors went away...

Only 1 error we are countering at the moment is (also I saw the identical error in the test PC)
Check alerts index pattern >

INFO: Index pattern id in cookie: yes [wazuh-alerts-]
INFO: Getting list of valid index patterns...
INFO: Valid index patterns found: 1
INFO: Found default index pattern with title [wazuh-alerts-]: yes
INFO: Checking the app default pattern exists: id [wazuh-alerts-]...
INFO: Default pattern with id [wazuh-alerts-] exists: yes
ACTION: Default pattern id [wazuh-alerts-] set as default index pattern
INFO: Checking the index pattern id [wazuh-alerts-] exists...
INFO: Index pattern id exists [wazuh-alerts-]: yes
INFO: Index pattern id in cookie: yes [wazuh-alerts-]
INFO: Checking if the index pattern id [wazuh-alerts-] exists...
INFO: Index pattern id [wazuh-alerts-] found: yes title [wazuh-alerts-]
INFO: Checking if exists a template compatible with the index pattern title [wazuh-alerts-]
INFO: Template found for the selected index-pattern title [wazuh-alerts-*]: no
ERROR: No template found for the selected index-pattern title [wazuh-alerts-]
INFO: Index pattern id in cookie: [wazuh-alerts-]
INFO: Getting index pattern data [wazuh-alerts-]...
INFO: Index pattern data found: [yes]
INFO: Refreshing index pattern fields: title [wazuh-alerts-], id [wazuh-alerts-]...
ACTION: Refreshed index pattern fields: title [wazuh-alerts-], id [wazuh-alerts-]
[Alerts index pattern] No template found for the selected index-pattern title [wazuh-alerts-]

I tried to add the templates manually, but still having the error. Am I missing any steps here?

curl https://raw.githubusercontent.com/wazuh/wazuh/v4.3.10/extensions/elasticsearch/7.x/wazuh-template.json | curl --noproxy '' -X PUT "https://localhost:9200/_template/wazuh" -H 'Content-Type: application/json' -d @- -u wazuh-wui:xx- -k

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 58530 100 58530 0 0 254k 0 --:--:-- --:--:-- --:--:-- 254k

Current Templates:

Index Patterns in GUI:

Hello @denizciftci-sec

This problem with the index pattern is due to the fact that it is created when, from the Wazuh manager container, Filebeat connects with Wazuh Indexer.

I recommend that you enter the Wazuh manager container and execute the following command

filebeat test output

If you have any errors, you should check the certificates that have been mounted, to see if they are correct.

Hi @vcerenu,

This is what we see in manager;

elasticsearch: https://wazuh.indexer:9200...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: 172.19.0.3
dial up... OK
TLS...
security: server's certificate chain verification is enabled
handshake... OK
TLS version: TLSv1.3
dial up... OK
talk to server... ERROR Get "https://wazuh.indexer:9200": Forbidden
root@wazuh:/tmp#

Hi @denizciftci-sec
This error when executing the filebeat test output command indicates that you have no connection between Wazuh manager and Wazuh indexer, so it will not be able to pass some things to it, including the missing index pattern.

You should check that the certificates that you are mounting to Wazuh manager do not have errors.

Hi @vcerenu, The problem was fixed. I was able to generate the certificates successfully by not chaning the IP address of certs.yml file on my test PC. When I moved the generated certifications to production one, it worked! thanks for the support. But still not able to fix the the container service which generated certificates.