How To Start Testing

• install the 16.04.4.2 ISO image (has old Logstash templates which we need to test removal of):
  https://github.com/Security-Onion-Solutions/security-onion/blob/master/old/Verify_ISO_16.04.4.2.md

• snapshot the VM if possible

• run Setup

• snapshot the VM if possible

• add the test PPA:

sudo add-apt-repository -y ppa:securityonion/test

• change DOCKERHUB from securityonionsolutions to securityonionsolutionstest (OSS license):

sudo sed -i 's|DOCKERHUB="securityonionsolutions"|DOCKERHUB="securityonionsolutionstest"|g' /etc/nsm/elasticdownload.conf

(OR change DOCKERHUB to securityonionsolutionselastest for Elastic Features license)

• update:

sudo soup

How To Verify Proper Elastic Operation

Please test in as many different combinations as possible:

• verify that traditional Logstash parsing correctly parses Zeek logs in JSON or TSV format and Sysmon logs via Winlogbeat and Wazuh

• verify that Elasticsearch Ingest parsing correctly parses Zeek logs in JSON format and Sysmon logs via Winlogbeat and Wazuh

• verify that ElastAlert works properly

• verify Kibana dashboards visualize those parsed logs correctly (for dashboards that have search hits, visualizations should show data and NO errors...for dashboards that have NO search hits, visualizations should show NO data and NO errors)

• verify Squert and Logout links work properly

• verify pivoting to Indicator dashboard

• verify that each Kibana dashboard has a default query in the upper left

• verify that lucene is still the default query language for both Dashboards and Discover

• verify that you can now switch from dark mode to light mode via Kibana Advanced Settings and that the old dark and light scripts are gone

• verify pivoting to CapMe works from all network data types

• verify templates look correct

• verify Curator close and delete work properly

• check elasticsearch, logstash, and kibana logs in /var/log/ for anything out of the ordinary

• so-import-pcap vs sosetup-minimal vs traditional Setup

• Setup GUI vs CLI

• Evaluation Mode vs Production Mode - when testing Evaluation Mode, make sure that Domainstats and Freqserver are generating data properly, here is a pcap that should generate data on the DomainStats dashboard:
  https://www.malware-traffic-analysis.net/2021/01/12/2021-01-12-Hancitor-infection-with-Cobalt-Strike.pcap.zip

• standalone vs distributed deployments

• new installation vs upgrade

• Elastic OSS vs Elastic Features license (use so-elastic-features to switch from OSS to Features)

• SSO vs Elastic native auth (use so-elastic-auth to switch to Elastic native auth)

• fully test all features in Kibana (both OSS and Features) to make sure we've got all the new URLs that Kibana added

• test upgrading a machine that already has Elastic native auth enabled

• test upgrading to Elastic 7.10.2 and then doing a full upgrade to Security Onion 2

• 16.04.4.2 includes the old Logstash templates. When you install this update, it should automatically remove those old Logstash templates so new logs should come in without having to manually remove old templates.

Please make sure so-curator-closed-delete-delete gets tested thoroughly in at least the following scenarios:

• if we haven't reached LOG_SIZE_LIMIT, it should do nothing
• if we have reached LOG_SIZE_LIMIT but there are no closed indices, it should do nothing
• if there are closed indices but we haven't reached LOG_SIZE_LIMIT, it should do thing
• if we have reached LOG_SIZE_LIMIT and there are closed indices, it should delete closed indices until we are below LOG_SIZE_LIMIT or there are no more closed indices

All testing conducted using the above guidance.

• verify that traditional Logstash parsing correctly parses Zeek logs in JSON or TSV format and Sysmon logs via Winlogbeat and Wazuh - No issues

• verify that Elasticsearch Ingest parsing correctly parses Zeek logs in JSON format and Sysmon logs via Winlogbeat and Wazuh - No issues

• verify that ElastAlert works properly - No issues

• verify Kibana dashboards visualize those parsed logs correctly - No issues

• verify Squert and Logout links work properly - No issues

• verify pivoting to Indicator dashboard - No issues

• verify that each Kibana dashboard has a default query in the upper left - No issues

• verify that lucene is still the default query language for both Dashboards and Discover - No issues

• verify that you can now switch from dark mode to light mode via Kibana Advanced Settings and that the old dark and light scripts are gone - No issues

• verify pivoting to CapMe works from all network data types - No issues

• verify templates look correct - No issues

• check elasticsearch, logstash, and kibana logs in /var/log/ for anything out of the ordinary - No issues

• so-import-pcap vs traditional Setup - No issues

• Setup GUI vs CLI - No issues

• Evaluation Mode vs Production Mode - No issues

• standalone vs distributed deployments - No issues

• Elastic OSS vs Elastic Features license (use so-elastic-features to switch from OSS to Features) - No issues

• SSO vs Elastic native auth (use so-elastic-auth to switch to Elastic native auth) - No issues

• fully test all features in Kibana (both OSS and Features) to make sure we've got all the new URLs that Kibana added - No issues

• test upgrading to Elastic 7.10.2 and then doing a full upgrade to Security Onion 2 - No issues (Standalone and Distributed)

• 16.04.4.2 includes the old Logstash templates. When you install this update, it should automatically remove those old Logstash templates so new logs should come in without having to manually remove old templates. - No issues

• verify Curator close and delete work properly - No issues

Please make sure so-curator-closed-delete-delete gets tested thoroughly in at least the following scenarios:

• if we haven't reached LOG_SIZE_LIMIT, it should do nothing - No issues

• if we have reached LOG_SIZE_LIMIT but there are no closed indices, it should do nothing - No issues

• if there are closed indices but we haven't reached LOG_SIZE_LIMIT, it should do nothing - No issues

• if we have reached LOG_SIZE_LIMIT and there are closed indices, it should delete closed indices until we are below LOG_SIZE_LIMIT or there are no more closed indices - No issues

Published:
https://blog.securityonion.net/2021/02/elastic-stack-7102-now-available-for.html